mirror of
https://github.com/tangerineturtle1/NX2navi.git
synced 2026-11-18 06:59:57 -04:00
Redraw the in-game menu with the game's own interface textures, and bring in the launcher's save import and faster builds.
In-game menu: the settings window, its tabs, rows, buttons and cursor, the performance overlay and the Preparing shaders box are drawn with the game's interface textures where the RomFS holds them (runtime/ui/menu_skin.cpp, perf_panel.cpp; plain shapes until they are decoded). Restart joins Continue and Quit, each behind a confirmation; VSync is an on/off switch; DLSS is listed only on NVIDIA GPUs; the menu sounds are quieter; Picture shape is gone and the setting descriptions are shorter; edge cases of input, layout and fonts are fixed. tools/menu_preview.py draws the menu without the game. Launcher: Import Saves takes a save from the game folder's saves/ folder or one the player picks; downloads fall back to Windows' own certificate check when Python's refuses a server; updates are faster; the title bar is drawn in the window and Navi is every icon. tools/make_game.py builds with CLANG64's clang where the launcher installed it, packs the RomFS beside the lift and the compile, takes a prebuilt runtime, and compiles the game's resources without cmd.exe, so a user folder with a space in its name builds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -10,7 +10,8 @@ Steps, each skipped when a run before finished it for the same inputs (state in
|
||||
2. exefs the five ExeFS files, every module's segments unpacked (ZBIC through the player's own `zbic` package)
|
||||
and stored plainly, as the runtime's loader reads them -> <out>/exefs
|
||||
3. romfs every RomFS file, checked against the container's sha256, into one image -> <out>/romfs/romfs.bin and
|
||||
index.json (16-byte aligned, in the byte order of the paths)
|
||||
index.json (16-byte aligned, in the byte order of the paths). When the lift and the compile follow, it
|
||||
runs on a thread of its own beside them: only the RomFS index and romfs.c wait for the image
|
||||
4. lift recomp/gen.py with recomp/patches-v111.txt -> <out>/build-cache/lift, plus the two SDK bring-up fixes
|
||||
5. compile the lifted C and runtime/ with MSYS2's clang, linked into <out>/ocarina.exe; the MSYS2 DLLs it loads,
|
||||
the shader translator and romfs-index.bin beside it
|
||||
@@ -20,8 +21,16 @@ Steps, each skipped when a run before finished it for the same inputs (state in
|
||||
The exefs and romfs steps depend only on the game files; lift and compile depend on this kit's code too. A newer kit
|
||||
therefore lifts and compiles again but keeps the unpacked game files, and --rebuild needs no game files at all. The
|
||||
compile step keeps every object with the dependency list clang wrote for it (build-cache/obj/objects.json) and
|
||||
compiles a file again only when it, a header it includes, the flags or the compiler changed. The new ocarina.exe
|
||||
replaces the old one only once it links, so a failed build leaves the last good one in place.
|
||||
compiles a file again only when it, a header it includes, the flags or the compiler changed; the kit's own folder
|
||||
is not part of that key, so a kit in another folder (the launcher keeps each version in its own) reuses the objects.
|
||||
A kit with lib/libnx2navirt.a built by --make-runtime-lib for the same runtime/ and the same compiler links the
|
||||
runtime from it and compiles only the runtime files that include headers written per install. The new ocarina.exe
|
||||
replaces the old one only once it links (with lld when the toolchain has it), so a failed build leaves the last good
|
||||
one in place. Compiles run on every logical processor, the largest files first. The lifted code is built for this
|
||||
PC's x86-64 level (cpu_level), with MSYS2 CLANG64's clang when it sits beside the UCRT64 folder and reports the same
|
||||
version (fast_clang: the same objects, about a fifth faster).
|
||||
|
||||
python tools/make_game.py --make-runtime-lib <kit>/lib --msys <ucrt64> # the release's prebuilt runtime
|
||||
|
||||
Nothing is written outside --out. The game folder then holds only what this PC built from the player's files: never
|
||||
share it. Needs Python 3.10+ with numpy, lz4, zstandard and zbic (pip), and MSYS2 UCRT64's clang and libraries
|
||||
@@ -43,6 +52,7 @@ import shutil
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
@@ -76,11 +86,14 @@ LIFT_FIXES = [
|
||||
LINK_LIBS = ['-lvulkan-1', '-lSDL3', '-lopus', '-lvpx', '-l:libdav1d.a', '-lzstd', '-l:libatomic.a', '-lbcrypt',
|
||||
'-limm32', '-luuid', '-lole32', '-luser32', '-lshell32', '-lgdi32', '-ladvapi32', '-lwinmm']
|
||||
SYSTEM_DLLS_SKIP = {'vulkan-1.dll'} # the graphics driver's loader, never a copy
|
||||
# GNU windres's preprocessor, named explicitly: left to find it, windres puts the full path of the gcc beside it on a
|
||||
# cmd.exe command line, and once that path and the .rc's both hold a space, cmd.exe drops the outer quotes and splits
|
||||
# the command at the first space. gcc is then found on PATH (the toolchain's bin first); the arguments are windres's own.
|
||||
WINDRES_PP = ['--preprocessor=gcc', '--preprocessor-arg=-E', '--preprocessor-arg=-xc', '--preprocessor-arg=-DRC_INVOKED']
|
||||
EXE_NAME = 'ocarina.exe'
|
||||
# Headers this build writes per install (gen.py, romfs_index.py): a runtime source that includes one, directly or
|
||||
# through a runtime header, is compiled on the player's PC; every other runtime source can come prebuilt.
|
||||
GENERATED_HEADERS = ('layout.h', 'patches.h', 'romfs_config.h')
|
||||
RUNTIME_LIB = 'libnx2navirt.a'
|
||||
RUNTIME_PATTERNS = ('*.c', '*.cpp', 'platform/*.c', 'ui/*.cpp', 'third_party/imgui/*.cpp',
|
||||
'third_party/imgui/backends/*.cpp')
|
||||
INCLUDE = re.compile(r'^[ \t]*#[ \t]*include[ \t]+"([^"]+)"', re.M)
|
||||
|
||||
# Runs a script the way `python script.py` does, with the script's folder first on sys.path. The launcher's
|
||||
# embeddable Python leaves the script folder off sys.path (its ._pth file), so every script this build starts goes
|
||||
@@ -89,24 +102,53 @@ RUN_SCRIPT = ('import os, runpy, sys; s = sys.argv[1]; sys.argv = sys.argv[1:];
|
||||
'sys.path.insert(0, os.path.dirname(os.path.abspath(s))); runpy.run_path(s, run_name="__main__")')
|
||||
|
||||
EVENTS = False
|
||||
OUT_LOCK = threading.Lock() # the RomFS step prints from its own thread (main)
|
||||
|
||||
|
||||
def say(msg):
|
||||
print(msg, flush=True)
|
||||
with OUT_LOCK:
|
||||
print(msg, flush=True)
|
||||
|
||||
|
||||
def emit(ev, **fields):
|
||||
"""One "@@{json}" line for the launcher (--events); nothing otherwise."""
|
||||
if EVENTS:
|
||||
print('@@' + json.dumps({'ev': ev, **fields}, ensure_ascii=False), flush=True)
|
||||
line = '@@' + json.dumps({'ev': ev, **fields}, ensure_ascii=False)
|
||||
with OUT_LOCK:
|
||||
print(line, flush=True)
|
||||
|
||||
|
||||
def fail(msg):
|
||||
emit('error', message=msg)
|
||||
print(f'\nerror: {msg}', file=sys.stderr, flush=True)
|
||||
with OUT_LOCK:
|
||||
print(f'\nerror: {msg}', file=sys.stderr, flush=True)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
class Background:
|
||||
"""One step on a thread of its own. join() waits for it and raises its failure (fail's SystemExit included) in
|
||||
the caller; failed() tells the caller early, so it can stop work that would be thrown away."""
|
||||
|
||||
def __init__(self, fn):
|
||||
self.exc = None
|
||||
self.thread = threading.Thread(target=self._run, args=(fn,), name='romfs', daemon=True)
|
||||
self.thread.start()
|
||||
|
||||
def _run(self, fn):
|
||||
try:
|
||||
fn()
|
||||
except BaseException as e: # noqa: B036 - handed to join()
|
||||
self.exc = e
|
||||
|
||||
def failed(self):
|
||||
return self.exc is not None
|
||||
|
||||
def join(self):
|
||||
self.thread.join()
|
||||
if self.exc is not None:
|
||||
raise self.exc
|
||||
|
||||
|
||||
def py_cmd(script, *args):
|
||||
"""The command line that runs a Python script of this kit with this interpreter (see RUN_SCRIPT)."""
|
||||
flags = ['-I'] if sys.flags.isolated else []
|
||||
@@ -317,11 +359,11 @@ def step_romfs(src, out, jobs):
|
||||
done += size
|
||||
emit('progress', step='romfs', done=done, total=total, unit='bytes')
|
||||
if start // batch % 40 == 0:
|
||||
say(f' {done / 1e9:5.1f} of {total / 1e9:.1f} GB, {time.time() - t0:.0f} s')
|
||||
say(f' RomFS {done / 1e9:5.1f} of {total / 1e9:.1f} GB, {time.time() - t0:.0f} s')
|
||||
part.replace(dst / 'romfs.bin')
|
||||
index = {'status': 'pass', 'romfs_size': pos, 'romfs_sha256': h.hexdigest(), 'files': entries}
|
||||
(dst / 'index.json').write_text(json.dumps(index), encoding='utf-8')
|
||||
say(f' image {pos / 1e9:.2f} GB in {time.time() - t0:.0f} s')
|
||||
say(f' RomFS image {pos / 1e9:.2f} GB in {time.time() - t0:.0f} s')
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- lift
|
||||
@@ -491,17 +533,33 @@ def parse_deps(text):
|
||||
|
||||
|
||||
class ObjectCache:
|
||||
"""Which objects are current: per object the key of its command and source, and the sha256 of every file its
|
||||
dependency list names (build-cache/obj/objects.json)."""
|
||||
"""Which objects are current: per object the key of its command and source, and the sha256 (with size and
|
||||
modification time, to skip hashing an untouched file) of every file its dependency list names
|
||||
(build-cache/obj/objects.json). Paths under a root are kept relative to it ({'<kit>': KIT}), so the same files
|
||||
in a kit at another place still count as the same."""
|
||||
|
||||
def __init__(self, path):
|
||||
def __init__(self, path, roots=None):
|
||||
self.path = path
|
||||
self.roots = {k: os.path.normcase(os.path.abspath(v)) for k, v in (roots or {}).items()}
|
||||
try:
|
||||
self.entries = json.loads(path.read_text(encoding='utf-8')) if path.is_file() else {}
|
||||
except (OSError, ValueError):
|
||||
self.entries = {}
|
||||
self._sha = {}
|
||||
|
||||
def norm(self, p):
|
||||
a = os.path.normcase(os.path.abspath(p))
|
||||
for name, root in self.roots.items():
|
||||
if a == root or a.startswith(root + os.sep):
|
||||
return name + a[len(root):].replace(os.sep, '/')
|
||||
return a
|
||||
|
||||
def real(self, p):
|
||||
for name, root in self.roots.items():
|
||||
if p.startswith(name):
|
||||
return root + p[len(name):].replace('/', os.sep)
|
||||
return p
|
||||
|
||||
def file_sha(self, p):
|
||||
p = os.path.normcase(os.path.abspath(p))
|
||||
if p not in self._sha:
|
||||
@@ -511,18 +569,37 @@ class ObjectCache:
|
||||
self._sha[p] = None
|
||||
return self._sha[p]
|
||||
|
||||
def _same(self, p, rec):
|
||||
path = self.real(p)
|
||||
if isinstance(rec, str): # a cache without sizes and times
|
||||
return self.file_sha(path) == rec
|
||||
digest, size, mtime = rec
|
||||
try:
|
||||
st = os.stat(path)
|
||||
except OSError:
|
||||
return False
|
||||
if st.st_size == size and st.st_mtime_ns == mtime:
|
||||
return True
|
||||
return st.st_size == size and self.file_sha(path) == digest
|
||||
|
||||
def current(self, obj, key):
|
||||
e = self.entries.get(obj.name)
|
||||
return (e is not None and e.get('key') == key and obj.is_file()
|
||||
and all(self.file_sha(p) == h for p, h in e.get('deps', {}).items()))
|
||||
and all(self._same(p, rec) for p, rec in e.get('deps', {}).items()))
|
||||
|
||||
def record(self, obj, key, dep_file, src):
|
||||
try:
|
||||
deps = parse_deps(Path(dep_file).read_text(encoding='utf-8', errors='replace'))
|
||||
except OSError:
|
||||
deps = []
|
||||
deps = deps or [str(src)]
|
||||
self.entries[obj.name] = {'key': key, 'deps': {p: self.file_sha(p) for p in deps}}
|
||||
out = {}
|
||||
for p in deps or [str(src)]:
|
||||
try:
|
||||
st = os.stat(p)
|
||||
out[self.norm(p)] = [self.file_sha(p), st.st_size, st.st_mtime_ns]
|
||||
except OSError:
|
||||
out[self.norm(p)] = [None, -1, -1]
|
||||
self.entries[obj.name] = {'key': key, 'deps': out}
|
||||
|
||||
def forget(self, obj):
|
||||
self.entries.pop(obj.name, None)
|
||||
@@ -539,22 +616,31 @@ def compiler_id(clang, env):
|
||||
return sha256_bytes(f'{p.stdout}\n{st.st_size}\n{int(st.st_mtime)}'.encode())
|
||||
|
||||
|
||||
def compile_all(srcs, cmd_for, cache, jobs, env, progress=None, salt=''):
|
||||
def compile_all(srcs, cmd_for, cache, jobs, env, progress=None, salt='', stop=None):
|
||||
"""Compile each (source, object) whose object is not current in cache. cmd_for(src, dst) -> argv without the
|
||||
dependency-file options; salt identifies the compiler. Returns (compiled, reused, failures) with failures as
|
||||
(src, stderr tail)."""
|
||||
(src, stderr tail). The cache's roots stand for their paths in the flags too. The largest sources start first:
|
||||
compile time grows with size, and a big file that started last would run on alone after the others finished.
|
||||
stop() true makes the compiles not yet started end at once, unrecorded (a step beside this one failed)."""
|
||||
todo, keys = [], {}
|
||||
for src, dst in srcs:
|
||||
cmd = cmd_for(src, dst)
|
||||
flags = [c for c in cmd if c not in (str(src), str(dst))]
|
||||
flags = [c for c in cmd[1:] if c not in (str(src), str(dst))]
|
||||
for name, root in cache.roots.items():
|
||||
flags = [name + os.path.normcase(f)[len(root):].replace(os.sep, '/') if os.path.normcase(f).startswith(root)
|
||||
else f for f in flags]
|
||||
flags = [os.path.basename(cmd[0])] + flags
|
||||
keys[dst] = sha256_bytes(json.dumps([salt, flags, cache.file_sha(src)]).encode())
|
||||
if not cache.current(dst, keys[dst]):
|
||||
todo.append((src, dst))
|
||||
reused = len(srcs) - len(todo)
|
||||
todo.sort(key=lambda job: job[0].stat().st_size, reverse=True)
|
||||
|
||||
def cc(job):
|
||||
src, dst = job
|
||||
dep = dst.with_suffix('.d')
|
||||
if stop and stop():
|
||||
return src, dst, dep, None, ''
|
||||
cmd = cmd_for(src, dst)
|
||||
cmd[1:1] = ['-MMD', '-MF', str(dep)]
|
||||
p = subprocess.run(cmd, capture_output=True, text=True, env=env, errors='replace')
|
||||
@@ -566,6 +652,8 @@ def compile_all(srcs, cmd_for, cache, jobs, env, progress=None, salt=''):
|
||||
with concurrent.futures.ThreadPoolExecutor(max(1, jobs)) as ex:
|
||||
for src, dst, dep, rc, err in ex.map(cc, todo):
|
||||
done += 1
|
||||
if rc is None: # not started (stop)
|
||||
continue
|
||||
if rc:
|
||||
bad.append((src, err))
|
||||
cache.forget(dst)
|
||||
@@ -577,33 +665,236 @@ def compile_all(srcs, cmd_for, cache, jobs, env, progress=None, salt=''):
|
||||
return len(todo), reused, bad
|
||||
|
||||
|
||||
def step_compile(out, cache_dir, msys, jobs):
|
||||
def runtime_sources():
|
||||
"""(source, object name) of every runtime file the Windows build compiles."""
|
||||
rt = KIT / 'runtime'
|
||||
out = []
|
||||
for pat in RUNTIME_PATTERNS:
|
||||
for p in sorted(rt.glob(pat)):
|
||||
if p.stem.endswith('_sdl') or p.name == 'linux.c':
|
||||
continue
|
||||
out.append((p, 'rt_' + p.relative_to(rt).as_posix().replace('/', '_').rsplit('.', 1)[0] + '.o'))
|
||||
return out
|
||||
|
||||
|
||||
def per_install_names(generated=GENERATED_HEADERS):
|
||||
"""The basenames of the generated headers and of every runtime header that includes one, at any depth."""
|
||||
rt = KIT / 'runtime'
|
||||
headers = {}
|
||||
for p in list(rt.rglob('*.h')) + list(rt.rglob('*.inc')):
|
||||
headers[p.name] = {os.path.basename(i) for i in INCLUDE.findall(p.read_text(encoding='utf-8', errors='replace'))}
|
||||
tainted = set(generated)
|
||||
grew = True
|
||||
while grew:
|
||||
grew = False
|
||||
for name, incs in headers.items():
|
||||
if name not in tainted and incs & tainted:
|
||||
tainted.add(name)
|
||||
grew = True
|
||||
return tainted
|
||||
|
||||
|
||||
def per_install(src, tainted):
|
||||
incs = {os.path.basename(i) for i in INCLUDE.findall(Path(src).read_text(encoding='utf-8', errors='replace'))}
|
||||
return bool(incs & tainted)
|
||||
|
||||
|
||||
def clang_version(clang, env):
|
||||
p = subprocess.run([str(clang), '--version'], capture_output=True, text=True, env=env)
|
||||
return (p.stdout.splitlines() or [''])[0].strip()
|
||||
|
||||
|
||||
def runtime_digest():
|
||||
return tree_digest(KIT / 'runtime', ('**/*',))
|
||||
|
||||
|
||||
# The x86-64 levels the lifted code is built and tested for (docs/building.md, "Checks"), as the macros clang defines
|
||||
# for a CPU that has them.
|
||||
CPU_V2 = {'__SSE3__', '__SSSE3__', '__SSE4_1__', '__SSE4_2__', '__POPCNT__', '__LAHF_SAHF__',
|
||||
'__GCC_HAVE_SYNC_COMPARE_AND_SWAP_16'}
|
||||
CPU_V3 = CPU_V2 | {'__AVX__', '__AVX2__', '__BMI__', '__BMI2__', '__F16C__', '__FMA__', '__LZCNT__', '__MOVBE__',
|
||||
'__XSAVE__'}
|
||||
|
||||
|
||||
def cpu_level(clang, env):
|
||||
"""This PC's x86-64 level for the lifted code: 3 (AVX2, FMA, BMI2), 2 (SSE4.2, POPCNT) or 0 when clang cannot
|
||||
tell. The game runs on the PC that builds it, so it takes this CPU's level, as tools/build_game.py --cpu does by
|
||||
hand. Without one the lifted code would be generic x86-64: every fused multiply-add of the game a call to
|
||||
runtime/fma.c's software fma, and a level the differential tests do not cover."""
|
||||
p = subprocess.run([str(clang), '-march=native', '-E', '-dM', '-x', 'c', os.devnull], capture_output=True,
|
||||
text=True, env=env, errors='replace')
|
||||
have = set(re.findall(r'^#define (__\w+) ', p.stdout, re.M)) if p.returncode == 0 else set()
|
||||
return 3 if CPU_V3 <= have else 2 if CPU_V2 <= have else 0
|
||||
|
||||
|
||||
def clang_ident(clang, env):
|
||||
"""(version, target) a clang reports, or None when it does not run."""
|
||||
try:
|
||||
p = subprocess.run([str(clang), '--version'], capture_output=True, text=True, env=env, errors='replace')
|
||||
except OSError:
|
||||
return None
|
||||
v, t = re.search(r'clang version (\S+)', p.stdout), re.search(r'^Target: (\S+)', p.stdout, re.M)
|
||||
return (v.group(1), t.group(1)) if p.returncode == 0 and v and t else None
|
||||
|
||||
|
||||
def fast_clang(msys, clang, env):
|
||||
"""MSYS2's CLANG64 clang beside the UCRT64 one (msys64/clang64, which the launcher installs): the same LLVM release
|
||||
built with profile-guided optimisation, about a fifth faster on the lifted code. With UCRT64's headers
|
||||
(--sysroot) it writes the same objects as UCRT64's clang, byte for byte but the COFF timestamp; it is used only
|
||||
when it reports the same version and target. None otherwise."""
|
||||
fast = Path(msys).parent / 'clang64' / 'bin' / 'clang.exe'
|
||||
if not fast.is_file():
|
||||
return None
|
||||
mine = clang_ident(clang, env)
|
||||
return fast if mine and clang_ident(fast, env) == mine else None
|
||||
|
||||
|
||||
def runtime_cmd(clang, clangxx, inc, level=0, fast=None, msys=None):
|
||||
"""fast: fast_clang's compiler for the lifted code, with msys as its sysroot."""
|
||||
def cmd_for(src, dst):
|
||||
cpp = src.suffix in ('.cpp', '.cxx')
|
||||
# the lifted code at -O1 (its size makes -O2 slow to build), the runtime at -O2; neither may assume strict
|
||||
# aliasing: the lifted code reaches guest memory through casts of one address to several types
|
||||
# (tools/build_game.py's flags). The lifted code is built for this PC's level (cpu_level); the runtime
|
||||
# stays generic, so the prebuilt one serves every PC (its vectors are 16 bytes, passed alike at every level).
|
||||
guest = dst.name.startswith(('main_', 'sdk_', 'subsdk0_'))
|
||||
march = [f'-march=x86-64-v{level}'] if guest and level else []
|
||||
cc = [str(fast), f'--sysroot={msys}'] if guest and fast else [str(clangxx if cpp else clang)]
|
||||
return [*cc, '-c', '-std=gnu++17' if cpp else '-std=gnu11', '-O1' if guest else '-O2',
|
||||
*march, '-fno-strict-aliasing', '-w', *inc, '-o', str(dst), str(src)]
|
||||
return cmd_for
|
||||
|
||||
|
||||
def make_runtime_lib(dest, msys, jobs, obj=None):
|
||||
"""--make-runtime-lib: compile every runtime source that does not include a per-install header into
|
||||
<dest>/libnx2navirt.a, with runtime-lib.json naming the runtime/ tree and the compiler it was built from. The
|
||||
objects stay in obj (default <dest>/obj), which a release keeps out of what it ships."""
|
||||
dest = Path(dest).resolve()
|
||||
dest.mkdir(parents=True, exist_ok=True)
|
||||
obj = Path(obj).resolve() if obj else dest / 'obj'
|
||||
obj.mkdir(parents=True, exist_ok=True)
|
||||
clang, clangxx = msys / 'bin' / 'clang.exe', msys / 'bin' / 'clang++.exe'
|
||||
env = dict(os.environ, PATH=str(msys / 'bin') + os.pathsep + os.environ.get('PATH', ''))
|
||||
rt = KIT / 'runtime'
|
||||
tainted = per_install_names()
|
||||
srcs = [(p, obj / name) for p, name in runtime_sources() if not per_install(p, tainted)]
|
||||
inc = ['-I', str(rt), '-I', str(rt / 'third_party' / 'imgui'), '-I', str(msys / 'include' / 'vma')]
|
||||
cache = ObjectCache(obj / 'objects.json', {'<kit>': KIT})
|
||||
t0 = time.time()
|
||||
compiled, reused, bad = compile_all(srcs, runtime_cmd(clang, clangxx, inc), cache, jobs, env,
|
||||
lambda d, t: (d % 20 == 0 or d == t) and say(f' {d} of {t}'),
|
||||
salt=compiler_id(clang, env))
|
||||
if bad:
|
||||
for src, err in bad[:3]:
|
||||
print(f'--- {src}\n{err}', file=sys.stderr)
|
||||
fail(f'{len(bad)} runtime files did not compile')
|
||||
lib = dest / RUNTIME_LIB
|
||||
lib.unlink(missing_ok=True)
|
||||
rsp = dest / 'ar.rsp'
|
||||
rsp.write_text('\n'.join('"%s"' % o.as_posix() for _, o in srcs), encoding='utf-8')
|
||||
ar = os.environ.get('NX2NAVI_AR') or str(msys / 'bin' / 'ar.exe')
|
||||
if subprocess.run([ar, 'rcs', str(lib), '@' + str(rsp)], env=env).returncode:
|
||||
fail('the runtime library could not be archived')
|
||||
meta = {'runtime': runtime_digest(), 'compiler': clang_version(clang, env),
|
||||
'objects': sorted(o.name for _, o in srcs)}
|
||||
(dest / 'runtime-lib.json').write_text(json.dumps(meta, indent=1), encoding='utf-8')
|
||||
rsp.unlink(missing_ok=True)
|
||||
say(f'{lib}: {len(srcs)} runtime files ({compiled} compiled, {reused} kept), {lib.stat().st_size / 1e6:.1f} MB, '
|
||||
f'{time.time() - t0:.0f} s')
|
||||
|
||||
|
||||
def prebuilt_runtime(clang, env):
|
||||
"""(library, object names it holds) when the kit's lib/ matches this runtime/ and this compiler, else None."""
|
||||
lib, meta = KIT / 'lib' / RUNTIME_LIB, KIT / 'lib' / 'runtime-lib.json'
|
||||
try:
|
||||
m = json.loads(meta.read_text(encoding='utf-8'))
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
if not lib.is_file() or m.get('runtime') != runtime_digest() or m.get('compiler') != clang_version(clang, env):
|
||||
say(' the prebuilt runtime does not match this kit or compiler: compiling runtime/')
|
||||
return None
|
||||
return lib, set(m.get('objects', []))
|
||||
|
||||
|
||||
def romfs_index(out, gen):
|
||||
if subprocess.run(py_cmd(KIT / 'tools' / 'romfs_index.py', '--romfs', out / 'romfs', '--out', gen)).returncode:
|
||||
fail('the RomFS index could not be written')
|
||||
|
||||
|
||||
def windres_command(msys, rco):
|
||||
"""MSYS2's windres on nx2navi.rc, run with runtime/ as the working directory (compile_resources says why)."""
|
||||
pp = ('-E', '-xc', '-DRC_INVOKED') # windres's own preprocessor flags, which --preprocessor replaces
|
||||
return [str(msys / 'bin' / 'windres.exe'), '--use-temp-file', f'--preprocessor={msys / "bin" / "gcc.exe"}',
|
||||
*(f'--preprocessor-arg={a}' for a in pp), '-O', 'coff', '-i', 'nx2navi.rc', '-o', str(rco)]
|
||||
|
||||
|
||||
def compile_resources(msys, rt, obj, env):
|
||||
"""runtime/nx2navi.rc into obj/nx2navi_rc.o: the manifest of the game's exe (per-monitor DPI awareness, so the
|
||||
picture keeps the monitor's real size rather than one Windows stretches) and its icon.
|
||||
|
||||
windres preprocesses the .rc with gcc through one command line it writes itself, quoting each path that holds a
|
||||
space. By default it hands that line to cmd.exe /c, which drops the first and the last double quote of a line
|
||||
holding more than two: gcc's quoted path loses its opening quote, and under a user folder with a space in its
|
||||
name (C:\\Users\\Ali Sayed) cmd.exe stops at that space ("'C:\\Users\\Ali' is not recognized").
|
||||
|
||||
With --use-temp-file windres starts gcc itself instead, no shell involved: it splits the line into arguments and
|
||||
passes each one to gcc intact. Its splitter reads a quoted first word right, but it cuts a quoted -I path at its
|
||||
spaces and reads past the end of the line after a quoted last word, so the line holds no path but gcc's, its
|
||||
first word: the .rc goes by its bare name from runtime/ as the working directory, where windres also finds the
|
||||
manifest and the icon, and there is no -I. windres's temporary files go to a folder of the build cache, deleted
|
||||
afterwards (it leaves two of them behind on every run).
|
||||
|
||||
The object is written under another name and replaces the old one only when its bytes differ, so an unchanged
|
||||
resource keeps its time and does not make the link key change (a rebuild with nothing new does not link again)."""
|
||||
rco, new = obj / 'nx2navi_rc.o', obj / 'nx2navi_rc.o.new'
|
||||
tmp = obj / 'windres-tmp'
|
||||
shutil.rmtree(tmp, ignore_errors=True)
|
||||
tmp.mkdir()
|
||||
try:
|
||||
r = subprocess.run(windres_command(msys, new), cwd=rt, env=dict(env, TMP=str(tmp), TEMP=str(tmp)),
|
||||
capture_output=True, text=True, errors='replace')
|
||||
finally:
|
||||
shutil.rmtree(tmp, ignore_errors=True)
|
||||
if r.returncode:
|
||||
new.unlink(missing_ok=True)
|
||||
fail(f'windres failed on runtime/nx2navi.rc: {r.stderr.strip()[:400]}')
|
||||
if rco.is_file() and rco.read_bytes() == new.read_bytes():
|
||||
new.unlink()
|
||||
else:
|
||||
os.replace(new, rco)
|
||||
return rco
|
||||
|
||||
|
||||
def step_compile(out, cache_dir, msys, jobs, romfs_job=None):
|
||||
"""romfs_job: the RomFS step still running beside this one (a first build). Everything that does not include
|
||||
romfs_config.h compiles meanwhile; the RomFS index and the few files that include it follow once it is done."""
|
||||
lift, gen, obj = cache_dir / 'lift', cache_dir / 'gen', cache_dir / 'obj'
|
||||
gen.mkdir(parents=True, exist_ok=True)
|
||||
obj.mkdir(parents=True, exist_ok=True)
|
||||
rt = KIT / 'runtime'
|
||||
clang, clangxx = msys / 'bin' / 'clang.exe', msys / 'bin' / 'clang++.exe'
|
||||
env = dict(os.environ, PATH=str(msys / 'bin') + os.pathsep + os.environ.get('PATH', ''))
|
||||
if subprocess.run(py_cmd(KIT / 'tools' / 'romfs_index.py', '--romfs', out / 'romfs', '--out', gen)).returncode:
|
||||
fail('the RomFS index could not be written')
|
||||
if not romfs_job:
|
||||
romfs_index(out, gen)
|
||||
srcs = [(p, obj / f'{mod}_{p.stem}.o') for mod in ('main', 'sdk', 'subsdk0') for p in sorted((lift / mod).glob('*.c'))]
|
||||
for pat in ('*.c', '*.cpp', 'platform/*.c', 'ui/*.cpp', 'third_party/imgui/*.cpp', 'third_party/imgui/backends/*.cpp'):
|
||||
for p in sorted(rt.glob(pat)):
|
||||
if p.stem.endswith('_sdl') or p.name == 'linux.c':
|
||||
continue
|
||||
srcs.append((p, obj / ('rt_' + p.relative_to(rt).as_posix().replace('/', '_').rsplit('.', 1)[0] + '.o')))
|
||||
prebuilt = prebuilt_runtime(clang, env)
|
||||
runtime = [(p, obj / name) for p, name in runtime_sources()]
|
||||
if prebuilt:
|
||||
runtime = [(p, o) for p, o in runtime if o.name not in prebuilt[1]]
|
||||
say(f' the runtime comes prebuilt; {len(runtime)} runtime files are compiled here')
|
||||
srcs += runtime
|
||||
srcs += [(modules_c(lift, out, gen), obj / 'modules.o'), (gen / 'shim.c', obj / 'shim.o')]
|
||||
inc = ['-I', str(rt), '-I', str(lift), '-I', str(gen), '-I', str(rt / 'third_party' / 'imgui'),
|
||||
'-I', str(msys / 'include' / 'vma')]
|
||||
cid = compiler_id(clang, env)
|
||||
|
||||
def cmd_for(src, dst):
|
||||
cpp = src.suffix in ('.cpp', '.cxx')
|
||||
# the lifted code at -O1 (its size makes -O2 slow to build), the runtime at -O2; neither may assume strict aliasing:
|
||||
# the lifted code reaches guest memory through casts of one address to several types (tools/build_game.py's flags)
|
||||
guest = dst.name.startswith(('main_', 'sdk_', 'subsdk0_'))
|
||||
return [str(clangxx if cpp else clang), '-c', '-std=gnu++17' if cpp else '-std=gnu11', '-O1' if guest else '-O2',
|
||||
'-fno-strict-aliasing', '-w', *inc, '-o', str(dst), str(src)]
|
||||
level = cpu_level(clang, env)
|
||||
say(f' the lifted code is built for x86-64-v{level}, this processor\'s level' if level else
|
||||
' the lifted code is built for generic x86-64: the processor\'s level could not be read')
|
||||
fast = fast_clang(msys, clang, env)
|
||||
if fast:
|
||||
cid += compiler_id(fast, env)
|
||||
say(f' the lifted code compiles with {fast}')
|
||||
cmd_for = runtime_cmd(clang, clangxx, inc, level, fast, msys)
|
||||
|
||||
t0 = time.time()
|
||||
last = [0.0]
|
||||
@@ -617,29 +908,42 @@ def step_compile(out, cache_dir, msys, jobs):
|
||||
say(f' compiled {done} of {total}, {now - t0:.0f} s')
|
||||
|
||||
say(f' {len(srcs)} files on {jobs} threads')
|
||||
cache = ObjectCache(obj / 'objects.json')
|
||||
compiled, reused, bad = compile_all(srcs, cmd_for, cache, jobs, env, progress, salt=cid)
|
||||
cache = ObjectCache(obj / 'objects.json', {'<kit>': KIT})
|
||||
late = []
|
||||
if romfs_job: # the lifted chunks include only cpu.h and simd.h: only the runtime and generated files are read
|
||||
tainted = per_install_names(('romfs_config.h',))
|
||||
late = [(s, o) for s, o in runtime + srcs[-2:] if per_install(s, tainted)]
|
||||
say(f' {len(late)} files wait for the RomFS index: {", ".join(s.name for s, _ in late)}')
|
||||
early = [j for j in srcs if j not in late]
|
||||
compiled, reused, bad = compile_all(early, cmd_for, cache, jobs, env, lambda d, t: progress(d, len(srcs)),
|
||||
salt=cid, stop=romfs_job.failed if romfs_job else None)
|
||||
if romfs_job:
|
||||
if romfs_job.thread.is_alive():
|
||||
say(' waiting for the RomFS image')
|
||||
romfs_job.join() # raises the RomFS step's failure; a failed compile still keeps the finished image
|
||||
if not bad:
|
||||
romfs_index(out, gen)
|
||||
c2, r2, bad = compile_all(late, cmd_for, cache, jobs, env,
|
||||
lambda d, t: progress(len(early) + d, len(srcs)), salt=cid)
|
||||
compiled, reused = compiled + c2, reused + r2
|
||||
say(f' {compiled} compiled, {reused} unchanged since the last build')
|
||||
if bad:
|
||||
for src, err in bad[:3]:
|
||||
print(f'--- {src}\n{err}', file=sys.stderr)
|
||||
fail(f'{len(bad)} files did not compile')
|
||||
|
||||
# the manifest (per-monitor DPI awareness: the picture at the monitor's real size, not one Windows stretches) and the
|
||||
# icon, runtime/nx2navi.rc through MSYS2's windres. windres runs its preprocessor through cmd.exe: run from the
|
||||
# runtime folder with relative names and gcc by bare name (WINDRES_PP), that command line holds no path, so a space
|
||||
# in the install folder (a user name) cannot split it.
|
||||
rco = (obj / 'nx2navi_rc.o').resolve()
|
||||
r = subprocess.run([str(msys / 'bin' / 'windres.exe'), *WINDRES_PP, '-I', '.', '-O', 'coff', '-i', 'nx2navi.rc',
|
||||
'-o', str(rco)], capture_output=True, text=True, env=env, errors='replace', cwd=rt)
|
||||
if r.returncode:
|
||||
fail(f'windres failed on runtime/nx2navi.rc: {r.stderr.strip()[:400]}')
|
||||
rco = compile_resources(msys, rt, obj, env)
|
||||
objs = [o for _, o in srcs] + [rco]
|
||||
exe = out / EXE_NAME
|
||||
rsp = cache_dir / 'link.rsp'
|
||||
rsp.write_text('\n'.join('"%s"' % o.as_posix() for o in objs), encoding='utf-8')
|
||||
link_key = sha256_bytes(json.dumps([LINK_LIBS, cid] + [[o.name, o.stat().st_size, o.stat().st_mtime_ns]
|
||||
for o in objs]).encode())
|
||||
# lld links several times faster than GNU ld; the stack matches tools/build_game.py's build
|
||||
ld = ['-fuse-ld=lld', '-Wl,--stack,16777216'] if (msys / 'bin' / 'ld.lld.exe').is_file() else []
|
||||
lib = ['-Wl,--whole-archive', str(prebuilt[0]), '-Wl,--no-whole-archive'] if prebuilt else []
|
||||
stamp = [[o.name, o.stat().st_size, o.stat().st_mtime_ns] for o in objs]
|
||||
if prebuilt:
|
||||
stamp.append([prebuilt[0].name, prebuilt[0].stat().st_size, sha256_file(prebuilt[0])])
|
||||
link_key = sha256_bytes(json.dumps([LINK_LIBS, ld, cid] + stamp).encode())
|
||||
state_link = cache_dir / 'link.key'
|
||||
if exe.is_file() and state_link.is_file() and state_link.read_text(encoding='utf-8') == link_key:
|
||||
say(' ocarina.exe is current, not linked again')
|
||||
@@ -647,8 +951,8 @@ def step_compile(out, cache_dir, msys, jobs):
|
||||
emit('phase', step='compile', phase='link')
|
||||
say(' linking')
|
||||
tmp = out / (EXE_NAME + '.new')
|
||||
p = subprocess.run([str(clangxx), '-o', str(tmp), '@' + str(rsp), '-L', str(msys / 'lib'), *LINK_LIBS],
|
||||
capture_output=True, text=True, env=env, errors='replace')
|
||||
p = subprocess.run([str(clangxx), *ld, '-o', str(tmp), '@' + str(rsp), *lib, '-L', str(msys / 'lib'),
|
||||
*LINK_LIBS], capture_output=True, text=True, env=env, errors='replace')
|
||||
if p.returncode:
|
||||
print(p.stderr[-4000:], file=sys.stderr)
|
||||
tmp.unlink(missing_ok=True)
|
||||
@@ -736,9 +1040,14 @@ def main():
|
||||
ap.add_argument('--msys', default=r'C:\msys64\ucrt64', help='MSYS2 UCRT64 folder (default C:\\msys64\\ucrt64)')
|
||||
ap.add_argument('--jobs', type=int, default=os.cpu_count() or 4, help='parallel compiles')
|
||||
ap.add_argument('--events', action='store_true', help='machine-readable progress lines on stdout ("@@" + JSON)')
|
||||
ap.add_argument('--make-runtime-lib', metavar='DIR', help='build the prebuilt runtime library of a release into DIR')
|
||||
ap.add_argument('--obj', help='with --make-runtime-lib: the folder for its objects')
|
||||
ap.add_argument('--until', choices=STEPS, help=argparse.SUPPRESS) # tests: stop after this step
|
||||
a = ap.parse_args()
|
||||
EVENTS = a.events
|
||||
if a.make_runtime_lib:
|
||||
make_runtime_lib(a.make_runtime_lib, Path(a.msys), a.jobs, a.obj)
|
||||
return
|
||||
given = bool(a.nup) + bool(a.exefs or a.romfs) + bool(a.rebuild)
|
||||
if given != 1 or (not a.nup and not a.rebuild and not (a.exefs and a.romfs)):
|
||||
ap.error('give --nup, both --exefs and --romfs, or --rebuild')
|
||||
@@ -774,29 +1083,60 @@ def main():
|
||||
state['kit'] = kit_version()
|
||||
keep_awake()
|
||||
t0 = time.time()
|
||||
romfs_job = None
|
||||
runs = {'exefs': lambda: step_exefs(src, out), 'romfs': lambda: step_romfs(src, out, min(a.jobs, 8)),
|
||||
'lift': lambda: step_lift(out, cache), 'compile': lambda: step_compile(out, cache, msys, a.jobs),
|
||||
'lift': lambda: step_lift(out, cache), 'compile': lambda: step_compile(out, cache, msys, a.jobs, romfs_job),
|
||||
'shaders': lambda: step_shaders(out)}
|
||||
for name in STEPS:
|
||||
key = step_key(name, src.identity, out)
|
||||
if state['steps'].get(name) == key:
|
||||
say(f'{name}: done before')
|
||||
emit('step', step=name, state='skip')
|
||||
else:
|
||||
# a step that runs makes every step after it run too: their keys hash its output, not its key
|
||||
for later in STEPS[STEPS.index(name):]:
|
||||
state['steps'].pop(later, None)
|
||||
say(f'{name}:')
|
||||
emit('step', step=name, state='start')
|
||||
t = time.time()
|
||||
runs[name]()
|
||||
state['steps'][name] = step_key(name, src.identity, out)
|
||||
state_lock = threading.Lock() # the RomFS step records itself from its own thread
|
||||
|
||||
def save():
|
||||
with state_lock:
|
||||
state_path.write_text(json.dumps(state, indent=1), encoding='utf-8')
|
||||
say(f'{name}: {time.time() - t:.0f} s')
|
||||
emit('step', step=name, state='done', seconds=round(time.time() - t, 1))
|
||||
if name == a.until:
|
||||
break
|
||||
state_path.write_text(json.dumps(state, indent=1), encoding='utf-8')
|
||||
|
||||
def run(name):
|
||||
t = time.time()
|
||||
runs[name]()
|
||||
with state_lock:
|
||||
state['steps'][name] = step_key(name, src.identity, out)
|
||||
save()
|
||||
say(f'{name}: {time.time() - t:.0f} s')
|
||||
emit('step', step=name, state='done', seconds=round(time.time() - t, 1))
|
||||
|
||||
try:
|
||||
for name in STEPS:
|
||||
key = step_key(name, src.identity, out)
|
||||
if state['steps'].get(name) == key:
|
||||
say(f'{name}: done before')
|
||||
emit('step', step=name, state='skip')
|
||||
else:
|
||||
# a step that runs makes every step after it run too: their keys hash its output, not its key
|
||||
with state_lock:
|
||||
for later in STEPS[STEPS.index(name):]:
|
||||
state['steps'].pop(later, None)
|
||||
say(f'{name}:')
|
||||
emit('step', step=name, state='start')
|
||||
if name == 'shaders' and romfs_job: # the shaders come out of the RomFS image
|
||||
romfs_job.join()
|
||||
if name == 'romfs' and a.until != 'romfs':
|
||||
# The RomFS image is mostly disk work, the lift and the compile are processor work: they run side
|
||||
# by side, and only the RomFS index and the files that include it wait for the image
|
||||
# (step_compile). A first build takes about as long as the longer of the two, not their sum.
|
||||
romfs_job = Background(lambda: run('romfs'))
|
||||
else:
|
||||
run(name)
|
||||
if romfs_job and romfs_job.failed():
|
||||
romfs_job.join()
|
||||
if name == a.until:
|
||||
break
|
||||
if romfs_job:
|
||||
romfs_job.join()
|
||||
except SystemExit:
|
||||
# a failed lift or compile: the RomFS image still finishes and is kept, so the next run starts after it
|
||||
if romfs_job and romfs_job.thread.is_alive():
|
||||
say('waiting for the RomFS image before stopping')
|
||||
romfs_job.thread.join()
|
||||
raise
|
||||
save()
|
||||
say(f'\nfinished in {(time.time() - t0) / 60:.1f} min: {out / EXE_NAME}')
|
||||
emit('end', exe=str(out / EXE_NAME), seconds=round(time.time() - t0, 1), kit=kit_version())
|
||||
|
||||
|
||||
@@ -12,12 +12,14 @@ checked against the sha256 in PINS.
|
||||
The release folder (--out) holds:
|
||||
|
||||
NX2navi/ the launcher folder exactly as NX2navi-Setup.exe installs it:
|
||||
NX2navi.exe launcher/native/nx2navi.c with the Triforce icon (launcher/ui/art.py)
|
||||
NX2navi.exe launcher/native/nx2navi.c with the Navi icon (launcher/ui/art.py)
|
||||
python/ python.org's embeddable Python 3.14.9, tkinter and Tcl/Tk 9.0 from its tcltk.msi,
|
||||
the packages of tools/requirements-launcher.txt, runtime-id.txt
|
||||
app/current.txt, app/<version>/ launcher/ (without native/), runtime/, recomp/, the tools make_game.py uses,
|
||||
bin/nx2navi_xlate.exe and .id (and amd_fidelityfx_vk.dll with --ffx), toolchain.lock.json (the MSYS2 packages of today,
|
||||
pinned), feed.json, VERSION, the licence files
|
||||
bin/nx2navi_xlate.exe and .id (and amd_fidelityfx_vk.dll with --ffx),
|
||||
lib/libnx2navirt.a (the runtime, prebuilt with the release's compiler),
|
||||
toolchain.lock.json (the MSYS2 packages of today, pinned), feed.json, VERSION, the
|
||||
licence files
|
||||
NX2navi-Setup-<version>.exe launcher/native/setup.c with the folder above appended as a zip: the one file
|
||||
players download
|
||||
NX2navi-app-<version>.zip app/<version> alone: what the launcher downloads to update itself
|
||||
@@ -196,10 +198,33 @@ def native_compiler(lock):
|
||||
[windres, '--target=x86_64-w64-mingw32'])
|
||||
|
||||
|
||||
def build_runtime_lib(folder, ver, out, wine):
|
||||
"""app/<version>/lib/libnx2navirt.a: the runtime compiled once here with the release's MSYS2 clang, so players
|
||||
compile only the few runtime files that include per-install headers (tools/make_game.py --make-runtime-lib). It
|
||||
needs MSYS2's own clang.exe: native on Windows, through Wine with --wine elsewhere; without either players compile
|
||||
runtime/ themselves."""
|
||||
msys = Layout(root=MAIN_ROOT / 'toolchain' / 'launcher-msys').msys
|
||||
app = folder / 'app' / ver
|
||||
args = ['--make-runtime-lib', app / 'lib', '--obj', out / 'native' / 'runtime-obj', '--msys', msys]
|
||||
if os.name == 'nt':
|
||||
run([sys.executable, app / 'tools' / 'make_game.py', *args])
|
||||
elif wine:
|
||||
def win(p):
|
||||
return 'Z:' + str(Path(p).resolve()).replace('/', '\\')
|
||||
env = dict(os.environ, WINEDEBUG='-all', NX2NAVI_AR=win(msys / 'bin' / 'ar.exe'))
|
||||
run(['wine', folder / 'python' / 'python.exe', win(app / 'tools' / 'make_game.py'),
|
||||
*[win(a) if isinstance(a, Path) else a for a in args]], env=env)
|
||||
else:
|
||||
say('note: no prebuilt runtime (it is built on Windows, or with --wine): players compile runtime/ themselves')
|
||||
return
|
||||
if not (app / 'lib' / 'libnx2navirt.a').is_file():
|
||||
sys.exit('the prebuilt runtime was not written')
|
||||
|
||||
|
||||
def build_native(work, cc, windres):
|
||||
from launcher.ui import art
|
||||
work.mkdir(parents=True, exist_ok=True)
|
||||
(work / 'nx2navi.ico').write_bytes(art.triforce_ico())
|
||||
(work / 'nx2navi.ico').write_bytes(art.navi_ico())
|
||||
shutil.copy2(ROOT / 'launcher' / 'native' / 'app.manifest', work / 'app.manifest')
|
||||
shutil.copy2(ROOT / 'launcher' / 'native' / 'res.rc', work / 'res.rc')
|
||||
res = (work / 'res.o').resolve()
|
||||
@@ -261,6 +286,8 @@ def main():
|
||||
ap.add_argument('--notes', type=Path, help='release notes (markdown) for the manifest and the GitHub release')
|
||||
ap.add_argument('--lock', type=Path, help='a toolchain.lock.json to reuse instead of resolving today\'s packages')
|
||||
ap.add_argument('--version', help='the release version instead of VERSION\'s (test releases)')
|
||||
ap.add_argument('--wine', action='store_true', help='on Linux: build the prebuilt runtime with MSYS2\'s clang '
|
||||
'through Wine')
|
||||
ap.add_argument('--publish', action='store_true', help='upload as GitHub release v<version> with gh')
|
||||
ap.add_argument('--draft', action='store_true', help='with --publish: as a draft')
|
||||
a = ap.parse_args()
|
||||
@@ -273,12 +300,15 @@ def main():
|
||||
shutil.rmtree(out, ignore_errors=True)
|
||||
folder = out / 'NX2navi'
|
||||
t0 = time.time()
|
||||
lock = json.loads(a.lock.read_text(encoding='utf-8')) if a.lock else toolchain.make_lock()
|
||||
say(f'release {build}: toolchain lock with {len(lock["packages"])} packages')
|
||||
lock = (json.loads(a.lock.read_text(encoding='utf-8')) if a.lock
|
||||
else toolchain.make_lock(fast_repo=toolchain.FAST_REPO))
|
||||
say(f'release {build}: toolchain lock with {len(lock["packages"])} packages'
|
||||
+ (f' (CLANG64 clang left out: {lock["fast_skipped"]})' if lock.get('fast_skipped') else ''))
|
||||
build_app(folder / 'app' / ver, ver, a.xlate, a.feed, lock, a.ffx)
|
||||
(folder / 'app' / 'current.txt').write_text(ver + '\n', encoding='utf-8')
|
||||
build_python(folder / 'python')
|
||||
cc, windres = native_compiler(lock)
|
||||
build_runtime_lib(folder, ver, out, a.wine)
|
||||
stub, setup_stub = build_native(out / 'native', cc, windres)
|
||||
shutil.copy2(stub, folder / 'NX2navi.exe')
|
||||
check_folder(folder)
|
||||
|
||||
427
tools/menu_preview.cpp
Normal file
427
tools/menu_preview.cpp
Normal file
@@ -0,0 +1,427 @@
|
||||
// Draws the in-game settings menu and the performance overlay without the game, for looking at their design: the
|
||||
// menu's own row code (runtime/ui/option_rows.cpp), its shapes (ui/menu_skin.cpp), the overlay panel
|
||||
// (ui/perf_panel.cpp) and the text catalog (ui_text.c), with the few runtime calls they make answered here, laid out as
|
||||
// menu.cpp layout_menu lays out the window. Dear ImGui's draw lists are rasterised on the CPU into an RGBA image.
|
||||
// Without an art folder the window has its plain border in place of the game's ornamented frame and every box is plain
|
||||
// shapes; with one (tools/menu_preview.py --romfs) the boxes use the game's textures as in the game. The given font
|
||||
// stands in for the game's either way.
|
||||
//
|
||||
// tools/menu_preview.py builds and runs it:
|
||||
// menu_preview <font.ttf> <scene: menu | overlay | boot> <width> <height> <out.rgba> [art folder]
|
||||
#include <math.h>
|
||||
#include <stdarg.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <time.h>
|
||||
|
||||
#include <vector>
|
||||
|
||||
#include "imgui.h"
|
||||
#include "imgui_internal.h"
|
||||
|
||||
#include "ui/menu_art.h"
|
||||
#include "ui/menu_skin.h"
|
||||
#include "ui/option_rows.h"
|
||||
#include "ui/perf_panel.h"
|
||||
|
||||
extern "C" {
|
||||
#include "cfg.h"
|
||||
#include "menu_sound.h"
|
||||
#include "ui_text.h"
|
||||
}
|
||||
|
||||
// ---- what the rows and the catalog ask of the runtime: a few settings with the menu's values and defaults
|
||||
|
||||
static const CfgMeta kMeta[] = {
|
||||
{"FULLSCREEN", "1", "0|1", 1}, {"WINDOW_SIZE", "", NULL, 1}, {"VK_PRESENT", "mailbox", "mailbox|fifo|immediate", 1},
|
||||
{"GPU_INDEX", "", NULL, 0}, {"PERF_OVERLAY", "off", "off|fps|full", 1},
|
||||
{"GAME_ASPECT", "16:9", "16:9|window|21:9|32:9", 0}, {"RENDER_SCALE", "", NULL, 0},
|
||||
{"UPSCALER", "bilinear", "bilinear|fsr", 0}, {"SHARPNESS", "50", "0..100", 1},
|
||||
{"ASPECT", "fit", "fit|stretch", 1}, {"MENU_LANG", "", NULL, 1},
|
||||
};
|
||||
static const struct { const char *key, *value; } kSet[] = {{"VK_PRESENT", "mailbox"}, {"PERF_OVERLAY", "full"}};
|
||||
|
||||
static const char *strip(const char *name) { return strncmp(name, "NX2NAVI_", 8) ? name : name + 8; }
|
||||
|
||||
extern "C" {
|
||||
const CfgMeta *cfg_meta(const char *key) {
|
||||
for (const CfgMeta &m : kMeta)
|
||||
if (!strcasecmp(m.key, strip(key))) return &m;
|
||||
return NULL;
|
||||
}
|
||||
const CfgMeta *cfg_meta_at(int i) { return i >= 0 && i < (int)(sizeof kMeta / sizeof *kMeta) ? &kMeta[i] : NULL; }
|
||||
const char *cfg_get(const char *name) {
|
||||
for (auto &s : kSet)
|
||||
if (!strcasecmp(s.key, strip(name))) return s.value;
|
||||
return NULL;
|
||||
}
|
||||
const char *cfg_value(const char *name) {
|
||||
const char *v = cfg_get(name);
|
||||
const CfgMeta *m = cfg_meta(name);
|
||||
return v ? v : m ? m->def : "";
|
||||
}
|
||||
int cfg_int(const char *name) { return atoi(cfg_value(name)); }
|
||||
double cfg_double(const char *name) { return atof(cfg_value(name)); }
|
||||
int cfg_bool(const char *name) { return strcmp(cfg_value(name), "0") != 0; }
|
||||
int cfg_from_env(const char *) { return 0; }
|
||||
void cfg_set(const char *, const char *) {}
|
||||
int cfg_save(void) { return 1; }
|
||||
int cfg_reset(const char *const *, int) { return 0; }
|
||||
unsigned cfg_version(void) { return 1; }
|
||||
void menu_sound_play(MenuSound) {}
|
||||
const char *ov_language(void) { return "en-US"; }
|
||||
const char *plat_ui_language(void) { return "en"; }
|
||||
uint64_t plat_now_ns(void) {
|
||||
struct timespec t;
|
||||
clock_gettime(CLOCK_MONOTONIC, &t);
|
||||
return (uint64_t)t.tv_sec * 1000000000ull + (uint64_t)t.tv_nsec;
|
||||
}
|
||||
void rt_log(const char *, ...) {}
|
||||
}
|
||||
|
||||
// ---- the game's art, from files menu_preview.py decoded: NN.art, a width and a height (32-bit) and the RGBA pixels of
|
||||
// the MenuArt NN, with the smaller levels the game uploads
|
||||
|
||||
struct Level {
|
||||
int w, h;
|
||||
std::vector<uint8_t> px;
|
||||
};
|
||||
static std::vector<Level> g_art[MENU_ART_COUNT];
|
||||
|
||||
static void art_load(const char *dir) {
|
||||
for (int i = 0; i < MENU_ART_COUNT; i++) {
|
||||
char path[1024];
|
||||
snprintf(path, sizeof path, "%s/%02d.art", dir, i);
|
||||
FILE *f = fopen(path, "rb");
|
||||
if (!f) continue;
|
||||
uint32_t size[2];
|
||||
if (fread(size, 4, 2, f) == 2) {
|
||||
Level top{(int)size[0], (int)size[1], std::vector<uint8_t>((size_t)size[0] * size[1] * 4)};
|
||||
if (fread(top.px.data(), 1, top.px.size(), f) == top.px.size()) {
|
||||
skin_art_prepare(i, top.px.data(), size[0], size[1]);
|
||||
g_art[i].push_back(std::move(top));
|
||||
while (g_art[i].size() < 6 && (g_art[i].back().w > 1 || g_art[i].back().h > 1)) {
|
||||
const Level &last = g_art[i].back();
|
||||
Level next{last.w > 1 ? last.w / 2 : 1, last.h > 1 ? last.h / 2 : 1, {}};
|
||||
next.px.resize((size_t)next.w * next.h * 4);
|
||||
skin_art_halve(last.px.data(), last.w, last.h, next.px.data(), next.w, next.h);
|
||||
g_art[i].push_back(std::move(next));
|
||||
}
|
||||
}
|
||||
}
|
||||
fclose(f);
|
||||
}
|
||||
}
|
||||
|
||||
// Texture ids in the draw lists: 1 is the font atlas, 2 and up the art.
|
||||
extern "C" int menu_art(MenuArt id, ImTextureID *tex, float *w, float *h) {
|
||||
if ((unsigned)id >= MENU_ART_COUNT || g_art[id].empty()) return 0;
|
||||
*tex = (ImTextureID)(intptr_t)(2 + id);
|
||||
if (w) *w = (float)g_art[id][0].w;
|
||||
if (h) *h = (float)g_art[id][0].h;
|
||||
return 1;
|
||||
}
|
||||
|
||||
// ---- the scenes
|
||||
|
||||
static void backdrop(float w, float h) {
|
||||
// a dim field and sky, as the game shows behind the menu; then the menu's own dimming (menu.cpp layout_menu)
|
||||
ImDrawList *bg = ImGui::GetBackgroundDrawList();
|
||||
bg->AddRectFilledMultiColor(ImVec2(0, 0), ImVec2(w, h * 0.55f), IM_COL32(62, 70, 74, 255), IM_COL32(70, 76, 78, 255),
|
||||
IM_COL32(52, 64, 52, 255), IM_COL32(48, 60, 50, 255));
|
||||
bg->AddRectFilledMultiColor(ImVec2(0, h * 0.55f), ImVec2(w, h), IM_COL32(52, 64, 52, 255), IM_COL32(48, 60, 50, 255),
|
||||
IM_COL32(30, 38, 30, 255), IM_COL32(28, 34, 28, 255));
|
||||
}
|
||||
|
||||
static void triforce(ImDrawList *dl, float x, float y, float h, ImU32 col) {
|
||||
float w = h * 1.1547f;
|
||||
ImVec2 t(x, y), l(x - w / 2, y + h), r(x + w / 2, y + h), ml(x - w / 4, y + h / 2), mr(x + w / 4, y + h / 2), mb(x, y + h);
|
||||
dl->AddTriangleFilled(t, mr, ml, col);
|
||||
dl->AddTriangleFilled(ml, mb, l, col);
|
||||
dl->AddTriangleFilled(mr, r, mb, col);
|
||||
}
|
||||
|
||||
static void scene_menu(float w, float h, float scale, ImFont *title, bool first) {
|
||||
backdrop(w, h);
|
||||
ImGui::GetBackgroundDrawList()->AddRectFilled(ImVec2(0, 0), ImVec2(w, h), IM_COL32(10, 6, 2, 150));
|
||||
float ww = fminf(w * 0.88f, 1400 * scale), wh = fminf(h * 0.86f, 820 * scale);
|
||||
ImGui::SetNextWindowPos(ImVec2(w * 0.5f, h * 0.5f), ImGuiCond_Always, ImVec2(0.5f, 0.5f));
|
||||
ImGui::SetNextWindowSize(ImVec2(ww, wh), ImGuiCond_Always);
|
||||
ImTextureID frame_tex;
|
||||
const bool framed = menu_art(MENU_ART_FRAME_DECO, &frame_tex, nullptr, nullptr);
|
||||
if (framed) ImGui::PushStyleVar(ImGuiStyleVar_WindowBorderSize, 0);
|
||||
ImGui::Begin("##menu", nullptr, ImGuiWindowFlags_NoDecoration | ImGuiWindowFlags_NoMove | ImGuiWindowFlags_NoSavedSettings);
|
||||
if (framed) ImGui::PopStyleVar();
|
||||
const ImVec2 win_p = ImGui::GetWindowPos(), win_s = ImGui::GetWindowSize();
|
||||
if (framed) { // as menu.cpp layout_menu draws it
|
||||
ImDrawList *fdl = ImGui::GetWindowDrawList();
|
||||
const float ox = 15 * scale, oy = 30 * scale;
|
||||
fdl->PushClipRectFullScreen();
|
||||
for (int turned = 0; turned < 2; turned++)
|
||||
menu_art_draw9_turned(fdl, MENU_ART_FRAME_DECO, ImVec2(win_p.x - ox, win_p.y - oy),
|
||||
ImVec2(win_p.x + win_s.x + ox, win_p.y + win_s.y + oy), 96 * scale, 0.2f,
|
||||
IM_COL32(236, 196, 112, 235), 0, turned);
|
||||
fdl->PopClipRect();
|
||||
}
|
||||
const ImVec4 gold(0.93f, 0.76f, 0.38f, 1);
|
||||
ImGui::PushFont(title);
|
||||
ImVec2 at = ImGui::GetCursorScreenPos();
|
||||
float th = ImGui::GetTextLineHeight();
|
||||
triforce(ImGui::GetWindowDrawList(), at.x + th * 0.6f, at.y + th * 0.02f, th * 0.96f, ImGui::GetColorU32(gold));
|
||||
ImGui::SetCursorScreenPos(ImVec2(at.x + th * 1.5f, at.y));
|
||||
ImGui::TextColored(gold, "Ocarina of Time");
|
||||
ImGui::SameLine();
|
||||
ImGui::TextDisabled("|");
|
||||
ImGui::SameLine();
|
||||
ImGui::TextUnformatted(menu_text(UI_MENU_TITLE));
|
||||
const float title_end = ImGui::GetItemRectMax().x;
|
||||
ImGui::PopFont();
|
||||
const char *version = "NX2navi 0.5.0 - game 1.1.1";
|
||||
float version_x = ImGui::GetWindowWidth() - ImGui::GetStyle().WindowPadding.x - ImGui::CalcTextSize(version).x;
|
||||
if (ImGui::GetWindowPos().x + version_x >= title_end + ImGui::GetStyle().ItemSpacing.x * 2) {
|
||||
ImGui::SameLine(version_x);
|
||||
ImGui::TextDisabled("%s", version);
|
||||
}
|
||||
const ImGuiStyle &st = ImGui::GetStyle();
|
||||
float bottom = ImGui::GetFrameHeight() + st.ItemSpacing.y * 2;
|
||||
static const UiTextId tabs[] = {UI_MENU_TAB_SCREEN, UI_MENU_TAB_GAME, UI_MENU_TAB_SOUND, UI_MENU_TAB_CONTROLS};
|
||||
if (ImGui::BeginTabBar("##tabs", ImGuiTabBarFlags_NoTabListScrollingButtons)) {
|
||||
for (int k = 0; k < 4; k++) {
|
||||
char label[96];
|
||||
snprintf(label, sizeof label, " %s ###tab%d", menu_text(tabs[k]), k);
|
||||
bool shown = skin_tab_item(label, menu_text(tabs[k]), 0, scale, title);
|
||||
if (!shown) continue;
|
||||
ImGui::Spacing();
|
||||
opt_reserve_bottom(bottom);
|
||||
if (first) opt_focus_first();
|
||||
// the Screen tab's first rows, as menu.cpp tab_screen shows them
|
||||
opt_begin("##screen", scale);
|
||||
opt_toggle("NX2NAVI_FULLSCREEN", UI_MENU_FULLSCREEN, UI_MENU_FULLSCREEN_DESC);
|
||||
static const char *const sizes[] = {"", "1920x1080"};
|
||||
static const char *const size_names[] = {"1920 \xC3\x97 1080", "1920 \xC3\x97 1080"};
|
||||
opt_choice_list("NX2NAVI_WINDOW_SIZE", UI_MENU_WINDOW_SIZE, sizes, size_names, 2, UI_MENU_WINDOW_SIZE_DESC, 0);
|
||||
opt_toggle_value("NX2NAVI_VK_PRESENT", UI_MENU_PRESENT_VSYNC, UI_MENU_PRESENT_DESC, "mailbox", "immediate");
|
||||
static const char *const gpus[] = {""};
|
||||
static const char *const gpu_names[] = {"NVIDIA GeForce RTX 3050 Ti Laptop GPU"}; // a long name
|
||||
opt_choice_list("NX2NAVI_GPU_INDEX", UI_MENU_GPU, gpus, gpu_names, 1, UI_MENU_GPU_DESC, 0);
|
||||
static const UiTextId perf[] = {UI_MENU_PERF_OFF, UI_MENU_PERF_FPS, UI_MENU_PERF_FULL};
|
||||
opt_choice("NX2NAVI_PERF_OVERLAY", UI_MENU_PERF_OVERLAY, perf, 3, UI_MENU_PERF_OVERLAY_DESC);
|
||||
opt_group(UI_MENU_GRAPHICS);
|
||||
static const UiTextId wide[] = {UI_MENU_GAME_ASPECT_16_9, UI_MENU_GAME_ASPECT_WINDOW, UI_MENU_GAME_ASPECT_21_9,
|
||||
UI_MENU_GAME_ASPECT_32_9};
|
||||
opt_choice("NX2NAVI_GAME_ASPECT", UI_MENU_GAME_ASPECT, wide, 4, UI_MENU_GAME_ASPECT_DESC);
|
||||
opt_slider("NX2NAVI_SHARPNESS", UI_MENU_SHARPNESS, 0, 100, UI_MENU_SHARPNESS_DESC, 5, "%.0f %%");
|
||||
opt_end();
|
||||
ImGui::EndTabItem();
|
||||
}
|
||||
ImGui::EndTabBar();
|
||||
}
|
||||
ImGui::Spacing();
|
||||
float y = ImGui::GetCursorPosY();
|
||||
{
|
||||
float sy = win_p.y + y, fh = ImGui::GetFrameHeight();
|
||||
menu_art_draw(ImGui::GetWindowDrawList(), MENU_ART_GUIDE_BAR, ImVec2(win_p.x + st.WindowPadding.x, sy),
|
||||
ImVec2(win_p.x + win_s.x - st.WindowPadding.x, sy + fh), IM_COL32(96, 70, 36, 170));
|
||||
}
|
||||
const char *cont = menu_text(UI_MENU_CONTINUE), *restart = menu_text(UI_MENU_RESTART_YES), *quit = menu_text(UI_MENU_QUIT);
|
||||
float bw = fmaxf(fmaxf(ImGui::CalcTextSize(cont).x, ImGui::CalcTextSize(restart).x), ImGui::CalcTextSize(quit).x) + st.FramePadding.x * 4;
|
||||
float gap = st.ItemSpacing.x + skin_cursor_reach(scale);
|
||||
ImGui::SetCursorPos(ImVec2(fmaxf(st.WindowPadding.x, ImGui::GetWindowWidth() - st.WindowPadding.x - bw * 3 - gap * 2), y));
|
||||
skin_widget_button(cont, ImVec2(bw, 0), scale);
|
||||
ImGui::SameLine(0, gap);
|
||||
skin_widget_button(restart, ImVec2(bw, 0), scale);
|
||||
ImGui::SameLine(0, gap);
|
||||
skin_widget_button(quit, ImVec2(bw, 0), scale);
|
||||
ImGui::End();
|
||||
}
|
||||
|
||||
static void scene_overlay(float w, float h, float scale) {
|
||||
backdrop(w, h);
|
||||
static PerfPanel p; // the numbers of the screenshot the overlay was redesigned from
|
||||
PerfStats &s = p.s;
|
||||
s.fps = 60, s.frame_ms_avg = 16.7f, s.frame_ms_max = 20.2f, s.low1_fps = 47, s.low01_fps = 8;
|
||||
for (int i = 0; i < PERF_HISTORY; i++) s.frame_ms[i] = 16.7f + 1.4f * sinf(i * 0.7f) + (i % 17 == 0 ? 3.5f : 0);
|
||||
snprintf(s.cpu_name, sizeof s.cpu_name, "AMD Ryzen 7 5800X3D 8-Core Processor");
|
||||
s.cpu_total_pct = 50, s.cpu_game_pct = 44, s.cpu_temp_c = -1;
|
||||
s.ram_used_mb = 23.0f * 1024, s.ram_total_mb = 31.9f * 1024, s.ram_game_mb = 8.6f * 1024;
|
||||
snprintf(s.gpu_name, sizeof s.gpu_name, "AMD Radeon RX 7900 XT");
|
||||
s.gpu_pct = 56, s.gpu_temp_c = 69, s.gpu_power_w = 107, s.gpu_clock_mhz = 1312, s.vram_used_mb = 5.3f * 1024,
|
||||
s.vram_total_mb = 20.0f * 1024;
|
||||
s.replay_ms = 0.60f, s.gpu_ms = 1.28f, s.pipe_wait_ms = 0, s.xlate_ms = 0, s.tex_uploads = 0;
|
||||
p.version = "NX2navi 0.5.0, game 1.1.1";
|
||||
snprintf(p.picture[0], 64, "1920x1080");
|
||||
snprintf(p.picture[1], 64, "2560x1440");
|
||||
snprintf(p.picture[3], 64, "bilinear");
|
||||
snprintf(p.picture[4], 64, "60 FPS");
|
||||
snprintf(p.picture[5], 64, "VSync");
|
||||
p.scene_known = false, p.fg_fps = 60, p.under_menu = false;
|
||||
perf_panel_draw(&p, w, scale);
|
||||
}
|
||||
|
||||
static void scene_boot(float w, float h, float scale, ImFont *title) {
|
||||
// the boot screen's box over the black the window shows then (menu.cpp menu_boot_frame), half way through the
|
||||
// pipelines
|
||||
ImGui::GetBackgroundDrawList()->AddRectFilled(ImVec2(0, 0), ImVec2(w, h), IM_COL32(0, 0, 0, 255));
|
||||
char step[160];
|
||||
snprintf(step, sizeof step, ui_text(UI_BOOT_PIPELINES), 1603LL, 3235LL);
|
||||
skin_boot_panel(ImGui::GetForegroundDrawList(), title, title->FontSize, ImGui::GetFont(), ImGui::GetFontSize(),
|
||||
ui_text(UI_BOOT_TITLE), step, ui_text(UI_BOOT_HINT), 1603.0f / 3235.0f, 0, w, h, scale);
|
||||
}
|
||||
|
||||
// ---- the rasteriser: every triangle of the draw lists, its colour times the font atlas, blended over the image
|
||||
|
||||
struct Image {
|
||||
int w, h;
|
||||
std::vector<float> px; // straight RGBA, 0..1
|
||||
};
|
||||
|
||||
// Twice the signed area of (e0, e1, p), computed with the edge's ends in one fixed order, so the two triangles that share
|
||||
// an edge get exactly opposite values for a pixel and none is shaded twice or left out along it.
|
||||
static float edge(ImVec2 e0, ImVec2 e1, float px, float py) {
|
||||
bool swap = e0.x > e1.x || (e0.x == e1.x && e0.y > e1.y);
|
||||
if (swap) {
|
||||
ImVec2 t = e0;
|
||||
e0 = e1, e1 = t;
|
||||
}
|
||||
float v = (e0.x - px) * (e1.y - py) - (e0.y - py) * (e1.x - px);
|
||||
return swap ? -v : v;
|
||||
}
|
||||
|
||||
// The atlas at (u, v), bilinear, clamped to its edges; 0..1.
|
||||
static void sample(const uint8_t *atlas, int aw, int ah, float u, float v, float out[4]) {
|
||||
float x = u * aw - 0.5f, y = v * ah - 0.5f;
|
||||
int x0 = (int)floorf(x), y0 = (int)floorf(y);
|
||||
float fx = x - x0, fy = y - y0;
|
||||
for (int k = 0; k < 4; k++) out[k] = 0;
|
||||
for (int j = 0; j < 2; j++)
|
||||
for (int i = 0; i < 2; i++) {
|
||||
int xx = x0 + i < 0 ? 0 : x0 + i >= aw ? aw - 1 : x0 + i, yy = y0 + j < 0 ? 0 : y0 + j >= ah ? ah - 1 : y0 + j;
|
||||
float wgt = (i ? fx : 1 - fx) * (j ? fy : 1 - fy);
|
||||
const uint8_t *t = atlas + 4 * ((size_t)yy * aw + xx);
|
||||
for (int k = 0; k < 4; k++) out[k] += wgt * t[k] / 255.0f;
|
||||
}
|
||||
}
|
||||
|
||||
// The art at (u, v) at level of detail lod, trilinear between its levels, as the game's sampler reads it.
|
||||
static void sample_art(const std::vector<Level> &levels, float u, float v, float lod, float out[4]) {
|
||||
const float top = (float)levels.size() - 1;
|
||||
lod = lod < 0 ? 0 : lod > top ? top : lod;
|
||||
int l0 = (int)floorf(lod), l1 = l0 + 1 < (int)levels.size() ? l0 + 1 : l0;
|
||||
float a[4], b[4], f = lod - l0;
|
||||
sample(levels[l0].px.data(), levels[l0].w, levels[l0].h, u, v, a);
|
||||
sample(levels[l1].px.data(), levels[l1].w, levels[l1].h, u, v, b);
|
||||
for (int k = 0; k < 4; k++) out[k] = a[k] + (b[k] - a[k]) * f;
|
||||
}
|
||||
|
||||
static void raster(const ImDrawData *dd, Image &img, const uint8_t *atlas, int aw, int ah) {
|
||||
for (int l = 0; l < dd->CmdListsCount; l++) {
|
||||
const ImDrawList *dl = dd->CmdLists[l];
|
||||
const ImDrawVert *vb = dl->VtxBuffer.Data;
|
||||
const ImDrawIdx *ib = dl->IdxBuffer.Data;
|
||||
for (const ImDrawCmd &cmd : dl->CmdBuffer) {
|
||||
if (cmd.UserCallback) continue;
|
||||
int cx0 = (int)fmaxf(0, cmd.ClipRect.x), cy0 = (int)fmaxf(0, cmd.ClipRect.y);
|
||||
int cx1 = (int)fminf((float)img.w, cmd.ClipRect.z), cy1 = (int)fminf((float)img.h, cmd.ClipRect.w);
|
||||
const intptr_t tex_id = (intptr_t)cmd.GetTexID();
|
||||
const std::vector<Level> *art = tex_id >= 2 && tex_id < 2 + MENU_ART_COUNT ? &g_art[tex_id - 2] : nullptr;
|
||||
for (unsigned i = 0; i + 2 < cmd.ElemCount; i += 3) {
|
||||
const ImDrawVert &a = vb[cmd.VtxOffset + ib[cmd.IdxOffset + i]], &b = vb[cmd.VtxOffset + ib[cmd.IdxOffset + i + 1]],
|
||||
&c = vb[cmd.VtxOffset + ib[cmd.IdxOffset + i + 2]];
|
||||
float area = (b.pos.x - a.pos.x) * (c.pos.y - a.pos.y) - (b.pos.y - a.pos.y) * (c.pos.x - a.pos.x);
|
||||
if (fabsf(area) < 1e-6f) continue;
|
||||
// the top-left rule: a pixel centre on an edge two triangles share is shaded by one of them only
|
||||
// (edge k is the one opposite vertex k, run in the triangle's own winding)
|
||||
const ImVec2 *vtx[3] = {&a.pos, &b.pos, &c.pos};
|
||||
bool top_left[3];
|
||||
for (int k = 0; k < 3; k++) {
|
||||
ImVec2 e0 = *vtx[(k + 1) % 3], e1 = *vtx[(k + 2) % 3];
|
||||
float dx = (e1.x - e0.x) * (area > 0 ? 1 : -1), dy = (e1.y - e0.y) * (area > 0 ? 1 : -1);
|
||||
top_left[k] = (dy == 0 && dx < 0) || dy > 0;
|
||||
}
|
||||
int x0 = (int)fmaxf((float)cx0, floorf(fminf(a.pos.x, fminf(b.pos.x, c.pos.x))));
|
||||
int x1 = (int)fminf((float)cx1, ceilf(fmaxf(a.pos.x, fmaxf(b.pos.x, c.pos.x))));
|
||||
int y0 = (int)fmaxf((float)cy0, floorf(fminf(a.pos.y, fminf(b.pos.y, c.pos.y))));
|
||||
int y1 = (int)fminf((float)cy1, ceilf(fmaxf(a.pos.y, fmaxf(b.pos.y, c.pos.y))));
|
||||
float lod = 0;
|
||||
if (art && !art->empty()) { // texels per screen pixel along the steeper axis
|
||||
const float tw = (float)(*art)[0].w, th = (float)(*art)[0].h;
|
||||
const float e1x = b.pos.x - a.pos.x, e1y = b.pos.y - a.pos.y, e2x = c.pos.x - a.pos.x, e2y = c.pos.y - a.pos.y;
|
||||
const float du1 = (b.uv.x - a.uv.x) * tw, dv1 = (b.uv.y - a.uv.y) * th, du2 = (c.uv.x - a.uv.x) * tw, dv2 = (c.uv.y - a.uv.y) * th;
|
||||
const float dudx = (du1 * e2y - du2 * e1y) / area, dvdx = (dv1 * e2y - dv2 * e1y) / area;
|
||||
const float dudy = (du2 * e1x - du1 * e2x) / area, dvdy = (dv2 * e1x - dv1 * e2x) / area;
|
||||
const float rho = fmaxf(sqrtf(dudx * dudx + dvdx * dvdx), sqrtf(dudy * dudy + dvdy * dvdy));
|
||||
lod = rho > 0 ? log2f(rho) : 0;
|
||||
}
|
||||
for (int y = y0; y < y1; y++)
|
||||
for (int x = x0; x < x1; x++) {
|
||||
float px = x + 0.5f, py = y + 0.5f;
|
||||
float w0 = edge(b.pos, c.pos, px, py) / area, w1 = edge(c.pos, a.pos, px, py) / area,
|
||||
w2 = edge(a.pos, b.pos, px, py) / area;
|
||||
if (w0 < 0 || w1 < 0 || w2 < 0) continue;
|
||||
if ((w0 == 0 && !top_left[0]) || (w1 == 0 && !top_left[1]) || (w2 == 0 && !top_left[2])) continue;
|
||||
float u = w0 * a.uv.x + w1 * b.uv.x + w2 * c.uv.x, v = w0 * a.uv.y + w1 * b.uv.y + w2 * c.uv.y;
|
||||
float col[4];
|
||||
for (int k = 0; k < 4; k++) {
|
||||
float ca = (a.col >> (8 * k)) & 255, cb = (b.col >> (8 * k)) & 255, cc = (c.col >> (8 * k)) & 255;
|
||||
col[k] = (w0 * ca + w1 * cb + w2 * cc) / 255.0f;
|
||||
}
|
||||
float t[4];
|
||||
if (art && !art->empty()) sample_art(*art, u, v, lod, t);
|
||||
else sample(atlas, aw, ah, u, v, t); // filtered, as the GPU samples the atlas's baked lines
|
||||
for (int k = 0; k < 4; k++) col[k] *= t[k];
|
||||
float *d = &img.px[4 * ((size_t)y * img.w + x)], sa = col[3];
|
||||
for (int k = 0; k < 3; k++) d[k] = col[k] * sa + d[k] * (1 - sa);
|
||||
d[3] = sa + d[3] * (1 - sa);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
if (argc != 6 && argc != 7) {
|
||||
fprintf(stderr, "usage: %s <font.ttf> <menu|overlay|boot> <width> <height> <out.rgba> [art folder]\n", argv[0]);
|
||||
return 2;
|
||||
}
|
||||
const char *font = argv[1], *scene = argv[2], *out = argv[5];
|
||||
if (argc == 7) art_load(argv[6]);
|
||||
int w = atoi(argv[3]), h = atoi(argv[4]);
|
||||
float scale = fmaxf(fminf((float)h, w * 0.68f), 720.0f) / 1080.0f; // as menu.cpp menu_scale at 100 % display scale
|
||||
ImGui::CreateContext();
|
||||
ImGuiIO &io = ImGui::GetIO();
|
||||
io.IniFilename = nullptr;
|
||||
io.DisplaySize = ImVec2((float)w, (float)h);
|
||||
io.DeltaTime = 1.0f / 60;
|
||||
io.ConfigFlags |= ImGuiConfigFlags_NavEnableKeyboard | ImGuiConfigFlags_NavEnableGamepad;
|
||||
io.Fonts->AddFontFromFileTTF(font, 26.0f * scale);
|
||||
ImFont *title = io.Fonts->AddFontFromFileTTF(font, 31.0f * scale);
|
||||
skin_atlas_add(io.Fonts);
|
||||
skin_atlas_fill(io.Fonts);
|
||||
uint8_t *atlas;
|
||||
int aw, ah;
|
||||
io.Fonts->GetTexDataAsRGBA32(&atlas, &aw, &ah);
|
||||
io.Fonts->SetTexID((ImTextureID)1);
|
||||
ImGuiStyle &st = ImGui::GetStyle();
|
||||
skin_style(st);
|
||||
st.ScaleAllSizes(scale);
|
||||
ImGui::SetCurrentFont(io.Fonts->Fonts[0]);
|
||||
for (int frame = 0; frame < 4; frame++) { // a few frames: sizes settle and the first row takes the cursor
|
||||
ImGui::NewFrame();
|
||||
if (!strcmp(scene, "menu")) scene_menu((float)w, (float)h, scale, title, frame == 0);
|
||||
else if (!strcmp(scene, "boot")) scene_boot((float)w, (float)h, scale, title);
|
||||
else scene_overlay((float)w, (float)h, scale);
|
||||
ImGui::Render();
|
||||
}
|
||||
Image img{w, h, std::vector<float>((size_t)w * h * 4, 0.0f)};
|
||||
raster(ImGui::GetDrawData(), img, atlas, aw, ah);
|
||||
FILE *f = fopen(out, "wb");
|
||||
if (!f) return 1;
|
||||
std::vector<uint8_t> bytes((size_t)w * h * 4);
|
||||
for (size_t i = 0; i < bytes.size(); i++) bytes[i] = (uint8_t)(fminf(1, fmaxf(0, img.px[i])) * 255 + 0.5f);
|
||||
fwrite(bytes.data(), 1, bytes.size(), f);
|
||||
fclose(f);
|
||||
ImGui::DestroyContext();
|
||||
return 0;
|
||||
}
|
||||
101
tools/menu_preview.py
Normal file
101
tools/menu_preview.py
Normal file
@@ -0,0 +1,101 @@
|
||||
"""Draw the in-game settings menu and the performance overlay without the game (tools/menu_preview.cpp).
|
||||
|
||||
python tools/menu_preview.py [--out build/menu-preview] [--font FILE] [--size 1920x1080] [--romfs FOLDER]
|
||||
|
||||
Builds tools/menu_preview.cpp with the menu's own row, shape, overlay and text code and Dear ImGui (host clang++, into
|
||||
build/menu-preview), runs it for each scene and writes menu.png, overlay.png and boot.png. Without --romfs no game
|
||||
files are read: the window has its plain border in place of the game's ornamented frame and the boxes are plain
|
||||
shapes. With --romfs, an extracted RomFS folder, the textures listed in runtime/ui/menu_art.cpp are decoded into --out
|
||||
and the boxes look as they do in the game (needs what tools/ui_textures.py needs). The font given (default: DejaVu
|
||||
Sans, else the first TrueType font found) stands in for the game's either way.
|
||||
"""
|
||||
import argparse
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
import zlib
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
RT = ROOT / 'runtime'
|
||||
SOURCES = [ROOT / 'tools' / 'menu_preview.cpp', RT / 'ui' / 'option_rows.cpp', RT / 'ui' / 'menu_skin.cpp',
|
||||
RT / 'ui' / 'perf_panel.cpp', RT / 'ui_text.c'] + sorted((RT / 'third_party' / 'imgui').glob('imgui*.cpp'))
|
||||
FONTS = ['/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf', 'C:/Windows/Fonts/segoeui.ttf', 'C:/Windows/Fonts/arial.ttf']
|
||||
|
||||
|
||||
def png(path, w, h, rgba):
|
||||
"""An 8-bit RGBA PNG of the raw pixels."""
|
||||
rows = b''.join(b'\0' + rgba[y * w * 4:(y + 1) * w * 4] for y in range(h))
|
||||
|
||||
def chunk(kind, data):
|
||||
return struct.pack('>I', len(data)) + kind + data + struct.pack('>I', zlib.crc32(kind + data) & 0xffffffff)
|
||||
Path(path).write_bytes(b'\x89PNG\r\n\x1a\n' + chunk(b'IHDR', struct.pack('>IIBBBBB', w, h, 8, 6, 0, 0, 0)) +
|
||||
chunk(b'IDAT', zlib.compress(rows, 6)) + chunk(b'IEND', b''))
|
||||
|
||||
|
||||
def build(out):
|
||||
cxx = shutil.which('clang++') or shutil.which('g++')
|
||||
if not cxx:
|
||||
sys.exit('needs clang++ or g++ on PATH')
|
||||
exe = out / ('menu_preview.exe' if os.name == 'nt' else 'menu_preview')
|
||||
objs = []
|
||||
for src in SOURCES:
|
||||
obj = out / (src.stem + '.o')
|
||||
if not obj.is_file() or obj.stat().st_mtime < max(src.stat().st_mtime, (RT / 'ui' / 'menu_skin.h').stat().st_mtime,
|
||||
(RT / 'ui' / 'menu_art.h').stat().st_mtime):
|
||||
lang = ['-x', 'c', '-std=gnu11'] if src.suffix == '.c' else ['-x', 'c++', '-std=gnu++17']
|
||||
cmd = [cxx, *lang, '-c', '-O2', '-w', '-I', str(RT), '-I', str(RT / 'third_party' / 'imgui'), str(src), '-o',
|
||||
str(obj)]
|
||||
if subprocess.run(cmd).returncode:
|
||||
sys.exit(f'{src.name} did not compile')
|
||||
objs.append(str(obj))
|
||||
if subprocess.run([cxx, *objs, '-o', str(exe), '-lm']).returncode:
|
||||
sys.exit('the link failed')
|
||||
return exe
|
||||
|
||||
|
||||
def decode_art(romfs, out):
|
||||
"""The textures of runtime/ui/menu_art.cpp kNames as out/art/NN.art for menu_preview.cpp; that folder."""
|
||||
sys.path.insert(0, str(ROOT / 'tools'))
|
||||
import ui_textures
|
||||
source = (RT / 'ui' / 'menu_art.cpp').read_text(encoding='utf-8')
|
||||
names = re.findall(r'"([^"]+)"', re.search(r'kNames\[MENU_ART_COUNT\] = \{(.*?)\};', source, re.S).group(1))
|
||||
data = ui_textures.Romfs(romfs).read(f'{ui_textures.ARCHIVES}/Common.Product.100.Nin_Ounce_NVN2.bntx.zs')
|
||||
textures = {t['name']: t for t in ui_textures.bntx_textures(data)}
|
||||
art = out / 'art'
|
||||
art.mkdir(parents=True, exist_ok=True)
|
||||
lib = ui_textures.astc_library(out)
|
||||
for i, name in enumerate(names):
|
||||
rgba = ui_textures.decode(textures[name], data, lib)
|
||||
(art / f'{i:02}.art').write_bytes(struct.pack('<II', rgba.shape[1], rgba.shape[0]) + rgba.tobytes())
|
||||
return art
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
ap.add_argument('--out', type=Path, default=ROOT / 'build' / 'menu-preview')
|
||||
ap.add_argument('--font', help='a TrueType font for the text')
|
||||
ap.add_argument('--size', default='1920x1080')
|
||||
ap.add_argument('--romfs', type=Path, help="an extracted RomFS folder: draw the game's textures")
|
||||
a = ap.parse_args()
|
||||
font = a.font or next((f for f in FONTS if Path(f).is_file()), None)
|
||||
if not font:
|
||||
sys.exit('no font found: give one with --font')
|
||||
w, h = (int(x) for x in a.size.lower().split('x'))
|
||||
a.out.mkdir(parents=True, exist_ok=True)
|
||||
exe = build(a.out)
|
||||
art = [str(decode_art(a.romfs, a.out))] if a.romfs else []
|
||||
for scene in ('menu', 'overlay', 'boot'):
|
||||
raw = a.out / f'{scene}.rgba'
|
||||
if subprocess.run([str(exe), font, scene, str(w), str(h), str(raw), *art]).returncode:
|
||||
sys.exit(f'the {scene} scene failed')
|
||||
png(a.out / f'{scene}.png', w, h, raw.read_bytes())
|
||||
raw.unlink()
|
||||
print(a.out / f'{scene}.png')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -1,6 +1,9 @@
|
||||
# Packages of the launcher release's bundled Python 3.14.9 (tools/make_launcher.py): what the launcher and
|
||||
# tools/make_game.py import. numpy: recomp/gen.py and the window's drawing (launcher/ui/art.py). lz4: NSO segments.
|
||||
# zstandard: tools/nup.py. zbic is not bundled: the launcher downloads it from PyPI on the player's PC.
|
||||
# zstandard: tools/nup.py. truststore: Windows' own certificate check for the launcher's downloads (launcher/net.py),
|
||||
# pinned to its wheel's sha256 too (uv checks the hashes a line gives). zbic is not bundled: the launcher downloads
|
||||
# it from PyPI on the player's PC.
|
||||
numpy==2.5.4
|
||||
lz4==4.4.5
|
||||
zstandard==0.25.0
|
||||
truststore==0.10.5 --hash=sha256:9aaaedaefaf06d8b206278cf8b5012bc897f485a874503501e12d776df78951c
|
||||
|
||||
245
tools/ui_textures.py
Normal file
245
tools/ui_textures.py
Normal file
@@ -0,0 +1,245 @@
|
||||
"""List and dump the game's interface textures, for choosing the art the settings menu draws (runtime/ui/menu_art.cpp).
|
||||
|
||||
python tools/ui_textures.py ROMFS [--out build/ui-textures] [--archive Common] [--match REGEX] [--sheet]
|
||||
|
||||
ROMFS is an extracted RomFS folder. Without --match it only lists; with it the matching textures become PNGs under
|
||||
--out, and --sheet also writes contact sheets of them (numbered; sheet.txt says which number is which). Reads files the
|
||||
way runtime/menu_assets.c does, and decodes ASTC with the runtime's own decoder (runtime/tex_astc.c, built into --out).
|
||||
Everything written is game data and stays under build/. Needs Python 3.14 (compression.zstd) and numpy.
|
||||
"""
|
||||
import argparse
|
||||
import ctypes
|
||||
import re
|
||||
import shutil
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
import zlib
|
||||
from compression import zstd
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
ARCHIVES = 'UI/Layout2Archive'
|
||||
# the second byte of a BNTX format: name, block size, bytes per block
|
||||
KINDS = {0x0b: ('RGBA8', 1, 4), 0x1a: ('BC1', 4, 8), 0x1b: ('BC2', 4, 16), 0x1c: ('BC3', 4, 16), 0x1d: ('BC4', 4, 8),
|
||||
0x1e: ('BC5', 4, 16), 0x20: ('BC7', 4, 16), 0x2d: ('ASTC4x4', 4, 16)}
|
||||
|
||||
|
||||
def u16(b, at): return struct.unpack_from('<H', b, at)[0]
|
||||
def u32(b, at): return struct.unpack_from('<I', b, at)[0]
|
||||
def u64(b, at): return struct.unpack_from('<Q', b, at)[0]
|
||||
|
||||
|
||||
def sarc_members(p):
|
||||
"""The members of a SARC archive as a name -> bytes dictionary."""
|
||||
hl, data = u16(p, 4), u32(p, 0xc)
|
||||
count, ents = u16(p, hl + 6), hl + 0xc
|
||||
names = ents + 16 * count + 8
|
||||
out = {}
|
||||
for i in range(count):
|
||||
e = ents + 16 * i
|
||||
at = names + (u32(p, e + 4) & 0xffff) * 4
|
||||
name = p[at:p.index(b'\0', at)].decode()
|
||||
out[name] = p[data + u32(p, e + 8):data + u32(p, e + 12)]
|
||||
return out
|
||||
|
||||
|
||||
class Romfs:
|
||||
def __init__(self, root):
|
||||
self.root = Path(root)
|
||||
self.dicts = None
|
||||
|
||||
def load_dicts(self):
|
||||
self.dicts = {}
|
||||
raw = zstd.decompress((self.root / 'Pack/ZsDic.pack.zs').read_bytes())
|
||||
for name, data in sarc_members(raw).items():
|
||||
d = zstd.ZstdDict(data)
|
||||
self.dicts[d.dict_id] = d
|
||||
|
||||
def read(self, rel):
|
||||
b = (self.root / rel).read_bytes()
|
||||
if not rel.endswith('.zs'):
|
||||
return b
|
||||
id_ = zstd.get_frame_info(b).dictionary_id
|
||||
if not id_:
|
||||
return zstd.decompress(b)
|
||||
if self.dicts is None:
|
||||
self.load_dicts()
|
||||
return zstd.decompress(b, zstd_dict=self.dicts[id_])
|
||||
|
||||
|
||||
def bntx_textures(b):
|
||||
"""Each texture of a BNTX file: name, format, size, block height, data offset and size."""
|
||||
out = []
|
||||
count, arr = u32(b, 0x24), u64(b, 0x28)
|
||||
for i in range(count):
|
||||
p = u64(b, arr + 8 * i)
|
||||
if b[p:p + 4] != b'BRTI':
|
||||
continue
|
||||
no = u64(b, p + 0x60)
|
||||
name = b[no + 2:no + 2 + u16(b, no)].decode()
|
||||
out.append({'name': name, 'fmt': u32(b, p + 0x1c), 'w': u32(b, p + 0x24), 'h': u32(b, p + 0x28),
|
||||
'th': u32(b, p + 0x34), 'size': u32(b, p + 0x50), 'data': u64(b, u64(b, p + 0x70))})
|
||||
return out
|
||||
|
||||
|
||||
def deswizzle(t, b, row, rows):
|
||||
"""The first level as tightly packed rows, in the Switch 2 GOB layout (runtime/tex_deswizzle.c tex_bl_copy)."""
|
||||
th = t['th']
|
||||
gobs, unit, block = (row + 63) // 64, 8 << th, 512 << th
|
||||
need = gobs * ((rows + unit - 1) // unit) * block
|
||||
bl = np.zeros(max(need, t['size']) // 16 + 1, dtype='V16')
|
||||
src = bytearray(bl.tobytes())
|
||||
src[:t['size']] = b[t['data']:t['data'] + t['size']]
|
||||
bl = np.frombuffer(bytes(src), dtype='V16')
|
||||
cols = (row + 15) // 16
|
||||
y = np.arange(rows)[:, None]
|
||||
x = (np.arange(cols) * 16)[None, :]
|
||||
at = (y // unit) * gobs * block + (x // 64) * block + ((y % unit) // 8) * 512 + \
|
||||
(x % 64 // 32) * 256 + (y % 8 // 4) * 128 + (x % 32 // 16) * 64 + (y % 4) * 16
|
||||
flat = np.frombuffer(bl[at // 16].tobytes(), dtype=np.uint8).reshape(rows, cols * 16)
|
||||
return flat[:, :row]
|
||||
|
||||
|
||||
def bc4(lin, wb, hb, w, h):
|
||||
"""White with the mask as alpha (runtime/menu_assets.c bc4_decode)."""
|
||||
blocks = lin.reshape(hb, wb, 8).astype(np.uint32)
|
||||
a, b = blocks[..., 0], blocks[..., 1]
|
||||
pal = np.zeros((hb, wb, 8), dtype=np.uint32)
|
||||
pal[..., 0], pal[..., 1] = a, b
|
||||
big = a > b
|
||||
for i in range(6):
|
||||
pal[..., 2 + i] = np.where(big, ((6 - i) * a + (i + 1) * b) // 7, 0)
|
||||
for i in range(4):
|
||||
pal[..., 2 + i] = np.where(big, pal[..., 2 + i], ((4 - i) * a + (i + 1) * b) // 5)
|
||||
pal[..., 6] = np.where(big, pal[..., 6], 0)
|
||||
pal[..., 7] = np.where(big, pal[..., 7], 255)
|
||||
bits = np.zeros((hb, wb), dtype=np.uint64)
|
||||
for i in range(6):
|
||||
bits |= blocks[..., 2 + i].astype(np.uint64) << np.uint64(8 * i)
|
||||
alpha = np.zeros((hb * 4, wb * 4), dtype=np.uint8)
|
||||
for i in range(16):
|
||||
sel = ((bits >> np.uint64(3 * i)) & np.uint64(7)).astype(np.intp)
|
||||
alpha[i // 4::4, i % 4::4] = np.take_along_axis(pal, sel[..., None], 2)[..., 0]
|
||||
rgba = np.full((h, w, 4), 255, dtype=np.uint8)
|
||||
rgba[..., 3] = alpha[:h, :w]
|
||||
return rgba
|
||||
|
||||
|
||||
def astc_library(out):
|
||||
"""The runtime's ASTC decoder as a library, built once into out."""
|
||||
dll = out / 'astc.dll'
|
||||
src = ROOT / 'runtime' / 'tex_astc.c'
|
||||
if not dll.is_file() or dll.stat().st_mtime < src.stat().st_mtime:
|
||||
cc = shutil.which('clang') or r'C:\msys64\ucrt64\bin\clang.exe'
|
||||
vma = Path(cc).resolve().parents[1] / 'include' / 'vma' # tex.h reaches the Vulkan allocator's header
|
||||
if subprocess.run([cc, '-shared', '-O2', '-static', '-w', '-I', str(ROOT / 'runtime'), '-I', str(vma), str(src), '-o',
|
||||
str(dll)]).returncode:
|
||||
sys.exit('tex_astc.c did not compile')
|
||||
lib = ctypes.CDLL(str(dll))
|
||||
lib.astc_decode_ldr.restype = None
|
||||
lib.astc_decode_ldr.argtypes = [ctypes.c_char_p, ctypes.c_uint32, ctypes.c_uint32, ctypes.c_uint32, ctypes.c_uint32,
|
||||
ctypes.c_bool, ctypes.c_void_p]
|
||||
return lib
|
||||
|
||||
|
||||
def decode(t, b, lib):
|
||||
kind = t['fmt'] >> 8
|
||||
if kind not in (0x0b, 0x1d, 0x2d):
|
||||
return None
|
||||
_, bs, bpb = KINDS[kind]
|
||||
w, h = t['w'], t['h']
|
||||
wb, hb = (w + bs - 1) // bs, (h + bs - 1) // bs
|
||||
lin = deswizzle(t, b, wb * bpb, hb)
|
||||
if kind == 0x1d:
|
||||
return bc4(lin, wb, hb, w, h)
|
||||
if kind == 0x0b:
|
||||
return lin.reshape(h, w, 4).copy()
|
||||
rgba = np.zeros((h, w, 4), dtype=np.uint8)
|
||||
lib.astc_decode_ldr(lin.tobytes(), w, h, 4, 4, (t['fmt'] & 0xff) == 6, rgba.ctypes.data)
|
||||
return rgba
|
||||
|
||||
|
||||
def png(path, a):
|
||||
h, w = a.shape[:2]
|
||||
rows = b''.join(b'\0' + a[y].tobytes() for y in range(h))
|
||||
|
||||
def chunk(kind, data):
|
||||
return struct.pack('>I', len(data)) + kind + data + struct.pack('>I', zlib.crc32(kind + data) & 0xffffffff)
|
||||
Path(path).write_bytes(b'\x89PNG\r\n\x1a\n' + chunk(b'IHDR', struct.pack('>IIBBBBB', w, h, 8, 6, 0, 0, 0)) +
|
||||
chunk(b'IDAT', zlib.compress(rows, 6)) + chunk(b'IEND', b''))
|
||||
|
||||
|
||||
DIGITS = ['111101101101111', '010110010010111', '111001111100111', '111001111001111', '101101111001001',
|
||||
'111100111001111', '111100111101111', '111001010010010', '111101111101111', '111101111001111']
|
||||
|
||||
|
||||
def label(sheet, x, y, n):
|
||||
for c in str(n):
|
||||
for j in range(5):
|
||||
for i in range(3):
|
||||
if DIGITS[int(c)][j * 3 + i] == '1':
|
||||
sheet[y + j * 2:y + j * 2 + 2, x + i * 2:x + i * 2 + 2] = (255, 255, 0, 255)
|
||||
x += 8
|
||||
|
||||
|
||||
def sheets(out, items, cell=220, cols=7, per=35):
|
||||
"""Contact sheets over a mid brown, so white masks and dark art both show."""
|
||||
lines = []
|
||||
for s in range(0, len(items), per):
|
||||
page = items[s:s + per]
|
||||
rows = (len(page) + cols - 1) // cols
|
||||
sheet = np.full((rows * cell, cols * cell, 4), (84, 70, 58, 255), dtype=np.uint8)
|
||||
for k, (name, a) in enumerate(page):
|
||||
n = s + k
|
||||
lines.append(f'{n} {name} {a.shape[1]}x{a.shape[0]}')
|
||||
step = max(1, max(a.shape[0], a.shape[1]) // (cell - 8) + (max(a.shape[:2]) % (cell - 8) > 0))
|
||||
t = a[::step, ::step]
|
||||
cy, cx = (k // cols) * cell + 4, (k % cols) * cell + 4
|
||||
al = t[..., 3:4].astype(np.float32) / 255
|
||||
bg = sheet[cy:cy + t.shape[0], cx:cx + t.shape[1], :3].astype(np.float32)
|
||||
sheet[cy:cy + t.shape[0], cx:cx + t.shape[1], :3] = (t[..., :3] * al + bg * (1 - al)).astype(np.uint8)
|
||||
label(sheet, cx, cy, n)
|
||||
png(out / f'sheet{s // per:02}.png', sheet)
|
||||
(out / 'sheet.txt').write_text('\n'.join(lines) + '\n')
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
ap.add_argument('romfs', type=Path)
|
||||
ap.add_argument('--out', type=Path, default=ROOT / 'build' / 'ui-textures')
|
||||
ap.add_argument('--archive', default='', help='only archives whose name contains this')
|
||||
ap.add_argument('--match', help='a regular expression: the textures to decode')
|
||||
ap.add_argument('--sheet', action='store_true', help='write contact sheets of what was decoded')
|
||||
a = ap.parse_args()
|
||||
fs = Romfs(a.romfs)
|
||||
a.out.mkdir(parents=True, exist_ok=True)
|
||||
lib = astc_library(a.out) if a.match else None
|
||||
decoded = []
|
||||
for f in sorted((a.romfs / ARCHIVES).glob('*.bntx.zs')):
|
||||
if a.archive not in f.name:
|
||||
continue
|
||||
b = fs.read(f'{ARCHIVES}/{f.name}')
|
||||
ts = bntx_textures(b)
|
||||
print(f'# {f.name}: {len(ts)} textures')
|
||||
for t in sorted(ts, key=lambda t: t['name']):
|
||||
kind = KINDS.get(t['fmt'] >> 8, (f"{t['fmt']:#x}",))[0]
|
||||
if not a.match:
|
||||
print(f"{t['name']} {kind} {t['w']}x{t['h']}")
|
||||
elif re.search(a.match, t['name']):
|
||||
rgba = decode(t, b, lib)
|
||||
if rgba is None:
|
||||
print(f"{t['name']}: format {kind} is not read by the runtime", file=sys.stderr)
|
||||
continue
|
||||
png(a.out / f"{t['name']}.png", rgba)
|
||||
decoded.append((t['name'], rgba))
|
||||
if a.match:
|
||||
print(f'{len(decoded)} textures written to {a.out}')
|
||||
if a.sheet and decoded:
|
||||
sheets(a.out, decoded)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Reference in New Issue
Block a user