Files
NX2navi/tools/make_kit.py
chrg6 61cd4784f8 Build the Linux kit against pinned Vulkan headers and without #embed, and try the compiler before the long steps.
A player's AppImage build of 0.7.1 stopped in its compile step after eight minutes of RomFS work: four
files did not compile, because the kit builds with the system's compiler and headers and theirs were
older than the runtime needs. vk_pipeline.c and vk_device.c name VK_KHR_pipeline_binary,
VK_KHR_dynamic_rendering_local_read and VK_KHR_shader_maximal_reconvergence, which their Vulkan headers
did not declare, and ui/menu.cpp takes its font in with #embed, which clang has from version 19.

- Vulkan headers: tools/vulkan_pin.py holds the release the other builds use (1.4.363, the archive and
  hash tools/make_release_linux.py already pinned, which now reads them from there). The AppImage carries
  its C headers in toolchain/vulkan-headers and builds against them whatever the system has; a kit or a
  source checkout uses the system's when they are at least that release and fetches the pinned ones
  otherwise (build_game.py vulkan_include). Their folder goes first on the include path of the runtime's
  sources only, so the lifted code's objects keep their keys, and a runtime object compiled before
  against the system's headers is taken over instead of compiled again. AppRun no longer asks for the
  system's Vulkan headers.
- #embed: tools/build_shaders.py also writes the font's bytes as runtime/ui/roboto_ttf.h (ignored by git,
  carried by a kit), and menu.cpp includes that where the compiler has no __has_embed. With #embed
  nothing changes.
- Before its long steps a Linux build that will compile now checks every runtime source that needs no
  header of the install with the compile step's own command (clang -fsyntax-only) and links an empty
  program against the game's libraries (make_game.py preflight): an old compiler, an old header or a
  missing development package stops it in the first seconds with the compiler's own errors. A failed
  compile names every file that failed, not only the three whose errors it prints.

Checked on Windows: tests/launcher_test.py passes with four new tests (117); the header of bytes and
#embed give the same 162,588 bytes, and both branches of menu.cpp compile; the early check passes on the
Windows file set with clang 23 and stops on files made to fail; the 34 local headers of release 363 are
the upstream tag's files by git blob id. Not run: anything on Linux, the pinned archive's download, a
compiler older than 23.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 12:42:17 -05:00

121 lines
6.9 KiB
Python

"""Assemble the build kit: what a player needs to build Ocarina of Time on their own PC with tools/make_game.py.
python tools/make_kit.py --xlate <dir with nx2navi_xlate.exe and .id> [--ffx amd_fidelityfx_vk.dll]
[--dlss nvngx_dlss.dll | --no-dlss] [--dlssg nx2navi_dlssg.dll [--streamline SDK]]
[--prompt-art FOLDER [FOLDER ...]] [--out build/kit] [--zip]
The kit holds this repository's runtime/ with its shaders compiled (tools/build_shaders.py builds the SPIR-V headers
here, so a player's PC needs no shader compiler; their sources go along), recomp/, the build tools, the shader
translator built from tools/xlate,
NVIDIA's DLSS library with its notice (tools/dlss_pin.py: downloaded from NVIDIA's DLSS SDK repository and checked,
or --dlss; not with --no-dlss), DLSS Frame Generation in bin/dlssg/ with --dlssg (nx2navi_dlssg.dll from
tools/build_dlssg.py beside NVIDIA Streamline's runtime, tools/streamline_pin.py: downloaded from NVIDIA's Streamline
SDK release and checked, or --streamline), AMD's FSR 3.1 library with --ffx, the button art with --prompt-art, and
README-BuildKit.md; no game code, data or anything generated from the game. check_content runs over the folder before
it is zipped.
--prompt-art names folders that hold XGamepad, P5Gamepad and Keyboard_Mouse, or those three folders themselves: the
pictures tools/pad_prompts.py reads go into the kit's prompt-art folder (tools/prompt_art.py art_files), and
make_game.py draws the Xbox, PlayStation, Steam Deck and keyboard and mouse prompts from them. Without it the kit builds the game
with the game's own prompts only.
"""
import argparse
import shutil
import sys
import zipfile
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
NAME = 'NX2navi-BuildKit'
TOOLS = ('make_game.py', 'nup.py', 'romfs_index.py', 'check_content.py', 'hostmem.py', 'prepare_shaders.py',
'shader_census.py', 'shader_list.py', 'zstd_compat.py', 'pad_prompts.py', 'prompt_art.py', 'button_icons.py', 'romfs_check.py',
'build_game.py', 'check_isa.py', 'build_shaders.py', 'make_dlaa_mask.py', 'make_mvobj_fs.py',
'pack_shader_cache.py', 'vulkan_pin.py') # make_game.py imports build_game's vma_include and vulkan_include on
# Linux for the VMA header and for Vulkan headers newer than the system's (which build_game takes from vulkan_pin), and
# build_game.py imports check_isa and build_shaders, which reads the two make_ scripts' shaders
TOP = ('LICENSE', 'LICENSE-ADDITIONS', 'THIRD_PARTY_NOTICES.md', 'VERSION', 'nx2navi.ini.example')
SKIP_DIRS = {'__pycache__', '.pytest_cache'}
SKIP_SUFFIXES = {'.o', '.obj', '.pyc', '.exe', '.dll', '.a', '.lib'}
def copy_tree(src, dst):
for p in sorted(src.rglob('*')):
rel = p.relative_to(src)
if any(part in SKIP_DIRS for part in rel.parts) or p.is_dir() or p.suffix.lower() in SKIP_SUFFIXES:
continue
(dst / rel).parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(p, dst / rel)
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument('--xlate', type=Path, required=True, help='folder with nx2navi_xlate.exe and nx2navi_xlate.id')
ap.add_argument('--ffx', type=Path, help="AMD's signed amd_fidelityfx_vk.dll (FidelityFX SDK 1.1.4, MIT) for FSR 3.1")
ap.add_argument('--dlss', type=Path, help="NVIDIA's nvngx_dlss.dll from disk instead of downloading it from "
"NVIDIA's DLSS SDK repository")
ap.add_argument('--no-dlss', action='store_true', help='leave NVIDIA DLSS out of the kit')
ap.add_argument('--dlssg', type=Path, metavar='DLL',
help='nx2navi_dlssg.dll (tools/build_dlssg.py) for DLSS Frame Generation')
ap.add_argument('--streamline', type=Path, metavar='SDK',
help='the Streamline SDK folder instead of downloading the SDK release with --dlssg')
ap.add_argument('--prompt-art', type=Path, nargs='+', metavar='FOLDER',
help='the button art: folders holding XGamepad, P5Gamepad and Keyboard_Mouse, or those folders')
ap.add_argument('--out', type=Path, default=ROOT / 'build' / 'kit')
ap.add_argument('--zip', action='store_true')
a = ap.parse_args()
sys.path.insert(0, str(ROOT / 'tools'))
import build_shaders
import dlss_pin
import prompt_art
import streamline_pin
if a.streamline and not a.dlssg:
sys.exit('--streamline goes with --dlssg')
if a.prompt_art:
prompt_art.art_files(a.prompt_art) # the art is all there before anything is downloaded
dlss = None if a.no_dlss else dlss_pin.resolve(a.dlss, ROOT / 'build' / 'dl')
sdk = streamline_pin.resolve(a.streamline, ROOT / 'build' / 'dl') if a.dlssg else None
kit = a.out / NAME
if kit.exists():
shutil.rmtree(kit)
kit.mkdir(parents=True)
build_shaders.build(ROOT / 'runtime') # the kit carries the shader headers: build the missing and the stale ones
for d in ('runtime', 'recomp'):
copy_tree(ROOT / d, kit / d)
(kit / 'tools').mkdir()
for t in TOOLS:
shutil.copy2(ROOT / 'tools' / t, kit / 'tools' / t)
for t in TOP:
if (ROOT / t).is_file():
shutil.copy2(ROOT / t, kit / t)
shutil.copy2(ROOT / 'docs' / 'README-BuildKit.md', kit / 'README-BuildKit.md')
(kit / 'bin').mkdir()
for n in ('nx2navi_xlate.exe', 'nx2navi_xlate.id'):
shutil.copy2(a.xlate / n, kit / 'bin' / n)
if a.ffx: # FSR 3.1 loads it from beside the game (runtime/vk_ffx.cpp); its licence is runtime/third_party/ffx_api
shutil.copy2(a.ffx, kit / 'bin' / 'amd_fidelityfx_vk.dll')
if dlss: # DLSS: NGX loads it from beside the game (runtime/vk_dlaa.c); make_game.py copies it there
dlss_pin.place(dlss, kit / 'bin')
if sdk: # DLSS Frame Generation (runtime/vk_dlssg.c): make_game.py place_dlssg copies it beside the game
streamline_pin.place(a.dlssg, sdk, kit / 'bin' / 'dlssg')
if a.prompt_art: # make_game.py draws the button prompts from it (its prompts step)
n, size = prompt_art.copy_art(a.prompt_art, kit / prompt_art.FOLDER)
print(f'button art: {n} pictures, {size / 1e6:.1f} MB')
import check_content
bad = [f'{p.relative_to(kit)}: {m}' for p in sorted(kit.rglob('*')) if p.is_file()
for m in check_content.problems_for(p, allow_large=p.suffix in ('.exe', '.dll'))] # the translator, the DLLs
if bad:
sys.exit('check_content refuses these kit files, not zipped:\n ' + '\n '.join(bad[:20]))
size = sum(p.stat().st_size for p in kit.rglob('*') if p.is_file())
print(f'{kit}: {size / 1e6:.1f} MB')
if a.zip:
z = a.out / f'{NAME}.zip'
with zipfile.ZipFile(z, 'w', zipfile.ZIP_DEFLATED, compresslevel=9) as zf:
for p in sorted(kit.rglob('*')):
if p.is_file():
zf.write(p, Path(NAME) / p.relative_to(kit))
print(f'{z}: {z.stat().st_size / 1e6:.1f} MB')
if __name__ == '__main__':
main()