From 89c453d7bb374975234e200f0f0a135e960fb6a8 Mon Sep 17 00:00:00 2001 From: epicmeme546 <178368684+epicmeme546@users.noreply.github.com> Date: Thu, 12 Nov 2026 08:11:57 -0500 Subject: [PATCH 1/4] Add a Mods screen to the launcher and build the mods' code patches into the launcher's game. Mods, on the home screen, opens the game's mods folder, adds a downloaded mod from its .zip, .7z or .rar file or its folder, switches each mod on or off, and sets the load order (mods_disabled and mods_order in nx2navi.ini, written as the game writes them). Each mod's description says what it changes, which other mod changes the same files and which of the two wins, and why the game leaves it off when it does. Before the game starts, Play puts the mods folder in order (launcher/mods.py prepare): an archive in it is unpacked into a folder of its name and kept, a mod whose files sit a folder or more down has them moved up, and a romfs, exefs or cheats folder put straight into the mods folder becomes a mod of its own. An archive that would write outside its folder is refused, and nothing is deleted. The launcher's build (tools/make_game.py) left the mods' code cheats and ExeFS patch files out, so those mods stayed off. The lift now turns them into patch sets first, as the setup does, with the files in the lift's key and build-cache/mod-code.json recording what it took up. Play builds the game again before it starts when an enabled mod brings code no build has seen. The shaders' key names the lift and the compile without the mods' code, so such a build compiles only what the patches change and leaves the shaders prepared. tools/mod_files.py is the one place the tools, the build and the launcher find a mod's parts. Tested: a zipped mod holding its own folder, put into a mods folder, was unpacked and moved up by the launcher's code, and the game served its file in place of the game's. The lifter took a code cheat and a pchtxt patch made for the game's main as patch sets. The launcher's and the setup's tests pass. Co-Authored-By: Claude Opus 5.5 --- docs/launcher.md | 36 ++- docs/releases/v0.9.0.md | 7 + launcher/jobs.py | 11 + launcher/mods.py | 505 ++++++++++++++++++++++++++++++++++++++++ launcher/ui/app.py | 277 +++++++++++++++++++++- launcher/ui/selftest.py | 30 ++- nx2navi.ini.example | 8 +- tests/launcher_test.py | 152 +++++++++++- tools/cheats.py | 23 +- tools/exefs_patches.py | 25 +- tools/make_game.py | 134 +++++++++-- tools/make_kit.py | 8 +- tools/make_launcher.py | 2 +- tools/make_release.py | 2 +- tools/mod_files.py | 99 ++++++++ 15 files changed, 1234 insertions(+), 85 deletions(-) create mode 100644 launcher/mods.py create mode 100644 tools/mod_files.py diff --git a/docs/launcher.md b/docs/launcher.md index e41048a8..66232499 100644 --- a/docs/launcher.md +++ b/docs/launcher.md @@ -88,7 +88,9 @@ identity (`NX2navi.Launcher`), so the taskbar groups and pins it under NX2navi r Screens (`launcher/ui/app.py`): first run (dump, install folder, Desktop Shortcut, Check for Updates at Start, Update Channel, Begin Installation); the steps of an install, update or rebuild with their progress, the build log's last lines and Cancel, and after a failure Try Again and Copy Report; home (Play, Updates, Game Folder, Saves, -Rebuild the Game, Customize Launcher, Settings); customize (Color Scheme, with Custom Background Color, Panel Color and +Mods, Rebuild the Game, Customize Launcher, Settings); mods (Open Mods Folder, Add a Mod, Add a Mod Folder, one On/Off +row for each mod, More Mods when they fill more than a page, Load Order, Back); load order (each mod with its place, +left and right to move it); customize (Color Scheme, with Custom Background Color, Panel Color and Button Color, Background, with a picture set Background Dimming and Remove Background, Animations, Sounds, Add to Steam on Windows, Back); the colour picker (Hue, Saturation, Brightness, Hex Code, Reset, Done); saves (Export Save, Add Log and Settings, Export To, Import a Save, Import a Save Folder, Open Saves Folder, Back); settings (update check, Install Updates Automatically, Hide Launcher While Playing, Keep @@ -338,6 +340,38 @@ copies the current save to `saves/backup ` and makes the import the s (`runtime/fs_host.c` `import_pending`), and the menu says where the old save went. An import in the launcher replaces one still waiting from the menu. +## Mods + +Mods (home, under Saves; `launcher/mods.py`) is where the player puts the mods they download. The game reads the game +folder's `mods` folder (`runtime/mods.c`; `mods_dir` in nx2navi.ini names another): one folder per mod, holding a +`romfs` folder with the game files it replaces, an `exefs` folder with code patches (`.ips`, `.pchtxt`) or a `cheats` +folder, either at its top or in the Atmosphere layout (`atmosphere/contents/0400486024DEC000/romfs` and the others +`tools/mod_files.py` lists). + +A mod goes in as it came. Play first puts the folder in order (`mods.prepare`): an archive (`.zip`, and `.7z` or +`.rar` through Windows' own `tar`, which reads them on Windows 10 and 11) is unpacked into a folder of its name unless +that folder is there already, and the archive stays; a mod folder whose files sit one or more folders down (an archive +that held the mod's own folder) gets them moved up; a `romfs`, `romfslite`, `exefs` or `cheats` folder put straight into +the mods folder becomes a mod of its own, *Unnamed mod*. An archive that would write outside its folder is refused. +Nothing is deleted. Add a Mod and Add a Mod Folder do the same for an archive or folder the player picks, which is +unpacked or copied into the mods folder under a free name; Open Mods Folder opens it. + +Each mod has an On/Off row; its description says what it changes (how many game files, code patches, cheats), which +other mod changes the same files and which of the two wins, and why the game leaves it off when it does: a code module +(`subsdk*`), a download with options to pick from (several folders that each hold a mod), a mod for another game or +another version, or nothing the game reads. Load Order sets which mod wins a file two of them change: the higher one. +The switches and the order are the game's own settings, `mods_disabled` and `mods_order` in nx2navi.ini, written as the +game writes them; they apply at the game's next start, so they wait while it runs. + +**Mods that change the game's code.** The game is lifted from the console's code ahead of time, so code cheats and +ExeFS patches cannot be applied while it runs: the build turns them into patch sets of the lifted code +(`tools/cheats.py`, `tools/exefs_patches.py`) that the game switches on with their mod, and the Cheats tab of the +game's menu switches each cheat. The lift step's key names those files, so a build lifts and compiles again when they +change (only the changed parts compile, and the shaders stay prepared), and `build-cache/mod-code.json` records the +files it took up. When an enabled mod brings code no build has seen, Play builds the game again first (a few minutes; +the home's Mods row says "Build Waiting") and starts it after. A Play from Steam then opens the window for that build. +Code mods are built from the game folder's own `mods` folder only. + ## Importing a save folder Import a Save Folder (Saves) makes another save the game's own. A save put into the game folder's `saves` folder (its diff --git a/docs/releases/v0.9.0.md b/docs/releases/v0.9.0.md index fa955e49..41f72df1 100644 --- a/docs/releases/v0.9.0.md +++ b/docs/releases/v0.9.0.md @@ -55,6 +55,13 @@ for the launcher; a GPU with Vulkan 1.3 and `VK_EXT_external_memory_host` (curre ## New +- **Mods in the launcher.** Mods, on the launcher's home screen, opens the mods folder and adds a mod you downloaded, + as its .zip, .7z or .rar file or its folder; one put into the mods folder by hand works as well. Before the game + starts, the launcher unpacks archives and moves a mod's files to where the game looks for them when they sit a folder + or two down. Each mod has an On/Off switch and says what it changes, which other mod changes the same files and + which of them wins, and why the game cannot use it when it cannot; Load Order sets which mod wins. A mod with code + patches or code cheats is built into the game before it starts, which takes a few minutes the first time: the + launcher's own build left those out until now. - **One Anti-aliasing setting**: the game's own SMAA, off, FXAA, SMAA at its Ultra preset, or TAA. - **Direct mouse camera on game 1.1.1**: the camera takes the mouse's own movement after the stick's dead zone. Contributed by Gaetarra (pull request #462). diff --git a/launcher/jobs.py b/launcher/jobs.py index a411236f..64dd095d 100644 --- a/launcher/jobs.py +++ b/launcher/jobs.py @@ -414,6 +414,17 @@ class RebuildJob(Job): self.note_scan(self.game_dir) +class ModsRebuildJob(RebuildJob): + """Play with a mod whose code cheats or ExeFS patches the game was not built with (launcher/mods.py + code_waiting): the game is built again with them, compiling only what they change, and then starts (App.play).""" + title = 'Building Your Mods In' + retune = False + + def __init__(self, ctx, game_dir): + super().__init__(ctx, game_dir) + self.after = 'play' + + class UpdateDownloadJob(Job): """Before the restart: download, check and stage the release, then switch app/current.txt to it.""" kind = 'update' diff --git a/launcher/mods.py b/launcher/mods.py new file mode 100644 index 00000000..296ff7b7 --- /dev/null +++ b/launcher/mods.py @@ -0,0 +1,505 @@ +"""The game's mods folder, for the launcher's Mods screen and Play. + +The game reads /mods (nx2navi.ini mods_dir names another folder; a relative one is under the game folder): one +folder per mod, each part of it where tools/mod_files.py says (runtime/mods.c). The load order and the switches are +the game's own settings, mods_order and mods_disabled in nx2navi.ini, which this module writes as runtime/cfg.c does: +the top mod wins a file two mods replace, and a mod nobody switched off is on. + +A mod as players download it often does not sit that way, so prepare() puts the mods folder in order before the game +starts, and add() does the same for an archive or a folder the player picks: +- an archive in the mods folder (.zip; .7z and .rar through the system's tar, which reads them on Windows 10 and 11) + is unpacked into a folder of its own name, unless that folder is there already. The archive stays: removing the + folder and the archive removes the mod, and switching it off keeps both; +- a mod folder with nothing of a mod at its top and one folder inside, down to where the mod's parts are, gets that + folder's contents (an archive unpacked into a folder of its own name, holding the mod's folder); +- a romfs, romfslite, exefs or cheats folder put straight into the mods folder becomes a mod of its own (the + Atmosphere layouts, atmosphere/, contents/ and the title's folder, already are one as they are). +Nothing is deleted: files only move inside the mods folder, and a mod that holds several folders with mod files (a +download with options to pick from) stays as it is, with a note. + +Mods that change the game's code (code cheats, ExeFS patches) need the game built with them (tools/make_game.py lift). +code_waiting() names the enabled ones whose files the last build did not take in (build-cache/mod-code.json), and +Play rebuilds the game first then. The build reads them from /mods only. +""" +import json +import os +import re +import shutil +import subprocess +import sys +import zipfile +from pathlib import Path + +from . import gameini + +sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools')) +import mod_files # noqa: E402 + +ARCHIVES = ('.zip', '.7z', '.rar') +LOOSE_PARTS = ('romfs', 'romfslite', 'exefs', 'cheats') # a mod's parts that are no mod when straight in mods/ +# what a folder holds at its top when a mod starts there (runtime/mods.c find_root's places) +MOD_TOP = {'romfs', 'romfslite', 'exefs', 'cheats', 'atmosphere', 'contents', mod_files.TITLE.lower()} +UNWRAP_DEPTH = 4 +UNNAMED = 'Unnamed mod' +STAND_INS = 'Damaged file stand-ins' # the launcher's own (tools/romfs_check.py), shown as such +TITLE_ID = re.compile(r'^[0-9A-Fa-f]{16}$') + + +class ModError(Exception): + pass + + +# ---------------------------------------------------------------- the folder and the settings + +def mods_dir(game_dir): + """The mods folder the game reads: nx2navi.ini mods_dir, a relative one under the game folder, else mods.""" + try: + d = gameini.get(game_dir, 'MODS_DIR') + except OSError: + d = '' + p = Path(d) if d else Path('mods') + return p if p.is_absolute() or (len(d) > 1 and d[1] == ':') else Path(game_dir) / p + + +def ensure_mods_dir(game_dir): + d = mods_dir(game_dir) + d.mkdir(parents=True, exist_ok=True) + return d + + +def _names(game_dir, key): + try: + v = gameini.get(game_dir, key) + except OSError: + v = '' + return [n for n in v.split('|') if n] if v else [] + + +def _quoted(v): + """A value as runtime/cfg.c put_value writes it: quoted when the blanks at its ends or the quotes around it would + be lost.""" + if v and (v[0] in ' \t' or v[-1] in ' \t' or (len(v) >= 2 and v[0] == v[-1] == '"')): + return f'"{v}"' + return v + + +def order(names, wanted): + """names in the game's order (runtime/mods.c by_order): the ones wanted names first, in its order, then the rest by + name (byte order of their UTF-8, as strcmp).""" + pos = {n: i for i, n in reversed(list(enumerate(wanted)))} + return sorted(names, key=lambda n: (0, pos[n], b'') if n in pos else (1, 0, n.encode('utf-8', 'surrogateescape'))) + + +def set_enabled(game_dir, name, on): + """Switch a mod on or off for the game's next start (nx2navi.ini mods_disabled).""" + off = [n for n in _names(game_dir, 'MODS_DISABLED') if n != name] + if not on: + off.append(name) + gameini.put(game_dir, 'mods', 'mods_disabled', _quoted('|'.join(off))) + + +def move(game_dir, name, d): + """Move a mod d places up (negative) or down in the load order (nx2navi.ini mods_order, every mod named).""" + names = [m.name for m in scan(game_dir, details=False)] + if name not in names: + return + i = names.index(name) + j = max(0, min(len(names) - 1, i + d)) + names.insert(j, names.pop(i)) + gameini.put(game_dir, 'mods', 'mods_order', _quoted('|'.join(names))) + + +# ---------------------------------------------------------------- what a mod holds + +class Mod: + def __init__(self, path): + self.path, self.name = path, path.name + self.enabled = True + self.romfs = mod_files.part(path, 'romfs') or mod_files.part(path, 'romfslite') + self.exefs = mod_files.part(path, 'exefs') + self.cheats = mod_files.part(path, 'cheats') + self.files = [] # RomFS paths it replaces, lower case, '/' separated (details) + self.patches = 0 # ExeFS patch files for this game's main + self.other_version = 0 # ExeFS patch files for another version + self.cheat_file = None # its cheat file for this game's main + self.code_module = False # subsdk* or main in its exefs: code the port cannot run + self.other_game = '' # the title id of another game its Atmosphere folders name + self.several = [] # folders that each hold a mod (a download with options) + self.unsupported = '' # why the last build could not take an ExeFS patch file of it + self.waiting = False # code the last build did not take in + + @property + def usable(self): + return not self.problem + + @property + def problem(self): + """Why the game leaves this mod off whatever its switch says (runtime/mods.c scan_cb), '' when it does not.""" + if '|' in self.name: + return 'Its folder name holds "|", which the load order list uses: rename the folder.' + if self.code_module: + return ('It holds a code module (subsdk or main in its exefs folder), which NX2navi cannot run: the game ' + 'leaves the whole mod off.') + if self.unsupported: + return (f'An ExeFS patch file of it cannot be applied ({self.unsupported}): the game leaves the whole mod ' + 'off.') + if not (self.files or self.romfs or self.patches or self.cheat_file): + if self.other_game: + return f'It is made for another game (title {self.other_game}), not Ocarina of Time.' + if self.several: + return (f'It holds {len(self.several)} folders with mod files ({", ".join(self.several[:3])}' + f'{", ..." if len(self.several) > 3 else ""}), a download with options to pick from: move ' + 'the one you want into the mods folder.') + if self.other_version: + return 'Its code patches are for another version of the game, not 1.1.1.' + return ('It holds nothing the game reads: a mod\'s files go into a romfs, exefs or cheats folder (or ' + 'atmosphere\\contents\\' + mod_files.TITLE + '\\romfs).') + return '' + + +def _main_id(game_dir): + try: + with open(Path(game_dir) / 'exefs' / 'main', 'rb') as f: + head = f.read(0x60) + except OSError: + return '' + return head[0x40:0x60].hex() if head[:4] == b'NSO0' and len(head) == 0x60 else '' + + +def _same_build(hex_id, main_id): + """hex_id (trailing zeros may be left out) names main's build id; True when main's is unknown (runtime/mods.c).""" + if not main_id: + return True + h = hex_id.lower() + return 0 < len(h) <= 64 and all(c in '0123456789abcdef' for c in h) and main_id.startswith(h.rstrip('0')) \ + and set(main_id[len(h):]) <= {'0'} + + +def _pchtxt_build(data): + """The build id a pchtxt file names (its last @nsobid- line before @stop), '' when none.""" + found = '' + for line in data.decode('utf-8-sig', errors='replace').splitlines(): + s = line.strip() + if s.startswith('@stop'): + break + if s.startswith('@nsobid-'): + found = re.match(r'@nsobid-([0-9A-Fa-f]*)', s).group(1) + return found + + +def _fnv(text): + h = 0x811C9DC5 + for b in text.encode('utf-8'): + h = ((h ^ b) * 0x01000193) & 0xFFFFFFFF + return f'{h:08X}' + + +def _status(game_dir, name): + """{key: (kind, note)} of a build status file (build-cache/exefs.status, tools/exefs_patches.py).""" + out = {} + try: + text = (Path(game_dir) / 'build-cache' / name).read_text(encoding='utf-8', errors='replace') + except OSError: + return out + for line in text.splitlines(): + f = line.split('\t') + if len(f) >= 5: + out[f[0]] = (f[2], f[4]) + return out + + +def _details(m, main_id, exefs_status): + if m.romfs: + top = len(m.romfs.parts) + m.files = sorted('/'.join(Path(root, f).parts[top:]).lower() + for root, _dirs, files in os.walk(m.romfs) for f in files) + if m.exefs: + for f in m.exefs.iterdir(): + n = f.name.lower() + if not f.is_file(): + continue + if n.startswith('subsdk') or n == 'main': + m.code_module = True + for mod, rel, f in mod_files.patch_files(m.path.parent): + if mod != m.name: + continue + try: + bid = f.stem if f.suffix.lower() == '.ips' else _pchtxt_build(f.read_bytes()) + except OSError: + bid = '' + ok = bool(bid) and _same_build(bid, main_id) + if not ok: + m.other_version += 1 + continue + m.patches += 1 + kind, note = exefs_status.get(_fnv(f'{mod}/{rel}'), ('', '')) + if kind == 'unsupported' and not m.unsupported: + m.unsupported = f'{f.name}: {note}' if note else f.name + contents = m.path / 'atmosphere' / 'contents' + for where in (contents, m.path / 'contents'): + if where.is_dir(): + others = [d.name for d in where.iterdir() if d.is_dir() and TITLE_ID.match(d.name) + and d.name.upper() != mod_files.TITLE] + if others and not m.romfs and not m.exefs and not m.cheats: + m.other_game = others[0].upper() + if not (m.romfs or m.exefs or m.cheats or _is_mod_top(m.path)): + m.several = [d.name for d in _mod_folders_below(m.path)] + + +def _mod_folders_below(folder, depth=2): + """The folders under folder (down to depth levels) that hold a mod at their top.""" + out = [] + try: + subs = sorted((d for d in folder.iterdir() if d.is_dir()), key=lambda d: d.name.lower()) + except OSError: + return out + for d in subs: + if _is_mod_top(d): + out.append(d) + elif depth > 1: + out += _mod_folders_below(d, depth - 1) + return out + + +def _is_mod_top(folder): + try: + return any(d.is_dir() and d.name.lower() in MOD_TOP for d in folder.iterdir()) + except OSError: + return False + + +def scan(game_dir, details=True): + """The mods of the mods folder in the game's load order, with their switches; with details what each holds.""" + folder = mods_dir(game_dir) + mods = [Mod(p) for p in mod_files.mod_folders(folder)] + off = set(_names(game_dir, 'MODS_DISABLED')) + ranked = order([m.name for m in mods], _names(game_dir, 'MODS_ORDER')) + mods.sort(key=lambda m: ranked.index(m.name)) + main_id = _main_id(game_dir) if details else '' + exefs_status = _status(game_dir, 'exefs.status') if details else {} + waiting = set(code_waiting(game_dir)) if details else set() + for m in mods: + m.enabled = m.name not in off + if details: + _details(m, main_id, exefs_status) + m.cheat_file = next((f for mod, f in mod_files.cheat_files(folder, main_id[:16].upper()) if mod == m.name), + None) if main_id else None + m.waiting = m.name in waiting + return mods + + +def overlaps(mods): + """{mod name: [(other mod name, files both replace, whether this one wins)]} among the enabled mods with RomFS + files: the mod higher in the load order wins a file two replace.""" + on = [m for m in mods if m.enabled and m.files] + out = {m.name: [] for m in on} + sets = {m.name: set(m.files) for m in on} + for i, a in enumerate(on): + for j, b in enumerate(on): + if a is not b: + n = len(sets[a.name] & sets[b.name]) + if n: + out[a.name].append((b.name, n, i < j)) + return out + + +# ---------------------------------------------------------------- the mods' code and the build + +def built_code(game_dir): + """{"/": sha256} of the mods' code the last build took in (tools/make_game.py MOD_RECORD).""" + try: + return json.loads((Path(game_dir) / 'build-cache' / 'mod-code.json').read_text(encoding='utf-8')) + except (OSError, ValueError): + return {} + + +def code_waiting(game_dir): + """The enabled mods whose code cheats or ExeFS patches the game was not built with (or with other ones). Only the + game folder's own mods folder is built from.""" + folder = Path(game_dir) / 'mods' + if mods_dir(game_dir) != folder or not folder.is_dir(): + return [] + bid = mod_files.main_build_id(Path(game_dir) / 'exefs' / 'main') + if not bid: + return [] + off = set(_names(game_dir, 'MODS_DISABLED')) + built = built_code(game_dir) + out = [] + for key, sha in mod_files.code_record(folder, bid).items(): + mod = key.split('/', 1)[0] + if mod not in off and built.get(key) != sha and mod not in out: + out.append(mod) + return out + + +# ---------------------------------------------------------------- putting downloads in order + +def _unique(folder, name, file=False): + """folder / name, or the first free "name 2", "name 3" ... (for a file "stem (2).txt" ...).""" + p, k = folder / name, 2 + stem, suffix = (Path(name).stem, Path(name).suffix) if file else (name, '') + while p.exists(): + p = folder / (f'{stem} ({k}){suffix}' if file else f'{name} {k}') + k += 1 + return p + + +def _merge_into(src, dst): + """Move src's entries into dst (made when missing) and remove src: a folder dst holds already is merged into, and + a file of a name dst holds already moves in under a free name (a readme beside the wrapped folder).""" + dst.mkdir(parents=True, exist_ok=True) + for e in list(src.iterdir()): + t = dst / e.name + if t.is_dir() and e.is_dir(): + _merge_into(e, t) + else: + shutil.move(str(e), str(_unique(dst, e.name, file=True) if t.exists() else t)) + src.rmdir() + + +def unwrap(folder): + """A mod folder with nothing of a mod at its top and one folder inside (files beside it, a readme, do not count), + down to UNWRAP_DEPTH levels until the mod starts: that folder's contents move up. True when it did.""" + inner = folder + for _ in range(UNWRAP_DEPTH): + if _is_mod_top(inner): + break + subs = [d for d in inner.iterdir() if d.is_dir()] + if len(subs) != 1: + return False + inner = subs[0] + else: + if not _is_mod_top(inner): + return False + if inner == folder: + return False + # the chain's top folder may share its name with a folder of the mod (MyMod/romfs/romfs): moved aside first. The + # mod's folders go up first, then whatever lay beside them on the way down (a readme), so nothing stays behind + top = next(p for p in [inner, *inner.parents] if p.parent == folder) + aside = _unique(folder, '.unwrapping') + top.rename(aside) + inner = aside / inner.relative_to(top) + _merge_into(inner, folder) + if aside.exists(): + _merge_into(aside, folder) + return True + + +def _safe_members(names): + for n in names: + p = n.replace('\\', '/') + if p.startswith('/') or re.match(r'^[A-Za-z]:', p) or '..' in p.split('/'): + raise ModError(f'it holds a file outside its own folder ({n})') + + +def _tar(): + if os.name == 'nt': + t = Path(os.environ.get('SystemRoot', r'C:\Windows')) / 'System32' / 'tar.exe' + return str(t) if t.is_file() else None + return shutil.which('bsdtar') + + +def unpack(archive, dest): + """Unpack an archive into dest (a new folder). ModError when it cannot be.""" + dest.mkdir(parents=True) + try: + if archive.suffix.lower() == '.zip': + try: + with zipfile.ZipFile(archive) as z: + _safe_members(z.namelist()) + z.extractall(dest) + except (zipfile.BadZipFile, NotImplementedError, RuntimeError) as e: + raise ModError(f'it is not a zip file the launcher can read ({e})') + else: + tar = _tar() + if not tar: + raise ModError(f'this PC has no tool for {archive.suffix} files: unpack it yourself into a folder of ' + 'the mods folder') + flags = {'creationflags': 0x08000000} if os.name == 'nt' else {} + r = subprocess.run([tar, '-tf', str(archive)], capture_output=True, text=True, errors='replace', **flags) + if r.returncode: + raise ModError(f'it could not be read ({r.stderr.strip()[:200] or "tar " + str(r.returncode)}): ' + 'unpack it yourself into a folder of the mods folder') + _safe_members(r.stdout.splitlines()) + r = subprocess.run([tar, '-xf', str(archive), '-C', str(dest)], capture_output=True, text=True, + errors='replace', **flags) + if r.returncode: + raise ModError(f'it could not be unpacked ({r.stderr.strip()[:200] or "tar " + str(r.returncode)}): ' + 'unpack it yourself into a folder of the mods folder') + except BaseException: + shutil.rmtree(dest, ignore_errors=True) + raise + + +def prepare(game_dir, log=None): + """Put the mods folder in order for the game (see the top of this file). Returns what it did, one sentence each.""" + folder = mods_dir(game_dir) + done = [] + if not folder.is_dir(): + return done + + def note(text): + done.append(text) + if log: + log(f'mods: {text}') + + for a in sorted(p for p in folder.iterdir() if p.is_file() and p.suffix.lower() in ARCHIVES): + name = a.stem.strip().rstrip('.') + target = folder / name + if not name or target.exists(): + continue + try: + unpack(a, target) + except (ModError, OSError) as e: + note(f'{a.name} was not unpacked: {e}') + continue + unwrap(target) + note(f'{a.name} unpacked into {target.name}.') + loose = [p for p in folder.iterdir() if p.is_dir() and p.name.lower() in LOOSE_PARTS] + if loose: + target = _unique(folder, UNNAMED) + target.mkdir() + for p in loose: + shutil.move(str(p), str(target / p.name.lower())) + note(f'{", ".join(p.name for p in loose)} moved into a mod folder of its own, {target.name}: rename it as you ' + 'like.') + for d in mod_files.mod_folders(folder): + # an Atmosphere layout straight in the mods folder is a mod as it is; a link is the player's own arrangement + if d.name.lower() in MOD_TOP or d.is_symlink() or getattr(d, 'is_junction', lambda: False)(): + continue + try: + if unwrap(d): + note(f'{d.name}: its files moved up to where the game looks for them.') + except OSError as e: + note(f'{d.name}: its files could not be moved up ({e}).') + return done + + +def add(game_dir, src, log=None): + """Add an archive or a folder the player picked as a mod: unpacked or copied into the mods folder under its name + (a free one), in order for the game. Returns the new mod's folder.""" + src = Path(src) + folder = ensure_mods_dir(game_dir) + if src.is_dir(): + try: + src.resolve().relative_to(folder.resolve()) + raise ModError('it is in the mods folder already') + except ValueError: + pass + name = src.name if src.name.lower() not in MOD_TOP else UNNAMED + target = _unique(folder, name) + if src.name.lower() in MOD_TOP: # the player picked the mod's romfs (or atmosphere) folder itself + target.mkdir() + shutil.copytree(src, target / src.name.lower() if src.name.lower() in LOOSE_PARTS else target / src.name) + else: + shutil.copytree(src, target) + elif src.suffix.lower() in ARCHIVES: + target = _unique(folder, src.stem.strip().rstrip('.') or UNNAMED) + unpack(src, target) + else: + raise ModError('it is no archive (.zip, .7z or .rar) and no folder') + unwrap(target) + if log: + log(f'mods: added {src} as {target.name}') + return target + diff --git a/launcher/ui/app.py b/launcher/ui/app.py index 93007a4d..f9c10bcc 100644 --- a/launcher/ui/app.py +++ b/launcher/ui/app.py @@ -37,7 +37,7 @@ import tkinter.font as tkfont from pathlib import Path from tkinter import filedialog -from .. import NAME, build, feed, gameini, gamefiles, jobs, saves, steam, versions, winsys +from .. import NAME, build, feed, gameini, gamefiles, jobs, mods, saves, steam, versions, winsys from ..context import Context from . import art, picture, sound @@ -54,7 +54,9 @@ P = art.PALETTE # the colour scheme in use: art.set_theme change FRAME_MS = 15 # between two steps of the animations; they move by the time that really passed SPARKS = 22 # the dust drifting over the backdrop DEPTH = {'first_run': 0, 'home': 0, 'job': 1, 'saves': 1, 'settings': 1, 'customize': 1, 'uninstall': 2, - 'color': 2} + 'color': 2, 'mods': 1, 'mod_order': 2} +MODS_PAGE = 7 # mods on a page of the Mods screen, beside its other rows +ORDER_PAGE = 10 # ... of the Load Order screen # the Custom scheme's picker (draw_picker), logical pixels: its box (x, top, width, height), the colour square and # the hue bar (x0, y0, x1, y1) PICKER_BOX = (614, 100, 336, 452) @@ -241,6 +243,9 @@ class App: self.save_message = '' # the last import's result, shown in the import rows' descriptions self.export_message = '' # the last Export Save result, shown in its description self.steam_message = '' # the last Add to Steam result, shown in its description + self.mods_message = '' # what the last change on the Mods screen did, shown in its descriptions + self.mods_page = 0 # the page of the Mods and Load Order screens + self.mods_failed = [] # the mods whose code a build before Play could not take in: Play no longer waits self.steam_busy = False # Add to Steam is making the pictures or writing Steam's files self.pick = None # the colour picker: {'slot', 'h', 's', 'v'} (HSV in 0..1) of the colour being picked self.pick_drag = None # 'square' or 'bar' while the mouse drags on the picker @@ -620,7 +625,7 @@ class App: return next((i for i, r in enumerate(self.rows_home()) if r.label == label), None) def back(self): - if self.screen in ('settings', 'uninstall', 'saves', 'customize', 'job', 'color'): + if self.screen in ('settings', 'uninstall', 'saves', 'customize', 'job', 'color', 'mods', 'mod_order'): self.sounds.play('back') if self.screen == 'color': self.pick_done() @@ -630,6 +635,10 @@ class App: self.show('settings') elif self.screen == 'saves': self.show('home', self.home_row('Saves')) + elif self.screen == 'mods': + self.show('home', self.home_row('Mods')) + elif self.screen == 'mod_order': + self.show('mods', self.mods_back_focus()) elif self.screen == 'customize': self.show('home', self.home_row('Customize Launcher')) elif self.screen == 'job' and self.job and self.job.result is None: @@ -762,6 +771,9 @@ class App: self.after_update_notice() if self.install_staged('a staged update waited for the launcher to start'): return + if a.play: # Play from Steam with a mod's code to build in first (play_only) + self.play() + return if self.settings.get('auto_check'): self.check_now() else: @@ -997,6 +1009,7 @@ class App: desc=('Open the game folder. The game keeps your save in its profile folder, its settings in ' 'nx2navi.ini and its logs in logs. Its saves folder holds saves to import and backups.',)), self.saves_row(gd), + self.mods_row(gd), Row('value', 'Rebuild the Game', rebuild_value, rebuild_style, on=self.start_rebuild, desc=('Build the game again from the files of the last build. Only what changed is compiled. Use it ' 'after changing build tools, or if the game stopped starting.',)), @@ -1522,6 +1535,228 @@ class App: text = f'Save imported: {r["files"]} files from {r["source"].name}.' return text + (f' Your old save is in saves\\{r["backup"].name}.' if r['backup'] else '') + # ---- mods + + def mod_list(self, gd, details=True): + """The game folder's mods in the game's load order ([] before the game is installed or when the folder cannot + be read).""" + if not gd or not Path(gd).is_dir(): + return [] + try: + return mods.scan(gd, details=details) + except OSError as e: + self.ctx.log(f'mods: {e}') + return [] + + def mods_row(self, gd): + """Mods: the screen of the game's mods; orange while a mod's code waits to be built into the game.""" + found = self.mod_list(gd, details=False) + try: + waiting = mods.code_waiting(gd) if found else [] + except OSError: + waiting = [] + on = sum(1 for m in found if m.enabled) + desc = ('Mods change the game: its pictures, models, sounds, text or code. Put a mod you downloaded into the ' + 'mods folder, its .zip, .7z or .rar file as it is or its folder, or select Mods and then Add a Mod: ' + 'Play takes it in. Switch each mod on or off, and choose which wins where two change the same file.',) + if waiting: + desc += (f'{", ".join(waiting)} changes the game\'s code: Play first builds the game again with it, which ' + 'takes a few minutes.',) + return Row('value', 'Mods', 'Build Waiting' if waiting else f'{on} On' if found else 'None', + 'orange_arrow' if waiting else 'arrow', on=lambda: self.show('mods'), desc=desc) + + def mods_page_count(self, n, per): + return max(1, -(-n // per)) + + def rows_mods(self): + gd = self.settings.get('game_dir') + folder = mods.mods_dir(gd) if gd else Path('mods') + found = self.mod_list(gd) + over = mods.overlaps(found) + pages = self.mods_page_count(len(found), MODS_PAGE) + self.mods_page = max(0, min(pages - 1, self.mods_page)) + msg = (self.mods_message,) if self.mods_message else () + rows = [ + Row('value', 'Open Mods Folder', str(folder), on=self.open_mods, + desc=msg + ('Open the mods folder: one folder for each mod. A mod you downloaded goes in as it came, ' + 'an archive (.zip, .7z or .rar) or a folder; Play unpacks archives into folders of their ' + 'own and moves a mod\'s files to where the game looks for them.', + 'To remove a mod, delete its folder and its archive, or switch it off here.')), + Row('value', 'Add a Mod', 'Choose a File', 'arrow', on=lambda: self.add_mod(folder=False), + desc=msg + ('Choose a mod you downloaded, a .zip, .7z or .rar file: it is unpacked into the mods ' + 'folder, ready for the game\'s next start.',)), + Row('value', 'Add a Mod Folder', 'Choose a Folder', 'arrow', on=lambda: self.add_mod(folder=True), + desc=msg + ('Choose the folder of a mod (one with a romfs, exefs or cheats folder in it, or an ' + 'atmosphere folder): it is copied into the mods folder.',)), + ] + if found: + rows.append(Row('divider')) + for m in found[self.mods_page * MODS_PAGE:(self.mods_page + 1) * MODS_PAGE]: + label = fit(self.f['label'], m.name, self.X(LABEL_W - 30)) + if m.usable: + rows.append(Row('toggle', label, on=lambda v, m=m: self.switch_mod(m, v), get=lambda m=m: m.enabled, + desc=lambda m=m: self.mod_desc(m, over))) + else: + rows.append(Row('value', label, 'Not Usable', 'salmon', on=lambda: None, + desc=lambda m=m: self.mod_desc(m, over))) + if pages > 1: + rows.append(Row('enum', 'More Mods', get=lambda: f'Page {self.mods_page + 1} of {pages}', + left=lambda: self.mods_turn(-1, 'mods'), right=lambda: self.mods_turn(1, 'mods'), + desc=(f'{len(found)} mods, {MODS_PAGE} to a page: left and right turn the pages.',))) + if len(found) > 1: + rows.append(Row('value', 'Load Order', '', 'arrow', on=lambda: self.show('mod_order'), + desc=('Where two mods change the same file, the one higher in the load order wins. Set ' + 'the order here.',))) + rows.append(Row('action', 'Back', on=lambda: self.show('home', self.home_row('Mods')), height=40, + desc=('Back to the main menu.',))) + return rows + + def mods_turn(self, d, screen): + self.mods_page += d + self.show(screen, next((i for i, r in enumerate(getattr(self, 'rows_' + screen)()) if r.label == 'More Mods'), + None)) + + def mod_desc(self, m, over): + lines = [self.mods_message] if self.mods_message else [] + if not m.usable: + lines.append(m.problem) + what = [] + if m.files: + what.append(f'replaces {len(m.files):,} of the game\'s files' if len(m.files) > 1 else + 'replaces one of the game\'s files') + if m.patches: + what.append(f'patches the game\'s code ({m.patches} patch file{"s" if m.patches > 1 else ""})') + if m.cheat_file: + what.append('brings cheats, which the Cheats tab of the game\'s menu switches on and off') + if what: + lines.append('It ' + (', '.join(what[:-1]) + ' and ' + what[-1] if len(what) > 1 else what[0]) + '.') + if m.name == mods.STAND_INS: + lines.append('The launcher made it: stand-ins for game files its check found damaged, so that the game ' + 'can load what needs them.') + if m.other_version: + lines.append(f'{m.other_version} of its code patch files are for another version of the game and are ' + 'left out.') + for other, n, wins in over.get(m.name, [])[:3]: + lines.append(f'{n:,} of its files are in {other} too: {"this mod" if wins else other} wins, being higher ' + 'in the load order.') + if m.waiting and m.enabled: + lines.append('Its code is not in the game yet: Play first builds the game again with it, which takes a ' + 'few minutes.') + if m.usable: + lines.append('On: the game loads it at its next start. Off: it stays in the mods folder, unused.') + return {'title': m.name, 'text': lines} + + def game_busy(self, gd): + return bool((self.game_proc and self.game_proc.poll() is None) or (gd and saves.game_running(gd))) + + def switch_mod(self, m, on): + gd = self.settings.get('game_dir') + if self.game_busy(gd): + self.mods_message = 'Quit the game first: it reads the mods when it starts.' + return + try: + mods.set_enabled(gd, m.name, on) + except OSError as e: + self.mods_message = f'The game\'s settings file could not be written: {e}' + return + m.enabled, self.mods_message = bool(on), '' + self.ctx.log(f'mods: {m.name} {"on" if on else "off"}') + + def open_mods(self): + gd = self.settings.get('game_dir') + if not gd or not Path(gd).is_dir(): + self.mods_message = 'Install the game first.' + self.show('mods', self.focus) + return + try: + folder = mods.ensure_mods_dir(gd) + except OSError as e: + self.mods_message = f'The mods folder could not be made: {e}' + self.show('mods', self.focus) + return + self.open(folder) + + def add_mod(self, folder): + gd = self.settings.get('game_dir') + if not gd or not Path(gd).is_dir(): + self.mods_message = 'Install the game first.' + else: + if folder: + src = filedialog.askdirectory(parent=self.root, title='Choose the folder of the mod', + initialdir=str(Path.home() / 'Downloads'), mustexist=True) + else: + src = filedialog.askopenfilename(parent=self.root, title='Choose the mod', + initialdir=str(Path.home() / 'Downloads'), + filetypes=[('Mod', '*.zip *.7z *.rar'), ('All files', '*.*')]) + if not src: + return + try: + target = mods.add(gd, src, log=self.ctx.log) + except (mods.ModError, OSError) as e: + self.mods_message = f'{Path(src).name} was not added: {e}.' + else: + added = next((m for m in self.mod_list(gd) if m.name == target.name), None) + self.mods_message = f'Added {target.name}.' + (f' {added.problem}' if added and not added.usable else + ' It is on: the game loads it at its next start.') + self.show('mods', self.focus) + + def rows_mod_order(self): + gd = self.settings.get('game_dir') + found = self.mod_list(gd, details=False) + pages = self.mods_page_count(len(found), ORDER_PAGE) + self.mods_page = max(0, min(pages - 1, self.mods_page)) + rows = [] + first = self.mods_page * ORDER_PAGE + for k, m in enumerate(found[first:first + ORDER_PAGE], first + 1): + rows.append(Row('enum', fit(self.f['label'], m.name, self.X(LABEL_W - 30)), get=lambda k=k: f'{k}', + left=lambda m=m: self.order_move(m.name, -1), right=lambda m=m: self.order_move(m.name, 1), + desc=lambda m=m, k=k: {'title': m.name, 'text': [ + f'Number {k} of {len(found)} in the load order{" (off)" if not m.enabled else ""}. ' + 'Left moves it up: it wins over the mods below it where two change the same file. ' + 'Right moves it down.'] + ([self.mods_message] if self.mods_message else [])})) + if pages > 1: + rows.append(Row('enum', 'More Mods', get=lambda: f'Page {self.mods_page + 1} of {pages}', + left=lambda: self.mods_turn(-1, 'mod_order'), right=lambda: self.mods_turn(1, 'mod_order'), + desc=(f'{len(found)} mods, {ORDER_PAGE} to a page: left and right turn the pages.',))) + rows.append(Row('action', 'Back', on=lambda: self.show('mods', self.mods_back_focus()), height=40, + desc=('Back to the mods.',))) + return rows + + def mods_back_focus(self): + return next((i for i, r in enumerate(self.rows_mods()) if r.label == 'Load Order'), None) + + def order_move(self, name, d): + gd = self.settings.get('game_dir') + if self.game_busy(gd): + self.mods_message = 'Quit the game first: it reads the mods when it starts.' + return + try: + mods.move(gd, name, d) + except OSError as e: + self.mods_message = f'The game\'s settings file could not be written: {e}' + return + self.mods_message = '' + names = [m.name for m in self.mod_list(gd, details=False)] + at = names.index(name) if name in names else 0 + self.mods_page = at // ORDER_PAGE + self.show('mod_order', at % ORDER_PAGE) + + def mods_before_play(self, gd): + """Put the mods folder in order (mods.prepare) and say whether the game must be built again for a mod's code + first: the mods that wait for it, [] when none do.""" + try: + for line in mods.prepare(gd, log=self.ctx.log): + self.mods_message = line + if 'was not unpacked' in line: # on the home screen too, when the launcher comes back + self.message = line + except OSError as e: + self.ctx.log(f'mods: the mods folder could not be put in order: {e}') + try: + return mods.code_waiting(gd) if build.can_rebuild(gd) else [] + except OSError as e: + self.ctx.log(f'mods: {e}') + return [] + def update_desc(self): r = self.release nv, mine = versions.display(r.version), versions.display(self.layout.version) @@ -1598,7 +1833,8 @@ class App: if self.screen == 'home': self.show('home', self.focus) - def play(self): + def play(self, built=False): + """built: the build for the mods' code (ModsRebuildJob) just ended, so the game starts without asking again.""" gd = self.settings.get('game_dir') exe = Path(gd) / 'ocarina.exe' if not exe.is_file(): @@ -1611,6 +1847,19 @@ class App: self.message = 'The game is already running.' self.render() return + waiting = [] if built else self.mods_before_play(gd) + if waiting and waiting != self.mods_failed: + # a mod's code cheats or ExeFS patches the game was not built with: built in first (tools/make_game.py + # lift), then the game starts (job_finished). After a build that failed for the same mods the game + # starts without them, so that a build that cannot finish never keeps the player from the game. + self.ctx.log(f'play: the game is built again first for the code of {", ".join(waiting)}') + job = jobs.ModsRebuildJob(self.ctx, gd) + job.waiting = waiting + self.run_job(job, 'home') + self.message = f'Building the game with the code of {", ".join(waiting)}; it starts after.' + return + if waiting: + self.ctx.log(f'play: without the code of {", ".join(waiting)}, whose build did not finish') try: self.game_proc = winsys.start_game(exe) except OSError as e: @@ -1914,6 +2163,14 @@ class App: self.message = 'The update is ready: start NX2navi again to finish it.' elif j.after == 'staged': self.message = 'The update was downloaded; a checkout keeps running from the repository.' + elif j.after == 'play': # the mods' code is built in (play): the game starts + if not self.settings.get('keep_cache'): + self.drop_build_cache() + self.message, self.mods_failed = '', [] + self.show('home', self.home_row('Play')) + self.play(built=True) + return + self.mods_failed = [] if not self.settings.get('keep_cache') and j.kind in ('install', 'update', 'rebuild'): self.drop_build_cache() if j.kind == 'install': @@ -1930,8 +2187,10 @@ class App: if j.kind == 'install' and self.settings.get('auto_check'): self.check_now() elif j.result == 'cancelled': + self.mods_failed = getattr(j, 'waiting', self.mods_failed) self.show(self.job_back) else: + self.mods_failed = getattr(j, 'waiting', self.mods_failed) self.show('job') def drop_build_cache(self): @@ -2447,6 +2706,16 @@ def play_only(ctx): if saves.game_running(gd): ctx.log('play from Steam: the game is already running') return 0 + try: + for line in mods.prepare(gd, log=ctx.log): + pass + waiting = mods.code_waiting(gd) if build.can_rebuild(gd) else [] + except OSError as e: + ctx.log(f'play from Steam: mods: {e}') + waiting = [] + if waiting: # the window builds the mods' code in, then starts the game (App.start with --play) + ctx.log(f'play from Steam: the game is built again first for the code of {", ".join(waiting)}') + return None try: proc = winsys.start_game(exe) except OSError as e: diff --git a/launcher/ui/selftest.py b/launcher/ui/selftest.py index 7eeec6c1..a5db2193 100644 --- a/launcher/ui/selftest.py +++ b/launcher/ui/selftest.py @@ -216,12 +216,40 @@ def states(app): app.apply_look() app.show('home', 0) + was_game = s.get('game_dir') + + def mods_screen(): # a game folder of made-up mods, removed when the selftest closes + import atexit + import shutil + import tempfile + game = Path(tempfile.mkdtemp(prefix='nx2navi-selftest-')) + atexit.register(shutil.rmtree, game, True) + for name, files in (('HD Textures', 40), ('Better Menus', 12), ('Classic Music', 3)): + for k in range(files): + f = game / 'mods' / name / 'romfs' / ('UI' if k < 6 else name.split()[0]) / f'file{k}.bin' + f.parent.mkdir(parents=True, exist_ok=True) + f.write_bytes(b'0') + (game / 'mods' / 'Old Plugin' / 'exefs').mkdir(parents=True) + (game / 'mods' / 'Old Plugin' / 'exefs' / 'subsdk9').write_bytes(b'0') + (game / 'nx2navi.ini').write_text('[mods]\nmods_disabled = Classic Music\n', encoding='utf-8') + s['game_dir'] = str(game) + app.mods_page, app.mods_message = 0, '' + app.show('mods', next(i for i, r in enumerate(app.rows_mods()) if r.label == 'HD Textures')) + + def mod_order(): + app.show('mod_order', 1) + + def restore_game(): + s['game_dir'] = was_game + app.show('home', 0) + return [('1-first-run', first_run), ('2-installing', installing), ('3-home', home), ('4-update-ready', update_ready), ('5-updating', updating), ('6-settings', settings), ('7-build-failed', failed), ('8-uninstall', uninstall), ('9-import-saves', import_saves), ('10-shaders', shaders), ('11-update-staged', update_staged), ('12-export-save', export_save), ('13-customize', customize), ('14-color-scheme', color_scheme), ('15-custom-picker', custom_picker), - ('16-first-scheme', restore_scheme)] + ('16-first-scheme', restore_scheme), ('17-mods', mods_screen), ('18-mod-order', mod_order), + ('19-home-again', restore_game)] def run(app, shots): diff --git a/nx2navi.ini.example b/nx2navi.ini.example index 74a27c13..931959ff 100644 --- a/nx2navi.ini.example +++ b/nx2navi.ini.example @@ -731,13 +731,13 @@ [mods] # Folder with the player's mods, one subfolder per mod (/romfs, /exefs, /cheats). A relative folder -# is under the game's folder. Default: mods next to the game's exe. The setup builds the ExeFS patches and code cheats -# of the mods folder next to the exe only. +# is under the game's folder. Default: mods next to the game's exe. The build takes the ExeFS patches and code cheats +# of the mods folder next to the exe only. The launcher's Mods screen adds mods and puts downloads in order. # mods_dir = # Load order, top first, names separated by |; a mod not listed goes below the listed ones, by name. The top mod wins a -# file two mods replace. Set it here: the in-game menu has no Mods tab. +# file two mods replace. The launcher's Mods screen sets it (Load Order); the in-game menu has no Mods tab. # mods_order = -# Mods switched off, names separated by |. +# Mods switched off, names separated by |. The launcher's Mods screen sets it. # mods_disabled = # auto: when an enabled mod has cheats that change the game's code and the game was built without them, the game starts # the setup's rebuild at start (a few minutes) and starts again after it. off: the game starts with the code it has. diff --git a/tests/launcher_test.py b/tests/launcher_test.py index c3b54d30..4a3efcbe 100644 --- a/tests/launcher_test.py +++ b/tests/launcher_test.py @@ -1863,7 +1863,8 @@ class MakeGameTest(Tmp): def test_shaders_key_names_the_translator_the_build_uses(self): """A source checkout has its translator in build/xlate/ (tools/build_xlate.py), not in the kit's bin/: the shaders key follows its identity, so a rebuilt translator prepares the shaders again. A kit's bin/ comes first - when both exist, and a kit with no translator is refused with the way to build one.""" + when both exist. A kit with no translator has a key that says so (the keys are worked out before the steps + run, steps_to_run), and the build refuses it with the way to build one when it needs the translator.""" from unittest import mock kit = self.tmp / 'kit' (kit / 'tools').mkdir(parents=True) @@ -1887,8 +1888,9 @@ class MakeGameTest(Tmp): self.assertEqual(key(), from_bin) shutil.rmtree(kit / 'bin') shutil.rmtree(xlate) + self.assertTrue(key().endswith('|missing')) with contextlib.redirect_stderr(io.StringIO()) as err, self.assertRaises(SystemExit): - key() + self.mg.xlate_source() self.assertIn('tools/build_xlate.py', err.getvalue()) def test_prompts_step(self): @@ -1958,12 +1960,13 @@ class MakeGameTest(Tmp): self.assertEqual(build()[2], 3) (art / 'T_X_B_White.png').write_bytes(b'picture two') self.assertEqual(build()[2], 4) - # the lift running again makes the compile and the shaders run, not the prompts + # the lift running again makes the compile run, not the shaders (their key names the lift and the + # compile they depend on, make_game.py UNAFFECTED) nor the prompts st = json.loads((self.out / 'build-cache' / 'state.json').read_text()) del st['steps']['lift'] (self.out / 'build-cache' / 'state.json').write_text(json.dumps(st)) others, ev, runs = build() - self.assertEqual((others, states(ev), runs), (['compile', 'lift', 'shaders'], ['skip'], 4)) + self.assertEqual((others, states(ev), runs), (['compile', 'lift'], ['skip'], 4)) # a kit without art: skipped, the folders an earlier build drew stay shutil.rmtree(kit / 'prompt-art') self.assertEqual(build(), ([], [{'ev': 'step', 'step': 'prompts', 'state': 'skip'}], 4)) @@ -3086,7 +3089,8 @@ class MakeGameTest(Tmp): code, _ = self.main_rebuild(lambda out, cache: None, stop) self.assertEqual(code, 1) st = json.loads((self.out / 'build-cache' / 'state.json').read_text()) - self.assertEqual(sorted(st['steps']), ['exefs', 'lift', 'romfs']) + # the shaders' record stays as the lift left it: the step's key names what it depends on (UNAFFECTED) + self.assertEqual(sorted(s for s in st['steps'] if s != 'shaders'), ['exefs', 'lift', 'romfs']) self.assertEqual(build.built_version(self.out), '') # no finished build on record self.assertTrue(build.can_rebuild(self.out)) self.assertTrue(build.installed(self.out)) @@ -4806,7 +4810,143 @@ class WindowTest(Tmp): r = subprocess.run([sys.executable, '-m', 'launcher', '--root', str(self.tmp / 'w'), '--selftest'], cwd=ROOT, capture_output=True, text=True, timeout=180) self.assertEqual(r.returncode, 0, r.stderr) - self.assertIn('16 screens drawn', r.stdout) + self.assertIn('19 screens drawn', r.stdout) + + +# ---------------------------------------------------------------- mods + +class ModsTest(Tmp): + """The mods folder as the launcher puts it in order for the game (launcher/mods.py) and the build takes its code + in (tools/make_game.py, tools/mod_files.py).""" + + def setUp(self): + super().setUp() + from launcher import mods + self.mods = mods + self.game = self.tmp / 'game' + made_up_game(self.game) + self.folder = self.game / 'mods' + self.folder.mkdir() + + def file(self, rel, text='x'): + p = self.folder / rel + p.parent.mkdir(parents=True, exist_ok=True) + p.write_text(text) + return p + + def tree(self): + return sorted(p.relative_to(self.folder).as_posix() for p in self.folder.rglob('*') if p.is_file()) + + def test_downloads_are_put_where_the_game_looks(self): + with zipfile.ZipFile(self.folder / 'Zipped.zip', 'w') as z: # a zip holding the mod's own folder + z.writestr('Zipped/romfs/Actor/a.bfres', 'a') + z.writestr('Zipped/README.md', 'r') + with zipfile.ZipFile(self.folder / 'Escape.zip', 'w') as z: + z.writestr('../outside.txt', 'e') + self.file('Wrapped/readme.txt') # wrapped twice, a readme on each level + self.file('Wrapped/Wrapped v2/readme.txt') + self.file('Wrapped/Wrapped v2/Wrapped/romfs/UI/b.txt') + self.file('romfs/Actor/c.bfres') # a mod's romfs straight in the mods folder + self.file('atmosphere/contents/0400486024DEC000/romfs/d.txt') # an Atmosphere layout: a mod as it is + self.file('Options/Option A/romfs/e') # a download with options: left alone + self.file('Options/Option B/romfs/e') + done = self.mods.prepare(self.game) + self.assertEqual(self.tree(), [ + 'Escape.zip', 'Options/Option A/romfs/e', 'Options/Option B/romfs/e', 'Unnamed mod/romfs/Actor/c.bfres', + 'Wrapped/readme (2).txt', 'Wrapped/readme.txt', 'Wrapped/romfs/UI/b.txt', 'Zipped.zip', + 'Zipped/README.md', 'Zipped/romfs/Actor/a.bfres', + 'atmosphere/contents/0400486024DEC000/romfs/d.txt']) + self.assertFalse((self.game / 'outside.txt').exists()) + self.assertTrue(any(line.startswith('Escape.zip was not unpacked') for line in done)) + self.assertEqual(self.mods.prepare(self.game), [line for line in done if 'Escape.zip' in line]) # once + found = {m.name: m for m in self.mods.scan(self.game)} + self.assertEqual(sorted(n for n, m in found.items() if m.usable), + ['Unnamed mod', 'Wrapped', 'Zipped', 'atmosphere']) + self.assertIn('options', found['Options'].problem) + self.assertEqual(found['Zipped'].files, ['actor/a.bfres']) + + def test_what_the_game_cannot_use_is_named(self): + self.file('Plugin/exefs/subsdk9') + self.file('Other/atmosphere/contents/01007EF00011E000/romfs/x') + self.file('Readme only/readme.txt') + self.file('Old patch/exefs/0123456789ABCDEF0123456789ABCDEF.ips', 'PATCHEOF') + self.file('Patch/exefs/E0F3CCFAF50065AE4EA01EB5AEB13B1A.ips', 'PATCHEOF') + found = {m.name: m for m in self.mods.scan(self.game)} + self.assertIn('code module', found['Plugin'].problem) + self.assertIn('01007EF00011E000', found['Other'].problem) + self.assertIn('nothing the game reads', found['Readme only'].problem) + self.assertIn('another version', found['Old patch'].problem) + self.assertEqual((found['Patch'].problem, found['Patch'].patches), ('', 1)) + + def test_switches_and_order_are_the_games_settings(self): + for name in ('beta', 'Alpha', ' Lead'): + self.file(f'{name}/romfs/same.txt', name) + self.assertEqual([m.name for m in self.mods.scan(self.game)], [' Lead', 'Alpha', 'beta']) # strcmp + self.mods.set_enabled(self.game, 'Alpha', False) + self.mods.move(self.game, 'beta', -2) + self.mods.set_enabled(self.game, ' Lead', False) + self.mods.set_enabled(self.game, 'Alpha', True) + self.assertEqual(gameini.get(self.game, 'MODS_ORDER'), 'beta| Lead|Alpha') + self.assertEqual(gameini.get(self.game, 'MODS_DISABLED'), ' Lead') + self.assertIn('mods_disabled = " Lead"', (self.game / 'nx2navi.ini').read_text()) # as runtime/cfg.c quotes + found = self.mods.scan(self.game) + self.assertEqual([(m.name, m.enabled) for m in found], [('beta', True), (' Lead', False), ('Alpha', True)]) + self.assertEqual(self.mods.overlaps(found), {'beta': [('Alpha', 1, True)], 'Alpha': [('beta', 1, False)]}) + + def test_a_picked_archive_or_folder_is_added(self): + src = self.tmp / 'Downloads' + src.mkdir() + with zipfile.ZipFile(src / 'Picked.zip', 'w') as z: + z.writestr('romfs/a.txt', 'a') + (src / 'Folder Mod' / 'Inner' / 'exefs').mkdir(parents=True) + (src / 'Folder Mod' / 'Inner' / 'exefs' / 'E0F3CCFAF50065AE.ips').write_text('PATCHEOF') + (src / 'romfs').mkdir() + (src / 'romfs' / 'b.txt').write_text('b') + self.assertEqual(self.mods.add(self.game, src / 'Picked.zip').name, 'Picked') + self.assertEqual(self.mods.add(self.game, src / 'Picked.zip').name, 'Picked 2') + self.assertEqual(self.mods.add(self.game, src / 'Folder Mod').name, 'Folder Mod') + self.assertEqual(self.mods.add(self.game, src / 'romfs').name, 'Unnamed mod') + self.assertEqual(self.tree(), ['Folder Mod/exefs/E0F3CCFAF50065AE.ips', 'Picked 2/romfs/a.txt', + 'Picked/romfs/a.txt', 'Unnamed mod/romfs/b.txt']) + self.assertTrue((src / 'Picked.zip').is_file()) # the player's download stays where it was + with self.assertRaises(self.mods.ModError): + self.mods.add(self.game, src / 'Folder Mod' / 'Inner' / 'exefs' / 'E0F3CCFAF50065AE.ips') + + def test_code_waits_for_a_build_that_saw_it(self): + import make_game + cheat = self.file('Cheats/cheats/E0F3CCFAF50065AE.txt', '[Moon jump]\n04000000 00000000 D503201F\n') + self.file('Textures/romfs/a.txt') + self.assertEqual(self.mods.code_waiting(self.game), ['Cheats']) + self.mods.set_enabled(self.game, 'Cheats', False) + self.assertEqual(self.mods.code_waiting(self.game), []) # a mod switched off waits for nothing + self.mods.set_enabled(self.game, 'Cheats', True) + (self.game / 'build-cache' / 'mod-code.json').write_text(json.dumps(make_game.mod_code(self.game))) + self.assertEqual(self.mods.code_waiting(self.game), []) # the last build took it up + cheat.write_text('[Moon jump]\n04000000 00000000 D65F03C0\n') + self.assertEqual(self.mods.code_waiting(self.game), ['Cheats']) # edited since + + def test_the_mods_code_is_in_the_lift_key_and_not_the_shaders_key(self): + import make_game + kit = self.tmp / 'kit' + for d in ('tools', 'recomp', 'runtime', 'bin'): + (kit / d).mkdir(parents=True) + for f in ('tools/prepare_shaders.py', 'tools/cheats.py', 'tools/exefs_patches.py', 'tools/mod_files.py', + 'recomp/patches-v111.txt', 'bin/' + make_game.XLATE_NAME, 'bin/nx2navi_xlate.id', 'VERSION'): + (kit / f).write_text('1\n') + + def keys(): + return {s: make_game.step_key(s, 'game', self.game) for s in ('lift', 'compile', 'shaders')} + with mock.patch.object(make_game, 'KIT', kit): + plain = keys() + self.file('Textures/romfs/a.txt') + self.assertEqual(keys(), plain) # files of the game's data: no build + self.assertNotIn('|mods ', plain['lift']) + self.file('Patch/exefs/E0F3CCFAF50065AE.ips', 'PATCHEOF') + with_patch = keys() + self.assertTrue(with_patch['lift'].startswith(plain['lift'] + '|mods ')) + self.assertNotEqual(with_patch['compile'], plain['compile']) + self.assertEqual(with_patch['shaders'], plain['shaders']) + self.assertEqual(list(make_game.mod_code(self.game)), ['Patch/exefs/E0F3CCFAF50065AE.ips']) if __name__ == '__main__': diff --git a/tools/cheats.py b/tools/cheats.py index 65df24de..db0c9c58 100644 --- a/tools/cheats.py +++ b/tools/cheats.py @@ -30,7 +30,7 @@ hash: the same over the cheat's words, so the runtime sees an edited cheat that A code cheat that calls or jumps into a code cave it writes (words where main has zeros) gets the cave's body inlined at the call site; a conditional skip of one store in the body (cbz wN, +8; str wN, ...) is dropped. -Standard library and recomp/ only. +Standard library, recomp/ and tools/mod_files.py only. """ import argparse import struct @@ -38,9 +38,11 @@ import sys from pathlib import Path sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'recomp')) +sys.path.insert(0, str(Path(__file__).resolve().parent)) import flags # noqa: E402 from lifter import Lifter, Unsupported # noqa: E402 from nso import Nso # noqa: E402 +from mod_files import cheat_files # noqa: E402 BASE = 0x7000000000 # main's base in the layout (recomp/layout.py); only pc-relative decoding sees it NOP = 0xD503201F @@ -90,25 +92,6 @@ def parse(text, mod): return out -TITLE = '0400486024DEC000' # the game's title id: a mod's Atmosphere folders are named after it - - -def cheat_files(mods, bid): - """[(mod folder name, cheat file)] of the folders of mods, by name: /cheats/.txt or its Atmosphere places - (runtime/mods.c find_root), the file name in any case.""" - out = [] - for mod in sorted((d for d in mods.iterdir() if d.is_dir()), key=lambda d: d.name) if mods.is_dir() else []: - for where in ('cheats', f'atmosphere/contents/{TITLE}/cheats', f'contents/{TITLE}/cheats', f'{TITLE}/cheats'): - folder = mod / where - if folder.is_dir(): - f = next((p for p in sorted(folder.iterdir()) if p.is_file() and p.name.lower() == f'{bid}.txt'.lower()), - None) - if f: - out.append((mod.name, f)) - break - return out - - def ops(words): """[(type, [words])] of a cheat; type is the opcode nibble, 0xC0.. for the extended ones. ValueError when the last opcode is cut short.""" diff --git a/tools/exefs_patches.py b/tools/exefs_patches.py index 5280c7b8..1183a5c7 100644 --- a/tools/exefs_patches.py +++ b/tools/exefs_patches.py @@ -29,7 +29,7 @@ comma separated, or "-": the runtime switches them off while the mod is on, so t A file the rules cannot express is unsupported as a whole: half of a patch is worse than none. -Standard library, recomp/ and tools/cheats.py only. +Standard library, recomp/, tools/cheats.py and tools/mod_files.py only. """ import argparse import re @@ -38,29 +38,10 @@ import sys from pathlib import Path sys.path.insert(0, str(Path(__file__).resolve().parent)) -from cheats import TITLE, Main, convert, fnv # noqa: E402 +from cheats import Main, convert, fnv # noqa: E402 +from mod_files import patch_files # noqa: E402 NSO_HEADER = 0x100 -EXEFS_PLACES = ('exefs', f'atmosphere/contents/{TITLE}/exefs', f'contents/{TITLE}/exefs', f'{TITLE}/exefs') - - -def patch_files(mods): - """[(mod folder name, path in the mod, file)] of the folders of mods, by name: the .ips and .pchtxt files of the - first exefs place that exists (runtime/mods.c find_root) and of atmosphere/exefs_patches//.""" - out = [] - for mod in sorted((d for d in mods.iterdir() if d.is_dir()), key=lambda d: d.name) if mods.is_dir() else []: - folders = [] - exefs = next((mod / w for w in EXEFS_PLACES if (mod / w).is_dir()), None) - if exefs: - folders.append(exefs) - patches = mod / 'atmosphere' / 'exefs_patches' - if patches.is_dir(): - folders += sorted(d for d in patches.iterdir() if d.is_dir()) - for folder in folders: - for f in sorted(folder.iterdir()): - if f.is_file() and f.suffix.lower() in ('.ips', '.pchtxt'): - out.append((mod.name, f.relative_to(mod).as_posix(), f)) - return out def same_build(name, build_id): diff --git a/tools/make_game.py b/tools/make_game.py index f271eea0..4a4c228d 100644 --- a/tools/make_game.py +++ b/tools/make_game.py @@ -12,7 +12,11 @@ Steps, each skipped when a run before finished it for the same inputs (state in 3. romfs every RomFS file, checked against the container's sha256, into one image -> /romfs/romfs.bin and index.json (16-byte aligned, in the byte order of the paths). When the lift and the compile follow, it runs on a thread of its own beside them: only the RomFS index and romfs.c wait for the image - 4. lift recomp/gen.py with recomp/patches-v111.txt -> /build-cache/lift, plus the two SDK bring-up fixes + 4. lift recomp/gen.py with recomp/patches-v111.txt -> /build-cache/lift, plus the two SDK bring-up fixes. + The code cheats and ExeFS patch files of the mods in /mods (tools/mod_files.py) become patch sets + first (tools/cheats.py, tools/exefs_patches.py), which the game switches on with their mod; the step's + key names those files, and build-cache/mod-code.json records the ones it took up (the launcher + rebuilds the game before it starts when a mod brings others, launcher/mods.py) 5. compile the lifted C and runtime/ with MSYS2's clang, linked into /ocarina.exe; the MSYS2 DLLs it loads, the shader translator and romfs-index.bin beside it 6. shaders every shader in the game files (tools/prepare_shaders.py, default bind state) translated, and each one's @@ -84,6 +88,7 @@ from pathlib import Path KIT = Path(__file__).resolve().parents[1] sys.path.insert(0, str(KIT / 'tools')) import nup # noqa: E402 +import mod_files # noqa: E402 import prompt_art # noqa: E402 TITLE_ID = 0x0400486024DEC000 @@ -99,7 +104,10 @@ OWN_KEY_STEPS = ('prompts', 'scan') # (a step that runs, a later step that reads nothing it writes): the later step is not made to run again. The lift # reads only the unpacked ExeFS (step_key), so the RomFS step leaves it alone: an install that was cut short while the # image was still being packed, the lift beside it already done, packs the image again but does not lift again. -UNAFFECTED = {('romfs', 'lift')} +# The shaders step's key names the lift and the compile it depends on without the mods' code (step_key): a lift and a +# compile that run for a mod's code cheats or ExeFS patches leave the shaders prepared, as they leave the game's +# shaders as they were, and one that runs for anything else (another kit) changes that key and prepares them again. +UNAFFECTED = {('romfs', 'lift'), ('lift', 'shaders'), ('compile', 'shaders')} # The Python packages tools/pad_prompts.py imports (pip name: import name), checked before a build with button art. PROMPT_PACKAGES = {'zstandard': 'zstandard', 'fonttools': 'fontTools', 'pillow': 'PIL'} # The line tools/pad_prompts.py prints for each overlay folder it wrote. @@ -517,12 +525,35 @@ def lift_digest(): return h.hexdigest() -def step_lift(out, cache): - """gen.py into build-cache/lift. The folder is kept: gen.py rewrites only the chunks whose text changed and - removes the ones it no longer writes.""" - lift = cache / 'lift' - cmd = py_cmd(KIT / 'recomp' / 'gen.py', '--exefs', out / 'exefs', '--out', lift, - '--patches', KIT / 'recomp' / 'patches-v111.txt') +# The mods' code (tools/mod_files.py): the tools that make patch sets of it, the patch file each writes, the status +# file the runtime reads in build-cache next to the exe (runtime/cheats.c, runtime/mods.c) and what the log calls it +MOD_STAGES = (('cheats.py', 'patches-cheats.txt', 'cheats.status', 'code cheats'), + ('exefs_patches.py', 'patches-exefs.txt', 'exefs.status', 'ExeFS patches')) +# build-cache/: {"/": sha256} of the files the last lift took up, built in or refused (the status files say +# which): the launcher builds the game again before Play only for files no build has seen (launcher/mods.py) +MOD_RECORD = 'mod-code.json' + + +def mod_code(out): + """{"/": sha256} of the code cheats and ExeFS patch files of /mods for main's build.""" + return mod_files.code_record(out / 'mods', mod_files.main_build_id(out / 'exefs' / 'main')) + + +def mod_digest(out): + """The mods' part of the lift's key: '' without mod code (the key of a game without mods stays what it was), else + the files by name and content and the tools that turn them into patch sets.""" + code = mod_code(out) + if not code: + return '' + h = hashlib.sha256(json.dumps(code, sort_keys=True).encode()) + for tool in ('cheats.py', 'exefs_patches.py', 'mod_files.py'): + h.update(sha256_file(KIT / 'tools' / tool).encode() if (KIT / 'tools' / tool).is_file() else b'no tool') + return h.hexdigest() + + +def run_lifter(out, lift, patches): + """gen.py with the patch file into lift; its exit code.""" + cmd = py_cmd(KIT / 'recomp' / 'gen.py', '--exefs', out / 'exefs', '--out', lift, '--patches', patches) p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, encoding='utf-8', errors='replace', bufsize=1) for line in p.stdout: @@ -531,8 +562,52 @@ def step_lift(out, cache): m = EMITTED.search(line) if m: emit('progress', step='lift', done=int(m.group(1)), total=int(m.group(2)), unit='chunks') - if p.wait(): - fail('the code generator failed (see above)') + return p.wait() + + +def mod_patches(out, cache, plain): + """The patch file with the mods' code cheats and ExeFS patches added (plain when there are none), the status files + written, and the mods' code. Each status file goes first: without it the runtime takes every such file as not + built. The files are the player's downloads, so one the tools refuse costs its stage, never the build.""" + patches, done = plain, [] + code = mod_code(out) + for tool, written, status, what in MOD_STAGES: + (cache / status).unlink(missing_ok=True) + if not code: + continue + cmd = py_cmd(KIT / 'tools' / tool, '--exefs', out / 'exefs', '--mods', out / 'mods', '--patches', patches, + '--out', cache / written, '--status', cache / status) + r = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, encoding='utf-8', + errors='replace') + for line in r.stdout.splitlines(): + say(f' {line}') + if r.returncode: + (cache / status).unlink(missing_ok=True) + say(f' the mods\' {what} could not be built in (see above): the game is built without them') + else: + patches = cache / written + done.append(status) + return patches, done, code + + +def step_lift(out, cache): + """gen.py into build-cache/lift. The folder is kept: gen.py rewrites only the chunks whose text changed and + removes the ones it no longer writes. The mods' code cheats and ExeFS patches go in as patch sets (mod_patches); + should the lifter refuse those, the game is lifted without them.""" + lift = cache / 'lift' + plain = KIT / 'recomp' / 'patches-v111.txt' + (cache / MOD_RECORD).unlink(missing_ok=True) + patches, done, code = mod_patches(out, cache, plain) + if run_lifter(out, lift, patches): + if patches == plain: + fail('the code generator failed (see above)') + say(' the code generator refused the mods\' code: the game is lifted without it') + for status in done: + (cache / status).unlink(missing_ok=True) + if run_lifter(out, lift, plain): + fail('the code generator failed (see above)') + if code: + (cache / MOD_RECORD).write_text(json.dumps(code, indent=1, sort_keys=True), encoding='utf-8') for mod, old, new in LIFT_FIXES: hits = [p for p in sorted((lift / mod).glob('*.c')) if old in p.read_text(encoding='utf-8')] if len(hits) != 1 or hits[0].read_text(encoding='utf-8').count(old) != 1: @@ -1559,6 +1634,15 @@ def xlate_source(): fail(f'{XLATE_NAME} is missing: build it with tools/build_xlate.py') +def xlate_id(): + """The shader translator's id file's sha256 for the shaders step's key, 'missing' without a translator: the key is + worked out before the steps run (steps_to_run), and the shaders step itself says what is missing when it runs.""" + for d in (KIT / 'bin', KIT / 'build' / 'xlate'): + if (d / XLATE_NAME).is_file(): + return sha256_file(d / 'nx2navi_xlate.id') + return 'missing' + + # The x86-64 level the lifted code was compiled for, beside the objects (lifted_level). LIFTED_BUILD = 'lifted-build.json' LIFTED_OBJECTS = ('main_', 'sdk_', 'subsdk0_') # the objects of the lifted code, by the module their names start with @@ -2208,10 +2292,12 @@ def step_scan(out): def step_key(step, identity, out=None): - """What a step's output depends on. exefs and romfs: the game files only; lift: the unpacked ExeFS and the - lifter; compile: the lift, the runtime sources, this script and the processor (cpu_tag); prompts: the game files, - the kit's button art, the version of the tools that draw it (PROMPT_TOOLS), the bind_ lines of nx2navi.ini, and - whether every folder it writes is still there; scan: the game files, the check's version (SCAN_TOOL) and whether its report is still there.""" + """What a step's output depends on. exefs and romfs: the game files only; lift: the unpacked ExeFS, the lifter and + the mods' code cheats and ExeFS patches (mod_digest); compile: the lift, the runtime sources, this script and the + processor (cpu_tag); shaders: the compile without the mods' code, the preparing script and the translator; + prompts: the game files, the kit's button art, the version of the tools that draw it (PROMPT_TOOLS), the bind_ + lines of nx2navi.ini, and whether every folder it writes is still there; scan: the game files, the check's version + (SCAN_TOOL) and whether its report is still there.""" if step in ('exefs', 'romfs'): return f'{step}-1|{identity}' if step == 'scan': @@ -2223,17 +2309,21 @@ def step_key(step, identity, out=None): written = all((out / f / 'manifest.tsv').is_file() for f in prompt_art.OVERLAYS) return (f'prompts-1|{identity}|{tree_digest(prompt_art_dir(), ("**/*.png",))}|{PROMPT_TOOLS}|{binds}' f'|{"written" if written else "missing"}') - lift = f'lift|{exefs_digest(out)}|{lift_digest()}' + base = f'lift|{exefs_digest(out)}|{lift_digest()}' + mods = mod_digest(out) if step in ('lift', 'compile') else '' + lift = f'{base}|mods {mods}' if mods else base if step == 'lift': return lift - compile_key = (f'compile|{lift}|{tree_digest(KIT / "runtime", ("**/*",))}|{sha256_file(Path(__file__))}|{kit_version()}' - f'|cpu {cpu_tag()}') + + def compile_of(lift): + return (f'compile|{lift}|{tree_digest(KIT / "runtime", ("**/*",))}|{sha256_file(Path(__file__))}' + f'|{kit_version()}|cpu {cpu_tag()}') if step == 'compile': - return compile_key - # shaders: the built game and the translator that prepared them, the one xlate_source finds (the kit's bin/, or a - # source checkout's build/xlate/) - return (f'shaders-1|{compile_key}|{sha256_file(KIT / "tools" / "prepare_shaders.py")}' - f'|{sha256_file(xlate_source() / "nx2navi_xlate.id")}') + return compile_of(lift) + # shaders: the built game without the mods' code (UNAFFECTED) and the translator that prepared them, the one + # xlate_source finds (the kit's bin/, or a source checkout's build/xlate/) + return (f'shaders-1|{compile_of(base)}|{sha256_file(KIT / "tools" / "prepare_shaders.py")}' + f'|{xlate_id()}') def out_path(arg): diff --git a/tools/make_kit.py b/tools/make_kit.py index f71a0cf5..bc304307 100644 --- a/tools/make_kit.py +++ b/tools/make_kit.py @@ -31,9 +31,11 @@ NAME = 'NX2navi-BuildKit' TOOLS = ('make_game.py', 'nup.py', 'romfs_index.py', 'check_content.py', 'hostmem.py', 'prepare_shaders.py', 'shader_census.py', 'shader_list.py', 'pipeline_list.py', 'zstd_compat.py', 'pad_prompts.py', 'prompt_art.py', 'button_icons.py', 'romfs_check.py', 'build_game.py', 'check_isa.py', 'build_shaders.py', 'make_dlaa_mask.py', 'make_mvobj_fs.py', - 'pack_shader_cache.py', 'vulkan_pin.py') # make_game.py imports build_game's vma_include and vulkan_include on -# Linux for the VMA header and for Vulkan headers newer than the system's (which build_game takes from vulkan_pin), and -# build_game.py imports check_isa and build_shaders, which reads the two make_ scripts' shaders + 'pack_shader_cache.py', 'vulkan_pin.py', 'mod_files.py', 'cheats.py', 'exefs_patches.py') +# make_game.py imports build_game's vma_include and vulkan_include on Linux for the VMA header and for Vulkan headers +# newer than the system's (which build_game takes from vulkan_pin), and build_game.py imports check_isa and +# build_shaders, which reads the two make_ scripts' shaders; the lift runs cheats.py and exefs_patches.py for the mods' +# code, and both import mod_files.py TOP = ('LICENSE', 'LICENSE-ADDITIONS', 'THIRD_PARTY_NOTICES.md', 'VERSION', 'nx2navi.ini.example') SKIP_DIRS = {'__pycache__', '.pytest_cache'} SKIP_SUFFIXES = {'.o', '.obj', '.pyc', '.exe', '.dll', '.a', '.lib'} diff --git a/tools/make_launcher.py b/tools/make_launcher.py index d154869a..98e8d815 100644 --- a/tools/make_launcher.py +++ b/tools/make_launcher.py @@ -80,7 +80,7 @@ PINS = { } TOOLS = ('make_game.py', 'nup.py', 'romfs_index.py', 'check_content.py', 'hostmem.py', 'prepare_shaders.py', 'shader_census.py', 'shader_list.py', 'pipeline_list.py', 'zstd_compat.py', 'pad_prompts.py', 'prompt_art.py', 'button_icons.py', 'romfs_check.py', - 'pack_shader_cache.py') + 'pack_shader_cache.py', 'mod_files.py', 'cheats.py', 'exefs_patches.py') TOP = ('LICENSE', 'LICENSE-ADDITIONS', 'THIRD_PARTY_NOTICES.md', 'nx2navi.ini.example') SKIP_DIRS = {'__pycache__', '.pytest_cache', 'native'} SKIP_SUFFIXES = {'.o', '.obj', '.pyc', '.exe', '.dll', '.a', '.lib'} diff --git a/tools/make_release.py b/tools/make_release.py index 0b36ee0b..9df4d11f 100644 --- a/tools/make_release.py +++ b/tools/make_release.py @@ -114,7 +114,7 @@ LLVM_SKIP_SUFFIXES = {'.idl', '.tlb'} TOOLS_FILES = ['build_game.py', 'build_shaders.py', 'button_icons.py', 'check_content.py', 'check_isa.py', 'extract.py', 'hostmem.py', 'make_romfs_patches.py', 'ncz.py', 'pack_shader_cache.py', 'prepare_shaders.py', 'romfs_index.py', 'shader_census.py', 'shader_list.py', 'pipeline_list.py', 'ultrawide_layout.py', 'cheats.py', - 'exefs_patches.py'] + 'exefs_patches.py', 'mod_files.py'] TOOLS_DIRS = ['vendor/macnativegames', 'data'] # data/: expected hashes RUNTIME_SUFFIXES = {'.h', '.inc', '.ico'} # .ico: the installer window icon diff --git a/tools/mod_files.py b/tools/mod_files.py new file mode 100644 index 00000000..14f85920 --- /dev/null +++ b/tools/mod_files.py @@ -0,0 +1,99 @@ +"""The player's mods folder as the game reads it (runtime/mods.c): where each mod keeps its parts, and the files of the +mods that change the game's code. + +One folder per mod (a symbolic link or junction to a folder counts). A part (romfs, romfslite, exefs, cheats) is the +first of /, /atmosphere/contents//<part>, <mod>/contents/<title>/<part> and +<mod>/<title>/<part> that is a folder (runtime/mods.c find_root). + +The files that change the game's code are its Atmosphere cheat file for main's build (<cheats>/<build id>.txt, the +name in any case) and the ExeFS patch files (.ips, .pchtxt) of its exefs part and of +<mod>/atmosphere/exefs_patches/<name>/. The lifted game cannot be patched while it runs, so the build turns them into +patch sets (tools/cheats.py, tools/exefs_patches.py) and the runtime switches those on with their mod; a file the +build did not see waits for the next one. tools/make_game.py builds them in and records what it built +(build-cache/mod-code.json, code_record); the launcher compares that record with the folder (launcher/mods.py) to +rebuild the game before it starts when a mod brought new ones. + +Standard library only: the launcher imports it from the kit's tools folder. +""" +import hashlib +from pathlib import Path + +TITLE = '0400486024DEC000' # the game's title id: a mod's Atmosphere folders are named after it +PLACES = ('{}', f'atmosphere/contents/{TITLE}/{{}}', f'contents/{TITLE}/{{}}', f'{TITLE}/{{}}') +PATCH_SUFFIXES = ('.ips', '.pchtxt') + + +def mod_folders(mods): + """The mod folders of the mods folder, by name (as the setup's tools list them: Python's order of the names).""" + mods = Path(mods) + if not mods.is_dir(): + return [] + return sorted((d for d in mods.iterdir() if d.is_dir()), key=lambda d: d.name) + + +def part(mod, name): + """The folder of a mod's part (romfs, romfslite, exefs, cheats): the first of its places that is a folder, or + None.""" + return next((Path(mod) / w.format(name) for w in PLACES if (Path(mod) / w.format(name)).is_dir()), None) + + +def cheat_files(mods, bid): + """[(mod folder name, cheat file)] of the folders of mods, by name: <cheats>/<bid>.txt, the name in any case. bid is + the first 8 bytes of main's build id in hex, as cheat files name themselves (tools/cheats.py build_id).""" + out = [] + for mod in mod_folders(mods): + folder = part(mod, 'cheats') + if folder: + f = next((p for p in sorted(folder.iterdir()) if p.is_file() and p.name.lower() == f'{bid}.txt'.lower()), + None) + if f: + out.append((mod.name, f)) + return out + + +def patch_files(mods): + """[(mod folder name, path in the mod, file)] of the folders of mods, by name: the .ips and .pchtxt files of the + exefs part and of atmosphere/exefs_patches/<name>/.""" + out = [] + for mod in mod_folders(mods): + folders = [] + exefs = part(mod, 'exefs') + if exefs: + folders.append(exefs) + patches = mod / 'atmosphere' / 'exefs_patches' + if patches.is_dir(): + folders += sorted(d for d in patches.iterdir() if d.is_dir()) + for folder in folders: + for f in sorted(folder.iterdir()): + if f.is_file() and f.suffix.lower() in PATCH_SUFFIXES: + out.append((mod.name, f.relative_to(mod).as_posix(), f)) + return out + + +def main_build_id(main): + """The first 8 bytes of main's build id in upper-case hex (the cheat files' name), '' when main cannot be read.""" + try: + with open(main, 'rb') as f: + head = f.read(0x48) + except OSError: + return '' + return head[0x40:0x48].hex().upper() if len(head) == 0x48 and head[:4] == b'NSO0' else '' + + +def code_files(mods, bid): + """[(mod folder name, path in the mod, file)] of every file of the mods folder that changes the game's code: the + cheat files for bid, then the ExeFS patch files.""" + return ([(m, f.relative_to(Path(mods) / m).as_posix(), f) for m, f in cheat_files(mods, bid)] + + patch_files(mods)) + + +def code_record(mods, bid): + """{"<mod>/<path in the mod>": sha256 of its bytes} of code_files: what a build that lifts them in takes. A file + that cannot be read is left out (the tools would fail on it too, and the next build tries again).""" + out = {} + for mod, rel, f in code_files(mods, bid): + try: + out[f'{mod}/{rel}'] = hashlib.sha256(f.read_bytes()).hexdigest() + except OSError: + pass + return out From 1c654ba0c141c049672003a976ce807638bf51a8 Mon Sep 17 00:00:00 2001 From: epicmeme546 <178368684+epicmeme546@users.noreply.github.com> Date: Thu, 12 Nov 2026 08:23:14 -0500 Subject: [PATCH 2/4] Let the Anti-aliasing row's Game SMAA, Off, FXAA and SMAA run together with FSR 4, FSR 3.1 or DLSS, and show the anti-aliasing in the performance overlay. Choosing one of them while a temporal upscaler was set turned the upscaler off, and the picture fell back to bilinear scaling. vk_aa.c no longer keeps the game's SMAA while DLSS or FSR runs. The chosen pass replaces the game's composition as without them: in the native modes after the upscaler, and in the scaling modes before it. In the menu, a choice leaves the upscaler and frame generation alone, and the row notes that the upscaler's TAA runs too. Picking DLSS or FSR under Upscaling keeps the Anti-aliasing choice. The TAA entry, listed while no temporal mode is set, now sets FSR 4 Native AA where FSR 4 runs instead of forcing FSR 3.1. The overlay gets an Anti-aliasing line under Upscaling, for example "TAA + SMAA (Ultra)". Checked on an RX 7900 XT in Hyrule Field with FSR 4.1 Native AA and Quality, and with FSR 3.1. The log has "the game's SMAA replaced by" smaa, off and fxaa while FSR 4.1.2 upscales, and the pictures are right. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --- docs/architecture.md | 4 +- docs/releases/v0.9.1.md | 10 +++++ nx2navi.ini.example | 7 ++-- runtime/ui/menu.cpp | 87 ++++++++++++++++++++++++--------------- runtime/ui/perf_panel.cpp | 7 ++-- runtime/ui/perf_panel.h | 6 +-- runtime/ui_text.c | 30 +++++++++----- runtime/ui_text.h | 2 +- runtime/vk_aa.c | 21 +++++----- 9 files changed, 109 insertions(+), 65 deletions(-) create mode 100644 docs/releases/v0.9.1.md diff --git a/docs/architecture.md b/docs/architecture.md index d9f0d735..539bd2c7 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -195,7 +195,9 @@ keeps per-core data without locks and relies on that, so the host models it (`ru composite (edges, blending weights, composition into the scene target as square-rooted RGB9E5 words). `off`, `fxaa` and `smaa` leave out the first two and replace the composition with `shaders/aa_post.comp` or SMAA at its Ultra preset (`shaders/smaa_*.comp`); the game's passes are known by a hash of their guest code (`vk_exec_draw.c` `aa_dispatch`). - `taa` is DLSS or FSR 3.1 at the scene's size; while either runs, the game's SMAA stays. + `taa` is DLSS or FSR at the scene's size with the game's SMAA. While DLSS or FSR runs (`dlaa`, any mode), the other + values still choose the pass: in the native modes it smooths the upscaler's picture after the bloom, in the scaling + modes it runs on the scene before the upscaler scales it. - HDR (`vk_hdr.c`): the game tonemaps its scene into a float target, upscales it, draws the HUD into a second float target and copies that into the 8-bit sRGB window texture it presents. With `NX2NAVI_HDR=1` the tonemap shader gets the function of `shaders/hdr_tonemap.frag` spliced in where its SDR curve clamps each channel, so the float picture diff --git a/docs/releases/v0.9.1.md b/docs/releases/v0.9.1.md new file mode 100644 index 00000000..4af20399 --- /dev/null +++ b/docs/releases/v0.9.1.md @@ -0,0 +1,10 @@ +## Changed + +- **The Anti-aliasing setting works with FSR 4, FSR 3.1 and DLSS.** Choosing Game SMAA, Off, FXAA or SMAA (Ultra) with + one of them on under Upscaling used to switch the upscaler off and leave the picture to bilinear scaling. Now the + upscaler stays, its TAA keeps running, and the chosen pass runs along with it (the row says "plus TAA from + Upscaling"): *Off* leaves the edges to the TAA alone, *SMAA (Ultra)* smooths what the TAA leaves. Picking FSR or DLSS + under Upscaling keeps the Anti-aliasing choice too, and the row's TAA entry now sets FSR 4 where it runs, no longer + FSR 3.1 in its place. +- **The performance overlay shows the anti-aliasing** under Upscaling, as *TAA + SMAA (Ultra)* and the like while FSR + or DLSS runs. diff --git a/nx2navi.ini.example b/nx2navi.ini.example index 931959ff..b9b7ce56 100644 --- a/nx2navi.ini.example +++ b/nx2navi.ini.example @@ -500,9 +500,10 @@ # more of the trails and leaves those edges a little less smooth. Changes while the game runs take effect at once. # dlss_char_mask = 65 # Anti-aliasing of the 3D picture: game (the game's own SMAA, as on the console), off, fxaa, smaa (SMAA at its Ultra -# preset in the game's place) or taa (DLSS or FSR 3.1 at full size: dlaa = 1 with dlss = dlaa, which the menu's -# Anti-aliasing row sets with it). While DLSS or FSR 3.1 runs, the game's SMAA stays whatever this says. Changes while -# the game runs take effect at once. +# preset in the game's place) or taa (DLSS or FSR at full size: dlaa = 1 with dlss = dlaa, which the menu's +# Anti-aliasing row sets with it, and the game's SMAA). With DLSS or FSR on (dlaa = 1, any mode) their TAA always runs, +# and game, off, fxaa or smaa runs along with it: off leaves the edges to the TAA alone. Changes while the game runs +# take effect at once. # aa = game # The FXAA switch of the game this runtime was first written for; Ocarina has no such pass, so it changes nothing here # (aa above is the setting). diff --git a/runtime/ui/menu.cpp b/runtime/ui/menu.cpp index beb9ccc3..e318b28e 100644 --- a/runtime/ui/menu.cpp +++ b/runtime/ui/menu.cpp @@ -923,6 +923,21 @@ static void upscaling_name(char *out, size_t size) { else snprintf(out, size, mode ? "DLSS %s" : "NVIDIA %s", menu_text(dlss[mode])); } +// The Anti-aliasing row's value for the performance overlay: what row_aa shows, with "TAA + " before it while DLSS's or +// FSR's TAA runs with it (as upscaling_name tells it: FSR 3.1 set where it cannot load runs nothing), and TAA alone +// for off then. +static void aa_name(char *out, size_t size) { + static const char *const values[] = {"game", "off", "fxaa", "smaa"}; + static const UiTextId names[] = {UI_MENU_AA_GAME, UI_MENU_FRAMEGEN_OFF, UI_MENU_AA_FXAA, UI_MENU_AA_SMAA}; + int i = 0; + for (int k = 0; k < 4; k++) + if (!strcmp(cfg_value("NX2NAVI_AA"), values[k])) i = k; // taa: the game's SMAA, with the upscaler or alone + const bool taa = cfg_bool("NX2NAVI_DLAA") && (strcmp(cfg_value("NX2NAVI_TAA"), "fsr3") || vk_ffx_available()); + if (taa && i == 1) snprintf(out, size, "TAA"); + else if (taa) snprintf(out, size, "TAA + %s", menu_text(names[i])); + else snprintf(out, size, "%s", menu_text(names[i])); +} + // The Upscaling row's NVIDIA DLSS waits for a restart (vk_dlaa_needs_restart), where DLSS is offered: the row's note and // the Restart button's count (restart_pending). static bool upscaling_restart(void) { return dlss_offered() && vk_dlaa_needs_restart(); } @@ -990,43 +1005,48 @@ static void row_upscaling(void) { // FSR 4 that failed in a row stays listed (it runs on this device) while FSR 3.1 serves: the row shows FSR 3.1, and // choosing FSR 4 again tries it anew if (e.kind == FSR4) vk_ffx4_retry(); - // the Anti-aliasing row follows: DLSS and FSR 3.1 are its TAA, and leaving them leaves TAA for the game's SMAA - if (e.kind >= DLSS) opt_apply("NX2NAVI_AA", "taa"); - else if (!strcmp(cfg_value("NX2NAVI_AA"), "taa")) opt_apply("NX2NAVI_AA", "game"); + // The Anti-aliasing row keeps its choice, which runs along with DLSS's or FSR's TAA (vk_aa.c vk_aa_mode); its TAA + // went with the temporal mode, so leaving that leaves the game's SMAA + if (e.kind < DLSS && !strcmp(cfg_value("NX2NAVI_AA"), "taa")) opt_apply("NX2NAVI_AA", "game"); } // Anti-aliasing (vk_aa.c NX2NAVI_AA): the game's own SMAA (the default), off, FXAA or SMAA in its place, or TAA, which is -// DLSS or FSR 3.1 at the scene's size, the Upscaling row's temporal modes. TAA is listed where one of them runs: NVIDIA -// DLAA where DLSS is offered, else FSR 3.1 Native AA where it loads. While a temporal mode is set (dlaa), the row shows -// TAA, noted as the Upscaling row's when that mode also scales. Choosing TAA sets the upscaler to the native mode; -// choosing another value turns the temporal upscaler off, with frame generation, which needs it, as the Upscaling row -// does for bilinear and FSR 1, and the picture is then fitted to the window as that row had it. +// DLSS or FSR at the scene's size, the Upscaling row's temporal modes. Without a temporal mode set (dlaa), TAA is listed +// where one runs: NVIDIA DLAA where DLSS is offered and taa does not name FSR, else FSR 4 Native AA where it runs, else +// FSR 3.1 Native AA where it loads; choosing it sets that native mode, as the Upscaling row would. While a temporal mode +// is set its TAA always runs, and the row picks what runs with it: the game's SMAA, nothing, FXAA or SMAA, noted as joined +// by the Upscaling row's TAA. A choice then leaves the upscaler and frame generation as they are. taa itself, set before +// or in the ini, shows as the game's SMAA, which is what runs with the upscaler (or without one, where it cannot run). static void row_aa(void) { enum { GAME, OFF, FXAA, SMAA, TAA }; static const char *const values[] = {"game", "off", "fxaa", "smaa", "taa"}; - const bool dlss = dlss_offered(), fsr = vk_ffx_available(), temporal = cfg_bool("NX2NAVI_DLAA"); - const bool fsr_set = !strcmp(cfg_value("NX2NAVI_TAA"), "fsr3"); + const bool dlss = dlss_offered(), fsr = vk_ffx_available(), fsr4 = fsr && vk_ffx4_available(); + const bool temporal = cfg_bool("NX2NAVI_DLAA"), fsr_set = !strcmp(cfg_value("NX2NAVI_TAA"), "fsr3"); const char *names[5] = {menu_text(UI_MENU_AA_GAME), menu_text(UI_MENU_FRAMEGEN_OFF), menu_text(UI_MENU_AA_FXAA), menu_text(UI_MENU_AA_SMAA), nullptr}; int n = 4; - if (dlss || fsr || temporal) - names[n++] = menu_text((temporal ? !fsr_set : dlss) ? UI_MENU_AA_TAA_DLSS : UI_MENU_AA_TAA_FSR); + enum { NONE, K_DLSS, K_FSR3, K_FSR4 }; + const int kind = dlss && (!fsr_set || !fsr) ? K_DLSS : fsr4 ? K_FSR4 : fsr ? K_FSR3 : NONE; + static char taa[96]; + if (!temporal && kind) { + if (kind == K_FSR4) snprintf(taa, sizeof taa, menu_text(UI_MENU_AA_TAA_FSR4), fsr4_label()); + else snprintf(taa, sizeof taa, "%s", menu_text(kind == K_DLSS ? UI_MENU_AA_TAA_DLSS : UI_MENU_AA_TAA_FSR)); + names[n++] = taa; + } int cur = GAME; - if (temporal) cur = TAA; - else - for (int i = 0; i < 4; i++) - if (!strcmp(cfg_value("NX2NAVI_AA"), values[i])) cur = i; - const char *note = temporal && strcmp(cfg_value("NX2NAVI_DLSS"), "dlaa") ? menu_text(UI_MENU_AA_UPSCALER) : nullptr; + for (int i = 0; i < 4; i++) + if (!strcmp(cfg_value("NX2NAVI_AA"), values[i])) cur = i; + // FSR 3.1 set where it cannot load runs nothing (the Upscaling row says so): no TAA joins the choice then + const char *note = temporal && (!fsr_set || fsr) ? menu_text(UI_MENU_AA_WITH_TAA) : nullptr; int nv = opt_choice_index("NX2NAVI_AA", UI_MENU_AA, names, n, cur, GAME, UI_MENU_AA_DESC, note); if (nv < 0 || nv >= n || nv == cur) return; - if (nv == TAA) { + if (nv == TAA) { // listed only without a temporal mode: the native one of that upscaler, as row_upscaling sets it + opt_apply("NX2NAVI_UPSCALER", "bilinear"); opt_apply("NX2NAVI_DLAA", "1"); opt_apply("NX2NAVI_DLSS", "dlaa"); - opt_apply("NX2NAVI_TAA", dlss ? "dlss" : "fsr3"); - if (!dlss) opt_apply("NX2NAVI_FSR4", "0"); - } else if (temporal) { - opt_apply("NX2NAVI_DLAA", "0"); - opt_apply("NX2NAVI_FRAMEGEN", "off"); + opt_apply("NX2NAVI_TAA", kind == K_DLSS ? "dlss" : "fsr3"); + if (kind != K_DLSS) opt_apply("NX2NAVI_FSR4", kind == K_FSR4 ? "1" : "0"); + if (kind == K_FSR4) vk_ffx4_retry(); } opt_apply("NX2NAVI_AA", values[nv]); } @@ -1886,27 +1906,28 @@ static void perf_picture(const MenuTarget *t, PerfPanel *p) { snprintf(v[1], sizeof v[1], menu_text(UI_MENU_OV_FMT_SIZE), (unsigned)t->image_w, (unsigned)t->image_h); snprintf(v[2], sizeof v[2], menu_text(UI_MENU_OV_FMT_SCENE), scene[0], scene[1], scene[0] * 100.0 / out[0]); upscaling_name(v[3], sizeof v[3]); + aa_name(v[4], sizeof v[4]); double hz = frame_step_limit_hz(); - snprintf(v[4], sizeof v[4], "%s", menu_text(UI_MENU_OV_NONE)); - if (hz > 0) snprintf(v[4], sizeof v[4], menu_text(UI_MENU_OV_FPS), hz); - snprintf(v[5], sizeof v[5], "%s", menu_text(strcmp(cfg_value("NX2NAVI_VK_PRESENT"), "immediate") ? UI_MENU_ON : UI_MENU_OFF)); - v[6][0] = v[7][0] = 0; + snprintf(v[5], sizeof v[5], "%s", menu_text(UI_MENU_OV_NONE)); + if (hz > 0) snprintf(v[5], sizeof v[5], menu_text(UI_MENU_OV_FPS), hz); + snprintf(v[6], sizeof v[6], "%s", menu_text(strcmp(cfg_value("NX2NAVI_VK_PRESENT"), "immediate") ? UI_MENU_ON : UI_MENU_OFF)); + v[7][0] = v[8][0] = 0; if (wide_available()) { const char *a = cfg_value("NX2NAVI_GAME_ASPECT"); - snprintf(v[6], sizeof v[6], "%s", menu_text(!strcmp(a, "window") ? UI_MENU_GAME_ASPECT_WINDOW : !strcmp(a, "21:9") ? UI_MENU_GAME_ASPECT_21_9 + snprintf(v[7], sizeof v[7], "%s", menu_text(!strcmp(a, "window") ? UI_MENU_GAME_ASPECT_WINDOW : !strcmp(a, "21:9") ? UI_MENU_GAME_ASPECT_21_9 : !strcmp(a, "32:9") ? UI_MENU_GAME_ASPECT_32_9 : UI_MENU_GAME_ASPECT_16_9)); const char *r = cfg_value("NX2NAVI_GAME_RES"); - if (!strcmp(r, "window")) snprintf(v[7], sizeof v[7], "%s", menu_text(UI_MENU_GAME_RES_NATIVE)); - else snprintf(v[7], sizeof v[7], "%sp", r); + if (!strcmp(r, "window")) snprintf(v[8], sizeof v[8], "%s", menu_text(UI_MENU_GAME_RES_NATIVE)); + else snprintf(v[8], sizeof v[8], "%sp", r); } const char *hdr = cfg_value("NX2NAVI_HDR"); - snprintf(v[8], sizeof v[8], "%s", menu_text(!strcmp(hdr, "native") ? UI_MENU_HDR_NATIVE : cfg_bool("NX2NAVI_HDR") ? UI_MENU_HDR_TONEMAPPED : UI_MENU_FRAMEGEN_OFF)); + snprintf(v[9], sizeof v[9], "%s", menu_text(!strcmp(hdr, "native") ? UI_MENU_HDR_NATIVE : cfg_bool("NX2NAVI_HDR") ? UI_MENU_HDR_TONEMAPPED : UI_MENU_FRAMEGEN_OFF)); const char *fg = cfg_value("NX2NAVI_FRAMEGEN"); const bool fsr_fg = !strcmp(fg, "fsr3") && vk_ffx_framegen_available(); // set but not loadable: off - snprintf(v[9], sizeof v[9], "%s", fsr_fg ? (vk_fg4_active() ? fg4_label() : "AMD FSR 3.1") + snprintf(v[10], sizeof v[10], "%s", fsr_fg ? (vk_fg4_active() ? fg4_label() : "AMD FSR 3.1") : !strcmp(fg, "dlss") ? "NVIDIA DLSS" : menu_text(UI_MENU_FRAMEGEN_OFF)); if (!strcmp(fg, "dlss") && vk_dlssg_active()) // its multiplier as set, clamped to what this GPU allows - snprintf(v[9], sizeof v[9], "NVIDIA DLSS %dx", + snprintf(v[10], sizeof v[10], "NVIDIA DLSS %dx", vk_dlssg_multiplier_of(cfg_value("NX2NAVI_FRAMEGEN_MULTIPLIER"), vk_dlssg_multiplier_max(nullptr), nullptr)); // The 3D scene's size is known only where resolution.c sets it (the builds with its patch sets); Ocarina sizes its // own scene dynamically and res_sizes would report a fixed 1600x900, so the line is left out diff --git a/runtime/ui/perf_panel.cpp b/runtime/ui/perf_panel.cpp index 5f959a16..32d72b12 100644 --- a/runtime/ui/perf_panel.cpp +++ b/runtime/ui/perf_panel.cpp @@ -165,14 +165,15 @@ void waits(const PerfStats &s) { } void picture(const PerfPanel *p) { - static const UiTextId labels[10] = {UI_MENU_OV_WINDOW, UI_MENU_OV_OUTPUT, UI_MENU_OV_SCENE, UI_MENU_UPSCALING, UI_MENU_OV_LIMIT, - UI_MENU_PRESENT_VSYNC, UI_MENU_GAME_ASPECT, UI_MENU_GAME_RES, UI_MENU_HDR, UI_MENU_FRAMEGEN}; + static const UiTextId labels[11] = {UI_MENU_OV_WINDOW, UI_MENU_OV_OUTPUT, UI_MENU_OV_SCENE, UI_MENU_UPSCALING, UI_MENU_AA, + UI_MENU_OV_LIMIT, UI_MENU_PRESENT_VSYNC, UI_MENU_GAME_ASPECT, UI_MENU_GAME_RES, UI_MENU_HDR, + UI_MENU_FRAMEGEN}; if (!ImGui::BeginTable("##perfp", 2, ImGuiTableFlags_SizingFixedFit)) return; float vw = 0; for (const auto &v : p->picture) vw = fmaxf(vw, ImGui::CalcTextSize(v).x); ImGui::TableSetupColumn("##p0", ImGuiTableColumnFlags_WidthStretch); ImGui::TableSetupColumn("##p1", ImGuiTableColumnFlags_WidthFixed, vw); - for (int i = 0; i < 10; i++) { + for (int i = 0; i < 11; i++) { if ((i == 2 && !p->scene_known) || !p->picture[i][0]) continue; label(labels[i]); ImGui::TableNextColumn(); diff --git a/runtime/ui/perf_panel.h b/runtime/ui/perf_panel.h index a3142a3f..b03d3cd5 100644 --- a/runtime/ui/perf_panel.h +++ b/runtime/ui/perf_panel.h @@ -10,9 +10,9 @@ extern "C" { typedef struct { PerfStats s; // perfmon_snapshot const char *version; // "NX2navi <version>, game <update>" - // window, game image, scene, then the settings as the menu names them: upscaling, frame limit, VSync, widescreen, - // resolution, HDR, frame generation. An empty string leaves its line out. - char picture[10][64]; + // window, game image, scene, then the settings as the menu names them: upscaling, anti-aliasing, frame limit, VSync, + // widescreen, resolution, HDR, frame generation. An empty string leaves its line out. + char picture[11][64]; bool scene_known; // the scene line is shown (resolution.c sets the scene's size) float fg_fps; // frames the display gets with frame generation, 0 without it bool under_menu; // the menu is open: the panel goes under its window diff --git a/runtime/ui_text.c b/runtime/ui_text.c index 61bb391d..9f7402bc 100644 --- a/runtime/ui_text.c +++ b/runtime/ui_text.c @@ -154,14 +154,16 @@ static const char *const text_en[UI_TEXT_COUNT] = { [UI_MENU_AA] = "Anti-aliasing", [UI_MENU_AA_DESC] = "How the edges of the 3D picture are smoothed. Game SMAA is the game's own, as on the console. Off " "leaves the edges as drawn. FXAA is quick and a little soft. SMAA finds more of the edges than the " - "game's and keeps detail sharp. TAA is DLSS or FSR 3.1 at full size: the steadiest edges in motion; " - "it sets the upscaler. Takes effect at once.", + "game's and keeps detail sharp. TAA is DLSS or FSR at full size: the steadiest edges in motion; it " + "sets the upscaler. With DLSS or FSR set under Upscaling their TAA always runs, and the " + "choice here runs along with it. Takes effect at once.", [UI_MENU_AA_GAME] = "Game SMAA", [UI_MENU_AA_FXAA] = "FXAA", [UI_MENU_AA_SMAA] = "SMAA (Ultra)", [UI_MENU_AA_TAA_DLSS] = "TAA (NVIDIA DLAA)", [UI_MENU_AA_TAA_FSR] = "TAA (AMD FSR 3.1)", - [UI_MENU_AA_UPSCALER] = "set by Upscaling", + [UI_MENU_AA_TAA_FSR4] = "TAA (%s)", // FSR 4's name as the Upscaling row gives it ("AMD FSR 4.1") + [UI_MENU_AA_WITH_TAA] = "plus TAA from Upscaling", [UI_MENU_FPS_LIMIT] = "Frame rate limit", [UI_MENU_FPS_LIMIT_DESC] = "The highest frame rate the game draws at. Frame generation adds its frames on top.", @@ -844,14 +846,16 @@ static const char *const text_ru[UI_TEXT_COUNT] = { [UI_MENU_AA] = "Сглаживание", [UI_MENU_AA_DESC] = "Как сглаживаются края 3D-картинки. SMAA игры — собственное, как на консоли. «Выкл.» оставляет края " "как есть. FXAA быстрое и слегка мягкое. SMAA находит больше краёв, чем игровое, и сохраняет " - "чёткость. TAA — это DLSS или FSR 3.1 в полном размере: самые устойчивые края в движении; задаёт " - "масштабирование. Действует сразу.", + "чёткость. TAA — это DLSS или FSR в полном размере: самые устойчивые края в движении; задаёт " + "масштабирование. Когда в Upscaling выбран DLSS или FSR, их TAA работает всегда, а " + "выбранное здесь — вместе с ним. Действует сразу.", [UI_MENU_AA_GAME] = "SMAA игры", [UI_MENU_AA_FXAA] = "FXAA", [UI_MENU_AA_SMAA] = "SMAA (Ultra)", [UI_MENU_AA_TAA_DLSS] = "TAA (NVIDIA DLAA)", [UI_MENU_AA_TAA_FSR] = "TAA (AMD FSR 3.1)", - [UI_MENU_AA_UPSCALER] = "задано масштабированием", + [UI_MENU_AA_TAA_FSR4] = "TAA (%s)", + [UI_MENU_AA_WITH_TAA] = "плюс TAA из Upscaling", [UI_MENU_FPS_LIMIT] = "Ограничение частоты кадров", [UI_MENU_FPS_LIMIT_DESC] = "Максимальная частота кадров, которые рисует игра. Генерация кадров добавляет свои сверх неё.", @@ -1836,14 +1840,16 @@ static const char *const text_pt_br[UI_TEXT_COUNT] = { [UI_MENU_AA] = "Antisserrilhado", [UI_MENU_AA_DESC] = "Como as bordas da imagem 3D são suavizadas. SMAA do jogo é o próprio, como no console. Desligado " "deixa as bordas como desenhadas. FXAA é rápido e um pouco suave. SMAA encontra mais bordas que o do " - "jogo e mantém os detalhes nítidos. TAA é o DLSS ou o FSR 3.1 em tamanho cheio: as bordas mais " - "estáveis em movimento; define o escalonamento. Vale na hora.", + "jogo e mantém os detalhes nítidos. TAA é o DLSS ou o FSR em tamanho cheio: as bordas mais " + "estáveis em movimento; define o escalonamento. Com DLSS ou FSR em Upscaling, o TAA deles " + "sempre roda, e a escolha daqui roda junto. Vale na hora.", [UI_MENU_AA_GAME] = "SMAA do jogo", [UI_MENU_AA_FXAA] = "FXAA", [UI_MENU_AA_SMAA] = "SMAA (Ultra)", [UI_MENU_AA_TAA_DLSS] = "TAA (NVIDIA DLAA)", [UI_MENU_AA_TAA_FSR] = "TAA (AMD FSR 3.1)", - [UI_MENU_AA_UPSCALER] = "definido pelo escalonamento", + [UI_MENU_AA_TAA_FSR4] = "TAA (%s)", + [UI_MENU_AA_WITH_TAA] = "mais o TAA do Upscaling", [UI_MENU_TAB_CHEATS] = "Cheats", [UI_MENU_CHEATS_NONE] = "Nenhum cheat para esta versão do jogo. Os cheats de um mod são um arquivo de cheats do " "Atmosphere na pasta cheats dele, com o nome da build do jogo: mods\\<mod>\\cheats\\%s. " @@ -2614,13 +2620,15 @@ static const char *const text_zh_hans[UI_TEXT_COUNT] = { [UI_MENU_FXAA_DESC] = "游戏自身的 FXAA 边缘平滑。会让细节略微模糊;开启 DLAA 时不需要。", [UI_MENU_AA] = "抗锯齿", [UI_MENU_AA_DESC] = "3D 画面边缘的平滑方式。游戏 SMAA 是游戏自带的,与主机相同。关闭则保留原始边缘。FXAA 快速,略显柔和。SMAA 比游戏自带的找到更多边缘,并保持细节清晰。TAA 即全尺寸的 DLSS 或 " - "FSR 3.1:运动中边缘最稳定;它会设置放大方式。立即生效。", + "FSR:运动中边缘最稳定;它会设置放大方式。在 Upscaling 中选择 DLSS 或 FSR 时,它们的 TAA 始终运行," + "此处的选择与其一同运行。立即生效。", [UI_MENU_AA_GAME] = "游戏 SMAA", [UI_MENU_AA_FXAA] = "FXAA", [UI_MENU_AA_SMAA] = "SMAA(极高)", [UI_MENU_AA_TAA_DLSS] = "TAA(NVIDIA DLAA)", [UI_MENU_AA_TAA_FSR] = "TAA(AMD FSR 3.1)", - [UI_MENU_AA_UPSCALER] = "由放大设置决定", + [UI_MENU_AA_TAA_FSR4] = "TAA(%s)", + [UI_MENU_AA_WITH_TAA] = "另加 Upscaling 的 TAA", [UI_MENU_TAB_CHEATS] = "金手指", [UI_MENU_CHEATS_NONE] = "此游戏版本没有金手指。模组的金手指是它的 cheats 文件夹中以游戏构建命名的 Atmosphere " "金手指文件:mods\\<模组>\\cheats\\%s。重启游戏后才会读取。对于修改游戏代码的金手指," diff --git a/runtime/ui_text.h b/runtime/ui_text.h index 5215dc93..9d098b1f 100644 --- a/runtime/ui_text.h +++ b/runtime/ui_text.h @@ -62,7 +62,7 @@ typedef enum { UI_MENU_DLSS_PRESET_DEFAULT, UI_MENU_DLSS_PRESET_K, UI_MENU_DLSS_PRESET_J, UI_MENU_DLSS_CHARS, UI_MENU_DLSS_CHARS_DESC, UI_MENU_DLSS_CHARS_OFF, UI_MENU_DLSS_CHARS_LIGHT, UI_MENU_DLSS_CHARS_MEDIUM, UI_MENU_DLSS_CHARS_FULL, UI_MENU_FXAA, UI_MENU_FXAA_DESC, UI_MENU_AA, UI_MENU_AA_DESC, UI_MENU_AA_GAME, UI_MENU_AA_FXAA, UI_MENU_AA_SMAA, UI_MENU_AA_TAA_DLSS, UI_MENU_AA_TAA_FSR, - UI_MENU_AA_UPSCALER, + UI_MENU_AA_TAA_FSR4, UI_MENU_AA_WITH_TAA, // Game UI_MENU_FPS_LIMIT, UI_MENU_FPS_LIMIT_DESC, UI_MENU_FPS_30, UI_MENU_FPS_40, UI_MENU_FPS_45, UI_MENU_FPS_60, UI_MENU_FPS_90, diff --git a/runtime/vk_aa.c b/runtime/vk_aa.c index d1aa5bd7..1798f013 100644 --- a/runtime/vk_aa.c +++ b/runtime/vk_aa.c @@ -10,9 +10,11 @@ // replace the composition (vk_exec_draw.c aa_dispatch) with one of the passes here, which read the same composite and // write the same words: off stores it as it is, fxaa smooths it first (shaders/aa_post.comp), smaa runs SMAA at its // Ultra preset (shaders/smaa_*.comp, the reference implementation in shaders/smaa.glsl with the area and search -// textures of third_party/smaa). taa is DLSS or FSR 3.1 at the scene's own size: the menu sets the upscaler for it -// (ui/menu.cpp row_aa). While DLSS or FSR 3.1 anti-aliases the scene (dlaa = 1 and it runs), the game's SMAA stays as it -// is whatever aa says: they take the picture after it, and their inputs are tuned with it. +// textures of third_party/smaa). taa is DLSS or FSR at the scene's own size: the menu sets the upscaler for it +// (ui/menu.cpp row_aa), and the game's SMAA runs with it. While DLSS or FSR anti-aliases or scales the scene (dlaa = 1), +// game, off, fxaa and smaa still choose the pass above: in their native modes the upscalers run before the bloom (vk_dlaa.c, +// "where DLAA runs") and the pass smooths their picture after it, in the scaling modes the pass runs on the scene first +// and the upscaler scales its result. off then leaves the upscaler's TAA alone on the edges. // // The game's passes are known by a hash of their guest code (vk_aa_role), which does not change with the translator or // its settings. Everything here runs on the replay thread. @@ -61,13 +63,13 @@ int vk_aa_role(uint64_t guest_hash) { static const char *const NAMES[] = {"game", "off", "fxaa", "smaa"}; -// DLSS or FSR 3.1 anti-aliases this frame's scene: dlaa = 1 and one of them runs (vk_dlaa.c) +// DLSS or FSR anti-aliases this frame's scene: dlaa = 1 and one of them runs (vk_dlaa.c) static int temporal(void) { return cfg_bool("NX2NAVI_DLAA") && (vk_dlaa_available() || vk_dlaa_fsr3()); } static struct { uint64_t frame_read; int mode; // this frame's (vk_aa_mode) - int said; // the last refusal logged: 1 + the mode refused, 0 none + int said; // taa was last logged as waiting for DLSS or FSR int composed; // the last composition seen was replaced (vk_aa_composed): the passes before it may go uint64_t frames_on, composes; // frames with a replacement asked for, compositions replaced int none_told; @@ -81,14 +83,13 @@ int vk_aa_mode(void) { int want = VK_AA_GAME; for (int i = 1; i < 4; i++) if (!strcmp(v, NAMES[i])) want = i; - int tmp = want != VK_AA_GAME && temporal(); - int said = tmp ? 1 + want : !strcmp(v, "taa") && !temporal() ? 1 + 4 : 0; + // taa is the game's SMAA with DLSS's or FSR's TAA; without them at work, the game's SMAA alone + int said = !strcmp(v, "taa") && !temporal(); if (said != A.said) { A.said = said; - if (said == 5) rt_log("aa: taa needs DLSS or FSR 3.1 at work (dlaa = 1): the game's SMAA stays"); - else if (said) rt_log("aa: %s waits while DLSS or FSR 3.1 anti-aliases the scene: the game's SMAA stays", NAMES[want]); + if (said) rt_log("aa: taa needs DLSS or FSR at work (dlaa = 1): the game's SMAA stays"); } - A.mode = tmp ? VK_AA_GAME : want; + A.mode = want; if (A.mode != VK_AA_GAME && !A.none_told && ++A.frames_on == 3600 && !A.composes) { A.none_told = 1; // a game version or a frame without the passes this file knows: nothing is replaced rt_log("aa: aa = %s found none of the game's SMAA passes in %llu frames: the picture stays the game's", From 2d09c220bfa2535d4c39021099792483bec3bee2 Mon Sep 17 00:00:00 2001 From: epicmeme546 <178368684+epicmeme546@users.noreply.github.com> Date: Thu, 12 Nov 2026 08:23:25 -0500 Subject: [PATCH 3/4] Fill the scene's last column or row from the one next to it after a jitter to the left or up, before DLSS or FSR reads the scene. With FSR 4 or FSR 3.1 a light line flickered along the bottom and right edges of the picture. The jitter moves the scene's viewport by under half a pixel, which keeps every pixel centre inside it. Yet on an RX 7900 XT, a viewport moved up left the last row undrawn in every scene pass, and one moved left the last column. A frame dump showed the depth's last row at its clear value and the light target's last row holding the fog's colour, which the upscaler blended into the edge. NX2NAVI_DLAA_EDGE_FILL=0 turns the fill off for testing. Hyrule Field, 3416x1440, present screenshots: the outermost rows averaged 40 against 20 just inside without the fill, and within 1 of the inside with it. The same held with FSR 4.1 Native AA and Quality, FSR 3.1, SMAA and HDR. Jitter off, and no upscaler, gave no line either. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --- docs/releases/v0.9.1.md | 8 ++++++++ runtime/cfg.c | 2 +- runtime/vk_dlaa.c | 35 +++++++++++++++++++++++++++++++++++ 3 files changed, 44 insertions(+), 1 deletion(-) diff --git a/docs/releases/v0.9.1.md b/docs/releases/v0.9.1.md index 4af20399..af787ab6 100644 --- a/docs/releases/v0.9.1.md +++ b/docs/releases/v0.9.1.md @@ -1,3 +1,11 @@ +## Fixed + +- **A flickering light line along the bottom and right edges of the picture with FSR 4, FSR 3.1 or DLSS.** To let the + upscaler build a smoother picture, the scene is drawn moved by a fraction of a pixel, a little differently each frame. + When it moved up or to the left, AMD's driver left the last row or column of the scene undrawn, the fog's colour stayed + there, and the upscaler blended it into the edge. That row or column is now filled from the one next to it before the + upscaler reads the scene. + ## Changed - **The Anti-aliasing setting works with FSR 4, FSR 3.1 and DLSS.** Choosing Game SMAA, Off, FXAA or SMAA (Ultra) with diff --git a/runtime/cfg.c b/runtime/cfg.c index 96d8867f..6459b8f0 100644 --- a/runtime/cfg.c +++ b/runtime/cfg.c @@ -44,7 +44,7 @@ static const char *const known[] = { "BIND_SKIP", "BIND_STATS", "BIND_UP", "BIND_X", "BIND_Y", "BIND_ZL", "BIND_ZR", "BLOCK_VIEWS_3D", "CACHE_DIE", "CLOTH_REALTIME", "COLOR_BRIGHTNESS", "COMPUTE_KEEP", "COLOR_CONTRAST", "COLOR_GAMMA", "COLOR_PRESET", "COLOR_SATURATION", "CONTACT_DEBUG", "CONTACT_SHADOWS", "CORE_DIAG", "CRIT", "CRIT_OUT", - "DESC_TEMPLATE", "DEVICE_PROFILE", "DISCORD_APP_ID", "DISCORD_IMAGE", "DISCORD_RPC", "DLAA", "DLAA_CAM_KEEP", "DLSS_IDLE_FREE", "DLAA_JITTER", "DLAA_MV", "DLAA_MASKDEBUG", "DLAA_MVDEBUG", "DLAA_OBJMV", "DLAA_SURVEY", "DLAA_TARGET_KEEP", "DLAA_TIMES", "DLSS", "DLSS_CHAR_MASK", "DLSS_PRESET", "DLSS_REACTIVE", "DUMP", "DUMP_BUF", "DUMP_DIR", + "DESC_TEMPLATE", "DEVICE_PROFILE", "DISCORD_APP_ID", "DISCORD_IMAGE", "DISCORD_RPC", "DLAA", "DLAA_CAM_KEEP", "DLSS_IDLE_FREE", "DLAA_EDGE_FILL", "DLAA_JITTER", "DLAA_MV", "DLAA_MASKDEBUG", "DLAA_MVDEBUG", "DLAA_OBJMV", "DLAA_SURVEY", "DLAA_TARGET_KEEP", "DLAA_TIMES", "DLSS", "DLSS_CHAR_MASK", "DLSS_PRESET", "DLSS_REACTIVE", "DUMP", "DUMP_BUF", "DUMP_DIR", "DUMP_LOG", "DUMP_RAW", "DUMP_SHADERS", "DUMP_FILE", "DS3", "LOG_CS_BINDS", "LOG_TEX_FLAGS", "LOG_MESH", "MESH", "POOL", "NATIVE_TENSOR", "LOG_RCOND", "LOG_RT_SIZES", "DUMP_WHITE", "DUMP_AFTER", "DUMP_MOVIE", "SUBMIT_FINISH", "ACQUIRE_WAIT", "CRASH_FILTER_GUARD", "CS_FALLBACK", "EXEFS", "FEEDBACK", "FIX_C0", "FIX_ZERO", "FLIGHT", "FLIGHT_AT", "FLIGHT_DIR", "FLIGHT_FRAMES", "GPU_TIMELINE", "FFX_PIPE_CACHE", "FLIGHT_SPIKE_MS", "FLIGHT_THUMBS", "FLIGHT_EVERY", "FLUSH_SPAN", "FLUSH_UPLOAD", "FOG_CULL", "FOLIAGE_SHADOW_CULL", "FOLIAGE_SHADOW_VRS", "FOV", "FOV_PATCH", "FG_HUDLESS_SHOW", "FG_MULT", "FPS60", "FPS_LIMIT", "FRAMEGEN", "FRAMEGEN_MULTIPLIER", "FREEZE_IDLE", "FREEZE_ASYNC", "FREEZE_BUDGET_US", "FREE_CORES", "FULLSCREEN", "FXAA", "GAME_ASPECT", "GAME_RES", "GPA", "GPA_EMPTY", "GPA_TWIN", "GPA_TWIN_MAX_KB", "MIRROR_CAPTURE_KB", "WB_SKIP_MB", "GPL", "GPL_DRAW_BUILD", "GPL_FEEDBACK", "PIPE_AHEAD", "PIPE_FAST_WORKERS", "PIPE_FRAME_WAIT_MS", "PIPE_LINK_DRAW_MS", "PIPE_SKIP_STEADY", "PROG_LEAD", "GPU", "GPU_INDEX", "GPU_PASSES", "GPU_NAME", "HDR", diff --git a/runtime/vk_dlaa.c b/runtime/vk_dlaa.c index 3854faab..57a2e897 100644 --- a/runtime/vk_dlaa.c +++ b/runtime/vk_dlaa.c @@ -1769,6 +1769,40 @@ static void release_kept(const struct Size *making, int all) { making->w, making->h, all ? "after a release" : "larger inputs"); } +// The scene's last column and row after a jitter to the left or up. The jitter moves the scene's viewport by under half a +// pixel (vk_dlaa_viewport), which keeps every pixel centre inside it, yet on an RX 7900 XT (AMD's Windows driver) the +// draws of a viewport moved left left the last column out, and of one moved up the last row: the driver evidently limits +// drawing to the whole pixels inside the viewport. In a frame dump (NX2NAVI_DUMP, Hyrule Field, 3416x1440, FSR 4 Native +// AA) the depth's last row kept its clear value through every scene pass and the light target held the fog's colour +// there, which the upscaler blended into a grey line along the bottom and right edges of the picture that flickered with +// the jitter; with NX2NAVI_DLAA_JITTER=0, and without DLSS or FSR, there was no line. Before DLSS or FSR reads the scene, +// such a column or row of the light target and of the depth (which the motion vectors are made from) is filled from the +// one next to it. NX2NAVI_DLAA_EDGE_FILL=0 (test): not filled. +static void edge_fill(VkCommandBuffer cb, const VkExecTarget *t, const VkExecTarget *z, float jx, float jy) { + static int on = -1; + if (on < 0) on = rt_knob("DLAA_EDGE_FILL", "1"); + uint32_t w = t->width, h = t->height; + if (!on || w < 2 || h < 2 || (jx >= 0 && jy >= 0)) return; + static int told; + if (!told++) rt_log("dlaa: the scene's last column or row after a jitter to the left or up is filled from the one before it"); + // the column first, then the row over the whole width, so the corner comes from the filled column + for (int pass = 0; pass < 2; pass++) { + if (pass ? jy >= 0 : jx >= 0) continue; + VkOffset3D from = pass ? (VkOffset3D){0, (int32_t)h - 2, 0} : (VkOffset3D){(int32_t)w - 2, 0, 0}; + VkOffset3D to = pass ? (VkOffset3D){0, (int32_t)h - 1, 0} : (VkOffset3D){(int32_t)w - 1, 0, 0}; + VkExtent3D ext = pass ? (VkExtent3D){w, 1, 1} : (VkExtent3D){1, h, 1}; + for (int i = 0; i < 2; i++) { + const VkExecTarget *g = i ? z : t; + if (!g->image || g->width != w || g->height != h) continue; + VkImageAspectFlags a = i ? VK_IMAGE_ASPECT_DEPTH_BIT : VK_IMAGE_ASPECT_COLOR_BIT; + VkImageCopy r = {.srcSubresource = {a, g->level, g->base_layer, 1}, .srcOffset = from, + .dstSubresource = {a, g->level, g->base_layer, 1}, .dstOffset = to, .extent = ext}; + vkCmdCopyImage(cb, g->image, VK_IMAGE_LAYOUT_GENERAL, g->image, VK_IMAGE_LAYOUT_GENERAL, 1, &r); + } + full_barrier(cb); + } +} + // Set when the scene the upscaler runs on has another size than the last one; vk_dlaa_resized_take reads it. static volatile int32_t size_changed; @@ -1799,6 +1833,7 @@ static void evaluate(VkExec *ex, const VkExecTarget *dst) { VkImageView zv = vk_exec_target_view(z->img, z->format, VK_IMAGE_VIEW_TYPE_2D, VK_IMAGE_ASPECT_DEPTH_BIT, z->level, z->base_layer, 1, 0); float jx = D.jit_used ? D.ux : 0.0f, jy = D.jit_used ? D.uy : 0.0f; + edge_fill(cb, t, z, jx, jy); // before the motion vectors read the depth static int no_mv = -1; if (no_mv < 0) no_mv = !rt_knob("DLAA_MV", "1"); // NX2NAVI_DLAA_MV=0: zero motion (test) int masked = 0, mv = no_mv ? 0 : mv_pass(ex, e, zv, jx, jy, char_bias(), &masked); From 37dc8a85b8e62a65015004af00cda0591f08401f Mon Sep 17 00:00:00 2001 From: epicmeme546 <178368684+epicmeme546@users.noreply.github.com> Date: Thu, 12 Nov 2026 08:25:25 -0500 Subject: [PATCH 4/4] Move the Mods bullet from the 0.9.0 notes to the 0.9.1 draft. 0.9.0 is tagged at fbd4067 without the launcher's Mods screen, which came after it (89c453d), so its bullet goes under "New" in docs/releases/v0.9.1.md, unchanged. docs/releases/v0.9.0.md is back to how it was at the tag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --- docs/releases/v0.9.0.md | 7 ------- docs/releases/v0.9.1.md | 10 ++++++++++ 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/docs/releases/v0.9.0.md b/docs/releases/v0.9.0.md index 41f72df1..fa955e49 100644 --- a/docs/releases/v0.9.0.md +++ b/docs/releases/v0.9.0.md @@ -55,13 +55,6 @@ for the launcher; a GPU with Vulkan 1.3 and `VK_EXT_external_memory_host` (curre ## New -- **Mods in the launcher.** Mods, on the launcher's home screen, opens the mods folder and adds a mod you downloaded, - as its .zip, .7z or .rar file or its folder; one put into the mods folder by hand works as well. Before the game - starts, the launcher unpacks archives and moves a mod's files to where the game looks for them when they sit a folder - or two down. Each mod has an On/Off switch and says what it changes, which other mod changes the same files and - which of them wins, and why the game cannot use it when it cannot; Load Order sets which mod wins. A mod with code - patches or code cheats is built into the game before it starts, which takes a few minutes the first time: the - launcher's own build left those out until now. - **One Anti-aliasing setting**: the game's own SMAA, off, FXAA, SMAA at its Ultra preset, or TAA. - **Direct mouse camera on game 1.1.1**: the camera takes the mouse's own movement after the stick's dead zone. Contributed by Gaetarra (pull request #462). diff --git a/docs/releases/v0.9.1.md b/docs/releases/v0.9.1.md index af787ab6..f830bc5e 100644 --- a/docs/releases/v0.9.1.md +++ b/docs/releases/v0.9.1.md @@ -1,3 +1,13 @@ +## New + +- **Mods in the launcher.** Mods, on the launcher's home screen, opens the mods folder and adds a mod you downloaded, + as its .zip, .7z or .rar file or its folder; one put into the mods folder by hand works as well. Before the game + starts, the launcher unpacks archives and moves a mod's files to where the game looks for them when they sit a folder + or two down. Each mod has an On/Off switch and says what it changes, which other mod changes the same files and + which of them wins, and why the game cannot use it when it cannot; Load Order sets which mod wins. A mod with code + patches or code cheats is built into the game before it starts, which takes a few minutes the first time: the + launcher's own build left those out until now. + ## Fixed - **A flickering light line along the bottom and right edges of the picture with FSR 4, FSR 3.1 or DLSS.** To let the