3 Commits

Author SHA1 Message Date
chrg6
62cc3b1312 Add release 0.12.0, the handover of installed launchers to Triforce: the source of its window, its tests and its notes.
The release is one launcher version that opens a single window in place of the launcher: Get Triforce starts the Triforce Setup the release carries, after checking its sha256, and Play NX2navi starts the game that is built on the PC. Nothing of NX2navi is removed. handover/ holds the source of that window as it was published; the drawn pictures, the Setup program and the release files are not kept in the repository (handover/README.md, "In the repository").

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-14 23:52:26 -05:00
chrg6
1c3fa7df2a Give Streamline constants and null tags on a frame without inputs while frame generation is on.
A review of the DLSS and frame generation paths against NVIDIA's guides (the DLSS programming guide 310.6.0, the
DLSS-FG guide 310.7.0, Streamline 2.14.16's guides and headers).

What follows them: the texture bias (log2(render width / output width) - 1 on top of the game's own, at each change
of the mode), the jitter (Halton, 8 x (output / render)^2 phases and 16 at least, in render pixels within half a
pixel, the scene drawn with it and the motion vectors made without it), the creation flags (HDR, vectors at the
render size, auto-exposure, depth not inverted) and the constants Streamline gets (unjittered matrices, the jitter
in pixels, the vectors' scale of one over the render size).

What did not: vk_dlssg.c holds a change of generation's state for FG_SETTLE_FRAMES frames, so for up to 12 frames
after the depth, the motion vectors or the camera went away (a menu, a fade, a load) generation stayed on and
nx2sl_frame set no constants and tagged nothing. Streamline's guide asks for the constants at every frame and for
null tags where depth or motion vectors are not valid; it logged "Unable to find 'common' constants for frame" and
"Failed to find global tag 'kBufferTypeDepth'" and used the frame before's (in every play log with frame
generation). Such a frame now gets the last frame's constants with reset set and the three tags as null. Not run in
a game yet: nx2navi_dlssg.dll builds and exports its 15 functions.

What the guides leave open: the sign of the jitter offsets against the shift of the picture, which NVIDIA says to
find by trying, and the bias reaches only the tiled, trilinear samplers, not every texture of the scene. For both
there is a test switch now, read once: NX2NAVI_DLAA_JITTER_TEST=<x>,<y> tells the upscaler the jitter times two
factors, and NX2NAVI_DLSS_LOD_BIAS_TEST=<bias> puts a bias in the rule's place. Neither was measured: the runs
that would do it start the game.

The line about the presents a second (the commit before) came at 59 a second under a limit of 60: it comes under 56
now.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-14 12:54:08 -05:00
chrg6
2dd363b86e Count the texture cache's short waits in seconds below 60 presents a second too.
The cache counts its short waits in presents (PRESENTS(n): n presents at 60 a second) and scales them by the presents
a second. Above 60 it always did. Below 60 the rate stopped at 60, so the 120 presents a released texture stays out
of the VRAM pressure's reach (STALE_FRAMES) were 4 s at 30 presents a second and 5.7 s at 21. The game lets go of
textures by the second, as the player moves, so what that window holds grows with its length.

A tester's log (RTX 4070 SUPER, 2560x1440, DLSS Quality with frame generation, texture detail Full, 0.10.2-rc4): the
card went to 11,198 MiB against a budget of 10,600 while the frame rate fell from 63 to 21 fps. The released textures
held 2,644 MiB against their bound of 384, and 2,105 MiB of that was released too lately to go (442 MiB when the
pressure began).

On an RTX 5090 with the budget taken as 10,600 MiB and the tester's output settings, a walk through Hyrule Field at a
30 fps limit: the released-too-lately textures were all in their first 120 presents (the new census line below) and
held 383 and 429 MiB on average at the memory reports of two runs, against 292 MiB at 60 fps. With this change they
are in their first 60 presents and hold 209 MiB. Textures made again after a drop: 295 and 523 before, 511 after.
Not shown: the tester's 21 fps and a card that is really over its budget, which this card does not reach; the memory
in use as the driver counts it stayed at 9.8 GiB in the capped runs.

The rate now follows the presents down to 15 a second (vk_tex_release.h trl_rate_sample, trl_rate, trl_presents),
once eight seconds in a row were under 60, and never in a run's first eight seconds; one slow second among fast ones
still shortens nothing. Every wait PRESENTS counts follows it. NX2NAVI_TEX_RATE_LOW=0 stops the rate at 60 as before,
and one line says when the rate crosses 60. Values at 60 presents a second and above are the ones before.

The census of the released textures over their bound gets a second line: what the ones released too lately wait for
(their first presents after the release, content only the GPU made, the gap before a take-back with or without a
lookup still reaching them). It is what showed which wait held the memory.

tests/tex_release_check has the rate's checks and a table of the wait by presents a second; they fail against the
rule before.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-14 11:33:02 -05:00
26 changed files with 2210 additions and 17 deletions

View File

@@ -30,7 +30,7 @@ The repository's additions are all rights reserved (`LICENSE-ADDITIONS`). It nev
- Every lifter change passes the differential tests against Unicorn ([docs/building.md](docs/building.md), "Checks"). After changing `recomp/jumptables.py`, `tests\jumptables_check.py` reports 0 disagreements on 1.2.1 and 1.4.3.
- `recomp/layout.py` alone decides module bases.
- Graphics bugs: [docs/maxwell.md](docs/maxwell.md) lists the Maxwell behaviour the Vulkan path must match, what goes wrong when it does not, and what our runtime does.
- Comments say what the code does and why, in words that stand on their own. Cite our code by file plus function (a line number may follow as a hint). Comments and docs do not name other projects or say where code came from: the license header of copied code and its `THIRD_PARTY_NOTICES.md` row carry that.
- Comments say what the code does and why, in words that stand on their own. Cite our code by file plus function (a line number may follow as a hint). Comments and docs do not name other projects or say where code came from: the license header of copied code and its `THIRD_PARTY_NOTICES.md` row carry that. `handover/` is the one exception: it is release 0.12.0, which hands installed launchers over to Triforce, and it names that project throughout.
## Writing

View File

@@ -85,6 +85,18 @@ cards).
it was found in): the first memory report asked Windows for a summary of the game's heap, and that summary holds
every allocation of the game while it is taken. It is now taken before play only (`mem_census = 1` asks for it in
play too, for a diagnosis).
- **Video memory staying over the card's budget once the frame rate falls under 60 fps** (reported on an RTX 4070
SUPER at 2560x1440 with DLSS and frame generation, where the frame rate went from 63 to 21 fps): a texture the game
has just let go of is kept out of the video memory pressure's reach for 2 seconds, counted in frames. Under 60 fps
the count did not shrink, so the 2 seconds were 4 at 30 fps and nearly 6 at 21, and the textures waiting them out
held 2.1 GB on that card. The wait is 2 seconds at any frame rate from 15 fps up now. On our own PC at a 30 fps
limit those textures held 209 MB on average where they held about 400. Not yet confirmed on a card that is really
short of video memory: please report how it goes (`tex_rate_low = 0` brings the old count back).
- **DLSS Frame Generation on frames without a scene** (the first frames of a menu, a fade or a load): generation
stayed on for up to 12 such frames without the camera data and the depth and motion buffers NVIDIA's library needs,
and the library logged errors and went on with the data of the frame before. Such a frame now tells it to start
anew and gives it no buffers, as NVIDIA's guide asks. Found in a review against NVIDIA's guides for DLSS and frame
generation, which the texture mip bias and the jitter were found to follow; not yet seen in a game by us.
- **Crashes and hangs found in a review of the whole runtime**; none was seen in our own play, and each fix has a check
of its own:
- a crash when the game read memory the lazy write-back was just handing to the GPU (NVIDIA and Intel cards);

24
docs/releases/v0.12.0.md Normal file
View File

@@ -0,0 +1,24 @@
NX2navi and Triforce have merged and go on as Triforce.
- This update turns NX2navi's launcher into the door to Triforce: one window that starts Triforce Setup, Triforce's
launcher.
- Nothing of NX2navi is removed or changed: your built game, your saves and your settings stay where they are.
- **Play NX2navi** in that window still starts the game you have, for as long as you want it.
- Triforce builds the game from your own files, as NX2navi did, and its Convert saves page brings your NX2navi saves
over.
## Good to know
- **Already have NX2navi:** the launcher updates itself to this version at its next check, as for any update. There is
nothing to download by hand.
- **New here:** get Triforce from https://triforceproject.com. This release carries no setup.
- **What is no longer in the launcher:** its own pages for settings, mods, rebuilding the game, saves and Add to
Steam. The game's own settings menu is unchanged.
- **Mods:** Play NX2navi starts the game as it is in its folder. A mod added after this update is not built in.
- **Steam:** a shortcut made with Add to Steam opens this window; choose Play NX2navi there.
- **Convert saves in Triforce v0.3.6** asks for the folder with your saves: it is the one the window shows as
*Your NX2navi game and saves* (Copy path).
- **Uninstalling NX2navi** from Windows' Apps list works as before and asks before it removes the game folder.
- **Linux:** the AppImage has no updater and is not changed by this release.
The window's source is in [`handover/`](https://github.com/tangerineturtle1/NX2navi/tree/main/handover).

5
handover/.gitignore vendored Normal file
View File

@@ -0,0 +1,5 @@
# Not kept in the repository (README.md, "In the repository"): the window's drawn pictures, the Triforce Setup a
# release carries, and what make_handover.py writes.
/bridge/launcher/art/
/bridge/launcher/triforce/TriforceSetup.exe
/release/

144
handover/README.md Normal file
View File

@@ -0,0 +1,144 @@
# NX2navi → Triforce: the launcher handover
NX2navi and Triforce have merged and go on as Triforce. This package is one NX2navi release that turns every
installed NX2navi launcher into the door to Triforce's launcher, through NX2navi's own update channel. Nothing new
has to be installed by hand, and nothing of NX2navi is removed from a player's PC.
It is made for whoever publishes NX2navi's releases (`tangerineturtle1/NX2navi`).
## What to do
1. Look at it first: the pictures in `preview/`, and the code in `bridge/launcher/` (four small Python files).
2. Upload the two files in `release/` to a new GitHub release **`v0.12.0`** of `tangerineturtle1/NX2navi`:
- `NX2navi-app-0.12.0.zip`
- `NX2navi-update.json`
3. Publish it as a **full release, not a pre-release**. Launchers on the Stable channel pass pre-releases over.
That is all. Every launcher sees it at its next check (when it opens, and every hour while it runs), downloads the
3.3 MB archive, checks its size and sha256 against the manifest and switches to it, exactly as for any update. With
Install Updates Automatically on (the default) that happens by itself while the game is not running.
`0.12.0` is newer than everything published as of 2026-11-14 (Stable `0.10.1`, Preview `0.11.0-rc4`). For another
number, see "Another version" below. The number must be plain (`0.12.0`, not `0.12.0-rc1`).
## What a player sees
After the update the launcher restarts, and instead of the old window this one opens (`preview/`):
- **Get Triforce** starts Triforce Setup, Triforce's launcher, which installs itself per user as NX2navi's did and
takes the player through building the game from their own files. When Triforce is already on the PC the button
reads **Open Triforce** and starts the installed one.
- **Play NX2navi** starts the game that is built on the PC (`ocarina.exe` in the game folder), as the launcher's
Play did. It is there for as long as the player wants it. It is not shown when no game was built.
- **Your game files** and **Your NX2navi game and saves**: the two paths from the launcher's settings, each with
Copy path, because Triforce Setup asks for the game files and its Convert saves page takes the saves from the
game folder.
- Links to triforceproject.com and the Discord.
The NX2navi shortcuts keep working and open this window. A Steam shortcut made by Add to Steam opens it too.
## What it touches, and what it does not
- **Not touched:** the built game, the saves in it, `data/launcher.json`, the build tools, the mods, the dump. The
old launcher version stays on the disk as `app/<old version>` (the stub's rollback needs it).
- **Written:** `app/0.12.0/` (the archive), `app/current.txt` and `app/previous.txt` (by the old launcher, as for any
update), `data/logs/handover.log`, `data/handover.json` (the time of the last update check), and, when the player
selects Get Triforce, a copy of `TriforceSetup.exe` in `%TEMP%\TriforceSetup\`.
- **Uninstalling NX2navi** (Windows' Apps list, which runs `NX2navi.exe --uninstall`) still works: the window asks,
then removes the shortcuts, the Apps entry and the same files and folders the launcher's own uninstall removed
(`winsys.OWNED`); the built game goes only on a second, separate Yes.
## Safety
- **A handover that cannot start rolls back.** `start.pyw` keeps the stub's contract: a failure before the window is
up ends the process with exit code 86 inside the stub's six seconds, and the stub goes back to the version before
and starts it with `--rolled-back`. That launcher then never installs this version by itself again
(`rolled_back`), as with any failed update.
- **The way back stays open.** Once an hour at most (and only while Check for Updates is on), the window asks the
same feed the launcher asked (`feed.json` of the version before; `feed_repo` in the settings still overrides it)
for a version newer than its own. When there is one it stages it the launcher's way: checked download,
`app/<version>`, a `runtime` the manifest names into `python.next`, then `previous.txt` and `current.txt`. So a
later NX2navi release, say `0.12.1`, replaces this window on every PC: a fixed handover, or a launcher again.
- **The Triforce Setup it carries is checked.** `launcher/triforce/setup.json` holds its size and sha256
(Triforce `v0.3.6`, sha256 `59fba57a…e7eb2e`, the file published at triforceproject.com); the window compares
before it starts it and starts nothing that differs. Setup updates itself from triforceproject.com when it
starts, so the carried copy does not have to be the newest.
- Only the standard library and tkinter are used, so it runs on the Python each installed launcher already has: the
manifest names no `runtime`, and no launcher downloads a Python for it.
- No game data and nothing derived from the game is in the archive.
## Looking at it before it is released
Pictures without showing a window or doing anything, from any installed launcher's Python:
```
python\pythonw.exe <unpacked NX2navi-app-0.12.0.zip>\launcher\start.pyw --selftest --shots <folder>
```
A private run through a real launcher: put the two release files in a release of another repository and set
`"feed_repo": "owner/name"` in that launcher's `data\launcher.json`, or serve them on the same PC and put
`{"type": "url", "url": "http://127.0.0.1:8000/NX2navi-update.json"}` in its `app\<version>\feed.json`.
## Another version, another Setup, other notes
Python 3.8 or later, no packages:
```
python make_handover.py --version 0.12.0
python make_handover.py --version 1.0.0 --setup TriforceSetup.exe --setup-version v0.3.7 --notes notes.md
```
It packs `bridge/` and writes `release/NX2navi-app-<version>.zip` and `release/NX2navi-update.json`. The window's
pictures are in `bridge/launcher/art/`; `tools/make_art.py` draws them again (Pillow and numpy) from `tools/art-src/`.
## What was tested, and what was not
`tests/e2e.py` (run 2026-11-14, all passed) copies an installed NX2navi launcher (versions 0.10.1 and 0.9.2 were on
the PC) and lets its own code and its own `NX2navi.exe` do the work against a feed on 127.0.0.1:
- `cli.py check` and `cli.py update` of both versions see, download, check, stage and switch to the handover;
- the path the window takes in the background (`feed.latest`, `updater.stage`, `staged_info`, `install_staged`);
- the real stub starts the handover with `--continue-update`, and nothing is rolled back;
- a handover that cannot start: exit code 86, and the stub goes back to the version before;
- a GitHub-shaped feed: offered on Stable as a full release, passed over as a pre-release;
- the way back: the handover stages a newer release, and refuses a file that does not match its sha256;
- the uninstall's plan is the same command line the launcher's own `winsys` builds for the same folder;
- the window drawn with NX2navi's own Python (3.14.9, Tk 9.0.4) at 125 % display scale.
Not tested: a release on the real GitHub repository; the 0.11.0 previews' launcher as a running program (in the
0.11.0-rc4 source, `feed.py`, `updater.py`, `versions.py`, `net.py`, `settings.py` and `start.pyw` are the same files
as 0.10.1's, byte for byte, and `cli.py` differs by six added lines); a person clicking through the window on a PC that has
an NX2navi game; Windows 10; display scales other than 125 %; the Linux AppImage, which has no launcher updater and
is not reached by this at all.
## For players who are new
This reaches installed launchers. `NX2navi-Setup-<version>.exe` on the old releases still installs the old launcher,
which then updates to the handover at its first check. The download links for new players should point at
https://triforceproject.com instead.
## Before publishing
Triforce Setup's Convert saves page finds NX2navi's saves by itself from the Triforce release after `v0.3.6` (in
`v0.3.6` the save folder is picked by hand there). The window tells players the page brings their saves over, so
publish this once that Triforce release is out. Nothing in this package has to change for it: Setup updates itself.
Triforce is free software under the GPL-3; its source, Triforce Setup's included, is at
https://git.triforceproject.com/triforce/source
## In the repository
This folder is the package's source. Three parts of the package are not kept here, and `make_handover.py` names the
one it misses when it is run without them:
- `bridge/launcher/art/`, the window's pictures: `tools/make_art.py` draws them from `tools/art-src/`.
- `bridge/launcher/triforce/TriforceSetup.exe`: `setup.json` beside it names the file release 0.12.0 carries (its
Triforce release, its size and its sha256). Put that file there, or give another with `--setup`.
- `release/`, which `make_handover.py` writes. The archive published as release 0.12.0 is `NX2navi-app-0.12.0.zip`,
3,333,988 bytes, sha256 `45c9c50d98fa4771c6d7330e8fa2353c9db1e8b65e3b37ef14174fb4b57e1793`.
`tests/e2e.py` ran again on 2026-11-14 against the launchers of 0.9.1, 0.9.2, 0.10.0, 0.10.1 and the 0.10.2 and
0.11.0 previews: all of its checks passed. The pictures the self-test takes with the window off the screen came out
empty on that PC (150 % display scale) while the window itself drew as in `preview/`, and the test does not look into
the pictures: look at them yourself. Release 0.12.0 was published on 2026-11-14 with Triforce `v0.3.6` as its newest
release, before the one "Before publishing" waits for: in `v0.3.6` the Convert saves page asks for the save folder.

View File

@@ -0,0 +1,18 @@
NX2navi is now Triforce
=======================
This version of NX2navi's launcher is the handover to Triforce: the two projects have merged and go on as Triforce.
launcher/start.pyw what NX2navi.exe starts, as in every version
launcher/handover.py the window: Get Triforce / Open Triforce, Play NX2navi, where your game files are
launcher/cli.py the command line's answer
launcher/selftest.py pictures of the window for a look before release (NX2navi.exe --selftest --shots DIR)
launcher/art/ the window's pictures (the Triforce crest; the fairy is NX2navi's logo)
launcher/triforce/ Triforce Setup, Triforce's launcher, as published at https://triforceproject.com,
with setup.json: its release, size and sha256. The window checks the file against
them before it starts it.
Triforce is free software under the GNU General Public License, version 3. Its source code, Triforce Setup's
included, is at https://git.triforceproject.com/triforce/source
Nothing in this folder is game data, and nothing in it is derived from the game.

View File

@@ -0,0 +1,20 @@
"""NX2navi's launcher without its window was launcher/cli.py, and its `update` command finishes an update by running
the NEW version's cli.py (`--root DIR continue-update OLD`). From this version on there is nothing left to finish:
NX2navi is now Triforce, and this version is the handover to it (handover.py). So every command says that and ends
with exit code 0, which lets an `update` started from a console end cleanly.
"""
import sys
TEXT = ('NX2navi is now Triforce: the two projects have merged and go on as Triforce.\n'
'This version of NX2navi\'s launcher is the handover: start NX2navi (its shortcut or NX2navi.exe) and choose\n'
'Get Triforce, or get Triforce from https://triforceproject.com\n'
'Nothing of NX2navi was removed: the built game, its saves and the settings are where they were.')
def main(argv=None):
print(TEXT, flush=True)
return 0
if __name__ == '__main__':
sys.exit(main())

View File

@@ -0,0 +1,883 @@
"""NX2navi is now Triforce: the window NX2navi's launcher becomes with this release.
NX2navi and Triforce have merged and go on as Triforce. This release takes the place of NX2navi's launcher in its own
update channel: the stub (NX2navi.exe) starts launcher/start.pyw of the version app/current.txt names, as it starts
every version, and start.pyw opens this window instead of the old launcher's. The window
- starts Triforce Setup, Triforce's launcher: the installed one when Triforce is on the PC already, else the copy
this release carries (launcher/triforce/TriforceSetup.exe, checked against the sha256 in setup.json beside it);
- still starts the NX2navi game that is built on the PC (ocarina.exe in the game folder), for whoever wants to go on
with it for now;
- shows where the player's game files and NX2navi game folder are, which Triforce Setup asks for.
It removes and changes nothing of NX2navi: the game folder, the saves in it, the settings (data/launcher.json) and the
build tools stay as they are, so Triforce Setup's Convert saves page finds the saves. Uninstalling NX2navi (Windows'
Apps list runs NX2navi.exe --uninstall) is done here as the old launcher did it.
It keeps the way back open: once an hour at most it asks the feed NX2navi's launcher asked (the GitHub releases of
feed.json's repository) for a version newer than this one and, when there is one, stages it exactly as the old
launcher did (checked download, app/<version>, previous.txt, current.txt), so the stub starts that version next time
and falls back to this one when it does not start. Whoever publishes NX2navi's releases can so replace this window.
Only the standard library and tkinter are used: the launcher's own Python (python/ beside NX2navi.exe) runs it.
"""
import ctypes
import hashlib
import json
import os
import re
import shutil
import subprocess
import sys
import tempfile
import threading
import time
import urllib.error
import urllib.parse
import urllib.request
import webbrowser
import zipfile
from pathlib import Path
HERE = Path(__file__).resolve().parent
APP = HERE.parent
ART = HERE / 'art'
SETUP_DIR = HERE / 'triforce'
SITE = 'https://triforceproject.com'
DISCORD = 'https://discord.gg/triforce'
DEFAULT_FEED = {'type': 'github', 'repo': 'tangerineturtle1/NX2navi'}
GITHUB_API = 'https://api.github.com'
MANIFEST = 'NX2navi-update.json'
CHECK_EVERY = 3600
CREATE_NO_WINDOW = 0x08000000
DETACHED_PROCESS = 0x00000008
CREATE_NEW_PROCESS_GROUP = 0x00000200
NX_UNINSTALL_KEY = r'Software\Microsoft\Windows\CurrentVersion\Uninstall\NX2navi'
TRIFORCE_UNINSTALL_KEY = r'Software\Microsoft\Windows\CurrentVersion\Uninstall\Triforce'
# what NX2navi's launcher put in its folder (its launcher/winsys.py OWNED): an uninstall deletes these and nothing else
OWNED = ('NX2navi.exe', 'app', 'python', 'python.next', 'pydeps', 'data', 'toolchain')
# ---------------------------------------------------------------- the launcher's folder
def launcher_root():
return Path(os.environ.get('NX2NAVI_LAUNCHER_ROOT') or APP.parents[1])
def read_text(path, default=''):
try:
return Path(path).read_text(encoding='utf-8-sig').strip()
except (OSError, UnicodeDecodeError):
return default
def write_text(path, text):
path = Path(path)
tmp = path.with_name(path.name + '.tmp')
tmp.write_text(text + '\n', encoding='utf-8')
os.replace(tmp, path)
def my_version():
return read_text(APP / 'VERSION').split('+', 1)[0].strip() or APP.name
def log(root, line):
"""One line into data/logs/handover.log; never an error."""
try:
logs = Path(root) / 'data' / 'logs'
logs.mkdir(parents=True, exist_ok=True)
with open(logs / 'handover.log', 'a', encoding='utf-8') as f:
f.write(time.strftime('%Y-%m-%d %H:%M:%S ') + line + '\n')
except OSError:
pass
def read_settings(root):
"""NX2navi's data/launcher.json as a dict ({} when it is not there or not JSON). Only read, never written."""
try:
data = json.loads((Path(root) / 'data' / 'launcher.json').read_text(encoding='utf-8-sig'))
return data if isinstance(data, dict) else {}
except (OSError, ValueError):
return {}
def nx_game(settings, root):
"""The NX2navi game built on this PC, as the path of its ocarina.exe, or None: the folder the settings name, else
the launcher's own game folder."""
for folder in (settings.get('game_dir'), Path(root) / 'game'):
try:
if folder and (Path(folder) / 'ocarina.exe').is_file():
return Path(folder) / 'ocarina.exe'
except (OSError, TypeError, ValueError):
pass
return None
def facts(root):
"""What the window shows and acts on, read from the PC."""
settings = read_settings(root)
game = nx_game(settings, root)
dump = settings.get('dump') if isinstance(settings.get('dump'), str) else ''
if dump and not os.path.exists(dump):
dump = ''
return {'triforce': triforce_installed(), 'game': game, 'dump': os.path.normpath(dump) if dump else '',
'game_dir': str(game.parent) if game else ''}
# ---------------------------------------------------------------- Triforce Setup
def triforce_installed():
"""The installed Triforce Setup (Triforce's launcher), or None: the folder its entry in Windows' Apps list names
(HKCU ...\\Uninstall\\Triforce, InstallLocation), else its default folder."""
folders = []
try:
import winreg
with winreg.OpenKey(winreg.HKEY_CURRENT_USER, TRIFORCE_UNINSTALL_KEY) as k:
folders.append(winreg.QueryValueEx(k, 'InstallLocation')[0])
except (OSError, ImportError):
pass
if os.environ.get('LOCALAPPDATA'):
folders.append(os.path.join(os.environ['LOCALAPPDATA'], 'Programs', 'Triforce'))
for f in folders:
try:
exe = Path(f) / 'TriforceSetup.exe'
if exe.is_file():
return exe
except (OSError, TypeError, ValueError):
pass
return None
def sha256_file(path):
h = hashlib.sha256()
with open(path, 'rb') as f:
for block in iter(lambda: f.read(1 << 20), b''):
h.update(block)
return h.hexdigest()
def bundled_setup():
"""(path, info) of the Triforce Setup this release carries, after checking it against setup.json (its size and
sha256, written when the release was made). OSError or ValueError when it is missing or is not that file."""
info = json.loads((SETUP_DIR / 'setup.json').read_text(encoding='utf-8'))
exe = SETUP_DIR / info['name']
if exe.stat().st_size != info['size'] or sha256_file(exe).lower() != info['sha256'].lower():
raise ValueError(f'{exe.name} is not the file this release was made with')
return exe, info
def start_detached(cmd, cwd):
flags = DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP if os.name == 'nt' else 0
return subprocess.Popen(cmd, cwd=str(cwd), stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL, close_fds=True, creationflags=flags)
def start_triforce(installed):
"""Start Triforce's launcher: the installed one, else this release's copy from a folder of its own in the temp
folder (Setup then installs itself, as it does for anyone who downloads it). Returns what was started."""
if installed:
start_detached([str(installed)], Path(installed).parent)
return str(installed)
src, info = bundled_setup()
folder = Path(tempfile.gettempdir()) / 'TriforceSetup'
folder.mkdir(parents=True, exist_ok=True)
exe = folder / info['name']
try:
same = exe.is_file() and exe.stat().st_size == info['size'] and sha256_file(exe).lower() == info['sha256'].lower()
except OSError:
same = False
if not same:
shutil.copyfile(src, exe)
start_detached([str(exe)], folder)
return str(exe)
def start_game(exe):
"""Start the NX2navi game from its own folder, as NX2navi's launcher did (its console stays hidden)."""
exe = Path(exe)
flags = CREATE_NO_WINDOW | CREATE_NEW_PROCESS_GROUP if os.name == 'nt' else 0
return subprocess.Popen([str(exe)], cwd=str(exe.parent), stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL, close_fds=True, creationflags=flags)
def copy_to_clipboard(text):
"""Windows' clipboard, set through Win32 so that it outlives this window. False when that did not work."""
if os.name != 'nt':
return False
try:
from ctypes import wintypes
u, k = ctypes.WinDLL('user32'), ctypes.WinDLL('kernel32')
k.GlobalAlloc.restype, k.GlobalAlloc.argtypes = wintypes.HGLOBAL, (wintypes.UINT, ctypes.c_size_t)
k.GlobalLock.restype, k.GlobalLock.argtypes = ctypes.c_void_p, (wintypes.HGLOBAL,)
k.GlobalUnlock.argtypes = (wintypes.HGLOBAL,)
u.OpenClipboard.argtypes = (wintypes.HWND,)
u.SetClipboardData.restype, u.SetClipboardData.argtypes = wintypes.HANDLE, (wintypes.UINT, wintypes.HANDLE)
data = ctypes.create_unicode_buffer(text)
if not u.OpenClipboard(None):
return False
try:
u.EmptyClipboard()
h = k.GlobalAlloc(0x0002, ctypes.sizeof(data)) # GMEM_MOVEABLE
p = k.GlobalLock(h)
ctypes.memmove(p, data, ctypes.sizeof(data))
k.GlobalUnlock(h)
return bool(u.SetClipboardData(13, h)) # CF_UNICODETEXT
finally:
u.CloseClipboard()
except (AttributeError, OSError):
return False
# ---------------------------------------------------------------- the way back: a newer release of the old feed
class UpdateError(Exception):
pass
_VERSION = re.compile(r'^\s*v?(\d+(?:\.\d+)*)(?:-([0-9A-Za-z.-]+))?(?:\+([0-9A-Za-z.-]+))?\s*$')
def parse_version(text):
"""As NX2navi's launcher orders versions (its launcher/versions.py): a pre-release sorts before its release."""
m = _VERSION.match(text or '')
if not m:
raise ValueError(f'not a version: {text!r}')
nums = tuple(int(x) for x in m.group(1).split('.'))
nums = nums + (0,) * (4 - len(nums)) if len(nums) < 4 else nums
return nums, (1,) if m.group(2) is None else (0, m.group(2))
def newer(a, b):
return parse_version(a) > parse_version(b)
def display(version):
return (version or '').split('+', 1)[0].strip()
def _is_local(url):
return urllib.parse.urlsplit(url).hostname in ('127.0.0.1', 'localhost', '::1')
def _allowed(url):
"""https, or plain http to this PC (tests)."""
scheme = urllib.parse.urlsplit(url).scheme
return scheme == 'https' or (scheme == 'http' and _is_local(url))
class _NoDowngrade(urllib.request.HTTPRedirectHandler):
def redirect_request(self, req, fp, code, msg, headers, newurl):
if not _allowed(newurl):
raise UpdateError(f'redirected to {newurl}, which is not https')
return super().redirect_request(req, fp, code, msg, headers, newurl)
_opener = None
def _open(url, headers=None, timeout=30):
global _opener
if not _allowed(url):
raise UpdateError(f'not an https address: {url}')
if _opener is None:
try: # Windows' own certificate check, when the launcher's Python brings the package for it
import truststore
truststore.inject_into_ssl()
except Exception:
pass
_opener = urllib.request.build_opener(_NoDowngrade)
req = urllib.request.Request(url, headers={'User-Agent': f'NX2navi-handover/{my_version()}', **(headers or {})})
return _opener.open(req, timeout=timeout)
def get_json(url, headers=None):
with _open(url, headers) as r:
return json.loads(r.read(8 << 20).decode('utf-8-sig'))
def load_feed(root, settings):
"""Where NX2navi's launcher looked for releases: the feed.json of the version before this one (or of any other
version in app/), with the settings' feed_repo over it, as there."""
feed = dict(DEFAULT_FEED)
apps = Path(root) / 'app'
names = [read_text(apps / 'previous.txt')]
try:
names += sorted(d.name for d in apps.iterdir() if d.is_dir())
except OSError:
pass
for name in names:
try:
if name and name != APP.name:
got = json.loads((apps / name / 'feed.json').read_text(encoding='utf-8-sig'))
if isinstance(got, dict) and got.get('type') in ('github', 'url'):
feed.update(got)
break
except (OSError, ValueError):
continue
own = APP / 'feed.json' # a release of this window may name its feed itself
try:
got = json.loads(own.read_text(encoding='utf-8-sig'))
if isinstance(got, dict) and got.get('type') in ('github', 'url'):
feed = got
except (OSError, ValueError):
pass
if isinstance(settings.get('feed_repo'), str) and settings.get('feed_repo'):
feed = {'type': 'github', 'repo': settings['feed_repo']}
return feed
def _check_manifest(m, where):
if not isinstance(m, dict) or m.get('format') != 1 or 'version' not in m or not isinstance(m.get('app'), dict):
raise UpdateError(f'{where}: not an NX2navi update manifest')
parse_version(m['version'])
return m
def latest(feed, channel='stable'):
"""(manifest, {file name: address}) of the newest release the feed offers on the channel, or None."""
if feed.get('type') == 'url':
url = feed['url']
m = _check_manifest(get_json(url), url)
base = url.rsplit('/', 1)[0] + '/'
return m, {e['name']: urllib.parse.urljoin(base, e['name']) for e in (m.get('app'), m.get('runtime'))
if isinstance(e, dict) and isinstance(e.get('name'), str)}
api = str(feed.get('api', GITHUB_API)).rstrip('/')
headers = {'Accept': 'application/vnd.github+json', 'X-GitHub-Api-Version': '2022-11-28'}
for r in get_json(f'{api}/repos/{feed["repo"]}/releases?per_page=20', headers):
if r.get('draft') or (r.get('prerelease') and channel != 'preview'):
continue
assets = {a['name']: a['browser_download_url'] for a in r.get('assets', [])}
if MANIFEST not in assets:
continue
with _open(assets[MANIFEST]) as f:
m = json.loads(f.read(8 << 20).decode('utf-8-sig'))
return _check_manifest(m, f'{r.get("tag_name")}/{MANIFEST}'), assets
return None
def _entry(manifest, key, assets):
"""(name, address, size, sha256) of the manifest's 'app' or 'runtime' file; refused without a plain file name, a
size and a sha256, as NX2navi's launcher refuses it."""
e = manifest.get(key) or {}
name, size, sha = e.get('name'), e.get('size'), e.get('sha256')
if (not isinstance(name, str) or not name or name in ('.', '..') or name != Path(name).name
or '/' in name or '\\' in name or ':' in name):
raise UpdateError(f'the manifest names its {key} file {name!r}, not a plain file name')
if (not isinstance(size, int) or isinstance(size, bool) or size <= 0 or not isinstance(sha, str)
or not re.fullmatch(r'[0-9a-fA-F]{64}', sha)):
raise UpdateError(f'the manifest gives no size and sha256 for {name}')
if not assets.get(name):
raise UpdateError(f'the release lacks {name}')
return name, assets[name], size, sha.lower()
def download(url, dest, size, sha):
dest = Path(dest)
dest.parent.mkdir(parents=True, exist_ok=True)
part = dest.with_name(dest.name + '.handover-part')
h, got = hashlib.sha256(), 0
try:
with _open(url, timeout=60) as r, open(part, 'wb') as f:
for block in iter(lambda: r.read(1 << 18), b''):
got += len(block)
if got > size:
raise UpdateError(f'{dest.name} is larger than the manifest says')
h.update(block)
f.write(block)
if got != size or h.hexdigest() != sha:
raise UpdateError(f'{dest.name} does not match the manifest (size or sha256)')
os.replace(part, dest)
finally:
try:
part.unlink()
except OSError:
pass
return dest
def safe_extract(zip_path, dest):
"""Unpack a zip into dest, refusing absolute names and names that climb out of it."""
dest = Path(dest)
root = dest.resolve()
with zipfile.ZipFile(zip_path) as z:
for info in z.infolist():
name = info.filename.replace('\\', '/')
if name.startswith('/') or ':' in name.split('/')[0] or '..' in name.split('/'):
raise UpdateError(f'{Path(zip_path).name}: unsafe path {info.filename}')
target = (dest / name).resolve()
if target != root and root not in target.parents:
raise UpdateError(f'{Path(zip_path).name}: unsafe path {info.filename}')
z.extractall(dest)
def stage(root, manifest, assets):
"""Make the release the version the stub starts next, as NX2navi's launcher did (its launcher/updater.py): the app
archive, checked, unpacks into app/<version>; a Python the release needs unpacks into python.next, which the stub
swaps in at its next start; then previous.txt names this version and current.txt the new one. Returns the
version. Nothing is switched when any step fails."""
root = Path(root)
apps, downloads = root / 'app', root / 'data' / 'downloads'
ver = display(manifest['version'])
name, url, size, sha = _entry(manifest, 'app', assets)
archive = download(url, downloads / name, size, sha)
part, final = apps / f'{ver}.part', apps / ver
shutil.rmtree(part, ignore_errors=True)
part.mkdir(parents=True)
try:
safe_extract(archive, part)
if display(read_text(part / 'VERSION')) != ver or not (part / 'launcher' / 'start.pyw').is_file():
raise UpdateError(f'{name} does not hold version {ver} with a launcher')
if final.exists():
if final.resolve() == APP.resolve():
raise UpdateError(f'{ver} is the version that runs now')
shutil.rmtree(final)
os.replace(part, final)
except BaseException:
shutil.rmtree(part, ignore_errors=True)
raise
finally:
try:
archive.unlink()
except OSError:
pass
rt = manifest.get('runtime')
if isinstance(rt, dict) and rt.get('id') and rt.get('id') != read_text(root / 'python' / 'runtime-id.txt'):
name, url, size, sha = _entry(manifest, 'runtime', assets)
archive = download(url, downloads / name, size, sha)
nxt = root / 'python.next'
part = root / 'python.next.part'
shutil.rmtree(nxt, ignore_errors=True)
shutil.rmtree(part, ignore_errors=True)
part.mkdir()
try:
safe_extract(archive, part)
if not (part / 'python.exe').is_file() or read_text(part / 'runtime-id.txt') != rt['id']:
raise UpdateError(f'{name} is not the runtime {rt["id"]}')
os.replace(part, nxt)
except BaseException:
shutil.rmtree(part, ignore_errors=True)
shutil.rmtree(final, ignore_errors=True)
raise
finally:
try:
archive.unlink()
except OSError:
pass
write_text(apps / 'previous.txt', APP.name)
write_text(apps / 'current.txt', ver)
return ver
def check_for_newer(root, settings=None, force=False):
"""Ask the feed for a version newer than this one and stage it. Returns the staged version, or '' when there is
none, the player turned update checks off, or the last check was less than an hour ago. Raises on a failure."""
root = Path(root)
settings = read_settings(root) if settings is None else settings
if not force:
if settings.get('auto_check') is False:
return ''
state = root / 'data' / 'handover.json'
try:
last = float(json.loads(state.read_text(encoding='utf-8')).get('last_check', 0))
except (OSError, ValueError, TypeError, AttributeError):
last = 0.0
if 0 <= time.time() - last < CHECK_EVERY:
return ''
try:
state.parent.mkdir(parents=True, exist_ok=True)
write_text(state, json.dumps({'last_check': time.time()}))
except OSError:
pass
channel = settings.get('channel') if settings.get('channel') in ('stable', 'preview') else 'stable'
found = latest(load_feed(root, settings), channel)
if not found or not newer(found[0]['version'], my_version()):
return ''
return stage(root, *found)
def check_in_background(root):
def work():
try:
ver = check_for_newer(root)
if ver:
log(root, f'a newer release, {ver}, is staged: NX2navi.exe starts it next time')
except Exception as e: # no network, a refused request, a damaged file: tried again at a later start
log(root, f'update check: {type(e).__name__}: {e}')
t = threading.Thread(target=work, name='handover-update', daemon=True)
t.start()
return t
# ---------------------------------------------------------------- uninstalling NX2navi
def message_box(text, title='NX2navi', flags=0x40):
try:
return ctypes.windll.user32.MessageBoxW(None, text, title, flags)
except (AttributeError, OSError):
print(text, file=sys.stderr)
return 0
def ansi_safe(path):
try:
str(path).encode('mbcs', errors='strict')
return True
except (UnicodeEncodeError, LookupError):
return False
def uninstall_plan(root, remove_game=''):
"""(paths to delete with all they hold, folders to delete only when that left them empty), as NX2navi's launcher
planned it (its launcher/winsys.py uninstall_plan)."""
root = Path(root)
gone = [Path(remove_game)] if remove_game else []
gone += [root / n for n in OWNED]
try:
gone += sorted(root.glob('python.old-*'))
except OSError:
pass
empty = [root]
if not ansi_safe(root): # the build tools then went to <system drive>\NX2navi\toolchain
tools = Path(os.environ.get('SystemDrive', 'C:') + '\\') / 'NX2navi' / 'toolchain'
gone.append(tools)
empty.append(tools.parent)
return gone, empty
def uninstall_script(gone, empty):
"""The cmd.exe line that deletes them once this process has exited. A path holding % is left out: cmd.exe expands
%NAME% even inside quotes."""
steps = ['ping -n 3 127.0.0.1 >nul']
for p in gone:
if '%' not in str(p):
steps.append(f'(if exist "{p}\\*" (rmdir /s /q "{p}") else if exist "{p}" (del /f /q "{p}"))')
steps += [f'rmdir "{p}" 2>nul' for p in empty if '%' not in str(p)]
return ' & '.join(steps)
def uninstall(root):
"""Windows' Apps list > NX2navi > Uninstall (NX2navi.exe --uninstall): asks, then removes the shortcuts, the Apps
entry and what the launcher put in its folder. The built game goes only when the player says so in a second
question; the game dump is never touched. Triforce is not touched either."""
root = Path(root)
if message_box('Remove NX2navi from this PC?\n\nIts launcher, its build tools and its shortcuts are removed. '
'Your game dump is never touched, and neither is Triforce.', 'Uninstall NX2navi', 0x24) != 6:
return 0
game = nx_game(read_settings(root), root)
remove_game = ''
if game and len(game.parent.parts) > 1:
if message_box(f'Also remove the NX2navi game in\n{game.parent}?\n\nYour NX2navi saves are in that folder. '
'Choose No to keep it, for instance until Triforce has taken your saves over.',
'Uninstall NX2navi', 0x124) == 6: # Yes / No, No preselected
remove_game = str(game.parent)
log(root, f'uninstall (game folder too: {bool(remove_game)})')
stub = root / 'NX2navi.exe'
if stub.is_file():
try:
subprocess.run([str(stub), '--shortcuts', 'off'], timeout=30, capture_output=True,
creationflags=CREATE_NO_WINDOW)
except (OSError, subprocess.TimeoutExpired):
pass
try:
import winreg
winreg.DeleteKey(winreg.HKEY_CURRENT_USER, NX_UNINSTALL_KEY)
except (OSError, ImportError):
pass
gone, empty = uninstall_plan(root, remove_game)
subprocess.Popen(f'cmd.exe /c "{uninstall_script(gone, empty)}"', cwd=os.environ.get('TEMP', 'C:\\'),
stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, close_fds=True,
creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP)
return 0
# ---------------------------------------------------------------- the window
W, H = 820, 480 # the composition, in units; tools/make_art.py draws the hall for it
SCALES = (1.0, 1.25, 1.5, 2.0)
TEXT_HI, TEXT, TEXT_DIM, TEXT_FAINT = '#F3EAD5', '#CDC2A8', '#948A70', '#5F5744'
GOLD, GOLD_300, LINE, LINE_GOLD = '#E0B23E', '#F7DC8C', '#2C2518', '#4A3C1B'
BAD = '#FF6B76'
HAIR = '\u200a'
def dpi_aware():
if os.name != 'nt':
return
try:
ctypes.windll.shcore.SetProcessDpiAwareness(1)
except (AttributeError, OSError):
try:
ctypes.windll.user32.SetProcessDPIAware()
except (AttributeError, OSError):
pass
def caps(text, gaps=1):
"""Letter-spaced capitals: hair spaces between the letters, as the Triforce client sets its labels."""
return (HAIR * gaps).join(text.upper())
class App:
def __init__(self, tk_root, root, found, argv=(), offscreen=False):
import tkinter as tk
import tkinter.font as tkfont
self.tk, self.tkfont = tk, tkfont
self.win, self.root, self.found, self.argv = tk_root, Path(root), dict(found), list(argv)
self.status, self.status_color = '', GOLD_300
self.game_proc = None
self.closing = False
dpi = self.win.winfo_fpixels('1i') / 96.0
self.s = max([s for s in SCALES if s <= dpi + 0.01] or [1.0])
tag = str(round(self.s * 100))
self.img = {n: tk.PhotoImage(master=self.win, file=str(ART / f'{n}_{tag}.png')) for n in ('hall', 'btn')}
self.img['btn_hi'] = tk.PhotoImage(master=self.win, file=str(ART / f'btn_{tag}_hi.png'))
self.icons = [tk.PhotoImage(master=self.win, file=str(ART / f'icon_{n}.png')) for n in (64, 32, 16)]
families = set(tkfont.families(self.win))
serif = next((f for f in ('Palatino Linotype', 'Book Antiqua', 'Georgia') if f in families), 'Times New Roman')
sans = next((f for f in ('Segoe UI', 'Tahoma') if f in families), 'Arial')
semi = 'Segoe UI Semibold' if 'Segoe UI Semibold' in families else sans
def font(family, px, weight='normal'):
return tkfont.Font(root=self.win, family=family, size=-max(8, round(px * self.s)), weight=weight)
self.f_title = font(serif, 31)
self.f_body = font(sans, 14)
self.f_small = font(sans, 12.5)
self.f_label = font(semi, 10.5, 'normal' if semi != sans else 'bold')
self.f_button = font(serif, 13, 'bold')
self.f_second = font(sans, 13)
w, h = round(W * self.s), round(H * self.s)
self.win.title('Triforce')
self.win.resizable(False, False)
self.win.configure(bg='#0A0908')
try:
self.win.iconphoto(True, *self.icons)
except tk.TclError:
pass
x = -32000 if offscreen else max(0, (self.win.winfo_screenwidth() - w) // 2)
y = -32000 if offscreen else max(0, (self.win.winfo_screenheight() - h) // 2 - round(20 * self.s))
self.win.geometry(f'{w}x{h}+{x}+{y}')
self.c = tk.Canvas(self.win, width=w, height=h, highlightthickness=0, bd=0, bg='#0A0908')
self.c.pack()
self.win.protocol('WM_DELETE_WINDOW', self.close)
self.win.bind('<Return>', lambda e: self.primary())
self.win.bind('<Escape>', lambda e: self.close())
self.draw()
self.win.after(60, self.dress)
# ---- drawing
def u(self, v):
return round(v * self.s)
def dress(self):
"""The title bar in the hall's colours (Windows 11; Windows 10 takes the dark frame)."""
if os.name != 'nt':
return
try:
user32, dwm = ctypes.windll.user32, ctypes.windll.dwmapi
user32.GetParent.restype, user32.GetParent.argtypes = ctypes.c_void_p, [ctypes.c_void_p]
hwnd = ctypes.c_void_p(user32.GetParent(self.win.winfo_id()) or self.win.winfo_id())
for attr, value in ((20, 1), (19, 1), (35, 0x00090A0B), (36, 0x003EB2E0), (34, 0x001E546E)):
v = ctypes.c_int(value)
dwm.DwmSetWindowAttribute(hwnd, attr, ctypes.byref(v), ctypes.sizeof(v))
except (AttributeError, OSError):
pass
def wrap(self, text, font, width):
lines, line = [], ''
for word in text.split():
trial = (line + ' ' + word).strip()
if line and font.measure(trial) > width:
lines.append(line)
line = word
else:
line = trial
return lines + [line] if line else lines
def fit(self, text, font, width):
"""A path shortened in its middle until it fits."""
if font.measure(text) <= width:
return text
head, tail = text[:len(text) // 2], text[len(text) // 2:]
while head and tail and font.measure(head + ' … ' + tail) > width:
if len(head) >= len(tail):
head = head[:-1]
else:
tail = tail[1:]
return head + ' … ' + tail
def link(self, tag, on, rest, hot):
self.c.tag_bind(tag, '<Enter>', lambda e: (self.c.itemconfigure(tag, fill=hot), self.c.configure(cursor='hand2')))
self.c.tag_bind(tag, '<Leave>', lambda e: (self.c.itemconfigure(tag, fill=rest), self.c.configure(cursor='')))
self.c.tag_bind(tag, '<Button-1>', lambda e: on())
def draw(self):
c, u = self.c, self.u
c.delete('all')
c.configure(cursor='')
c.create_image(0, 0, anchor='nw', image=self.img['hall'])
left, col = u(52), u(348)
y = u(54)
# the eyebrow: a short rule, then the label
c.create_line(left, y + u(6), left + u(20), y + u(6), fill=LINE_GOLD)
c.create_text(left + u(30), y, anchor='nw', text=caps('NX2navi × Triforce'), font=self.f_label, fill=GOLD)
y += u(28)
c.create_text(left - u(1), y, anchor='nw', text='NX2navi is now Triforce', font=self.f_title, fill=TEXT_HI)
y += u(52)
body = ('The two projects have merged and go on under one name. Triforce is your launcher from here: it '
'builds the game from your own files, as NX2navi did, and its Convert saves page brings your '
'NX2navi saves over.')
for line in self.wrap(body, self.f_body, col):
c.create_text(left, y, anchor='nw', text=line, font=self.f_body, fill=TEXT)
y += u(22)
y += u(14)
rows = []
if self.found.get('dump'):
rows.append(('Your game files', self.found['dump'], 'copy_dump'))
if self.found.get('game_dir'):
rows.append(('Your NX2navi game and saves', self.found['game_dir'], 'copy_game'))
for label, path, tag in rows:
c.create_text(left, y, anchor='nw', text=caps(label), font=self.f_label, fill=TEXT_DIM)
copy_w = self.f_small.measure('Copy path')
c.create_text(left, y + u(17), anchor='nw', text=self.fit(path, self.f_small, col - copy_w - u(18)),
font=self.f_small, fill=TEXT)
c.create_text(left + col, y + u(17), anchor='ne', text='Copy path', font=self.f_small, fill=TEXT_DIM,
tags=(tag,))
self.link(tag, lambda p=path: self.copy(p), TEXT_DIM, GOLD_300)
y += u(44)
# the actions, on a fixed line near the foot
by = u(370)
if self.status: # (two lines at most, between the rows above, which end at 316, and the buttons)
for i, line in enumerate(self.wrap(self.status, self.f_small, col)[:2]):
c.create_text(left, by - u(44) + i * u(18), anchor='nw', text=line, font=self.f_small,
fill=self.status_color)
bw, bh = u(196), u(44)
c.create_image(left, by, anchor='nw', image=self.img['btn'], tags=('primary', 'primary_bar'))
label = 'Open Triforce' if self.found.get('triforce') else 'Get Triforce'
c.create_text(left + bw // 2, by + bh // 2 + u(1), text=caps(label, 2), font=self.f_button, fill='#F9E7A8',
tags=('primary',))
c.create_text(left + bw // 2, by + bh // 2, text=caps(label, 2), font=self.f_button, fill='#2A1C05',
tags=('primary',))
c.tag_bind('primary', '<Enter>', lambda e: (c.itemconfigure('primary_bar', image=self.img['btn_hi']),
c.configure(cursor='hand2')))
c.tag_bind('primary', '<Leave>', lambda e: (c.itemconfigure('primary_bar', image=self.img['btn']),
c.configure(cursor='')))
c.tag_bind('primary', '<Button-1>', lambda e: self.primary())
if self.found.get('game'):
sx, sw, sh = left + bw + u(14), u(138), u(36)
sy = by + (bh - sh) // 2
c.create_rectangle(sx, sy, sx + sw, sy + sh, outline=LINE, fill='#0C0B09', tags=('second', 'second_box'))
c.create_text(sx + sw // 2, sy + sh // 2, text='Play NX2navi', font=self.f_second, fill=TEXT,
tags=('second', 'second_text'))
c.tag_bind('second', '<Enter>', lambda e: (c.itemconfigure('second_box', outline='#8C7230'),
c.itemconfigure('second_text', fill=TEXT_HI),
c.configure(cursor='hand2')))
c.tag_bind('second', '<Leave>', lambda e: (c.itemconfigure('second_box', outline=LINE),
c.itemconfigure('second_text', fill=TEXT),
c.configure(cursor='')))
c.tag_bind('second', '<Button-1>', lambda e: self.play())
# the foot: what stays, and where Triforce lives
fy = u(H - 36)
c.create_line(left, fy - u(12), left + col, fy - u(12), fill=LINE)
c.create_text(left, fy, anchor='nw', font=self.f_small, fill=TEXT_DIM,
text='Nothing of NX2navi is removed or changed.')
# (under the crest, centred on its column: tools/make_art.py CX)
site_w, dot_w, disc_w = (self.f_small.measure(t) for t in ('triforceproject.com', '·', 'Discord'))
gap = u(12)
sx = u(632) - (site_w + dot_w + disc_w + 2 * gap) // 2
c.create_text(sx, fy, anchor='nw', text='triforceproject.com', font=self.f_small, fill=TEXT_DIM, tags=('site',))
self.link('site', lambda: self.browse(SITE), TEXT_DIM, GOLD_300)
c.create_text(sx + site_w + gap, fy, anchor='nw', text='·', font=self.f_small, fill=TEXT_FAINT)
c.create_text(sx + site_w + dot_w + 2 * gap, fy, anchor='nw', text='Discord', font=self.f_small,
fill=TEXT_DIM, tags=('discord',))
self.link('discord', lambda: self.browse(DISCORD), TEXT_DIM, GOLD_300)
# ---- actions
def say(self, text, bad=False):
self.status, self.status_color = text, BAD if bad else GOLD_300
self.draw()
def copy(self, path):
self.say('Copied. Paste it where Triforce Setup asks for it.' if copy_to_clipboard(path)
else 'The path could not be copied.', bad=False)
def browse(self, url):
try:
webbrowser.open(url)
except Exception:
self.say(url)
def primary(self):
if self.closing:
return
try:
started = start_triforce(self.found.get('triforce'))
except Exception as e:
log(self.root, f'Triforce Setup could not be started: {type(e).__name__}: {e}')
self.say('Triforce Setup could not be started from here. Get it from triforceproject.com, which opens '
'now.', bad=True)
self.browse(SITE)
return
log(self.root, f'started {started}')
self.closing = True
self.say('Triforce is opening.')
self.win.after(700, self.win.destroy)
def play(self):
if self.closing or not self.found.get('game'):
return
try:
self.game_proc = start_game(self.found['game'])
except OSError as e:
log(self.root, f'{self.found["game"]}: {e}')
self.say(f'The NX2navi game could not be started: {e}', bad=True)
return
log(self.root, f'started {self.found["game"]}')
self.closing = True
self.win.withdraw()
self.wait_game()
def wait_game(self):
"""Stay (hidden) while the game runs, as the old launcher did: whoever started this window (Steam, with
--play) sees the game as running for that time."""
if self.game_proc is not None and self.game_proc.poll() is None:
self.win.after(1000, self.wait_game)
else:
self.win.destroy()
def close(self):
self.win.destroy()
def main(argv=None):
"""The window. Returns the process's exit code; raises when the window cannot be made (start.pyw then lets the
stub go back to the version before)."""
argv = list(sys.argv[1:] if argv is None else argv)
root = launcher_root()
if '--uninstall' in argv:
return uninstall(root)
selftest = os.environ.get('NX2NAVI_HANDOVER_TEST') or ('--selftest' in argv and '.') or ''
if '--shots' in argv and argv.index('--shots') + 1 < len(argv):
selftest = argv[argv.index('--shots') + 1]
dpi_aware()
try:
ctypes.windll.shell32.SetCurrentProcessExplicitAppUserModelID('Triforce.NX2naviHandover')
except (AttributeError, OSError):
pass
import tkinter as tk
win = tk.Tk(className='Triforce')
app = App(win, root, facts(root), argv, offscreen=bool(selftest))
log(root, f'window up: version {my_version()}, arguments {argv}, Triforce installed: '
f'{bool(app.found["triforce"])}, NX2navi game: {bool(app.found["game"])}')
if selftest:
import selftest as st
win.after(300, lambda: st.run(app, selftest))
else:
check_in_background(root)
win.mainloop()
return 0

View File

@@ -0,0 +1,126 @@
"""Pictures of the handover window without showing it and without doing anything: for whoever wants to look at it
before it is released, and for the tests.
NX2navi.exe --selftest --shots DIR through the stub of an installed launcher
pythonw start.pyw --selftest --shots DIR directly
NX2NAVI_HANDOVER_TEST=DIR the same from the environment (a test that starts the stub)
The window opens far off the screen, each state is drawn and saved as handover-<state>.png (the window as Windows
draws it, title bar included), state.json says what was found on this PC, and the process ends with exit code 0.
Started through the environment it first stays until it is eight seconds old, longer than the stub waits for a
failed start. No button is pressed, nothing is started, downloaded or written outside DIR.
"""
import ctypes
import json
import os
import struct
import time
import zlib
from pathlib import Path
import handover
BORN = time.time()
def write_png(path, w, h, rgb):
def chunk(kind, data):
return struct.pack('>I', len(data)) + kind + data + struct.pack('>I', zlib.crc32(kind + data))
rows = b''.join(b'\0' + rgb[y * w * 3:(y + 1) * w * 3] for y in range(h))
Path(path).write_bytes(b'\x89PNG\r\n\x1a\n' + chunk(b'IHDR', struct.pack('>IIBBBBB', w, h, 8, 2, 0, 0, 0))
+ chunk(b'IDAT', zlib.compress(rows, 6)) + chunk(b'IEND', b''))
def snap(win, path):
"""The window as Windows draws it (PrintWindow: it need not be on the screen). False where that is not possible."""
if os.name != 'nt':
return False
from ctypes import wintypes
user32, gdi32 = ctypes.WinDLL('user32'), ctypes.WinDLL('gdi32')
P = ctypes.c_void_p
user32.GetParent.restype, user32.GetParent.argtypes = P, [P]
user32.GetWindowRect.argtypes = [P, ctypes.POINTER(wintypes.RECT)]
user32.GetWindowDC.restype, user32.GetWindowDC.argtypes = P, [P]
user32.ReleaseDC.argtypes = [P, P]
user32.PrintWindow.argtypes = [P, P, ctypes.c_uint]
gdi32.CreateCompatibleDC.restype, gdi32.CreateCompatibleDC.argtypes = P, [P]
gdi32.CreateCompatibleBitmap.restype, gdi32.CreateCompatibleBitmap.argtypes = P, [P, ctypes.c_int, ctypes.c_int]
gdi32.SelectObject.restype, gdi32.SelectObject.argtypes = P, [P, P]
gdi32.GetDIBits.argtypes = [P, P, ctypes.c_uint, ctypes.c_uint, P, P, ctypes.c_uint]
gdi32.DeleteObject.argtypes = [P]
gdi32.DeleteDC.argtypes = [P]
hwnd = user32.GetParent(win.winfo_id()) or win.winfo_id()
r = wintypes.RECT()
user32.GetWindowRect(hwnd, ctypes.byref(r))
w, h = r.right - r.left, r.bottom - r.top
if w <= 0 or h <= 0:
return False
dc = user32.GetWindowDC(hwnd)
mem = gdi32.CreateCompatibleDC(dc)
bmp = gdi32.CreateCompatibleBitmap(dc, w, h)
old = gdi32.SelectObject(mem, bmp)
try:
user32.PrintWindow(hwnd, mem, 2) # PW_RENDERFULLCONTENT
header = struct.pack('<IiiHHIIiiII', 40, w, -h, 1, 32, 0, 0, 0, 0, 0, 0)
info = ctypes.create_string_buffer(header, 40 + 12)
buf = ctypes.create_string_buffer(w * h * 4)
if not gdi32.GetDIBits(mem, bmp, 0, h, buf, info, 0):
return False
raw = buf.raw
rgb = bytearray(w * h * 3)
rgb[0::3], rgb[1::3], rgb[2::3] = raw[2::4], raw[1::4], raw[0::4]
write_png(path, w, h, bytes(rgb))
return True
finally:
gdi32.SelectObject(mem, old)
gdi32.DeleteObject(bmp)
gdi32.DeleteDC(mem)
user32.ReleaseDC(hwnd, dc)
SAMPLE = {'dump': r'D:\Games\dumps\My game dump [0400486024dec000][v131072].nup', 'game_dir': r'D:\Games\NX2navi',
'game': Path(r'D:\Games\NX2navi\ocarina.exe')}
STATES = [
('new', dict(SAMPLE, triforce=None), '', False),
('triforce-installed', dict(SAMPLE, triforce=Path(r'C:\Triforce\TriforceSetup.exe')), '', False),
('nothing-built', {'triforce': None, 'game': None, 'dump': '', 'game_dir': ''}, '', False),
('copied', dict(SAMPLE, triforce=None), 'Copied. Paste it where Triforce Setup asks for it.', False),
('problem', dict(SAMPLE, triforce=None),
'Triforce Setup could not be started from here. Get it from triforceproject.com, which opens now.', True),
]
def run(app, out):
out = Path(out)
out.mkdir(parents=True, exist_ok=True)
real = dict(app.found)
shots = {}
for _ in range(5):
app.win.update()
time.sleep(0.1)
try: # the first picture Windows gives of a window is an empty frame: take it and throw it away
snap(app.win, out / 'first-frame.png')
(out / 'first-frame.png').unlink()
except Exception:
pass
for name, found, status, bad in STATES + [('this-pc', real, '', False)]:
app.found, app.status, app.status_color = dict(found), status, handover.BAD if bad else handover.GOLD_300
app.draw()
app.win.update()
time.sleep(0.15)
app.win.update()
try:
shots[name] = bool(snap(app.win, out / f'handover-{name}.png'))
except Exception as e:
shots[name] = f'{type(e).__name__}: {e}'
state = {'version': handover.my_version(), 'arguments': app.argv, 'launcher_root': str(app.root),
'scale': app.s, 'tk': str(app.win.tk.call('info', 'patchlevel')),
'found': {k: (str(v) if v else '') for k, v in real.items()}, 'shots': shots}
try:
_, info = handover.bundled_setup()
state['setup'] = {'ok': True, **info}
except Exception as e:
state['setup'] = {'ok': False, 'error': f'{type(e).__name__}: {e}'}
(out / 'state.json').write_text(json.dumps(state, indent=1), encoding='utf-8')
wait = max(0.0, 8.0 - (time.time() - BORN)) if os.environ.get('NX2NAVI_HANDOVER_TEST') else 0.0
app.win.after(int(wait * 1000) + 50, app.win.destroy)

View File

@@ -0,0 +1,93 @@
"""What NX2navi.exe runs: python\\pythonw.exe app\\<version>\\launcher\\start.pyw [arguments].
From this version on that is the handover to Triforce (handover.py beside this file), not NX2navi's launcher.
The contract with the stub (NX2navi's launcher/native/nx2navi.c) is kept as the launcher kept it: a failure before
the window is up is written to data/logs/crash.log and ends the process with exit code 86 within the six seconds the
stub waits, so the stub goes back to the version before (app/previous.txt), which is still on the PC. Only when there
is no version to go back to, or the stub no longer waits, the failure is shown in a message box instead, with the
address Triforce is at.
"""
import os
import sys
import traceback
HERE = os.path.dirname(os.path.abspath(__file__))
APP = os.path.dirname(HERE)
STARTUP_FAILED = 86
STUB_WAITS = 5.0 # the stub waits 6 s for the exit code (QUICK_MS); a second is left for the exit itself
def launcher_root():
return os.environ.get('NX2NAVI_LAUNCHER_ROOT') or os.path.dirname(os.path.dirname(APP))
def rollback_target(root):
"""The version the stub goes back to on exit code 86: the one app/previous.txt names, when it is another than
this one and holds a launcher. '' when there is none."""
try:
with open(os.path.join(root, 'app', 'previous.txt'), encoding='utf-8-sig') as f:
prev = (f.read().splitlines() or [''])[0].strip()
except (OSError, UnicodeDecodeError):
return ''
if not prev or prev == os.path.basename(APP):
return ''
return prev if os.path.isfile(os.path.join(root, 'app', prev, 'launcher', 'start.pyw')) else ''
def process_age():
"""Seconds since Windows made this process, which is when the stub began to wait. Infinite where that cannot be
read."""
try:
import ctypes
from ctypes import wintypes
k = ctypes.windll.kernel32
k.GetCurrentProcess.restype = wintypes.HANDLE
k.GetProcessTimes.argtypes = [wintypes.HANDLE] + [ctypes.POINTER(wintypes.FILETIME)] * 4
made, ended, kernel, user, now = (wintypes.FILETIME() for _ in range(5))
if not k.GetProcessTimes(k.GetCurrentProcess(), *(ctypes.byref(t) for t in (made, ended, kernel, user))):
return float('inf')
k.GetSystemTimeAsFileTime(ctypes.byref(now))
ticks = [t.dwHighDateTime << 32 | t.dwLowDateTime for t in (made, now)]
return max(0.0, (ticks[1] - ticks[0]) / 1e7)
except Exception:
return float('inf')
def crash(text, box):
try:
logs = os.path.join(launcher_root(), 'data', 'logs')
os.makedirs(logs, exist_ok=True)
with open(os.path.join(logs, 'crash.log'), 'a', encoding='utf-8') as f:
f.write(text + '\n')
except OSError:
pass
if not box:
return
try:
import ctypes
ctypes.windll.user32.MessageBoxW(
None, 'NX2navi is now Triforce, but the window that takes you there could not open.\n\n'
'Get Triforce from https://triforceproject.com\n\n' + text[-900:], 'NX2navi', 0x30)
except Exception:
print(text, file=sys.stderr)
def run():
sys.path.insert(0, HERE) # the embeddable Python puts only its ._pth entries on sys.path
argv = sys.argv[1:]
try:
import handover
code = handover.main(argv)
except SystemExit:
raise
except Exception:
root = launcher_root()
quiet = bool(rollback_target(root)) and '--play' not in argv and process_age() < STUB_WAITS
crash(traceback.format_exc(), box=not quiet)
sys.exit(STARTUP_FAILED if quiet else 0)
sys.exit(code or 0)
if __name__ == '__main__':
run()

View File

@@ -0,0 +1,7 @@
{
"name": "TriforceSetup.exe",
"version": "v0.3.6",
"size": 635904,
"sha256": "59fba57af1e9b15e66bfe3a2e1fb0127d1e7deb0c4b8688206c2935a66e7eb2e",
"from": "https://triforceproject.com"
}

105
handover/make_handover.py Normal file
View File

@@ -0,0 +1,105 @@
"""Packs the handover as a release NX2navi's launcher installs by itself.
python make_handover.py --version 0.12.0 [--setup TriforceSetup.exe --setup-version v0.3.6] [--notes notes.md]
Needs only Python 3.8 or later (no packages). It writes into release/:
NX2navi-app-<version>.zip the app archive: VERSION and launcher/ (the handover window, bridge/ of this folder)
NX2navi-update.json the manifest NX2navi's launcher reads (format 1: version, notes, and the archive's
name, size and sha256)
Upload both files to a GitHub release v<version> of the repository the launchers ask (tangerineturtle1/NX2navi),
as a full release, not a pre-release: launchers on the Stable channel pass pre-releases over, and a version with a
pre-release tag (0.12.0-rc1) counts as one whatever the release says. <version> must be newer than every version
published so far, the previews included.
The manifest names no "runtime": the handover runs on whatever Python the installed launcher has, so no launcher
downloads a Python for it.
"""
import argparse
import hashlib
import json
import re
import shutil
import sys
import time
import zipfile
from pathlib import Path
ROOT = Path(__file__).resolve().parent
BRIDGE = ROOT / 'bridge'
VERSION = re.compile(r'^\d+(?:\.\d+){1,3}$')
NOTES = """NX2navi and Triforce have merged and go on as Triforce.
- This update turns NX2navi's launcher into the door to Triforce: one window that starts Triforce Setup, Triforce's launcher.
- Nothing of NX2navi is removed or changed: your built game, your saves and your settings stay where they are.
- Play NX2navi in that window still starts the game you have, for as long as you want it.
- Triforce builds the game from your own files, as NX2navi did, and its Convert saves page brings your NX2navi saves over.
"""
def sha256_file(path):
h = hashlib.sha256()
with open(path, 'rb') as f:
for block in iter(lambda: f.read(1 << 20), b''):
h.update(block)
return h.hexdigest()
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument('--version', required=True, help='the release version, such as 0.12.0: plain numbers, newer than '
'every NX2navi version published so far')
ap.add_argument('--setup', type=Path, help='another TriforceSetup.exe to carry than the one in bridge/')
ap.add_argument('--setup-version', help='the Triforce release that TriforceSetup.exe is from (with --setup)')
ap.add_argument('--notes', type=Path, help='release notes (markdown) for the manifest instead of the built-in ones')
ap.add_argument('--out', type=Path, default=ROOT / 'release')
a = ap.parse_args()
if not VERSION.match(a.version):
sys.exit(f'{a.version}: give plain numbers such as 0.12.0. A version with a tag (0.12.0-rc1) is a '
'pre-release to the launcher, which only the Preview channel installs.')
ver = a.version
setup_dir = BRIDGE / 'launcher' / 'triforce'
info_file = setup_dir / 'setup.json'
info = json.loads(info_file.read_text(encoding='utf-8')) if info_file.is_file() else {}
if a.setup:
if not a.setup_version:
sys.exit('--setup needs --setup-version (the Triforce release the file is from)')
shutil.copyfile(a.setup, setup_dir / 'TriforceSetup.exe')
info['version'] = a.setup_version
exe = setup_dir / 'TriforceSetup.exe'
if not exe.is_file():
sys.exit(f'{exe} is missing: give --setup')
info = {'name': exe.name, 'version': info.get('version', ''), 'size': exe.stat().st_size,
'sha256': sha256_file(exe), 'from': 'https://triforceproject.com'}
info_file.write_text(json.dumps(info, indent=1) + '\n', encoding='utf-8')
for need in ('start.pyw', 'handover.py', 'cli.py', 'selftest.py', 'art/hall_100.png'):
if not (BRIDGE / 'launcher' / need).is_file():
sys.exit(f'bridge/launcher/{need} is missing')
a.out.mkdir(parents=True, exist_ok=True)
for old in list(a.out.glob('NX2navi-app-*.zip')) + list(a.out.glob('NX2navi-update.json')):
old.unlink()
archive = a.out / f'NX2navi-app-{ver}.zip'
stamp = time.localtime()[:6]
with zipfile.ZipFile(archive, 'w', zipfile.ZIP_DEFLATED, compresslevel=9) as z:
z.writestr(zipfile.ZipInfo('VERSION', stamp), ver + '\n')
for p in sorted(BRIDGE.rglob('*')):
if p.is_file() and '__pycache__' not in p.parts and p.name != 'VERSION':
z.write(p, p.relative_to(BRIDGE).as_posix())
notes = a.notes.read_text(encoding='utf-8') if a.notes else NOTES
manifest = {'format': 1, 'version': ver, 'build': ver, 'notes': notes,
'app': {'name': archive.name, 'size': archive.stat().st_size, 'sha256': sha256_file(archive)},
'made': time.strftime('%Y-%m-%d %H:%M')}
(a.out / 'NX2navi-update.json').write_text(json.dumps(manifest, indent=1), encoding='utf-8')
with zipfile.ZipFile(archive) as z:
names = z.namelist()
print(f'{archive.name}: {archive.stat().st_size / 1e6:.2f} MB, {len(names)} files, sha256 {manifest["app"]["sha256"]}')
print(f'carries {info["name"]} {info["version"]} ({info["size"]} bytes, sha256 {info["sha256"]})')
print(f'NX2navi-update.json: version {ver}')
print(f'upload both files from {a.out} to GitHub release v{ver} (a full release, not a pre-release)')
return 0
if __name__ == '__main__':
sys.exit(main())

Binary file not shown.

After

Width:  |  Height:  |  Size: 587 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 587 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 587 KiB

342
handover/tests/e2e.py Normal file
View File

@@ -0,0 +1,342 @@
"""The handover against NX2navi's real launcher, on a copy, without a window on the screen.
python tests/e2e.py [--nx2navi DIR] [--work DIR]
DIR is an installed NX2navi launcher (default %LOCALAPPDATA%\\Programs\\NX2navi). It is only read: its stub, its
Python and the launcher code of each version in app/ are copied into a work folder, where the launcher's own code
then finds this handover in a feed on 127.0.0.1 and installs it, and its own stub starts it.
A each launcher version's command line (`cli.py check`, `cli.py update`): sees the handover, downloads, checks,
stages and switches to it, and the handover's cli.py ends the update with exit code 0
B the stub (NX2navi.exe) starts the handover: it is still up when the stub stops waiting, nothing is rolled back
C a handover that cannot start: exit code 86 inside the stub's wait, and the stub goes back to the version before
D the path the launcher's window takes in the background (feed.latest, updater.stage, staged_info, install_staged)
E a GitHub-shaped feed, Stable channel: found as a full release, passed over as a pre-release
F the way back: the handover stages a newer release of the feed, as the launcher did
G the handover's own parts: what it finds on the PC, the Setup it carries and starts, the uninstall plan against
the launcher's own
"""
import argparse
import functools
import http.server
import importlib.util
import json
import os
import shutil
import subprocess
import sys
import tempfile
import threading
import time
from pathlib import Path
HERE = Path(__file__).resolve().parent
PROJECT = HERE.parent
FAILED = []
def check(name, ok, detail=''):
print((' ok ' if ok else ' FAIL ') + name + (f': {detail}' if detail and not ok else ''), flush=True)
if not ok:
FAILED.append(name)
class Handler(http.server.SimpleHTTPRequestHandler):
releases = []
def log_message(self, *a):
pass
def do_GET(self):
if self.path.startswith('/api/repos/'):
body = json.dumps(self.releases).encode()
self.send_response(200)
self.send_header('Content-Type', 'application/json')
self.send_header('Content-Length', str(len(body)))
self.end_headers()
self.wfile.write(body)
return
super().do_GET()
def serve(folder):
srv = http.server.ThreadingHTTPServer(('127.0.0.1', 0), functools.partial(Handler, directory=str(folder)))
threading.Thread(target=srv.serve_forever, daemon=True).start()
return srv, f'http://127.0.0.1:{srv.server_address[1]}'
def make_release(version, out):
r = subprocess.run([sys.executable, '-B', str(PROJECT / 'make_handover.py'), '--version', version, '--out', str(out)],
capture_output=True, text=True)
if r.returncode:
sys.exit(r.stdout + r.stderr)
def clean_env(**more):
env = {k: v for k, v in os.environ.items() if not k.startswith('NX2NAVI_')}
env.update(more)
return env
def make_root(work, nx, name, version, feed):
"""A launcher folder like the installed one, holding only that version of the launcher."""
root = work / name
shutil.rmtree(root, ignore_errors=True)
(root / 'app' / version).mkdir(parents=True)
shutil.copy2(nx / 'NX2navi.exe', root / 'NX2navi.exe')
if not (work / 'python').is_dir():
shutil.copytree(nx / 'python', work / 'python')
# (a junction to the one copy: the stub wants python\ beside it, and 100 MB need not be copied per test)
subprocess.run(['cmd', '/c', 'mklink', '/J', str(root / 'python'), str(work / 'python')], capture_output=True)
src = nx / 'app' / version
for part in ('launcher', 'tools'):
shutil.copytree(src / part, root / 'app' / version / part, ignore=shutil.ignore_patterns('__pycache__'))
for f in ('VERSION', 'toolchain.lock.json'):
if (src / f).is_file():
shutil.copy2(src / f, root / 'app' / version / f)
(root / 'app' / version / 'feed.json').write_text(json.dumps(feed), encoding='utf-8')
(root / 'app' / 'current.txt').write_text(version + '\n', encoding='utf-8')
(root / 'data').mkdir()
(root / 'data' / 'launcher.json').write_text(json.dumps({'game_dir': '', 'dump': '', 'channel': 'stable'}),
encoding='utf-8')
return root
def text(path):
try:
return Path(path).read_text(encoding='utf-8-sig').strip()
except OSError:
return ''
def run_py(root, version, args, **kw):
py = root / 'python' / 'python.exe'
return subprocess.run([str(py), str(root / 'app' / version / 'launcher' / 'cli.py'), *args], capture_output=True,
text=True, cwd=str(root), env=clean_env(), timeout=300, **kw)
def load_handover(app_dir):
sys.path.insert(0, str(app_dir / 'launcher'))
spec = importlib.util.spec_from_file_location('handover', app_dir / 'launcher' / 'handover.py')
mod = importlib.util.module_from_spec(spec)
sys.modules['handover'] = mod
spec.loader.exec_module(mod)
return mod
def main():
ap = argparse.ArgumentParser()
ap.add_argument('--nx2navi', type=Path, default=Path(os.environ.get('LOCALAPPDATA', '')) / 'Programs' / 'NX2navi')
ap.add_argument('--work', type=Path, default=Path(tempfile.gettempdir()) / 'nx2navi-handover-e2e')
a = ap.parse_args()
nx, work = a.nx2navi, a.work
if not (nx / 'NX2navi.exe').is_file():
sys.exit(f'{nx} is not an installed NX2navi launcher')
work.mkdir(parents=True, exist_ok=True)
versions = sorted(d.name for d in (nx / 'app').iterdir() if (d / 'launcher' / 'updater.py').is_file())
print(f'NX2navi launcher versions on this PC: {", ".join(versions)}')
NEW, NEWER = '0.12.0', '0.13.0'
feed_dir, feed2_dir = work / 'feed', work / 'feed2'
for d in (feed_dir, feed2_dir):
shutil.rmtree(d, ignore_errors=True)
make_release(NEW, feed_dir)
make_release(NEWER, feed2_dir)
srv, base = serve(feed_dir)
srv2, base2 = serve(feed2_dir)
url_feed = {'type': 'url', 'url': f'{base}/NX2navi-update.json'}
for v in versions:
print(f'\nA the command line of launcher {v}')
root = make_root(work, nx, f'root-a-{v}', v, url_feed)
r = run_py(root, v, ['check'])
check('check sees the handover', f'update available: {NEW}' in r.stdout, r.stdout + r.stderr)
r = run_py(root, v, ['update'])
check('update ends with exit code 0', r.returncode == 0, r.stdout[-600:] + r.stderr[-600:])
check('the handover\'s cli.py finished it', 'NX2navi is now Triforce' in r.stdout, r.stdout[-400:])
check('current.txt names the handover', text(root / 'app' / 'current.txt') == NEW)
check('previous.txt names the old version', text(root / 'app' / 'previous.txt') == v)
check('the old version is still on the disk', (root / 'app' / v / 'launcher' / 'start.pyw').is_file())
check('the archive unpacked whole', all((root / 'app' / NEW / p).is_file() for p in (
'VERSION', 'launcher/start.pyw', 'launcher/handover.py', 'launcher/triforce/TriforceSetup.exe',
'launcher/art/hall_100.png')))
print(f'B the stub starts the handover (after launcher {v})')
shots = work / f'shots-{v}'
shutil.rmtree(shots, ignore_errors=True)
t0 = time.time()
r = subprocess.run([str(root / 'NX2navi.exe'), '--continue-update', v], cwd=str(root),
env=clean_env(NX2NAVI_HANDOVER_TEST=str(shots)), timeout=60)
waited = time.time() - t0
check('the stub ends with 0 after its six seconds', r.returncode == 0 and 5.5 < waited < 9, f'{r.returncode} after {waited:.1f} s')
for _ in range(40):
if (shots / 'state.json').is_file():
break
time.sleep(0.25)
state = json.loads(text(shots / 'state.json') or '{}')
check('the window came up with the stub\'s arguments', state.get('arguments') == ['--continue-update', v], str(state.get('arguments')))
check('it knows its launcher folder', os.path.normcase(state.get('launcher_root', '')) == os.path.normcase(str(root)))
check('its pictures were taken', bool(state.get('shots')) and all(v2 is True for v2 in state['shots'].values()), str(state.get('shots')))
check('the Setup it carries is intact', state.get('setup', {}).get('ok') is True, str(state.get('setup')))
time.sleep(3)
check('nothing was rolled back', text(root / 'app' / 'current.txt') == NEW and text(root / 'app' / 'previous.txt') == v)
check('no crash was logged', not (root / 'data' / 'logs' / 'crash.log').exists(), text(root / 'data' / 'logs' / 'crash.log')[-500:])
print(f'C a handover that cannot start (after launcher {v})')
(root / 'app' / NEW / 'launcher' / 'handover.py').rename(root / 'app' / NEW / 'launcher' / 'handover.py.off')
dummy = root / 'app' / '0.0.9' / 'launcher'
dummy.mkdir(parents=True)
(dummy / 'start.pyw').write_text(
'import os, sys\nopen(os.path.join(os.path.dirname(__file__), "started.txt"), "w").write(" ".join(sys.argv[1:]))\n',
encoding='utf-8')
(root / 'app' / 'previous.txt').write_text('0.0.9\n', encoding='utf-8')
t0 = time.time()
r = subprocess.run([str(root / 'NX2navi.exe')], cwd=str(root), env=clean_env(), timeout=60)
waited = time.time() - t0
for _ in range(40):
if (dummy / 'started.txt').is_file():
break
time.sleep(0.25)
check('the stub went back to the version before', text(root / 'app' / 'current.txt') == '0.0.9', text(root / 'app' / 'current.txt'))
check('and started it with --rolled-back', text(dummy / 'started.txt') == f'--rolled-back {NEW}', text(dummy / 'started.txt'))
check('no message box held the exit code back', waited < 14, f'{waited:.1f} s')
check('the failure is in crash.log', 'handover' in text(root / 'data' / 'logs' / 'crash.log'))
print(f'D the background path of launcher {v} (stage, then install when the game is not running)')
root = make_root(work, nx, f'root-d-{v}', v, url_feed)
script = work / 'stage_test.py'
script.write_text(
'import json, sys\n'
'sys.path.insert(0, sys.argv[1])\n'
'from launcher import feed, updater\n'
'from launcher.paths import Layout\n'
'from launcher.settings import Settings\n'
'lay = Layout().ensure()\n'
's = Settings(lay.settings)\n'
'rel = feed.latest(feed.load(lay, s), s.get("channel", "stable"))\n'
'updater.stage(lay, rel)\n'
'ver = rel.version\n'
'out = {"version": ver, "prerelease": updater.prerelease(rel), "staged": bool(updater.staged_info(lay, ver)),\n'
' "found": (updater.staged(lay, "stable") or {}).get("version"), "runtime_needed": updater.runtime_needed(lay, rel)}\n'
'updater.install_staged(lay, ver)\n'
'out["current"], out["previous"] = updater.current(lay), updater.previous(lay)\n'
'print(json.dumps(out))\n', encoding='utf-8')
r = subprocess.run([str(root / 'python' / 'python.exe'), str(script), str(root / 'app' / v)], capture_output=True,
text=True, cwd=str(root), env=clean_env(NX2NAVI_LAUNCHER_ROOT=str(root)), timeout=300)
try:
out = json.loads(r.stdout.strip().splitlines()[-1])
except (ValueError, IndexError):
out = {}
check('staged complete and found again', out.get('staged') is True and out.get('found') == NEW, r.stdout[-300:] + r.stderr[-900:])
check('a full release to the Stable channel, no Python to fetch', out.get('prerelease') is False and out.get('runtime_needed') is False, str(out))
check('installed: current and previous switched', out.get('current') == NEW and out.get('previous') == v, str(out))
v = versions[-1]
print(f'\nE a GitHub-shaped feed (launcher {v})')
man = json.loads(text(feed_dir / 'NX2navi-update.json'))
assets = [{'name': n, 'url': f'{base}/{n}', 'browser_download_url': f'{base}/{n}'}
for n in ('NX2navi-update.json', man['app']['name'])]
older = {'tag_name': 'v0.11.0-rc4', 'draft': False, 'prerelease': True, 'body': '', 'html_url': '', 'assets': []}
gh_feed = {'type': 'github', 'repo': 'someone/NX2navi', 'api': f'{base}/api'}
root = make_root(work, nx, 'root-e', v, gh_feed)
Handler.releases = [{'tag_name': f'v{NEW}', 'draft': False, 'prerelease': False, 'body': 'notes', 'html_url': '', 'assets': assets}, older]
r = run_py(root, v, ['check'])
check('a full release is offered on Stable', f'update available: {NEW}' in r.stdout, r.stdout + r.stderr)
Handler.releases = [{'tag_name': f'v{NEW}', 'draft': False, 'prerelease': True, 'body': 'notes', 'html_url': '', 'assets': assets}, older]
r = run_py(root, v, ['check'])
check('a pre-release is passed over on Stable', 'update available' not in r.stdout, r.stdout + r.stderr)
print('\nF the way back: the handover stages a newer release of the launcher\'s feed')
root = make_root(work, nx, 'root-f', v, {'type': 'url', 'url': f'{base2}/NX2navi-update.json'})
r = run_py(root, v, ['update'])
# (that installed 0.13.0 from the second feed; set the folder up as a PC on 0.12.0 whose old version asks that feed)
shutil.rmtree(root / 'app' / NEWER, ignore_errors=True)
shutil.unpack_archive(str(feed_dir / f'NX2navi-app-{NEW}.zip'), str(root / 'app' / NEW), 'zip')
(root / 'app' / 'current.txt').write_text(NEW + '\n', encoding='utf-8')
(root / 'app' / 'previous.txt').write_text(v + '\n', encoding='utf-8')
os.environ['NX2NAVI_LAUNCHER_ROOT'] = str(root)
h = load_handover(root / 'app' / NEW)
check('it reads the old version\'s feed', h.load_feed(root, {}).get('url', '').startswith(base2), str(h.load_feed(root, {})))
found = h.latest(h.load_feed(root, {}), 'stable')
check('it finds the newer release', bool(found) and found[0]['version'] == NEWER)
try:
staged = h.check_for_newer(root, force=True)
except Exception as e:
staged = f'{type(e).__name__}: {e}'
check('it staged it', staged == NEWER, str(staged))
check('current.txt names the newer release, previous.txt the handover',
text(root / 'app' / 'current.txt') == NEWER and text(root / 'app' / 'previous.txt') == NEW)
check('the newer release is whole', (root / 'app' / NEWER / 'launcher' / 'start.pyw').is_file() and text(root / 'app' / NEWER / 'VERSION') == NEWER)
check('nothing newer than itself: nothing staged', h.check_for_newer(root, {'feed_repo': ''}, force=True) in ('', NEWER) and
not h.newer(NEW, NEWER) and h.newer('0.12.0', '0.11.0-rc4') and h.newer('0.11.0', '0.11.0-rc4') and not h.newer('0.12.0-rc1', '0.12.0'))
bad = dict(found[0], app=dict(found[0]['app'], sha256='0' * 64))
(root / 'app' / 'current.txt').write_text(NEW + '\n', encoding='utf-8')
shutil.rmtree(root / 'app' / NEWER, ignore_errors=True)
try:
h.stage(root, bad, found[1])
refused = False
except h.UpdateError:
refused = True
check('a file that does not match its sha256 is refused and nothing is switched',
refused and text(root / 'app' / 'current.txt') == NEW and not (root / 'app' / NEWER).exists())
print('\nG the handover\'s own parts')
game = work / 'fake game'
shutil.rmtree(game, ignore_errors=True)
game.mkdir()
(game / 'ocarina.exe').write_bytes(b'MZ')
dump = work / 'fake dump.nup'
dump.write_bytes(b'x')
(root / 'data' / 'launcher.json').write_text(json.dumps({'game_dir': str(game), 'dump': str(dump).replace('\\', '/')}), encoding='utf-8')
f = h.facts(root)
check('it finds the NX2navi game and the dump from the launcher\'s settings',
f['game'] == game / 'ocarina.exe' and f['game_dir'] == str(game) and f['dump'] == str(dump), str(f))
started = []
h.start_detached = lambda cmd, cwd: started.append((cmd, str(cwd)))
temp = work / 'temp'
shutil.rmtree(temp, ignore_errors=True)
temp.mkdir()
h.tempfile.gettempdir = lambda: str(temp)
exe = h.start_triforce(None)
info = json.loads(text(root / 'app' / NEW / 'launcher' / 'triforce' / 'setup.json'))
check('Get Triforce starts the carried Setup from its own temp folder, checked',
started and started[-1][0] == [exe] and Path(exe).parent == temp / 'TriforceSetup' and h.sha256_file(exe) == info['sha256'], str(started))
h.start_triforce(Path(r'C:\somewhere\TriforceSetup.exe'))
check('Open Triforce starts the installed one', started[-1][0] == [r'C:\somewhere\TriforceSetup.exe'])
carried = root / 'app' / NEW / 'launcher' / 'triforce' / 'TriforceSetup.exe'
carried.write_bytes(carried.read_bytes() + b'x')
try:
h.start_triforce(None)
refused = False
except (ValueError, OSError):
refused = True
check('a changed Setup is not started', refused and len(started) == 2)
# the uninstall plan against the launcher's own, on the same folder
script = work / 'plan_test.py'
script.write_text(
'import json, sys\n'
'sys.path.insert(0, sys.argv[1])\n'
'from launcher import winsys\n'
'from launcher.paths import Layout\n'
'lay = Layout()\n'
'out = []\n'
'for game in ("", sys.argv[2]):\n'
' gone, empty = winsys.uninstall_plan(lay, game)\n'
' out.append(winsys.uninstall_script(gone, empty))\n'
'print(json.dumps(out))\n', encoding='utf-8')
r = subprocess.run([str(root / 'python' / 'python.exe'), str(script), str(root / 'app' / v), str(game)], capture_output=True,
text=True, cwd=str(root), env=clean_env(NX2NAVI_LAUNCHER_ROOT=str(root)), timeout=120)
try:
theirs = json.loads(r.stdout.strip().splitlines()[-1])
except (ValueError, IndexError):
theirs = [r.stderr[-600:]]
mine = [h.uninstall_script(*h.uninstall_plan(root, g)) for g in ('', str(game))]
check('its uninstall removes exactly what the launcher\'s own would', mine == theirs, f'\n mine {mine}\n theirs {theirs}')
srv.shutdown()
srv2.shutdown()
print(f'\n{"ALL PASSED" if not FAILED else str(len(FAILED)) + " FAILED: " + "; ".join(FAILED)}')
return 1 if FAILED else 0
if __name__ == '__main__':
sys.exit(main())

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 277 KiB

240
handover/tools/make_art.py Normal file
View File

@@ -0,0 +1,240 @@
"""Draws the pictures of the handover window into bridge/launcher/art (run once, by whoever changes the look).
python tools/make_art.py needs Pillow and numpy; reads tools/art-src/triforce.png and nx2navi_logo.png
The window (bridge/launcher/handover.py) is one fixed composition of 820 x 480 units. Everything that needs soft light
(the hall, the halo behind the crest, the crest, the wordmark, the fairy) is drawn here, once per display scale, so the
player's PC only puts text and buttons on top of a PNG: no picture library is needed there.
hall_<scale>.png the whole background, <scale> = 100, 125, 150, 200 (percent)
btn_<scale>.png the gold bar of the primary button, and btn_<scale>_hi.png for the mouse over it
icon_64/32/16.png the crest for the title bar and the taskbar
The look is the Triforce start client's ("the relic in a dark hall"): a warm near-black page, one light from above, a
gold halo and a dial's arc behind the painted crest, the wordmark under it. The fairy is NX2navi's logo, kept pixel
exact (whole cells, hard edges), arriving beside the crest.
"""
import math
import sys
from pathlib import Path
import numpy as np
from PIL import Image, ImageDraw, ImageFilter, ImageFont
ROOT = Path(__file__).resolve().parents[1]
SRC = ROOT / 'tools' / 'art-src'
OUT = ROOT / 'bridge' / 'launcher' / 'art'
W, H = 820, 480 # the window, in units (pixels at 100 %)
CX, CREST_W, CREST_TOP = 632, 228, 70 # the crest's centre line, width and top: handover.py leaves this column free
SCALES = (1.0, 1.25, 1.5, 2.0)
SS = 2 # everything soft is drawn at twice the size and scaled down
INK = (0x0A, 0x09, 0x08)
GOLD = (0xE0, 0xB2, 0x3E)
GOLD_100, GOLD_300, GOLD_700 = (0xFF, 0xF4, 0xCC), (0xF7, 0xDC, 0x8C), (0x9A, 0x6A, 0x18)
def grid(h, w):
yy, xx = np.mgrid[0:h, 0:w].astype(np.float32)
return xx + 0.5, yy + 0.5
def ramp(t, stops):
return np.interp(t, [p for p, _ in stops], [a for _, a in stops]).astype(np.float32)
def over(base, color, alpha):
c = np.array(color, np.float32) / 255
return base * (1 - alpha[..., None]) + c * alpha[..., None]
def mask_of(size, draw):
"""An 8-bit mask drawn with PIL, as a float array 0..1."""
m = Image.new('L', size, 0)
draw(ImageDraw.Draw(m))
return np.asarray(m, np.float32) / 255
def crest_art(width):
im = Image.open(SRC / 'triforce.png').convert('RGBA')
return im.resize((width, round(width * im.height / im.width)), Image.LANCZOS)
def navi_cells():
"""NX2navi's logo as its 22 x 21 cells (the source picture is each cell 54 x 54)."""
im = Image.open(SRC / 'nx2navi_logo.png').convert('RGBA')
cw, ch = im.width // 22, im.height // 21
cells = Image.new('RGBA', (22, 21))
for y in range(21):
for x in range(22):
cells.putpixel((x, y), im.getpixel((x * cw + cw // 2, y * ch + ch // 2)))
return cells
def wordmark(k):
"""TRIFORCE in engraved gold with its shadow, as an RGBA picture at k pixels per unit."""
font = ImageFont.truetype('palab.ttf', round(21 * k))
gap = 0.40 * 21 * k
letters = 'TRIFORCE'
widths = [font.getlength(c) for c in letters]
total = sum(widths) + gap * (len(letters) - 1)
pad = round(12 * k)
size = (round(total) + 2 * pad, round(34 * k) + 2 * pad)
text = Image.new('L', size, 0)
d = ImageDraw.Draw(text)
x = pad
for c, w in zip(letters, widths):
d.text((x, pad), c, font=font, fill=255)
x += w + gap
a = np.asarray(text, np.float32) / 255
box = text.getbbox()
t = np.clip((np.arange(size[1], dtype=np.float32) - box[1]) / max(1, box[3] - box[1]), 0, 1)
stops = [(0, (0xFF, 0xF1, 0xB8)), (0.42, (0xE8, 0xB8, 0x40)), (0.56, (0xB5, 0x7E, 0x1C)), (1, (0xF3, 0xCF, 0x6A))]
rgb = np.stack([np.interp(t, [p for p, _ in stops], [c[i] for _, c in stops]) for i in range(3)], -1)
rgb = np.repeat(rgb[:, None, :], size[0], 1).astype(np.float32)
shadow = np.asarray(Image.fromarray((a * 255).astype(np.uint8)).filter(ImageFilter.GaussianBlur(3 * k)),
np.float32) / 255
shadow = np.roll(shadow, round(2 * k), 0) * 0.8
alpha = a + shadow * (1 - a)
color = np.where(alpha[..., None] > 0, rgb * (a / np.maximum(alpha, 1e-6))[..., None], 0)
out = np.dstack([color, alpha * 255]).clip(0, 255).astype(np.uint8)
return Image.fromarray(out, 'RGBA'), total, pad
def hall(scale):
k = scale * SS
w, h = round(W * k), round(H * k)
xx, yy = grid(h, w)
img = np.empty((h, w, 3), np.float32)
img[:] = np.array(INK, np.float32) / 255
crest = crest_art(round(CREST_W * k))
cw, ch = crest.size
cl, ct = round(CX * k - cw / 2), round(CREST_TOP * k)
ccx, ccy = CX * k, ct + ch / 2
# 2. the light from above
img = over(img, (0xFF, 0xEC, 0xAA), ramp(yy / h, [(0, 0.045), (0.45, 0)]))
# 3. the halo behind the crest
halo_r = 1.2 * cw
t = np.sqrt((xx - ccx) ** 2 + (yy - ccy) ** 2) / halo_r
img = over(img, GOLD, ramp(t, [(0, 0.24), (0.45, 0.075), (1, 0)]))
# 4. the lattice of triangles, only inside the halo
side, row = 56 * k, 56 * k * math.sqrt(3) / 2
def lattice(d):
lw = max(1, round(k * 0.9))
y = ccy % row - row
while y < h:
d.line([(0, y), (w, y)], fill=255, width=lw)
y += row
slope = math.tan(math.radians(60))
x0 = ccx - (ccy / slope)
n = int(w / side) + 6
for i in range(-n, n):
xa = x0 + i * side
d.line([(xa, 0), (xa + h / slope, h)], fill=255, width=lw)
xb = x0 + 2 * (ccy / slope) + i * side
d.line([(xb, 0), (xb - h / slope, h)], fill=255, width=lw)
img = over(img, GOLD, mask_of((w, h), lattice) * 0.06 * ramp(t, [(0, 1), (0.5, 0.5), (0.95, 0)]))
# 6. the vignette (before the arc, which sits in front of it)
tv = np.sqrt(((xx - w / 2) / (0.78 * w)) ** 2 + ((yy - h / 2) / (0.78 * h)) ** 2)
img = over(img, (0, 0, 0), ramp(tv, [(0, 0), (0.55, 0), (1, 0.55)]))
# 5. the arc over the crest: a ring and a ring of marks, fading out on the way down
acx, acy = ccx, ccy + 0.06 * ch
r_in, r_out = 0.66 * cw, 0.705 * cw
def ring(d):
d.ellipse([acx - r_in, acy - r_in, acx + r_in, acy + r_in], outline=255, width=max(1, round(k * 0.9)))
def marks(d):
for i in range(120):
a = 2 * math.pi * i / 120
ca, sa = math.cos(a), math.sin(a)
d.line([(acx + (r_out - 1.5 * k) * ca, acy + (r_out - 1.5 * k) * sa),
(acx + (r_out + 1.5 * k) * ca, acy + (r_out + 1.5 * k) * sa)], fill=255, width=max(1, round(k * 0.9)))
box_top, box_h = acy - r_out, 2 * r_out
fade = ramp((yy - box_top) / box_h, [(0, 1), (0.42, 1), (0.74, 0), (1, 0)])
img = over(img, GOLD, mask_of((w, h), ring) * 0.17 * fade)
img = over(img, GOLD, mask_of((w, h), marks) * 0.28 * fade)
# the floor glow under the crest
fx, fy = ccx, ct + ch - 2 * k
tf = np.sqrt(((xx - fx) / (0.43 * cw)) ** 2 + ((yy - fy) / (15 * k)) ** 2)
img = over(img, GOLD, ramp(tf, [(0, 0.26), (0.55, 0.08), (1, 0)]))
# the fairy's light, where she will be put (soft, so it is drawn here; her pixels go on after the scaling down)
n = max(2, round(2 * scale)) # pixels per cell of the logo, in the finished picture
nx, ny = round((CX - CREST_W / 2 - 34) * scale), round((CREST_TOP + 26) * scale)
ncx, ncy = (nx + 11 * n) * SS, (ny + 10.5 * n) * SS
tn = np.sqrt((xx - ncx) ** 2 + (yy - ncy) ** 2) / (46 * k)
img = over(img, (0x8F, 0xC0, 0xFF), ramp(tn, [(0, 0.30), (0.4, 0.10), (1, 0)]))
pic = Image.fromarray((img.clip(0, 1) * 255).astype(np.uint8), 'RGB').convert('RGBA')
pic.alpha_composite(crest, (cl, ct))
wm, total, pad = wordmark(k)
wm_top = ct + ch + round(22 * k)
pic.alpha_composite(wm, (round(ccx - total / 2) - pad, wm_top - pad))
# the rule with its diamond, under the wordmark
rule_y, half = wm_top + round(46 * k), 90 * k
rule = np.asarray(pic, np.float32) / 255
rgb = rule[..., :3]
line = ((np.abs(yy - rule_y) < max(0.5, k * 0.5)) & (np.abs(xx - ccx) < half)).astype(np.float32)
rgb = over(rgb, GOLD, line * 0.55 * ramp(np.abs(xx - ccx) / half, [(0, 1), (0.7, 1), (1, 0)]))
diamond = ((np.abs(xx - ccx) + np.abs(yy - rule_y)) < 3.2 * k).astype(np.float32)
rgb = over(rgb, GOLD_300, diamond)
# down to the display's size, in float, then dithered: a dark gradient bands in 8 bits without it
small = np.asarray(Image.fromarray((rgb.clip(0, 1) * 255).astype(np.uint8), 'RGB')
.resize((round(W * scale), round(H * scale)), Image.LANCZOS), np.float32)
rng = np.random.default_rng(7)
small = small + (rng.random(small.shape, np.float32) - rng.random(small.shape, np.float32)) * 0.6
out = Image.fromarray(small.clip(0, 255).astype(np.uint8), 'RGB').convert('RGBA')
# the fairy: whole cells, hard edges, and a short trail of sparks she came along
cells = navi_cells().resize((22 * n, 21 * n), Image.NEAREST)
out.alpha_composite(cells, (nx, ny))
d = ImageDraw.Draw(out, 'RGBA')
# (the trail stays right of x = 420: the text column of handover.py ends at 400)
trail = [(-14, 28, (0x8F, 0xC0, 0xFF), 160), (-26, 40, (0xFF, 0xFF, 0xFF), 125), (-40, 46, (0x8F, 0xC0, 0xFF), 95),
(-52, 60, (0xC8, 0xE0, 0xFF), 65), (-64, 66, (0x8F, 0xC0, 0xFF), 38)]
for dx, dy, c, a in trail:
px, py = nx + round(dx * scale / n) * n, ny + round(dy * scale / n) * n
d.rectangle([px, py, px + n - 1, py + n - 1], fill=c + (a,))
return out.convert('RGB')
def button(scale, hot):
k = scale * SS
w, h = round(196 * k), round(44 * k)
t = (np.arange(h, dtype=np.float32) + 0.5) / h
stops = [(0, GOLD_100), (0.12, GOLD_300), (0.48, GOLD), (1, GOLD_700)]
rgb = np.stack([np.interp(t, [p for p, _ in stops], [c[i] for _, c in stops]) for i in range(3)], -1)
rgb = np.repeat(rgb[:, None, :], w, 1).astype(np.float32)
if hot:
rgb = rgb * 0.82 + 255 * 0.18
rgb[:max(1, round(k * 0.8))] = rgb[:max(1, round(k * 0.8))] * 0.5 + np.array(GOLD_100, np.float32) * 0.5
a = Image.new('L', (w, h), 0)
ImageDraw.Draw(a).rounded_rectangle([0, 0, w - 1, h - 1], radius=round(2 * k), fill=255)
pic = Image.fromarray(rgb.clip(0, 255).astype(np.uint8), 'RGB').convert('RGBA')
pic.putalpha(a)
return pic.resize((round(196 * scale), round(44 * scale)), Image.LANCZOS)
def main():
OUT.mkdir(parents=True, exist_ok=True)
for s in SCALES:
tag = f'{round(s * 100)}'
hall(s).save(OUT / f'hall_{tag}.png', optimize=True)
button(s, False).save(OUT / f'btn_{tag}.png', optimize=True)
button(s, True).save(OUT / f'btn_{tag}_hi.png', optimize=True)
print(f'scale {tag}: hall {(OUT / f"hall_{tag}.png").stat().st_size // 1024} KB')
crest = Image.open(SRC / 'triforce.png').convert('RGBA')
side = max(crest.size)
square = Image.new('RGBA', (side, side))
square.alpha_composite(crest, ((side - crest.width) // 2, (side - crest.height) // 2))
for n in (64, 32, 16):
square.resize((n, n), Image.LANCZOS).save(OUT / f'icon_{n}.png', optimize=True)
return 0
if __name__ == '__main__':
sys.exit(main())

View File

@@ -432,6 +432,13 @@
# game does this all the time: 52,000 second uploads in three minutes without it, 600 with it. 0: only the small ones,
# as before (for a comparison; nothing should look different). Read at start.
# tex_same_big = 1
# The texture cache counts its short waits in frames: a texture the game let go of may be dropped under video memory
# pressure 2 seconds later, one nothing sampled for 2 seconds counts as idle, and so on. Above 60 fps those frame
# counts always grew with the frame rate, so that the seconds stayed. Below 60 fps they did not shrink: the 2 seconds
# were 4 at 30 fps and nearly 6 at 21, and the textures waiting them out held about three times the video memory on a
# card that was already over its budget, which kept the frame rate down. 1 (the default) counts the seconds below 60
# fps too, down to 15 fps, once the game has been under 60 fps for 8 seconds; 0: as before. Read at start.
# tex_rate_low = 1
# Texture compaction (experimental). Video memory is taken in blocks (vma_block_mb), and a block goes back to the
# driver only once everything in it is freed: after a long session the blocks hold textures scattered between holes,
# and the game holds a gigabyte or two of video memory it does not use. 1: textures no shader sampled for ten seconds

View File

@@ -50,7 +50,7 @@ static const char *const known[] = {
"BIND_SKIP", "BIND_STATS", "BIND_UP", "BIND_X", "BIND_Y",
"BIND_ZL", "BIND_ZR", "BLOCK_VIEWS_3D", "CACHE_DIE", "CLOTH_REALTIME", "COLOR_BRIGHTNESS", "COMPUTE_KEEP", "COLOR_CONTRAST", "COLOR_GAMMA",
"COLOR_PRESET", "COLOR_SATURATION", "CONTACT_DEBUG", "CONTACT_SHADOWS", "CORE_DIAG", "CRIT", "CRIT_OUT",
"DMABUF", "DMABUF_MB", "DRAW_CRUMBS", "GPA_LATE", "DESC_TEMPLATE", "DEVICE_PROFILE", "DISCORD_APP_ID", "DISCORD_IMAGE", "DISCORD_RPC", "DLAA", "DLAA_CAM_KEEP", "DLAA_CUT", "DLSS_IDLE_FREE", "DLAA_EDGE_FILL", "DLAA_AT_UPSCALER", "FFX_DEBUG", "FFX_KEYS", "FSR3_SHARPEN", "DLAA_JITTER", "DLAA_MV", "DLAA_MASKDEBUG", "DLAA_MVDEBUG", "DLAA_OBJMV", "DLAA_SURVEY", "DLAA_TARGET_KEEP", "DLAA_TIMES", "DLSS", "DLSS_CHAR_MASK", "DLSS_PRESET", "DLSS_PREBLOOM", "DLSS_RANGE", "DLSS_REACTIVE", "DUMP", "DUMP_BUF", "DUMP_DIR",
"DMABUF", "DMABUF_MB", "DRAW_CRUMBS", "GPA_LATE", "DESC_TEMPLATE", "DEVICE_PROFILE", "DISCORD_APP_ID", "DISCORD_IMAGE", "DISCORD_RPC", "DLAA", "DLAA_CAM_KEEP", "DLAA_CUT", "DLSS_IDLE_FREE", "DLAA_EDGE_FILL", "DLAA_AT_UPSCALER", "FFX_DEBUG", "FFX_KEYS", "FSR3_SHARPEN", "DLAA_JITTER", "DLAA_JITTER_TEST", "DLSS_LOD_BIAS_TEST", "DLAA_MV", "DLAA_MASKDEBUG", "DLAA_MVDEBUG", "DLAA_OBJMV", "DLAA_SURVEY", "DLAA_TARGET_KEEP", "DLAA_TIMES", "DLSS", "DLSS_CHAR_MASK", "DLSS_PRESET", "DLSS_PREBLOOM", "DLSS_RANGE", "DLSS_REACTIVE", "DUMP", "DUMP_BUF", "DUMP_DIR",
"UPSCALER_IDLE_WAIT",
"DUMP_LOG", "DUMP_RAW", "DUMP_SHADERS", "DUMP_FILE", "DS3", "LOG_CS_BINDS", "LOG_TEX_FLAGS", "LOG_MESH", "MESH", "POOL", "NATIVE_TENSOR", "LOG_RCOND", "LOG_RT_SIZES", "DUMP_WHITE", "DUMP_AFTER", "DUMP_MOVIE", "SUBMIT_FINISH", "ACQUIRE_WAIT", "CRASH_FILTER_GUARD", "CS_FALLBACK", "EXEFS", "FEEDBACK", "FIX_C0", "FIX_ZERO", "FLIGHT", "FLIGHT_AT", "FLIGHT_DIR", "FLIGHT_FRAMES", "GPU_TIMELINE",
"FFX_PIPE_CACHE", "FLIGHT_SPIKE_MS", "FREEZE_QUIET_MS", "FREEZE_LOAD_HOLD_MS", "STUTTER_MAP", "PAR_JOBS", "PIPE_RELEASE", "FLIGHT_THUMBS", "FLIGHT_EVERY", "FLIGHT_FLASH", "FLIGHT_FLASH_DIFF", "FLIGHT_FLASH_CELLS", "FLUSH_SPAN", "FLUSH_UPLOAD", "FOG_CULL", "FOLIAGE_SHADOW_CULL", "FOLIAGE_SHADOW_VRS", "FOV", "FOV_PATCH", "FG_HUDLESS_SHOW", "FG_MULT", "FPS60", "FPS_LIMIT", "FRAMEGEN", "FRAMEGEN_MULTIPLIER", "FREEZE_IDLE", "FREEZE_ASYNC", "FREEZE_BUDGET_US", "FREE_CORES",
@@ -66,7 +66,7 @@ static const char *const known[] = {
"MVOBJ_ALL", "MVOBJ_BANKS", "MVOBJ_EQUAL", "MVOBJ_EXACT", "MVOBJ_EXEC", "MVOBJ_NOC4", "MVOBJ_ONLY", "MVOBJ_TIMES", "MVPREV_CHECK", "MVPREV_GROW", "MVPREV_SKIP", "MVPREV_TRUST", "MV_CHECK",
"MV_SURVEY",
"HANDLE_CENSUS",
"PIPE_SEED_BOOT", "ONE_PER_CORE", "OPERATION_MODE", "OUTPUT_RES", "PAD_LAYOUT", "PACE_HZ", "PASS_CRUMBS", "PATCH_OUTPUT", "PATCH_RES_FIXED", "PATCH_SCENE", "PATCH_WIDE_ASPECT", "PATCH_WIDE_HUD", "PATCH_WIDE_MOVIE", "PATCH_WIDE_OUT", "PATCH_FREECAM", "PATCH_OCCLUSION_OFF", "PATCH_WIDE_SAVE", "PATCH_WIDE_SCENE", "PCSS", "PCSS_K", "PERF_OVERLAY", "PERF_SLOW_MS", "PIPELINE_CACHE", "PIPELINE_CACHE_MAX_MB", "PIPELINE_CACHE_PLAY", "PIPELINE_CACHE_PLAY_MB", "PIPE_BINARIES", "PIPE_BINARIES_MAX_MB", "PIPE_FEEDBACK", "PIPE_MEMO", "PIPE_PREBUILD", "PIPE_SEED", "PIPE_SKIP", "PIPE_SYNC", "PIPE_WAIT", "POWER_FULL", "TEX_SAME", "TEX_SAME_BIG", "TEX_FLAT3D", "TEX_KEEP_TAKEN", "TEX_HARD_ORDER", "TEX_STALE_KEPT", "ASTC_SCRATCH_TRIM", "PRESENT_HELD", "HELD_SYNC",
"PIPE_SEED_BOOT", "ONE_PER_CORE", "OPERATION_MODE", "OUTPUT_RES", "PAD_LAYOUT", "PACE_HZ", "PASS_CRUMBS", "PATCH_OUTPUT", "PATCH_RES_FIXED", "PATCH_SCENE", "PATCH_WIDE_ASPECT", "PATCH_WIDE_HUD", "PATCH_WIDE_MOVIE", "PATCH_WIDE_OUT", "PATCH_FREECAM", "PATCH_OCCLUSION_OFF", "PATCH_WIDE_SAVE", "PATCH_WIDE_SCENE", "PCSS", "PCSS_K", "PERF_OVERLAY", "PERF_SLOW_MS", "PIPELINE_CACHE", "PIPELINE_CACHE_MAX_MB", "PIPELINE_CACHE_PLAY", "PIPELINE_CACHE_PLAY_MB", "PIPE_BINARIES", "PIPE_BINARIES_MAX_MB", "PIPE_FEEDBACK", "PIPE_MEMO", "PIPE_PREBUILD", "PIPE_SEED", "PIPE_SKIP", "PIPE_SYNC", "PIPE_WAIT", "POWER_FULL", "TEX_SAME", "TEX_SAME_BIG", "TEX_FLAT3D", "TEX_KEEP_TAKEN", "TEX_RATE_LOW", "TEX_HARD_ORDER", "TEX_STALE_KEPT", "ASTC_SCRATCH_TRIM", "PRESENT_HELD", "HELD_SYNC",
"PRESENT_ORDER",
"PREPARE_PIPELINES", "PREWARM_FULL", "PREWARM_FULL_S", "PREWARM_WAIT_MS", "PREWARM_XLATE", "PROBE_MOVIE", "PROBE_MOVIE_SHOT", "PROFILE", "PROFILE_CHAINS", "PROFILE_DIR", "PROFILE_FRAMES", "PROFILE_REPLAY", "PROFILE_SLOW",
"POOL_FAST", "POOL_GROUP", "POOL_GROUP_CHECK", "POOL_PRESNAP", "POOL_PRESNAP_RUNS",

View File

@@ -237,6 +237,26 @@ extern "C" NX2SL_API int nx2sl_frame(VkCommandBuffer cb, const Nx2SlImage *depth
uint32_t max = g_max.load();
int n = enable > 0 ? enable : 0;
if (n > (int)(max ? max : 1)) n = (int)(max ? max : 1);
static sl::Constants last; // the constants of the last frame that had its inputs
static bool have_last;
if (n && !(depth && mv && cam) && have_last) {
// Generation is on for this present and the frame has no depth, motion vectors or camera: a menu, a fade or a
// load, while vk_dlssg.c holds the change of state for a few frames. Streamline's guide asks for the common
// constants at every frame and for null tags wherever depth or motion vectors are not valid
// (ProgrammingGuideDLSS_G.md 5.2 and 7.0). This frame gave neither, and Streamline logged "Unable to find
// 'common' constants for frame" and "Failed to find global tag 'kBufferTypeDepth'" and went on with what the
// frame before had set. It gets the last frame's constants again, as a new start, and the three tags as null.
sl::Constants c = last;
c.reset = sl::Boolean::eTrue;
p_constants(c, *t, vp);
sl::ResourceTag none[3] = {sl::ResourceTag(nullptr, sl::kBufferTypeDepth, sl::ResourceLifecycle::eValidUntilPresent),
sl::ResourceTag(nullptr, sl::kBufferTypeMotionVectors, sl::ResourceLifecycle::eValidUntilPresent),
sl::ResourceTag(nullptr, sl::kBufferTypeHUDLessColor, sl::ResourceLifecycle::eValidUntilPresent)};
if ((t = token(frame))) p_tag(*t, vp, none, 3, (sl::CommandBuffer *)cb);
static int said;
if (!said++) say("dlssg: a frame without depth and motion vectors while generation is on gets the last constants "
"as a new start and null tags");
}
if (n && depth && mv && cam) {
sl::Constants c{};
matrix(c.cameraViewToClip, cam->view_to_clip);
@@ -259,6 +279,7 @@ extern "C" NX2SL_API int nx2sl_frame(VkCommandBuffer cb, const Nx2SlImage *depth
c.motionVectors3D = sl::Boolean::eFalse;
c.reset = cam->reset ? sl::Boolean::eTrue : sl::Boolean::eFalse;
p_constants(c, *t, vp);
last = c, have_last = true;
sl::Resource rd = resource(depth), rm = resource(mv), rh = hudless ? resource(hudless) : sl::Resource{};
sl::Extent ed{0, 0, depth->width, depth->height}, em{0, 0, mv->width, mv->height};
sl::Extent eh{0, 0, hudless ? hudless->width : 0, hudless ? hudless->height : 0};

View File

@@ -2451,6 +2451,21 @@ static void evaluate(VkExec *ex, const VkExecTarget *dst) {
VkImageView zv = vk_exec_target_view(z->img, z->format, VK_IMAGE_VIEW_TYPE_2D, VK_IMAGE_ASPECT_DEPTH_BIT, z->level,
z->base_layer, 1, 0);
float jx = D.jit_used ? D.ux : 0.0f, jy = D.jit_used ? D.uy : 0.0f;
// NX2NAVI_DLAA_JITTER_TEST=<x>,<y> (test, read once): the upscaler is told the jitter the scene was drawn with
// times these two factors. NVIDIA's DLSS guide gives no sign for the offsets against the projection's shift and
// says to find it by trying (8.4.1: its development library negates, halves or doubles an axis by a key): the
// picture of a still scene is sharp and stands still only with the right ones. The scene, the motion vectors and
// frame generation keep the real jitter.
static float jt[2] = {1.0f, 1.0f};
static int jt_read;
if (!jt_read) {
jt_read = 1;
const char *s = cfg_get("NX2NAVI_DLAA_JITTER_TEST");
if (s && *s && sscanf(s, "%f,%f", &jt[0], &jt[1]) == 2)
rt_log("dlaa: the upscaler is told the jitter times %g, %g (NX2NAVI_DLAA_JITTER_TEST, a test)", jt[0], jt[1]);
else jt[0] = jt[1] = 1.0f;
}
const float tjx = jx * jt[0], tjy = jy * jt[1];
edge_fill(cb, t, z, D.jit_used); // before the motion vectors read the depth
static int no_mv = -1;
if (no_mv < 0) no_mv = !rt_knob("DLAA_MV", "1"); // NX2NAVI_DLAA_MV=0: zero motion (test)
@@ -2482,7 +2497,7 @@ static void evaluate(VkExec *ex, const VkExecTarget *dst) {
fm = {e->mv, VK_FORMAT_R16G16_SFLOAT, w, h, 1}, fo = {e->out, VK_FORMAT_R16G16B16A16_SFLOAT, ow, oh, 1};
VkFfxCamera fcam;
if (!vk_dlaa_ffx_camera(&fcam)) memset(&fcam, 0, sizeof fcam); // no camera this frame: FSR's fixed values
if (!vk_ffx_upscale(cb, &fc, &fz, &fm, &fo, jx, jy, D.reset, dt > 0 && dt < 100 ? (float)dt : 16.67f, &fcam))
if (!vk_ffx_upscale(cb, &fc, &fz, &fm, &fo, tjx, tjy, D.reset, dt > 0 && dt < 100 ? (float)dt : 16.67f, &fcam))
return dl_fail("FSR 3.1", 0);
static int told;
if (!told++) rt_log("dlaa: FSR 3.1 at %ux%u -> %ux%u (scene format %d, depth format %d)", w, h, ow, oh,
@@ -2557,8 +2572,8 @@ own_feature:
p_setp(p, "Output", &ro);
p_setp(p, "Depth", &rz);
p_setp(p, "MotionVectors", &rm);
p_setf(p, "Jitter.Offset.X", jx);
p_setf(p, "Jitter.Offset.Y", jy);
p_setf(p, "Jitter.Offset.X", tjx);
p_setf(p, "Jitter.Offset.Y", tjy);
p_setf(p, "Sharpness", 0.0f);
p_seti(p, "Reset", D.reset);
// One bias mask: the reactive one (NX2NAVI_DLSS_REACTIVE, a test) where it ran, else the characters' (mv_pass), else
@@ -3221,8 +3236,20 @@ static void dlss_mode(void) {
bias_on = -1;
}
int b = dl_active();
if (b != bias_on)
bias_on = b, vk_sampler_dlss_bias(b ? res_dlss_lod_bias() + (jitter_on() ? JITTER_LOD_BIAS : 0.0f) : 0.0f);
if (b != bias_on) {
// NX2NAVI_DLSS_LOD_BIAS_TEST=<bias> (test, read once): this bias in the place of the rule's while an upscaler
// runs, 0 for none: what the rule's bias does to the picture is the difference between two runs.
static float test;
static int test_on = -1;
if (test_on < 0) {
const char *s = cfg_get("NX2NAVI_DLSS_LOD_BIAS_TEST");
test_on = s && *s;
if (test_on) test = (float)atof(s), rt_log("dlaa: texture LOD bias %g while an upscaler runs "
"(NX2NAVI_DLSS_LOD_BIAS_TEST, a test)", test);
}
float rule = res_dlss_lod_bias() + (jitter_on() ? JITTER_LOD_BIAS : 0.0f);
bias_on = b, vk_sampler_dlss_bias(b ? (test_on ? test : rule) : 0.0f);
}
}

View File

@@ -139,3 +139,36 @@ static inline uint32_t trl_clock_age(const TrlClock *c, uint64_t frame, uint32_t
uint32_t s = c->sec[lo % TRL_CLOCK_N];
return now_sec > s ? now_sec - s : 0;
}
// The rate the short waits go by. vk_texture.c counts them in presents (PRESENTS(n): n presents at 60 a second: how
// long after its release a texture may go, how long without a sample makes one idle) and scales them by the presents
// a second, kept as a multiple of 60 in 1/256. Above 60 they always were scaled. Below 60 they were not: the rate
// stopped at 60, so "120 presents" was 4 s at 30 presents a second and 5.7 s at 21, and every texture the game let go
// of stayed out of the VRAM pressure's reach that much longer. The game lets go by the second, as the player moves, so
// what the window holds grows with its length: in a tester's log (RTX 4070 SUPER at 2560x1440, 2026-11-14) it held
// 442 MiB at 60 presents a second and 2,105 MiB at 21, on a card then 600 MiB over its budget. The rate follows the
// presents down to TRL_RATE_MIN now (low 1; 0: stops at 60, as before).
#define TRL_RATE_60 256u
#define TRL_RATE_MIN 64u // 15 presents a second: fewer count as 15
#define TRL_RATE_MAX 2048u // 480 presents a second
static inline uint32_t trl_rate_sample(uint64_t presents, uint64_t ns, int low) {
if (!ns) return TRL_RATE_60;
uint64_t r = presents * 256ull * 1000000000ull / (ns * 60);
uint32_t least = low ? TRL_RATE_MIN : TRL_RATE_60;
return r < least ? least : r > TRL_RATE_MAX ? TRL_RATE_MAX : (uint32_t)r;
}
// The rate of the last n samples (one a second): the highest of them, so that a slow second or a few (an area load, a
// movie's start) shorten nothing, and only n slow seconds in a row do. A sample not taken yet (0) counts as 60 a
// second: the first n seconds of a run never go below it.
static inline uint32_t trl_rate(const uint32_t *ring, int n) {
uint32_t m = TRL_RATE_MIN;
for (int i = 0; i < n; i++) {
uint32_t v = ring[i] ? ring[i] : TRL_RATE_60;
if (v > m) m = v;
}
return m;
}
// n presents at 60 a second, in presents at the rate: the same wall time.
static inline uint64_t trl_presents(uint64_t n, uint32_t rate) { return n * rate >> 8; }

View File

@@ -179,11 +179,17 @@ static uint32_t submit_epoch = 1; // bumped per queue submit: draw bindings ch
// were set at 60 presents a second. Without a limit the game presents 85 to 240 a second (a menu), and "120 presents"
// was then half a second and "30 s without a sample" 7 to 20 s: under memory pressure textures went three times as
// readily as meant, and the ones still on screen showed their dark stand-ins until they were made again.
// tex_rate is the presents per second as a multiple of 60 in 1/256 (256 at 60 or fewer), the highest of the last RATE_N
// seconds so that one slow second does not shorten the thresholds; PRESENTS(n) is n presents at 60 in today's presents.
// tex_rate is the presents per second as a multiple of 60 in 1/256, the highest of the last RATE_N seconds so that one
// slow second does not shorten the thresholds; PRESENTS(n) is n presents at 60 in today's presents.
// NX2NAVI_TEX_RATE_LOW (default 1; 0: as before): the rate follows the presents below 60 a second too, down to 15
// (vk_tex_release.h trl_rate_sample). It stopped at 60, so at 21 presents a second a texture stayed out of the VRAM
// pressure's reach for 5.7 s after its release and not 2, and the released ones held 2.1 GiB of a 12 GiB card that
// was over its budget.
#define RATE_N 8
static uint32_t tex_rate = 256;
#define PRESENTS(n) ((uint64_t)(n) * tex_rate >> 8)
#define RATE_SAY 16
#define RATE_SAY_UNDER 240u // the log's line: under 56 presents a second (a limit of 60 measures 59 now and then)
static uint32_t tex_rate = TRL_RATE_60;
#define PRESENTS(n) trl_presents((uint64_t)(n), tex_rate)
// The wall clock of the presents (vk_tex_release.h TrlClock): rate_tick notes the present count about once a second,
// so that the released textures' hold (drop_aged) and the take-back ages (vk_texture_reuse_log) count seconds, not
@@ -215,12 +221,21 @@ static void rate_tick(void) { // each submission, tex_lock held
if (!t0) t0 = t, f0 = f, pclock_t0 = t, trl_clock_tick(&pclock, f, 0);
if (t - t0 < 1000000000ull) return;
trl_clock_tick(&pclock, f, (uint32_t)(play_ns / 1000000000ull));
uint64_t r = (f - f0) * 256ull * 1000000000ull / ((t - t0) * 60);
ring[at++ % RATE_N] = r < 256 ? 256 : r > 2048 ? 2048 : (uint32_t)r;
static int low = -1, said;
if (low < 0) low = rt_knob("TEX_RATE_LOW", "1") != 0;
ring[at++ % RATE_N] = trl_rate_sample(f - f0, t - t0, low);
t0 = t, f0 = f;
uint32_t m = 256;
for (int i = 0; i < RATE_N; i++)
if (ring[i] > m) m = ring[i];
uint32_t m = trl_rate(ring, RATE_N);
// one line at each crossing of RATE_SAY_UNDER, RATE_SAY times a run: the log then shows which waits the drops went by
static int slow;
if ((m < RATE_SAY_UNDER) != slow && said < RATE_SAY) {
said++, slow = m < RATE_SAY_UNDER;
if (slow)
rt_log("vk: the game presents %u frames a second, fewer than 60, for %d seconds: the textures' waits count "
"their seconds in these (2 s are %llu presents, not 120; NX2NAVI_TEX_RATE_LOW=0: 120)",
m * 60 >> 8, RATE_N, (unsigned long long)trl_presents(120, m));
else rt_log("vk: the game presents 60 frames a second or more again: the textures' waits count 60 or more");
}
tex_rate = m;
}
// Mirrored textures the GPU wrote into, sorted by address (vk_texture_cpu_stale, vk_texture_write_back). Changed under
@@ -3425,6 +3440,12 @@ static void rel_census(uint64_t cap, int room) { // tex_lock held
enum { YOUNG, SLOT, SLOT_OVER, SAMPLED, LOOKED, NEXT, KINDS };
uint64_t by[KINDS] = {0}, gpu = 0;
int n[KINDS] = {0}, ngpu = 0;
// the young ones by what they wait for: fewer than STALE_FRAMES since the release; content only the GPU made
// (GPU_STALE_FRAMES or the gap); intact and waiting out the gap (stale_after) with nothing reaching them, which
// the VRAM pressure takes; intact, waiting it out and still reached
enum { Y_NEW, Y_GPU, Y_GAP, Y_REACHED, YKINDS };
uint64_t yby[YKINDS] = {0}, gap_max = 0;
int yn[YKINDS] = {0};
for (const Tex *e = rel_head; e; e = e->rel_next) {
if (!e->bytes) continue;
int shown = e->ph && pref[e->ph - 1].refs;
@@ -3434,6 +3455,12 @@ static void rel_census(uint64_t cap, int room) { // tex_lock held
: !e->overwritten && !idle(e, now) ? LOOKED : NEXT;
by[k] += e->bytes, n[k]++;
if (!comes_back_intact(e)) gpu += e->bytes, ngpu++;
if (k == YOUNG) {
int y = now - e->rel_frame < STALE_FRAMES ? Y_NEW
: !comes_back_intact(e) && !e->overwritten ? Y_GPU : idle(e, now) ? Y_GAP : Y_REACHED;
yby[y] += e->bytes, yn[y]++;
if (y >= Y_GAP && stale_after(e) > gap_max) gap_max = stale_after(e);
}
}
const double M = 1048576.0;
rt_log("vk: released textures over their bound%s: %d entries, %.0f MiB of %.0f MiB (%d of them, %.0f MiB, hold content "
@@ -3443,6 +3470,15 @@ static void rel_census(uint64_t cap, int room) { // tex_lock held
room ? " (kept past it while the card has room)" : "", nreleased, rel_bytes / M, cap / M,
ngpu, gpu / M, n[YOUNG], by[YOUNG] / M, n[SAMPLED], by[SAMPLED] / M, n[SLOT], by[SLOT] / M, n[SLOT_OVER],
by[SLOT_OVER] / M, n[LOOKED], by[LOOKED] / M, n[NEXT], by[NEXT] / M);
if (n[YOUNG])
rt_log("vk: the ones released too lately wait: %d, %.0f MiB, the first %llu presents after a release (2 s at the "
"%u presents a second the waits count); %d, %.0f MiB, with content only the GPU made, %llu presents or their own "
"gap; %d, %.0f MiB, intact, for 8 times their longest gap before a take-back (%llu presents at most) with "
"nothing reaching them: the VRAM pressure takes these; %d, %.0f MiB, intact, for their gap and still "
"reached",
yn[Y_NEW], yby[Y_NEW] / M, (unsigned long long)STALE_FRAMES, tex_rate * 60 >> 8, yn[Y_GPU], yby[Y_GPU] / M,
(unsigned long long)GPU_STALE_FRAMES, yn[Y_GAP], yby[Y_GAP] / M, (unsigned long long)gap_max, yn[Y_REACHED],
yby[Y_REACHED] / M);
}
static void sweep_pressure(void) { // tex_lock held

View File

@@ -60,6 +60,55 @@ static void clock_checks(void) {
"short run: %u %u %u", trl_clock_age(&c2, 50, 10), trl_clock_age(&c2, 0, 10), trl_clock_age(&c2, 251, 10));
}
// ---- the rate the short waits go by (trl_rate_sample, trl_rate, trl_presents)
static void rate_checks(void) {
const uint64_t S = 1000000000ull;
CHECK(trl_rate_sample(60, S, 1) == 256 && trl_rate_sample(60, S, 0) == 256, "60 presents a second");
CHECK(trl_rate_sample(240, S, 1) == 1024 && trl_rate_sample(240, S, 0) == 1024, "240 a second");
CHECK(trl_rate_sample(1000, S, 1) == TRL_RATE_MAX, "1000 a second counts as 480");
CHECK(trl_rate_sample(30, S, 1) == 128, "30 a second: %u", trl_rate_sample(30, S, 1));
CHECK(trl_rate_sample(21, S, 1) == 89, "21 a second: %u", trl_rate_sample(21, S, 1));
CHECK(trl_rate_sample(5, S, 1) == TRL_RATE_MIN && trl_rate_sample(0, S, 1) == TRL_RATE_MIN, "under 15 a second: 15");
CHECK(trl_rate_sample(30, S, 0) == 256 && trl_rate_sample(0, S, 0) == 256, "the rule before: never under 60");
CHECK(trl_rate_sample(100, 0, 1) == 256, "no time passed: 60");
CHECK(trl_rate_sample(63, 3 * S, 1) == 89, "21 a second over three seconds: %u", trl_rate_sample(63, 3 * S, 1));
// the highest of the samples; one not taken yet counts as 60, so a run's first seconds never go below it
enum { N = 8 };
uint32_t ring[N] = {0};
CHECK(trl_rate(ring, N) == 256, "no sample yet: %u", trl_rate(ring, N));
for (int i = 0; i < N - 1; i++) ring[i] = trl_rate_sample(21, S, 1);
CHECK(trl_rate(ring, N) == 256, "seven slow seconds and one not sampled: %u", trl_rate(ring, N));
ring[N - 1] = trl_rate_sample(21, S, 1);
CHECK(trl_rate(ring, N) == 89, "eight slow seconds: %u", trl_rate(ring, N));
ring[3] = trl_rate_sample(60, S, 1);
CHECK(trl_rate(ring, N) == 256, "one second at 60 among them: %u", trl_rate(ring, N));
ring[3] = trl_rate_sample(144, S, 1);
CHECK(trl_rate(ring, N) == trl_rate_sample(144, S, 1), "one second at 144 among them");
for (int i = 0; i < N; i++) ring[i] = trl_rate_sample(120, S, 1);
ring[5] = trl_rate_sample(2, S, 1); // an area load: one second with two presents
CHECK(trl_rate(ring, N) == 512, "a slow second among fast ones shortens nothing: %u", trl_rate(ring, N));
// 120 presents at 60 a second are 2 s at every rate from 15 a second up: STALE_FRAMES, the wait after a release
static const uint32_t fps[] = {15, 21, 30, 45, 60, 90, 144, 240};
printf("the wait after a release (120 presents at 60 a second), by the presents a second:\n");
printf(" presents/s before: presents, seconds now: presents, seconds\n");
for (unsigned i = 0; i < sizeof fps / sizeof *fps; i++) {
uint64_t was = trl_presents(120, trl_rate_sample(fps[i], S, 0)), now = trl_presents(120, trl_rate_sample(fps[i], S, 1));
double was_s = (double)was / fps[i], now_s = (double)now / fps[i];
printf(" %10u %16llu %7.2f %13llu %7.2f\n", fps[i], (unsigned long long)was, was_s,
(unsigned long long)now, now_s);
CHECK(now_s > 1.9 && now_s <= 2.0, "%u presents a second: %.2f s", fps[i], now_s);
if (fps[i] >= 60) CHECK(was == now, "%u presents a second: %llu presents, %llu before", fps[i],
(unsigned long long)now, (unsigned long long)was);
}
// under 15 a second the wait is 15 a second's: 30 presents, longer than 2 s
CHECK(trl_presents(120, trl_rate_sample(10, S, 1)) == 30, "10 presents a second: 30 presents");
// the tester's log (vk_tex_release.h): 21 presents a second. Before, 120 presents were 5.7 s there
double before = (double)trl_presents(120, trl_rate_sample(21, S, 0)) / 21;
CHECK(before > 5.7 && before < 5.72, "the rule before at 21 a second: %.2f s", before);
// the longer waits scale the same way: GPU_STALE_FRAMES (1800 presents at 60, 30 s)
CHECK(trl_presents(1800, trl_rate_sample(30, S, 1)) == 900, "30 s at 30 presents a second: 900 presents");
}
// ---- the rules
static void rule_checks(void) {
CHECK(trl_hold(TRL_HOLD_S, TRL_STORAGE) == 60 && trl_hold(TRL_HOLD_S, TRL_GPU) == 120, "holds 60 and 120 s");
@@ -335,6 +384,7 @@ static void hard_checks(void) {
int main(void) {
clock_checks();
rate_checks();
rule_checks();
taken_back_checks();
session_checks();