diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index a7bc7df1..665d911d 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -117,11 +117,15 @@ folder's `licenses/` and in each component's own folder. | CPython 3.14.9 embeddable package; includes `vcruntime140*.dll` (Microsoft, redistributable with Python), OpenSSL, libffi, SQLite, zlib, libmpdec | https://www.python.org/downloads/release/python-3149/ | PSF-2.0 | `python/` | | `tkinter` and Tcl/Tk 9.0 (`_tkinter.pyd`, `tcl90.dll`, `tcl9tk90.dll`, `zlib1.dll`, `libtommath.dll`, `tcl/`), from python.org's 3.14.9 `amd64/tcltk.msi`, unpacked, never run | https://www.python.org/ftp/python/3.14.9/amd64/ | PSF-2.0; Tcl/Tk licence (BSD-style); libtommath public domain or BSD; zlib licence | `python/` | | numpy 2.5.4 (with its bundled OpenBLAS), lz4 4.4.5, zstandard 0.25.0 | PyPI binary wheels, pins in `tools/requirements-launcher.txt` | BSD-3-Clause; lz4 BSD-3-Clause; zstandard BSD-3-Clause | `python/Lib/site-packages/` | +| The runtime of this repository compiled into `lib/libnx2navirt.a` (`tools/make_game.py --make-runtime-lib`), with the third-party code it contains as listed in the tables above (Dear ImGui, the copied runtime parts, FidelityFX FSR 1, Vulkan Memory Allocator and the others) and inline code from the toolchain's C++ headers | this repository; MSYS2 UCRT64 toolchain of `toolchain.lock.json` | as listed above for each part; libstdc++ headers GPL-3.0 with the GCC Runtime Library Exception | `app//lib/` | +| truststore 0.10.5, unmodified (Windows' own certificate check for `launcher/net.py`) | https://pypi.org/project/truststore/0.10.5/ (`truststore-0.10.5-py3-none-any.whl`, pinned with its sha256 in `tools/requirements-launcher.txt`), Copyright (c) 2022 Seth Michael Larson | MIT | `python/Lib/site-packages/truststore/`, its licence in `truststore-0.10.5.dist-info/licenses/` | | zlib 1.4 (MSYS2 `mingw-w64-ucrt-x86_64-zlib`), static library | https://zlib.net, Copyright (C) Jean-loup Gailly and Mark Adler | zlib | linked into `NX2navi-Setup.exe` (its unzip) | | mingw-w64 CRT startup code and winpthreads 14.0.0, GCC 16.2.0's `libgcc` (MSYS2 packages), static | https://www.mingw-w64.org, https://gcc.gnu.org | mingw-w64: ZPL-2.1 / MIT / public domain; libgcc: GPL-3.0 with the GCC Runtime Library Exception | linked into `NX2navi.exe` and `NX2navi-Setup.exe` | Downloaded on the player's PC by the launcher, not distributed: the MSYS2 UCRT64 packages that `toolchain.lock.json` pins (clang and LLVM, Apache-2.0 with LLVM exception; GCC and binutils, GPL-3.0; the libraries of the "Compiler runtime in a source build" section and of `README-BuildKit.md`), from -https://repo.msys2.org/mingw/ucrt64/; and zbic 1.0.1.post1 (GPL-2.0), from PyPI. The game build links them as a -source build does. +https://repo.msys2.org/mingw/ucrt64/; the MSYS2 CLANG64 clang it pins beside them (clang, LLVM and libc++, +Apache-2.0 with LLVM exception; libffi MIT; zlib; zstd BSD-3-Clause; libxml2 MIT; libiconv LGPL-2.1+), from +https://repo.msys2.org/mingw/clang64/, which only compiles the lifted code; and zbic 1.0.1.post1 (GPL-2.0), from +PyPI. The game build links them as a source build does. diff --git a/docs/README-BuildKit.md b/docs/README-BuildKit.md index 3f3f5302..d762f517 100644 --- a/docs/README-BuildKit.md +++ b/docs/README-BuildKit.md @@ -49,9 +49,9 @@ With extracted folders instead, use `--exefs "C:\path\exefs" --romfs "C:\path\ro check that your dump is the right version, add `--check` (it builds nothing). The build checks the version, unpacks the program files, packs the 26 GB RomFS image (every file is checked against -the dump's own checksums), turns the game's code into C and compiles it (about 6 minutes on a 16-thread Ryzen 7; longer on fewer cores), -then prepares every one of the game's shaders for your graphics card (about 5 more minutes), so the game does not stop -to compile shaders while you play. +the dump's own checksums) while it turns the game's code into C and compiles it (about 6 minutes on a 16-thread +Ryzen 7; longer on fewer cores), then prepares every one of the game's shaders for your graphics card (about 5 more +minutes), so the game does not stop to compile shaders while you play. The game it builds is for this PC's processor. You can use your PC meanwhile. If it stops, run the same command again: finished steps are skipped. @@ -62,7 +62,9 @@ When it says `finished`, start `C:\Games\Ocarina\ocarina.exe`. You can delete `C - **F1** opens the settings menu. Settings are saved in `nx2navi.ini` beside the game; saves go to `profile\`. - Every shader was prepared during the build. After an update of your graphics driver, the next start shows - "Preparing shaders" once while the driver rebuilds them. + "Preparing shaders" once while the driver rebuilds them. With a driver that has Vulkan's graphics pipeline library + it lasts at most about 20 seconds once the shaders are translated, and the remaining pipelines are built in the + background while you play. - **Screen** in the menu has Upscaling (DLSS on NVIDIA RTX cards, FSR 3.1 on any card), Frame generation (FSR 3.1; it needs DLSS or FSR 3.1 upscaling on), HDR (Native is the game's own HDR; turn HDR on in Windows first) and the resolution. Those marked "after restart" take effect when the game starts again. diff --git a/docs/architecture.md b/docs/architecture.md index 9e163323..7036cbca 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -85,9 +85,13 @@ keeps per-core data without locks and relies on that, so the host models it (`ru - Shaders: Ampere (SM86) code is lowered to Maxwell (`tools/xlate/AmpereLowering.cs`) and translated to SPIR-V by the pinned shader translator (`tools/vendor/ryujinx`, 1.3.2) with our patches (`tools/xlate/patches/`), built with .NET 10 NativeAOT as `nx2navi_xlate.exe` and run as separate processes. Translations are cached on disk; shaders and pipelines seen in - play are prepared at the next boot. The game waits for all of it (`runtime/boot.c` `boot_wait`): about half a second + play are prepared at the next boot. The game waits for it (`runtime/boot.c` `boot_wait`): about half a second when warm; after a translator, GPU or driver change a "Preparing shaders" screen shows the progress (`ui/menu.cpp` - `menu_boot_frame`). `NX2NAVI_PREWARM_WAIT_MS` caps the wait, after which the rest finishes in the background. + `menu_boot_frame`). With the graphics pipeline library the boot builds recorded pipelines in record order for at + most 20 s after the shaders are ready and leaves the rest to the pipeline workers below normal priority; a draw + that meets one before it is built gets a fast-linked pipeline (`vk_pipeline.c` `PREWARM_PIPE_MS`). + `NX2NAVI_PREWARM_FULL=1`, and drivers without the library, wait for every pipeline. `NX2NAVI_PREWARM_WAIT_MS` caps + the whole wait, after which the rest finishes in the background. - Mesh and task shaders run as `VK_EXT_mesh_shader`; indirect mesh draws read NVN's `{count, first}` record through a ring of Vulkan `{x, y, z}` records filled on the GPU. - Pipelines: `VK_EXT_graphics_pipeline_library` where the driver has it: stage libraries compiled ahead, a fast link diff --git a/docs/building.md b/docs/building.md index 7e955617..ce7e0e6b 100644 --- a/docs/building.md +++ b/docs/building.md @@ -228,6 +228,13 @@ Before a commit: `git config core.hooksPath .githooks` runs it before each local commit. - `build\\nx2navi.exe --selftest`: layout, dispatch table and host-function registry, without game files. +A change to the in-game menu's look (`runtime/ui/menu_skin.cpp`, `option_rows.cpp`, `perf_panel.cpp`) can be looked at +without game files: `python tools/menu_preview.py` builds the menu's own drawing code with the host's clang++ and +writes the Screen tab and the full performance overlay to `build/menu-preview/` (a stand-in font, and the window's +plain border in place of the game's ornamented frame). With `--romfs ` it draws the game's +interface textures as the game does; `python tools/ui_textures.py ` lists the textures there and dumps those +matching `--match` to `build/ui-textures/`. + A change to the button prompts (`tools/make_romfs_patches.py --prompts`, `tools/button_icons.py`) passes `.venv-py314\Scripts\python tests\romfs_prompts_test.py`, which runs the tool on a RomFS it makes itself (no game files); a change to the settings passes `nx2navi.exe --probe-cfg`. diff --git a/docs/launcher.md b/docs/launcher.md index c06d6625..ba2fc390 100644 --- a/docs/launcher.md +++ b/docs/launcher.md @@ -23,20 +23,23 @@ game. Saves and settings are never touched. ## Look -The window follows the game's options menu, with colours sampled from it: a wooden title band with the Triforce and -the name, bronze label cells, dark value cells, orange On/Off choices, a white glowing cursor around the selected +The window follows the game's options menu, with colours sampled from it: a wooden band across the whole top with +Navi and the name, which is also the window's title bar (no system title bar: drag the band to move the window, +minimise and close sit at its right end, Alt+F4 closes), bronze label cells, dark value cells, orange On/Off choices, a white glowing cursor around the selected row, the menu's filigree dividers and a description box for the selected row. All text is in the game's own Cetra Display: the launcher decodes it from the player's dump or built game into `data/cache/fonts` (as `runtime/menu_assets.c` does for the in-game menu) and uses the system's interface font until then. Every picture is drawn in code (`launcher/ui/art.py`); no texture, font or sound of the game is in the repository or the release. -The Triforce is also the icon of `NX2navi.exe`, `NX2navi-Setup.exe`, the window and the taskbar -(`art.triforce_ico`, written into the release by `tools/make_launcher.py`). +Navi (`art.NAVI`, 22 x 21 pixels, drawn in whole pixels at every size) is also the icon of `NX2navi.exe`, +`NX2navi-Setup.exe`, the window and its taskbar button (`art.navi_ico`, written into the release by +`tools/make_launcher.py`). The window sets that icon on itself, and the setup's shortcuts carry the window's taskbar +identity (`NX2navi.Launcher`), so the taskbar groups and pins it under NX2navi rather than Python. Screens (`launcher/ui/app.py`): first run (dump, install folder, Desktop Shortcut, Check for Updates at Start, Update Channel, Begin Installation); the steps of an install, update or rebuild with their progress, the build log's -last lines and Cancel, and after a failure Try Again and Copy Report; home (Play, Updates, Game Folder, Rebuild the -Game, Settings); settings (update check, Hide Launcher While Playing, Keep Build Cache, Update Channel, the folders, +last lines and Cancel, and after a failure Try Again and Copy Report; home (Play, Updates, Game Folder, Import Saves, +Rebuild the Game, Settings); settings (update check, Hide Launcher While Playing, Keep Build Cache, Update Channel, the folders, Open Logs Folder, Uninstall, Back); uninstall (keep or remove the built game). Controls: the mouse (hover moves the cursor, a click selects); the keyboard (arrows, Enter or Space, Esc, F5 checks @@ -48,7 +51,7 @@ for updates); an XInput gamepad (d-pad or left stick, A, B, Y). ``` NX2navi.exe launcher/native/nx2navi.c: swaps in a staged Python, starts app\\launcher\start.pyw -python/ python.org's embeddable Python 3.14.9 with tkinter, numpy, lz4 and zstandard +python/ python.org's embeddable Python 3.14.9 with tkinter, numpy, lz4, zstandard and truststore app/current.txt the version the stub starts; app/previous.txt the one before it app// launcher/, runtime/, recomp/, tools/, bin/nx2navi_xlate.exe, VERSION, toolchain.lock.json (the pinned build tools), feed.json (where updates come from) @@ -56,7 +59,8 @@ pydeps/ zbic, downloaded from PyPI on this PC toolchain/msys64/ucrt64 MSYS2's clang and libraries, downloaded from the MSYS2 repository on this PC toolchain/pkgs/ the downloaded package files data/launcher.json settings; data/logs/launcher.log (and crash.log); data/cache/fonts -game/ the built game, unless another folder was chosen (its layout: README-BuildKit.md) +game/ the built game, unless another folder was chosen (its layout: README-BuildKit.md), with + saves/ for saves to import (below) ``` A path the Windows ANSI code page cannot hold breaks the MSYS2 tools, so the toolchain then goes to @@ -67,12 +71,23 @@ A path the Windows ANSI code page cannot hold breaks the MSYS2 tools, so the too `jobs.InstallJob`, one row per step: 1. **Gather Build Tools** (`launcher/toolchain.py`): the packages `toolchain.lock.json` pins (MSYS2 UCRT64 clang, - gcc, SDL3, Opus, libvpx, dav1d, zstd, Vulkan headers and loader, VMA and their dependencies: 31 packages, 170 MB), + gcc, lld, SDL3, Opus, libvpx, dav1d, zstd, Vulkan headers and loader, VMA and their dependencies: about 32 + packages, 175 MB; and, when the release found the same clang version there, MSYS2 CLANG64's clang with the DLLs + it loads: 9 packages, about 100 MB), each checked against its sha256 and unpacked into `toolchain/msys64.new`, which replaces `toolchain/msys64` once complete. No MSYS2 shell or pacman runs. Then zbic: the wheel for the bundled Python from PyPI, checked against PyPI's sha256, into `pydeps/`. 2. **Inspect Your Game** to **Compile**: `tools/make_game.py --nup --out --msys --events`, - whose `@@` progress lines drive the rows (check, exefs, romfs, lift, compile). + whose `@@` progress lines drive the rows (check, exefs, romfs, lift, compile). How it keeps the build short: + - The RomFS image (26 GB, mostly disk work) is packed beside the lift and the compile (processor work), so a first + build takes about as long as the longer of the two. Only `romfs.c`, which includes the RomFS index's + `romfs_config.h`, waits for the image. + - Every logical processor compiles, the largest files first, so no big file starts last and runs on alone. + - The lifted code compiles with MSYS2 CLANG64's clang when the toolchain has it: the same LLVM release as + UCRT64's, built with profile-guided optimisation, about a fifth faster on this code and writing the same objects + (it reads UCRT64's headers; `make_game.py` `fast_clang` checks that both report the same version and target). + - The lifted code is compiled for the processor's own level, x86-64-v3 (AVX2, FMA) or v2 (`cpu_level`): the game + runs on the PC that builds it, and a generic build would do every fused multiply-add of the game in software. 3. **Prepare the Shaders**: make_game.py's last step translates every shader of the game into the game's shader cache and the driver's own cache (`tools/prepare_shaders.py`), so play does not stop to compile them. 4. The game's Cetra Display is decoded for the window. @@ -100,12 +115,19 @@ The stub swaps `python.next` in while no Python runs, then starts the new versio `jobs.ContinueUpdateJob`: 4. **Update Build Tools**: only when the new release pins other packages. -5. **Translate the Code** and **Compile Changed Files**: `make_game.py --rebuild`. The exefs and romfs steps depend - only on the game files, so they are kept and the dump is not needed again; the lift runs again and the compile - step compiles only the files whose source, included headers, flags or compiler changed (`build-cache/obj/ - objects.json`, from clang's dependency files). -6. **Swap In the New Build**: the link writes `ocarina.exe.new`, which replaces `ocarina.exe` only once it is - complete. A failed rebuild leaves the old build playable; Rebuild the Game tries again. +5. **Translate the Code** and **Compile Changed Files**: `make_game.py --rebuild`. Most updates compile almost + nothing: + - The exefs and romfs steps depend only on the game files: they are kept, and the dump is not needed again. + - The lift runs only when the lifter, its patch table or the bring-up fixes changed. + - The runtime comes prebuilt with the release (`app//lib/libnx2navirt.a`, compiled by + `make_game.py --make-runtime-lib` with the release's own compiler). Only the few runtime files that include a + header written per install (`layout.h`, `patches.h`, `romfs_config.h`) are compiled on the player's PC. + - Every other file compiles only when its source, a header it includes, the flags or the compiler changed + (`build-cache/obj/objects.json`, from clang's dependency files). The kit's folder is not part of that key, so the + objects of the version before are reused although each version lives in its own folder. The lifted code + includes only `cpu.h` and `simd.h`: a change there compiles it all again, which is rare. +6. **Swap In the New Build**: lld links `ocarina.exe.new`, which replaces `ocarina.exe` only once it is complete. A + failed rebuild leaves the old build playable; Rebuild the Game tries again. 7. **Prepare the Shaders** again for the new build; shaders the translator already prepared come from the cache. 8. Every version but the current and the previous one is deleted. @@ -114,6 +136,32 @@ When the new version fails to start (exit code 86 from `start.pyw`, or any error runs. Keep Build Cache off deletes the lifted C and the objects (about 5 GB) after each build but keeps `build-cache/state.json`, so updates still rebuild without the dump (compiling everything). +## Importing a save + +Import Saves (home, under Game Folder; `launcher/saves.py`) makes another save the game's own. The game folder has +a `saves` folder with a README for it: a save put there (its files, or the folder that holds them) is taken first, +and the row then reads "From the Saves Folder"; otherwise selecting the row asks for the folder. It recognises a +Switch save backup (the save's files), an emulator's save folder (nested folders named by user and title ids), and +another NX2navi game folder or its profile folder (the account folder inside). A folder with more than 4,000 files +or 1 GB is refused as picked by mistake. + +The game keeps its save in `profile/save/`, named after `profile/account.uid` (made as the game would when +it never ran). The import copies the save the game would load now (a finished commit still waiting in +`profile/save-tx`, else the account folder) to `saves/backup `, puts the new files in place and empties +`profile/save-tx`, so the game loads the imported save. A save taken from the saves folder moves to +`saves/imported `. It refuses while the game runs. + +## Downloads + +Every download (`launcher/net.py`) comes over HTTPS and is checked against a sha256. A server's certificate is checked +first by Python's OpenSSL against the certificates in Windows' stores and, when that refuses it, by Windows' own +certificate check (the truststore package of the bundled Python; not under Wine, whose crypt32 passes a chain that +stops at an issuer it does not know). OpenSSL sees only the roots the PC already holds, and it takes an expired +cross-signature still in a store before the server's valid one ("certificate has expired" on a valid chain); Windows +weighs every chain as its own programs do and downloads a missing root from Windows Update. A certificate both refuse +stops the step with what each check said, the PC's clock against the server's (the Date of a plain-HTTP request to +it), and what to check: the date, time and time zone, or security software that scans encrypted connections. + ## Uninstalling Settings > Uninstall NX2navi, or Settings > Apps > NX2navi (which runs `NX2navi.exe --uninstall`): the launcher @@ -130,7 +178,9 @@ Built Game" is On. The dump is never touched. ## Releases 1. Bump `VERSION`; build the shader translator (`tools/build_xlate.py`). -2. With Python 3.14, numpy (the icon is drawn with it, `launcher/ui/art.py`) and uv: +2. With Python 3.14, numpy (the icon is drawn with it, `launcher/ui/art.py`) and uv, on Windows (the prebuilt runtime + needs MSYS2's own clang; on Linux add `--wine` to run it through Wine, else the release ships without it and + players compile `runtime/` themselves): ```powershell uv run --no-project --python 3.14 --with numpy tools\make_launcher.py --xlate build\xlate --ffx amd_fidelityfx_vk.dll --feed owner/name --notes notes.md @@ -140,7 +190,8 @@ Built Game" is On. The dump is never touched. On Linux it cross-compiles the two programs with clang, lld and llvm-windres against the same MSYS2 packages and unpacks Tcl/Tk with msiextract (msitools). It writes into `build/launcher-release`: `NX2navi-Setup-.exe` - (the setup program with the launcher folder appended as a zip), `NX2navi-app-.zip`, + (the setup program with the launcher folder appended as a zip), `NX2navi-app-.zip` (with the prebuilt + runtime), `NX2navi-runtime-.zip` and `NX2navi-update.json`. `check_content` runs over the folder first. 3. Upload those four files to GitHub release `v` of the feed repository, by hand or with `--publish` (`gh release create`; add `--draft` to look before it goes out). Launchers see it at their next start. @@ -151,10 +202,15 @@ repository keeps old package files, so a release keeps building with the compile ## Tests `python tests/launcher_test.py` (Python 3.14): versions, settings, the toolchain resolver and installer against a -local fake repository, resumed and checked downloads, the GitHub and URL feeds, staging, switching, rollback and +local fake repository, resumed and checked downloads, the certificate checks against a local HTTPS server (its +certificates made with the `openssl` command), the GitHub and URL feeds, staging, switching, rollback and pruning, unsafe archives, the font decoding on a system font wrapped like the game's, `make_game.py` on a synthetic -`.nup` (events, skipped steps, `--rebuild` without the dump, step keys), the compile step's object cache with the -system's C compiler, the install job with a fake `make_game.py`, and the window's selftest when a display is there. +`.nup` (events, skipped steps, `--rebuild` without the dump, step keys), the RomFS step beside the lift and compile +and its failure stopping the build, the compile step's object cache with the system's C compiler and its +largest-first order, its windres call under folders with spaces in their names, CLANG64's clang in the lock and the +install, the save import (the account folder, the save folders it recognises, backup, the commit journal, the saves +folder, refusals), the install job with a fake +`make_game.py`, and the window's selftest when a display is there. ## Requirements and limits diff --git a/launcher/__init__.py b/launcher/__init__.py index 95636a8c..a0ba4159 100644 --- a/launcher/__init__.py +++ b/launcher/__init__.py @@ -1,8 +1,9 @@ """The NX2navi launcher: installs the build tools, builds the game from the player's own dump, starts it and keeps the port up to date (docs/launcher.md). -Standard library only, plus numpy for drawing the window (launcher/ui/art.py) and zstandard where Python lacks -compression.zstd. tkinter draws the window; everything that does work runs without it (launcher/cli.py). +Standard library only, plus numpy for drawing the window (launcher/ui/art.py), zstandard where Python lacks +compression.zstd, and truststore for Windows' own certificate check (launcher/net.py). tkinter draws the window; +everything that does work runs without it (launcher/cli.py). """ NAME = 'NX2navi' diff --git a/launcher/native/common.h b/launcher/native/common.h index 68e2300f..2faa3dd4 100644 --- a/launcher/native/common.h +++ b/launcher/native/common.h @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -15,6 +16,12 @@ #define PATHN 1024 #define APP_NAME L"NX2navi" #define UNINSTALL_KEY L"Software\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\NX2navi" +/* The launcher window's taskbar identity (launcher/winsys.py app_id): a shortcut that carries it is the one the + taskbar groups and pins the window under, with the shortcut's icon. */ +#define APP_USER_MODEL_ID L"NX2navi.Launcher" +static const PROPERTYKEY NX_PKEY_AppUserModel_ID = { + {0x9F4C2855, 0x9F79, 0x4B39, {0xA8, 0xD0, 0xE1, 0xD4, 0x2D, 0xE1, 0xD5, 0xF3}}, 5}; +static const IID NX_IID_IPropertyStore = {0x886d8eeb, 0x8cf2, 0x4446, {0x8d, 0x02, 0xcd, 0xba, 0x1d, 0xbd, 0xcf, 0x99}}; static inline void exe_dir(wchar_t *out, size_t n) { @@ -105,6 +112,16 @@ static inline int make_shortcut(const wchar_t *lnk, const wchar_t *target, const IShellLinkW_SetWorkingDirectory(sl, workdir); IShellLinkW_SetIconLocation(sl, target, 0); IShellLinkW_SetDescription(sl, desc); + IPropertyStore *ps = NULL; + if (SUCCEEDED(IShellLinkW_QueryInterface(sl, &NX_IID_IPropertyStore, (void **)&ps))) { + PROPVARIANT pv; + memset(&pv, 0, sizeof pv); + pv.vt = VT_LPWSTR; + pv.pwszVal = (LPWSTR)APP_USER_MODEL_ID; + IPropertyStore_SetValue(ps, &NX_PKEY_AppUserModel_ID, &pv); + IPropertyStore_Commit(ps); + IPropertyStore_Release(ps); + } if (SUCCEEDED(IShellLinkW_QueryInterface(sl, &IID_IPersistFile, (void **)&pf))) { ok = SUCCEEDED(IPersistFile_Save(pf, lnk, TRUE)); IPersistFile_Release(pf); diff --git a/launcher/native/res.rc b/launcher/native/res.rc index 585e16c2..8c9f5c7c 100644 --- a/launcher/native/res.rc +++ b/launcher/native/res.rc @@ -1,4 +1,4 @@ -/* The icon (the Triforce, written by tools/make_launcher.py from launcher/ui/art.py triforce_ico into the build - folder, found through -I) and the manifest of NX2navi.exe and NX2navi-Setup.exe. */ +/* The icon (Navi, written by tools/make_launcher.py from launcher/ui/art.py navi_ico into the build folder, + found through -I) and the manifest of NX2navi.exe and NX2navi-Setup.exe. */ 1 ICON "nx2navi.ico" 1 24 "app.manifest" diff --git a/launcher/native/setup.c b/launcher/native/setup.c index 43299e55..30151617 100644 --- a/launcher/native/setup.c +++ b/launcher/native/setup.c @@ -32,6 +32,16 @@ typedef struct { static Setup S; static HWND g_wnd; +/* Navi as launcher/ui/art.py NAVI draws it: W white, g grey, K black, B and L blue, '.' transparent. */ +static const char *const NAVI[21] = { + "WWWW..............WWWW", "WggWW............WWggW", "WgggWW..........WWgggW", "WgWggWW........WWggWgW", + "WggWggW........WggWggW", "WgWgWgWW......WWgWgWgW", "WggggWgWKKKKKKWgWggggW", "WWggWggKBBBBBBKggWggWW", + ".WggggKKBBLLBBKKggggW.", ".WWggKKBBLLLLBBKKggWW.", "..WWgKBBLLWWLLBBKgWW..", "...WWKBBLWWWWLBBKWW...", + ".....KBBLLWWLLBBK.....", ".....KBBBLLLLBBBK.....", ".....WKKBBBBBBKKW.....", "....WWgKKBBBBKKgWW....", + "...WWgggKKKKKKgggWW...", "...WggWgW....WgWggW...", "...WgWggW....WggWgW...", "...WWggWW....WWggWW...", + "....WWWW......WWWW....", +}; + static void fail_msg(const wchar_t *fmt, const wchar_t *arg) { swprintf(S.error, 512, fmt, arg ? arg : L""); @@ -258,35 +268,37 @@ static LRESULT CALLBACK wndproc(HWND w, UINT m, WPARAM wp, LPARAM lp) FillRect(dc, &r, edge); RECT in = {r.left + 2, r.top + 2, r.right - 2, r.bottom - 2}; FillRect(dc, &in, bg); - /* the Triforce: three gold triangles */ - HBRUSH gold = CreateSolidBrush(RGB(0xe8, 0xaf, 0x3d)); - HPEN none = CreatePen(PS_NULL, 0, 0); - HGDIOBJ ob = SelectObject(dc, gold), op = SelectObject(dc, none); - int cx = 52, top = 28, s = 44; - POINT t1[3] = {{cx, top}, {cx + s / 4, top + s * 433 / 1000}, {cx - s / 4, top + s * 433 / 1000}}; - POINT t2[3] = {{cx - s / 4, top + s * 433 / 1000 + 1}, {cx, top + s * 866 / 1000}, {cx - s / 2, top + s * 866 / 1000}}; - POINT t3[3] = {{cx + s / 4, top + s * 433 / 1000 + 1}, {cx + s / 2, top + s * 866 / 1000}, {cx, top + s * 866 / 1000}}; - Polygon(dc, t1, 3); - Polygon(dc, t2, 3); - Polygon(dc, t3, 3); - SelectObject(dc, ob); - SelectObject(dc, op); + /* Navi, the launcher's emblem: the artwork's 22 x 21 pixels, 4 x 4 screen pixels each */ + for (int y = 0; y < 21; y++) + for (int x = 0; x < 22; x++) { + COLORREF col; + switch (NAVI[y][x]) { + case 'W': col = RGB(255, 255, 255); break; + case 'g': col = RGB(207, 216, 220); break; + case 'K': col = RGB(0, 0, 0); break; + case 'B': col = RGB(2, 136, 209); break; + case 'L': col = RGB(79, 195, 247); break; + default: continue; + } + HBRUSH b = CreateSolidBrush(col); + RECT px = {16 + 4 * x, 18 + 4 * y, 20 + 4 * x, 22 + 4 * y}; + FillRect(dc, &px, b); + DeleteObject(b); + } HFONT big = CreateFontW(-26, 0, 0, 0, FW_SEMIBOLD, 0, 0, 0, DEFAULT_CHARSET, 0, 0, CLEARTYPE_QUALITY, 0, L"Segoe UI"); HFONT small = CreateFontW(-16, 0, 0, 0, FW_NORMAL, 0, 0, 0, DEFAULT_CHARSET, 0, 0, CLEARTYPE_QUALITY, 0, L"Segoe UI"); SetBkMode(dc, TRANSPARENT); SetTextColor(dc, RGB(0xf3, 0xec, 0xdc)); HGDIOBJ of = SelectObject(dc, big); - TextOutW(dc, 96, 30, APP_NAME, (int)wcslen(APP_NAME)); + TextOutW(dc, 120, 30, APP_NAME, (int)wcslen(APP_NAME)); SelectObject(dc, small); SetTextColor(dc, RGB(0xe9, 0xdc, 0xc0)); wchar_t line[96]; swprintf(line, 96, L"Installing version %ls\x2026 %llu MB", S.version, (unsigned long long)(S.written >> 20)); - TextOutW(dc, 96, 66, line, (int)wcslen(line)); + TextOutW(dc, 120, 66, line, (int)wcslen(line)); SelectObject(dc, of); DeleteObject(big); DeleteObject(small); - DeleteObject(gold); - DeleteObject(none); DeleteObject(bg); DeleteObject(edge); EndPaint(w, &ps); diff --git a/launcher/net.py b/launcher/net.py index 15b7a101..c758fbeb 100644 --- a/launcher/net.py +++ b/launcher/net.py @@ -1,17 +1,37 @@ """HTTPS for the launcher: JSON requests and file downloads that resume, report progress, can be cancelled and are -checked against a sha256. Standard library only (urllib); Windows' own certificate store verifies the servers.""" +checked against a sha256. The standard library (urllib), and on Windows the truststore package of the launcher's +Python for Windows' own certificate check. + +Two checks can accept a server's certificate, tried in turn by request(): + +- Python's: OpenSSL with the certificates of Windows' ROOT and CA stores (ssl.create_default_context). It knows only + the certificates already in those stores, while Windows holds few roots and downloads the others the first time a + Windows program needs them. And it takes a stored certificate before the server's copy of the same name and key: a + store that still holds an old, expired cross-signature of the root a server's chain leads to makes it report + "certificate has expired" for a valid chain (repo.msys2.org's chain on a PC whose CA store still holds the + "ISRG Root X2" cross-signature that expired in September 2025). +- Windows': Windows' own chain engine, which Windows programs check servers with (CertGetCertificateChain and + CertVerifyCertificateChainPolicy, through truststore.SSLContext). It weighs every path the stores and the server + allow, prefers a valid one, and downloads a missing root from Windows Update. + +A certificate both refuse ends the request with CertificateError: what each check said, whether the PC's clock agrees +with the server's, and what to do.""" +import email.utils import hashlib import json import os import ssl import time import urllib.error +import urllib.parse import urllib.request +from datetime import timezone from pathlib import Path from . import NAME CHUNK = 1 << 20 +CLOCK_TOLERANCE = 3600 # seconds between this PC's clock and a server's beyond which this PC's is wrong class Cancelled(Exception): @@ -22,26 +42,127 @@ class DownloadError(Exception): pass +class CertificateError(DownloadError): + """No certificate check accepts a server's certificate. Asking again gets the same answer, so download() does + not retry it.""" + + def user_agent(version='dev'): return f'{NAME}-Launcher/{version}' -_CTX = None +def _under_wine(): + try: + import ctypes + return hasattr(ctypes.WinDLL('ntdll'), 'wine_get_version') + except Exception: + return False -def _context(): - global _CTX - if _CTX is None: - _CTX = ssl.create_default_context() - return _CTX +def _windows_check(): + """An SSLContext that leaves the certificate to Windows' own verifier, or None off Windows, in a Python without + truststore (a checkout's own Python) and under Wine. Wine's crypt32 accepts a chain that stops at an issuer it + does not know (CERT_TRUST_IS_PARTIAL_CHAIN passes its SSL policy), and its roots are the host's, which Python's + check reads already.""" + if os.name != 'nt' or _under_wine(): + return None + try: + import truststore + return truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + except Exception: # not installed, or crypt32 not reachable through ctypes + return None + + +_CHECKS = None # [(name, SSLContext)] in the order request() tries them +_ACCEPTED = {} # host -> index into _CHECKS of the check that accepted its certificate last + + +def _checks(): + global _CHECKS + if _CHECKS is None: + checks = [("Python's check", ssl.create_default_context())] + windows = _windows_check() + if windows is not None: + checks.append(("Windows' check", windows)) + _CHECKS = checks + return _CHECKS def request(url, headers=None, timeout=30, version='dev'): + """The open response to a GET of url. An https URL is tried with each certificate check in turn, first the one + that accepted the host last, until one accepts the server; CertificateError when none does.""" h = {'User-Agent': user_agent(version)} h.update(headers or {}) - req = urllib.request.Request(url, headers=h) - ctx = _context() if url.startswith('https:') else None - return urllib.request.urlopen(req, timeout=timeout, context=ctx) + if not url.startswith('https:'): + return urllib.request.urlopen(urllib.request.Request(url, headers=h), timeout=timeout) + host = urllib.parse.urlsplit(url).hostname or '' + checks = _checks() + first = min(_ACCEPTED.get(host, 0), len(checks) - 1) + refused = [] + for i in [first] + [i for i in range(len(checks)) if i != first]: + name, ctx = checks[i] + try: + r = urllib.request.urlopen(urllib.request.Request(url, headers=h), timeout=timeout, context=ctx) + except urllib.error.URLError as e: + if not isinstance(e.reason, ssl.SSLCertVerificationError): + raise + refused.append((name, e.reason)) + continue + _ACCEPTED[host] = i + return r + raise CertificateError(certificate_problem(url, refused, version)) from refused[0][1] + + +def server_clock(url, timeout=10, version='dev'): + """The time (epoch seconds) of the Date header a plain-HTTP HEAD request to url answers with, without following a + redirect, or None without an answer. A PC whose clock is wrong cannot check any certificate, so its clock is + compared over HTTP; the answer is not authenticated and only chooses the advice of an error message.""" + class Stay(urllib.request.HTTPRedirectHandler): + def redirect_request(self, *args, **kwargs): + return None + + req = urllib.request.Request(url, method='HEAD', headers={'User-Agent': user_agent(version)}) + try: + with urllib.request.build_opener(Stay).open(req, timeout=timeout) as r: + date = r.headers.get('Date') + except urllib.error.HTTPError as e: # the redirect to https, or any other status: its Date is as good + date = e.headers.get('Date') + e.close() + except Exception: # no answer, whatever the reason (refused, timed out, not HTTP) + return None + if not date: + return None + try: + when = email.utils.parsedate_to_datetime(date) + except (TypeError, ValueError): + return None + return (when if when.tzinfo else when.replace(tzinfo=timezone.utc)).timestamp() + + +def _said(err): + said = getattr(err, 'verify_message', None) or (str(err.args[-1]) if err.args else type(err).__name__) + return said.strip().rstrip('.') + + +def _local(t): + return time.strftime('%Y-%m-%d %H:%M', time.localtime(t)) + + +def certificate_problem(url, refused, version='dev'): + """The message of a CertificateError for url: what each check in refused ([(name, error)]) said, then the advice + that fits this PC's clock.""" + host = urllib.parse.urlsplit(url).hostname or url + said = '; '.join(f'{name}: {_said(e)}' for name, e in refused) + text = f'{url}: the certificate of {host} was not accepted ({said}).' + where = 'Windows Settings > Time & language > Date & time' if os.name == 'nt' else "the system's settings" + ours, theirs = time.time(), server_clock(f'http://{host}/', version=version) + if theirs is not None and abs(ours - theirs) > CLOCK_TOLERANCE: + return (f"{text} This PC's clock reads {_local(ours)}, but {host} says it is {_local(theirs)}: set the date, " + f'time and time zone in {where}, then try again.') + clock = (f"This PC's clock agrees with {host}'s." if theirs is not None else + f"Check that this PC's date, time and time zone are right ({where}).") + return (f'{text} {clock} Security software that scans encrypted connections (HTTPS or SSL scanning) or a proxy ' + 'may be replacing the certificate: let NX2navi through it or turn that scanning off, then try again.') def get_bytes(url, headers=None, timeout=30, version='dev', limit=64 << 20): @@ -100,7 +221,7 @@ def download(url, dest, sha256=None, size=None, progress=None, cancel=None, head if progress: progress(done, total or done) break - except Cancelled: + except (Cancelled, CertificateError): raise except urllib.error.HTTPError as e: if e.code == 416 and have: # the part file is complete or longer than the file: start over diff --git a/launcher/saves.py b/launcher/saves.py new file mode 100644 index 00000000..fdf032ed --- /dev/null +++ b/launcher/saves.py @@ -0,0 +1,221 @@ +"""Importing a save into the built game, and the saves folder of the game folder. + +The game keeps its save in /profile/save// (runtime/fs_host.c storage_init). The folder is named after +the local account in /profile/account.uid: 16 bytes, two little-endian 64-bit halves, the low one first; the +name is the high half then the low half, 16 hex digits each (runtime/ov_system.c load_uid). A game that never ran has +no account yet; the import makes one as the game would. Saves are committed through /profile/save-tx/, where +a finished commit (ready-) is newer than save/: an import clears that folder, so the game loads the +imported save. + +/saves is the player's side. Import Saves takes a save put there first (its files, or the folder that holds +them); otherwise the player picks a folder. Before the save is replaced it is copied to saves/backup