commit 34b36834417a30fc884eab9c1a18ebaee13ff419 Author: ruckus7382sfgsgdf <355121112+ruckus7382sfgsgdf@users.noreply.github.com> Date: Tue Nov 10 19:32:48 2026 -0600 Triforce v0.2.0 Co-Authored-By: Claude Opus 5.5 diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..cc23192 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,2 @@ +* text=auto eol=lf +*.bat text eol=crlf diff --git a/.github/ISSUE_TEMPLATE/1-game-problem.yml b/.github/ISSUE_TEMPLATE/1-game-problem.yml new file mode 100644 index 0000000..c9da25c --- /dev/null +++ b/.github/ISSUE_TEMPLATE/1-game-problem.yml @@ -0,0 +1,111 @@ +name: Game problem +description: Graphics, crashes, black screens, stutter, sound, controls or saves +labels: [bug] +body: + - type: markdown + attributes: + value: | + Thanks for testing. Before you start: + - **Check the [newest release](https://github.com/NotRexed/Hookshot/releases/latest) first.** Every release includes all the fixes before it, and yours may already be fixed. + - **Search the open issues.** If yours is there, add your details to it instead. + - English, please. + - **Never attach game files**: no hookshot.exe, image.bin, the `.bin` caches, romfs or exefs files, keys or saves. + - type: input + id: version + attributes: + label: Hookshot version + description: The version you built (such as `v2.0.0`, shown by the launcher and on the Releases page), or the first line of `hookshot_log.txt` if it starts with `hookshot`. + placeholder: v2.0.0 + validations: + required: true + - type: dropdown + id: kind + attributes: + label: What kind of problem + options: + - Graphics (missing, black or wrong-coloured things, lighting, flicker) + - Crash, freeze or black screen + - Low fps or stutter + - Sound, controls or saves + - Something else + validations: + required: true + - type: textarea + id: what + attributes: + label: What happened + description: >- + What you saw, and what you expected instead. For low FPS or stutter, note the FPS before and after + and whether it happens on a second pass through the same spot. See [Low FPS or stutter](https://github.com/NotRexed/Hookshot#low-fps-or-stutter) + for what to compare and which logs to attach. + validations: + required: true + - type: input + id: where + attributes: + label: Where in the game + description: The area, cutscene or menu, and what you were doing. + placeholder: Kokiri Forest, walking out of Link's house + validations: + required: true + - type: dropdown + id: often + attributes: + label: How often + options: + - Every time + - Sometimes + - Once so far + validations: + required: true + - type: input + id: gpu + attributes: + label: Graphics card and driver version + placeholder: RTX 3080 Ti, NVIDIA 596.21 + validations: + required: true + - type: input + id: system + attributes: + label: CPU, RAM and operating system + placeholder: Ryzen 7 7700, 32 GB, Windows 11 + validations: + required: true + - type: textarea + id: settings + attributes: + label: Changes from the defaults + description: Insert-menu options you changed, and the lines in `hookshot_env.txt` if you have one. Leave empty if none. + - type: dropdown + id: injectors + attributes: + label: Other software hooked into the game + description: ReShade, shader-cache tools (SCSKiller and the like), overlays or frame-generation injectors change what Hookshot does. Please test without them before reporting. + options: + - No, nothing hooked in + - Yes (named under "Changes from the defaults") + validations: + required: true + - type: textarea + id: log + attributes: + label: hookshot_log.txt + description: Drag the file here. It is beside hookshot.exe and holds the last run; the four runs before are `hookshot_log.1.txt` (the one before) to `.4.txt`. If you started the game again after the problem, attach `hookshot_log.1.txt` too. It has no game files in it. + validations: + required: true + - type: textarea + id: media + attributes: + label: Screenshots or video + - type: checkboxes + id: checks + attributes: + label: Before you send it + options: + - label: I'm on the newest release, or the problem is still there on it. + required: true + - label: I searched the open issues for this problem. + required: true + - label: I'm not attaching game files, keys, hookshot.exe, the .bin caches or saves. + required: true diff --git a/.github/ISSUE_TEMPLATE/2-build-problem.yml b/.github/ISSUE_TEMPLATE/2-build-problem.yml new file mode 100644 index 0000000..bba13cb --- /dev/null +++ b/.github/ISSUE_TEMPLATE/2-build-problem.yml @@ -0,0 +1,67 @@ +name: Build problem +description: Hookshot Builder or make_game.py stops or gives an error +labels: [bug] +body: + - type: markdown + attributes: + value: | + Before you start: check the [newest release](https://github.com/NotRexed/Hookshot/releases/latest) and the [build requirements](https://github.com/NotRexed/Hookshot#what-you-need) (Visual Studio Build Tools with "Desktop development with C++" and the Windows SDK, about 60 GB free), and search the open issues. English, please. Never attach game files. + - type: input + id: version + attributes: + label: Builder version + description: The version you downloaded (such as `v2.0.0`, on the Releases page and in the builder). + placeholder: v2.0.0 + validations: + required: true + - type: dropdown + id: how + attributes: + label: How you built + options: + - Hookshot Builder app (Windows) + - Command line (nx2/make_game.py) + - Linux AppImage + validations: + required: true + - type: dropdown + id: input + attributes: + label: Game files given to it + options: + - A .nup + - romfs and exefs folders + validations: + required: true + - type: textarea + id: error + attributes: + label: The error + description: Copy the text the builder shows, with the failed step's details (text, not a screenshot of it). + render: text + validations: + required: true + - type: input + id: toolchain + attributes: + label: Visual Studio or Build Tools version, and Windows SDK version + description: Windows only. Visual Studio Installer shows both. + placeholder: Build Tools 2022 17.14, Windows SDK 10.0.26100 + - type: input + id: system + attributes: + label: Operating system and free space on the output drive + placeholder: "Windows 11 24H2, 210 GB free on D:" + validations: + required: true + - type: checkboxes + id: checks + attributes: + label: Before you send it + options: + - label: I'm on the newest release, or it still fails on it. + required: true + - label: I searched the open issues for this error. + required: true + - label: I'm not attaching game files or anything built from them. + required: true diff --git a/.github/ISSUE_TEMPLATE/3-idea.yml b/.github/ISSUE_TEMPLATE/3-idea.yml new file mode 100644 index 0000000..7e560fc --- /dev/null +++ b/.github/ISSUE_TEMPLATE/3-idea.yml @@ -0,0 +1,19 @@ +name: Idea or feature request +description: Something Hookshot doesn't do yet +labels: [enhancement] +body: + - type: markdown + attributes: + value: Search the open issues first; if someone asked already, add to theirs. English, please. + - type: textarea + id: idea + attributes: + label: The idea + description: What you want, and why it would help. + validations: + required: true + - type: textarea + id: more + attributes: + label: Anything else + description: How another port or emulator does it, links, pictures. diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000..41181df --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,8 @@ +blank_issues_enabled: false +contact_links: + - name: Newest release + url: https://github.com/NotRexed/Hookshot/releases/latest + about: Every release includes all the fixes before it. Try the newest before reporting. + - name: Issue guidelines + url: https://github.com/NotRexed/Hookshot/issues/159 + about: What to check before opening an issue. diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 0000000..90bb41c --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,14 @@ +# The actions the workflows use are pinned by commit (the version in a comment beside each): Dependabot proposes the +# newer ones once a week, pin and comment together. A release is proposed only once it is a week old (cooldown): a +# compromised action version is usually found and pulled within days, before it reaches these workflows. +version: 2 +updates: + - package-ecosystem: github-actions + directory: / + schedule: + interval: weekly + cooldown: + default-days: 7 + groups: + actions: + patterns: ['*'] diff --git a/.github/workflows/linux-appimage.yml b/.github/workflows/linux-appimage.yml new file mode 100644 index 0000000..335e09a --- /dev/null +++ b/.github/workflows/linux-appimage.yml @@ -0,0 +1,166 @@ +# The Linux AppImage (x86_64): Hookshot Builder with the compiler, Python, libraries and the Vulkan back end it builds the +# game with on the player's PC (scripts/linux/deps.sh, scripts/linux/appimage.sh). No game data is involved. +# Built in Ubuntu 22.04, the oldest system it is for (glibc 2.35). A workflow artifact on every run; on a tag, also an +# asset of that tag's release (scripts/release_upload.sh, shared with the Windows builder workflow). +name: Linux AppImage +on: + push: + branches: [main] + tags: ['*'] + pull_request: + workflow_dispatch: +permissions: + contents: read +concurrency: + group: linux-appimage-${{ github.ref }} + # (a pull request's older run is cancelled; a push's or a tag's never is) + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +jobs: + appimage: + # (builder-parts holds the installer's parts: no AppImage there, and it must never be made "latest") + if: github.ref_name != 'builder-parts' && github.ref_name != 'linux-parts' + runs-on: ubuntu-latest + container: ubuntu:22.04 + # (about 4 minutes with the dependencies cached; building them all, as a release does, takes longer) + timeout-minutes: 60 + steps: + - name: System packages + env: + DEBIAN_FRONTEND: noninteractive + run: | + # Retry the entire bootstrap: even the first index refresh or the toolchain setup can fail while a mirror syncs. + # Every attempt refreshes the signed indexes; no stale-index or authentication checks are bypassed. + for try in 1 2 3; do + # (libstdc++ 13, from the toolchain PPA: C++20 ; linked in statically) + if apt-get update && + apt-get install -y --no-install-recommends software-properties-common gpg-agent && + add-apt-repository -y ppa:ubuntu-toolchain-r/test && + apt-get install -y --no-install-recommends g++-13 libstdc++-13-dev && + apt-get install -y --no-install-recommends git ca-certificates curl xz-utils zstd file unzip python3 \ + build-essential gcc-12 g++-12 libstdc++-12-dev nasm meson ninja-build cmake pkg-config \ + libx11-dev libxext-dev libxrandr-dev libxcursor-dev libxi-dev libxss-dev libxfixes-dev libxinerama-dev \ + libxkbcommon-dev libwayland-dev wayland-protocols libdecor-0-dev libegl-dev libgles-dev libdrm-dev libgbm-dev \ + libpulse-dev libasound2-dev libpipewire-0.3-dev libudev-dev libdbus-1-dev libxft-dev libfontconfig1-dev zlib1g-dev; then + break + else + package_status=$? + fi + if [ "$try" = 3 ]; then exit "$package_status"; fi + sleep 20 + done + + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + persist-credentials: false + + - name: Trust the checkout + run: git config --global --add safe.directory "$GITHUB_WORKSPACE" + + # (a release builds what the AppImage carries from its checked downloads: no built deps from a cache, which other + # runs write. Other runs take the built deps) + - if: ${{ !startsWith(github.ref, 'refs/tags/') }} + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: build-linux/deps + key: linux-deps-${{ hashFiles('scripts/linux/deps.sh', 'requirements.txt', 'nx2/prompt_requirements.txt') }} + - if: ${{ startsWith(github.ref, 'refs/tags/') }} + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: build-linux/deps/downloads + key: linux-deps-downloads-${{ hashFiles('scripts/linux/deps.sh') }} + + - name: Dependencies (LLVM, Python, FFmpeg with dav1d, SDL2, Vulkan headers) + env: + GH_TOKEN: ${{ github.token }} + run: scripts/linux/deps.sh + + # work/: the lists and the play profile made on the development PC (addresses and counts, no game code). When the + # repository does not carry them, the ones in the latest release's app part (named by its hookshot-setup.txt) are + # used; without either the game is built all the same, a little slower in play. + # (not this tag's own release: the Windows builder workflow can make it before this step runs. A fork with no release + # takes upstream's. Downloads are retried: GitHub answers 504 now and then) + - name: Play profile + env: + GH_TOKEN: ${{ github.token }} + SKIP_TAG: ${{ startsWith(github.ref, 'refs/tags/') && github.ref_name || '' }} + run: | + if [ -f work/hot_order.txt ]; then echo "work/ is in the repository"; exit 0; fi + python3 -I scripts/release_parts.py previous /tmp/setup.txt --repo "$GITHUB_REPOSITORY" --repo NotRexed/Hookshot \ + --skip-tag "$SKIP_TAG" || + { echo "no release with a setup list: built without the profile"; exit 0; } + python3 -I scripts/release_parts.py profile /tmp/setup.txt work || + { echo "the profile did not download: built without it"; exit 0; } + ls -la work work/image + + - name: AppImage + run: scripts/linux/appimage.sh + + # (AMD FSR 4's Linux side, a build option the AppImage leaves off (gpu/vk/vk_fsr4.cpp: dlopen, the older backend + # description, the unit exposure; the descriptor buffers in vk_main.cpp): built once more in a copy of the back end's + # work folder, where only vk_fsr4.cpp compiles again. The AppImage above is not changed) + - name: hookshot_vk.so with AMD FSR 4 (build check) + shell: bash # (this container's default is sh, which has no PIPESTATUS) + run: | + cp -r build-linux/vk-work "$RUNNER_TEMP/vk-fsr" + rm -f "$RUNNER_TEMP/vk-fsr/sa/build/hookshot_vk.so" # (the AppImage's own, without FSR: the test below is this build's) + D="$PWD/build-linux/deps" + OOT_WORK="$RUNNER_TEMP/vk-fsr" EMSDK="$D/emsdk" RX_CPM="$D/cpm" RX_MARCH=x86-64-v3 RX_VK_ONLY=1 RX_RELEASE=1 HOOKSHOT_FSR=1 \ + "$D/python/bin/python3" -I nx2/build.py --standalone --jobs "$(nproc)" | + tee "$RUNNER_TEMP/vk-fsr.log" + test "${PIPESTATUS[0]}" -eq 0 + grep -q "with AMD FSR 4" "$RUNNER_TEMP/vk-fsr.log" + test -f "$RUNNER_TEMP/vk-fsr/sa/build/hookshot_vk.so" + + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: Hookshot-linux-x86_64 + path: build-linux/*.AppImage + if-no-files-found: error + + # (a player's build started from the AppImage itself, with no game, up to where the lifted game is needed: a file the + # AppImage does not carry fails here, not in every player's build. 2026-11-10: third_party/fonts, #1567. After the upload, + # so a failing image can be looked at; a tag's release needs this job to pass) + - name: A player's build from the AppImage + run: scripts/linux/check_player_build.sh + + # (a tag's update parts, for the release job: scripts/linux/make_parts.py. Not on other runs: about 240 MB) + - if: startsWith(github.ref, 'refs/tags/') + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: Hookshot-linux-parts + path: build-linux/parts/ + if-no-files-found: error + + release: + name: Release asset + if: startsWith(github.ref, 'refs/tags/') && github.ref_name != 'builder-parts' && github.ref_name != 'linux-parts' + needs: appimage + runs-on: ubuntu-24.04 + timeout-minutes: 15 + permissions: + contents: write + env: + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + steps: + - name: Check out source + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: Hookshot-linux-x86_64 + path: dist + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: Hookshot-linux-parts + path: dist/parts + # (the parts to the linux-parts release, which is never "latest"; a part already there is not uploaded again) + - name: Update parts + run: scripts/linux/publish_parts.sh dist/parts + # (the release made here or by the Windows builder workflow, whichever is first; a -rc tag is a prerelease and + # never "latest") + - name: AppImage to the release + run: scripts/release_upload.sh "$GITHUB_REF_NAME" dist/*.AppImage dist/parts/hookshot-linux.txt diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..f129b61 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,63 @@ +# The version, tag and changelog entry for a push to main (stable) or next (release candidates). The tag starts the +# Linux AppImage and Windows builder workflows, which publish the GitHub release with their files. The rules and the one-time v0.1.0 tag are in +# docs/versioning.md. No game data is involved. +name: Release + +on: + push: + branches: [main, next] + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: release-${{ github.ref }} + cancel-in-progress: false + +jobs: + release: + # (a manual run on a branch, not on a tag: semantic-release needs a release branch) + if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/next' + runs-on: ubuntu-latest + timeout-minutes: 15 + # (the release commit and the tag are pushed back) + permissions: + contents: write + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + # the last release tag and the commits since it are part of the analysis + fetch-depth: 0 + # the release commit and the tag are pushed with this token; RELEASE_TOKEN lets the tag start the AppImage + # and Windows builder workflows, which publish the GitHub release + token: ${{ secrets.RELEASE_TOKEN || secrets.GITHUB_TOKEN }} + + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: 24 + + - name: Release tooling + run: npm install --no-audit --no-fund --ignore-scripts + + # semantic-release starts at 1.0.0 when it finds no tag. The first version is 0.1.0, tagged by hand on the merge + # that introduces the scheme, so nothing is released before that tag exists. + - name: First release tag + id: first + run: | + if git describe --tags --match 'v[0-9]*' >/dev/null 2>&1; then + echo "released=true" >> "$GITHUB_OUTPUT" + else + echo "released=false" >> "$GITHUB_OUTPUT" + echo "::notice::No version tag yet. Tag the first release by hand (docs/versioning.md)." + fi + + # RELEASE_TOKEN is a fine-grained personal access token with contents: write. It is preferred over GITHUB_TOKEN + # because a tag pushed by GITHUB_TOKEN does not start the workflows that publish the release. + # Without the secret, run them by hand and attach their artifacts. + - name: semantic-release + if: steps.first.outputs.released == 'true' + env: + GITHUB_TOKEN: ${{ secrets.RELEASE_TOKEN || secrets.GITHUB_TOKEN }} + run: npx semantic-release diff --git a/.github/workflows/source-regressions.yml b/.github/workflows/source-regressions.yml new file mode 100644 index 0000000..bb730b4 --- /dev/null +++ b/.github/workflows/source-regressions.yml @@ -0,0 +1,232 @@ +# Every game-free check, on every pull request to main and every push to it: tests/run_all.py runs each +# tests/test_*.py and nx2/test_*.py (all of them, even after a failure), the C++ unit tests under gpu/vk/tests and a +# byte-compile of every tracked Python file, and writes a table of them to the run's summary. A check that needs a GPU, +# the game or MSVC says so, and why, in its own file (RUN_ALL_SKIP). No game data is involved. +name: Game-free source regressions + +on: + push: + branches: [main] + pull_request: + branches: [main] + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +jobs: + python: + name: Python source checks (${{ matrix.os }}) + strategy: + fail-fast: false + matrix: + os: [ubuntu-24.04, windows-2022] + runs-on: ${{ matrix.os }} + timeout-minutes: 20 + steps: + - name: Check out source + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + # (the versions scripts/linux/deps.sh builds the Vulkan back end with) + - name: Check out pinned SPIR-V headers + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: KhronosGroup/SPIRV-Headers + ref: 04f10f650d514df88b76d25e83db360142c7b174 # vulkan-sdk-1.4.341.0 + path: _deps/spirv-headers + persist-credentials: false + - name: Check out pinned Vulkan headers + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: KhronosGroup/Vulkan-Headers + ref: 74d8a6cb930c68ef617b202c3ff3c59d919e086b # v1.4.345 + path: _deps/vulkan-headers + persist-credentials: false + - name: Set up Python + uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0 + with: + python-version: '3.12' + # (the one package a test imports: nx2/test_nup_codec.py's zstandard. The runner itself needs none) + - name: Python packages for the tests + run: python -m pip install --disable-pip-version-check --no-deps zstandard==0.25.0 + - name: Compilers and the SPIR-V validator + if: runner.os == 'Linux' + run: sudo apt-get update && sudo apt-get install -y g++ clang spirv-tools + - name: Pinned glslang for generated shader comparisons + if: runner.os == 'Linux' + shell: bash + run: | + # The checked-in shader headers are generated with glslang16.6.0. + # Older distro compilers can lower discard differently even for unchanged GLSL. + tool_dir="$RUNNER_TEMP/glslang-16.6.0" + mkdir -p "$tool_dir" + curl -fL --retry 3 -o "$tool_dir/glslang.tar.gz" https://github.com/KhronosGroup/glslang/releases/download/16.6.0/glslang-16.6.0-linux-x86_64-release.tar.gz + echo '2c34071f56ecf39d16233294d3739cf43f981fd39080f2fd9dccb782101191ce '"$tool_dir/glslang.tar.gz" | sha256sum --check --strict + tar -xzf "$tool_dir/glslang.tar.gz" -C "$tool_dir" + "$tool_dir/bin/glslangValidator" --version + echo "$tool_dir/bin" >> "$GITHUB_PATH" + + # (the steps below all run when one before them failed, so one run shows every failure) + - name: Check Windows native source regressions + if: ${{ !cancelled() && runner.os == 'Windows' }} + shell: cmd + run: | + @echo off + set "VSWHERE=%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" + if not exist "%VSWHERE%" ( + echo Visual Studio locator not found. + exit /b 1 + ) + set "VSROOT=" + for /f "usebackq delims=" %%i in (`call "%VSWHERE%" -latest -products * -version "[17.0,18.0)" -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath`) do set "VSROOT=%%i" + if not defined VSROOT ( + echo Visual Studio 2022 C++ tools not found. + exit /b 1 + ) + if not exist "%VSROOT%\Common7\Tools\VsDevCmd.bat" ( + echo Visual Studio developer command prompt not found. + exit /b 1 + ) + call "%VSROOT%\Common7\Tools\VsDevCmd.bat" -arch=x64 -host_arch=x64 + if errorlevel 1 exit /b 1 + where cl + if errorlevel 1 exit /b 1 + cl /nologo /std:c++17 /W4 /WX /EHsc /Fo"%TEMP%\present_rect_test.obj" /Fe:"%TEMP%\present_rect_test.exe" gpu\vk\tests\present_rect_test.cpp + if errorlevel 1 exit /b 1 + "%TEMP%\present_rect_test.exe" + if errorlevel 1 exit /b 1 + python -I -B tests/test_invalidate_vcore.py --cxx cl + if errorlevel 1 exit /b 1 + - name: Set up .NET for Windows Builder checks + if: ${{ !cancelled() && runner.os == 'Windows' }} + uses: actions/setup-dotnet@0f4a354dd5547a28c70b7853dbd95d5b67886e73 # v6.1.0 + with: + dotnet-version: '8.0.x' + - name: Check Windows Builder worker delivery + if: ${{ !cancelled() && runner.os == 'Windows' }} + shell: pwsh + run: | + dotnet restore tests/builder_workers/BuilderWorkersTests.csproj --configfile tests/builder_workers/NuGet.Config + dotnet run --project tests/builder_workers/BuilderWorkersTests.csproj --configuration Release --no-restore -- (Get-Command python).Source + - name: Compile Windows Builder + if: ${{ !cancelled() && runner.os == 'Windows' }} + run: dotnet build builder/HookshotBuilder/HookshotBuilder.csproj --configuration Debug -p:NuGetAudit=false + # (--strict: a check whose compiler, headers or package is missing here fails instead of being skipped) + - name: Every test + if: ${{ !cancelled() }} + env: + HOOKSHOT_SPIRV_HEADERS: ${{ github.workspace }}/_deps/spirv-headers/include + HOOKSHOT_VULKAN_HEADERS: ${{ github.workspace }}/_deps/vulkan-headers/include + run: python -I -B tests/run_all.py --strict + + spirv-global16: + name: SPIR-V global narrow load/store lowering + runs-on: ubuntu-24.04 + timeout-minutes: 15 + steps: + - name: Check out source + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Check out pinned SPIR-V headers + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: KhronosGroup/SPIRV-Headers + ref: 496543121ce6419f23d6fa5d7194ba66c36212d2 + path: _deps/spirv-headers + persist-credentials: false + - name: Install Clang C++20 and SPIR-V validator + run: sudo apt-get update && sudo apt-get install -y clang spirv-tools + - name: Configure the focused real-IR fixture + run: >- + CC=clang CXX=clang++ cmake -S tests/global_load_16 -B ${{ runner.temp }}/global-load-16 + -DREPO_ROOT=${{ github.workspace }} + -DSPIRV_HEADERS_ROOT=${{ github.workspace }}/_deps/spirv-headers + - name: Build the focused fixture + run: cmake --build ${{ runner.temp }}/global-load-16 --parallel 2 + - name: Emit and inspect real IR modules + run: ${{ runner.temp }}/global-load-16/global_load_16_ir ${{ runner.temp }}/global-load-16/modules + - name: Check real-IR compare-exchange loop contracts + env: + RX_VK_FOLLOW_STORES: '1' + run: ${{ runner.temp }}/global-load-16/global_load_16_ir --cas-loops ${{ runner.temp }}/global-load-16/modules + - name: Validate every emitted module + shell: bash + # (--uniform-buffer-standard-layout: the back end's device has uniformBufferStandardLayout, gpu/vk/vk_main.cpp + # creates it with every feature the GPU reports, so constant buffers are u32 arrays of stride 4) + run: | + for module in "${{ runner.temp }}"/global-load-16/modules/*.spv; do + spirv-val --target-env vulkan1.3 --uniform-buffer-standard-layout "$module" + done + - name: Configure the authored direct surface atomic fixture + run: >- + CC=clang CXX=clang++ cmake -S tests/native_suatom -B ${{ runner.temp }}/native-suatom + -DREPO_ROOT=${{ github.workspace }} + -DSPIRV_HEADERS_ROOT=${{ github.workspace }}/_deps/spirv-headers + - name: Build the direct surface atomic fixture + run: cmake --build ${{ runner.temp }}/native-suatom --parallel 2 + - name: Translate authored direct surface atomic programs + run: ${{ runner.temp }}/native-suatom/native_suatom ${{ runner.temp }}/native-suatom/modules + - name: Validate every direct surface atomic module + shell: bash + run: | + for module in "${{ runner.temp }}"/native-suatom/modules/*.spv; do + spirv-val --target-env vulkan1.3 "$module" + done + + spirv-f2i-immediate: + name: Native F2I immediate lowering + runs-on: ubuntu-24.04 + timeout-minutes: 10 + steps: + - name: Check out source + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Check out pinned SPIR-V headers + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: KhronosGroup/SPIRV-Headers + ref: 496543121ce6419f23d6fa5d7194ba66c36212d2 + path: _deps/spirv-headers + persist-credentials: false + - name: Install Clang and validator + run: sudo apt-get update && sudo apt-get install -y clang spirv-tools + - name: Build the focused fixture + run: | + CC=clang CXX=clang++ cmake -S tests/f2i_immediate -B "${{ runner.temp }}/f2i" -DREPO_ROOT="${{ github.workspace }}" -DSPIRV_HEADERS_ROOT="${{ github.workspace }}/_deps/spirv-headers" + cmake --build "${{ runner.temp }}/f2i" --parallel 2 + - name: Emit and validate authored modules + run: | + "${{ runner.temp }}/f2i/f2i_immediate" "${{ runner.temp }}/f2i/modules" + for module in "${{ runner.temp }}"/f2i/modules/*.spv; do + spirv-val --target-env vulkan1.3 --uniform-buffer-standard-layout "$module" + done + + actionlint: + name: Workflow lint (actionlint) + runs-on: ubuntu-24.04 + timeout-minutes: 5 + steps: + - name: Check out source + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + # (the release's own binary, checked against its published SHA-256; shellcheck from the runner image checks the + # run: scripts) + - name: actionlint 1.7.12 + env: + VERSION: 1.7.12 + SHA256: 8aca8db96f1b94770f1b0d72b6dddcb1ebb8123cb3712530b08cc387b349a3d8 + run: | + curl -fsSL --retry 5 --retry-delay 10 -o "$RUNNER_TEMP/actionlint.tar.gz" \ + "https://github.com/rhysd/actionlint/releases/download/v$VERSION/actionlint_${VERSION}_linux_amd64.tar.gz" + echo "$SHA256 $RUNNER_TEMP/actionlint.tar.gz" | sha256sum -c - + tar -xzf "$RUNNER_TEMP/actionlint.tar.gz" -C "$RUNNER_TEMP" actionlint + - name: Lint the workflows + run: '"$RUNNER_TEMP/actionlint" -color' diff --git a/.github/workflows/vulkan-dlss.yml b/.github/workflows/vulkan-dlss.yml new file mode 100644 index 0000000..b3753bb --- /dev/null +++ b/.github/workflows/vulkan-dlss.yml @@ -0,0 +1,127 @@ +# NVIDIA DLSS is a local build's option: NVIDIA's SDK is never part of Hookshot or its downloads. This only checks that +# hookshot_vk.dll still compiles and links with it, through llvm-mingw as players build it. It runs by hand, and on a push or +# pull request only when a file DLSS's build depends on changed (the paths below), not on every push. scripts/ngx_sdk.py +# fetches the SDK at its pinned release, SHA-256 checked, without DLSS's runtime: NVIDIA's license is accepted for that run +# alone (--accept-nvidia-license), and nothing of it is uploaded or cached. +# AMD FSR 4 (HOOKSHOT_FSR=1: AMD's MIT headers, nothing downloaded) is built in the same run: both engines share +# vk_temporal.cpp. +name: Vulkan back end with NVIDIA DLSS + +on: + push: + branches: [main, dev] + paths: + - 'gpu/vk/vk_dlss*' + - 'gpu/vk/vk_temporal*' + - 'gpu/vk/vk_fsr4*' + - 'third_party/fidelityfx/**' + - 'gpu/vk/vk_reflex*' + - 'gpu/vk/vk_ngx_msvc_rt.cpp' + - 'gpu/vk/upscaler_rows.h' + - 'gpu/vk/dlss_math.h' + - 'gpu/vk/dlss_checks.h' + - 'gpu/vk/shaders/dlss_*' + - 'host/mingw_msvc_shims.c' + - 'nx2/oot_jitter.py' + - 'nx2/build.py' + - 'nx2/make_game.py' + - 'scripts/ngx_sdk.py' + - '.github/workflows/vulkan-dlss.yml' + pull_request: + paths: + - 'gpu/vk/vk_dlss*' + - 'gpu/vk/vk_temporal*' + - 'gpu/vk/vk_fsr4*' + - 'third_party/fidelityfx/**' + - 'gpu/vk/vk_reflex*' + - 'gpu/vk/vk_ngx_msvc_rt.cpp' + - 'gpu/vk/upscaler_rows.h' + - 'gpu/vk/dlss_math.h' + - 'gpu/vk/dlss_checks.h' + - 'gpu/vk/shaders/dlss_*' + - 'host/mingw_msvc_shims.c' + - 'nx2/oot_jitter.py' + - 'nx2/build.py' + - 'nx2/make_game.py' + - 'scripts/ngx_sdk.py' + - '.github/workflows/vulkan-dlss.yml' + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +defaults: + run: + shell: bash + +jobs: + vulkan-dlss: + name: Windows Vulkan back end with NVIDIA DLSS and AMD FSR 4 (llvm-mingw, link check) + runs-on: windows-2022 + timeout-minutes: 45 + steps: + - name: Check out source + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Check out pinned Vulkan headers + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: KhronosGroup/Vulkan-Headers + ref: 74d8a6cb930c68ef617b202c3ff3c59d919e086b # v1.4.345 + path: _deps/cpm/vulkanheaders/v1.4.345 + persist-credentials: false + - name: Check out pinned SPIR-V headers + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: KhronosGroup/SPIRV-Headers + ref: 04f10f650d514df88b76d25e83db360142c7b174 # vulkan-sdk-1.4.341.0 + path: _deps/cpm/spirv-headers/vulkan-sdk-1.4.341.0 + persist-credentials: false + - name: Check out pinned VulkanMemoryAllocator + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator + ref: 1d8f600fd424278486eade7ed3e877c99f0846b1 # v3.3.0 + path: _deps/cpm/vulkanmemoryallocator/v3.3.0 + persist-credentials: false + - name: Set up Python + uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0 + with: + python-version: '3.12' + - name: llvm-mingw version + id: llvm + run: echo "name=$(python -I scripts/release_parts.py llvm-mingw --name)" >> "$GITHUB_OUTPUT" + - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ${{ runner.temp }}/llvm + key: ${{ steps.llvm.outputs.name }} + - name: llvm-mingw + run: echo "HOOKSHOT_MINGW=$(python -I scripts/release_parts.py llvm-mingw "$RUNNER_TEMP/llvm")" >> "$GITHUB_ENV" + - name: NVIDIA's DLSS SDK (headers and nvsdk_ngx_s.lib) + run: python -I scripts/ngx_sdk.py --platform windows --build-only --accept-nvidia-license "$RUNNER_TEMP/ngx-sdk" + - name: hookshot_vk.dll with NVIDIA DLSS and AMD FSR 4 + env: + RX_VK_ONLY: '1' + RX_RELEASE: '1' + HOOKSHOT_FSR: '1' + run: | + OOT_WORK="$RUNNER_TEMP/vk-work" RX_CPM="$GITHUB_WORKSPACE/_deps/cpm" HOOKSHOT_NGX_SDK="$RUNNER_TEMP/ngx-sdk" \ + python -I nx2/build.py --standalone --jobs "$NUMBER_OF_PROCESSORS" | tee "$RUNNER_TEMP/vk-build.log" + test "${PIPESTATUS[0]}" -eq 0 + grep -q "with NVIDIA DLSS" "$RUNNER_TEMP/vk-build.log" + grep -q "with AMD FSR 4" "$RUNNER_TEMP/vk-build.log" + # (what it loads is only Windows' own: the system DLLs and the UCRT's api-ms-win-crt-*. NVIDIA's runtime, nvngx_dlss.dll, + # is loaded by NGX at run time from the game folder's ngx folder, and its core, _nvngx.dll, from the driver's) + - name: Imports + run: | + dll="$RUNNER_TEMP/vk-work/sa/build/hookshot_vk.dll" + "$HOOKSHOT_MINGW/bin/llvm-objdump" -p "$dll" | sed -n 's/^ *DLL Name: //p' | tr -d '\r' | sort -u | tee "$RUNNER_TEMP/imports.txt" + if grep -viE '^(kernel32|user32|gdi32|advapi32)\.dll$|^api-ms-win-crt-[a-z0-9-]+\.dll$' "$RUNNER_TEMP/imports.txt"; then + echo "::error::hookshot_vk.dll with NVIDIA DLSS imports a DLL that is not Windows' own (above)" + exit 1 + fi diff --git a/.github/workflows/windows-builder.yml b/.github/workflows/windows-builder.yml new file mode 100644 index 0000000..334e487 --- /dev/null +++ b/.github/workflows/windows-builder.yml @@ -0,0 +1,241 @@ +# What a Windows player downloads, made without game data: HookshotBuilder-Setup.exe (builder/setup/setup.c, with +# llvm-mingw) and the Hookshot Builder package (dotnet publish, versioned from the tag as docs/versioning.md says), split +# into the parts the setup downloads by scripts/make_setup_parts.py. The Python, emsdk, CPM and wgpu-native parts, and the +# third_party part's prebuilt files, are the latest release's (scripts/release_parts.py). The Windows Vulkan back end is +# compiled too, with the same llvm-mingw: players build it on their PC, this only checks that it still builds. +# A workflow artifact on every run; on a tag, the new parts go to the builder-parts release and the setup +# (HookshotBuilder.exe) and its list (hookshot-setup.txt) to the tag's release, beside the Linux AppImage. +name: Windows builder + +on: + push: + branches: [main] + tags: ['*'] + pull_request: + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +defaults: + run: + shell: bash + +jobs: + package: + name: Hookshot Builder package and setup + # (builder-parts holds the installer's parts: no package there) + if: github.ref_name != 'builder-parts' + runs-on: windows-2022 + timeout-minutes: 30 + steps: + - name: Check out source + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + # (the version is git describe's: the tags and the history since them) + fetch-depth: 0 + persist-credentials: false + - name: Set up Python + uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0 + with: + python-version: '3.12' + - name: Set up .NET + uses: actions/setup-dotnet@0f4a354dd5547a28c70b7853dbd95d5b67886e73 # v6.1.0 + with: + dotnet-version: '8.0.x' + + - name: llvm-mingw version + id: llvm + run: echo "name=$(python -I scripts/release_parts.py llvm-mingw --name)" >> "$GITHUB_OUTPUT" + # (a release unpacks llvm-mingw from its checked download, not from a cache other runs write) + - if: ${{ !startsWith(github.ref, 'refs/tags/') }} + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ${{ runner.temp }}/llvm + key: ${{ steps.llvm.outputs.name }} + - name: llvm-mingw + run: echo "LLVM_MINGW=$(python -I scripts/release_parts.py llvm-mingw "$RUNNER_TEMP/llvm")" >> "$GITHUB_ENV" + + # (a tag is its own version; any other commit is git describe's: v2.0.1-3-g1a2b3c4, version 2.0.1-3-g1a2b3c4) + - name: Version + run: | + if [[ $GITHUB_REF == refs/tags/* ]]; then + tag=$GITHUB_REF_NAME + else + tag=$(git describe --tags --match 'v[0-9]*' 2>/dev/null || echo "v0.0.0-$(git rev-parse --short HEAD)") + fi + echo "TAG=$tag" >> "$GITHUB_ENV" + echo "VERSION=${tag#v}" >> "$GITHUB_ENV" + echo "version: $tag" + + # (the list of the newest stable release other than this tag: its Python, emsdk, CPM, wgpu-native and third_party + # parts and its play profile. A fork with no release takes upstream's. A release must have one; any other run goes + # on without, and its package lacks those parts) + - name: The latest release's setup list + env: + GH_TOKEN: ${{ github.token }} + run: | + if python -I scripts/release_parts.py previous "$RUNNER_TEMP/previous-setup.txt" \ + --repo "$GITHUB_REPOSITORY" --repo NotRexed/Hookshot --skip-tag "$TAG"; then + echo "PREVIOUS=$RUNNER_TEMP/previous-setup.txt" >> "$GITHUB_ENV" + elif [[ $GITHUB_REF == refs/tags/* ]]; then + exit 1 + else + echo "::warning::no earlier release's setup list: the package has only the app and .NET parts" + fi + + # (work/hookshot.ico is the builder's icon, builder/HookshotBuilder/HookshotBuilder.csproj) + - name: Play profile + if: env.PREVIOUS != '' + run: python -I scripts/release_parts.py profile "$PREVIOUS" work + + # (--dependent-load-flag 0x800, LOAD_LIBRARY_SEARCH_SYSTEM32: the libraries it imports come from System32, not from + # the folder it was downloaded to) + - name: HookshotBuilder-Setup.exe + working-directory: builder/setup + run: | + "$LLVM_MINGW/bin/x86_64-w64-mingw32-windres" setup.rc -O coff -o "$RUNNER_TEMP/setup_rc.o" + "$LLVM_MINGW/bin/x86_64-w64-mingw32-clang" -O2 -mwindows -municode setup.c "$RUNNER_TEMP/setup_rc.o" \ + -lwinhttp -lbcrypt -lcomctl32 -lshell32 -lole32 -Wl,--dependent-load-flag=0x800 \ + -o "$RUNNER_TEMP/HookshotBuilder-Setup.exe" + ls -l "$RUNNER_TEMP/HookshotBuilder-Setup.exe" + + - name: Hookshot Builder + run: >- + dotnet publish builder/HookshotBuilder/HookshotBuilder.csproj --configuration Release + -p:Version="$VERSION" -p:NuGetAudit=false --output "$RUNNER_TEMP/publish" + + - name: Package folder + run: >- + python -I scripts/release_parts.py package "$RUNNER_TEMP/package" + --publish "$RUNNER_TEMP/publish" --setup "$RUNNER_TEMP/HookshotBuilder-Setup.exe" + ${PREVIOUS:+--list "$PREVIOUS" --python-packages nx2/prompt_requirements.txt} + + - name: Setup parts + run: | + python -I scripts/make_setup_parts.py "$RUNNER_TEMP/package" "$TAG" dist \ + --base "https://github.com/$GITHUB_REPOSITORY/releases/download/builder-parts" ${PREVIOUS:+--reuse "$PREVIOUS"} + # (a release holds the setup as HookshotBuilder.exe: builder/HookshotBuilder/Updater.cs) + cp "$RUNNER_TEMP/HookshotBuilder-Setup.exe" dist/HookshotBuilder.exe + # (RecompLauncher installs this zip: the setup beside its recomplauncher.json, named for its system and processor) + python -I - "dist/Hookshot-$TAG-Windows-x64.zip" <<'PY' + import sys, zipfile + with zipfile.ZipFile(sys.argv[1], "w", zipfile.ZIP_DEFLATED) as z: + z.write("dist/HookshotBuilder.exe", "HookshotBuilder.exe") + z.write("recomplauncher.json", "recomplauncher.json") + PY + cat dist/hookshot-setup.txt + ls -l dist + + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: HookshotBuilder-windows + path: dist/ + if-no-files-found: error + + vulkan: + name: Windows Vulkan back end (llvm-mingw) + if: github.ref_name != 'builder-parts' + runs-on: windows-2022 + timeout-minutes: 45 + steps: + - name: Check out source + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + # (the headers in the CPM cache layout nx2/build.py reads, at the versions scripts/linux/deps.sh pins) + - name: Check out pinned Vulkan headers + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: KhronosGroup/Vulkan-Headers + ref: 74d8a6cb930c68ef617b202c3ff3c59d919e086b # v1.4.345 + path: _deps/cpm/vulkanheaders/v1.4.345 + persist-credentials: false + - name: Check out pinned SPIR-V headers + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: KhronosGroup/SPIRV-Headers + ref: 04f10f650d514df88b76d25e83db360142c7b174 # vulkan-sdk-1.4.341.0 + path: _deps/cpm/spirv-headers/vulkan-sdk-1.4.341.0 + persist-credentials: false + - name: Check out pinned VulkanMemoryAllocator + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator + ref: 1d8f600fd424278486eade7ed3e877c99f0846b1 # v3.3.0 + path: _deps/cpm/vulkanmemoryallocator/v3.3.0 + persist-credentials: false + - name: Set up Python + uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0 + with: + python-version: '3.12' + - name: llvm-mingw version + id: llvm + run: echo "name=$(python -I scripts/release_parts.py llvm-mingw --name)" >> "$GITHUB_OUTPUT" + # (a release unpacks llvm-mingw from its checked download, not from a cache other runs write) + - if: ${{ !startsWith(github.ref, 'refs/tags/') }} + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ${{ runner.temp }}/llvm + key: ${{ steps.llvm.outputs.name }} + - name: llvm-mingw + run: echo "HOOKSHOT_MINGW=$(python -I scripts/release_parts.py llvm-mingw "$RUNNER_TEMP/llvm")" >> "$GITHUB_ENV" + # (as scripts/linux/appimage.sh builds Linux's: the back end alone, no lift, no game) + - name: hookshot_vk.dll + env: + RX_VK_ONLY: '1' + RX_RELEASE: '1' + run: | + OOT_WORK="$RUNNER_TEMP/vk-work" RX_CPM="$GITHUB_WORKSPACE/_deps/cpm" \ + python -I nx2/build.py --standalone --jobs "$NUMBER_OF_PROCESSORS" + ls -l "$RUNNER_TEMP/vk-work/sa/build/hookshot_vk.dll" + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: hookshot_vk-windows + path: ${{ runner.temp }}/vk-work/sa/build/hookshot_vk.dll + if-no-files-found: error + + release: + name: Release assets + # (off until the maintainer switches it on: releases are published by hand with HookshotBuilder.exe and hookshot-setup.txt, + # and a second upload of them on the tag would race that. Set vars.CI_RELEASE_WINDOWS to 'true' to turn it on) + if: startsWith(github.ref, 'refs/tags/') && github.ref_name != 'builder-parts' && vars.CI_RELEASE_WINDOWS == 'true' + needs: [package, vulkan] + runs-on: ubuntu-24.04 + timeout-minutes: 15 + permissions: + contents: write + env: + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + steps: + - name: Check out source + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: HookshotBuilder-windows + path: dist + # (named by their SHA-256: a part already there is the same file and is not uploaded again. Before the list, which + # names them) + - name: New parts to builder-parts + run: | + if ! gh release view builder-parts >/dev/null 2>&1; then + gh release create builder-parts --prerelease --latest=false --target "$GITHUB_SHA" --title builder-parts \ + --notes "The parts HookshotBuilder-Setup.exe downloads, named by each release's hookshot-setup.txt." + fi + have=$(gh release view builder-parts --json assets --jq '.assets[].name') + for f in dist/HookshotBuilder-*.zip; do + if grep -qxF "$(basename "$f")" <<<"$have"; then + echo "$(basename "$f"): already there" + else + gh release upload builder-parts "$f" + fi + done + - name: Setup and its list to the release + run: scripts/release_upload.sh "$GITHUB_REF_NAME" dist/HookshotBuilder.exe dist/hookshot-setup.txt dist/Hookshot-*-Windows-x64.zip diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..b736876 --- /dev/null +++ b/.gitignore @@ -0,0 +1,43 @@ +# ---- never commit the game or anything made from it ---- +# (decompressed modules, the loaded image, lifted C, test builds, romfs/exefs copies, dumps, keys) +work/ +gen/ +image/ +exefs/ +romfs/ +*.bin +*.keys +*.nso +*.nca +*.nsp +*.xci +gamefile/ +*.nup +*.nsz +*.xcz + +# ---- build output and caches ---- +build/ +__pycache__/ +*.pyc +*.o +*.obj +*.exe +*.dll +*.lib +*.exp +*.pdb +*.ilk +*.sha1 +*.log + +# ---- Linux builds ---- +build-linux/ +*.so + +# ---- NVIDIA's DLSS SDK (scripts/ngx_sdk.py): NVIDIA's files, never committed ---- +ngx-sdk/ + +# Triforce Setup's build output (setup\build.ps1, setup\pack_tools.ps1) +setup/out/ +build1.done diff --git a/.releaserc.json b/.releaserc.json new file mode 100644 index 0000000..180d021 --- /dev/null +++ b/.releaserc.json @@ -0,0 +1,39 @@ +{ + "branches": [ + "main", + { + "name": "next", + "prerelease": "rc" + } + ], + "tagFormat": "v${version}", + "plugins": [ + [ + "@semantic-release/commit-analyzer", + { + "preset": "conventionalcommits" + } + ], + [ + "@semantic-release/release-notes-generator", + { + "preset": "conventionalcommits" + } + ], + [ + "@semantic-release/exec", + { + "prepareCmd": "python3 scripts/changelog.py ${nextRelease.version} ${lastRelease.gitTag}" + } + ], + [ + "@semantic-release/git", + { + "assets": [ + "changelog.txt" + ], + "message": "chore(release): ${nextRelease.version} [skip ci] [skip release]\n\n${nextRelease.notes}" + } + ] + ] +} diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..8c5945d --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,25 @@ +# Notes for AI coding agents + +**Start with [docs/workflow.md](docs/workflow.md):** what to work on first (severity x reach x confidence, reach from +the hardware people run), the hardware tiers a change must not break, and the steps from claim to release. The open work, scored and sorted, +is in [docs/worklist.md](docs/worklist.md): pick from its top. + +Follow [CONTRIBUTING.md](CONTRIBUTING.md): its rules apply to agents as to everyone, above all that no game files or +anything made from them leave the PC. The [development guide](docs/development.md) has the layout and the commands; the +[design notes](docs/design.md) have the architecture and the rules the GPU replay keeps. + +What CONTRIBUTING.md does not spell out, and agents trip on: + +- **Tests read the source.** Many checks in `tests/` cut a function out of a source file by its text and compile it + with stubs (17 read `gpu/vk/vk_menu.cpp` alone). Renaming or reshaping a function can break a check far from your + change: run `python -I -B tests/run_all.py` and update the check in the same PR rather than weakening it. +- **The renderer's interface is versioned.** A change to `MkwVkHost` or `MkwVkRenderer` in `gpu/vk/vk_api.h` bumps + `MKWVK_API_VERSION`. +- **Generated files are generated.** The SPIR-V headers in `gpu/vk/shaders/` (`*_spv.h`) are compiled from the shader + beside them; regenerate them with glslang rather than editing them (the shaders' tests show the command). +- **Two platforms.** Windows builds with llvm-mingw, Linux with clang/gcc. Keep host code portable, and paths UTF-8 + (no `CP_ACP`; `tests/test_utf8_paths.py`). +- **No game in CI.** Checks must run without a dump. A run in the game is for you to do locally and report. + +Pull requests: **open the description with why the PR is needed**, in one or two plain sentences, and **include +screenshots when the change shows on screen** (CONTRIBUTING.md, *Commits and pull requests*). diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..34e60ae --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,286 @@ +# Contributing to Hookshot + +Hookshot is a static recompilation of the Switch 2 *The Legend of Zelda: Ocarina of Time* remake: the game's AArch64 +code is lifted to C, the Horizon OS and NVN2 are implemented natively, and the GPU work is replayed on Vulkan. No +emulator is involved, and the game is built on your PC from your own copy. + +This page covers what to send, how to build and test it, and the rules that keep the project out of trouble. The +[development guide](docs/development.md) has the full pipeline and its commands; the [design notes](docs/design.md) +have the architecture and the lessons the code follows. + +## The rule with no exceptions + +**No game files, keys, or anything made from them leave your PC.** The repository does not contain the game and +never will. That includes: + +- the `.nup`, `romfs`, `exefs`, `.nso`, `.nca`, `.nsp`, `.xci` or `.nsz`/`.xcz` files, and any title keys; +- `work/` and everything in it: the extracted dump, `image.bin`, the lifted C, test builds, runs and screenshots; +- `hookshot.exe`, `hookshot_vk.dll`, `hookshot_programs.bin` (and its `hookshot_programs.idx`), `hookshot_pipelines.bin`, + `mkw_vk_pipelines.bin` and other build output; +- saves (`user/`), `hookshot_options.ini`, `hookshot_env.txt` and `language.txt`. + +Do not open an issue or a PR asking where to get the game, how to decrypt it, or for someone else's files. A +screenshot, a video and `hookshot_log.txt` are fine and often necessary; the log holds no game data. + +`.gitignore` covers most of this, but check `git status` before you commit. If you are not sure whether a file is +game data, leave it out. + +## Ways to help + +What to work on first, and the steps from claiming a problem to its release, are in the [workflow](docs/workflow.md): +priorities by severity and by how many players run the affected hardware, and the hardware tiers a change must not +break. Claim in Discord #dev-work before you start. + +| | What | +|---|---| +| **Test on other hardware** | NVIDIA cards are the most tested; the README's [status table](readme.md#project-status) says what works where, and [tested configurations](docs/tested-configurations.md) lists the runs behind it (build, hardware, driver, where, fps). Reports from AMD and Intel cards, other NVIDIA drivers, other distributions and SteamOS are useful even when everything works: add a row there. | +| **Play further** | The tested area is the start of the game. Reaching new areas, cutscenes, menus and items finds problems the developer has not seen. | +| **Report what is broken** | Use the [issue forms](https://github.com/NotRexed/Hookshot/issues/new/choose): game problem, build problem or idea. Include the version, your hardware, where you were and `hookshot_log.txt`. | +| **Send a fix** | Fork, fix, test, open a pull request. Small, focused changes are easiest to review and merge. | +| **Improve the docs** | The README, the development guide and the design notes are part of the project. Corrections and clearer steps are welcome. | + +## Before you open an issue or a PR + +- Try the [newest release](https://github.com/NotRexed/Hookshot/releases/latest) first. It includes every fix before it. +- Search the open issues. If yours is already there, add your details instead of opening another. +- English, please. +- One topic per issue or pull request. Do not mix a fix with a refactor or a batch of unrelated changes. +- Report the version you actually ran: the release name, or the first line of `hookshot_log.txt`. +- Test without ReShade, shader-cache tools, overlays and frame-generation injectors when you report a graphics or + performance problem. They change what Hookshot does. + +## Development setup + +You need your own supported dump: program `0400486024DEC000`, update v131072, build `e0f3ccfa`. Other versions do +not work, and the build reads the dump where it lies without changing it. Builds need about 60 GB free and a run uses +about 10 GB of RAM. Build on an SSD if you can: a hard drive works, but takes much longer, and the builder warns you +before it starts. + +### Windows, from the command line + +1. Install Python 3 from [python.org](https://www.python.org/downloads/) (not the Microsoft Store build: `python -I` + does not see packages installed there) and the pipeline's packages: + + ```powershell + python -m pip install -r requirements.txt + ``` + + (the second line: the button prompts' packages. Without them the game shows the Switch's icons) + +2. Get [llvm-mingw](https://github.com/mstorsjo/llvm-mingw) (the `ucrt-x86_64` build) and set `HOOKSHOT_MINGW` to its + folder. Without it the build falls back to emsdk's clang at `C:\emsdk` with Visual Studio's C++ tools and the + Windows SDK (`HOOKSHOT_MSVC=1` forces that route). + +3. Copy the pieces a clone does not carry from the `tools` folder Hookshot Builder installs. Run `HookshotBuilder.exe` + from the latest release once (releases no longer carry a `HookshotBuilder.zip`): the folder is + `%LOCALAPPDATA%\HookshotBuilder\versions\\HookshotBuilder\tools`. + + | From `tools` | To | Variable | + |---|---|---| + | `tools\cpm` | anywhere | `RX_CPM` points at it (Vulkan headers and FFmpeg; without it, system headers or `HOOKSHOT_*_HEADERS`, the standalone build stops) | + | `tools\wgpu-native` | anywhere | `WGPU_NATIVE` points at it (the WebGPU fallback renderer; without it the game is built without one) | + | `tools\third_party` | `third_party\` | (its `.dll`, `.exe`, `.lib` and `.sha1` files, to the same places: `a64emit`, `lld-link`, `zbic` and SDL2) | + | `tools\work` | `work\` | (hot-function list, link order, profile and icon; without it the game builds but runs slower) | + +4. Run everything with `python -I` from the repository root. + +### Linux + +Either use the `Hookshot--x86_64.AppImage` from the Releases page, or build the environment and the AppImage +yourself: + +```sh +scripts/linux/deps.sh # LLVM, Python packages, FFmpeg with dav1d, SDL2, Vulkan headers +scripts/linux/appimage.sh # the AppImage, in build-linux/ +``` + +The AppImage needs glibc 2.35 or newer and a Vulkan 1.3 driver with mesh shaders. `HOOKSHOT_JOBS` sets how many files +compile at once. The README's [Linux section](readme.md#on-linux) has the details and the runtime paths. + +### NVIDIA DLSS (optional) + +DLSS (`gpu/vk/vk_dlss.cpp`) is built in only with NVIDIA's DLSS SDK, which is NVIDIA's, under NVIDIA's own license, and +is never committed, bundled or uploaded: not in a PR, not in a release, not in a CI artifact. Fetch it for your own +builds with `scripts/ngx_sdk.py` (it prints NVIDIA's license notice and needs `--accept-nvidia-license` or a typed yes), +then build with `HOOKSHOT_NGX_SDK=`: + +```powershell +python -I scripts/ngx_sdk.py --platform windows C:\path\ngx-sdk # headers, nvsdk_ngx_s.lib, nvngx_dlss.dll +$env:HOOKSHOT_NGX_SDK = "C:\path\ngx-sdk" +python -I nx2/make_game.py --nup "C:\path\your-game.nup" --out "C:\Games\Hookshot-dlss" +``` + +On Windows the llvm-mingw build links NVIDIA's static `nvsdk_ngx_s.lib` (built for Microsoft's static runtime) into +`hookshot_vk.dll`, with `gpu/vk/vk_ngx_msvc_rt.cpp` and `host/mingw_msvc_shims.c` for what it needs of that runtime. +`make_game.py` copies `nvngx_dlss.dll` into the game folder's `ngx` folder; never put NGX's core, `_nvngx.dll`, beside +the game (it is the driver's). On Linux it is `--platform linux` (or `scripts/linux/ngx-sdk.sh`), with `libnvsdk_ngx.a` +and `libnvidia-ngx-dlss.so`. A Windows game with DLSS can be built on Linux too, with `RX_TARGET=windows` as below. +CI's `vulkan-dlss` workflow (`.github/workflows/vulkan-dlss.yml`) checks only that `hookshot_vk.dll` still links with the +SDK. It runs by hand, and on a push or pull request only when a file DLSS's build depends on changes; it accepts NVIDIA's +license for that run alone and uploads nothing. + +Never share a game folder built with the SDK. A change to the pinned release updates the file table and SHA-256s in +`scripts/ngx_sdk.py` (and `tests/test_ngx_sdk.py`), and the release named in the builder's license text +(`builder/HookshotBuilder/MainWindow.xaml.cs`). + +### Hookshot Builder + +The builder is a .NET 8 WPF app. Open `builder/HookshotBuilder.sln` in Visual Studio 2022 with the .NET desktop +workload, or build it from a developer prompt: + +```powershell +dotnet build builder/HookshotBuilder.sln +``` + +You only need Visual Studio for the builder itself. The game build uses llvm-mingw. + +The Windows app's **Build workers** choice applies to builds started from that window. **Automatic** keeps the existing +`HOOKSHOT_JOBS` override, or the build tools' free-memory policy when no override is set. A number overrides it for that +build's child process only; more workers need more memory and are not always faster. This choice is not saved: a new +window, including one started to update a game, uses Automatic again. + +## Building + +| Command | What it does | +|---|---| +| `python -I nx2/make_game.py --nup "C:\path\your-game.nup" --out "C:\Games\Hookshot" --language en-US` | Build a complete game folder from a `.nup`. A build takes about 40 minutes. | +| `python -I nx2/make_game.py --romfs "C:\path\romfs" --exefs "C:\path\exefs" --out "C:\Games\Hookshot"` | The same, from extracted folders. | +| `python -I nx2/make_game.py --check --nup "C:\path\your-game.nup"` | Check the game and its version without building. | +| `python -I nx2/dump.py --dump --set` | Record the dump in `work/dump.txt`. | +| `python -I nx2/lift.py --standalone --write` | Lift the game to C in `work/sa/gen`. | +| `python -I nx2/build.py --standalone --jobs 4` | Compile `work/sa/build/hookshot.exe` and `hookshot_vk.dll`. | +| `powershell -File scripts/run-vk.ps1 -Window -Seconds 120` | Run the standalone build with the Vulkan back end and take screenshots in `work/runs//shots`. | +| `python -I scripts/prof.py ` | Read a profile sampled with `HOS_PROFILE`, `HOS_SAMPLE` and `HOS_SAMPLE_FILE`. | + +Use a separate `--work` folder or `OOT_WORK` if you keep more than one dump around. The lifted C and every build are +local artifacts; they are never committed. + +## Testing + +### Game-free checks + +These need no dump and are what CI runs, on every pull request and every push to `main`. One command runs them all: + +```powershell +python -I -B tests/run_all.py +``` + +It runs every `tests/test_*.py` and `nx2/test_*.py` in its own process, the C++ unit tests under `gpu/vk/tests` and a +byte-compile of every tracked Python file, goes on after a failure, and ends with a table (slowest first) and every +failure's output. Name a check to run only the ones whose path contains it (`python -I -B tests/run_all.py ub_`). A new +`test_*.py` file is picked up by itself: no workflow to edit. + +Some checks compile a small C or C++ fixture and need a compiler (`g++`, `gcc` or `clang++` on PATH), the SPIR-V +validator (`spirv-val`), the SPIR-V or Vulkan headers (`HOOKSHOT_SPIRV_HEADERS` / `HOOKSHOT_VULKAN_HEADERS`, or the CPM +cache in `RX_CPM` or `build-linux/deps/cpm`), or the `zstandard` package. A check whose tool is missing is skipped and +says so; CI passes `--strict`, which fails it instead. A check that cannot run in CI (a GPU, the game, a Visual Studio +prompt), or runs on one OS only, says so in its own file with `RUN_ALL_SKIP`; `--no-skip` runs it anyway. + +### Checks that need your dump or a run + +These are slow and local, and they need `work/dump.txt` from `nx2/dump.py --set`: + +| Command | What it checks | +|---|---| +| `python -I nx2/test_insn.py` | Every instruction form the game uses, Unicorn against the lifted C. | +| `python -I nx2/test_leaf.py --calls` | Whole functions and call trees on the relocated image, native against Unicorn. | +| `python -I nx2/census.py --nx2` | The instruction census per module. | +| `python -I nx2/lower_test.py work/lowfail` | Shader programs that failed to lower. | +| `powershell -File scripts/run-vk.ps1 -Window` | An in-game run: screenshots, the flash detector, frame dumps and the profiler. | + +For a graphics fix, look at the screenshots in `work/runs//shots` and delete them afterwards; they reach +gigabytes and are derived from the game. The switches for scripted input, screenshots, frame dumps and profiles are in +the development guide's [reference](docs/development.md#switches-for-testing-and-debugging). + +### When you add a fix or a check + +- Add a game-free test where the behavior can be tested without the dump. The existing tests are small Python files + under `tests/` or `nx2/` that read source files and run them, or compile a tiny C++ fixture. +- Name it `tests/test_*.py` or `nx2/test_*.py`: `tests/run_all.py` picks it up by itself and CI runs it on every pull + request, so there is no workflow to edit. What it needs from the runner it declares in its own file, after its + docstring and imports, as plain literals `tests/run_all.py` reads without importing the test (its docstring has + the details); there is no central list to add a line to: + + ```python + RUN_ALL_ARGS = ["--cxx", "{cxx}", "--headers", "{vulkan_headers}"] # a compiler, headers, spirv-val ... + RUN_ALL_MODULES = ["zstandard"] # packages it imports + RUN_ALL_SKIP = {"windows": "POSIX signals: the Linux guard"} # cannot run in CI there (a GPU, the game, + # one OS only), and why + ``` + + `{name}` is a tool the runner finds (`cxx`, `vulkan_headers`, `spirv_headers`, `spirv_val`, `glslang`, `dotnet`); + a check whose tool is missing is skipped, or fails with `--strict`. A test that needs none of this declares nothing. +- Keep the test fast: every check shares one CI job with a 20-minute limit. Make it pass on both Ubuntu and Windows + where it can. +- If the fix is in the renderer or the runtime, say what you saw before and after, on which GPU and driver, and in + which part of the game. + +## Code and data conventions + +- Match the file you are editing: there is no formatter configuration, so use the indentation, comment style and + naming around you. Do not reformat unrelated code in the same change. +- Keep OoT-specific code in new files where possible. `nx2/`, `host/` and `gpu/` are kept close to mkw-recomp so + upstream fixes can be applied as a patch; keeping the paths and the shared files recognizable is what makes that + work. Each sync names its new mkw-recomp base commit in the commit message. +- Do not modify `third_party/`. Its parts are unmodified upstream copies with their notices in `PROVENANCE.md`; add a + new folder and a provenance note rather than editing one. +- Treat the local mkw-recomp and Eglantine checkouts as read-only references: read from them, run them, never edit + them. +- New switches use the `RX_` prefix in the renderer and `HOS_` in the runtime. They are off, or neutral, unless + asked for, and they do not cost anything while off. Document them where the surrounding switches are documented. +- The GPU replay rules in [design notes, section 4](docs/design.md#4-the-lessons-from-mkw-as-rules-here) are not + suggestions: nothing the replay reads may come from live guest memory, waits are honoured per queue, and texture + and sampler registrations stay in submission order. A change that reads guest memory at replay time is a + regression, even if it looks right on one frame. +- Python runs with `-I`. Tests stay on the standard library so CI needs no install step; the pipeline's existing + dependencies (lz4, numpy, capstone, unicorn, zstandard) are the exception, not a reason to add more. So are the + button prompts' (fontTools, Pillow, astc-encoder-py, texture2ddecoder: `nx2/prompt_requirements.txt`), accepted for + `nx2/prompt_assets.py` and `nx2/prompt_draw.py` alone, at build time. Maintainers' agents reviewing a change: check + that these four are still used only there, and flag any other use. Keep line endings LF (`.gitattributes`). +- New code has to be GPL-compatible. See below. + +## Commits and pull requests + +1. Fork the repository and branch from `main`. Name the branch for the change: `fix/...`, `linux/...`, `builder/...`, + `docs/...`. +2. Keep the pull request to one topic. Bug fixes do not carry refactors, and two unrelated fixes are two PRs. +3. Write the title as a short imperative sentence, as the history does: "Stop scanning storage-buffer tables after + one pass", "Linux: a crash report gives the whole host stack". Start the body with why the PR is needed, in one or + two plain sentences a player could follow: the bug, the missing feature or the cost it removes ("The menu's text + is cut off at 4K", "First starts spend 20 s waiting on a few shaders"). Then say what changed, how it was tested + (GPU, driver, game area, fps before and after), and link the issue it fixes. +4. Include screenshots when the change shows on screen: graphics, the menu, the start-up screen, an overlay, a + message. Before and after for a fix, the new screen for a feature, and the sizes or settings it was checked at. A + screenshot of the game is fine; the rule above is about game files, not pictures of the game. +5. Run the relevant checks. CI must pass; a change that only works on your machine is not ready. +6. Rebase or merge `main` when it has moved, and resolve conflicts in your branch rather than in the review. +7. Do not include build output, dumps, generated files or a `.gitignore` bypass. Check `git status`. + +Commit subjects decide the next release: write them as Conventional Commits (`feat:`, `fix:`, and `!` or +`BREAKING CHANGE:` for a break). The version rules, branches and changelog format are in +[docs/versioning.md](docs/versioning.md). + +Maintainer commits record AI assistance with `Assisted-by:` and `Co-Authored-By:` trailers. If you used a tool, add +the trailers that describe it; if you wrote the change yourself, there is nothing to add. + +## Licensing + +Hookshot is free software under the [GNU General Public License, version 3 or later](LICENSE). By submitting a change +you agree to license it under those terms. Only send code you can license that way. Do not paste code from a project +with an incompatible license, and do not paste decompiled game code or anything generated from the dump into a PR. + +The third-party parts (Eden, suyu, Sirit, the Vulkan Memory Allocator, SDL, Opus, Dear ImGui, zstd, the Libertinus +fonts, zbic and the a64-kit tools) keep their own notices. Preserve them, and do not relicense their code. + +NVIDIA's DLSS SDK is not one of them: it is never in the repository, and nothing of it may be added (see +[NVIDIA DLSS](#nvidia-dlss-optional)). + +## Where to talk + +- **Issues:** [use the forms](https://github.com/NotRexed/Hookshot/issues/new/choose). +- **Discord:** https://discord.gg/EajnD2AJD9. Use the `hookshot-discussion` channel; the server is mainly for + web ports, and that channel is where Hookshot talk belongs. +- **Newest release:** https://github.com/NotRexed/Hookshot/releases/latest. + +Thanks for testing, reporting and sending fixes. Every report from a different GPU and every area played further than +before moves the port forward. diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..f288702 --- /dev/null +++ b/LICENSE @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. diff --git a/assets/evil_hookshot.png b/assets/evil_hookshot.png new file mode 100644 index 0000000..61dd685 Binary files /dev/null and b/assets/evil_hookshot.png differ diff --git a/assets/readme/divider.svg b/assets/readme/divider.svg new file mode 100644 index 0000000..16fb2c1 --- /dev/null +++ b/assets/readme/divider.svg @@ -0,0 +1,7 @@ + + + + + + + diff --git a/assets/readme/emerald.svg b/assets/readme/emerald.svg new file mode 100644 index 0000000..54c6f75 --- /dev/null +++ b/assets/readme/emerald.svg @@ -0,0 +1,4 @@ + + + + diff --git a/assets/readme/gold.svg b/assets/readme/gold.svg new file mode 100644 index 0000000..be1a8e9 --- /dev/null +++ b/assets/readme/gold.svg @@ -0,0 +1,3 @@ + + + diff --git a/assets/readme/ruby.svg b/assets/readme/ruby.svg new file mode 100644 index 0000000..552d5b3 --- /dev/null +++ b/assets/readme/ruby.svg @@ -0,0 +1,4 @@ + + + + diff --git a/assets/readme/sapphire.svg b/assets/readme/sapphire.svg new file mode 100644 index 0000000..5401a65 --- /dev/null +++ b/assets/readme/sapphire.svg @@ -0,0 +1,4 @@ + + + + diff --git a/assets/readme/title.svg b/assets/readme/title.svg new file mode 100644 index 0000000..7eb07b9 --- /dev/null +++ b/assets/readme/title.svg @@ -0,0 +1,5 @@ + + +TRIFORCE + + diff --git a/assets/triforce.png b/assets/triforce.png new file mode 100644 index 0000000..7349f55 Binary files /dev/null and b/assets/triforce.png differ diff --git a/builder/.gitignore b/builder/.gitignore new file mode 100644 index 0000000..547a964 --- /dev/null +++ b/builder/.gitignore @@ -0,0 +1,4 @@ +bin/ +obj/ +.vs/ +*.user diff --git a/builder/HookshotBuilder.sln b/builder/HookshotBuilder.sln new file mode 100644 index 0000000..93ac539 --- /dev/null +++ b/builder/HookshotBuilder.sln @@ -0,0 +1,22 @@ + +Microsoft Visual Studio Solution File, Format Version 12.00 +# Visual Studio Version 17 +VisualStudioVersion = 17.14.0.0 +MinimumVisualStudioVersion = 10.0.40219.1 +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "HookshotBuilder", "HookshotBuilder\HookshotBuilder.csproj", "{6F1C2D94-7A3B-4E58-9C11-2D4B8E0A7F31}" +EndProject +Global + GlobalSection(SolutionConfigurationPlatforms) = preSolution + Debug|Any CPU = Debug|Any CPU + Release|Any CPU = Release|Any CPU + EndGlobalSection + GlobalSection(ProjectConfigurationPlatforms) = postSolution + {6F1C2D94-7A3B-4E58-9C11-2D4B8E0A7F31}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {6F1C2D94-7A3B-4E58-9C11-2D4B8E0A7F31}.Debug|Any CPU.Build.0 = Debug|Any CPU + {6F1C2D94-7A3B-4E58-9C11-2D4B8E0A7F31}.Release|Any CPU.ActiveCfg = Release|Any CPU + {6F1C2D94-7A3B-4E58-9C11-2D4B8E0A7F31}.Release|Any CPU.Build.0 = Release|Any CPU + EndGlobalSection + GlobalSection(SolutionProperties) = preSolution + HideSolutionNode = FALSE + EndGlobalSection +EndGlobal diff --git a/builder/HookshotBuilder/App.xaml b/builder/HookshotBuilder/App.xaml new file mode 100644 index 0000000..0c91136 --- /dev/null +++ b/builder/HookshotBuilder/App.xaml @@ -0,0 +1,425 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + pack://application:,,,/Fonts/#Inter + pack://application:,,,/Fonts/#Inter + Cascadia Mono, Consolas + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/builder/HookshotBuilder/App.xaml.cs b/builder/HookshotBuilder/App.xaml.cs new file mode 100644 index 0000000..221e04f --- /dev/null +++ b/builder/HookshotBuilder/App.xaml.cs @@ -0,0 +1,49 @@ +using System.IO; +using System.Text; +using System.Windows; + +namespace HookshotBuilder; + +public partial class App : Application +{ + // (RecompLauncher) --cli : no window; its lines on stdout, and its result as the exit code. + // Why it stopped is one "error " line on stdout (the stream a launcher reads) and again on stderr + protected override void OnStartup(StartupEventArgs e) + { + if (e.Args.Length > 0 && e.Args[0] == "--cli") + { + Environment.Exit(Cli(e.Args.Skip(1).ToList())); + return; + } + base.OnStartup(e); + } + + static int Cli(List args) + { + using var stdout = new StreamWriter(Console.OpenStandardOutput(), new UTF8Encoding(false)) { AutoFlush = true }; + var tools = Tools.Find(); + if (tools == null) + { + const string missing = "error The build tools were not found next to this program."; + stdout.WriteLine(missing); + Console.Error.WriteLine(missing); + return 1; + } + var done = new ManualResetEventSlim(); + string? failure = null, said = null; // (said: make_game's own "error" line in --prepare mode, already passed on to stdout) + var runner = new Runner(tools, args); + runner.Line += line => + { + if (line.StartsWith("error ")) said = line[6..]; + lock (stdout) stdout.WriteLine(line); + }; + runner.Finished += result => { failure = result; done.Set(); }; + runner.Start(); + done.Wait(); + if (failure == null) return 0; + var reason = failure == "cancelled" ? failure : said ?? failure; + if (said == null) lock (stdout) stdout.WriteLine("error " + reason); + Console.Error.WriteLine("error " + reason); + return 1; + } +} diff --git a/builder/HookshotBuilder/BuildWorkers.cs b/builder/HookshotBuilder/BuildWorkers.cs new file mode 100644 index 0000000..6f3017c --- /dev/null +++ b/builder/HookshotBuilder/BuildWorkers.cs @@ -0,0 +1,29 @@ +using System.Diagnostics; +using System.Globalization; + +namespace HookshotBuilder; + +/// A choice for this window only. Automatic leaves the build tools' inherited override and memory policy alone. +static class BuildWorkers +{ + public const string Automatic = "Automatic"; + + public static IEnumerable Choices(int processors) + { + yield return Automatic; + for (var count = 1; count <= Math.Max(1, processors); count++) yield return count; + } + + public static int? Selected(object? value) => value switch + { + string text when text == Automatic => null, + int count when count > 0 => count, + _ => throw new ArgumentException("Choose Automatic or a positive number of build workers."), + }; + + public static void Apply(ProcessStartInfo info, int? count) + { + if (count is <= 0) throw new ArgumentOutOfRangeException(nameof(count), "Build workers must be positive."); + if (count is int selected) info.Environment["HOOKSHOT_JOBS"] = selected.ToString(CultureInfo.InvariantCulture); + } +} diff --git a/builder/HookshotBuilder/BuilderSettings.cs b/builder/HookshotBuilder/BuilderSettings.cs new file mode 100644 index 0000000..53804bd --- /dev/null +++ b/builder/HookshotBuilder/BuilderSettings.cs @@ -0,0 +1,23 @@ +namespace HookshotBuilder; + +/// What the player chose last time (the install folder, the game files, the shaders' tick): builder-settings.json beside the versions +sealed class BuilderSettings +{ + public string? Out { get; set; } + public string? Nup { get; set; } + public string? Romfs { get; set; } + public string? Exefs { get; set; } + public bool Dlss { get; set; } // (NVIDIA DLSS ticked: NVIDIA's license was accepted) + public bool Fsr { get; set; } // (AMD FSR 4 ticked) + public bool PrepareLater { get; set; } // ("Prepare shaders now" unticked) + + public static BuilderSettings? Load(string path) + { + try { return System.Text.Json.JsonSerializer.Deserialize(System.IO.File.ReadAllText(path)); } + catch (Exception) { return null; } + } + + /// (the updates, from the launcher's Update and from the --update start after a self-update, begin before ApplySettings sets + /// the tick from these settings: they read the saved choice itself. None saved: prepared, as the tick's default) + public static bool PrepareNow(string path) => Load(path)?.PrepareLater != true; +} diff --git a/builder/HookshotBuilder/Disk.cs b/builder/HookshotBuilder/Disk.cs new file mode 100644 index 0000000..b772edb --- /dev/null +++ b/builder/HookshotBuilder/Disk.cs @@ -0,0 +1,55 @@ +using System.IO; +using System.Runtime.InteropServices; +using Microsoft.Win32.SafeHandles; + +namespace HookshotBuilder; + +/// Whether a folder is on a hard drive (a spinning disk) or an SSD: a build reads and writes tens of gigabytes, and an SSD makes +/// that much faster. The answer is a warning only (the player may have no other drive), never a stop. +static class Disk +{ + /// True on a hard drive, false on an SSD, null when the drive cannot be told (a network share, an unknown volume) + public static bool? Spinning(string path) + { + try + { + var root = Path.GetPathRoot(Path.GetFullPath(path)); + if (string.IsNullOrEmpty(root)) return null; + using var volume = CreateFile(@"\\.\" + root.TrimEnd('\\'), 0, FileShare.ReadWrite, IntPtr.Zero, FileMode.Open, + FileOptions.None, IntPtr.Zero); + if (volume.IsInvalid) return null; + var query = new StoragePropertyQuery { PropertyId = SeekPenalty, QueryType = 0 }; + var answer = new DeviceSeekPenaltyDescriptor(); + // (a spinning disk says it incurs a seek penalty; an SSD says it does not) + if (!DeviceIoControl(volume, QueryProperty, ref query, Marshal.SizeOf(), ref answer, + Marshal.SizeOf(), out _, IntPtr.Zero)) + return null; + return answer.IncursSeekPenalty; + } + catch (Exception e) when (e is IOException or UnauthorizedAccessException or ArgumentException) { return null; } + } + + const uint QueryProperty = 0x002D1400; // (IOCTL_STORAGE_QUERY_PROPERTY) + const uint SeekPenalty = 7; // (StorageDeviceSeekPenaltyProperty) + + [StructLayout(LayoutKind.Sequential)] + struct StoragePropertyQuery // (STORAGE_PROPERTY_QUERY: AdditionalParameters[1], padded to a DWORD) + { + public uint PropertyId, QueryType, Additional; + } + + [StructLayout(LayoutKind.Sequential)] + struct DeviceSeekPenaltyDescriptor + { + public uint Version, Size; + [MarshalAs(UnmanagedType.U1)] public bool IncursSeekPenalty; + } + + [DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)] + static extern SafeFileHandle CreateFile(string name, uint access, FileShare share, IntPtr security, FileMode mode, + FileOptions flags, IntPtr template); + + [DllImport("kernel32.dll", SetLastError = true)] + static extern bool DeviceIoControl(SafeFileHandle device, uint code, ref StoragePropertyQuery input, int inputSize, + ref DeviceSeekPenaltyDescriptor output, int outputSize, out uint returned, IntPtr overlapped); +} diff --git a/builder/HookshotBuilder/Finder.cs b/builder/HookshotBuilder/Finder.cs new file mode 100644 index 0000000..61e4940 --- /dev/null +++ b/builder/HookshotBuilder/Finder.cs @@ -0,0 +1,145 @@ +using System.IO; + +namespace HookshotBuilder; + +/// A .nup found on this PC +sealed record Dump(string Path, string Where, long Size, DateTime Modified); + +/// The player's game files where people keep them, and the game's shortcuts. +static class Finder +{ + /// .nup files in Downloads, Desktop, Documents, the home folder and the top of each drive, one folder level down too, + /// newest first. (Only names and sizes are read here: the version is checked one file at a time afterwards.) + public static List FindDumps() + { + var home = Environment.GetFolderPath(Environment.SpecialFolder.UserProfile); + var places = new List<(string Dir, string Name)> + { + (Path.Combine(home, "Downloads"), "Downloads"), + (Environment.GetFolderPath(Environment.SpecialFolder.DesktopDirectory), "Desktop"), + (Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments), "Documents"), + (home, "Home"), + }; + foreach (var d in DriveInfo.GetDrives()) + { + try + { + if (d.IsReady && d.DriveType is DriveType.Fixed or DriveType.Removable) places.Add((d.RootDirectory.FullName, d.Name.TrimEnd('\\'))); + } + catch (Exception) { } + } + var seen = new HashSet(StringComparer.OrdinalIgnoreCase); + var found = new List(); + foreach (var (dir, name) in places) + { + if (!Directory.Exists(dir) || !seen.Add(Path.GetFullPath(dir))) continue; + var dirs = new List { dir }; + try + { + dirs.AddRange(Directory.EnumerateDirectories(dir) + .Where(s => !Path.GetFileName(s).StartsWith('.') && !Path.GetFileName(s).StartsWith('$') + && !string.Equals(Path.GetFileName(s), "Windows", StringComparison.OrdinalIgnoreCase) + && !Path.GetFileName(s).StartsWith("Program", StringComparison.OrdinalIgnoreCase)) + .Take(200)); + } + catch (Exception) { } + foreach (var sub in dirs) + { + try + { + foreach (var f in Directory.EnumerateFiles(sub, "*.nup")) + { + var info = new FileInfo(f); + if (seen.Add(info.FullName)) found.Add(new Dump(info.FullName, name, info.Length, info.LastWriteTime)); + } + } + catch (Exception) { } + } + } + return found.OrderByDescending(d => d.Modified).ToList(); + } + + public static string Ago(DateTime t) + { + var d = DateTime.Now - t; + if (d.TotalDays < 1 && t.Date == DateTime.Today) return "modified today"; + if (d.TotalDays < 2) return "modified yesterday"; + if (d.TotalDays < 14) return $"{(int)d.TotalDays} days ago"; + if (d.TotalDays < 60) return $"{(int)(d.TotalDays / 7)} weeks ago"; + return t.ToString("d MMM yyyy"); + } + + public static string Size(long bytes) => bytes >= 1L << 30 ? $"{bytes / (double)(1L << 30):0.0} GB" : $"{bytes >> 20} MB"; + + // ---- shortcuts to the built game ---- + + public static string DesktopLink => Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.DesktopDirectory), "Hookshot.lnk"); + public static string StartLink => Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Programs), "Hookshot.lnk"); + + /// Made or removed as the two ticks say (Windows' own shell object writes the .lnk) + /// The shortcuts the player has, made again (onto the folder's launcher now that it has one); none made where there was none + public static void RefreshShortcuts(string gameFolder) => Shortcuts(gameFolder, File.Exists(DesktopLink), File.Exists(StartLink)); + + public static void Shortcuts(string gameFolder, bool desktop, bool start) + { + // (hookshot.exe: the game's launcher in a folder built since it came (Play, and Update when a newer release is out), else the + // game. The icon is the game's own: bin\hookshot.exe when it is there) + var exe = Path.Combine(gameFolder, "hookshot.exe"); + var icon = File.Exists(Path.Combine(gameFolder, "bin", "hookshot.exe")) ? Path.Combine(gameFolder, "bin", "hookshot.exe") : exe; + foreach (var (link, want) in new[] { (DesktopLink, desktop), (StartLink, start) }) + { + try + { + if (!want) + { + if (File.Exists(link)) File.Delete(link); + continue; + } + if (!File.Exists(exe)) continue; + // (the shell's own shortcut object (IShellLinkW + IPersistFile): the scripting host's, late-bound through dynamic, + // made none for players and said nothing) + var sl = (IShellLinkW)new ShellLink(); + sl.SetPath(exe); + sl.SetWorkingDirectory(gameFolder); + sl.SetIconLocation(icon, 0); + sl.SetDescription("The Legend of Zelda: Ocarina of Time, built by Hookshot"); + Directory.CreateDirectory(Path.GetDirectoryName(link)!); + ((System.Runtime.InteropServices.ComTypes.IPersistFile)sl).Save(link, true); + } + catch (Exception e) + { + ShortcutError = $"{Path.GetFileName(link)}: {e.Message}"; + } + } + } + + /// The last shortcut that could not be made, for the log + public static string? ShortcutError; + + [System.Runtime.InteropServices.ComImport, System.Runtime.InteropServices.Guid("00021401-0000-0000-C000-000000000046")] + class ShellLink { } + + [System.Runtime.InteropServices.ComImport, System.Runtime.InteropServices.Guid("000214F9-0000-0000-C000-000000000046"), + System.Runtime.InteropServices.InterfaceType(System.Runtime.InteropServices.ComInterfaceType.InterfaceIsIUnknown)] + interface IShellLinkW + { + void GetPath([System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.LPWStr)] System.Text.StringBuilder file, int max, IntPtr fd, uint flags); + void GetIDList(out IntPtr ppidl); + void SetIDList(IntPtr pidl); + void GetDescription([System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.LPWStr)] System.Text.StringBuilder name, int max); + void SetDescription([System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.LPWStr)] string name); + void GetWorkingDirectory([System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.LPWStr)] System.Text.StringBuilder dir, int max); + void SetWorkingDirectory([System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.LPWStr)] string dir); + void GetArguments([System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.LPWStr)] System.Text.StringBuilder args, int max); + void SetArguments([System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.LPWStr)] string args); + void GetHotkey(out short hotkey); + void SetHotkey(short hotkey); + void GetShowCmd(out int show); + void SetShowCmd(int show); + void GetIconLocation([System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.LPWStr)] System.Text.StringBuilder path, int max, out int index); + void SetIconLocation([System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.LPWStr)] string path, int index); + void SetRelativePath([System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.LPWStr)] string path, uint reserved); + void Resolve(IntPtr hwnd, uint flags); + void SetPath([System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.LPWStr)] string file); + } +} diff --git a/builder/HookshotBuilder/Fonts/Inter-Bold.ttf b/builder/HookshotBuilder/Fonts/Inter-Bold.ttf new file mode 100644 index 0000000..9fb9b75 Binary files /dev/null and b/builder/HookshotBuilder/Fonts/Inter-Bold.ttf differ diff --git a/builder/HookshotBuilder/Fonts/Inter-LICENSE.txt b/builder/HookshotBuilder/Fonts/Inter-LICENSE.txt new file mode 100644 index 0000000..9b2ca37 --- /dev/null +++ b/builder/HookshotBuilder/Fonts/Inter-LICENSE.txt @@ -0,0 +1,92 @@ +Copyright (c) 2016 The Inter Project Authors (https://github.com/rsms/inter) + +This Font Software is licensed under the SIL Open Font License, Version 1.1. +This license is copied below, and is also available with a FAQ at: +http://scripts.sil.org/OFL + +----------------------------------------------------------- +SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007 +----------------------------------------------------------- + +PREAMBLE +The goals of the Open Font License (OFL) are to stimulate worldwide +development of collaborative font projects, to support the font creation +efforts of academic and linguistic communities, and to provide a free and +open framework in which fonts may be shared and improved in partnership +with others. + +The OFL allows the licensed fonts to be used, studied, modified and +redistributed freely as long as they are not sold by themselves. The +fonts, including any derivative works, can be bundled, embedded, +redistributed and/or sold with any software provided that any reserved +names are not used by derivative works. The fonts and derivatives, +however, cannot be released under any other type of license. The +requirement for fonts to remain under this license does not apply +to any document created using the fonts or their derivatives. + +DEFINITIONS +"Font Software" refers to the set of files released by the Copyright +Holder(s) under this license and clearly marked as such. This may +include source files, build scripts and documentation. + +"Reserved Font Name" refers to any names specified as such after the +copyright statement(s). + +"Original Version" refers to the collection of Font Software components as +distributed by the Copyright Holder(s). + +"Modified Version" refers to any derivative made by adding to, deleting, +or substituting -- in part or in whole -- any of the components of the +Original Version, by changing formats or by porting the Font Software to a +new environment. + +"Author" refers to any designer, engineer, programmer, technical +writer or other person who contributed to the Font Software. + +PERMISSION AND CONDITIONS +Permission is hereby granted, free of charge, to any person obtaining +a copy of the Font Software, to use, study, copy, merge, embed, modify, +redistribute, and sell modified and unmodified copies of the Font +Software, subject to the following conditions: + +1) Neither the Font Software nor any of its individual components, +in Original or Modified Versions, may be sold by itself. + +2) Original or Modified Versions of the Font Software may be bundled, +redistributed and/or sold with any software, provided that each copy +contains the above copyright notice and this license. These can be +included either as stand-alone text files, human-readable headers or +in the appropriate machine-readable metadata fields within text or +binary files as long as those fields can be easily viewed by the user. + +3) No Modified Version of the Font Software may use the Reserved Font +Name(s) unless explicit written permission is granted by the corresponding +Copyright Holder. This restriction only applies to the primary font name as +presented to the users. + +4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font +Software shall not be used to promote, endorse or advertise any +Modified Version, except to acknowledge the contribution(s) of the +Copyright Holder(s) and the Author(s) or with their explicit written +permission. + +5) The Font Software, modified or unmodified, in part or in whole, +must be distributed entirely under this license, and must not be +distributed under any other license. The requirement for fonts to +remain under this license does not apply to any document created +using the Font Software. + +TERMINATION +This license becomes null and void if any of the above conditions are +not met. + +DISCLAIMER +THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT +OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE +COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, +INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL +DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM +OTHER DEALINGS IN THE FONT SOFTWARE. diff --git a/builder/HookshotBuilder/Fonts/Inter-Medium.ttf b/builder/HookshotBuilder/Fonts/Inter-Medium.ttf new file mode 100644 index 0000000..458cd06 Binary files /dev/null and b/builder/HookshotBuilder/Fonts/Inter-Medium.ttf differ diff --git a/builder/HookshotBuilder/Fonts/Inter-Regular.ttf b/builder/HookshotBuilder/Fonts/Inter-Regular.ttf new file mode 100644 index 0000000..b7aaca8 Binary files /dev/null and b/builder/HookshotBuilder/Fonts/Inter-Regular.ttf differ diff --git a/builder/HookshotBuilder/Fonts/Inter-SemiBold.ttf b/builder/HookshotBuilder/Fonts/Inter-SemiBold.ttf new file mode 100644 index 0000000..47f8ab1 Binary files /dev/null and b/builder/HookshotBuilder/Fonts/Inter-SemiBold.ttf differ diff --git a/builder/HookshotBuilder/Glass.cs b/builder/HookshotBuilder/Glass.cs new file mode 100644 index 0000000..1f70798 --- /dev/null +++ b/builder/HookshotBuilder/Glass.cs @@ -0,0 +1,63 @@ +using System.Runtime.InteropServices; +using System.Windows; +using System.Windows.Interop; +using System.Windows.Media; +using Microsoft.Win32; + +namespace HookshotBuilder; + +/// Windows 11's acrylic behind the window, dark title colours and round corners (DWM). Windows 10 has none of it, and +/// neither does a light Windows: both get the plain Slate window, which the light palette is made for. +static class Glass +{ + [DllImport("dwmapi.dll")] static extern int DwmSetWindowAttribute(IntPtr hwnd, int attribute, ref int value, int size); + [DllImport("dwmapi.dll")] static extern int DwmExtendFrameIntoClientArea(IntPtr hwnd, ref Margins margins); + [DllImport("user32.dll")] static extern IntPtr GetWindowLongPtrW(IntPtr hwnd, int index); + [DllImport("user32.dll")] static extern IntPtr SetWindowLongPtrW(IntPtr hwnd, int index, IntPtr value); + + [StructLayout(LayoutKind.Sequential)] + struct Margins { public int Left, Right, Top, Bottom; } + + const int DarkMode = 20, CornerPreference = 33, SystemBackdrop = 38; + const int Round = 2, Acrylic = 3; + + /// Windows draws the acrylic in the system's Windows mode. Its light variant is brighter than anything the installer's + /// tint can darken enough: the fixed Slate palette has light text and InkFaint (the inactive steps, the "Supported + /// version" badge, the details) needs the dark page behind it. A light system gets that page, as Windows 10 does. + static bool LightBackdrop() + { + try + { + using var key = Registry.CurrentUser.OpenSubKey(@"Software\Microsoft\Windows\CurrentVersion\Themes\Personalize"); + var value = key?.GetValue("SystemUsesLightTheme") ?? key?.GetValue("AppsUseLightTheme"); + return value is int on && on != 0; + } + catch + { + return false; + } + } + + /// true: acrylic is on (the window's background must stay see-through) + public static bool Apply(Window w) + { + var hwnd = new WindowInteropHelper(w).Handle; + if (hwnd == IntPtr.Zero) return false; + // (a framed window: Windows draws the backdrop only behind one. Without the system menu it draws no caption buttons of its + // own behind the window's; WindowStyle None lost the acrylic, and the window was flat grey) + const int GwlStyle = -16; + const long SysMenu = 0x00080000; + SetWindowLongPtrW(hwnd, GwlStyle, (IntPtr)((long)GetWindowLongPtrW(hwnd, GwlStyle) & ~SysMenu)); + var on = 1; + DwmSetWindowAttribute(hwnd, DarkMode, ref on, sizeof(int)); + var corner = Round; + DwmSetWindowAttribute(hwnd, CornerPreference, ref corner, sizeof(int)); + if (LightBackdrop()) return false; // (the plain Slate page: light acrylic cannot carry the palette) + if (Environment.OSVersion.Version.Build < 22621 || SystemParameters.HighContrast) return false; // (22H2: the backdrop attribute) + if (HwndSource.FromHwnd(hwnd) is { CompositionTarget: { } target }) target.BackgroundColor = Colors.Transparent; + var all = new Margins { Left = -1, Right = -1, Top = -1, Bottom = -1 }; + DwmExtendFrameIntoClientArea(hwnd, ref all); + var backdrop = Acrylic; + return DwmSetWindowAttribute(hwnd, SystemBackdrop, ref backdrop, sizeof(int)) == 0; + } +} diff --git a/builder/HookshotBuilder/HookshotBuilder.csproj b/builder/HookshotBuilder/HookshotBuilder.csproj new file mode 100644 index 0000000..62431a5 --- /dev/null +++ b/builder/HookshotBuilder/HookshotBuilder.csproj @@ -0,0 +1,34 @@ + + + + WinExe + net8.0-windows + true + enable + enable + HookshotBuilder + HookshotBuilder + 0.1.0 + Hookshot Builder + app.manifest + en + ..\..\work\hookshot.ico + + + + + win-x64 + true + false + none + + + + + + + + + + diff --git a/builder/HookshotBuilder/Images/hookshot.png b/builder/HookshotBuilder/Images/hookshot.png new file mode 100644 index 0000000..05168dc Binary files /dev/null and b/builder/HookshotBuilder/Images/hookshot.png differ diff --git a/builder/HookshotBuilder/MainWindow.xaml b/builder/HookshotBuilder/MainWindow.xaml new file mode 100644 index 0000000..96fff7f --- /dev/null +++ b/builder/HookshotBuilder/MainWindow.xaml @@ -0,0 +1,360 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/builder/HookshotBuilder/MainWindow.xaml.cs b/builder/HookshotBuilder/MainWindow.xaml.cs new file mode 100644 index 0000000..f22beea --- /dev/null +++ b/builder/HookshotBuilder/MainWindow.xaml.cs @@ -0,0 +1,1363 @@ +using System.Diagnostics; +using System.IO; +using Path = System.Windows.Shapes.Path; +using System.Text.RegularExpressions; +using System.Windows; +using System.Windows.Controls; +using System.Windows.Media; +using System.Windows.Media.Animation; +using System.Windows.Media.Effects; +using System.Windows.Shapes; +using System.Windows.Threading; +using Microsoft.Win32; + +namespace HookshotBuilder; + +/// The installer: one 960x600 window. It opens with the hookshot in the middle while it looks for the game files, then walks +/// through Game files > Build > Play. The game's update offer (Updater.cs) opens the same window on its update screen. +public partial class MainWindow : Window +{ + const long NeededBytes = 45L << 30; // (measured 2026-11-09: the game data 24.4 GB, the shader caches ~8 GB, the game 0.4 GB, the build files ~5 GB at their peak; the rest headroom) + Language lang = Strings.Default(); + string T(S s) => (int)s < lang.Text.Length ? lang.Text[(int)s] : Strings.All[0].Text[(int)s]; + + readonly DispatcherTimer clock = new() { Interval = TimeSpan.FromSeconds(1) }; + readonly Stopwatch elapsed = new(); + readonly Stopwatch compileClock = new(); + readonly Dictionary rows = new(); + readonly Dictionary checks = new(StringComparer.OrdinalIgnoreCase); + readonly List dumps = new(); + readonly System.Text.StringBuilder log = new(); + Runner? runner; + string? current; + string finished = ""; + string? chosenNup, romfs, exefs; + // (issue 650) "Using the folders ..." above the list of .nup files, while romfs and exefs are the game files: FillDumps clears the + // list, so the note is made there again each time (it was gone as soon as the list was first filled) + string? foldersNote; + TextBlock? foldersNoteBlock; + bool updating, checking; + double compiled; + int filesDone, filesOf; + int filesChanged = -1, filesCompiled; + double compileEta = -1; + long thisCompile, previousTotal = -1; + string lastFailure = ""; + + public MainWindow() + { + InitializeComponent(); + OutPath.Text = System.IO.Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.UserProfile), "Games", "Hookshot"); + LanguageBox.ItemsSource = Strings.All; + LanguageBox.SelectedItem = lang; + WorkerBox.ItemsSource = BuildWorkers.Choices(Environment.ProcessorCount); + WorkerBox.SelectedIndex = 0; + clock.Tick += (_, _) => Tick(); + Drop += Dropped; + SourceInitialized += (_, _) => + { + if (!Glass.Apply(this)) // (Windows 10, or no acrylic: plain Slate) + { + Background = (Brush)FindResource("Page"); + Root.Background = (Brush)FindResource("Page"); + } + }; + Loaded += (_, _) => Start(); + var tools = Tools.Find(); + ToolsNote.Text = tools != null + ? "Nothing else to install: Hookshot's own build tools are here, downloaded and checked by its setup." + : "Hookshot's build tools are missing from its folder. Press Build and Hookshot offers to download them again."; + SetStep(0); + Refresh(); + } + + // ================= the start ================= + + async void Start() + { + // (development) HOOKSHOT_TEST_SHORTCUTS=: both shortcuts made for it, the result in the log beside the app, and out + if (Environment.GetEnvironmentVariable("HOOKSHOT_TEST_SHORTCUTS") is { Length: > 0 } testFolder) + { + Finder.Shortcuts(testFolder, true, true); + File.WriteAllText(System.IO.Path.Combine(AppContext.BaseDirectory, "shortcut-test.txt"), + $"desktop {File.Exists(Finder.DesktopLink)} start {File.Exists(Finder.StartLink)} error {Finder.ShortcutError}"); + Application.Current.Shutdown(); + return; + } + Animate(); + var started = Stopwatch.StartNew(); + if (Environment.GetCommandLineArgs().Contains("--update")) + { + SplashText.Text = "Getting the update…"; + await Task.Delay(900); + ShowMain(); + Updater.FromCommandLine(this); + return; + } + if (LaunchArg() is { } launchFolder) // (Hookshot Launcher.exe in the game folder) + { + SplashText.Text = "Starting Hookshot…"; + await Task.Delay(700); + ShowMain(); + ShowLauncher(launchFolder); + return; + } + SplashText.Text = "Looking for your game files…"; + var found = await Task.Run(Finder.FindDumps); + dumps.AddRange(found); + SplashText.Text = found.Count switch { 0 => "No game files found yet", 1 => "Found 1 game file", _ => $"Found {found.Count} game files" }; + var wait = 1700 - (int)started.ElapsedMilliseconds; + if (wait > 0) await Task.Delay(wait); + ApplySettings(); + FillDumps(); + ShowMain(); + if (Environment.GetEnvironmentVariable("HOOKSHOT_UI_PREVIEW") is { Length: > 0 } preview) { Preview(preview); return; } + CheckAll(); + latestTag = Environment.GetEnvironmentVariable("HOOKSHOT_LATEST_TAG") is { Length: > 0 } fake ? (fake == "none" ? null : fake) : await Updater.LatestTag(); + RefreshMode(); + } + + // ================= what the button does: build, update the game, play it, or get the newer builder ================= + // (as NX2navi's launcher: whatever version the player opens, it checks the latest release and says what is there) + + enum Mode { Build, UpdateGame, UpToDate, UpdateBuilder } + Mode mode = Mode.Build; + string? latestTag, updateTarget; + bool sourcePicked; // (the player chose game files here: a build from them, even over a game that is up to date) + + /// The release a game folder was built with (its hookshot_version.txt), when it keeps its game files for updates + static string? GameVersion(string folder) + { + try + { + if (!Directory.Exists(System.IO.Path.Combine(folder, ".hookshot_exefs")) || !Directory.Exists(System.IO.Path.Combine(folder, "romfs"))) + return null; + foreach (var line in File.ReadAllLines(System.IO.Path.Combine(folder, "hookshot_version.txt"))) + if (line.StartsWith("version=")) return line[8..].Trim() is { Length: > 0 } v ? v : null; + } + catch (Exception) { } + return null; + } + + void RefreshMode() + { + if (BuildButton == null || updating || runner != null) return; + var game = OutPath.Text.Length > 2 ? GameVersion(OutPath.Text) : null; + var builder = Tools.Find()?.Version; + // (the newest release known: GitHub's, else this builder's own. v2.0.0 took a GitHub that did not answer for "up to date") + updateTarget = latestTag ?? builder; + if (game != null && updateTarget != null && game != updateTarget && !sourcePicked) mode = Mode.UpdateGame; + else if (game != null && !sourcePicked) mode = Mode.UpToDate; + else if (latestTag != null && builder != null && builder != latestTag) mode = Mode.UpdateBuilder; + else mode = Mode.Build; + switch (mode) + { + case Mode.UpdateGame: + FilesTitle.Text = "Update available"; + FilesSub.Text = $"Hookshot {updateTarget} is available. Your game is {game}: Update builds it again from the game files kept in " + + "its folder, compiling only what changed."; + BuildButton.Content = "Update"; + break; + case Mode.UpToDate: + FilesTitle.Text = "Your game is up to date"; + FilesSub.Text = latestTag == null + ? $"Your game is {game}, as this builder. GitHub did not answer, so newer versions could not be checked." + : $"{game} is the latest version. Pick game files below only to build it again from scratch."; + BuildButton.Content = "Play"; + break; + case Mode.UpdateBuilder: + FilesTitle.Text = "Build your game"; + FilesSub.Text = $"Hookshot Builder {latestTag} is available (this one is {builder}): update it first, then build."; + BuildButton.Content = "Update builder"; + break; + default: + FilesTitle.Text = "Build your game"; + FilesSub.Text = "Pick your game files and where the game goes. Hookshot does the rest, once."; + BuildButton.Content = "Build the game"; + break; + } + BuildButton.IsEnabled = mode != Mode.Build || (SourceReady && OutPath.Text.Length > 2); + AboutText.Visibility = mode is Mode.Build ? Visibility.Visible : Visibility.Collapsed; + } + + // ---- the launcher: Hookshot Launcher.exe in a game folder starts this with --launch (builder/setup/setup.c) ---- + string launchFolder = ""; + + static string? LaunchArg() + { + var a = Environment.GetCommandLineArgs(); + var i = Array.IndexOf(a, "--launch"); + return i >= 0 && i + 1 < a.Length ? a[i + 1].TrimEnd('\\', '/') : null; + } + + async void ShowLauncher(string folder) + { + launchFolder = folder; + OutPath.Text = folder; + StepPill.Visibility = Visibility.Collapsed; + var game = GameVersion(folder) ?? ReadVersion(folder); + LaunchVersion.Text = (game != null ? $"Hookshot {game}" : "Hookshot") + " · " + folder; + LaunchPlay.IsEnabled = File.Exists(GameExe(folder)); + ShowPage(LaunchPage); + latestTag = Environment.GetEnvironmentVariable("HOOKSHOT_LATEST_TAG") is { Length: > 0 } fake ? (fake == "none" ? null : fake) : await Updater.LatestTag(); + updateTarget = latestTag ?? Tools.Find()?.Version; // (GitHub's newest, else this builder's own) + if (updateTarget == null) LaunchStatus.Text = "Could not check for updates (GitHub did not answer)"; + else if (game != null && game != updateTarget && GameVersion(folder) != null) + { + LaunchStatus.Text = $"Update available: {updateTarget}"; + LaunchStatus.Foreground = new SolidColorBrush(Color.FromRgb(0xB9, 0xC8, 0xFF)); + LaunchStatus.FontWeight = FontWeights.SemiBold; + LaunchUpdate.Visibility = Visibility.Visible; + } + else if (game != null && game != updateTarget) + LaunchStatus.Text = $"{updateTarget} is out: build this folder again with the Hookshot Builder to get it"; + else LaunchStatus.Text = latestTag == null ? "Up to date with this builder (GitHub did not answer)" : "Up to date"; + } + + /// (a folder built before it kept its game files has a version too: only shown) + static string? ReadVersion(string folder) + { + try + { + foreach (var line in File.ReadAllLines(System.IO.Path.Combine(folder, "hookshot_version.txt"))) + if (line.StartsWith("version=")) return line[8..].Trim() is { Length: > 0 } v ? v : null; + } + catch (Exception) { } + return null; + } + + /// The game in a folder: bin\hookshot.exe below its launcher (hookshot.exe), or hookshot.exe in a folder built before the launcher + static string GameExe(string folder) + { + var game = System.IO.Path.Combine(folder, "bin", "hookshot.exe"); + return File.Exists(game) ? game : System.IO.Path.Combine(folder, "hookshot.exe"); + } + + /// (the launcher already checked for updates: the game does not ask again) + static void StartGame(string folder) + { + var si = new ProcessStartInfo(GameExe(folder)) { UseShellExecute = false, WorkingDirectory = folder }; + si.Environment["HOOKSHOT_NO_UPDATE_CHECK"] = "1"; + Process.Start(si); + } + + void LaunchUpdateClick(object sender, RoutedEventArgs e) => Updater.UpdateFolder(this, launchFolder, updateTarget, fetched: false); + + void LaunchPlayClick(object sender, RoutedEventArgs e) + { + StartGame(launchFolder); + Application.Current.Shutdown(); + } + + void LaunchFolderClick(object sender, RoutedEventArgs e) => + Process.Start(new ProcessStartInfo(launchFolder) { UseShellExecute = true }); + + // ---- what the player chose last time (BuilderSettings.cs): builder-settings.json beside the versions ---- + static string SettingsPath => System.IO.Path.Combine( + Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "HookshotBuilder", "builder-settings.json"); + + void ApplySettings() + { + var s = BuilderSettings.Load(SettingsPath); + if (s == null) return; + PrepareTick.IsChecked = !s.PrepareLater; + if (!string.IsNullOrEmpty(s.Out)) OutPath.Text = s.Out; + DlssTick.IsChecked = s.Dlss; + FsrTick.IsChecked = s.Fsr; + if (s.Romfs != null && s.Exefs != null && Directory.Exists(s.Romfs) && File.Exists(System.IO.Path.Combine(s.Exefs, "main"))) + { + romfs = s.Romfs; + exefs = s.Exefs; + foldersNote = $"Using the folders {romfs} and {exefs} (as last time)"; + } + else if (s.Nup != null && File.Exists(s.Nup)) + { + if (!dumps.Any(d => string.Equals(d.Path, s.Nup, StringComparison.OrdinalIgnoreCase))) + { + var info = new FileInfo(s.Nup); + dumps.Insert(0, new Dump(info.FullName, "Last time", info.Length, info.LastWriteTime)); + } + chosenNup = s.Nup; + } + } + + void SaveSettings() + { + try + { + var s = new BuilderSettings { Out = OutPath.Text, Nup = romfs == null ? chosenNup : null, Romfs = romfs, Exefs = exefs, + Dlss = DlssTick.IsChecked == true, Fsr = FsrTick.IsChecked == true, + PrepareLater = PrepareTick.IsChecked != true }; + Directory.CreateDirectory(System.IO.Path.GetDirectoryName(SettingsPath)!); + File.WriteAllText(SettingsPath, System.Text.Json.JsonSerializer.Serialize(s)); + } + catch (Exception) { } + } + + /// (development) HOOKSHOT_UI_PREVIEW=build|ready|problem|update: one screen with made-up progress, nothing run + void Preview(string what) + { + switch (what) + { + case "build": + MakeRows(update: false, shaders: true); + SetStep(1); + ShowPage(BuildPage); + rows["read"].Begin(); rows["read"].Done(); rows["read"].Small("Version checked"); + rows["lift"].Begin(); rows["lift"].Done(); + rows["compile"].Begin(); rows["compile"].Small("31 of 58 files"); + SetProgress(0.54); + BuildNote.Text = "About 19 minutes left · your PC stays usable"; + break; + case "ready": + finished = OutPath.Text; + ShowReady(); + break; + case "problem": + ShowProblem("Files of the builder's own compiler are missing (clang.exe in tools\\llvm-mingw\\bin).", tools: true); + break; + case "update": + BeginUpdate(OutPath.Text, "honest-eta"); + rows["download"].Done(); rows["download"].Small("3 MB · build tools unchanged"); + rows["read"].Begin(); rows["read"].Done(); + rows["lift"].Begin(); rows["lift"].Done(); + rows["compile"].Begin(); rows["compile"].Small("7 of 516 files"); + SetProgress(0.62); + BuildNote.Text = "About 3 minutes left. Your current game stays as it is until the new one is complete."; + break; + } + } + + void Animate() + { + var forever = RepeatBehavior.Forever; + var ease = new SineEase { EasingMode = EasingMode.EaseInOut }; + HaloScale.BeginAnimation(ScaleTransform.ScaleXProperty, new DoubleAnimation(0.94, 1.04, TimeSpan.FromSeconds(1.2)) { AutoReverse = true, RepeatBehavior = forever, EasingFunction = ease }); + HaloScale.BeginAnimation(ScaleTransform.ScaleYProperty, new DoubleAnimation(0.94, 1.04, TimeSpan.FromSeconds(1.2)) { AutoReverse = true, RepeatBehavior = forever, EasingFunction = ease }); + Halo.BeginAnimation(OpacityProperty, new DoubleAnimation(0.6, 1, TimeSpan.FromSeconds(1.2)) { AutoReverse = true, RepeatBehavior = forever, EasingFunction = ease }); + BobLift.BeginAnimation(TranslateTransform.YProperty, new DoubleAnimation(0, -6, TimeSpan.FromSeconds(1.2)) { AutoReverse = true, RepeatBehavior = forever, EasingFunction = ease }); + BobTurn.BeginAnimation(RotateTransform.AngleProperty, new DoubleAnimation(-4, 2, TimeSpan.FromSeconds(1.2)) { AutoReverse = true, RepeatBehavior = forever, EasingFunction = ease }); + Sweep.BeginAnimation(TranslateTransform.XProperty, new DoubleAnimation(-88, 208, TimeSpan.FromSeconds(1.2)) { RepeatBehavior = forever, EasingFunction = ease }); + } + + /// The hookshot fades out and the first step fades in, in the same window + void ShowMain() + { + var fade = new DoubleAnimation(0, TimeSpan.FromMilliseconds(450)); + fade.Completed += (_, _) => { Splash.Visibility = Visibility.Collapsed; StopSplash(); }; + Splash.BeginAnimation(OpacityProperty, fade); + SplashScale.BeginAnimation(ScaleTransform.ScaleXProperty, new DoubleAnimation(1.04, TimeSpan.FromMilliseconds(500))); + SplashScale.BeginAnimation(ScaleTransform.ScaleYProperty, new DoubleAnimation(1.04, TimeSpan.FromMilliseconds(500))); + var ease = new CubicEase { EasingMode = EasingMode.EaseOut }; + Main.BeginAnimation(OpacityProperty, new DoubleAnimation(1, TimeSpan.FromMilliseconds(500)) { BeginTime = TimeSpan.FromMilliseconds(150) }); + MainLift.BeginAnimation(TranslateTransform.YProperty, new DoubleAnimation(0, TimeSpan.FromMilliseconds(600)) { BeginTime = TimeSpan.FromMilliseconds(150), EasingFunction = ease }); + Main.IsHitTestVisible = true; + } + + void StopSplash() + { + foreach (var (o, p) in new (Animatable, DependencyProperty)[] { (HaloScale, ScaleTransform.ScaleXProperty), (HaloScale, ScaleTransform.ScaleYProperty), + (BobLift, TranslateTransform.YProperty), (BobTurn, RotateTransform.AngleProperty), (Sweep, TranslateTransform.XProperty) }) + o.BeginAnimation(p, null); + Halo.BeginAnimation(OpacityProperty, null); + } + + // ================= the step pill: Game files > Build > Play ================= + + void SetStep(int on) + { + StepItems.Children.Clear(); + var names = new[] { "Game files", "Build", "Play" }; + for (var i = 0; i < names.Length; i++) + { + if (i > 0) + StepItems.Children.Add(new Path { Data = Geometry.Parse("M0,0 L4,4 L0,8"), Stroke = (Brush)FindResource("InkSoft"), StrokeThickness = 1.6, + Opacity = 0.45, VerticalAlignment = VerticalAlignment.Center, Margin = new Thickness(5, 0, 5, 0) }); + var done = i < on; + var text = new TextBlock { Text = names[i], FontSize = 13, VerticalAlignment = VerticalAlignment.Center, + FontWeight = i == on ? FontWeights.SemiBold : FontWeights.Medium, + Foreground = (Brush)FindResource(i == on ? "Ink" : done ? "InkSoft" : "InkFaint") }; + var inner = new StackPanel { Orientation = Orientation.Horizontal, VerticalAlignment = VerticalAlignment.Center }; + if (done) + { + var tick = new Grid { Width = 22, Height = 22, Margin = new Thickness(0, 0, 8, 0) }; + tick.Children.Add(new Ellipse { Fill = (Brush)FindResource("GoSoft") }); + tick.Children.Add(new Path { Data = Geometry.Parse("M0,4 L3.5,7.5 L10,0"), Stroke = (Brush)FindResource("Go"), StrokeThickness = 1.8, + HorizontalAlignment = HorizontalAlignment.Center, VerticalAlignment = VerticalAlignment.Center }); + inner.Children.Add(tick); + } + inner.Children.Add(text); + var pill = new Border { Height = 30, CornerRadius = new CornerRadius(15), Padding = done ? new Thickness(4, 0, 14, 0) : new Thickness(14, 0, 14, 0), Child = inner }; + if (i == on) + { + text.Foreground = Brushes.White; + pill.Background = new LinearGradientBrush(Color.FromRgb(0x4B, 0x70, 0xF0), Color.FromRgb(0x35, 0x57, 0xD6), 90); + pill.Effect = new DropShadowEffect { Color = Color.FromRgb(0x3D, 0x63, 0xE6), BlurRadius = 18, ShadowDepth = 6, Direction = 270, Opacity = 0.6 }; + } + StepItems.Children.Add(pill); + } + StepPill.Visibility = Visibility.Visible; + VersionChip.Visibility = Visibility.Collapsed; + } + + void ShowPage(FrameworkElement page) + { + foreach (var p in new FrameworkElement[] { FilesPage, BuildPage, ReadyPage, ProblemPage, LaunchPage }) + p.Visibility = p == page ? Visibility.Visible : Visibility.Collapsed; + page.Opacity = 0; + page.BeginAnimation(OpacityProperty, new DoubleAnimation(1, TimeSpan.FromMilliseconds(250))); + } + + // ================= 1: game files ================= + + void FillDumps() + { + DumpList.Children.Clear(); + foldersNoteBlock = null; + if (romfs != null && exefs != null && foldersNote != null) + { + foldersNoteBlock = new TextBlock { Text = foldersNote, Style = (Style)FindResource("Note"), Margin = new Thickness(16, 12, 16, 12) }; + DumpList.Children.Add(foldersNoteBlock); + } + if (dumps.Count == 0) + { + DumpList.Children.Add(new TextBlock + { + Text = "No .nup files in Downloads, Desktop, Documents or the top folders of your drives. Browse to yours, or drop it on this window.", + Style = (Style)FindResource("Note"), Margin = new Thickness(16, 14, 16, 14), + }); + return; + } + for (var i = 0; i < dumps.Count; i++) + { + var d = dumps[i]; + if (i > 0) DumpList.Children.Add(new Border { Height = 1, Background = (Brush)FindResource("GlassLine") }); + var name = new TextBlock { Text = System.IO.Path.GetFileName(d.Path), FontWeight = FontWeights.SemiBold, TextTrimming = TextTrimming.CharacterEllipsis }; + var meta = new TextBlock { Text = $"{d.Where} · {Finder.Size(d.Size)} · {Finder.Ago(d.Modified)}", Style = (Style)FindResource("Faint") }; + var texts = new StackPanel { Children = { name, meta }, Margin = new Thickness(0, 0, 12, 0) }; + // (issue 563: "Can't be read" said why only in the chip's tooltip, which players did not find; the reason is in the row) + if (checks.TryGetValue(d.Path, out var why) && why is { Ok: false } && !why.Value.Text.StartsWith("This is build")) + texts.Children.Add(new TextBlock { Text = why.Value.Text, Style = (Style)FindResource("Faint"), TextWrapping = TextWrapping.Wrap, + Foreground = (Brush)FindResource("Bad"), Margin = new Thickness(0, 4, 0, 0) }); + var grid = new Grid { ColumnDefinitions = { new ColumnDefinition(), new ColumnDefinition { Width = GridLength.Auto } } }; + var chip = Chip(d.Path); + Grid.SetColumn(chip, 1); + grid.Children.Add(texts); + grid.Children.Add(chip); + var row = new RadioButton { Style = (Style)FindResource("DumpRow"), GroupName = "Dumps", Content = grid, Tag = d.Path, ToolTip = d.Path, + IsChecked = string.Equals(d.Path, chosenNup, StringComparison.OrdinalIgnoreCase) }; + row.Checked += (_, _) => Choose(d.Path); + DumpList.Children.Add(row); + } + } + + Border Chip(string path) + { + checks.TryGetValue(path, out var c); + var (text, brush) = c == null ? ("Checking…", "InkFaint") + : c.Value.Ok ? ("✓ Supported version", "Go") + : c.Value.Text.StartsWith("This is build") ? ("Not the supported version", "Warn") + : ("Can't be read", "Bad"); + var chip = new Border { Style = (Style)FindResource("Chip"), Child = new TextBlock { Text = text, FontSize = 12, Foreground = (Brush)FindResource(brush), VerticalAlignment = VerticalAlignment.Center } }; + if (c is { Ok: false }) chip.ToolTip = c.Value.Text; + if (c is { Ok: true }) chip.Background = (Brush)FindResource("GoSoft"); + return chip; + } + + /// Each found file's version, one at a time (the check unpacks into one probe folder); the first good one is chosen + async void CheckAll() + { + if (checking) return; + checking = true; + var tools = Tools.Find(); + while (tools != null && dumps.FirstOrDefault(d => !checks.ContainsKey(d.Path)) is { } d) + { + checks[d.Path] = null; + FillDumps(); + var r = await Runner.Check(tools, new[] { "--check", "--nup", d.Path }); + checks[d.Path] = r; + if (!r.Ok && r.Text.Contains("antivirus", StringComparison.OrdinalIgnoreCase)) + { + checking = false; + ShowProblem(r.Text, tools: true); + return; + } + // (issue 650: not over the romfs and exefs folders the player chose, here or last time: the found .nup showed as chosen) + if (chosenNup == null && romfs == null && r.Ok) chosenNup = d.Path; + FillDumps(); + Refresh(); + } + if (chosenNup == null && romfs == null && dumps.Count > 0 && tools != null) { chosenNup = dumps[0].Path; FillDumps(); } + checking = false; + Refresh(); + } + + void Choose(string path) + { + sourcePicked = true; + chosenNup = path; + romfs = exefs = null; + if (foldersNoteBlock != null) DumpList.Children.Remove(foldersNoteBlock); // (not the whole list again: this row's own event) + foldersNoteBlock = null; + Refresh(); + } + + void AddDump(string path, string where) + { + var info = new FileInfo(path); + if (!dumps.Any(d => string.Equals(d.Path, info.FullName, StringComparison.OrdinalIgnoreCase))) + dumps.Insert(0, new Dump(info.FullName, where, info.Length, info.LastWriteTime)); + sourcePicked = true; + chosenNup = info.FullName; + romfs = exefs = null; + FillDumps(); + CheckAll(); + Refresh(); + } + + bool SourceReady => romfs != null && exefs != null + || chosenNup != null && checks.TryGetValue(chosenNup, out var c) && c is { Ok: true }; + + void PathsChanged(object sender, RoutedEventArgs e) => Refresh(); + + void Refresh() + { + if (BuildButton == null) return; + BuildButton.IsEnabled = SourceReady && OutPath.Text.Length > 2; + ShowWork(); + try + { + var root = System.IO.Path.GetPathRoot(System.IO.Path.GetFullPath(OutPath.Text))!; + var free = new DriveInfo(root).AvailableFreeSpace; + SpaceBar.Value = Math.Min(1, NeededBytes / (double)Math.Max(1, free)); + SpaceBar.Foreground = (Brush)FindResource(free >= NeededBytes ? "Go" : "Bad"); + SpaceText.Text = $"Needs 45 GB · {root.TrimEnd('\\')} has {free >> 30} GB free"; + SpaceText.Foreground = (Brush)FindResource(free >= NeededBytes ? "InkSoft" : "Bad"); + } + catch (Exception) + { + SpaceText.Text = "Needs 45 GB"; + } + RefreshMode(); + } + + /// The game folder or the build files folder on a hard drive: the build works, but much more slowly than on an SSD. Null + /// when both are on an SSD (or their drive cannot be told, as on a network share) + string? HardDriveWarning() + { + foreach (var (what, folder) in new[] { ("game folder", OutPath.Text), ("build files folder", Storage.Work) }) + { + if (Disk.Spinning(folder) != true) continue; + string root; + try { root = System.IO.Path.GetPathRoot(System.IO.Path.GetFullPath(folder))!.TrimEnd('\\'); } + catch (Exception e) when (e is ArgumentException or NotSupportedException or PathTooLongException) { root = folder; } + return $"The {what} is on a hard drive ({root}). An SSD builds the game much faster: build there if you can.\n\n" + + "Choose OK to build here anyway, or Cancel to choose another folder."; + } + return null; + } + + void Dropped(object sender, DragEventArgs e) + { + if (runner != null || e.Data.GetData(DataFormats.FileDrop) is not string[] { Length: > 0 } paths) return; + foreach (var p in paths) + { + if (p.EndsWith(".nup", StringComparison.OrdinalIgnoreCase) && File.Exists(p)) AddDump(p, "Dropped"); + else if (Directory.Exists(System.IO.Path.Combine(p, "romfs")) && Directory.Exists(System.IO.Path.Combine(p, "exefs"))) + UseFolders(System.IO.Path.Combine(p, "romfs"), System.IO.Path.Combine(p, "exefs")); + } + } + + void BrowseNup(object sender, RoutedEventArgs e) + { + var d = new OpenFileDialog { Filter = "Game file (*.nup)|*.nup", Title = "Choose your game file" }; + if (d.ShowDialog(this) == true) AddDump(d.FileName, "Chosen"); + } + + /// The other way in: an unpacked game, its romfs and exefs folders (or one folder holding both) + void BrowseFolders(object sender, RoutedEventArgs e) + { + var both = PickFolder("Choose the folder with romfs and exefs (or the romfs folder)"); + if (both == null) return; + if (Directory.Exists(System.IO.Path.Combine(both, "romfs")) && Directory.Exists(System.IO.Path.Combine(both, "exefs"))) + { + UseFolders(System.IO.Path.Combine(both, "romfs"), System.IO.Path.Combine(both, "exefs")); + return; + } + var ex = PickFolder("Now choose the exefs folder"); + if (ex != null) UseFolders(both, ex); + } + + void UseFolders(string r, string x) + { + if (!File.Exists(System.IO.Path.Combine(x, "main"))) + { + MessageBox.Show(this, "That exefs folder has no file named main.", "Hookshot"); + return; + } + sourcePicked = true; + romfs = r; + exefs = x; + chosenNup = null; + foldersNote = $"Using the folders {r} and {x}"; + FillDumps(); + Refresh(); + } + + static string? PickFolder(string title) + { + var d = new OpenFolderDialog { Title = title }; + return d.ShowDialog() == true ? d.FolderName : null; + } + + void BrowseOut(object sender, RoutedEventArgs e) => OutPath.Text = PickFolder("Choose where the game goes") ?? OutPath.Text; + + void BrowseWork(object sender, RoutedEventArgs e) + { + if (PickFolder("Choose where Hookshot keeps its build files") is not { } picked) return; + ChangeWork(picked); + } + + void ResetWork(object sender, RoutedEventArgs e) => ChangeWork(null); + + void ChangeWork(string? picked) + { + try { Storage.MoveWork(picked); } + catch (Exception x) when (x is IOException or UnauthorizedAccessException) + { + MessageBox.Show(this, $"Hookshot cannot keep its build files there: {x.Message}", "Hookshot"); + } + Refresh(); + } + + /// How much the build files take and where, said before the build (they stay after it, for quick updates) + void ShowWork() + { + if (WorkNote == null) return; + var work = Storage.Work; + WorkReset.Visibility = Storage.WorkChosen ? Visibility.Visible : Visibility.Collapsed; + WorkNote.Text = Storage.KeepBuildFiles + ? $"Build files: about {Storage.Gb(Storage.BuildFilesBytes)}, kept in {work} so that updates are quick (HOOKSHOT_KEEP_BUILD_FILES)." + : $"Build files: about {Storage.Gb(Storage.BuildFilesBytes)} in {work} while it builds; when the game is ready about 1 GB stays (the compiled code) so that updates compile only what changed."; + WorkNote.Foreground = (Brush)FindResource("InkSoft"); + try + { + var root = System.IO.Path.GetPathRoot(System.IO.Path.GetFullPath(work))!; + var free = new DriveInfo(root).AvailableFreeSpace; + if (free < Storage.BuildFilesBytes) + { + WorkNote.Text += $" {root.TrimEnd('\\')} has only {Storage.Gb(free)} free."; + WorkNote.Foreground = (Brush)FindResource("Bad"); + } + } + catch (Exception) { } + } + + void LanguageChanged(object sender, SelectionChangedEventArgs e) + { + if (LanguageBox.SelectedItem is not Language l) return; + lang = l; + CancelButton.Content = T(S.Cancel); + BackButton.Content = T(S.Back); + OpenButton.Content = T(S.OpenFolder); + PlayText.Text = T(S.Play); + ShowLog.Content = T(ShowLog.IsChecked == true ? S.HideDetails : S.ShowDetails); + } + + // ================= 2: the build ================= + + async void StartBuild(object sender, RoutedEventArgs e) + { + switch (mode) + { + case Mode.UpdateGame: // (the kept game files; the newer builder first when this one is older) + SaveSettings(); + Updater.UpdateFolder(this, OutPath.Text, updateTarget, fetched: false); + return; + case Mode.UpToDate: + finished = OutPath.Text; + Play(sender, e); + return; + case Mode.UpdateBuilder: + SaveSettings(); + BuildButton.IsEnabled = false; + BuildButton.Content = "Getting the new builder…"; + try { await Updater.RunLatestSetup(this, Array.Empty()); } + catch (Exception ex) + { + MessageBox.Show(this, ex.Message + " You can also download HookshotBuilder.exe from the releases page.", "Hookshot"); + RefreshMode(); + } + return; + } + SaveSettings(); + var tools = Tools.Find(); + if (tools == null) + { + ShowProblem("The build tools are not in Hookshot's folder.", tools: true); + return; + } + var args = new List(); + if (romfs != null && exefs != null) args.AddRange(new[] { "--romfs", romfs, "--exefs", exefs }); + else if (chosenNup != null) args.AddRange(new[] { "--nup", chosenNup }); + else return; + args.AddRange(new[] { "--out", OutPath.Text, "--language", lang.Code }); + if (FsrTick.IsChecked == true) args.Add("--fsr"); // (an --update of the folder keeps it: make_game.py's mark) + prepareNow = PrepareTick.IsChecked == true; + Run(tools, ShaderStep.Args(args, prepareNow), dlss: DlssTick.IsChecked == true); + } + + const string DlssLicense = + "NVIDIA DLSS is NVIDIA's, not Hookshot's. With it ticked, the build downloads NVIDIA's DLSS SDK (release v310.9.16) from " + + "github.com/NVIDIA/DLSS, under NVIDIA's own license, the NVIDIA RTX SDKs license:\n\n" + + "https://github.com/NVIDIA/DLSS/blob/9981c6350dc58c0ce21f024fd65e31e851c31689/LICENSE.txt\n\n" + + "Its files stay on this PC (in the builder's ngx-sdk folder, and nvngx_dlss.dll in the game folder's ngx folder). " + + "Hookshot never includes or shares them: never share a game folder built with them.\n\n" + + "Do you accept NVIDIA's license?"; + + /// The NVIDIA DLSS tick: NVIDIA's license first, and the tick stays only when it is accepted + void DlssChanged(object sender, RoutedEventArgs e) + { + if (DlssTick.IsChecked == true && + MessageBox.Show(this, DlssLicense, "NVIDIA DLSS", MessageBoxButton.YesNo, MessageBoxImage.Information) != MessageBoxResult.Yes) + DlssTick.IsChecked = false; + SaveSettings(); // (kept at once, not only when Build is pressed: ticked and closed, it is still ticked next time) + } + + static readonly (string Id, string Build, string Update)[] Stages = + { + ("download", "", "Download the update"), + ("read", "Read your game files", "Read the game files kept in the folder"), + ("lift", "Translate the game to PC code", "Translate the game to PC code"), + ("compile", "Compile", "Compile changed files"), + ("finish", "Put the game together", "Swap in the new game"), + (ShaderStep.Id, "Prepare the shaders", "Prepare the shaders"), + }; + static readonly Dictionary WithShaders = new() + { + ["download"] = (0, 0.04), ["read"] = (0.04, 0.04), ["lift"] = (0.08, 0.14), ["compile"] = (0.22, 0.68), ["finish"] = (0.90, 0.02), + [ShaderStep.Id] = (0.92, 0.08), // (hookshot --prepare-shaders: minutes on some GPUs, the first start's wait moved here) + }; + static readonly Dictionary WithoutShaders = new() + { + ["download"] = (0, 0.04), ["read"] = (0.04, 0.04), ["lift"] = (0.08, 0.14), ["compile"] = (0.22, 0.74), ["finish"] = (0.96, 0.04), + }; + Dictionary Weight = WithoutShaders; + // (this build prepares the shaders: the tick for a build, the saved choice for an update. The game's @@stage shaders came, and + // make_game's line when it left them to the first start: the row's end says which) + bool prepareNow, shadersBegan; + string? shadersLater; + TimeSpan? builtIn; // (at @@times: the build without the shaders' minutes, as build_times.json keeps it) + + void MakeRows(bool update, bool shaders) + { + StageList.Children.Clear(); + rows.Clear(); + Weight = shaders ? WithShaders : WithoutShaders; + var list = Stages.Where(s => (update || s.Id != "download") && (shaders || s.Id != ShaderStep.Id)).ToList(); + for (var i = 0; i < list.Count; i++) + { + var row = new StepRow(i + 1, update ? list[i].Update : list[i].Build, last: i == list.Count - 1); + rows[list[i].Id] = row; + StageList.Children.Add(row.Root); + } + if (update) rows["read"].Small("Your game files are kept: the .nup isn't needed"); + } + + /// dlss: NVIDIA DLSS, its license accepted: NVIDIA's SDK fetched into Storage.NgxSdk first (only what is missing), and built in + void Run(Tools tools, List args, bool keepRows = false, bool dlss = false) + { + // (a warning, not a stop: the player may have no other folder with the room. An update keeps its rows and its own flow) + if (!keepRows && HardDriveWarning() is { } slow && + MessageBox.Show(this, slow, "Hookshot", MessageBoxButton.OKCancel, MessageBoxImage.Warning) != MessageBoxResult.OK) + return; + int? workers; + try { workers = BuildWorkers.Selected(WorkerBox.SelectedItem); } + catch (ArgumentException e) + { + MessageBox.Show(this, e.Message, "Hookshot"); + return; + } + if (!keepRows) + { + updating = false; + MakeRows(update: false, shaders: prepareNow); + SetStep(1); + BuildTitle.Text = "Building your game"; + UpdateBar.Visibility = Visibility.Collapsed; + RingBox.Visibility = Visibility.Visible; + } + log.Clear(); + Log.Clear(); + current = null; + finished = ""; + compiled = 0; + filesDone = filesOf = 0; + filesChanged = -1; + filesCompiled = 0; + compileEta = -1; + thisCompile = 0; + previousTotal = -1; + shadersBegan = false; + shadersLater = null; + builtIn = null; + ShowPage(BuildPage); + CancelButton.Visibility = Visibility.Visible; + SetProgress(updating ? Weight["read"].From : 0); + elapsed.Restart(); + compileClock.Reset(); + clock.Start(); + Tick(); + if (Storage.WorkChosen && !args.Contains("--work")) args.AddRange(new[] { "--work", Storage.Work }); // (the build files where the player chose) + if (dlss && !args.Contains("--fetch-ngx-sdk")) args.AddRange(new[] { "--fetch-ngx-sdk", "--accept-nvidia-license" }); + runner = new Runner(tools, args, workers, dlss ? Storage.NgxSdk : null); + runner.Line += l => Dispatcher.BeginInvoke(() => Line(l)); + runner.Finished += failure => Dispatcher.BeginInvoke(() => Ended(failure)); + runner.Start(); + } + + static readonly Regex FilesLine = new(@"^(\d+) files, "), FileLine = new(@"^\s+\S+\s+\d+ MB\s+\d+s"); + + void Line(string line) + { + if (line.StartsWith("@@stage ")) + { + var id = line.Split(' ')[1]; + if (current != null && rows.TryGetValue(current, out var was)) was.Done(); + current = id; + if (rows.TryGetValue(id, out var row)) row.Begin(); + if (id == "compile") compileClock.Restart(); + if (id == ShaderStep.Id) shadersBegan = true; + if (Weight.TryGetValue(id, out var w)) SetProgress(w.From); + } + else if (line.StartsWith("@@progress ")) + { + if (double.TryParse(line[11..], System.Globalization.NumberStyles.Float, System.Globalization.CultureInfo.InvariantCulture, out var f) && current != null + && Weight.TryGetValue(current, out var w)) + { + if (current == "compile") compiled = f; + SetProgress(w.From + w.Size * Math.Clamp(f, 0, 1)); + } + return; + } + else if (line.StartsWith("@@plan ") || line.StartsWith("@@file ") || line.StartsWith("@@eta ") || line.StartsWith("@@times ")) + { + Number(line); + return; + } + else if (line.StartsWith("@@done ")) finished = line[7..]; + else if (ShaderStep.Reason(line) is { } later) shadersLater = later; + else if (FilesLine.Match(line) is { Success: true } m) filesOf = int.Parse(m.Groups[1].Value); + else if (FileLine.IsMatch(line)) filesDone++; + if (current == "compile" && filesChanged < 0 && filesOf > 0 && rows.TryGetValue("compile", out var c)) + c.Small($"{filesDone} of {filesOf} files"); + AddLog(line); + } + + /// The compile timer's lines: build.py says which files this run really rebuilds and how long each took before (the + /// countdown), and make_game.py gives this build's times beside the last one's. Not shown in details: the compiler's + /// own lines there already carry each file's seconds. + void Number(string line) + { + var p = line.Split(' '); + switch (p[0]) + { + case "@@plan" when p.Length == 3 && int.TryParse(p[1], out var changed) && int.TryParse(p[2], out var total): + filesChanged = changed; + filesOf = total; + filesCompiled = 0; + ChangedFiles(); + break; + case "@@file" when p.Length == 4 && (p[3] is "new" or "again"): + filesCompiled++; + ChangedFiles(); + break; + case "@@eta" when p.Length == 2 && double.TryParse(p[1], System.Globalization.NumberStyles.Float, + System.Globalization.CultureInfo.InvariantCulture, out var eta): + compileEta = eta; + break; + case "@@times" when p.Length == 4: + builtIn = elapsed.Elapsed; + long.TryParse(p[2], out thisCompile); + long.TryParse(p[3], out previousTotal); + break; + } + } + + void ChangedFiles() + { + if (filesChanged < 0 || !rows.TryGetValue("compile", out var row)) return; + row.Small(filesChanged == 0 ? "Everything is already built" + : filesChanged >= filesOf ? $"{filesCompiled} of {filesChanged} files" + : $"{filesCompiled} of {filesChanged} changed files"); + } + + void AddLog(string line) + { + log.AppendLine(line); + if (log.Length > 600_000) log.Remove(0, log.Length - 400_000); + if (Log.Visibility == Visibility.Visible) + { + Log.AppendText(line + "\n"); + Log.ScrollToEnd(); + } + } + + void SetProgress(double f) + { + f = Math.Clamp(f, 0, 1); + RingText.Text = $"{(int)(f * 100)}%"; + UpdateBar.Value = f; + const double r = 55, c = 60; + if (f >= 0.999) { RingArc.Data = new EllipseGeometry(new Point(c, c), r, r); return; } + var a = f * 2 * Math.PI; + var end = new Point(c + r * Math.Sin(a), c - r * Math.Cos(a)); + var fig = new PathFigure { StartPoint = new Point(c, c - r), IsClosed = false }; + fig.Segments.Add(new ArcSegment(end, new Size(r, r), 0, f > 0.5, SweepDirection.Clockwise, true)); + RingArc.Data = f < 0.002 ? null : new PathGeometry(new[] { fig }); + } + + void Tick() + { + foreach (var r in rows.Values) r.Tick(); + if (runner == null) return; + var tail = updating ? ". Your current game stays as it is until the new one is complete." : " · your PC stays usable"; + if (current == "compile") + { + // (the timer of the compile itself, and the countdown build.py works out from what this PC measured before) + var left = compileEta >= 0 ? TimeLeft(compileEta) + : compiled > 0.03 ? TimeLeft(compileClock.Elapsed.TotalSeconds / compiled * (1 - compiled) + 60) + : "Estimating…"; + BuildNote.Text = $"Compiling {Clock(compileClock.Elapsed)} · {left}{tail}"; + return; + } + BuildNote.Text = (updating ? "Working…" : "Usually about 45 minutes") + tail; + } + + static string Clock(TimeSpan t) => $"{(int)t.TotalMinutes}:{t.Seconds:00}"; + + static string TimeLeft(double seconds) + { + if (seconds < 45) return "Less than a minute left"; + if (seconds < 90) return "About a minute left"; + var minutes = (int)Math.Round(seconds / 60); + return minutes >= 60 ? $"About {minutes / 60} h {minutes % 60} min left" : $"About {minutes} minutes left"; + } + + void Ended(string? failure) + { + clock.Stop(); + elapsed.Stop(); + runner = null; + if (finished.Length > 0) + { + foreach (var (id, r) in rows) + { + if (id != ShaderStep.Id) r.Done(); + else if (ShaderStep.Skipped(shadersBegan, shadersLater) is { } why) { r.Skipped(); r.Small(why); } + else r.Done(); + } + SetProgress(1); + ShowReady(); + return; + } + if (failure == "cancelled") + { + SetStep(0); + ShowPage(FilesPage); + return; + } + if (current != null && rows.TryGetValue(current, out var row)) row.Stopped(); + ShowProblem(failure ?? "The build tools stopped without saying why.", tools: failure?.Contains("antivirus", StringComparison.OrdinalIgnoreCase) == true); + } + + void LogToggled(object sender, RoutedEventArgs e) + { + var show = ShowLog.IsChecked == true; + Log.Visibility = show ? Visibility.Visible : Visibility.Collapsed; + StepCard.Visibility = show ? Visibility.Collapsed : Visibility.Visible; + ShowLog.Content = T(show ? S.HideDetails : S.ShowDetails); + if (show) + { + Log.Text = log.ToString(); + Log.ScrollToEnd(); + } + } + + void CancelBuild(object sender, RoutedEventArgs e) + { + download?.Cancel(); + runner?.Cancel(); + } + + // ================= 3: play ================= + + void ShowReady() + { + SetStep(2); + ReadyTitle.Text = updating ? "Your game is updated" : "Your game is ready"; + var compiledNote = thisCompile > 0 ? $" (compiled in {Span(TimeSpan.FromSeconds(thisCompile))})" : ""; + ReadyNote.Text = $"{(updating ? "Updated" : "Built")} in {Span(elapsed.Elapsed)}{compiledNote}{Against(builtIn ?? elapsed.Elapsed)} · {finished}"; + if (!updating && Environment.GetEnvironmentVariable("HOOKSHOT_UI_PREVIEW") == null) + Finder.Shortcuts(finished, DesktopTick.IsChecked == true, StartTick.IsChecked == true); + else if (updating && Environment.GetEnvironmentVariable("HOOKSHOT_UI_PREVIEW") == null) + { // (the ones the player has, onto the folder's launcher; the boxes say which there are, and ticking one makes it) + DesktopTick.IsChecked = File.Exists(Finder.DesktopLink); + StartTick.IsChecked = File.Exists(Finder.StartLink); + Finder.RefreshShortcuts(finished); + } + if (Finder.ShortcutError != null) AddLog("shortcut not made: " + Finder.ShortcutError); + ShowPage(ReadyPage); + _ = FreeAfterBuild(); + } + + // (issue: 5 GB and more left in %LOCALAPPDATA% and never said. Said here, with the way to let it go) + async Task ShowStorageUse() + { + StorageRow.Visibility = Visibility.Collapsed; + var (total, freeable) = await Task.Run(Storage.Measure); + if (total < 100L << 20) return; + // (both places when the build files are in a folder the player chose: the total counts both) + var where = Storage.WorkChosen ? $"{Storage.Base} and {Storage.Work}" : Storage.Base; + StorageNote.Text = freeable >= 100L << 20 + ? $"Hookshot Builder keeps {Storage.Gb(total)} on this PC ({where}). {Storage.Gb(freeable)} of it only makes the next " + + "update quicker and can be freed: your game keeps playing, and the next update then takes about as long as this build." + : $"Hookshot Builder keeps {Storage.Gb(total)} on this PC ({where}): the builder itself."; + FreeButton.Visibility = freeable >= 100L << 20 ? Visibility.Visible : Visibility.Collapsed; + FreeButton.IsEnabled = true; + FreeButton.Content = "Free up space"; + StorageRow.Visibility = Visibility.Visible; + } + + // (players asked for the space back: the translated code, the setup's and updater's downloads and older builder versions go as soon + // as the game is ready. The compiled objects stay (~1 GB): without them every update compiled all ~600 files again, an hour and + // more (issue 1287). Free up space lets them go too. HOOKSHOT_KEEP_BUILD_FILES=1 keeps everything) + async Task FreeAfterBuild() + { + if (!Storage.KeepBuildFiles) + { + var left = await Task.Run(() => Storage.Free(afterBuild: true)); + if (left >= 100L << 20) AddLog($"build files: {Storage.Gb(left)} could not be deleted (a file in use); they go after the next build"); + } + await ShowStorageUse(); + } + + async void FreeSpace(object sender, RoutedEventArgs e) + { + FreeButton.IsEnabled = false; + FreeButton.Content = "Freeing…"; + var left = await Task.Run(() => Storage.Free()); + await ShowStorageUse(); + if (left >= 100L << 20) + StorageNote.Text += $" ({Storage.Gb(left)} could not be deleted: a file is in use. Close the game and the builder, and delete the folder's contents by hand.)"; + } + + void ShowStorage(object sender, RoutedEventArgs e) => + Process.Start(new ProcessStartInfo("explorer.exe", $"\"{(Storage.WorkChosen ? Storage.Work : Storage.Base)}\"") { UseShellExecute = true }); // (the bigger one) + + void ShortcutsChanged(object sender, RoutedEventArgs e) + { + if (finished.Length > 0) Finder.Shortcuts(finished, DesktopTick.IsChecked == true, StartTick.IsChecked == true); + } + + static string Span(TimeSpan t) => t.TotalHours >= 1 ? $"{(int)t.TotalHours} h {t.Minutes} min" : t.TotalMinutes >= 1 ? $"{(int)t.TotalMinutes} minutes" : $"{t.Seconds} seconds"; + + /// This build beside the last one on this PC (build_times.json, through make_game.py's @@times): "last time 45 minutes + /// (4 minutes faster)" + string Against(TimeSpan now) + { + if (previousTotal <= 0) return ""; + var last = TimeSpan.FromSeconds(previousTotal); + var diff = last.TotalSeconds - now.TotalSeconds; + var how = Math.Abs(diff) < 30 ? "about the same" + : diff > 0 ? $"{Span(TimeSpan.FromSeconds(diff))} faster" : $"{Span(TimeSpan.FromSeconds(-diff))} slower"; + return $" · last time {Span(last)} ({how})"; + } + + void OpenFolder(object sender, RoutedEventArgs e) => + Process.Start(new ProcessStartInfo("explorer.exe", $"\"{finished}\"") { UseShellExecute = true }); + + void Play(object sender, RoutedEventArgs e) + { + StartGame(finished); + WindowState = WindowState.Minimized; + } + + // ================= when something goes wrong ================= + + bool problemIsTools; + + void ShowProblem(string why, bool tools) + { + lastFailure = why; + problemIsTools = tools; + ProblemSteps.Children.Clear(); + ProblemSub.Text = updating ? "Your current game is unchanged and still plays." : "Nothing was changed in your game folder."; + if (tools) + { + ProblemTitle.Text = "The build tools need repairing"; + ProblemHead.Text = "Your antivirus removed part of the build tools"; + ProblemText.Text = why + "\n\nAntivirus programs sometimes flag unsigned developer tools by mistake."; + var base_ = System.IO.Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "HookshotBuilder"); + foreach (var (n, s) in new[] + { + (1, "Open Windows Security → Protection history, and restore the removed files."), + (2, $"Or add {base_} to its exclusions."), + (3, "Then press Repair: Hookshot downloads the tools again from their official releases and checks each one."), + }) + ProblemSteps.Children.Add(new TextBlock { Text = $"{n}. {s}", Style = (Style)FindResource("Note"), FontSize = 14, Margin = new Thickness(0, 6, 0, 0) }); + } + else + { + ProblemTitle.Text = updating ? "The update stopped" : "The build stopped"; + ProblemHead.Text = "What went wrong"; + ProblemText.Text = why; + } + RepairButton.Visibility = tools ? Visibility.Visible : Visibility.Collapsed; + ProblemDetails.Content = "Show details"; + if (!updating) SetStep(1); // (the update keeps its versions in the head) + ShowPage(ProblemPage); + } + + void ProblemShowDetails(object sender, RoutedEventArgs e) + { + var lines = log.ToString().Split('\n'); + ProblemSteps.Children.Clear(); + ProblemSteps.Children.Add(new TextBox + { + Text = string.Join("\n", lines.Skip(Math.Max(0, lines.Length - 80))), IsReadOnly = true, FontFamily = (FontFamily)FindResource("Mono"), FontSize = 11, + Foreground = (Brush)FindResource("InkSoft"), Background = new SolidColorBrush(Color.FromArgb(0x33, 0, 0, 0)), BorderThickness = new Thickness(0), + Padding = new Thickness(8), TextWrapping = TextWrapping.NoWrap, HorizontalScrollBarVisibility = ScrollBarVisibility.Auto, + }); + } + + void CopyForReport(object sender, RoutedEventArgs e) + { + var lines = log.ToString().Split('\n'); + var text = $"Hookshot Builder {Tools.Find()?.Version ?? "(source)"} on Windows {Environment.OSVersion.Version}\n" + + $"{ProblemTitle.Text}: {lastFailure}\n\n" + string.Join("\n", lines.Skip(Math.Max(0, lines.Length - 200))); + try { Clipboard.SetText(text); CopyReport.Content = "Copied"; } catch (Exception) { } + } + + void Back(object sender, RoutedEventArgs e) + { + SetStep(0); + ShowPage(FilesPage); + } + + /// The setup (HookshotBuilder-Setup.exe beside this app) checks every part against its checksum and downloads the missing ones + void Repair(object sender, RoutedEventArgs e) + { + var setup = System.IO.Path.Combine(AppContext.BaseDirectory, "HookshotBuilder-Setup.exe"); + if (File.Exists(setup)) + { + Process.Start(new ProcessStartInfo(setup, "--repair") { UseShellExecute = false, WorkingDirectory = AppContext.BaseDirectory }); + Application.Current.Shutdown(); + return; + } + Process.Start(new ProcessStartInfo("https://github.com/NotRexed/Hookshot/releases/latest") { UseShellExecute = true }); + } + + // ================= the update (Updater.cs): the game started this app with --update ================= + + CancellationTokenSource? download; + + public void BeginUpdate(string folder, string? tag) + { + updating = true; + OutPath.Text = folder; + finished = ""; + var to = tag ?? "the latest version"; + var from = GameVersion(folder) ?? Tools.Find()?.Version; + StepPill.Visibility = Visibility.Collapsed; + VersionChip.Visibility = Visibility.Visible; + VersionChipText.Text = from != null && from != to ? $"{from} → {to}" : to; + prepareNow = BuilderSettings.PrepareNow(SettingsPath); + MakeRows(update: true, shaders: prepareNow); + BuildTitle.Text = "Updating your game"; + BuildNote.Text = "Getting the update…"; + RingBox.Visibility = Visibility.Collapsed; + UpdateBar.Visibility = Visibility.Visible; + SetProgress(0); + rows["download"].Begin(); + CancelButton.Visibility = Visibility.Visible; + ShowPage(BuildPage); + } + + public CancellationToken DownloadToken => (download = new CancellationTokenSource()).Token; + + public void DownloadProgress(double f) + { + rows.GetValueOrDefault("download")?.Small($"{(int)(f * 100)}%"); + SetProgress(Weight["download"].Size * f); + } + + public void Note(string line) + { + AddLog(line); + rows.GetValueOrDefault("download")?.Small(line); + } + + public void DownloadFailed(string why) + { + download = null; + rows.GetValueOrDefault("download")?.Stopped(); + ShowProblem(why, tools: false); + ProblemTitle.Text = "The update could not be downloaded"; + } + + /// The new tools are here: the folder built again with them (only what changed is compiled) + internal void UpdateBuild(Tools tools, string folder) + { + download = null; + if (rows.TryGetValue("download", out var d)) { d.Done(); d.Small("Build tools checked"); } + // (the saved tick also opts an existing game into DLSS. After a self-update ApplySettings has not run yet; + // a folder already built with DLSS keeps it, its license accepted then) + var dlss = BuilderSettings.Load(SettingsPath)?.Dlss == true || + (File.Exists(System.IO.Path.Combine(folder, "ngx", "nvngx_dlss.dll")) && + File.Exists(System.IO.Path.Combine(Storage.NgxSdk, "LICENSE.txt"))); + // (prepareNow: the saved choice, read in BeginUpdate. The launcher's Update and the --update start after a self-update never + // ran ApplySettings, so the tick there is only its default) + Run(tools, ShaderStep.Args(new List { "--update", folder }, prepareNow), keepRows: true, dlss: dlss); + } + + // ================= the window ================= + + void Minimize(object sender, RoutedEventArgs e) => WindowState = WindowState.Minimized; + + void CloseWindow(object sender, RoutedEventArgs e) + { + if (runner != null && MessageBox.Show(this, T(S.CloseQuestion), "Hookshot", MessageBoxButton.YesNo) != MessageBoxResult.Yes) return; + runner?.Cancel(); + Close(); + } +} + +/// One step of the build: its mark (done, now, waiting), its name, a line under it and its time. +sealed class StepRow +{ + public readonly Grid Root = new() { Margin = new Thickness(0, 10, 0, 10) }; + readonly Grid mark = new() { Width = 22, Height = 22, VerticalAlignment = VerticalAlignment.Top, Margin = new Thickness(0, 1, 0, 0) }; + readonly Ellipse disc = new(), halo = new() { Width = 32, Height = 32, Margin = new Thickness(-5) }; + readonly TextBlock number = new() { FontSize = 12, FontWeight = FontWeights.SemiBold, HorizontalAlignment = HorizontalAlignment.Center, VerticalAlignment = VerticalAlignment.Center }; + readonly Path tick = new() { Data = Geometry.Parse("M0,4 L3.5,7.5 L10,0"), StrokeThickness = 1.8, HorizontalAlignment = HorizontalAlignment.Center, VerticalAlignment = VerticalAlignment.Center }; + readonly TextBlock name = new() { FontWeight = FontWeights.SemiBold }; + readonly TextBlock small = new() { FontSize = 12, Visibility = Visibility.Collapsed, TextTrimming = TextTrimming.CharacterEllipsis }; + readonly TextBlock time = new() { FontSize = 12, Text = "—", VerticalAlignment = VerticalAlignment.Top, Margin = new Thickness(0, 2, 0, 0) }; + readonly Stopwatch watch = new(); + readonly int n; + bool running; + + static Brush B(string key) => (Brush)Application.Current.FindResource(key); + + public StepRow(int n, string text, bool last) + { + this.n = n; + Root.ColumnDefinitions.Add(new ColumnDefinition { Width = new GridLength(34) }); + Root.ColumnDefinitions.Add(new ColumnDefinition()); + Root.ColumnDefinitions.Add(new ColumnDefinition { Width = GridLength.Auto }); + mark.Children.Add(halo); + mark.Children.Add(disc); + mark.Children.Add(number); + mark.Children.Add(tick); + tick.Stroke = B("Go"); + time.FontFamily = (FontFamily)Application.Current.FindResource("Mono"); + var texts = new StackPanel { Children = { name, small } }; + Grid.SetColumn(texts, 1); + Grid.SetColumn(time, 2); + if (!last) + { + var link = new Rectangle { Width = 1, Fill = B("GlassLine"), HorizontalAlignment = HorizontalAlignment.Left, Margin = new Thickness(10.5, 28, 0, -19) }; + Root.Children.Add(link); + } + Root.Children.Add(mark); + Root.Children.Add(texts); + Root.Children.Add(time); + name.Text = text; + Wait(); + } + + public void Small(string text) + { + small.Text = text; + small.Visibility = Visibility.Visible; + } + + void Wait() + { + disc.Fill = Brushes.Transparent; + disc.Stroke = B("GlassLineStrong"); + disc.StrokeThickness = 1.5; + halo.Fill = Brushes.Transparent; + number.Text = ""; + tick.Visibility = Visibility.Collapsed; + name.Foreground = B("InkFaint"); + name.FontWeight = FontWeights.Medium; + small.Foreground = B("InkFaint"); + time.Foreground = B("InkFaint"); + } + + public void Begin() + { + running = true; + watch.Restart(); + disc.Fill = B("Brand"); + disc.Stroke = null; + halo.Fill = B("BrandSoft"); + number.Text = n.ToString(); + number.Foreground = Brushes.White; + tick.Visibility = Visibility.Collapsed; + name.Foreground = B("Ink"); + name.FontWeight = FontWeights.SemiBold; + time.Foreground = B("Ink"); + Tick(); + } + + public void Tick() + { + if (running) time.Text = $"{(int)watch.Elapsed.TotalMinutes}:{watch.Elapsed.Seconds:00}"; + } + + public void Done() + { + if (running) Tick(); + running = false; + watch.Stop(); + disc.Fill = B("GoSoft"); + disc.Stroke = null; + halo.Fill = Brushes.Transparent; + number.Text = ""; + tick.Visibility = Visibility.Visible; + name.Foreground = B("Ink"); + name.FontWeight = FontWeights.SemiBold; + time.Foreground = B("InkFaint"); + } + + /// (not done: left to later, the shaders' step when they were not prepared now) + public void Skipped() + { + if (running) Tick(); + running = false; + watch.Stop(); + disc.Fill = Brushes.Transparent; + disc.Stroke = B("GlassLineStrong"); + disc.StrokeThickness = 1.5; + halo.Fill = Brushes.Transparent; + number.Text = "–"; + number.Foreground = B("InkFaint"); + tick.Visibility = Visibility.Collapsed; + name.Foreground = B("InkFaint"); + name.FontWeight = FontWeights.Medium; + time.Foreground = B("InkFaint"); + } + + public void Stopped() + { + running = false; + watch.Stop(); + disc.Fill = B("BadSoft"); + halo.Fill = Brushes.Transparent; + number.Text = "!"; + number.Foreground = B("Bad"); + } +} diff --git a/builder/HookshotBuilder/Runner.cs b/builder/HookshotBuilder/Runner.cs new file mode 100644 index 0000000..b47a1a8 --- /dev/null +++ b/builder/HookshotBuilder/Runner.cs @@ -0,0 +1,184 @@ +using System.Diagnostics; +using System.IO; + +namespace HookshotBuilder; + +/// Where nx2/make_game.py and its Python are: a "tools" folder beside the app when packaged, or the source tree it sits in. +sealed record Tools(string Root, string Python, string[]? PythonArgs = null) +{ + /// Which release these tools are (tools/version.txt, written when the release is packed); null from the source tree + public string? Version => File.Exists(Path.Combine(Root, "version.txt")) ? File.ReadAllText(Path.Combine(Root, "version.txt")).Trim() : null; + + public static Tools? Find() + { + var at = AppContext.BaseDirectory; + var bundled = Path.Combine(at, "tools"); + if (File.Exists(Path.Combine(bundled, "nx2", "make_game.py"))) + { + var py = Path.Combine(bundled, "python", "python.exe"); + return new Tools(bundled, File.Exists(py) ? py : "python"); + } + // (the source tree the app was built in: its builder\ folder holds the app. Not any parent folder with an nx2\ + // folder, such as one another user of the PC made at the drive's root) + for (var d = new DirectoryInfo(at); d != null; d = d.Parent) + if (File.Exists(Path.Combine(d.FullName, "nx2", "make_game.py")) && + at.StartsWith(Path.Combine(d.FullName, "builder") + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase)) + { + var (exe, args) = PythonOnPath(); + return new Tools(d.FullName, exe, args); + } + return null; + } + + /// The Python for a source tree: Windows' Store shortcut is not an interpreter (it only opens the Store), so the py + /// launcher is tried when "python" is that. CONTRIBUTING: Python 3 from python.org, not the Microsoft Store build. + static (string Exe, string[]? Args) PythonOnPath() + { + var python = FirstOnPath("python.exe"); + if (python != null && !IsStoreShortcut(python)) return (python, null); + var py = FirstOnPath("py.exe"); + if (py != null) return (py, new[] { "-3" }); + var later = AllOnPath("python.exe").FirstOrDefault(p => !IsStoreShortcut(p)); + return (later ?? python ?? "python", null); + } + + static IEnumerable AllOnPath(string name) + { + foreach (var dir in (Environment.GetEnvironmentVariable("PATH") ?? "").Split(Path.PathSeparator)) + if (!string.IsNullOrWhiteSpace(dir)) + { + var path = Path.Combine(dir.Trim('"'), name); + if (File.Exists(path)) yield return path; + } + } + + static string? FirstOnPath(string name) => AllOnPath(name).FirstOrDefault(); + + static bool IsStoreShortcut(string path) => path.Contains(@"\Microsoft\WindowsApps\", StringComparison.OrdinalIgnoreCase); +} + +/// nx2/make_game.py as a child process: its lines as they come, and how it ended. +sealed class Runner +{ + readonly Tools tools; + readonly List args; + readonly int? workers; + readonly string? ngxSdk; + Process? process; + string? error; + bool cancelled; + + public event Action? Line; + public event Action? Finished; + + /// ngxSdk: NVIDIA DLSS (the builder's tick): the folder of NVIDIA's DLSS SDK, as HOOKSHOT_NGX_SDK; null builds without it + public Runner(Tools tools, List args, int? workers = null, string? ngxSdk = null) + { + this.tools = tools; + this.args = args; + if (workers is <= 0) throw new ArgumentOutOfRangeException(nameof(workers), "Build workers must be positive."); + this.workers = workers; + this.ngxSdk = ngxSdk; + } + + static ProcessStartInfo Info(Tools tools, IEnumerable args, int? workers = null, string? ngxSdk = null) + { + var si = new ProcessStartInfo(tools.Python) + { + WorkingDirectory = tools.Root, + UseShellExecute = false, + CreateNoWindow = true, + RedirectStandardOutput = true, + RedirectStandardError = true, + }; + si.Environment["PYTHONIOENCODING"] = "utf-8"; + BuildWorkers.Apply(si, workers); + // (written into the game folder's hookshot_version.txt: the game starts this app again to update, host/hos_update.c) + if (Environment.ProcessPath != null) si.Environment["HOOKSHOT_BUILDER_EXE"] = Environment.ProcessPath; + // (NVIDIA DLSS: only when ticked. Without it, one set on the PC is not passed on either: a build is with DLSS only when asked) + if (ngxSdk != null) si.Environment["HOOKSHOT_NGX_SDK"] = ngxSdk; + else si.Environment.Remove("HOOKSHOT_NGX_SDK"); + // (the folder's own compiler, headers and libraries, when it has them: nothing of the PC's but Windows' own) + var sets = new[] { ("EMSDK", "emsdk"), ("RX_CPM", "cpm"), ("WGPU_NATIVE", "wgpu-native") }; + foreach (var (name, folder) in sets) + { + var path = Path.Combine(tools.Root, folder); + if (Directory.Exists(path)) si.Environment[name] = path; + } + foreach (var a in tools.PythonArgs ?? Array.Empty()) si.ArgumentList.Add(a); + si.ArgumentList.Add("-I"); + si.ArgumentList.Add(Path.Combine(tools.Root, "nx2", "make_game.py")); + foreach (var a in args) si.ArgumentList.Add(a); + return si; + } + + public void Start() + { + Task.Run(() => + { + try + { + using var p = Process.Start(Info(tools, args, workers, ngxSdk))!; + process = p; + p.ErrorDataReceived += (_, e) => { if (e.Data != null) Line?.Invoke(e.Data); }; + p.BeginErrorReadLine(); + try { p.PriorityClass = ProcessPriorityClass.BelowNormal; } catch (Exception) { } + while (p.StandardOutput.ReadLine() is { } line) + { + if (line.StartsWith("@@error ")) error = line[8..]; + Line?.Invoke(line); + } + p.WaitForExit(); + Finished?.Invoke(cancelled ? "cancelled" : p.ExitCode == 0 ? null : error ?? "The build tools stopped without saying why. See the details."); + } + catch (Exception e) + { + Finished?.Invoke("The build tools could not start: " + e.Message); + } + }); + } + + public void Cancel() + { + cancelled = true; + try { process?.Kill(entireProcessTree: true); } catch (Exception) { } + } + + /// Is this the supported game and version? The answer in plain words. + public static async Task<(bool Ok, string Text)> Check(Tools tools, IEnumerable args) + { + try + { + using var p = Process.Start(Info(tools, args))!; + string? lastError = null; + p.ErrorDataReceived += (_, e) => { if (!string.IsNullOrWhiteSpace(e.Data)) lastError = e.Data; }; + p.BeginErrorReadLine(); + var result = (false, "The game files could not be read."); + var answered = false; + while (await p.StandardOutput.ReadLineAsync() is { } line) + { + if (line.StartsWith("@@check ok ")) { result = (true, line[11..]); answered = true; } + else if (line.StartsWith("@@check bad ")) { result = (false, line[12..]); answered = true; } + else if (line.StartsWith("@@error ")) { result = (false, line[8..]); answered = true; } + } + await p.WaitForExitAsync(); + // (a step that stopped before it could say why: the reason is on stderr, which used to be dropped here) + if (!answered) + { + if (lastError != null && lastError.Contains("Microsoft Store", StringComparison.OrdinalIgnoreCase)) + return (false, "Python 3 was not found: the 'python' command on this PC is the Microsoft Store shortcut, which only opens " + + "the Store. Install Python 3 from python.org (not the Store version), or use the full Hookshot Builder package."); + return (false, lastError ?? "The game tools stopped without saying why (exit code " + p.ExitCode + ")."); + } + return result; + } + catch (Exception) + { + // (the package's own Python missing: an antivirus took it, most often, after a false alarm on the download) + if (tools.Python == "python" && Directory.Exists(Path.Combine(tools.Root, "python"))) + return (false, "The builder's own Python (tools/python/python.exe) is missing. Your antivirus probably removed it: restore it " + + "from quarantine, or unpack the zip again into a folder your antivirus allows, then try again."); + return (false, "Python 3 was not found. Install it from python.org (not the Microsoft Store build), or use the full Hookshot Builder package."); + } + } +} diff --git a/builder/HookshotBuilder/ShaderStep.cs b/builder/HookshotBuilder/ShaderStep.cs new file mode 100644 index 0000000..e906f00 --- /dev/null +++ b/builder/HookshotBuilder/ShaderStep.cs @@ -0,0 +1,30 @@ +using System.Text.RegularExpressions; + +namespace HookshotBuilder; + +/// The build's last step, the shaders' preparation (nx2/make_game.py prepare_shaders): the argument that leaves it to the game's +/// first start, and what its row says once the build is done +static class ShaderStep +{ + public const string Id = "shaders"; + public const string Later = "--no-prepare-shaders"; + // (make_game.py NOT_PREPARED: a plain line, so a builder that does not know it only shows it) + static readonly Regex NotPrepared = new(@"^the shaders were not prepared now \((.*)\): "); + + /// make_game's arguments with the player's choice (unticked: the game prepares its shaders at its first start) + public static List Args(List args, bool prepare) + { + if (!prepare) args.Add(Later); + return args; + } + + /// Why make_game left the shaders to the first start, when this line says so; else null + public static string? Reason(string line) => NotPrepared.Match(line) is { Success: true } m ? m.Groups[1].Value : null; + + /// The line under the row of a finished build when the shaders were not prepared, or null when they were (a check mark). + /// began: the game's @@stage shaders came; reason: make_game's not-prepared line + public static string? Skipped(bool began, string? reason) => + reason != null ? $"Left to the game's first start: {reason}" + : began ? null + : "Left to the game's first start"; +} diff --git a/builder/HookshotBuilder/Storage.cs b/builder/HookshotBuilder/Storage.cs new file mode 100644 index 0000000..8c8f2f7 --- /dev/null +++ b/builder/HookshotBuilder/Storage.cs @@ -0,0 +1,171 @@ +using System.IO; +using System.Runtime.InteropServices; +using Microsoft.Win32.SafeHandles; + +namespace HookshotBuilder; + +/// What the builder keeps on the PC, and letting it go. In %LOCALAPPDATA%\HookshotBuilder: the builder itself (versions\: the one +/// installed last, and the one before it), the setup's downloaded parts (the next builder update downloads only those that changed). +/// In the build files folder (that same folder, or one the player chose: make_game.py --work): the translated code and its compiled +/// objects (sa\, objcache\: the next update compiles only what changed). None of it is needed to play: the game folder holds all the +/// game uses, and its copy of the game files (.hookshot_exefs) is what updates build from +static class Storage +{ + public static string Base => Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "HookshotBuilder"); + + /// About what a build leaves in the build files folder (the translated code and its compiled objects): said before the build + public const long BuildFilesBytes = 4L << 30; + + /// HOOKSHOT_KEEP_BUILD_FILES=1: the build files stay after a build (quick updates); otherwise they go when the game is ready + public static bool KeepBuildFiles => Environment.GetEnvironmentVariable("HOOKSHOT_KEEP_BUILD_FILES") is { Length: > 0 } v && v != "0"; + + /// Where builds keep their files: the folder the player chose (work.txt), else the builder's own + public static string Work + { + get + { + try + { + var chosen = File.ReadAllText(Path.Combine(Base, "work.txt")).Trim(); + if (chosen.Length > 2 && Path.IsPathFullyQualified(chosen)) return chosen; + } + catch (Exception e) when (e is IOException or UnauthorizedAccessException) { } + return Base; + } + } + + public static bool WorkChosen => !Same(Work, Base); + + /// (NVIDIA DLSS, only when ticked) NVIDIA's DLSS SDK, downloaded under NVIDIA's license by the build (make_game.py + /// --fetch-ngx-sdk, scripts/ngx_sdk.py): HOOKSHOT_NGX_SDK. NVIDIA's files, never part of the builder or its downloads + public static string NgxSdk => Path.Combine(Base, "ngx-sdk"); + + /// The build files go to a folder of their own in the one the player picked (null: back to the builder's own). What the last + /// builds left at the old place goes: the next build would not look for it there + public static void MoveWork(string? picked) + { + var old = Work; + var to = picked == null ? Base : Path.Combine(picked, "Hookshot build files"); + if (Same(old, to)) return; + Directory.CreateDirectory(to); + Directory.CreateDirectory(Base); + if (picked == null) File.Delete(Path.Combine(Base, "work.txt")); + else File.WriteAllText(Path.Combine(Base, "work.txt"), to); + foreach (var d in WorkParts(old)) Delete(d); + if (!Same(old, Base)) + try { Directory.Delete(old); } catch (Exception e) when (e is IOException or UnauthorizedAccessException) { } // (only when empty) + } + + /// What a build makes in its work folder (make_game.py, lift.py, build.py) + static IEnumerable WorkParts(string work) => + new[] { "objcache", "sa", "exefs", "probe", "image", "system_headers.txt", "hot_funcs.txt", "hot_order.txt", "hookshot_pieces.prof", "hookshot.res" } + .Select(n => Path.Combine(work, n)); + + /// The builder versions that stay: the one installed last (current.txt) and the one running + static bool Kept(string versionDir) + { + try + { + if (File.ReadAllText(Path.Combine(Base, "versions", "current.txt")).Trim() == Path.GetFileName(versionDir)) return true; + } + catch (Exception e) when (e is IOException or UnauthorizedAccessException) { } + var running = Path.GetFullPath(AppContext.BaseDirectory).TrimEnd('\\') + "\\"; + return running.StartsWith(Path.GetFullPath(versionDir).TrimEnd('\\') + "\\", StringComparison.OrdinalIgnoreCase); + } + + /// What Free lets go. afterBuild: what goes once the game is ready, the compiled objects of the build (objcache, which make_game.py + /// prunes to them) and their system headers' stamp kept, so that the next update compiles only what changed (issue 1287: every + /// update compiled all ~600 files again, an hour and more). The Free up space button lets those go too + static IEnumerable Freeable(bool afterBuild = false) + { + foreach (var d in afterBuild ? new[] { "sa", "exefs", "probe" } : new[] { "objcache", "sa", "exefs", "probe" }) + yield return Path.Combine(Work, d); + foreach (var d in new[] { "downloads", "update" }) + yield return Path.Combine(Base, d); + var versions = Path.Combine(Base, "versions"); + if (Directory.Exists(versions)) + foreach (var v in Directory.EnumerateDirectories(versions)) + if (!Kept(v)) yield return v; + } + + /// Bytes in the builder's folder and the build files folder, and in what Free lets go (an object hard-linked in both sa\build and + /// objcache counted once) + public static (long total, long freeable) Measure() + { + var seen = new HashSet<(uint, ulong)>(); + long freeable = 0; + foreach (var d in Freeable()) freeable += Size(d, seen); + seen.Clear(); + var total = Size(Base, seen) + (WorkChosen ? Size(Work, seen) : 0); + return (Math.Max(total, freeable), freeable); + } + + /// Lets go of what Measure counts as freeable; returns the bytes it could not delete (a file in use) + public static long Free(bool afterBuild = false) + { + foreach (var d in Freeable(afterBuild)) Delete(d); + if (!afterBuild) Delete(Path.Combine(Work, "system_headers.txt")); // (the stamp of the compiled objects' system headers goes with them) + if (!afterBuild) return Measure().freeable; + var seen = new HashSet<(uint, ulong)>(); // (what of the after-build set is still there: a file in use) + long left = 0; + foreach (var d in Freeable(afterBuild: true)) left += Size(d, seen); + return left; + } + + public static string Gb(long bytes) => bytes >= 1L << 30 ? $"{bytes / (double)(1L << 30):0.0} GB" : $"{bytes >> 20} MB"; + + static bool Same(string a, string b) => + string.Equals(Path.GetFullPath(a).TrimEnd('\\'), Path.GetFullPath(b).TrimEnd('\\'), StringComparison.OrdinalIgnoreCase); + + static void Delete(string path) + { + try + { + if (Directory.Exists(path)) Directory.Delete(path, recursive: true); + else if (File.Exists(path)) File.Delete(path); + } + catch (Exception e) when (e is IOException or UnauthorizedAccessException) { } + } + + static long Size(string dir, HashSet<(uint, ulong)> seen) + { + if (!Directory.Exists(dir)) return 0; + long n = 0; + IEnumerable files; + try + { + files = new DirectoryInfo(dir).EnumerateFiles("*", new EnumerationOptions { RecurseSubdirectories = true, IgnoreInaccessible = true }); + } + catch (Exception e) when (e is IOException or UnauthorizedAccessException) { return 0; } + foreach (var f in files) + { + try + { + if (f.Length == 0) continue; + if (Identity(f.FullName) is { } id && !seen.Add(id)) continue; + n += f.Length; + } + catch (Exception e) when (e is IOException or UnauthorizedAccessException) { } + } + return n; + } + + [StructLayout(LayoutKind.Sequential, Pack = 4)] // (BY_HANDLE_FILE_INFORMATION: its FILETIMEs are two DWORDs, no padding) + struct FileInfoByHandle + { + public uint Attributes; + public long Created, Accessed, Written; + public uint Volume, SizeHigh, SizeLow, Links, IndexHigh, IndexLow; + } + + [DllImport("kernel32.dll", SetLastError = true)] + static extern bool GetFileInformationByHandle(SafeFileHandle h, out FileInfoByHandle info); + + /// The file's volume and index (the same for each of its hard links); null for one with a single link, which needs no check + static (uint, ulong)? Identity(string path) + { + using var h = File.OpenHandle(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete); + if (!GetFileInformationByHandle(h, out var i) || i.Links < 2) return null; + return (i.Volume, ((ulong)i.IndexHigh << 32) | i.IndexLow); + } +} diff --git a/builder/HookshotBuilder/Strings.cs b/builder/HookshotBuilder/Strings.cs new file mode 100644 index 0000000..4a95447 --- /dev/null +++ b/builder/HookshotBuilder/Strings.cs @@ -0,0 +1,174 @@ +using System.Globalization; + +namespace HookshotBuilder; + +/// One of the game's languages: its code in language.txt, its own name, and the app's text in it. +sealed record Language(string Code, string Name, string[] Text) +{ + public override string ToString() => Name; +} + +enum S +{ + Subtitle, GameFiles, Nup, Folders, DropHint, SaveTo, Build, Browse, Needs, Free, Checking, VersionOk, VersionBad, ToolsMissing, PythonMissing, + Building, Patience, Elapsed, Read, Lift, Compile, Finish, ShowDetails, HideDetails, Cancel, Back, OpenFolder, Play, Ready, BuiltIn, + Stopped, Cancelled, CancelledNote, CloseQuestion, LanguageLabel, + // (the update, English only for now: Strings.Text falls back to it) + Download, Updating, UpdateFailed, +} + +static class Strings +{ + static readonly string[] En = + { + "Turns your own copy of Ocarina of Time into a game folder you can start on PC.", "Game files", ".nup file", "romfs and exefs", + "You can also drop a file or folder onto this window.", "Save the game to", "Build", "Browse", "Takes about 45 minutes and needs 45 GB.", + "{0} has {1} GB free.", "Checking the game version...", "This is the version Hookshot supports.", "This version isn't supported. Hookshot needs build e0f3ccfa.", + "The Hookshot tools were not found next to this app.", "Python was not found.", + "Building", "This takes a while, and your PC stays usable.", "Elapsed", "Reading your game files", "Translating the game to PC code", "Compiling", + "Putting the game folder together", "Show details", "Hide details", "Cancel", "Back", "Open folder", "Play", "Your game is ready", "built in", + "The build stopped", "Build cancelled", "You can start again from the same folder.", "Stop the build and close?", "Language", + "Downloading the new version", "Updating your game", "The update could not be downloaded", + }; + static readonly string[] Fr = + { + "Transforme votre copie d'Ocarina of Time en un dossier de jeu à lancer sur PC.", "Fichiers du jeu", "Fichier .nup", "romfs et exefs", + "Vous pouvez aussi déposer un fichier ou un dossier sur cette fenêtre.", "Enregistrer le jeu dans", "Créer", "Parcourir", "Prend environ 45 minutes et nécessite 45 Go.", + "{0} : {1} Go libres.", "Vérification de la version du jeu...", "Cette version est prise en charge par Hookshot.", "Cette version n'est pas prise en charge. Hookshot a besoin de la version e0f3ccfa.", + "Les outils Hookshot sont introuvables à côté de cette application.", "Python est introuvable.", + "Création en cours", "Cela prend du temps, et votre PC reste utilisable.", "Temps écoulé", "Lecture de vos fichiers de jeu", "Traduction du jeu en code PC", "Compilation", + "Assemblage du dossier du jeu", "Afficher les détails", "Masquer les détails", "Annuler", "Retour", "Ouvrir le dossier", "Jouer", "Votre jeu est prêt", "créé en", + "La création s'est arrêtée", "Création annulée", "Vous pouvez recommencer depuis le même dossier.", "Arrêter la création et fermer ?", "Langue", + }; + static readonly string[] De = + { + "Macht aus deiner eigenen Kopie von Ocarina of Time einen Spielordner für den PC.", "Spieldateien", ".nup-Datei", "romfs und exefs", + "Du kannst auch eine Datei oder einen Ordner auf dieses Fenster ziehen.", "Spiel speichern unter", "Erstellen", "Durchsuchen", "Dauert etwa 45 Minuten und benötigt 45 GB.", + "{0} hat {1} GB frei.", "Spielversion wird geprüft...", "Diese Version wird von Hookshot unterstützt.", "Diese Version wird nicht unterstützt. Hookshot benötigt Build e0f3ccfa.", + "Die Hookshot-Werkzeuge wurden neben dieser App nicht gefunden.", "Python wurde nicht gefunden.", + "Wird erstellt", "Das dauert eine Weile, dein PC bleibt nutzbar.", "Vergangen", "Spieldateien werden gelesen", "Spiel wird in PC-Code übersetzt", "Kompilieren", + "Spielordner wird zusammengestellt", "Details anzeigen", "Details ausblenden", "Abbrechen", "Zurück", "Ordner öffnen", "Spielen", "Dein Spiel ist fertig", "erstellt in", + "Die Erstellung wurde angehalten", "Erstellung abgebrochen", "Du kannst im selben Ordner neu beginnen.", "Erstellung beenden und schließen?", "Sprache", + }; + static readonly string[] It = + { + "Trasforma la tua copia di Ocarina of Time in una cartella di gioco da avviare su PC.", "File di gioco", "File .nup", "romfs ed exefs", + "Puoi anche trascinare un file o una cartella su questa finestra.", "Salva il gioco in", "Crea", "Sfoglia", "Richiede circa 45 minuti e 45 GB.", + "{0} ha {1} GB liberi.", "Controllo della versione del gioco...", "Questa versione è supportata da Hookshot.", "Questa versione non è supportata. Hookshot richiede la build e0f3ccfa.", + "Gli strumenti di Hookshot non sono stati trovati accanto a questa app.", "Python non è stato trovato.", + "Creazione in corso", "Ci vuole un po', e il PC resta utilizzabile.", "Trascorso", "Lettura dei file di gioco", "Traduzione del gioco in codice PC", "Compilazione", + "Assemblaggio della cartella di gioco", "Mostra dettagli", "Nascondi dettagli", "Annulla", "Indietro", "Apri cartella", "Gioca", "Il tuo gioco è pronto", "creato in", + "La creazione si è interrotta", "Creazione annullata", "Puoi ricominciare dalla stessa cartella.", "Interrompere la creazione e chiudere?", "Lingua", + }; + static readonly string[] Es = + { + "Convierte tu copia de Ocarina of Time en una carpeta de juego para iniciar en PC.", "Archivos del juego", "Archivo .nup", "romfs y exefs", + "También puedes soltar un archivo o una carpeta en esta ventana.", "Guardar el juego en", "Crear", "Examinar", "Tarda unos 45 minutos y necesita 45 GB.", + "{0} tiene {1} GB libres.", "Comprobando la versión del juego...", "Hookshot admite esta versión.", "Esta versión no es compatible. Hookshot necesita la build e0f3ccfa.", + "No se encontraron las herramientas de Hookshot junto a esta aplicación.", "No se encontró Python.", + "Creando", "Tarda un rato, y tu PC sigue siendo utilizable.", "Transcurrido", "Leyendo tus archivos del juego", "Traduciendo el juego a código de PC", "Compilando", + "Preparando la carpeta del juego", "Mostrar detalles", "Ocultar detalles", "Cancelar", "Atrás", "Abrir carpeta", "Jugar", "Tu juego está listo", "creado en", + "La creación se detuvo", "Creación cancelada", "Puedes empezar de nuevo desde la misma carpeta.", "¿Detener la creación y cerrar?", "Idioma", + }; + static readonly string[] Pt = + { + "Transforma sua cópia de Ocarina of Time em uma pasta de jogo para iniciar no PC.", "Arquivos do jogo", "Arquivo .nup", "romfs e exefs", + "Você também pode soltar um arquivo ou uma pasta nesta janela.", "Salvar o jogo em", "Criar", "Procurar", "Leva cerca de 45 minutos e precisa de 45 GB.", + "{0} tem {1} GB livres.", "Verificando a versão do jogo...", "Esta versão é compatível com o Hookshot.", "Esta versão não é compatível. O Hookshot precisa da build e0f3ccfa.", + "As ferramentas do Hookshot não foram encontradas ao lado deste aplicativo.", "O Python não foi encontrado.", + "Criando", "Isso leva um tempo, e seu PC continua utilizável.", "Decorrido", "Lendo seus arquivos do jogo", "Traduzindo o jogo para código de PC", "Compilando", + "Montando a pasta do jogo", "Mostrar detalhes", "Ocultar detalhes", "Cancelar", "Voltar", "Abrir pasta", "Jogar", "Seu jogo está pronto", "criado em", + "A criação parou", "Criação cancelada", "Você pode recomeçar a partir da mesma pasta.", "Parar a criação e fechar?", "Idioma", + }; + static readonly string[] Nl = + { + "Maakt van je eigen exemplaar van Ocarina of Time een spelmap om op pc te starten.", "Spelbestanden", ".nup-bestand", "romfs en exefs", + "Je kunt ook een bestand of map op dit venster slepen.", "Spel opslaan in", "Maken", "Bladeren", "Duurt ongeveer 45 minuten en vereist 45 GB.", + "{0} heeft {1} GB vrij.", "Spelversie controleren...", "Deze versie wordt door Hookshot ondersteund.", "Deze versie wordt niet ondersteund. Hookshot heeft build e0f3ccfa nodig.", + "De Hookshot-tools zijn niet naast deze app gevonden.", "Python is niet gevonden.", + "Bezig met maken", "Dit duurt even, en je pc blijft bruikbaar.", "Verstreken", "Je spelbestanden lezen", "Spel vertalen naar pc-code", "Compileren", + "Spelmap samenstellen", "Details tonen", "Details verbergen", "Annuleren", "Terug", "Map openen", "Spelen", "Je spel is klaar", "gemaakt in", + "Het maken is gestopt", "Maken geannuleerd", "Je kunt opnieuw beginnen vanuit dezelfde map.", "Maken stoppen en sluiten?", "Taal", + }; + static readonly string[] Pl = + { + "Zamienia Twoją kopię Ocarina of Time w folder gry, który uruchomisz na PC.", "Pliki gry", "Plik .nup", "romfs i exefs", + "Możesz też upuścić plik lub folder na to okno.", "Zapisz grę w", "Utwórz", "Przeglądaj", "Trwa około 45 minut i wymaga 45 GB.", + "{0}: wolne {1} GB.", "Sprawdzanie wersji gry...", "Ta wersja jest obsługiwana przez Hookshot.", "Ta wersja nie jest obsługiwana. Hookshot wymaga kompilacji e0f3ccfa.", + "Nie znaleziono narzędzi Hookshot obok tej aplikacji.", "Nie znaleziono Pythona.", + "Tworzenie", "To chwilę potrwa, a komputer pozostanie w użyciu.", "Upłynęło", "Odczytywanie plików gry", "Tłumaczenie gry na kod PC", "Kompilowanie", + "Składanie folderu gry", "Pokaż szczegóły", "Ukryj szczegóły", "Anuluj", "Wstecz", "Otwórz folder", "Graj", "Twoja gra jest gotowa", "utworzono w", + "Tworzenie zostało przerwane", "Anulowano tworzenie", "Możesz zacząć ponownie z tego samego folderu.", "Przerwać tworzenie i zamknąć?", "Język", + }; + static readonly string[] Ru = + { + "Превращает вашу копию Ocarina of Time в папку с игрой для запуска на ПК.", "Файлы игры", "Файл .nup", "romfs и exefs", + "Также можно перетащить файл или папку в это окно.", "Сохранить игру в", "Собрать", "Обзор", "Занимает около 45 минут и требует 45 ГБ.", + "{0}: свободно {1} ГБ.", "Проверка версии игры...", "Эта версия поддерживается Hookshot.", "Эта версия не поддерживается. Hookshot нужна сборка e0f3ccfa.", + "Инструменты Hookshot не найдены рядом с этим приложением.", "Python не найден.", + "Сборка", "Это займёт время, а компьютером можно пользоваться.", "Прошло", "Чтение файлов игры", "Перевод игры в код для ПК", "Компиляция", + "Подготовка папки с игрой", "Показать подробности", "Скрыть подробности", "Отмена", "Назад", "Открыть папку", "Играть", "Ваша игра готова", "собрано за", + "Сборка остановлена", "Сборка отменена", "Можно начать заново из той же папки.", "Остановить сборку и закрыть?", "Язык", + }; + static readonly string[] Ja = + { + "お持ちの Ocarina of Time を、PC で起動できるゲームフォルダーに変換します。", "ゲームファイル", ".nup ファイル", "romfs と exefs", + "ファイルやフォルダーをこのウィンドウにドロップすることもできます。", "ゲームの保存先", "ビルド", "参照", "約 45 分かかり、45 GB 必要です。", + "{0} の空き容量: {1} GB", "ゲームのバージョンを確認しています...", "このバージョンは Hookshot に対応しています。", "このバージョンには対応していません。Hookshot にはビルド e0f3ccfa が必要です。", + "このアプリの横に Hookshot ツールが見つかりません。", "Python が見つかりません。", + "ビルド中", "時間がかかりますが、PC はそのまま使えます。", "経過時間", "ゲームファイルを読み込んでいます", "ゲームを PC 用コードに変換しています", "コンパイル中", + "ゲームフォルダーを作成しています", "詳細を表示", "詳細を隠す", "キャンセル", "戻る", "フォルダーを開く", "プレイ", "ゲームの準備ができました", "所要時間", + "ビルドが停止しました", "ビルドをキャンセルしました", "同じフォルダーからやり直せます。", "ビルドを中止して閉じますか?", "言語", + }; + static readonly string[] Ko = + { + "가지고 있는 Ocarina of Time을 PC에서 실행할 수 있는 게임 폴더로 만듭니다.", "게임 파일", ".nup 파일", "romfs 및 exefs", + "파일이나 폴더를 이 창에 끌어다 놓을 수도 있습니다.", "게임 저장 위치", "빌드", "찾아보기", "약 45분이 걸리며 45GB가 필요합니다.", + "{0} 여유 공간: {1}GB", "게임 버전을 확인하는 중...", "Hookshot이 지원하는 버전입니다.", "지원되지 않는 버전입니다. Hookshot에는 빌드 e0f3ccfa가 필요합니다.", + "이 앱 옆에서 Hookshot 도구를 찾을 수 없습니다.", "Python을 찾을 수 없습니다.", + "빌드 중", "시간이 걸리지만 PC는 계속 사용할 수 있습니다.", "경과 시간", "게임 파일을 읽는 중", "게임을 PC 코드로 변환하는 중", "컴파일 중", + "게임 폴더를 만드는 중", "자세히 보기", "자세히 숨기기", "취소", "뒤로", "폴더 열기", "플레이", "게임이 준비되었습니다", "소요 시간", + "빌드가 중단되었습니다", "빌드가 취소되었습니다", "같은 폴더에서 다시 시작할 수 있습니다.", "빌드를 중단하고 닫을까요?", "언어", + }; + static readonly string[] ZhHans = + { + "将你自己的 Ocarina of Time 变成可在电脑上启动的游戏文件夹。", "游戏文件", ".nup 文件", "romfs 和 exefs", + "也可以将文件或文件夹拖放到此窗口。", "游戏保存位置", "生成", "浏览", "大约需要 45 分钟和 45 GB 空间。", + "{0} 可用空间 {1} GB。", "正在检查游戏版本...", "Hookshot 支持此版本。", "不支持此版本。Hookshot 需要构建版本 e0f3ccfa。", + "在此应用旁边找不到 Hookshot 工具。", "找不到 Python。", + "正在生成", "这需要一些时间,期间电脑仍可正常使用。", "已用时间", "正在读取游戏文件", "正在将游戏转换为电脑代码", "正在编译", + "正在整理游戏文件夹", "显示详细信息", "隐藏详细信息", "取消", "返回", "打开文件夹", "开始游戏", "你的游戏已准备就绪", "用时", + "生成已停止", "生成已取消", "你可以从同一文件夹重新开始。", "要停止生成并关闭吗?", "语言", + }; + static readonly string[] ZhHant = + { + "將你自己的 Ocarina of Time 變成可在電腦上啟動的遊戲資料夾。", "遊戲檔案", ".nup 檔案", "romfs 和 exefs", + "也可以將檔案或資料夾拖放到此視窗。", "遊戲儲存位置", "建置", "瀏覽", "大約需要 45 分鐘和 45 GB 空間。", + "{0} 可用空間 {1} GB。", "正在檢查遊戲版本...", "Hookshot 支援此版本。", "不支援此版本。Hookshot 需要建置版本 e0f3ccfa。", + "在此應用程式旁找不到 Hookshot 工具。", "找不到 Python。", + "正在建置", "這需要一些時間,期間電腦仍可正常使用。", "已用時間", "正在讀取遊戲檔案", "正在將遊戲轉換為電腦程式碼", "正在編譯", + "正在整理遊戲資料夾", "顯示詳細資訊", "隱藏詳細資訊", "取消", "返回", "開啟資料夾", "開始遊戲", "你的遊戲已準備就緒", "用時", + "建置已停止", "建置已取消", "你可以從同一資料夾重新開始。", "要停止建置並關閉嗎?", "語言", + }; + + public static readonly Language[] All = + { + new("en-US", "English (US)", En), new("en-GB", "English (UK)", En), new("fr", "Français", Fr), new("fr-CA", "Français (Canada)", Fr), + new("de", "Deutsch", De), new("it", "Italiano", It), new("es", "Español (España)", Es), new("es-419", "Español (Latinoamérica)", Es), + new("pt-BR", "Português (Brasil)", Pt), new("nl", "Nederlands", Nl), new("pl", "Polski", Pl), new("ru", "Русский", Ru), + new("ja", "日本語", Ja), new("ko", "한국어", Ko), new("zh-Hans", "简体中文", ZhHans), new("zh-Hant", "繁體中文", ZhHant), + }; + + /// The language closest to Windows' own: the exact one, then the same family, else English (US). + public static Language Default() + { + var c = CultureInfo.CurrentUICulture; + var name = c.Name; + var pick = All.FirstOrDefault(l => l.Code.Equals(name, StringComparison.OrdinalIgnoreCase)); + if (pick != null) return pick; + if (name.StartsWith("zh")) return All.First(l => l.Code == (name.Contains("TW") || name.Contains("HK") || name.Contains("Hant") ? "zh-Hant" : "zh-Hans")); + if (name.StartsWith("es-") && name != "es-ES") return All.First(l => l.Code == "es-419"); + if (name.StartsWith("pt")) return All.First(l => l.Code == "pt-BR"); + return All.FirstOrDefault(l => l.Code == c.TwoLetterISOLanguageName) ?? All[0]; + } +} diff --git a/builder/HookshotBuilder/Updater.cs b/builder/HookshotBuilder/Updater.cs new file mode 100644 index 0000000..7c2c814 --- /dev/null +++ b/builder/HookshotBuilder/Updater.cs @@ -0,0 +1,349 @@ +using System.Diagnostics; +using System.IO; +using System.IO.Compression; +using System.Net.Http; +using System.Text.Json; +using System.Windows; + +namespace HookshotBuilder; + +/// The game's update offer (host/hos_update.c) starts this app with --update --tag . When these tools are +/// not that release, the release's HookshotBuilder.zip is downloaded and unpacked beside this app, and the new app is started with +/// --fetched to build the folder; when they are, the folder is built again here (nx2/make_game.py --update: from the game files the +/// folder keeps). +static class Updater +{ + const string Latest = "https://api.github.com/repos/NotRexed/Hookshot/releases/latest"; + const string Asset = "HookshotBuilder.zip"; + const string SetupAsset = "HookshotBuilder.exe"; // (the setup, builder/setup/setup.c: a release holds it and its list) + const string SetupList = "hookshot-setup.txt"; // (scripts/make_setup_parts.py) + static readonly TimeSpan[] RetryDelays = { TimeSpan.FromSeconds(3), TimeSpan.FromSeconds(6), TimeSpan.FromSeconds(12), TimeSpan.FromSeconds(20) }; + + public static void FromCommandLine(MainWindow w) + { + var a = Environment.GetCommandLineArgs(); + string? folder = null, tag = null; + var fetched = false; + for (var i = 1; i < a.Length; i++) + { + if (a[i] == "--update" && i + 1 < a.Length) folder = a[++i]; + else if (a[i] == "--tag" && i + 1 < a.Length) tag = a[++i]; + else if (a[i] == "--fetched") fetched = true; + } + if (folder == null) return; + UpdateFolder(w, folder, tag, fetched); + } + + /// The latest release's tag, quietly (null: GitHub did not answer); a few tries, 20 s at most + public static async Task LatestTag() + { + using var http = new HttpClient { Timeout = TimeSpan.FromSeconds(8) }; + http.DefaultRequestHeaders.UserAgent.ParseAdd("HookshotBuilder"); + for (var attempt = 0; attempt < 3; attempt++) + { + try + { + using var request = new HttpRequestMessage(HttpMethod.Get, Latest); + request.Headers.Accept.ParseAdd("application/vnd.github+json"); + using var response = await http.SendAsync(request); + if (response.IsSuccessStatusCode) + { + using var doc = JsonDocument.Parse(await response.Content.ReadAsStringAsync()); + return doc.RootElement.GetProperty("tag_name").GetString(); + } + if ((int)response.StatusCode < 500) break; // (the API's limit, 60 an hour a PC: the list below instead) + } + catch (Exception) { } + await Task.Delay(TimeSpan.FromSeconds(2 << attempt)); + } + // (v2.0.0: when the API did not answer, the builder said a game was up to date. The release's setup list names its tag in + // its first line, from GitHub's download service, not the API) + for (var attempt = 0; attempt < 2; attempt++) + { + try + { + var list = await http.GetStringAsync("https://github.com/NotRexed/Hookshot/releases/latest/download/" + SetupList); + foreach (var line in list.Split('\n')) + if (line.StartsWith("tag ")) return line[4..].Trim(); + } + catch (Exception) { } + await Task.Delay(TimeSpan.FromSeconds(2)); + } + return null; + } + + /// The latest release's setup (HookshotBuilder.exe) downloaded and started with args: it installs the new builder (only the + /// parts that changed) and starts it with the same args; this one closes. (no args: the new builder opens as it does) + public static async Task RunLatestSetup(MainWindow w, IEnumerable args) + { + var token = w.DownloadToken; + using var http = new HttpClient { Timeout = Timeout.InfiniteTimeSpan }; + http.DefaultRequestHeaders.UserAgent.ParseAdd("HookshotBuilder"); + w.Note("Asking GitHub for the latest release"); + string? setupUrl = null, setupApiUrl = null; + long setupSize = 0; + using (var response = await GetWithRetries(w, http, Latest, token, "application/vnd.github+json", TimeSpan.FromSeconds(30))) + using (var doc = JsonDocument.Parse(await response.Content.ReadAsStringAsync(token))) + foreach (var asset in doc.RootElement.GetProperty("assets").EnumerateArray()) + if (asset.GetProperty("name").GetString() == SetupAsset) + { + setupUrl = asset.GetProperty("browser_download_url").GetString(); + setupApiUrl = asset.GetProperty("url").GetString(); + setupSize = asset.GetProperty("size").GetInt64(); + } + if (setupUrl == null) throw new InvalidOperationException("The latest release has no " + SetupAsset + "."); + var cache = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "HookshotBuilder", "update"); + Directory.CreateDirectory(cache); + var setup = Path.Combine(cache, SetupAsset); + w.Note("Downloading the new Hookshot Builder"); + await DownloadWithRetries(w, http, setupUrl, setupApiUrl, setup, setupSize, token); + var si = new ProcessStartInfo(setup) { UseShellExecute = false, WorkingDirectory = cache }; + foreach (var arg in args) si.ArgumentList.Add(arg); + Process.Start(si); + Application.Current.Shutdown(); + } + + /// The game in folder built again with the latest release: here when these tools are it (or fetched), else after the setup + /// installed it (it starts the new builder with --update folder --tag t --fetched) + public static async void UpdateFolder(MainWindow w, string folder, string? tag, bool fetched) + { + w.BeginUpdate(folder, tag); + var tools = Tools.Find(); + if (tools == null) + { + w.DownloadFailed("The Hookshot tools were not found next to this app."); + return; + } + // (just downloaded, but another release than asked for: the setup installed the release before it (GitHub's latest link lags + // right after a publish), and building here kept players on the old version while the builder said it updated) + if (fetched && tag != null && tools.Version != null && tools.Version != tag) + { + w.DownloadFailed($"The update installed {tools.Version}, not {tag}: GitHub's download links can lag right after a release. Try Update again in a few minutes."); + return; + } + // (already the new release, or just downloaded: no second download) + if (fetched || (tag != null && tools.Version == tag)) + { + w.UpdateBuild(tools, folder); + return; + } + try + { + var token = w.DownloadToken; + using var http = new HttpClient { Timeout = Timeout.InfiniteTimeSpan }; + http.DefaultRequestHeaders.UserAgent.ParseAdd("HookshotBuilder"); + w.Note("Asking GitHub for the latest release"); + string? newTag = null, url = null, setupUrl = null; + string? apiUrl = null, setupApiUrl = null; + long size = 0, setupSize = 0; + var hasList = false; + using (var response = await GetWithRetries(w, http, Latest, token, "application/vnd.github+json", TimeSpan.FromSeconds(30))) + using (var doc = JsonDocument.Parse(await response.Content.ReadAsStringAsync(token))) + { + newTag = doc.RootElement.GetProperty("tag_name").GetString(); + foreach (var asset in doc.RootElement.GetProperty("assets").EnumerateArray()) + { + var assetName = asset.GetProperty("name").GetString(); + if (assetName == Asset) + { + url = asset.GetProperty("browser_download_url").GetString(); + apiUrl = asset.GetProperty("url").GetString(); + size = asset.GetProperty("size").GetInt64(); + } + else if (assetName == SetupAsset) + { + setupUrl = asset.GetProperty("browser_download_url").GetString(); + setupApiUrl = asset.GetProperty("url").GetString(); + setupSize = asset.GetProperty("size").GetInt64(); + } + else if (assetName == SetupList) hasList = true; + } + } + if (newTag == null || (url == null && setupUrl == null)) throw new InvalidOperationException("The latest release has no " + Asset + "."); + if (newTag == tools.Version) + { + w.UpdateBuild(tools, folder); + return; + } + var name = string.Concat(newTag.Where(c => char.IsLetterOrDigit(c) || c == '-' || c == '_' || c == '.')); + // (hookshot) the name is a folder under builders\ that the update replaces: "..", "." or a name with a leading dot + // would be the HookshotBuilder folder itself or not a version (the same rule as the setup's and Linux's lists) + if (name.Length == 0 || name.Length > 128 || name[0] == '.') + throw new InvalidOperationException("The latest release's tag is not a version name."); + var cache = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "HookshotBuilder", "update"); + Directory.CreateDirectory(cache); + // (a release with the setup and its list: the setup downloads only the parts that changed since the version installed (a few MB + // when only the builder's code did), checks them and starts the new builder with these arguments. The whole zip is the fallback) + if (setupUrl != null && hasList) + { + w.Note($"Downloading the update for {newTag}"); + var setup = Path.Combine(cache, SetupAsset); + await DownloadWithRetries(w, http, setupUrl, setupApiUrl, setup, setupSize, token); + var ssi = new ProcessStartInfo(setup) { UseShellExecute = false, WorkingDirectory = cache }; + foreach (var arg in new[] { "--update", folder, "--tag", newTag, "--fetched" }) ssi.ArgumentList.Add(arg); + Process.Start(ssi); + Application.Current.Shutdown(); + return; + } + if (url == null) throw new InvalidOperationException("The latest release has no " + Asset + "."); + var zip = Path.Combine(cache, "HookshotBuilder-" + name + ".zip"); + w.Note($"Downloading {Asset} of {newTag} ({size >> 20} MB)"); + await DownloadWithRetries(w, http, url, apiUrl, zip, size, token, w.DownloadProgress); + w.Note("Unpacking the new version"); + // (beside this app when that folder takes writing, else in the app data folder) + var parent = Directory.GetParent(AppContext.BaseDirectory.TrimEnd('\\', '/'))!.FullName; + string target; + try + { + target = await Unpack(zip, Path.Combine(parent, "HookshotBuilder " + name)); + } + catch (Exception e) when (e is UnauthorizedAccessException or IOException) + { + target = await Unpack(zip, Path.Combine(Path.GetDirectoryName(cache)!, "builders", name)); + } + try { File.Delete(zip); } catch (Exception) { } + var exe = Path.Combine(target, "HookshotBuilder.exe"); + if (!File.Exists(exe)) throw new FileNotFoundException("The download has no HookshotBuilder.exe (your antivirus may have removed it)."); + var si = new ProcessStartInfo(exe) { UseShellExecute = false, WorkingDirectory = target }; + foreach (var arg in new[] { "--update", folder, "--tag", newTag, "--fetched" }) si.ArgumentList.Add(arg); + Process.Start(si); + Application.Current.Shutdown(); + } + catch (OperationCanceledException) + { + w.DownloadFailed("The download was cancelled. Your game folder is unchanged."); + } + catch (Exception e) + { + w.DownloadFailed(e.Message + " Your game folder is unchanged; you can also download the new version from GitHub yourself."); + } + } + + static async Task GetWithRetries(MainWindow w, HttpClient http, string url, CancellationToken token, + string? accept = null, TimeSpan? timeout = null) + { + Exception? last = null; + for (var attempt = 0; attempt <= RetryDelays.Length; attempt++) + { + try + { + using var request = new HttpRequestMessage(HttpMethod.Get, url); + if (accept != null) request.Headers.Accept.ParseAdd(accept); + using var ask = timeout is null ? null : new CancellationTokenSource(timeout.Value); + using var both = ask is null ? null : CancellationTokenSource.CreateLinkedTokenSource(token, ask.Token); + var response = await http.SendAsync(request, HttpCompletionOption.ResponseContentRead, both?.Token ?? token); + if (response.IsSuccessStatusCode) return response; + + var failure = HttpFailure(response); + response.Dispose(); + if (!IsTransient(failure)) throw failure; + last = failure; + } + catch (OperationCanceledException e) when (!token.IsCancellationRequested && timeout is not null) + { + last = e; + } + catch (HttpRequestException e) when (IsTransient(e)) + { + last = e; + } + + if (attempt == RetryDelays.Length) break; + w.Note($"GitHub did not answer{StatusDetail(last)}. Retrying in {(int)RetryDelays[attempt].TotalSeconds} s..."); + await Task.Delay(RetryDelays[attempt], token); + } + throw new HttpRequestException($"GitHub's release API did not respond after {RetryDelays.Length + 1} attempts: {last?.Message}", + last, (last as HttpRequestException)?.StatusCode); + } + + static async Task DownloadWithRetries(MainWindow w, HttpClient http, string url, string? apiUrl, string path, long size, + CancellationToken token, Action? progress = null) + { + Exception? last = null; + for (var attempt = 0; attempt <= RetryDelays.Length; attempt++) + { + try + { + await DownloadOnce(http, url, path, size, token, progress, apiAsset: false); + return; + } + catch (Exception e) when (IsTransient(e)) + { + last = e; + if (apiUrl != null) w.Note($"Release download failed{StatusDetail(e)}; trying GitHub's API asset URL."); + } + + if (apiUrl != null) + { + try + { + await DownloadOnce(http, apiUrl, path, size, token, progress, apiAsset: true); + return; + } + catch (Exception e) when (IsTransient(e)) + { + last = e; + } + } + + if (attempt == RetryDelays.Length) break; + w.Note($"GitHub's download servers did not answer{StatusDetail(last)}. Retrying in {(int)RetryDelays[attempt].TotalSeconds} s..."); + await Task.Delay(RetryDelays[attempt], token); + } + throw new HttpRequestException($"GitHub's asset download failed after {RetryDelays.Length + 1} attempts: {last?.Message}", + last, (last as HttpRequestException)?.StatusCode); + } + + static async Task DownloadOnce(HttpClient http, string url, string path, long size, CancellationToken token, + Action? progress, bool apiAsset) + { + using var request = new HttpRequestMessage(HttpMethod.Get, url); + if (apiAsset) request.Headers.Accept.ParseAdd("application/octet-stream"); + using var response = await http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, token); + if (!response.IsSuccessStatusCode) throw HttpFailure(response); + + var total = response.Content.Headers.ContentLength ?? size; + await using var from = await response.Content.ReadAsStreamAsync(token); + await using var to = File.Create(path); + var buffer = new byte[1 << 20]; + long have = 0; + var shown = DateTime.MinValue; + int n; + while (true) + { + try { n = await from.ReadAsync(buffer, token); } + catch (IOException e) { throw new HttpRequestException("The download connection ended unexpectedly.", e); } + if (n == 0) break; + await to.WriteAsync(buffer.AsMemory(0, n), token); + have += n; + if (progress != null && total > 0 && DateTime.UtcNow - shown > TimeSpan.FromMilliseconds(200)) + { + shown = DateTime.UtcNow; + progress(Math.Min(1.0, (double)have / total)); + } + } + if (total > 0 && have != total) throw new HttpRequestException($"The download ended early ({have >> 20} of {total >> 20} MB)."); + } + + static HttpRequestException HttpFailure(HttpResponseMessage response) => + new($"GitHub returned HTTP {(int)response.StatusCode} ({response.ReasonPhrase}).", null, response.StatusCode); + + static bool IsTransient(Exception error) => + error is HttpRequestException request && (request.StatusCode is null || (int)request.StatusCode >= 500); + + static string StatusDetail(Exception? error) => + error is HttpRequestException { StatusCode: { } status } ? $" (HTTP {(int)status})" : ""; + + /// The zip's HookshotBuilder folder, as target (an older one there replaced) + static async Task Unpack(string zip, string target) + { + var part = target + ".part"; + if (Directory.Exists(part)) Directory.Delete(part, true); + await Task.Run(() => ZipFile.ExtractToDirectory(zip, part)); + var inner = Path.Combine(part, "HookshotBuilder"); + if (Directory.Exists(target)) Directory.Delete(target, true); + Directory.Move(Directory.Exists(inner) ? inner : part, target); + if (Directory.Exists(part)) Directory.Delete(part, true); + return target; + } +} diff --git a/builder/HookshotBuilder/app.manifest b/builder/HookshotBuilder/app.manifest new file mode 100644 index 0000000..f76a9dd --- /dev/null +++ b/builder/HookshotBuilder/app.manifest @@ -0,0 +1,16 @@ + + + + + + + + + + + + + PerMonitorV2 + + + diff --git a/builder/linux/AppRun b/builder/linux/AppRun new file mode 100755 index 0000000..1652f5e --- /dev/null +++ b/builder/linux/AppRun @@ -0,0 +1,38 @@ +#!/bin/sh +# The Hookshot AppImage: the builder window (builder/linux/hookshot_builder.py) with everything it builds the game with - +# Python and the tools, clang and lld, a glibc 2.35 sysroot, FFmpeg, SDL2 and the Vulkan back end, built already. +# Hookshot.AppImage the builder window (Play is in the window, once a game is built) +# Hookshot.AppImage --play start the game the builder saved, without opening the window +# Hookshot.AppImage --play start the game in that folder +# Hookshot.AppImage the same +# Hookshot.AppImage --cli nx2/make_game.py itself, e.g. --cli --nup game.nup --out ~/Games/Hookshot +# Hookshot.AppImage --launch the game folder's launcher: Play, and Update when a newer release is out +# Hookshot.AppImage --update download the new release (only the parts that changed) and build that folder again +HERE="$(dirname "$(readlink -f "$0")")" +LIB="$HERE/usr/lib/hookshot" +export HOOKSHOT_TOOLS="$HERE/usr/share/hookshot" +export EMSDK="$LIB/emsdk" +export HOOKSHOT_SYSROOT="$LIB/sysroot" +export HOOKSHOT_FFMPEG="$LIB/ffmpeg" +export HOOKSHOT_VK_PREBUILT="$LIB/hookshot_vk.so" +export HOOKSHOT_SDL2="$LIB/libSDL2-2.0.so.0" +# (written into the game folder's hookshot_version.txt, so its launcher finds this builder) +if [ -n "${APPIMAGE:-}" ]; then + export HOOKSHOT_BUILDER_EXE="$APPIMAGE" +else + export HOOKSHOT_BUILDER_EXE="$HERE/AppRun" +fi +PY="$HERE/usr/python/bin/python3" +if [ "${1:-}" = "--cli" ]; then + shift + exec "$PY" -I "$HOOKSHOT_TOOLS/nx2/make_game.py" "$@" +fi +# Play is the same AppImage: the start script drops this image's libraries, then runs the built game. +if [ "${1:-}" = "--play" ]; then + shift + exec "$HOOKSHOT_TOOLS/builder/linux/hookshot-launch" "$@" +fi +if [ -n "${1:-}" ] && [ -d "$1" ]; then + exec "$HOOKSHOT_TOOLS/builder/linux/hookshot-launch" "$1" +fi +exec "$PY" -I "$HOOKSHOT_TOOLS/builder/linux/hookshot_builder.py" "$@" diff --git a/builder/linux/hookshot-builder.desktop b/builder/linux/hookshot-builder.desktop new file mode 100644 index 0000000..8b5c72b --- /dev/null +++ b/builder/linux/hookshot-builder.desktop @@ -0,0 +1,14 @@ +[Desktop Entry] +Type=Application +Name=Hookshot Builder +Comment=Builds Ocarina of Time for PC from your own copy of the game +Exec=AppRun +Icon=hookshot +Categories=Game; +Terminal=false +Actions=Play; + +[Desktop Action Play] +Name=Play +Exec=AppRun --play +Icon=hookshot diff --git a/builder/linux/hookshot-folder b/builder/linux/hookshot-folder new file mode 100755 index 0000000..ffe1343 --- /dev/null +++ b/builder/linux/hookshot-folder @@ -0,0 +1,42 @@ +#!/bin/sh +# The game folder's launcher, what hookshot.exe is on Windows: Play, and Update when a newer release is out. +# The game itself is bin/hookshot. This file is copied to hookshot in the folder (nx2/make_game.py). +# +# It starts the builder installed by an update ($XDG_DATA_HOME/hookshot/builder), or the builder= line in +# hookshot_version.txt (the AppImage that built the folder). With neither, it starts the game. +set -eu +HERE=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) + +installed() { + root=${HOOKSHOT_BUILDER_HOME:-${XDG_DATA_HOME:-$HOME/.local/share}}/hookshot/builder + [ -f "$root/current" ] || return 1 + tag=$(tr -d ' \r\n' < "$root/current") + [ -n "$tag" ] && [ -x "$root/versions/$tag/AppRun" ] || return 1 + printf '%s\n' "$root/versions/$tag/AppRun" +} + +builder_line() { + [ -f "$HERE/hookshot_version.txt" ] || return 1 + path= + while IFS= read -r line || [ -n "$line" ]; do + case $line in + builder=*) path=${line#builder=} ;; + esac + done < "$HERE/hookshot_version.txt" + [ -n "$path" ] && [ -x "$path" ] || return 1 + printf '%s\n' "$path" +} + +run=$(installed || true) +if [ -z "$run" ]; then + run=$(builder_line || true) +fi +if [ -n "$run" ]; then + exec "$run" --launch "$HERE" +fi +if [ -x "$HERE/bin/hookshot" ]; then + cd "$HERE" + exec ./bin/hookshot +fi +echo "Hookshot: the game (bin/hookshot) is missing from this folder. Build it again with Hookshot Builder." >&2 +exit 1 diff --git a/builder/linux/hookshot-launch b/builder/linux/hookshot-launch new file mode 100755 index 0000000..1c6c18f --- /dev/null +++ b/builder/linux/hookshot-launch @@ -0,0 +1,68 @@ +#!/bin/sh +# Start the game Hookshot Builder wrote, from that folder, without the AppImage's libraries. +# The builder AppImage calls this for --play. The window's Play button does the same thing. +# +# hookshot-launch the out= folder in ~/.config/hookshot/builder.ini +# hookshot-launch /path/to/the/game that folder +# HOOKSHOT_LAUNCH_CONFIG= read that ini instead (a test) +set -eu + +ini_path() { + if [ -n "${HOOKSHOT_LAUNCH_CONFIG:-}" ]; then + printf '%s\n' "$HOOKSHOT_LAUNCH_CONFIG" + else + printf '%s\n' "${XDG_CONFIG_HOME:-$HOME/.config}/hookshot/builder.ini" + fi +} + +# the last out= line, as the builder keeps the last value it wrote +out_of() { + out= + while IFS= read -r line || [ -n "$line" ]; do + case $line in + out=*) out=${line#out=} ;; + esac + done < "$1" + [ -n "$out" ] || return 1 + printf '%s\n' "$out" +} + +ready() { + [ -n "${1:-}" ] && [ -f "$1/image.bin" ] && { [ -x "$1/bin/hookshot" ] || [ -f "$1/hookshot" ]; } +} + +folder=${1:-} +if [ -z "$folder" ]; then + ini=$(ini_path) + if [ -f "$ini" ]; then + folder=$(out_of "$ini") || folder= + fi +fi + +if ! ready "$folder"; then + echo "Hookshot: no built game to start. Build one with Hookshot Builder, or pass its folder." >&2 + # (HOOKSHOT_LAUNCH_CONFIG is a test: it must not open a dialog) + if [ -z "${HOOKSHOT_LAUNCH_CONFIG:-}" ]; then + if command -v zenity >/dev/null 2>&1; then + zenity --error --title=Hookshot --text="No built game to start. Build one with Hookshot Builder first." || true + elif command -v kdialog >/dev/null 2>&1; then + kdialog --error "No built game to start. Build one with Hookshot Builder first." || true + fi + fi + exit 1 +fi + +# The AppImage runtime and the builder put their own libraries on the search path. The game loads +# SDL from beside itself, so those names go before it starts. Same set as the builder's Play button. +names=$(env | awk -F= '$1 ~ /^(HOOKSHOT_|PYTHON|APPDIR|APPIMAGE|ARGV0|OWD)/ || $1 == "LD_LIBRARY_PATH" { print $1 }') +for name in $names; do + unset "$name" +done + +cd "$folder" +# (--play already chose Play. The game's own update question stays for a bin/hookshot the player started themselves) +export HOOKSHOT_NO_UPDATE_CHECK=1 +if [ -x ./bin/hookshot ]; then + exec ./bin/hookshot +fi +exec ./hookshot diff --git a/builder/linux/hookshot_builder.py b/builder/linux/hookshot_builder.py new file mode 100644 index 0000000..a01bafc --- /dev/null +++ b/builder/linux/hookshot_builder.py @@ -0,0 +1,1208 @@ +"""Hookshot Builder for Linux: what builder/HookshotBuilder (WPF) is on Windows, in Tk. It runs nx2/make_game.py and +follows its @@ lines (make_game.py's docstring), then offers to start the game. The AppImage starts it (builder/linux/AppRun), +with the compiler, sysroot and libraries it brings in the environment. + + hookshot_builder.py the window. Its button is Build, Update, Play, or Update builder + hookshot_builder.py --play the game built last time, if there is one (else the window) + hookshot_builder.py --launch that folder's launcher: Play, and Update when a newer release is out + hookshot_builder.py --update download the new release (only parts that changed) and build the folder again +""" +import math +import os +import queue +import re +import shutil +import signal +import subprocess +import sys +import threading +import time +import tkinter as tk +from tkinter import filedialog, messagebox, ttk + +HERE = os.path.dirname(os.path.abspath(__file__)) +ROOT = os.environ.get("HOOKSHOT_TOOLS") or os.path.dirname(os.path.dirname(HERE)) +sys.path.insert(0, HERE) +sys.path.insert(0, os.path.join(ROOT, "nx2")) +sys.path.insert(0, os.path.join(ROOT, "third_party", "a64-kit", "tools")) +import linux_update # noqa: E402 (the release check and the part download) +import make_game # noqa: E402 (LANGUAGES, BUILD_ID) +import toolchain # noqa: E402 (default_jobs: RX_JOBS) + +CONFIG = os.path.join(os.environ.get("XDG_CONFIG_HOME") or os.path.expanduser("~/.config"), "hookshot", "builder.ini") +STAGES = {"read": "Reading your game files", "lift": "Translating the game to PC code", "compile": "Compiling", + "finish": "Putting the game folder together", "shaders": "Preparing shaders"} +DLSS_LICENSE = ( + "NVIDIA DLSS is NVIDIA's, not Hookshot's. With it ticked, the build downloads NVIDIA's DLSS SDK (release v310.9.16) from " + "github.com/NVIDIA/DLSS, under NVIDIA's own license, the NVIDIA RTX SDKs license:\n\n" + "https://github.com/NVIDIA/DLSS/blob/9981c6350dc58c0ce21f024fd65e31e851c31689/LICENSE.txt\n\n" + "Its files stay on this PC (in the builder's ngx-sdk folder, and libnvidia-ngx-dlss.so in the game folder's ngx folder). " + "Hookshot never includes or shares them: never share a game folder built with them.\n\n" + "Do you accept NVIDIA's license?" +) + + +def ngx_sdk_dir(): + """Where NVIDIA's DLSS SDK goes when the player ticks NVIDIA DLSS: beside the builder's versions, kept across updates.""" + return os.path.join(linux_update.data_root(), "ngx-sdk") + + +def folder_has_dlss(folder): + """Whether a game folder was built with NVIDIA DLSS (its runtime in ngx/, and NVIDIA's license in ngx-sdk).""" + if not folder: + return False + ngx_dir = os.path.join(os.path.expanduser(folder), "ngx") + try: + has_runtime = any(name.startswith("libnvidia-ngx-dlss.so") for name in os.listdir(ngx_dir)) + except OSError: + return False + return has_runtime and os.path.isfile(os.path.join(ngx_sdk_dir(), "LICENSE.txt")) + + +def dlss_args(dlss): + return ["--fetch-ngx-sdk", "--accept-nvidia-license"] if dlss else [] + + +def make_game_env(dlss, environ=None): + """The environment make_game.py runs in. HOOKSHOT_NGX_SDK is set only when DLSS is chosen (one set in the shell is not + passed on when unticked, as on Windows), and HOOKSHOT_VK_PREBUILT is dropped so build.py compiles hookshot_vk.so with NGX.""" + env = dict(os.environ if environ is None else environ) + if dlss: + env["HOOKSHOT_NGX_SDK"] = ngx_sdk_dir() + env.pop("HOOKSHOT_VK_PREBUILT", None) + else: + env.pop("HOOKSHOT_NGX_SDK", None) + return env + + +def left_text(seconds): + """The countdown make_game.py and build.py work out from this PC's earlier builds.""" + if seconds < 45: + return "less than a minute left" + if seconds < 90: + return "about a minute left" + minutes = int(round(seconds / 60)) + return "about %d h %d min left" % (minutes // 60, minutes % 60) if minutes >= 60 else "about %d minutes left" % minutes + + +def load_config(): + cfg = {} + try: + for line in open(CONFIG, encoding="utf-8"): + k, eq, v = line.rstrip("\n").partition("=") + if eq: + cfg[k] = v + except OSError: + pass + return cfg + + +def save_config(cfg): + os.makedirs(os.path.dirname(CONFIG), exist_ok=True) + with open(CONFIG, "w", encoding="utf-8") as f: + f.writelines("%s=%s\n" % kv for kv in cfg.items()) + + +def game_ready(folder): + return linux_update.game_exe(folder) is not None + + +def play(folder): + """the game, on its own (it outlives this window), from its folder. The launcher already asked about updates.""" + exe = linux_update.game_exe(folder) + if not exe: + return + env = make_game.game_env() # (as make_game's shader preparation and hookshot-launch: the game's own SDL, the system's libraries) + env["HOOKSHOT_NO_UPDATE_CHECK"] = "1" + subprocess.Popen([exe], cwd=folder, env=env, start_new_session=True, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + + +def desktop_exec(path): + """One Exec argument. A % in the path is a field code, so "100% Games" is rejected and the menu entry does not run. + Inside the quotes the Desktop Entry spec wants the characters double quote, backtick, dollar and backslash each + preceded by a backslash, and every backslash written twice because the file's string escaping is applied first + (so a literal backslash is four, and a dollar is two backslashes and the dollar).""" + escaped = path.replace("\\", "\\\\\\\\") + for char in ('"', "`", "$"): + escaped = escaped.replace(char, "\\\\" + char) + return '"%s"' % escaped.replace("%", "%%") + + +APP_ID = "io.github.NotRexed.Hookshot" # (the game's window names itself so: host/posix/display_sdl.c's HOS_APP_ID) + + +def add_to_menu(folder): + """an entry in the applications menu that starts the game""" + if any(ch in folder for ch in "\r\n\0"): # (a line break would start another key of the entry) + raise ValueError("the game folder's name has a line break") + icon = os.path.join(folder, "hookshot.png") + src = os.path.join(ROOT, "assets", "hookshot.png") + if os.path.exists(src): + shutil.copyfile(src, icon) + apps = os.path.join(os.environ.get("XDG_DATA_HOME") or os.path.expanduser("~/.local/share"), "applications") + os.makedirs(apps, exist_ok=True) + with open(os.path.join(apps, "hookshot.desktop"), "w", encoding="utf-8") as f: + f.write("[Desktop Entry]\nType=Application\nName=Hookshot\nComment=The Legend of Zelda: Ocarina of Time, built from your copy\n" + "Exec=%s\nPath=%s\nIcon=%s\nCategories=Game;\nTerminal=false\nStartupWMClass=%s\n" + % (desktop_exec(os.path.join(folder, "hookshot")), folder, icon, APP_ID)) + + +def free_gb(path): + path = os.path.expanduser(path or "") # (the field takes ~/Games/Hookshot, as build() and apply_mode() read it) + while path and not os.path.exists(path): + path = os.path.dirname(path) + try: + return shutil.disk_usage(path or "/").free / 2 ** 30 + except OSError: + return None + + +def slow_disk_warning(folder): + """The game folder on a hard drive: the build works, but much more slowly than on an SSD. None for an SSD or a drive whose + kind cannot be told (a network share)""" + if make_game.spinning_disk(os.path.expanduser(folder)) is not True: + return None + return ("%s is on a hard drive. An SSD builds the game much faster: build there if you can.\n\n" + "Build here anyway?" % os.path.expanduser(folder)) + + +def xft_dpi(): + """the DPI KDE and GNOME hand X apps when the desktop is scaled. Tk on X11 and XWayland never reads it.""" + env = {k: v for k, v in os.environ.items() if k != "LD_LIBRARY_PATH"} + try: + out = subprocess.run(["xrdb", "-query"], env=env, capture_output=True, text=True, timeout=2).stdout + except (OSError, subprocess.SubprocessError): + return None + found = re.search(r"^Xft\.dpi:\s*([0-9.]+)", out, re.M) + return float(found.group(1)) if found else None + + +def ui_scale(screen_height, dpi=None, environ=os.environ): + """how much bigger than 96 DPI to draw the window. HOOKSHOT_UI_SCALE, then GDK_SCALE or QT_SCALE_FACTOR above 1 + (Wayland sessions export GDK_SCALE=1 whatever the screen), then Xft.dpi; with none of them (gamescope, a bare X + session) a 1440p or 4K screen is still scaled by its height.""" + for name, least in (("HOOKSHOT_UI_SCALE", 0), ("GDK_SCALE", 1), ("QT_SCALE_FACTOR", 1)): + try: + value = float(environ.get(name, "")) + except ValueError: + continue + if value > least: + return min(max(value, 0.5), 4.0) + if dpi: + return min(max(dpi / 96, 0.5), 4.0) + return min(screen_height / 1080, 4.0) if screen_height >= 1440 else 1.0 + + +# ---- the look: the Windows builder's (builder/HookshotBuilder/App.xaml, "Slate"): its colours, Inter, its sizes in 96 DPI +# pixels, its rounded buttons, fields and cards. Tk draws no rounded corners or alpha, so they are painted into images here, +# antialiased and laid over the colour they sit on, and ttk's clam theme draws everything else ---- +PAGE, RAISED, RAISED_HOVER, LINE_STRONG = "#14161a", "#1c1f24", "#23272d", "#3d434c" +INK, INK_SOFT, INK_FAINT = "#edeff2", "#a2a9b3", "#6e7681" +BRAND, BRAND_HOVER, BRAND_INK = "#3d63e6", "#4b70f0", "#dfe6ff" +GO, GO_HOVER, GO_INK, BAD = "#5fb08a", "#70c09a", "#06150e", "#e2776b" +UI_FONTS = ("Inter", "Inter Variable", "Segoe UI", "Cantarell", "Noto Sans", "DejaVu Sans", "Liberation Sans") +MONO_FONTS = ("Cascadia Mono", "JetBrains Mono", "DejaVu Sans Mono", "Noto Sans Mono", "Liberation Mono") + + +def mix(color, under, alpha): + """color at alpha over under: the WPF window's #AARRGGBB glass brushes, flattened onto what is behind them""" + if alpha >= 1: + return color + a, b = int(color[1:], 16), int(under[1:], 16) + return "#%02x%02x%02x" % tuple(round(((a >> s) & 255) * alpha + ((b >> s) & 255) * (1 - alpha)) for s in (16, 8, 0)) + + +WHITE = "#ffffff" +GLASS = mix(WHITE, PAGE, 0x0D / 255) # cards, fields +GLASS_LINE = mix(WHITE, PAGE, 0x17 / 255) # their edges, and a hovered quiet button +QUIET = mix(WHITE, PAGE, 0x08 / 255) # Cancel, Change, Open folder +QUIET_LINE = mix(WHITE, PAGE, 0x1F / 255) +QUIET_HOVER_LINE = mix(WHITE, PAGE, 0x2A / 255) +TICK_LINE = mix(WHITE, PAGE, 0x40 / 255) +TRACK = mix(WHITE, PAGE, 0x12 / 255) # a bar's empty part +LOG = mix("#000000", GLASS, 0x33 / 255) # the build's lines + + +def box(x, y, w, h, r): + """a distance function: from a point to the box at x, y of w x h with corners of radius r (negative inside)""" + cx, cy, hx, hy = x + w / 2, y + h / 2, w / 2 - r, h / 2 - r + + def dist(px, py): + qx, qy = abs(px - cx) - hx, abs(py - cy) - hy + return math.hypot(max(qx, 0.0), max(qy, 0.0)) + min(max(qx, qy), 0.0) - r + return dist, (x, y, x + w, y + h) + + +def ring(cx, cy, r, width): + def dist(px, py): + return abs(math.hypot(px - cx, py - cy) - (r - width / 2)) - width / 2 + return dist, (cx - r, cy - r, cx + r, cy + r) + + +def stroke(points, width): + """a line through points, width wide (a tick, a chevron)""" + pairs = list(zip(points, points[1:])) + + def dist(px, py): + best = 1e9 + for (ax, ay), (bx, by) in pairs: + dx, dy = bx - ax, by - ay + t = max(0.0, min(1.0, ((px - ax) * dx + (py - ay) * dy) / (dx * dx + dy * dy))) + best = min(best, math.hypot(px - ax - t * dx, py - ay - t * dy)) + return best - width / 2 + xs, ys = [p[0] for p in points], [p[1] for p in points] + return dist, (min(xs) - width, min(ys) - width, max(xs) + width, max(ys) + width) + + +def triangle(points): + """a filled convex triangle (Play's), its points clockwise""" + edges = [] + for (ax, ay), (bx, by) in zip(points, points[1:] + points[:1]): + length = math.hypot(bx - ax, by - ay) + edges.append((ax, ay, (by - ay) / length, -(bx - ax) / length)) + + def dist(px, py): + return max((px - ax) * nx + (py - ay) * ny for ax, ay, nx, ny in edges) + xs, ys = [p[0] for p in points], [p[1] for p in points] + return dist, (min(xs), min(ys), max(xs), max(ys)) + + +def paint(w, h, under, layers): + """Photo image data, w x h over the colour under: each layer (colour, (distance function, bounds)) laid on in turn, + antialiased by its distance. A colour may be a function of y (the Windows step's gradient).""" + rows = [] + for j in range(h): + y = j + 0.5 + row = [] + for i in range(w): + x = i + 0.5 + color = under + for paint_color, (dist, (x0, y0, x1, y1)) in layers: + if x < x0 - 1 or x > x1 + 1 or y < y0 - 1 or y > y1 + 1: + continue + cover = 0.5 - dist(x, y) + if cover > 0: + color = mix(paint_color(y) if callable(paint_color) else paint_color, color, min(cover, 1.0)) + row.append(color) + rows.append("{%s}" % " ".join(row)) + return " ".join(rows) + + +def framed(w, h, r, fill, line, width): + """the layers of a rounded box: its edge, and its fill inside it""" + layers = [(line, box(0, 0, w, h, r))] if line and width else [] + return layers + [(fill, box(width, width, w - 2 * width, h - 2 * width, max(r - width, 0)) if line and width else + box(0, 0, w, h, r))] + + +def _arg(name): + args = sys.argv[1:] + if name in args: + index = args.index(name) + if index + 1 < len(args): + return args[index + 1] + return None + + +class Builder(tk.Tk): + def __init__(self, launch=None, update=None, fetched=False, tag=None): + super().__init__() + self.title("Hookshot Builder") + self.apply_scale() + self.apply_theme() + self.minsize(self.px(760), self.px(520)) + self.cfg = load_config() + self.proc = None + self.lines = queue.Queue() + self.started = 0.0 + self.launch_folder = launch + self.update_folder = update + self.fetched = fetched + self.update_tag = tag + self.mode = "build" + self.latest = None + self.source_picked = False + self.dlss_toggled = False + self.tools = linux_update.tools_tag(linux_update.running_root() or ROOT) + self.status = tk.StringVar() + self.kind = tk.StringVar(value=self.cfg.get("kind", "nup")) + self.nup = tk.StringVar(value=self.cfg.get("nup", "")) + self.romfs = tk.StringVar(value=self.cfg.get("romfs", "")) + self.exefs = tk.StringVar(value=self.cfg.get("exefs", "")) + self.out = tk.StringVar(value=self.cfg.get("out", os.path.expanduser("~/Games/Hookshot"))) + names = [n for _, n in make_game.LANGUAGES] + code = self.cfg.get("language", "en-US") + self.language = tk.StringVar(value=dict(make_game.LANGUAGES).get(code, names[0])) + self.prepare = tk.BooleanVar(value=self.cfg.get("prepare", "1") != "0") # (make_game.py's last step: off, at the first start) + self.dlss = tk.BooleanVar(value=self.cfg.get("dlss", "0") == "1") + self.check_text = tk.StringVar() + self.space_text = tk.StringVar() + self.stage_text = tk.StringVar() + self.time_text = tk.StringVar() + self.page = None + self.build_button = None + if update is not None: + self.after(200, self.begin_update) + elif launch: + self.show_launch(launch) + else: + self.show_form() + self.after(200, self.lookup_latest) + self.after(100, self.pump) + self.protocol("WM_DELETE_WINDOW", self.close) + + def apply_scale(self): + """Fonts sized in points follow tk scaling; a named font sized in pixels (Tk 8.6 on X11: -12) does not, so it is + multiplied.""" + self.scale = ui_scale(self.winfo_screenheight(), xft_dpi()) + if self.scale == 1.0: + return + self.tk.call("tk", "scaling", self.scale * 96 / 72) + for name in self.tk.splitlist(self.tk.call("font", "names")): + size = int(self.tk.call("font", "configure", name, "-size")) + if size < 0: + self.tk.call("font", "configure", name, "-size", -max(1, round(-size * self.scale))) + + def px(self, n): + return int(round(n * self.scale)) + + # ---- the look (the colours and images above, as ttk styles) ---- + def apply_theme(self): + px = self.px + self.images = {} + self.configure(background=PAGE) + families = {name.lower(): name for name in self.tk.splitlist(self.tk.call("font", "families"))} + ui = next((families[n.lower()] for n in UI_FONTS if n.lower() in families), None) + mono = next((families[n.lower()] for n in MONO_FONTS if n.lower() in families), None) + named = set(self.tk.splitlist(self.tk.call("font", "names"))) + + def font(name, size, weight="normal", family=ui): + options = ["-size", -px(size), "-weight", weight] + (["-family", family] if family else []) + self.tk.call("font", "configure" if name in named else "create", name, *options) + for name, size, weight in (("HsUi", 14, "normal"), ("HsStrong", 14, "bold"), ("HsNote", 13, "normal"), + ("HsNoteStrong", 13, "bold"), ("HsSmall", 12, "normal"), ("HsLabel", 12, "bold"), + ("HsBrand", 15, "bold"), ("HsTitle", 24, "bold"), ("HsDisplay", 34, "bold"), + ("HsPlay", 16, "bold"), ("TkDefaultFont", 13, "normal"), ("TkTextFont", 13, "normal"), + ("TkMenuFont", 13, "normal"), ("TkHeadingFont", 13, "bold"), ("TkCaptionFont", 14, "bold")): + font(name, size, weight) + font("HsMono", 12, family=mono) + font("HsLog", 11, family=mono) + if mono: + font("TkFixedFont", 12, family=mono) + + # (the classic widgets and dialogs: the file chooser, the DLSS license's question, the language list) + for pattern, value in (("*Background", PAGE), ("*Foreground", INK), ("*selectBackground", BRAND), + ("*selectForeground", WHITE), ("*insertBackground", INK), ("*highlightBackground", PAGE), + ("*highlightColor", BRAND), ("*activeBackground", RAISED_HOVER), ("*activeForeground", INK), + ("*troughColor", TRACK), ("*Dialog.msg.font", "HsUi"), + ("*Dialog.msg.wrapLength", px(460)), ("*Dialog.dtl.font", "HsNote"), + ("*TCombobox*Listbox.background", RAISED), ("*TCombobox*Listbox.foreground", INK), + ("*TCombobox*Listbox.selectBackground", RAISED_HOVER), + ("*TCombobox*Listbox.selectForeground", BRAND_INK), ("*TCombobox*Listbox.font", "HsNote"), + ("*TCombobox*Listbox.borderWidth", 0), ("*TCombobox*Listbox.highlightThickness", 0), + ("*TCombobox*Listbox.relief", "flat")): + self.option_add(pattern, value) + + s = self.style = ttk.Style(self) + s.theme_use("clam") + s.configure(".", background=PAGE, foreground=INK, font="HsUi", bordercolor=QUIET_LINE, lightcolor=PAGE, + darkcolor=PAGE, troughcolor=TRACK, focuscolor=BRAND, selectbackground=BRAND, selectforeground=WHITE, + fieldbackground=GLASS, insertcolor=INK, arrowcolor=INK_SOFT, borderwidth=0) + s.map(".", foreground=[("disabled", INK_FAINT)], background=[("disabled", PAGE)]) + for name, options in (("TFrame", {}), ("Card.TFrame", {"background": GLASS}), + ("TLabel", {}), ("Title.TLabel", {"font": "HsTitle"}), ("Display.TLabel", {"font": "HsDisplay"}), + ("Sub.TLabel", {"foreground": INK_SOFT}), ("Note.TLabel", {"foreground": INK_SOFT, "font": "HsNote"}), + ("Small.TLabel", {"foreground": INK_SOFT, "font": "HsSmall"}), + ("Faint.TLabel", {"foreground": INK_FAINT, "font": "HsSmall"}), + ("Section.TLabel", {"foreground": INK_SOFT, "font": "HsLabel"}), + ("Status.TLabel", {"foreground": BRAND_INK, "font": "HsNote"}), + ("Stage.TLabel", {"background": GLASS, "font": "HsStrong"}), + ("Bad.Stage.TLabel", {"foreground": BAD}), ("Go.Stage.TLabel", {"foreground": GO}), + ("Card.Note.TLabel", {"background": GLASS})): + s.configure(name, **options) + + # buttons: Quiet (also every plain ttk button, the dialogs' among them), Primary (blue: the step's action), Go (green: Play) + def button(style, fill, hover, line, hover_line, focus_line, foreground, font, padding, radius=8, disabled=None): + r, w = px(radius), px(1) + n = 2 * r + 6 + image = {k: self.photo("%s-%s" % (style, k), n, n, PAGE, framed(n, n, r, f, l, w)) + for k, f, l in (("normal", fill, line), ("hover", hover, hover_line), ("focus", fill, focus_line), + ("disabled", disabled or fill, disabled or line))} + s.element_create(style + ".box", "image", image["normal"], ("disabled", image["disabled"]), + ("pressed", image["hover"]), ("active", image["hover"]), ("focus", image["focus"]), + border=r + 2, padding=0, sticky="nsew") + s.layout(style + ".TButton", [(style + ".box", {"sticky": "nsew", "children": [ + ("Button.padding", {"sticky": "nsew", "children": [("Button.label", {"sticky": "nsew"})]})]})]) + s.configure(style + ".TButton", foreground=foreground, font=font, padding=padding, anchor="center", background=fill) + s.map(style + ".TButton", foreground=[("disabled", INK_FAINT)], background=[("disabled", disabled or fill)]) + button("Quiet", QUIET, GLASS_LINE, QUIET_LINE, QUIET_HOVER_LINE, BRAND_HOVER, INK, "HsStrong", (px(18), px(6))) + button("Primary", BRAND, BRAND_HOVER, mix(WHITE, BRAND, 0x33 / 255), mix(WHITE, BRAND_HOVER, 0x33 / 255), + mix(WHITE, BRAND, 0xAA / 255), WHITE, "HsStrong", (px(22), px(6)), disabled=mix(WHITE, PAGE, 0x14 / 255)) + button("Go", GO, GO_HOVER, mix(WHITE, GO, 0x40 / 255), mix(WHITE, GO_HOVER, 0x40 / 255), mix(WHITE, GO, 0xAA / 255), + GO_INK, "HsPlay", (px(30), px(10)), radius=10) + s.layout("TButton", s.layout("Quiet.TButton")) + s.configure("TButton", **{k: s.lookup("Quiet.TButton", k) for k in ("foreground", "font", "padding", "anchor", "background")}) + s.map("TButton", foreground=[("disabled", INK_FAINT)]) + # (a text link: Open the game folder) + s.layout("Link.TButton", [("Button.padding", {"sticky": "nsew", "children": [("Button.label", {"sticky": "nsew"})]})]) + s.configure("Link.TButton", foreground=INK_FAINT, font="HsNoteStrong", padding=0, background=PAGE) + s.map("Link.TButton", foreground=[("active", INK)], background=[("active", PAGE)]) + + # ticks and the game files' choice (the Windows Tick and DumpRow) + b, t = px(18), px(8) + for name, under, layers in ( + ("tick", PAGE, framed(b, b, px(5), PAGE, TICK_LINE, px(1.5))), + ("tick-on", PAGE, framed(b, b, px(5), BRAND, BRAND, px(1.5)) + + [(WHITE, stroke([(px(2.5), px(8.5)), (px(6.5), px(12.5)), (px(14.5), px(3.5))], px(2)))]), + ("radio", GLASS, [(INK_FAINT, ring(b / 2, b / 2, b / 2, px(2)))]), + ("radio-on", GLASS, [(BRAND, ring(b / 2, b / 2, b / 2, px(2))), (BRAND, ring(b / 2, b / 2, px(4), px(4)))])): + self.photo(name, b + px(14 if name.startswith("radio") else 8), b, under, layers) + s.element_create("Hs.tick", "image", self.images["tick"], ("selected", self.images["tick-on"]), sticky="w") + s.element_create("Hs.radio", "image", self.images["radio"], ("selected", self.images["radio-on"]), sticky="w") + s.layout("TCheckbutton", [("Checkbutton.padding", {"sticky": "nsew", "children": [ + ("Hs.tick", {"side": "left", "sticky": ""}), ("Checkbutton.label", {"side": "left", "sticky": "nsew"})]})]) + s.configure("TCheckbutton", foreground=INK, font="HsNote", padding=(0, px(2))) + s.map("TCheckbutton", background=[("active", PAGE)], foreground=[("active", WHITE)]) + s.layout("Row.TRadiobutton", [("Radiobutton.padding", {"sticky": "nsew", "children": [ + ("Hs.radio", {"side": "left", "sticky": ""}), ("Radiobutton.label", {"side": "left", "sticky": "nsew"})]})]) + s.configure("Row.TRadiobutton", background=GLASS, foreground=INK, font="HsUi", padding=(px(16), px(13))) + s.map("Row.TRadiobutton", background=[("active", GLASS)], foreground=[("active", WHITE)]) + + # fields: the paths (monospaced, as on Windows) and the language + r, w, n = px(8), px(1), px(8) * 2 + 6 + field = self.photo("field", n, n, PAGE, framed(n, n, r, GLASS, GLASS_LINE, w)) + field_hover = self.photo("field-hover", n, n, PAGE, framed(n, n, r, GLASS, QUIET_HOVER_LINE, w)) + field_focus = self.photo("field-focus", n, n, PAGE, framed(n, n, r, GLASS, BRAND, w)) + s.element_create("Hs.field", "image", field, ("focus", field_focus), ("hover", field_hover), border=r + 2, padding=0, + sticky="nsew") + s.layout("TEntry", [("Hs.field", {"sticky": "nsew", "children": [ + ("Entry.padding", {"sticky": "nsew", "children": [("Entry.textarea", {"sticky": "nsew"})]})]})]) + s.configure("TEntry", foreground=INK_SOFT, padding=(px(12), px(8)), fieldbackground=GLASS, insertwidth=max(1, px(1))) + chevron = self.photo("chevron", px(8 + 14), px(8), GLASS, + [(INK_SOFT, stroke([(px(1), px(2)), (px(5), px(6)), (px(9), px(2))], px(1.4)))]) + s.element_create("Hs.chevron", "image", chevron, sticky="") + s.layout("TCombobox", [("Hs.field", {"sticky": "nsew", "children": [ + ("Hs.chevron", {"side": "right", "sticky": ""}), + ("Combobox.padding", {"sticky": "nsew", "children": [("Combobox.textarea", {"sticky": "nsew"})]})]})]) + s.configure("TCombobox", foreground=INK, padding=(px(12), px(7), px(4), px(7)), arrowsize=px(12)) + s.map("TCombobox", fieldbackground=[("readonly", GLASS)], foreground=[("readonly", INK)], + selectbackground=[("readonly", GLASS)], selectforeground=[("readonly", INK)]) + s.configure("ComboboxPopdownFrame", background=RAISED, bordercolor=LINE_STRONG, relief="solid", borderwidth=1) + + # cards (the found list's, the build's steps on Windows) + r, n = px(10), px(10) * 2 + 6 + self.photo("card", n, n, PAGE, framed(n, n, r, GLASS, GLASS_LINE, px(1))) + s.element_create("Hs.card", "image", self.images["card"], border=r + 2, padding=0, sticky="nsew") + s.layout("Card.TFrame", [("Hs.card", {"sticky": "nsew", "children": [("Frame.padding", {"sticky": "nsew"})]})]) + + # bars: the free space (green, red when short) and a step's progress (blue), 6 px and round + h = max(4, px(6)) + # (the parts' names end in trough and pbar: the progress bar finds the part it sizes by that name) + for name, under, color in (("HsTrack.trough", PAGE, TRACK), ("HsGo.pbar", PAGE, GO), ("HsBad.pbar", PAGE, BAD), + ("HsCardTrack.trough", GLASS, mix(WHITE, GLASS, 0x12 / 255)), ("HsCard.pbar", GLASS, BRAND)): + self.photo(name, h + 4, h, under, [(color, box(0, 0, h + 4, h, h / 2))]) + s.element_create(name, "image", self.images[name], border=(h // 2 + 1, 0), padding=0, sticky="nsew") + for style, track in (("Go", "HsTrack"), ("Bad", "HsTrack"), ("Card", "HsCardTrack")): + s.layout(style + ".Horizontal.TProgressbar", [(track + ".trough", {"sticky": "nsew", "children": [ + ("Hs%s.pbar" % style, {"side": "left", "sticky": "ns"})]})]) + s.configure(style + ".Horizontal.TProgressbar", thickness=h, background=GLASS) + + # the log's scroll bar: a thumb on the log, no arrows + s.layout("Log.Vertical.TScrollbar", [("Vertical.Scrollbar.trough", {"sticky": "ns", "children": [ + ("Vertical.Scrollbar.thumb", {"expand": "1", "sticky": "nsew"})]})]) + s.configure("Log.Vertical.TScrollbar", troughcolor=LOG, background=RAISED_HOVER, bordercolor=LOG, lightcolor=RAISED_HOVER, + darkcolor=RAISED_HOVER, gripcount=0, arrowsize=px(10)) + s.map("Log.Vertical.TScrollbar", background=[("active", LINE_STRONG)], lightcolor=[("active", LINE_STRONG)], + darkcolor=[("active", LINE_STRONG)]) + + # Play's triangle, and the hookshot of the window's head + self.photo("play", px(11) + px(10), px(14), GO, + [(GO_INK, triangle([(px(0.5), px(0.5)), (px(11), px(7)), (px(0.5), px(13.5))]))]) + self.load_logo(px(28)) + + def photo(self, name, w, h, under, layers): + image = tk.PhotoImage(master=self, width=int(w), height=int(h)) + image.put(paint(int(w), int(h), under, layers)) + self.images[name] = image + return image + + def load_logo(self, size): + """assets/hookshot.png (the Windows head's and the game's icon), size pixels wide over the page: averaged down with its + alpha on Tk 9, every nth pixel on an older Tk""" + self.images["logo"] = None + path = os.path.join(ROOT, "assets", "hookshot.png") + try: + full = tk.PhotoImage(master=self, file=path) + except (tk.TclError, OSError): + return + w, h = full.width(), full.height() + try: + rows = [self.tk.splitlist(row) for row in + self.tk.splitlist(self.tk.call(full, "data", "-format", "default -colorformat rgba"))] + except tk.TclError: + self.images["logo"] = full.subsample(max(1, w // size)) + return + tw, th = size, max(1, round(h * size / w)) + page = [int(PAGE[i:i + 2], 16) for i in (1, 3, 5)] + out = [] + for j in range(th): + y0, y1 = j * h // th, max(j * h // th + 1, (j + 1) * h // th) + line = [] + for i in range(tw): + x0, x1 = i * w // tw, max(i * w // tw + 1, (i + 1) * w // tw) + acc, n = [0.0, 0.0, 0.0, 0.0], 0 + for y in range(y0, y1): + for x in range(x0, x1): + v = int(rows[y][x][1:], 16) + a = (v & 255) / 255 + acc[0] += (v >> 24 & 255) * a + acc[1] += (v >> 16 & 255) * a + acc[2] += (v >> 8 & 255) * a + acc[3] += a + n += 1 + alpha = acc[3] / n + line.append("#%02x%02x%02x" % tuple(min(255, round(acc[k] / n + page[k] * (1 - alpha))) for k in range(3))) + out.append("{%s}" % " ".join(line)) + logo = tk.PhotoImage(master=self, width=tw, height=th) + logo.put(" ".join(out)) + self.images["logo"] = logo + + def header(self, parent, step=None, chip=None): + """the window's head, as Windows draws it: the hookshot and Hookshot at the left, and in the middle the steps (Game files > + Build > Play) or, for the launcher, the game's version""" + px = self.px + canvas = tk.Canvas(parent, height=px(64), background=PAGE, highlightthickness=0, borderwidth=0) + canvas.pack(fill="x") + x = px(28) + if self.images.get("logo") is not None: + canvas.create_image(x, px(32), image=self.images["logo"], anchor="w") + x += self.images["logo"].width() + px(10) + canvas.create_text(x, px(32), text="Hookshot", font="HsBrand", fill=INK, anchor="w") + x += int(self.tk.call("font", "measure", "HsBrand", "Hookshot")) + if chip: + canvas.create_text(x + px(12), px(32), text=chip, font="HsSmall", fill=INK_SOFT, anchor="w") + self.header_canvas = canvas + if step is not None: + self.set_step(step) + canvas.bind("", lambda event: self.center_steps(event.width)) + return canvas + + def set_step(self, step): + canvas = getattr(self, "header_canvas", None) + if canvas is None or not canvas.winfo_exists(): + return + canvas.delete("steps") + pill, labels = self.step_pill(step) + canvas.create_image(0, self.px(12), image=pill, anchor="nw", tags=("steps", "pill")) + for lx, text, font, color in labels: + canvas.create_text(lx, self.px(32), text=text, font=font, fill=color, anchor="w", tags="steps") + self.center_steps(canvas.winfo_width()) + + def center_steps(self, width): + canvas = self.header_canvas + pill = canvas.bbox("pill") + if pill and width > 1: + canvas.move("steps", (width - (pill[2] - pill[0])) // 2 - pill[0], 0) + + def step_pill(self, on): + """Windows' SetStep: the steps in a pill, the one now in blue, the ones done with a green tick. One image (the shapes), + and the labels with their x, for the canvas to write over it""" + px = self.px + names = ("Game files", "Build", "Play") + container = mix(WHITE, PAGE, 0x0B / 255) + layers, labels = [], [] + x = px(4) + for i, name in enumerate(names): + if i: + layers.append((mix(INK_SOFT, container, 0.45), stroke([(x + px(5), px(16)), (x + px(9), px(20)), (x + px(5), px(24))], px(1.6)))) + x += px(14) + font = "HsNoteStrong" if i == on else "HsNote" + width = int(self.tk.call("font", "measure", font, name)) + done = i < on + pill_w = (px(4) + px(22) + px(8) if done else px(14)) + width + px(14) + if i == on: + layers.append((lambda y: mix("#3557d6", "#4b70f0", min(1, max(0, (y - px(5)) / px(30)))), + box(x, px(5), pill_w, px(30), px(15)))) + if done: + cx, cy = x + px(4) + px(11), px(20) + layers.append((mix(GO, container, 0x26 / 255), box(cx - px(11), cy - px(11), px(22), px(22), px(11)))) + layers.append((GO, stroke([(cx - px(5), cy), (cx - px(1.5), cy + px(3.5)), (cx + px(5), cy - px(3.5))], px(1.8)))) + text_x = x + (px(4) + px(22) + px(8) if done else px(14)) + labels.append((text_x, name, font, WHITE if i == on else INK_SOFT if done else INK_FAINT)) + x += pill_w + width = x + px(4) + image = self.photo("steps-%d" % on, width, px(40), PAGE, + framed(width, px(40), px(20), container, GLASS_LINE, px(1)) + layers) + return image, labels + + def section(self, parent, text, row, column=0, columnspan=1, top=24): + """the Windows builder's section labels: FOUND ON THIS PC, INSTALL THE GAME TO""" + label = ttk.Label(parent, text=text.upper(), style="Section.TLabel") + label.grid(row=row, column=column, columnspan=columnspan, sticky="w", pady=(self.px(top), self.px(10))) + return label + + # ---- the form ---- + def show_form(self): + self.title("Hookshot Builder") + if self.page: + self.page.destroy() + px = self.px + page = self.page = ttk.Frame(self) + page.pack(fill="both", expand=True) + self.header(page, step=0) + f = ttk.Frame(page, padding=(px(48), px(6), px(48), px(28))) + f.pack(fill="both", expand=True) + f.columnconfigure(0, weight=1, minsize=px(580)) # (every page px(800) wide: the window keeps its width, as on Windows) + f.columnconfigure(1, minsize=px(20)) + f.columnconfigure(2, minsize=px(200)) + ttk.Label(f, text="Build your game", style="Title.TLabel").grid(row=0, column=0, columnspan=3, sticky="w") + ttk.Label(f, text="Pick your game files and where the game goes. Hookshot does the rest, once.", style="Sub.TLabel", + wraplength=px(800)).grid(row=1, column=0, columnspan=3, sticky="w", pady=(px(6), 0)) + if game_ready(self.cfg.get("out")): + ttk.Button(f, text="Play the game built in %s" % self.cfg["out"], command=lambda: self.play_and_close(self.cfg["out"])).grid( + row=2, column=0, columnspan=3, sticky="w", pady=(px(16), 0)) + self.section(f, "Game files", 3, columnspan=3) + kinds = ttk.Frame(f, style="Card.TFrame", padding=px(5)) + kinds.grid(row=4, column=0, columnspan=3, sticky="we") + kinds.columnconfigure(0, weight=1) + ttk.Radiobutton(kinds, text=".nup file", value="nup", variable=self.kind, command=self.show_form, + style="Row.TRadiobutton", cursor="hand2").grid(row=0, column=0, sticky="we") + tk.Frame(kinds, height=max(1, px(1)), background=GLASS_LINE).grid(row=1, column=0, sticky="we", padx=px(12)) + ttk.Radiobutton(kinds, text="romfs and exefs folders", value="folders", variable=self.kind, command=self.show_form, + style="Row.TRadiobutton", cursor="hand2").grid(row=2, column=0, sticky="we") + paths = ttk.Frame(f) + paths.grid(row=5, column=0, columnspan=3, sticky="we", pady=(px(10), 0)) + paths.columnconfigure(1, weight=1) + if self.kind.get() == "nup": + self.path_row(paths, 0, ".nup file", self.nup, lambda: self.pick_file(self.nup)) + else: + self.path_row(paths, 0, "romfs", self.romfs, lambda: self.pick_dir(self.romfs)) + self.path_row(paths, 1, "exefs", self.exefs, lambda: self.pick_dir(self.exefs)) + ttk.Label(f, textvariable=self.check_text, style="Small.TLabel", wraplength=px(800)).grid( + row=6, column=0, columnspan=3, sticky="w", pady=(px(6), 0)) + + self.section(f, "Install the game to", 7, top=18) + self.section(f, "Game language", 7, column=2, top=18) + out = ttk.Frame(f) + out.grid(row=8, column=0, sticky="we") + out.columnconfigure(1, weight=1) + self.path_row(out, 0, None, self.out, lambda: self.pick_dir(self.out), button="Change") + ttk.Combobox(f, textvariable=self.language, values=[n for _, n in make_game.LANGUAGES], state="readonly", + font="HsNote", cursor="hand2").grid(row=8, column=2, sticky="we") + + options = ttk.Frame(f) + options.grid(row=9, column=0, columnspan=3, sticky="we", pady=(px(22), 0)) + ttk.Checkbutton(options, text="Prepare shaders now", variable=self.prepare, cursor="hand2").pack(anchor="w") + ttk.Label(options, text="A few minutes at the end of the build. Unticked, the game prepares them at its first start, " + "behind a black window.", style="Small.TLabel", wraplength=px(660)).pack(anchor="w", padx=(px(26), 0), pady=(px(2), 0)) + ttk.Checkbutton(options, text="NVIDIA DLSS (downloads NVIDIA's SDK under NVIDIA's license)", variable=self.dlss, + command=self.on_dlss_changed, cursor="hand2").pack(anchor="w", pady=(px(12), 0)) + ttk.Label(options, text="For NVIDIA RTX graphics cards. NVIDIA's files stay on this PC: never share a game folder built with them.", + style="Small.TLabel", wraplength=px(660)).pack(anchor="w", padx=(px(26), 0), pady=(px(2), 0)) + + f.rowconfigure(10, weight=1, minsize=px(18)) + ttk.Label(f, textvariable=self.status, style="Status.TLabel", wraplength=px(800)).grid( + row=11, column=0, columnspan=3, sticky="w", pady=(0, px(12))) + foot = ttk.Frame(f) + foot.grid(row=12, column=0, columnspan=3, sticky="we") + foot.columnconfigure(1, weight=1) + self.space_bar = ttk.Progressbar(foot, maximum=1.0, length=px(160), style="Go.Horizontal.TProgressbar") + self.space_bar.grid(row=0, column=0) + ttk.Label(foot, textvariable=self.space_text, style="Small.TLabel").grid(row=0, column=1, sticky="w", padx=(px(10), 0)) + self.about_label = ttk.Label(foot, text="About 45 min", style="Faint.TLabel") + self.about_label.grid(row=0, column=2, padx=(px(14), px(14))) + self.build_button = ttk.Button(foot, text="Build the game", command=self.build, style="Primary.TButton", cursor="hand2") + self.build_button.grid(row=0, column=3) + self.out.trace_add("write", lambda *a: self.update_space()) + self.update_space() + self.apply_mode() + + def path_row(self, f, row, label, var, browse, button="Browse…"): + if label: + ttk.Label(f, text=label, style="Small.TLabel").grid(row=row, column=0, sticky="w", padx=(0, self.px(12))) + ttk.Entry(f, textvariable=var, font="HsMono").grid(row=row, column=1, sticky="we", pady=self.px(3)) + ttk.Button(f, text=button, command=browse, cursor="hand2").grid(row=row, column=2, padx=(self.px(10), 0), pady=self.px(3)) + + def pick_file(self, var): + p = filedialog.askopenfilename(title="Your game's .nup", filetypes=[("Switch 2 game package", "*.nup"), ("All files", "*")]) + if p: + var.set(p) + self.source_picked = True + self.apply_mode() + self.check_version() + + def pick_dir(self, var): + p = filedialog.askdirectory(title="Choose a folder", initialdir=var.get() or os.path.expanduser("~")) + if p: + var.set(p) + if var is self.exefs: + self.source_picked = True + self.apply_mode() + self.check_version() + + def update_space(self): + gb = free_gb(self.out.get()) + self.space_text.set("Needs 45 GB · %.0f GB free there" % gb if gb is not None else "Needs 45 GB") + bar = getattr(self, "space_bar", None) # (as on Windows: the 45 GB's share of what is free, green when it fits) + try: + if bar is not None and bar.winfo_exists(): + bar.configure(value=min(1.0, 45 / max(gb, 0.1)) if gb is not None else 0, + style="Go.Horizontal.TProgressbar" if gb is None or gb >= 45 else "Bad.Horizontal.TProgressbar") + except tk.TclError: + pass + self.apply_mode() + + def on_dlss_changed(self): + if self.dlss.get() and not messagebox.askyesno("NVIDIA DLSS", DLSS_LICENSE): + self.dlss.set(False) + else: + self.dlss_toggled = True + self.cfg["dlss"] = "1" if self.dlss.get() else "0" + save_config(self.cfg) + self.apply_mode() + + def check_version(self): + args = ["--nup", self.nup.get()] if self.kind.get() == "nup" else ["--exefs", self.exefs.get()] + self.check_text.set("Checking the game version...") + + def run(): + r = subprocess.run([sys.executable, "-I", os.path.join(ROOT, "nx2", "make_game.py"), "--check"] + args, + capture_output=True, text=True, cwd=ROOT) + said = [] + for l in r.stdout.splitlines(): + t = make_game.reported(l) + if t is not None: + said.append(t) + tail = [l for l in (r.stderr or "").splitlines() if l.strip()] + text = said[-1] if said else (tail[-1] if tail else "The version could not be read.") + self.lines.put(("check", text)) + threading.Thread(target=run, daemon=True).start() + + def lookup_latest(self): + def run(): + forced = os.environ.get("HOOKSHOT_LATEST_TAG") + tag = None if forced == "none" else (forced or linux_update.latest_tag()) + self.lines.put(("latest", tag or "")) + threading.Thread(target=run, daemon=True).start() + + def apply_mode(self): + if self.launch_folder and self.build_button is None: + self.apply_launch() + return + folder = self.out.get().strip() + expanded = os.path.expanduser(folder) if folder else "" + game, kept = linux_update.game_version(expanded) if expanded else (None, False) + self.mode = linux_update.choose_mode(game, self.tools, self.latest, self.source_picked, kept) + dlss_changed = bool(kept and game and not self.source_picked and getattr(self, "dlss_toggled", False) and + hasattr(self, "dlss") and bool(self.dlss.get()) != folder_has_dlss(expanded)) + if self.mode == "up-to-date" and dlss_changed: + self.mode = "update-game" + button = self.build_button + if button is None: + return + try: + if not button.winfo_exists(): + return + except tk.TclError: + return + button.configure(text={"update-game": "Update", "up-to-date": "Play", "update-builder": "Update builder", + "build": "Build the game"}[self.mode]) + about = getattr(self, "about_label", None) # (Windows: the time only for a build) + try: + if about is not None and about.winfo_exists(): + about.configure(text="About 45 min" if self.mode == "build" else "") + except tk.TclError: + pass + target = linux_update.update_target(self.latest, self.tools) + if self.mode == "update-game" and dlss_changed and game == target: + self.status.set("Update builds your game again %s NVIDIA DLSS." + % ("with" if self.dlss.get() else "without")) + elif self.mode == "update-game": + self.status.set("Hookshot %s is available. Your game is %s. Update downloads only what changed, then compiles what changed." + % (target, game)) + elif self.mode == "up-to-date" and self.latest is None: + self.status.set("Your game is %s, as this builder. GitHub did not answer, so newer versions could not be checked." % game) + elif self.mode == "up-to-date": + self.status.set("%s is the latest version." % game) + elif self.mode == "update-builder": + self.status.set("Hookshot Builder %s is available (this one is %s). Update it first, then build." % (self.latest, self.tools or "this one")) + else: + self.status.set("") + + def apply_launch(self): + if not self.launch_folder or not hasattr(self, "launch_status"): + return + game, kept = linux_update.game_version(self.launch_folder) + target = linux_update.update_target(self.latest, self.tools) + show = False + if target and game and game != target and kept: + self.launch_status.set("Update available: %s" % target) + show = True + elif target and game and game != target: + self.launch_status.set("%s is out. Build this folder again with Hookshot Builder to get it." % target) + elif game and self.latest is None: + self.launch_status.set("Up to date with this builder (GitHub did not answer)") + elif game: + self.launch_status.set("Up to date") + else: + self.launch_status.set("Could not check for updates (GitHub did not answer)" if self.latest is None else "Hookshot") + if hasattr(self, "launch_update"): + self.launch_update.pack(side="right", padx=self.px(4)) if show else self.launch_update.pack_forget() + + def show_launch(self, folder): + if self.page: + self.page.destroy() + self.title("Hookshot") + px = self.px + page = self.page = ttk.Frame(self) + page.pack(fill="both", expand=True) + game, _kept = linux_update.game_version(folder) + self.header(page, chip=game) + # (the Windows launcher's page: the game's name in the middle, Update beside Play when there is one, the folder below) + f = ttk.Frame(page, padding=(px(48), px(24), px(48), px(56))) + f.pack(fill="both", expand=True) + ttk.Frame(f, width=px(800), height=0).pack() + ttk.Label(f, text="THE LEGEND OF ZELDA", style="Section.TLabel").pack() + ttk.Label(f, text="Ocarina of Time", style="Display.TLabel").pack(pady=(px(4), 0)) + ttk.Label(f, text=folder, style="Sub.TLabel", wraplength=px(600), justify="center").pack(pady=(px(8), 0)) + self.launch_status = tk.StringVar(value="Checking for updates...") + ttk.Label(f, textvariable=self.launch_status, style="Note.TLabel", wraplength=px(600), justify="center").pack(pady=(px(16), 0)) + buttons = ttk.Frame(f) + buttons.pack(pady=(px(28), 0)) + self.launch_update = ttk.Button(buttons, text="Update", style="Primary.TButton", cursor="hand2", + command=lambda: self.begin_update(folder)) + ttk.Button(buttons, text="Play", style="Go.TButton", image=self.images["play"], compound="left", cursor="hand2", + command=lambda: self.play_and_close(folder)).pack(side="right", padx=px(4)) + ttk.Button(f, text="Open the game folder", style="Link.TButton", cursor="hand2", + command=lambda: subprocess.Popen(["xdg-open", folder])).pack(pady=(px(22), 0)) + self.lookup_latest() + + def begin_update(self, folder=None): + if folder is not None: + self.update_folder = folder + self.show_progress("Updating your game" if self.update_folder else "Updating Hookshot Builder") + folder = self.update_folder + # (a game older than this builder is built again with it: nothing to download, and GitHub need not answer. So is one + # from a builder that is not an older release, e.g. from the Actions tab: issue 902) + target = self.update_tag or linux_update.update_target(self.latest, self.tools) + fetched = self.fetched or bool(folder and self.tools and target == self.tools) + + def run(): + try: + if not fetched: + root = linux_update.running_root() + if root is None: + raise RuntimeError("These tools are a source checkout, so they cannot download a release. Start the AppImage to update.") + manifest = linux_update.startup_manifest(root) + if manifest.tag != linux_update.tools_tag(root): + # (as Windows does: GitHub's latest list lags right after a publish, and installing it would put + # the release before the one asked for over these tools) + if self.update_tag and manifest.tag != self.update_tag: + raise RuntimeError("GitHub offered %s, not %s: its download links can lag right after a release. " + "Try Update again in a few minutes." % (manifest.tag, self.update_tag)) + source =linux_update.read_manifest(root) + apprun = linux_update.install(manifest, root, source, note=lambda message: self.lines.put(("note", message))) + for name in ("APPDIR", "APPIMAGE", "ARGV0", "OWD", "LD_LIBRARY_PATH"): + os.environ.pop(name, None) + args = [apprun] + if folder: + args += ["--update", folder, "--tag", manifest.tag, "--fetched"] + os.execv(apprun, args) + self.lines.put(("do-build", folder or "")) + except Exception as error: + self.lines.put(("update-error", str(error))) + + threading.Thread(target=run, daemon=True).start() + + def start_make_update(self, folder): + if not folder: + self.show_form() + self.status.set("This builder is the latest version.") + self.lookup_latest() + return + dlss = bool(self.dlss.get()) if getattr(self, "dlss_toggled", False) else folder_has_dlss(folder) + cmd = [sys.executable, "-I", os.path.join(ROOT, "nx2", "make_game.py"), "--update", folder] + \ + ([] if self.prepare.get() else ["--no-prepare-shaders"]) + dlss_args(dlss) # (as the last build chose) + self.heading.configure(text="Updating your game") + self.stage_text.set("Compiling what changed") + self.started = time.time() + self.proc = subprocess.Popen(cmd, cwd=ROOT, env=make_game_env(dlss), stdout=subprocess.PIPE, stderr=subprocess.STDOUT, + text=True, bufsize=1, encoding="utf-8", errors="replace", start_new_session=True) + threading.Thread(target=self.read, args=(self.proc,), daemon=True).start() + self.tick() + + # ---- the build ---- + def build(self): + mode = getattr(self, "mode", "build") + if mode == "up-to-date": + folder = os.path.expanduser(self.out.get().strip()) + if game_ready(folder): + self.play_and_close(folder) + return + if mode == "update-game": + self.fetched = False + self.begin_update(os.path.expanduser(self.out.get().strip())) + return + if mode == "update-builder": + self.fetched = False + self.update_folder = "" + self.begin_update("") + return + out = os.path.expanduser(self.out.get().strip()) + if self.kind.get() == "nup": + if not os.path.isfile(self.nup.get()): + return messagebox.showerror("Hookshot", "Choose your game's .nup file.") + src = ["--nup", self.nup.get()] + else: + if not (os.path.isdir(self.romfs.get()) and os.path.isdir(self.exefs.get())): + return messagebox.showerror("Hookshot", "Choose your game's romfs and exefs folders.") + src = ["--romfs", self.romfs.get(), "--exefs", self.exefs.get()] + if not out: + return messagebox.showerror("Hookshot", "Choose where to save the game.") + try: + jobs = make_game.requested_jobs(os.environ.get("HOOKSHOT_JOBS")) + if jobs is None and os.environ.get("RX_JOBS"): # (the older setting, still honoured) + jobs = toolchain.default_jobs() + except ValueError as error: + return messagebox.showerror("Hookshot", str(error)) + code = next((c for c, n in make_game.LANGUAGES if n == self.language.get()), "en-US") + dlss = bool(self.dlss.get()) if hasattr(self, "dlss") else False + self.cfg.update(kind=self.kind.get(), nup=self.nup.get(), romfs=self.romfs.get(), exefs=self.exefs.get(), out=out, language=code, + prepare="1" if self.prepare.get() else "0", **({"dlss": "1" if dlss else "0"} if hasattr(self, "dlss") else {})) + save_config(self.cfg) + # (issue 531) Without HOOKSHOT_JOBS or RX_JOBS, make_game chooses as on Windows, from the memory free now; this window chose + # fewer, 80% of the total memory at 3.5 GB each, and the CPU sat at 29% + cmd = [sys.executable, "-I", os.path.join(ROOT, "nx2", "make_game.py")] + src + ["--out", out, "--language", code] + \ + ([] if jobs is None else ["--jobs", str(jobs)]) + ([] if self.prepare.get() else ["--no-prepare-shaders"]) + \ + (dlss_args(True) if dlss else []) + slow = slow_disk_warning(out) # (a warning, not a stop: a hard drive still builds, only much slower) + if slow is not None and not messagebox.askokcancel("Hookshot", slow): + return + self.show_progress() + self.started = time.time() + env_kw = {"env": make_game_env(dlss)} if (dlss or "HOOKSHOT_NGX_SDK" in os.environ) else {} + self.proc = subprocess.Popen(cmd, cwd=ROOT, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, + text=True, bufsize=1, encoding="utf-8", errors="replace", start_new_session=True, **env_kw) + threading.Thread(target=self.read, args=(self.proc,), daemon=True).start() + + def read(self, proc): + for line in proc.stdout: + self.lines.put(("line", line.rstrip("\n"))) + self.lines.put(("exit", proc.wait())) + + def show_progress(self, title="Building your game"): + if self.page: + self.page.destroy() + self.changed, self.compiled_files = -1, 0 + self.eta = None + self.compile_started = None + self.times = None + self.shaders_later = False + px = self.px + page = self.page = ttk.Frame(self) + page.pack(fill="both", expand=True) + self.header(page, step=1) + f = ttk.Frame(page, padding=(px(48), px(6), px(48), px(28))) + f.pack(fill="both", expand=True) + f.columnconfigure(0, weight=1, minsize=px(800)) + f.rowconfigure(3, weight=1) + self.heading = ttk.Label(f, text=title, style="Title.TLabel") + self.heading.grid(row=0, column=0, sticky="w") + ttk.Label(f, text="This takes a while, and your PC stays usable.", style="Sub.TLabel").grid( + row=1, column=0, sticky="w", pady=(px(6), 0)) + # (the step, its bar and the clock in a card, as Windows' steps; the lines make_game.py prints below it) + card = ttk.Frame(f, style="Card.TFrame", padding=(px(20), px(16))) + card.grid(row=2, column=0, sticky="we", pady=(px(22), px(14))) + card.columnconfigure(0, weight=1) + self.stage_label = ttk.Label(card, textvariable=self.stage_text, style="Stage.TLabel", wraplength=px(760)) + self.stage_label.grid(row=0, column=0, sticky="w") + self.bar = ttk.Progressbar(card, maximum=1.0, mode="determinate", style="Card.Horizontal.TProgressbar") + self.bar.grid(row=1, column=0, sticky="we", pady=(px(12), px(10))) + ttk.Label(card, textvariable=self.time_text, style="Card.Note.TLabel").grid(row=2, column=0, sticky="w") + lines = tk.Frame(f, background=LOG, highlightthickness=max(1, px(1)), highlightbackground=GLASS_LINE, + highlightcolor=GLASS_LINE) + lines.grid(row=3, column=0, sticky="nsew") + lines.columnconfigure(0, weight=1) + lines.rowconfigure(0, weight=1) + self.log = tk.Text(lines, height=12, wrap="none", font="HsLog", background=LOG, foreground=INK_SOFT, + borderwidth=0, highlightthickness=0, padx=px(10), pady=px(8), insertbackground=INK, + selectbackground=BRAND, selectforeground=WHITE) + self.log.grid(row=0, column=0, sticky="nsew") + scroll = ttk.Scrollbar(lines, orient="vertical", command=self.log.yview, style="Log.Vertical.TScrollbar") + scroll.grid(row=0, column=1, sticky="ns") + self.log.configure(yscrollcommand=scroll.set) + self.buttons = ttk.Frame(f) + self.buttons.grid(row=4, column=0, sticky="e", pady=(px(14), 0)) + ttk.Button(self.buttons, text="Cancel", command=self.cancel, cursor="hand2").pack(side="right") + self.stage_text.set("Starting") + self.tick() + + def tick(self): + if self.proc and self.proc.poll() is None: + s = int(time.time() - self.started) + text = "Elapsed %d:%02d:%02d" % (s // 3600, s // 60 % 60, s % 60) + if self.compile_started: + c = int(time.time() - self.compile_started) + text += " · compiling %d:%02d" % (c // 60, c % 60) + if self.eta is not None: + text += " · " + left_text(self.eta) + self.time_text.set(text) + self.after(1000, self.tick) + + def pump(self): + try: + while True: + kind, v = self.lines.get_nowait() + if kind == "check": + self.check_text.set(v) + elif kind == "latest": + self.latest = v or None + self.apply_mode() + elif kind == "note": + if hasattr(self, "stage_text"): + self.stage_text.set(v) + elif kind == "do-build": + self.start_make_update(v) + elif kind == "update-error": + if hasattr(self, "stage_text"): + self.stage_text.set(v + " Your game folder is unchanged.") + if hasattr(self, "buttons"): + for widget in self.buttons.winfo_children(): + widget.destroy() + ttk.Button(self.buttons, text="Back", command=self.show_form, cursor="hand2").pack(side="right", padx=self.px(4)) + if hasattr(self, "stage_label"): + self.stage_label.configure(style="Bad.Stage.TLabel") + elif kind == "line": + self.on_line(v) + elif kind == "exit": + self.on_exit(v) + except queue.Empty: + pass + self.after(100, self.pump) + + def on_line(self, line): + if line.startswith("@@stage "): + stage = line.split(" ", 2)[1] + self.stage_text.set(STAGES.get(stage, line.split(" ", 2)[-1])) + self.bar.configure(mode="determinate", value=0) + # (the compile's clock and countdown are the compile's: a later step, the minutes-long shaders one, without them) + self.compile_started = time.time() if stage == "compile" else None + self.eta = None + elif line.startswith("@@progress "): + try: + self.bar.configure(value=float(line.split()[1])) + except ValueError: + pass + elif line.startswith("@@plan "): + parts = line.split(" ") + if len(parts) == 3: + try: + self.changed, total = int(parts[1]), int(parts[2]) + except ValueError: + pass + else: + self.stage_text.set("Compiling: everything is already built" if not self.changed else + "Compiling %d changed files of %d" % (self.changed, total) if self.changed < total else + "Compiling %d files" % total) + return + elif line.startswith("@@file "): + parts = line.split(" ") + if len(parts) == 4 and parts[3] in ("new", "again"): + self.compiled_files += 1 + if self.changed > 0: + self.stage_text.set("Compiling %d of %d changed files" % (self.compiled_files, self.changed)) + return + elif line.startswith("@@eta "): + try: + self.eta = float(line.split()[1]) + except (ValueError, IndexError): + pass + return + elif line.startswith("@@times "): + parts = line.split(" ") + if len(parts) == 4: + try: + self.times = (float(parts[1]), float(parts[2]), float(parts[3])) + except ValueError: + pass + return + elif line.startswith("@@done "): + self.built = line[7:] + elif line.startswith("@@error "): + self.error = line[8:] + elif line.startswith(make_game.NOT_PREPARED): + self.shaders_later = True + if hasattr(self, "log"): + self.log.insert("end", line + "\n") + self.log.see("end") + + def on_exit(self, code): + self.proc = None + for w in self.buttons.winfo_children(): + w.destroy() + folder = getattr(self, "built", None) + if code == 0 and folder and game_ready(folder): + s = int(time.time() - self.started) + text = "Your game is ready, built in %d:%02d" % (s // 3600, s // 60 % 60) + if self.times: + total, compiled, previous = self.times + text += ", compiled in %d:%02d" % (int(compiled) // 60, int(compiled) % 60) + if previous > 0: + text += " · last time %d:%02d" % (int(previous) // 3600, int(previous) // 60 % 60) + if self.shaders_later: + text += " · the shaders are prepared at its first start" + self.stage_text.set(text) + self.bar.configure(value=1.0) + try: + add_to_menu(folder) + except (OSError, ValueError): # (no menu entry; the game is built) + pass + ttk.Button(self.buttons, text="Play", style="Go.TButton", image=self.images["play"], compound="left", cursor="hand2", + command=lambda: self.play_and_close(folder)).pack(side="right", padx=self.px(4)) + ttk.Button(self.buttons, text="Open folder", cursor="hand2", + command=lambda: subprocess.Popen(["xdg-open", folder])).pack(side="right", padx=self.px(4)) + self.heading.configure(text="Your game is ready") + self.stage_label.configure(style="Go.Stage.TLabel") + self.set_step(2) + else: + self.stage_text.set("The build stopped: %s" % getattr(self, "error", "see the lines above")) + ttk.Button(self.buttons, text="Back", command=self.show_form, cursor="hand2").pack(side="right", padx=self.px(4)) + self.heading.configure(text="The build stopped") + self.stage_label.configure(style="Bad.Stage.TLabel") + + def cancel(self): + if self.proc and self.proc.poll() is None: + os.killpg(self.proc.pid, signal.SIGTERM) + self.error = "Build cancelled. You can start again from the same folder." + + def play_and_close(self, folder): + play(folder) + self.destroy() + + def close(self): + if self.proc and self.proc.poll() is None: + if not messagebox.askyesno("Hookshot", "Stop the build and close?"): + return + os.killpg(self.proc.pid, signal.SIGTERM) + self.destroy() + + +def main(): + if "--play" in sys.argv[1:]: + folder = _arg("--play") or load_config().get("out") + if game_ready(folder): + play(folder) + return + launch = _arg("--launch") + update = _arg("--update") + if update is not None: + Builder(update=update, fetched="--fetched" in sys.argv[1:], tag=_arg("--tag")).mainloop() + elif launch: + Builder(launch=launch).mainloop() + else: + Builder().mainloop() + + +if __name__ == "__main__": + main() diff --git a/builder/linux/linux_update.py b/builder/linux/linux_update.py new file mode 100644 index 0000000..8fbeffe --- /dev/null +++ b/builder/linux/linux_update.py @@ -0,0 +1,546 @@ +"""Linux builder updates, the same way Windows installs (builder/setup/setup.c, scripts/make_setup_parts.py). + +A release publishes hookshot-linux.txt. Each part is a zip named by its SHA-256 and kept on the linux-parts release, so a +part whose files did not change is not downloaded again. The player's copy lives at +$XDG_DATA_HOME/hookshot/builder/versions/ (AppRun at its root, the same layout as the AppImage). An unchanged part +is copied from the builder that is running now (the mounted AppImage, or the version installed last time). + + tag + part + tree the whole directory + file one file in that directory (repeat; instead of tree) +""" +import hashlib +import os +import re +import shutil +import stat +import time +import urllib.error +import urllib.request +import zipfile + +from part_links import SYSROOT_PART, is_absolute_link, relative_sysroot_link + +LIST_NAME = "hookshot-linux.txt" +LATEST_LIST = "https://github.com/NotRexed/Hookshot/releases/latest/download/" + LIST_NAME +LATEST_API = "https://api.github.com/repos/NotRexed/Hookshot/releases/latest" + + +class Part(object): + def __init__(self, sha, size, dest, url, files): + self.sha, self.size, self.dest, self.url, self.files = sha, size, dest, url, files # files None: the whole directory + + def same(self, other): + return other is not None and other.sha == self.sha and other.dest == self.dest and other.files == self.files + + +class Manifest(object): + def __init__(self, tag, parts): + self.tag, self.parts = tag, parts + + def by_sha(self, sha): + for part in self.parts: + if part.sha == sha: + return part + return None + + +def data_root(): + base = os.environ.get("HOOKSHOT_BUILDER_HOME") or os.environ.get("XDG_DATA_HOME") or os.path.expanduser("~/.local/share") + return os.path.join(base, "hookshot", "builder") + + +def parse_list(text, local=True): + """The setup list. A part owns the following tree line, or the file lines, until the next part. local: a list from + this PC (the tools' own, HOOKSHOT_LINUX_LIST), whose parts may be file:// links; a downloaded one's are https""" + tag, parts, cur = "", [], None + for raw in text.splitlines(): + line = raw.strip() + if not line or line.startswith("#"): + continue + if line.startswith("tag "): + tag = line[4:].strip() + elif line.startswith("part "): + bits = line.split(" ") + if len(bits) != 5: + raise ValueError("a part line needs sha, size, directory and url") + cur = Part(bits[1], int(bits[2]), bits[3], bits[4], []) + parts.append(cur) + elif line == "tree": + if cur is None: + raise ValueError("tree with no part") + cur.files = None + elif line.startswith("file "): + if cur is None or cur.files is None: + raise ValueError("file with no part") + cur.files.append(line[5:].strip()) + else: + raise ValueError("unknown list line") + if not tag or not parts: + raise ValueError("the setup list has no release") + # (the tag names a folder under versions/ that an update removes and remakes, and a part's names are paths in it: + # they stay inside it) + if not re.fullmatch(r"[0-9A-Za-z._+-]{1,128}", tag) or tag in (".", ".."): + raise ValueError("the setup list's tag is not a version name") + for part in parts: + if not re.fullmatch(r"[0-9a-f]{64}", part.sha) or not part.url.startswith(("https://", "file://") if local else "https://"): + raise ValueError("a part line has a bad checksum or link") + if part.dest != ".": + _safe_name(part.dest) + for name in part.files or (): + _safe_name(name) + return Manifest(tag, parts) + + +def release_key(tag): + """A version to compare: a release tag (v2.0.5, v2.1.0-rc.1) or git describe's v2.0.5-3-g1a2b3c4[-dirty] (a build from + the Actions tab or a checkout: after v2.0.5, before any later release). None for a build that names no release: a bare + commit (digits alone too: no v and no dot), dev, or no version at all.""" + m = re.match(r"^(?=v|\d+\.)v?(\d+(?:\.\d+)*)(?:-(?!\d+-g|dirty$)([0-9A-Za-z]+(?:\.[0-9A-Za-z]+)*))?(-\d+-g[0-9a-f]+)?(-dirty)?$", + (tag or "").strip()) + if not m: + return None + # (a prerelease before its release; commits after a tag, or changes to it, after that tag and before the next) + return tuple(int(n) for n in m.group(1).split(".")), m.group(2) is None, bool(m.group(3) or m.group(4)) + + +def update_target(latest, tools): + """The newest version this builder knows of: the latest release when it is newer than these tools, else the tools' + own. (issue 902) Any version other than the latest release was made to update before it could build, so a test + build from the Actions tab, newer than the release, was taken back to it. One made after an older release is still + offered the newer one.""" + if not latest or not tools: + return latest or tools + have, out = release_key(tools), release_key(latest) + return latest if have and out and out > have else tools + + +def choose_mode(game, tools, latest, source_picked, kept): + """What the builder's button does. The same order as the Windows builder (MainWindow.RefreshMode): an older game + that still has its files is updated, a game at the newest known release is played, and an older builder updates + itself before a new build. latest None: GitHub did not answer, so the builder's own version is the newest known. + Newest is update_target's: a builder that is not an older release is not made to update.""" + target = update_target(latest, tools) + if kept and game and target and game != target and not source_picked: + return "update-game" + if game and not source_picked: + return "up-to-date" + if latest and tools and target != tools: + return "update-builder" + return "build" + + +def tools_tag(root): + """The release these tools are, from the file a package writes (VERSION, or version.txt). A source checkout has neither.""" + if not root: + return "" + for rel in ("usr/share/hookshot/VERSION", "usr/share/hookshot/version.txt", "VERSION", "version.txt"): + path = os.path.join(root, rel) + try: + with open(path, encoding="utf-8") as f: + line = f.readline().strip() + except OSError: + continue + if line: + return line + return "" + + +def running_root(): + """The AppDir that is running: the AppImage's mount, or an installed version (its AppRun beside usr/).""" + appdir = os.environ.get("APPDIR") + if appdir and os.path.isfile(os.path.join(appdir, "AppRun")): + return appdir + tools = os.environ.get("HOOKSHOT_TOOLS") + if tools: + root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(tools)))) # (/usr/share/hookshot) + if os.path.isfile(os.path.join(root, "AppRun")): + return root + return None + + +def read_manifest(root): + path = os.path.join(root, LIST_NAME) if root else "" + try: + with open(path, encoding="utf-8") as f: + return parse_list(f.read()) + except (OSError, ValueError): + return None + + +def game_version(folder): + """The release a folder was built with, when it still has the game files an update builds from.""" + if not folder: + return None, False + kept = os.path.isdir(os.path.join(folder, ".hookshot_exefs")) and os.path.isdir(os.path.join(folder, "romfs")) + version = None + try: + with open(os.path.join(folder, "hookshot_version.txt"), encoding="utf-8") as f: + for line in f: + if line.startswith("version="): + version = line[8:].strip() or None + break + except OSError: + pass + return version, kept + + +def game_exe(folder): + """bin/hookshot under the launcher, or hookshot in a folder built before the launcher.""" + if not folder: + return None + nested = os.path.join(folder, "bin", "hookshot") + if os.path.isfile(nested) and os.path.isfile(os.path.join(folder, "image.bin")): + return nested + flat = os.path.join(folder, "hookshot") + if os.path.isfile(flat) and os.path.isfile(os.path.join(folder, "image.bin")): + return flat + return None + + +MAX_DOWNLOAD = 4 << 30 # (no part or list is this big: a download that is stops there) + + +def _read_at_most(response, limit): + data = response.read(limit + 1) + if len(data) > limit: + raise RuntimeError("the download is larger than expected") + return data + + +def _https(url): + """Only https links are fetched (never file:, ftp: or a plain http: link from a release's answer).""" + if not isinstance(url, str) or not url.startswith("https://"): + raise ValueError("not an https link: %r" % (url,)) + return url + + +def _request(url, timeout, limit=MAX_DOWNLOAD): + # (no second argument: Request's second is the POST body, and a str body failed every download before connecting) + req = urllib.request.Request(_https(url), headers={"User-Agent": "Hookshot", "Accept": "application/vnd.github+json"}) + with urllib.request.urlopen(req, timeout=timeout) as response: + return _read_at_most(response, limit) + + +def _api_download(url): + """A GitHub release link that failed: the same file through the API (scripts/linux/deps.sh does this for 504s).""" + prefix = "https://github.com/" + mark = "/releases/download/" + if not url.startswith(prefix) or mark not in url: + return None + repo, _, rest = url[len(prefix):].partition(mark) + tag, _, name = rest.partition("/") + if not repo or not tag or not name or "/" in name: + return None + info = _request("https://api.github.com/repos/%s/releases/tags/%s" % (repo, tag), 30) + import json + for asset in json.loads(info.decode("utf-8")).get("assets", []): + if asset.get("name") == name: + req = urllib.request.Request(_https(asset.get("url")), headers={ + "User-Agent": "Hookshot", "Accept": "application/octet-stream"}) + with urllib.request.urlopen(req, timeout=120) as response: + return _read_at_most(response, MAX_DOWNLOAD) + return None + + +def fetch_bytes(url, timeout=60): + """The url's bytes. A GitHub download that the server fails (5xx) is tried again, then through the API.""" + delay, last = 2, "the download failed" + for attempt in range(4): + try: + return _request(url, timeout) + except urllib.error.HTTPError as error: + last = "GitHub answered %s" % error.code + if error.code < 500 and error.code != 429: + break + except Exception as error: + last = str(error) or error.__class__.__name__ + if attempt < 3: + time.sleep(delay) + delay = min(delay * 2, 20) + try: + got = _api_download(url) + except Exception as error: + got = None + last = str(error) or last + if got is not None: + return got + raise RuntimeError(last) + + +def latest_tag(get=None): + """The latest release's tag, or None when GitHub did not answer. The API first (60 an hour); the setup list's tag + otherwise, which is a file download and not that limit. Same fallback as the Windows builder.""" + get = get or (lambda url: fetch_bytes(url, 8)) + try: + import json + body = json.loads(get(LATEST_API).decode("utf-8")) + tag = body.get("tag_name") + if tag: + return tag + except Exception: + pass + try: + text = get(LATEST_LIST).decode("utf-8") + return parse_list(text, local=False).tag + except Exception: + return None + + +def _sha256(data): + return hashlib.sha256(data).hexdigest() + + +def _clone(src, dst, root, rooted_links=None): + """A copy of src at dst. A hard link when the two are on one disk (the installed copy); a real copy from the + AppImage's mount, which does not link.""" + if os.path.islink(src): + link_to = os.readlink(src) + if rooted_links is not None and is_absolute_link(link_to): + rooted_links.append((dst, link_to)) + else: + _symlink(root, link_to, dst) + return + try: + os.link(src, dst) + return + except OSError: + shutil.copy2(src, dst) + + +def _clone_tree(src, dst, rebase_sysroot=False, root=None): + root = dst if root is None else root + rooted_links = [] if rebase_sysroot else None + for dirpath, dirnames, filenames in os.walk(src, followlinks=False): + rel = os.path.relpath(dirpath, src) + out = dst if rel == os.curdir else os.path.join(dst, rel) + os.makedirs(out, exist_ok=True) + kept = [] + for name in dirnames: + link = os.path.join(dirpath, name) + if os.path.islink(link): + _clone(link, os.path.join(out, name), root, rooted_links) + else: + kept.append(name) + dirnames[:] = kept + for name in filenames: + _clone(os.path.join(dirpath, name), os.path.join(out, name), root, rooted_links) + if rooted_links is not None: + _finish_sysroot_links(dst, rooted_links) + + +def _safe_name(name): + if not name or name.startswith("/") or ".." in name.split("/"): + raise ValueError("the zip has a path outside its folder") + return name + + +def _check_part_directory(path): + if os.path.realpath(path) != os.path.abspath(path): + raise ValueError("the update would write through a symbolic link: %s" % path) + + +def _symlink(root, link_to, target): + # The whole staging tree is the boundary: a valid toolchain link may name a different part or a later entry. + root = os.path.abspath(root) + # (".." only at the start: "x/../.." is checked as written, but the system resolves x first, and x may be a link the + # same zip or a later part makes, so the check and the system would disagree) + parts = link_to.split("/") + ups = 0 + while ups < len(parts) and parts[ups] == "..": + ups += 1 + if ".." in parts[ups:]: + raise ValueError("an update symlink climbs after a name: %s -> %s" % (target, link_to)) + destination = os.path.abspath(os.path.join(os.path.dirname(target), link_to)) + real_root = os.path.realpath(root) + if (is_absolute_link(link_to) or os.path.commonpath((root, destination)) != root or + os.path.commonpath((real_root, os.path.realpath(destination))) != real_root): + raise ValueError("an update symlink leaves its staging folder: %s -> %s" % (target, link_to)) + os.symlink(link_to, target) + + +def _finish_sysroot_links(dest, rooted_links): + while rooted_links: + pending = [] + for target, link_to in rooted_links: + try: + relative = relative_sysroot_link(dest, target, link_to) + except FileNotFoundError: + pending.append((target, link_to)) + continue + _check_part_directory(os.path.dirname(target)) + os.symlink(relative, target) + if len(pending) == len(rooted_links): + raise ValueError("sysroot links have missing or circular targets: %s" % + ", ".join("%s -> %s" % (target, link_to) for target, link_to in pending)) + rooted_links = pending + + +def _unpack(data, dest, files, rebase_sysroot=False, root=None): + root = dest if root is None else root + os.makedirs(dest, exist_ok=True) + rooted_links = [] + with zipfile.ZipFile(__import__("io").BytesIO(data)) as zipped: + for info in zipped.infolist(): + name = _safe_name(info.filename) + if not name or name.endswith("/"): + continue + if files is not None and name not in files: + raise ValueError("the zip has %s, which this part does not list" % name) + target = os.path.join(dest, name) + parent = os.path.dirname(target) + _check_part_directory(parent) + os.makedirs(parent, exist_ok=True) + mode = info.external_attr >> 16 + if stat.S_ISLNK(mode): + link_to = zipped.read(info).decode("utf-8") + # Released sysroots contain /lib links rooted inside the sysroot; other parts must reject absolute links. + if is_absolute_link(link_to): + if not rebase_sysroot: + raise ValueError("a zip symlink has an absolute target: %s -> %s" % (name, link_to)) + rooted_links.append((target, link_to)) + else: + _symlink(root, link_to, target) + else: + # A part can ship symlinks, but cannot overwrite one (or any earlier entry). + with open(target, "xb") as out: + out.write(zipped.read(info)) + if mode & 0o777: + # (the modes the parts are made with: no group- or world-writable files from a zip) + os.chmod(target, 0o755 if mode & 0o111 else 0o644) + _finish_sysroot_links(dest, rooted_links) + + +def _place(part, source_root, source, dest_root, get, note): + # Resolve installation parent links without following the staging directory itself. + parent, staging = os.path.split(os.path.abspath(dest_root)) + staging_root = os.path.join(os.path.realpath(parent), staging) + dest = os.path.join(staging_root, _safe_name(part.dest)) + _check_part_directory(dest) + old = source.by_sha(part.sha) if source else None + src = os.path.join(source_root, part.dest) if old and old.same(part) and source_root else "" + ready = src and os.path.isdir(src) and (part.files is None or all( + os.path.lexists(os.path.join(src, name)) for name in part.files)) + rebase_sysroot = part.dest == SYSROOT_PART + if ready: + note("Keeping %s (unchanged)" % part.dest) + if part.files is None: + _clone_tree(src, dest, rebase_sysroot, staging_root) + else: + os.makedirs(dest, exist_ok=True) + rooted_links = [] if rebase_sysroot else None + for name in part.files: + _clone(os.path.join(src, name), os.path.join(dest, name), staging_root, rooted_links) + if rooted_links is not None: + _finish_sysroot_links(dest, rooted_links) + return + note("Downloading %s (%d MB)" % (part.dest, part.size >> 20)) + data = get(part.url) + if len(data) != part.size or _sha256(data) != part.sha: + raise RuntimeError("%s did not download correctly (its checksum differs)" % part.dest) + _unpack(data, dest, part.files, rebase_sysroot, staging_root) + + +def _check_links_inside(dest): + """Every link in the finished staging tree resolves inside it, whatever order the parts made them in.""" + real = os.path.realpath(dest) + for dirpath, dirnames, filenames in os.walk(dest, followlinks=False): + for name in dirnames + filenames: + path = os.path.join(dirpath, name) + if os.path.islink(path) and os.path.commonpath((real, os.path.realpath(path))) != real: + raise ValueError("an update symlink leaves its staging folder: %s" % path) + + +def _finish_tree(dest): + """AppRun at the root, from the tools just installed, so this folder starts the way the AppImage does.""" + _check_links_inside(dest) + for name in ("AppRun", "hookshot.png", "hookshot-builder.desktop"): + if os.path.islink(os.path.join(dest, name)): # (written below: never through a link a part made) + raise ValueError("the update has a link where %s goes" % name) + tools = os.path.join(dest, "usr", "share", "hookshot") + apprun = os.path.join(dest, "AppRun") + shutil.copyfile(os.path.join(tools, "builder", "linux", "AppRun"), apprun) + os.chmod(apprun, 0o755) # (rwxr-xr-x: the launcher must run; only its owner may change it) + icon = os.path.join(tools, "assets", "hookshot.png") + if os.path.isfile(icon): + shutil.copyfile(icon, os.path.join(dest, "hookshot.png")) + desktop = os.path.join(tools, "builder", "linux", "hookshot-builder.desktop") + if os.path.isfile(desktop): + shutil.copyfile(desktop, os.path.join(dest, "hookshot-builder.desktop")) + return apprun + + +def install(manifest, source_root, source, note=None, get=None): + """versions/ from the list. Unchanged parts come from source_root. The new folder replaces nothing until it + is complete, and the version installed before stays until this one is.""" + note = note or (lambda message: None) + get = get or fetch_bytes + root = data_root() + versions = os.path.join(root, "versions") + dest = os.path.join(versions, manifest.tag) + partial = dest + ".part" + if os.path.islink(partial): + raise ValueError("the update staging folder is a symbolic link: %s" % partial) + if os.path.isdir(partial): + shutil.rmtree(partial) + os.makedirs(versions, exist_ok=True) + try: + for part in manifest.parts: + _place(part, source_root, source, partial, get, note) + # The publisher keeps the list outside all hashed parts. Never follow or overwrite a part-supplied leaf. + with open(os.path.join(partial, LIST_NAME), "x", encoding="utf-8", newline="\n") as handle: + handle.write("tag %s\n" % manifest.tag) + for part in manifest.parts: + handle.write("part %s %d %s %s\n" % (part.sha, part.size, part.dest, part.url)) + handle.write("tree\n" if part.files is None else "".join("file %s\n" % name for name in part.files)) + apprun = _finish_tree(partial) + except Exception: + shutil.rmtree(partial, ignore_errors=True) + raise + if os.path.isdir(dest): + shutil.rmtree(dest) + os.rename(partial, dest) + current = os.path.join(root, "current") + tmp = current + ".new" + with open(tmp, "w", encoding="utf-8", newline="\n") as handle: + handle.write(manifest.tag + "\n") + os.replace(tmp, current) + for name in os.listdir(versions): + if name != manifest.tag: + shutil.rmtree(os.path.join(versions, name), ignore_errors=True) + return os.path.join(dest, "AppRun") if os.path.isfile(os.path.join(dest, "AppRun")) else apprun + + +def installed_apprun(): + root = data_root() + try: + with open(os.path.join(root, "current"), encoding="utf-8") as handle: + tag = handle.read().strip() + except OSError: + return "" + path = os.path.join(root, "versions", tag, "AppRun") + return path if tag and os.path.isfile(path) else "" + + +def fetch_manifest(get=None): + get = get or (lambda url: fetch_bytes(url, 30)) + override = os.environ.get("HOOKSHOT_LINUX_LIST") + if override: + with open(override, encoding="utf-8") as handle: + return parse_list(handle.read()) + return parse_list(get(LATEST_LIST).decode("utf-8"), local=False) + + +def startup_manifest(root, get=None): + """The list begin_update installs on launch: the latest release, except when these tools ship a list of their own + that is that same release and no older than it (a self-contained -rc or Actions tab AppImage). Reinstalling an + older release's parts over those tools replaces files the working tools had (issue 990: a -rc shipped + gpu/vk/oot_cheat_items.h, the release's zip did not, and every launch lost it). The comparison is update_target's: + a builder that is not an older release still updates. A HOOKSHOT_LINUX_LIST is a deliberate override: it wins.""" + latest = fetch_manifest(get) + if not os.environ.get("HOOKSHOT_LINUX_LIST"): + own = read_manifest(root) + if own and own.tag == tools_tag(root) and update_target(latest.tag, own.tag) == own.tag: + return own + return latest diff --git a/builder/linux/part_links.py b/builder/linux/part_links.py new file mode 100644 index 0000000..d356356 --- /dev/null +++ b/builder/linux/part_links.py @@ -0,0 +1,26 @@ +"""Safe handling of the rooted links in Linux sysroot update parts.""" +import os + + +SYSROOT_PART = "usr/lib/hookshot/sysroot" + + +def is_absolute_link(target): + return target.startswith("/") or os.path.isabs(target) + + +def relative_sysroot_link(root, link, target): + """Rebase a sysroot-absolute link onto an existing target within the same sysroot.""" + if not target.startswith("/") or target.startswith("//"): + raise ValueError("not a rooted Linux sysroot link: %s -> %s" % (link, target)) + root = os.path.abspath(root) + destination = os.path.abspath(os.path.join(root, target.lstrip("/"))) + if os.path.commonpath((root, destination)) != root: + raise ValueError("a sysroot link leaves its root: %s -> %s" % (link, target)) + real_root = os.path.realpath(root) + real_destination = os.path.realpath(destination) + if os.path.commonpath((real_root, real_destination)) != real_root: + raise ValueError("a sysroot link resolves outside its root: %s -> %s" % (link, target)) + if not os.path.exists(destination): + raise FileNotFoundError("a sysroot link target is missing: %s -> %s" % (link, target)) + return os.path.relpath(destination, os.path.dirname(link)) diff --git a/builder/msi/HookshotBuilder.wxs b/builder/msi/HookshotBuilder.wxs new file mode 100644 index 0000000..8c9e2eb --- /dev/null +++ b/builder/msi/HookshotBuilder.wxs @@ -0,0 +1,49 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/builder/py/gen_strings.py b/builder/py/gen_strings.py new file mode 100644 index 0000000..6e4fa9c --- /dev/null +++ b/builder/py/gen_strings.py @@ -0,0 +1,57 @@ +"""Writes hookshot_builder/strings.py from builder/HookshotBuilder/Strings.cs (the WPF builder's languages): the same text, the same +order. Run again whenever Strings.cs changes: python -I builder/py/gen_strings.py""" +import json +import os +import re + +HERE = os.path.dirname(os.path.abspath(__file__)) +src = open(os.path.join(HERE, "..", "HookshotBuilder", "Strings.cs"), encoding="utf-8").read() +enum = re.search(r"enum S\s*\{(.*?)\}", src, re.S).group(1) +keys = [k.strip() for k in re.sub(r"//.*", "", enum).replace("\n", " ").split(",") if k.strip()] +arrays = {} +for m in re.finditer(r"static readonly string\[\] (\w+) =\s*\{(.*?)\n \};", src, re.S): + arrays[m.group(1)] = [json.loads('"' + s + '"') for s in re.findall(r'"((?:[^"\\]|\\.)*)"', m.group(2))] +langs = re.findall(r'new\("([^"]+)", "([^"]+)", (\w+)\)', src) + +out = ['"""The builder\'s languages, generated by gen_strings.py from builder/HookshotBuilder/Strings.cs: one of the game\'s languages', + 'is its code in language.txt, its own name, and the app\'s text in it. Do not edit by hand."""', + "", "KEYS = " + repr(keys), "", "TEXT = {"] +for name, arr in arrays.items(): + out.append(" %r: %s," % (name, json.dumps(arr, ensure_ascii=False))) +out += ["}", "", "LANGUAGES = [ # (code, its own name, the TEXT it uses)"] +out += [" (%r, %r, %r)," % l for l in langs] +out += ["]", ""] +out.append(''' + +def text(lang, key): + """A string in the language (a LANGUAGES entry), English where it has none (the update's lines are English only)""" + i = KEYS.index(key) + arr = TEXT[lang[2]] + return arr[i] if i < len(arr) else TEXT["En"][i] + + +def default_language(): + """The language closest to Windows' own: the exact one, then the same family, else English (US)""" + name = "" + try: + import ctypes + buf = ctypes.create_unicode_buffer(85) + if ctypes.windll.kernel32.GetUserDefaultLocaleName(buf, 85): + name = buf.value + except Exception: + pass + for lang in LANGUAGES: + if lang[0].lower() == name.lower(): + return lang + if name.startswith("zh"): + pick = "zh-Hant" if any(t in name for t in ("TW", "HK", "Hant")) else "zh-Hans" + elif name.startswith("es-") and name != "es-ES": + pick = "es-419" + elif name.startswith("pt"): + pick = "pt-BR" + else: + pick = name.split("-")[0] + return next((lang for lang in LANGUAGES if lang[0] == pick), LANGUAGES[0]) +''') +open(os.path.join(HERE, "hookshot_builder", "strings.py"), "w", encoding="utf-8", newline="\n").write("\n".join(out)) +print(len(keys), "keys;", {k: len(v) for k, v in arrays.items()}, len(langs), "languages") diff --git a/builder/py/hookshot_builder/__init__.py b/builder/py/hookshot_builder/__init__.py new file mode 100644 index 0000000..168273f --- /dev/null +++ b/builder/py/hookshot_builder/__init__.py @@ -0,0 +1,6 @@ +"""Hookshot Builder, in Python: the same window as the WPF builder (builder/HookshotBuilder), run by python.org's own python.exe and +Qt's DLLs (both signed by their makers), with no executable of Hookshot's own to download. It also does what the setup exe did +(builder/setup/setup.c): it reads a release's hookshot-setup.txt, downloads the parts it names, checks each one's SHA-256 and +unpacks them into %LOCALAPPDATA%\\HookshotBuilder\\versions\\.""" +APP_ID = "NotRexed.Hookshot.Builder" # (Windows' AppUserModelID: the taskbar groups and shows the window as Hookshot, not Python) +REPO = "NotRexed/Hookshot" diff --git a/builder/py/hookshot_builder/assets/Fonts/Inter-Bold.ttf b/builder/py/hookshot_builder/assets/Fonts/Inter-Bold.ttf new file mode 100644 index 0000000..9fb9b75 Binary files /dev/null and b/builder/py/hookshot_builder/assets/Fonts/Inter-Bold.ttf differ diff --git a/builder/py/hookshot_builder/assets/Fonts/Inter-LICENSE.txt b/builder/py/hookshot_builder/assets/Fonts/Inter-LICENSE.txt new file mode 100644 index 0000000..9b2ca37 --- /dev/null +++ b/builder/py/hookshot_builder/assets/Fonts/Inter-LICENSE.txt @@ -0,0 +1,92 @@ +Copyright (c) 2016 The Inter Project Authors (https://github.com/rsms/inter) + +This Font Software is licensed under the SIL Open Font License, Version 1.1. +This license is copied below, and is also available with a FAQ at: +http://scripts.sil.org/OFL + +----------------------------------------------------------- +SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007 +----------------------------------------------------------- + +PREAMBLE +The goals of the Open Font License (OFL) are to stimulate worldwide +development of collaborative font projects, to support the font creation +efforts of academic and linguistic communities, and to provide a free and +open framework in which fonts may be shared and improved in partnership +with others. + +The OFL allows the licensed fonts to be used, studied, modified and +redistributed freely as long as they are not sold by themselves. The +fonts, including any derivative works, can be bundled, embedded, +redistributed and/or sold with any software provided that any reserved +names are not used by derivative works. The fonts and derivatives, +however, cannot be released under any other type of license. The +requirement for fonts to remain under this license does not apply +to any document created using the fonts or their derivatives. + +DEFINITIONS +"Font Software" refers to the set of files released by the Copyright +Holder(s) under this license and clearly marked as such. This may +include source files, build scripts and documentation. + +"Reserved Font Name" refers to any names specified as such after the +copyright statement(s). + +"Original Version" refers to the collection of Font Software components as +distributed by the Copyright Holder(s). + +"Modified Version" refers to any derivative made by adding to, deleting, +or substituting -- in part or in whole -- any of the components of the +Original Version, by changing formats or by porting the Font Software to a +new environment. + +"Author" refers to any designer, engineer, programmer, technical +writer or other person who contributed to the Font Software. + +PERMISSION AND CONDITIONS +Permission is hereby granted, free of charge, to any person obtaining +a copy of the Font Software, to use, study, copy, merge, embed, modify, +redistribute, and sell modified and unmodified copies of the Font +Software, subject to the following conditions: + +1) Neither the Font Software nor any of its individual components, +in Original or Modified Versions, may be sold by itself. + +2) Original or Modified Versions of the Font Software may be bundled, +redistributed and/or sold with any software, provided that each copy +contains the above copyright notice and this license. These can be +included either as stand-alone text files, human-readable headers or +in the appropriate machine-readable metadata fields within text or +binary files as long as those fields can be easily viewed by the user. + +3) No Modified Version of the Font Software may use the Reserved Font +Name(s) unless explicit written permission is granted by the corresponding +Copyright Holder. This restriction only applies to the primary font name as +presented to the users. + +4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font +Software shall not be used to promote, endorse or advertise any +Modified Version, except to acknowledge the contribution(s) of the +Copyright Holder(s) and the Author(s) or with their explicit written +permission. + +5) The Font Software, modified or unmodified, in part or in whole, +must be distributed entirely under this license, and must not be +distributed under any other license. The requirement for fonts to +remain under this license does not apply to any document created +using the Font Software. + +TERMINATION +This license becomes null and void if any of the above conditions are +not met. + +DISCLAIMER +THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT +OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE +COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, +INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL +DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM +OTHER DEALINGS IN THE FONT SOFTWARE. diff --git a/builder/py/hookshot_builder/assets/Fonts/Inter-Medium.ttf b/builder/py/hookshot_builder/assets/Fonts/Inter-Medium.ttf new file mode 100644 index 0000000..458cd06 Binary files /dev/null and b/builder/py/hookshot_builder/assets/Fonts/Inter-Medium.ttf differ diff --git a/builder/py/hookshot_builder/assets/Fonts/Inter-Regular.ttf b/builder/py/hookshot_builder/assets/Fonts/Inter-Regular.ttf new file mode 100644 index 0000000..b7aaca8 Binary files /dev/null and b/builder/py/hookshot_builder/assets/Fonts/Inter-Regular.ttf differ diff --git a/builder/py/hookshot_builder/assets/Fonts/Inter-SemiBold.ttf b/builder/py/hookshot_builder/assets/Fonts/Inter-SemiBold.ttf new file mode 100644 index 0000000..47f8ab1 Binary files /dev/null and b/builder/py/hookshot_builder/assets/Fonts/Inter-SemiBold.ttf differ diff --git a/builder/py/hookshot_builder/assets/hookshot.ico b/builder/py/hookshot_builder/assets/hookshot.ico new file mode 100644 index 0000000..6c7aca6 Binary files /dev/null and b/builder/py/hookshot_builder/assets/hookshot.ico differ diff --git a/builder/py/hookshot_builder/assets/hookshot.png b/builder/py/hookshot_builder/assets/hookshot.png new file mode 100644 index 0000000..05168dc Binary files /dev/null and b/builder/py/hookshot_builder/assets/hookshot.png differ diff --git a/builder/py/hookshot_builder/core.py b/builder/py/hookshot_builder/core.py new file mode 100644 index 0000000..f8d07fe --- /dev/null +++ b/builder/py/hookshot_builder/core.py @@ -0,0 +1,557 @@ +"""What the WPF builder's Runner.cs, Storage.cs, Finder.cs and BuildWorkers.cs do, the same rules: the build tools, nx2/make_game.py as a +child process, what the builder keeps on the PC, the game files found on it, and the game's shortcuts.""" +import ctypes +import json +import os +import shutil +import string +import subprocess +import sys +import threading +import time +from ctypes import wintypes + +from . import paths + +CREATE_NO_WINDOW = 0x08000000 +BELOW_NORMAL_PRIORITY_CLASS = 0x00004000 + + +# ---------------- the build tools (Runner.cs: Tools) ---------------- + +class Tools: + """Where nx2/make_game.py and its Python are: the installed version's tools, or the source tree this app sits in""" + + def __init__(self, root, python, python_args=None): + self.root, self.python, self.python_args = root, python, python_args or [] + + @property + def version(self): + """Which release these tools are (tools/version.txt, written when the release is packed); None from the source tree""" + return paths.read_small(os.path.join(self.root, "version.txt")) or None + + @staticmethod + def find(): + from . import setup_parts + env = os.environ.get("HOOKSHOT_TOOLS") # (development: these tools) + cands = [env] if env else [] + cur = setup_parts.current() + if cur: + cands.append(os.path.join(cur, "HookshotBuilder", "tools")) + for c in cands: + if c and os.path.isfile(os.path.join(c, "nx2", "make_game.py")): + py = os.path.join(c, "python", "python.exe") + return Tools(c, py if os.path.isfile(py) else "python") + d = paths.APP_DIR + while True: # (a source checkout: builder/py/hookshot_builder in it) + if os.path.isfile(os.path.join(d, "nx2", "make_game.py")): + return Tools(d, sys.executable) + up = os.path.dirname(d) + if up == d: + return None + d = up + + +def build_env(tools, workers=None, ngx_sdk=None): + env = dict(os.environ) + env["PYTHONIOENCODING"] = "utf-8" + if workers: + env["HOOKSHOT_JOBS"] = str(int(workers)) + # (written into the game folder's hookshot_version.txt: the game starts this app again to update, host/hos_update.c) + exe, args = paths.launcher_command() + env["HOOKSHOT_BUILDER_EXE"] = exe + env["HOOKSHOT_BUILDER_ARGS"] = subprocess.list2cmdline(args) + # (NVIDIA DLSS: only when ticked. Without it, one set on the PC is not passed on either) + if ngx_sdk: + env["HOOKSHOT_NGX_SDK"] = ngx_sdk + else: + env.pop("HOOKSHOT_NGX_SDK", None) + for name, folder in (("EMSDK", "emsdk"), ("RX_CPM", "cpm"), ("WGPU_NATIVE", "wgpu-native")): + p = os.path.join(tools.root, folder) + if os.path.isdir(p): + env[name] = p + return env + + +def build_command(tools, args): + return [tools.python, *tools.python_args, "-I", os.path.join(tools.root, "nx2", "make_game.py"), *args] + + +class Runner: + """nx2/make_game.py as a child process: its lines as they come (on_line, from a worker thread), and how it ended (on_finished: + None, "cancelled", or why it stopped)""" + + def __init__(self, tools, args, workers=None, ngx_sdk=None, on_line=None, on_finished=None): + if workers is not None and workers <= 0: + raise ValueError("Build workers must be positive.") + self.tools, self.args, self.workers, self.ngx_sdk = tools, args, workers, ngx_sdk + self.on_line, self.on_finished = on_line or (lambda l: None), on_finished or (lambda f: None) + self.process, self.error, self.cancelled = None, None, False + + def start(self): + threading.Thread(target=self._run, daemon=True).start() + + def _run(self): + try: + p = subprocess.Popen(build_command(self.tools, self.args), cwd=self.tools.root, + env=build_env(self.tools, self.workers, self.ngx_sdk), stdout=subprocess.PIPE, + stderr=subprocess.PIPE, stdin=subprocess.DEVNULL, + creationflags=CREATE_NO_WINDOW | BELOW_NORMAL_PRIORITY_CLASS) + except OSError as e: + self.on_finished("The build tools could not start: %s" % e) + return + self.process = p + + def errs(): + for raw in p.stderr: + self.on_line(raw.decode("utf-8", "replace").rstrip("\r\n")) + threading.Thread(target=errs, daemon=True).start() + for raw in p.stdout: + line = raw.decode("utf-8", "replace").rstrip("\r\n") + if line.startswith("@@error "): + self.error = line[8:] + self.on_line(line) + p.wait() + self.on_finished("cancelled" if self.cancelled else None if p.returncode == 0 else + self.error or "The build tools stopped without saying why. See the details.") + + def cancel(self): + self.cancelled = True + p = self.process + if p and p.poll() is None: # (the whole tree: the compilers it started too) + subprocess.run(["taskkill", "/T", "/F", "/PID", str(p.pid)], capture_output=True, creationflags=CREATE_NO_WINDOW) + + +def check(tools, args): + """Is this the supported game and version? (ok, the answer in plain words)""" + try: + p = subprocess.run(build_command(tools, args), cwd=tools.root, env=build_env(tools), capture_output=True, + stdin=subprocess.DEVNULL, creationflags=CREATE_NO_WINDOW) + except OSError: + if tools.python == "python" and os.path.isdir(os.path.join(tools.root, "python")): + return False, ("The builder's own Python (tools/python/python.exe) is missing. Your antivirus probably removed it: restore " + "it from quarantine, then try again.") + return False, "Python 3 was not found. Install it from python.org (not the Microsoft Store build)." + result, answered = (False, "The game files could not be read."), False + for line in p.stdout.decode("utf-8", "replace").splitlines(): + if line.startswith("@@check ok "): + result, answered = (True, line[11:]), True + elif line.startswith("@@check bad "): + result, answered = (False, line[12:]), True + elif line.startswith("@@error "): + result, answered = (False, line[8:]), True + if not answered: + errs = [l for l in p.stderr.decode("utf-8", "replace").splitlines() if l.strip()] + last = errs[-1] if errs else None + if last and "microsoft store" in last.lower(): + return False, ("Python 3 was not found: the 'python' command on this PC is the Microsoft Store shortcut, which only opens " + "the Store. Install Python 3 from python.org (not the Store version).") + return False, last or "The game tools stopped without saying why (exit code %d)." % p.returncode + return result + + +# ---------------- build workers (BuildWorkers.cs) ---------------- + +AUTOMATIC = "Automatic" + + +def worker_choices(): + return [AUTOMATIC] + list(range(1, max(1, os.cpu_count() or 1) + 1)) + + +# ---------------- what the builder keeps on the PC (Storage.cs) ---------------- + +BUILD_FILES_BYTES = 4 << 30 + + +def keep_build_files(): + v = os.environ.get("HOOKSHOT_KEEP_BUILD_FILES", "") + return bool(v) and v != "0" + + +def _same(a, b): + return os.path.normcase(os.path.abspath(a)).rstrip("\\/") == os.path.normcase(os.path.abspath(b)).rstrip("\\/") + + +def work(): + """Where builds keep their files: the folder the player chose (work.txt), else the builder's own""" + chosen = paths.read_small(os.path.join(paths.base(), "work.txt")) + return chosen if len(chosen) > 2 and os.path.isabs(chosen) else paths.base() + + +def work_chosen(): + return not _same(work(), paths.base()) + + +def ngx_sdk(): + return os.path.join(paths.base(), "ngx-sdk") + + +WORK_PARTS = ("objcache", "sa", "exefs", "probe", "image", "system_headers.txt", "hot_funcs.txt", "hot_order.txt", + "hookshot_pieces.prof", "hookshot.res") + + +def _delete(p): + try: + if os.path.isdir(p): + shutil.rmtree(p) + elif os.path.isfile(p): + os.remove(p) + except OSError: + shutil.rmtree(p, ignore_errors=True) if os.path.isdir(p) else None + + +def move_work(picked): + """The build files go to a folder of their own in the one the player picked (None: back to the builder's own); what the last + builds left at the old place goes""" + old = work() + to = paths.base() if picked is None else os.path.join(picked, "Hookshot build files") + if _same(old, to): + return + os.makedirs(to, exist_ok=True) + os.makedirs(paths.base(), exist_ok=True) + if picked is None: + try: + os.remove(os.path.join(paths.base(), "work.txt")) + except OSError: + pass + else: + paths.write_small(os.path.join(paths.base(), "work.txt"), to) + for n in WORK_PARTS: + _delete(os.path.join(old, n)) + if not _same(old, paths.base()): + try: + os.rmdir(old) + except OSError: + pass + + +def _kept_version(d): + from . import setup_parts + cur = setup_parts.current() + return bool(cur) and _same(cur, d) + + +def freeable(after_build=False): + for d in (("sa", "exefs", "probe") if after_build else ("objcache", "sa", "exefs", "probe")): + yield os.path.join(work(), d) + for d in ("downloads", "update"): + yield os.path.join(paths.base(), d) + v = paths.versions() + if os.path.isdir(v): + for name in os.listdir(v): + p = os.path.join(v, name) + if os.path.isdir(p) and not _kept_version(p): + yield p + + +def _size(d, seen): + n = 0 + for root, _dirs, files in os.walk(d): + for f in files: + try: + st = os.stat(os.path.join(root, f)) + except OSError: + continue + if not st.st_size: + continue + if st.st_nlink > 1: # (an object hard-linked in both sa\build and objcache counted once) + key = (st.st_dev, st.st_ino) + if key in seen: + continue + seen.add(key) + n += st.st_size + return n + + +def measure(): + """(bytes in the builder's folder and the build files folder, bytes Free lets go)""" + seen = set() + free = sum(_size(d, seen) for d in freeable()) + seen = set() + total = _size(paths.base(), seen) + (_size(work(), seen) if work_chosen() else 0) + return max(total, free), free + + +def free(after_build=False): + """Lets go of what measure counts as freeable; returns the bytes it could not delete (a file in use)""" + for d in list(freeable(after_build)): + _delete(d) + if not after_build: + _delete(os.path.join(work(), "system_headers.txt")) + return measure()[1] + seen = set() + return sum(_size(d, seen) for d in freeable(True)) + + +def gb(n): + return "%.1f GB" % (n / (1 << 30)) if n >= 1 << 30 else "%d MB" % (n >> 20) + + +def free_space(path): + try: + root = os.path.splitdrive(os.path.abspath(path))[0] + "\\" + return root, shutil.disk_usage(root).free + except OSError: + return None, None + + +def spinning(path): + """True on a hard drive, False on an SSD, None when the drive cannot be told (Disk.cs's IOCTL: the volume's seek penalty). + A build reads and writes tens of gigabytes, and an SSD makes that much faster: a warning only, never a stop.""" + try: + from ctypes import wintypes + root = os.path.splitdrive(os.path.abspath(os.path.expanduser(path)))[0] + if not root: + return None + + class Query(ctypes.Structure): + _fields_ = [("PropertyId", wintypes.DWORD), ("QueryType", wintypes.DWORD), ("Additional", wintypes.DWORD)] + + class SeekPenalty(ctypes.Structure): + _fields_ = [("Version", wintypes.DWORD), ("Size", wintypes.DWORD), ("IncursSeekPenalty", ctypes.c_ubyte)] + + k = ctypes.windll.kernel32 + k.CreateFileW.argtypes = [wintypes.LPCWSTR, wintypes.DWORD, wintypes.DWORD, ctypes.c_void_p, + wintypes.DWORD, wintypes.DWORD, wintypes.HANDLE] + k.CreateFileW.restype = wintypes.HANDLE + k.DeviceIoControl.argtypes = [wintypes.HANDLE, wintypes.DWORD, ctypes.c_void_p, wintypes.DWORD, + ctypes.c_void_p, wintypes.DWORD, ctypes.POINTER(wintypes.DWORD), ctypes.c_void_p] + k.DeviceIoControl.restype = wintypes.BOOL + k.CloseHandle.argtypes = [wintypes.HANDLE] + handle = k.CreateFileW("\\\\.\\" + root, 0, 0x1 | 0x2, None, 3, 0, None) # (read + write share, OPEN_EXISTING) + if not handle or handle == wintypes.HANDLE(-1).value: + return None + try: + query, answer, returned = Query(7, 0, 0), SeekPenalty(), wintypes.DWORD() + ok = k.DeviceIoControl(handle, 0x002D1400, ctypes.byref(query), ctypes.sizeof(query), ctypes.byref(answer), + ctypes.sizeof(answer), ctypes.byref(returned), None) + if not ok or returned.value < ctypes.sizeof(answer): + return None + return bool(answer.IncursSeekPenalty) + finally: + k.CloseHandle(handle) + except (OSError, ValueError, AttributeError): + return None + + +# ---------------- the player's game files (Finder.cs) ---------------- + +class Dump: + def __init__(self, path, where, size, modified): + self.path, self.where, self.size, self.modified = path, where, size, modified + + +def _known_folder(fid): + buf = ctypes.c_wchar_p() + guid = (ctypes.c_byte * 16).from_buffer_copy(fid) + if ctypes.windll.shell32.SHGetKnownFolderPath(guid, 0, None, ctypes.byref(buf)) == 0: + p = buf.value + ctypes.windll.ole32.CoTaskMemFree(buf) + return p + return None + + +def _guid(s): + import uuid + return uuid.UUID(s).bytes_le + + +FOLDER_DESKTOP = _guid("B4BFCC3A-DB2C-424C-B029-7FE99A87C641") +FOLDER_DOCUMENTS = _guid("FDD39AD0-238F-46AF-ADB4-6C85480369C7") +FOLDER_DOWNLOADS = _guid("374DE290-123F-4565-9164-39C4925E467B") +FOLDER_PROGRAMS = _guid("A77F5D77-2E2B-44C3-A6A2-ABA601054A51") + + +def find_dumps(): + """.nup files in Downloads, Desktop, Documents, the home folder and the top of each drive, one folder level down too, newest first""" + home = os.path.expanduser("~") + places = [(_known_folder(FOLDER_DOWNLOADS) or os.path.join(home, "Downloads"), "Downloads"), + (_known_folder(FOLDER_DESKTOP) or os.path.join(home, "Desktop"), "Desktop"), + (_known_folder(FOLDER_DOCUMENTS) or os.path.join(home, "Documents"), "Documents"), (home, "Home")] + mask = ctypes.windll.kernel32.GetLogicalDrives() + for i, letter in enumerate(string.ascii_uppercase): + if mask >> i & 1: + root = letter + ":\\" + if ctypes.windll.kernel32.GetDriveTypeW(root) in (2, 3): # (removable, fixed) + places.append((root, letter + ":")) + seen, found = set(), [] + for d, name in places: + if not d or not os.path.isdir(d) or os.path.normcase(os.path.abspath(d)) in seen: + continue + seen.add(os.path.normcase(os.path.abspath(d))) + dirs = [d] + try: + subs = [e.path for e in os.scandir(d) if e.is_dir() and not e.name.startswith((".", "$")) + and e.name.lower() != "windows" and not e.name.lower().startswith("program")] + dirs += subs[:200] + except OSError: + pass + for sub in dirs: + try: + for e in os.scandir(sub): + if e.is_file() and e.name.lower().endswith(".nup"): + full = os.path.abspath(e.path) + if os.path.normcase(full) not in seen: + seen.add(os.path.normcase(full)) + st = e.stat() + found.append(Dump(full, name, st.st_size, st.st_mtime)) + except OSError: + pass + return sorted(found, key=lambda x: -x.modified) + + +def ago(t): + import datetime + now, then = datetime.datetime.now(), datetime.datetime.fromtimestamp(t) + days = (now - then).total_seconds() / 86400 + if days < 1 and then.date() == now.date(): + return "modified today" + if days < 2: + return "modified yesterday" + if days < 14: + return "%d days ago" % int(days) + if days < 60: + return "%d weeks ago" % int(days / 7) + return "%d %s" % (then.day, then.strftime("%b %Y")) + + +def size_text(n): + return "%.1f GB" % (n / (1 << 30)) if n >= 1 << 30 else "%d MB" % (n >> 20) + + +# ---------------- the game folder ---------------- + +def _version_line(folder): + for line in paths.read_small(os.path.join(folder, "hookshot_version.txt")).splitlines(): + if line.startswith("version="): + return line[8:].strip() or None + return None + + +def game_version(folder): + """The release a game folder was built with, when it keeps its game files for updates""" + if not os.path.isdir(os.path.join(folder, ".hookshot_exefs")) or not os.path.isdir(os.path.join(folder, "romfs")): + return None + return _version_line(folder) + + +def read_version(folder): + """(a folder built before it kept its game files has a version too: only shown)""" + return _version_line(folder) + + +def game_exe(folder): + g = os.path.join(folder, "bin", "hookshot.exe") + return g if os.path.isfile(g) else os.path.join(folder, "hookshot.exe") + + +def start_game(folder): + """(the launcher already checked for updates: the game does not ask again)""" + env = dict(os.environ, HOOKSHOT_NO_UPDATE_CHECK="1") + subprocess.Popen([game_exe(folder)], cwd=folder, env=env) + + +def open_folder(folder): + subprocess.Popen(["explorer.exe", folder]) + + +# ---------------- shortcuts (Finder.cs: IShellLinkW + IPersistFile, Windows' own shell object writes the .lnk) ---------------- + +def desktop_link(): + return os.path.join(_known_folder(FOLDER_DESKTOP) or os.path.join(os.path.expanduser("~"), "Desktop"), "Hookshot.lnk") + + +def start_link(): + return os.path.join(_known_folder(FOLDER_PROGRAMS) or "", "Hookshot.lnk") + + +def folder_link(game_folder): + """The game folder's own launcher: Play, and Update left of it when a newer release is out (the setup exe's launch mode)""" + return os.path.join(game_folder, "Hookshot.lnk") + + +shortcut_error = None + + +def _com_call(obj, index, restype, *args): + vtbl = ctypes.cast(obj, ctypes.POINTER(ctypes.POINTER(ctypes.c_void_p))).contents + fn = ctypes.WINFUNCTYPE(restype, ctypes.c_void_p, *[type(a) for a in args])(vtbl[index]) + return fn(obj, *args) + + +def make_link(link, target, args, workdir, icon, description): + ole32 = ctypes.windll.ole32 + ole32.CoInitialize(None) + clsid = (ctypes.c_byte * 16).from_buffer_copy(_guid("00021401-0000-0000-C000-000000000046")) + iid_link = (ctypes.c_byte * 16).from_buffer_copy(_guid("000214F9-0000-0000-C000-000000000046")) + iid_file = (ctypes.c_byte * 16).from_buffer_copy(_guid("0000010B-0000-0000-C000-000000000046")) + sl = ctypes.c_void_p() + hr = ole32.CoCreateInstance(clsid, None, 1, iid_link, ctypes.byref(sl)) + if hr != 0: + raise OSError("CoCreateInstance(ShellLink) failed: 0x%08x" % (hr & 0xFFFFFFFF)) + try: + W = ctypes.c_wchar_p + _com_call(sl, 20, ctypes.HRESULT, W(target)) # SetPath + _com_call(sl, 11, ctypes.HRESULT, W(args)) # SetArguments + _com_call(sl, 9, ctypes.HRESULT, W(workdir)) # SetWorkingDirectory + _com_call(sl, 17, ctypes.HRESULT, W(icon), ctypes.c_int(0)) # SetIconLocation + _com_call(sl, 7, ctypes.HRESULT, W(description)) # SetDescription + pf = ctypes.c_void_p() + _com_call(sl, 0, ctypes.HRESULT, ctypes.cast(iid_file, ctypes.c_void_p), ctypes.byref(pf)) # QueryInterface + try: + os.makedirs(os.path.dirname(link), exist_ok=True) + _com_call(pf, 6, ctypes.HRESULT, W(link), wintypes.BOOL(True)) # IPersistFile::Save + finally: + _com_call(pf, 2, ctypes.c_ulong) + finally: + _com_call(sl, 2, ctypes.c_ulong) + + +def shortcuts(game_folder, desktop, start, folder=True): + """Made or removed as the ticks say: each starts this app's launcher for the folder (Play, and Update when a newer release is + out), with the game's own icon""" + global shortcut_error + exe, args = paths.launcher_command() + args = subprocess.list2cmdline(args + ["--launch", game_folder]) + icon = os.path.join(game_folder, "bin", "hookshot.exe") + if not os.path.isfile(icon): + icon = os.path.join(game_folder, "hookshot.exe") + for link, want in ((desktop_link(), desktop), (start_link(), start), (folder_link(game_folder), folder)): + try: + if not want: + if os.path.isfile(link): + os.remove(link) + continue + if not os.path.isfile(icon): + continue + make_link(link, exe, args, game_folder, icon, "The Legend of Zelda: Ocarina of Time, built by Hookshot") + except OSError as e: + shortcut_error = "%s: %s" % (os.path.basename(link), e) + + +def refresh_shortcuts(game_folder): + shortcuts(game_folder, os.path.isfile(desktop_link()), os.path.isfile(start_link())) + + +# ---------------- what the player chose last time (builder-settings.json, the WPF builder's) ---------------- + +def settings_path(): + return os.path.join(paths.base(), "builder-settings.json") + + +def load_settings(): + try: + with open(settings_path(), encoding="utf-8") as f: + return json.load(f) + except (OSError, ValueError): + return {} + + +def save_settings(s): + try: + os.makedirs(paths.base(), exist_ok=True) + with open(settings_path(), "w", encoding="utf-8") as f: + json.dump(s, f) + except OSError: + pass diff --git a/builder/py/hookshot_builder/paths.py b/builder/py/hookshot_builder/paths.py new file mode 100644 index 0000000..629bd23 --- /dev/null +++ b/builder/py/hookshot_builder/paths.py @@ -0,0 +1,45 @@ +"""Where things are: the builder's own folder in %LOCALAPPDATA%, and this app's folder.""" +import os +import sys + +APP_DIR = os.path.dirname(os.path.abspath(__file__)) +ASSETS = os.path.join(APP_DIR, "assets") + + +def base(): + """%LOCALAPPDATA%\\HookshotBuilder (HOOKSHOT_SETUP_BASE=: another, for testing installs and updates)""" + b = os.environ.get("HOOKSHOT_SETUP_BASE") + if b: + return b + return os.path.join(os.environ.get("LOCALAPPDATA") or os.path.expanduser("~"), "HookshotBuilder") + + +def versions(): + return os.path.join(base(), "versions") + + +def read_small(path): + try: + with open(path, encoding="utf-8", errors="replace") as f: + return f.read().strip() + except OSError: + return "" + + +def write_small(path, text): + try: + os.makedirs(os.path.dirname(path), exist_ok=True) + with open(path, "w", encoding="utf-8", newline="\n") as f: + f.write(text) + except OSError: + pass + + +def launcher_command(): + """How this app is started again (the game folder's launcher, the game's update offer, a shortcut): the Python that runs it + (pythonw.exe beside python.exe when there is one: no console window) and the app's start script""" + exe = sys.executable + w = os.path.join(os.path.dirname(exe), "pythonw.exe") + if os.path.isfile(w): + exe = w + return exe, ["-I", os.path.join(os.path.dirname(APP_DIR), "hookshot_builder_start.py")] diff --git a/builder/py/hookshot_builder/setup_parts.py b/builder/py/hookshot_builder/setup_parts.py new file mode 100644 index 0000000..6f431a7 --- /dev/null +++ b/builder/py/hookshot_builder/setup_parts.py @@ -0,0 +1,337 @@ +"""The setup's work (builder/setup/setup.c, the same rules): a release's hookshot-setup.txt names its tag and its parts (one zip each, +with its SHA-256 and size); each part is downloaded (or moved over from the installed version when its checksum did not change), +checked and unpacked into versions\\.part, which becomes versions\\ once complete. The version installed stays, the others +and the downloads go. + +hookshot-setup.txt, one item a line: + tag + part + run (not needed here: the tools are what this app uses)""" +import hashlib +import json +import os +import re +import shutil +import time +import urllib.error +import urllib.request +import zipfile + +from . import REPO, paths + +UA = "HookshotBuilder-Setup/2.0" +WAITS = (3, 6, 12, 20, 30, 45, 60) # (8 tries over ~3 minutes: GitHub's release downloads answered 504 for hours on 2026-11-08) +OWN = ("python", "emsdk", "cpm", "third_party", "wgpu-native") + + +class SetupError(Exception): + pass + + +class Part: + def __init__(self, sha, size, dest, frm, to, url): + self.sha, self.size, self.dest, self.frm, self.to, self.url = sha, int(size), dest, frm, to, url + + +def retryable(status): + """an answer worth trying again: none, a server error, a download cut off half way (200), a timeout or the API's rate limit""" + return not status or status >= 500 or status in (200, 408, 429, 403) + + +class Http: + def __init__(self): + self.status = 0 + + def get(self, url, sink=None, expect=0, accept=None, progress=None, cancel=None): + """GET following redirects (never to http); the bytes to sink, or returned. Sets self.status; raises on failure""" + if not url.startswith("https://"): + raise SetupError("Only https downloads are allowed: " + url) + req = urllib.request.Request(url, headers={"User-Agent": UA, **({"Accept": accept} if accept else {})}) + self.status = 0 + try: + with urllib.request.urlopen(req, timeout=60) as r: + if not r.geturl().startswith("https://"): + raise SetupError("A download was redirected away from https") + self.status = r.status + got, out = 0, [] + while True: + if cancel and cancel(): + raise SetupError("cancelled") + b = r.read(1 << 16) + if not b: + break + got += len(b) + if sink: + sink(b) + else: + out.append(b) + if progress and expect: + progress(min(1.0, got / expect)) + if expect and got != expect: + self.status = 200 # (cut off half way: worth another try) + raise SetupError("The download ended early (%d of %d MB)" % (got >> 20, expect >> 20)) + return None if sink else b"".join(out) + except urllib.error.HTTPError as e: + self.status = e.code + raise SetupError("GitHub returned HTTP %d" % e.code) from None + except (urllib.error.URLError, OSError, TimeoutError) as e: + if not isinstance(e, SetupError): + self.status = self.status if self.status and self.status != 200 else 0 + raise SetupError("Could not reach GitHub (%s)" % getattr(e, "reason", e)) from None + raise + + +def api_link_for(http, url): + """https://github.com///releases/download// (or .../latest/download/) -> the API's own link to + that file, which redirects to a fresh download (GitHub's cached release links answered 504 for hours while its API answered)""" + m = re.match(r"https://github\.com/([^/]+/[^/]+)/releases/(?:download/([^/]+)|latest/download)/([^/?]+)$", url) + if not m: + return None + repo, tag, name = m.groups() + api = "https://api.github.com/repos/%s/releases/%s" % (repo, ("tags/" + tag) if tag else "latest") + try: + doc = json.loads(http.get(api, accept="application/vnd.github+json")) + for a in doc.get("assets", []): + if a.get("name") == name and str(a.get("url", "")).startswith("https://api.github.com/"): + return a["url"] + except (SetupError, ValueError): + pass + return None + + +def manifest_url(tag=None): + """(an update to a named release reads THAT release's list: the latest/download link can answer with the release before it for + a while after a publish. HOOKSHOT_SETUP_LIST=: another list, for testing a release before it is the latest)""" + e = os.environ.get("HOOKSHOT_SETUP_LIST") + if e: + return e + if tag: + return "https://github.com/%s/releases/download/%s/hookshot-setup.txt" % (REPO, tag) + return "https://github.com/%s/releases/latest/download/hookshot-setup.txt" % REPO + + +def read_manifest(status, tag=None, cancel=None): + """(tag, parts) of the release's list, tried again on a server error or no answer""" + http, url = Http(), manifest_url(tag) + text, last = None, "" + for attempt in range(len(WAITS) + 1): + if attempt: + status("GitHub didn't answer (%s). Trying again in %d s..." % (http.status or "no answer", WAITS[attempt - 1])) + time.sleep(WAITS[attempt - 1]) + status("Looking for the latest Hookshot Builder...") + try: + text = http.get(url, cancel=cancel).decode("utf-8", "replace") + break + except SetupError as e: + last = str(e) + first = http.status + if not first or first >= 500: # (the API's link instead) + api = api_link_for(http, url) + if api: + try: + text = http.get(api, accept="application/octet-stream", cancel=cancel).decode("utf-8", "replace") + break + except SetupError: + http.status = first + if not retryable(http.status): + break + if text is None: + if http.status == 404: + raise SetupError("The latest Hookshot release has no setup files yet. Download it from the releases page for now: " + "https://github.com/%s/releases/latest" % REPO) + if http.status: + raise SetupError("GitHub's download servers are failing right now (an error 500 or above, tried 8 times over 3 minutes). " + "This is on GitHub's side: try again in a few minutes, or later today.") + raise SetupError("Could not reach GitHub to get the Hookshot Builder (%s). Check your internet connection and try again." % last) + got_tag, parts = "", [] + for line in text.splitlines(): + p = line.split() + if len(p) >= 2 and p[0] == "tag": + got_tag = p[1] + elif len(p) == 7 and p[0] == "part": + parts.append(Part(*p[1:])) + if not got_tag or not parts: + raise SetupError("The release's setup list could not be read. Try again later.") + if tag and tag != got_tag: + raise SetupError("GitHub answered with the setup list of %s, not %s (its download links can lag right after a release). " + "Try the update again in a few minutes." % (got_tag, tag)) + return got_tag, parts + + +def part_folder(p): + """The folder a part unpacks into, relative to a version's folder (None: the part has none of its own)""" + if p.to != "-" and p.dest != ".": + return os.path.join(p.dest, p.to).replace("/", os.sep) + b = p.url.rsplit("/", 1)[-1] + for own in OWN: + if b.startswith("HookshotBuilder-%s-" % own): + return os.path.join("HookshotBuilder", "tools", own) + return None + + +def reuse_part(tag, p, install): + """A part whose checksum the installed version lists in its parts.txt (and whose folder it has) is moved over from it instead of + downloaded: that version is deleted after the install anyway""" + rel = part_folder(p) + was = paths.read_small(os.path.join(paths.versions(), "current.txt")) + if not rel or not was or was == tag: + return False + if p.sha not in paths.read_small(os.path.join(paths.versions(), was, "parts.txt")): + return False + src, dst = os.path.join(paths.versions(), was, rel), os.path.join(install, rel) + if not os.path.exists(src): + return False + os.makedirs(os.path.dirname(dst), exist_ok=True) + try: + os.replace(src, dst) + return True + except OSError: + return False + + +def rmtree(p): + shutil.rmtree(p, ignore_errors=True) + + +def installed(tag): + d = os.path.join(paths.versions(), tag) + return d if os.path.isfile(os.path.join(d, "installed.txt")) and os.path.isfile( + os.path.join(d, "HookshotBuilder", "tools", "nx2", "make_game.py")) else None + + +def current(): + """The version installed last, its folder (None: none yet)""" + tag = paths.read_small(os.path.join(paths.versions(), "current.txt")) + return installed(tag) if tag else None + + +def install(status, progress=lambda f: None, tag=None, repair=False, cancel=None): + """Installs the release (the latest, or tag) unless it is there already; returns (tag, its folder)""" + tag, parts = read_manifest(status, tag, cancel) + versions, downloads = paths.versions(), os.path.join(paths.base(), "downloads") + final = os.path.join(versions, tag) + if repair or not installed(tag): + work = final + ".part" + rmtree(work) + rmtree(final) + os.makedirs(work, exist_ok=True) + os.makedirs(downloads, exist_ok=True) + n = len(parts) + for i, p in enumerate(parts, 1): + if cancel and cancel(): + raise SetupError("cancelled") + if reuse_part(tag, p, work): + status("Part %d of %d unchanged: kept from the installed version" % (i, n)) + continue + zpath = os.path.join(downloads, p.sha + ".zip") + if not (os.path.isfile(zpath) and sha256_file(zpath) == p.sha): + download_part(status, progress, p, i, n, zpath, cancel) + status("Unpacking part %d of %d..." % (i, n)) + progress(-1) + dest = work if p.dest == "." else os.path.join(work, p.dest.replace("/", os.sep)) + os.makedirs(dest, exist_ok=True) + try: + with zipfile.ZipFile(zpath) as z: + z.extractall(dest) + except (OSError, zipfile.BadZipFile) as e: + raise SetupError("Unpacking part %d of %d failed: %s" % (i, n, e)) from None + if p.frm != "-" and p.to != "-": # (llvm-mingw--ucrt-x86_64 -> llvm-mingw) + src, dst = os.path.join(dest, p.frm), os.path.join(dest, p.to) + if not os.path.exists(dst): + try: + os.replace(src, dst) + except OSError: + raise SetupError("Could not set up part %d of %d (renaming %s)." % (i, n, p.frm)) from None + if not os.path.isfile(os.path.join(work, "HookshotBuilder", "tools", "nx2", "make_game.py")): + raise SetupError("The build tools are missing after unpacking. Your antivirus may have removed them: allow the folder %s " + "and try again." % paths.base()) + try: + os.replace(work, final) + except OSError: + raise SetupError("Could not finish installing into %s." % final) from None + paths.write_small(os.path.join(final, "installed.txt"), tag) + write_parts(final, parts) + elif not os.path.isfile(os.path.join(final, "parts.txt")): + write_parts(final, parts) + prune(tag, parts) + rmtree(downloads) # (installed: its zips are not kept; an update moves unchanged folders over instead) + return tag, final + + +def download_part(status, progress, p, i, n, zpath, cancel): + http, part = Http(), zpath + ".part" + for attempt in range(len(WAITS) + 1): + if attempt: + status("Part %d of %d: GitHub didn't answer (%s). Trying again in %d s..." % (i, n, http.status or "no answer", + WAITS[attempt - 1])) + time.sleep(WAITS[attempt - 1]) + status("Downloading part %d of %d (%d MB)..." % (i, n, p.size >> 20)) + h = hashlib.sha256() + try: + with open(part, "wb") as f: + def sink(b): + f.write(b) + h.update(b) + url = api_link_for(http, p.url) if attempt % 2 == 1 else None # (the API's link, every other try) + if url: + http.get(url, sink, p.size, "application/octet-stream", progress, cancel) + else: + http.get(p.url, sink, p.size, None, progress, cancel) + except SetupError as e: + if str(e) == "cancelled": + raise + if not retryable(http.status): + break + continue + except OSError: + raise SetupError("Could not write to %s." % os.path.dirname(zpath)) from None + progress(-1) + if h.hexdigest() != p.sha: + os.remove(part) + raise SetupError("Part %d of %d did not download correctly (its checksum differs). Try again." % (i, n)) + os.replace(part, zpath) + return + try: + os.remove(part) + except OSError: + pass + raise SetupError(("Downloading part %d of %d failed: GitHub's download servers are failing right now (tried 8 times over 3 " + "minutes). This is on GitHub's side. The parts already downloaded are kept: try again in a few minutes." + if retryable(http.status) else + "Downloading part %d of %d failed. Check your connection and free space, then try again.") % (i, n)) + + +def sha256_file(path): + h = hashlib.sha256() + with open(path, "rb") as f: + for b in iter(lambda: f.read(1 << 20), b""): + h.update(b) + return h.hexdigest() + + +def write_parts(install_dir, parts): + """The checksums of the parts this version holds in folders of their own (reuse_part reads them at the next update)""" + paths.write_small(os.path.join(install_dir, "parts.txt"), "".join(p.sha + "\n" for p in parts if part_folder(p))) + + +def prune(tag, parts): + """Only the version installed stays (a folder still in use cannot be renamed: it goes at a later run), and downloads the list + no longer names go""" + versions = paths.versions() + was = paths.read_small(os.path.join(versions, "current.txt")) + if was != tag: + if was: + paths.write_small(os.path.join(versions, "previous.txt"), was) + paths.write_small(os.path.join(versions, "current.txt"), tag) + for name in os.listdir(versions): + p = os.path.join(versions, name) + if not os.path.isdir(p) or name == tag: + continue + if name.endswith(".gone"): + rmtree(p) + continue + try: + os.replace(p, p + ".gone") + rmtree(p + ".gone") + except OSError: + pass diff --git a/builder/py/hookshot_builder/strings.py b/builder/py/hookshot_builder/strings.py new file mode 100644 index 0000000..f218a7e --- /dev/null +++ b/builder/py/hookshot_builder/strings.py @@ -0,0 +1,71 @@ +"""The builder's languages, generated by gen_strings.py from builder/HookshotBuilder/Strings.cs: one of the game's languages +is its code in language.txt, its own name, and the app's text in it. Do not edit by hand.""" + +KEYS = ['Subtitle', 'GameFiles', 'Nup', 'Folders', 'DropHint', 'SaveTo', 'Build', 'Browse', 'Needs', 'Free', 'Checking', 'VersionOk', 'VersionBad', 'ToolsMissing', 'PythonMissing', 'Building', 'Patience', 'Elapsed', 'Read', 'Lift', 'Compile', 'Finish', 'ShowDetails', 'HideDetails', 'Cancel', 'Back', 'OpenFolder', 'Play', 'Ready', 'BuiltIn', 'Stopped', 'Cancelled', 'CancelledNote', 'CloseQuestion', 'LanguageLabel', 'Download', 'Updating', 'UpdateFailed'] + +TEXT = { + 'En': ["Turns your own copy of Ocarina of Time into a game folder you can start on PC.", "Game files", ".nup file", "romfs and exefs", "You can also drop a file or folder onto this window.", "Save the game to", "Build", "Browse", "Takes about 45 minutes and needs 45 GB.", "{0} has {1} GB free.", "Checking the game version...", "This is the version Hookshot supports.", "This version isn't supported. Hookshot needs build e0f3ccfa.", "The Hookshot tools were not found next to this app.", "Python was not found.", "Building", "This takes a while, and your PC stays usable.", "Elapsed", "Reading your game files", "Translating the game to PC code", "Compiling", "Putting the game folder together", "Show details", "Hide details", "Cancel", "Back", "Open folder", "Play", "Your game is ready", "built in", "The build stopped", "Build cancelled", "You can start again from the same folder.", "Stop the build and close?", "Language", "Downloading the new version", "Updating your game", "The update could not be downloaded"], + 'Fr': ["Transforme votre copie d'Ocarina of Time en un dossier de jeu à lancer sur PC.", "Fichiers du jeu", "Fichier .nup", "romfs et exefs", "Vous pouvez aussi déposer un fichier ou un dossier sur cette fenêtre.", "Enregistrer le jeu dans", "Créer", "Parcourir", "Prend environ 45 minutes et nécessite 45 Go.", "{0} : {1} Go libres.", "Vérification de la version du jeu...", "Cette version est prise en charge par Hookshot.", "Cette version n'est pas prise en charge. Hookshot a besoin de la version e0f3ccfa.", "Les outils Hookshot sont introuvables à côté de cette application.", "Python est introuvable.", "Création en cours", "Cela prend du temps, et votre PC reste utilisable.", "Temps écoulé", "Lecture de vos fichiers de jeu", "Traduction du jeu en code PC", "Compilation", "Assemblage du dossier du jeu", "Afficher les détails", "Masquer les détails", "Annuler", "Retour", "Ouvrir le dossier", "Jouer", "Votre jeu est prêt", "créé en", "La création s'est arrêtée", "Création annulée", "Vous pouvez recommencer depuis le même dossier.", "Arrêter la création et fermer ?", "Langue"], + 'De': ["Macht aus deiner eigenen Kopie von Ocarina of Time einen Spielordner für den PC.", "Spieldateien", ".nup-Datei", "romfs und exefs", "Du kannst auch eine Datei oder einen Ordner auf dieses Fenster ziehen.", "Spiel speichern unter", "Erstellen", "Durchsuchen", "Dauert etwa 45 Minuten und benötigt 45 GB.", "{0} hat {1} GB frei.", "Spielversion wird geprüft...", "Diese Version wird von Hookshot unterstützt.", "Diese Version wird nicht unterstützt. Hookshot benötigt Build e0f3ccfa.", "Die Hookshot-Werkzeuge wurden neben dieser App nicht gefunden.", "Python wurde nicht gefunden.", "Wird erstellt", "Das dauert eine Weile, dein PC bleibt nutzbar.", "Vergangen", "Spieldateien werden gelesen", "Spiel wird in PC-Code übersetzt", "Kompilieren", "Spielordner wird zusammengestellt", "Details anzeigen", "Details ausblenden", "Abbrechen", "Zurück", "Ordner öffnen", "Spielen", "Dein Spiel ist fertig", "erstellt in", "Die Erstellung wurde angehalten", "Erstellung abgebrochen", "Du kannst im selben Ordner neu beginnen.", "Erstellung beenden und schließen?", "Sprache"], + 'It': ["Trasforma la tua copia di Ocarina of Time in una cartella di gioco da avviare su PC.", "File di gioco", "File .nup", "romfs ed exefs", "Puoi anche trascinare un file o una cartella su questa finestra.", "Salva il gioco in", "Crea", "Sfoglia", "Richiede circa 45 minuti e 45 GB.", "{0} ha {1} GB liberi.", "Controllo della versione del gioco...", "Questa versione è supportata da Hookshot.", "Questa versione non è supportata. Hookshot richiede la build e0f3ccfa.", "Gli strumenti di Hookshot non sono stati trovati accanto a questa app.", "Python non è stato trovato.", "Creazione in corso", "Ci vuole un po', e il PC resta utilizzabile.", "Trascorso", "Lettura dei file di gioco", "Traduzione del gioco in codice PC", "Compilazione", "Assemblaggio della cartella di gioco", "Mostra dettagli", "Nascondi dettagli", "Annulla", "Indietro", "Apri cartella", "Gioca", "Il tuo gioco è pronto", "creato in", "La creazione si è interrotta", "Creazione annullata", "Puoi ricominciare dalla stessa cartella.", "Interrompere la creazione e chiudere?", "Lingua"], + 'Es': ["Convierte tu copia de Ocarina of Time en una carpeta de juego para iniciar en PC.", "Archivos del juego", "Archivo .nup", "romfs y exefs", "También puedes soltar un archivo o una carpeta en esta ventana.", "Guardar el juego en", "Crear", "Examinar", "Tarda unos 45 minutos y necesita 45 GB.", "{0} tiene {1} GB libres.", "Comprobando la versión del juego...", "Hookshot admite esta versión.", "Esta versión no es compatible. Hookshot necesita la build e0f3ccfa.", "No se encontraron las herramientas de Hookshot junto a esta aplicación.", "No se encontró Python.", "Creando", "Tarda un rato, y tu PC sigue siendo utilizable.", "Transcurrido", "Leyendo tus archivos del juego", "Traduciendo el juego a código de PC", "Compilando", "Preparando la carpeta del juego", "Mostrar detalles", "Ocultar detalles", "Cancelar", "Atrás", "Abrir carpeta", "Jugar", "Tu juego está listo", "creado en", "La creación se detuvo", "Creación cancelada", "Puedes empezar de nuevo desde la misma carpeta.", "¿Detener la creación y cerrar?", "Idioma"], + 'Pt': ["Transforma sua cópia de Ocarina of Time em uma pasta de jogo para iniciar no PC.", "Arquivos do jogo", "Arquivo .nup", "romfs e exefs", "Você também pode soltar um arquivo ou uma pasta nesta janela.", "Salvar o jogo em", "Criar", "Procurar", "Leva cerca de 45 minutos e precisa de 45 GB.", "{0} tem {1} GB livres.", "Verificando a versão do jogo...", "Esta versão é compatível com o Hookshot.", "Esta versão não é compatível. O Hookshot precisa da build e0f3ccfa.", "As ferramentas do Hookshot não foram encontradas ao lado deste aplicativo.", "O Python não foi encontrado.", "Criando", "Isso leva um tempo, e seu PC continua utilizável.", "Decorrido", "Lendo seus arquivos do jogo", "Traduzindo o jogo para código de PC", "Compilando", "Montando a pasta do jogo", "Mostrar detalhes", "Ocultar detalhes", "Cancelar", "Voltar", "Abrir pasta", "Jogar", "Seu jogo está pronto", "criado em", "A criação parou", "Criação cancelada", "Você pode recomeçar a partir da mesma pasta.", "Parar a criação e fechar?", "Idioma"], + 'Nl': ["Maakt van je eigen exemplaar van Ocarina of Time een spelmap om op pc te starten.", "Spelbestanden", ".nup-bestand", "romfs en exefs", "Je kunt ook een bestand of map op dit venster slepen.", "Spel opslaan in", "Maken", "Bladeren", "Duurt ongeveer 45 minuten en vereist 45 GB.", "{0} heeft {1} GB vrij.", "Spelversie controleren...", "Deze versie wordt door Hookshot ondersteund.", "Deze versie wordt niet ondersteund. Hookshot heeft build e0f3ccfa nodig.", "De Hookshot-tools zijn niet naast deze app gevonden.", "Python is niet gevonden.", "Bezig met maken", "Dit duurt even, en je pc blijft bruikbaar.", "Verstreken", "Je spelbestanden lezen", "Spel vertalen naar pc-code", "Compileren", "Spelmap samenstellen", "Details tonen", "Details verbergen", "Annuleren", "Terug", "Map openen", "Spelen", "Je spel is klaar", "gemaakt in", "Het maken is gestopt", "Maken geannuleerd", "Je kunt opnieuw beginnen vanuit dezelfde map.", "Maken stoppen en sluiten?", "Taal"], + 'Pl': ["Zamienia Twoją kopię Ocarina of Time w folder gry, który uruchomisz na PC.", "Pliki gry", "Plik .nup", "romfs i exefs", "Możesz też upuścić plik lub folder na to okno.", "Zapisz grę w", "Utwórz", "Przeglądaj", "Trwa około 45 minut i wymaga 45 GB.", "{0}: wolne {1} GB.", "Sprawdzanie wersji gry...", "Ta wersja jest obsługiwana przez Hookshot.", "Ta wersja nie jest obsługiwana. Hookshot wymaga kompilacji e0f3ccfa.", "Nie znaleziono narzędzi Hookshot obok tej aplikacji.", "Nie znaleziono Pythona.", "Tworzenie", "To chwilę potrwa, a komputer pozostanie w użyciu.", "Upłynęło", "Odczytywanie plików gry", "Tłumaczenie gry na kod PC", "Kompilowanie", "Składanie folderu gry", "Pokaż szczegóły", "Ukryj szczegóły", "Anuluj", "Wstecz", "Otwórz folder", "Graj", "Twoja gra jest gotowa", "utworzono w", "Tworzenie zostało przerwane", "Anulowano tworzenie", "Możesz zacząć ponownie z tego samego folderu.", "Przerwać tworzenie i zamknąć?", "Język"], + 'Ru': ["Превращает вашу копию Ocarina of Time в папку с игрой для запуска на ПК.", "Файлы игры", "Файл .nup", "romfs и exefs", "Также можно перетащить файл или папку в это окно.", "Сохранить игру в", "Собрать", "Обзор", "Занимает около 45 минут и требует 45 ГБ.", "{0}: свободно {1} ГБ.", "Проверка версии игры...", "Эта версия поддерживается Hookshot.", "Эта версия не поддерживается. Hookshot нужна сборка e0f3ccfa.", "Инструменты Hookshot не найдены рядом с этим приложением.", "Python не найден.", "Сборка", "Это займёт время, а компьютером можно пользоваться.", "Прошло", "Чтение файлов игры", "Перевод игры в код для ПК", "Компиляция", "Подготовка папки с игрой", "Показать подробности", "Скрыть подробности", "Отмена", "Назад", "Открыть папку", "Играть", "Ваша игра готова", "собрано за", "Сборка остановлена", "Сборка отменена", "Можно начать заново из той же папки.", "Остановить сборку и закрыть?", "Язык"], + 'Ja': ["お持ちの Ocarina of Time を、PC で起動できるゲームフォルダーに変換します。", "ゲームファイル", ".nup ファイル", "romfs と exefs", "ファイルやフォルダーをこのウィンドウにドロップすることもできます。", "ゲームの保存先", "ビルド", "参照", "約 45 分かかり、45 GB 必要です。", "{0} の空き容量: {1} GB", "ゲームのバージョンを確認しています...", "このバージョンは Hookshot に対応しています。", "このバージョンには対応していません。Hookshot にはビルド e0f3ccfa が必要です。", "このアプリの横に Hookshot ツールが見つかりません。", "Python が見つかりません。", "ビルド中", "時間がかかりますが、PC はそのまま使えます。", "経過時間", "ゲームファイルを読み込んでいます", "ゲームを PC 用コードに変換しています", "コンパイル中", "ゲームフォルダーを作成しています", "詳細を表示", "詳細を隠す", "キャンセル", "戻る", "フォルダーを開く", "プレイ", "ゲームの準備ができました", "所要時間", "ビルドが停止しました", "ビルドをキャンセルしました", "同じフォルダーからやり直せます。", "ビルドを中止して閉じますか?", "言語"], + 'Ko': ["가지고 있는 Ocarina of Time을 PC에서 실행할 수 있는 게임 폴더로 만듭니다.", "게임 파일", ".nup 파일", "romfs 및 exefs", "파일이나 폴더를 이 창에 끌어다 놓을 수도 있습니다.", "게임 저장 위치", "빌드", "찾아보기", "약 45분이 걸리며 45GB가 필요합니다.", "{0} 여유 공간: {1}GB", "게임 버전을 확인하는 중...", "Hookshot이 지원하는 버전입니다.", "지원되지 않는 버전입니다. Hookshot에는 빌드 e0f3ccfa가 필요합니다.", "이 앱 옆에서 Hookshot 도구를 찾을 수 없습니다.", "Python을 찾을 수 없습니다.", "빌드 중", "시간이 걸리지만 PC는 계속 사용할 수 있습니다.", "경과 시간", "게임 파일을 읽는 중", "게임을 PC 코드로 변환하는 중", "컴파일 중", "게임 폴더를 만드는 중", "자세히 보기", "자세히 숨기기", "취소", "뒤로", "폴더 열기", "플레이", "게임이 준비되었습니다", "소요 시간", "빌드가 중단되었습니다", "빌드가 취소되었습니다", "같은 폴더에서 다시 시작할 수 있습니다.", "빌드를 중단하고 닫을까요?", "언어"], + 'ZhHans': ["将你自己的 Ocarina of Time 变成可在电脑上启动的游戏文件夹。", "游戏文件", ".nup 文件", "romfs 和 exefs", "也可以将文件或文件夹拖放到此窗口。", "游戏保存位置", "生成", "浏览", "大约需要 45 分钟和 45 GB 空间。", "{0} 可用空间 {1} GB。", "正在检查游戏版本...", "Hookshot 支持此版本。", "不支持此版本。Hookshot 需要构建版本 e0f3ccfa。", "在此应用旁边找不到 Hookshot 工具。", "找不到 Python。", "正在生成", "这需要一些时间,期间电脑仍可正常使用。", "已用时间", "正在读取游戏文件", "正在将游戏转换为电脑代码", "正在编译", "正在整理游戏文件夹", "显示详细信息", "隐藏详细信息", "取消", "返回", "打开文件夹", "开始游戏", "你的游戏已准备就绪", "用时", "生成已停止", "生成已取消", "你可以从同一文件夹重新开始。", "要停止生成并关闭吗?", "语言"], + 'ZhHant': ["將你自己的 Ocarina of Time 變成可在電腦上啟動的遊戲資料夾。", "遊戲檔案", ".nup 檔案", "romfs 和 exefs", "也可以將檔案或資料夾拖放到此視窗。", "遊戲儲存位置", "建置", "瀏覽", "大約需要 45 分鐘和 45 GB 空間。", "{0} 可用空間 {1} GB。", "正在檢查遊戲版本...", "Hookshot 支援此版本。", "不支援此版本。Hookshot 需要建置版本 e0f3ccfa。", "在此應用程式旁找不到 Hookshot 工具。", "找不到 Python。", "正在建置", "這需要一些時間,期間電腦仍可正常使用。", "已用時間", "正在讀取遊戲檔案", "正在將遊戲轉換為電腦程式碼", "正在編譯", "正在整理遊戲資料夾", "顯示詳細資訊", "隱藏詳細資訊", "取消", "返回", "開啟資料夾", "開始遊戲", "你的遊戲已準備就緒", "用時", "建置已停止", "建置已取消", "你可以從同一資料夾重新開始。", "要停止建置並關閉嗎?", "語言"], +} + +LANGUAGES = [ # (code, its own name, the TEXT it uses) + ('en-US', 'English (US)', 'En'), + ('en-GB', 'English (UK)', 'En'), + ('fr', 'Français', 'Fr'), + ('fr-CA', 'Français (Canada)', 'Fr'), + ('de', 'Deutsch', 'De'), + ('it', 'Italiano', 'It'), + ('es', 'Español (España)', 'Es'), + ('es-419', 'Español (Latinoamérica)', 'Es'), + ('pt-BR', 'Português (Brasil)', 'Pt'), + ('nl', 'Nederlands', 'Nl'), + ('pl', 'Polski', 'Pl'), + ('ru', 'Русский', 'Ru'), + ('ja', '日本語', 'Ja'), + ('ko', '한국어', 'Ko'), + ('zh-Hans', '简体中文', 'ZhHans'), + ('zh-Hant', '繁體中文', 'ZhHant'), +] + + + +def text(lang, key): + """A string in the language (a LANGUAGES entry), English where it has none (the update's lines are English only)""" + i = KEYS.index(key) + arr = TEXT[lang[2]] + return arr[i] if i < len(arr) else TEXT["En"][i] + + +def default_language(): + """The language closest to Windows' own: the exact one, then the same family, else English (US)""" + name = "" + try: + import ctypes + buf = ctypes.create_unicode_buffer(85) + if ctypes.windll.kernel32.GetUserDefaultLocaleName(buf, 85): + name = buf.value + except Exception: + pass + for lang in LANGUAGES: + if lang[0].lower() == name.lower(): + return lang + if name.startswith("zh"): + pick = "zh-Hant" if any(t in name for t in ("TW", "HK", "Hant")) else "zh-Hans" + elif name.startswith("es-") and name != "es-ES": + pick = "es-419" + elif name.startswith("pt"): + pick = "pt-BR" + else: + pick = name.split("-")[0] + return next((lang for lang in LANGUAGES if lang[0] == pick), LANGUAGES[0]) diff --git a/builder/py/hookshot_builder/theme.py b/builder/py/hookshot_builder/theme.py new file mode 100644 index 0000000..934194d --- /dev/null +++ b/builder/py/hookshot_builder/theme.py @@ -0,0 +1,461 @@ +"""The WPF builder's App.xaml in Qt: the Slate palette, the Inter fonts, and the controls drawn as its templates draw them (the same +sizes, radii, colours, glows and hover states).""" +import math +import os + +from PySide6.QtCore import QPointF, QRectF, QSize, Qt, Signal +from PySide6.QtGui import QBrush, QColor, QFont, QFontDatabase, QLinearGradient, QPainter, QPainterPath, QPen +from PySide6.QtWidgets import (QAbstractButton, QComboBox, QFrame, QGraphicsDropShadowEffect, QLabel, QLineEdit, QListView, + QSizePolicy, QStyledItemDelegate, QWidget) + +from . import paths + + +def argb(s): + """#AARRGGBB or #RRGGBB, as WPF writes them""" + s = s.lstrip("#") + if len(s) == 8: + return QColor(int(s[2:4], 16), int(s[4:6], 16), int(s[6:8], 16), int(s[0:2], 16)) + return QColor(int(s[0:2], 16), int(s[2:4], 16), int(s[4:6], 16)) + + +C = {k: argb(v) for k, v in { + "Page": "#14161A", "Raised": "#1C1F24", "RaisedHover": "#23272D", "Line": "#2C3037", "LineStrong": "#3D434C", + "Ink": "#EDEFF2", "InkSoft": "#A2A9B3", "InkFaint": "#6E7681", "Brand": "#3D63E6", "BrandHover": "#4B70F0", + "BrandInk": "#DFE6FF", "Go": "#5FB08A", "GoHover": "#70C09A", "GoInk": "#06150E", "Warn": "#E2B86B", "Bad": "#E2776B", + "Glass": "#0DFFFFFF", "GlassHover": "#17FFFFFF", "GlassLine": "#17FFFFFF", "GlassLineStrong": "#2AFFFFFF", + "GoSoft": "#265FB08A", "BadSoft": "#26E2776B", "WarnSoft": "#1FE2B86B", "BrandSoft": "#333D63E6", "White": "#FFFFFF", +}.items()} + +UI = "Inter" +MONO = "Cascadia Mono" +W_MEDIUM, W_SEMIBOLD, W_BOLD = QFont.Weight.Medium, QFont.Weight.DemiBold, QFont.Weight.Bold + + +def load_fonts(): + global MONO + for f in ("Inter-Regular.ttf", "Inter-Medium.ttf", "Inter-SemiBold.ttf", "Inter-Bold.ttf"): + QFontDatabase.addApplicationFont(os.path.join(paths.ASSETS, "Fonts", f)) + if MONO not in QFontDatabase.families(): + MONO = "Consolas" + + +def font(size=14, weight=QFont.Weight.Normal, family=None): + f = QFont(family or UI) + f.setPixelSize(int(size)) + f.setWeight(weight) + f.setHintingPreference(QFont.HintingPreference.PreferNoHinting) # (WPF's Ideal text formatting) + f.setStyleStrategy(QFont.StyleStrategy.PreferAntialias) + return f + + +def css(c): + return "rgba(%d,%d,%d,%d)" % (c.red(), c.green(), c.blue(), c.alpha()) + + +def label(text="", size=14, weight=QFont.Weight.Normal, color="Ink", wrap=False, mono=False): + l = QLabel(text) + l.setFont(font(size, weight, MONO if mono else None)) + set_color(l, color) + l.setWordWrap(wrap) + l.setTextInteractionFlags(Qt.TextInteractionFlag.NoTextInteraction) + l.setAttribute(Qt.WidgetAttribute.WA_TranslucentBackground) + return l + + +def set_color(l, color): + c = C[color] if isinstance(color, str) else color + l.setStyleSheet("color: %s; background: transparent;" % css(c)) + + +# the text styles of App.xaml +def heading(text=""): + return label(text, 24, W_BOLD) + + +def sub(text=""): + l = label(text, 14, color="InkSoft", wrap=True) + l.setContentsMargins(0, 6, 0, 0) + return l + + +def note(text="", size=13): + return label(text, size, color="InkSoft", wrap=True) + + +def faint(text="", size=12): + return label(text, size, color="InkFaint") + + +def section(text): + l = label(text, 12, W_SEMIBOLD, "InkSoft") + l.setContentsMargins(0, 24, 0, 10) + return l + + +def glow(widget, color, blur, depth, opacity): + """WPF's DropShadowEffect (Direction 270: straight down)""" + e = QGraphicsDropShadowEffect(widget) + c = QColor(argb(color) if isinstance(color, str) else color) + c.setAlphaF(opacity) + e.setColor(c) + e.setBlurRadius(blur * 1.2) + e.setOffset(0, depth) + widget.setGraphicsEffect(e) + return e + + +def rounded(rect, r): + p = QPainterPath() + p.addRoundedRect(rect, r, r) + return p + + +class Card(QFrame): + """Card: Glass background, GlassLine border, corner radius 10""" + + def __init__(self, radius=10, bg="Glass", line="GlassLine"): + super().__init__() + self.radius, self.bg, self.line = radius, bg, line + self.setAttribute(Qt.WidgetAttribute.WA_TranslucentBackground) + + def paintEvent(self, e): + p = QPainter(self) + p.setRenderHint(QPainter.RenderHint.Antialiasing) + r = QRectF(self.rect()).adjusted(0.5, 0.5, -0.5, -0.5) + p.fillPath(rounded(r, self.radius), C[self.bg] if isinstance(self.bg, str) else self.bg) + p.setPen(QPen(C[self.line] if isinstance(self.line, str) else self.line, 1)) + p.drawPath(rounded(r, self.radius)) + + +class Button(QAbstractButton): + """Quiet / Primary / GoButton / LinkButton / Link / Caption, as App.xaml's templates""" + KINDS = { + # kind: (height, padding x, radius, font size, weight) + "quiet": (40, 18, 8, 14, W_SEMIBOLD), "primary": (40, 20, 8, 14, W_SEMIBOLD), "go": (50, 34, 10, 16, W_BOLD), + "link": (None, 0, 0, 13, W_SEMIBOLD), "toggle": (None, 0, 0, 13, QFont.Weight.Normal), "caption": (36, 0, 8, 12, QFont.Weight.Normal), + } + + def __init__(self, text="", kind="quiet", icon=None, color=None, size=None): + super().__init__() + self._glow = None + self.kind, self.icon, self.hover, self._color = kind, icon, False, color + h, self.padx, self.radius, fs, fw = self.KINDS[kind] + self.setFont(font(size or fs, fw)) + self.setText(text) + self.setCursor(Qt.CursorShape.PointingHandCursor if kind != "caption" else Qt.CursorShape.ArrowCursor) + self.setFocusPolicy(Qt.FocusPolicy.TabFocus if kind in ("quiet", "primary", "go") else Qt.FocusPolicy.NoFocus) + self.setAttribute(Qt.WidgetAttribute.WA_TranslucentBackground) + self.fixed_h = h + if kind == "caption": + self.setFixedSize(46, 36) + self._glow = None + if kind == "primary": + self._glow = glow(self, "#3D63E6", 22, 6, 0.55) + elif kind == "go": + self._glow = glow(self, "#5FB08A", 30, 8, 0.5) + self.setSizePolicy(QSizePolicy.Policy.Fixed, QSizePolicy.Policy.Fixed) + + def setText(self, t): + super().setText(t) + self.updateGeometry() + self.update() + + def set_color(self, color): + self._color = color + self.update() + + def sizeHint(self): + fm = self.fontMetrics() + w = fm.horizontalAdvance(self.text()) + if self.icon == "play": + w += 21 + if self.kind in ("link", "toggle"): + return QSize(w + 1, fm.height() + 2) + if self.kind == "caption": + return QSize(46, 36) + return QSize(w + 2 * self.padx + 2, self.fixed_h) + + def enterEvent(self, e): + self.hover = True + self.update() + + def leaveEvent(self, e): + self.hover = False + self.update() + + def changeEvent(self, e): + if self._glow is not None: + self._glow.setEnabled(self.isEnabled()) + super().changeEvent(e) + + def paintEvent(self, e): + p = QPainter(self) + p.setRenderHint(QPainter.RenderHint.Antialiasing) + p.setRenderHint(QPainter.RenderHint.TextAntialiasing) + r = QRectF(self.rect()).adjusted(0.5, 0.5, -0.5, -0.5) + k, on, hov, focus = self.kind, self.isEnabled(), self.hover and self.isEnabled(), self.hasFocus() + ink = C["Ink"] + if k == "quiet": + bg = C["GlassHover"] if hov else argb("#08FFFFFF") + line = C["BrandHover"] if focus else C["GlassLineStrong"] if hov else argb("#1FFFFFFF") + p.fillPath(rounded(r, 8), bg) + p.setPen(QPen(line, 1)) + p.drawPath(rounded(r, 8)) + ink = C["Ink"] if on else C["InkFaint"] + elif k == "primary": + bg = (C["BrandHover"] if hov else C["Brand"]) if on else argb("#14FFFFFF") + p.fillPath(rounded(r, 8), bg) + p.setPen(QPen(argb("#AAFFFFFF") if focus else argb("#33FFFFFF"), 1)) + p.drawPath(rounded(r, 8)) + ink = C["White"] if on else C["InkFaint"] + elif k == "go": + p.fillPath(rounded(r, 10), C["GoHover"] if hov else C["Go"]) + p.setPen(QPen(argb("#40FFFFFF"), 1)) + p.drawPath(rounded(r, 10)) + ink = C["GoInk"] + elif k == "link": + ink = C["White"] if hov else (self._color or C["BrandInk"]) + elif k == "toggle": + ink = C["Ink"] if hov else (self._color or C["InkSoft"]) + elif k == "caption": + if self.hover: + p.fillPath(rounded(r, 8), C["GlassHover"]) + ink = C["Ink"] if self.hover else argb("#80FFFFFF") + p.setPen(QPen(ink, 1.2 if self.icon == "close" else 1)) + cx, cy = self.width() / 2, self.height() / 2 + if self.icon == "min": + p.fillRect(QRectF(cx - 5, cy, 10, 1), ink) + else: + p.drawLine(QPointF(cx - 5, cy - 5), QPointF(cx + 5, cy + 5)) + p.drawLine(QPointF(cx + 5, cy - 5), QPointF(cx - 5, cy + 5)) + return + p.setFont(self.font()) + p.setPen(ink) + fm = self.fontMetrics() + tw = fm.horizontalAdvance(self.text()) + if self.icon == "play": # (the triangle M0,0 L11,7 L0,14, 10 px before the text) + total = 11 + 10 + tw + x0 = (self.width() - total) / 2 + tri = QPainterPath() + cy = self.height() / 2 + tri.moveTo(x0, cy - 7) + tri.lineTo(x0 + 11, cy) + tri.lineTo(x0, cy + 7) + tri.closeSubpath() + p.fillPath(tri, ink) + p.drawText(QRectF(x0 + 21, 0, tw + 2, self.height()), Qt.AlignmentFlag.AlignVCenter | Qt.AlignmentFlag.AlignLeft, self.text()) + return + align = Qt.AlignmentFlag.AlignCenter if k in ("quiet", "primary", "go") else Qt.AlignmentFlag.AlignLeft | Qt.AlignmentFlag.AlignVCenter + p.drawText(QRectF(self.rect()), align, self.text()) + + +class Tick(QAbstractButton): + """CheckBox Tick: an 18x18 box, radius 5, its mark when checked, the text in InkSoft 13""" + + def __init__(self, text, checked=False): + super().__init__() + self.setCheckable(True) + self.setChecked(checked) + self.setText(text) + self.setFont(font(13)) + self.setCursor(Qt.CursorShape.PointingHandCursor) + self.setFocusPolicy(Qt.FocusPolicy.NoFocus) + + def sizeHint(self): + return QSize(26 + self.fontMetrics().horizontalAdvance(self.text()) + 2, max(18, self.fontMetrics().height())) + + def paintEvent(self, e): + p = QPainter(self) + p.setRenderHint(QPainter.RenderHint.Antialiasing) + y = (self.height() - 18) / 2 + box = QRectF(0.75, y + 0.75, 16.5, 16.5) + if self.isChecked(): + p.fillPath(rounded(box, 5), C["Brand"]) + p.setPen(QPen(C["Brand"], 1.5)) + p.drawPath(rounded(box, 5)) + p.setPen(QPen(C["White"], 2)) + path = QPainterPath(QPointF(2, y + 8)) + path.lineTo(6, y + 12) + path.lineTo(14, y + 3) + p.drawPath(path) + else: + p.setPen(QPen(argb("#40FFFFFF"), 1.5)) + p.drawPath(rounded(box, 5)) + p.setFont(self.font()) + p.setPen(C["InkSoft"]) + p.drawText(QRectF(26, 0, self.width() - 26, self.height()), Qt.AlignmentFlag.AlignVCenter | Qt.AlignmentFlag.AlignLeft, + self.text()) + + +class Bar(QWidget): + """ProgressBar Bar: 6 high, track #12FFFFFF, the indicator in Foreground, both radius 3""" + + def __init__(self, color="Brand", width=None): + super().__init__() + self.value, self.color = 0.0, color + self.setFixedHeight(6) + if width: + self.setFixedWidth(width) + + def set(self, v, color=None): + self.value = max(0.0, min(1.0, v)) + if color: + self.color = color + self.update() + + def paintEvent(self, e): + p = QPainter(self) + p.setRenderHint(QPainter.RenderHint.Antialiasing) + r = QRectF(self.rect()) + p.fillPath(rounded(r, 3), argb("#12FFFFFF")) + if self.value > 0: + p.fillPath(rounded(QRectF(0, 0, r.width() * self.value, r.height()), 3), C[self.color]) + + +class Chip(QLabel): + """Border Chip: 26 high, radius 13, #0DFFFFFF with #14FFFFFF, 10 px each side""" + + def __init__(self, text, color, soft=None): + super().__init__(text) + self.setFont(font(12)) + self.soft = soft + self.setStyleSheet("color: %s; background: transparent; padding: 0 10px;" % css(C[color])) + self.setFixedHeight(26) + + def paintEvent(self, e): + p = QPainter(self) + p.setRenderHint(QPainter.RenderHint.Antialiasing) + r = QRectF(self.rect()).adjusted(0.5, 0.5, -0.5, -0.5) + p.fillPath(rounded(r, 13), C[self.soft] if self.soft else argb("#0DFFFFFF")) + p.setPen(QPen(argb("#14FFFFFF"), 1)) + p.drawPath(rounded(r, 13)) + p.end() + super().paintEvent(e) + + +class PathEdit(QLineEdit): + """TextBox Path: mono 12 in InkSoft, Glass with GlassLine (Brand when focused), radius 8, 40 high""" + + def __init__(self): + super().__init__() + self.setFont(font(12, family=MONO)) + self.setFixedHeight(40) + self.setStyleSheet( + "QLineEdit { color: %s; background: %s; border: 1px solid %s; border-radius: 8px; padding: 0 11px;" + " selection-background-color: %s; }" + "QLineEdit:focus { border-color: %s; }" % (css(C["InkSoft"]), css(C["Glass"]), css(C["GlassLine"]), css(C["Brand"]), + css(C["Brand"]))) + + +class _ItemDelegate(QStyledItemDelegate): + """ComboBoxItem ChoiceItem: radius 6, 10/7 padding, RaisedHover under the mouse, BrandInk when chosen""" + + def __init__(self, combo): + super().__init__(combo) + self.combo = combo + + def sizeHint(self, opt, idx): + return QSize(opt.rect.width(), 7 + 7 + opt.fontMetrics.height()) + + def paint(self, p, opt, idx): + from PySide6.QtWidgets import QStyle + p.save() + p.setRenderHint(QPainter.RenderHint.Antialiasing) + if opt.state & QStyle.StateFlag.State_MouseOver or opt.state & QStyle.StateFlag.State_Selected: + p.fillPath(rounded(QRectF(opt.rect), 6), C["RaisedHover"]) + p.setFont(font(13)) + p.setPen(C["BrandInk"] if idx.row() == self.combo.currentIndex() else C["Ink"]) + p.drawText(QRectF(opt.rect).adjusted(10, 0, -10, 0), Qt.AlignmentFlag.AlignVCenter | Qt.AlignmentFlag.AlignLeft, + idx.data()) + p.restore() + + +class Choice(QComboBox): + """ComboBox Choice: 40 high, #08FFFFFF with #1FFFFFFF (GlassHover under the mouse), radius 8, its chevron; the list in Raised + with LineStrong, radius 8, padding 4""" + + def __init__(self): + super().__init__() + self.setFixedHeight(40) + self.setFont(font(13)) + self.setCursor(Qt.CursorShape.PointingHandCursor) + self.hover = False + v = QListView() + v.setItemDelegate(_ItemDelegate(self)) + v.setMouseTracking(True) + self.setView(v) + self.setStyleSheet( + "QComboBox { background: transparent; border: none; color: %s; }" + "QComboBox::drop-down { width: 0; border: none; } QComboBox::down-arrow { image: none; }" + "QComboBox QAbstractItemView { background: %s; border: 1px solid %s; border-radius: 8px; padding: 4px; outline: none;" + " color: %s; }" % (css(C["Ink"]), css(C["Raised"]), css(C["LineStrong"]), css(C["Ink"]))) + self.view().window().setWindowFlags(Qt.WindowType.Popup | Qt.WindowType.FramelessWindowHint | + Qt.WindowType.NoDropShadowWindowHint) + self.view().window().setAttribute(Qt.WidgetAttribute.WA_TranslucentBackground) + + def enterEvent(self, e): + self.hover = True + self.update() + + def leaveEvent(self, e): + self.hover = False + self.update() + + def showPopup(self): + super().showPopup() + w = self.view().window() # (4 px below the box, as WPF's Popup margin) + w.move(w.x(), self.mapToGlobal(self.rect().bottomLeft()).y() + 4) + + def paintEvent(self, e): + p = QPainter(self) + p.setRenderHint(QPainter.RenderHint.Antialiasing) + r = QRectF(self.rect()).adjusted(0.5, 0.5, -0.5, -0.5) + p.fillPath(rounded(r, 8), C["GlassHover"] if self.hover else argb("#08FFFFFF")) + p.setPen(QPen(C["Brand"], 2) if self.hasFocus() and self.property("focus_ring") else QPen(argb("#1FFFFFFF"), 1)) + p.drawPath(rounded(r, 8)) + p.setPen(QPen(C["InkSoft"], 1.3)) + x, y = self.width() - 12 - 8, self.height() / 2 - 2 + path = QPainterPath(QPointF(x, y)) + path.lineTo(x + 4, y + 4) + path.lineTo(x + 8, y) + p.drawPath(path) + p.setFont(self.font()) + p.setPen(C["Ink"]) + p.drawText(QRectF(12, 0, self.width() - 40, self.height()), Qt.AlignmentFlag.AlignVCenter | Qt.AlignmentFlag.AlignLeft, + self.currentText()) + + +class Ring(QWidget): + """The build's ring: a #14FFFFFF circle of 10, the Brand arc over it (round caps), the percent in the middle""" + + def __init__(self): + super().__init__() + self.setFixedSize(120, 120) + self.f = 0.0 + + def set(self, f): + self.f = max(0.0, min(1.0, f)) + self.update() + + def paintEvent(self, e): + p = QPainter(self) + p.setRenderHint(QPainter.RenderHint.Antialiasing) + p.setPen(QPen(argb("#14FFFFFF"), 10)) + p.drawEllipse(QRectF(5, 5, 110, 110)) + if self.f >= 0.002: + pen = QPen(C["Brand"], 10) + pen.setCapStyle(Qt.PenCapStyle.RoundCap) + p.setPen(pen) + p.drawArc(QRectF(5, 5, 110, 110), 90 * 16, -int(self.f * 360 * 16)) + p.setPen(C["Ink"]) + p.setFont(font(24, W_BOLD)) + p.drawText(QRectF(self.rect()), Qt.AlignmentFlag.AlignCenter, "%d%%" % int(self.f * 100)) + + +def tick_path(cx, cy): + """M0,4 L3.5,7.5 L10,0, centred on (cx, cy)""" + path = QPainterPath(QPointF(cx - 5, cy)) + path.lineTo(cx - 1.5, cy + 3.5) + path.lineTo(cx + 5, cy - 4) + return path diff --git a/builder/py/hookshot_builder/updater.py b/builder/py/hookshot_builder/updater.py new file mode 100644 index 0000000..b27c6b3 --- /dev/null +++ b/builder/py/hookshot_builder/updater.py @@ -0,0 +1,111 @@ +"""The game's update offer and the builder's own (Updater.cs, the same rules). The game (host/hos_update.c) starts this app with +--update --tag . When these tools are not that release, the release is installed (setup_parts: only the +parts that changed are downloaded) and this app starts again on it with --fetched to build the folder; when they are, the folder is +built again here (nx2/make_game.py --update: from the game files the folder keeps).""" +import json +import os +import subprocess +import threading +import time +import urllib.request + +from . import REPO, paths, setup_parts +from .core import Tools + +LATEST = "https://api.github.com/repos/%s/releases/latest" % REPO + + +def latest_tag(): + """The latest release's tag, quietly (None: GitHub did not answer); a few tries, 20 s at most""" + if os.environ.get("HOOKSHOT_LATEST_TAG"): # (development: a made-up latest release, "none" for no answer) + t = os.environ["HOOKSHOT_LATEST_TAG"] + return None if t == "none" else t + for attempt in range(3): + try: + req = urllib.request.Request(LATEST, headers={"User-Agent": "HookshotBuilder", "Accept": "application/vnd.github+json"}) + with urllib.request.urlopen(req, timeout=8) as r: + return json.loads(r.read()).get("tag_name") + except urllib.error.HTTPError as e: + if e.code < 500: # (the API's limit, 60 an hour a PC: the list below instead) + break + except (OSError, ValueError): + pass + time.sleep(2 << attempt) + # (v2.0.0: when the API did not answer, the builder said a game was up to date. The release's setup list names its tag in its + # first line, from GitHub's download service, not the API) + for _ in range(2): + try: + req = urllib.request.Request(setup_parts.manifest_url(), headers={"User-Agent": "HookshotBuilder"}) + with urllib.request.urlopen(req, timeout=8) as r: + for line in r.read().decode("utf-8", "replace").splitlines(): + if line.startswith("tag "): + return line[4:].strip() + except OSError: + pass + time.sleep(2) + return None + + +def restart(args): + """This app started again (on the version just installed: hookshot_builder_start.py picks it) with args, and this one closes""" + exe, base_args = paths.launcher_command() + subprocess.Popen([exe, *base_args, *args], cwd=os.path.dirname(exe)) + + +class Install: + """The release installed on a worker thread; ui is the window (note, download_progress, download_failed, and done(tag) called + on the window's thread through ui.post)""" + + def __init__(self, ui, tag=None, then=None): + self.ui, self.tag, self.then, self.cancelled = ui, tag, then, False + + def start(self): + threading.Thread(target=self._run, daemon=True).start() + return self + + def cancel(self): + self.cancelled = True + + def _run(self): + post = self.ui.post + try: + tag, _ = setup_parts.install(lambda s: post(self.ui.note, s), lambda f: post(self.ui.download_progress, f) if f >= 0 else None, + tag=self.tag, cancel=lambda: self.cancelled) + post(self.then, tag) + except setup_parts.SetupError as e: + msg = str(e) + post(self.ui.download_failed, "The download was cancelled. Your game folder is unchanged." if msg == "cancelled" else + msg + " Your game folder is unchanged.") + except Exception as e: # (anything else: said, not a silent window) + post(self.ui.download_failed, "%s. Your game folder is unchanged." % e) + + +def update_folder(ui, folder, tag, fetched): + """The game in folder built again with the latest release: here when these tools are it (or fetched), else after installing it + (this app then starts again on it with --update folder --tag t --fetched)""" + ui.begin_update(folder, tag) + tools = Tools.find() + # (just installed, but another release than asked for: GitHub's latest link lags right after a publish) + if fetched and tag and tools and tools.version and tools.version != tag: + ui.download_failed("The update installed %s, not %s: GitHub's download links can lag right after a release. Try Update again " + "in a few minutes." % (tools.version, tag)) + return + if tools and (fetched or (tag and tools.version == tag)): + ui.update_build(tools, folder) + return + ui.note("Asking GitHub for the latest release") + + def installed(new_tag): + restart(["--update", folder, "--tag", new_tag, "--fetched"]) + ui.quit_app() + ui.download = Install(ui, tag, installed).start() + + +def run_latest_setup(ui): + """The latest release installed, then this app started again on it (the builder's own update)""" + ui.note("Asking GitHub for the latest release") + + def installed(_tag): + restart([]) + ui.quit_app() + ui.download = Install(ui, None, installed).start() diff --git a/builder/py/hookshot_builder/window.py b/builder/py/hookshot_builder/window.py new file mode 100644 index 0000000..b818e56 --- /dev/null +++ b/builder/py/hookshot_builder/window.py @@ -0,0 +1,1812 @@ +"""The installer window: MainWindow.xaml / MainWindow.xaml.cs of the WPF builder in Qt, the same 960x600 window, pages and logic. It +opens with the hookshot in the middle while it looks for the game files (and, at the first start or after an update, gets the build +tools), then walks through Game files > Build > Play. The game's update offer opens it on its update screen, and the game folder's +launcher (Hookshot.lnk, --launch ) on the launcher page.""" +import ctypes +import math +import os +import re +import sys +import time +import winreg + +from PySide6.QtCore import (QEasingCurve, QObject, QPointF, QPropertyAnimation, QRectF, QSize, Qt, QTimer, QUrl, Signal) +from PySide6.QtGui import (QColor, QDesktopServices, QGuiApplication, QIcon, QLinearGradient, QPainter, QPainterPath, QPen, + QPixmap, QRadialGradient) +from PySide6.QtWidgets import (QAbstractButton, QApplication, QFileDialog, QFrame, QGraphicsOpacityEffect, QGridLayout, + QHBoxLayout, QLabel, QMessageBox, QPlainTextEdit, QScrollArea, QSizePolicy, QStackedLayout, + QVBoxLayout, QWidget) + +from . import core, paths, setup_parts, strings, updater +from .theme import (C, MONO, W_BOLD, W_MEDIUM, W_SEMIBOLD, Bar, Button, Card, Chip, Choice, PathEdit, Ring, Tick, argb, css, + faint, font, glow, heading, label, note, rounded, section, set_color, sub, tick_path) + +NEEDED_BYTES = 45 << 30 # (the game data 24.4 GB, the shader caches ~8 GB, the game 0.4 GB, the build files ~5 GB at their peak) + +STAGES = [("download", "", "Download the update"), ("read", "Read your game files", "Read the game files kept in the folder"), + ("lift", "Translate the game to PC code", "Translate the game to PC code"), ("compile", "Compile", "Compile changed files"), + ("finish", "Put the game together", "Swap in the new game"), ("shaders", "Prepare the shaders", "Prepare the shaders")] +WITHOUT_SHADERS = {"download": (0, 0.04), "read": (0.04, 0.04), "lift": (0.08, 0.14), "compile": (0.22, 0.74), "finish": (0.96, 0.04)} +# (the game's --prepare-shaders run, make_game.py's last step: minutes on some GPUs, the first start's wait moved here) +WITH_SHADERS = {"download": (0, 0.04), "read": (0.04, 0.04), "lift": (0.08, 0.14), "compile": (0.22, 0.68), "finish": (0.90, 0.02), + "shaders": (0.92, 0.08)} +SHADERS_LATER = "--no-prepare-shaders" +NOT_PREPARED = re.compile(r"^the shaders were not prepared now \((.*)\): ") # (make_game.py NOT_PREPARED) + +DLSS_LICENSE = ( + "NVIDIA DLSS is NVIDIA's, not Hookshot's. With it ticked, the build downloads NVIDIA's DLSS SDK (release v310.9.16) from " + "github.com/NVIDIA/DLSS, under NVIDIA's own license, the NVIDIA RTX SDKs license:\n\n" + "https://github.com/NVIDIA/DLSS/blob/9981c6350dc58c0ce21f024fd65e31e851c31689/LICENSE.txt\n\n" + "Its files stay on this PC (in the builder's ngx-sdk folder, and nvngx_dlss.dll in the game folder's ngx folder). " + "Hookshot never includes or shares them: never share a game folder built with them.\n\nDo you accept NVIDIA's license?") + + +class Poster(QObject): + """Calls from worker threads run on the window's thread""" + call = Signal(object, object) + + def __init__(self): + super().__init__() + self.call.connect(lambda fn, args: fn(*args)) + + +def clock(secs): + return "%d:%02d" % (int(secs) // 60, int(secs) % 60) + + +def span(secs): + secs = int(secs) + if secs >= 3600: + return "%d h %d min" % (secs // 3600, secs % 3600 // 60) + if secs >= 60: + return "%d minutes" % (secs // 60) + return "%d seconds" % secs + + +def time_left(seconds): + if seconds < 45: + return "Less than a minute left" + if seconds < 90: + return "About a minute left" + m = int(round(seconds / 60)) + return "About %d h %d min left" % (m // 60, m % 60) if m >= 60 else "About %d minutes left" % m + + +# ================= small pieces drawn as the XAML draws them ================= + +class Splash(QWidget): + """The start: the hookshot (178 wide) over its pulsing halo, bobbing; "Hookshot", the status, and the sweeping bar""" + + def __init__(self, hook): + super().__init__() + self.hook, self.t0, self.text, self.scale = hook, time.monotonic(), "Starting…", 1.0 + self.timer = QTimer(self, timeout=self.update) + self.timer.start(16) + + def stop(self): + self.timer.stop() + + def paintEvent(self, e): + p = QPainter(self) + p.setRenderHints(QPainter.RenderHint.Antialiasing | QPainter.RenderHint.SmoothPixmapTransform | QPainter.RenderHint.TextAntialiasing) + t = time.monotonic() - self.t0 + ping = (1 - math.cos(math.pi * ((t / 1.2) % 2 if (t / 1.2) % 2 <= 1 else 2 - (t / 1.2) % 2))) / 2 # (sine in-out, auto-reverse) + cx, cy = self.width() / 2, self.height() / 2 + p.translate(cx, cy) + p.scale(self.scale, self.scale) + top = -(260 + 16 + 27 + 6 + 20 + 16 + 3) / 2 # (the stack: 260 hook area, the title, the status, the bar) + hc = QPointF(0, top + 130) + r = (260 - 48) / 2 * (0.94 + 0.10 * ping) # (the halo: Margin 24, scale 0.94..1.04, opacity 0.6..1) + g = QRadialGradient(hc, r) + g.setColorAt(0, argb("#663D63E6")) + g.setColorAt(0.68, argb("#003D63E6")) + g.setColorAt(1, argb("#003D63E6")) + p.setOpacity(0.6 + 0.4 * ping) + p.setBrush(g) + p.setPen(Qt.PenStyle.NoPen) + p.drawEllipse(hc, r, r) + p.setOpacity(1) + w = 178 + h = self.hook.height() * w / self.hook.width() + p.save() + p.translate(hc.x(), hc.y() - 6 * ping) # (BobLift 0..-6, BobTurn -4..2) + p.rotate(-4 + 6 * ping) + shadow = QRectF(-w / 2, -h / 2 + 14, w, h) # (DropShadow 24 blur, 14 down, 0.6: the image's own, softened) + p.setOpacity(0.35) + p.drawPixmap(shadow, self.shadow_pixmap(), QRectF(self.shadow_pixmap().rect())) + p.setOpacity(1) + p.drawPixmap(QRectF(-w / 2, -h / 2, w, h), self.hook, QRectF(self.hook.rect())) + p.restore() + y = top + 260 + 16 + p.setPen(C["Ink"]) + p.setFont(font(22, W_SEMIBOLD)) + p.drawText(QRectF(-200, y, 400, 28), Qt.AlignmentFlag.AlignCenter, "Hookshot") + y += 28 + 6 + p.setPen(argb("#A6FFFFFF")) + p.setFont(font(13)) + p.drawText(QRectF(-300, y, 600, 20), Qt.AlignmentFlag.AlignCenter, self.text) + y += 20 + 16 + track = QRectF(-100, y, 200, 3) + p.setClipPath(rounded(track, 1.5)) + p.fillPath(rounded(track, 1.5), argb("#14FFFFFF")) + f = (t / 1.2) % 1 + f = (1 - math.cos(math.pi * f)) / 2 + x = -100 - 88 + 296 * f # (Sweep -88..208, sine in-out, repeating) + lg = QLinearGradient(x, 0, x + 80, 0) + lg.setColorAt(0, argb("#007F9BFF")) + lg.setColorAt(0.5, argb("#FF7F9BFF")) + lg.setColorAt(1, argb("#007F9BFF")) + p.fillRect(QRectF(x, y, 80, 3), lg) + + _shadow = None + + def shadow_pixmap(self): + if Splash._shadow is None: + img = self.hook.toImage().convertToFormat(self.hook.toImage().Format.Format_ARGB32_Premultiplied) + sp = QPixmap(img.size()) + sp.fill(Qt.GlobalColor.transparent) + q = QPainter(sp) + q.drawImage(0, 0, img) + q.setCompositionMode(QPainter.CompositionMode.CompositionMode_SourceIn) + q.fillRect(sp.rect(), QColor(0, 0, 0)) + q.end() + Splash._shadow = sp.scaled(sp.width() // 6, sp.height() // 6, Qt.AspectRatioMode.IgnoreAspectRatio, + Qt.TransformationMode.SmoothTransformation) + return Splash._shadow + + +class StepPill(QWidget): + """The head's pill: Game files > Build > Play (done: a green tick disc; now: the blue pill with its glow)""" + + def __init__(self): + super().__init__() + self.on = 0 + self.setFixedHeight(40) + self.names = ["Game files", "Build", "Play"] + self._layout() + + def set(self, on): + self.on = on + self._layout() + self.update() + + def _layout(self): + self.items = [] + x = 4 + 1 + for i, n in enumerate(self.names): + if i: + x += 5 + 4 + 5 # (the chevron, 5 each side) + fw = W_SEMIBOLD if i == self.on else W_MEDIUM + from PySide6.QtGui import QFontMetricsF + tw = QFontMetricsF(font(13, fw)).horizontalAdvance(n) + done = i < self.on + w = (4 + 22 + 8 + tw + 14) if done else (14 + tw + 14) + self.items.append((x, w, n, done, fw)) + x += w + self.setFixedWidth(int(x + 4 + 1 + 0.5)) + + def paintEvent(self, e): + p = QPainter(self) + p.setRenderHints(QPainter.RenderHint.Antialiasing | QPainter.RenderHint.TextAntialiasing) + r = QRectF(self.rect()).adjusted(0.5, 0.5, -0.5, -0.5) + p.fillPath(rounded(r, 20), argb("#0BFFFFFF")) + p.setPen(QPen(argb("#17FFFFFF"), 1)) + p.drawPath(rounded(r, 20)) + for i, (x, w, n, done, fw) in enumerate(self.items): + if i: + cx = x - 5 - 2 + p.setPen(QPen(QColor(C["InkSoft"].red(), C["InkSoft"].green(), C["InkSoft"].blue(), int(255 * 0.45)), 1.6)) + path = QPainterPath(QPointF(cx - 2, 16)) + path.lineTo(cx + 2, 20) + path.lineTo(cx - 2, 24) + p.drawPath(path) + pill = QRectF(x, 5, w, 30) + if i == self.on: # (LinearGradient #4B70F0 -> #3557D6 at 90 deg, and its glow) + for k in range(8, 0, -1): + gc = QColor(0x3D, 0x63, 0xE6, int(0.6 * 255 / 8 / k * 2)) + p.fillPath(rounded(pill.adjusted(-k, -k + 6, k, k + 6), 15 + k), gc) + lg = QLinearGradient(pill.topLeft(), pill.bottomLeft()) + lg.setColorAt(0, argb("#4B70F0")) + lg.setColorAt(1, argb("#3557D6")) + p.fillPath(rounded(pill, 15), lg) + tx = x + 14 + if done: + d = QRectF(x + 4, 9, 22, 22) + p.setBrush(C["GoSoft"]) + p.setPen(Qt.PenStyle.NoPen) + p.drawEllipse(d) + p.setPen(QPen(C["Go"], 1.8)) + p.setBrush(Qt.BrushStyle.NoBrush) + p.drawPath(tick_path(d.center().x(), d.center().y() + 0.25)) + tx = x + 4 + 22 + 8 + p.setFont(font(13, fw)) + p.setPen(C["White"] if i == self.on else C["InkSoft"] if done else C["InkFaint"]) + p.drawText(QRectF(tx, 5, w, 30), Qt.AlignmentFlag.AlignVCenter | Qt.AlignmentFlag.AlignLeft, n) + + +class Mark(QWidget): + """A build step's mark: waiting (a ring), now (the number on Brand with its halo), done (a tick on GoSoft), stopped (! on BadSoft)""" + + def __init__(self, n): + super().__init__() + self.n, self.state = n, "wait" + self.setFixedSize(34, 34) + + def paintEvent(self, e): + p = QPainter(self) + p.setRenderHints(QPainter.RenderHint.Antialiasing | QPainter.RenderHint.TextAntialiasing) + c = QPointF(11, 12) + s = self.state + if s == "now": + p.setPen(Qt.PenStyle.NoPen) + p.setBrush(C["BrandSoft"]) + p.drawEllipse(c, 16, 16) + p.setBrush(C["Brand"]) + p.drawEllipse(c, 11, 11) + p.setPen(C["White"]) + p.setFont(font(12, W_SEMIBOLD)) + p.drawText(QRectF(c.x() - 11, c.y() - 11, 22, 22), Qt.AlignmentFlag.AlignCenter, str(self.n)) + elif s == "done": + p.setPen(Qt.PenStyle.NoPen) + p.setBrush(C["GoSoft"]) + p.drawEllipse(c, 11, 11) + p.setPen(QPen(C["Go"], 1.8)) + p.setBrush(Qt.BrushStyle.NoBrush) + p.drawPath(tick_path(c.x(), c.y() + 0.25)) + elif s == "stopped": + p.setPen(Qt.PenStyle.NoPen) + p.setBrush(C["BadSoft"]) + p.drawEllipse(c, 11, 11) + p.setPen(C["Bad"]) + p.setFont(font(12, W_SEMIBOLD)) + p.drawText(QRectF(c.x() - 11, c.y() - 11, 22, 22), Qt.AlignmentFlag.AlignCenter, "!") + else: + p.setPen(QPen(C["GlassLineStrong"], 1.5)) + p.setBrush(Qt.BrushStyle.NoBrush) + p.drawEllipse(c, 10.25, 10.25) + if s == "skipped": + p.setPen(C["InkFaint"]) + p.setFont(font(12, W_SEMIBOLD)) + p.drawText(QRectF(c.x() - 11, c.y() - 11, 22, 22), Qt.AlignmentFlag.AlignCenter, "–") + + +class StepRow(QWidget): + """One step of the build: its mark (done, now, waiting), its name, a line under it and its time""" + + def __init__(self, n, text, last): + super().__init__() + self.running, self.start = False, 0.0 + self.last = last + g = QGridLayout(self) + g.setContentsMargins(0, 10, 0, 10) + g.setHorizontalSpacing(0) + g.setVerticalSpacing(0) + self.mark = Mark(n) + g.addWidget(self.mark, 0, 0, 2, 1, Qt.AlignmentFlag.AlignTop) + self.name = label(text, 14, W_SEMIBOLD) + self.small = label("", 12) + self.small.hide() + g.addWidget(self.name, 0, 1) + g.addWidget(self.small, 1, 1) + self.time = label("—", 12, mono=True) + g.addWidget(self.time, 0, 2, Qt.AlignmentFlag.AlignTop) + g.setColumnStretch(1, 1) + self.wait() + + def paintEvent(self, e): + if self.last: + return + p = QPainter(self) # (the link down to the next mark: 1 wide at 10.5, from 28 below the top to 19 past the bottom) + p.fillRect(QRectF(10.5, 10 + 28, 1, self.height() - 38 + 10), C["GlassLine"]) + + def small_text(self, t): + self.small.setText(t) + self.small.show() + + def wait(self): + self.mark.state = "wait" + self.mark.update() + self.name.setFont(font(14, W_MEDIUM)) + set_color(self.name, "InkFaint") + set_color(self.small, "InkFaint") + set_color(self.time, "InkFaint") + + def begin(self): + self.running, self.start = True, time.monotonic() + self.mark.state = "now" + self.mark.update() + self.name.setFont(font(14, W_SEMIBOLD)) + set_color(self.name, "Ink") + set_color(self.time, "Ink") + self.tick() + + def tick(self): + if self.running: + self.time.setText(clock(time.monotonic() - self.start)) + + def done(self): + if self.running: + self.tick() + self.running = False + self.mark.state = "done" + self.mark.update() + self.name.setFont(font(14, W_SEMIBOLD)) + set_color(self.name, "Ink") + set_color(self.time, "InkFaint") + + def stopped(self): + self.running = False + self.mark.state = "stopped" + self.mark.update() + + def skipped(self): + """(not done: left to later, the shaders' step when they were not prepared now)""" + if self.running: + self.tick() + self.running = False + self.mark.state = "skipped" + self.mark.update() + self.name.setFont(font(14, W_MEDIUM)) + set_color(self.name, "InkFaint") + set_color(self.time, "InkFaint") + + +class DumpRow(QAbstractButton): + """A found .nup: the whole row is the choice (RadioButton DumpRow: 16/13 padding, the ring and its Brand dot, #08FFFFFF hover)""" + + def __init__(self, dump, chip, why=None): + super().__init__() + self.setCheckable(True) + self.setAutoExclusive(True) + self.setCursor(Qt.CursorShape.PointingHandCursor) + self.setToolTip(dump.path) + self.hover = False + lay = QHBoxLayout(self) + lay.setContentsMargins(16 + 18 + 14, 13, 16, 13) + texts = QVBoxLayout() + texts.setSpacing(0) + name = label(os.path.basename(dump.path), 14, W_SEMIBOLD) + name.setTextFormat(Qt.TextFormat.PlainText) + name.setSizePolicy(QSizePolicy.Policy.Ignored, QSizePolicy.Policy.Preferred) + texts.addWidget(name) + texts.addWidget(faint("%s · %s · %s" % (dump.where, core.size_text(dump.size), core.ago(dump.modified)))) + if why: + w = faint(why) + w.setWordWrap(True) + set_color(w, "Bad") + w.setContentsMargins(0, 4, 0, 0) + texts.addWidget(w) + lay.addLayout(texts, 1) + lay.addSpacing(12) + lay.addWidget(chip, 0, Qt.AlignmentFlag.AlignVCenter) + + def enterEvent(self, e): + self.hover = True + self.update() + + def leaveEvent(self, e): + self.hover = False + self.update() + + def paintEvent(self, e): + p = QPainter(self) + p.setRenderHint(QPainter.RenderHint.Antialiasing) + if self.hover: + p.fillRect(self.rect(), argb("#08FFFFFF")) + c = QPointF(16 + 9, self.height() / 2) + p.setPen(QPen(C["Brand"] if self.isChecked() else C["InkFaint"], 2)) + p.setBrush(Qt.BrushStyle.NoBrush) + p.drawEllipse(c, 8, 8) + if self.isChecked(): + p.setPen(Qt.PenStyle.NoPen) + p.setBrush(C["Brand"]) + p.drawEllipse(c, 4, 4) + + +class Hline(QWidget): + def __init__(self): + super().__init__() + self.setFixedHeight(1) + + def paintEvent(self, e): + QPainter(self).fillRect(self.rect(), C["GlassLine"]) + + +class Glyph(QWidget): + """The small round badges: the "i" before the tools note, the "!" of a problem, the ready page's tick""" + + def __init__(self, kind): + super().__init__() + self.kind = kind + self.setFixedSize({"info": QSize(16, 16), "bad": QSize(28, 28), "ready": QSize(124, 124)}[kind]) + + def paintEvent(self, e): + p = QPainter(self) + p.setRenderHints(QPainter.RenderHint.Antialiasing | QPainter.RenderHint.TextAntialiasing) + r = QRectF(self.rect()) + if self.kind == "info": + p.setPen(Qt.PenStyle.NoPen) + p.setBrush(C["BrandSoft"]) + p.drawEllipse(r) + p.setPen(argb("#B9C8FF")) + p.setFont(font(10, W_BOLD)) + p.drawText(r, Qt.AlignmentFlag.AlignCenter, "i") + elif self.kind == "bad": + p.setPen(Qt.PenStyle.NoPen) + p.setBrush(C["BadSoft"]) + p.drawEllipse(r) + p.setPen(C["Bad"]) + p.setFont(font(14, W_BOLD)) + p.drawText(r, Qt.AlignmentFlag.AlignCenter, "!") + else: # (84 wide: GoSoft with its 60-blur green glow, a ring of 10 at -10, the tick) + c = r.center() + for k in range(30, 0, -2): + p.setPen(Qt.PenStyle.NoPen) + p.setBrush(QColor(0x5F, 0xB0, 0x8A, int(255 * 0.55 * 0.06 * (1 - k / 30)))) + p.drawEllipse(c, 42 + k, 42 + k) + p.setBrush(C["GoSoft"]) + p.drawEllipse(c, 42, 42) + p.setBrush(Qt.BrushStyle.NoBrush) + p.setPen(QPen(argb("#125FB08A"), 10)) + p.drawEllipse(c, 47, 47) + pen = QPen(C["Go"], 4) + pen.setCapStyle(Qt.PenCapStyle.RoundCap) + pen.setJoinStyle(Qt.PenJoinStyle.RoundJoin) + p.setPen(pen) + path = QPainterPath(QPointF(c.x() - 14, c.y())) + path.lineTo(c.x() - 4, c.y() + 10) + path.lineTo(c.x() + 14, c.y() - 10) + p.drawPath(path) + + +class LogBox(QPlainTextEdit): + """The details: mono 11 in InkSoft on #33000000, GlassLine border, 10 padding, no wrapping""" + + def __init__(self, border=True): + super().__init__() + self.setReadOnly(True) + self.setFont(font(11, family=MONO)) + self.setLineWrapMode(QPlainTextEdit.LineWrapMode.NoWrap) + self.setStyleSheet("QPlainTextEdit { color: %s; background: rgba(0,0,0,51); border: %s; border-radius: 0; padding: 10px; }" + % (css(C["InkSoft"]), "1px solid " + css(C["GlassLine"]) if border else "none") + SCROLLBARS) + + +SCROLLBARS = ("QScrollBar:vertical { background: transparent; width: 10px; margin: 2px; }" + "QScrollBar::handle:vertical { background: rgba(255,255,255,40); border-radius: 3px; min-height: 24px; }" + "QScrollBar::handle:vertical:hover { background: rgba(255,255,255,70); }" + "QScrollBar:horizontal { background: transparent; height: 10px; margin: 2px; }" + "QScrollBar::handle:horizontal { background: rgba(255,255,255,40); border-radius: 3px; min-width: 24px; }" + "QScrollBar::add-line, QScrollBar::sub-line { width: 0; height: 0; }" + "QScrollBar::add-page, QScrollBar::sub-page { background: transparent; }") + + +def scroll(widget, max_h=None): + s = QScrollArea() + s.setWidget(widget) + s.setWidgetResizable(True) + s.setHorizontalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAlwaysOff) + s.setFrameShape(QFrame.Shape.NoFrame) + s.setStyleSheet("QScrollArea { background: transparent; } QScrollArea > QWidget > QWidget { background: transparent; }" + SCROLLBARS) + s.viewport().setAutoFillBackground(False) + widget.setAutoFillBackground(False) + if max_h: + s.setMaximumHeight(max_h) + return s + + +def hbox(*items, spacing=0, margins=(0, 0, 0, 0)): + l = QHBoxLayout() + l.setSpacing(spacing) + l.setContentsMargins(*margins) + for it in items: + if it is None: + l.addStretch(1) + elif isinstance(it, int): + l.addSpacing(it) + elif isinstance(it, QWidget): + l.addWidget(it) + else: + l.addLayout(it) + return l + + +def vbox(*items, spacing=0, margins=(0, 0, 0, 0)): + l = QVBoxLayout() + l.setSpacing(spacing) + l.setContentsMargins(*margins) + for it in items: + if it is None: + l.addStretch(1) + elif isinstance(it, int): + l.addSpacing(it) + elif isinstance(it, QWidget): + l.addWidget(it) + else: + l.addLayout(it) + return l + + +def holder(layout): + w = QWidget() + w.setAttribute(Qt.WidgetAttribute.WA_TranslucentBackground) + w.setLayout(layout) + return w + + +# ================= the window ================= + +class Window(QWidget): + def __init__(self, args): + super().__init__() + self.args = args + self.poster = Poster() + self.lang = strings.default_language() + self.glass = False + self.download = None + self.runner = None + self.rows = {} + self.checks = {} + self.dumps = [] + self.log_lines = [] + self.current = None + self.finished = "" + self.chosen_nup = self.romfs = self.exefs = None + self.folders_note = None + self.updating = self.checking = self.source_picked = False + self.compiled = 0.0 + self.files_done = self.files_of = 0 + self.files_changed, self.files_compiled = -1, 0 + self.compile_eta, self.this_compile, self.previous_total = -1.0, 0, -1 + self.last_failure = "" + self.mode = "build" + self.latest_tag = self.update_target = None + self.launch_folder = "" + self.problem_is_tools = False + self.elapsed0 = self.compile0 = None + self.prepare_now, self.shaders_began, self.shaders_later, self.built_in = True, False, None, None + self.weight = WITHOUT_SHADERS + self.setWindowTitle("Hookshot") + self.setWindowIcon(QIcon(os.path.join(paths.ASSETS, "hookshot.ico"))) + self.setFixedSize(960, 600) + self.setAttribute(Qt.WidgetAttribute.WA_TranslucentBackground) + self.setAcceptDrops(True) + self.hook = QPixmap(os.path.join(paths.ASSETS, "hookshot.png")) + self.clock = QTimer(self, interval=1000, timeout=self.tick) + self._build_ui() + self.out_path.setText(os.path.join(os.path.expanduser("~"), "Games", "Hookshot")) + self.out_path.textChanged.connect(self.refresh) + tools = core.Tools.find() + self.tools_note.setText("Nothing else to install: Hookshot's own build tools are here, downloaded and checked by its setup." + if tools else "Hookshot's build tools are missing from its folder. Press Build and Hookshot downloads " + "them again.") + self.set_step(0) + self.refresh() + + def post(self, fn, *args): + self.poster.call.emit(fn, args) + + def T(self, key): + return strings.text(self.lang, key) + + # ---------------- the native window: Windows 11's acrylic behind it (Glass.cs), our own caption ---------------- + + def showEvent(self, e): + super().showEvent(e) + if not getattr(self, "_native", False): + self._native = True + self.glass = apply_glass(int(self.winId())) + self.update() + QTimer.singleShot(0, self.start) + + def nativeEvent(self, kind, message): + if kind == b"windows_generic_MSG": + msg = ctypes.wintypes.MSG.from_address(int(message)) + if msg.message == 0x0083 and msg.wParam: # WM_NCCALCSIZE: the whole window is the client area (no title bar) + return True, 0 + if msg.message == 0x0084: # WM_NCHITTEST: the top 64 px (but the buttons in it) moves the window + x, y = ctypes.c_short(msg.lParam & 0xFFFF).value, ctypes.c_short((msg.lParam >> 16) & 0xFFFF).value + pos = self.mapFromGlobal(self.screen_point(x, y)) + if 0 <= pos.y() < 64: + w = self.childAt(pos) + if w is None or not isinstance(w, (QAbstractButton,)) and w.parent() is not None and not isinstance( + w.parent(), QAbstractButton): + return True, 2 # HTCAPTION + return super().nativeEvent(kind, message) + + def screen_point(self, x, y): + from PySide6.QtCore import QPoint + dpr = self.devicePixelRatioF() + return QPoint(int(x / dpr), int(y / dpr)) + + def paintEvent(self, e): + p = QPainter(self) + r = QRectF(self.rect()) + if not self.glass: # (Windows 10, a light Windows, or no acrylic: the plain Slate page) + p.fillRect(r, C["Page"]) + lg = QLinearGradient(0, 0, 0, r.height()) + lg.setColorAt(0, argb("#401A1D24")) + lg.setColorAt(1, argb("#3312141A")) + p.fillRect(r, lg) + + # ---------------- the layout (MainWindow.xaml) ---------------- + + def _build_ui(self): + stack = QStackedLayout(self) + stack.setStackingMode(QStackedLayout.StackingMode.StackAll) + self.splash = Splash(self.hook) + self.main = QWidget() + self.main.setAttribute(Qt.WidgetAttribute.WA_TranslucentBackground) + self.main.hide() # (shown as the splash fades: a fade of its own would hide the glows inside it, Qt draws one effect deep) + self.main.setAttribute(Qt.WidgetAttribute.WA_TransparentForMouseEvents, True) + stack.addWidget(self.main) + stack.addWidget(self.splash) + self.splash.raise_() + + # the head: the hookshot, and the steps (or the update's versions) + head = QWidget(self.main) + head.setGeometry(0, 0, 960, 64) + icon = QLabel(head) + icon.setPixmap(self.hook.scaledToWidth(56, Qt.TransformationMode.SmoothTransformation)) + icon.setScaledContents(True) + icon.setGeometry(28, 32 - int(28 * self.hook.height() / self.hook.width() / 2), 28, + int(28 * self.hook.height() / self.hook.width())) + glow(icon, "#000000", 6, 2, 0.5) + title = label("Hookshot", 15, W_BOLD) + title.setParent(head) + title.move(28 + 28 + 10, 32 - title.sizeHint().height() // 2) + self.version_chip = label("", 12, color="InkSoft") + self.version_chip.setParent(head) + self.version_chip.move(title.x() + title.sizeHint().width() + 12, 32 - 8) + self.version_chip.hide() + self.step_pill = StepPill() + self.step_pill.setParent(head) + self.step_pill.move((960 - self.step_pill.width()) // 2, 12) + + body = QWidget(self.main) + body.setGeometry(64, 64 + 16, 960 - 128, 600 - 64 - 16 - 34) + self.pages = QStackedLayout(body) + self.files_page = self._files_page() + self.build_page = self._build_page() + self.ready_page = self._ready_page() + self.launch_page = self._launch_page() + self.problem_page = self._problem_page() + for pg in (self.files_page, self.build_page, self.ready_page, self.launch_page, self.problem_page): + self.pages.addWidget(pg) + + # the window's own buttons, over everything + self.caption = QWidget(self) + self.caption.setGeometry(960 - 6 - 92, 6, 92, 36) + mn, cl = Button(kind="caption", icon="min"), Button(kind="caption", icon="close") + mn.setToolTip("Minimize") + cl.setToolTip("Close") + mn.clicked.connect(self.showMinimized) + cl.clicked.connect(self.close_window) + self.caption.setLayout(hbox(mn, cl)) + self.caption.raise_() + + def _files_page(self): + self.files_title, self.files_sub = heading("Build your game"), sub("Pick your game files and where the game goes. Hookshot does " + "the rest, once.") + self.dump_list = QVBoxLayout() + self.dump_list.setSpacing(0) + dump_holder = holder(self.dump_list) + card = Card() + self.dump_scroll = scroll(dump_holder, 128) + card.setLayout(vbox(self.dump_scroll)) + browse = Button("Browse…", "link") + browse.clicked.connect(self.browse_nup) + other = Button("romfs and exefs folders", "link", color=C["InkSoft"]) + other.clicked.connect(self.browse_folders) + drop = faint(" · or drop the file on this window · ", 13) + links = hbox(note("Not listed?"), 8, browse, drop, other, None) + self.out_path = PathEdit() + change = Button("Change", "quiet") + change.clicked.connect(self.browse_out) + self.work_note = note("", 12) + self.work_change = Button("Change", "link", size=12) + self.work_change.clicked.connect(self.browse_work) + self.work_reset = Button("Default", "link", size=12) + self.work_reset.clicked.connect(lambda: self.change_work(None)) + left = vbox(section("INSTALL THE GAME TO"), hbox(self.out_path, 10, change), 8, + hbox(self.work_note, 12, self.work_change, 12, self.work_reset)) + self.language_box = Choice() + for code, name, _t in strings.LANGUAGES: + self.language_box.addItem(name) + self.language_box.setCurrentIndex(strings.LANGUAGES.index(self.lang)) + self.language_box.currentIndexChanged.connect(self.language_changed) + lang_col = holder(vbox(section("GAME LANGUAGE"), self.language_box)) + lang_col.setFixedWidth(200) + row3 = hbox(left, 20) + row3.addWidget(lang_col, 0, Qt.AlignmentFlag.AlignTop) + self.tools_note = note("", 12) + self.worker_box = Choice() + for c in core.worker_choices(): + self.worker_box.addItem(str(c)) + self.worker_box.setProperty("focus_ring", True) + self.worker_box.setToolTip("Automatic uses available memory and any existing worker override. More workers need more memory and " + "may not be faster.") + self.prepare_tick = Tick("Prepare shaders now", True) + self.prepare_tick.setToolTip("Prepares the game's shaders at the end of the build, a few minutes (longer on some GPUs). Unticked, " + "the game prepares them at its first start, behind a black window.") + self.prepare_tick.clicked.connect(self.save_settings) + work_col = holder(vbox(section("BUILD WORKERS"), self.worker_box, 12, self.prepare_tick)) + work_col.setFixedWidth(160) + row4 = hbox(hbox(Glyph("info"), 8, self.tools_note), 20) + row4.addWidget(work_col, 0, Qt.AlignmentFlag.AlignTop) + self.dlss_tick = Tick("NVIDIA DLSS (downloads NVIDIA's SDK under NVIDIA's license)") + self.dlss_tick.clicked.connect(self.dlss_changed) + dnote = note("For NVIDIA RTX graphics cards. NVIDIA's files stay on this PC: never share a game folder built with them.", 12) + dnote.setContentsMargins(26, 4, 0, 0) + found = section("FOUND ON THIS PC") + content = holder(vbox(self.files_title, self.files_sub, found, card, 10, links, row3, 12, row4, 14, self.dlss_tick, dnote, None)) + self.space_bar = Bar("Go", 160) + self.space_text = note("", 12) + self.space_text.setWordWrap(False) + self.about_text = faint("About 45 min") + self.build_button = Button("Build the game", "primary") + self.build_button.setEnabled(False) + self.build_button.clicked.connect(self.start_build) + foot = hbox(self.space_bar, 10, self.space_text, None, self.about_text, 14, self.build_button) + page = holder(vbox(scroll(content), 12, foot)) + return page + + def _build_page(self): + self.ring = Ring() + self.build_title, self.build_note = heading("Building your game"), sub("") + self.update_bar = Bar("Brand") + self.update_bar.setContentsMargins(0, 18, 0, 0) + self.update_bar.hide() + self.ring_box = holder(hbox(self.ring, 28)) # (collapsed with its gap in the update) + top = hbox(self.ring_box, vbox(None, self.build_title, self.build_note, 18, self.update_bar, None)) + self.stage_list = QVBoxLayout() + self.stage_list.setSpacing(0) + self.step_card = Card() + self.step_card.setLayout(vbox(self.stage_list, margins=(18, 6, 18, 6))) + self.log_box = LogBox() + self.log_box.hide() + mid = vbox(self.step_card, self.log_box, None) + self.show_log = Button(self.T("ShowDetails"), "toggle") + self.show_log.setCheckable(True) + self.show_log.toggled.connect(self.log_toggled) + self.cancel_button = Button(self.T("Cancel"), "quiet") + self.cancel_button.clicked.connect(self.cancel_build) + return holder(vbox(top, 22, mid, 14, hbox(self.show_log, None, self.cancel_button))) + + def _ready_page(self): + self.ready_title = heading("Your game is ready") + self.ready_note = sub("") + self.ready_note.setAlignment(Qt.AlignmentFlag.AlignCenter) + self.ready_note.setWordWrap(False) + self.play_button = Button(self.T("Play"), "go", icon="play") + self.play_button.clicked.connect(self.play) + self.open_button = Button(self.T("OpenFolder"), "quiet") + self.open_button.setFixedHeight(50) + self.open_button.fixed_h = 50 + self.open_button.clicked.connect(lambda: core.open_folder(self.finished)) + self.desktop_tick = Tick("Desktop shortcut", True) + self.start_tick = Tick("Start menu entry", True) + self.desktop_tick.clicked.connect(self.shortcuts_changed) + self.start_tick.clicked.connect(self.shortcuts_changed) + self.storage_note = note("") + self.storage_note.setAlignment(Qt.AlignmentFlag.AlignCenter) + self.storage_note.setMaximumWidth(560) + self.free_button = Button("Free up space", "link") + self.free_button.clicked.connect(self.free_space) + show = Button("Show folder", "link") + show.clicked.connect(lambda: core.open_folder(core.work() if core.work_chosen() else paths.base())) + self.storage_row = holder(vbox(hbox(None, self.storage_note, None), 8, hbox(None, self.free_button, 18, show, None))) + self.storage_row.hide() + col = vbox(None, hbox(None, Glyph("ready"), None), 24 - 20, hbox(None, self.ready_title, None), hbox(None, self.ready_note, None), + 28, hbox(None, self.play_button, 12, self.open_button, None), 22, + hbox(None, self.desktop_tick, 18, self.start_tick, None), 26, self.storage_row, None) + return holder(col) + + def _launch_page(self): + z = section("THE LEGEND OF ZELDA") + z.setContentsMargins(0, 0, 0, 0) + t = heading("Ocarina of Time") + t.setFont(font(34, W_BOLD)) + self.launch_version = sub("") + self.launch_version.setContentsMargins(0, 8, 0, 0) + self.launch_version.setWordWrap(False) + self.launch_status = label("Checking for updates…", 13, color="InkSoft") + self.launch_update = Button("Update", "primary") + self.launch_update.setFixedHeight(50) + self.launch_update.fixed_h, self.launch_update.padx = 50, 26 + self.launch_update.hide() + self.launch_update.clicked.connect(lambda: updater.update_folder(self, self.launch_folder, self.update_target, False)) + self.launch_play = Button(self.T("Play"), "go", icon="play") + self.launch_play.clicked.connect(self.launch_play_click) + fold = Button("Open the game folder", "link", color=C["InkFaint"]) + fold.clicked.connect(lambda: core.open_folder(self.launch_folder)) + return holder(vbox(None, hbox(None, z, None), 4, hbox(None, t, None), hbox(None, self.launch_version, None), 16, + hbox(None, self.launch_status, None), 28, hbox(None, self.launch_update, 12, self.launch_play, None), 22, + hbox(None, fold, None), None)) + + def _problem_page(self): + self.problem_title, self.problem_sub = heading("The build stopped"), sub("Nothing was changed in your game folder.") + self.problem_head = label("", 16, W_SEMIBOLD, wrap=True) + self.problem_head.setMaximumWidth(740) + self.problem_text = note("", 14) + self.problem_steps = QVBoxLayout() + self.problem_steps.setSpacing(0) + self.problem_steps.setContentsMargins(0, 8, 0, 0) + box = ProblemCard() + head = hbox(Glyph("bad"), 12) + head.addWidget(self.problem_head, 1) + self.problem_scroll = scroll(holder(vbox(self.problem_text, self.problem_steps)), 190) + box.setLayout(vbox(head, 10, self.problem_scroll, margins=(24, 22, 24, 22))) + self.problem_details = Button("Show details", "link", color=C["InkSoft"]) + self.problem_details.clicked.connect(self.problem_show_details) + self.copy_report = Button("Copy for a bug report", "link", color=C["InkSoft"]) + self.copy_report.clicked.connect(self.copy_for_report) + self.back_button = Button(self.T("Back"), "quiet") + self.back_button.clicked.connect(self.back) + self.repair_button = Button("Repair", "primary") + self.repair_button.clicked.connect(self.repair) + foot = hbox(self.problem_details, faint(" · ", 13), self.copy_report, None, self.back_button, 10, self.repair_button) + return holder(vbox(self.problem_title, self.problem_sub, 22, box, None, 14, foot)) + + # ================= the start ================= + + def start(self): + if os.environ.get("HOOKSHOT_TEST_SHORTCUTS"): # (development: both shortcuts made for it, the result beside the app) + folder = os.environ["HOOKSHOT_TEST_SHORTCUTS"] + core.shortcuts(folder, True, True) + paths.write_small(os.path.join(paths.APP_DIR, "shortcut-test.txt"), "desktop %s start %s error %s" % ( + os.path.isfile(core.desktop_link()), os.path.isfile(core.start_link()), core.shortcut_error)) + QApplication.quit() + return + self.t_start = time.monotonic() + a = self.args + if a.update: + self.splash.text = "Getting the update…" + QTimer.singleShot(900, lambda: (self.show_main(), updater.update_folder(self, a.update, a.tag, a.fetched))) + return + if a.launch: + self.splash.text = "Starting Hookshot…" + QTimer.singleShot(700, lambda: (self.show_main(), self.show_launcher(a.launch.rstrip("\\/")))) + return + if a.repair or not core.Tools.find(): # (the first start, or Repair: the build tools, as the setup got them) + self.splash.text = "Getting the build tools…" + self.get_tools(a.repair) + return + self.find_files() + + def get_tools(self, repair): + def status(s): + self.splash.text = s + + def run(): + try: + setup_parts.install(lambda s: self.post(status, s), tag=None, repair=repair) + self.post(self.find_files) + except setup_parts.SetupError as e: + self.post(self.tools_failed, str(e)) + import threading + threading.Thread(target=run, daemon=True).start() + + def tools_failed(self, why): + self.show_main() + self.show_problem(why, tools=False) + self.problem_title.setText("The build tools could not be downloaded") + self.problem_sub.setText("Nothing was changed on this PC.") + self.repair_button.show() + self.repair_button.setText("Try again") + + def find_files(self): + self.splash.text = "Looking for your game files…" + import threading + + def run(): + found = core.find_dumps() + self.post(self.found_files, found) + threading.Thread(target=run, daemon=True).start() + + def found_files(self, found): + self.dumps.extend(found) + n = len(found) + self.splash.text = "No game files found yet" if n == 0 else "Found 1 game file" if n == 1 else "Found %d game files" % n + wait = 1.7 - (time.monotonic() - self.t_start) + QTimer.singleShot(max(0, int(wait * 1000)), self.found_go) + + def found_go(self): + self.apply_settings() + self.fill_dumps() + self.show_main() + if os.environ.get("HOOKSHOT_UI_PREVIEW"): + self.preview(os.environ["HOOKSHOT_UI_PREVIEW"]) + return + self.check_all() + import threading + + def run(): + tag = updater.latest_tag() + self.post(self.got_latest, tag) + threading.Thread(target=run, daemon=True).start() + + def got_latest(self, tag): + self.latest_tag = tag + self.refresh_mode() + + def show_main(self): + """The hookshot fades out and the first step fades in, in the same window""" + if getattr(self, "_shown_main", False): + return + self._shown_main = True + fx = QGraphicsOpacityEffect(self.splash) + self.splash.setGraphicsEffect(fx) + a = QPropertyAnimation(fx, b"opacity", self) + a.setDuration(450) + a.setStartValue(1.0) + a.setEndValue(0.0) + a.finished.connect(lambda: (self.splash.hide(), self.splash.stop())) + a.start() + self._splash_anim = a + t0 = time.monotonic() + + def grow(): + f = min(1.0, (time.monotonic() - t0) / 0.5) + self.splash.scale = 1 + 0.04 * f + if f < 1: + QTimer.singleShot(16, grow) + grow() + self.main.move(0, 10) + lift = QPropertyAnimation(self.main, b"pos", self) + lift.setDuration(600) + lift.setStartValue(self.main.pos()) + from PySide6.QtCore import QPoint + lift.setEndValue(QPoint(0, 0)) + lift.setEasingCurve(QEasingCurve.Type.OutCubic) + self._lift_anim = lift + QTimer.singleShot(150, lambda: (self.main.show(), self.splash.raise_(), self.caption.raise_(), lift.start())) + self.main.setAttribute(Qt.WidgetAttribute.WA_TransparentForMouseEvents, False) + + def show_page(self, page): + self.pages.setCurrentWidget(page) + + def set_step(self, on): + self.step_pill.set(on) + self.step_pill.move((960 - self.step_pill.width()) // 2, 12) + self.step_pill.show() + self.version_chip.hide() + + # ================= what the button does: build, update the game, play it, or get the newer builder ================= + + def refresh_mode(self): + if self.updating or self.runner: + return + out = self.out_path.text() + game = core.game_version(out) if len(out) > 2 else None + tools = core.Tools.find() + builder = tools.version if tools else None + self.update_target = self.latest_tag or builder + if game and self.update_target and game != self.update_target and not self.source_picked: + self.mode = "update_game" + elif game and not self.source_picked: + self.mode = "up_to_date" + elif self.latest_tag and builder and builder != self.latest_tag: + self.mode = "update_builder" + else: + self.mode = "build" + if self.mode == "update_game": + self.files_title.setText("Update available") + self.files_sub.setText("Hookshot %s is available. Your game is %s: Update builds it again from the game files kept in its " + "folder, compiling only what changed." % (self.update_target, game)) + self.build_button.setText("Update") + elif self.mode == "up_to_date": + self.files_title.setText("Your game is up to date") + self.files_sub.setText("Your game is %s, as this builder. GitHub did not answer, so newer versions could not be checked." % game + if not self.latest_tag else + "%s is the latest version. Pick game files below only to build it again from scratch." % game) + self.build_button.setText("Play") + elif self.mode == "update_builder": + self.files_title.setText("Build your game") + self.files_sub.setText("Hookshot Builder %s is available (this one is %s): update it first, then build." % (self.latest_tag, + builder)) + self.build_button.setText("Update builder") + else: + self.files_title.setText("Build your game") + self.files_sub.setText("Pick your game files and where the game goes. Hookshot does the rest, once.") + self.build_button.setText("Build the game") + self.build_button.setEnabled(self.mode != "build" or (self.source_ready() and len(out) > 2)) + self.about_text.setVisible(self.mode == "build") + + # ---- the launcher: the game folder's Hookshot.lnk starts this with --launch ---- + + def show_launcher(self, folder): + self.launch_folder = folder + self.out_path.setText(folder) + self.step_pill.hide() + game = core.game_version(folder) or core.read_version(folder) + self.launch_version.setText(("Hookshot %s" % game if game else "Hookshot") + " · " + folder) + self.launch_play.setEnabled(os.path.isfile(core.game_exe(folder))) + self.show_page(self.launch_page) + import threading + + def run(): + tag = updater.latest_tag() + self.post(self.launcher_latest, folder, game, tag) + threading.Thread(target=run, daemon=True).start() + + def launcher_latest(self, folder, game, tag): + self.latest_tag = tag + tools = core.Tools.find() + self.update_target = tag or (tools.version if tools else None) + if not self.update_target: + self.launch_status.setText("Could not check for updates (GitHub did not answer)") + elif game and game != self.update_target and core.game_version(folder): + self.launch_status.setText("Update available: %s" % self.update_target) + self.launch_status.setFont(font(13, W_SEMIBOLD)) + set_color(self.launch_status, argb("#B9C8FF")) + self.launch_update.show() + elif game and game != self.update_target: + self.launch_status.setText("%s is out: build this folder again with the Hookshot Builder to get it" % self.update_target) + else: + self.launch_status.setText("Up to date with this builder (GitHub did not answer)" if not tag else "Up to date") + + def launch_play_click(self): + core.start_game(self.launch_folder) + QApplication.quit() + + # ---- what the player chose last time ---- + + def apply_settings(self): + s = core.load_settings() + if not s: + return + if s.get("Out"): + self.out_path.setText(s["Out"]) + self.dlss_tick.setChecked(bool(s.get("Dlss"))) + self.prepare_tick.setChecked(not s.get("PrepareLater")) + r, x, n = s.get("Romfs"), s.get("Exefs"), s.get("Nup") + if r and x and os.path.isdir(r) and os.path.isfile(os.path.join(x, "main")): + self.romfs, self.exefs = r, x + self.folders_note = "Using the folders %s and %s (as last time)" % (r, x) + elif n and os.path.isfile(n): + if not any(os.path.normcase(d.path) == os.path.normcase(n) for d in self.dumps): + st = os.stat(n) + self.dumps.insert(0, core.Dump(os.path.abspath(n), "Last time", st.st_size, st.st_mtime)) + self.chosen_nup = n + + def save_settings(self): + core.save_settings({"Out": self.out_path.text(), "Nup": None if self.romfs else self.chosen_nup, "Romfs": self.romfs, + "Exefs": self.exefs, "Dlss": self.dlss_tick.isChecked(), + "PrepareLater": not self.prepare_tick.isChecked()}) + + def preview(self, what): + """(development) HOOKSHOT_UI_PREVIEW=build|ready|problem|update|launch: one screen with made-up progress, nothing run""" + if what == "build": + self.make_rows(False) + self.set_step(1) + self.show_page(self.build_page) + self.rows["read"].begin(), self.rows["read"].done(), self.rows["read"].small_text("Version checked") + self.rows["lift"].begin(), self.rows["lift"].done() + self.rows["compile"].begin(), self.rows["compile"].small_text("31 of 58 files") + self.set_progress(0.54) + self.build_note.setText("About 19 minutes left · your PC stays usable") + elif what == "ready": + self.finished = self.out_path.text() + self.elapsed0 = time.monotonic() - 2700 + self.show_ready() + elif what == "problem": + self.show_problem("Files of the builder's own compiler are missing (clang.exe in tools\\llvm-mingw\\bin).", tools=True) + elif what == "update": + self.begin_update(self.out_path.text(), "honest-eta") + self.rows["download"].done(), self.rows["download"].small_text("3 MB · build tools unchanged") + self.rows["read"].begin(), self.rows["read"].done() + self.rows["lift"].begin(), self.rows["lift"].done() + self.rows["compile"].begin(), self.rows["compile"].small_text("7 of 516 files") + self.set_progress(0.62) + self.build_note.setText("About 3 minutes left. Your current game stays as it is until the new one is complete.") + elif what == "launch": + self.show_launcher(self.out_path.text()) + + # ================= 1: game files ================= + + def fill_dumps(self): + while self.dump_list.count(): + it = self.dump_list.takeAt(0) + if it.widget(): + it.widget().deleteLater() + self.folders_label = None + if self.romfs and self.exefs and self.folders_note: + l = note(self.folders_note) + l.setContentsMargins(16, 12, 16, 12) + self.folders_label = l + self.dump_list.addWidget(l) + if not self.dumps: + l = note("No .nup files in Downloads, Desktop, Documents or the top folders of your drives. Browse to yours, or drop it on " + "this window.") + l.setContentsMargins(16, 14, 16, 14) + self.dump_list.addWidget(l) + self.fit_dumps() + return + for i, d in enumerate(self.dumps): + if i: + self.dump_list.addWidget(Hline()) + c = self.checks.get(d.path, "missing") + why = c[1] if isinstance(c, tuple) and not c[0] and not c[1].startswith("This is build") else None + row = DumpRow(d, self.chip(d.path), why) + row.setChecked(self.chosen_nup is not None and os.path.normcase(d.path) == os.path.normcase(self.chosen_nup)) + row.clicked.connect(lambda _=False, p=d.path: self.choose(p)) + self.dump_list.addWidget(row) + self.fit_dumps() + + def fit_dumps(self): + """The list as tall as its rows, at most 128 (then it scrolls)""" + def fit(): + self.dump_list.activate() + h = sum(self.dump_list.itemAt(i).widget().sizeHint().height() for i in range(self.dump_list.count()) + if self.dump_list.itemAt(i).widget() is not None and not self.dump_list.itemAt(i).widget().isHidden()) + self.dump_scroll.setFixedHeight(max(1, min(128, h + 1))) + self.dump_scroll.setVerticalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAsNeeded if h > 128 else + Qt.ScrollBarPolicy.ScrollBarAlwaysOff) + fit() + QTimer.singleShot(0, fit) + + def chip(self, path): + c = self.checks.get(path) + if c is None: + ch = Chip("Checking…", "InkFaint") + elif c[0]: + ch = Chip("✓ Supported version", "Go", "GoSoft") + elif c[1].startswith("This is build"): + ch = Chip("Not the supported version", "Warn") + else: + ch = Chip("Can't be read", "Bad") + if c is not None and not c[0]: + ch.setToolTip(c[1]) + return ch + + def check_all(self): + """Each found file's version, one at a time; the first good one is chosen""" + if self.checking: + return + tools = core.Tools.find() + nxt = next((d for d in self.dumps if d.path not in self.checks), None) if tools else None + if nxt is None: + if self.chosen_nup is None and self.romfs is None and self.dumps and tools: + self.chosen_nup = self.dumps[0].path + self.fill_dumps() + self.refresh() + return + self.checking = True + self.checks[nxt.path] = None + self.fill_dumps() + import threading + + def run(): + r = core.check(tools, ["--check", "--nup", nxt.path]) + self.post(self.checked, nxt.path, r) + threading.Thread(target=run, daemon=True).start() + + def checked(self, path, r): + self.checking = False + self.checks[path] = r + if not r[0] and "antivirus" in r[1].lower(): + self.show_problem(r[1], tools=True) + return + if self.chosen_nup is None and self.romfs is None and r[0]: + self.chosen_nup = path + self.fill_dumps() + self.refresh() + self.check_all() + + def choose(self, path): + self.source_picked = True + self.chosen_nup = path + self.romfs = self.exefs = None + if self.folders_label is not None: + self.folders_label.deleteLater() + self.folders_label = None + self.refresh() + + def add_dump(self, path, where): + path = os.path.abspath(path) + if not any(os.path.normcase(d.path) == os.path.normcase(path) for d in self.dumps): + st = os.stat(path) + self.dumps.insert(0, core.Dump(path, where, st.st_size, st.st_mtime)) + self.source_picked = True + self.chosen_nup = path + self.romfs = self.exefs = None + self.fill_dumps() + self.check_all() + self.refresh() + + def source_ready(self): + if self.romfs and self.exefs: + return True + c = self.checks.get(self.chosen_nup) if self.chosen_nup else None + return bool(c and c[0]) + + def refresh(self, *_): + self.build_button.setEnabled(self.source_ready() and len(self.out_path.text()) > 2) + self.show_work() + root, freeb = core.free_space(self.out_path.text()) + if freeb is not None: + self.space_bar.set(min(1.0, NEEDED_BYTES / max(1, freeb)), "Go" if freeb >= NEEDED_BYTES else "Bad") + self.space_text.setText("Needs 45 GB · %s has %d GB free" % (root.rstrip("\\"), freeb >> 30)) + set_color(self.space_text, "InkSoft" if freeb >= NEEDED_BYTES else "Bad") + else: + self.space_text.setText("Needs 45 GB") + self.refresh_mode() + + def dragEnterEvent(self, e): + if e.mimeData().hasUrls() and not self.runner: + e.acceptProposedAction() + + def dropEvent(self, e): + for u in e.mimeData().urls(): + p = u.toLocalFile() + if p.lower().endswith(".nup") and os.path.isfile(p): + self.add_dump(p, "Dropped") + elif os.path.isdir(os.path.join(p, "romfs")) and os.path.isdir(os.path.join(p, "exefs")): + self.use_folders(os.path.join(p, "romfs"), os.path.join(p, "exefs")) + + def browse_nup(self): + f, _ = QFileDialog.getOpenFileName(self, "Choose your game file", "", "Game file (*.nup)") + if f: + self.add_dump(os.path.normpath(f), "Chosen") + + def browse_folders(self): + both = QFileDialog.getExistingDirectory(self, "Choose the folder with romfs and exefs (or the romfs folder)") + if not both: + return + both = os.path.normpath(both) + if os.path.isdir(os.path.join(both, "romfs")) and os.path.isdir(os.path.join(both, "exefs")): + self.use_folders(os.path.join(both, "romfs"), os.path.join(both, "exefs")) + return + ex = QFileDialog.getExistingDirectory(self, "Now choose the exefs folder") + if ex: + self.use_folders(both, os.path.normpath(ex)) + + def use_folders(self, r, x): + if not os.path.isfile(os.path.join(x, "main")): + QMessageBox.information(self, "Hookshot", "That exefs folder has no file named main.") + return + self.source_picked = True + self.romfs, self.exefs, self.chosen_nup = r, x, None + self.folders_note = "Using the folders %s and %s" % (r, x) + self.fill_dumps() + self.refresh() + + def browse_out(self): + d = QFileDialog.getExistingDirectory(self, "Choose where the game goes") + if d: + self.out_path.setText(os.path.normpath(d)) + + def browse_work(self): + d = QFileDialog.getExistingDirectory(self, "Choose where Hookshot keeps its build files") + if d: + self.change_work(os.path.normpath(d)) + + def change_work(self, picked): + try: + core.move_work(picked) + except OSError as e: + QMessageBox.information(self, "Hookshot", "Hookshot cannot keep its build files there: %s" % e) + self.refresh() + + def show_work(self): + w = core.work() + self.work_reset.setVisible(core.work_chosen()) + self.work_note.setText( + "Build files: about %s, kept in %s so that updates are quick (HOOKSHOT_KEEP_BUILD_FILES)." % (core.gb(core.BUILD_FILES_BYTES), w) + if core.keep_build_files() else + "Build files: about %s in %s while it builds; when the game is ready about 1 GB stays (the compiled code) so that updates " + "compile only what changed." % (core.gb(core.BUILD_FILES_BYTES), w)) + set_color(self.work_note, "InkSoft") + root, freeb = core.free_space(w) + if freeb is not None and freeb < core.BUILD_FILES_BYTES: + self.work_note.setText(self.work_note.text() + " %s has only %s free." % (root.rstrip("\\"), core.gb(freeb))) + set_color(self.work_note, "Bad") + + def language_changed(self, i): + self.lang = strings.LANGUAGES[i] + self.cancel_button.setText(self.T("Cancel")) + self.back_button.setText(self.T("Back")) + self.open_button.setText(self.T("OpenFolder")) + self.play_button.setText(self.T("Play")) + self.show_log.setText(self.T("HideDetails" if self.show_log.isChecked() else "ShowDetails")) + + # ================= 2: the build ================= + + def start_build(self): + if self.mode == "update_game": + self.save_settings() + updater.update_folder(self, self.out_path.text(), self.update_target, False) + return + if self.mode == "up_to_date": + self.finished = self.out_path.text() + self.play() + return + if self.mode == "update_builder": + self.save_settings() + self.build_button.setEnabled(False) + self.build_button.setText("Getting the new builder…") + self.begin_update(self.out_path.text(), self.latest_tag) + updater.run_latest_setup(self) + return + self.save_settings() + tools = core.Tools.find() + if not tools: + self.show_problem("The build tools are not in Hookshot's folder.", tools=True) + return + if self.romfs and self.exefs: + args = ["--romfs", self.romfs, "--exefs", self.exefs] + elif self.chosen_nup: + args = ["--nup", self.chosen_nup] + else: + return + args += ["--out", self.out_path.text(), "--language", self.lang[0]] + self.prepare_now = self.prepare_tick.isChecked() + if not self.prepare_now: + args.append(SHADERS_LATER) + self.run(tools, args, dlss=self.dlss_tick.isChecked()) + + def dlss_changed(self): + if self.dlss_tick.isChecked(): + b = QMessageBox.question(self, "NVIDIA DLSS", DLSS_LICENSE, QMessageBox.StandardButton.Yes | QMessageBox.StandardButton.No) + if b != QMessageBox.StandardButton.Yes: + self.dlss_tick.setChecked(False) + self.save_settings() # (kept at once: ticked and closed, it is still ticked next time) + + def make_rows(self, update, shaders=False): + while self.stage_list.count(): + it = self.stage_list.takeAt(0) + if it.widget(): + it.widget().deleteLater() + self.rows = {} + self.weight = WITH_SHADERS if shaders else WITHOUT_SHADERS + lst = [s for s in STAGES if (update or s[0] != "download") and (shaders or s[0] != "shaders")] + for i, (sid, b, u) in enumerate(lst): + row = StepRow(i + 1, u if update else b, i == len(lst) - 1) + self.rows[sid] = row + self.stage_list.addWidget(row) + if update: + self.rows["read"].small_text("Your game files are kept: the .nup isn't needed") + + def hard_drive_warning(self): + """The game folder or the build files folder on a hard drive: the build works, but much more slowly than on an SSD. + None when both are on an SSD (or their drive cannot be told, as on a network share)""" + for what, folder in (("game folder", self.out_path.text()), ("build files folder", core.work())): + if core.spinning(folder) is not True: + continue + root = os.path.splitdrive(os.path.abspath(folder))[0] or folder + return ("The %s is on a hard drive (%s). An SSD builds the game much faster: build there if you can.\n\n" + "Choose OK to build here anyway, or Cancel to choose another folder." % (what, root)) + return None + + def run(self, tools, args, keep_rows=False, dlss=False): + # (a warning, not a stop: the player may have no other folder with the room. An update keeps its own flow) + slow = None if keep_rows else self.hard_drive_warning() + if slow and QMessageBox.question(self, "Hookshot", slow, QMessageBox.StandardButton.Ok | + QMessageBox.StandardButton.Cancel) != QMessageBox.StandardButton.Ok: + return + sel = self.worker_box.currentText() + workers = None if sel == core.AUTOMATIC else int(sel) + if not keep_rows: + self.updating = False + self.make_rows(False, self.prepare_now) + self.set_step(1) + self.build_title.setText("Building your game") + self.update_bar.hide() + self.ring_box.show() + self.log_lines = [] + self.log_box.clear() + self.current, self.finished = None, "" + self.compiled, self.files_done, self.files_of = 0.0, 0, 0 + self.files_changed, self.files_compiled, self.compile_eta = -1, 0, -1.0 + self.this_compile, self.previous_total = 0, -1 + self.shaders_began, self.shaders_later, self.built_in = False, None, None + self.show_page(self.build_page) + self.cancel_button.show() + self.set_progress(self.weight["read"][0] if self.updating else 0) + self.elapsed0, self.compile0 = time.monotonic(), None + self.clock.start() + self.tick() + if core.work_chosen() and "--work" not in args: + args += ["--work", core.work()] + if dlss and "--fetch-ngx-sdk" not in args: + args += ["--fetch-ngx-sdk", "--accept-nvidia-license"] + self.runner = core.Runner(tools, args, workers, core.ngx_sdk() if dlss else None, + on_line=lambda l: self.post(self.line, l), on_finished=lambda f: self.post(self.ended, f)) + self.runner.start() + + FILES_LINE = re.compile(r"^(\d+) files, ") + FILE_LINE = re.compile(r"^\s+\S+\s+\d+ MB\s+\d+s") + + def line(self, line): + if line.startswith("@@stage "): + sid = line.split(" ")[1] + if self.current in self.rows: + self.rows[self.current].done() + self.current = sid + if sid in self.rows: + self.rows[sid].begin() + if sid == "compile": + self.compile0 = time.monotonic() + if sid == "shaders": + self.shaders_began = True + if sid in self.weight: + self.set_progress(self.weight[sid][0]) + elif line.startswith("@@progress "): + try: + f = float(line[11:]) + except ValueError: + return + if self.current in self.weight: + if self.current == "compile": + self.compiled = f + w = self.weight[self.current] + self.set_progress(w[0] + w[1] * max(0.0, min(1.0, f))) + return + elif line.startswith(("@@plan ", "@@file ", "@@eta ", "@@times ")): + self.number(line) + return + elif line.startswith("@@done "): + self.finished = line[7:] + elif NOT_PREPARED.match(line): + self.shaders_later = NOT_PREPARED.match(line).group(1) + elif self.FILES_LINE.match(line): + self.files_of = int(self.FILES_LINE.match(line).group(1)) + elif self.FILE_LINE.match(line): + self.files_done += 1 + if self.current == "compile" and self.files_changed < 0 and self.files_of > 0 and "compile" in self.rows: + self.rows["compile"].small_text("%d of %d files" % (self.files_done, self.files_of)) + self.add_log(line) + + def number(self, line): + p = line.split(" ") + if p[0] == "@@plan" and len(p) == 3 and p[1].isdigit() and p[2].isdigit(): + self.files_changed, self.files_of, self.files_compiled = int(p[1]), int(p[2]), 0 + self.changed_files() + elif p[0] == "@@file" and len(p) == 4 and p[3] in ("new", "again"): + self.files_compiled += 1 + self.changed_files() + elif p[0] == "@@eta" and len(p) == 2: + try: + self.compile_eta = float(p[1]) + except ValueError: + pass + elif p[0] == "@@times" and len(p) == 4: + try: + self.this_compile, self.previous_total = int(p[2]), int(p[3]) + self.built_in = time.monotonic() - (self.elapsed0 or time.monotonic()) # (without the shaders' minutes) + except ValueError: + pass + + def changed_files(self): + if self.files_changed < 0 or "compile" not in self.rows: + return + self.rows["compile"].small_text("Everything is already built" if self.files_changed == 0 else + "%d of %d files" % (self.files_compiled, self.files_changed) if self.files_changed >= self.files_of + else "%d of %d changed files" % (self.files_compiled, self.files_changed)) + + def add_log(self, line): + self.log_lines.append(line) + if len(self.log_lines) > 12000: + del self.log_lines[:4000] + if self.log_box.isVisible(): + self.log_box.appendPlainText(line) + + def set_progress(self, f): + f = max(0.0, min(1.0, f)) + self.ring.set(f) + self.update_bar.set(f) + + def tick(self): + for r in self.rows.values(): + r.tick() + if not self.runner: + return + tail = ". Your current game stays as it is until the new one is complete." if self.updating else " · your PC stays usable" + if self.current == "compile" and self.compile0: + el = time.monotonic() - self.compile0 + left = (time_left(self.compile_eta) if self.compile_eta >= 0 else + time_left(el / self.compiled * (1 - self.compiled) + 60) if self.compiled > 0.03 else "Estimating…") + self.build_note.setText("Compiling %s · %s%s" % (clock(el), left, tail)) + return + self.build_note.setText(("Working…" if self.updating else "Usually about 45 minutes") + tail) + + def ended(self, failure): + self.clock.stop() + self.runner = None + if self.finished: + for sid, r in self.rows.items(): + why = None if sid != "shaders" else ("Left to the game's first start: %s" % self.shaders_later if self.shaders_later else + None if self.shaders_began else "Left to the game's first start") + if why: + r.skipped() + r.small_text(why) + else: + r.done() + self.set_progress(1) + self.show_ready() + return + if failure == "cancelled": + self.set_step(0) + self.show_page(self.files_page) + return + if self.current in self.rows: + self.rows[self.current].stopped() + self.show_problem(failure or "The build tools stopped without saying why.", tools=bool(failure and "antivirus" in failure.lower())) + + def log_toggled(self, show): + self.log_box.setVisible(show) + self.step_card.setVisible(not show) + self.show_log.setText(self.T("HideDetails" if show else "ShowDetails")) + if show: + self.log_box.setPlainText("\n".join(self.log_lines)) + self.log_box.verticalScrollBar().setValue(self.log_box.verticalScrollBar().maximum()) + + def cancel_build(self): + if self.download: + self.download.cancel() + if self.runner: + self.runner.cancel() + + # ================= 3: play ================= + + def show_ready(self): + self.set_step(2) + self.ready_title.setText("Your game is updated" if self.updating else "Your game is ready") + took = time.monotonic() - (self.elapsed0 or time.monotonic()) + compiled = " (compiled in %s)" % span(self.this_compile) if self.this_compile > 0 else "" + self.ready_note.setText("%s in %s%s%s · %s" % ("Updated" if self.updating else "Built", span(took), compiled, self.against(self.built_in or took), + self.finished)) + if not os.environ.get("HOOKSHOT_UI_PREVIEW"): + if not self.updating: + core.shortcuts(self.finished, self.desktop_tick.isChecked(), self.start_tick.isChecked()) + else: # (the ones the player has, onto the folder's launcher; the ticks say which there are) + self.desktop_tick.setChecked(os.path.isfile(core.desktop_link())) + self.start_tick.setChecked(os.path.isfile(core.start_link())) + core.refresh_shortcuts(self.finished) + if core.shortcut_error: + self.add_log("shortcut not made: " + core.shortcut_error) + self.show_page(self.ready_page) + import threading + + def run(): + if not core.keep_build_files(): + left = core.free(after_build=True) + if left >= 100 << 20: + self.post(self.add_log, "build files: %s could not be deleted (a file in use); they go after the next build" % + core.gb(left)) + self.post(self.show_storage_use, *core.measure()) + threading.Thread(target=run, daemon=True).start() + + def show_storage_use(self, total, freeable, extra=""): + self.storage_row.hide() + if total < 100 << 20: + return + where = "%s and %s" % (paths.base(), core.work()) if core.work_chosen() else paths.base() + self.storage_note.setText( + "Hookshot Builder keeps %s on this PC (%s). %s of it only makes the next update quicker and can be freed: your game keeps " + "playing, and the next update then takes about as long as this build." % (core.gb(total), where, core.gb(freeable)) + if freeable >= 100 << 20 else "Hookshot Builder keeps %s on this PC (%s): the builder itself." % (core.gb(total), where)) + self.storage_note.setText(self.storage_note.text() + extra) + self.free_button.setVisible(freeable >= 100 << 20) + self.free_button.setEnabled(True) + self.free_button.setText("Free up space") + self.storage_row.show() + + def free_space(self): + self.free_button.setEnabled(False) + self.free_button.setText("Freeing…") + import threading + + def run(): + left = core.free() + extra = (" (%s could not be deleted: a file is in use. Close the game and the builder, and delete the folder's contents by " + "hand.)" % core.gb(left)) if left >= 100 << 20 else "" + self.post(self.show_storage_use, *core.measure(), extra) + threading.Thread(target=run, daemon=True).start() + + def shortcuts_changed(self): + if self.finished: + core.shortcuts(self.finished, self.desktop_tick.isChecked(), self.start_tick.isChecked()) + + def against(self, now): + if self.previous_total <= 0: + return "" + diff = self.previous_total - now + how = "about the same" if abs(diff) < 30 else "%s faster" % span(diff) if diff > 0 else "%s slower" % span(-diff) + return " · last time %s (%s)" % (span(self.previous_total), how) + + def play(self): + core.start_game(self.finished) + self.showMinimized() + + # ================= when something goes wrong ================= + + def show_problem(self, why, tools): + self.last_failure, self.problem_is_tools = why, tools + while self.problem_steps.count(): + it = self.problem_steps.takeAt(0) + if it.widget(): + it.widget().deleteLater() + self.problem_sub.setText("Your current game is unchanged and still plays." if self.updating else + "Nothing was changed in your game folder.") + if tools: + self.problem_title.setText("The build tools need repairing") + self.problem_head.setText("Your antivirus removed part of the build tools") + self.problem_text.setText(why + "\n\nAntivirus programs sometimes flag unsigned developer tools by mistake.") + for n, s in ((1, "Open Windows Security → Protection history, and restore the removed files."), + (2, "Or add %s to its exclusions." % paths.base()), + (3, "Then press Repair: Hookshot downloads the tools again from their official releases and checks each one.")): + l = note("%d. %s" % (n, s), 14) + l.setContentsMargins(0, 6, 0, 0) + self.problem_steps.addWidget(l) + else: + self.problem_title.setText("The update stopped" if self.updating else "The build stopped") + self.problem_head.setText("What went wrong") + self.problem_text.setText(why) + self.repair_button.setText("Repair") + self.repair_button.setVisible(tools) + self.problem_details.setText("Show details") + if not self.updating: + self.set_step(1) + self.show_page(self.problem_page) + QTimer.singleShot(0, self.fit_problem) + + def fit_problem(self): + w = self.problem_scroll.widget() + w.adjustSize() + h = w.layout().sizeHint().height() if w.layout() else w.sizeHint().height() + h = w.layout().heightForWidth(self.problem_scroll.viewport().width()) if w.layout() and w.layout().hasHeightForWidth() else h + self.problem_scroll.setFixedHeight(max(20, min(190, h + 2))) + + def problem_show_details(self): + while self.problem_steps.count(): + it = self.problem_steps.takeAt(0) + if it.widget(): + it.widget().deleteLater() + box = LogBox(border=False) + box.setPlainText("\n".join(self.log_lines[-80:])) + box.setMinimumHeight(160) + self.problem_steps.addWidget(box) + self.problem_scroll.setFixedHeight(190) + + def copy_for_report(self): + tools = core.Tools.find() + import platform + text = "Hookshot Builder %s on Windows %s\n%s: %s\n\n%s" % (tools.version if tools and tools.version else "(source)", + platform.version(), self.problem_title.text(), self.last_failure, + "\n".join(self.log_lines[-200:])) + QGuiApplication.clipboard().setText(text) + self.copy_report.setText("Copied") + + def back(self): + self.set_step(0) + self.show_page(self.files_page) + + def repair(self): + """The build tools checked against their checksums and downloaded again, after this window closes""" + updater.restart(["--repair"]) + QApplication.quit() + + # ================= the update (updater.py) ================= + + def begin_update(self, folder, tag): + self.updating = True + self.out_path.setText(folder) + self.finished = "" + to = tag or "the latest version" + tools = core.Tools.find() + frm = core.game_version(folder) or (tools.version if tools else None) + self.step_pill.hide() + self.version_chip.setText("%s → %s" % (frm, to) if frm and frm != to else to) + self.version_chip.adjustSize() + self.version_chip.show() + self.prepare_now = not core.load_settings().get("PrepareLater") # (the saved choice: an update never shows the tick) + self.make_rows(True, self.prepare_now) + self.build_title.setText("Updating your game") + self.build_note.setText("Getting the update…") + self.ring_box.hide() + self.update_bar.show() + self.set_progress(0) + self.rows["download"].begin() + self.cancel_button.show() + self.show_page(self.build_page) + + def download_progress(self, f): + if "download" in self.rows: + self.rows["download"].small_text("%d%%" % int(f * 100)) + self.set_progress(self.weight["download"][1] * f) + + def note(self, line): + self.add_log(line) + if "download" in self.rows: + self.rows["download"].small_text(line) + + def download_failed(self, why): + self.download = None + if "download" in self.rows: + self.rows["download"].stopped() + self.show_problem(why, tools=False) + self.problem_title.setText("The update could not be downloaded") + + def update_build(self, tools, folder): + """The new tools are here: the folder built again with them (only what changed is compiled)""" + self.download = None + if "download" in self.rows: + self.rows["download"].done() + self.rows["download"].small_text("Build tools checked") + dlss = os.path.isfile(os.path.join(folder, "ngx", "nvngx_dlss.dll")) and os.path.isfile(os.path.join(core.ngx_sdk(), "LICENSE.txt")) + self.run(tools, ["--update", folder] + ([] if self.prepare_now else [SHADERS_LATER]), keep_rows=True, dlss=dlss) + + def quit_app(self): + QApplication.quit() + + # ================= the window ================= + + def close_window(self): + if self.runner and QMessageBox.question(self, "Hookshot", self.T("CloseQuestion"), + QMessageBox.StandardButton.Yes | QMessageBox.StandardButton.No) != QMessageBox.StandardButton.Yes: + return + if self.runner: + self.runner.cancel() + self.close() + + +class ProblemCard(QFrame): + """The problem's box: radius 10, #55E2776B border, #14E2776B fading to clear by 70%""" + + def paintEvent(self, e): + p = QPainter(self) + p.setRenderHint(QPainter.RenderHint.Antialiasing) + r = QRectF(self.rect()).adjusted(0.5, 0.5, -0.5, -0.5) + lg = QLinearGradient(r.topLeft(), r.bottomLeft()) + lg.setColorAt(0, argb("#14E2776B")) + lg.setColorAt(0.7, argb("#00E2776B")) + lg.setColorAt(1, argb("#00E2776B")) + p.fillPath(rounded(r, 10), lg) + p.setPen(QPen(argb("#55E2776B"), 1)) + p.drawPath(rounded(r, 10)) + + +# ================= Glass.cs: Windows 11's acrylic, dark title colours and round corners (DWM) ================= + +def light_backdrop(): + try: + with winreg.OpenKey(winreg.HKEY_CURRENT_USER, r"Software\Microsoft\Windows\CurrentVersion\Themes\Personalize") as k: + for name in ("SystemUsesLightTheme", "AppsUseLightTheme"): + try: + return winreg.QueryValueEx(k, name)[0] != 0 + except OSError: + continue + except OSError: + pass + return False + + +class MARGINS(ctypes.Structure): + _fields_ = [("l", ctypes.c_int), ("r", ctypes.c_int), ("t", ctypes.c_int), ("b", ctypes.c_int)] + + +def apply_glass(hwnd): + """true: acrylic is on (the window's background stays see-through)""" + import ctypes.wintypes + dwm, user = ctypes.windll.dwmapi, ctypes.windll.user32 + GWL_STYLE = -16 + style = user.GetWindowLongPtrW(ctypes.c_void_p(hwnd), GWL_STYLE) + # (a framed window: Windows draws the backdrop only behind one; without the system menu it draws no caption buttons of its own) + style = (style | 0x00C00000 | 0x00040000) & ~0x00080000 # (WS_CAPTION | WS_THICKFRAME, no WS_SYSMENU) + style &= ~0x00040000 # (no resizing border: the window is always 960x600) + user.SetWindowLongPtrW(ctypes.c_void_p(hwnd), GWL_STYLE, ctypes.c_void_p(style)) + user.SetWindowPos(ctypes.c_void_p(hwnd), None, 0, 0, 0, 0, 0x0027) # (SWP_FRAMECHANGED | NOMOVE | NOSIZE | NOZORDER) + on = ctypes.c_int(1) + dwm.DwmSetWindowAttribute(ctypes.c_void_p(hwnd), 20, ctypes.byref(on), 4) # dark mode + corner = ctypes.c_int(2) + dwm.DwmSetWindowAttribute(ctypes.c_void_p(hwnd), 33, ctypes.byref(corner), 4) # round corners + if light_backdrop(): + return False + if sys.getwindowsversion().build < 22621: # (22H2: the backdrop attribute) + return False + hc = ctypes.wintypes.BOOL() + m = MARGINS(-1, -1, -1, -1) + dwm.DwmExtendFrameIntoClientArea(ctypes.c_void_p(hwnd), ctypes.byref(m)) + backdrop = ctypes.c_int(3) # acrylic + return dwm.DwmSetWindowAttribute(ctypes.c_void_p(hwnd), 38, ctypes.byref(backdrop), 4) == 0 diff --git a/builder/py/hookshot_builder_start.py b/builder/py/hookshot_builder_start.py new file mode 100644 index 0000000..aff6a73 --- /dev/null +++ b/builder/py/hookshot_builder_start.py @@ -0,0 +1,103 @@ +"""Starts Hookshot Builder: python.exe -I hookshot_builder_start.py [--update --tag [--fetched]] [--launch +] [--repair]. Qt for Python is the "qt" folder beside this file's (the package), or HOOKSHOT_QT (development).""" +import argparse +import ctypes +import os +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) + + +def settle(): + """The package (python, qt, app) copied once into %LOCALAPPDATA%\\HookshotBuilder\\app and started from there: the shortcuts, the + game folder's launcher and the game's update offer point at that copy, which stays where it is whatever happens to the folder + the download was unpacked into. True: the copy was started, this one ends""" + import shutil + import subprocess + root = os.path.dirname(HERE) + version = os.path.join(HERE, "VERSION") + if not os.path.isfile(version) or not os.path.isdir(os.path.join(root, "python")) or os.environ.get("HOOKSHOT_NO_SETTLE"): + return False # (a source checkout: run where it is) + base = os.environ.get("HOOKSHOT_SETUP_BASE") or os.path.join(os.environ.get("LOCALAPPDATA") or os.path.expanduser("~"), + "HookshotBuilder") + home = os.path.join(base, "app") + if os.path.normcase(os.path.abspath(root)) == os.path.normcase(os.path.abspath(home)): + return False + want = open(version, encoding="utf-8").read().strip() + try: + have = open(os.path.join(home, "app", "VERSION"), encoding="utf-8").read().strip() + except OSError: + have = "" + if have != want: + part = home + ".part" + shutil.rmtree(part, ignore_errors=True) + try: + shutil.copytree(root, part, ignore=shutil.ignore_patterns("__pycache__")) + shutil.rmtree(home, ignore_errors=True) + os.replace(part, home) + except OSError: + shutil.rmtree(part, ignore_errors=True) + return False # (could not copy: run from here) + exe = os.path.join(home, "python", "pythonw.exe") + if not os.path.isfile(exe): + exe = os.path.join(home, "python", "python.exe") + subprocess.Popen([exe, "-I", os.path.join(home, "app", "hookshot_builder_start.py"), *sys.argv[1:]], cwd=home) + return True + + +def main(): + p = argparse.ArgumentParser(add_help=False) + p.add_argument("--update") + p.add_argument("--tag") + p.add_argument("--fetched", action="store_true") + p.add_argument("--launch") + p.add_argument("--repair", action="store_true") + args, _ = p.parse_known_args() + if settle(): + return + # (the app's code: the newest of this folder's and the installed release's, so a release's fixes reach the window without a new + # download of the package; the package's own when the release has none) + sys.path.insert(0, HERE) + try: + from hookshot_builder import setup_parts + cur = setup_parts.current() + newer = os.path.join(cur, "HookshotBuilder", "tools", "builder_py") if cur else None + if newer and os.path.isfile(os.path.join(newer, "hookshot_builder", "window.py")) and not os.environ.get("HOOKSHOT_OWN_APP"): + for m in [m for m in sys.modules if m == "hookshot_builder" or m.startswith("hookshot_builder.")]: + del sys.modules[m] + sys.path.insert(0, newer) + except Exception: + pass + qt = os.environ.get("HOOKSHOT_QT") or os.path.join(os.path.dirname(HERE), "qt") + if os.path.isdir(qt): + sys.path.insert(0, qt) + os.add_dll_directory(qt) + # (the taskbar groups and shows the window by this id and its own icon, not as python.exe) + from hookshot_builder import APP_ID + try: + ctypes.windll.shell32.SetCurrentProcessExplicitAppUserModelID(APP_ID) + except (AttributeError, OSError): + pass + try: # (a console the start left, when started by python.exe rather than pythonw.exe: not shown) + if ctypes.windll.kernel32.GetConsoleWindow() and not os.environ.get("HOOKSHOT_CONSOLE"): + ctypes.windll.kernel32.FreeConsole() + except OSError: + pass + from PySide6.QtCore import Qt + from PySide6.QtGui import QGuiApplication, QIcon + from PySide6.QtWidgets import QApplication + QGuiApplication.setHighDpiScaleFactorRoundingPolicy(Qt.HighDpiScaleFactorRoundingPolicy.PassThrough) + app = QApplication(sys.argv[:1]) + app.setApplicationName("Hookshot") + app.setApplicationDisplayName("Hookshot") + from hookshot_builder import paths, theme + from hookshot_builder.window import Window + app.setWindowIcon(QIcon(os.path.join(paths.ASSETS, "hookshot.ico"))) + theme.load_fonts() + w = Window(args) + w.show() + sys.exit(app.exec()) + + +if __name__ == "__main__": + main() diff --git a/builder/py/shots.py b/builder/py/shots.py new file mode 100644 index 0000000..3fcf709 --- /dev/null +++ b/builder/py/shots.py @@ -0,0 +1,58 @@ +"""Renders the window's pages to PNGs without showing it (QT_QPA_PLATFORM=offscreen): python -I shots.py . For comparing +the layout with the WPF builder's screenshots.""" +import os +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, HERE) +qt = os.environ.get("HOOKSHOT_QT") +if qt: + sys.path.insert(0, qt) + os.add_dll_directory(os.path.join(qt, "PySide6")) +os.environ.setdefault("QT_QPA_PLATFORM", "offscreen") +out = sys.argv[1] +os.makedirs(out, exist_ok=True) + +from PySide6.QtCore import QTimer # noqa: E402 +from PySide6.QtWidgets import QApplication # noqa: E402 + +app = QApplication(sys.argv[:1]) +from hookshot_builder import theme, window # noqa: E402 + +theme.load_fonts() + + +class Args: + update = tag = launch = None + fetched = repair = False + + +pages = ["files", "build", "ready", "problem", "update", "launch", "splash"] +state = {"i": 0, "w": None} + + +def next_page(): + if state["w"] is not None: + state["w"].grab().save(os.path.join(out, pages[state["i"] - 1] + ".png")) + state["w"].close() + if state["i"] >= len(pages): + app.quit() + return + name = pages[state["i"]] + state["i"] += 1 + if name not in ("files", "splash"): + os.environ["HOOKSHOT_UI_PREVIEW"] = name + else: + os.environ.pop("HOOKSHOT_UI_PREVIEW", None) + w = window.Window(Args()) + if name == "splash": + w.start = lambda: None + w.splash.text = "Looking for your game files…" + w.show() + state["w"] = w + QTimer.singleShot(2600 if name != "splash" else 600, next_page) + + +QTimer.singleShot(0, next_page) +app.exec() +print("wrote", out) diff --git a/builder/setup/.gitignore b/builder/setup/.gitignore new file mode 100644 index 0000000..774f008 --- /dev/null +++ b/builder/setup/.gitignore @@ -0,0 +1,2 @@ +*.exe +*.o diff --git a/builder/setup/hookshot.ico b/builder/setup/hookshot.ico new file mode 100644 index 0000000..6c7aca6 Binary files /dev/null and b/builder/setup/hookshot.ico differ diff --git a/builder/setup/setup.c b/builder/setup/setup.c new file mode 100644 index 0000000..721934a --- /dev/null +++ b/builder/setup/setup.c @@ -0,0 +1,881 @@ +/* HookshotBuilder-Setup.exe: the one file players download. It reads the latest release's hookshot-setup.txt, downloads + the parts it names (the builder and its tools from this repository's release, llvm-mingw from its own official + release), checks each one's SHA-256, unpacks them with Windows' own tar.exe into %LOCALAPPDATA%\HookshotBuilder\ + and starts the builder there. Parts already downloaded and checked are used again; a release already installed just + starts. Files fetched by a program carry no "downloaded from the internet" mark, and the download itself holds no + scripts, which is what antivirus programs' machine-learning guesses (Trojan:Script/...!ml) fired on. + + hookshot-setup.txt, one item a line: + tag + part + run + Build: clang -O2 -mwindows -municode setup.c setup_rc.o -lwinhttp -lbcrypt -lcomctl32 -lshell32 -lole32 -o HookshotBuilder-Setup.exe */ +#define WIN32_LEAN_AND_MEAN +#ifndef UNICODE +#define UNICODE +#endif +#define _UNICODE +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define WM_STATUS (WM_APP + 1) +#define WM_DONE (WM_APP + 2) +#define MAX_PARTS 16 + +static HWND g_win, g_text, g_bar; +static wchar_t g_status[512]; +static volatile LONG g_percent = -1; +static DWORD g_http_status; /* (the last answer: 0 when the server was not reached) */ +static wchar_t g_cmdline[2048]; +/* (RecompLauncher) --cli : no window, the builder's lines on this process's stdout and stderr, its exit code + as ours; --play : the game itself, waited for. Neither is used by the builder's own start, which is unchanged. */ +static int g_headless; +static DWORD g_exit; + +static void say_error(const wchar_t *msg) { + char u[2048]; + const int n = WideCharToMultiByte(CP_UTF8, 0, msg, -1, u, (int)sizeof u - 8, NULL, NULL); + const HANDLE h = GetStdHandle(STD_ERROR_HANDLE); + DWORD wrote; + if (n > 0 && h && h != INVALID_HANDLE_VALUE) WriteFile(h, "error ", 6, &wrote, NULL), WriteFile(h, u, n - 1, &wrote, NULL), WriteFile(h, "\n", 1, &wrote, NULL); +} + +typedef struct { + char sha[65]; + unsigned long long size; + char dest[260], from[260], to[260], url[1024]; +} Part; +/* the next non-empty line of a text, cut in place (its \r or \n made 0); *next moves past it. NULL at the end + (strtok's work without its hidden state) */ +static char *next_line(char **next) { + char *p = *next; + while (*p == '\r' || *p == '\n') p++; + if (!*p) return *next = p, (char *)NULL; + char *line = p; + while (*p && *p != '\r' && *p != '\n') p++; + if (*p) *p++ = 0; + *next = p; + return line; +} +/* a name from the setup list (a tag, or with path set a relative path with / or \ between its parts): letters, digits + and . _ + - only, no ".." part, no drive, no root */ +static int list_name_ok(const char *s, int path) { + if (!*s || *s == '/' || *s == '\\') return 0; + for (const char *q = s; *q; q++) + if (!((*q >= 'a' && *q <= 'z') || (*q >= 'A' && *q <= 'Z') || (*q >= '0' && *q <= '9') || strchr("._+-", *q) || + (path && (*q == '/' || *q == '\\')))) + return 0; + for (const char *q = s; *q;) { /* (each part: not "." or "..") */ + const size_t n = path ? strcspn(q, "/\\") : strlen(q); + if ((n == 1 && q[0] == '.') || (n == 2 && q[0] == '.' && q[1] == '.') || !n) return 0; + q += n; + if (*q) q++; + } + return 1; +} + +static void status(const wchar_t *fmt, ...) { + va_list ap; + va_start(ap, fmt); + vswprintf(g_status, 512, fmt, ap); + va_end(ap); + PostMessageW(g_win, WM_STATUS, 0, 0); +} + +static void fail(const wchar_t *fmt, ...) { + wchar_t msg[1024]; + va_list ap; + va_start(ap, fmt); + vswprintf(msg, 1024, fmt, ap); + va_end(ap); + if (g_headless) say_error(msg), ExitProcess(1); + MessageBoxW(g_win, msg, L"Hookshot Builder setup", MB_OK | MB_ICONERROR); + PostMessageW(g_win, WM_DONE, 1, 0); + ExitThread(1); +} + +/* ---- HTTPS GET into a sink, following redirects (GitHub's release links go to its download host) ---- */ +typedef int (*Sink)(void *user, const void *data, DWORD n); + +static int http_get(const char *url8, Sink sink, void *user, unsigned long long expect, const wchar_t *accept) { + wchar_t url[1024]; + MultiByteToWideChar(CP_UTF8, 0, url8, -1, url, 1024); + URL_COMPONENTS uc = {sizeof uc}; + wchar_t host[256], path[1024]; + uc.lpszHostName = host, uc.dwHostNameLength = 256; + uc.lpszUrlPath = path, uc.dwUrlPathLength = 1024; + if (!WinHttpCrackUrl(url, 0, 0, &uc)) return 0; + HINTERNET s = WinHttpOpen(L"HookshotBuilder-Setup/1.0", WINHTTP_ACCESS_TYPE_AUTOMATIC_PROXY, NULL, NULL, 0); + if (!s) s = WinHttpOpen(L"HookshotBuilder-Setup/1.0", WINHTTP_ACCESS_TYPE_DEFAULT_PROXY, NULL, NULL, 0); + if (!s) return 0; + /* (a stalled download fails after a minute without data, with the message, instead of waiting forever: a GitHub 504 hour) */ + WinHttpSetTimeouts(s, 30000, 30000, 30000, 60000); + DWORD policy = WINHTTP_OPTION_REDIRECT_POLICY_DISALLOW_HTTPS_TO_HTTP; /* (a redirect never drops to clear text) */ + WinHttpSetOption(s, WINHTTP_OPTION_REDIRECT_POLICY, &policy, sizeof policy); + HINTERNET c = WinHttpConnect(s, host, uc.nPort, 0); + LPCWSTR types[2] = {accept, NULL}; + HINTERNET r = c ? WinHttpOpenRequest(c, L"GET", path, NULL, WINHTTP_NO_REFERER, accept ? types : WINHTTP_DEFAULT_ACCEPT_TYPES, + uc.nScheme == INTERNET_SCHEME_HTTPS ? WINHTTP_FLAG_SECURE : 0) + : NULL; + int ok = 0; + if (r && WinHttpSendRequest(r, WINHTTP_NO_ADDITIONAL_HEADERS, 0, NULL, 0, 0, 0) && WinHttpReceiveResponse(r, NULL)) { + DWORD code = 0, len = sizeof code; + WinHttpQueryHeaders(r, WINHTTP_QUERY_STATUS_CODE | WINHTTP_QUERY_FLAG_NUMBER, NULL, &code, &len, NULL); + g_http_status = code; + if (code == 200) { + static char buf[1 << 16]; + unsigned long long got = 0; + ok = 1; + for (;;) { + DWORD n = 0; + if (!WinHttpReadData(r, buf, sizeof buf, &n)) { ok = 0; break; } + if (!n) break; + if (!sink(user, buf, n)) { ok = 0; break; } + got += n; + if (expect) InterlockedExchange(&g_percent, (LONG)(got * 100 / expect)), PostMessageW(g_win, WM_STATUS, 0, 0); + } + if (expect && got != expect) ok = 0; + } + } + if (r) WinHttpCloseHandle(r); + if (c) WinHttpCloseHandle(c); + WinHttpCloseHandle(s); + return ok; +} + +typedef struct { char *p; size_t n, cap; } Mem; +static int mem_sink(void *u, const void *d, DWORD n) { + Mem *m = (Mem *)u; + if (m->n + n + 1 > m->cap) { + m->cap = (m->n + n + 1) * 2; + m->p = (char *)realloc(m->p, m->cap); + if (!m->p) return 0; + } + memcpy(m->p + m->n, d, n); + m->n += n; + m->p[m->n] = 0; + return 1; +} + +typedef struct { HANDLE f; BCRYPT_HASH_HANDLE h; } FileSink; +static int file_sink(void *u, const void *d, DWORD n) { + FileSink *s = (FileSink *)u; + DWORD w = 0; + if (!WriteFile(s->f, d, n, &w, NULL) || w != n) return 0; + return BCryptHashData(s->h, (PUCHAR)d, n, 0) == 0; +} + +static void hex(const unsigned char *d, int n, char *out) { + for (int i = 0; i < n; i++) sprintf(out + 2 * i, "%02x", d[i]); +} + +/* the SHA-256 of a file already on disk (a part downloaded before) */ +static int file_sha256(const wchar_t *path, char out[65]) { + BCRYPT_ALG_HANDLE a; + BCRYPT_HASH_HANDLE h; + if (BCryptOpenAlgorithmProvider(&a, BCRYPT_SHA256_ALGORITHM, NULL, 0)) return 0; + BCryptCreateHash(a, &h, NULL, 0, NULL, 0, 0); + HANDLE f = CreateFileW(path, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_FLAG_SEQUENTIAL_SCAN, NULL); + int ok = f != INVALID_HANDLE_VALUE; + static unsigned char buf[1 << 20]; + DWORD n; + while (ok && ReadFile(f, buf, sizeof buf, &n, NULL) && n) BCryptHashData(h, buf, n, 0); + if (f != INVALID_HANDLE_VALUE) CloseHandle(f); + unsigned char d[32]; + BCryptFinishHash(h, d, 32, 0); + BCryptDestroyHash(h); + BCryptCloseAlgorithmProvider(a, 0); + if (ok) hex(d, 32, out); + return ok; +} + +typedef struct { + DWORD code, start_error, status_error, capture_error; + char error[512]; +} RunResult; + +static RunResult run_wait(wchar_t *cmd, const wchar_t *dir) { + RunResult result = {0}; + HANDLE rd = NULL, wr = NULL, null = INVALID_HANDLE_VALUE; + SECURITY_ATTRIBUTES sa = {sizeof sa, NULL, TRUE}; + STARTUPINFOW si = {sizeof si}; + PROCESS_INFORMATION pi; + si.dwFlags = STARTF_USESHOWWINDOW, si.wShowWindow = SW_HIDE; + if (CreatePipe(&rd, &wr, &sa, 0)) { + if (SetHandleInformation(rd, HANDLE_FLAG_INHERIT, 0)) + null = CreateFileW(L"NUL", GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_WRITE, &sa, OPEN_EXISTING, 0, NULL); + if (null != INVALID_HANDLE_VALUE) { + si.dwFlags |= STARTF_USESTDHANDLES; + si.hStdInput = si.hStdOutput = null; /* (setup has no console: supply valid inherited handles) */ + si.hStdError = wr; + } else { + result.capture_error = GetLastError(); + CloseHandle(rd), CloseHandle(wr); + rd = wr = NULL; + } + } else result.capture_error = GetLastError(); + /* Capturing a diagnostic is optional: if pipe setup fails, keep the original child-process route. */ + if (!CreateProcessW(NULL, cmd, NULL, NULL, wr != NULL, CREATE_NO_WINDOW, NULL, dir, &si, &pi)) { + result.start_error = GetLastError(); + if (rd) CloseHandle(rd); + if (wr) CloseHandle(wr); + if (null != INVALID_HANDLE_VALUE) CloseHandle(null); + return result; + } + if (wr) CloseHandle(wr); /* (otherwise EOF never arrives on the parent's reader) */ + if (null != INVALID_HANDLE_VALUE) CloseHandle(null); + if (rd) { + char buf[512]; + DWORD n; + size_t kept = 0; + while (ReadFile(rd, buf, sizeof buf, &n, NULL) && n) { + size_t copy = n < sizeof result.error - 1 - kept ? n : sizeof result.error - 1 - kept; + memcpy(result.error + kept, buf, copy); + kept += copy; /* (keep draining after this buffer fills, so a noisy child cannot block) */ + } + CloseHandle(rd); + } + WaitForSingleObject(pi.hProcess, INFINITE); + if (!GetExitCodeProcess(pi.hProcess, &result.code)) result.status_error = GetLastError(); + CloseHandle(pi.hThread), CloseHandle(pi.hProcess); + return result; +} + +static void mkdirs(const wchar_t *path) { + wchar_t p[MAX_PATH]; + wcsncpy(p, path, MAX_PATH - 1); + p[MAX_PATH - 1] = 0; + for (wchar_t *q = p + 3; *q; q++) + if (*q == L'\\') *q = 0, CreateDirectoryW(p, NULL), *q = L'\\'; + CreateDirectoryW(p, NULL); +} + +/* a folder and everything in it (a version folder, a half-unpacked one) */ +static void rmtree(const wchar_t *dir) { + wchar_t pat[MAX_PATH], p[MAX_PATH]; + WIN32_FIND_DATAW fd; + swprintf(pat, MAX_PATH, L"%ls\\*", dir); + HANDLE h = FindFirstFileW(pat, &fd); + if (h != INVALID_HANDLE_VALUE) { + do { + if (!wcscmp(fd.cFileName, L".") || !wcscmp(fd.cFileName, L"..")) continue; + swprintf(p, MAX_PATH, L"%ls\\%ls", dir, fd.cFileName); + if ((fd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) && !(fd.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT)) rmtree(p); + else if (fd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) RemoveDirectoryW(p); /* (a link: never what it points to) */ + else SetFileAttributesW(p, FILE_ATTRIBUTE_NORMAL), DeleteFileW(p); + } while (FindNextFileW(h, &fd)); + FindClose(h); + } + RemoveDirectoryW(dir); +} + +static void read_small(const wchar_t *path, char *out, int n) { + out[0] = 0; + HANDLE f = CreateFileW(path, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL); + if (f == INVALID_HANDLE_VALUE) return; + DWORD got = 0; + ReadFile(f, out, (DWORD)n - 1, &got, NULL); + CloseHandle(f); + out[got] = 0; + for (char *q = out; *q; q++) + if (*q == '\r' || *q == '\n') { *q = 0; break; } +} + +static void write_small(const wchar_t *path, const char *text) { + HANDLE f = CreateFileW(path, GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, 0, NULL); + if (f == INVALID_HANDLE_VALUE) return; + DWORD n; + WriteFile(f, text, (DWORD)strlen(text), &n, NULL); + CloseHandle(f); +} + +/* A part that unpacks into a folder of its own (the tools the builder ships: Python, emsdk, the CPM packages, third_party, + wgpu-native, llvm-mingw): that folder, relative to a version's folder, in rel (0: the part has none, e.g. the builder's own code). + Named from its download (HookshotBuilder--.zip) or, for llvm-mingw, from its dest and rename. */ +static int part_folder(const Part *p, wchar_t *rel, int n) { + if (strcmp(p->to, "-") && strcmp(p->dest, ".")) { /* (llvm-mingw: dest HookshotBuilder/tools, renamed to llvm-mingw) */ + swprintf(rel, n, L"%hs\\%hs", p->dest, p->to); + } else { + const char *b = strrchr(p->url, '/'); + b = b ? b + 1 : p->url; + static const char *const own[] = {"python", "emsdk", "cpm", "third_party", "wgpu-native"}; + int found = 0; + for (int i = 0; i < 5 && !found; i++) { + char want[64]; + snprintf(want, sizeof want, "HookshotBuilder-%s-", own[i]); + if (!strncmp(b, want, strlen(want))) swprintf(rel, n, L"HookshotBuilder\\tools\\%hs", own[i]), found = 1; + } + if (!found) return 0; + } + for (wchar_t *q = rel; *q; q++) if (*q == L'/') *q = L'\\'; + return 1; +} + +/* (players asked for less in %LOCALAPPDATA%: the downloaded zips are not kept beside the unpacked version any more. So that an update + still downloads only what changed, a part whose checksum the installed version lists in its parts.txt (and whose folder it has) + is moved over from it instead of downloaded: that version is deleted after the install anyway) */ +static int reuse_part(const wchar_t *versions, const char *tag, const Part *p, const wchar_t *install) { + wchar_t rel[MAX_PATH], cur[MAX_PATH], list[MAX_PATH], from[MAX_PATH], to[MAX_PATH], parent[MAX_PATH]; + char was[128]; + if (!part_folder(p, rel, MAX_PATH)) return 0; + swprintf(cur, MAX_PATH, L"%ls\\current.txt", versions); + read_small(cur, was, sizeof was); + if (!was[0] || !strcmp(was, tag)) return 0; + swprintf(list, MAX_PATH, L"%ls\\%hs\\parts.txt", versions, was); + HANDLE f = CreateFileW(list, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL); + if (f == INVALID_HANDLE_VALUE) return 0; + char buf[8192]; + DWORD got = 0; + ReadFile(f, buf, sizeof buf - 1, &got, NULL); + CloseHandle(f); + buf[got] = 0; + if (!strstr(buf, p->sha)) return 0; + swprintf(from, MAX_PATH, L"%ls\\%hs\\%ls", versions, was, rel); + swprintf(to, MAX_PATH, L"%ls\\%ls", install, rel); + if (GetFileAttributesW(from) == INVALID_FILE_ATTRIBUTES) return 0; + wcscpy(parent, to); + wchar_t *slash = wcsrchr(parent, L'\\'); + if (slash) *slash = 0, mkdirs(parent); + return MoveFileW(from, to) != 0; +} + +/* The checksums of the parts this version holds in folders of their own (reuse_part reads them at the next update) */ +static void write_parts(const wchar_t *install, const Part *parts, int nparts) { + wchar_t list[MAX_PATH], rel[MAX_PATH]; + swprintf(list, MAX_PATH, L"%ls\\parts.txt", install); + HANDLE f = CreateFileW(list, GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, 0, NULL); + if (f == INVALID_HANDLE_VALUE) return; + for (int i = 0; i < nparts; i++) { + if (!part_folder(&parts[i], rel, MAX_PATH)) continue; + char line[128]; + const int n = snprintf(line, sizeof line, "%s\n", parts[i].sha); + DWORD put; + WriteFile(f, line, (DWORD)n, &put, NULL); + } + CloseHandle(f); +} + +/* (as NX2navi's launcher keeps them: the version just installed and the one before it stay, older versions go, and so do downloads + the list no longer names; a part unchanged since the last release keeps its checksum and stays cached) */ +static void prune(const wchar_t *versions, const wchar_t *downloads, const char *tag, const Part *parts, int nparts) { + wchar_t cur[MAX_PATH], prev[MAX_PATH], pat[MAX_PATH], p[MAX_PATH]; + char was[128], before[128]; + swprintf(cur, MAX_PATH, L"%ls\\current.txt", versions); + swprintf(prev, MAX_PATH, L"%ls\\previous.txt", versions); + read_small(cur, was, sizeof was); + if (strcmp(was, tag)) { + if (was[0]) write_small(prev, was); + write_small(cur, tag); + } + read_small(prev, before, sizeof before); + WIN32_FIND_DATAW fd; + swprintf(pat, MAX_PATH, L"%ls\\*", versions); + HANDLE h = FindFirstFileW(pat, &fd); + if (h != INVALID_HANDLE_VALUE) { + do { + if (!(fd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) || !wcscmp(fd.cFileName, L".") || !wcscmp(fd.cFileName, L"..")) continue; + char name[260]; + WideCharToMultiByte(CP_UTF8, 0, fd.cFileName, -1, name, sizeof name, NULL, NULL); + /* (issue 611: only the version installed stays; the one before was 1.3 GB nobody rolled back to. A folder still in use + (the builder that started this, closing; a build it left) cannot be renamed: it stays, and goes at a later run) */ + (void)before; + if (!strcmp(name, tag)) continue; + swprintf(p, MAX_PATH, L"%ls\\%ls", versions, fd.cFileName); + wchar_t gone[MAX_PATH]; + swprintf(gone, MAX_PATH, L"%ls.gone", p); + if (wcslen(fd.cFileName) > 5 && !wcscmp(fd.cFileName + wcslen(fd.cFileName) - 5, L".gone")) rmtree(p); + else if (MoveFileW(p, gone)) rmtree(gone); + } while (FindNextFileW(h, &fd)); + FindClose(h); + } + swprintf(pat, MAX_PATH, L"%ls\\*.zip", downloads); + h = FindFirstFileW(pat, &fd); + if (h != INVALID_HANDLE_VALUE) { + do { + char name[260]; + WideCharToMultiByte(CP_UTF8, 0, fd.cFileName, -1, name, sizeof name, NULL, NULL); + int keep = 0; + for (int i = 0; i < nparts; i++) + if (!strncmp(name, parts[i].sha, 64)) keep = 1; + if (!keep) swprintf(p, MAX_PATH, L"%ls\\%ls", downloads, fd.cFileName), DeleteFileW(p); + } while (FindNextFileW(h, &fd)); + FindClose(h); + } +} + +static void w(const char *s8, wchar_t *out, int n) { MultiByteToWideChar(CP_UTF8, 0, s8, -1, out, n); } + +/* (HOOKSHOT_SETUP_LIST=: another list, for testing a release before it is the latest) */ +/* ---- The GitHub API's link for a release file (after PR 546 by nicholasgower, issue 544). GitHub answered the release + page's cached link (releases/latest/download/...) with 504 for hours on 2026-11-08 while its API still answered: the API + gives the file's own asset link, which redirects to a fresh download. Used when the release link fails. ---- */ +static int json_space(char c) { return c == 32 || c == 9 || c == 10 || c == 13; } + +static int json_string_after(const char *p, const char *end, char *out, int n) { + while (p < end && json_space(*p)) p++; + if (p >= end || *p != ':') return 0; + for (p++; p < end && json_space(*p); p++) {} + if (p >= end || *p != '"') return 0; + int i = 0; + for (p++; p < end && *p != '"'; p++) { + if (*p == '\\' && p + 1 < end) p++; + if (i + 1 < n) out[i++] = *p; + } + if (p >= end) return 0; + out[i] = 0; + return 1; +} + +/* The API link of the asset called name in a release's JSON (in an asset object its own "url" comes before its "name") */ +static int asset_from_json(const char *json, size_t len, const char *name, char *out, int n) { + const char *end = json + len, *at = NULL; + char v[600]; + for (const char *p = json; (p = strstr(p, "\"name\"")) && p < end; p += 6) + if (json_string_after(p + 6, end, v, sizeof v) && !strcmp(v, name)) { at = p; break; } + if (!at) return 0; + const char *url = NULL; + for (const char *p = json; p + 5 <= at; p++) + if (!strncmp(p, "\"url\"", 5)) url = p; + return url && json_string_after(url + 5, end, out, n) && !strncmp(out, "https://api.github.com/", 23); +} + +/* repo "owner/name", tag NULL for the latest release */ +static int api_asset_url(const char *repo, const char *tag, const char *name, char *out, int n) { + char api[1024]; + if (tag) snprintf(api, sizeof api, "https://api.github.com/repos/%s/releases/tags/%s", repo, tag); + else snprintf(api, sizeof api, "https://api.github.com/repos/%s/releases/latest", repo); + Mem m = {0}; + const int found = http_get(api, mem_sink, &m, 0, L"application/vnd.github+json") && m.p && asset_from_json(m.p, m.n, name, out, n); + free(m.p); + return found; +} + +/* https://github.com///releases/download// or .../releases/latest/download/ -> its API link */ +static int api_link_for(const char *url, char *out, int n) { + static const char pre[] = "https://github.com/"; + char repo[256], tag[256], name[260]; + if (strncmp(url, pre, sizeof pre - 1)) return 0; + const char *p = url + sizeof pre - 1, *slash = strchr(p, '/'); + if (!slash || !(slash = strchr(slash + 1, '/'))) return 0; + if ((size_t)(slash - p) >= sizeof repo) return 0; + memcpy(repo, p, slash - p), repo[slash - p] = 0; + const char *t; + int latest = 0; + if (!strncmp(slash, "/releases/latest/download/", 26)) latest = 1, t = slash + 26; + else if (!strncmp(slash, "/releases/download/", 19)) { + t = slash + 19; + const char *e = strchr(t, '/'); + if (!e || (size_t)(e - t) >= sizeof tag) return 0; + memcpy(tag, t, e - t), tag[e - t] = 0; + t = e + 1; + } else return 0; + if (!*t || strlen(t) >= sizeof name || strpbrk(t, "/?#")) return 0; + strcpy(name, t); + return api_asset_url(repo, latest ? NULL : tag, name, out, n); +} + +/* an answer worth trying again: none (no connection), a server error, a download cut off half way (200), a timeout, or the API's + rate limit (403/429) */ +static int retryable(DWORD status) { + return !status || status >= 500 || status == 200 || status == 408 || status == 429 || status == 403; +} + +/* (the release the builder asked for: --tag in the arguments, "" when none) */ +static void wanted_tag(char *out, size_t n) { + out[0] = 0; + const wchar_t *t = wcsstr(g_cmdline, L"--tag "); + if (!t) return; + t += 6; + while (*t == L' ' || *t == L'"') t++; + size_t k = 0; + while (*t && *t != L' ' && *t != L'"' && k + 1 < n) { + const wchar_t c = *t++; + if (!((c >= L'0' && c <= L'9') || (c >= L'a' && c <= L'z') || (c >= L'A' && c <= L'Z') || c == L'.' || c == L'-' || c == L'_')) + { out[0] = 0; return; } + out[k++] = (char)c; + } + out[k] = 0; +} + +/* (an update to a named release reads THAT release's list: the latest/download link can answer with the release before it for a + while after a publish, and the setup then installed the old version again and started it with --fetched, which builds without + checking: players pressed Update and stayed on v2.0.3, 2026-11-08) */ +static const char *manifest_url(void) { + static char u[1024]; + const char *e = getenv("HOOKSHOT_SETUP_LIST"); + char tag[128]; + wanted_tag(tag, sizeof tag); + if (e && *e) snprintf(u, sizeof u, "%s", e); + else if (tag[0]) snprintf(u, sizeof u, "https://github.com/NotRexed/Hookshot/releases/download/%s/hookshot-setup.txt", tag); + else snprintf(u, sizeof u, "%s", "https://github.com/NotRexed/Hookshot/releases/latest/download/hookshot-setup.txt"); + return u; +} + +static DWORD WINAPI work(LPVOID unused) { + (void)unused; + /* (--repair, from the builder's Repair button: every part checked again and the version unpacked anew, after the builder that + started this has closed. It is not passed on) */ + int repair = 0; + wchar_t *flag = wcsstr(g_cmdline, L"--repair"); + if (flag) { + repair = 1; + wmemmove(flag, flag + 8, wcslen(flag + 8) + 1); + Sleep(2000); + } + wchar_t base[MAX_PATH], downloads[MAX_PATH]; + if (SHGetFolderPathW(NULL, CSIDL_LOCAL_APPDATA, NULL, 0, base) != S_OK) fail(L"No local application data folder."); + wcscat(base, L"\\HookshotBuilder"); + { /* (HOOKSHOT_SETUP_BASE=: install there instead, for testing installs and updates without touching the real one) */ + const wchar_t *tb = _wgetenv(L"HOOKSHOT_SETUP_BASE"); + if (tb && *tb) wcsncpy(base, tb, MAX_PATH - 1), base[MAX_PATH - 1] = 0; + } + swprintf(downloads, MAX_PATH, L"%ls\\downloads", base); + mkdirs(downloads); + + status(L"Looking for the latest Hookshot Builder..."); + Mem man = {0}; + /* (tried again on a server error or no answer: GitHub's release downloads answered 504 for an hour, 2026-11-08, and every + start failed at once) */ + /* (low-vram, issue 558: 5 tries in 41 s were not enough while GitHub's site and API both failed on and off for hours; 8 tries + over ~3 minutes now, alternating the release link and the API's) */ + static const int waits[] = {3, 6, 12, 20, 30, 45, 60}; + enum { TRIES = 8 }; + int ok = 0; + for (int attempt = 0; attempt < TRIES && !ok; attempt++) { + if (attempt) { + status(L"GitHub didn't answer (%lu). Trying again in %d s...", g_http_status, waits[attempt - 1]); + Sleep(waits[attempt - 1] * 1000); + status(L"Looking for the latest Hookshot Builder..."); + } + free(man.p); + man = (Mem){0}; + g_http_status = 0; + ok = http_get(manifest_url(), mem_sink, &man, 0, NULL) && man.p; + const DWORD first = g_http_status; + char api[1024]; + if (!ok && (!first || first >= 500) && api_link_for(manifest_url(), api, sizeof api)) { /* (the API's link instead) */ + free(man.p); + man = (Mem){0}; + ok = http_get(api, mem_sink, &man, 0, L"application/octet-stream") && man.p; + if (!ok) g_http_status = first; + } + if (!ok && !retryable(g_http_status)) break; /* (404 and the like: trying again changes nothing) */ + } + if (!ok) + fail(g_http_status == 404 + ? L"The latest Hookshot release has no setup files yet. Download HookshotBuilder.zip from the releases page for now: " + L"https://github.com/NotRexed/Hookshot/releases/latest" + : g_http_status ? L"GitHub's download servers are failing right now (an error 500 or above, tried 8 times over 3 minutes). " + L"This is on GitHub's side: try again in a few minutes, or later today." + : L"Could not reach GitHub to get the Hookshot Builder. Check your internet connection and try again."); + + char tag[128] = "", run[260] = ""; + Part parts[MAX_PARTS]; + int nparts = 0; + for (char *next = man.p, *line; (line = next_line(&next)) != NULL;) { + if (!strncmp(line, "tag ", 4)) sscanf(line + 4, "%127s", tag); + else if (!strncmp(line, "run ", 4)) sscanf(line + 4, "%259s", run); + else if (!strncmp(line, "part ", 5) && nparts < MAX_PARTS) { + Part *p = &parts[nparts]; + if (sscanf(line + 5, "%64s %llu %259s %259s %259s %1023s", p->sha, &p->size, p->dest, p->from, p->to, p->url) == 6) nparts++; + } + } + if (!tag[0] || !run[0] || !nparts) fail(L"The release's setup list could not be read. Try again later."); + { /* (the list's names become paths under the install folder, a folder this removes and remakes: they stay inside it) */ + int bad = !list_name_ok(tag, 0) || !list_name_ok(run, 1); + for (int i = 0; i < nparts && !bad; i++) { + const Part *p = &parts[i]; + bad = strlen(p->sha) != 64 || strspn(p->sha, "0123456789abcdef") != 64 || _strnicmp(p->url, "https://", 8) || + (strcmp(p->dest, ".") && !list_name_ok(p->dest, 1)) || (strcmp(p->from, "-") && !list_name_ok(p->from, 0)) || + (strcmp(p->to, "-") && !list_name_ok(p->to, 0)); + } + if (bad) fail(L"The release's setup list has a name this setup does not accept. Try again later."); + } + { + char want[128]; + wanted_tag(want, sizeof want); + if (want[0] && strcmp(want, tag)) { + wchar_t ww[128], wt[128]; + MultiByteToWideChar(CP_UTF8, 0, want, -1, ww, 128), MultiByteToWideChar(CP_UTF8, 0, tag, -1, wt, 128); + fail(L"GitHub answered with the setup list of %ls, not %ls (its download links can lag right after a release). Try the " + L"update again in a few minutes.", wt, ww); + } + } + + wchar_t install[MAX_PATH], marker[MAX_PATH], wtag[128], wrun[260], exe[MAX_PATH], versions[MAX_PATH], final_dir[MAX_PATH]; + w(tag, wtag, 128), w(run, wrun, 260); + swprintf(versions, MAX_PATH, L"%ls\\versions", base); + swprintf(final_dir, MAX_PATH, L"%ls\\%ls", versions, wtag); + swprintf(install, MAX_PATH, L"%ls\\%ls", versions, wtag); + swprintf(marker, MAX_PATH, L"%ls\\installed.txt", install); + swprintf(exe, MAX_PATH, L"%ls\\%ls", install, wrun); + for (wchar_t *q = exe; *q; q++) if (*q == L'/') *q = L'\\'; + + if (repair || GetFileAttributesW(marker) == INVALID_FILE_ATTRIBUTES || GetFileAttributesW(exe) == INVALID_FILE_ATTRIBUTES) { + /* (unpacked into .part and renamed once complete: a setup stopped half way never leaves a version that looks installed) */ + swprintf(install, MAX_PATH, L"%ls\\%ls.part", versions, wtag); + rmtree(install); + rmtree(final_dir); + mkdirs(install); + for (int i = 0; i < nparts; i++) { + Part *p = &parts[i]; + wchar_t zip[MAX_PATH], wsha[65]; + w(p->sha, wsha, 65); + swprintf(zip, MAX_PATH, L"%ls\\%ls.zip", downloads, wsha); + if (reuse_part(versions, tag, p, install)) { /* (unchanged since the installed version: its folder moved over) */ + status(L"Part %d of %d unchanged: kept from the installed version", i + 1, nparts); + continue; + } + char have[65] = ""; + if (!(GetFileAttributesW(zip) != INVALID_FILE_ATTRIBUTES && file_sha256(zip, have) && !strcmp(have, p->sha))) { + wchar_t part[MAX_PATH]; + swprintf(part, MAX_PATH, L"%ls.part", zip); + int got = 0; + for (int attempt = 0; attempt < TRIES && !got; attempt++) { /* (a server error or a dropped connection: again) */ + if (attempt) { + status(L"Part %d of %d: GitHub didn't answer (%lu). Trying again in %d s...", i + 1, nparts, g_http_status, + waits[attempt - 1]); + Sleep(waits[attempt - 1] * 1000); + } + status(L"Downloading part %d of %d (%llu MB)...", i + 1, nparts, p->size >> 20); + FileSink fs; + BCRYPT_ALG_HANDLE a; + fs.f = CreateFileW(part, GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_FLAG_SEQUENTIAL_SCAN, NULL); + if (fs.f == INVALID_HANDLE_VALUE) fail(L"Could not write to %ls.", downloads); + BCryptOpenAlgorithmProvider(&a, BCRYPT_SHA256_ALGORITHM, NULL, 0); + BCryptCreateHash(a, &fs.h, NULL, 0, NULL, 0, 0); + g_http_status = 0; + /* (after a failed try the API's link to the same file, every other try) */ + char api[1024]; + if (attempt % 2 == 1 && api_link_for(p->url, api, sizeof api)) + got = http_get(api, file_sink, &fs, p->size, L"application/octet-stream"); + else + got = http_get(p->url, file_sink, &fs, p->size, NULL); + CloseHandle(fs.f); + unsigned char d[32]; + BCryptFinishHash(fs.h, d, 32, 0); + BCryptDestroyHash(fs.h); + BCryptCloseAlgorithmProvider(a, 0); + InterlockedExchange(&g_percent, -1); + hex(d, 32, have); + /* (a download cut off half way answered 200 and stopped the tries at once: only a definite no stops them) */ + if (!got && !retryable(g_http_status)) break; + } + if (!got) DeleteFileW(part), fail(retryable(g_http_status) + ? L"Downloading part %d of %d failed: GitHub's download servers are failing right now (tried 8 times over 3 " + L"minutes). This is on GitHub's side. The parts already downloaded are kept: run HookshotBuilder.exe again in a " + L"few minutes and it carries on from here." + : L"Downloading part %d of %d failed. Check your connection and free space, then try again.", i + 1, nparts); + if (strcmp(have, p->sha)) DeleteFileW(part), fail(L"Part %d of %d did not download correctly (its checksum differs). Try again.", i + 1, nparts); + /* (the checked file or nothing: an old .zip that could not be removed is never unpacked instead) */ + if (!MoveFileExW(part, zip, MOVEFILE_REPLACE_EXISTING)) { + const DWORD e = GetLastError(); + DeleteFileW(part); + fail(L"Could not place part %d of %d in %ls (Windows error %lu).", i + 1, nparts, downloads, e); + } + } + status(L"Unpacking part %d of %d...", i + 1, nparts); + wchar_t dest[MAX_PATH], wdest[260], tarcmd[2 * MAX_PATH + 64], sys[MAX_PATH]; + w(p->dest, wdest, 260); + if (!strcmp(p->dest, ".")) wcscpy(dest, install); + else swprintf(dest, MAX_PATH, L"%ls\\%ls", install, wdest); + for (wchar_t *q = dest; *q; q++) if (*q == L'/') *q = L'\\'; + mkdirs(dest); + GetSystemDirectoryW(sys, MAX_PATH); + swprintf(tarcmd, 2 * MAX_PATH + 64, L"\"%ls\\tar.exe\" -xf \"%ls\" -C \"%ls\"", sys, zip, dest); + RunResult tar = run_wait(tarcmd, dest); + if (tar.start_error) fail(L"Could not start Windows' tar.exe (Windows error %lu).\nArchive: %ls\nDestination: %ls", tar.start_error, zip, dest); + if (tar.status_error) fail(L"Could not read Windows' tar.exe exit status (Windows error %lu).\nArchive: %ls\nDestination: %ls", tar.status_error, zip, dest); + if (tar.code) { + wchar_t detail[512] = L""; + if (tar.capture_error) swprintf(detail, 512, L"Could not capture stderr (Windows error %lu).", tar.capture_error); + /* Best-effort UTF-8: tar may use an ANSI/OEM code page, or the byte cap may split a sequence. */ + else MultiByteToWideChar(CP_UTF8, 0, tar.error, -1, detail, 512); + wchar_t heading[1024] = L""; + swprintf(heading, 1024, L"Unpacking part %d of %d failed: Windows' tar.exe exited with code %lu.\nArchive: %ls\nDestination: %ls\n%ls:\n", + i + 1, nparts, tar.code, zip, dest, + tar.capture_error ? L"Stderr unavailable" : L"Tar stderr excerpt (up to 511 bytes captured)"); + heading[1023] = 0; + /* Keep both paths; the displayed excerpt uses only the space left in fail()'s 1024-wide message. */ + fail(L"%ls%.*ls", heading, (int)(1023 - wcslen(heading)), detail); + } + if (strcmp(p->from, "-") && strcmp(p->to, "-")) { /* (llvm-mingw--ucrt-x86_64 -> llvm-mingw) */ + wchar_t from[MAX_PATH], to[MAX_PATH], wf[260], wt[260]; + w(p->from, wf, 260), w(p->to, wt, 260); + swprintf(from, MAX_PATH, L"%ls\\%ls", dest, wf); + swprintf(to, MAX_PATH, L"%ls\\%ls", dest, wt); + if (GetFileAttributesW(to) == INVALID_FILE_ATTRIBUTES && !MoveFileW(from, to)) + fail(L"Could not set up part %d of %d (renaming %ls).", i + 1, nparts, wf); + } + } + wchar_t staged_exe[MAX_PATH]; + swprintf(staged_exe, MAX_PATH, L"%ls\\%ls", install, wrun); + for (wchar_t *q = staged_exe; *q; q++) if (*q == L'/') *q = L'\\'; + if (GetFileAttributesW(staged_exe) == INVALID_FILE_ATTRIBUTES) + fail(L"The builder is missing after unpacking (%ls). Your antivirus may have removed it: allow the folder %ls and run the setup again.", staged_exe, base); + if (!MoveFileW(install, final_dir)) fail(L"Could not finish installing into %ls.", final_dir); + wcscpy(install, final_dir); + write_small(marker, tag); + write_parts(install, parts, nparts); + } + { /* (a version an older setup installed has no parts.txt: written now, so the next update can move its folders over) */ + wchar_t have_list[MAX_PATH]; + swprintf(have_list, MAX_PATH, L"%ls\\parts.txt", final_dir); + if (GetFileAttributesW(have_list) == INVALID_FILE_ATTRIBUTES) write_parts(final_dir, parts, nparts); + } + prune(versions, downloads, tag, parts, nparts); + rmtree(downloads); /* (installed: its zips are not kept; an update moves unchanged folders over instead, reuse_part) */ + + status(L"Starting the Hookshot Builder..."); + wchar_t cmd[4096], dir[MAX_PATH]; + wcscpy(dir, exe); + wchar_t *slash = wcsrchr(dir, L'\\'); + if (slash) *slash = 0; + swprintf(cmd, 4096, L"\"%ls\" %ls", exe, g_cmdline); + STARTUPINFOW si = {sizeof si}; + PROCESS_INFORMATION pi; + BOOL inherit = FALSE; + if (g_headless) { /* (the builder writes to the same pipes we were given) */ + si.hStdInput = GetStdHandle(STD_INPUT_HANDLE), si.hStdOutput = GetStdHandle(STD_OUTPUT_HANDLE), si.hStdError = GetStdHandle(STD_ERROR_HANDLE); + if (si.hStdOutput && si.hStdOutput != INVALID_HANDLE_VALUE && si.hStdError && si.hStdError != INVALID_HANDLE_VALUE) { + if (!si.hStdInput || si.hStdInput == INVALID_HANDLE_VALUE) si.hStdInput = NULL; + SetHandleInformation(si.hStdOutput, HANDLE_FLAG_INHERIT, HANDLE_FLAG_INHERIT); + SetHandleInformation(si.hStdError, HANDLE_FLAG_INHERIT, HANDLE_FLAG_INHERIT); + si.dwFlags = STARTF_USESTDHANDLES, inherit = TRUE; + } + } + if (!CreateProcessW(NULL, cmd, NULL, NULL, inherit, 0, NULL, dir, &si, &pi)) fail(L"Could not start %ls.", exe); + if (g_headless) WaitForSingleObject(pi.hProcess, INFINITE), GetExitCodeProcess(pi.hProcess, &g_exit); + CloseHandle(pi.hThread), CloseHandle(pi.hProcess); + PostMessageW(g_win, WM_DONE, 0, 0); + return 0; +} + +static LRESULT CALLBACK wndproc(HWND h, UINT msg, WPARAM wp, LPARAM lp) { + switch (msg) { + case WM_STATUS: { + SetWindowTextW(g_text, g_status); + const LONG p = g_percent; + if (p < 0) SendMessageW(g_bar, PBM_SETMARQUEE, TRUE, 30); + else SendMessageW(g_bar, PBM_SETMARQUEE, FALSE, 0), SendMessageW(g_bar, PBM_SETPOS, (WPARAM)p, 0); + return 0; + } + case WM_DONE: DestroyWindow(h); return 0; + case WM_DESTROY: PostQuitMessage(0); return 0; + } + return DefWindowProcW(h, msg, wp, lp); +} + +/* (launch mode) "Hookshot Launcher.exe" in a game folder (nx2/make_game.py puts this file there under that name): the installed + builder opened as the game's launcher (--launch : Play, and Update when a newer release is out), at once, with no + window, download or network of its own. With no builder installed yet, the setup runs as usual and then opens the launcher. */ +static int launch_installed(const wchar_t *args) { + wchar_t base[MAX_PATH], cur[MAX_PATH], exe[MAX_PATH], dir[MAX_PATH], cmd[4096]; + if (SHGetFolderPathW(NULL, CSIDL_LOCAL_APPDATA, NULL, 0, base) != S_OK) return 0; + char tag[128] = ""; + swprintf(cur, MAX_PATH, L"%ls\\HookshotBuilder\\versions\\current.txt", base); + read_small(cur, tag, sizeof tag); + if (!tag[0]) return 0; + wchar_t wtag[128]; + MultiByteToWideChar(CP_UTF8, 0, tag, -1, wtag, 128); + swprintf(dir, MAX_PATH, L"%ls\\HookshotBuilder\\versions\\%ls\\HookshotBuilder", base, wtag); + swprintf(exe, MAX_PATH, L"%ls\\HookshotBuilder.exe", dir); + if (GetFileAttributesW(exe) == INVALID_FILE_ATTRIBUTES) return 0; + swprintf(cmd, 4096, L"\"%ls\" %ls", exe, args); + STARTUPINFOW si = {sizeof si}; + PROCESS_INFORMATION pi; + if (!CreateProcessW(NULL, cmd, NULL, NULL, FALSE, 0, NULL, dir, &si, &pi)) return 0; + CloseHandle(pi.hThread), CloseHandle(pi.hProcess); + return 1; +} + +/* (RecompLauncher's Play) --play : bin\hookshot.exe there, started like the game folder's launcher starts it but without + its update question, and waited for, so the launcher knows when the session ends. */ +static int play_folder(const wchar_t *args) { + wchar_t folder[MAX_PATH], cmd[MAX_PATH + 32]; + while (*args == L' ') args++; + const int quoted = *args == L'"'; + size_t k = 0; + for (args += quoted; *args && !(quoted ? *args == L'"' : *args == L' ') && k + 1 < MAX_PATH; ) folder[k++] = *args++; + folder[k] = 0; + swprintf(cmd, MAX_PATH + 32, L"\"%ls\\bin\\hookshot.exe\"", folder); + _wputenv_s(L"HOOKSHOT_NO_UPDATE_CHECK", L"1"); + STARTUPINFOW si = {sizeof si}; + PROCESS_INFORMATION pi; + DWORD code = 1; + if (!k || !CreateProcessW(NULL, cmd, NULL, NULL, FALSE, 0, NULL, folder, &si, &pi)) { + g_headless = 1; + say_error(L"No built game to start. Build it first (the game folder needs bin\\hookshot.exe)."); + return 1; + } + WaitForSingleObject(pi.hProcess, INFINITE), GetExitCodeProcess(pi.hProcess, &code); + CloseHandle(pi.hThread), CloseHandle(pi.hProcess); + return (int)code; +} + +int WINAPI wWinMain(HINSTANCE inst, HINSTANCE prev, LPWSTR cmdline, int show) { + (void)prev; + /* (libraries loaded from here on come from System32 only, never from the folder the setup was downloaded to) */ + SetDefaultDllDirectories(LOAD_LIBRARY_SEARCH_SYSTEM32); + wcsncpy(g_cmdline, cmdline ? cmdline : L"", 2047); + { + const wchar_t *c = g_cmdline; + while (*c == L' ') c++; + if (!wcsncmp(c, L"--play", 6) && (c[6] == L' ' || !c[6])) return play_folder(c + 6); + if (!wcsncmp(c, L"--cli", 5) && (c[5] == L' ' || !c[5])) g_headless = 1, show = SW_HIDE; + } + { + wchar_t self[MAX_PATH]; + GetModuleFileNameW(NULL, self, MAX_PATH); + const wchar_t *name = wcsrchr(self, L'\\'); + name = name ? name + 1 : self; + if (!_wcsicmp(name, L"hookshot.exe") && !wcsstr(g_cmdline, L"--")) { /* (the game folder's launcher) */ + wchar_t *slash = wcsrchr(self, L'\\'); + if (slash) *slash = 0; + swprintf(g_cmdline, 2047, L"--launch \"%ls\"", self); + if (launch_installed(g_cmdline)) return 0; + /* (no builder installed on this PC: the game itself, at once; nothing is downloaded to show a Play button) */ + wchar_t game[MAX_PATH], cmd[MAX_PATH + 4]; + swprintf(game, MAX_PATH, L"%ls\\bin\\hookshot.exe", self); + swprintf(cmd, MAX_PATH + 4, L"\"%ls\"", game); + STARTUPINFOW si = {sizeof si}; + PROCESS_INFORMATION pi; + if (CreateProcessW(NULL, cmd, NULL, NULL, FALSE, 0, NULL, self, &si, &pi)) { + CloseHandle(pi.hThread), CloseHandle(pi.hProcess); + return 0; + } + MessageBoxW(NULL, L"The game (bin\\hookshot.exe) is missing from this folder. Build it again with the Hookshot Builder.", L"Hookshot", + MB_ICONERROR); + return 1; + } + if (wcsstr(g_cmdline, L"--launch") && launch_installed(g_cmdline)) return 0; + } + INITCOMMONCONTROLSEX icc = {sizeof icc, ICC_PROGRESS_CLASS}; + InitCommonControlsEx(&icc); + WNDCLASSW wc = {0}; + wc.lpfnWndProc = wndproc, wc.hInstance = inst, wc.lpszClassName = L"HookshotSetup"; + wc.hIcon = LoadIconW(inst, MAKEINTRESOURCEW(1)); + wc.hCursor = LoadCursorW(NULL, (LPCWSTR)IDC_ARROW); + wc.hbrBackground = (HBRUSH)(COLOR_WINDOW + 1); + RegisterClassW(&wc); + const int ww = 460, wh = 140; + g_win = CreateWindowW(L"HookshotSetup", L"Hookshot Builder setup", WS_OVERLAPPED | WS_CAPTION | WS_SYSMENU | WS_MINIMIZEBOX, + (GetSystemMetrics(SM_CXSCREEN) - ww) / 2, (GetSystemMetrics(SM_CYSCREEN) - wh) / 2, ww, wh, NULL, NULL, inst, NULL); + g_text = CreateWindowW(L"STATIC", L"Starting...", WS_CHILD | WS_VISIBLE, 16, 16, ww - 48, 22, g_win, NULL, inst, NULL); + g_bar = CreateWindowW(PROGRESS_CLASSW, NULL, WS_CHILD | WS_VISIBLE | PBS_MARQUEE, 16, 48, ww - 48, 20, g_win, NULL, inst, NULL); + HFONT f = CreateFontW(-15, 0, 0, 0, FW_NORMAL, 0, 0, 0, DEFAULT_CHARSET, 0, 0, CLEARTYPE_QUALITY, 0, L"Segoe UI"); + SendMessageW(g_text, WM_SETFONT, (WPARAM)f, TRUE); + SendMessageW(g_bar, PBM_SETMARQUEE, TRUE, 30); + ShowWindow(g_win, show); + CloseHandle(CreateThread(NULL, 0, work, NULL, 0, NULL)); + MSG m; + while (GetMessageW(&m, NULL, 0, 0) > 0) TranslateMessage(&m), DispatchMessageW(&m); + return (int)g_exit; +} diff --git a/builder/setup/setup.manifest b/builder/setup/setup.manifest new file mode 100644 index 0000000..52b4a3b --- /dev/null +++ b/builder/setup/setup.manifest @@ -0,0 +1,7 @@ + + + + + true + + diff --git a/builder/setup/setup.rc b/builder/setup/setup.rc new file mode 100644 index 0000000..402bbf7 --- /dev/null +++ b/builder/setup/setup.rc @@ -0,0 +1,22 @@ +1 ICON "hookshot.ico" +1 24 "setup.manifest" +1 VERSIONINFO +FILEVERSION 1,0,0,0 +PRODUCTVERSION 1,0,0,0 +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "040904b0" + BEGIN + VALUE "CompanyName", "Hookshot" + VALUE "FileDescription", "Hookshot Builder setup" + VALUE "ProductName", "Hookshot Builder" + VALUE "FileVersion", "1.0.0" + VALUE "OriginalFilename", "HookshotBuilder-Setup.exe" + END + END + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x409, 1200 + END +END diff --git a/changelog.txt b/changelog.txt new file mode 100644 index 0000000..89a6d14 --- /dev/null +++ b/changelog.txt @@ -0,0 +1,21 @@ +--------------------------------------------------------------------------------------------------- +Version: 0.1.0 +Date: 07. 11. 2026 + Major Features: + - First release. + - The game's AArch64 code is lifted to C and compiled natively on the player's PC from their own copy; no emulator is involved. + - Horizon OS and NVN2 are implemented natively, and the GPU work is replayed on Vulkan. + - Hookshot Builder on Windows and the Linux AppImage build the game without Visual Studio. + Features: + - Kokiri Forest: reaching gameplay and walking around. + - Sound, controller, keyboard and saves. + - In-game menu with frame generation and upscaling. + - Movies, with a fallback when the PC has no AV1 decoder. + - Linux: the AppImage carries the compiler, Python and the Vulkan back end; the built game runs without it. + Changes: + - Versioning starts at 0.1.0. Releases follow Semantic Versioning and semantic-release conventions; the changelog uses Factorio's format. + Info: + - Tested on one Windows PC (Ryzen 5 7600X, RTX 4060 Ti) and one Linux PC, mostly around the start of the game. + - AMD and Intel graphics cards and the rest of the game are not tested yet. + - A build needs about 60 GB free, and a run uses about 10 GB of RAM. + - The repository contains no game files. The game is built from your own dump. diff --git a/docs/capture-dimensions.md b/docs/capture-dimensions.md new file mode 100644 index 0000000..2d2a0d6 --- /dev/null +++ b/docs/capture-dimensions.md @@ -0,0 +1,29 @@ +# Screenshot dimension validation + +`RX_VK_SHOT_W` and `RX_VK_SHOT_H` accept positive decimal integers. Missing variables +retain the1280x720 default. Empty, signed, whitespace-containing, partially numeric, +zero and overflowing values disable screenshots for that process and report the +invalid request once. They do not fall back to a different requested capture size. + +Before any screenshot allocation, the dimensions must fit the device's +`maxImageDimension2D`, the RGBA readback's CPU addressable size, and the existing +PNG encoder. That encoder writes one IDAT with stored deflate blocks: the checked +size includes its zlib header, five bytes per block and Adler32. The +[PNG chunk limit](https://www.w3.org/TR/png-3/#5Chunk-layout) is2^31-1 bytes. +This is a format/capability bound; it does not add a memory budget or guarantee +that a valid request can be allocated on a particular GPU. + +Rejected requests never enter the capture producer. Window presentation continues; +headless presentation keeps its existing offscreen policy and produces no capture +for an invalid size. The fixed-size binary frame logger is independent. Valid +standard, ultrawide, odd and small dimensions retain their requested sizes. + +`tests/test_capture_dimensions.py` compiles production setup, the real capture +entry/presentation conditions, and the PNG writer. A Vulkan allocation model checks +rejection before calls, reports once, and preserves window/headless routing. A +small real PNG validates chunk CRCs, decoded scanlines and stored-block sizing. +Large dimension/format boundaries use handles only, never large allocations. + +Resource readiness in #1472 remains separate. In a combined source tree its sliced +setup test must include the actual dimension guard and initialized physical-device +limits before exercising allocation failures; it must not stub the guard away. diff --git a/docs/capture-readback.md b/docs/capture-readback.md new file mode 100644 index 0000000..58b434d --- /dev/null +++ b/docs/capture-readback.md @@ -0,0 +1,35 @@ +# Capture copies and CPU visibility + +Normal screenshots, generated-frame screenshots and binary frame-log slots are +GPU-written mapped staging buffers. Their contents reach CPU consumers through +three distinct requirements: + +1. Record a copy-write to host-read dependency for the copied buffer range. +2. Wait for its owning GPU submission or presentation-slot fence. +3. Invalidate that allocation range before reading a non-coherent host cache. + +`capture_host_read_barrier` uses COPY/TRANSFER_WRITE to HOST/HOST_READ, offset0 +and the complete copied byte count, with no queue ownership transfer. Each +capture producer records it immediately after its copy. Existing timeline/fence +waits remain in place. `invalidate_mapped` delegates cache atom alignment and +coherent-memory handling to VMA. + +`shot_pixels` owns normal screenshot bytes after the timeline wait. A pending +screenshot already copied to CPU memory keeps using that snapshot if its parity +slot is reused; it must not invalidate or reread the reused GPU slot. Generated +screenshots invalidate their own allocation after the same completed value. +Frame-log collection invalidates only a used slot after its fence wait and +preserves timestamp, sequence and kind metadata. + +The source-sliced regression executes the actual three copy/dependency tails +and the CPU consumers against authored device bytes, host-domain availability +and coherent/non-coherent caches. Four phases (normal, owned snapshot, made, +binary log), two cache modes and two host-visibility modes give16 scenarios. +Permissive host visibility remains a positive control; explicit visibility +detects a missing producer dependency even on coherent memory. These models +verify protocol and ownership, not physical non-coherent GPU memory or driver +behavior. PNG encoding and generic device-loss/wait-error handling are outside +this test; no performance or intended visual change is claimed. + +References: [Khronos host readback](https://docs.vulkan.org/guide/latest/synchronization_examples.html#cpu-read-back-of-data-written-by-a-compute-shader) +and [VMA cache management](https://gpuopen-librariesandsdks.github.io/VulkanMemoryAllocator/html/memory_mapping.html). diff --git a/docs/capture-slot-ownership.md b/docs/capture-slot-ownership.md new file mode 100644 index 0000000..e004362 --- /dev/null +++ b/docs/capture-slot-ownership.md @@ -0,0 +1,33 @@ +# Pending screenshot slot ownership + +Normal debug screenshots use two readback buffers, selected by present parity. +A pending capture may outlive the next non-shot present when the GPU is behind. +Another shot two presents later selects the same buffer. Submitting that new copy +before reading the old capture lets a fast GPU replace the pixels, even though +the later read waits for the older timeline value. The filename/flash analysis +then identifies the old frame while consuming the new frame's bytes. + +Before recording a new copy into a pending capture's slot, snapshot that capture +into CPU-owned pixels. Its existing consumer moves the snapshot instead of reading +the reused buffer. Another slot still follows the asynchronous path, and no +pending capture adds no wait. Re-arming starts with no saved snapshot, so the old +pixels cannot be attached to the new frame. The generated-shot path and capture +format/metadata remain unchanged. + +`do_present` is serialized by the replay job consumer; synchronous replay calls +hold `sync_order_mu`. The pending state remains in that producer/consumer flow. +There is at most one pending snapshot, with the same pixel allocation the normal +consumer already makes, moved earlier only when slot reuse requires it. + +`tests/test_capture_slot_ownership.py` compiles the actual copy-selection, snapshot, +read and pending-consume statements with a queued GPU model. It covers a deferred +capture across a non-shot frame, same/opposite parity, fast/late new completion, +no-pending captures and re-arming across sparse intervals. It checks pixels and +metadata together, and verifies that different slots do not wait before submit. +This models legal GPU ordering; it does not claim to reproduce a hardware race. + +This change concerns buffer lifetime. Host-cache visibility is the separate +`invalidate_mapped` contract tracked in PR1502; if that change lands first, retain +its normal-shot invalidation inside `shot_pixels`, after the wait and before the +copy into CPU-owned storage. PR1025's on-demand path still needs the same lifetime +rule when integrated; request paths do not change readback ownership. diff --git a/docs/compiler-memory-accounting.md b/docs/compiler-memory-accounting.md new file mode 100644 index 0000000..9fb3597 --- /dev/null +++ b/docs/compiler-memory-accounting.md @@ -0,0 +1,19 @@ +# Compiler memory accounting + +The build memory gate counts a live compiler driver and its descendants, rather +than just the small driver process. `rss_mb` returns zero for a process not yet +attached, unavailable counters, or a root whose `Popen.poll()` already reports +completion. Retaining a Popen/OS handle must not let stale process-tree counters +or a subsequently reused PID contribute memory after that root has ended. + +This check precedes the platform counter query. It does not change compiler peak +estimates, worker floors, reserves, lease release or live child accounting. A +process can still exit during a later process-tree snapshot; this is not an atomic +snapshot or a complete PID-ancestry race solution. + +`tests/test_rss_liveness.py` retains an actual completed Popen and models a stale +positive backend sample. The known exit takes precedence without querying that +backend; live-tree sampling and unavailable-counter fallback are also checked. +`test_mem_gate_children.py` separately checks a real child holding300MiB, and the +existing memory-gate/lease regressions cover scheduling behavior. These tests need +no compiler, game dump or game execution. diff --git a/docs/design.md b/docs/design.md new file mode 100644 index 0000000..6c94a9a --- /dev/null +++ b/docs/design.md @@ -0,0 +1,265 @@ +# Hookshot (oot-recomp): design + +A native x86-64 Windows and Linux build of the Switch 2 *The Legend of Zelda: Ocarina of Time* remake: the game's AArch64 +code translated to C ahead of time from the user's own decrypted dump and compiled with clang, the Horizon OS and +NVN2 implemented natively, and the GPU work replayed on Vulkan. No CPU or GPU emulation at any point: the same +model as mkw-recomp (Mario Kart World), which this project forks. + +This document is the plan. The numbers in it that depend on the binaries are filled in from the dump (section 9); +until it exists they are marked *dump*. + +## 1. Short answer + +**The game is a Switch 2 title, so the base is mkw-recomp, not TOTKPort.** Public reports before its release +(2026-11-05) describe a Switch 2 exclusive, by Nintendo, 1080p to 1440p docked and 720p handheld, with the ocarina +playable by humming into the console's microphone. A Switch 2 executable means: + +| | Switch 2 (this game, expected) | Switch 1 (TOTK, TOTKPort's base) | +|---|---|---| +| CPU code | ARMv8.2+ (Cortex-A78C): PAC returns, LSE atomics, CAS, LDAPR | ARMv8.0 (Cortex-A57) | +| Relocations | DT_RELR packed + RELA | RELA | +| GPU API | NVN2 (API 231.x): native images, DLSS, mesh/task programs | NVN 55.x | +| Shaders | Ampere SASS (SM86) in `Nin_Ounce_NVN2` packages | Maxwell SASS in `Nin_NX_NVN` packages | +| Services | nx2 gs/gpu/vi2 sessions, no nvdrv in the npdm | nvdrv, vi:u | + +mkw-recomp already handles every row of the left column, from the lifter (`nx2/insn.py`, `nx2/nso2.py`) to the +Vulkan back end with Eden's SM86 recompiler (`gpu/vk`), and its standalone runtime plays MKW races at 45-55 fps. +TOTKPort's NVN1 layer and Maxwell path would have to be rebuilt for Switch 2 first. + +`nx2/dump.py` checks this on the real dump: six independent signals (program id prefix, ACID layout, nvdrv in the +services, DT_RELR, ARMv8.1+ instructions, NVN2 proc names) must all say Switch 2. If they say Switch 1, section 3 +changes: TOTKPort's `hos/nvn_native.c` (NVN 55.15) would sit under this repository's Vulkan back end instead. + +## 2. What is reused (checked 2026-11-04) + +| From | What | Where here | +|---|---|---| +| **mkw-recomp** `7336f0a` (`..\mkw-recomp`, local) | lift driver and its fixes, Switch 2 forms, DT_RELR, Eden's image layout, 64-bit guest window, loop-header polls, stable chunks, Unicorn tests, build; the standalone runtime; the native NVN2 layer; the Vulkan back end | `nx2/`, `host/`, `gpu/`, `third_party/` (commit `86871a1`, byte-identical) | +| **a64-kit** (Rexcomp's AArch64 kit, TOTKPort `60de030`, via mkw-recomp) | NSO loader, image builder, lifter, suyu's translator, instruction harness | `third_party/a64-kit` (unmodified) | +| **Eden's shader recompiler** (Eglantine build 181, via mkw-recomp) | SM86 -> IR -> SPIR-V, task/mesh programs | `gpu/vk/eden` | +| **TOTKPort** (`Desktop\Ports\TOTK\TOTKPort`) | the original HLE runtime and NVN layer that mkw-recomp's are forks of | reference only | +| **Eglantine** (`..\Eglantine`, read and run only) | Switch 2 service protocols, the reference renderer, an A/B oracle | reference only; its CPM cache supplies the Vulkan/SPIR-V headers, VMA, FFmpeg | + +Keeping the fork's paths identical to mkw-recomp's is deliberate: MKW is the reference port and gets fixes every +day (the queue/event ordering and pool captures of 2026-11-04 among them). A later mkw-recomp commit comes over with + +``` +git -C ../mkw-recomp diff 7336f0a -- nx2 host gpu third_party > mkw.patch +git apply -3 mkw.patch +``` + +which works because the base blobs are in this repository. Each sync names its new mkw-recomp base in its commit +message. OoT-specific code goes into new files where it can, so the shared files stay easy to patch. + +## 3. Architecture + +``` + exefs/{rtld,main,subsdk*,sdk} + main.npdm (the user's dump, read in place: work/dump.txt) + │ a64-kit tools/nso.py + nx2/nso2.py (DT_RELR) + ▼ + modules: LZ4 segments, MOD0, dynamic section, symbols, RELA + RELR, .eh_frame_hdr starts + │ nx2/image.py (Eden's layout from 0x80000000: rtld, main, subsdk*, sdk; we are rtld) + ▼ + image: relocated, imports bound across modules; the HLE set hooked (nx2/standalone.py) + │ a64-kit lift/a64_lift.py + nx2/insn.py + nx2/lift_fixes.py + nx2/poll.py + nx2/chunks.py + ▼ + one C function per guest function (work/sa/gen, ~1 GB, local), 64-bit guest window (nx2/rtgen.py) + │ clang (llvm-mingw, emsdk or the system's) -O1, the hot functions -O2 with the play profile, lld (nx2/build.py --standalone) + ▼ + hookshot.exe = lifted game + host/ (Horizon: threads, SVCs, memory, fs, hid, audio, vi, gs, NVN2) + hookshot_vk.dll = the Vulkan back end (gpu/vk), loaded by host/rx_backend.c unless RX_BACKEND names another renderer +``` + +- **Standalone from the start.** MKW went emulator-hosted first (Eglantine with a recompiled CPU backend), then + standalone. That detour is not needed any more: the standalone runtime exists and is the user's rule anyway + ("100% native"). Eglantine stays the reference to compare against, never part of the process. +- **The SDK is replaced at the nn:: boundary** (TOTKPort's model): `HLE()` functions in `host/` stand + in for the SDK's OS-facing functions; the rest of the SDK (libc, nn::util, nn::mem, the NVN2 driver's front end) + runs as lifted code. Unimplemented OS-facing imports get generated stubs that log their first call. +- **Memory.** A 512 GB reservation; the image at Eden's `0x80000000` (the lift's addresses), the heap from 64 GB, + address regions growing down from 512 GB; committed as handed out or on first touch (`host/hos.h`). +- **GPU.** `host/nvn_native.c` implements the NVN2 procs over guest memory in place: objects live where the game + puts them, command buffers record an op stream (`gpu/render/rx_nvn.h`), queue submits hand it to the back end. + `gpu/vk` replays it on Vulkan 1.3: pools imported as host memory, SM86 programs through Eden's recompiler to + SPIR-V, mesh/task programs with `VK_EXT_mesh_shader`. The WebGPU renderer (`gpu/render`) stays in the Windows exe as the + fallback when Vulkan is missing, as in mkw-recomp, when the build has wgpu-native (`WGPU_NATIVE`). + +## 4. The lessons from MKW, as rules here + +Each of these cost days in mkw-recomp; the fork has the fixes, and these are the checks that keep them true for a +second game. + +1. **Cross-queue `nvnCommandBufferWaitEvent` is honoured.** MKW's queues pass stages back and forth with events; + replaying them in arrival order on one Vulkan queue flashed. `gpu/vk/vk_replay.cpp` `submit_job` replays each + submission as far as its waits allow, with per-queue state (`RX_VK_EVENT_WAITS=0` turns it off, for A/B only). + *Check early:* log OoT's queues and event waits on the first boot (which queues, how many waits per frame). +2. **Nothing the replay reads comes from live guest memory.** Games write per-frame data ahead of the GPU: every + read is from submit-time captures, and pools are captured from their creation, not their first GPU use + (`gpu/vk/vk_memory.cpp`, `RX_VK_CAPTURE_POOL_MB`). *Check:* the mirror sync log's live-sync count stays ~0. +3. **Texture and sampler registrations are queued in order with the submits**, never applied out of band. +4. **GPU timestamps drive dynamic resolution.** OoT is reported at 1080p-1440p docked, so it has a resolution + controller fed by timestamp reports: the fake GPU time (the counter reports the replay writes, `gpu/vk/vk_replay.cpp` + `NOP_REPORT_COUNTER`: the 31.25 MHz tick clock since the frame's present, times `RX_VK_TS_SCALE`, 0.1 by default) must be + scaled so the controller settles where the PC is, not at its floor. *Check:* the render size it picks over time. +5. **Clears after draws, and draws that sample a target their pass already wrote (or write one an earlier draw of the + pass sampled), get passes of their own** (`vk_draw.cpp` `clear_after_draws`, `draw_hazards`). A draw that samples + a target it writes itself stays in the pass. +6. **2D views of 3D textures need `VK_EXT_image_2d_view_of_3d`** (the RTX 4060 Ti has it). +7. **The CPU saturates first.** Ryzen 5 7600X, 6 cores / 12 threads: the game's threads, the replay, captures and + pipeline compiles share it, and extra helper threads have lowered fps before. Thread counts are measured, never + raised by default (`HOS_VCORES`, `RX_VK_CAPTURE_THREADS`, `HOS_THREAD_TIMES=`, the `HOS_SAMPLE` profiler + with `scripts/prof.py`). + +Also inherited, and game-specific in MKW: MKW's frame-pacer hooks at fixed guest addresses are gated on MKW's build +id (`hos_game_is_mkw`); OoT gets its own once its pacer is found, never by address coincidence. + +## 5. What OoT needs that MKW did not (checked on the dump, 2026-11-04) + +| | Found | Plan | +|---|---|---| +| Dump format | one `.nup` package (base v65536 + update v131072, each an exefs/ and romfs/ tree, data stored as is) | `nx2/nup.py` extracts the newest title to `work/dump` | +| NSO compression | `main` is ZBIC (NSO flags bit 7, firmware 22.0.0+: zstd with its own magic and BIC-coded tables); rtld, sdk, subsdk0 are LZ4 | `nx2/unpack.py` rewrites ZBIC NSOs as plain ones (hashes checked), decoder `third_party/zbic` | +| SDK / NVN2 version | 22.2.8 (MKW: 21.4.0); same 46 services and 49 SVCs as MKW | 26 NVN proc names MKW never asked for, 14 not in `host/nvn_native.c` yet (stubs that stop the game only when called; DLSS parameter procs, window acquire timeout, HDR metadata, `nvnProgramBuilderGetShaders`, ...); the device limits in `nvn_native.c` are MKW sdk's (`devinfo[]`) and may need OoT's | +| Instructions | 4 UDOT (FEAT_DotProd) sites, the only forms suyu's translator lacked | `nx2/insn.py` `dot()` | +| Engine | romfs laid out like TOTK's (agl, gsys, Phive, Banc, Bake), Havok 2025.2, NintendoWare G3d2/Vfx3/Ui, SMAA, `.zs` (zstd) files | romfs read from `work/dump/romfs` | +| Microphone (the ocarina by humming) | to check at runtime (nn::audio input; `audsmx:u`, `auddmg:u` are in the services, as in MKW) | first a stub that hears silence; then a host microphone (WASAPI) or a key that plays the note | +| Cutscene video / voice | `Movie/` in the romfs | MKW's paths; `HOS_NO_MOVIES=1` to skip while booting | +| DLSS | NVN2 DLSS procs present (66 DLSS / mesh-task names) | a bilinear stand-in first (as Eglantine), FSR or NVIDIA's own DLSS later | + +## 6. Stages and milestones + +| | What | Done when | +|---|---|---| +| **S0** | the dump located and characterized (`nx2/dump.py --set`) | the six signals agree; program id, build ids, module sizes, SDK and NVN versions, romfs size recorded in section 9 | +| **S1** | loader + RELR, image in Eden's layout, discovery, coverage, census; instruction test (every form the game uses) and function tests (self-contained and call trees) against Unicorn | 100 % of discovered instructions translated (UDF aside), no BR without a jump table, every form bit-exact, every comparable function case identical | +| **S2** | the standalone lift (`lift.py --standalone --write`) and a full build (`build.py --standalone`): hookshot.exe and hookshot_vk.dll link | the boot runs `_init_libc*`, `nnosInitialize`, `nninitStartup`, the modules' DT_INIT, `nnMain` and reaches `nvnBootstrapLoader` | +| **S3** | NVN2 for this game: the procs it asks for, device limits, its window; the Vulkan replay | **the first frame**: a `run-vk.ps1` screenshot shows what the game draws first (a logo or loading screen) | +| **S4** | title screen and menus: romfs, save data, hid, audio (muted in tests), movies, the stubs it hits | the title screen and the file select are reachable by scripted input (`HOS_INPUT_SCRIPT` / `HOS_INPUT_LIVE`) | +| **S5** | gameplay: correctness (the flash detector `RX_VK_FLASH=1`, frame dumps) and speed (the profiler) | Kokiri Forest plays without flashes at the game's own frame rate | +| **S6** | PC: resolution, frame rate, controls, the microphone, DLSS/FSR | the user's call | + +## 7. Testing + +- **Instruction forms** (`nx2/test_insn.py`): every form the game's code contains, random states, Unicorn's + ARMv8.x "max" CPU against the lifted C: X0-X30, SP, NZCV, V0-V31, FPSR and memory compared. +- **Whole functions** (`nx2/test_leaf.py`, `--calls` for call trees): random functions of every module on the real + relocated image, natively and in Unicorn, on the 64-bit window with polls firing; registers, V registers and + every changed page compared. +- **The production build** (`nx2/test_dll.py`, when a `hookshot_game.dll` is built). +- **In the game**: `scripts/run-vk.ps1` screenshots, the flash detector, frame dumps (`RX_VK_DUMP*`), the sampled + profiler (`HOS_SAMPLE` + `scripts/prof.py`), and Eglantine as the A/B reference for what a frame should look like. + +## 8. Rules + +- No game files in the repository: decompressed modules, the image, lifted C, builds, romfs copies, dumps and keys + stay in `work/` (ignored) or in the dump's own folder. Nothing is pushed anywhere. +- Commits end with `Assisted-by: Claude Opus 5.5` and `Co-Authored-By: Claude Opus 5.5 `. +- `..\Eglantine` is read and run, never modified; mkw-recomp and the other ports are read and copied from, never + edited. +- One game run at a time on this PC (mkw-recomp's runs take ~10 GB): `run-vk.ps1` refuses while `mkw.exe` or + another `hookshot.exe` runs, and while Rainbow Six Siege is open; no heavy builds while Siege is open either. Runs are + muted (`HOS_AUDIO=0`), at the priority `-Priority` gives (High by default), and screenshot folders are deleted once looked at. +- The lifted C and every build are GPL (suyu's translator is GPL-2.0-or-later; Eden's code is GPL-3.0). + +## 9. Status + +2026-11-04 (before the dump): fork base committed (`86871a1` = mkw-recomp `7336f0a`), the tools made to take any +dump, the runtime made game-neutral, the stage-1 tools checked on MKW's dump as a stand-in. + +2026-11-04, the dump (`gamefile/...[0400486024dec000][v131072].nup`, 25.6 GB; extracted to `work/dump`, 26.2 GB): +- **S0 done.** Switch 2 by all seven signals. Program id 0400486024DEC000, update v131072 (base v65536). + SDK and NVN2 22.2.8. 39-bit address space, main thread priority 44 core 0, 1 MB stack, system resource 48 MB. + + | module | build id | .text | .rodata | .data | .bss | eh starts | + |---|---|---|---|---|---|---| + | rtld | 8e213f228d12c7b0f8c21c3ce81b95e214417369 | 10,048 | 2,532 | 49,160 | 65,528 | 42 | + | main | e0f3ccfaf50065ae4ea01eb5aeb13b1a45da3348 | 63,201,744 | 12,603,520 | 5,217,152 | 1,414,272 | 247,520 | + | subsdk0 | b0110087a7b339ef74588d3bd0b4dd5d5c7abd0f | 542,576 | 234,548 | 69,648 | 65,520 | 1,039 | + | sdk | 1643a838814f9c10c696f930ed681e7fa3fb9af8 | 8,906,832 | 6,093,948 | 435,448 | 1,862,408 | 35,786 | + + romfs: 186,583 files, 24.4 GiB (Tex 8.8, TerrainArc 5.6, Model 2.6, Sound 2.0 GiB), 10,154 `Nin_Ounce_NVN2` + shader files. +- **S1:** the lift: 295,178 functions, 17,668,191 instructions, 100.0000 % translated (after UDOT), 1,475 jump + tables, no BR without one. Instruction test: 742 forms, 44,172 cases, all pass. +- **S2:** the standalone lift: 294,450 functions in 57 chunks (1.4 GB of C), 485 HLE functions, 243 OS imports + stubbed, 22 sdk NVN functions routed to `host/nvn2.c`. + The build links (`hookshot.exe`, 426 MB; ~25 min for a full build with 4 jobs, a few minutes after a relift). +- **S3 done (2026-11-04, boot13).** The title screen renders at 60 fps: black for ~23 s (~1,400 presents) while it + loads, then the logo, smoke and the menu (New Game / Options / ZELDA NOTES Setting), through the Vulkan back end. + It runs the 150 s out with no crash. What it took, in boot order: + - TLS: the owner handle at TLR+0x110 too (SDK 22). NVN API version 231.40 (main 0x8348c0f0 checks minor >= 40). + - svcGetInfo 20 (UserExceptionContextAddress); the address arbiter (svc 0x34/0x35, WaitIfEqual64 too). + - StandardSteadyClock::now returns in x0. ADR / ADRP+ADD code addresses as entries, but only outside the + functions .eh_frame covers (all of them: 1.68 M duplicated instructions and a 122 MB chunk; now 79 entries), + for the formatter callback 0x824c5560 that nothing else reaches. + - SDK 22 overloads left to the guest SDK, which reached kernel calls the runtime does not have: + CreateThread(..., prio, idealCore), ReleaseLightSemaphore(sem, count), MountRom(name), + GetDirectoryEntryCount. + - nn::os fibers on Win32 fibers, each with its own guest context (nn::async's FiberTaskCoordinator on the + nn.npln threads); a fiber function's NULL return goes back to the thread. + - Offline answers: ConnectToNamedPort gives a session whose requests are answered empty (sm:); + nsd::GetEnvironmentIdentifier "lp1"; socket::detail::Poll nothing ready (gRPC calls it through a pointer); + EnsureNetworkServiceAccountIdTokenCacheAsync fails. + - nvnQueueWaitSync (an ordered no-op), SetCopyImageStride; no virtual core held through host file I/O + (mkw-recomp 16bc3bc: the ARes loaders' 250 ms stalls are gone, longest wait now 0.7 ms). +- **Open at S3:** nvnCommandBufferPushComputeConstants is not implemented; one NVN2 program (e409733c45530d75, + stage 5) does not lower (an unsupported native opcode at +0x4330); 70 imports answered by the generic stub + (audio effects, prepo, nifm, socket/ssl setup, vi refresh rate, hid gyro drift mode, pl::RequestLoadNxFont). + Next: input to get past the title (S4: New Game into Kokiri Forest), then the open items above. +- **S4 (2026-11-05): gameplay.** New Game → the movies (AV1 through Media Foundation) → Kokiri Forest, with the + HUD, at 30-36 fps. What it took (commits through 7e88949 and the ones after): + - Push constants (nvnCommandBufferPushComputeConstants) in c0 after the grid; ClearBuffer; prepo stubs. + - setjmp/longjmp: the lifter wraps every call of a function that calls setjmp in a host setjmp of its own + (host/hos_setjmp.c, assembly, returns_twice: clang's `__builtin_longjmp` restored rbp to the frame address, + 128 bytes off Win64's frame pointer), and longjmp's BR comes back through a64_ijump_hook. + - HMMA on the host's tensor cores: HMMA_nx2 → IR MatrixMulAddF16 → OpCooperativeMatrixMulAddKHR 16x8x16 f16. + gpu/vk/vk_coopmat.cpp measures the device's element order (NVIDIA: A's registers 1 and 2 swapped) and checks + a multiply. OoT's networks (the movie upscaler, the gameplay TAAU) run at full speed (the movie at 60 fps). + - Write-backs of GPU-written textures keep the pages the CPU wrote since (SceneLowPriority read texels as + pointers). Storage buffer tables: GetCbuf(i, x*16 + base) addresses get a 16-entry descriptor array that the + global memory fallbacks search (the froxel fog); the 32-bit global atomics got fallbacks too. + - Program variants remember the storage images' integer-ness too: OoT writes R11G11B10 and RGBA8 targets + through R32UI views, and a variant made while the slot held a float texture wrote packed words as floats + (the yellow cast: NaN scene before the TAAU). + - Extended registers: programs using ~249 GPRs (vertex and mesh) keep their high GPRs in IR registers 256 + n + (REGMAP_nx2 prefixes, lower_to_maxwell.h RegisterMapPrefix; R246+ and R239 renamed to markers first). + 222 of the 240 dumped failing programs lower now (the offline harness: `nx2/lower_test.py work/lowfail`). +- **Open at S4:** an intermittent Vulkan device loss (nvlddmkm event 153) shortly after the first gameplay frames, + never with RX_VK_CHECKPOINTS=1 (five runs) and every time without (four); mesh output clamps did not change it. + Six programs still do not lower (CALL linkage 0x944, a texture gather mode, an LDG mode). Gameplay is CPU-bound + at 30-36 fps (the replay thread ~9 ms a frame, the GPU ~10 ms): the game's threads next. + +## 10. Findings, 2026-11-06 + +- White flash: one NaN in the scene target goes through the bloom's averages to the whole picture for a frame, the HUD over it. + `GuardFragmentOutputs` keeps float colour outputs finite (RX_VK_NAN_GUARD=0: off). Where the NaN comes from is not known. +- Discards: a nested early return after a demote wrote its colours (`structured_control_flow.cpp`, from Lichii). That was the box + around Link (`1e26755d`, the per-character shadow projector, drawn as a box that discards outside) and most of the milky look. +- The OoT fog (0.25) and exposure (0.65) factors are still needed: with them off the fog is far too thick. Fog's cause is open. +- Fps is CPU-bound: RX_VK_NO_WRITEBACK cut GPU time from 10 to 3.6 ms and left fps alone. HOS_PIN_BY_CORE 35 fps, HOS_SPIN_US=200 + slower, narrow barriers no change, RX_VK_WAIT_GPU no help. +- Tools: RX_VK_FRAMELOG + scripts/framelog.py (every frame shown), RX_VK_PROBE=2 (the passes at speed), RX_VK_SLOW_LOG, HOS_WATCH. +- A release build (`RX_RELEASE=1` at build time) precompiles before the game starts; test builds do not unless RX_VK_PRECOMPILE=1. + hookshot_programs.bin (with hookshot_programs.idx, the list of its records), hookshot_pipelines.bin and mkw_vk_pipelines.bin go + beside the executable. + +## 11. Findings, 2026-11-06, day + +- Builds: `nx2/split_hot.py` puts code_hot.c in eight files that compile side by side, 79 minutes to about 6. `RX_PGO=gen` counts branches + (host/pgo_rt.c writes LLVM raw format 11, the toolchain has no compiler-rt profile library), `HOS_PGO=` with `.reset` and + `.dump` files drives a run, `nx2/pgo_pieces.py` fits the merged profile to the pieces, `RX_PGO=` uses it. `RX_VK_ONLY=1`: the renderer alone. +- Dark frames, open: lit surfaces come out black for one frame, geometry and depth intact. 5% with `RX_VK_FRAMEGEN=0`, 2.9% with the menu's Off, about 0.1% at 4x. + Not the cause: render conditions, clear flags, ClearBuffer order, GPU lag, write-back, the real-time step, the ambient cubemap. + `HOS_COND_RECORD` and `RX_VK_COND_ALWAYS` look like fixes by count and are not: the picture goes white. Tools: the frame log's dark trap dumps probes, passes and conditions; `RX_VK_PROBE_MIN`, `RX_VK_PROBE_UBO_HEX`. + Also not the cause: colour write masks (`wm` in the pass trace: f on every lit pass), constants (`RX_VK_PROBE_UBO_HEX`), queue order (`RX_VK_EVENT_WAITS=0`), and the capture in + synchronous mode (`RX_VK_SYNC_CAPTURE=1`, which does take 380,000 pages per 600 submits). `RX_VK_QUEUE_STATE=0` stops the game at the Load menu, and `RX_VK_MIRROR=0` is 8 fps: neither says anything. + What holds: geometry and depth are right (black shapes against the fog), the prepass and the lit passes run with the same draw counts, and the lit passes output black. + The lit materials' four storage buffers (c0+0x510..0x540) keep their addresses, sizes and non-zero counts in dark frames as seen in guest memory; the 720 KB one is all zeros in good + frames too, so it is no cause, though its compute shader (4b03b304) runs every frame. Guest memory is not what the GPU reads, so these need comparing on the GPU's side. + Next: their mirrors' contents read back on the GPU's timeline, and the bound textures past the centre texel. `RX_VK_SUBMIT_DELAY_US=3000` shows none, but at 33 fps, where they never show: + repeat with a delay that keeps ~45 fps. The MKW session's mirror page skip fired 0 times here and was removed. + Not an instrument's doing: with nothing attached, photographed from outside the window (37 s, ~2200 photographs): 16 with `RX_VK_FRAMEGEN=0`, none at the user's 4x. With + `RX_VK_SUBMIT_DELAY_US`: 7 at 300, 0 at 1000; a repeat at 1000 with shots still had 7 events against 50-60. So it is a race with the game's threads: less overlap, fewer. The delay is a test, not a fix. +- The builder's first full build: 43 minutes at below-normal priority, 0 failures; the folder starts from a double click and reaches the title at 60 fps. +- Render conditions here gate single lights and read reports earlier in the same stream, so they cannot be read at record time. + diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..cd4eb4e --- /dev/null +++ b/docs/development.md @@ -0,0 +1,221 @@ +# Hookshot (oot-recomp) + +**Hookshot** is the port's name (as Skybound is TOTKPort's and BlueShell is mkw-recomp's); the maintainer's repository +folder stays `oot-recomp`. The game builds as `hookshot.exe` + `hookshot_vk.dll` (`hookshot` + `hookshot_vk.so` on Linux). + +A static recompilation of the Switch 2 *The Legend of Zelda: Ocarina of Time* remake to native x86-64 Windows and Linux: +the game's code lifted to C (NotRexed's a64-kit, Rexcomp's AArch64 kit), the Horizon OS and NVN2 implemented +natively, the GPU work replayed on Vulkan. It is a fork of mkw-recomp (`..\mkw-recomp`, Mario Kart World), whose +paths it keeps so MKW's fixes apply here. + +**No game files are in this repository.** The dump is read where it lies (`work/dump.txt`); everything made from it +(modules, image, lifted C, builds, runs) goes to `work/`, which git ignores. + +Status: the game plays; the README's [status table](../readme.md#project-status) says how far. The design, its stages and +the lessons carried over from MKW: [design.md](design.md). + +## Use + +Requirements: Python 3 with `pip install -r requirements.txt` (`lz4`, `numpy`, `capstone`, `unicorn` 2.1, +`zstandard`, and the button-prompt packages); a compiler: on Windows llvm-mingw +(`HOOKSHOT_MINGW`, or an `llvm-mingw` folder in the clone), else emsdk's clang at `C:\emsdk` (or `EMSDK`) with Visual +Studio's C++ tools; on Linux clang and lld. For the Vulkan back end: distro Vulkan / SPIR-V / VMA headers under +`/usr`, or `HOOKSHOT_VULKAN_HEADERS` / `HOOKSHOT_SPIRV_HEADERS` / `HOOKSHOT_VMA_HEADERS`, or a CPM package cache +(`RX_CPM`, else `cpm` in the clone, else the old caches in mkw-recomp's `work/eg` or `..\Eglantine\.cache\cpm`). +Without those the standalone build stops. [CONTRIBUTING.md](../CONTRIBUTING.md#development-setup) has the rest. + +Run everything with `python -I` from this folder. + +| Command | What | +|---|---| +| `python -I nx2/nup.py --extract work/dump` | the `.nup` dump package to `work/dump/exefs` + `romfs` (newest title; ZBIC NSOs made plain by `nx2/unpack.py`) | +| `python -I nx2/dump.py --dump --set` | what the dump is (Switch 1 or 2, ids, modules, SDK, NVN); `--set` records it in `work/dump.txt` | +| `python -I nx2/lift.py [--write]` | image + discovery + coverage (`work/image/coverage.txt`); `--write` emits the C to `work/gen` | +| `python -I nx2/census.py --nx2` | instruction census per module, after the Switch 2 forms | +| `python -I nx2/test_insn.py [--per 6 --states 12]` | every instruction form the game uses, Unicorn vs lifted C | +| `python -I nx2/test_leaf.py [--count 3000 --states 4] [--calls]` | random whole functions (or call trees), Unicorn vs recompiled native code on the real image | +| `python -I nx2/lift.py --standalone --write` then `python -I nx2/build.py --standalone --jobs 4` | the standalone game: `work/sa/build/hookshot.exe` + `hookshot_vk.dll` | +| `powershell -File scripts/run-vk.ps1 [-Window] [-Seconds 120] [-Name x]` | a muted test run on Vulkan, with screenshots in `work/runs//shots` | +| `python -I scripts/prof.py ` | a profile sampled with `HOS_PROFILE=` + `HOS_SAMPLE=` + `HOS_SAMPLE_FILE=` (the sampler runs only with `HOS_PROFILE` set), named from the linker maps | + +The dump can also come from `OOT_DUMP` (its folder) or `OOT_EXEFS`; `OOT_WORK` moves the work folder. + +The built game's own arguments (`image.bin`, `--romfs`, `--user`, `--trace`, `--prepare-shaders`) are in the README's +[Running the game from the command line](../readme.md#running-the-game-from-the-command-line). + +Scripted test input (`HOS_INPUT_SCRIPT` and `HOS_INPUT_LIVE`) is applied before the window-focus check. A log saying +the game window is not in front does not mean those presses were discarded. `HOS_HEADLESS` and +`HOS_WINDOW_NOACTIVATE` change window creation, not this input routing. The game must still reach a state that reads +and responds to the supplied controller input. + +Physical keyboard and mouse input stays with the foreground window. Physical controllers can also play in the +background when the existing "Controller in background" option or `HOS_INPUT_BACKGROUND=1` enables them; +`HOS_INPUT_BACKGROUND=0` overrides that option. With `HOS_INPUT_EXCLUSIVE` set (to any value, `0` too) the script +alone drives the game until 5 s after its last press; without it the player's devices add to the script's presses. It +is not a focus override. + +## Switches for testing and debugging + +The running game reads its switches from the environment or from `hookshot_env.txt` beside `image.bin` (one +`NAME=VALUE` a line, `#` for notes; `HOS_ENV_FILE=` reads that file instead). The build reads its own from the +environment. The README has the ones players are asked to try (`RX_VK_ASTC_RGBA8`, `RX_VK_LOW_SPEC`, +`RX_VK_MOVIE_UPSCALE`, `RX_VK_SKIP_CODE`, `HOS_OOT_CHEATS`). There are several hundred more, most of them one-off +experiments: `grep -rn 'getenv("' host gpu/vk` lists them, and the comment where each is read says what it does. +"Set" below means any value, `0` included. + +`RX_VK_MESH_NOCULL=` turns off the culling in the task shaders whose native code key starts with one of +the prefixes. On a Steam Deck, `RX_VK_MESH_NOCULL=2c56dc1006942f74` avoids the same hang without leaving anything out: +that task shader walks a coarse depth grid cell by cell for every meshlet and ends with +`visible = (result & 1) | (c3[2320] & 4)`, bit 4 being the game's own "draw everything". The switch makes that answer +true, and dead code elimination removes the grid walk, so every meshlet is drawn and the rasterizer clips what is off +screen. It costs some GPU time on cards that cull quickly, so it is off unless set. The log says +"culling off, every meshlet drawn" for each task shader it changed. + +### Running a test + +| Switch | What it does | +|---|---| +| `HOS_HEADLESS` (set) | No window: the renderer draws offscreen, for screenshots and frame dumps | +| `HOS_WINDOW_SIZE=x` | The window's size in pixels (1280x720 by default, wider for an ultrawide picture) | +| `HOS_WINDOW_NOACTIVATE` (set) | The window opens behind the others and does not take the focus (Windows) | +| `HOS_FULLSCREEN=1` | Starts in fullscreen; the Insert menu's Fullscreen row is then left alone | +| `HOS_AUDIO=0` | No sound device is opened (the mixer still runs); `scripts/run-vk.ps1` sets it unless `-Sound` | +| `HOS_LANGUAGE=` | The game's language for this run, a code from `language.txt` | +| `HOS_OUTPUT_RES=<0..3>`, `HOS_HANDHELD=1`, `HOS_ASPECT=` | Over the options file: the output resolution (0 2048x1152, 1 2560x1440, 2 3840x2160, 3 1920x1080), the handheld profile, a picture wider than 16:9 (2.3889 for 3440x1440) | +| `HOS_DYNRES=0`, `HOS_REALTIME_STEP=0`, `HOS_TEXT_SPEED=<0..3>` | Over the options file: Resolution 100% (non-zero: Auto), Game speed Per frame (1: Real time), Text speed | +| `RX_VK_FRAMEGEN=0`, `RX_VK_FRAMEGEN_MULT=<2..8>`, `RX_VK_PACE=0`, `RX_VK_UPSCALE=<0..2>` | No frame generation whatever the menu says; over the options file: its multiplier, frame pacing, the upscaling filter | +| `RX_VK_DEVICE=` | The graphics card listed as `vk: device ` in the log, over the menu's GPU device | +| `HOOKSHOT_NO_UPDATE_CHECK=1` | No look for a newer release at the start | +| `HOS_TRACE=1` | Every Horizon call logged, as `--trace` does | +| `RX_BACKEND=` | Unset or `vulkan`: `hookshot_vk.dll`; any other value but `eden`: the WebGPU renderer, when the exe was built with one | + +### Input for scripted runs + +| Switch | What it does | +|---|---| +| `HOS_INPUT_SCRIPT=` | Presses timed from the game's first controller read | +| `HOS_INPUT_LIVE=` | A file the test appends lines of presses to, read every 30 ms; each line's times count from when it is read. A line `!cheat ` sends a Cheats page action, `!synclog` logs the NVN syncs (`!dump`, `!shots`, `!rdoc` and `!flasharm` are the WebGPU renderer's) | +| `HOS_INPUT_FRAME_CLOCK=1` | The script's time runs at 60 game frames a second instead of the wall clock | +| `HOS_INPUT_EXCLUSIVE`, `HOS_INPUT_BACKGROUND`, `HOS_INPUT_SCRIPT_BACKGROUND` (set) | As above; the last lets every device count while the window is behind another one | +| `RX_VK_MENU_SCRIPT=:,...` | Timed Insert menu keys: `INS`, `UP`, `DOWN`, `LEFT`, `RIGHT`, `ESC`, `ENTER`, `TABL`, `TABR` | +| `RX_VK_UI_PROMPTS=pad` or `keys` | The menu's button prompts for a controller or the keys, whatever was used last (screenshots) | +| `RX_VK_UI_GAME_FONT=0` | The menu, the counter and the start-up screen in the included Libertinus, not the game's own font | + +The presses are `: