mirror of
https://github.com/tangerineturtle1/NX2navi.git
synced 2026-11-18 06:59:57 -04:00
Merge branch 'main' of https://github.com/tangerineturtle1/NX2navi into test-all-fixes
# Conflicts: # runtime/cfg.c
This commit is contained in:
@@ -55,6 +55,18 @@ Customize Launcher (on the home screen) makes the look the player's own:
|
||||
it is silent until the player's game has been read. Moving the cursor makes no sound. They are played from memory
|
||||
(PlaySound on Windows; `pw-play`, `paplay` or `aplay` elsewhere).
|
||||
|
||||
- Add to Steam (Windows): the game in the player's Steam library as *The Legend of Zelda Ocarina of Time*, started
|
||||
from there without the launcher's window (`launcher/steam.py`). Its pictures are made from the player's game when
|
||||
the row is selected (`launcher/gameart.py`): Navi as the game's HUD draws her for the icon, the title screen's logo,
|
||||
and a frame of the first title movie (the moonlit field) as the background, the cover and the wide tile, the movie
|
||||
decoded with the toolchain's `dav1d.exe` and the logo with the runtime's own ASTC decoder built with its clang. They
|
||||
go into each signed-in account's `config/grid`, the shortcut into its `shortcuts.vdf` (other shortcuts kept as they
|
||||
are, ours updated when it is there; the file before the first change kept as `shortcuts.vdf.nx2navi-backup`).
|
||||
Steam reads that file only when it starts and writes it back when it closes, so a running Steam is asked to close
|
||||
first and started again after. The shortcut runs the launcher's `pythonw.exe` on `data/steam_play.pyw`, which
|
||||
starts the launcher of `app/current.txt` with `--play`: the game starts as Play starts it, and the process waits
|
||||
for it, so Steam counts the game as running while it is.
|
||||
|
||||
The choices are `theme`, `background`, `background_dim`, `animations` and `sounds` in `data/launcher.json`.
|
||||
|
||||
Navi (`art.NAVI`, 22 x 21 pixels, drawn in whole pixels at every size) is also the icon of `NX2navi.exe`,
|
||||
@@ -66,7 +78,7 @@ Screens (`launcher/ui/app.py`): first run (dump, install folder, Desktop Shortcu
|
||||
Update Channel, Begin Installation); the steps of an install, update or rebuild with their progress, the build log's
|
||||
last lines and Cancel, and after a failure Try Again and Copy Report; home (Play, Updates, Game Folder, Saves,
|
||||
Rebuild the Game, Customize Launcher, Settings); customize (Color Scheme, Background, with a picture set Background
|
||||
Dimming and Remove Background, Animations, Sounds, Back); saves (Export Save, Add Log and Settings, Export To, Import a Save, Import a Save Folder,
|
||||
Dimming and Remove Background, Animations, Sounds, Add to Steam on Windows, Back); saves (Export Save, Add Log and Settings, Export To, Import a Save, Import a Save Folder,
|
||||
Open Saves Folder, Back); settings (update check, Install Updates Automatically, Hide Launcher While Playing, Keep
|
||||
Build Cache, Update Channel, the folders, Open Logs Folder, Uninstall, Back); uninstall (keep or remove the built game).
|
||||
A screen with more rows than the window holds draws them closer together (`fit_rows`).
|
||||
|
||||
@@ -75,7 +75,7 @@ Source shorthands:
|
||||
| 3.10 | **BFE / BFI fields past bit 31** (as PTX `bfe` / `bfi`): a field that runs past bit 31 is cut there; one that starts past it extracts 0 (signed: bit 31 repeated) and inserts nothing. Terrain vertex shaders (42 variants) have taken the position from a register, `((2y + x) * 8) & 0xff` with 5-bit x and y, so up to 248. | `OpBitField*Extract` / `OpBitFieldInsert` undefined when offset or offset + count exceeds 32; AMD's bitfield instructions take offset and width modulo 32. | The translator passes the fields through. Terrain walls and ground went missing on the Steam Deck and a Linux RX 570 (RADV). Ours: clamped where the field is not a constant inside the word (translator patch 0012). RX 9070 on Windows drew the terrain the same before and after. | PTX ISA `bfe`, `bfi` |
|
||||
| 3.11 | **SHF by 0** gives the unshifted word (C for `SHF.L`, A for `SHF.R`). | The translator's funnel shifts the other word by 32: undefined in SPIR-V; AMD shifts take the low 5 bits and give A \| C. | Gives terrain gaps at a distance. Ours: a select on a zero amount (translator patch 0013). In the shaders checked, every register SHF pairs its word with 0, so the result was right already. | — |
|
||||
| 3.12 | **Registers read before any write** on a path (paths the structurizer adds included) have no defined value in the game's code; the NVIDIA driver has been reported to give 0. | An `OpVariable` without an initializer loads an undefined value; Mesa folds undefined values freely. | Ours: every function-local variable starts at 0 (`OpConstantNull`, job flag `zero_locals`, translator patch 0016) on every driver; `NX2NAVI_ZERO_LOCALS=0` turns it off. On NVIDIA an undefined read takes another warp lane's value: Ocarina's clustered-light loop lit its floors in 4x8-pixel tiles (one warp each) and darkened Link. It was off while DLAA was on, because zeroing moves the SPIR-V hash of the game's FXAA shader that `vk_dlaa.c` `vk_dlaa_skip_draw` matches (`FXAA_FS` holds the zeroed hash too); with it off the tiles showed under DLSS and DLAA, and the shader cache keys, which include the flag, changed with the upscaler, so shaders prepared with one setting missed under the other. Checked on an RTX 5090: zeroed, DLAA keeps its camera motion (`NX2NAVI_MV_CHECK`) and the floor is clean. | — |
|
||||
| 3.13 | **NaN, infinity and signed zero follow IEEE 754**, and the game's code leans on them: its compiler writes an unordered comparison as `x > c \|\| isnan(x)`, a normalisation of a zero-length vector gives an infinity that a later test catches, and integer masks (0xffffffff) pass through float moves and selects. | SPIR-V without the `SignedZeroInfNanPreserve` execution mode lets the driver assume that no float is NaN or infinite. AMD's compilers act on it; NVIDIA's driver keeps the IEEE results. | The isnan tests of the game and of the translator (patches 0027, 0031) are only safe with the mode. On an RX 9070 XT a select between 1 and the bits 0xffffffff, converted as a signed integer, gave 2^32 instead of -1 and blew shadow casters up (issue #156); RX 6000 cards drew black and white blocks on Navi and on Link next to her (#41, #110); AMD GPUs drew Navi's sparkles without her orb. Ours: `vk_pipeline.c` `spirv_nan_preserve` adds the mode for 32-bit floats (16 and 64 where the module declares them and the driver can) when the module is created, on every driver except NVIDIA's (`NX2NAVI_VK_NAN_PRESERVE`). | VKSPEC (SPIR-V environment, precision and operation of SPIR-V instructions) |
|
||||
| 3.13 | **NaN, infinity and signed zero follow IEEE 754**, and the game's code leans on them: its compiler writes an unordered comparison as `x > c \|\| isnan(x)`, a normalisation of a zero-length vector gives an infinity that a later test catches, and integer masks (0xffffffff) pass through float moves and selects. | SPIR-V without the `SignedZeroInfNanPreserve` execution mode lets the driver assume that no float is NaN or infinite. AMD's compilers act on it; NVIDIA's driver keeps the IEEE results. | The isnan tests of the game and of the translator (patches 0027, 0031) are only safe with the mode. On an RX 9070 XT a select between 1 and the bits 0xffffffff, converted as a signed integer, gave 2^32 instead of -1 and blew shadow casters up (issue #156); RX 6000 cards drew black and white blocks on Navi and on Link next to her (#41, #110); AMD GPUs drew Navi's sparkles without her orb. Ours: `vk_pipeline.c` `spirv_nan_preserve` adds the mode for 32-bit floats (16 and 64 where the module declares them and the driver can) when the module is created, on every driver except NVIDIA's (`NX2NAVI_VK_NAN_PRESERVE`). On AMD's Windows driver for RDNA 1 and 2 ("AMD proprietary shader compiler", `vk_caps.amd_sc`), where the black and white blocks were reported, `spirv_nan_bits` also turns every 32-bit `OpIsNan` into an integer test of its bits and a float select with a NaN-bits constant into a select of the bits, which no float assumption can change (`NX2NAVI_VK_NAN_BITS`). | VKSPEC (SPIR-V environment, precision and operation of SPIR-V instructions) |
|
||||
|
||||
## 4. Memory model
|
||||
|
||||
|
||||
35
docs/releases/v0.7.2.md
Normal file
35
docs/releases/v0.7.2.md
Normal file
@@ -0,0 +1,35 @@
|
||||
Draft notes for the release after 0.7.1. The version number is not decided yet.
|
||||
|
||||
## Faster
|
||||
|
||||
- **AMD cards: about a fifth more frames in open areas, and more even frame times.** The game asks the video card
|
||||
which objects are visible, about a hundred questions a frame in Hyrule Field, and waits for a frame to be finished
|
||||
on the card before it lets the next one start. The answers were fetched one call at a time, on the thread that also
|
||||
reports "finished", and on AMD's Windows driver each of those calls goes into the kernel: every frame was reported
|
||||
finished about 4 ms late, with every thread of the game waiting. They are now fetched a few hundred at a time.
|
||||
Standing in Hyrule Field on an RX 7900 XT (driver 26.10.1, 3440x1440, FSR): 53 fps instead of 45, and the longest
|
||||
frame of a minute 46 ms instead of 55 (`oq_batch = 0` in `nx2navi.ini` brings the old readback back).
|
||||
- **No more frames drawn one present late.** The game signals from the CPU when a frame's data is written, and its
|
||||
render thread submits the frame's commands without waiting for that signal. When the signal came a millisecond or
|
||||
two after the submit call, in about every other frame in Hyrule Field, the frame was set aside and drawn at its
|
||||
present, and frames of 23 and 15 ms took turns. The submit call now waits for the signal, 10 ms at most
|
||||
(`gate_wait_ms`): 54.5 fps instead of 53 in the same place and the longest frame 33 ms instead of 46; with
|
||||
`vk_recorder = 1`, which AMD cards do not have on by default, 58.6 fps.
|
||||
|
||||
## New
|
||||
|
||||
- **Add to Steam** in Customize Launcher (Windows): the game joins your Steam library as *The Legend of Zelda Ocarina
|
||||
of Time*, with Navi as its icon, the title screen's moonlit field as its background and cover, and the game's logo,
|
||||
all made from your own game files when you select the row. Playing it from Steam starts the game straight away,
|
||||
without the launcher. Steam closes for a moment while the shortcut is written and starts again with it.
|
||||
|
||||
## Fixed
|
||||
|
||||
- Black and white blocks on AMD RX 5000 and RX 6000 cards: on Navi and on Link next to her, on bright floors near her,
|
||||
and on the bright outdoors seen through a door or window from inside a house. Every float colour a shader writes is
|
||||
checked for NaN, so that one bad pixel cannot spread through the blur and bloom passes; those checks, and the
|
||||
game's own NaN tests in nearly nine of ten shaders, are float comparisons that a shader compiler may decide can
|
||||
never be true. The Windows driver for these cards uses AMD's older shader compiler (the log reads "AMD proprietary
|
||||
shader compiler"); with it each NaN test is now made on the bits of the number instead, which no compiler can drop.
|
||||
It is on by default only with that driver (`vk_nan_bits`); the first start after the update compiles the pipelines
|
||||
again. Untested on those cards before this release: please report whether the blocks are gone.
|
||||
358
launcher/gameart.py
Normal file
358
launcher/gameart.py
Normal file
@@ -0,0 +1,358 @@
|
||||
"""The pictures of the game's Steam shortcut (launcher/steam.py), made from the player's own game: nothing ships.
|
||||
|
||||
icon Navi as the game's HUD draws her (HUDNaviIcon_Body_00 over the glow HUDNaviIcon_Flare_00), 128 x 128
|
||||
logo the title screen's logo with the Master Sword and the Hylian Shield (Logo_USEU_00), cut to its outline
|
||||
hero the title screen's moonlit field, a frame of its first movie (Event/Movie/Dm_Title_0005.webm), 1920 x 620
|
||||
portrait that frame with the logo over it, 600 x 900 (the library's cover)
|
||||
wide the same at 920 x 430 (the library's wide tile)
|
||||
|
||||
The textures come from the interface archive UI/Layout2Archive/Title...bntx.zs as runtime/menu_assets.c reads them:
|
||||
a BNTX's BRTI records give format, size, block height and data; the level is block linear in the Switch 2 GOB layout
|
||||
(runtime/tex_deswizzle.c tex_bl_copy); BC4 masks become white with the mask as alpha; ASTC 4x4 goes through the
|
||||
runtime's own decoder, runtime/tex_astc.c, built here into a library with the toolchain's clang (the launcher builds
|
||||
the game with it, so it is on every PC that has a game to add). The movie is WebM with AV1 video: the frames from the
|
||||
key frame before TITLE_AT up to it go into an IVF file for the toolchain's dav1d.exe, whose last picture (8-bit 4:2:0,
|
||||
limited range BT.709) becomes the background. Every picture is written as PNG with art.png_bytes; nothing here needs
|
||||
an imaging package.
|
||||
"""
|
||||
import ctypes
|
||||
import hashlib
|
||||
import os
|
||||
import struct
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
from . import gamefiles
|
||||
|
||||
TITLE_ARCHIVE = 'UI/Layout2Archive/Title.Product.100.Nin_Ounce_NVN2.bntx.zs'
|
||||
COMMON_ARCHIVE = 'UI/Layout2Archive/Common.Product.100.Nin_Ounce_NVN2.bntx.zs'
|
||||
LOGO, NAVI, NAVI_GLOW = 'Logo_USEU_00', 'HUDNaviIcon_Body_00', 'HUDNaviIcon_Flare_00'
|
||||
TITLE_MOVIE = 'Event/Movie/Dm_Title_0005.webm'
|
||||
TITLE_AT = 12.0 # seconds into it: the field under the full moon, as the title screen shows it first
|
||||
KIND_BC4, KIND_ASTC4 = 0x1d, 0x2d
|
||||
NO_WINDOW = 0x08000000 if os.name == 'nt' else 0
|
||||
|
||||
|
||||
class ArtError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- BNTX
|
||||
|
||||
def _u16(b, at): return struct.unpack_from('<H', b, at)[0]
|
||||
def _u32(b, at): return struct.unpack_from('<I', b, at)[0]
|
||||
def _u64(b, at): return struct.unpack_from('<Q', b, at)[0]
|
||||
|
||||
|
||||
def bntx_textures(b):
|
||||
"""Each texture of a BNTX file: name, format, size, block height, data offset and size."""
|
||||
out = []
|
||||
count, arr = _u32(b, 0x24), _u64(b, 0x28)
|
||||
for i in range(count):
|
||||
p = _u64(b, arr + 8 * i)
|
||||
if b[p:p + 4] != b'BRTI':
|
||||
continue
|
||||
no = _u64(b, p + 0x60)
|
||||
name = b[no + 2:no + 2 + _u16(b, no)].decode('utf-8', 'replace')
|
||||
out.append({'name': name, 'fmt': _u32(b, p + 0x1c), 'w': _u32(b, p + 0x24), 'h': _u32(b, p + 0x28),
|
||||
'th': _u32(b, p + 0x34), 'size': _u32(b, p + 0x50), 'data': _u64(b, _u64(b, p + 0x70))})
|
||||
return out
|
||||
|
||||
|
||||
def deswizzle(t, b, row, rows):
|
||||
"""The first level as tightly packed rows of `row` bytes (runtime/tex_deswizzle.c tex_bl_copy)."""
|
||||
th = t['th']
|
||||
gobs, unit, block = (row + 63) // 64, 8 << th, 512 << th
|
||||
need = gobs * ((rows + unit - 1) // unit) * block
|
||||
src = bytearray(max(need, t['size']) + 16)
|
||||
src[:t['size']] = b[t['data']:t['data'] + t['size']]
|
||||
bl = np.frombuffer(bytes(src), dtype='V16')
|
||||
cols = (row + 15) // 16
|
||||
y = np.arange(rows)[:, None]
|
||||
x = (np.arange(cols) * 16)[None, :]
|
||||
at = (y // unit) * gobs * block + (x // 64) * block + ((y % unit) // 8) * 512 + \
|
||||
(x % 64 // 32) * 256 + (y % 8 // 4) * 128 + (x % 32 // 16) * 64 + (y % 4) * 16
|
||||
flat = np.frombuffer(bl[at // 16].tobytes(), dtype=np.uint8).reshape(rows, cols * 16)
|
||||
return flat[:, :row]
|
||||
|
||||
|
||||
def bc4(lin, wb, hb, w, h):
|
||||
"""White with the mask as alpha (runtime/menu_assets.c bc4_decode)."""
|
||||
blocks = lin.reshape(hb, wb, 8).astype(np.uint32)
|
||||
a, b = blocks[..., 0], blocks[..., 1]
|
||||
pal = np.zeros((hb, wb, 8), dtype=np.uint32)
|
||||
pal[..., 0], pal[..., 1] = a, b
|
||||
big = a > b
|
||||
for i in range(6):
|
||||
pal[..., 2 + i] = np.where(big, ((6 - i) * a + (i + 1) * b) // 7, 0)
|
||||
for i in range(4):
|
||||
pal[..., 2 + i] = np.where(big, pal[..., 2 + i], ((4 - i) * a + (i + 1) * b) // 5)
|
||||
pal[..., 6] = np.where(big, pal[..., 6], 0)
|
||||
pal[..., 7] = np.where(big, pal[..., 7], 255)
|
||||
bits = np.zeros((hb, wb), dtype=np.uint64)
|
||||
for i in range(6):
|
||||
bits |= blocks[..., 2 + i].astype(np.uint64) << np.uint64(8 * i)
|
||||
alpha = np.zeros((hb * 4, wb * 4), dtype=np.uint8)
|
||||
for i in range(16):
|
||||
sel = ((bits >> np.uint64(3 * i)) & np.uint64(7)).astype(np.intp)
|
||||
alpha[i // 4::4, i % 4::4] = np.take_along_axis(pal, sel[..., None], 2)[..., 0]
|
||||
rgba = np.full((h, w, 4), 255, dtype=np.uint8)
|
||||
rgba[..., 3] = alpha[:h, :w]
|
||||
return rgba
|
||||
|
||||
|
||||
def astc_library(layout):
|
||||
"""runtime/tex_astc.c of this release as a library in data/cache, built once per source with the toolchain."""
|
||||
src = layout.app / 'runtime' / 'tex_astc.c'
|
||||
if not src.is_file():
|
||||
raise ArtError(f'{src} is missing')
|
||||
tag = hashlib.sha256(src.read_bytes() + (layout.app / 'runtime' / 'tex.h').read_bytes()).hexdigest()[:12]
|
||||
dll = layout.cache / f'astc-{tag}.dll'
|
||||
if not dll.is_file():
|
||||
clang = layout.msys / 'bin' / 'clang.exe'
|
||||
if not clang.is_file():
|
||||
raise ArtError('the build tools are not installed (Rebuild the Game installs them)')
|
||||
env = dict(os.environ, PATH=str(layout.msys / 'bin') + os.pathsep + os.environ.get('PATH', ''))
|
||||
tmp = dll.with_name(dll.name + '.tmp')
|
||||
r = subprocess.run([str(clang), '-shared', '-O2', '-static', '-w', '-I', str(layout.app / 'runtime'),
|
||||
'-I', str(layout.msys / 'include' / 'vma'), str(src), '-o', str(tmp)],
|
||||
capture_output=True, text=True, env=env, creationflags=NO_WINDOW)
|
||||
if r.returncode or not tmp.is_file():
|
||||
raise ArtError(f'the texture decoder did not build: {(r.stderr or r.stdout).strip()[-300:]}')
|
||||
os.replace(tmp, dll)
|
||||
lib = ctypes.CDLL(str(dll))
|
||||
lib.astc_decode_ldr.restype = None
|
||||
lib.astc_decode_ldr.argtypes = [ctypes.c_char_p, ctypes.c_uint32, ctypes.c_uint32, ctypes.c_uint32,
|
||||
ctypes.c_uint32, ctypes.c_bool, ctypes.c_void_p]
|
||||
return lib
|
||||
|
||||
|
||||
def texture(bntx, name, lib):
|
||||
"""The named texture as an HxWx4 uint8 array (BC4 or ASTC 4x4), or None when the file has no such texture."""
|
||||
for t in bntx_textures(bntx):
|
||||
if t['name'] != name:
|
||||
continue
|
||||
kind = t['fmt'] >> 8
|
||||
w, h = t['w'], t['h']
|
||||
wb, hb = (w + 3) // 4, (h + 3) // 4
|
||||
if kind == KIND_BC4:
|
||||
return bc4(deswizzle(t, bntx, wb * 8, hb), wb, hb, w, h)
|
||||
if kind == KIND_ASTC4:
|
||||
lin = deswizzle(t, bntx, wb * 16, hb)
|
||||
rgba = np.zeros((h, w, 4), dtype=np.uint8)
|
||||
lib.astc_decode_ldr(lin.tobytes(), w, h, 4, 4, (t['fmt'] & 0xff) == 6, rgba.ctypes.data)
|
||||
return rgba
|
||||
raise ArtError(f'{name} has a texture format this launcher does not read ({t["fmt"]:#x})')
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- the title movie
|
||||
|
||||
_MASTERS = {0x18538067, 0x1F43B675, 0xA0, 0x1654AE6B, 0xAE, 0xE0} # Segment, Cluster, BlockGroup, Tracks, TrackEntry, Video
|
||||
|
||||
|
||||
def _vint(b, at, keep_marker):
|
||||
first = b[at]
|
||||
n = 1
|
||||
while n <= 8 and not first & (0x80 >> (n - 1)):
|
||||
n += 1
|
||||
if n > 8:
|
||||
raise ArtError('the movie is not WebM')
|
||||
v = first if keep_marker else first & (0xff >> n)
|
||||
for i in range(1, n):
|
||||
v = v << 8 | b[at + i]
|
||||
return v, n
|
||||
|
||||
|
||||
def webm_frames(b):
|
||||
"""(time in ms, key frame, data) of every block of a single-track WebM, and the video's width and height."""
|
||||
out, info = [], {}
|
||||
|
||||
def walk(at, end, cluster):
|
||||
while at < end:
|
||||
eid, n1 = _vint(b, at, True)
|
||||
size, n2 = _vint(b, at + n1, False)
|
||||
body = at + n1 + n2
|
||||
if size == (1 << (7 * n2)) - 1: # unknown size: to the end of the parent
|
||||
size = end - body
|
||||
if eid in _MASTERS:
|
||||
walk(body, min(body + size, end), cluster)
|
||||
elif eid == 0xE7:
|
||||
cluster[0] = int.from_bytes(b[body:body + size], 'big')
|
||||
elif eid == 0xB0:
|
||||
info['w'] = int.from_bytes(b[body:body + size], 'big')
|
||||
elif eid == 0xBA:
|
||||
info['h'] = int.from_bytes(b[body:body + size], 'big')
|
||||
elif eid in (0xA3, 0xA1): # SimpleBlock (its flags say key frame), Block
|
||||
_track, n = _vint(b, body, False)
|
||||
rel = struct.unpack_from('>h', b, body + n)[0]
|
||||
key = eid == 0xA3 and bool(b[body + n + 2] & 0x80)
|
||||
out.append((cluster[0] + rel, key, b[body + n + 3:body + size]))
|
||||
at = body + size
|
||||
|
||||
walk(0, len(b), [0])
|
||||
if not out or 'w' not in info or 'h' not in info:
|
||||
raise ArtError('the title movie has no video frames')
|
||||
return out, info['w'], info['h']
|
||||
|
||||
|
||||
def movie_frame(layout, data, at_s, work):
|
||||
"""The picture of the movie `data` at at_s seconds as an HxWx3 uint8 array, decoded by the toolchain's dav1d."""
|
||||
dav1d = layout.msys / 'bin' / 'dav1d.exe'
|
||||
if not dav1d.is_file():
|
||||
raise ArtError('the build tools are not installed (Rebuild the Game installs them)')
|
||||
frames, w, h = webm_frames(data)
|
||||
upto = [f for f in frames if f[0] <= at_s * 1000] or frames[:1]
|
||||
k0 = max((i for i, f in enumerate(upto) if f[1]), default=0)
|
||||
sel = upto[k0:]
|
||||
ivf, y4m = Path(work) / 'title.ivf', Path(work) / 'title.y4m'
|
||||
with open(ivf, 'wb') as f:
|
||||
f.write(b'DKIF' + struct.pack('<HH4sHHIIII', 0, 32, b'AV01', w, h, 1000, 1, len(sel), 0))
|
||||
for t, _key, d in sel:
|
||||
f.write(struct.pack('<IQ', len(d), max(t, 0)) + d)
|
||||
r = subprocess.run([str(dav1d), '-q', '-i', str(ivf), '-o', str(y4m)], capture_output=True, text=True,
|
||||
cwd=str(dav1d.parent), creationflags=NO_WINDOW)
|
||||
if r.returncode or not y4m.is_file():
|
||||
raise ArtError(f'the title movie did not decode: {(r.stderr or r.stdout).strip()[-300:]}')
|
||||
b = y4m.read_bytes()
|
||||
at = b.rfind(b'FRAME\n')
|
||||
if at < 0 or len(b) < at + 6 + w * h * 3 // 2:
|
||||
raise ArtError('the title movie did not decode to a picture')
|
||||
at += 6
|
||||
y = np.frombuffer(b, np.uint8, w * h, at).reshape(h, w).astype(np.float32)
|
||||
cw, ch = (w + 1) // 2, (h + 1) // 2
|
||||
u = np.frombuffer(b, np.uint8, cw * ch, at + w * h).reshape(ch, cw).astype(np.float32)
|
||||
v = np.frombuffer(b, np.uint8, cw * ch, at + w * h + cw * ch).reshape(ch, cw).astype(np.float32)
|
||||
u = (u.repeat(2, 0).repeat(2, 1)[:h, :w] - 128) * (255 / 224)
|
||||
v = (v.repeat(2, 0).repeat(2, 1)[:h, :w] - 128) * (255 / 224)
|
||||
y = (y - 16) * (255 / 219)
|
||||
rgb = np.stack([y + 1.5748 * v, y - 0.1873 * u - 0.4681 * v, y + 1.8556 * u], -1) # BT.709
|
||||
return np.clip(rgb + 0.5, 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- pictures
|
||||
|
||||
def resize(img, w, h):
|
||||
"""img (HxWxC uint8 or float) at w x h: whole-pixel box averaging first when it shrinks by 2 or more, then
|
||||
bilinear. Returns float32."""
|
||||
a = img.astype(np.float32)
|
||||
fy, fx = a.shape[0] // h, a.shape[1] // w
|
||||
if fy >= 2 or fx >= 2:
|
||||
fy, fx = max(fy, 1), max(fx, 1)
|
||||
H, W = a.shape[0] // fy * fy, a.shape[1] // fx * fx
|
||||
a = a[:H, :W].reshape(H // fy, fy, W // fx, fx, -1).mean(axis=(1, 3))
|
||||
ys = (np.arange(h) + 0.5) * a.shape[0] / h - 0.5
|
||||
xs = (np.arange(w) + 0.5) * a.shape[1] / w - 0.5
|
||||
y0 = np.clip(np.floor(ys).astype(int), 0, a.shape[0] - 1)
|
||||
x0 = np.clip(np.floor(xs).astype(int), 0, a.shape[1] - 1)
|
||||
y1, x1 = np.minimum(y0 + 1, a.shape[0] - 1), np.minimum(x0 + 1, a.shape[1] - 1)
|
||||
wy = np.clip(ys - y0, 0, 1)[:, None, None]
|
||||
wx = np.clip(xs - x0, 0, 1)[None, :, None]
|
||||
top = a[y0][:, x0] * (1 - wx) + a[y0][:, x1] * wx
|
||||
bottom = a[y1][:, x0] * (1 - wx) + a[y1][:, x1] * wx
|
||||
return top * (1 - wy) + bottom * wy
|
||||
|
||||
|
||||
def premultiplied_resize(rgba, w, h):
|
||||
"""Resize with the colours weighted by alpha, so transparent texels add no dark fringe; uint8 straight alpha out."""
|
||||
a = rgba.astype(np.float32)
|
||||
a[..., :3] *= a[..., 3:4] / 255
|
||||
r = resize(a, w, h)
|
||||
r[..., :3] /= np.maximum(r[..., 3:4], 1e-3) / 255
|
||||
return np.clip(r + 0.5, 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
def trim(rgba, pad=8):
|
||||
"""rgba cut to the box of its visible texels, pad texels around."""
|
||||
ys, xs = np.nonzero(rgba[..., 3] > 4)
|
||||
if not len(ys):
|
||||
return rgba
|
||||
y0, y1 = max(ys.min() - pad, 0), min(ys.max() + 1 + pad, rgba.shape[0])
|
||||
x0, x1 = max(xs.min() - pad, 0), min(xs.max() + 1 + pad, rgba.shape[1])
|
||||
return rgba[y0:y1, x0:x1]
|
||||
|
||||
|
||||
def place(dst, src, x, y):
|
||||
"""src (uint8 RGBA, straight alpha) over dst (uint8 RGB) at x, y."""
|
||||
h, w = src.shape[:2]
|
||||
s = src.astype(np.float32)
|
||||
d = dst[y:y + h, x:x + w].astype(np.float32)
|
||||
al = s[..., 3:4] / 255
|
||||
dst[y:y + h, x:x + w] = np.clip(d * (1 - al) + s[..., :3] * al + 0.5, 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
def crop(frame, w, h, cx, cy):
|
||||
"""The largest w:h part of frame centred near (cx, cy) as fractions, scaled to w x h (uint8 RGB)."""
|
||||
H, W = frame.shape[:2]
|
||||
if W * h > H * w:
|
||||
ch, cw = H, H * w // h
|
||||
else:
|
||||
cw, ch = W, W * h // w
|
||||
x0 = int(np.clip(cx * W - cw / 2, 0, W - cw))
|
||||
y0 = int(np.clip(cy * H - ch / 2, 0, H - ch))
|
||||
return np.clip(resize(frame[y0:y0 + ch, x0:x0 + cw], w, h) + 0.5, 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
def with_logo(bg, logo, width_share, cy):
|
||||
"""bg (RGB) with the logo scaled to width_share of its width, centred at height cy (a fraction)."""
|
||||
H, W = bg.shape[:2]
|
||||
lw = int(W * width_share)
|
||||
lh = int(lw * logo.shape[0] / logo.shape[1])
|
||||
if lh > H * 0.8:
|
||||
lh = int(H * 0.8)
|
||||
lw = int(lh * logo.shape[1] / logo.shape[0])
|
||||
small = premultiplied_resize(logo, lw, lh)
|
||||
out = bg.copy()
|
||||
place(out, small, (W - lw) // 2, int(np.clip(cy * H - lh / 2, 0, H - lh)))
|
||||
return out
|
||||
|
||||
|
||||
def navi_icon(body, glow, size=128):
|
||||
"""Navi's HUD icon on her glow, size x size RGBA: the glow tinted the pale blue of her light."""
|
||||
out = np.zeros((size, size, 4), dtype=np.float32)
|
||||
g = resize(glow, size, size)
|
||||
g[..., :3] = np.array([200, 228, 255], dtype=np.float32)
|
||||
g[..., 3] *= 0.85
|
||||
b = resize(body, int(size * 0.8), int(size * 0.8))
|
||||
for layer, at in ((g, 0), (b, (size - b.shape[0]) // 2)):
|
||||
h, w = layer.shape[:2]
|
||||
d = out[at:at + h, at:at + w]
|
||||
sa, da = layer[..., 3:4] / 255, d[..., 3:4] / 255
|
||||
oa = sa + da * (1 - sa)
|
||||
d[..., :3] = (layer[..., :3] * sa + d[..., :3] * da * (1 - sa)) / np.maximum(oa, 1e-6)
|
||||
d[..., 3:4] = oa * 255
|
||||
return np.clip(out + 0.5, 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
def rgba_of(rgb):
|
||||
return np.concatenate([rgb, np.full(rgb.shape[:2] + (1,), 255, dtype=np.uint8)], -1)
|
||||
|
||||
|
||||
def make(layout, game_dir='', dump=''):
|
||||
"""{name: HxWx4 uint8} for icon, logo, hero, portrait and wide, from the player's game. ArtError when the game's
|
||||
files or the tools to read them are not there."""
|
||||
romfs = gamefiles.open_romfs(layout, game_dir, dump)
|
||||
if romfs is None:
|
||||
raise ArtError('the game\'s files were not found (the game folder\'s romfs, or the dump)')
|
||||
zs = gamefiles.ZsReader(romfs)
|
||||
title, common = zs.read(TITLE_ARCHIVE), zs.read(COMMON_ARCHIVE)
|
||||
movie = romfs.read(TITLE_MOVIE)
|
||||
if not title or not common or not movie:
|
||||
raise ArtError('the game\'s title screen files were not found')
|
||||
lib = astc_library(layout)
|
||||
logo = texture(title, LOGO, lib)
|
||||
body, glow = texture(common, NAVI, lib), texture(common, NAVI_GLOW, lib)
|
||||
if logo is None or body is None or glow is None:
|
||||
raise ArtError('the game\'s logo or Navi\'s icon was not found')
|
||||
logo = trim(logo)
|
||||
with tempfile.TemporaryDirectory(dir=str(layout.cache)) as work:
|
||||
frame = movie_frame(layout, movie, TITLE_AT, work)
|
||||
# the moon sits right of the middle and high: every cut keeps it, and the hero keeps the horizon too
|
||||
hero = crop(frame, 1920, 620, 0.5, 0.42)
|
||||
portrait = with_logo(crop(frame, 600, 900, 0.68, 0.5), logo, 0.88, 0.70)
|
||||
wide = with_logo(crop(frame, 920, 430, 0.5, 0.45), logo, 0.52, 0.55)
|
||||
return {'icon': navi_icon(body, glow), 'logo': logo, 'hero': rgba_of(hero), 'portrait': rgba_of(portrait),
|
||||
'wide': rgba_of(wide)}
|
||||
329
launcher/steam.py
Normal file
329
launcher/steam.py
Normal file
@@ -0,0 +1,329 @@
|
||||
"""Add to Steam (Customize Launcher): the game as a non-Steam shortcut in the player's Steam library, with Navi as its
|
||||
icon and the game's title screen and logo as its pictures (launcher/gameart.py). Windows only.
|
||||
|
||||
Steam keeps the shortcuts of each account in <Steam>/userdata/<account id>/config/shortcuts.vdf, a binary key-value
|
||||
file: a map is 0x00 name 0x00 ... 0x08, a string 0x01 name 0x00 value 0x00, a 32-bit integer 0x02 name 0x00 value
|
||||
(0x07 is a 64-bit one). The file is the map "shortcuts" holding one map per shortcut, named "0", "1", ... Each holds
|
||||
appid, AppName, Exe, StartDir, icon, LaunchOptions and Steam's own fields; the other shortcuts are read and written
|
||||
back as they are, and a shortcut of ours already there (by its name or its id) is updated in place. Its pictures go
|
||||
into config/grid named by the shortcut's id: <id>p.png the cover, <id>.png the wide tile, <id>_hero.png the
|
||||
background, <id>_logo.png the logo, and the icon file the shortcut names. The id is CRC-32 of Exe and AppName as
|
||||
Steam writes them, with the top bit set.
|
||||
|
||||
Steam reads the file when it starts and writes it back when it closes, so a running Steam would neither see the new
|
||||
shortcut nor keep it: Steam is asked to close (steam.exe -shutdown), the files are written, and Steam starts again.
|
||||
|
||||
The shortcut starts the launcher's pythonw.exe on data/steam_play.pyw, which starts this launcher with --play: the game
|
||||
runs from its own folder with its console hidden, and the Python process waits for it, so Steam shows the game as
|
||||
running for as long as it is (the game's own Restart included). Starting ocarina.exe directly would open a console
|
||||
window beside the game, and NX2navi.exe ends a few seconds after it started the launcher.
|
||||
"""
|
||||
import os
|
||||
import shutil
|
||||
import struct
|
||||
import subprocess
|
||||
import time
|
||||
import zlib
|
||||
from pathlib import Path
|
||||
|
||||
NAME = 'The Legend of Zelda Ocarina of Time'
|
||||
PLAY_SCRIPT = 'steam_play.pyw'
|
||||
NO_WINDOW = 0x08000000 if os.name == 'nt' else 0
|
||||
STEAM64_BASE = 76561197960265728
|
||||
|
||||
PLAY_TEXT = '''"""Written by the NX2navi launcher (Customize Launcher > Add to Steam): Steam starts the game through this file.
|
||||
It runs the launcher of app/current.txt with --play, which starts the game and waits for it."""
|
||||
import os
|
||||
import runpy
|
||||
import sys
|
||||
|
||||
root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
with open(os.path.join(root, 'app', 'current.txt'), encoding='utf-8') as f:
|
||||
version = f.read().strip()
|
||||
os.environ['NX2NAVI_LAUNCHER_ROOT'] = root
|
||||
sys.argv = [os.path.join(root, 'app', version, 'launcher', 'start.pyw'), '--play'] + sys.argv[1:]
|
||||
runpy.run_path(sys.argv[0], run_name='__main__')
|
||||
'''
|
||||
|
||||
|
||||
class SteamError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- binary VDF
|
||||
|
||||
T_MAP, T_STR, T_INT, T_FLOAT, T_PTR, T_WSTR, T_COLOR, T_U64, T_END = 0, 1, 2, 3, 4, 5, 6, 7, 8
|
||||
|
||||
|
||||
def vdf_load(b):
|
||||
"""A binary VDF as a list of [type, key, value]; a map's value is such a list. Raises SteamError when malformed."""
|
||||
def cstr(at):
|
||||
end = b.index(b'\0', at)
|
||||
return b[at:end].decode('utf-8', 'surrogateescape'), end + 1
|
||||
|
||||
def walk(at, top):
|
||||
items = []
|
||||
while True:
|
||||
if at >= len(b):
|
||||
if top:
|
||||
return items, at
|
||||
raise SteamError('shortcuts.vdf ends inside a map')
|
||||
t = b[at]
|
||||
at += 1
|
||||
if t == T_END:
|
||||
return items, at
|
||||
key, at = cstr(at)
|
||||
if t == T_MAP:
|
||||
v, at = walk(at, False)
|
||||
elif t == T_STR:
|
||||
v, at = cstr(at)
|
||||
elif t in (T_INT, T_FLOAT, T_PTR, T_COLOR):
|
||||
v, at = b[at:at + 4], at + 4
|
||||
elif t == T_U64:
|
||||
v, at = b[at:at + 8], at + 8
|
||||
else:
|
||||
raise SteamError(f'shortcuts.vdf holds a value of type {t}')
|
||||
items.append([t, key, v])
|
||||
|
||||
try:
|
||||
items, at = walk(0, True)
|
||||
except ValueError:
|
||||
raise SteamError('shortcuts.vdf is cut short') from None
|
||||
return items
|
||||
|
||||
|
||||
def vdf_dump(items, top=True):
|
||||
out = bytearray()
|
||||
for t, key, v in items:
|
||||
out += bytes([t]) + key.encode('utf-8', 'surrogateescape') + b'\0'
|
||||
if t == T_MAP:
|
||||
out += vdf_dump(v, False)
|
||||
elif t == T_STR:
|
||||
out += v.encode('utf-8', 'surrogateescape') + b'\0'
|
||||
else:
|
||||
out += v
|
||||
if not top or items:
|
||||
out += bytes([T_END])
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _get(items, key):
|
||||
for it in items:
|
||||
if it[1].lower() == key.lower():
|
||||
return it
|
||||
return None
|
||||
|
||||
|
||||
def _set(items, t, key, v):
|
||||
it = _get(items, key)
|
||||
if it:
|
||||
it[0], it[2] = t, v
|
||||
else:
|
||||
items.append([t, key, v])
|
||||
|
||||
|
||||
def appid(exe, name):
|
||||
"""The id Steam gives a shortcut: CRC-32 of its Exe and AppName fields with the top bit set (unsigned)."""
|
||||
return (zlib.crc32((exe + name).encode('utf-8')) & 0xFFFFFFFF) | 0x80000000
|
||||
|
||||
|
||||
def add_shortcut(vdf_bytes, exe, start_dir, options, icon, name=NAME):
|
||||
"""shortcuts.vdf (b'' for none) with our shortcut added or updated; returns (new bytes, id)."""
|
||||
items = vdf_load(vdf_bytes) if vdf_bytes else []
|
||||
root = _get(items, 'shortcuts')
|
||||
if root is None or root[0] != T_MAP:
|
||||
items = [[T_MAP, 'shortcuts', []]]
|
||||
root = items[0]
|
||||
sid = appid(exe, name)
|
||||
entry = None
|
||||
for it in root[2]:
|
||||
if it[0] != T_MAP:
|
||||
continue
|
||||
n, a = _get(it[2], 'AppName'), _get(it[2], 'appid')
|
||||
if (n and n[0] == T_STR and n[2] == name) or (a and a[0] == T_INT and struct.unpack('<I', a[2])[0] == sid):
|
||||
entry = it[2]
|
||||
break
|
||||
if entry is None:
|
||||
used = {it[1] for it in root[2]}
|
||||
k = 0
|
||||
while str(k) in used:
|
||||
k += 1
|
||||
entry = []
|
||||
root[2].append([T_MAP, str(k), entry])
|
||||
for key, t, v in (('IsHidden', T_INT, 0), ('AllowDesktopConfig', T_INT, 1), ('AllowOverlay', T_INT, 1),
|
||||
('OpenVR', T_INT, 0), ('Devkit', T_INT, 0), ('DevkitGameID', T_STR, ''),
|
||||
('DevkitOverrideAppID', T_INT, 0), ('LastPlayTime', T_INT, 0), ('FlatpakAppID', T_STR, '')):
|
||||
entry.append([t, key, struct.pack('<I', v) if t == T_INT else v])
|
||||
entry.append([T_MAP, 'tags', []])
|
||||
_set(entry, T_INT, 'appid', struct.pack('<I', sid))
|
||||
_set(entry, T_STR, 'AppName', name)
|
||||
_set(entry, T_STR, 'Exe', exe)
|
||||
_set(entry, T_STR, 'StartDir', start_dir)
|
||||
_set(entry, T_STR, 'icon', icon)
|
||||
_set(entry, T_STR, 'LaunchOptions', options)
|
||||
# Steam's order: appid and the strings first, then the switches and tags
|
||||
order = ['appid', 'AppName', 'Exe', 'StartDir', 'icon', 'ShortcutPath', 'LaunchOptions']
|
||||
entry.sort(key=lambda it: order.index(it[1]) if it[1] in order else len(order) + (it[1] == 'tags'))
|
||||
if _get(entry, 'ShortcutPath') is None:
|
||||
entry.insert(5, [T_STR, 'ShortcutPath', ''])
|
||||
return vdf_dump(items), sid
|
||||
|
||||
|
||||
def has_shortcut(vdf_bytes, name=NAME):
|
||||
try:
|
||||
root = _get(vdf_load(vdf_bytes), 'shortcuts') if vdf_bytes else None
|
||||
except SteamError:
|
||||
return False
|
||||
return bool(root and root[0] == T_MAP and any(
|
||||
it[0] == T_MAP and (_get(it[2], 'AppName') or [0, 0, None])[2] == name for it in root[2]))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- Steam on this PC
|
||||
|
||||
def find_steam():
|
||||
"""(Steam's folder, steam.exe) from the registry, else the usual folder; None when Steam is not installed."""
|
||||
if os.name != 'nt':
|
||||
return None
|
||||
folder = exe = None
|
||||
try:
|
||||
import winreg
|
||||
with winreg.OpenKey(winreg.HKEY_CURRENT_USER, r'Software\Valve\Steam') as k:
|
||||
for name in ('SteamPath', 'SteamExe'):
|
||||
try:
|
||||
v = winreg.QueryValueEx(k, name)[0]
|
||||
except OSError:
|
||||
continue
|
||||
if name == 'SteamPath':
|
||||
folder = Path(v)
|
||||
else:
|
||||
exe = Path(v)
|
||||
except OSError:
|
||||
pass
|
||||
if folder is None:
|
||||
folder = Path(os.environ.get('ProgramFiles(x86)', r'C:\Program Files (x86)')) / 'Steam'
|
||||
exe = exe or folder / 'steam.exe'
|
||||
if not (folder / 'userdata').is_dir() and not exe.is_file():
|
||||
return None
|
||||
return folder, exe
|
||||
|
||||
|
||||
def accounts(folder):
|
||||
"""The config folders of the accounts that signed in to Steam on this PC (config/loginusers.vdf), else of every
|
||||
account folder under userdata."""
|
||||
ud = Path(folder) / 'userdata'
|
||||
if not ud.is_dir():
|
||||
return []
|
||||
dirs = {p.name: p / 'config' for p in ud.iterdir() if p.is_dir() and p.name.isdigit() and p.name != '0'}
|
||||
signed = []
|
||||
try:
|
||||
text = (Path(folder) / 'config' / 'loginusers.vdf').read_text(encoding='utf-8', errors='replace')
|
||||
import re
|
||||
signed = [str(int(i) - STEAM64_BASE) for i in re.findall(r'"(7656119\d{10})"', text)]
|
||||
except (OSError, ValueError):
|
||||
pass
|
||||
picked = [dirs[a] for a in signed if a in dirs] or list(dirs.values())
|
||||
return sorted(set(picked))
|
||||
|
||||
|
||||
def added(folder=None):
|
||||
"""True when an account of this PC has our shortcut."""
|
||||
found = (folder, None) if folder else find_steam()
|
||||
if not found:
|
||||
return False
|
||||
for cfg in accounts(found[0]):
|
||||
try:
|
||||
if has_shortcut((cfg / 'shortcuts.vdf').read_bytes()):
|
||||
return True
|
||||
except OSError:
|
||||
pass
|
||||
return False
|
||||
|
||||
|
||||
def steam_running():
|
||||
if os.name != 'nt':
|
||||
return False
|
||||
try:
|
||||
r = subprocess.run(['tasklist', '/FI', 'IMAGENAME eq steam.exe', '/NH', '/FO', 'CSV'], capture_output=True,
|
||||
text=True, creationflags=NO_WINDOW, timeout=15)
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
return False
|
||||
return '"steam.exe"' in r.stdout.lower()
|
||||
|
||||
|
||||
def close_steam(exe, timeout=45.0):
|
||||
"""Ask Steam to close and wait for it; False when it is still running after timeout seconds."""
|
||||
subprocess.Popen([str(exe), '-shutdown'], stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL, creationflags=NO_WINDOW, close_fds=True)
|
||||
end = time.time() + timeout
|
||||
while time.time() < end:
|
||||
time.sleep(1)
|
||||
if not steam_running():
|
||||
time.sleep(2) # it writes its files on the way out
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def write_atomic(path, data):
|
||||
tmp = path.with_name(path.name + '.nx2navi-tmp')
|
||||
tmp.write_bytes(data)
|
||||
os.replace(tmp, path)
|
||||
|
||||
|
||||
def add(layout, game_dir, dump='', log=print, restart=True):
|
||||
"""Add the game to Steam (or update the shortcut): the pictures from the player's game, the shortcut in every
|
||||
account's shortcuts.vdf, Steam closed first and started again. Returns the sentence the launcher shows."""
|
||||
from . import gameart
|
||||
from .ui import art
|
||||
found = find_steam()
|
||||
if not found:
|
||||
raise SteamError('Steam is not installed on this PC.')
|
||||
folder, exe_steam = found
|
||||
cfgs = accounts(folder)
|
||||
if not cfgs:
|
||||
raise SteamError('Steam has no account on this PC yet: sign in to Steam once, then try again.')
|
||||
game = Path(game_dir) / 'ocarina.exe'
|
||||
if not game.is_file():
|
||||
raise SteamError('The game is not built yet.')
|
||||
try:
|
||||
pics = gameart.make(layout, game_dir, dump)
|
||||
except gameart.ArtError as e:
|
||||
raise SteamError(f'The pictures could not be made: {e}.') from None
|
||||
script = layout.data / PLAY_SCRIPT
|
||||
write_atomic(script, PLAY_TEXT.encode('utf-8'))
|
||||
python = Path(layout.python) / 'pythonw.exe'
|
||||
if not python.is_file(): # a checkout runs under its own interpreter
|
||||
import sys
|
||||
python = Path(sys.executable).with_name('pythonw.exe')
|
||||
exe, start = f'"{python}"', f'"{layout.root}"'
|
||||
options = f'"{script}"'
|
||||
was_running = steam_running()
|
||||
if was_running:
|
||||
log('steam: asking Steam to close')
|
||||
if not close_steam(exe_steam):
|
||||
raise SteamError('Steam did not close. Close Steam yourself, then select Add to Steam again.')
|
||||
sid = appid(exe, NAME)
|
||||
try:
|
||||
for cfg in cfgs:
|
||||
grid = cfg / 'grid'
|
||||
grid.mkdir(parents=True, exist_ok=True)
|
||||
icon = grid / f'{sid}_icon.png'
|
||||
for name, file in (('icon', icon.name), ('portrait', f'{sid}p.png'), ('wide', f'{sid}.png'),
|
||||
('hero', f'{sid}_hero.png'), ('logo', f'{sid}_logo.png')):
|
||||
write_atomic(grid / file, art.png_bytes(pics[name]))
|
||||
vdf = cfg / 'shortcuts.vdf'
|
||||
try:
|
||||
old = vdf.read_bytes()
|
||||
except FileNotFoundError:
|
||||
old = b''
|
||||
new, got = add_shortcut(old, exe, start, options, str(icon))
|
||||
if old and not (cfg / 'shortcuts.vdf.nx2navi-backup').exists():
|
||||
shutil.copyfile(vdf, cfg / 'shortcuts.vdf.nx2navi-backup')
|
||||
write_atomic(vdf, new)
|
||||
log(f'steam: shortcut {got} in {vdf}')
|
||||
finally: # Steam comes back also when a file could not be written
|
||||
if was_running and restart:
|
||||
subprocess.Popen([str(exe_steam)], stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL, creationflags=0x00000008, close_fds=True) # detached
|
||||
if was_running and restart:
|
||||
return f'Added to Steam as {NAME}. Steam is starting again with it in the library.'
|
||||
return f'Added to Steam as {NAME}. It is in the library the next time Steam starts.'
|
||||
@@ -9,7 +9,7 @@ flat cells and the orange choices are canvas items. Screens:
|
||||
|
||||
first_run the dump, the install folder, three choices and Begin Installation
|
||||
home Play, Updates, the game folder, Saves, Rebuild the Game, Customize Launcher, Settings
|
||||
customize the colour scheme, the player's own background picture, animations and sounds
|
||||
customize the colour scheme, the player's own background picture, animations and sounds; Add to Steam
|
||||
saves Export Save (a zip for a bug report or another player), Import a Save (such a zip), a save folder
|
||||
job the steps of an install, update or rebuild, with their progress, and Cancel; after a failure Try Again
|
||||
and Copy Report
|
||||
@@ -35,7 +35,7 @@ import tkinter.font as tkfont
|
||||
from pathlib import Path
|
||||
from tkinter import filedialog
|
||||
|
||||
from .. import NAME, build, feed, gameini, gamefiles, jobs, saves, versions, winsys
|
||||
from .. import NAME, build, feed, gameini, gamefiles, jobs, saves, steam, versions, winsys
|
||||
from ..context import Context
|
||||
from . import art, picture, sound
|
||||
|
||||
@@ -170,6 +170,8 @@ class App:
|
||||
self.look_message = '' # why the last background picture was not taken, in Customize Launcher's descriptions
|
||||
self.save_message = '' # the last import's result, shown in the import rows' descriptions
|
||||
self.export_message = '' # the last Export Save result, shown in its description
|
||||
self.steam_message = '' # the last Add to Steam result, shown in its description
|
||||
self.steam_busy = False # Add to Steam is making the pictures or writing Steam's files
|
||||
self.dumps = None
|
||||
self.dump_search = None
|
||||
self.uninstall_game = False
|
||||
@@ -956,11 +958,75 @@ class App:
|
||||
left=lambda: sounds(-1), right=lambda: sounds(1),
|
||||
desc=('The game\'s select and back sounds. Low by default.',)),
|
||||
Row('divider'),
|
||||
]
|
||||
if os.name == 'nt':
|
||||
rows += [self.steam_row(), Row('divider')]
|
||||
rows += [
|
||||
Row('action', 'Back', on=lambda: self.show('home', self.home_row('Customize Launcher')), height=40,
|
||||
desc=('Back to the main menu.',)),
|
||||
]
|
||||
return rows
|
||||
|
||||
def steam_row(self):
|
||||
"""Add to Steam: the game in the Steam library, started from there without the launcher (launcher/steam.py)."""
|
||||
found = steam.find_steam()
|
||||
desc = ('Add Ocarina of Time to your Steam library as The Legend of Zelda Ocarina of Time, with Navi as its '
|
||||
'icon and the title screen\'s moonlit field and logo as its pictures, all taken from your game. '
|
||||
'Playing it from Steam starts the game straight away.',
|
||||
'Steam closes while the shortcut is written and starts again with it. Select the row again after an '
|
||||
'update of the game to refresh it.')
|
||||
if self.steam_message:
|
||||
desc = (self.steam_message,) + desc
|
||||
if self.steam_busy:
|
||||
value, style = 'Adding…', 'dim'
|
||||
elif not found:
|
||||
value, style = 'Steam not found', 'dim'
|
||||
else:
|
||||
value, style = ('Added' if steam.added(found[0]) else ''), 'arrow'
|
||||
return Row('value', 'Add to Steam', value, style, on=self.add_to_steam, desc=desc)
|
||||
|
||||
def add_to_steam(self):
|
||||
import threading
|
||||
if self.steam_busy:
|
||||
return
|
||||
gd = self.settings.get('game_dir')
|
||||
if not steam.find_steam():
|
||||
self.steam_message = 'Steam is not installed on this PC.'
|
||||
self.sounds.play('back')
|
||||
self.show('customize', self.focus)
|
||||
return
|
||||
if not gd or not (Path(gd) / 'ocarina.exe').is_file():
|
||||
self.steam_message = 'The game is not built yet.'
|
||||
self.sounds.play('back')
|
||||
self.show('customize', self.focus)
|
||||
return
|
||||
self.steam_busy = True
|
||||
self.steam_message = 'Making the pictures from your game and adding it to Steam…'
|
||||
self.sounds.play('select')
|
||||
self.show('customize', self.focus)
|
||||
dump = self.settings.get('dump')
|
||||
|
||||
def work():
|
||||
try:
|
||||
text, ok = steam.add(self.layout, gd, dump, self.ctx.log), True
|
||||
except steam.SteamError as e:
|
||||
text, ok = str(e), False
|
||||
except Exception as e: # a Steam folder in a state nobody expected: say so instead of dying
|
||||
import traceback
|
||||
self.ctx.log('steam: ' + traceback.format_exc())
|
||||
text, ok = f'Adding to Steam failed: {e}', False
|
||||
self.ctx.log(f'steam: {text}')
|
||||
self.root.after(0, lambda: self.steam_done(text, ok))
|
||||
threading.Thread(target=work, daemon=True).start()
|
||||
|
||||
def steam_done(self, text, ok):
|
||||
self.steam_busy = False
|
||||
self.steam_message = text
|
||||
if not ok:
|
||||
self.sounds.play('back')
|
||||
if self.screen == 'customize':
|
||||
self.show('customize', self.focus)
|
||||
|
||||
def choose_background(self):
|
||||
start = self.settings.get('background')
|
||||
start = str(Path(start).parent) if start and Path(start).parent.is_dir() else \
|
||||
@@ -1987,6 +2053,8 @@ def parse_args(argv):
|
||||
ap.add_argument('--root', help='the launcher folder (tests)')
|
||||
ap.add_argument('--selftest', action='store_true', help='draw every screen and close')
|
||||
ap.add_argument('--shots', help='with --selftest: save a PNG of each screen into this folder')
|
||||
ap.add_argument('--play', action='store_true',
|
||||
help='start the game without the window and wait for it (the Steam shortcut, launcher/steam.py)')
|
||||
a, _unknown = ap.parse_known_args(argv)
|
||||
return a
|
||||
|
||||
@@ -2016,12 +2084,40 @@ def single_instance(layout, wait):
|
||||
time.sleep(0.25)
|
||||
|
||||
|
||||
def play_only(ctx):
|
||||
"""--play, from the Steam shortcut (launcher/steam.py): the game starts from its folder with its console hidden, as
|
||||
Play starts it, and this process waits until it and the copies its own Restart starts have ended, so Steam shows
|
||||
the game as running all that time. None when there is no built game: the launcher's window opens instead."""
|
||||
gd = ctx.settings.get('game_dir')
|
||||
exe = Path(gd) / 'ocarina.exe' if gd else None
|
||||
if not exe or not exe.is_file():
|
||||
return None
|
||||
if saves.game_running(gd):
|
||||
ctx.log('play from Steam: the game is already running')
|
||||
return 0
|
||||
try:
|
||||
proc = winsys.start_game(exe)
|
||||
except OSError as e:
|
||||
ctx.log(f'play from Steam: {exe}: {e}')
|
||||
return None
|
||||
ctx.log(f'play from Steam: {exe}')
|
||||
time.sleep(2)
|
||||
while proc.poll() is None or saves.game_running(gd):
|
||||
time.sleep(1)
|
||||
ctx.log(f'play from Steam: the game ended (code {proc.returncode})')
|
||||
return 0
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
a = parse_args(sys.argv[1:] if argv is None else argv)
|
||||
winsys.dpi_aware()
|
||||
winsys.app_id()
|
||||
from ..paths import Layout
|
||||
ctx = Context(Layout(root=a.root) if a.root else None)
|
||||
if a.play and not a.selftest:
|
||||
code = play_only(ctx)
|
||||
if code is not None:
|
||||
return code
|
||||
lock = None
|
||||
if not a.selftest:
|
||||
lock = single_instance(ctx.layout, 20 if (a.continue_update or a.rolled_back) else 2)
|
||||
|
||||
@@ -78,6 +78,13 @@
|
||||
# recorder_core = auto
|
||||
# Submissions handed to the GPU queue together in one call, 1 to 16 (1 = each one at once).
|
||||
# submit_batch = 4
|
||||
# Occlusion query results are read back from the video card in batches, 0 or 1. Default: on. 0 reads each query with a
|
||||
# call of its own, as before, which on AMD's Windows driver held every frame's "GPU done" back by about 4 ms.
|
||||
# oq_batch = 1
|
||||
# How many milliseconds a submit call waits for the game's own signal that the frame's data is written, when the
|
||||
# frame's commands begin with a wait for that signal (0 to 100; 0: the frame is set aside at once and drawn at its
|
||||
# present, as before).
|
||||
# gate_wait_ms = 10
|
||||
# Dynamic resolution, 0 or 1, applied at once: with a DLSS upscaling mode, the scene resolution follows the GPU time
|
||||
# per frame toward dyn_res_target milliseconds (1 to 100), one DLSS step at a time, each held for dyn_res_step frames
|
||||
# (8 to 600), from the DLSS mode set down to performance. Default: off.
|
||||
@@ -292,6 +299,11 @@
|
||||
# sparkles), black and white blocks on her and on Link, shadows blown up. Default: on, except on NVIDIA's driver,
|
||||
# which keeps them anyway; 1 also asks NVIDIA's driver. The first start after a change compiles the pipelines again.
|
||||
# vk_nan_preserve = 1
|
||||
# Test for NaN in the game's shaders on the bits of the number instead of as a float, 0 or 1, so that no shader
|
||||
# compiler can leave a test out. Default: on with AMD's driver for RX 5000 and RX 6000 cards (the one whose shader
|
||||
# compiler reads "AMD proprietary shader compiler" in the log), where the black and white blocks were reported; off
|
||||
# elsewhere. The first start after a change compiles the pipelines again.
|
||||
# vk_nan_bits = 1
|
||||
|
||||
[display]
|
||||
# Device profile: auto (the Steam Deck when this is one), desktop or deck. On a Steam Deck the keys this file does not
|
||||
|
||||
@@ -48,7 +48,7 @@ static const char *const known[] = {
|
||||
"HDR_EXPOSURE", "HDR_FLARE", "HDR_GAMMA", "HDR_GAME_NITS", "HDR_GRADE", "HDR_HIGHLIGHT_SAT", "HDR_HIGHLIGHTS", "HDR_HUE",
|
||||
"HDR_LIVE",
|
||||
"HDR_PEAK_NITS", "HDR_SATURATION", "HDR_SHADOWS", "HDR_TONEMAP", "HDR_UI_NITS", "HIGHFPS_FIX", "INPUT", "INPUT_RECORD", "INPUT_SCRIPT", "LANG", "LIB_PREBUILD",
|
||||
"BIG_BUILD_FREE_MB", "BIG_BUILD_MAX_MB", "LIB_PREBUILD_BIG", "LIB_PREBUILD_BIG_BG", "LIB_PREBUILD_KEEP", "LIB_PREBUILD_THREADS", "LIB_WARM", "PIPE_LINK_MS", "FRAME_HOLD_MS", "PACE_SMOOTH", "LOCALE", "LOD_GUARD", "LSTICK_DEADZONE", "MEMORY", "MEM_CENSUS", "MENU_LANG", "MENU_SOUND_VOLUME", "MENU_TEST",
|
||||
"BIG_BUILD_FREE_MB", "BIG_BUILD_MAX_MB", "LIB_PREBUILD_BIG", "LIB_PREBUILD_BIG_BG", "LIB_PREBUILD_KEEP", "LIB_PREBUILD_THREADS", "LIB_WARM", "PIPE_LINK_MS", "FRAME_HOLD_MS", "PACE_SMOOTH", "OQ_BATCH", "GATE_WAIT_MS", "LOCALE", "LOD_GUARD", "LSTICK_DEADZONE", "MEMORY", "MEM_CENSUS", "MENU_LANG", "MENU_SOUND_VOLUME", "MENU_TEST",
|
||||
"MENU_CURSOR", "MENU_CURSOR_LOG", "MENU_TEST_CLICK", "MENU_TEST_DIR", "MENU_TEST_SET", "MINIMAL_LAYOUT", "MICROPHONE", "MIRROR_GAP", "MODS_DIR", "MODS_DISABLED", "MODS_ORDER", "MODS_REBUILD", "MONITOR", "MOTION", "MOUSE_MODE", "MOUSE_DIRECT_PATCH", "MOUSE_SENS", "MOVIE_HOST", "MOVIE_PACE",
|
||||
"MVOBJ_ALL", "MVOBJ_BANKS", "MVOBJ_EQUAL", "MVOBJ_EXACT", "MVOBJ_EXEC", "MVOBJ_NOC4", "MVOBJ_ONLY", "MVOBJ_TIMES", "MVPREV_CHECK", "MVPREV_GROW", "MVPREV_SKIP", "MVPREV_TRUST", "MV_CHECK",
|
||||
"MV_SURVEY",
|
||||
@@ -65,7 +65,7 @@ static const char *const known[] = {
|
||||
"SHOTS_FULL", "SPEED_PROBE", "SPIN", "SPIN_HANDOFF_US", "SPLIT_DRAWS", "SPLIT_IDLE", "SPLIT_IN_PASS", "SPLIT_US", "SPV_MARK_TEST", "SPV_OVERRIDE", "STICK_DEADZONE", "STICK_SENS",
|
||||
"SWKBD_ANSWER",
|
||||
"SET_AT", "SWKBD_TEST", "TAA", "TEST_PAD", "TEST_PAD_AT", "TEST_PREPARE_SHADERS", "TEX_ALIAS", "DEPTH_ALIAS", "DEPTH_ALIAS_LOG", "TEX_DISUSE", "TEX_GEN_UNDO", "TEX_MEMO", "TEX_VRAM_PRESSURE", "TEX_RELEASED_MB", "TEX_REUSE", "TEX_COMMIT_FREE_MB", "TEX_PROC_COMMIT_MB", "TEX_EVICT", "TEX_COMMIT_EVICT", "MIRROR_EVICT", "VRAM_BUDGET_MB", "TIMER_1MS", "TRACE", "TXD_TRANSPOSE", "UBO_SNAPSHOT", "EVO_TRACE", "EVENT_AT_SYNC", "UPSCALER", "VK_DYN_A2C", "VK_DYN_VERTEX", "VK_NAN_PRESERVE", "VK_NOOPT", "VK_NO_FLOAT64",
|
||||
"VK_PRESENT", "VK_RECONVERGE", "VK_RECORDER", "VK_SSBO_ALIGN", "VK_SUBGROUP", "VK_SUBGROUP_PIN", "VK_VALIDATION", "VOLUME", "VRAM_HEADROOM", "VS_OUTS",
|
||||
"VK_PRESENT", "VK_RECONVERGE", "VK_NAN_BITS", "VK_RECORDER", "VK_SSBO_ALIGN", "VK_SUBGROUP", "VK_SUBGROUP_PIN", "VK_VALIDATION", "VOLUME", "VRAM_HEADROOM", "VS_OUTS",
|
||||
"WALK_KEY", "WALK_SCALE", "WB_EXACT", "WB_FENCE_LIMIT", "WB_MERGE", "WB_NO_REUP", "WB_MERGE_MAX", "WB_PROBE", "WB_SPLIT", "WB_LAZY", "WB_LAZY_TOUCH", "WINDOW_SIZE", "XLATE", "XLATE_ID", "XLATE_KEEP", "XLATE_PROCS",
|
||||
"ZSTD_CHECK", "ZSTD_NATIVE",
|
||||
"BIND_REUSE", "DYN_DELTA", "STG_ARENA", "SUBMIT_BATCH", "ASTC_GPU_DESWIZZLE", "ASTC_SCRATCH_KEEP", "ASTC_ARENA",
|
||||
@@ -118,14 +118,14 @@ static const struct { const char *key, *section; } sections[] = {
|
||||
{"TIMER_1MS", "performance"}, {"PIPE_WAIT", "performance"}, {"PIPE_SKIP", "performance"}, {"ONESHOT_BUDGET", "performance"},
|
||||
{"PIPE_FRAME_WAIT_MS", "performance"}, {"PIPE_SKIP_STEADY", "performance"},
|
||||
{"CACHE_DIE", "performance"}, {"ONE_PER_CORE", "performance"},
|
||||
{"LIB_PREBUILD", "performance"}, {"LIB_PREBUILD_THREADS", "performance"}, {"LIB_WARM", "performance"}, {"PIPE_LINK_MS", "performance"}, {"PIPE_LINK_DRAW_MS", "performance"}, {"PIPE_AHEAD", "performance"}, {"PIPE_FAST_WORKERS", "performance"}, {"GPL_FEEDBACK", "performance"}, {"FRAME_HOLD_MS", "performance"}, {"PACE_SMOOTH", "performance"}, {"PIPE_PREBUILD", "performance"},
|
||||
{"LIB_PREBUILD", "performance"}, {"LIB_PREBUILD_THREADS", "performance"}, {"LIB_WARM", "performance"}, {"PIPE_LINK_MS", "performance"}, {"PIPE_LINK_DRAW_MS", "performance"}, {"PIPE_AHEAD", "performance"}, {"PIPE_FAST_WORKERS", "performance"}, {"GPL_FEEDBACK", "performance"}, {"FRAME_HOLD_MS", "performance"}, {"PACE_SMOOTH", "performance"}, {"OQ_BATCH", "performance"}, {"GATE_WAIT_MS", "performance"}, {"PIPE_PREBUILD", "performance"},
|
||||
{"COMPUTE_KEEP", "performance"},
|
||||
{"PREWARM_XLATE", "performance"}, {"GPL", "performance"}, {"ASTC", "performance"}, {"MEMORY", "performance"},
|
||||
{"TEX_ALIAS", "performance"}, {"DEPTH_ALIAS", "performance"}, {"DEPTH_ALIAS_LOG", "diagnostics"}, {"TEX_DISUSE", "performance"}, {"TEX_GEN_UNDO", "performance"}, {"TEX_REUSE", "performance"}, {"TEX_VRAM_PRESSURE", "performance"}, {"TEX_PROC_COMMIT_MB", "performance"}, {"ASTC_BC", "performance"}, {"LOD_GUARD", "performance"}, {"FIX_C0", "performance"}, {"FIX_ZERO", "performance"},
|
||||
{"BIND_SKIP", "performance"}, {"SHADER_CACHE_FLUSH", "performance"}, {"SHADER_CARRY", "performance"}, {"SHADER_MEMO", "performance"}, {"PIPE_MEMO", "performance"}, {"TEX_MEMO", "performance"}, {"DESC_TEMPLATE", "performance"},
|
||||
{"PUSH_SKIP", "performance"}, {"SPLIT_DRAWS", "performance"}, {"SPLIT_IDLE", "performance"}, {"SPLIT_IN_PASS", "performance"}, {"SPLIT_US", "performance"}, {"VK_VALIDATION", "performance"},
|
||||
{"VK_DYN_VERTEX", "performance"}, {"VK_DYN_A2C", "performance"}, {"VK_SUBGROUP_PIN", "performance"}, {"VK_RECONVERGE", "performance"},
|
||||
{"VK_NAN_PRESERVE", "performance"},
|
||||
{"VK_NAN_PRESERVE", "performance"}, {"VK_NAN_BITS", "performance"},
|
||||
{"TRACE", "diagnostics"}, {"FLIGHT", "diagnostics"}, {"PASS_CRUMBS", "diagnostics"}, {"FLIGHT_SPIKE_MS", "diagnostics"},
|
||||
{"WB_FENCE_LIMIT", "diagnostics"}, {"WB_MERGE", "diagnostics"}, {"WB_EXACT", "diagnostics"}, {"MEM_CENSUS", "diagnostics"}, {"WB_NO_REUP", "diagnostics"}, {"WB_PROBE", "diagnostics"}, {"WB_SPLIT", "performance"}, {"SHADOW_BULK_WRITE", "performance"}, {"WB_LAZY", "performance"}, {"BARRIER_GAME", "performance"}, {"WB_LAZY_TOUCH", "diagnostics"}, {"FLUSH_UPLOAD", "diagnostics"}, {"GPA_TWIN_MAX_KB", "diagnostics"},
|
||||
{"WB_MERGE_MAX", "diagnostics"}, {"MV_SURVEY", "diagnostics"}, {"TEXUP_LOG_MS", "diagnostics"}, {"VA_REPORT", "diagnostics"},
|
||||
@@ -239,6 +239,8 @@ static const CfgMeta metas[] = {
|
||||
{"PIPE_FAST_WORKERS", "1", "0|1", RESTART}, // vk_pipeline.c lib_worker, fast_take
|
||||
{"FRAME_HOLD_MS", "120", "0..500", RESTART}, // vk_backend.c frame_hold
|
||||
{"PACE_SMOOTH", "1", "0|1", RESTART}, // vk_backend.c pace_smooth
|
||||
{"OQ_BATCH", "1", "0|1", RESTART}, // vk_exec_xfer.c oq_take
|
||||
{"GATE_WAIT_MS", "10", "0..100", RESTART}, // vk_exec.c vk_exec_gate_wait
|
||||
{"LIB_PREBUILD", "1", "0|1", RESTART}, {"LIB_PREBUILD_THREADS", "", "1..64", RESTART},
|
||||
{"LIB_WARM", "1", "0|1|force", RESTART}, // vk_pipeline.c lw_start: the driver's cache of stage libraries, warmed when cold
|
||||
{"PIPE_PREBUILD", "1", "0|1", RESTART}, {"PREWARM_XLATE", "", "1..64", RESTART}, {"GPL", "1", "0|1", RESTART},
|
||||
@@ -265,6 +267,7 @@ static const CfgMeta metas[] = {
|
||||
{"VK_DYN_A2C", "", "0|1", RESTART}, {"VK_SUBGROUP_PIN", "1", "0|1", RESTART},
|
||||
{"VK_RECONVERGE", "1", "0|1", RESTART}, // vk_pipeline.c spirv_reconverge
|
||||
{"VK_NAN_PRESERVE", "", "0|1", RESTART}, // vk_pipeline.c spirv_nan_preserve; unset: every driver but NVIDIA's
|
||||
{"VK_NAN_BITS", "", "0|1", RESTART}, // vk_pipeline.c spirv_nan_bits; unset: AMD's driver for RDNA 1 and 2
|
||||
{"VK_NOOPT", "0", "0|1", RESTART}, // vk_pipeline.c noopt_flag
|
||||
// diagnostics
|
||||
{"TRACE", "", NULL, RESTART}, {"FLIGHT", "0", "0|1", RESTART}, {"FLIGHT_SPIKE_MS", "100", "0..60000", RESTART},
|
||||
|
||||
@@ -387,6 +387,7 @@ static uint64_t submit_sync(const uint64_t *handles, int n) {
|
||||
if (!vk_exec_held()) ph_flush(&PH.after);
|
||||
}
|
||||
vk_present_drain();
|
||||
vk_exec_gate_wait(handles, n); // a submission that begins with a wait for the CPU's signal: before its copies
|
||||
vk_texture_new_submit();
|
||||
uint64_t f = vk_sync_submit(handles, n);
|
||||
// The submission stopped at a wait whose signal the CPU gave while it was being replayed (vk_exec_xfer.c "events
|
||||
|
||||
@@ -1192,6 +1192,72 @@ static int cont_ready(const Cont *c) {
|
||||
return !c->at || vk_exec_event_met(nvn_op_payload(c->at)) || vk_exec_submit_seq - c->since > CONT_MAX;
|
||||
}
|
||||
|
||||
// ---- the gate: a submission that begins with a wait for the CPU's signal (NX2NAVI_GATE_WAIT_MS, default 10; 0 off)
|
||||
// Ocarina's main queue begins with a wait for an event the game signals from the CPU once it has written what the
|
||||
// frame reads (vk_exec_xfer.c "events the CPU signals"). The render thread submits that queue without waiting for the
|
||||
// signal, and polls the event afterwards, a millisecond at a time, before it presents. In Hyrule Field the signal
|
||||
// came 1 to 2 ms after the submit call in about every other frame: the main thread first waits for the previous
|
||||
// frame's fence and reads back what it needs. The submission then took its copies (the mirror sync and the uniform
|
||||
// snapshot, 1.1 to 1.7 ms), met the wait unmet and was held; the render thread polled, presented, and the present
|
||||
// replayed the frame (vk_backend.c held_resume) under a value the game's fence does not stand for, so the game's next
|
||||
// wait for that fence returned with the GPU still at work and the frame after was replayed inside its submit call
|
||||
// again: frames of 23 and 15 ms in turn where each takes 17.
|
||||
// So when a submission would stop at such a wait before it has recorded anything, the submit call waits for the
|
||||
// signal here, before the copies are taken, as the console's GPU waits at the wait itself: the copies are then
|
||||
// taken after the game has finished writing, and the frame is replayed in the call the game's fence stands for.
|
||||
// Only state may precede the wait in the command buffer, and the event must be one the CPU has signalled before;
|
||||
// the wait ends after NX2NAVI_GATE_WAIT_MS at the latest, and the submission is then held as before.
|
||||
static struct { uint64_t n, ns, late; } GW; // since the last perf line: waits, their time, waits that ran out
|
||||
|
||||
static const void *gate_first_wait(const op_hdr *h) {
|
||||
for (int k = 0; k < 256; k++) {
|
||||
if (h->size < sizeof *h || (h->size & 7) || h->op >= OP_COUNT_) return NULL;
|
||||
const void *a = nvn_op_payload(h);
|
||||
switch (h->op) {
|
||||
case OP_WAIT_EVENT: return a;
|
||||
case OP_JUMP: h = (const op_hdr *)(uintptr_t)((const p_jump *)a)->target; continue;
|
||||
// state the walk only stores: nothing is recorded and nothing reaches the CPU before the wait
|
||||
case OP_BEGIN: case OP_SET_RT: case OP_VIEWPORTS: case OP_SCISSOR: case OP_DEPTH_RANGE: case OP_BIND_PROGRAM:
|
||||
case OP_BIND_STATE: case OP_BIND_VATTR: case OP_BIND_VSTREAM: case OP_BIND_VBUF: case OP_BIND_UBO:
|
||||
case OP_BIND_SSBO: case OP_BIND_TEX: case OP_BIND_SAMP: case OP_BIND_IMG: case OP_STENCIL: case OP_POLY_OFFSET:
|
||||
case OP_SET_TEX_POOL: case OP_SET_SAMP_POOL: case OP_COPY_STRIDE: case OP_BLEND_COLOR: case OP_DEPTH_CLAMP:
|
||||
case OP_RASTER_DISCARD: case OP_PRIM_RESTART: case OP_PUSH_CC:
|
||||
break;
|
||||
default: return NULL; // a draw, a copy, a signal, a call: the walk decides, as before
|
||||
}
|
||||
h = nvn_op_next(h);
|
||||
}
|
||||
return NULL;
|
||||
}
|
||||
|
||||
void vk_exec_gate_wait(const uint64_t *handles, int n) {
|
||||
static int64_t cap_ns = -1;
|
||||
if (cap_ns < 0) {
|
||||
int ms = cfg_int("NX2NAVI_GATE_WAIT_MS");
|
||||
cap_ns = ms > 0 ? (int64_t)ms * 1000000 : 0;
|
||||
}
|
||||
if (!cap_ns || n < 1 || !handles[0] || !gfx_submit_queue || !evo_enabled()) return;
|
||||
const void *w = gate_first_wait(nvn_op_first(handles[0]));
|
||||
if (!w || vk_exec_event_holds_now(w) || !vk_exec_event_cpu_known((const void *)((const p_wait_event *)w)->val)) return;
|
||||
uint64_t t0 = kern_now_ns(), now = t0;
|
||||
for (unsigned i = 0; !vk_exec_event_holds_now(w); i++) {
|
||||
if ((i & 15) == 15 && (now = kern_now_ns()) - t0 >= (uint64_t)cap_ns) {
|
||||
GW.late++;
|
||||
break;
|
||||
}
|
||||
if (i < 20000) plat_pause();
|
||||
else plat_yield();
|
||||
}
|
||||
GW.n++, GW.ns += kern_now_ns() - t0;
|
||||
if (GW.n >= 600) {
|
||||
if (rt_trace & RT_TRACE_PERF)
|
||||
rt_log("perf: gate: %llu submissions waited for the CPU's signal before their copies, %.2f ms on average; "
|
||||
"%llu waited %lld ms without it and were held", (unsigned long long)GW.n, (double)GW.ns / (double)GW.n / 1e6,
|
||||
(unsigned long long)GW.late, (long long)(cap_ns / 1000000));
|
||||
GW.n = GW.ns = GW.late = 0;
|
||||
}
|
||||
}
|
||||
|
||||
int vk_exec_held_ready(void) {
|
||||
for (int s = 0; evo_pending && s < EVO_QUEUES; s++)
|
||||
if (evq[s].head && cont_ready(evq[s].head)) return 1;
|
||||
|
||||
@@ -252,6 +252,11 @@ extern int vk_exec_signalled; // set by every replayed SignalEve
|
||||
// Events the CPU signals (vk_exec_xfer.c "events the CPU signals").
|
||||
void vk_exec_event_cpu(const void *event); // any thread: nvnEventSignal is about to write the event's value
|
||||
void vk_exec_event_sync(int begin); // vk_exec_record: its mirror sync and uniform copy are taken now / it ends
|
||||
int vk_exec_event_cpu_known(const void *event); // the CPU has signalled this event before
|
||||
int vk_exec_event_holds_now(const void *payload); // an OP_WAIT_EVENT's condition on the value in memory now
|
||||
// vk_backend.c submit_sync, before a submission is recorded: waits for the CPU's signal when the submission begins
|
||||
// with a wait for it (vk_exec.c "the gate").
|
||||
void vk_exec_gate_wait(const uint64_t *handles, int n);
|
||||
int vk_exec_event_at_sync(void); // NX2NAVI_EVENT_AT_SYNC
|
||||
uint64_t vk_exec_event_late_take(void); // waits only a CPU signal after the copies would have passed, since the last call
|
||||
// Since the last call, which resets them: waits, waits reached before their signal, those of them a later replayed
|
||||
|
||||
@@ -265,38 +265,103 @@ void vk_exec_oq_draw(VkExec *ex) {
|
||||
|
||||
void vk_exec_oq_pass_end(VkExec *ex) { oq_close(ex); }
|
||||
|
||||
static void oq_batch_drop(void); // the results kept by the completion thread (oq_take)
|
||||
|
||||
void vk_exec_xfer_shutdown(void) {
|
||||
if (oq_state > 0) vkDestroyQueryPool(vk_ctx()->device, oq_pool, NULL);
|
||||
oq_pool = VK_NULL_HANDLE, oq_state = 0, oq_next = 0, oq_counting = oq_reported = oq_keep = 0, oq_open = -1;
|
||||
oq_nseg = 0, oq_lost = 0, oq_count = 0;
|
||||
memset((void *)oq_busy, 0, sizeof oq_busy);
|
||||
oq_batch_drop();
|
||||
}
|
||||
|
||||
// The completion thread reads the results in batches (NX2NAVI_OQ_BATCH, default 1; 0: every segment by itself, as
|
||||
// before). The game tests one proxy per report, so a segment holds one query, and a frame of Hyrule Field has about a
|
||||
// hundred of them: one vkGetQueryPoolResults and one vkResetQueryPool each. On an RX 7900 XT (26.10.1) each of those
|
||||
// calls goes into the kernel, and the two hundred took 3.9 ms of every frame on the completion thread, which runs a
|
||||
// submission's callbacks before it lets the submission's waiters go: the game's main thread waits for the previous
|
||||
// frame's fence before it lets the next frame's GPU work start, so the frame after one replayed inside its submit
|
||||
// call began 4 to 6 ms late with every thread idle. Slots come from the ring in order, so the slots of the segments
|
||||
// that follow lie behind the one asked for: one call reads OQ_BATCH slots from there on with their availability, the
|
||||
// available ones are kept (the results stay in the pool until the reset), and the segments after it take theirs from
|
||||
// what is kept. A slot not available in the batch (its part had not run yet) is read again when its segment comes.
|
||||
// The slots taken are reset in runs, one vkResetQueryPool for each: a run ends when a slot that does not continue it
|
||||
// is taken or a new batch is read. Until then its slots stay busy for the replay, at most OQ_BATCH of the ring's
|
||||
// OQ_N, also while the game reports nothing.
|
||||
#define OQ_BATCH 256
|
||||
static struct {
|
||||
uint32_t first, n; // the slots kept: [first, first + n)
|
||||
uint64_t v[OQ_BATCH * 2]; // per slot: samples, available
|
||||
uint8_t taken[OQ_BATCH];
|
||||
uint32_t r0, rn; // the run of taken slots still to be reset
|
||||
uint64_t calls, slots, segs; // since the last perf line: result calls, slots they read, segments served
|
||||
} OQB;
|
||||
static int oq_batch_mode = -1;
|
||||
|
||||
static void oq_batch_drop(void) { OQB.n = OQB.rn = 0; }
|
||||
|
||||
static void oq_reset_flush(VkDevice dev) {
|
||||
if (!OQB.rn) return;
|
||||
vkResetQueryPool(dev, oq_pool, OQB.r0, OQB.rn);
|
||||
for (uint32_t k = 0; k < OQB.rn; k++) __atomic_store_n(&oq_busy[OQB.r0 + k], 0, __ATOMIC_RELEASE);
|
||||
OQB.rn = 0;
|
||||
}
|
||||
|
||||
static void oq_read(VkDevice dev, uint32_t first, uint32_t n) {
|
||||
VkResult r = vkGetQueryPoolResults(dev, oq_pool, first, n, sizeof OQB.v[0] * 2 * n, OQB.v, 2 * sizeof OQB.v[0],
|
||||
VK_QUERY_RESULT_64_BIT | VK_QUERY_RESULT_WITH_AVAILABILITY_BIT);
|
||||
if (r != VK_SUCCESS && r != VK_NOT_READY) {
|
||||
vk_device_fault_report(r);
|
||||
rt_fatal("vk: occlusion query results failed (%d)", (int)r);
|
||||
}
|
||||
OQB.first = first, OQB.n = n;
|
||||
memset(OQB.taken, 0, n);
|
||||
OQB.calls++, OQB.slots += n;
|
||||
}
|
||||
|
||||
// The samples of slot q, whose submission has completed; *avail 0 when the query has no result.
|
||||
static uint64_t oq_take(VkDevice dev, uint32_t q, int *avail) {
|
||||
uint32_t k = q - OQB.first;
|
||||
if (!(q >= OQB.first && k < OQB.n && !OQB.taken[k] && OQB.v[2 * k + 1])) {
|
||||
// not kept, or kept as unavailable (read before its part ran): from this slot on, up to the replay's next slot
|
||||
oq_reset_flush(dev); // before the kept results go: the run's slots are among them
|
||||
uint32_t next = __atomic_load_n(&oq_next, __ATOMIC_RELAXED) % OQ_N, n = OQ_N - q;
|
||||
if (next > q && next - q < n) n = next - q;
|
||||
if (n > OQ_BATCH) n = OQ_BATCH;
|
||||
if (oq_batch_mode == 0) n = 1;
|
||||
oq_read(dev, q, n);
|
||||
k = 0;
|
||||
}
|
||||
OQB.taken[k] = 1;
|
||||
if (OQB.rn && q != OQB.r0 + OQB.rn) oq_reset_flush(dev);
|
||||
if (!OQB.rn) OQB.r0 = q;
|
||||
OQB.rn++;
|
||||
*avail = OQB.v[2 * k + 1] != 0;
|
||||
return OQB.v[2 * k];
|
||||
}
|
||||
|
||||
static void oq_cb(void *arg) {
|
||||
OqSegment *s = arg;
|
||||
VkDevice dev = vk_ctx()->device;
|
||||
uint64_t sum = 0, v[64 * 2];
|
||||
for (uint32_t i = 0; i < s->n;) {
|
||||
uint32_t j = i + 1;
|
||||
while (j < s->n && j - i < 64 && s->q[j] == s->q[j - 1] + 1) j++;
|
||||
if (oq_batch_mode < 0) oq_batch_mode = cfg_bool("NX2NAVI_OQ_BATCH");
|
||||
uint64_t sum = 0;
|
||||
for (uint32_t i = 0; i < s->n; i++) {
|
||||
// The fence has passed, so a query that is still unavailable will not become available: waiting would block the
|
||||
// completion thread, and with it every later callback, for good. It counts as visible (one sample).
|
||||
VkResult r = vkGetQueryPoolResults(dev, oq_pool, s->q[i], j - i, sizeof v, v, 2 * sizeof v[0],
|
||||
VK_QUERY_RESULT_64_BIT | VK_QUERY_RESULT_WITH_AVAILABILITY_BIT);
|
||||
if (r != VK_SUCCESS && r != VK_NOT_READY) {
|
||||
vk_device_fault_report(r);
|
||||
rt_fatal("vk: occlusion query results failed (%d)", (int)r);
|
||||
int avail;
|
||||
uint64_t v = oq_take(dev, s->q[i], &avail);
|
||||
if (avail) sum += v;
|
||||
else {
|
||||
sum += 1;
|
||||
LOG_ONCE("vk: occlusion query slot %u not available after its fence, counted as visible", s->q[i]);
|
||||
}
|
||||
vkResetQueryPool(dev, oq_pool, s->q[i], j - i);
|
||||
for (uint32_t k = i; k < j; k++) {
|
||||
if (v[2 * (k - i) + 1]) sum += v[2 * (k - i)];
|
||||
else {
|
||||
sum += 1;
|
||||
LOG_ONCE("vk: occlusion query slot %u not available after its fence, counted as visible", s->q[k]);
|
||||
}
|
||||
__atomic_store_n(&oq_busy[s->q[k]], 0, __ATOMIC_RELEASE);
|
||||
}
|
||||
i = j;
|
||||
}
|
||||
if (s->n && ++OQB.segs >= 12000) {
|
||||
if (rt_trace & RT_TRACE_PERF)
|
||||
rt_log("perf: occlusion queries: %llu segments read with %llu result calls of %.1f slots on average",
|
||||
(unsigned long long)OQB.segs, (unsigned long long)OQB.calls,
|
||||
OQB.calls ? (double)OQB.slots / (double)OQB.calls : 0.0);
|
||||
OQB.calls = OQB.slots = OQB.segs = 0;
|
||||
}
|
||||
if (s->reset) oq_count = 0;
|
||||
else {
|
||||
@@ -412,6 +477,22 @@ void vk_exec_event_cpu(const void *event) {
|
||||
plat_mutex_unlock(&cpu_ev_lock);
|
||||
}
|
||||
|
||||
// 1 when the CPU has signalled this event before (nvnEventSignal): its waits are gates the game opens from the CPU.
|
||||
int vk_exec_event_cpu_known(const void *event) {
|
||||
plat_mutex_lock(&cpu_ev_lock);
|
||||
int i = 0;
|
||||
while (i < cpu_ev_n && cpu_ev[i].val != event) i++;
|
||||
int known = i < cpu_ev_n;
|
||||
plat_mutex_unlock(&cpu_ev_lock);
|
||||
return known;
|
||||
}
|
||||
|
||||
// The wait's condition on the value in memory now, whatever the replay has seen. Any thread.
|
||||
int vk_exec_event_holds_now(const void *payload) {
|
||||
const p_wait_event *p = payload;
|
||||
return ev_holds(__atomic_load_n(p->val, __ATOMIC_ACQUIRE), p->mode, p->value);
|
||||
}
|
||||
|
||||
void vk_exec_event_sync(int begin) {
|
||||
plat_mutex_lock(&cpu_ev_lock);
|
||||
ev_sync_seq = begin ? cpu_ev_seq : UINT64_MAX;
|
||||
|
||||
@@ -87,6 +87,8 @@ static struct {
|
||||
VkShaderStageFlags sg32_stages; // stages that accept requiredSubgroupSize = 32 (0: not available)
|
||||
int reconverge; // modules with subgroup ops get MaximallyReconvergesKHR (spirv_reconverge)
|
||||
uint32_t nan_preserve; // float widths that get SignedZeroInfNanPreserve (spirv_nan_preserve): NP_16|NP_32|NP_64
|
||||
int nan_bits; // NaN tests as integer bit tests (spirv_nan_bits)
|
||||
volatile uint32_t nan_bits_tests, nan_bits_selects, nan_bits_modules; // rewritten so far (the first log line)
|
||||
VkDynamicState dyn[32];
|
||||
uint32_t n_dyn;
|
||||
PFN_vkCmdPushDescriptorSetKHR push;
|
||||
@@ -658,6 +660,12 @@ static void nan_preserve_init(VkCtx *c) {
|
||||
(fc.shaderSignedZeroInfNanPreserveFloat32 ? NP_32 : 0) |
|
||||
(fc.shaderSignedZeroInfNanPreserveFloat64 ? NP_64 : 0);
|
||||
P.nan_preserve = want && (can & NP_32) ? can : 0;
|
||||
// NX2NAVI_VK_NAN_BITS: 1 every driver, 0 none; unset: AMD's driver for RDNA 1 and 2 (spirv_nan_bits)
|
||||
const char *nb = cfg_get("NX2NAVI_VK_NAN_BITS");
|
||||
P.nan_bits = nb && *nb ? strcmp(nb, "0") != 0 : vk_caps.amd_sc;
|
||||
if (P.nan_bits)
|
||||
rt_log("vk_pipeline: game shaders test for NaN on the bits of the number, not as a float (%s)",
|
||||
nb && *nb ? "NX2NAVI_VK_NAN_BITS=1" : "AMD's driver for RDNA 1 and 2");
|
||||
if (P.nan_preserve)
|
||||
rt_log("vk_pipeline: game shaders keep NaN, infinity and signed zero (SignedZeroInfNanPreserve for %s32%s-bit "
|
||||
"floats%s)", can & NP_16 ? "16, " : "", can & NP_64 ? ", 64" : "", set ? ", NX2NAVI_VK_NAN_PRESERVE=1" : "");
|
||||
@@ -1478,6 +1486,125 @@ static uint32_t *spirv_nan_preserve(const uint32_t *w, uint64_t n, uint32_t widt
|
||||
return o;
|
||||
}
|
||||
|
||||
// NaN tests that no compiler can drop (P.nan_bits). The game's shaders and the translator's own guards test for NaN with
|
||||
// OpIsNan: 48,717 of an install's 54,959 translated modules, 26 tests each on average (the game's unordered
|
||||
// comparisons, and the guard that keeps NaN out of every float colour output, patch 0027, so that one NaN texel does not
|
||||
// spread through bloom and blur into blocks of black and white). SignedZeroInfNanPreserve asks the driver to keep NaN,
|
||||
// but whether a compiler that reasons about floats keeps an isnan through every optimisation is up to it. The reports
|
||||
// of black and white blocks, on Navi, on Link next to her and on bright surfaces seen from inside a house, come from
|
||||
// RX 5000 and 6000 cards (#41, #110, #117), whose Windows driver compiles with "AMD proprietary shader compiler"
|
||||
// (vk_caps.amd_sc); RDNA 3 and 4 (LLPC) draw the same scenes clean. It is not known whether that compiler keeps the
|
||||
// mode: on it each float32 test becomes integer arithmetic on the bits, which means the same under any float
|
||||
// assumption:
|
||||
// isnan(x) -> (bitcast<uint>(x) & 0x7fffffff) > 0x7f800000
|
||||
// for scalars and vectors of 32-bit floats; tests of 16- and 64-bit floats stay. A float32 select with a constant
|
||||
// operand whose bits are a NaN (an integer mask such as 0xffffffff that the translator types as float) becomes a
|
||||
// select of the bits (issue #156: on an RX 9070 XT without the mode such a select between 1 and -1 gave 2^32).
|
||||
// NULL: nothing to change (or no memory); *tests and *selects count what changed.
|
||||
static uint32_t *spirv_nan_bits(const uint32_t *w, uint64_t n, uint64_t *out_n, uint32_t *tests, uint32_t *selects) {
|
||||
enum { OP_UNDEF = 1, OP_TYPE_BOOL = 20, OP_SPEC_CONSTANT = 50, OP_SPEC_CONSTANT_COMPOSITE = 51, OP_STORE = 62,
|
||||
OP_BITCAST = 124, OP_IS_NAN = 156, OP_SELECT = 169, OP_UGREATER = 172, OP_BITWISE_AND = 199 };
|
||||
*tests = *selects = 0;
|
||||
if (n < 5 || w[3] >= (1u << 22)) return NULL;
|
||||
uint32_t bound = w[3];
|
||||
uint32_t *rtype = calloc(bound, sizeof *rtype), *vec_comp = calloc(bound, sizeof *vec_comp);
|
||||
uint8_t *f32 = calloc(bound, 1), *vec_n = calloc(bound, 1), *nan_const = calloc(bound, 1);
|
||||
uint32_t *o = NULL;
|
||||
if (!rtype || !vec_comp || !f32 || !vec_n || !nan_const) goto out;
|
||||
uint32_t tu = 0, uvec[5] = {0}; // uint32 and its vectors by component count
|
||||
uint64_t fn_at = 0;
|
||||
int vec_used[5] = {0};
|
||||
// Ids that are not results of the instruction's second word, inside functions: first assignment wins anyway, as
|
||||
// a definition comes before its uses (OpPhi operands aside, which this never reads).
|
||||
for (uint64_t i = 5; i < n;) {
|
||||
uint32_t op = w[i] & 0xffff, l = w[i] >> 16;
|
||||
if (!l || i + l > n) goto out;
|
||||
const uint32_t *a = w + i + 1;
|
||||
if (!fn_at) {
|
||||
if (op == SPV_OP_FUNCTION) fn_at = i;
|
||||
else if (op == SPV_OP_TYPE_FLOAT && l >= 3 && a[0] < bound && a[1] == 32) f32[a[0]] = 1;
|
||||
else if (op == SPV_OP_TYPE_INT && l >= 4 && a[1] == 32 && a[2] == 0 && !tu) tu = a[0];
|
||||
else if (op == SPV_OP_TYPE_VECTOR && l >= 4 && a[0] < bound && a[2] >= 2 && a[2] <= 4) {
|
||||
vec_comp[a[0]] = a[1], vec_n[a[0]] = (uint8_t)a[2];
|
||||
if (tu && a[1] == tu && !uvec[a[2]]) uvec[a[2]] = a[0];
|
||||
} else if ((op == SPV_OP_CONSTANT || op == SPV_OP_CONSTANT_COMPOSITE || op == SPV_OP_CONSTANT_NULL ||
|
||||
op == OP_UNDEF || op == SPV_OP_VARIABLE || op == OP_SPEC_CONSTANT ||
|
||||
op == OP_SPEC_CONSTANT_COMPOSITE) && l >= 3 && a[1] < bound) {
|
||||
rtype[a[1]] = a[0];
|
||||
if (op == SPV_OP_CONSTANT && l == 4 && a[0] < bound && f32[a[0]] && (a[2] & 0x7fffffffu) > 0x7f800000u)
|
||||
nan_const[a[1]] = 1;
|
||||
}
|
||||
}
|
||||
if (fn_at) {
|
||||
int no_result = op == OP_STORE || op == 63 || op == 64 || op == 8 || op == 317 || (op >= 246 && op <= 255) ||
|
||||
op == 99 || op == 224 || op == 225 || op == 228 || op == 56 || op == 218 || op == 219 ||
|
||||
op == 5294 || op == 5295 || op == 4416 || op == 5380 || op == 5364 || op == 5365;
|
||||
if (!no_result && op != SPV_OP_FUNCTION && l >= 3 && a[1] < bound && !rtype[a[1]]) rtype[a[1]] = a[0];
|
||||
if (op == OP_IS_NAN && l == 4 && a[2] < bound) {
|
||||
uint32_t t = rtype[a[2]];
|
||||
if (t < bound && f32[t]) ++*tests;
|
||||
else if (t < bound && vec_n[t] && vec_comp[t] < bound && f32[vec_comp[t]]) ++*tests, vec_used[vec_n[t]] = 1;
|
||||
} else if (op == OP_SELECT && l == 6 && a[0] < bound && f32[a[0]] && a[3] < bound && a[4] < bound &&
|
||||
(nan_const[a[3]] || nan_const[a[4]])) {
|
||||
++*selects;
|
||||
}
|
||||
}
|
||||
i += l;
|
||||
}
|
||||
if (!fn_at || (!*tests && !*selects)) goto out;
|
||||
// the declarations go right before the first function: uint32 and its vectors where missing, then the constants
|
||||
uint32_t next = bound, decl[64], nd = 0;
|
||||
if (!tu) tu = next++, decl[nd++] = 4u << 16 | SPV_OP_TYPE_INT, decl[nd++] = tu, decl[nd++] = 32, decl[nd++] = 0;
|
||||
uint32_t c_mag = next++, c_inf = next++, cv_mag[5] = {0}, cv_inf[5] = {0};
|
||||
for (int k = 2; k <= 4; k++)
|
||||
if (vec_used[k] && !uvec[k])
|
||||
uvec[k] = next++, decl[nd++] = 4u << 16 | SPV_OP_TYPE_VECTOR, decl[nd++] = uvec[k], decl[nd++] = tu, decl[nd++] = (uint32_t)k;
|
||||
decl[nd++] = 4u << 16 | SPV_OP_CONSTANT, decl[nd++] = tu, decl[nd++] = c_mag, decl[nd++] = 0x7fffffffu;
|
||||
decl[nd++] = 4u << 16 | SPV_OP_CONSTANT, decl[nd++] = tu, decl[nd++] = c_inf, decl[nd++] = 0x7f800000u;
|
||||
for (int k = 2; k <= 4; k++) {
|
||||
if (!vec_used[k]) continue;
|
||||
cv_mag[k] = next++, cv_inf[k] = next++;
|
||||
decl[nd++] = (uint32_t)(3 + k) << 16 | SPV_OP_CONSTANT_COMPOSITE, decl[nd++] = uvec[k], decl[nd++] = cv_mag[k];
|
||||
for (int j = 0; j < k; j++) decl[nd++] = c_mag;
|
||||
decl[nd++] = (uint32_t)(3 + k) << 16 | SPV_OP_CONSTANT_COMPOSITE, decl[nd++] = uvec[k], decl[nd++] = cv_inf[k];
|
||||
for (int j = 0; j < k; j++) decl[nd++] = c_inf;
|
||||
}
|
||||
uint64_t cap = n + nd + 10ull * *tests + 12ull * *selects;
|
||||
o = malloc(cap * 4);
|
||||
if (!o) goto out;
|
||||
memcpy(o, w, fn_at * 4);
|
||||
uint64_t p = fn_at;
|
||||
memcpy(o + p, decl, nd * 4), p += nd;
|
||||
for (uint64_t i = fn_at; i < n;) {
|
||||
uint32_t op = w[i] & 0xffff, l = w[i] >> 16;
|
||||
const uint32_t *a = w + i + 1;
|
||||
uint32_t t = op == OP_IS_NAN && l == 4 && a[2] < bound ? rtype[a[2]] : 0;
|
||||
int k = t && t < bound ? (f32[t] ? 1 : vec_n[t] && vec_comp[t] < bound && f32[vec_comp[t]] ? vec_n[t] : 0) : 0;
|
||||
if (k) {
|
||||
uint32_t ut = k == 1 ? tu : uvec[k], mag = k == 1 ? c_mag : cv_mag[k], inf = k == 1 ? c_inf : cv_inf[k];
|
||||
uint32_t bits = next++, low = next++;
|
||||
o[p++] = 4u << 16 | OP_BITCAST, o[p++] = ut, o[p++] = bits, o[p++] = a[2];
|
||||
o[p++] = 5u << 16 | OP_BITWISE_AND, o[p++] = ut, o[p++] = low, o[p++] = bits, o[p++] = mag;
|
||||
o[p++] = 5u << 16 | OP_UGREATER, o[p++] = a[0], o[p++] = a[1], o[p++] = low, o[p++] = inf;
|
||||
} else if (op == OP_SELECT && l == 6 && a[0] < bound && f32[a[0]] && a[3] < bound && a[4] < bound &&
|
||||
(nan_const[a[3]] || nan_const[a[4]])) {
|
||||
uint32_t x = next++, y = next++, s = next++;
|
||||
o[p++] = 4u << 16 | OP_BITCAST, o[p++] = tu, o[p++] = x, o[p++] = a[3];
|
||||
o[p++] = 4u << 16 | OP_BITCAST, o[p++] = tu, o[p++] = y, o[p++] = a[4];
|
||||
o[p++] = 6u << 16 | OP_SELECT, o[p++] = tu, o[p++] = s, o[p++] = a[2], o[p++] = x, o[p++] = y;
|
||||
o[p++] = 4u << 16 | OP_BITCAST, o[p++] = a[0], o[p++] = a[1], o[p++] = s;
|
||||
} else {
|
||||
memcpy(o + p, w + i, l * 4), p += l;
|
||||
}
|
||||
i += l;
|
||||
}
|
||||
o[3] = next;
|
||||
*out_n = p;
|
||||
out:
|
||||
free(rtype), free(vec_comp), free(f32), free(vec_n), free(nan_const);
|
||||
return o;
|
||||
}
|
||||
|
||||
static int spirv_mark_list_test(const char *list); // texture pool marks, below
|
||||
|
||||
// --selftest (main.c): spirv_nan_preserve on a small fragment module, without a device. Returns the number of failures.
|
||||
@@ -1523,6 +1650,42 @@ int vk_pipeline_spirv_selftest(void) {
|
||||
failures++;
|
||||
}
|
||||
printf("spirv: SignedZeroInfNanPreserve rewrite: %s\n", failures ? "FAIL" : "ok");
|
||||
{ // spirv_nan_bits: a scalar and a vec2 isnan, a select on a NaN constant; no uint32 type in the module
|
||||
static const uint32_t m[] = {
|
||||
0x07230203, 0x00010300, 0, 20, 0,
|
||||
2u << 16 | 17, 1, // OpCapability Shader
|
||||
3u << 16 | 14, 0, 1, // OpMemoryModel Logical GLSL450
|
||||
5u << 16 | 15, 5, 1, 0x6e69616d, 0, // OpEntryPoint GLCompute %1 "main"
|
||||
2u << 16 | 19, 2, // %2 void
|
||||
3u << 16 | 33, 3, 2, // %3 fn
|
||||
3u << 16 | 22, 4, 32, // %4 float
|
||||
2u << 16 | 20, 5, // %5 bool
|
||||
4u << 16 | 23, 6, 4, 2, // %6 vec2
|
||||
4u << 16 | 23, 7, 5, 2, // %7 bvec2
|
||||
4u << 16 | 43, 4, 8, 0xffffffffu, // %8 = NaN bits
|
||||
4u << 16 | 43, 4, 9, 0x3f800000u, // %9 = 1.0
|
||||
5u << 16 | 44, 6, 10, 8, 9, // %10 = vec2(%8, %9)
|
||||
3u << 16 | 41, 5, 11, // %11 = OpConstantTrue
|
||||
5u << 16 | 54, 2, 1, 0, 3, // %1 = OpFunction
|
||||
2u << 16 | 248, 12, // OpLabel
|
||||
4u << 16 | 156, 5, 13, 9, // %13 = OpIsNan bool %9
|
||||
4u << 16 | 156, 7, 14, 10, // %14 = OpIsNan bvec2 %10
|
||||
6u << 16 | 169, 4, 15, 11, 9, 8, // %15 = OpSelect float %11 %9 %8
|
||||
1u << 16 | 253, 1u << 16 | 56}; // OpReturn, OpFunctionEnd
|
||||
uint64_t on = 0;
|
||||
uint32_t tests = 0, sels = 0, isnan_left = 0, ugt = 0, int32 = 0, bitcasts = 0;
|
||||
uint32_t *o = spirv_nan_bits(m, sizeof m / 4, &on, &tests, &sels);
|
||||
for (uint64_t i = 5; o && i < on;) {
|
||||
uint32_t op = o[i] & 0xffff, l = o[i] >> 16;
|
||||
if (!l) break;
|
||||
isnan_left += op == 156, ugt += op == 172, int32 += op == 21 && o[i + 2] == 32, bitcasts += op == 124;
|
||||
i += l;
|
||||
}
|
||||
int bad = !o || tests != 2 || sels != 1 || isnan_left || ugt != 2 || int32 != 1 || bitcasts != 5 || o[3] <= 20;
|
||||
printf("spirv: integer NaN tests rewrite: %s\n", bad ? "FAIL" : "ok");
|
||||
failures += bad;
|
||||
free(o);
|
||||
}
|
||||
const char *list = cfg_get("NX2NAVI_SPV_MARK_TEST");
|
||||
if (list && *list) failures += spirv_mark_list_test(list);
|
||||
return failures;
|
||||
@@ -1867,6 +2030,18 @@ static VkShaderModule make_module(const uint32_t *w, uint64_t words, int early_o
|
||||
free(fixed);
|
||||
w = fixed = np, words = np_n;
|
||||
}
|
||||
uint64_t nb_n = 0;
|
||||
uint32_t nb_tests = 0, nb_sels = 0;
|
||||
uint32_t *nb = P.nan_bits ? spirv_nan_bits(w, words, &nb_n, &nb_tests, &nb_sels) : NULL;
|
||||
if (nb) {
|
||||
free(fixed);
|
||||
w = fixed = nb, words = nb_n;
|
||||
__atomic_fetch_add(&P.nan_bits_tests, nb_tests, __ATOMIC_RELAXED);
|
||||
__atomic_fetch_add(&P.nan_bits_selects, nb_sels, __ATOMIC_RELAXED);
|
||||
if (__atomic_add_fetch(&P.nan_bits_modules, 1, __ATOMIC_RELAXED) == 1)
|
||||
rt_log("vk_pipeline: first module with NaN tests made integer: %u tests, %u selects of NaN bits", nb_tests,
|
||||
nb_sels);
|
||||
}
|
||||
if (P.feedback >= 2) mm_hash = fnv(FNV0, w, (size_t)words * 4);
|
||||
VkShaderModuleCreateInfo ci = {.sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO, .codeSize = (size_t)words * 4, .pCode = w};
|
||||
VkShaderModule m = VK_NULL_HANDLE;
|
||||
|
||||
@@ -18,6 +18,7 @@ Run: python tests/launcher_test.py (Python 3.14 for the zstd tests; the window
|
||||
a source checkout's build/xlate/), the prompts step with a stand-in prompt tool, the object cache of the compile
|
||||
step with the system's C compiler, and its windres call under folders with spaces (a script in windres's place)
|
||||
- jobs: an install driven by a fake make_game.py, with and without the prompts row
|
||||
- Add to Steam: the binary shortcuts file read and written back unchanged, a shortcut added and updated in place
|
||||
- the look: the colour schemes, the background picture's fitting and kept copy, the sound effects (low by default,
|
||||
only the game's own select and back sounds, decoded from a made-up .bwav)
|
||||
- the window: --selftest draws every screen (skipped without a display)
|
||||
@@ -3048,6 +3049,60 @@ def have_display():
|
||||
|
||||
|
||||
@unittest.skipUnless(have_display(), 'needs tkinter, numpy and a display')
|
||||
class SteamTest(Tmp):
|
||||
"""launcher/steam.py on made-up Steam files: the binary shortcuts file read and written back unchanged, our
|
||||
shortcut added beside another and updated in place, its id as Steam computes it, the accounts that signed in."""
|
||||
|
||||
def other(self):
|
||||
from launcher import steam
|
||||
exe = '"C:\\Games\\Other\\other.exe"'
|
||||
entry = [[steam.T_INT, 'appid', struct.pack('<I', steam.appid(exe, 'Other'))], [steam.T_STR, 'AppName', 'Other'],
|
||||
[steam.T_STR, 'Exe', exe], [steam.T_STR, 'StartDir', '"C:\\Games\\Other\\"'],
|
||||
[steam.T_STR, 'icon', ''], [steam.T_INT, 'LastPlayTime', struct.pack('<I', 1234)],
|
||||
[steam.T_STR, 'sortas', ''], [steam.T_MAP, 'tags', [[steam.T_STR, '0', 'Favorite']]]]
|
||||
return steam.vdf_dump([[steam.T_MAP, 'shortcuts', [[steam.T_MAP, '0', entry]]]])
|
||||
|
||||
def test_round_trip_add_and_update(self):
|
||||
from launcher import steam
|
||||
b = self.other()
|
||||
self.assertEqual(b[-4:], b'\x08\x08\x08\x08') # tags, the shortcut, shortcuts, the file
|
||||
self.assertEqual(steam.vdf_dump(steam.vdf_load(b)), b)
|
||||
self.assertFalse(steam.has_shortcut(b))
|
||||
new, sid = steam.add_shortcut(b, '"C:\\NX2navi\\python\\pythonw.exe"', '"C:\\NX2navi"',
|
||||
'"C:\\NX2navi\\data\\steam_play.pyw"', 'C:\\grid\\x_icon.png')
|
||||
self.assertEqual(sid, steam.appid('"C:\\NX2navi\\python\\pythonw.exe"', steam.NAME))
|
||||
self.assertTrue(sid & 0x80000000)
|
||||
self.assertTrue(new.startswith(b[:-2])) # the other shortcut stays byte for byte
|
||||
self.assertTrue(steam.has_shortcut(new))
|
||||
items = steam.vdf_load(new)[0][2]
|
||||
self.assertEqual([k for _, k, _ in items], ['0', '1'])
|
||||
ours = {k: v for _, k, v in items[1][2]}
|
||||
self.assertEqual(ours['AppName'], 'The Legend of Zelda Ocarina of Time')
|
||||
self.assertEqual(struct.unpack('<I', ours['appid'])[0], sid)
|
||||
self.assertEqual(ours['LaunchOptions'], '"C:\\NX2navi\\data\\steam_play.pyw"')
|
||||
self.assertEqual(ours['AllowOverlay'], struct.pack('<I', 1))
|
||||
again, _ = steam.add_shortcut(new, '"D:\\NX2navi\\python\\pythonw.exe"', '"D:\\NX2navi"', '', 'x')
|
||||
items = steam.vdf_load(again)[0][2]
|
||||
self.assertEqual(len(items), 2) # moved launcher: the same shortcut, updated
|
||||
self.assertEqual({k: v for _, k, v in items[1][2]}['StartDir'], '"D:\\NX2navi"')
|
||||
with self.assertRaises(steam.SteamError):
|
||||
steam.vdf_load(b[:-5])
|
||||
|
||||
def test_new_file_and_accounts(self):
|
||||
from launcher import steam
|
||||
new, _ = steam.add_shortcut(b'', '"x.exe"', '"."', '', '')
|
||||
self.assertTrue(steam.has_shortcut(new))
|
||||
root = self.tmp / 'Steam'
|
||||
for a in ('0', '1046310423', '22'):
|
||||
(root / 'userdata' / a / 'config').mkdir(parents=True)
|
||||
(root / 'config').mkdir()
|
||||
(root / 'config' / 'loginusers.vdf').write_text('"users"\n{\n\t"76561199006576151"\n\t{\n\t}\n}\n')
|
||||
self.assertEqual(steam.accounts(root), [root / 'userdata' / '1046310423' / 'config'])
|
||||
(root / 'config' / 'loginusers.vdf').unlink()
|
||||
self.assertEqual(steam.accounts(root), sorted([root / 'userdata' / '1046310423' / 'config',
|
||||
root / 'userdata' / '22' / 'config']))
|
||||
|
||||
|
||||
class WhilePlayingTest(Tmp):
|
||||
def test_the_games_restart_does_not_bring_the_launcher_back(self):
|
||||
try:
|
||||
|
||||
Reference in New Issue
Block a user