diff --git a/docs/README-BuildKit.md b/docs/README-BuildKit.md index bf37ea75..c5330081 100644 --- a/docs/README-BuildKit.md +++ b/docs/README-BuildKit.md @@ -99,9 +99,11 @@ On NVIDIA, start the game with `NX2NAVI_SHADOW=0` if the 3D world stays black wh profile, logs and settings are in the `--out` folder as on Windows. On NVIDIA the driver keeps the shaders it compiled in `nvidia-cache` beside `ocarina`, not in `~/.cache/nvidia`, where -one run adds at most 1 GiB while the preparation at the end of the build makes about 1.5 GB. The game sets the folder -and a limit of 8 GiB itself (`__GL_SHADER_DISK_CACHE_PATH` and `__GL_SHADER_DISK_CACHE_SIZE` in the environment take -precedence). Deleting the folder costs one slow start; `./ocarina --warm-shaders` fills it again. +one run adds at most 1 GiB while the preparation at the end of the build makes about 1.5 GB. The game makes and sets +the folder unless the environment sets `__GL_SHADER_DISK_CACHE_PATH`, and a limit of 8 GiB unless it sets +`__GL_SHADER_DISK_CACHE_SIZE`. When it cannot make or write the folder, it sets neither and the driver keeps its own +folder and limit; the log says which (`vk: NVIDIA's shader disk cache`). Deleting the folder costs one slow start; +`./ocarina --warm-shaders` fills it again. `tools/make_appimage.py` packs this kit into one `NX2navi-Linux--x86_64.AppImage` with its own Python, for players without a source checkout: it checks for the packages above, installs zbic with pip on its first run, picks diff --git a/docs/building.md b/docs/building.md index 3b561114..47243cf7 100644 --- a/docs/building.md +++ b/docs/building.md @@ -235,7 +235,10 @@ NVIDIA's Linux driver compiles into a disk cache of its own. A process adds at m folder, `~/.cache/nvidia`, is shared with every program. `--prepare-shaders` of game 1.1.1 builds 42,142 variants, about 1.5 GB: on the default limit about 30,700 of them were stored and the rest compiled again at their first draw in play. `runtime/vk_device.c` therefore points `__GL_SHADER_DISK_CACHE_PATH` at `nvidia-cache` beside `ocarina` and the size at -8 GiB before the first Vulkan call, each unless the environment sets it, as it does `MESA_SHADER_CACHE_DIR` for RADV. The +8 GiB, each unless the environment sets it, as it does `MESA_SHADER_CACHE_DIR` for RADV, and only while NVIDIA's kernel +driver is loaded (`/proc/driver/nvidia/version`). The driver does not make that folder and caches nothing in a folder it +cannot write, so the game makes it, and when it cannot make or write it sets neither variable. A game run sets them in +`main` before any of its threads starts, since a thread can load the Vulkan driver before the instance is made. The folder can be deleted; `ocarina --warm-shaders` fills it again. Memory: on Windows the GPU imports guest memory and `GetWriteWatch` tracks the CPU's writes to it. On Linux, diff --git a/docs/installer.md b/docs/installer.md index ae1d5cc3..7587c906 100644 --- a/docs/installer.md +++ b/docs/installer.md @@ -202,9 +202,10 @@ bin/ nx2navi_xlate, hactool, ninja, libSDL3.so.0 - The setup's temp folder is `.setup-temp` inside the install folder (`--temp` changes it): the AppImage is mounted read-only, and a Steam Deck's `/home` is often too full for the generated C while the game goes to the SD card. - The game keeps Mesa's shader cache in `mesa-cache` beside `nx2navi` unless `MESA_SHADER_CACHE_DIR` is set, for the - same reason: RADV fills its default folder in `~/.cache` up to 1 GB. NVIDIA's driver cache goes to `nvidia-cache` - beside it with a limit of 8 GiB unless `__GL_SHADER_DISK_CACHE_PATH` or `__GL_SHADER_DISK_CACHE_SIZE` is set: the - default limit of 1 GiB is smaller than what the shader preparation builds. + same reason: RADV fills its default folder in `~/.cache` up to 1 GB. With NVIDIA's driver, its cache goes to + `nvidia-cache` beside it unless `__GL_SHADER_DISK_CACHE_PATH` is set, with a limit of 8 GiB unless + `__GL_SHADER_DISK_CACHE_SIZE` is set: the default limit of 1 GiB is smaller than what the shader preparation builds. + When that folder cannot be made or written, the driver keeps its own folder and limit. - The window follows `LANG` for English or Russian, cancels through SIGTERM to the CLI's process group (the CLI handles it as Ctrl+Break on Windows) and opens the finished folder with `xdg-open`. - Requirements beyond Windows': Linux 6.7 or later (the write watch, `docs/building.md`) and glibc 2.34. diff --git a/runtime/host.h b/runtime/host.h index 81effc41..85155251 100644 --- a/runtime/host.h +++ b/runtime/host.h @@ -389,4 +389,7 @@ int probe_movie(void); // movie_probe.c: hos int vk_pipeline_prepare_shaders(void); // vk_pipeline.c: --prepare-shaders (tools/prepare_shaders.py) int vk_pipeline_warm_shaders(void); // vk_pipeline.c: --warm-shaders (tools/build_game.py) int vk_pipeline_probe_prewarm(void); // vk_pipeline.c: --probe-prewarm +// vk_device.c: the drivers' own shader caches beside the exe (MESA_SHADER_CACHE_DIR, on NVIDIA's Linux driver +// __GL_SHADER_DISK_CACHE_PATH and _SIZE) in the environment, once; before any thread can load a Vulkan driver +void vk_driver_cache_env(void); int vk_pipeline_spirv_selftest(void); // vk_pipeline.c: --selftest, the SPIR-V rewrites without a device diff --git a/runtime/main.c b/runtime/main.c index 9f1e6d1d..db0a5e98 100644 --- a/runtime/main.c +++ b/runtime/main.c @@ -611,6 +611,7 @@ int main(int argc, char **argv) { log_open(!mode); if (!mode) { cfg.boot = cfg.seconds > 0; // --seconds starts the game like --boot, with its checkpoints + vk_driver_cache_env(); // before the threads below and rt_boot's, any of which may load the Vulkan driver discord_init(); // the Rich Presence thread (discord_rpc.c); it reads nx2navi.ini itself and idles while off rt_boot(); return 0; diff --git a/runtime/vk_device.c b/runtime/vk_device.c index 55a2cff9..9167b87f 100644 --- a/runtime/vk_device.c +++ b/runtime/vk_device.c @@ -4,6 +4,10 @@ #include #include #include +#ifdef __linux__ +#include +#include +#endif #include "cfg.h" #include "host.h" @@ -79,30 +83,56 @@ static void mesa_cache_beside_exe(void) { // variants of game 1.1.1 into it in one process: with the default limit it stored about 30,700 of them and the rest // were compiled again by the first start after it. So unless the environment sets them, the cache goes to nvidia-cache // beside the exe, which nothing else writes, with a limit of NV_CACHE_BYTES. The driver makes GLCache inside that -// folder but not the folder, so it is made here. +// folder but not the folder, so it is made here; pointed at a folder that is missing or not writable it caches nothing +// at all, so when that folder cannot be made or written neither variable is set and the driver keeps its own folder +// and limit. Without NVIDIA's kernel driver (no /proc/driver/nvidia/version, the test play.sh makes) the variables do +// nothing and the folder would stay empty, so nothing is done. #define NV_CACHE_BYTES "8589934592" // 8 GiB static void nvidia_cache_beside_exe(void) { + if (plat_fs_kind("/proc/driver/nvidia/version") != PLAT_FS_FILE) return; char dir[PLAT_PATH_MAX * 2], size[32]; - if (!plat_getenv("__GL_SHADER_DISK_CACHE_PATH", dir, sizeof dir)) { + int own_dir = !plat_getenv("__GL_SHADER_DISK_CACHE_PATH", dir, sizeof dir); + if (own_dir) { int len = plat_exe_path(dir, sizeof dir - 16); char *cut = len ? plat_last_sep(dir) : NULL; - if (cut) { - strcpy(cut + 1, "nvidia-cache"); - plat_fs_mkdir(dir); // already there is fine - plat_setenv("__GL_SHADER_DISK_CACHE_PATH", dir); + if (!cut) return; + strcpy(cut + 1, "nvidia-cache"); + int made = plat_fs_mkdir(dir), there = made == PLAT_FS_OK || made == PLAT_FS_EXISTS; + int err = !there ? plat_fs_os_error() : access(dir, W_OK | X_OK) ? errno : 0; + if (err) { + rt_log("vk: NVIDIA's shader disk cache stays in the driver's own folder with its own limit: %s %s (%s)", dir, + there ? "is not writable" : "cannot be made", strerror(err)); + return; } + plat_setenv("__GL_SHADER_DISK_CACHE_PATH", dir); } - if (!plat_getenv("__GL_SHADER_DISK_CACHE_SIZE", size, sizeof size)) - plat_setenv("__GL_SHADER_DISK_CACHE_SIZE", NV_CACHE_BYTES); + int own_size = !plat_getenv("__GL_SHADER_DISK_CACHE_SIZE", size, sizeof size); + if (own_size) plat_setenv("__GL_SHADER_DISK_CACHE_SIZE", NV_CACHE_BYTES); + rt_log("vk: NVIDIA's shader disk cache in %s (%s), up to %s bytes (%s)", dir, + own_dir ? "beside the exe" : "__GL_SHADER_DISK_CACHE_PATH", own_size ? NV_CACHE_BYTES : size, + own_size ? "the game's limit" : "__GL_SHADER_DISK_CACHE_SIZE"); } #endif -static int create_instance(void) { - vk_dlssg_early(); // DLSS Frame Generation: Streamline before any Vulkan call of the process (its slInit asks so) +// A driver may read these variables when it loads, and a thread can load the Vulkan driver before create_instance: +// widescreen.c's wide_init asks for the monitor's size (fullscreen with a monitor chosen and the picture at the +// window's size), and SDL's load of the Vulkan library there lists the instance extensions, which loads the drivers, +// while the prewarm thread may be in create_instance; setenv beside other threads' getenv is not safe either. main +// therefore calls this for a game run before any thread of the game starts; create_instance calls it for the other +// modes. Both calls happen before other threads exist or under vk_init's lock, so the flag needs no atomics. +void vk_driver_cache_env(void) { + static int done; + if (done) return; + done = 1; mesa_cache_beside_exe(); #ifdef __linux__ nvidia_cache_beside_exe(); #endif +} + +static int create_instance(void) { + vk_driver_cache_env(); + vk_dlssg_early(); // DLSS Frame Generation: Streamline before any Vulkan call of the process (its slInit asks so) const char *ext[12]; uint32_t n_ext = (uint32_t)gfx_window_vk_extensions(ext, 7); const char *layers[] = {"VK_LAYER_KHRONOS_validation"};