Import guest memory as dma-bufs through udmabuf on Linux instead of keeping shadow copies of it.

On Linux every CPU-visible pool had a host-visible shadow copy, because amdgpu treats a host-pointer import as a userptr object that it looks up again at every submission and invalidates whenever the write watch or a freeze re-protects its pages. Now the watched reservations are shared mappings of one memfd, /dev/udmabuf makes dma-bufs of page ranges of it, and the device imports them (VK_KHR_external_memory_fd, VK_EXT_external_memory_dma_buf). A dma-buf pins its pages once and is not tied to the process's page tables, so protection changes leave the GPU's mapping alone, and nothing is copied.

RADV puts every imported memory object into every submission, so pools share their imports: guest addresses are cut into 64 MiB spans (NX2NAVI_DMABUF_MB), and each pool is a buffer bound into its span's memory at its offset. A new span is cut back at the spans already there, so spans overlap only where a pool itself lies across an older span's edge, since amdgpu counts every span's pages in its GTT use. A decommit punches the pages out of the memfd, or zeroes them in place where a dma-buf pins them, and the pages a freed span pinned that nothing committed go back to the system.

It is on by default where /dev/udmabuf opens, udmabuf's size_limit_mb is 1024 or more, the device has both extensions and NVIDIA's kernel module is not loaded. A start-up probe imports one page as the pools will be; any refusal keeps the shadow copies, and the log says why. NX2NAVI_DMABUF=0 keeps the copies. tests/platform_dmabuf_test.c checks the shared mapping without a GPU, and docs/building.md ("Memory") describes the imports and what they need.

RX 6800 XT (RADV, Mesa 26.2.6), one build, interleaved pairs with NX2NAVI_DMABUF=1 and 0: Kokiri Forest 60.7 against 40.3 fps (3 pairs, +51%, standard error 0.4), 1% low 50.9 against 34.0, frames over 25 ms 5 against 768; Hyrule Field 55.7 against 42.4 fps (2 pairs, +31%, standard error 0.4). No pool was left without a span, no span was refused, and --probe-vkmem passes with and without the imports.
This commit is contained in:
Vasilis Manetas
2026-11-12 18:12:08 +01:00
committed by chrg6
parent 677eb5c242
commit eaf10677fa
10 changed files with 750 additions and 15 deletions

View File

@@ -258,6 +258,17 @@ uint64_t plat_mem_written(uint64_t addr, uint64_t n, uint64_t *pages, uint64_t m
// pages in one call, where the watch works by protecting pages (Linux), so the stores do not fault page by page. The
// pages count as written. Returns 0 when there is nothing to clear or it cannot be done; the write is valid either way.
int plat_mem_prepare_write(uint64_t addr, uint64_t n);
// Guest memory as dma-bufs (Linux, NX2NAVI_DMABUF, on by default; vk_memory.c "dma-buf imports"): the watched
// reservations are shared mappings of one memfd, and /dev/udmabuf makes a dma-buf of a page range of it, which the GPU
// driver imports like its own memory. plat_mem_dmabuf_ok: 1 when that is so, with udmabuf's size limit in bytes in *limit and the
// first page of a shared reservation in *any; 0 with the reason in *why ("" when NX2NAVI_DMABUF=0 turned it off).
// plat_mem_dmabuf: a new dma-buf file descriptor over [addr, addr + n) (whole pages inside one watched reservation),
// -1 with errno on failure. Its pages stay pinned (a decommit zeroes them in place) until plat_mem_dmabuf_done is
// told, after the last user of the dma-buf (the imported memory) is gone; the pages of its range that nothing has
// committed then go back to the system. Elsewhere these report 0 and -1.
int plat_mem_dmabuf_ok(uint64_t *limit, uint64_t *any, const char **why);
int plat_mem_dmabuf(uint64_t addr, uint64_t n);
void plat_mem_dmabuf_done(uint64_t addr, uint64_t n);
// A write that faulted on a read-only page at addr goes to fn first; fn returns 1 when it made the page writable and
// the write should run again, 0 to let the fault go on (to the crash handler). One hook per process.
void plat_mem_fault_hook(int (*fn)(uint64_t addr));