bug: intermittent GGML_ASSERT(bufs.size() == 1) during simulator session load when the useMmap option is passed, regardless of its value

[savagemanage]

Issue description

Passing the useMmap option to loadModel intermittently fail-fasts the process during the simulator session load with GGML_ASSERT(bufs.size() == 1); the value does not matter, and omitting the option never reproduced in 290 runs.

Expected Behavior

loadModel completes, and passing useMmap explicitly behaves the same as omitting it.

In particular, useMmap: false should not differ from omitting the option, since AddonModel already forces mmap off for the simulator session.

Actual Behavior

The process is terminated by a failed assertion during the simulator session load, before the real model load:

llama-model.cpp:1708: GGML_ASSERT(bufs.size() == 1) failed

It is not a catchable exception, so the application cannot retry or fall back. The process fail-fasts with 0xC0000409. In a bash loop on Windows this surfaces as exit code 127.

Full output of a failing run with getLlama({ logLevel: LlamaLogLevel.debug, debug: true }) and NODE_LLAMA_CPP_DEBUG=1:

loadOpts={"modelPath":"...Qwen_Qwen3-1.7B-Q4_K_M.gguf","gpuLayers":"auto","useMmap":true}
load_backend: loaded CPU backend from ...\ggml-cpu-alderlake.dll
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 8187 MiB):
  Device 0: NVIDIA GeForce RTX 4060 Laptop GPU, compute capability 8.9, VMM: yes
load_backend: loaded CUDA backend from ...\fallback\ggml-cuda.dll
llama gpu=cuda logLevel=debug
[llama:debug] Loading model for simulator session. gpuLayers=29 useMmap=true
D:\a\node-llama-cpp\node-llama-cpp\llama\llama.cpp\src\llama-model.cpp:1708: GGML_ASSERT(bufs.size() == 1) failed

No llama_model_loader: output appears before the assert. Successful runs in the same loop print the normal load output and exit 0.

A symbolized stack is not available: the prebuilt binaries ship without PDBs, and a local source build fails on this machine with MSVC C1060. The unsymbolized chain from a crash dump is llama-addon.node -> llama.cuda.b10068.dll -> ggml-base.dll -> ucrtbase.dll, consistent with GGML_ASSERT reaching abort().

How often

About 4.5% of invocations with identical inputs, and only when the useMmap option is passed. The value does not matter.

Main measurement: 300 invocations in a single shuffled loop, 100 per condition, so that condition and wall-clock time are not confounded. gpuLayers: "auto" throughout.

useMmap	asserts / 100	crash positions in the 1-300 sequence
true	3	97, 103, 243
false	6	43, 50, 228, 231, 275, 300
omitted	0	none

Crash positions are spread across the sequence, so this is not drift over time.

Including earlier block runs, cumulative:

	asserts / runs	rate
useMmap passed (either value)	19 / 420	4.5%
useMmap omitted	0 / 290	0%

Given 19 crashes across 710 runs, the probability that all 19 land in the useMmap group by chance is 3.9e-5.

true versus false is indistinguishable (3 vs 6 out of 100 each, binomial p = 0.51), so the mmap setting itself does not look like the mechanism. What correlates is whether the option was supplied at all. gpuLayers showed no effect in any of these runs.

Steps to reproduce

Fresh directory, npm i node-llama-cpp@3.19.1 only. No other dependencies.

Model: https://huggingface.co/bartowski/Qwen_Qwen3-1.7B-GGUF
File: Qwen_Qwen3-1.7B-Q4_K_M.gguf
sha256: 72c5c3cb38fa32d5256e2fe30d03e7a64c6c79e668ad84057e3bd66e250b24fb

curl -L -o Qwen_Qwen3-1.7B-Q4_K_M.gguf \
  "https://huggingface.co/bartowski/Qwen_Qwen3-1.7B-GGUF/resolve/main/Qwen_Qwen3-1.7B-Q4_K_M.gguf"

repro-min.mjs:

import { getLlama } from "node-llama-cpp";
 
const path = process.argv[2];
if (!path) throw new Error("usage: node repro-min.mjs <model.gguf> [useMmap]");
const mmap = process.argv[3]; // "true" | "false" | omit
 
const opts = { modelPath: path, gpuLayers: "auto" };
if (mmap === "true") opts.useMmap = true;
if (mmap === "false") opts.useMmap = false;
 
const t0 = performance.now();
const llama = await getLlama({ gpu: "cuda", build: "never", usePrebuiltBinaries: true, skipDownload: true });
const model = await llama.loadModel(opts);
console.log(`ok ${Math.round(performance.now() - t0)}ms ${model.typeDescription} gpuLayers=${model.gpuLayers} useMmapArg=${mmap ?? "omit"}`);
await model.dispose();
await llama.dispose();

Run 100 times and count non-zero exits:

for i in $(seq 1 100); do node repro-min.mjs "$MODEL" true; echo exit=$?; done
# control: drop the trailing argument

Expect roughly 3-6 crashes per 100 with true or false, and none when the argument is omitted. A single run will usually succeed, so the loop is necessary.

My Environment

Dependency	Version
Operating System	Windows 10.0.26200 (x64)
CPU	13th Gen Intel(R) Core(TM) i7-13620H
Node.js version	v22.18.0
Typescript version	5.9.3
node-llama-cpp version	3.19.1
Prebuilt binaries	@node-llama-cpp/win-x64-cuda@3.19.1, @node-llama-cpp/win-x64-cuda-ext@3.19.1
Bundled llama.cpp	b10068 / 571d0d540, Clang 20.1.8, Windows x86_64
GPU	NVIDIA GeForce RTX 4060 Laptop GPU, 8187 MiB, cc 8.9
Driver / CUDA	576.02 / 12.9
Model	Qwen_Qwen3-1.7B-Q4_K_M.gguf, single file, no shards, no mmproj

npx --yes node-llama-cpp inspect gpu output:

OS: Windows 10.0.26200 (x64)
Node: 22.18.0 (x64)
node-llama-cpp: 3.19.1
Prebuilt binaries: b10068
CUDA: available
Vulkan: available
CUDA device: NVIDIA GeForce RTX 4060 Laptop GPU
CUDA used VRAM: 13.29% (1.06GB/8GB)
CUDA free VRAM: 86.7% (6.93GB/8GB)
Vulkan devices: NVIDIA GeForce RTX 4060 Laptop GPU, Intel(R) UHD Graphics
Vulkan used VRAM: 43355331159.44% (16384PB/39.63GB)
Vulkan free VRAM: 0% (0B/39.63GB)
CPU model: 13th Gen Intel(R) Core(TM) i7-13620H
Math cores: 10
Used RAM: 50.28% (32.04GB/63.71GB)
Free RAM: 49.71% (31.67GB/63.71GB)
Wired RAM: 2.32% (1.48GB/63.71GB)
Used swap: 55.93% (37.87GB/67.71GB)
Max swap size: 67.71GB
mmap: supported

Additional Context

Why this looks contradictory

llama_model::memory_breakdown() only asserts when no_alloc is set:

for (const auto & [ctx, bufs] : pimpl->ctxs_bufs) {
    if (hparams.no_alloc) {
        GGML_ASSERT(bufs.size() == 1);

The only load branch that pushes more than one buffer is the mmap / buffer_from_host_ptr path, which is guarded by GGML_ASSERT(!ml.no_alloc). Every other branch pushes exactly one.

And AddonModel already forces mmap off for the simulator session (llama/addon/AddonModel.cpp:442-444 in 3.19.1):

if (model_params.no_alloc) {
    model_params.use_mlock = false;
    model_params.use_mmap = false;

So on this path bufs.size() should be 1 by construction. Either the forcing does not reach the failing path, or the extra buffer comes from somewhere other than the mmap branch.

Where the paths diverge

Instrumenting the JS layer, the three conditions differ like this (3 runs each, consistent):

	explicit true	explicit false	omitted
resolved value	true	false	"auto"
simulator AddonModel calls per invocation	pinned to true, 3-4	pinned to false, 3-4	both true and false, 7-8
final real load	useMmap: true	useMmap: false	useMmap: true

Resolution happens at LlamaModel.js:474-478, with defaultUseMmap = "auto" at LlamaModel.js:18. The simulator loads are issued from GgufInsights.js:1001-1007, always with noAlloc: true and useMlock: false in all three conditions.

Two things stand out.

1. The omitted case issues simulator loads with both mmap values and still never asserted, while each pinned value asserts on its own. It also issues about twice as many simulator loads per invocation, so "more load attempts" does not explain it either. Counting per simulator load rather than per invocation:

	simulator loads	asserts	per-load rate
useMmap passed	~1470	19	1.29%
omitted	~2223	0	0%

If the omitted path had the same per-load rate, the probability of seeing zero asserts in 2223 loads would be about 3e-13.

2. The final real load is identical between explicit true and omitted (useMmap: true, no noAlloc). The only difference is in the simulator phase, which matches the debug log showing the crash during "Loading model for simulator session".

That points at the resolution path rather than the mmap value, but I do not know this codebase well enough to say where. I have not been able to instrument the C++ side (AddonModel.cpp:442-444, before and after the forcing) because the local source build fails with C1060.

Unrelated observation in inspect gpu

The Vulkan VRAM figures in the output above look wrong:

Vulkan used VRAM: 43355331159.44% (16384PB/39.63GB)
Vulkan free VRAM: 0% (0B/39.63GB)

16384 PB looks like an overflow or underflow in the Vulkan memory query, on a machine with a discrete NVIDIA GPU plus an Intel UHD iGPU. I have no evidence it is connected to the assert, and all repro runs use the CUDA backend. Mentioning it only because it appears in the output this template asks for. Happy to open it separately if useful.

Workaround

Omitting the useMmap option entirely. On our machine that costs about 16% on load time (3810 ms to 4427 ms, single run each).

Not tested

• CPU backend (gpu: false)
• Other models or quantizations
• Other GPUs, drivers, or platforms
• Whether the rate changes across llama.cpp releases

Happy to help

I am willing to work on a fix but do not know this codebase yet. If you can point me at the right layer, I can take it from there. I am also happy to run more measurements on this machine.

Relevant Features Used

• Metal support
• CUDA support
• Vulkan support
• Grammar
• Function calling

Are you willing to resolve this issue by submitting a Pull Request?

Yes, I have the time, but I don't know how to start. I would need guidance.