Skip to content

Releases: ggml-org/llama.cpp

b10188

Choose a tag to compare

@github-actions github-actions released this 30 Jul 08:56
d0bfb19

metal: fix memory unwire if model is freed without any GPU operations (#26082)

  • metal: fix memory leak if model is freed without any GPU operations

  • metal: run dummy work only if residency sets are used

  • metal: wrap function in #if defined

  • metal: measure system-wide wired memory in test

  • metal: always build regression test

Co-authored-by: YiChen Lv 63285796+forforever73@users.noreply.github.com


Co-authored-by: YiChen Lv 63285796+forforever73@users.noreply.github.com

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b10186

Choose a tag to compare

@github-actions github-actions released this 30 Jul 07:35
32703b4

b10184

Choose a tag to compare

@github-actions github-actions released this 30 Jul 04:40
64d528b

b10182

Choose a tag to compare

@github-actions github-actions released this 29 Jul 17:01
afeebe1

llama: move suppress_tokens handling to common/sampling (#26276)

  • llama: move suppress_tokens handling to common/sampling

  • address security issues

  • rm has_logit_bias

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b10181

Choose a tag to compare

@github-actions github-actions released this 29 Jul 15:09
caa596a

ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory (#26141)

ggml_cuda_should_use_mmq() selects MMQ purely from the quantization
type. The current MMQ configurations are designed and maintained against
a minimum of 48 KiB per-block shared memory, the limit provided by
NVIDIA Pascal GPUs and later. On devices that report less, no supported
MMQ tile fits and mul_mat_q_switch_J() aborts when every tile size
exceeds the device's per-block shared memory budget.

Disable MMQ when smpbo < 48 KiB so the caller falls back to the BLAS
path instead of hitting GGML_ABORT. Some current MUSA QY1 devices
report only 28 KiB and are covered by this guard.

Reproduced on a Moore Threads MTT S70 (arch mp_21, 28 KiB shared memory
per block) with an RWKV-7 0.1B Q8_0 model:

$ llama-bench -m rwkv7-g1d-0.1b-Q8_0.gguf -p 128 -n 0
J_best=0
ggml/src/ggml-cuda/template-instances/../mmq.cuh:1521: fatal error
(core dumped)

Only prefill (batch > 1) is affected; token generation is fine. After
the fix the same device falls back to the BLAS path:

Q8_0 pp128 1470.7 t/s, tg8 55.3 t/s (was: abort)
FP16 unchanged
Q4_K_M unchanged

This matches a -DGGML_CUDA_FORCE_CUBLAS=ON build (pp128 1464.2 t/s),
which confirms the fallback path is the one being taken.

This is not MUSA-specific: any device with less than 48 KiB per-block
shared memory is affected.

Co-authored-by: KakaruHayate KakaruHayate@users.noreply.github.com

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b10180

Choose a tag to compare

@github-actions github-actions released this 29 Jul 14:34
11b068d

sycl: contiguous fast path + 32-bit index math for unary elementwise ops (#25946)

  • sycl: contiguous fast path + 32-bit index math for unary elementwise ops

  • sycl: use fastdiv for elementwise index math

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b10179

Choose a tag to compare

@github-actions github-actions released this 29 Jul 13:50
e2f59ed

b10178

Choose a tag to compare

@github-actions github-actions released this 29 Jul 13:01
992c325

server : add trace logging for slot similarity checking (#26271)

Adds trace logging in server-context.cpp for slot similarity checking
during prompt cache slot selection, including skip reasons and similarity
calculation details.

Assisted-by: llama.cpp:Qwen3.6-27B

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b10176

Choose a tag to compare

@github-actions github-actions released this 29 Jul 08:49
f5b9bd3

b10175

Choose a tag to compare

@github-actions github-actions released this 29 Jul 08:05
60bccc3