Skip to content

Releases: ggml-org/llama.cpp

Release list

b11946

b11946 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 28 Oct 01:58
b11946
8ca4060
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

server : add realtime conversation and transcription on /v1/realtime (#31269)

  • server : add realtime conversation and transcription on /v1/realtime

Co-authored-by: Xuan Son Nguyen son@huggingface.co
Assisted-by: Claude Opus 5.5

  • examples : default the realtime client to the server port 9931

Assisted-by: Claude Opus 5.5

  • server : sample realtime responses right after their own decode

Slots respond in turns on one context: sampling at the start of the next step read the logits of whichever slot decoded last, so concurrent sessions mixed their outputs.

Assisted-by: Claude Opus 5.5

  • move more stuff to server-realtime, making it self-contained

  • examples : fix the type check of the realtime client

Assisted-by: Claude Opus 5.5

  • examples : use the model card prompts in the realtime client

Assisted-by: Claude Opus 5.5

Co-authored-by: Xuan Son Nguyen son@huggingface.co

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b11945

b11945 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 28 Oct 01:41
b11945
d05706b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B (#31275)

  • model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B

Assisted-by: Claude Opus 5.5

  • convert : keep the speech task templates out of the d1 GGUFs

Assisted-by: Claude Opus 5.5

  • small refactoring: no_batching

Co-authored-by: Xuan Son Nguyen son@huggingface.co

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b11944

b11944 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 28 Oct 01:24
b11944
1b4554e
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

server: expose the reasoning effort levels of the chat template (#31271)

  • server: expose the reasoning effort levels of the chat template

The capability probe renders a user turn for each candidate level
(minimal, low, medium, high, xhigh, max) and for an unknown level. A
level is honored when it renders without raising and differently from
the unknown level, which shows the template fallback. Levels sharing a
render are aliases and the highest one is kept.

The caps serialize to JSON, and chat_template_caps in /props reports
the list as reasoning_effort_levels, from lowest to highest.

  • jinja: keep the caps free of the json dependency

The caps keep reporting their flags as a map, and the effort levels
stay a plain member. common_chat_templates_get_caps() builds the JSON
the server reports from both, so jinja depends on json only for the
probe inputs.

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b11941

b11941 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 28 Oct 01:03
b11941
19ff817
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B (#31257)

  • model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B

Co-authored-by: Xuan Son Nguyen son@huggingface.co
Assisted-by: Claude Opus 5.5

  • convert : read the LFM3-Audio config.json as released

A transformers that knows lfm3_audio rewrites the config into another layout (no lfm/encoder keys).

Assisted-by: Claude Opus 5.5

  • mtmd : disable flash attention for the LFM3-Audio depthformer

Assisted-by: Claude Opus 5.5

  • tts : add a voice option for LFM3-Audio and LFM3-Omni

The model card offers a female and a male voice; --tts-voice for llama-tts, "voice" for the server /tts.

Assisted-by: Claude Opus 5.5

  • clean up template handling

  • convert : detect LFM3-Audio and LFM3-Omni by model_type

Assisted-by: Claude Opus 5.5

  • move banned tokens --> logits bias --> mtmd_helper

  • leave mtmd-cli unchanged

  • no more template_asr_system

  • left over

  • llama : keep the LFM3-Audio encoder convs and codec unquantized

Assisted-by: Claude Opus 5.5

  • common, mtmd : shorten comments

Assisted-by: Claude Opus 5.5

Co-authored-by: Xuan Son Nguyen son@huggingface.co

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b11940

b11940 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 28 Oct 00:32
b11940
eb7818c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

common : support deferred tools and tool_reference content parts (#31256)

  • common : support deferred tools and tool_reference content parts

Assisted-by: Claude Opus 5.5

  • common : credit the deferred tools co-author

Co-authored-by: Aldehir Rojas 765227+aldehir@users.noreply.github.com

Assisted-by: Claude Opus 5.5

  • tests : fix the type check of the deferred tools test

Assisted-by: Claude Opus 5.5

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b11938

b11938 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 27 Oct 23:57
b11938
19cce69
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B (#31252)

  • model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B

Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co
Co-authored-by: Xuan-Son Nguyen son@huggingface.co
Assisted-by: Claude Opus 5.5

  • tests : set the SWA rope base of the LFM3 test model

Assisted-by: Claude Opus 5.5

  • llama : enable recurrent state rollback for LFM3

Assisted-by: Claude Opus 5.5

  • Apply batched suggestions from code review

Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co
Co-authored-by: Tarek Dakhran github@dakhran.com

  • gguf-py : align the lfm3vl mapping comments

Assisted-by: Claude Opus 5.5

  • llama : fix CRLF line endings from the review suggestions

Assisted-by: Claude Opus 5.5

  • tests : add the LFM3 Metal fusion baseline

Assisted-by: Claude Opus 5.5

  • llama : keep the LFM3-VL depthwise conv kernels in F32 when quantizing

Assisted-by: Claude Opus 5.5

Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co
Co-authored-by: Xuan-Son Nguyen son@huggingface.co

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b11936

b11936 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 27 Oct 20:11
b11936
27b7a8a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

llama : build the k-pool sparse selection independently of the kq_mask shape (#31243)

  • llama : build the k-pool sparse selection independently of the kq_mask shape

qwen4exp (QSA) and glm5-next pick cache cells by index and no longer read the kq_mask values, but they still shaped the selection like the dense kq_mask. Build the selection as [n_kv, n_tokens] over the cache cells instead, and split it into the streams of the K view right before the attention.

This lets the dense mask change its layout (e.g. per-sequence row lists) without touching the sparse layers.

  • improve comment

Co-authored-by: Georgi Gerganov ggerganov@gmail.com

  • llama : drop the unused build_fused_top_k_mask args, review nits

Co-authored-by: Georgi Gerganov ggerganov@gmail.com

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b11935

b11935 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 27 Oct 17:15
b11935
7ecb57f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

rpc : catch std::length_error in recv_msg (#31204)

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b11934

b11934 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 27 Oct 15:50
b11934
8542e94
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

tests : speed up tensor data generation and model saving (#31145)

  • tests : speed up tensor data generation and model saving

Refactor save_models to create only the model, not a llama_context,
which is not needed for saving. Replace the slow std::normal_distribution
draw with a splitmix64 + box-muller normal generator.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)

  • tests : use a shared pool of normal draws for tensor data

Generate a pool of normal draws once with std::mt19937 and
std::normal_distribution, then have every tensor copy a cyclic slice of
it instead of running a per-tensor box-muller. The name-keyed slice
selection keeps the draw deterministic.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)

  • tests : log the size of saved models

Report the produced file size in bytes and MB after each save.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)

  • tests : simplify the model save log

Put the model name, target and size on one line per saved model.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b11933

b11933 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 27 Oct 15:27
b11933
922f53e
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

vulkan: fix CI llvmpipe by allowing dim 3 broadcast in MUL_MAT (#31239)

  • vulkan: allow dim 3 broadcast in MUL_MAT

supports_op rejected every MUL_MAT whose src0 and src1 differ in ne3,
although mul_mm, mul_mmq and mul_mat_vec already broadcast src0 over
dim 3 through broadcast3. Such a matmul then ran on the CPU with a
copy of the weight, and the split layout changed with the number of
streams in the ubatch.

Keep the strict check for MUL_MAT_ID only, and route the broadcast
cases away from mul_mat_vec_nc, the one path that requires equal ne3.

  • tests : harden MUL_MAT with permuted dim 3 broadcast cases

Harden the MUL_MAT coverage with a weight broadcast over dim 3 against
a permuted per-stream activation, the shape of an MLA output
projection with a non unified KV cache, for a single token, several
heads and several tokens per stream.

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading