Findings [1] 2026-09-29 b11259 common : stop accepting draft tokens at EOG (#29638) common : stop accepting draft tokens at EOG cont : remove the test Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51199980 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
Findings [1] 2026-09-28 b11236 batch: migrate speculative, mtmd and server to batch_ext (#29385) adapt common add common_batch wip wip: spec cont common_speculative_process server_batch to use common_batch rm some stale calls Assisted-by: Claude Fable 5.1 migrate mtmd handle imrope, handle return val of add()/add_embd() add spec zeros vector add warning on zero fill path Website:…
Findings [1] 2026-09-27 b11222 common : avoid side effects around params parsing (#29537) register --rpc unconditionally and call llama_supports_rpc() only from its handler print server "initialization ..." log after args are parsed Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50580981 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
Findings [1] 2026-09-26 b11201 Revert "Change max context length for auto-fitting with unified KV (#28849)" (#29437) This reverts commit b04d4e5. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50433369 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan)…
Findings [1] 2026-09-25 b11189 opencl: add bin kernel kernel_gemm_noshuffle_q5_k_f32_32b_trans_ila_a8_bin, kernel_gemm_noshuffle_q5_k_q8_1_dp4a_ila_a8_bin (#29401) opencl: add A8 Q5_K non-MoE non dp4a + dp4a binary kernel opencl: fix s transpose - s only transposed for bin kernels Co-authored-by: Li He lih@qti.qualcomm.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50281199 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
Findings [1] 2026-09-24 b11163 llama: add llama_batch_ext (#24669) (wip) add llama_batch_ext wip updated design updated impl change signature unused var demo common_prompt_batch_decode fix pos tmp disable test-batch-alloc fix compat nits: add const no more pos_max add comment about llama_batch_ext_set_embd_state handle n_embd_out properly rename api --> embd_token llama_embd stub llama_batch_ext_set_embd_state support both token + embd…
Findings [1] 2026-09-23 b11147 opencl: add A8 Q6_K non-MoE dp4a binary kernel (#29057) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49630180 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12)…
Findings [1] 2026-09-22 b11112 server: support input_image in function_call_output (#20663) (#22575) server: support input_image in function_call_output (#20663) server: fix if statement spacing server: avoid repeated type lookup Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49330467 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…
What Happened
Over the last update cycle ggml/llama.cpp received a set of operational, portability and performance changes that materially affect how open weights and inference engines are deployed in production:
llama-server gained environment‑variable control via new LLAMA_ARG_* mappings (e.g., LLAMA_ARG_TEMP, LLAMA_ARG_TOP_P, LLAMA_ARG_REPEAT_PENALTY), enabling systemd/EnvironmentFile driven configuration for runtime sampling parameters; documentation was regenerated…
What Happened
Kernel specialization for variable HC: A metal backend change allows dsv4_hc_pre kernels to accept arbitrary hc values (previously hardcoded to 4). The patch passes n_hc as a function constant, generates per-n_hc pipeline variants and adds tests for many hc values — fixing a fallback-to-CPU failure for models that vary hc by…
What Happened
Over the last development cycle the open-source inference ecosystem delivered two parallel waves of work: (1) broad, low-level backend and kernel hardening in the llama.cpp/ggml ecosystem that expands supported accelerators and fixes stability/performance bugs, and (2) a major inference/runtime release that adds schedulers, memory/residency improvements, new models and language/runtime tooling for high‑throughput production…
What Happened
Over the last set of commits to ggml / llama.cpp the community shipped multiple engineering changes that materially affect inference performance, portability and robustness for open‑weight models. Key changes include:
New binary matmul kernels (including A8 Q6_K non‑MoE and IQ3_S MMQ kernels) and layout fixes that broaden high‑performance kernel coverage across…