Open-Source Models & Communities

What Happened SGLang 0.5.21 expands support for open-weight and other models, including DeepSeek-V4.1 Flash and several vision and diffusion models. It also makes its Rust radix-tree cache core the default, improves prefill/decode serving and KV-cache handling, and adds classification and candidate-scoring APIs. Its reported 22% improvement in first-token time for DeepSeek-V4.1 on long prompts and…

What Happened Recent llama.cpp releases focus on inference reliability and model compatibility, rather than announcing new model weights. A direct-I/O change avoids making a second full-size copy of each tensor during memory mapping, while another fixes a workqueue race that could leave read and write state out of sync. Backend changes address a ROCm hardware-detection…

Findings [1] 2026-09-29 b11259 common : stop accepting draft tokens at EOG (#29638) common : stop accepting draft tokens at EOG cont : remove the test Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51199980 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…

Findings [1] 2026-09-28 b11236 batch: migrate speculative, mtmd and server to batch_ext (#29385) adapt common add common_batch wip wip: spec cont common_speculative_process server_batch to use common_batch rm some stale calls Assisted-by: Claude Fable 5.1 migrate mtmd handle imrope, handle return val of add()/add_embd() add spec zeros vector add warning on zero fill path Website:…

Findings [1] 2026-09-27 b11222 common : avoid side effects around params parsing (#29537) register --rpc unconditionally and call llama_supports_rpc() only from its handler print server "initialization ..." log after args are parsed Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50580981 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…

Findings [1] 2026-09-26 b11201 Revert "Change max context length for auto-fitting with unified KV (#28849)" (#29437) This reverts commit b04d4e5. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50433369 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan)…

Findings [1] 2026-09-25 b11189 opencl: add bin kernel kernel_gemm_noshuffle_q5_k_f32_32b_trans_ila_a8_bin, kernel_gemm_noshuffle_q5_k_q8_1_dp4a_ila_a8_bin (#29401) opencl: add A8 Q5_K non-MoE non dp4a + dp4a binary kernel opencl: fix s transpose - s only transposed for bin kernels Co-authored-by: Li He lih@qti.qualcomm.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50281199 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…

Findings [1] 2026-09-24 b11163 llama: add llama_batch_ext (#24669) (wip) add llama_batch_ext wip updated design updated impl change signature unused var demo common_prompt_batch_decode fix pos tmp disable test-batch-alloc fix compat nits: add const no more pos_max add comment about llama_batch_ext_set_embd_state handle n_embd_out properly rename api --> embd_token llama_embd stub llama_batch_ext_set_embd_state support both token + embd…

Findings [1] 2026-09-23 b11147 opencl: add A8 Q6_K non-MoE dp4a binary kernel (#29057) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49630180 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12)…

Findings [1] 2026-09-22 b11112 server: support input_image in function_call_output (#20663) (#22575) server: support input_image in function_call_output (#20663) server: fix if statement spacing server: avoid repeated type lookup Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49330467 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…