Skip to content Skip to sidebar Skip to footer

Open-Source Models & Communities — September 29, 2026

Findings [1] 2026-09-29 b11259 common : stop accepting draft tokens at EOG (#29638) common : stop accepting draft tokens at EOG cont : remove the test Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51199980 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…

Read More

Open-Source Models & Communities — September 28, 2026

Findings [1] 2026-09-28 b11236 batch: migrate speculative, mtmd and server to batch_ext (#29385) adapt common add common_batch wip wip: spec cont common_speculative_process server_batch to use common_batch rm some stale calls Assisted-by: Claude Fable 5.1 migrate mtmd handle imrope, handle return val of add()/add_embd() add spec zeros vector add warning on zero fill path Website:…

Read More

Open-Source Models & Communities — September 27, 2026

Findings [1] 2026-09-27 b11222 common : avoid side effects around params parsing (#29537) register --rpc unconditionally and call llama_supports_rpc() only from its handler print server "initialization ..." log after args are parsed Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50580981 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…

Read More

Open-Source Models & Communities — September 26, 2026

Findings [1] 2026-09-26 b11201 Revert "Change max context length for auto-fitting with unified KV (#28849)" (#29437) This reverts commit b04d4e5. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50433369 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan)…

Read More

Open-Source Models & Communities — September 25, 2026

Findings [1] 2026-09-25 b11189 opencl: add bin kernel kernel_gemm_noshuffle_q5_k_f32_32b_trans_ila_a8_bin, kernel_gemm_noshuffle_q5_k_q8_1_dp4a_ila_a8_bin (#29401) opencl: add A8 Q5_K non-MoE non dp4a + dp4a binary kernel opencl: fix s transpose - s only transposed for bin kernels Co-authored-by: Li He lih@qti.qualcomm.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50281199 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…

Read More

Open-Source Models & Communities — September 24, 2026

Findings [1] 2026-09-24 b11163 llama: add llama_batch_ext (#24669) (wip) add llama_batch_ext wip updated design updated impl change signature unused var demo common_prompt_batch_decode fix pos tmp disable test-batch-alloc fix compat nits: add const no more pos_max add comment about llama_batch_ext_set_embd_state handle n_embd_out properly rename api --> embd_token llama_embd stub llama_batch_ext_set_embd_state support both token + embd…

Read More

Open-Source Models & Communities — September 23, 2026

Findings [1] 2026-09-23 b11147 opencl: add A8 Q6_K non-MoE dp4a binary kernel (#29057) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49630180 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12)…

Read More

Open-Source Models & Communities — September 22, 2026

Findings [1] 2026-09-22 b11112 server: support input_image in function_call_output (#20663) (#22575) server: support input_image in function_call_output (#20663) server: fix if statement spacing server: avoid repeated type lookup Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49330467 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…

Read More

Illustration for the Kimbodo News & Research briefing “How Recent llama.cpp Changes Make Cross‑Platform Inference More Practical — and What That Means for Production AI” (Open-Source Models & Communities).

How Recent llama.cpp Changes Make Cross‑Platform Inference More Practical — and What That Means for Production AI

What Happened Over the last update cycle ggml/llama.cpp received a set of operational, portability and performance changes that materially affect how open weights and inference engines are deployed in production: llama-server gained environment‑variable control via new LLAMA_ARG_* mappings (e.g., LLAMA_ARG_TEMP, LLAMA_ARG_TOP_P, LLAMA_ARG_REPEAT_PENALTY), enabling systemd/EnvironmentFile driven configuration for runtime sampling parameters; documentation was regenerated…

Read More

Illustration for the Kimbodo News & Research briefing “Why Recent llama.cpp Upgrades Make Open Models Safer, Faster and Easier to Deploy in Production” (Open-Source Models & Communities).

Why Recent llama.cpp Upgrades Make Open Models Safer, Faster and Easier to Deploy in Production

What Happened Kernel specialization for variable HC: A metal backend change allows dsv4_hc_pre kernels to accept arbitrary hc values (previously hardcoded to 4). The patch passes n_hc as a function constant, generates per-n_hc pipeline variants and adds tests for many hc values — fixing a fallback-to-CPU failure for models that vary hc by…

Read More

How New Open-Source Inference Tooling Widens Hardware Reach and Lowers Latency — Practical Steps to Deploy Safely

What Happened Over the last development cycle the open-source inference ecosystem delivered two parallel waves of work: (1) broad, low-level backend and kernel hardening in the llama.cpp/ggml ecosystem that expands supported accelerators and fixes stability/performance bugs, and (2) a major inference/runtime release that adds schedulers, memory/residency improvements, new models and language/runtime tooling for high‑throughput production…

Read More

How Recent llama.cpp Engine Updates Improve Cross‑Platform Inference Performance and Reliability

What Happened Over the last set of commits to ggml / llama.cpp the community shipped multiple engineering changes that materially affect inference performance, portability and robustness for open‑weight models. Key changes include: New binary matmul kernels (including A8 Q6_K non‑MoE and IQ3_S MMQ kernels) and layout fixes that broaden high‑performance kernel coverage across…

Read More

Engineering Founded & Customer Focused

KIMBODO LLC. © 2026. All Rights Reserved.