What Happened
The developments in these notes are concentrated in llama.cpp, not a new open-weight model release. Runtime support was added for Prism Bonsai 2 27B, including loading and converter changes. OpenCL GPU decode support for Gemma-4 now covers E4B and E2B, alongside flash-attention and decode optimizations. These are compatibility changes; they do not establish…
What Happened
The documented updates center on llama.cpp rather than a broad wave of new model releases. The most consequential fix prevents a prompt-cache update from replacing the context in a busy server slot: when a request names an occupied id_slot, the server now waits instead of altering that slot while generation is in progress…
What Happened
Recent llama.cpp changes focus on inference correctness and deployment tooling rather than a new model release. Fixes address an out-of-bounds write in a tensor operation, a CUDA rounding discrepancy under MSVC, a sampling-graph mismatch that could abort decoding, and OpenCL compilation failures on Adreno A6x devices. [10] [4] [9] [5]
Other changes remove…
What Happened
The developments in these notes are concentrated in llama.cpp; they do not establish new model-weight releases or changes in Hugging Face, Ollama, vLLM, SGLang, EleutherAI or LAION. They show how community work on inference backends can affect production performance and reliability.
A CUDA TOP_K proposal selects different algorithms by workload size. In a…
What Happened
The developments in these notes are concentrated in llama.cpp. They document inference-engine changes, not a new open-weight model release or a substantive update from Hugging Face, Ollama, vLLM, SGLang, EleutherAI, or LAION.
On Apple Silicon, llama.cpp extended a Metal matrix-multiplication kernel to BF16 and several quantized formats. In M3 Ultra tests, it improved…
What Happened
The developments in these notes are concentrated in llama.cpp and ggml, not new open-weight model releases. llama.cpp added support for the pplx-decider model, while text, vision and audio support for embeddinggemma2 is tracked as a request rather than a confirmed release. [5] [8]
Inference changes include RPC support for tensor split mode, with…
What Happened
Two major open-source inference releases expand the choices for running models outside a managed API. llama.cpp v0.6.0 adds model and multimodal support, new processing and precision APIs, and performance improvements across several hardware backends. Its server now accepts typed vision, audio, and video inputs for embeddings and reports model input and output modalities…
What Happened
The documented activity centers on llama.cpp rather than new model weights. Recent changes include a fix for a chat tool-call parser use-after-free and double-free, a fix for a CUDA mixture-of-experts memory fault when expert count greatly exceeds microbatch size, improved Vulkan matrix-vector tuning for RDNA4 GPUs, and vectorized BF16, FP16, and FP32 tinyBLAS…
What Happened
Recent open-source activity in these notes centers on llama.cpp and its inference backends, not on a new open-weight model release. The changes add text-only support for the Clef decision model and fused MoE support for gemma-4, while improving Qwen3.5 MoE execution through the OpenVINO backend [8][6]. No new Hugging Face, Ollama, vLLM, SGLang,…
What Happened
SGLang 0.5.21 expands support for open-weight and other models, including DeepSeek-V4.1 Flash and several vision and diffusion models. It also makes its Rust radix-tree cache core the default, improves prefill/decode serving and KV-cache handling, and adds classification and candidate-scoring APIs. Its reported 22% improvement in first-token time for DeepSeek-V4.1 on long prompts and…
What Happened
Recent llama.cpp releases focus on inference reliability and model compatibility, rather than announcing new model weights. A direct-I/O change avoids making a second full-size copy of each tensor during memory mapping, while another fixes a workqueue race that could leave read and write state out of sync. Backend changes address a ROCm hardware-detection…
What Happened
Over the last set of community updates the llama.cpp / ggml ecosystem and adjacent tooling have added broad platform support, new numeric formats, and robustness fixes while vllm and Ollama pushed complementary standards and runtime features:
Numerical and precision work: BF16 support was expanded in ggml for unary, GLU, binary and…