Open-Source Models & Communities

What Happened The developments in these notes are concentrated in llama.cpp and its ggml backends. They document inference and deployment tooling changes, not a new open-weight model release. There is no substantive update here to attribute to Ollama, vLLM, SGLang, EleutherAI, or LAION. SYCL tensor-split loading now batches contiguous rows into pinned staging slots, with…

What Happened Recent llama.cpp builds focus on reliability, model compatibility and targeted acceleration rather than a new model release. Fixes address truncated GGUF data during merge, dropped slices in 4D tensor quantization, excessive CPU and memory use in a sampling path, and grammar recursion that could overflow the stack. Qwen conversion and memory-cache behavior also…

What Happened The developments documented here concern llama.cpp and ggml inference tooling, not a verified release of new model weights. A fix passes quantization scales into three feed-forward call sites for MiniMax M3.1 Flash; without them, routed-expert weights could overwhelm the residual and produce garbled output [3]. Related changes simplify mixture-of-experts (MoE) rescaling and block…

What Happened The documented activity is concentrated in llama.cpp, not new open-weight model releases. It adds MiniCPM-OCR 1.0 support, Nemotron compatibility for non-standard grouped-query attention and tokenization, and fixes a DFlash drafting failure with qwen4exp targets [2][6][4]. Inference and platform work includes Adreno OpenCL depthwise convolution, SYCL convolution correctness fixes, and Intel XMX changes aimed…

What Happened The developments in these notes are concentrated in llama.cpp, not new model-weight releases. They include a fused QSA scoring and top-k masking operation used by several attention paths, with fallbacks retained; changes that keep a row-setting operation on the GPU; faster Q8 dequantization on SYCL; and vectorized repacking for Snapdragon Hexagon targets [2][8][1][10].…

What Happened Recent llama.cpp changes focus on inference correctness and deployment coverage rather than a new model release. Its Anthropic-compatible /v1/messages endpoint now passes a requested id_slot through to the server, so a proxy can direct a conversation to the slot holding its KV cache. Previously, automatic slot selection could trigger a full re-prefill for…

What Happened SGLang 0.5.22 expanded support for language, embedding, image and video models, including DeepSeek-V4.1 and EmbeddingGemma 2. It also updated distributed serving, caches, hardware backends and API behavior. Its reported performance gains include up to 2.3× more DeepSeek-V4.1 throughput per H200 GPU, but those figures should be treated as workload-specific benchmarks, not general capacity…

What Happened The clearest developments are in inference software, not new open weights. vLLM v0.32.0 expands model, embedding, vision, ASR, Transformers-backend, and speculative-decoding support. It also adds low-precision KV-cache options and serving optimizations. Its release notes report 43–85% higher short-request throughput in benchmarks using least-loaded API-worker routing; that result should not be assumed for other…

What Happened Recent llama.cpp builds add support for Kolibri-1, a German/English mixture-of-experts reasoning model with 78 billion parameters and 3.46 billion active per token. The integration covers its FP8 checkpoint, tokenizer, expert routing, and alternating sliding-window and full-attention layers [7]. Tooling also moved forward. Discover Models offers an optional, searchable Hugging Face Hub view with…

What Happened Reflection’s Beam, its first open-weight model, is available through Ollama Cloud for software engineering, reasoning and long-running agent tasks. Cloud access is free for a limited time; that offer is separate from the model’s open-weight status. [11] Recent llama.cpp work adds conversion and server support for IBM’s Granite Switch 4.1 8B decision model…

What Happened llama.cpp 0.7.0 expands support for text and multimodal models, including K2 Horizon/MoVA, Prism Bonsai 2, EmbeddingGemma-2, LiquidAI d1, MiniCPM-V 4.7 and new Nemotron models. It also adds speculative-decoding options, embedding-memory improvements and a multi-GPU cache for host-offloaded mixture-of-experts models. The reported gains—up to roughly 2.2× faster decoding and 10× lower embedding VRAM use—depend…