What Happened
The developments documented here are primarily inference-engine changes, not new open-weight model releases. In llama.cpp and ggml, a CUDA fix addresses a build failure introduced after chunked CUDA graphs, while a qwen4exp fix corrects a multi-stream tensor-shape mismatch that could trigger a graph-reserve assertion [1][3]. Other changes add a Hexagon CEIL operation for…
What Happened
The documented activity is concentrated in llama.cpp, not new open-weight model releases. It adds MiniCPM-OCR 1.0 support, Nemotron compatibility for non-standard grouped-query attention and tokenization, and fixes a DFlash drafting failure with qwen4exp targets [2][6][4].
Inference and platform work includes Adreno OpenCL depthwise convolution, SYCL convolution correctness fixes, and Intel XMX changes aimed…
What Happened
The developments in these notes are concentrated in llama.cpp, not new model-weight releases. They include a fused QSA scoring and top-k masking operation used by several attention paths, with fallbacks retained; changes that keep a row-setting operation on the GPU; faster Q8 dequantization on SYCL; and vectorized repacking for Snapdragon Hexagon targets [2][8][1][10].…
What Happened
Recent llama.cpp changes focus on inference correctness and deployment coverage rather than a new model release. Its Anthropic-compatible /v1/messages endpoint now passes a requested id_slot through to the server, so a proxy can direct a conversation to the slot holding its KV cache. Previously, automatic slot selection could trigger a full re-prefill for…
What Happened
SGLang 0.5.22 expanded support for language, embedding, image and video models, including DeepSeek-V4.1 and EmbeddingGemma 2. It also updated distributed serving, caches, hardware backends and API behavior. Its reported performance gains include up to 2.3× more DeepSeek-V4.1 throughput per H200 GPU, but those figures should be treated as workload-specific benchmarks, not general capacity…
What Happened
The clearest developments are in inference software, not new open weights. vLLM v0.32.0 expands model, embedding, vision, ASR, Transformers-backend, and speculative-decoding support. It also adds low-precision KV-cache options and serving optimizations. Its release notes report 43–85% higher short-request throughput in benchmarks using least-loaded API-worker routing; that result should not be assumed for other…
What Happened
Recent llama.cpp builds add support for Kolibri-1, a German/English mixture-of-experts reasoning model with 78 billion parameters and 3.46 billion active per token. The integration covers its FP8 checkpoint, tokenizer, expert routing, and alternating sliding-window and full-attention layers [7].
Tooling also moved forward. Discover Models offers an optional, searchable Hugging Face Hub view with…
What Happened
Reflection’s Beam, its first open-weight model, is available through Ollama Cloud for software engineering, reasoning and long-running agent tasks. Cloud access is free for a limited time; that offer is separate from the model’s open-weight status. [11]
Recent llama.cpp work adds conversion and server support for IBM’s Granite Switch 4.1 8B decision model…
What Happened
llama.cpp 0.7.0 expands support for text and multimodal models, including K2 Horizon/MoVA, Prism Bonsai 2, EmbeddingGemma-2, LiquidAI d1, MiniCPM-V 4.7 and new Nemotron models. It also adds speculative-decoding options, embedding-memory improvements and a multi-GPU cache for host-offloaded mixture-of-experts models. The reported gains—up to roughly 2.2× faster decoding and 10× lower embedding VRAM use—depend…
What Happened
The reported activity is concentrated in inference tooling, not new model weights. The notes identify no new weight releases or substantive Hugging Face, vLLM, SGLang, EleutherAI, or LAION announcements. llama.cpp changes span AMD and Adreno kernels, Vulkan compatibility, embedding graphs, template output, and RPC diagnostics [1][2][3][4][5][6][7][8][9].
Hardware support is getting more precise. Vulkan…
What Happened
The developments documented here are concentrated in llama.cpp, not new model-weight releases. They include multi-device Vulkan access to GPU-visible host memory, a change to avoid graph reallocations in the “no cache” path, and an initial Q8_0 matrix-multiplication prototype for zdnn. Qwen3.8 Flash Next can now accept image embeddings for MTP. These are compatibility…
What Happened
Recent llama.cpp changes focus on backend correctness, model compatibility and inference performance rather than a new open-weight model release. Support for vLLM D3 models and the pplx-decider model was added alongside tighter validation; this is llama.cpp compatibility work, not a vLLM inference-engine release [3][4].
Other changes address specific hardware paths: Hexagon NPU DMA…