Skip to content Skip to footer

Which AI Library Updates Matter for Production: LiteLLM Security Fixes and Ollama Runtime Changes

What Happened

LiteLLM: Version 1.108.14 LTS fixes Claude tool search disappearing on Anthropic Responses API follow-up turns, which could cause 400 errors, and adds OpenAI input_json content-part support on /v1/decisions [1]. Recent releases across the 1.106, 1.108 LTS, 1.109, and 1.110 lines also fix GPT-6.2 prompt-cache misses and a /key/info authorization flaw that could expose other userless keys by hash or alias [4][5][6][8]. Version 1.110.2 additionally addresses Responses API stream restarts or duplication, JWT mapping delays, Anthropic request errors, pricing, and memory growth [8]. Version 1.111.0-rc.3 remains a release candidate and includes further auth, spend-tracking, SSO, Postgres connection, and memory fixes [7].

Ollama: Version 0.40.17 adds Inkling 2 Mini, a model reported to support text, images, tools, and thinking. Its release notes also describe deferred tool loading in launch integrations, concurrent Apple-silicon requests, Vulkan and Windows iGPU fixes, and CLI and API corrections. The reported SWE-Bench Verified score is a model claim, not evidence of performance on a particular workload [2]. Related mlxrunner work adapts saved Metal prefix history to available memory and reclaims cold history under pressure [10].

Streamlit: A 1.67.1 development build was listed without release notes; there is no basis here to identify its changes [9]. No breaking changes are documented in these notes, but that is not a compatibility guarantee.

Why It Matters to Businesses

The LiteLLM /key/info fix is the clearest security-driven upgrade reason. Cache, streaming, and tool-search fixes also affect operating cost and reliability: missed cache hits can increase token charges, while malformed follow-up turns or duplicate streams can disrupt agent workflows [1][5][6][8]. Ollama’s concurrency and memory changes may improve local-model capacity, but require testing against the hardware and request patterns used in production [2][10].

Kimbodo Engineering Perspective

We would treat these as separate decisions, not a blanket dependency refresh. Prioritize a patched LiteLLM line where userless keys or the affected API paths are in use; choose LTS when patch stability matters, and keep 1.111.0-rc.3 out of production unless a required fix justifies release-candidate risk [1][7][8]. Test Ollama as a runtime change: better memory reclamation is valuable only if it preserves latency and reuse under concurrent load [2][10]. A Streamlit nightly without documented changes should not enter a production upgrade plan [9].

How We Would Implement It

  • Inventory deployed LiteLLM versions, key types, provider routes, caching settings, and image variants. Select a patched release and pin its image by digest rather than relying on the moving lts tag [1][5][6][8].
  • Verify LiteLLM images with the release-specified Cosign certificate identity and GitHub Actions OIDC issuer, including database or non-root variants where deployed [1][4][8].
  • In staging, test /key/info isolation, JWT mapping updates, GPT-6.2 cache hits with and without store: false, Anthropic follow-up tool calls, and streamed Responses API requests. Compare token accounting, errors, memory, and Postgres connections before promotion [1][6][7][8].
  • Benchmark Ollama on target Apple-silicon, Vulkan, or Windows iGPU hosts with concurrent requests and memory pressure. Check model reloads, tool schemas, and thinking controls in the integrations the business uses [2][10].

Risks, Costs and Security

Leaving affected LiteLLM deployments unpatched risks cross-key information exposure; cache misses and untracked spend can also distort costs [5][7][8]. Conversely, upgrading a gateway or local-model runtime can change provider behavior, memory use, and latency even when release notes say no upgrade action is required. Use canary traffic, observable cost and error baselines, and a rollback-ready image digest. Do not treat an unannotated nightly or a release candidate as equivalent to a tested stable release [7][9].

Where Kimbodo Comes In

Kimbodo builds and operates this in production for businesses — see our AI Application Development practice, or Estimate My AI Application.

Sources

  1. [1] 1.108.14 (LTS)
  2. [2] v0.40.17
  3. [4] v1.109.3
  4. [5] v1.106.12
  5. [6] 1.108.13 (LTS)
  6. [7] v1.111.0-rc.3
  7. [8] v1.110.2
  8. [9] 1.67.1.dev20261119
  9. [10] v0.40.17-rc2: mlxrunner: adapt saved history to memory pressure (#19678)

Leave a comment

0.0/5

Engineering Founded & Customer Focused

KIMBODO LLC. © 2026. All Rights Reserved.