How to pick and operate agent frameworks (LangChain, AutoGen, Semantic Kernel, Claude Code and peers) for production AI

What Happened

Multiple agent frameworks and tooling projects aim to simplify building “agentic” applications: orchestrating models, tools, retrieval, memory and multi-step plans. Common capabilities across these projects include tool adapters, planner/chain abstractions, session/state management, retrieval-augmented generation (RAG) integrations, and connectors to vector databases and external APIs.

Separately, a recent maintenance release for Anthropic’s developer tooling fixed two reliability issues: Claude Code failed to launch on macOS 12 (Monterey) because of a regression, and remote/scheduled sessions could fail with the error “user messages must have non-empty content” caused by a re-sent permission approval that could not be applied [1]. This is a concrete example of how subtle session and permission state bugs in agent runtimes can break scheduled and remote workflows [1].

Why It Matters to Businesses

• Operational reliability: Agents introduce additional state, scheduling and external-tool dependencies beyond single-turn LLM calls. Session and permission bugs can silently break automations and scheduled jobs (example: Claude Code fix) [1].
• Time-to-value vs. control trade-off: Frameworks like LangChain prioritize rapid prototyping and many connectors; SDKs (Semantic Kernel, OpenAI Agents) trade off some convenience for tighter integration and more deterministic runtime behavior.
• Cost and latency: Multi-step chains, retrieval calls and tool executions increase token usage, API calls and tail latency; design choices materially affect run cost and user experience.
• Security and compliance: Agents can execute actions and call external systems. Without strict auth, sandboxing and auditing, agents amplify data-exfiltration and regulatory risk.
• Vendor lock-in and portability: Provider-specific agent SDKs (Claude Code, OpenAI Agents SDK, Semantic Kernel) can accelerate delivery but increase migration cost compared with provider-agnostic frameworks (LangChain, AutoGen, LlamaIndex).

Kimbodo Engineering Perspective

When building production agentic systems we evaluate frameworks on three pragmatic axes: runtime determinism and observability, extensibility for secure tool integration, and cost/latency control.

• LangChain / LangGraph: Best for rapid iteration and wide connector availability. Use when you need many built-in adapters (APIs, data sources). Expect to add significant production-grade logging, retries and state management layers.
• LlamaIndex: Use when the primary challenge is indexing and retrieval of enterprise data. It’s data-first and pairs with vector stores for RAG scenarios.
• AutoGen / CrewAI / multi-agent libraries: Useful when you need explicit multi-agent roles, structured dialogues, or orchestration patterns. They reduce glue code for complex agent interactions but increase operational complexity.
• Semantic Kernel / OpenAI Agents SDK / Claude Code: Prefer these when you want tighter, vendor-optimized execution models, built-in planners, and potentially managed session handling. But plan for vendor-specific behaviors and upgrade testing (the Claude Code session bug is an example of a vendor runtime regression you must detect) [1].
• PydanticAI / DSPy / schema-first tooling: Adopt these for strict input/output validation and typed tool interfaces — they reduce hallucination-driven errors by enforcing shape and type constraints before actions execute.

Trade-offs we make at Kimbodo: prefer modular architectures that allow swapping the agent orchestration layer without reworking tool adapters; invest early in observability (structured traces for chains and tool calls); and prototype on framework-agnostic stacks, then lock to a vendor SDK only if runtime guarantees or integrations justify it.

How We Would Implement It

Concrete architecture and rollout plan we use for production agentic applications:

Reference architecture

• Model & provider layer: Abstract a provider interface (OpenAI, Anthropic, open models) behind a small adapter. Keep tokenization, batching and rate limiting centralized.
• Agent orchestration layer: Start with a framework that matches your needs (LangChain/AutoGen for connectors and rapid prototyping; Semantic Kernel or OpenAI Agents SDK for tighter runtimes). Implement a thin runtime shim so you can change frameworks without rewriting tool adapters.
• Tool adapter & execution layer: Each tool implements a typed contract (use Pydantic-style schemas) with explicit auth, quota enforcement, and execution sandboxes for untrusted code or shell calls.
• Data & retrieval: Vector DB + index management using LlamaIndex or direct integrations. Persist metadata and provenance with each retrieval for audit and RLHF data collection.
• Session & state store: Use Redis/ElastiCache or DynamoDB for session state, locking and permission approvals. Implement optimistic retries and idempotency to avoid issues like empty-message failures from permission re-sends (see example) [1].
• Orchestration & scheduling: Use a workflow runner (e.g., serverless functions + queued tasks or Airflow/Temporal) for long-running or scheduled agents, with heartbeat and retry policies.
• Observability & provenance: Emit structured traces for each chain step, tool call, and model response. Correlate with user/session IDs and store deterministic logs for replay and audits.

Implementation steps

1. Define clear success criteria for the agent (SLOs for latency, accuracy and cost).
2. Prototype core flows with a framework that minimizes initial engineering (LangChain/AutoGen + vector DB for RAG).
3. Introduce typed tool contracts (Pydantic-style schemas) and enforce them at the adapter boundary.
4. Add session/state persistence and staged rollouts for scheduled tasks; include regression tests that simulate permission re-sends and session edge-cases to catch bugs like the one fixed in Claude Code [1].
5. Harden security (least privilege for tool credentials, secrets manager, encrypted storage) and add red-team tests for prompt injection and data exfiltration.
6. Move from prototype to production by swapping in the chosen runtime SDK (e.g., OpenAI Agents SDK or vendor SDK) only after validating runtime guarantees and implementing additional observability hooks.

Risks, Costs and Security

• Operational risks: Session state regressions, scheduler edge-cases and permission race conditions can break workflows; add end-to-end tests and synthetic monitoring to detect these issues early (example: scheduled sessions failing due to permission re-sends) [1].
• Security risks: Tool invocation can be abused for data exfiltration or remote code execution. Mitigations: sandboxing, least-privilege credentials, typed tool schemas, runtime policy enforcement, and thorough auditing of all tool outputs.
• Cost risks: Multi-step plans increase token and API costs. Mitigations: token budgets per session, caching of RAG results, and fallback heuristics to reduce unnecessary model calls.
• Compliance & provenance: Agents must log inputs, decisions, and tool outputs for compliance. Maintain immutable audit trails and data retention policies aligned with regulation.
• Supply chain & vendor risk: Relying on vendor SDKs speeds delivery but requires a plan for breaking changes and regressions. Maintain an adapter layer and automated integration tests to reduce migration friction.

Implement monitoring that alerts on semantic failure modes (sudden change in successful tool calls, spike in empty responses, retries) and include playbooks for rolling back framework or provider updates when regression indicators appear — the recent Claude Code fixes illustrate the need for that discipline [1].

Sources

1. [1] v2.1.258