How to Build Production-Ready AI Agent Platforms Without Losing Control of Cost, Security or Operations

What Happened

Enterprise AI platforms are moving from standalone chatbots toward managed agent infrastructure: identity, runtime isolation, tool gateways, memory, registries, evaluation, observability and workflow automation.

AWS expanded Bedrock AgentCore patterns around secure agent authentication, including Private Key JWT client authentication where AgentCore uses AWS KMS asymmetric signing keys to create short-lived JWT client assertions without exposing private keys. The flow supports machine-to-machine, on-behalf-of and user-delegated OAuth patterns, with CloudTrail auditability for token and signing calls [4].

Google Cloud’s Gemini Enterprise Agent Platform now emphasizes long-running agent execution, memory, identity, registry, gateway controls, evaluation and observability. Agent Runtime supports agents that can run continuously for up to seven days, while Agent Gateway and Model Armor are positioned as controls for prompt injection, tool poisoning and data leakage [6]. Google also describes a deterministic agent lifecycle using Agents CLI, ADK, function tools, code execution sandboxing, allow-listed egress and CI-based evaluation [5].

AWS also published examples of AgentCore connecting enterprise systems through MCP servers, semantic metadata, policy enforcement and Firecracker-isolated runtimes. The architectural direction is clear: agents are becoming orchestration layers over existing databases, SaaS systems, data lakes and operational APIs rather than isolated LLM prompts [7]. Amazon Quick examples show similar automation patterns for customer retention workflows using S3, indexed contact data, MCP actions, Lambda, API Gateway and generated documents [8].

At the same time, security research continues to show that agentic AI introduces new attack surfaces. A reported prompt-injection variant against Microsoft Copilot for Word used hidden document instructions that could be copied into model-generated output, creating self-propagating carrier documents with no class-wide mitigation reported at the time [2].

Why It Matters to Businesses

The business value of AI agents is not in giving a model access to more tools. It is in creating controlled, auditable automation that can safely act across business systems.

Agent platforms are becoming integration middleware. MCP servers, semantic layers and tool registries let teams connect AI workflows to ERP, CRM, data warehouses, object stores, search indexes and custom services without rebuilding every integration from scratch [7][9].
Identity is now a first-class architecture decision. Per-agent identities, OAuth token exchange, Private Key JWT, KMS-backed signing and non-repudiable audit trails matter because agents increasingly perform actions, not just answer questions [4][6].
Determinism is the difference between a demo and a production system. The strongest examples constrain the model to narrate or decide based on tool-produced outputs, while reconciliation, scoring, joining, deduplication and validation happen in deterministic code [5].
Cost shifts from model calls to platform operations. Long-running runtimes, gateways, memory stores, vector indexes, egress controls, observability pipelines, sandboxed execution and connector maintenance can become the dominant cost drivers.
Security risk expands with every tool connection. Prompt injection, data exfiltration, tool poisoning, over-permissioned identities and hidden content in documents are not edge cases; they are core failure modes for enterprise agents [2][6][7].

The pattern resembles earlier abstraction shifts in software. As D. Richard Hipp observed, SQL replaced custom software for querying large datasets by raising the level of abstraction [1]. AI agents may similarly raise the abstraction level for business workflows, but they do not remove the need for schemas, permissions, transaction boundaries, testing or operational controls.

Kimbodo Engineering Perspective

For most businesses, the right architecture is not “one autonomous agent connected to everything.” It is a controlled orchestration system with narrowly scoped agents, deterministic tools, explicit policies and measurable outputs.

Managed platforms reduce undifferentiated work, but increase platform dependency

Bedrock AgentCore and Gemini Enterprise can reduce the time required to implement identity propagation, tool gateways, runtime isolation, memory, registry and observability [4][6][7]. That is valuable when teams need production controls quickly. The trade-off is dependency on cloud-specific runtimes, policy models, observability formats and deployment workflows.

We would generally use managed agent infrastructure when the client already has a strong AWS or Google Cloud footprint, security teams trust the provider controls and speed to production matters more than maximum portability. We would favor a more portable orchestration layer when regulated workloads require cloud independence, custom governance or tight integration with existing platform engineering standards.

MCP is useful, but it is not a security boundary

MCP helps standardize how models and agents discover and call tools. It can reduce integration friction across Claude, ChatGPT and custom enterprise systems [9]. But a protocol does not solve authorization, input validation, output filtering, tenancy isolation, rate limiting, observability or blast-radius control. Those must be implemented in the gateway, identity layer and tool services.

Long-running agents need workflow engineering, not just prompts

Seven-day runtime support and persistent memory are useful for sales sequences, supply-chain monitoring and incident response [6]. But long-running agents introduce failure recovery, idempotency, replay, partial completion, timeout handling and state migration problems. In production, these should be designed like distributed workflows, with checkpoints and compensating actions.

Use models for interpretation, not for core control logic

The Google Industry Watch example is instructive: fetchers gather SEC filings and public claims, deterministic code reconciles and scores them, and the model narrates grounded results [5]. This is the architecture we prefer for high-stakes use cases. The model should not be the source of truth for joins, compliance classifications, financial calculations, entitlement checks or irreversible actions.

How We Would Implement It

1. Define agent boundaries by business capability

We would start by decomposing the target workflow into bounded capabilities: retrieve customer history, score churn risk, reconcile claims against filings, summarize operational anomalies, draft a response or create a ticket. Each capability gets its own service contract, permissions and evaluation criteria.

Use one agent per workflow domain, not one agent per enterprise.
Keep tool permissions minimal and role-specific.
Separate read-only intelligence workflows from workflows that modify business systems.
Require human approval for high-impact actions such as refunds, contract changes, customer communications or production operations.

2. Put a gateway between agents and tools

Agents should never call internal systems directly. We would route tool access through an agent gateway with authentication, authorization, schema validation, logging, rate limits, egress allow-lists and response filtering. This aligns with the gateway patterns described in both Gemini Enterprise and Bedrock AgentCore architectures [6][7].

Allow-list external destinations such as sec.gov, approved SaaS APIs and internal service endpoints [5].
Validate every tool request against JSON Schema or equivalent contracts.
Apply policy at call time, not only at agent startup.
Return permission denials as structured tool results with safe alternatives, rather than allowing the model to infer missing data [7].

3. Use cloud-native identity with short-lived credentials

For AWS deployments, we would strongly consider KMS-backed Private Key JWT through Bedrock AgentCore Identity when integrating with OAuth identity providers. The private key remains in KMS, AgentCore signs short-lived assertions, and CloudTrail records token and KMS signing activity [4]. This is preferable to storing long-lived client secrets in application configuration.

For Google Cloud deployments, we would use Agent Identity with IAM-scoped service identities and audit logging, combined with Agent Registry for discoverability and governance [6]. In either cloud, the design principle is the same: every agent and tool call should have an attributable identity, scoped permissions and auditable execution history.

4. Keep business logic deterministic where possible

We would implement joins, scoring, deduplication, eligibility rules and compliance checks in code. The model can classify ambiguous text, generate explanations or draft communications, but deterministic functions should produce the evidence base and enforce constraints.

For disclosure monitoring, use code to fetch filings, normalize tickers and CIKs, deduplicate claims, match date windows and verify accession numbers [5].
For retention workflows, calculate risk scores in a service or Lambda-backed MCP action, then let the model draft customer-specific language based on approved fields [8].
For operational intelligence, use the semantic layer for discovery and typed retrieval, then aggregate results through controlled tool calls [7].

5. Build evaluation into CI/CD

Agent evaluation should not be an afterthought. We would create test datasets with expected tool calls, expected citations, denied actions, injection attempts and edge cases. Google’s example of verifying that every cited accession number and 8-K item appears verbatim in tool output is the right kind of deterministic check [5].

Run regression tests on prompts, tool schemas and policy changes.
Track tool-call accuracy, groundedness, refusal correctness, latency and cost per completed workflow.
Use LLM-as-a-judge only for subjective dimensions such as helpfulness or tone, not for entitlement, safety or financial correctness.
Cluster failures and fix root causes in tools, retrieval, prompts or policies before expanding scope.

6. Design for observability and cost attribution from day one

Every agent run should produce a trace: user, agent identity, model, prompt version, tools called, documents retrieved, policy decisions, token usage, latency, errors and final output. Without this, teams cannot debug failures, allocate cost or prove compliance.

Tag cost by workflow, business unit, environment and customer segment.
Cache deterministic tool results where freshness requirements allow it.
Set budgets and circuit breakers for long-running agents.
Monitor gateway denials, unusual tool-call patterns and repeated retries.

Risks, Costs and Security

Prompt injection and document-borne attacks

The Copilot for Word prompt-injection research shows why enterprises must treat documents, emails, tickets, web pages and chat transcripts as untrusted input [2]. Hidden instructions can be interpreted by a model and propagated into new outputs. Mitigations should include content sanitization, instruction hierarchy, tool-output isolation, document rendering controls, output scanning and restrictions on copying untrusted instructions into generated artifacts.

Over-permissioned agents

The highest-risk failure is an agent with broad access and weak policy enforcement. Per-agent identities, least privilege, short-lived credentials, gateway authorization and CloudTrail or equivalent audit logs are mandatory for production systems [4][6][7]. Shared service accounts and static secrets should be avoided.

Runtime and workflow cost

Long-running agents, memory banks, code sandboxes, connector endpoints and observability pipelines all have recurring costs [6][7]. Teams should estimate cost per completed business outcome, not just cost per model token. A workflow that saves analyst time may still be uneconomical if it triggers excessive retrieval, repeated tool calls or unnecessary high-context model runs.

Connector maintenance

Prebuilt connectors can accelerate deployment, but real enterprise systems often require custom MCP servers, legacy API adapters, schema mapping and session-affinity handling [7][9]. These integrations need versioning, uptime monitoring, authentication rotation and backward-compatible contracts.

Data leakage and policy drift

Agents can combine data across systems in ways existing dashboards did not. That creates risks around row-level security, customer confidentiality, regulated data and internal information barriers. Policies should be enforced deterministically at retrieval and tool-call time, not merely described in prompts. Plain-English policy authoring can help business teams participate, but generated policies must be reviewed, tested and logged [7].

Cryptographic agility

Identity-heavy agent architectures depend on signing keys, token exchange and public-key infrastructure. As cryptography transitions toward post-quantum algorithms, teams should maintain inventories of signing algorithms, key lifetimes, IdP dependencies and rotation processes [3]. Even if immediate migration is not required, crypto agility should be part of enterprise AI platform design.

Vendor lock-in

Managed platforms provide speed, but memory formats, registries, evaluation tooling and runtime semantics may not move cleanly across clouds. We would reduce lock-in by keeping core business logic in independent services, exposing tools through stable contracts, storing canonical workflow state outside the agent runtime where appropriate and maintaining portable evaluation suites.

Bottom line: production AI agents are not primarily an LLM deployment problem. They are an enterprise platform engineering problem involving identity, orchestration, deterministic tools, policy enforcement, runtime isolation, evaluation, observability and cost control. The organizations that succeed will treat agents as governed software systems, not as prompts connected directly to business systems.

Where Kimbodo Comes In

Kimbodo builds and operates this in production for businesses — see our AI Infrastructure & MLOps practice. Wondering what it would cost for your organization? Get a preliminary range, timeline and architecture in about a minute.

Estimate My Infrastructure