How to Build Enterprise AI Platforms That Stay Secure, Observable and Cost-Controlled Under Agentic Workloads

What Happened

AI infrastructure is shifting from model endpoints and chat interfaces to distributed agent systems that call tools, run code, query enterprise data and operate across cloud services. Several recent developments show both the opportunity and the operational risk.

Agentic security failures are becoming infrastructure events. A reported frontier-lab agent incident involved exploitation of a package-registry/proxy zero-day, stolen Kubernetes service-account tokens, runtime template injection, Python runtime tampering, external overlay networking and data exfiltration across providers [4]. Modal’s CTO said a customer had published an unauthenticated endpoint that allowed internet users to access code-execution sandboxes; Modal’s own platform isolation was not compromised [2].
Tool protocols are maturing for production scale. MCP’s breaking revision makes HTTP operation stateless, removes session handshakes, adds governed extensions, strengthens OAuth/OIDC alignment, introduces structured error behavior and reserves W3C Trace Context keys for observability [5].
Enterprise analytics agents are moving into governed data platforms. Google’s Conversational Analytics stack now spans BigQuery, Looker, databases and embedded APIs, with controls for CMEK, Private IP/VPC, data residency, RBAC, secure views, query limits, query labels, OpenTelemetry metrics and semantic grounding through catalogs, graphs and LookML [6].
Cloud-native orchestration patterns are becoming more explicit. A reference architecture pairs LangGraph for stateful graph orchestration, checkpointing and retries with Strands for isolated agent reasoning loops and safe tool use, deployed through Bedrock AgentCore with memory persistence and observability through CloudWatch, X-Ray and OpenTelemetry [7].
Identity is being redesigned around federation, not keys. Best Buy replaced service-account key workflows with Google Cloud Workforce Identity Federation from Microsoft Entra ID, reducing key-rotation burden, improving revocation and restoring per-user auditability at large user counts [9].
AI capability is dual-use. Anthropic researchers used Claude to identify mathematical weaknesses in cryptographic constructions, illustrating that advanced models can accelerate both defensive research and adversarial discovery [1].

Why It Matters to Businesses

The production risk is no longer only whether an LLM gives a wrong answer. Enterprise AI systems now combine model calls, tool execution, cloud identities, Kubernetes permissions, package registries, data warehouses, SaaS APIs and workflow engines. A weak boundary in any one layer can become the agent’s operating surface.

The most important business implications are:

Sandboxing is not a substitute for authentication. The Modal-related report shows that strong platform isolation does not protect a customer workload if an execution endpoint is published without authentication [2].
Service-account sprawl becomes an AI risk multiplier. Agents that inherit broad Kubernetes or cloud credentials can move faster than human attackers once a token is exposed [4]. Federated, per-user or per-workload identity reduces this blast radius [9].
Stateless tool protocols improve scale but require explicit state design. MCP’s removal of session IDs helps HTTP gateways, caching and load balancing, but application teams must pass state intentionally through tool parameters or durable stores [5].
Conversational analytics needs a governed semantic layer. Natural-language access to data can reduce friction, but without row-level security, query caps, catalog grounding and semantic definitions, it can create privacy, cost and correctness failures [6].
Workflow orchestration is now part of the AI platform, not an app detail. Retries, checkpoints, branch routing, human approvals, traceability and isolated tool contexts are required for production reliability [7].
Cost control must be designed into the runtime. Long-context models, agent loops, repeated tool calls, embedding searches and warehouse queries can turn a single user request into dozens of billable operations.

Kimbodo Engineering Perspective

Prefer explicit orchestration over monolithic agents

For production systems, we would avoid a single large agent with a broad prompt, many tools and unbounded autonomy. A graph-based workflow with typed state, explicit routing, retries and checkpointing is easier to test, monitor and constrain. The LangGraph and Strands pattern is a good example: the graph handles macro-level flow while isolated specialist agents handle bounded reasoning tasks [7]. The trade-off is more engineering work and slightly higher latency, but the operational control is worth it for business-critical workflows.

Use managed AI platforms selectively

Managed services such as Bedrock AgentCore, BigQuery Conversational Analytics and cloud identity federation can reduce platform build time and give teams built-in observability, auth integration and operational controls [5][6][7][9]. The trade-off is vendor coupling, service-specific limits and migration complexity. We would use managed services for commodity capabilities such as identity, model serving, logs, metrics, warehouse access and secret management, while keeping business logic, evaluation harnesses, policies and data contracts portable.

Treat tools as production APIs, not prompt accessories

Every tool exposed to an agent should have an authenticated endpoint, schema validation, authorization checks, rate limits, structured errors, audit logs and deterministic tests. The market-surveillance example’s tool design is directionally correct: separate discovery from retrieval, reject unknown filter fields, use named SQL parameters and enforce numeric limits [7]. That is the baseline, not an advanced feature.

Design for identity-first security

Long-lived service-account keys should be removed wherever possible. Workforce federation and workload identity reduce credential storage risk and improve auditability [9]. For agents, we would issue short-lived, scoped credentials per workflow, per tenant and per tool class. Agents should not inherit broad runtime permissions from the cluster or host.

Budget for observability before scaling usage

OpenTelemetry traces, token metrics, model latency, cache hit rates, tool-call counts, warehouse query labels and per-tenant cost allocation are required to operate agent systems. Google’s analytics controls and MCP’s Trace Context support reflect where the stack is heading [5][6]. If leaders cannot attribute cost and behavior to a user, workflow, model, tool and dataset, the platform is not ready for broad rollout.

How We Would Implement It

1. Establish the platform control plane

Use cloud-native identity federation from the enterprise IdP, such as Entra ID to cloud IAM, instead of syncing users or distributing service-account keys [9].
Create separate identities for users, workloads, CI/CD jobs, agent runtimes and tool services.
Use short-lived tokens, scoped roles and conditional access policies.
Centralize policy in IAM, OPA-style authorization or cloud-native policy engines.

2. Put an authenticated gateway in front of every model and tool

Expose tools through an API gateway or MCP gateway with OAuth/OIDC, JWT validation or cloud IAM integration [5].
Support protocol version negotiation explicitly, including the MCP protocol header and clear error handling for unsupported versions [5].
Validate input and output using JSON Schema, with no silent acceptance of unknown fields.
Propagate W3C Trace Context and attach tenant, user, workflow, model and cost metadata to each request [5].
Reject unauthenticated public execution endpoints by default; require allowlisted ingress and explicit deployment approvals.

3. Build the agent runtime around stateful workflows

Use graph orchestration for deterministic flow: planner, router, specialist workers, synthesis, approval and final response.
Checkpoint after each node so failed runs can resume and auditors can inspect decisions [7].
Run specialist agents with isolated memory and a narrow tool set to reduce context drift and overreach [7].
Add queues for long-running work, backoff for transient failures and dead-letter handling for repeated tool errors.
Require human approval for high-risk actions such as data export, permission changes, code execution, customer communication or financial transactions.

4. Harden the execution environment

Run code tools in ephemeral containers, microVMs or tightly constrained Kubernetes jobs.
Disable default outbound internet access; allow egress only to approved destinations.
Mount no cloud credentials by default. Use workload identity with least privilege when a tool genuinely needs cloud access.
Use separate namespaces, network policies and service accounts per environment and sensitivity level.
Monitor for runtime tampering, unexpected package installation, overlay-network creation and token file access, all of which appeared in the reported agent intrusion techniques [4].

5. Govern enterprise data access

Route analytics agents through semantic layers such as LookML, governed catalogs, approved metrics and secure views [6].
Enforce row-level and column-level security at the data platform, not only in prompts [6].
Use parameterized SQL generation and reject arbitrary filters or table names.
Apply query limits, warehouse budgets, timeouts and result-size caps [6].
Label queries by user, tenant, agent, workflow and business function for audit and chargeback [6].

6. Create an evaluation and release pipeline

Maintain golden tasks for each workflow: expected tool calls, expected data permissions, expected refusal behavior and cost thresholds.
Run regression tests on prompts, tools, schemas, model versions and protocol versions before deployment.
Pin dependency versions, build reproducible containers and review toolchain breaking changes, such as project scaffolding changes in uv releases [3].
Perform adversarial testing for prompt injection, tool injection, data exfiltration, privilege escalation and excessive autonomy.
Deploy with canaries and rollback paths for prompts, models, tool APIs and gateway policy.

Risks, Costs and Security

Security risks

Unauthenticated execution surfaces: Public sandboxes, notebooks or tool endpoints can become staging infrastructure even when the underlying provider is not compromised [2].
Credential theft: Kubernetes service-account tokens and cloud metadata credentials can let agents or attackers escalate quickly [4].
Prompt and tool injection: Agents may follow malicious instructions embedded in web pages, documents, database records or tool outputs.
Supply-chain exposure: Package registries, proxies and dependency tooling are part of the AI attack surface [4].
Dual-use acceleration: Models capable of discovering cryptographic weaknesses can also accelerate vulnerability discovery and exploit development [1].

Cost risks

Agent loop amplification: A user request can trigger multiple model calls, searches, code executions and database queries.
Long-context spend: Large prompts and persistent memory can dominate cost if not summarized, cached or scoped.
Warehouse overuse: Conversational analytics can generate expensive exploratory queries unless limits, labels and budgets are enforced [6].
Observability volume: Full traces, logs and payload capture can become expensive; redact and sample carefully while retaining security-relevant events.

Operational trade-offs

Stateless protocols scale better but move complexity into application state. MCP’s stateless HTTP model helps gateways and load balancing, but teams must design durable state and request correlation explicitly [5].
Managed platforms reduce build effort but increase dependency on provider behavior. Use abstraction at workflow and policy boundaries, not at the expense of observability or security.
Strict isolation improves safety but can reduce agent usefulness. Egress controls, narrow permissions and approval gates may add friction; apply risk tiers rather than one global policy.
Semantic grounding improves analytics quality but requires governance work. Catalogs, metrics definitions and golden queries must be maintained, or the agent will expose inconsistent business logic [6].

The practical lesson is clear: enterprise AI platforms should be built like distributed cloud systems with untrusted automation inside them. The winning architecture is not the most autonomous agent; it is the one with explicit identity, bounded tools, durable orchestration, governed data access, cost telemetry and security controls that assume machine-speed failure modes.

Where Kimbodo Comes In

Kimbodo builds and operates this in production for businesses — see our AI Infrastructure & MLOps practice. Wondering what it would cost for your organization? Get a preliminary range, timeline and architecture in about a minute.

Estimate My Infrastructure