How to Build Governed Enterprise AI Agents Without Losing Control of Cost, Data or Operations

What Happened

Enterprise AI platforms are moving from isolated chatbots toward orchestrated agent systems that connect governed data, legacy applications, office tools, cloud observability and model gateways.

Google is pushing governed analytics into agent workflows. BigQuery Graph lets teams map existing relational tables into a property graph without ETL, then define measures so agents can reason over multi-hop business relationships without producing duplicated or incorrect aggregates from naïve joins [5]. Looker’s semantic layer is also being integrated into Gemini Enterprise so users can ask questions in natural language while the system generates governed, version-controlled SQL against approved business definitions, preserving row-level and column-level controls [13].

AWS is expanding Amazon Bedrock AgentCore into a broader enterprise agent runtime. AgentCore Observability can now receive OpenTelemetry traces, logs, session data, token metrics and latency metrics from agents running outside AWS by using AWS Distro for OpenTelemetry, SigV4-signed OTLP exports and CloudWatch endpoints [6]. AgentCore Browser Tool adds managed Chromium automation for legacy web applications, combining Playwright/CDP browser control, Strands Agents and vision-capable foundation models for workflows that still depend on server-rendered HTML, SSO, MFA and manual data entry [7]. AWS also published a due diligence reference architecture with supervisor and specialist agents, Bedrock Knowledge Bases, AgentCore Gateway, evaluations, guardrails, CloudWatch, X-Ray, private networking, KMS and least-privilege IAM [8].

AI is also being embedded directly into enterprise productivity surfaces. Amazon Quick for Microsoft 365 brings cloud-hosted agents into Word, Excel, PowerPoint and Outlook, inheriting existing Quick data sources, permissions and regional isolation while recording before-and-after snapshots for document changes [9].

At the model and tooling layer, the pace of change remains high. The llm-gemini plugin added support for Gemini 3.7 Flash, additional Flash variants and Gemini embedding models [3]. DeepSeek V4 Pro 0813 appeared as an API-only release through OpenRouter, with large model artifacts later posted to Hugging Face [12]. Smaller infrastructure releases also illustrate production fragility: sqlite-utils 4.2 added more capable table transformation support [2], then 4.2.1 quickly fixed a crash caused by a missing dependency declaration for typing-extensions [1].

Why It Matters to Businesses

The key shift is that enterprise AI value is no longer determined only by model quality. It depends on whether agents can safely operate across governed data, business applications, identity systems, audit logs and cost controls.

Governed metrics reduce hallucinated decisions. Semantic layers and graph measures help agents answer with approved definitions instead of inventing joins, filters or KPI logic [5][13].
Legacy automation is becoming viable, but not simple. Browser agents can automate systems with no modern API, but they require session isolation, audit trails, selector resilience, human approval and error recovery [7].
Observability must cross cloud boundaries. Many enterprises will run agents in AWS, Azure, GCP, SaaS platforms, Kubernetes clusters and on-premises environments. A single telemetry model based on OpenTelemetry is becoming essential [6].
Agent systems need evaluations, not just logs. The AgentCore due diligence example uses citation checking, pass/fail evaluator results and trace IDs, which is closer to how regulated AI workflows need to operate [8].
Embedding AI inside office tools changes risk exposure. Agents in Excel, Outlook and PowerPoint operate near sensitive business content, so permission inheritance, regional isolation and change auditability become core requirements rather than optional controls [9].
Model optionality is useful, but operationally expensive. New hosted and downloadable models give teams flexibility, but every model adds evaluation, routing, latency, data handling, quota and cost-management work [3][12].

For business leaders, the practical lesson is to avoid treating AI agents as standalone applications. Production-grade agent platforms need the same rigor as payment systems, data platforms and customer-facing cloud services.

Kimbodo Engineering Perspective

We would not start by asking, “Which agent framework should we use?” We would start by separating the system into control planes and risk domains: identity, data access, model access, tool execution, observability, evaluation, audit and cost management.

Managed platforms accelerate delivery, but can hide coupling

AgentCore, Gemini Enterprise, Looker, BigQuery and Quick offer useful managed capabilities: hosted runtime components, semantic governance, native telemetry, office integration and browser automation [5][6][7][9][13]. The trade-off is platform coupling. If all agent memory, tools, traces, evaluations and orchestration are tied to one cloud-specific abstraction, migration and multi-cloud operations become harder.

Our preference is to use managed services where they remove undifferentiated operational burden, but keep portable boundaries: OpenTelemetry for traces, standard IAM/OIDC patterns for identity, model gateways for LLM access, policy-as-code for authorization and versioned data contracts for business metrics.

Semantic layers beat prompt engineering for business analytics

For analytics agents, prompt tuning alone is not enough. If “revenue,” “active customer,” “gross margin” or “supplier delay” are not formally modeled, an LLM will eventually produce a plausible but wrong answer. BigQuery Graph measures and Looker’s governed semantic layer address the right problem: constrain the agent to approved relationships and calculations [5][13].

The engineering trade-off is upfront modeling effort. Teams must define entities, measures, relationships, ownership, test cases and release workflows. That cost is justified when the AI output influences pricing, supply chain, finance, compliance, sales forecasting or executive reporting.

Browser agents are a bridge, not a substitute for APIs

Managed browser automation is valuable for legacy systems that cannot expose APIs quickly. The Bedrock Browser Tool architecture is directionally sound: isolated browser containers, SSO-aware access, screenshots, transcripts, CloudWatch and CloudTrail auditability, and human-in-the-loop confidence gates [7].

However, browser automation remains brittle. DOM changes, timing issues, CAPTCHA, MFA flows, rate limits and ambiguous UI states can break workflows. We would use browser agents as a controlled automation layer for high-value back-office processes, while maintaining a roadmap to replace fragile UI automation with APIs, events or database-backed integration.

Observability must include reasoning, tools and cost

Traditional APM is insufficient for agents. A useful trace must show the user intent, selected model, prompt template version, retrieved documents, tool calls, browser actions, policy decisions, token usage, latency, retries, evaluator results and final answer. AgentCore’s external OpenTelemetry path is important because many production agents will not live inside one vendor runtime [6].

We would standardize on OpenTelemetry from day one, then export to the customer’s chosen backend: CloudWatch, Datadog, Grafana, Honeycomb, New Relic, OpenSearch or a SIEM. The data model should support incident response, cost attribution and compliance review.

Small dependency mistakes still break AI platforms

The sqlite-utils 4.2.1 fix is a useful reminder: AI systems depend on ordinary software supply chains [1]. A missing dependency declaration can crash production just as effectively as a failed model endpoint. Agent platforms often combine Python packages, browser runtimes, cloud SDKs, vector databases, database drivers, model plugins and observability agents. Version pinning, software bills of materials, integration tests and staged rollouts are mandatory.

How We Would Implement It

Reference architecture

For an enterprise AI platform spanning analytics, workflow automation and knowledge work, we would use the following architecture:

Identity and access: OIDC/SAML for users, workload identity for services, least-privilege IAM, short-lived credentials and policy enforcement before every tool call.
Agent runtime: containerized agents on Kubernetes, ECS, Cloud Run or a managed agent runtime depending on the client’s cloud posture. Use separate runtimes for low-risk Q&A, privileged workflow automation and regulated decision support.
Model gateway: a central service that routes to Bedrock, Gemini, OpenAI-compatible APIs, self-hosted models or OpenRouter-style providers. It enforces allowlists, budgets, logging, redaction and fallback policies.
Data plane: governed SQL through Looker/semantic models, graph-backed business relationships where multi-hop context matters, vector search for unstructured retrieval and strict row/column-level permissions [5][13].
Tool gateway: typed, permissioned tools for CRM, ERP, ticketing, email, document systems, databases and browser automation. Each tool has input schemas, output schemas, audit logging and policy checks.
Browser automation tier: isolated ephemeral browser sessions for legacy applications, with screenshots, transcripts, state snapshots, human approval thresholds and retention controls [7].
Evaluation service: offline test sets, online sampling, citation checks, regression tests, jailbreak tests and workflow-specific pass/fail evaluators, similar to the due diligence reference architecture [8].
Observability backbone: OpenTelemetry traces, logs and metrics across all agents, tools and model calls, exported to the enterprise monitoring stack or CloudWatch-compatible endpoints [6].
Audit and evidence store: immutable storage for prompts, model versions, retrieved sources, tool inputs, tool outputs, approvals and final actions, with retention policies aligned to legal and regulatory requirements.

Implementation steps

1. Classify use cases by risk. Separate read-only assistants, analytics copilots, document drafting, browser automation and agents that can change business records. Do not give all agents the same permissions.
2. Establish the semantic contract. Define approved metrics, dimensions, entity relationships and ownership. Use Looker-style semantic governance or graph measures where business context spans multiple hops [5][13].
3. Build the model gateway before scaling usage. Centralize model credentials, routing, budget enforcement, prompt templates, rate limits and per-tenant logging. Add support for new models only after evaluation, not simply because they are available [3][12].
4. Instrument every agent path. Add OpenTelemetry spans for sessions, prompts, model calls, retrieval, tool calls, browser actions, approvals and errors. Propagate session IDs consistently across services [6].
5. Implement tool permissions as code. Agents should request actions through typed tools. The tool gateway should check user identity, data scope, business policy, environment and approval requirements before execution.
6. Add human approval where the business impact is high. For financial changes, customer communications, claims processing, procurement actions or legacy system updates, use confidence thresholds and mandatory review queues [7].
7. Create evaluation gates for deployment. Require regression tests for answer quality, citation accuracy, policy compliance, latency and cost before promoting a new prompt, model, retriever or agent version [8].
8. Run progressive rollouts. Start with shadow mode, then read-only recommendations, then limited write actions, then broader automation after telemetry proves reliability.
9. Operate with FinOps controls. Track cost by user, department, agent, model, tool and workflow. Set budgets, alerts, caching rules and fallback models for lower-value requests.
10. Treat dependencies as production risk. Pin versions, scan packages, test upgrades and keep rollback paths for SDKs, model plugins, browser libraries and database tools [1][2][10].

Risks, Costs and Security

Major risks

Incorrect business logic: Agents can produce confident but wrong analytics if they bypass governed metrics or infer relationships from flat tables [5][13].
Over-permissioned tools: An agent with broad database, browser or email access can turn a prompt injection into a business-impacting incident.
Browser automation failure: UI changes, stale selectors, hidden validation rules and SSO changes can break workflows or cause incorrect submissions [7].
Observability gaps: Without full traces, teams cannot reconstruct why an agent made a recommendation or action [6].
Data residency and leakage: Office integrations, model APIs and multi-cloud agents must preserve regional boundaries, tenant isolation and permission inheritance [9].
Dependency and SDK drift: Fast-moving AI packages can introduce breaking changes, missing dependencies or changed model behavior [1][3].

Cost drivers

Token volume: Long prompts, multi-agent loops, retrieval stuffing and repeated reasoning calls can dominate spend.
Model tier selection: Premium reasoning models should be reserved for tasks that need them; routine extraction, classification and drafting can often use smaller or faster models.
Browser sessions: Managed Chromium automation consumes compute, storage and observability resources, especially when screenshots and transcripts are retained [7].
Telemetry volume: Full traces are valuable but can become expensive. Sampling and retention policies should vary by risk level.
Data duplication: Avoid unnecessary ETL when in-place graph mapping or governed semantic access can solve the problem [5].
Self-hosted models: Downloadable model weights may look attractive, but GPU capacity, scaling, monitoring, patching and utilization risk can outweigh API costs unless usage is predictable and high [12].

Security controls we would require

Least-privilege identity: Use workload identity, short-lived credentials and scoped tool permissions. Avoid static cloud keys except for controlled development.
Prompt injection defenses: Treat retrieved content, webpages, emails and documents as untrusted input. Enforce tool policies outside the model.
Data access enforcement: Preserve row-level, column-level and document-level permissions through the entire agent chain [13].
Immutable audit trails: Store prompts, outputs, approvals, tool calls, browser screenshots, document diffs and evaluator results where required [7][8][9].
Network isolation: Use private subnets, private endpoints, egress controls and KMS encryption for regulated workloads [8].
Evaluation and rollback: Every model, prompt, retriever and tool change should have test coverage, deployment history and a rollback path.

The production lesson is clear: enterprise AI platforms are becoming distributed systems with probabilistic components. The winning architecture is not the one with the most agents; it is the one that constrains agents with governed data, observable execution, secure tools, measured cost and controlled rollout.

Where Kimbodo Comes In

Kimbodo builds and operates this in production for businesses — see our AI Infrastructure & MLOps practice. Wondering what it would cost for your organization? Get a preliminary range, timeline and architecture in about a minute.

Estimate My Infrastructure