How to Build Production AI Platforms Without Losing Control of Access, Quality and Cost

What Happened

Recent AWS and Google Cloud releases move more AI infrastructure into managed services: retrieval, agent execution, tool integration, web search and database-native inference. The architectural opportunity is to reduce custom orchestration and infrastructure work. The operational challenge is that application teams still own authorization, answer quality, release decisions and recovery.

• Managed retrieval: Amazon Bedrock Managed Knowledge Base handles ingestion infrastructure, storage, indexing, retrieval and scaling. Its migration tooling can provision a replacement alongside an existing knowledge base, but does not ingest content or cut over applications. [1]
• Composable agents: AWS demonstrates A2A interoperability between agents using different frameworks and accounts, with OAuth authentication and discoverable Agent Cards. AgentCore also offers direct web search without requiring a Gateway. [2][8]
• More deployment choices: Java teams can deploy Spring AI applications to AgentCore or optionally EKS, while Codex can access supported OpenAI models through Bedrock using AWS identity and regional configuration. [5][6]
• Inference closer to data: AlloyDB’s preview ai.classify and ai.extract functions expose Gemini-powered classification and structured extraction through SQL. [10]

Why It Matters to Businesses

Managed services reduce infrastructure ownership, not accountability. A retrieval service can operate correctly while an assistant exposes another tenant’s documents or produces an incomplete answer. Bedrock Managed Knowledge Base trusts the user identity supplied by its caller; it does not authenticate the end user itself. [3]

Quality metrics also need business context. In an HR assistant reference implementation, chunk-level precision was 1.00 across three questions, but fact-level precision ranged from 0.09 to 0.59. A prompt change improved one answer and worsened another; missing policy coverage required a retrieval or content fix, not more prompt tuning. [4]

Costs become more granular as applications assemble managed capabilities. Managed Knowledge Base lists storage at $5/GB-month and retrieval at $1 per 1,000 requests. Agentic retrieval adds $4 per 1,000 queries plus underlying retrieval calls. AgentCore web search costs $7 per 1,000 queries. These are component prices, not the total cost of answering a user request. [1][8]

For illustration, 100 GB of knowledge-base storage, one million standard retrieval requests and 100,000 web searches would total $2,200 at those rates, before model inference, runtime, other applicable service charges and additional retrieval activity. Teams should budget per completed business task, including retries and escalation, rather than per model call.

Kimbodo Engineering Perspective

Start with the smallest architecture that meets the requirement

We would default to one application, one agent and a limited tool surface. Multi-agent orchestration earns its complexity when separate ownership, security boundaries or specialist workflows justify it. A2A solves interoperability across frameworks; it does not establish whether a remote agent’s output is trustworthy or sufficient. The AWS demonstration uses synthetic findings and is not a compliance certification. [2]

A gateway should likewise solve a documented problem. Direct APIs are appropriate when the application already owns retries, caching and error handling. A gateway becomes useful for centralized tool policies or routing. AWS makes this distinction for web search and recommends starting Codex pilots with native AWS access unless a gateway is needed. [6][8]

Choose placement by workload, not platform uniformity

SQL-native inference is worth testing for bounded classification and extraction over database-resident text. AlloyDB supports micro-batching for these functions in WHERE clauses, but the functions remain in preview. We would benchmark accuracy, query latency and cost before putting them on a critical transaction path. [10]

For Java agents, a managed runtime can reduce operational work; EKS may suit organizations with existing Kubernetes controls and operating expertise. The Spring AI example supports both paths, but distributed tracing and automated evaluation are suggested extensions, not implemented capabilities. Deployment readiness must include those gaps. [5]

How We Would Implement It

• Define acceptance criteria first. Establish 20–30 business-approved examples for one task. Separate hard constraints, release gates and monitoring metrics. Evaluate repeated runs and release readiness by intent, rather than relying on one aggregate score. [4]
• Keep identity outside model control. Validate OAuth tokens at ingress. For retrieval, overwrite userContext from a verified identity claim in a trusted interceptor. Add a policy check and a knowledge-base resource policy restricting retrieval to the authorized gateway role. Test direct-call bypasses as well as normal requests. [3]
• Provision in parallel. Review migration settings classified as 1:1, Adjusted or Blocked. Approve the scoped IAM permissions and provisioning script, then create the replacement without changing the original service. Explicitly schedule ingestion and application integration. [1]
• Validate retrieval and cutover separately. Test representative questions, filters, permissions and missing-content cases. Update the knowledge-base ID and managedSearchConfiguration only after validation, and retain the original through a defined rollback window. [1]
• Instrument the complete request path. Capture model, tool, retrieval and workflow traces, subject to data-handling policy. Record the evaluator, test set, agent version, thresholds and release decision. Separate operational telemetry from billing attribution; AWS billing records and gateway allocation records serve different purposes. [4][6]
• Prove failure behavior. Test expired tokens, unauthorized tools, timeouts, unavailable dependencies and access revocation. Withhold failing intents or route them to a human unless a safeguarded limited release was approved in advance. [2][4][6]

Risks, Costs and Security

Authorization requires complete path coverage. Gateway policies do not protect direct retrieval calls by themselves. ACL-enabled sources return no documents without userContext, while sources without ACLs remain visible to all users. ACL changes also require synchronization or reindexing, creating an operational dependency for access updates. [3]

External tools expand the trust boundary. Treat remote agents, MCP servers and retrieved web content as untrusted inputs. Restrict credentials and tool permissions independently of prompts. Repository guidance is not an enforcement boundary for coding agents; validate supported managed controls and test deny, timeout and disconnected behavior. [6]

Observability is not enforcement. Cost dashboards identify spending but do not stop it. Where hard request-path budgets or rate limits are required, use an enforcement layer that supports them and test its failure behavior. Direct web search also has a default quota of 10 requests per second per account per Region, which needs capacity planning. [6][8]

Training requires a separate integrity strategy. Infrastructure health checks cannot establish arithmetic correctness. HyperPod research demonstrates in-loop gradient, replay and weight checks, but also documents missed faults and substantial overhead: deterministic audits added about 10% wall-clock time every 100 steps versus 35% every 10 steps in the tested model. Checkpoint validation and rollback therefore remain essential. [7]

The practical objective is not maximum platform consolidation. It is a smaller operational burden with explicit identity boundaries, measurable acceptance criteria, attributable costs and a tested recovery path.

Sources

1. [1] Migrate to Amazon Bedrock Managed Knowledge Base with your coding agent
2. [2] Build composable, interoperable multi-agent systems with A2A on AWS
3. [3] Fine-grained access control for Amazon Bedrock Managed Knowledge Base tools
4. [4] From prototype to production: Engineering agents with Agent DLC
5. [5] Build autonomous AI agents with Amazon Bedrock and Spring AI
6. [6] OpenAI Codex on Amazon Bedrock
7. [7] Detecting and recovering from silent data corruption on Amazon SageMaker HyperPod
8. [8] Introducing the Web Search API on Amazon Bedrock AgentCore
9. [10] Introducing new AI functions for AlloyDB: ai.classify and ai.extract