What Happened
Three AWS implementation examples illustrate distinct enterprise AI deployment patterns: a managed agent application, a controlled cloud-remediation system, and a dedicated model-evaluation workflow.
- Document generation: An Amazon Bedrock AgentCore supervisor routes requests to presentation or summary tools. A .NET 8 API on Lambda manages conversations, while specialized functions generate files in S3. Retrieval uses Titan Text Embeddings v2 and OpenSearch Serverless to ground Claude Sonnet 4.6 responses. Internal testing produced a six-slide presentation from a 90-page report in 70–90 seconds, with approximately $0.04 in model costs per generation. [1]
- Cloud remediation: Tamnoon’s Tami uses four specialized agents to investigate findings, assess safety, plan changes, and generate fixes. Actions classified as SAFE may execute automatically; RISKY or AWAITING DATA cases go to human reviewers. Execution uses short-lived, tenant-scoped credentials rather than giving credentials to the AI. [2]
- Model deployment: SageMaker JumpStart provides a repeatable discovery, deployment, testing, and endpoint-deletion workflow. An example deploys a Qwen vision-language model using SageMaker Python SDK v3, then measures response latency and approximate generation rate. GPU quota, permissions, model licenses, and model-specific request formats remain deployment prerequisites. [3]
Why It Matters to Businesses
Choosing a model is only one part of choosing an operating model. Managed inference reduces the need to provision model-serving infrastructure. Dedicated endpoints give teams more direct control over deployment configuration, but introduce capacity planning, GPU availability, endpoint lifecycle management, and potential idle costs. The right choice depends on workload behavior, not merely benchmark quality. [2][3]
Latency also determines application architecture. The presentation example’s 70–90-second generation time exceeds API Gateway’s cited default 29-second timeout. That makes asynchronous execution, polling, or streaming an architectural requirement rather than a user-interface enhancement. [1]
For action-taking systems, business value depends on limiting the consequences of mistakes. Tamnoon reports up to 90% reduction in critical cloud exposure within 90 days and up to 100× remediation efficiency. These are company-reported outcomes, not transferable guarantees; the more useful engineering lesson is the separation of investigation, safety assessment, approval, and execution. [2]
Kimbodo Engineering Perspective
We would separate three concerns: model inference, workflow orchestration, and privileged execution. A model can propose an answer or change, but application code should enforce authorization, validate outputs, and determine which operations are permitted.
Multi-agent designs are justified when different stages need distinct tools, permissions, or review criteria. They are not inherently safer: additional agents introduce inference costs, failure paths, and coordination overhead. For a simple document-generation workflow, deterministic routing may be easier to test than open-ended planning.
Retrieval grounding improves access to relevant evidence, but does not establish factual correctness. Generated presentations still need checks for citation validity, unsupported claims, numerical consistency, and file integrity. Content guardrails and PII filtering address different risks from factual validation. [1]
Likewise, model benchmarks must represent the intended workload. JumpStart’s example reports 153 words in 9.07 seconds, or 16.9 words per second, for one request. That is a useful smoke test, not evidence of concurrent throughput, tail latency, or production economics. [3]
How We Would Implement It
1. Establish a workload-specific deployment baseline
Evaluate managed inference and a dedicated endpoint against the same representative tasks. Pin model versions where supported, record prompt and configuration versions, and measure task success, latency percentiles, concurrency behavior, and total cost per successful task. Confirm licenses, regional availability, and GPU quotas before committing to a model. [2][3]
2. Put durable orchestration behind the API
For long-running generation, return a job identifier and process work through a queue-backed workflow. Persist status, use idempotency keys, bound retries, and support polling or streaming for progress. Store completed artifacts in tenant-scoped S3 locations and issue short-lived download URLs only after authorization. The reference application uses five-minute presigned URLs. [1]
3. Separate reasoning from execution
Give investigation tools read-only access. Require proposed changes to include scope, preconditions, rollback instructions, and expected effects. Combine model-based assessment with deterministic policy checks; send uncertain or high-impact changes for approval. Only an execution service should obtain short-lived credentials for the approved operation. This follows the control separation demonstrated by Tamnoon. [2]
4. Build deployment and evaluation into operations
Use infrastructure as code, tenant-isolation tests, structured audit logs, and regression evaluations. Track retrieval failures, tool errors, approval rates, and cost by tenant and workflow. Automate teardown of experimental endpoints and review dependency-aware cleanup for agent runtimes, storage, and retrieval infrastructure. [1][3]
Risks, Costs and Security
- Model cost is not application cost. The approximately $0.04 presentation figure covers model usage, not the full cost of retrieval infrastructure, storage, orchestration, logging, networking, or operations. Compare all-in costs at expected utilization. [1]
- Endpoint convenience can conceal idle spending. Evaluation environments need explicit expiry and deletion policies. JumpStart’s workflow ends with endpoint deletion for a reason. [3]
- Public access and authorization are separate controls. The presentation architecture leaves the OpenSearch Serverless data plane publicly accessible for Bedrock Knowledge Bases, with IAM controlling access. Teams must review that design against their network policies rather than assume every component is private. [1]
- Documents and findings are untrusted input. Treat retrieved instructions as data, constrain tool access, and prevent document content from changing permissions or execution policy.
- Safety claims require operational evidence. Tamnoon reports zero production issues across millions of workloads. Buyers should still validate rollback behavior, approval controls, tenant isolation, and audit completeness in their own environment. [2]
The production decision is not “agents or endpoints.” It is which combination delivers acceptable task quality, predictable operating cost, and enforceable limits on failure.
Where Kimbodo Comes In
Kimbodo builds and operates this in production for businesses — see our AI Infrastructure & MLOps practice, or Estimate My Infrastructure.
