What Happened
Recent enterprise AI implementations highlight a practical architecture pattern: use managed infrastructure for execution, but retain explicit control over workflow state, validation, spending and human approval.
- Document processing: An AWS reference implementation uses eight specialized agents on Amazon Bedrock AgentCore to classify, extract and validate documents. Known formats go directly to extraction; unfamiliar formats trigger the creation and refinement of reusable instructions. SES or S3 handles intake, while EventBridge, Lambda and DynamoDB coordinate processing. [1]
- AI cost controls: BigQuery introduced generally available daily input and output token quotas for its LLM-backed AI functions, excluding embedding functions. Quotas apply at project and user-within-project levels; output usage includes thinking tokens. [2]
- Internal support: ShipStation Global’s Deckhand combines an AgentCore-hosted agent, enterprise knowledge retrieval and Lambda-backed tools to answer Slack questions and create confirmed Jira tickets. The company reports an estimated 406 hours saved in its first three months. [4]
- Engineering infrastructure: Anthropic replaced Git with its custom Scion system as merged changes grew from roughly 2,500 to more than 30,000 per day. Its CI infrastructure performed approximately 25 million checkouts daily. [3]
Why It Matters to Businesses
The business value comes from completing controlled workflows, not merely generating answers. Document extraction needs schema and business-rule validation. Help desk automation needs reliable escalation, appropriate permissions and the judgment to remain silent when a human takes over. ShipStation implemented those interaction boundaries alongside its model and retrieval components. [1][4]
Cost predictability also requires more than selecting a cheaper model. BigQuery’s default project limits are 200 billion input and 20 billion output tokens daily—platform allowances, not sensible starting budgets for most pilots. Teams should lower quotas to match approved workloads and account for output and thinking tokens when forecasting spend. [2]
Operational scale can move the bottleneck outside inference. Anthropic’s source-control redesign illustrates how AI-intensive development can stress repository storage, checkouts and review systems. It does not establish that ordinary enterprise teams need a custom version-control platform. [3]
Kimbodo Engineering Perspective
Use deterministic orchestration around probabilistic components
We would keep job state, retries, authorization and acceptance criteria outside the model. Agents can classify unfamiliar documents or propose corrections; application code should decide whether required fields, totals and business rules pass. Eight agents may suit the AWS demonstration, but agent count is not a success metric. Begin with the smallest workflow that meets quality requirements and add specialization only when evaluations justify the extra calls and failure paths. [1]
Treat learned instructions as production configuration
The document example reused a correction rule after detecting an incorrectly formatted item code. That is useful adaptation, but an instruction that fixes one document can damage another format. We would version instructions, test proposed changes against representative documents and require approval for changes affecting financial or compliance-sensitive fields. [1]
Adopt managed services without surrendering control
Managed runtimes reduce infrastructure work, but workflow rules, evaluation datasets and integration contracts should remain independently testable. Likewise, Anthropic’s five-month migration by a seven-person team demonstrates a substantial engineering commitment—not a shortcut. Optimize standard tooling and measure bottlenecks before considering bespoke infrastructure. [3]
How We Would Implement It
- Establish measurable acceptance criteria. For documents, track field accuracy, validation failures, manual-review rate and cost per accepted document. For support, track verified resolution, escalation quality and employee feedback—not just response volume.
- Separate intake from execution. Place authenticated document or chat events behind a durable queue. Assign an idempotency key, persist workflow state and make retries safe so duplicate events cannot create duplicate tickets or business records.
- Route before invoking expensive reasoning. Send recognized document formats through validated extraction instructions; route unknown formats to instruction generation and review. Start with the AWS example’s suggested 5–10 representative documents per type, then expand evaluation coverage for exceptions. [1]
- Keep tools narrow. Expose task-specific operations rather than broad infrastructure access. Require confirmation before ticket creation, enforce file limits and stop automated replies when a person takes over, following Deckhand’s interaction safeguards. [4]
- Enforce workload budgets. In BigQuery, configure project and per-user token quotas through supported administration tools or infrastructure as code. Sample workloads with AI.COUNT_TOKENS and constrain output where supported. Its estimate excludes output, thinking tokens, AI-function instructions and non-text inputs, so reserve headroom. [2]
- Make interruption recoverable. Persist completed work and distinguish accepted, failed and unprocessed records. BigQuery quota exhaustion can reject new queries or stop new LLM calls during a query; partial-result behavior depends on the function. [2]
- Gate releases on evaluations. Version models, prompts, extraction instructions and retrieval settings. Compare changes against a fixed test set, deploy gradually and preserve a rollback path.
Risks, Costs and Security
Reference costs are not production budgets. AWS estimates roughly $58 monthly for 10 training documents and 1,000 processed documents in US East (N. Virginia). That estimate should not be generalized to different document complexity, retry rates, review requirements or operational staffing. Similarly, a reported 15–20-minute infrastructure deployment is not a production-readiness timeline. [1]
Token quotas limit consumption, not total application cost. BigQuery embedding functions are excluded from these quotas, and surrounding storage, processing and integration costs still require separate controls. Pair quotas with billing monitoring and verify which services actually support enforceable spending limits; do not assume an alert stops expenditure. [2]
Documents, Slack messages and retrieved content must be treated as untrusted input. Enforce authorization in tool code, apply retrieval permissions, isolate tenant data and prevent document text from changing system instructions. Store secrets outside prompts and redact sensitive material from operational logs.
Finally, plan retention and recovery explicitly. The document implementation’s cleanup process permanently deletes documents, job history, validation data and learned instructions. Export required records before teardown and test recovery before enabling automated cleanup. [1]
Where Kimbodo Comes In
Kimbodo builds and operates this in production for businesses — see our AI Infrastructure & MLOps practice, or Estimate My Infrastructure.
