What Happened
On The Vergecast, Nilay Patel argued that current AI products do not deliver benefits clear enough to justify their costs. He pointed to data-center noise, low-quality generated content and agents that do not yet work as users expect. He also asserted that companies spend more on these products than they earn. These are commentary claims, not a benchmark study or a verified financial assessment of enterprise AI deployments. [1]
The relevant engineering question is narrower: can an AI application deliver measurable business value after inference, infrastructure, oversight and failure costs are included?
Why It Matters to Businesses
Model access is not the same as a viable production service. An inexpensive inference call can become an expensive workflow when it triggers repeated retrieval, tool calls, retries and human corrections.
- Measure cost per accepted outcome: Include model usage, infrastructure, review and rework—not just cost per token.
- Validate user benefit: Compare completion time, quality and adoption against the existing process.
- Treat autonomy as an investment: Additional agent steps should improve outcomes enough to justify their latency, expense and operational risk.
Patel’s demand for benefits that are evident through use is a useful product test, even though his commentary does not establish which enterprise architectures succeed economically. [1]
Kimbodo Engineering Perspective
Our recommendation is to start with a bounded workflow, not a general-purpose agent platform. Extracting information, preparing a draft or recommending a next action is easier to evaluate and control than allowing an agent to execute an open-ended business process.
Choose deployment architecture from measured constraints. Managed model APIs can reduce initial infrastructure work and suit uncertain demand. Self-hosted models can offer greater runtime control and may improve economics at sustained utilization, but require capacity planning, model serving expertise, patching and on-call coverage. Neither option guarantees lower total cost.
Kubernetes is appropriate when an existing platform team needs scheduling, isolation and operational consistency across multiple services. It is not a prerequisite for an initial AI application. Managed containers, queues and hosted inference may offer a simpler path until scale or control requirements justify additional orchestration.
How We Would Implement It
Establish the outcome and baseline
Select one workflow and define acceptance criteria, latency targets, escalation rules and a maximum cost per accepted result. Build an evaluation set from representative, authorized examples, including ambiguous requests and failure cases.
Separate application control from model execution
Use an authenticated application API, a workflow service and a model-access layer. Keep authorization, business rules and tool permissions in application code rather than prompts. Use queued workers for long-running tasks; reserve synchronous execution for work that fits the user-facing latency budget.
Add only the components the workflow needs
- Retrieval: Add it when answers require enterprise knowledge, with document-level access checks and traceable references.
- Model routing: Evaluate smaller models for simpler tasks before introducing routing complexity.
- Execution controls: Set limits on tokens, tool calls, retries and elapsed time. Require approval for consequential actions.
- Release controls: Version prompts, models and retrieval settings; run regression evaluations before deployment and retain a rollback path.
Trace each workflow across model calls, retrieval and tools. Record latency, usage, errors and acceptance outcomes so infrastructure decisions reflect successful work rather than request volume alone.
Risks, Costs and Security
For self-hosted inference, account for idle accelerator capacity, redundancy, startup delays and engineering support. For managed APIs, account for usage variability, rate limits, provider dependencies and data-handling terms. Test both against realistic traffic before committing to capacity.
Retrieved documents and tool outputs are untrusted inputs. Enforce least-privilege access, isolate tenant data, keep credentials outside prompts and validate proposed actions independently. Minimize sensitive content in logs and define retention policies.
Scale only after quality, security and unit economics pass explicit thresholds. If a workflow still needs extensive correction or supervision, improve its scope and controls before buying more infrastructure.
Where Kimbodo Comes In
Kimbodo builds and operates this in production for businesses — see our AI Infrastructure & MLOps practice, or Estimate My Infrastructure.
