How to Build Production AI Platforms That Cut Integration Work Without Losing Control

What Happened

Recent enterprise AI examples show a clear shift: businesses are moving from isolated LLM assistants to orchestrated AI platforms that generate code, manage data workflows, enforce policy, and operate under human approval.

Formula 1 and AWS built an agentic Data Accelerator for Customer 360 data onboarding. The platform uses Amazon Bedrock AgentCore with Claude Sonnet 4.6, Lambda, EventBridge, MWAA, S3, Redshift, SageMaker Unified Studio, GitHub, and Jira. A business requirements document uploaded to S3 triggers agents that generate configuration, infrastructure pull requests, DBT transformations, governance metadata, and GDPR tags. Human reviewers approve the generated pull requests before deployment. The result was a reduction from 6–8 weeks per data source to roughly 40 minutes of code generation plus review and deployment time, with agents handling about 95% of onboarding work [4].

Google described a similar production pattern for mainframe modernization: use AI to assess legacy systems, extract business rules, map dependencies, and support iterative modernization rather than attempting a high-risk “big bang” migration. The approach combines Gemini models, mainframe assessment tooling, agent workflows, deterministic transformation where appropriate, and Dual Run validation against live production traffic to compare legacy and modernized outputs before cutover [5].

AWS also added automated policy refinement for Amazon Bedrock Guardrails Automated Reasoning checks. The system translates natural language into variables, validates outputs with formal logic, and proposes policy changes through an asynchronous review workflow. Importantly, no production policy changes occur without explicit approval [6].

At the edge of the stack, smaller local models and agents are becoming more relevant. The LFM2.5-2.6B release points to a continued trend toward deploying local agents closer to users, devices, or private environments when latency, privacy, or cost make cloud-only inference unattractive [1]. At the same time, practitioner reports show model upgrades can break agent systems that previously worked, as in the case where a reusable tool reportedly performed well through one model version and then “fell apart” with a later one [2].

Why It Matters to Businesses

The business value is not “LLMs can write code.” The value is that AI can now sit inside controlled operational loops: generate artifacts, open pull requests, attach tests, route approvals, update tickets, observe failures, and propose fixes. That changes the economics of data onboarding, platform engineering, analytics operations, and application modernization.

• Integration backlogs become automation targets. F1’s 18-month backlog was not solved by a chatbot. It was solved by decomposing onboarding into repeatable artifacts: configuration, infrastructure, transformations, metadata, access controls, and review workflows [4].
• Legacy modernization becomes less binary. Google’s approach separates assessment, rewrite, deterministic conversion, data migration, and live equivalence testing. That allows teams to modernize workload by workload instead of betting the business on a single migration event [5].
• Governance has to become executable. Policies expressed only in PDFs do not scale. Formal reasoning, generated guardrail policies, review screens, fidelity reports, and approval gates make policy part of the delivery system [6].
• Human review remains a control, not a bottleneck. The best examples keep humans in the approval path for generated code, infrastructure, and policy changes [4][6]. The human role shifts from manual authoring to validation, exception handling, and accountability.
• Model choice becomes an infrastructure decision. Cloud frontier models, smaller local models, open-source models, and task-specific agents each have different cost, latency, privacy, and reliability profiles [1][8].

This also changes risk. A person who simply forwards AI output without reading or validating it becomes a “meat proxy,” adding latency without judgment [3]. In production AI systems, that failure mode appears as rubber-stamped pull requests, untested generated pipelines, stale embeddings, unsafe permissions, and undocumented model behavior.

Kimbodo Engineering Perspective

The strongest production pattern is not a fully autonomous agent. It is an agent-assisted delivery system with deterministic boundaries. The agent should generate, inspect, explain, and propose. The platform should test, diff, enforce policy, log, and require approval for irreversible changes.

For enterprise teams, the main architecture decision is where to place autonomy. We would allow high autonomy for reversible, low-risk actions such as documentation generation, schema profiling, test generation, and draft pull requests. We would restrict autonomy for production deployments, access control changes, data deletion, customer-facing decisions, financial calculations, and policy updates.

The Formula 1 example is instructive because the agents did not bypass software engineering discipline. They produced GitHub pull requests, Jira tickets, lineage views, DBT changes, governance tags, and deployment artifacts [4]. That is the right direction: make AI output enter existing delivery systems rather than creating a parallel shadow process.

The Google mainframe pattern is also realistic because it avoids a single modernization strategy. Some workloads should be rewritten. Some should be converted like-for-like. Some should be left alone until there is a business case. Dual Run validation is especially important for regulated or high-value workloads because AI-generated modernization must be proven against production behavior, not just reviewed in design meetings [5].

We would be cautious about overfitting a platform to one model. The report of a tool working well through one model version and degrading with a later upgrade is a common production lesson: LLM behavior is not a stable API [2]. Teams need model version pinning, regression tests, evaluation suites, fallback models, and release gates for model changes.

How We Would Implement It

1. Build a control plane around agents

We would separate the AI control plane from execution systems. The control plane manages prompts, model routing, tool permissions, workflow state, evaluation results, audit logs, and approvals. Execution systems remain conventional: CI/CD, infrastructure-as-code, data orchestration, DBT, Kubernetes, cloud functions, warehouses, and ticketing.

• Workflow orchestration: Use Temporal, AWS Step Functions, MWAA/Airflow, Argo Workflows, or EventBridge depending on the environment and workload type.
• Artifact storage: Store inputs, generated outputs, evaluation traces, and approvals in object storage with immutable versioning.
• Code path: Require agents to open pull requests rather than writing directly to production branches.
• Approval path: Route high-risk changes through code owners, data owners, security, or compliance reviewers.

2. Use task-specific agents, not one general agent

For a data onboarding platform, we would define separate skills or agents for requirements parsing, schema inference, infrastructure generation, DBT modeling, test creation, lineage updates, data quality rules, privacy tagging, and deployment notes. This mirrors the modular skill architecture used in the F1 implementation, where new skills could be added without changing the core loop [4].

Each agent should have a narrow tool set. A governance-tagging agent does not need permission to deploy infrastructure. A schema-analysis agent does not need access to customer PII values if column names, samples, and profiling statistics are sufficient.

3. Put policy and reasoning in the pipeline

We would encode platform rules as executable checks: naming standards, IAM boundaries, data classification rules, encryption requirements, network restrictions, retention policy, and approved data movement paths. Where natural-language policy must be converted into machine-enforced rules, systems such as Bedrock Guardrails Automated Reasoning can help refine logic and identify ambiguity, but changes should remain approval-based [6].

The practical pattern is: generate policy, test policy, show diffs, review fidelity, approve, then apply. This is safer than letting agents interpret compliance requirements ad hoc at runtime.

4. Create an observability graph

AI-generated operations need business-readable observability. For data platforms, we would build a context graph connecting source systems, object storage paths, transformations, warehouse tables, dashboards, data owners, schema versions, quality checks, tickets, and deployments. F1’s clickable lineage graph and agentic root-cause analysis based on a JSON context graph are good examples of this pattern [4].

This graph becomes the agent’s map of the platform. It also becomes the human operator’s audit trail when something breaks.

5. Treat model deployment as a portfolio

We would not default every task to the largest hosted model. A production AI platform should route by sensitivity, latency, cost, and reasoning need.

• Frontier cloud models: Best for complex reasoning, code generation, planning, and ambiguous requirements.
• Smaller hosted models: Useful for classification, extraction, summarization, and predictable structured outputs.
• Local or private models: Useful where data residency, latency, offline operation, or unit economics matter [1].
• Deterministic code: Prefer rules, SQL, parsers, validators, and conventional services where behavior must be exact.

6. Modernize legacy systems iteratively

For mainframe or legacy modernization, we would start with assessment and dependency mapping, then select a thin vertical slice for modernization. The pilot should include production-like data, equivalence tests, rollback paths, and business-owner signoff. Dual Run or parallel execution is the right validation pattern for critical workloads because it compares new outputs with legacy outputs under real transaction conditions [5].

Risks, Costs and Security

Model drift and version risk: LLM behavior can change across versions. Pin model versions where possible, maintain regression suites, compare outputs before upgrades, and keep fallback routes. Do not assume an agent that worked last month will behave the same after a model change [2].

Approval theater: Human-in-the-loop controls fail when reviewers do not understand the output. Reviewers need diffs, tests, lineage, risk labels, and rollback instructions. Otherwise they become the “meat proxy” problem in enterprise form [3].

Cloud cost expansion: Agentic workflows can multiply inference calls through planning, retries, tool use, evaluation, and self-correction. Control cost with token budgets, caching, batch processing, smaller models for routine tasks, prompt compression, structured outputs, and hard workflow timeouts.

GPU and inference trade-offs: Self-hosting can reduce marginal inference cost at scale but adds operational burden: GPU scheduling, autoscaling, model serving, patching, monitoring, capacity planning, and security hardening. Managed model APIs reduce platform work but can become expensive and may create data governance constraints.

Data leakage: Agents often need broad context to be useful. Use least-privilege access, short-lived credentials, data minimization, redaction, VPC or private networking where appropriate, and strict logging controls. F1’s pattern of least privilege, short-lived tokens, audit trails, human review, automated tests, and isolation is the right baseline [4].

Prompt injection and tool abuse: Any agent that reads external documents, tickets, code, webpages, or data files can be manipulated. Tool calls should be permissioned, validated, and scoped. Agents should not be able to exfiltrate secrets, change IAM, or deploy unreviewed infrastructure.

Generated code supply-chain risk: AI-generated pull requests can introduce unsafe dependencies, vulnerable containers, insecure Terraform, or license issues. Run SAST, dependency scanning, container scanning, policy-as-code, secret detection, and infrastructure plan review on every generated change.

Modernization correctness: Legacy systems often encode decades of undocumented business rules. AI-assisted extraction is useful, but production cutover requires deterministic validation, reconciliation, exception analysis, and parallel operation for critical paths [5].

The practical conclusion is clear: AI infrastructure should be designed as a controlled production system, not a collection of prompts. The winning architecture combines agents, workflow orchestration, formal checks, observability, Git-based review, model routing, and security boundaries. That is how businesses get the speed benefits of AI without giving up operational control.

Sources

1. [1] Deploy local agents everywhere with LFM2.5-2.6B
2. [2] Quoting Steve Yegge
3. [3] Don't be a meat proxy
4. [4] From weeks to minutes: How Formula 1® uses agentic AI on AWS to accelerate data operations
5. [5] Real-world mainframe modernization with AI: A safe, scalable path from mainframe to cloud
6. [6] Automated Reasoning policy refinement in Amazon Bedrock
7. [7] Quoting David Crawshaw's prompt
8. [8] Devtools must be open source (exe.dev)