What Happened
Google is expanding two capabilities that move AI operations closer to enterprise data: agent observability in BigQuery and Looker, and model-powered text processing inside AlloyDB. Together, they offer a way to reduce separate telemetry pipelines and application-side inference code—not replace the orchestration layer of an AI platform. [1][2]
- Agent observability: BigQuery conversation logging, agent-monitoring dashboards and Looker System Activity dashboards are generally available. Coverage includes first-party agents and custom agents built with ADK or LangGraph, with telemetry stored in customer-owned BigQuery tables. Optional prompt and response logging adds content-level visibility. [1]
- SQL-based inference: AlloyDB’s preview functions, ai.classify and ai.extract, categorize text and produce structured JSON using Gemini. They support individual and batched inputs and require PostgreSQL 18 with google_ml_integration 1.6.1 or later. [2]
The maturity distinction matters: BigQuery’s native evaluations and daily token quotas remain in preview, as do the two AlloyDB functions. Production designs should not treat every announced control as generally available. [1][2]
Why It Matters to Businesses
The opportunity is fewer integration boundaries. Keeping agent telemetry in BigQuery lets teams investigate behavior, compare evaluation results and analyze operating costs without a separate export pipeline. Moving suitable classification and extraction workloads into AlloyDB can reduce application code for prompts, parsing and retries. [1][2]
Neither approach guarantees lower total cost. BigQuery analysis consumes query resources; AI-assisted log analysis adds inference work. SQL inference introduces model consumption into database queries. Teams should measure cost per successful business task, not just token price or database spend.
These capabilities are most relevant to organizations already operating on Google Cloud. For teams using other databases or multi-cloud infrastructure, migration and integration costs may exceed the savings from consolidation.
Kimbodo Engineering Perspective
Our architectural recommendation is to separate business execution, operational evidence and evaluation, even when they share a cloud platform. Application orchestration should still own workflow state, authorization, retries and failure recovery. BigQuery can hold evidence about those workflows; AlloyDB can perform bounded text transformations.
SQL inference fits repeatable tasks such as document categorization and field extraction. It is less suitable as the control plane for a multi-step agent. Putting a model call behind SQL simplifies the interface, but does not remove latency variability, output errors or the need for validation.
AlloyDB’s smart batching improves inference efficiency when these functions appear in WHERE clauses. That optimization is useful, but it should not be generalized to every query shape; broader query pushdowns are planned enhancements. Benchmark the actual workload rather than assuming uniform savings. [2]
How We Would Implement It
1. Establish a minimal operational record
Enable supported observability and instrument custom agents using the BigQueryAgentAnalyticsPlugin where appropriate. Define consistent identifiers for requests, agents, releases and tenants, alongside latency, token usage, errors and task outcomes. Start with operational metadata rather than full conversation content. [1]
2. Build a controlled investigation and evaluation path
Deploy usage and latency dashboards, then benchmark agents through Prism and the Conversational Analytics API. Store evaluation results alongside production telemetry to compare releases against representative workloads. BigQuery supports investigating agent_events with AI.GENERATE, embeddings and vector search; run such analysis as a budgeted workload, not an unrestricted background process. [1]
3. Introduce SQL inference off the critical path
For an eligible AlloyDB deployment, begin with asynchronous classification or extraction over a bounded dataset. Persist results with model and processing metadata, validate extracted JSON against application schemas, and route failures for retry or review. Compare single-row and batched execution before placing inference in customer-facing request paths. [2]
4. Enforce budgets outside preview features
Use BigQuery scan caps and monitoring, supplemented by application-level request limits, concurrency controls and spend alerts. Treat preview daily token quotas as an additional safeguard rather than the sole production budget boundary. [1]
Risks, Costs and Security
- Content exposure: Operational telemetry can avoid customer content, but optional prompt and response logs—and full traces—need separate review. Apply least-privilege access, retention limits and appropriate IAM, CMEK and VPC Service Controls protections. [1]
- Unpredictable consumption: Repeated SQL queries may repeat inference unless the application deliberately persists and reuses results. Bound input volumes and test execution behavior before scaling.
- Incorrect outputs: Structured JSON is not proof of factual accuracy. Validate required fields, business rules and source consistency. Line-level provenance is planned, not a current guarantee. [2]
- Operational coupling: Database-integrated inference and warehouse-native monitoring simplify pipelines but increase dependence on platform capabilities, permissions and release cycles. Keep application contracts and evaluation datasets portable.
The production lesson: move AI work closer to data when it removes measurable integration overhead, but retain explicit controls for execution, quality and spending. A simpler pipeline is valuable only if its failures and costs remain visible.
Where Kimbodo Comes In
Kimbodo builds and operates this in production for businesses — see our AI Infrastructure & MLOps practice, or Estimate My Infrastructure.
