What Happened
BigQuery has made daily token quotas generally available for its LLM-backed SQL functions, giving administrators a way to limit generation activity at both project and user levels. When a quota is exhausted, BigQuery stops new LLM calls. The embedding functions AI.GENERATE_EMBEDDING, ML.GENERATE_EMBEDDING and AI.EMBED are excluded. [1]
The default daily limits are substantial: 200 billion input and 20 billion output tokens per project, plus 150 billion input and 15 billion output tokens per user within a project. Output accounting includes thinking tokens. Administrators can adjust limits through Google Cloud’s quota interface, Cloud Quotas API, gcloud or Terraform. [1]
The production implication: quotas provide a consumption backstop, not a complete dollar-budget controller. Teams still need workload measurement, model-specific pricing, admission controls and recovery behavior.
Why It Matters to Businesses
LLM-backed SQL makes it easier to apply generation across large datasets. It also makes a broad query, oversized prompt or repeated job a potential inference-cost incident. Per-user quotas can constrain individual consumers; project quotas provide a shared ceiling. However, these controls do not cover the excluded embedding functions or establish a budget for the entire application. [1]
Token limits must be translated into money using the selected model and execution mode. At the cited Gemini 3.1 Flash-Lite batch prices of $0.125 per million input tokens and $0.75 per million output tokens, approximately 754 million input and 7.54 million output tokens cost $100. That estimate assumes a measured 100:1 input-to-output ratio; a different ratio changes the economics. [1]
Business owners therefore need two decisions: how much consumption to permit, and what service should continue when that allowance runs out.
Kimbodo Engineering Perspective
We would treat native quotas as the final enforcement layer, not the scheduler. Allowing an uncontrolled workload to reach its quota can protect against further token consumption while still interrupting business processes.
- Separate workloads with different priorities. Use distinct projects where independent quota boundaries justify the additional IAM, deployment and monitoring overhead.
- Budget for output uncertainty. AI.COUNT_TOKENS estimates input, but excludes output and thinking tokens, function-added instructions and non-text inputs. It is a lower bound, not a complete cost estimate. [1]
- Measure before scaling. Run a representative sample to establish actual input/output ratios, completion rates and cost per useful result.
- Limit generation deliberately. Set max_output_tokens where supported, then test whether shorter responses still satisfy the task. A cheaper incomplete answer is not necessarily a cheaper business outcome. [1]
How We Would Implement It
1. Establish enforceable workload boundaries
Inventory generation and embedding jobs, their owners, execution identities and service priorities. Start with conservative per-user limits and a project ceiling, managed through Terraform where practical. Because the native controls are per project and per user, do not assume they provide application-tenant isolation without verifying how execution identities map to quota accounting. [1]
2. Put admission control ahead of generation
For batch workloads, use an orchestrator to select bounded partitions, estimate input with AI.COUNT_TOKENS and admit work against an internal allowance. Add headroom based on measured output and estimation gaps. Reserve allowance before dispatch so concurrent jobs cannot all spend against the same remaining budget.
3. Record actual consumption and useful results
Maintain a workload ledger containing job identifiers, model configuration, estimated and observed consumption, completion status and attributable cost. Track cost per successfully processed record alongside daily spend. Monitor embedding workloads separately because these generation quotas exclude them. [1]
4. Make quota exhaustion recoverable
Checkpoint completed records and resume only unfinished work. Treat quota exhaustion as a pause or escalation condition rather than an immediate retry loop. For interactive applications, define a controlled fallback, such as a deferred response or previously validated result, instead of exposing a raw infrastructure failure.
Risks, Costs and Security
- Budget precision: Token ceilings do not guarantee a fixed dollar limit across different model prices, execution modes or input/output mixes. Recalculate allowances when configurations change.
- Availability: A shared project ceiling can stop unrelated workloads. Isolation reduces that coupling but increases operational overhead.
- Incomplete coverage: Embedding consumption and other application infrastructure require their own cost controls. [1]
- Administrative exposure: Restrict quota-change permissions, audit changes and require review for increases. Otherwise, an authorized change can weaken the intended spending boundary.
- Data exposure: Cost controls do not secure prompts or outputs. Apply least-privilege data access and avoid placing sensitive payloads in routine operational logs.
The strongest design combines native quotas with measured workload sizing, application-level admission control and restartable execution. That makes spending more predictable without turning every budget boundary into a production outage.
Where Kimbodo Comes In
Kimbodo builds and operates this in production for businesses — see our AI Infrastructure & MLOps practice, or Estimate My Infrastructure.
