What Happened
AI workloads are shifting toward agentic and long-context interactions (multi‑hour transcripts, long documents, multi-agent state), which greatly increases sequence length and the compute spent in attention. Recent analysis shows attention now dominates inference time as context grows, making attention design — not just kernel engineering — a first-order determinant of throughput and latency…
