Token prices fell 98%. Your AI bill didn’t.
Here’s the number that should be on every CFO’s desk this week: the price of an AI token has fallen roughly 98% in three years — and enterprise AI bills went up anyway. Four things worth two minutes of your time.
01The paradox your budget is losing to
GPT-4-equivalent output that cost ~$20 per million tokens in late 2022 now runs about $0.40 — a ~98% drop. Yet across the businesses Ramp tracks, token usage grew ~1,001% since January 2025 while spend grew ~497%, and average monthly AI token spend is up 13x. Falling unit prices don’t lower the bill; they invite more usage and pricier models.
If you’re budgeting AI on per-token price trends, you’re forecasting the wrong variable. The bill is driven by volume and model-mix, not sticker price.
Source: ramp.com/blog/ai-token-cost-for-businesses
Closing that gap
The reason falling unit prices don’t reach the invoice is that most teams can’t see spend as it happens — by team, model, and use case — or put policy around it before the usage runs. That visibility-plus-guardrails layer is what we built Chompute to be: the AI control plane for enterprise AI spend, policy, and governance. If the paradox above describes your budget, it’s worth a look — chompute.ai.
02Reserved GPU capacity just got more expensive
While token prices fall, the infrastructure underneath is moving the other way. AWS raised prices on EC2 Capacity Blocks — its reserved GPU compute for ML — by ~20% effective early July, the second increase this year after a ~15% hike in January. AWS attributes it plainly to “supply/demand patterns.”
The savings you see at the API layer can be quietly offset by scarcity pricing at the infrastructure layer.
03You’re probably paying for idle GPUs
Inference now consumes more compute than training for the first time — an estimated 55–80% of enterprise GPU spend. And most of it sits idle: Cast AI’s 2026 measurements put average enterprise GPU utilization around 5% across production clusters, while 80–85% of enterprises miss their AI infrastructure forecasts by more than 25%.
The cheapest token is the one you don’t waste capacity to serve. Utilization — not price shopping — is where the real money is hiding.
Source: finout.io/blog/top-6-ai-cost-drivers-and-genai-cost-examples-in-2026
04AI cost governance is now table stakes
The State of FinOps 2026 report finds 98% of organizations now actively manage AI spend, up from 63% a year ago, and names “FinOps for AI” the #1 forward-looking priority and the #1 skillset teams need to build. The metric practitioners are pushing: value per token, not cost per token.
Managing AI spend is no longer optional or niche — it’s a standard finance function.
Source: data.finops.org
Bottom line
Per-token price is the most-watched and least-useful AI cost number. The bill is set by volume, model-mix, idle capacity, and — increasingly — infrastructure scarcity. The organizations staying ahead aren’t the ones chasing the cheapest model; they’re the ones putting visibility and policy around usage before the invoice arrives.
Sources
- Ramp — How Much Do AI Tokens Cost Businesses? 2026 Spending Benchmarks
- Amazon EC2 Capacity Blocks for ML — Pricing
- SDxCentral — AWS Raises GPU Capacity Block Prices Again
- Finout — Top 6 AI Cost Drivers and GenAI Cost Examples in 2026
- State of FinOps 2026 Report
AI Cost Brief is your weekly read on what AI actually costs: pricing, infrastructure, and FinOps. Brought to you by chompute.ai.