AI GPU Utilization and Rental Rates: 2026 Data

The market is simultaneously scarce at the frontier and underused inside ordinary enterprises. What looks like a shortage or a glut depends on which utilization definition, which silicon generation, and which buyer you measure. Public price and survey series can be checked; fleet-level GPU utilization and the training-versus-inference split mostly cannot.

Utilization is not one number, and the “100%” and “5%” headlines describe different machines

Bank of America argued on October 16, 2026 that deployed AI infrastructure is running near 100% utilization, and that rental rates on even older A100, H100, and B200 generations are up 12–37% year to date—evidence of shortage, not the idle fiber of 2000. That claim is about capacity that has already been sold into training clusters and long-term cloud contracts, where a kernel is almost always running and the customer is paying whether or not the job is efficient.

Inside enterprises that own or provision their own GPUs, the picture is the opposite. A VentureBeat Research survey of 573 technical leaders found 86% of operators reporting utilization of 50% or less. Cast.ai’s 2026 Kubernetes report, covering 23,000 clusters through April 2026, put average GPU utilization at about 5% (Azure 2%, AWS 5%, Google Cloud 6%). Those are not contradictory once the metric is unpacked. NVIDIA’s own NVML “GPU utilization” only records whether at least one kernel ran during a sample interval; it is not a share of peak FLOPS. A small job can pin the gauge at 90%+ while using a fraction of the silicon. Older academic traces make the same split: Microsoft’s Philly study found hardware utilization of about 52% on GPUs that were in use; Alibaba’s PAI traces showed high allocation with much lower actual activity; dedicated LLM training clusters have reported median utilization of 97–99%.

Power waste sits on top of that. NVIDIA’s Rod Evans said in mid-October 2026 that only about 60–70% of electricity drawn from the grid ends up doing AI work: roughly 20% lost to cooling, conversion, and distribution, and about 10% to misconfigured racks, failures, and restarts. A contracted 100 MW site may deliver closer to 93 MW, and less at peak hours.

What this means for anyone entering the market: quoting “utilization” without saying whether you mean reserved, billed, kernel-busy, or useful throughput is not analysis. The scarce asset is liquid, interconnect-dense, power-backed cluster time. The surplus is poorly scheduled enterprise GPUs and Kubernetes overprovisioning. A neocloud that only reports occupancy, and an enterprise that only reports nvidia-smi, are talking past each other.

• BofA: near-100% on deployed AI infrastructure; A100/H100/B200 rents up 12–37% YTD. [1] [2]
• Enterprise self-report: 86% at ≤50% utilization (n=573). [3]
• Kubernetes fleets: ~5% average GPU utilization across 23,000 clusters. [4]
• NVML utilization is kernel presence, not peak performance or profit. [5] [6]
• Grid-to-useful-compute: ~60–70%. [7]

Price signals split: GPU hours have firmed in 2026; tokens have collapsed

Independent rate panels as of October 18–19, 2026 show a wide but coherent ladder. Compute Credit Index Research’s guaranteed on-demand panel (U.S. and Europe) printed B200 at $14.24 per GPU-hour on hyperscalers, $8.11 on neoclouds, and $8.36 on marketplaces; H200 at $10.30 / $4.50 / $5.31; H100 at $11.25 / $3.34 / $3.07; A100 at $4.45 / $1.82 / $1.58. Interruptible grades are far cheaper—H100 interruptible at $3.30 hyperscaler and $2.15 neocloud—because the operator can reclaim the box. Other public medians land in the same neighborhood: cloud-gpus.com on October 19 had B200 on-demand at $8.34, H200 SXM at $5.53, and H100 SXM at $3.99, with floors near $2 on marketplaces and hyperscalers often 2–3× the specialist floor.

Direction matters more than the level. getdeploying’s median on-demand series was up 21% over the year to the week of October 19, 2026, and up 9% over the prior four weeks. That is the opposite of a glut tape. Earlier in the cycle, H100 lead times had collapsed from about 52 weeks to under 8 by March 2026, and spot H100 prices had fallen sharply from the 2023–24 peak. The 2026 rebound is selective: newest rack-scale Blackwell remains allocated, while older generations are no longer collapsing in price because inference and batch work still pay for them.

Token prices moved the other way. BenchLM’s frontier token-price index stood at 15.5 on October 19, 2026, 84.5% below its March 2023 base. OpenAI cut GPT-6 Sol to $2 / $10 per million input/output tokens and Luna to $0.10 / $0.50, roughly half prior promotional rates. Anthropic held an Opus-class line for several releases and then cut it; open-weight models run a large share of volume at a steep discount to closed frontier SKUs. Ramp’s economist Ara Kharazian reported the effective blended price down 41% to $0.68 per million tokens from a 2026 peak of $1.15 in March, with cheaper standard models driving volume and frontier models’ token share falling from 53% to 45%.

The trap is equating cheaper tokens with cheaper work. One 2026 cost series found list prices down about 91% from Q1 2023 to Q3 2026, but real cost per completed task down only about 63%, and up roughly 80% from a Q4 2024 trough, because reasoning models burn more tokens per task. Falling API prices can raise, not lower, the GPU bill if agent loops multiply calls.

What this means: a buyer who shops only hyperscaler list rates overpays; a seller who underwrites residual value off 2024 H100 peaks will be wrong in both directions. The public, checkable series are list and marketplace rates. Contracted one-year deals and take-or-pay prices are only partly visible.

• CCIR guaranteed ladder, October 18, 2026. [8] [9]
• Median on-demand snapshot, October 19, 2026. [10]
• On-demand medians up 21% year over year. [11]
• Frontier token index −84.5% from March 2023. [12]
• Effective price $0.68/M tokens; frontier share 45%. [13]
• List prices down far more than cost per completed task. [14]

Power, not wafers, is the binding constraint on new capacity

Morgan Stanley’s October 2026 work puts the U.S. data-center power deficit at about 32 GW through 2028, roughly 34% of projected requirements, even after on-site generation and fuel cells. The bank says NVIDIA has confirmed the thesis: more semiconductors are being built than there are places to plug them in. Large, creditworthy players are expected to take about 94% of global data-center capacity additions in 2026, falling to 64% in 2027 and 57% in 2028—a squeeze on smaller developers, not a chip shortage in the 2023 sense. A related Morgan Stanley note cut the expected U.S. share of global compute from 60% to 55% because overseas power cannot absorb that 32 GW gap, and flagged a possible widening toward 68 GW by 2029 in one framing.

Hyperscalers are still describing a demand-constrained tape, not an inventory build. Combined 2026 capex guidance for the four largest was cited in mid-October at $720–745 billion, roughly double the 2025 pace. Amazon has said a roughly $220 billion plan will not cover 2026 demand and that it expects to stay constrained into 2027, monetizing capacity as fast as it is installed. Microsoft has said Azure demand continues to surpass supply (Azure growth was cited at 43% in the latest quarter in that note). Alphabet has raised its capex range to deliver capacity faster. TSMC has said packaging is tight enough to limit customer growth. Those are management statements, not audited utilization, but they are the opposite of the fiber-era pattern in which suppliers cut prices into unsold inventory.

The forward revenue math is the bear case, and it is a forecast, not a measurement. Gartner’s October 14, 2026 release said AI providers are expected to spend nearly $1 trillion on infrastructure in 2026 and another $6 trillion from 2027 to 2030. To cover that spend, financing, and a 12% return, data-center owners would need about $8.5 trillion of cumulative revenue through 2030. Gartner forecasts no more than $3.6 trillion of enterprise and consumer AI revenue attributable to those data centers, and says more than $3.5 trillion of spending could be exposed to impairment by the end of 2030. It also expects more than 80% of neoclouds to cease independent operations by 2030 through acquisition, partnership, or failure—not necessarily bankruptcy. Hyperscalers can absorb a miss because AI is not their only P&L. A single-purpose GPU lessor cannot.

What this means: the scarce input for a new entrant is an interconnection and a credit rating, not a purchase order for H100s. Projects that cannot energize on the hyperscaler timetable will slip even if the chips arrive. The Gartner gap is a returns problem, not proof that today’s racks are empty.

• 32 GW U.S. shortfall through 2028, ~34% of need. [15] [15] [16]
• Hyperscaler guidance and “demand exceeds supply” statements. [17] [17]
• Gartner: $8.5 trillion required vs ≤$3.6 trillion attributable revenue; >$3.5 trillion impairment exposure; >80% of neoclouds exit independence by 2030. [18] [19]

Enterprise adoption is real, narrow, and still mostly a productivity story rather than an EBIT story

The “MIT 95% of pilots fail” line is a misread of Project NANDA’s GenAI Divide (fieldwork January–June 2025, published mid-2025). The funnel in the report was: about 60% of organizations investigated custom or task-specific tools, 20% reached a pilot, and 5% of all organizations had a successful production deployment with measurable impact. Among the minority that actually piloted, roughly one in four cleared a high bar—marked P&L or productivity effect within six months. The executive summary’s “95% of organizations getting zero return” is a different claim from “95% of pilots fail.” The more useful finding was operational: partner-built tools reached deployment far more often than internal builds (on the order of two-to-one in the coverage of the report), and generic chat tools were widely used by individuals while workflow-embedded tools stalled.

McKinsey’s 2026 State of AI survey (1,719 respondents, 97 countries, May 4–June 8, 2026) shows the same split a year later. About 80% of respondents say AI improved their own productivity. Only 37% attribute any EBIT impact to AI, flat versus the prior year. About 6% qualify as high performers (at least 5% of EBIT from AI, and described as significant). Scaling is uneven: 44% say AI is scaling across the enterprise, up from 38%; 40% of companies with $1 billion-plus revenue say they are scaling agents, up from 27%, while smaller firms are flat. About one in five organizations already cites AI operating costs as a constraint. McKinsey’s separate cost work says moving a tool from pilot to production nearly quadruples spend, because unguarded model access, cloud compute, and integration dominate the license fee. These are self-reported perceptions, not audited earnings.

The Census Bureau’s Business Trends and Outlook Survey is the cleanest public adoption series, and the question wording is the story. Through late 2025 the survey asked whether the firm used AI in producing goods or services, and the rate sat near 10%. In November 2025 the question became “any business function,” and measured use nearly doubled. The 2026 AI-supplement working paper (reference period November 2025–January 2026) found 18% of firms using AI in at least one function, 32% on an employment-weighted basis, with 22% expected within six months. Very large firms in information, professional services, and finance ran 50–60% (60–70% employment-weighted). Among users, 57% used AI in three or fewer functions, most often sales and marketing. Worker task use showed up in 23% of firms (41% employment-weighted), mostly writing, document analysis, and search. AI-related employment decreases were reported in only 2% of firms. A later biweekly wave (September 21–October 4, 2026) put the share at 23.8%.

Spend data sees a different population. Ramp, tracking card and vendor payments at tens of thousands of U.S. businesses, has published paid-AI adoption well above Census—56.1% as of August 2026 in one write-up, versus Census near 22%—because a $16–$20 monthly ChatGPT seat counts as adoption in transaction data and may not count as “using AI in a business function” on a Census form. Ramp’s October 2026 index also showed the tension in the price war: token volumes rose even as spend at the heaviest users softened from a July high (top 1% at $9,568 per employee per month in September, down 3.1% from $9,873), before early-October spend ticked to a new peak. Median firm spend is tiny; the bill is concentrated.

What this means: competing on “enterprises are not adopting” is wrong. Competing on “enterprises have proven scaled ROI” is also wrong. The wedge is narrow workflows with a P&L owner, usually bought rather than built, in large knowledge-sector firms. Broad agent rollouts that quadruple the bill before the process is redesigned are how the 37% EBIT figure stays flat.

• MIT NANDA funnel and the 95% misquote. [20] [21] [22]
• McKinsey 2026: 80% individual productivity, 37% any EBIT, ~6% high performers. [23] [24] [25]
• Pilot-to-production spend nearly quadruples. [26]
• Census: 18% / 32% employment-weighted (Nov 2025–Jan 2026); question change doubled the series; 23.8% in early October 2026. [27] [28] [29]
• Ramp vs Census and the volume-versus-spend split. [30] [31] [32]

Inference has likely passed training on spend, not yet as a public census of watts

Gartner’s AI-optimized infrastructure-as-a-service forecast for 2026 is about $42.3 billion, nearly double 2025, with inference at $23.3 billion and training at $19 billion—the first year inference is the larger line, at 55%, and a sign that buyers are serving models in production rather than only training them. Deloitte’s compute-share estimates put inference near one-third of AI compute in 2023, about half in 2025, and roughly two-thirds in 2026. Futurum’s platforms sizing had managed inference already ahead in calendar 2025 ($23.1 billion vs $16.3 billion for training) and at 71.7% of that infrastructure slice by 2030 in its base case.

Power models are more cautious and should not be averaged with dollar models. A McKinsey data-center estimate cited in 2026 coverage had 2025 training demand at 23.1 GW and inference at 20.9 GW—training still slightly larger on watts—then inference growing faster (about 35% a year vs 22%) to 93.3 GW versus 62.2 GW by 2030. Lifetime compute for a deployed model is even more inference-heavy, because the training run happens once and serving runs for years. None of these splits is a regulatory filing. They are vendor and analyst constructs with different denominators (IaaS dollars, managed-platform dollars, IT power, GPU-hours).

The mechanism that links this to prices: inference is bursty, latency-sensitive, and willing to run on last-generation memory-rich GPUs, which is why H100 and H200 rents can rise even as frontier training clusters hoard Blackwell. Training still sets the peak rack density—GB200-class halls around 130 kW per rack, with roadmaps far higher—which is exactly what makes the power queue the constraint.

What this means: underwriting a data center as if 2023’s training mix still dominates will mis-size both the interconnect and the customer. Inference customers care about cost per million tokens and time-to-first-token, not cluster scale. That is a different sales motion, and a different residual-value story for older GPUs.

• Gartner: $23.3 billion inference vs $19 billion training in 2026 IaaS. [33] [34]
• Deloitte compute share and McKinsey GW paths. [35]
• Futurum inversion already in CY2025. [36]

Circular financing is large enough to contaminate “demand,” but it is not the whole tape

The circular loop is now in filings, not just diagrams. NVIDIA has taken equity in customers, committed to buy cloud services from them, and extended guarantees tied to sites that must run NVIDIA hardware. Coverage of its disclosures cites on the order of $36 billion in cloud-service commitments and guarantees capped around $105 billion for an OpenAI-leased campus, plus a $2 billion CoreWeave stake and a residual-capacity purchase arrangement reported at $6.3 billion through 2032. Microsoft’s relationship with OpenAI includes a very large Azure purchase commitment (reported around $250 billion incremental) that shows up as Microsoft cloud revenue. Amazon, Google, and AMD have analogous stake-plus-compute structures with Anthropic and OpenAI. Proponents call this a flywheel that accelerates scarce supply. Critics note that supplier, investor, landlord, and customer are the same set of firms, so one dollar can be counted as capex, revenue, and backlog more than once.

CoreWeave is the stress case that shows both sides. At the end of Q2 2026 it reported a $104.2 billion revenue backlog with names such as Microsoft, OpenAI, Meta, and Anthropic, and management said that backlog more than supports its debt. It also carries on the order of $50 billion of debt, guided 2026 capex at $35–39 billion, and saw five-year default protection widen to roughly 1,000 basis points in mid-October 2026 reporting. A take-or-pay contract is real revenue even if the customer’s end users never show up. It is not the same thing as a marketplace clearing price.

How to separate the two, using what is actually public:

• Closer to end demand. Census BTOS (representative, repeated, question-sensitive). Ramp and similar card/API spend (selected sample of firms that already use a spend platform, but it is money leaving a bank account to OpenAI or Anthropic, not a supplier investing in a customer). Marketplace GPU rents that rise when labs are not the only bidder. Hyperscaler statements that incremental capacity is sold as it is energized—still management commentary, but paired with Azure/AWS growth rates in filings.
• Contaminated or subsidized. Equity investments that round-trip into chip or cloud purchases. Residual-capacity backstops. Guarantees that only pay if a named lab leases the site. Backlog concentrated in three or four labs that are themselves financed by the same vendors. Internal research clusters that are “100% utilized” and produce no external rental revenue.
• Ambiguous. Token-volume explosions during a price war. Volume can be real usage and still fail to cover depreciation if the price per token falls faster than volume rises—the pattern Ramp and Bank of America both flagged in late summer and early fall 2026.

A central-bank-style claim that a majority of AI-firm investment comes from other AI firms has circulated in secondary write-ups; treat it as contested until the primary statistical release is in hand. The filing-level circular amounts above do not need that statistic to be material.

What is reliably public, and what is not

Metric	How public	Reliability caveat
API list prices	Provider rate cards, daily	Reliable for the SKU; blend and “effective” indexes depend on mix and cache assumptions
On-demand GPU rents	Marketplace listings, hyperscaler price pages, third-party panels (CCIR, cloud-gpus, getdeploying)	Reliable as offers, not as cleared volume; config, region, and interruptibility must be separated
Hyperscaler capex and cloud growth	10-Q / earnings	Reliable; “demand exceeds supply” is commentary
NVIDIA investments, guarantees, purchase commitments	10-Q footnotes	Reliable as disclosed amounts; economic substance of circularity is interpretation
Neocloud backlog and debt	Issuer filings (e.g. CoreWeave)	Contracted revenue, not collected cash; customer concentration is the risk
U.S. firm AI use	Census BTOS, public microdata and working papers	Best representative series; wording change in Nov 2025 breaks the old time series
Enterprise ROI	McKinsey, MIT NANDA, S&P, IDC	Self-report; samples and success definitions differ; not audited EBIT
Fleet GPU utilization	Almost never	Operator claims and nvidia-smi are not interchangeable with billed or productive hours
Inference vs training split	Gartner, Deloitte, McKinsey, Futurum	Analyst estimates; dollars and gigawatts disagree on the exact crossover year
Power shortfall	Sell-side (Morgan Stanley) and some grid telemetry	Model-dependent; UK distribution data shows booked vs used capacity can diverge sharply
“$8.5T vs $3.6T” returns gap	Gartner press release, Oct 14, 2026	A forecast with an assumed 12% hurdle, not an observed loss

The practical reading as of October 20, 2026: frontier cluster time and energized power are scarce, which is why rents on deployed GPUs have risen this year and hyperscalers are still raising capex. Ordinary enterprise GPUs are often idle, which is why utilization surveys look disastrous and why ROI has not followed individual productivity. Token prices are in a real price war that boosts usage faster than spend. A large slice of the most cited “demand”—backlog, chip purchase commitments, cloud revenue between related parties—is contractually real and economically circular. The metrics that can adjudicate the argument without taking a bank’s word for it are Census adoption, card-level API spend, and independent GPU rental panels. Fleet utilization and the training/inference watt split are not in that set.

---

Recent Findings Supplement (October 2026)

Recent data (primarily October 2026 publications, with some September–mid-2026 reports) highlight a widening split between high-utilization hyperscaler/neocloud deployments and widespread underutilization or inefficiency in enterprise and single-tenant settings.[1]

Sedai analysis (Oct 13) found roughly one-third of GPUs operating below 15% utilization, with even systems reporting >95% standard utilization often achieving only ~38% kernel efficiency—leading to estimates of ~$70,000 annual waste per H100-class GPU at 30% effective use. A VentureBeat Intelligence survey (referenced Oct 15) reported 81% of enterprises running AI workloads at ≤50% GPU utilization and 31% below 25%, yet 63% planned infrastructure expansion and only 10% had finance-reviewed ROI figures.[2]

In contrast, Oracle reported 97.9% GPU fleet utilization in its most recent quarter (after adding >300,000 GPUs), with renewed capacity resold at a 20% premium. Bank of America (Oct 19) cited near-100% AI infrastructure utilization and GPU rental price increases of 12–37% year-to-date as evidence of shortage rather than surplus.[3][4]

A new National Compute Grid initiative (launched Oct 13) aims to pool idle capacity, noting single-tenant data centers average <15% utilization and targeting 2 GW by 2030 from an initial 760 MW pipeline (with >$5B in reservations).[5]

This divergence suggests compute scarcity is concentrated among well-capitalized players with committed workloads, while broader buildout risks creating stranded or inefficient capacity. Enterprises face challenges in achieving billable/effective utilization versus raw allocation; new pooled marketplaces or optimization tools (e.g., Sedai) could help entrants compete on efficiency rather than raw scale. Public metrics remain fragmented—enterprise surveys show low averages, while hyperscaler self-reports and rental trackers show tightness.

GPU rental rates have risen overall amid contract lengthening and newer-generation scarcity, even as older capacity shows signs of softening and token/API prices continue sharp declines.[6]

Median on-demand H100 pricing reached ~$3.81–$4.14 per GPU-hour across providers as of mid-to-late October 2026 data (+21% year-over-year; +9% in recent weeks per one tracker), with spreads as wide as 5.9× between cheapest and priciest offers. Earlier 2026 contract examples included ~$4.30/GPU-hour for multi-year B300 clusters. Newer chips (B200/B300) command premiums; spot/interruptible rates are lower but volatile.[7][8]

Token prices have fallen rapidly (e.g., LLM usage benchmark at $0.97 per million tokens after a 29% drop in one month; broader deflation of ~47% in some gateway averages), driven by competition and cheaper open-weight models. Goldman Sachs (early September) warned this could contribute to compute oversupply if usage growth does not outpace price declines.[9]

Rising rentals signal persistent demand for scarce new capacity and long-term contracts, but falling token prices and older-GPU softening point to potential bifurcation—newer silicon remains tight while commodity tiers face pressure. Entrants should monitor generation-specific dynamics and consider flexible or pooled offerings; hyperscalers and specialists with locked-in deals hold advantages. Rental price indices from multiple independent trackers (e.g., Price of Compute, getdeploying) provide relatively reliable public data here.

Power and grid constraints have emerged as the dominant near-term bottleneck, with Nvidia highlighting operational inefficiencies and analysts projecting significant U.S. deficits favoring large players.[10]

Nvidia (Oct 14) stated AI data centers deliver useful compute on only 60–70% of grid power drawn (20% lost to cooling/distribution/conversion; up to 10%+ to failures, restarts, and misconfigurations), with potential for recovery through better design. Morgan Stanley (Oct 17) estimated a 32 GW U.S. data center power deficit through 2028 (~34% of projected needs), warning it will concentrate capacity additions among creditworthy hyperscalers and neoclouds (projected 94% of 2026 global additions, declining later).[11]

Grid interconnection delays of 5–10 years are common; events like a July 2026 PJM fault affecting 3.9 GW of load and regional pauses (e.g., ERCOT) underscore risks. Projects are shifting or using on-site generation (e.g., xAI turbines). Broader pipeline credibility is low—Bernstein estimated only ~35% of planned U.S. capacity as realistic, with dozens of large projects delayed or blocked in 2026.[12]

Power access now differentiates winners more than chip availability alone; smaller players risk being squeezed out or forced to Asia/other regions. New efficiency tools (e.g., dynamic power allocation yielding 24%+ token throughput gains in tests) and flexible-load approaches offer mitigation paths. Public data on deficits and queues (from banks, grid operators, and trackers) is increasingly cited but often modeled rather than fully realized.

McKinsey’s 2026 State of AI survey (fielded May–June, 1,719 respondents) shows enterprise scaling advancing but ROI lagging, consistent with earlier MIT NANDA findings of limited P&L impact; Census data provides a broader U.S. baseline.[13]

McKinsey: 44% of organizations scaling AI enterprise-wide (up from 38% prior year); 37% report any EBIT impact from AI (flat year-over-year); only 6% are “high performers” (≥5% EBIT contribution with significant impact). Individual productivity gains are widespread (80% report improvement), but ~20% cite operating/token costs as a constraint. Agentic AI scaling is rising among large firms.[13]

MIT Project NANDA’s 2025 “GenAI Divide” report (widely referenced in 2026 coverage) found 95% of organizations saw no measurable P&L return from GenAI deployments (based on interviews, surveys, and public cases); clarifications note this reflects limited documented impact rather than outright pilot failure rates. U.S. Census Bureau (Sep 2026): 23.2% of businesses using AI in any function.[14][15]

Adoption is real and broadening, but translating to enterprise ROI remains difficult—high performers redesign workflows; most do not. This supports distinguishing subsidized/circular demand (e.g., internal hyperscaler usage or pre-committed lab capacity) from organic enterprise pull. Surveys like McKinsey and Census offer the most reliable public benchmarks here.

Inference/token demand has surged dramatically, outpacing many forecasts and shifting spend toward inference (now majority of AI-optimized IaaS), with agentic workloads driving higher per-task consumption.[16]

OpenRouter routed ~23.8T tokens/day in October 2026 (33× year-over-year); agentic traffic reached ~7.3T tokens (7-day avg.), up 14× in six months and surpassing human prompting. Inference accounted for 55% of 2026 AI-optimized IaaS spend (~$23.3B vs. $19B training) and is projected to rise further. Open-weight models handled 71.5% of Vercel gateway tokens (23% of spend) in September.[17][18]

Gartner projects agentic workflow inference costs rising >5× through 2028 despite per-token deflation, due to 5–30× higher token use versus basic chatbots. Volume growth (e.g., Google/Microsoft hyperscaler reports of multi-fold increases) appears demand-driven rather than purely subsidized.[19]

This represents tangible usage growth (especially agentic/coding), helping differentiate real demand from circular buildout. Public trackers like OpenRouter/Vercel provide granular, timely signals; hyperscaler commentary offers directional confirmation but less granularity.

Overall, recent evidence points to persistent tightness at the frontier (high utilization, rising rentals for new capacity, exploding inference volume) coexisting with inefficiencies, project delays, and ROI challenges elsewhere. Power/grid issues and execution gaps are the clearest new constraints. Metrics from rental trackers, McKinsey/Census surveys, and grid reports are among the more reliable public sources; utilization and pipeline figures vary significantly by segment and self-reporting. For competitors, advantages accrue to those optimizing existing assets, securing power early, or proving measurable ROI in agentic/inference use cases.