.png){kind=small}
The economics of AI are undergoing a rapid and fundamental shift.
When organizations first started using generative AI, it felt almost free thanks to relatively simple use cases and subsidized token prices offered by API providers. Now, boardrooms are confronting rapidly growing AI bills, even as the price per token has plummeted.
The paradox is explained by AI’s new economic battleground: inference costs. While organizations continue to invest heavily in training and fine-tuning models, inference is the cost that compounds every time AI is used. As enterprises deploy increasingly token-hungry agentic AI applications, inference workloads have soared.
Microsoft research found that agentic coding tasks require 1,000x more tokens than code reasoning or code chat. Even though the cost of achieving GPT-4-level performance plunged from $30 per million tokens in 2023 to under $0.10 this year, those savings are being more than offset by the explosion in token consumption.
The shift is already reshaping the economics of AI. Gartner forecasts spending on inference-focused applications to reach $20.6 billion in 2026, up from $9.2 billion in 2025, overtaking training for the first time. Inference workloads are projected to account for roughly two-thirds of all AI compute by 2026, up from a third in 2023.
.png)
The result is that AI inference has gone from an afterthought to one of AI's largest and most unpredictable operating costs. As those costs grow, the practice of treating token usage as a proxy for productivity, known as “token-maxing,” is increasingly untenable. Uber, for example, reportedly burned through its entire 2026 AI budget in just four months as employees rushed to adopt advanced AI coding tools.
These new economics mean that leaders have to think much more strategically about how to control inference costs without compromising on quality.
A big part of the answer lies in gaining more control over the AI infrastructure layer. Much of the cost of inference is determined before any application code runs, by the hardware that models run on and the energy that feeds them.
Organizations that secure and control the infrastructure underpinning their AI workloads will gain a powerful advantage in the era of spiraling usage demand and costs.
That case starts with understanding where software-layer fixes run out of road.
Software optimizes to a floor it can’t move
Solutions at the software layer can help limit costs. Tiered routing, where each AI request is routed to the cheapest model capable of handling it, can substantially reduce blended per-token costs. One analysis of 2.4 billion enterprise API calls found a median blended cost of $2.31 per million tokens with tiered routing, versus $18.40 for frontier-only.
But that comparison can be deceptive. A cost dashboard tracks tokens, not outcomes. If a cheaper model produces a weaker answer, users often re-ask the question, add context, or escalate to human support to get the result they needed, and each of those steps adds cost the dashboard doesn't capture. A lower blended cost per token doesn't guarantee a lower cost per successful outcome.
The deeper limitation of software-layer solutions is that they can only optimize toward the cost floor set by infrastructure.
How infrastructure sets the cost floor
Using purpose-built infrastructure gives companies greater control over the economics of inference. A vertically integrated provider like Nscale brings together more of the AI stack, from power generation and data centers to cloud infrastructure and inference services, running them as a unified system, and reducing the cumulative cost added at each layer.
In a typical stack, by contrast, each layer may be controlled by a different provider, each of which needs to make its own profit margin. Those margins are baked into the per-token price charged by API providers.
The benefits of an integrated stack extend beyond reducing integration overhead. AI infrastructure performs best when each layer is designed to reinforce the next, creating opportunities to optimize performance, cost, and operational efficiency across the system. That also enables faster experimentation with new hardware, models, and serving frameworks, helping organizations identify more efficient architectures and improve the cost of serving AI over time.
Working with an integrated stack provider also gives AI-native firms access to innovative approaches to power generation that improve the cost and predictability of AI infrastructure. Local grids are struggling to deliver enough power to new data center sites, with connection queues and permitting issues often stalling projects for years. Berkeley Lab research puts the U.S. grid connection backlog at roughly 2,060 GW, with typical projects waiting around five years to connect. These bottlenecks are creating artificial scarcity, and therefore high prices, across the market. It’s a constraint that doesn’t need to exist given the strength of demand.
Behind-the-meter power generation offers a way to remove those constraints. By generating power on site, privately funded, without drawing from the grid or competing with homes, data centers can scale at the speed demand requires. Nscale is adopting this approach in the US, while also locating infrastructure in regions with abundant, affordable renewable energy, such as Norway and Iceland, where reliable power helps improve both the cost and predictability of AI infrastructure. Together, these approaches give customers faster access to capacity, and pricing that reflects the underlying economics of compute rather than grid-related scarcity.
Infrastructure control isn't only about power and hardware, it also determines which models an organization can afford to run and how much control it has over them.
Scale changes the economics of frontier APIs
Frontier model APIs are often the fastest way to achieve high-quality outputs, making them a natural starting point for many organizations. But as AI workloads move into production and inference volumes grow, the economics of AI begin to change. The convenience of managed APIs comes at a premium, making it harder to optimize the cost of inference at scale while limiting control over where and how AI is run.
Open-weight models offer a different path. Rather than relying on a provider's managed API, organizations can fine-tune models for their own use cases and choose the infrastructure that best balances cost, performance, latency, and governance. As demand grows, the same tuned model can move from shared inference infrastructure to dedicated GPU capacity, improving cost efficiency, operational control, and long-term inference economics.
Making that transition requires more than choosing a lower-cost GPU provider. It involves evaluating models, tuning them for production workloads, testing quality, implementing guardrails, and designing the serving architecture around the application. The goal isn't simply to reduce the price of inference today, but to build an AI stack that becomes more efficient as usage grows. That's where the right infrastructure partner becomes valuable. At Nscale, we help AI-native organizations move from frontier APIs to open-weight models running on our inference endpoints, supporting every stage of that transition.
The economics beneath every token
The shift underway is bigger than a change in AI pricing. It changes where competitive advantage is created. The decisions organizations make today will shape the economics of every AI application they build tomorrow.
Organizations will win by controlling the economics beneath every token. Those that master those economics early will scale faster, adapt more freely, and build advantages that become harder to replicate.
.png)
.jpeg){kind=small}
.png)
