The next phase of AI spending is already underway

The AI trade has been easy to understand so far. Investors piled into chipmakers and model developers, betting that demand for compute and large language models would define the next cycle of growth. That bet worked.

But it is also incomplete. A quieter and potentially more consequential shift is beginning to take hold across enterprise budgets. And most investors are not paying enough attention to it yet.

Phase one was about speed, not discipline

The first phase of the AI cycle was defined by urgency. Companies rushed to experiment with generative AI, launching pilots as quickly as possible. Cost discipline took a back seat. Enterprises were willing to spend aggressively just to understand what AI could do.

That willingness shows up in the numbers. Global AI spending is forecast to total $2.5 trillion in 2026, according to Gartner. AI infrastructure alone accounts for $401 billion of that, as technology providers race to build the foundations enterprises need to run AI at scale.

But experimentation has limits. Eventually, prototypes have to become production systems. And that is where the economics start to change.

Running AI is a fundamentally different problem

Deploying an AI model is not the hard part. Keeping it running efficiently is.

“The bill that surprises most enterprises isn’t the one from launching AI. It’s the one from running it,” Guilhem Tesseyre told TheStreet. Tesseyre is the CTO and co-founder of Zencore, a Google Cloud Premier Partner founded by former Google engineers that has delivered more than 300 AI and cloud projects across over 200 enterprise customers.

Related: Elon Musk has a shocking message on AI and robots

The surprise comes from multiple directions simultaneously. Serving AI models requires round-the-clock availability for inference, even when those resources sit idle. Maintaining multiple model versions across geographies adds management overhead that compounds fast. When an enterprise AI solution gains traction internally, usage patterns shift quickly, and without proactive planning, operating costs inflate before anyone catches them.

Data operations add another layer. Enterprises building proprietary models are managing massive datasets across duplicated storage, cross-region data movement, and multiple networks. Each adds cost that compounds quietly. Token usage, without proper monitoring, can spiral in scenarios involving large context windows such as codebases or extended video formats.

One in five organizations misses its AI spend forecast by more than 50%, according to CloudZero. AI-native companies, the organizations most immersed in AI, have the worst forecast accuracy of any segment: 36% miss by 50% or more. The most committed organizations are also the most surprised by what it costs to sustain.

The architecture problem enterprises did not see coming

The cost surprise is not just a finance problem. It is an engineering problem with financial consequences.

“The biggest mistake enterprises make is trying to fit AI into a process that wasn’t designed for it. The smarter move is to redesign the process with AI in mind from the start,” Tesseyre said.

More AI:

Micron sits at the center of a red-hot chip rallyIBM CEO sends blunt message on AI and quantum computingAnthropic CEO makes shocking admission about AI

Most enterprise cloud environments were built for traditional applications where demand is predictable and resource usage is easier to optimize. AI changes that equation. It introduces variability, continuous retraining, and new dependencies on data quality and accessibility.

Tesseyre identifies three failure patterns that surface consistently: the absence of a solid MLOps foundation with automated pipelines and model validation; data architecture not designed with AI access patterns in mind, where retrofitting requires significant upfront work; and change management, which is less technical but just as consequential. Most enterprise environments were not built with AI as a core assumption, and inserting AI into processes designed without it generates friction that is substantially harder to resolve than redesigning from scratch.

Legacy infrastructure compounds the difficulty. Most enterprise technology decisions were made years before AI workloads existed at meaningful scale, according to Deloitte. That is an entirely different type of demand at the most foundational infrastructure levels.

A new spending cycle is emerging for investors to watch

“GPU spend is now a board-level conversation. Every technology leader we work with has a dedicated AI budget, and the pressure to justify it is real,” Tesseyre said.

That matches what the data shows. Organizations reporting AI as an active FinOps concern jumped from 31% in 2024 to 63% in 2025, according to CloudZero. AI and ML workloads now represent 22% of total cloud costs at SaaS and IT companies. Token leaderboards and token budgets are becoming standard management tools across enterprise teams.

The hyperscalers are responding. Amazon, Alphabet, Microsoft, Meta, and Oracle are collectively forecast to exceed $600 billion in capital expenditure in 2026, with roughly $450 billion tied to AI infrastructure, according to CloudZero. And 42% of enterprises say optimizing AI workflows is their top spending priority for 2026, according to NVIDIA. Optimization has overtaken expansion as the primary stated enterprise priority. The first wave was about acquiring capability. The second is about making it sustainable.

The next wave of AI spending looks nothing like the first one

Termmee/Getty Images

Where the real investment is now flowing

“Model experimentation gets the headlines. But you can’t build reliable AI without a modernized data estate underneath it. That’s where the real work is happening,” Tesseyre added.

That insight is consistent with what Zencore sees across its client base. The critical first steps in almost every enterprise AI engagement involve database modernization, migrating legacy data stores to the cloud, and building the pipelines that feed the AI layer. That foundational work is unglamorous but non-negotiable. Model experimentation tends to come slightly after, or in parallel, once teams realize that models are only as good as the data grounding them.

Deloitte’s 2026 enterprise AI report reinforces this view. Legacy data and infrastructure architectures cannot power real-time, autonomous AI. Modernization must create a living AI backbone that adapts dynamically to business and regulatory change, according to Deloitte. Organizations that fail to invest in that foundation will find their models constrained by the quality of the data underneath them.

Key figures on AI spending and infrastructure in 2026:Global AI spending forecast for 2026: $2.5 trillion, with AI infrastructure accounting for $401 billion, according to GartnerCombined hyperscaler capex for 2026: more than $600 billion, with $450 billion tied to AI infrastructure, according to CloudZeroInference costs dropped 280-fold over two years, but overall AI spending grew explosively as usage outpaced savings, according to DeloitteSome enterprises are now seeing monthly AI bills in the tens of millions of dollars, with agentic AI’s continuous inference sending token costs spiraling, Deloitte notedData center systems spending forecast to jump 55.8% in 2026, the largest single growth category in IT, according to GartnerRoughly 70% of large enterprises now maintain a dedicated FinOps or cloud economics team, CloudZero confirmedWhat this means for investors

Phase one of the AI trade rewarded those who bought the infrastructure buildout early. Phase two rewards those who understand where the sustained spending goes next.

The companies positioned to benefit are not the most visible names in AI. They are the ones solving the operational, architectural, and data problems that emerge once enterprises move past the pilot stage. Cloud optimization tools, data infrastructure platforms, MLOps providers, and specialized engineering firms are all part of this next wave.

What is not in doubt is the direction. AI spending is entering a more complex, more scrutinized phase. The days of deploying first and optimizing later are running out. And the investors who recognize that shift early are likely to find it matters more than the first wave ever did.

Related: JPMorgan executive says one thing is keeping AI in check

#phase #spending #underway