Google’s concern with bottlenecks shows the arrival of a new stage of AI, says analyst

Google’s head of artificial intelligence infrastructure has warned that the company needs to expand its technology to accommodate a huge influx of users and complex requests being handled by AI products — and that could be a sign that fears of a bubble are overblown.

Amin Vahdat, a vice president who leads Google’s global AI and infrastructure team, said during a presentation at an all-hands meeting on Nov. 6 that the company needs to double its fulfillment capacity every six months, reaching a thousandfold increase four to five years from now, CNBC reported.

Also read: The advancement of AI and layoffs raise the question: has the era of robot unemployment arrived?

This refers to Google’s ability to ensure that Gemini and other AI products that rely on Google Cloud still work well when queried by a growing number of users. This is different from computing, or the physical infrastructure involved in training AI.

A Google spokesperson told Fortune that “demand for AI services means we are being asked to provide significantly greater computing capacity, which we are driving through efficiencies in hardware, software and model optimizations, as well as new investments,” pointing to the company’s Ironwood chips as an example of proprietary hardware providing improvements in computing capacity.

In previous years, every hyperscaler—large-capacity services like Google Cloud, Amazon, and Microsoft Azure—raced to ramp up compute in anticipation of a wave of AI users.

Now, the users have arrived, said Shay Boloor, chief market strategist at Futurum Equities. But as each company ramps up its AI offerings, fulfillment capability is emerging as the next big challenge to tackle.

“We are entering the second stage of AI, where fulfillment capacity matters even more than computing capacity, because computation creates the model, but fulfillment capacity determines how widely and how quickly that model can actually reach users,” he told Fortune.

Google, with its vast capital expenditures and past strategic moves to develop its own AI chips, is likely capable of doubling its fulfillment capacity every six months, Boloor said.

Still, Google and its competitors face an uphill battle, he added, especially as AI products begin to handle more complex requests, including advanced search and video queries.

“The bottleneck is not caused by a lack of plans, it is actually physical limitations such as power, cooling, network bandwidth and the time required to build turnkey data centers,” he said.

However, the fact that Google is apparently facing so much demand for its AI infrastructure, with pressure to double its fulfillment capacity so quickly could be a sign that the gloomy predictions made by AI doomsayers are not entirely correct, Boloor said.

Such concerns caused all three major US stock indexes to fall 1.9% or more last week — including the Nasdaq, which is heavily linked to the technology sector.

“This isn’t speculative enthusiasm, it’s just unmet demand waiting in line,” he said. “If things are slowing down a little more than many people expected, it is because everyone is limited in computing and service capacity.”