Demand for artificial intelligence (AI) chips, particularly ultra-powerful GPUs from Nvidia (NASDAQ: NVDA) used to train the most advanced AI models, appears to be insatiable. Nvidia’s data center segment generated more than $30 billion in revenue during the third quarter alone, up by nearly a factor of 10 compared to two years ago. Tech giants are scrambling to build AI data centers packed with Nvidia’s GPUs, spending untold billions on hardware.
Up until now, each new AI model to come out of OpenAI, one of those tech giants, or anyone else who’s caught AI fever has meaningfully improved on its predecessor. OpenAI’s GPT-4 is far more capable than GPT-3, and Alphabet‘s Gemini AI models blow its older models out of the water. But those improvements came at a cost.
Start Your Mornings Smarter! Wake up with Breakfast news in your inbox every market day. Sign Up For Free »
GPT-4 is estimated to have cost around $100 million to train, whereas GPT-3 may have cost just a few million dollars. Anthropic CEO Dario Amodei expects the next generation of AI models to cost around $1 billion to produce. Buying and then running many thousands of high-powered GPUs is expensive, and collecting mass amounts of training data is no picnic, either.
Large language models like GPT-4 work by predicting the next token in the output. This works pretty well in a lot of cases. The best AI models can produce high-quality text, generate convincing images, and even appear to do some fairly advanced reasoning. More training data generally produces better results, as does more time churning through that data in the training process.
However, AI companies appear to be reaching a limit. The rate of improvement in AI models is slowing, even with more data and more computational horsepower. Marc Andreessen, co-founder of venture capital firm a16z, recently noted that AI models seem to be hitting a ceiling in capabilities, regardless of the amount of data or computing power thrown at them.
A major breakthrough could help AI companies push through this ceiling, but it’s also possible that LLMs just aren’t capable of much more. Soaring demand for AI chips is driven by the idea that training a $1 billion or a $10 billion AI model makes financial sense. What if it doesn’t?
If AI models have largely topped out in terms of capabilities, the frantic multibillion-dollar AI investments being made by tech giants in an effort to not fall behind may never pay off in terms of revenue or profit. The hangover from this overinvestment could be brutal for companies like Nvidia as demand for AI chips dries up.
