Modern AI requires a massive amount of processing power, especially when training LLM models. Today, the billions of trillions of calculations required are performed primarily by GPUs, whose parallel processing is well-suited for the task. No company has benefited more from this AI boom than Nvidia. Big tech companies and enterprises alike are fighting for access to its chips, which are well-suited for training and running advanced AI models.
For more than 10 years, Nvidia has built an almost unassailable lead in developing silicon that can perform complex AI tasks such as image, facial and speech recognition, as well as generating text for AI chatbots such as ChatGPT. It has achieved this dominance by recognizing the AI trend early on, adapting its chip hardware to suit those AI tasks and investing in its CUDA software ecosystem.
Nvidia keeps raising the bar. To maintain its leading position, the company now offers customers access to its specialized GPU-based computers, computing services and other tools on a “as-a-Service” basis. To all intents and purposes, this has turned Nvidia from being a chip vendor into a one-stop shop for AI development. However, there are two crucial factors that motivate Nvidia’s rivals, both established chip vendors and start-ups, and that is high pricing and the fear of vendor lock-in by its customers. Clearly, no company wants to be beholden to a dominant vendor and so more competition seems inevitable. The intense demand for AI services and the desire to diversify reliance on a single company are driving the ambitions of rival big chip vendors as well as numerous start-ups.
In 2017, Google’s Tensor Processing Unit (TPU), a chip designed specifically for Deep Learning. demonstrated that it was possible for new players to build domain-specific chips with better performance, lower power consumption and cost compared to Nvidia’s general-purpose GPUs. Now, the emergence of generative AI with its unique and heavy computational requirements presents new opportunities for domain-specific ASICs vendors.
Many AI chip start-ups believe that their new silicon innovations exceed Nvidia GPUs in performance and have a significantly lower power consumption since they have been designed specifically for the training and processing of deep neural networks. However, achieving commercial success has proven to be much more challenging, particularly with the explosion in foundation models that followed the launch of OpenAI’s ChatGPT. As a result, many start-ups have recently had to re-engineer their designs to handle the massive number of parameters needed for LLMs. Others have changed their business models to become service providers rather than chip vendors.
Counterpoint Research’s recent report “AI Chip Start-Ups – Can Domain-Specific Chips Impact Nvidia’s Dominance?” provides an overview of the AI chip start-up market and highlights the opportunities and challenges facing new players entering the burgeoning AI chip market.
Table of Contents
Introduction
Nvidia – A One Stop Ship for AI
The Compute Challenge
Established AI Chips Start-Ups
- Cerebras
- Groq
- SambaNova
- Tenstorrent
Emerging AI Chip Start-Ups
- Rivos, Enflame Technology, Rain AI, etc.
Key Challenges for Start-Ups
- Technical complexity
- Funding difficulties
- The Nvidia effect
- Open Standards
- Big Tech
Analyst Viewpoint
