One of the biggest trends impacting hyperscalers is the end of Dennard Scaling and thus the end of Moore’s Law. This means that it is no longer possible to get 2x performance gains from next-generations CPUs anymore. At the same time, the popularity of AI applications such as ChatGPT is significantly impacting silicon design and chip designers must look for alternative high-performance silicon solutions. In recent years, workload footprints have continued to expand at a phenomenal rate forcing data center operators to innovate to keep up with scaling demands while simultaneously focusing on improving ROI. While the size of a typical AI cluster today is around 25,000 GPUs, this is expected to increase dramatically to 100,000 GPUs and beyond over the next few years.

Current AI solutions adopt a “one-size-fits-all” approach to hardware development. However, AI models have evolved from the AlexNet and ResNet models of a few years ago to transformer-based LLMs today. AI models are still changing rapidly which means that customers need a hardware architecture that can adapt to this change otherwise new AI accelerators will become obsolete very quickly.

With ChatGPT driving up the cost of computing, hyperscalers are looking to increase compute efficiency, i.e. TFLOPS/Watt/mm2. Hyperscalers also need to optimize their compute solutions in order to get a competitive advantage over rivals. This is why many of these hyperscalers have been developing custom silicon solutions, including various RISC-V-based solutions.

RISC-V is a fledgling ISA which offers some unique benefits compared to established ISAs such as x86 and ARM. In particular, it is a clean-slate, modular ISA that is extendable and customizable. As an open standard, it removes barriers to entry, which encourages innovation and allows companies of all sizes to design custom CPUs tailored to their specific needs - without the burdens of high licensing/royalty costs. Unlike proprietary ISAs, the RISC-V instruction set can be used and changed by anyone, creating the ability to extend the processor exactly as a use case requires.

In essence, RISC-V provides designers with the flexibility to customize and construct processors tailored to their specific end applications, enabling optimization of power, performance and area (PPA) for those particular uses. This greater capacity to differentiate creates competition in processors, ultimately leading to more innovation.

Hyperscalers have been using custom silicon as an alternative to GPUs for selected workloads for several years. With the growth in AI/ML models expected to continue unabated, Counterpoint Research believe that this trend will accelerate and that RISC-V will play an increasing role in AI/ML workload acceleration in data centers as well as non-AI/ML workload acceleration applications over the next few years.

Report: RISC-V: Emerging Opportunities in Data Center Markets

Counterpoint Research will shortly publish a report that investigates the main opportunities for RISC-V in the data center, detailing the various applications and use cases where RISC-V is a viable alternative to existing proprietary ISAs. The report highlights the benefits that RISC-V offers and the key trends driving the adoption of RISC-V in the key data center market segment. The report also provides details of the chip and software ecosystems, including detailed profiles of selected key RISC-V vendors.

Table of Contents (Provisional)

Key Takeaways

Introduction

Key Trends and Drivers

Key Benefits of RISC-V

Tech Overview

RISC-V In the Data Center: Market Opportunities

Accelerating Data Center Workloads with RISC-V

DSA via RISC-V and the Role of Chiplets

RISC-V Chip Vendor Ecosystem

RISC-V Software Ecosystem

Key Challenges

Analyst Viewpoint