The $Trillion Bottleneck: NVIDIA's NVHBM and the Fight for the Base Die
- NVHBM moves the memory controller into the HBM base die and replaces the JEDEC bus with a private die to die link. NVIDIA designs that layer. The Big Three memory makers build it to order.
- The architecture is not new. Marvell specified the same thing in December 2024 and in more detail. What NVIDIA has that Marvell does not is a named customer, Annapurna on Trainium4, and a rack standard to attach it to.
- Five days later MediaTek adopted NVLink Fusion and will offer it to its own XPU customers, with $3.5bn of NVIDIA money attached. NVHBM went from an architecture with one customer to a platform with a channel.
- Only the 67% PHY area reduction follows from something we can check, which is pin count. The bandwidth, area and performance claims are normalised against HBM4E, which is not shipping, and no baseline has been disclosed.
- The commercial point is dollar content. NVIDIA can now sell the fabric, the rack, the system design, the software and the memory subsystem into a custom accelerator programme without building the accelerator, and MediaTek carries it to customers NVIDIA does not cover directly.
- Broadcom is hit and helped at once. NVIDIA is now in its business and Broadcom has no base die answer, but with Marvell on NVLink Fusion and MediaTek reselling the platform, Broadcom is the only large design house left that does not come with NVIDIA attached.
On 26 August, NVIDIA extended NVLink Fusion with NVHBM, a custom high-bandwidth memory technology built around a proprietary base die and interface that NVIDIA designs but does not make. Amazon's Annapurna Labs is the first named partner, beginning with Trainium4. The base die will be manufactured by the “Big Three” memory partners against a single NVIDIA design that covers the memory controller, the interface and the link protocol. None of the three have been named so far.
The difference is where the controller sits. Conventional HBM puts the memory controller on the accelerator die; NVHBM moves it into the base die of the 3D DRAM stack and replaces JEDEC's wide parallel bus with a narrower serialised link.
It is a real architectural step, but the goal is commercial: to extend NVIDIA’s clout into the XPU segment of the market. The base die is where accelerator differentiation now happens — and whoever controls that layer controls the platform around it.
NVHBM is thus not really a memory product. It is NVIDIA selling the same custom silicon engagement Broadcom and Marvell built their accelerator businesses on. Everything around the compute die, which is the one piece hyperscalers insist on owning.
Focus where the Controller Lives
HBM has always been a compromise between two dies owned by two different companies. The DRAM vendor supplies the stack and its base die, and the accelerator designer supplies the controller and PHY. JEDEC standardises the seam between them, and the price of that standardisation is a very wide, very slow interface. The HBM4 runs over two thousand signal pins across thirty-two 64-bit channels.
NVHBM dissolves the seam. With the controller inside the base die, the link between logic and memory no longer has to be a JEDEC-compliant memory bus at all. It becomes a dedicated die-to-die interface that NVIDIA can design for speed rather than interoperability. NVIDIA now owns the design of that whole layer. The memory maker simply builds it to order.
Third in the Trillion Dollar Bottleneck series, the first two notes looked at how the industry is attacking the memory wall through structure and packaging — Intel's XBM and ZAM against TSMC CoWoS, and Qualcomm's HBC. This one is different: NVIDIA is now selling memory architecture to the people building alternatives to its GPUs. The technology is not new. The distribution is.
Read the full report: The $Trillion Bottleneck: NVIDIA's NVHBM and the Fight for the Base Die
Get a Cloud AI industry briefing from our live webinar: Cloud AI – Trillion-Dollar Infrastructure, Companies & Compute Trends
Similar blogs from the series:
The $Trillion Bottleneck: Intel Bets EMIB, ZAM and XBM Against TSMC’s CoWoS
The $Trillion Bottleneck: Inside Qualcomm's High Bandwidth Compute (HBC) Bet
Receive our insightful weekly newsletter and stay ahead of the competition.
Author
Neil Shah
Neil is a sought-after frequently-quoted Industry Analyst with a wide spectrum of rich multifunctional experience. He is a knowledgeable, adept, and accomplished strategist. In the last 18 years he has offered expert strategic advice that has been highly regarded across different industries especially in telecom. Prior to Counterpoint, Neil worked at Strategy Analytics as a Senior Analyst (Telecom). Neil also had an opportunity to work with Philips Electronics in multiple roles. He is also an IEEE Certified Wireless Professional with a Master of Science (Telecommunications & Business) from the University of Maryland, College Park, USA.