The world’s leading full-stack AI systems company once again laid out a strong foundation of offerings from semi to software to help industry accelerate, monetize and create value in the Agentic AI era.   

Key announcements from NVIDIA on the occasion included:   

• New Vera Rubin chips 
• Racks enhanced with Groq 3 LPX (SRAM-driven ultra-premium acceleration) 
• BlueField-4 STX (brings in massive warm KV Cache for longer-context AI) 
• NemoClaw (security shell around OpenClaw)  
• AI Grids (for verticals like Telco)  
• Nemotron 3 (throughput booster)  
• DSX AI platform (accelerated AI factories)  
• Open models and libraries 
• New NVIDIA Cosmos 3 world models, NVIDIA Isaac simulation frameworks and NVIDIA Isaac GR00T N models (accelerating Physical AI) 

Counterpoint has planned a series of analyses related to GTC 2026, and in this post, we will focus on three key drivers that will reshape the performance metrics, token economics, and applications for the Agentic AI infrastructure to be deployed over the next couple of years. 

Groq 3 LPX: A New Paradigm in Low-Latency Inferencing for Vera Rubin Platform

The Groq Strategic Acquisition:

In December 2025, NVIDIA entered into a strategic partnership with Groq, an AI chip startup known for its innovative Language Processing Unit (LPU) architecture, for a perpetual non-exclusive licensing for Groq’s entire patent portfolio, software stack and almost a full leadership and engineering talent acquisition for $20 billion, almost at a 3x multiple.

Why?

As the AI industry moves from training to inferencing, and in parallel from Generative AI to Agentic AI, latency-sensitive inferencing becomes supercritical for real-time agentic experiences. The “Von Neumann” bottleneck of separation between the processing unit and memory remains a primary constraint. While HBM solves this for GPUs by offering impressive bandwidth closest to the GPU, there is still latency (up to 500ms) associated with fetching data from the off-chip memory, bringing in “jitter”.

Groq Brings in Deterministic Low-Latency SRAM-driven Inferencing

The Groq LPU’s approach involves offering “deterministic” inferencing, which leverages Static RAM (SRAM) instead of DRAM, distributed across the compute and enabling 150 TB/s or more bandwidth and almost sub-40ms latency, which cannot be perceived by a human reading the response and feels almost real-time. The Groq LPU processes a single user’s request with maximum efficiency, unlike GPU-HBM batch processing.

In a 1U tray, there are 8x Groq3 LPUs, Groq’s RealScale C2C spine connectors, a host CPU, BlueField-4 DPU, and 32LPU C2C optical links for scale-up.

At rack scale, Groq 3 LPX delivers 256 interconnected NVIDIA Groq 3 LPU accelerators, unlocking 315 PFLOPs packing 128GB SRAM with 40 PB/s on-chip bandwidth and 640TB/s scale-up bandwidth.

Analyst Takes:

• The most important metrics Groq 3 LPU focuses on are time-to-first-token, tokens per second per user, and tail latency.
• The increasing requirements for some AI models for multi-step chain thoughts and longer contexts drive the need for sequential token generation, where the LPU shines.
• NVIDIA has made “low-latency deterministic” inferencing a product feature, now enabling its customers to offer premium Agentic AI experiences.
• Groq 3 LPX is thus a big differentiator for NVIDIA, offering a heterogeneous inferencing infrastructure solution to its customers.
• The capex for adding an LPX will be key to understanding the ROI for the same.
• We will deep dive into Groq 3 architecture and identify the potential highly interactive applications where delay is immediately visible.

Tokenomics: Heterogeneous Architecture for Agentic Era

NVIDIA’s move to heterogeneous AI Infrastructure for inferencing is a function of the evolving nature of workloads from the Generative to the Agentic era. From context-heavy prefill to low-latency decode, the workloads are no longer uniform and require more than an HBM-GPU combination. So, the addition of LPU takes care of the interactive agentic decode for latency-sensitive small batch sizes, which could otherwise stall or induce contention or jitter.

Boosting Throughput (TPS per MW) and Interactivity (TPS per User)

Combined Rubin and LPU architecture allows NVIDIA to offer higher AI factory throughput as well as greater interactivity when needed, respectively. NVIDIA estimates that the combined system can deliver more than 35x performance efficiency than Blackwell.

• The heterogeneous AI infrastructure approach unlocks “interactive inferencing” and is optimized for trillion-parameter models.
• The addition of LPX to Vera Rubin helps change the economics of token generation, boosting throughput for heavy workloads as well as interactivity for the latency-sensitive Agentic AI workloads such as coding assistants, chatbots and copilots.
• This expands infrastructure to newer tiers for monetization from token generation.

CMX: NVIDIA Context Memory Storage with BlueField-4 STX Boosts KV Cache Capacity for Long-Context Agentic AI Workloads

Agentic AI Demands More KV Cache Capacity

Agentic AI workflows are driving context windows to millions of tokens, which need real-time data and context to ensure quick, coherent conversations and tasks. As context expands, conventional storage and data paths may slow inference and limit GPU efficiency. As context windows increase, Key-Value (KV) Cache capacity needs to increase to retrieve and recalculate the history faster. The need for AI-native KV Cache also increases. It’s time to rethink how context will be placed across different memory tiers from G1 - GPU HBM, G2 – LPDRAM and G3 - local SSDs to G4 - shared storage. The G4 provides capacity but introduces latency and power inefficiencies, and thus a lack of scalability for KV Cache to be stored in this tier.

NVIDIA CMX Platform Solves Pod-Level KV Cache Sharing Problem

To solve this challenge, NVIDIA introduced CMX context memory storage, an AI‑native context tier for long‑context agentic AI reasoning. This is a new storage tier (G3.5) at the pod level to bridge the gap between G1 and G4, dedicated to KV Cache and acting as a “long-term memory” for multiple agents simultaneously. [RG2.1]It is close enough to efficiently pre-fill the HBM or DRAM frequently from the storage. This approach minimises capacity and bandwidth demand on G4, while ensuring that critical application-level history is maintained where necessary. This is powered by NVIDIA BlueField-4 STX reference architecture.

NVIDIA Dynamo, DOCA Memos, and NVIDIA Inference Transfer Library (NIXL) manage prefill and decode, while coordinating advanced context-sharing AI-native KV Cache strategies across the AI nodes, thereby boosting Agentic inferencing. This improves tokens per second, reduces time to first token, and enables pod-wide context reuse for multi-turn, multi-agent workloads.

• STX brings in contextual intelligence with closely connected high-performance storage to enable frequent exchange of context data between the storage and GPU memory, thus supporting pod-wide long context KV Caches and model weight slices.
• This dedicated context memory tier extends GPU memory capacity and boosts tokens per second for long-context multi-agent inferencing.
• This drives more efficiency in terms of power with more tokens per watt, reducing
   recomputation, GPU idle time and thus boosting scalability.
• It also reduces unit cost for long-context and Agentic AI inferencing.
• This makes NVL72, powered by Vera+Rubin+LPX+STX, one of the most powerful infrastructure solutions for real-time, high-throughput and long-context Agentic AI communication.
• The entire system is liquid-cooled and super-easy to install.