AI Infra Summit is rapidly becoming one of the more important events on the calendar with multiple companies providing new insights into data center infrastructure. While speakers and companies covered a broad spectrum of topics, here are five we believe bear heightened scrutiny.
Key takeaways:
- NPO technologies will arrive in a few months, with CPO following soon after. How fast next-generation optics take off will be one of the critical issues for infrastructure through the remainder of the decade.
- Switch systems and ASICs have largely improved through incremental speed bumps. New designs from Cornelis, Eridu and others will try to upend that.
- While hyperscalers remain the primary customers for AI servers, expect to see companies increasingly target large enterprises.
- Samsung, Nvidia, and AWS, among others, are working on ways to lower the memory wall through new base die designs, and HBM and HBF architectures.
Expect to see technologies for fine-tuning power consumption.
1. NPO Hits Soon and CPO Is Hot on its Heels
Near-packaged optics (NPO) from Lightmatter and other vendors will start sampling in late 2026/early 2027, with commercial shipments in the second half of that year. Integrating optics on the board through NPO instead of inside the XPU package simplifies deployment. NPO concept is gaining commercial momentum with established module makers like TeraHop entering the NPO market via Open CPX designs.
CPO, however, won’t be far behind. A number of vendors are aiming for initial commercial shipments of CPO for scale-up in 2028 with some such as Coherent planning on initial shipments of CPO for scale-out at the end of this year and CPO for scale-up in late 2027. Ayar Labs, among others, are also preparing optical engines and other components for next-generation optics. (As always in optics, volume shipments for new technologies follow a few years after the initial hullabaloo.)
Ayar Labs Chief Strategy Officer Vivek Gupta shows the company’s TeraPHY optical engine for scale-up. The target is volume shipments in 2028.
Modules, meanwhile, will also continue to grow rapidly. Credo, among others, are readying to ramp 1.6T optical transceivers for the expected ramp of 102.4T switches. The XPO MSA is also working to extend the roadmap for pluggable transceivers with module designs that can deliver 12.8T of bandwidth.
By 2030, expect to see stark differences between scale-up and scale-out technology. Scale-up will likely be dominated by NPO/CPO technology. Adoption of NPO/CPO for scale-out will likely span a spectrum with some hyperscalers shifting to NPO/CPO as their default technology and others continuing to rely on a blend of modules and a more gradual introduction of NPO/CPO.
One of the largest factors determining the pace of the shift to NPO/CPO will be the success of the module industry in developing 400G/lane products. Several companies have spun up 400G/lane demonstrations and Broadcom released a 1.6T 400G/lane DSP in March 2026. Still, designing and producing 400G/lane modules that meet the power, performance and reach requirements in volume will be a challenge and likely require new modulation schemes and other changes in technology. Meanwhile, hyperscalers will already have a generation's worth of experience with integrated optics.
2029-2030 will also mark the beginning of intensive testing and possible commercialization for optics in scale-in networks.
2. Out-of-the-Box Ideas for Switching Are Coming
When it comes to innovation, switching has lagged processors, memories and other product categories, argues Lisa Spelman, CEO of Cornelis Networks. GPUs spent around 12.5% of their time waiting for the network. In a 100k-GPU cluster, that can mean $1.68B in wasted GPU hours and gigawatt-hours of power per year.
Cornelis’ solution is to integrate RISC-V cores into the switch and NIC to help optimize data transmission. In its hypothetical 100k-GPU system, an additional 1.1 million RISC-V cores equipped with 1.2TB of additional SRAM could increase network utilization from its typical 50% level to 60% for a 1.8-4x performance improvement on dollar per token.
Eridu, meanwhile, outlined its plans for bringing switch ASICs and systems with 10x more ports than conventional switches. Get an in-depth profile on the company’s technology in our recent Optics, Interconnects & Networking Monthly Supply Chain report.
3. Inference Goes to the Enterprise
Some companies build their AI strategies around the big five. Rebellions targets the big 500.
The company has created a portfolio of modular servers, racks, and pods that it claims can deliver the lowest total cost of ownership for inference. But just as important, it’s relatively straightforward to integrate into the data center operations of enterprises.
Rebellion’s RebelServer, for example, typically draws only 4kW, a thermal ceiling that makes it possible to deploy the system in standard data centers in Southeast Asia and other geographies without retrofits, says Chief Business Officer Marshall Choy. That’s critical, he adds, because a growing number of enterprises will seek to keep their AI infrastructure in-house. The fewer headaches, the lower the barriers to adoption. AI, he adds, will potentially follow the adoption pattern seen in the cloud with a large number of applications ultimately remaining inside an organization. Rebellions counts SK Telecom and KT Cloud as two customers taking the enterprise-centric approach to inference.
The modular inference platform from Rebellions. The RebelServer contains eight inference processors (which Rebellions calls NPUs) and two AMD EPYC CPUs. The NPU servers slide into the purpose-built rack fit and multiple racks can be tied together in a pod.
In a completely separate presentation, Dave Patterson, former UC Berkeley professor and current Google exec, noted that the search giant continues to use TPU-based systems that are over six years old. Google is far from being an ordinary enterprise, but the observation that the lifetime of AI servers is actually somewhat long provides further fodder for the idea that local AI may have legs.
4. Memory: Size (and Packaging) Matter
To accommodate the growing need for more memory close to the processor, Samsung Semiconductor outlined its plans for zHBM and zNAND-O for, respectively, stacking HBM and flash inside chip packages with TSVs.
Through its optimized design, Samsung said zHBM is up to 10x more dense than conventional HBM. 30% less data movement also means fewer metal layers, lower power consumption and 10-25% less thermal resistance.
By contrast, zNAND-O will deliver 7x better read power efficiency and 7x read bandwidth than conventional flash. Samsung is actively engaged with XPU customers on the design requirements for zHBM. Sampling for zNAND-O starts in 2028.
NVIDIA and Annapurna Labs, the chip group inside AWS, meanwhile, are collaborating on a custom base die for HBM that reduces the physical HBM footprint. By reducing the base die footprint, the available area for compute can increase by up to 30%.
When in doubt, go up. Samsung’s zNAND-O proposes to bring flash into the processor package for TBs of additional storage. Complementing zHBM, zNAND-O will constitute a new layer of storage.
5. A Dimmer Switch for Data Centers
Power has become the primary problem for AI and a large part of the effort to reduce it has revolved around producing semiconductors and systems that can effectively perform more work per watt. The big appeal of CPO, after all, is that it can reduce power from 14 picojoules per bit to 5 or less.
A growing number of companies, however, are looking at ways to dynamically control power to better match workloads in real-time.
One of the more noteworthy achievements: Nvidia DSM MaxLPS actively micromanages power consumption per rack. By precisely allocating power, it can raise the number of racks in a 1GW data center from 4,000 to 5,600 and throughput from 1.5B tokens per second to 2.1B.
Want more market intelligence?
Counterpoint’s Optics, Interconnects & Networking – Monthly Supply Chain & Market Pulse Intelligence tracks the technologies, vendors and supply-chain developments shaping next-generation AI data center connectivity.
Covering the ecosystem from substrate to rack, each edition analyzes scale-up, scale-out and scale-across interconnects, highlighting technology transitions, supply constraints, competitive dynamics and emerging opportunities.
Combined with Counterpoint’s quantitative trackers, the service helps clients understand how large the market is, who is positioned to capture it, why the market is changing and what to watch next.
Contact us to learn more.