Counterpoint analysts recently attended AMD’s ‘Advancing AI’ event in San Francisco where CEO Lisa Su unveiled the company’s latest cutting-edge end-to-end portfolio to advance AI from cloud to edge.

The launches included AMD’s fifth-generation x86-based server CPU EPYC Turin, which aims to extend the company’s overall server leadership beyond a third of the market. The latest Turin EPYC is seeing significant performance improvements across various applications, from database, media processing, supercomputing and HPC (like dense linear solvers and molecular dynamics) to machine learning and E2E AI performance when paired with GPU.

Data center architects face key challenges on the quantum of memory to be designed as it varies depending on use cases and workloads, from analytics to database to AI. Flexibility to scale up server memory for higher-performance workloads is becoming more important while maintaining lower TCO.

With partners such as Supermicro and Micron, AMD also showcased at the event a host of different server configurations to address the above mentioned challenges for the data center architects. For example, the below-featured demo highlights how Micron’s CXL2.0 module-based server designs from Supermicro featuring the AMD Turin CPU help scale up and lower the TCO while boosting memory performance across different workload.

The Interview

Key takeaways from the demo showcased by Micron engineering leader Eishan Mirakhur:

• High-capacity RDIMMs and CXL 2.0 memory modules from Micron designed and integrated in Supermicro’s H13 petascale server platform help expand memory capacity from 1.5 TB to 2.5 TB, a 66% increase with four 256GB CXL 2.0 modules.
• This additional memory enhances the CPU’s performance, like for multi-tenant workloads for virtual machines, leading to improved core utilization. It drives the overall efficiency and reduces the TCO in a scale-up scenario, which also simplifies solutions.
• It also features DDR5 128 GB RDIMMs that support speeds of up to 7,200 mega transfers and E3.S memory modules capable of 37 GB/s with up to 256 GB per module. This is ideal for database workloads, enterprise VMs, HPC and the latest AI RAG deployment and pipeline.

• This particular chassis has decided to go with four memory modules to bring one terabyte of additional memory expansion, but it is up to the vendor or the design requirements to decide the number of CXL cards to be deployed.

  Key analyst takeaways

• Over the decades, CPU performance has been growing at a higher pace than the memory bandwidth. This has created a bottleneck for data-hungry applications. Limited support for memory channels and expansion capabilities makes it difficult to scale up the server infrastructure efficiently.
• Having realized the different memory challenges, often referred to as the “memory wall”, CPU vendors such as AMD have been increasing support for memory expansion for open standard-based Compute Express Link (CXL) interfaces, bringing in flexibility to boost memory bandwidth and capacity, and enable prudent memory tiering to reduce latency gaps and the overall TCO.
• The integration of Micron’s CXL 2.0 solutions with partners brings additional flexibility to the data center architects in adding memory capacity “on the go” to achieve performance and cost goals at the same time in a multi-tenant workload environment.
• This resource efficiency also makes the data center more sustainable, helping the architects achieve those goals.
• • With CXL 3.1 specced in, it will bring newer CXL fabric improvements, from Port-Based Routing (PBR) switching supporting multiple topologies to Global Integrated Memory (GIM) concept that allows multiple hosts to map CXL memory modules, added level of security, acceleration and better scale-out expansions as well.