AMD GPU Architecture #6: CDNA 2

Series Roadmap

The Two Problems CDNA 1 Left Behind

CDNA 1 (MI100) inherited GCN’s compute foundation and began matrix acceleration with Matrix Core. It beat the A100 on FP64/FP32 vector. But it had two weaknesses.

• No FP64 matrix operation: Matrix Core accelerated only FP16/BF16/FP32. FP64 ran on the vector pipeline alone, with no answer to the A100’s FP64 Tensor Core for scientific computing centered on double-precision dense linear algebra.
• The limit of a single die: MI100 was a 120-CU monolith. More performance meant a bigger die, and large dies suffer collapsing yields.

CDNA 2 (Aldebaran, MI250/MI250X, November 2021) attacked both head-on: surpass the die limit with MCM, and complete HPC performance with an FP64 Matrix Core.

MCM: AMD’s First Multi-Die GPU

CDNA 2’s headline is MCM (Multi-Chip Module) — AMD’s first integration of two GPU dies in a single package.

The MI250X places two GCDs (Graphics Compute Dies) on an OAM package. Each GCD has its own compute units and HBM2e memory.

• Per GCD: 110 CUs, 7,040 SP, 64 GB HBM2e
• MI250X total: 220 CUs, 14,080 SP, 128 GB HBM2e, 3.2 TB/s
• Die-to-die link: Infinity Fabric, 400 GB/s bidirectional (200 GB/s each direction)

It Is Not a Unified GPU: The Key Nuance

The MI250X’s two GCDs do not merge into one logical GPU. They appear as two separate GPUs to the OS and the programmer.

How software sees a single MI250X:
┌──────────────────────────────────────────┐
│  MI250X (physically one card)              │
│                                          │
│  GPU 0 (GCD 0)  ←→  GPU 1 (GCD 1)       │
│  64 GB          IF   64 GB              │
│  separate addr space  separate addr space  │
│                                          │
│  → programs see 2 GPUs and must          │
│    explicitly partition data             │
└──────────────────────────────────────────┘

A single Frontier node holds four MI250X cards. Software sees them as eight GPUs (eight GCDs). For one GCD to access another’s memory, it traverses Infinity Fabric, creating NUMA-like (non-uniform memory access) characteristics.

This design was a pragmatic way around the die-size limit, but it left the programmer to handle two dies explicitly. Merging the two dies into one logical GPU becomes the mission of the next generation, CDNA 3 (MI300).

FP64 Matrix Core: Completing HPC Performance

CDNA 2’s second key feature is the addition of the FP64 Matrix Core.

CDNA 1’s Matrix Core did not support FP64 matrix operations. CDNA 2 added them and simultaneously raised FP64 vector operations to full rate.

FP64 performance evolution (per card):
  MI100 (CDNA 1)  : FP64 vector 11.5 TFLOPS,  no FP64 Matrix
  MI250X (CDNA 2) : FP64 vector 47.9 TFLOPS,  FP64 Matrix 95.7 TFLOPS
                    ↑ vector ~4× (2 dies + full-rate FP64 per die)
                    ↑ Matrix new (2× the vector rate)

FP64 jumping ~4× over MI100 is the product of two factors: the die count doubled (2×), and per-die FP64 throughput doubled to full rate (2×). On top of that, the FP64 Matrix Core adds 2× the vector rate (95.7 TFLOPS).

This double-precision performance made CDNA 2 a force in HPC. Scientific computing’s dense linear algebra is dominated by FP64 matrix multiply. Against the A100’s FP64 Tensor Core (19.5 TFLOPS), the MI250X’s FP64 Matrix (95.7 TFLOPS) leads by roughly 5×.

Memory and Interconnect

HBM2e Memory

The MI250X connects four HBM2e stacks per GCD — eight stacks, 128 GB total — over an 8192-bit interface at 3.2 TB/s, about 2.6× the CDNA 1 figure (1.23 TB/s).

3rd-Gen Infinity Fabric: Extending to the CPU

CDNA 2’s Infinity Fabric extended beyond GPU-to-GPU links to a coherent CPU-GPU connection.

Frontier node configuration:
┌────────────────────────────────────────────────┐
│  EPYC CPU (64-core, 3rd Gen)                     │
│     │  Coherent Infinity Fabric (36+36 GB/s/GCD) │
│     ├── MI250X #1 (GCD 0, GCD 1)                │
│     ├── MI250X #2 (GCD 2, GCD 3)                │
│     ├── MI250X #3 (GCD 4, GCD 5)                │
│     └── MI250X #4 (GCD 6, GCD 7)                │
│  → 8 GCDs per node, unified CPU-GPU memory space │
└────────────────────────────────────────────────┘

CPU and GPU connect coherently over Infinity Fabric. Without going through PCIe, they share memory while maintaining cache coherence — easing bottlenecks in HPC workloads with frequent CPU-GPU data movement. AMD realized with Infinity Fabric, during the CDNA 2 era, a direction similar to what NVIDIA later built with NVLink-C2C in Grace-Hopper.

MI250X Specs:

Frontier: The World’s First Exascale System

CDNA 2’s significance goes beyond benchmark numbers. The MI250X is the compute engine of Frontier.

Frontier, at Oak Ridge National Laboratory (ORNL), took the Top500 #1 spot in June 2022 as the world’s first system to break exascale (over 1 EFlop/s).

• Node config: 1 EPYC CPU + 4 MI250X (8 GCDs)
• Scale: over 9,000 nodes
• Performance: over 1.1 EFlop/s on the HPL benchmark
• Efficiency: also ranked high on the Green500 (power efficiency)

Frontier proved AMD had secured a top-tier system in datacenter compute. The same MI250X powers Europe’s LUMI supercomputer. It was a turning point showing AMD as a real alternative in the top-tier HPC market NVIDIA had dominated.

Comparison with NVIDIA A100

At the MI250X launch (November 2021), the competitor was NVIDIA’s A100 (Ampere). The H100 (Hopper) arrived the following year.

Specification Comparison

Domain Breakdown

HPC (FP64)

The MI250X dominates. Its 95.7 TFLOPS FP64 Matrix is nearly 5× the A100’s. It also leads on memory capacity (128 vs. 80 GB) and bandwidth (3.2 vs. 2.0 TB/s). Frontier’s exascale achievement is the proof of this advantage.

AI/Deep Learning

A mixed picture. The MI250X (383) beats the A100 (312) on dense FP16. But the A100 extends to 624 TFLOPS with structured sparsity and supports the TF32 training format. In effective AI performance, the A100 often held the edge.

Programming Model

The A100 has the advantage. As a single logical GPU, it is simpler to program. The MI250X requires handling two GCDs explicitly, with the programmer managing die-to-die data movement — complexity that raised the burden of software porting and optimization.

Software

CUDA’s ecosystem still led. However, Frontier’s deployment began validating ROCm on a large production system, and HPC library support improved.

Summary

CDNA 2 lifted AMD to the top of HPC. It surpassed the die limit with MCM, completed double-precision performance with the FP64 Matrix Core, and achieved the world’s first exascale with Frontier. But the split-MCM design, where two GCDs appear as separate GPUs, left programming complexity behind. Merging those dies into one logical GPU — and putting the CPU in the same package — becomes the next generation’s challenge.

Next: CDNA 3 — XCD chiplets, a CPU+GPU integrated APU (MI300A), and El Capitan