B200 vs MI325X
MI325X has 1.3× the memory; B200 has 1.3× the bandwidth; MI325X rents for 1.6× less.
The NVIDIA B200 (Blackwell, 2024) and the AMD Instinct MI325X (CDNA 3, 2024) come from different vendors and different design goals, but they get cross-shopped for a reason. On capacity, the MI325X carries 1.3x the memory (256 vs 192 GB), which sets what each can hold at all. On memory bandwidth, the number that governs LLM serving speed, the B200 leads at 8 vs 6 TB/s. Raw compute favors the B200 by 1.7x, which matters for training, prefill-heavy traffic, and diffusion work more than for chat-style decoding.
Price is where it settles: the MI325X rents from $2.40/hr against $3.75/hr for the B200, a 1.6x gap that the performance numbers above only partly close. Per gigabyte of VRAM, the MI325X is the cheaper rental, worth knowing if your model is capacity-bound rather than speed-bound. For how these numbers translate into tokens per dollar, the inference cost estimator runs both cards against any model. Everything below is the underlying data.
NVIDIA · Blackwell · 2024
NVIDIA B200
NVIDIA's Blackwell flagship: 192 GB of HBM3e and roughly double Hopper's throughput per chip.
From $3.75/hr at Packet.ai
AMD · CDNA 3 · 2024
AMD Instinct MI325X
256 GB of HBM3e, the largest memory pool on any single accelerator you can rent.
From $2.40/hr at Vast.ai
Head to head
■ B200 ■ MI325X, bars share one scale across the whole catalog.
| B200 | MI325X | |
|---|---|---|
| Memory | 192 GB HBM3e | 256 GB HBM3e |
| Bandwidth | 8 TB/s | 6 TB/s |
| FP16 dense | 2,250 TF | 1,307 TF |
| FP8 dense | 4,500 TF | 2,614 TF |
| TDP | 1000 W | 1000 W |
| Interconnect | NVLink 5 · 1.8 TB/s | Infinity Fabric · 896 GB/s |
| Cheapest rental | $3.75/hr | $2.40/hr |
| $/hr per GB VRAM | $2.0¢ | $0.9¢ |
Rule of thumb: for LLM serving, prefer the GPU with more memory bandwidth per dollar; for training and prefill-heavy work, prefer FLOPS per dollar; and if the model doesn't fit in VRAM, none of the other numbers matter. Sanity-check with the VRAM calculator.