H200 vs MI300X
MI300X has 1.4× the memory; MI300X rents for 1.2× less.
The NVIDIA H200 (Hopper, 2024) and the AMD Instinct MI300X (CDNA 3, 2023) come from different vendors and different design goals, but they get cross-shopped for a reason. On capacity, the MI300X carries 1.4x the memory (192 vs 141 GB), which sets what each can hold at all. Bandwidth is nearly even (4.8 vs 5.3 TB/s), so serving speed per GPU will be similar.
Price is where it settles: the MI300X rents from $1.85/hr against $2.30/hr for the H200, a 1.2x gap that the performance numbers above only partly close. Per gigabyte of VRAM, the MI300X is the cheaper rental, worth knowing if your model is capacity-bound rather than speed-bound. For how these numbers translate into tokens per dollar, the inference cost estimator runs both cards against any model. Everything below is the underlying data.
NVIDIA · Hopper · 2024
NVIDIA H200
An H100 with 76% more memory and 43% more bandwidth, the practical choice for large-model inference.
From $2.30/hr at Nebius
AMD · CDNA 3 · 2023
AMD Instinct MI300X
AMD's answer to Hopper: 192 GB on one GPU, a 70B model in FP16 fits with room to spare.
From $1.85/hr at Vast.ai
Head to head
■ H200 ■ MI300X, bars share one scale across the whole catalog.
| H200 | MI300X | |
|---|---|---|
| Memory | 141 GB HBM3e | 192 GB HBM3 |
| Bandwidth | 4.8 TB/s | 5.3 TB/s |
| FP16 dense | 990 TF | 1,307 TF |
| FP8 dense | 1,979 TF | 2,614 TF |
| TDP | 700 W | 750 W |
| Interconnect | NVLink 4 · 900 GB/s | Infinity Fabric · 896 GB/s |
| Cheapest rental | $2.30/hr | $1.85/hr |
| $/hr per GB VRAM | $1.6¢ | $1.0¢ |
Rule of thumb: for LLM serving, prefer the GPU with more memory bandwidth per dollar; for training and prefill-heavy work, prefer FLOPS per dollar; and if the model doesn't fit in VRAM, none of the other numbers matter. Sanity-check with the VRAM calculator.