H100 SXM vs MI300X
MI300X has 2.4× the memory; MI300X has 1.6× the bandwidth.
The NVIDIA H100 SXM (Hopper, 2022) and the AMD Instinct MI300X (CDNA 3, 2023) come from different vendors and different design goals, but they get cross-shopped for a reason. On capacity, the MI300X carries 2.4x the memory (192 vs 80 GB), which sets what each can hold at all. On memory bandwidth, the number that governs LLM serving speed, the MI300X leads at 5.3 vs 3.35 TB/s.
Price is where it settles: the MI300X rents from $1.85/hr against $1.90/hr for the H100 SXM, close enough that price should not decide. Per gigabyte of VRAM, the MI300X is the cheaper rental, worth knowing if your model is capacity-bound rather than speed-bound. For how these numbers translate into tokens per dollar, the inference cost estimator runs both cards against any model. Everything below is the underlying data.
NVIDIA · Hopper · 2022
NVIDIA H100 SXM
The workhorse of the AI boom. Still the most widely available serious training and inference GPU.
From $1.90/hr at Hyperstack
AMD · CDNA 3 · 2023
AMD Instinct MI300X
AMD's answer to Hopper: 192 GB on one GPU, a 70B model in FP16 fits with room to spare.
From $1.85/hr at Vast.ai
Head to head
■ H100 SXM ■ MI300X, bars share one scale across the whole catalog.
| H100 SXM | MI300X | |
|---|---|---|
| Memory | 80 GB HBM3 | 192 GB HBM3 |
| Bandwidth | 3.35 TB/s | 5.3 TB/s |
| FP16 dense | 990 TF | 1,307 TF |
| FP8 dense | 1,979 TF | 2,614 TF |
| TDP | 700 W | 750 W |
| Interconnect | NVLink 4 · 900 GB/s | Infinity Fabric · 896 GB/s |
| Cheapest rental | $1.90/hr | $1.85/hr |
| $/hr per GB VRAM | $2.4¢ | $1.0¢ |
What one can run that the other can't
Only on H100 SXM
Nothing, MI300X runs everything H100 SXM does.
Only on MI300X
- Mixtral 8x22B (8-bit)
- gpt-oss-120b (8-bit)
Rule of thumb: for LLM serving, prefer the GPU with more memory bandwidth per dollar; for training and prefill-heavy work, prefer FLOPS per dollar; and if the model doesn't fit in VRAM, none of the other numbers matter. Sanity-check with the VRAM calculator.