H100 SXM vs GH200
GH200 has 1.2× the memory; GH200 has 1.2× the bandwidth; GH200 rents for 1.4× less.
The NVIDIA H100 SXM (Hopper, 2022) and the NVIDIA GH200 (Grace Hopper, 2023) are close contemporaries from adjacent generations, so the details decide. On capacity, the GH200 carries 1.2x the memory (96 vs 80 GB), which sets what each can hold at all. On memory bandwidth, the number that governs LLM serving speed, the GH200 leads at 4 vs 3.35 TB/s.
Price is where it settles: the GH200 rents from $1.35/hr against $1.90/hr for the H100 SXM, a 1.4x gap that the performance numbers above only partly close. Per gigabyte of VRAM, the GH200 is the cheaper rental, worth knowing if your model is capacity-bound rather than speed-bound. For how these numbers translate into tokens per dollar, the inference cost estimator runs both cards against any model. Everything below is the underlying data.
NVIDIA · Hopper · 2022
NVIDIA H100 SXM
The workhorse of the AI boom. Still the most widely available serious training and inference GPU.
From $1.90/hr at Hyperstack
NVIDIA · Grace Hopper · 2023
NVIDIA GH200
A Hopper GPU welded to a Grace CPU: 576 GB of unified fast memory for models that spill past VRAM.
From $1.35/hr at Vast.ai
Head to head
■ H100 SXM ■ GH200, bars share one scale across the whole catalog.
| H100 SXM | GH200 | |
|---|---|---|
| Memory | 80 GB HBM3 | 96 GB HBM3 + 480GB LPDDR5X |
| Bandwidth | 3.35 TB/s | 4 TB/s |
| FP16 dense | 990 TF | 990 TF |
| FP8 dense | 1,979 TF | 1,979 TF |
| TDP | 700 W | 700 W |
| Interconnect | NVLink 4 · 900 GB/s | NVLink-C2C · 900 GB/s CPU↔GPU |
| Cheapest rental | $1.90/hr | $1.35/hr |
| $/hr per GB VRAM | $2.4¢ | $1.4¢ |
What one can run that the other can't
Only on H100 SXM
Nothing, GH200 runs everything H100 SXM does.
Only on GH200
- gpt-oss-120b (4-bit)
Rule of thumb: for LLM serving, prefer the GPU with more memory bandwidth per dollar; for training and prefill-heavy work, prefer FLOPS per dollar; and if the model doesn't fit in VRAM, none of the other numbers matter. Sanity-check with the VRAM calculator.