H100 SXM vs A100 80GB
H100 SXM has 1.7× the bandwidth; A100 80GB rents for 2.2× less.
The NVIDIA H100 SXM (Hopper, 2022) and the NVIDIA A100 80GB (Ampere, 2020) are 2 years and a silicon generation apart, which makes the price gap the heart of the story. Memory is effectively a wash at 80 vs 80 GB, so capacity does not decide this one. On memory bandwidth, the number that governs LLM serving speed, the H100 SXM leads at 3.35 vs 2 TB/s. Raw compute favors the H100 SXM by 3.2x, which matters for training, prefill-heavy traffic, and diffusion work more than for chat-style decoding. The H100 SXM also speaks native FP8 while the A100 80GB tops out at BF16/INT8, a generational gap explained in CUDA cores vs Tensor Cores.
Price is where it settles: the A100 80GB rents from $0.87/hr against $1.90/hr for the H100 SXM, a 2.2x gap that the performance numbers above only partly close. Per gigabyte of VRAM, the A100 80GB is the cheaper rental, worth knowing if your model is capacity-bound rather than speed-bound. The longer decision guide for this matchup is A100 vs H100 for inference. Everything below is the underlying data.
NVIDIA · Hopper · 2022
NVIDIA H100 SXM
The workhorse of the AI boom. Still the most widely available serious training and inference GPU.
From $1.90/hr at Hyperstack
NVIDIA · Ampere · 2020
NVIDIA A100 80GB
The GPU that started the LLM era, now a value pick for fine-tuning and mid-size inference.
From $0.87/hr at Vast.ai
Head to head
■ H100 SXM ■ A100 80GB, bars share one scale across the whole catalog.
| H100 SXM | A100 80GB | |
|---|---|---|
| Memory | 80 GB HBM3 | 80 GB HBM2e |
| Bandwidth | 3.35 TB/s | 2 TB/s |
| FP16 dense | 990 TF | 312 TF |
| FP8 dense | 1,979 TF | - |
| TDP | 700 W | 400 W |
| Interconnect | NVLink 4 · 900 GB/s | NVLink 3 · 600 GB/s |
| Cheapest rental | $1.90/hr | $0.87/hr |
| $/hr per GB VRAM | $2.4¢ | $1.1¢ |
Rule of thumb: for LLM serving, prefer the GPU with more memory bandwidth per dollar; for training and prefill-heavy work, prefer FLOPS per dollar; and if the model doesn't fit in VRAM, none of the other numbers matter. Sanity-check with the VRAM calculator.