H100 SXM vs Gaudi 3
Gaudi 3 has 1.6× the memory; H100 SXM rents for 1.2× less.
The NVIDIA H100 SXM (Hopper, 2022) and the Intel Gaudi 3 (Gaudi, 2024) come from different vendors and different design goals, but they get cross-shopped for a reason. On capacity, the Gaudi 3 carries 1.6x the memory (128 vs 80 GB), which sets what each can hold at all. Bandwidth is nearly even (3.35 vs 3.7 TB/s), so serving speed per GPU will be similar.
Price is where it settles: the H100 SXM rents from $1.90/hr against $2.30/hr for the Gaudi 3, a 1.2x gap that the performance numbers above only partly close. Per gigabyte of VRAM, the Gaudi 3 is the cheaper rental, worth knowing if your model is capacity-bound rather than speed-bound. The full case for and against Intel's part is in Gaudi 3: the accelerator nobody is fighting over. Everything below is the underlying data.
NVIDIA · Hopper · 2022
NVIDIA H100 SXM
The workhorse of the AI boom. Still the most widely available serious training and inference GPU.
From $1.90/hr at Hyperstack
Intel · Gaudi · 2024
Intel Gaudi 3
Intel's price-performance play, with Ethernet-native scale-out instead of proprietary interconnect.
From $2.30/hr at FluidStack
Head to head
■ H100 SXM ■ Gaudi 3, bars share one scale across the whole catalog.
| H100 SXM | Gaudi 3 | |
|---|---|---|
| Memory | 80 GB HBM3 | 128 GB HBM2e |
| Bandwidth | 3.35 TB/s | 3.7 TB/s |
| FP16 dense | 990 TF | 918 TF |
| FP8 dense | 1,979 TF | 1,835 TF |
| TDP | 700 W | 900 W |
| Interconnect | NVLink 4 · 900 GB/s | 24×200GbE RoCE on-chip |
| Cheapest rental | $1.90/hr | $2.30/hr |
| $/hr per GB VRAM | $2.4¢ | $1.8¢ |
What one can run that the other can't
Only on H100 SXM
Nothing, Gaudi 3 runs everything H100 SXM does.
Only on Gaudi 3
- Mixtral 8x22B (4-bit)
- gpt-oss-120b (4-bit)
Rule of thumb: for LLM serving, prefer the GPU with more memory bandwidth per dollar; for training and prefill-heavy work, prefer FLOPS per dollar; and if the model doesn't fit in VRAM, none of the other numbers matter. Sanity-check with the VRAM calculator.