RTX 3090 vs RTX 4090

The NVIDIA RTX 3090 (Ampere, 2020) and the NVIDIA RTX 4090 (Ada Lovelace, 2022) are 2 years and a silicon generation apart, which makes the price gap the heart of the story. Memory is effectively a wash at 24 vs 24 GB, so capacity does not decide this one. Bandwidth is nearly even (0.936 vs 1.01 TB/s), so serving speed per GPU will be similar. Raw compute favors the RTX 4090 by 2.3x, which matters for training, prefill-heavy traffic, and diffusion work more than for chat-style decoding. The RTX 4090 also speaks native FP8 while the RTX 3090 tops out at BF16/INT8, a generational gap explained in CUDA cores vs Tensor Cores.

Price is where it settles: the RTX 3090 rents from $0.18/hr against $0.34/hr for the RTX 4090, a 1.9x gap that the performance numbers above only partly close. Per gigabyte of VRAM, the RTX 3090 is the cheaper rental, worth knowing if your model is capacity-bound rather than speed-bound. Bear in mind the RTX 3090 is a consumer card rented mostly through marketplaces, with the trust and reliability trade-offs that implies, while the RTX 4090 is datacenter silicon with ECC HBM and standard hosting. For how these numbers translate into tokens per dollar, the inference cost estimator runs both cards against any model. Everything below is the underlying data.