Hardware · 19 devices · 19 reference systems
Buy for the models you want to run.
Memory decides which models fit, so hardware is grouped the same way as the models page: a machine in a group runs every model in that group.
Server & multi-GPU1 device · 2 systems
Usable memory for a model: graphics cards keep about 95% of their memory for it, unified-memory Macs about 75%.
A 32 GB graphics card
22–30 GBRuns every model that needs up to 30 GB.
- 48 GBVRAM~73B estimateparameters~$3,500build estimateDual RTX 3090SystemMulti-GPU build that splits 70B-class models across cards.Desktop · 2× RTX 3090 + Ryzen 9 7950X · 1 measurement · What can it run?
A 64 GB Mac or two graphics cards
30–46 GBRuns every model that needs up to 46 GB.
- Datacenter card for large models and many users.NVIDIA · Datacenter GPU · 2020 · 1 measurement80 GBVRAM~120B estimateparametersNo launch price
- 80 GBVRAM~120B estimateparametersNo estimateA100 80GB serverSystemHolds 70B-class and larger models in memory.Server · A100 80GB + EPYC 7763 · 1 measurement · What can it run?
Memory bandwidth, usable-memory scales, price per GB and side-by-side scanning are in the Technical view. Prices are launch MSRPs or editorial estimates, not current prices.