RTX 5090 workstation (96 GB DDR5)

What can I run? · 8K context

Change hardwareCurrently: RTX 5090 workstation (96 GB DDR5)

Build your own setupChoose GPUs or a chip and how much memory you have

Main deviceHow many

Second device (optional)How many

System RAM (GB)Unified memory (GB, Mac/APU)

16run well

2with offload

1too large

Usable memory~31 GB

Speeds marked measured come from reference results and verified community runs on this exact system; others are estimates.

Runs in accelerator memory 16

Fits entirely in GPU or unified memory — the fast path.

Model

Recommended download

Memory

Generation speed

Qwen2.5 32B InstructQwen Team (Alibaba Cloud) · 32.8B

2 other options

Q4_K_M · Qwen Team (Alibaba Cloud)	llama.cpp	Runs well	21.9 GiB	61 tok/s measured
AWQ 4-bit · Qwen Team (Alibaba Cloud)	SGLang	Runs well	20.8 GiB	~60 tok/s est.

Q5_K_M via llama.cpp21.7 GiB · LM Studio Community

25.1 of 30.4 GiB

~49 tok/s est.

Qwen2.5-Coder 32B InstructQwen Team (Alibaba Cloud) · 32.8B

1 other option

Q8_0 · LM Studio Community	llama.cpp	With offload	36.2 GiB	~8 tok/s est.

Q4_K_M via llama.cpp18.7 GiB · Qwen Team (Alibaba Cloud)

21.9 of 30.4 GiB

~57 tok/s est.

DeepSeek-R1-Distill-Qwen-32BQwen Team (Alibaba Cloud) · 32.8B

Q4_K_M via llama.cpp18.7 GiB · LM Studio Community

21.9 of 30.4 GiB

~57 tok/s est.

Qwen3 30B-A3BQwen Team (Alibaba Cloud) · 30.5B (3.3B active)

3 other options

Q4_K_M · Unsloth AI	llama.cpp	Runs well	19.3 GiB	~456 tok/s est.
Q8_0 · LM Studio Community	llama.cpp	With offload	32.7 GiB	~124 tok/s est.
Q8_0 · Qwen Team (Alibaba Cloud)	llama.cpp	With offload	32.7 GiB	~122 tok/s est.

Q4_K_M via llama.cpp17.3 GiB · Qwen Team (Alibaba Cloud)

19.2 of 30.4 GiB

~458 tok/s est.

Gemma 3 27B ITGoogle DeepMind · 27.4B

Q4_K_M via llama.cpp15.6 GiB · LM Studio Community

20.6 of 30.4 GiB

~67 tok/s est.

Gemma 2 27B ITGoogle DeepMind · 27.2B

Q4_K_M via llama.cpp15.5 GiB · LM Studio Community

19.5 of 30.4 GiB

~68 tok/s est.

Mistral Small 24B Instruct 2501Mistral AI · 23.6B

1 other option

Q4_K_M · LM Studio Community	llama.cpp	Runs well	15.7 GiB	~78 tok/s est.

Q6_K via llama.cpp18.0 GiB · LM Studio Community

20.5 of 30.4 GiB

~59 tok/s est.

Qwen2.5 14B InstructQwen Team (Alibaba Cloud) · 14.8B

2 other options

AWQ 4-bit · Qwen Team (Alibaba Cloud)	SGLang	Runs well	10.2 GiB	~129 tok/s est.
Q4_K_M · Qwen Team (Alibaba Cloud)	llama.cpp	Runs well	10.7 GiB	~122 tok/s est.

Q6_K via llama.cpp11.3 GiB · LM Studio Community

13.7 of 30.4 GiB

~92 tok/s est.

Phi-4Microsoft · 14.7B

1 other option

Q4_K_M · LM Studio Community	llama.cpp	Runs well	10.7 GiB	~123 tok/s est.

Q8_0 via llama.cpp14.5 GiB · LM Studio Community

17.1 of 30.4 GiB

~72 tok/s est.

Gemma 2 9B ITGoogle DeepMind · 9.24B

1 other option

Q4_K_M · LM Studio Community	llama.cpp	Runs well	8.6 GiB	~188 tok/s est.

BF16 via SGLang17.2 GiB · Google DeepMind

21.0 of 30.4 GiB

~61 tok/s est.

Hermes 3 Llama 3.1 8BMeta · 8.03B

2 other options

Q4_K_M · NousResearch	llama.cpp	Runs well	6.3 GiB	~213 tok/s est.
Q6_K · NousResearch	llama.cpp	Runs well	7.9 GiB	~163 tok/s est.

Q8_0 via llama.cpp8.0 GiB · NousResearch

9.8 of 30.4 GiB

~128 tok/s est.

Llama 3.1 8B InstructMeta · 8.03B

3 other options

Q4_K_M · LM Studio Community	llama.cpp	Runs well	6.3 GiB	~214 tok/s est.
FP8 · RedHatAI	SGLang	Runs well	10.3 GiB	~121 tok/s est.
BF16 · Meta	SGLang	Runs well	17.1 GiB	~70 tok/s est.

Q8_0 via llama.cpp7.9 GiB · LM Studio Community

9.8 of 30.4 GiB

~128 tok/s est.

Qwen2.5 7B InstructQwen Team (Alibaba Cloud) · 7.62B

3 other options

AWQ 4-bit · Qwen Team (Alibaba Cloud)	SGLang	Runs well	5.2 GiB	~237 tok/s est.
Q4_K_M · Qwen Team (Alibaba Cloud)	llama.cpp	Runs well	5.5 GiB	~224 tok/s est.
BF16 · Qwen Team (Alibaba Cloud)	SGLang	Runs well	15.7 GiB	~74 tok/s est.

Q8_0 via llama.cpp7.5 GiB · Qwen Team (Alibaba Cloud)

8.8 of 30.4 GiB

~135 tok/s est.

DeepSeek-R1-Distill-Qwen-7BQwen Team (Alibaba Cloud) · 7.62B

1 other option

Q4_K_M · LM Studio Community	llama.cpp	Runs well	5.2 GiB	~224 tok/s est.

BF16 via SGLang14.2 GiB · DeepSeek

15.5 of 30.4 GiB

~74 tok/s est.

Mistral 7B Instruct v0.3Mistral AI · 7.25B

2 other options

Q4_K_M · LM Studio Community	llama.cpp	Runs well	5.8 GiB	~235 tok/s est.
BF16 · Mistral AI	SGLang	Runs well	15.5 GiB	~78 tok/s est.

Q8_0 via llama.cpp7.2 GiB · LM Studio Community

9.0 of 30.4 GiB

~141 tok/s est.

Mixtral 8x7B Instruct v0.1Mistral AI · 46.7B (12.9B active)

Q4_K_M via llama.cpp26.6 GiB · LM Studio CommunityTight fit

29.1 of 30.4 GiB

~138 tok/s est.

Runs with partial offload to system RAM 2

Too big for accelerator memory alone; part of it runs from system RAM, which is much slower.

Model

Recommended download

Memory

Generation speed

Llama 3.1 70B InstructMeta · 70.6B

Q4_K_M via llama.cpp40.2 GiB · LM Studio Community

44.8 of 30.4 GiB · 14.4 in RAM

~4 tok/s est.

Llama 3.3 70B InstructMeta · 70.6B

1 other option

Q4_K_M · LM Studio Community	llama.cpp	With offload	44.8 GiB	~4 tok/s est.

Q3_K_M via llama.cpp32.1 GiB · Unsloth AI

36.4 of 30.4 GiB · 6.0 in RAM

~8 tok/s est.

Too large for this hardware1 model variant

DeepSeek-R1 671B · 3 downloads checked

How this is calculatedMemory, fit and speed estimates

Memory = download size + fp16 KV cache for 8K tokens + runtime overhead. Usable memory is 95% of dedicated VRAM, a device-specific share of unified memory (75% by default), and 80% of system RAM for runtimes that can offload. Runtimes must load the file format and support a backend present on the hardware.

Estimated speed is bounded by memory bandwidth ÷ bytes read per token (active parameters for mixture-of-experts), shown in italics with est. and a ±35% range. Measured speeds are medians of reference results and verified public community runs on the same system, download and runtime. For each model we recommend the highest-precision download that fits, preferring not to offload.

RTX 5090 workstation (96 GB DDR5)

Laptops

Mini PCs

Desktops

Servers

Runs in accelerator memory 16

Runs with partial offload to system RAM 2