Dual RTX 3090 (128 GB DDR5)

What can I run? · 8K context

Change hardwareCurrently: Dual RTX 3090 (128 GB DDR5)

Build your own setupChoose GPUs or a chip and how much memory you have

Main deviceHow many

Second device (optional)How many

System RAM (GB)Unified memory (GB, Mac/APU)

18run well

0with offload

1too large

Usable memory~46 GB

Speeds marked measured come from reference results and verified community runs on this exact system; others are estimates.

Runs in accelerator memory 18

Fits entirely in GPU or unified memory — the fast path.

Model

Recommended download

Memory

Generation speed

Mixtral 8x7B Instruct v0.1Mistral AI · 46.7B (12.9B active)

Q4_K_M via llama.cpp26.6 GiB · LM Studio Community

29.1 of 45.6 GiB

~69 tok/s est.

Qwen2.5 32B InstructQwen Team (Alibaba Cloud) · 32.8B

2 other options

AWQ 4-bit · Qwen Team (Alibaba Cloud)	SGLang	Runs well	20.8 GiB	~30 tok/s est.
Q4_K_M · Qwen Team (Alibaba Cloud)	llama.cpp	Runs well	21.9 GiB	~28 tok/s est.

Q5_K_M via llama.cpp21.7 GiB · LM Studio Community

25.1 of 45.6 GiB

~24 tok/s est.

Qwen2.5-Coder 32B InstructQwen Team (Alibaba Cloud) · 32.8B

1 other option

Q4_K_M · Qwen Team (Alibaba Cloud)	llama.cpp	Runs well	21.9 GiB	~28 tok/s est.

Q8_0 via llama.cpp32.4 GiB · LM Studio Community

36.2 of 45.6 GiB

~16 tok/s est.

DeepSeek-R1-Distill-Qwen-32BQwen Team (Alibaba Cloud) · 32.8B

Q4_K_M via llama.cpp18.7 GiB · LM Studio Community

21.9 of 45.6 GiB

~28 tok/s est.

Qwen3 30B-A3BQwen Team (Alibaba Cloud) · 30.5B (3.3B active)

4 other options

Q4_K_M · Qwen Team (Alibaba Cloud)	llama.cpp	Runs well	19.2 GiB	~227 tok/s est.
Q4_K_M · Unsloth AI	llama.cpp	Runs well	19.3 GiB	~226 tok/s est.
FP8 · Qwen Team (Alibaba Cloud)	SGLang	Runs well	31.2 GiB	~149 tok/s est.
Q8_0 · Qwen Team (Alibaba Cloud)	llama.cpp	Runs well	32.7 GiB	~143 tok/s est.

Q8_0 via llama.cpp30.2 GiB · LM Studio Community

32.7 of 45.6 GiB

~143 tok/s est.

Gemma 3 27B ITGoogle DeepMind · 27.4B

Q4_K_M via llama.cpp15.6 GiB · LM Studio Community

20.6 of 45.6 GiB

~33 tok/s est.

Gemma 2 27B ITGoogle DeepMind · 27.2B

Q4_K_M via llama.cpp15.5 GiB · LM Studio Community

19.5 of 45.6 GiB

~34 tok/s est.

Mistral Small 24B Instruct 2501Mistral AI · 23.6B

1 other option

Q4_K_M · LM Studio Community	llama.cpp	Runs well	15.7 GiB	~39 tok/s est.

Q6_K via llama.cpp18.0 GiB · LM Studio Community

20.5 of 45.6 GiB

~29 tok/s est.

Qwen2.5 14B InstructQwen Team (Alibaba Cloud) · 14.8B

3 other options

AWQ 4-bit · Qwen Team (Alibaba Cloud)	SGLang	Runs well	10.2 GiB	~64 tok/s est.
Q4_K_M · Qwen Team (Alibaba Cloud)	llama.cpp	Runs well	10.7 GiB	~60 tok/s est.
Q6_K · LM Studio Community	llama.cpp	Runs well	13.7 GiB	~46 tok/s est.

BF16 via SGLang27.5 GiB · Qwen Team (Alibaba Cloud)

30.6 of 45.6 GiB

~19 tok/s est.

Phi-4Microsoft · 14.7B

2 other options

Q4_K_M · LM Studio Community	llama.cpp	Runs well	10.7 GiB	~61 tok/s est.
BF16 · Microsoft	SGLang	Runs well	30.5 GiB	~19 tok/s est.

Q8_0 via llama.cpp14.5 GiB · LM Studio Community

17.1 of 45.6 GiB

~36 tok/s est.

Gemma 2 9B ITGoogle DeepMind · 9.24B

1 other option

Q4_K_M · LM Studio Community	llama.cpp	Runs well	8.6 GiB	~93 tok/s est.

BF16 via SGLang17.2 GiB · Google DeepMind

21.0 of 45.6 GiB

~30 tok/s est.

Hermes 3 Llama 3.1 8BMeta · 8.03B

2 other options

Q4_K_M · NousResearch	llama.cpp	Runs well	6.3 GiB	~106 tok/s est.
Q6_K · NousResearch	llama.cpp	Runs well	7.9 GiB	~81 tok/s est.

Q8_0 via llama.cpp8.0 GiB · NousResearch

9.8 of 45.6 GiB

~64 tok/s est.

Llama 3.1 8B InstructMeta · 8.03B

3 other options

Q4_K_M · LM Studio Community	llama.cpp	Runs well	6.3 GiB	~106 tok/s est.
FP8 · RedHatAI	SGLang	Runs well	10.3 GiB	~60 tok/s est.
BF16 · Meta	SGLang	Runs well	17.1 GiB	~35 tok/s est.

Q8_0 via llama.cpp7.9 GiB · LM Studio Community

9.8 of 45.6 GiB

~64 tok/s est.

Qwen2.5 7B InstructQwen Team (Alibaba Cloud) · 7.62B

3 other options

AWQ 4-bit · Qwen Team (Alibaba Cloud)	SGLang	Runs well	5.2 GiB	~118 tok/s est.
Q4_K_M · Qwen Team (Alibaba Cloud)	llama.cpp	Runs well	5.5 GiB	~111 tok/s est.
BF16 · Qwen Team (Alibaba Cloud)	SGLang	Runs well	15.7 GiB	~37 tok/s est.

Q8_0 via llama.cpp7.5 GiB · Qwen Team (Alibaba Cloud)

8.8 of 45.6 GiB

~67 tok/s est.

DeepSeek-R1-Distill-Qwen-7BQwen Team (Alibaba Cloud) · 7.62B

1 other option

Q4_K_M · LM Studio Community	llama.cpp	Runs well	5.2 GiB	~111 tok/s est.

BF16 via SGLang14.2 GiB · DeepSeek

15.5 of 45.6 GiB

~37 tok/s est.

Mistral 7B Instruct v0.3Mistral AI · 7.25B

2 other options

Q4_K_M · LM Studio Community	llama.cpp	Runs well	5.8 GiB	~116 tok/s est.
BF16 · Mistral AI	SGLang	Runs well	15.5 GiB	~38 tok/s est.

Q8_0 via llama.cpp7.2 GiB · LM Studio Community

9.0 of 45.6 GiB

~70 tok/s est.

Llama 3.1 70B InstructMeta · 70.6B

Q4_K_M via llama.cpp40.2 GiB · LM Studio CommunityTight fit

44.8 of 45.6 GiB

~13 tok/s est.

Llama 3.3 70B InstructMeta · 70.6B

2 other options

Q3_K_M · Unsloth AI	llama.cpp	Runs well	36.4 GiB	~17 tok/s est.
AWQ 4-bit · Unsloth AI	SGLang	Tight fit	42.3 GiB	~14 tok/s est.

Q4_K_M via llama.cpp40.2 GiB · LM Studio CommunityTight fit

44.8 of 45.6 GiB

17.6 tok/s measured

Too large for this hardware1 model variant

DeepSeek-R1 671B · 3 downloads checked

How this is calculatedMemory, fit and speed estimates

Memory = download size + fp16 KV cache for 8K tokens + runtime overhead. Usable memory is 95% of dedicated VRAM, a device-specific share of unified memory (75% by default), and 80% of system RAM for runtimes that can offload. Runtimes must load the file format and support a backend present on the hardware.

Estimated speed is bounded by memory bandwidth ÷ bytes read per token (active parameters for mixture-of-experts), shown in italics with est. and a ±35% range. Measured speeds are medians of reference results and verified public community runs on the same system, download and runtime. For each model we recommend the highest-precision download that fits, preferring not to offload.

Dual RTX 3090 (128 GB DDR5)

Laptops

Mini PCs

Desktops

Servers

Runs in accelerator memory 18