Menu
mutinai
What can I run?
Public preview · figures are illustrative fixture data, attributed to their source · data to May 2025 · read-only

RTX 4090 workstation (64 GB DDR5)

What can I run? · 8K context
Change hardwareCurrently: RTX 4090 workstation (64 GB DDR5)

Laptops

MacBook Pro M3 Max 64 GBMacBook Pro M4 Max 128 GB

Mini PCs

GB10 mini workstationMac mini M4 Pro 48 GBRyzen AI Max+ 395 mini PC

Desktops

CPU-only Ryzen 9 7950XDual RTX 3090Mac Studio M2 Ultra 192 GBRTX 4060 Ti 16GB budget buildRTX 4090 workstationRTX 5090 workstationRX 7900 XTX desktop

Servers

A100 80GB server
Build your own setupChoose GPUs or a chip and how much memory you have
15run well
3with offload
1too large

Speeds marked measured come from reference results and verified community runs on this exact system; others are estimates.

Runs in accelerator memory 15

Fits entirely in GPU or unified memory — the fast path.

ModelRecommended downloadMemoryGeneration speed
Variant Qwen3 30B-A3BQwen Team (Alibaba Cloud) · 30.5B (3.3B active)
3 other options
Q4_K_M · Unsloth AIOllamaRuns well19.3 GiB~257 tok/s est.
Q8_0 · LM Studio Communityllama.cppWith offload32.7 GiB~40 tok/s est.
Q8_0 · Qwen Team (Alibaba Cloud)llama.cppWith offload32.7 GiB~39 tok/s est.
Q4_K_M via llama.cpp17.3 GiB · Qwen Team (Alibaba Cloud)
19.2 of 22.8 GiB
~258 tok/s est.
Variant Gemma 2 27B ITGoogle DeepMind · 27.2BQ4_K_M via llama.cpp15.5 GiB · LM Studio Community
19.5 of 22.8 GiB
~38 tok/s est.
Variant Mistral Small 24B Instruct 2501Mistral AI · 23.6B
1 other option
Q4_K_M · LM Studio Communityllama.cppRuns well15.7 GiB~44 tok/s est.
Q6_K via llama.cpp18.0 GiB · LM Studio Community
20.5 of 22.8 GiB
~33 tok/s est.
Variant Qwen2.5 14B InstructQwen Team (Alibaba Cloud) · 14.8B
2 other options
AWQ 4-bit · Qwen Team (Alibaba Cloud)SGLangRuns well10.2 GiB~73 tok/s est.
Q4_K_M · Qwen Team (Alibaba Cloud)llama.cppRuns well10.7 GiB~69 tok/s est.
Q6_K via llama.cpp11.3 GiB · LM Studio Community
13.7 of 22.8 GiB
~52 tok/s est.
Variant Phi-4Microsoft · 14.7B
1 other option
Q4_K_M · LM Studio Communityllama.cppRuns well10.7 GiB~69 tok/s est.
Q8_0 via llama.cpp14.5 GiB · LM Studio Community
17.1 of 22.8 GiB
~41 tok/s est.
Variant Hermes 3 Llama 3.1 8BMeta · 8.03B
2 other options
Q4_K_M · NousResearchllama.cppRuns well6.3 GiB~120 tok/s est.
Q6_K · NousResearchllama.cppRuns well7.9 GiB~92 tok/s est.
Q8_0 via llama.cpp8.0 GiB · NousResearch
9.8 of 22.8 GiB
~72 tok/s est.
Variant Llama 3.1 8B InstructMeta · 8.03B
3 other options
Q4_K_M · LM Studio Communityllama.cppRuns well6.3 GiB128 tok/s measured
FP8 · RedHatAISGLangRuns well10.3 GiB~68 tok/s est.
BF16 · MetaSGLangRuns well17.1 GiB~40 tok/s est.
Q8_0 via llama.cpp7.9 GiB · LM Studio Community
9.8 of 22.8 GiB
~72 tok/s est.
Variant Qwen2.5 7B InstructQwen Team (Alibaba Cloud) · 7.62B
3 other options
AWQ 4-bit · Qwen Team (Alibaba Cloud)SGLangRuns well5.2 GiB~133 tok/s est.
Q4_K_M · Qwen Team (Alibaba Cloud)llama.cppRuns well5.5 GiB~126 tok/s est.
BF16 · Qwen Team (Alibaba Cloud)SGLangRuns well15.7 GiB~42 tok/s est.
Q8_0 via llama.cpp7.5 GiB · Qwen Team (Alibaba Cloud)
8.8 of 22.8 GiB
~76 tok/s est.
Variant DeepSeek-R1-Distill-Qwen-7BQwen Team (Alibaba Cloud) · 7.62B
1 other option
Q4_K_M · LM Studio Communityllama.cppRuns well5.2 GiB~126 tok/s est.
BF16 via SGLang14.2 GiB · DeepSeek
15.5 of 22.8 GiB
~42 tok/s est.
Variant Mistral 7B Instruct v0.3Mistral AI · 7.25B
2 other options
Q4_K_M · LM Studio Communityllama.cppRuns well5.8 GiB~132 tok/s est.
BF16 · Mistral AISGLangRuns well15.5 GiB~44 tok/s est.
Q8_0 via llama.cpp7.2 GiB · LM Studio Community
9.0 of 22.8 GiB
~80 tok/s est.
Variant Qwen2.5 32B InstructQwen Team (Alibaba Cloud) · 32.8B
2 other options
AWQ 4-bit · Qwen Team (Alibaba Cloud)SGLangTight fit20.8 GiB~34 tok/s est.
Q5_K_M · LM Studio Communityllama.cppWith offload25.1 GiB~14 tok/s est.
Q4_K_M via llama.cpp18.7 GiB · Qwen Team (Alibaba Cloud)Tight fit
21.9 of 22.8 GiB
~32 tok/s est.
Variant Qwen2.5-Coder 32B InstructQwen Team (Alibaba Cloud) · 32.8B
1 other option
Q8_0 · LM Studio Communityllama.cppWith offload36.2 GiB~4 tok/s est.
Q4_K_M via llama.cpp18.7 GiB · Qwen Team (Alibaba Cloud)Tight fit
21.9 of 22.8 GiB
~32 tok/s est.
Variant DeepSeek-R1-Distill-Qwen-32BQwen Team (Alibaba Cloud) · 32.8BQ4_K_M via llama.cpp18.7 GiB · LM Studio CommunityTight fit
21.9 of 22.8 GiB
~32 tok/s est.
Variant Gemma 3 27B ITGoogle DeepMind · 27.4BQ4_K_M via llama.cpp15.6 GiB · LM Studio CommunityTight fit
20.6 of 22.8 GiB
~38 tok/s est.
Variant Gemma 2 9B ITGoogle DeepMind · 9.24B
1 other option
Q4_K_M · LM Studio Communityllama.cppRuns well8.6 GiB~106 tok/s est.
BF16 via SGLang17.2 GiB · Google DeepMindTight fit
21.0 of 22.8 GiB
~34 tok/s est.

Runs with partial offload to system RAM 3

Too big for accelerator memory alone; part of it runs from system RAM, which is much slower.

ModelRecommended downloadMemoryGeneration speed
Variant Llama 3.1 70B InstructMeta · 70.6BQ4_K_M via llama.cpp40.2 GiB · LM Studio Community
44.8 of 22.8 GiB · 22.0 in RAM
~2 tok/s est.
Variant Llama 3.3 70B InstructMeta · 70.6B
1 other option
Q4_K_M · LM Studio Communityllama.cppWith offload44.8 GiB~2 tok/s est.
Q3_K_M via llama.cpp32.1 GiB · Unsloth AI
36.4 of 22.8 GiB · 13.6 in RAM
~4 tok/s est.
Variant Mixtral 8x7B Instruct v0.1Mistral AI · 46.7B (12.9B active)Q4_K_M via llama.cpp26.6 GiB · LM Studio Community
29.1 of 22.8 GiB · 6.3 in RAM
~24 tok/s est.
Too large for this hardware1 model variant
How this is calculatedMemory, fit and speed estimates

Memory = download size + fp16 KV cache for 8K tokens + runtime overhead. Usable memory is 95% of dedicated VRAM, a device-specific share of unified memory (75% by default), and 80% of system RAM for runtimes that can offload. Runtimes must load the file format and support a backend present on the hardware.

Estimated speed is bounded by memory bandwidth ÷ bytes read per token (active parameters for mixture-of-experts), shown in italics with est. and a ±35% range. Measured speeds are medians of reference results and verified public community runs on the same system, download and runtime. For each model we recommend the highest-precision download that fits, preferring not to offload.