Menu
mutinai
What can I run?
Public preview · figures are illustrative fixture data, attributed to their source · data to May 2025 · read-only

RTX 4060 Ti 16GB budget build (32 GB DDR5)

What can I run? · 8K context
Change hardwareCurrently: RTX 4060 Ti 16GB budget build (32 GB DDR5)

Laptops

MacBook Pro M3 Max 64 GBMacBook Pro M4 Max 128 GB

Mini PCs

GB10 mini workstationMac mini M4 Pro 48 GBRyzen AI Max+ 395 mini PC

Desktops

CPU-only Ryzen 9 7950XDual RTX 3090Mac Studio M2 Ultra 192 GBRTX 4060 Ti 16GB budget buildRTX 4090 workstationRTX 5090 workstationRX 7900 XTX desktop

Servers

A100 80GB server
Build your own setupChoose GPUs or a chip and how much memory you have
8run well
9with offload
2too large

Speeds marked measured come from reference results and verified community runs on this exact system; others are estimates.

Runs in accelerator memory 8

Fits entirely in GPU or unified memory — the fast path.

ModelRecommended downloadMemoryGeneration speed
Variant Phi-4Microsoft · 14.7B
1 other option
Q8_0 · LM Studio Communityllama.cppWith offload17.1 GiB~9 tok/s est.
Q4_K_M via llama.cpp8.3 GiB · LM Studio Community
10.7 of 15.2 GiB
~20 tok/s est.
Variant Gemma 2 9B ITGoogle DeepMind · 9.24BQ4_K_M via llama.cpp5.3 GiB · LM Studio Community
8.6 of 15.2 GiB
~30 tok/s est.
Variant Hermes 3 Llama 3.1 8BMeta · 8.03B
2 other options
Q4_K_M · NousResearchllama.cppRuns well6.3 GiB~34 tok/s est.
Q6_K · NousResearchllama.cppRuns well7.9 GiB~26 tok/s est.
Q8_0 via llama.cpp8.0 GiB · NousResearch
9.8 of 15.2 GiB
~21 tok/s est.
Variant Llama 3.1 8B InstructMeta · 8.03B
2 other options
Q4_K_M · LM Studio Communityllama.cppRuns well6.3 GiB~34 tok/s est.
FP8 · RedHatAISGLangRuns well10.3 GiB~20 tok/s est.
Q8_0 via llama.cpp7.9 GiB · LM Studio Community
9.8 of 15.2 GiB
~21 tok/s est.
Variant Qwen2.5 7B InstructQwen Team (Alibaba Cloud) · 7.62B
2 other options
AWQ 4-bit · Qwen Team (Alibaba Cloud)SGLangRuns well5.2 GiB~38 tok/s est.
Q4_K_M · Qwen Team (Alibaba Cloud)llama.cppRuns well5.5 GiB~36 tok/s est.
Q8_0 via llama.cpp7.5 GiB · Qwen Team (Alibaba Cloud)
8.8 of 15.2 GiB
~22 tok/s est.
Variant DeepSeek-R1-Distill-Qwen-7BQwen Team (Alibaba Cloud) · 7.62BQ4_K_M via llama.cpp4.3 GiB · LM Studio Community
5.2 of 15.2 GiB
~36 tok/s est.
Variant Mistral 7B Instruct v0.3Mistral AI · 7.25B
1 other option
Q4_K_M · LM Studio Communityllama.cppRuns well5.8 GiB~38 tok/s est.
Q8_0 via llama.cpp7.2 GiB · LM Studio Community
9.0 of 15.2 GiB
~23 tok/s est.
Variant Qwen2.5 14B InstructQwen Team (Alibaba Cloud) · 14.8B
2 other options
Q4_K_M · Qwen Team (Alibaba Cloud)llama.cppRuns well10.7 GiB25.5 tok/s measured
AWQ 4-bit · Qwen Team (Alibaba Cloud)SGLangRuns well10.2 GiB~21 tok/s est.
Q6_K via llama.cpp11.3 GiB · LM Studio CommunityTight fit
13.7 of 15.2 GiB
~15 tok/s est.

Runs with partial offload to system RAM 9

Too big for accelerator memory alone; part of it runs from system RAM, which is much slower.

ModelRecommended downloadMemoryGeneration speed
Variant Llama 3.3 70B InstructMeta · 70.6BQ3_K_M via llama.cpp32.1 GiB · Unsloth AI
36.4 of 15.2 GiB · 21.2 in RAM
~2 tok/s est.
Variant Mixtral 8x7B Instruct v0.1Mistral AI · 46.7B (12.9B active)Q4_K_M via llama.cpp26.6 GiB · LM Studio Community
29.1 of 15.2 GiB · 13.9 in RAM
~11 tok/s est.
Variant Qwen2.5 32B InstructQwen Team (Alibaba Cloud) · 32.8B
1 other option
Q5_K_M · LM Studio Communityllama.cppWith offload25.1 GiB~4 tok/s est.
Q4_K_M via llama.cpp18.7 GiB · Qwen Team (Alibaba Cloud)
21.9 of 15.2 GiB · 6.7 in RAM
~5 tok/s est.
Variant Qwen2.5-Coder 32B InstructQwen Team (Alibaba Cloud) · 32.8B
1 other option
Q8_0 · LM Studio Communityllama.cppWith offload36.2 GiB~2 tok/s est.
Q4_K_M via llama.cpp18.7 GiB · Qwen Team (Alibaba Cloud)
21.9 of 15.2 GiB · 6.7 in RAM
~5 tok/s est.
Variant DeepSeek-R1-Distill-Qwen-32BQwen Team (Alibaba Cloud) · 32.8BQ4_K_M via llama.cpp18.7 GiB · LM Studio Community
21.9 of 15.2 GiB · 6.7 in RAM
~5 tok/s est.
Variant Qwen3 30B-A3BQwen Team (Alibaba Cloud) · 30.5B (3.3B active)
3 other options
Q4_K_M · Unsloth AIllama.cppWith offload19.3 GiB~50 tok/s est.
Q8_0 · LM Studio Communityllama.cppWith offload32.7 GiB~22 tok/s est.
Q8_0 · Qwen Team (Alibaba Cloud)llama.cppWith offload32.7 GiB~21 tok/s est.
Q4_K_M via llama.cpp17.3 GiB · Qwen Team (Alibaba Cloud)
19.2 of 15.2 GiB · 4.0 in RAM
~50 tok/s est.
Variant Gemma 3 27B ITGoogle DeepMind · 27.4BQ4_K_M via llama.cpp15.6 GiB · LM Studio Community
20.6 of 15.2 GiB · 5.4 in RAM
~6 tok/s est.
Variant Gemma 2 27B ITGoogle DeepMind · 27.2BQ4_K_M via llama.cpp15.5 GiB · LM Studio Community
19.5 of 15.2 GiB · 4.3 in RAM
~7 tok/s est.
Variant Mistral Small 24B Instruct 2501Mistral AI · 23.6B
1 other option
Q6_K · LM Studio Communityllama.cppWith offload20.5 GiB~6 tok/s est.
Q4_K_M via llama.cpp13.4 GiB · LM Studio Community
15.7 of 15.2 GiB · 0.5 in RAM
~12 tok/s est.
Too large for this hardware2 model variants
How this is calculatedMemory, fit and speed estimates

Memory = download size + fp16 KV cache for 8K tokens + runtime overhead. Usable memory is 95% of dedicated VRAM, a device-specific share of unified memory (75% by default), and 80% of system RAM for runtimes that can offload. Runtimes must load the file format and support a backend present on the hardware.

Estimated speed is bounded by memory bandwidth ÷ bytes read per token (active parameters for mixture-of-experts), shown in italics with est. and a ±35% range. Measured speeds are medians of reference results and verified public community runs on the same system, download and runtime. For each model we recommend the highest-precision download that fits, preferring not to offload.