System · desktop

RTX 4090 workstation (64 GB DDR5)

Runs models up to ~35B parameters at 4-bit entirely on the GPU. Single 24 GB GPU desktop.

GPU memory: 24 GB
System RAM: 64 GB
Runs: 15 of 19model variants, 8K
Estimated cost: ~$3,200

What it runs

Largest models that fit without spilling into system memory, at an 8K contextHow much text the model can consider at once, counted in tokens — roughly ¾ of a word each..

Everything →

Variant
Qwen2.5 32B Instruct
32.8B · Q4_K_M via llama.cpp
Tight fit
~32 tok/s est.
Variant
Qwen2.5-Coder 32B Instruct
32.8B · Q4_K_M via llama.cpp
Tight fit
~32 tok/s est.
Variant
DeepSeek-R1-Distill-Qwen-32B
32.8B · Q4_K_M via llama.cpp
Tight fit
~32 tok/s est.
Variant
Qwen3 30B-A3B
30.5B · Q4_K_M via llama.cpp
Runs well
~258 tok/s est.
Variant
Gemma 3 27B IT
27.4B · Q4_K_M via llama.cpp
Tight fit
~38 tok/s est.
Variant
Gemma 2 27B IT
27.2B · Q4_K_M via llama.cpp
Runs well
~38 tok/s est.
Variant
Mistral Small 24B Instruct 2501
23.6B · Q6_K via llama.cpp
Runs well
~33 tok/s est.
Variant
Qwen2.5 14B Instruct
14.8B · Q6_K via llama.cpp
Runs well
~52 tok/s est.
Variant
Phi-4
14.7B · Q8_0 via llama.cpp
Runs well
~41 tok/s est.
Variant
Gemma 2 9B IT
9.24B · BF16 via SGLang
Tight fit
~34 tok/s est.

Components

1× Hardware NVIDIA GeForce RTX 4090 gpu · 24 GB
1× Hardware AMD Ryzen 9 7950X cpu
System RAM bandwidth 96 GB/s

Measured performance

Model · quantization	Runtime	Context	Measurements	Source
Llama 3.1 8B Instruct Q4_K_M (LM Studio Community)	llama.cppcuda · b4600	4K	12,100 tok/s · Prompt processing (512) 128 tok/s · Generation (128)	Mutinai illustrative fixtures
Qwen3 30B-A3B Q4_K_M (Unsloth AI)	llama.cppcuda · b5300	4K	3,400 tok/s · Prompt processing (512) 152 tok/s · Generation (128)	Mutinai illustrative fixtures

Community results

Contributions are not open yet, so there is nothing here from members.

Reviews

Contributions are not open yet, so there is nothing here from members.