Hardware · CPU · AMD

AMD Ryzen 9 7950X

Runs models from system RAM. Slower than a GPU, but not limited by graphics memory.

What it runs

Largest models without offloading on the Dual RTX 3090 (128 GB DDR5), 8K context.

Variant
Llama 3.1 70B Instruct
70.6B · Q4_K_M via llama.cpp
Tight fit
~13 tok/s est.
Variant
Llama 3.3 70B Instruct
70.6B · Q4_K_M via llama.cpp
Tight fit
17.6 tok/s measured
Variant
Mixtral 8x7B Instruct v0.1
46.7B · Q4_K_M via llama.cpp
Runs well
~69 tok/s est.
Variant
Qwen2.5 32B Instruct
32.8B · Q5_K_M via llama.cpp
Runs well
~24 tok/s est.
Variant
Qwen2.5-Coder 32B Instruct
32.8B · Q8_0 via llama.cpp
Runs well
~16 tok/s est.
Variant
DeepSeek-R1-Distill-Qwen-32B
32.8B · Q4_K_M via llama.cpp
Runs well
~28 tok/s est.
Variant
Qwen3 30B-A3B
30.5B · Q8_0 via llama.cpp
Runs well
~143 tok/s est.
Variant
Gemma 3 27B IT
27.4B · Q4_K_M via llama.cpp
Runs well
~33 tok/s est.
Variant
Gemma 2 27B IT
27.2B · Q4_K_M via llama.cpp
Runs well
~34 tok/s est.
Variant
Mistral Small 24B Instruct 2501
23.6B · Q6_K via llama.cpp
Runs well
~29 tok/s est.

System Dual RTX 3090 (128 GB DDR5)
Runs models up to ~73B parameters at 4-bit entirely on the GPU. What runs
System RTX 5090 workstation (96 GB DDR5)
Runs models up to ~47B parameters at 4-bit entirely on the GPU. What runs
System RTX 4090 workstation (64 GB DDR5)
Runs models up to ~35B parameters at 4-bit entirely on the GPU. What runs
System RX 7900 XTX desktop (64 GB DDR5)
Runs models up to ~35B parameters at 4-bit entirely on the GPU. What runs
System RTX 4060 Ti 16GB budget build (32 GB DDR5)
Runs models up to ~22B parameters at 4-bit entirely on the GPU. What runs
System CPU-only Ryzen 9 7950X (128 GB DDR5)
Runs models up to ~170B parameters at 4-bit in system RAM (CPU only). What runs

Throughput on systems containing this device.

Model · quantization	System	Runtime	Context	Measurements	Source
Llama 3.1 8B Instruct Q4_K_M (LM Studio Community)	CPU-only Ryzen 9 7950X (128 GB DDR5)	llama.cppcpu · b4600	4K	92 tok/s · Prompt processing (512) 12.4 tok/s · Generation (128)	Mutinai illustrative fixtures
Llama 3.3 70B Instruct Q4_K_M (LM Studio Community)	Dual RTX 3090 (128 GB DDR5)	llama.cppcuda · b4600	4K	390 tok/s · Prompt processing (512) 17.6 tok/s · Generation (128)	Mutinai illustrative fixtures
Qwen2.5 14B Instruct Q4_K_M	RTX 4060 Ti 16GB budget build (32 GB DDR5)	llama.cppcuda · b4600	4K	1,250 tok/s · Prompt processing (512) 25.5 tok/s · Generation (128)	Mutinai illustrative fixtures
Llama 3.1 8B Instruct Q4_K_M (LM Studio Community)	RTX 4090 workstation (64 GB DDR5)	llama.cppcuda · b4600	4K	12,100 tok/s · Prompt processing (512) 128 tok/s · Generation (128)	Mutinai illustrative fixtures
Qwen3 30B-A3B Q4_K_M (Unsloth AI)	RTX 4090 workstation (64 GB DDR5)	llama.cppcuda · b5300	4K	3,400 tok/s · Prompt processing (512) 152 tok/s · Generation (128)	Mutinai illustrative fixtures
Qwen2.5 32B Instruct Q4_K_M	RTX 5090 workstation (96 GB DDR5)	llama.cppcuda · b4600	4K	2,900 tok/s · Prompt processing (512) 61 tok/s · Generation (128)	Mutinai illustrative fixtures
Llama 3.1 8B Instruct Q4_K_M (LM Studio Community)	RX 7900 XTX desktop (64 GB DDR5)	llama.cpprocm · b4600	4K	3,050 tok/s · Prompt processing (512) 94 tok/s · Generation (128)	Mutinai illustrative fixtures

Contributions are not open yet, so there is nothing here from members.

Contributions are not open yet, so there is nothing here from members.

No source records linked.