System · desktop

RTX 4060 Ti 16GB budget build (32 GB DDR5)

Runs models up to ~22B parameters at 4-bit entirely on the GPU. Entry-level 16 GB GPU system.

GPU memory: 16 GB
System RAM: 32 GB
Runs: 8 of 19model variants, 8K
Estimated cost: ~$1,400

What it runs

Largest models that fit without spilling into system memory, at an 8K contextHow much text the model can consider at once, counted in tokens — roughly ¾ of a word each..

Everything →

Variant
Qwen2.5 14B Instruct
14.8B · Q6_K via llama.cpp
Tight fit
~15 tok/s est.
Variant
Phi-4
14.7B · Q4_K_M via llama.cpp
Runs well
~20 tok/s est.
Variant
Gemma 2 9B IT
9.24B · Q4_K_M via llama.cpp
Runs well
~30 tok/s est.
Variant
Hermes 3 Llama 3.1 8B
8.03B · Q8_0 via llama.cpp
Runs well
~21 tok/s est.
Variant
Llama 3.1 8B Instruct
8.03B · Q8_0 via llama.cpp
Runs well
~21 tok/s est.
Variant
Qwen2.5 7B Instruct
7.62B · Q8_0 via llama.cpp
Runs well
~22 tok/s est.
Variant
DeepSeek-R1-Distill-Qwen-7B
7.62B · Q4_K_M via llama.cpp
Runs well
~36 tok/s est.
Variant
Mistral 7B Instruct v0.3
7.25B · Q8_0 via llama.cpp
Runs well
~23 tok/s est.

Components

1× Hardware NVIDIA GeForce RTX 4060 Ti 16GB gpu · 16 GB
1× Hardware AMD Ryzen 9 7950X cpu
System RAM bandwidth 96 GB/s

Measured performance

Model · quantization	Runtime	Context	Measurements	Source
Qwen2.5 14B Instruct Q4_K_M	llama.cppcuda · b4600	4K	1,250 tok/s · Prompt processing (512) 25.5 tok/s · Generation (128)	Mutinai illustrative fixtures

Community results

Contributions are not open yet, so there is nothing here from members.

Reviews

Contributions are not open yet, so there is nothing here from members.