Tool · Inference runtime · Apple
MLX-LM
Run LLMs with MLX
- License
- PermissiveMIT License
- Language
- Python
- Latest release
- 22 Apr 2026MLX-LM v0.31.3
- Measurements
- 1with this runtime
Relationships
No relationships recorded.
Releases & news
- MLX-LM v0.31.3Lots of bugfixes
- MLX-LM v0.31.2Caching system prompt and user messages for non-trimmable caches
- MLX-LM v0.31.0Fix save/load of CacheList by @angeloskath in https://github.com/ml-explore/mlx-lm/pull/886
- MLX-LM v0.30.7Fix Kimi Linear by @kernelpool in https://github.com/ml-explore/mlx-lm/pull/853
- MLX-LM v0.30.6Transformers v5 by @awni in https://github.com/ml-explore/mlx-lm/pull/811
- MLX-LM v0.30.5import logging as it throws no logging error in place of actual error by @Maanas-Verma in https://github.com/ml-explore/mlx-lm/pull/778
- MLX-LM v0.30.4Add AWQ/GPTQ weight transformation utilities by @ericcurtin in https://github.com/ml-explore/mlx-lm/pull/730
- MLX-LM v0.30.2Fix mlx-lm release by @awni in https://github.com/ml-explore/mlx-lm/pull/733
- MLX-LM v0.30.1custom dsv32 chat template by @awni in https://github.com/ml-explore/mlx-lm/pull/693
- MLX-LM v0.30.0fix: server busy-waiting during idle request polling by @zenyr in https://github.com/ml-explore/mlx-lm/pull/674
Measured performance with this runtime
Throughput members and projects recorded, in tokens per secondHow fast the model writes its answer; about 10 tokens per second reads like comfortable typing..
| Model · quantization | System | Runtime | Context | Measurements | Source |
|---|---|---|---|---|---|
| Llama 3.3 70B Instruct MLX 4-bit (MLX Community) | MacBook Pro M4 Max 128 GB | MLX-LMmetal · 0.21.0 | 4K | 11.2 tok/s · Generation throughput 41 GB · Peak memory 110 tok/s · Prompt throughput 4,200 ms · Time to first token | Mutinai illustrative fixtures |
Reviews
Contributions are not open yet, so there is nothing here from members.
Sources & history
Sources
- GitHub (live source, 2 records, 13 Sept 2026)
External identifiers
- github ml-explore/mlx-lm
Field history
- homepageUrl = null — GitHub (current)
- homepageUrl = null — GitHub
- primaryLanguage = "Python" — GitHub (current)
- primaryLanguage = "Python" — GitHub
- summary = "Run LLMs with MLX" — GitHub (current)
- summary = "Run LLMs with MLX" — GitHub