Tools
13 of 13ToolWorks withLicenseMeasurements
Runtime · turboderp · Python
A fast inference library for running LLMs locally on modern consumer-class GPUs
Works withexl2NVIDIA GPUs
LicensePermissive
Measurements—
Runtime · ggml.org · C++
LLM inference in C/C++
Works withggufNVIDIA GPUs, AMD GPUs, Apple silicon, most GPUs (Vulkan), CPUs
LicensePermissive
Runtime · Apple · Python
Run LLMs with MLX
Works withmlxApple silicon
LicensePermissive
Runtime · Ollama · Go
Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
Works withggufNVIDIA GPUs, AMD GPUs, Apple silicon, CPUs
LicensePermissive
Measurements—
Runtime · SGLang Project · Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Works withsafetensorsNVIDIA GPUs, AMD GPUs
LicensePermissive
Measurements—
Runtime · vLLM Project · Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Works withsafetensorsNVIDIA GPUs, AMD GPUs
LicensePermissive
Chat interface · Open WebUI · Python
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
Works withAny OpenAI-compatible model API
LicenseRestricted
Measurements—
Coding assistant · Aider · Python
aider is AI pair programming in your terminal
Works withAny OpenAI-compatible model API
LicensePermissive
Measurements—
Coding assistant · Continue · TypeScript
open-source coding agent
Works withAny OpenAI-compatible model API
LicensePermissive
Measurements—
Fine-tuning · Axolotl AI · Python
Go ahead and axolotl questions
Works withAny OpenAI-compatible model API
LicensePermissive
Measurements—
Fine-tuning · Unsloth AI · Python
Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.
Works withAny OpenAI-compatible model API
LicensePermissive
Measurements—
Evaluation · EleutherAI · Python
A framework for few-shot evaluation of language models.
Works withAny OpenAI-compatible model API
LicensePermissive
Measurements—
Gateway · BerriAI · Python
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
Works withAny OpenAI-compatible model API
LicensePermissive
Measurements—