Vllm Mlx
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
FreeOpen source
What you get
Vllm Mlx is a free, open-source AI tool published on GitHub by waybarrios.
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
GitHub stars: 1,633
License: Apache-2.0 (free to use)
Language: Python
Topics: anthropic, anthropic-api, apple-silicon, claude-code, continuous-batching, inference-server, llm, local-llm
Last updated: 2026-10-20
How to get it: open the repository on GitHub, read the README and follow the setup steps. HEEEXY lists it in the Lab because it is popular, actively maintained and openly licensed.
More for local ai
- Local Deep Research: ~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
- MakazhanAlpamys Soup: Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
- Dograh Hq Dograh: Open source voice AI platform. Self-hosted alternative to Vapi and Retell. On Prem, BYOK across Speech to Speech or LLM/STT/TTS, with a visual workflow builder, MCP native and telephony support.