MLX on a Mac

Apple-silicon model serving; the MLX-LM CLI server, the oMLX inference server, or Inco AI's Splash engine · AI / LLMs

MLX runs large language models natively on Apple Silicon; unified M-series memory, often faster and lighter than llama.cpp on a Mac, with models from the mlx-community org on Hugging Face. FrontierStack manages both ways to serve them, in this one pane: MLX-LM (pip install mlx-lm, then mlx_lm.server --model … --port 8080; a plain OpenAI-compatible API) and oMLX ([jundot/omlx](https://github.com/jundot/omlx), Apache-2.0); an MLX inference server built for throughput: continuous batching and a two-tier KV cache (hot in RAM, cold on SSD), serving LLM, vision, OCR, embedding and reranker models on :8000 with OpenAI-compatible endpoints (/v1/chat/completions, /v1/embeddings, /v1/rerank) and the Anthropic /v1/messages API, plus an admin dashboard and chat at /admin. It is run from its native menu-bar app or its own CLI (omlx start / omlx stop, brew services start omlx); install the signed .dmg from GitHub Releases, or brew tap jundot/omlx https://github.com/jundot/omlx && brew install jundot/omlx/omlx. macOS 15+. Point LibreChat, OpenCode, Claude Code or the Model Costs pane at either; all three appear in the AI-harness model picker when running. The pane also runs Splash (Inco AI's open-source Apple-silicon engine, brew install incoai/tap/splash; OpenAI *and* Anthropic APIs; M3+ Mac, macOS 26.4+, 36 GB+ memory); not MLX-format models, but the same kind of local server; FrontierStack starts it on :8100 so it does not collide with oMLX on :8000. Underneath all of it is MLX itself, Apple's open-source array/ML framework ([ml-explore/mlx](https://github.com/ml-explore/mlx)); a NumPy-like API with lazy evaluation and a PyTorch-style nn module, for on-device training and inference; pip install mlx mlx-lm installs the framework with the LLM toolkit, and MLX-VLM adds vision models. MLX-VLM is also how the pane serves Bonsai 2 (PrismML's ternary models; 2B to 27B, a 9B in 3.5 GB, text + image, 262k context): pick one in the model list and it starts mlx_vlm.server instead, since mlx-lm can't load those packs yet.

Run MLX with FrontierStack

Install or connect MLX; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.

Related in AI / LLMs

• Vane (Perplexica)
• Needle 2
• TurboFieldfare
• Cerebras
• Darkbloom
• vLLM
• llama.cpp Server
• Modular MAX