AI / LLMs: 32 solutions compared
Which AI / LLMs tools can you self-host, run on a Mac or connect to? FrontierStack's catalogue covers 32 AI / LLMs options: AnythingLLM, Cerebras, ChatLLM Teams, CLOVA OCR (NAVER Cloud), CLOVA Studio (HyperCLOVA X), CrewAI (open source) and others. For 29 of them FrontierStack installs the service on a Mac or server, or connects to one you already run, and checks status, ports and certificates where they apply; 3 are hosted services FrontierStack connects to with an API token and monitors.
| Name | Type | What FrontierStack does |
|---|---|---|
| AnythingLLM | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| Cerebras | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| ChatLLM Teams | Hosted service | Connect with an API token; reachability and status alerts |
| CLOVA OCR (NAVER Cloud) | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| CLOVA Studio (HyperCLOVA X) | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| CrewAI (open source) | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| Darkbloom | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| Dyad | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| FriendliAI (프렌들리AI) | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| Hugging Face | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| IBM watsonx.ai | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| Jan | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| LangChain (open source) | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| LibreChat | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| LiteLLM | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| llama.cpp Server | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| LocalAI | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| Mem0 | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| MLX | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| Modular MAX | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| Needle 2 | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| NotebookLM | Hosted service | Connect with an API token; reachability and status alerts |
| Open Computer | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| Open Notebook | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| OpenCode | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| Papago Translation (NAVER Cloud) | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| PrivateGPT | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| Tetrate Agent Router | Hosted service | Connect with an API token; reachability and status alerts |
| TurboFieldfare | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| Upstage (Solar) | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| Vane (Perplexica) | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
| vLLM | Self-hosted or connected | Install or connect; status, ports and certificate checks where they apply |
AnythingLLM
Self-hosted RAG chat app
AnythingLLM is an self-hosted app for chatting with your documents; workspaces, RAG over any vector DB, agents, and pluggable LLM backends (Ollama, OpenAI, local servers). Run the Docker image (UI on :3001) or the desktop app; it exposes a developer REST API so you can wire its workspaces and chat into other services. Pairs with the vector DBs and local LLM servers in this app.
Install or connect AnythingLLM; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
Cerebras
Very fast hosted inference on wafer-scale hardware
Cerebras runs open models on its wafer-scale engine, which makes it markedly faster than GPU-hosted inference for the same weights; the practical benefit is latency, not model quality, so it suits interactive and agentic work where waiting is the cost. The API is OpenAI-compatible at https://api.cerebras.ai/v1 with a bearer key, so FrontierStack drives it through the normal AI provider path: add a Cerebras key in Model Costs and it appears in the assistant's model picker alongside every other provider. Models include gpt-oss-120b, Llama 3.3 70B and Qwen 3. A live monitor watches Cerebras's public status page for incidents. Note this is a cloud service: prompts leave this Mac, so the prompt firewall and redaction apply as they do to any external provider.
Install or connect Cerebras; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply. Full page →
ChatLLM Teams
Abacus.AI's multi-model team chat & agent workspace (cloud)
ChatLLM Teams (Abacus.AI) is a single subscription that puts the frontier models; Claude, GPT, Gemini, Grok, Llama and others; behind one team chat UI, so a team pays per seat instead of holding a key with every provider. Beyond chat it bundles document/RAG chat, image and video generation, web search, code execution, and AI Agents built from its AgentStudio, plus Slack/Teams/email connectors. A team workspace shares prompts, agents and chatbots across seats. Cloud-hosted at chatllm.abacus.ai; no local install. FrontierStack watches the seats and agents in your workspace through the Abacus.AI platform API and puts them in Alerts: pin it to be warned when the API stops answering or your deployed agents change. Because ChatLLM replaces per-provider keys, the spend it represents sits outside Model Costs; track that subscription in Subscriptions.
ChatLLM Teams is hosted, so FrontierStack does not try to install it. Connect an API token to check reachability and published status beside the services you host yourself; FrontierStack alerts you if it stops answering. Credentials stay in Keychain.
CLOVA OCR (NAVER Cloud)
NAVER CLOVA OCR; general, template & document OCR
Create an OCR 'domain' in the CLOVA OCR builder, generate its Secret Key and click API Gateway auto-integration; each domain gets a unique Invoke URL. Calls send the domain secret in X-OCR-SECRET. Two values must be stored (URL + secret).
Install or connect CLOVA OCR (NAVER Cloud); then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
CLOVA Studio (HyperCLOVA X)
Naver HyperCLOVA X LLM API (OpenAI-compatible)
API keys are issued in the NCP console's CLOVA Studio > API 키 menu as Test or Service keys. Service keys need a separate service-app application, and the key is shown only once at issuance. Base URL is https://clovastudio.stream.ntruss.com, with an OpenAI-compatible path /v1/openai (chat/completions, embeddings, models). Keys from before 2025-01 (X-NCP-CLOVASTUDIO-API-KEY + X-NCP-APIGW-API-KEY on clovastudio.apigw.ntruss.com) are deprecated.
Install or connect CLOVA Studio (HyperCLOVA X); then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
CrewAI (open source)
Multi-agent crews; Python framework + CLI (local)
The open-source CrewAI framework: define a *crew* of role-playing agents with goals, tools and tasks, then run it locally. It's a Python library plus a CLI, not a server; install with uv tool install crewai (or pip install crewai), scaffold with crewai create crew, and execute with crewai run; projects are config-first (agents/tasks files) with Python for the glue. Because there's no daemon or port, FrontierStack manages it as a local toolchain: it detects the crewai CLI and its version, and running crews show up in the Agent Platforms board's local-instance sweep. Distinct from the CrewAI AMP pane in Agent Platforms (formerly CrewAI Enterprise), which watches a crew deployed to CrewAI's hosted platform through that crew's own URL and token; use this one when the crews run on your own machine or server.
Install or connect CrewAI (open source); then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply. Full page →
Darkbloom
Private AI inference marketplace on idle Apple Silicon Macs
Darkbloom (Eigen Labs) routes encrypted inference requests to idle Apple Silicon Macs; prompts stay unreadable to the Macs' operators; at roughly half typical API prices (pay-per-token, no subscription; public alpha). It has two sides and FrontierStack supports both. Calling it: the API is OpenAI-compatible at https://api.darkbloom.dev/v1 with a bearer key from console.darkbloom.dev; add a Darkbloom key in Model Costs and it appears in the assistant's model picker (gemma-4-26b, gpt-oss-20b; MoE models up to 239B, 128K context). Earning from it: an Apple Silicon Mac (macOS 14+, 48 GB+ RAM recommended) can serve models when idle; install the provider CLI with curl -fsSL https://api.darkbloom.dev/install.sh | bash (installs to ~/.darkbloom with a /usr/local/bin symlink; no launchd service), then darkbloom login, darkbloom models download <id> and darkbloom start; darkbloom doctor verifies the machine, and earnings live at console.darkbloom.dev/earn. This pane detects the installed CLI and a running provider process. A live monitor watches Darkbloom's status page for coordinator incidents. Cloud caveats: even encrypted, prompts leave this Mac, so the prompt firewall and redaction apply as with any external provider; and the provider credentials in ~/.darkbloom (wallet_key, auth_token) are secrets; keep that folder out of backups you share.
Install or connect Darkbloom; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply. Full page →
Dyad
Local open-source AI app builder (desktop)
Dyad is a local, open-source AI app builder; a bring-your-own-key desktop alternative to Lovable/Bolt that generates and edits apps on your machine. It's an Electron desktop app for Mac/Windows, not a server (no web UI or port). Apache-2.0 for the core, FSL-1.1 for its src/pro module.
Install or connect Dyad; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
FriendliAI (프렌들리AI)
Seoul-founded LLM inference cloud (hosts LG EXAONE)
FriendliAI (founded in Seoul) runs OpenAI-compatible serverless and dedicated inference. LG AI Research partnered with it to offer EXAONE/K-EXAONE via API. Personal access tokens are created in Friendli Suite > Personal Settings; the SDK reads FRIENDLI_TOKEN. FriendliAI is also the official API host for LG AI Research's EXAONE models.
Install or connect FriendliAI (프렌들리AI); then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
Hugging Face
Model hub, Inference API & local tooling
Hugging Face is the hub for open models, datasets and Spaces, plus the Transformers library and serving stacks. Install the CLI (brew install huggingface-cli) to log in and pull/push models for the local servers here (MLX-LM, vLLM, Ollama). Hosted Inference Endpoints and the Inference API give a REST endpoint to hook into other services; self-host high-throughput serving with Text Generation Inference (TGI). Keep your access token in Keychain.
Install or connect Hugging Face; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
IBM watsonx.ai
IBM's enterprise AI platform; Granite + third-party models (cloud/on-prem)
IBM watsonx.ai is the model/studio half of the watsonx family (alongside watsonx.data for the lakehouse and watsonx.governance for model risk). It serves IBM's open Granite models plus curated third-party and Hugging Face models, with prompt-tuning, an agent framework and a REST inference API (/ml/v1/text/generation and chat/embeddings variants under https://{region}.ml.cloud.ibm.com, authenticated with an IAM bearer token minted from an API key, plus a project_id or space_id). Available as IBM Cloud SaaS, on AWS, or self-managed on-prem via watsonx.ai Software on Red Hat OpenShift/Cloud Pak for Data; the usual pick for regulated shops that need models running inside their own estate. Store the API key in Keychain; it's a standard OpenAI-alternative endpoint for the harness once configured.
Install or connect IBM watsonx.ai; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply. Full page →
Jan
Private, offline AI desktop app (OpenAI-compatible)
Jan is an open-source ChatGPT alternative that runs models 100% offline on your Mac. Its desktop app includes a local API server (OpenAI-compatible, default :1337) you can switch on in settings; add that in AI Models as a network endpoint to use Jan's models in the assistant.
Install or connect Jan; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
LangChain (open source)
Agent framework; LangChain, LangGraph and Deep Agents (Python/TypeScript)
LangChain is the open-source (MIT) framework for building LLM apps and agents in Python and TypeScript, with LangGraph underneath for durable, stateful agent graphs and Deep Agents for long-running planners with subagents and a filesystem. It's a set of libraries, not a server; pip install langchain langgraph or npm i langchain @langchain/langgraph. FrontierStack detects the LangGraph CLI (langgraph, pip install -U "langgraph-cli[inmem]"), whose langgraph dev starts a local Agent Server on :2024 that the LangSmith Deployment pane in Agent Platforms monitors (assistants, threads and runs). Traces, errors and cost go to LangSmith; add its key on the LangSmith service to watch them. The terminal coding agent built on Deep Agents ships separately as dcode.
Install or connect LangChain (open source); then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply. Full page →
LibreChat
Self-hosted multi-provider AI chat UI
LibreChat is the leading self-hosted ChatGPT-style interface; one polished web UI over OpenAI, Anthropic, Google, and local models via Ollama, with multi-user auth, presets, agents/tools, RAG file chat and search. Runs via Docker Compose (Node + MongoDB); web on :3080. Pairs with the AI Models pane (Ollama) for fully local chat, and Model Costs for tracking the API spend it generates.
Install or connect LibreChat; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
LiteLLM
Proxy/gateway for 100+ LLM APIs (OpenAI-compatible)
LiteLLM is a unified gateway that puts an OpenAI-compatible API in front of 100+ providers (OpenAI, Anthropic, Bedrock, Azure, Ollama, vLLM…). Run the proxy (pip install 'litellm[proxy]' then litellm --config config.yaml) on :4000 for one endpoint with routing, fallbacks, rate limits, key management and spend tracking; a natural front door for the Model Costs pane. Self-host or use the Python SDK.
Install or connect LiteLLM; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
llama.cpp Server
Lightweight local LLM server (OpenAI-compatible)
llama.cpp is the lean C/C++ inference engine for GGUF models, with first-class Apple Silicon (Metal) support. brew install llama.cpp, then run llama-server -m model.gguf to serve an OpenAI-compatible API on :8080 (/v1/chat/completions); point the AI Administrator, LibreChat or OpenCode at it. Add it as a provider in AI Administrator (llama.cpp Server). Download GGUF models from Hugging Face.
Install or connect llama.cpp Server; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
LocalAI
Self-hosted, OpenAI-compatible inference server
LocalAI is a free, drop-in replacement for the OpenAI API that runs LLMs, embeddings, image and audio models on your own CPU/GPU; no cloud, no key. Run it via Docker or the binary; it serves an OpenAI-compatible API (default :8080). Add it in AI Models as a network endpoint to drive it from the assistant.
Install or connect LocalAI; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
Mem0
Memory layer for AI agents; self-hosted (Docker) or cloud
Mem0 is a memory engine for AI apps and agents: it extracts, stores and retrieves durable facts across sessions so an assistant remembers preferences and history instead of restarting cold every conversation. The same engine runs two ways; the open-source server on your own infrastructure (Docker Compose; make bootstrap brings up the server, dashboard and first API key), or the managed Mem0 Platform. The self-hosted stack wants Postgres + pgvector for the vector store plus an LLM/embeddings provider (defaults to OpenAI models, but any OpenAI-compatible endpoint; including a local Ollama or LiteLLM; works, which keeps memories on your own hardware). Python/TypeScript SDKs and a REST API; pairs naturally with the vector databases and LiteLLM already in this catalogue.
Install or connect Mem0; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply. Full page →
MLX
Apple-silicon model serving; the MLX-LM CLI server, the oMLX inference server, or Inco AI's Splash engine
MLX runs large language models natively on Apple Silicon; unified M-series memory, often faster and lighter than llama.cpp on a Mac, with models from the mlx-community org on Hugging Face. FrontierStack manages both ways to serve them, in this one pane: MLX-LM (pip install mlx-lm, then mlx_lm.server --model … --port 8080; a plain OpenAI-compatible API) and oMLX ([jundot/omlx](https://github.com/jundot/omlx), Apache-2.0); an MLX inference server built for throughput: continuous batching and a two-tier KV cache (hot in RAM, cold on SSD), serving LLM, vision, OCR, embedding and reranker models on :8000 with OpenAI-compatible endpoints (/v1/chat/completions, /v1/embeddings, /v1/rerank) and the Anthropic /v1/messages API, plus an admin dashboard and chat at /admin. It is run from its native menu-bar app or its own CLI (omlx start / omlx stop, brew services start omlx); install the signed .dmg from GitHub Releases, or brew tap jundot/omlx https://github.com/jundot/omlx && brew install jundot/omlx/omlx. macOS 15+. Point LibreChat, OpenCode, Claude Code or the Model Costs pane at either; all three appear in the AI-harness model picker when running. The pane also runs Splash (Inco AI's open-source Apple-silicon engine, brew install incoai/tap/splash; OpenAI *and* Anthropic APIs; M3+ Mac, macOS 26.4+, 36 GB+ memory); not MLX-format models, but the same kind of local server; FrontierStack starts it on :8100 so it does not collide with oMLX on :8000. Underneath all of it is MLX itself, Apple's open-source array/ML framework ([ml-explore/mlx](https://github.com/ml-explore/mlx)); a NumPy-like API with lazy evaluation and a PyTorch-style nn module, for on-device training and inference; pip install mlx mlx-lm installs the framework with the LLM toolkit, and MLX-VLM adds vision models. MLX-VLM is also how the pane serves Bonsai 2 (PrismML's ternary models; 2B to 27B, a 9B in 3.5 GB, text + image, 262k context): pick one in the model list and it starts mlx_vlm.server instead, since mlx-lm can't load those packs yet.
Install or connect MLX; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply. Full page →
Modular MAX
Modular's inference server; OpenAI-compatible on :8000
MAX is Modular's inference platform: it serves models from Hugging Face behind an OpenAI-compatible API, so anything that speaks /v1/chat/completions can use it. Install the platform with pip install modular, or max[all] via pip/uv (max-all on conda/pixi); Modular recommends pixi for the most reliable environment and uv for the tightest Python interop. Then max serve --model-path <hf-model> listens on :8000; point the AI Administrator at it as a custom endpoint, or LibreChat and the Model Costs pane. Serving performs best on NVIDIA/AMD GPUs under Linux; on this Mac expect fewer compatible models and slower generation. MAX and the Mojo language are free under Modular's Community License; Modular's paid products are its hosted services; Modular Cloud (per token or GPU hour), Bring Your Own Cloud (per minute, in your VPC) and Enterprise (custom); none of which are needed to self-host here. The Mojo language itself lives under Runtimes, where it installs on its own.
Install or connect Modular MAX; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply. Full page →
Needle 2
45M on-device model for tool calling and structured extraction
Needle 2 (Cactus Compute, Apache-2.0) is a 45M-parameter foundation model small enough to run on any Mac; the weights are a single 14 MB .cact file baked into the inference engine. It is built for tool calling, device use and structured extraction: text in, JSON out, with confidence-gated responses, built-in tool retrieval from large catalogs, and bounded memory from a 256-token sliding window. Install it with pip install cactus-needle; needle playground serves its own web UI on 127.0.0.1:7860. FrontierStack's Needle pane installs it, runs the playground and extracts structured fields from a passage of text. It also fine-tunes: needle generate-data, needle finetune and needle build produce a tuned .cact you can point the playground at (add the [train,metal] extra for Apple Silicon GPU training). Note it is not an OpenAI-compatible server and not an assistant model; it appears in the AI Models picker but is deliberately refused for the assistant's tools, because at 45M parameters with a 256-token window it cannot drive them reliably.
Install or connect Needle 2; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply. Full page →
NotebookLM
Google's source-grounded research notebook (cloud)
NotebookLM (Google) is an AI research assistant grounded in *your* sources; upload PDFs, docs, URLs, slides or pasted text and it answers with citations, writes summaries and study guides, and generates podcast-style Audio Overviews. Hosted at notebooklm.google.com (free); the paid/Enterprise tier runs on Google Agentspace/Vertex AI. No local install or public REST API; it's a web app, so this entry links the site and docs.
NotebookLM is hosted, so FrontierStack does not try to install it. Connect an API token to check reachability and published status beside the services you host yourself; FrontierStack alerts you if it stops answering. Credentials stay in Keychain.
Open Computer
Virtual OS for AI agents; QEMU VM per agent (Mintplex Labs)
Open Computer (Mintplex Labs, the AnythingLLM team) gives an AI agent a whole disposable computer instead of a shell on yours: each agent gets an isolated QEMU virtual machine (Debian base, ~2.9 GB, ~100 MB per-agent overlay) with a browser, terminal, editor and file manager, and a live UI so you can watch and take over in real time. The point is to stop running agents with --dangerously-skip-permissions against your own host; a bad step wrecks a throwaway VM, not your machine. It drives native apps via accessibility APIs rather than screenshots (about 60% fewer tokens than vision-based computer use), supports bring-your-own harness (OpenClaw, Hermes…) and any OpenAI-compatible model, local or cloud, and can fork an agent with its current state. macOS ARM64 and Windows x64; run with open-computer create/up agent --dev. AGPL-3.0. Needs a model with at least a 16K context.
Install or connect Open Computer; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply. Full page →
Open Notebook
Open-source, self-hosted NotebookLM alternative
Open Notebook is a privacy-first, open-source (MIT) research notebook; the self-hostable answer to Google's NotebookLM. Add links, PDFs, slides and YouTube as sources; get AI summaries, grounded Q&A and podcast-style audio, with you in control of what the AI sees. Supports 18+ providers (OpenAI, Anthropic, Google, Groq, Mistral, DeepSeek, xAI…) and local models via Ollama / LM Studio. This pane offers one-click Docker self-host (app + SurrealDB, web UI on :8502) and a from-source developer setup (uv + Node). Pairs with the AI Models pane for fully local use.
Install or connect Open Notebook; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
OpenCode
Open-source AI coding agent (terminal)
OpenCode is an open-source AI coding agent for the terminal; point it at a repo and it reads, edits and runs code agentically, with a TUI, LSP awareness and your choice of provider (Anthropic/OpenAI/Google or local models via Ollama). Install with Homebrew (brew install opencode, or npm i -g opencode-ai), then run opencode inside a project. Log its usage in Model Costs.
Install or connect OpenCode; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
Papago Translation (NAVER Cloud)
Naver Papago machine translation (NAVER Cloud)
The free Papago API on Naver Developers ended on 29 Feb 2024; it is now an NCP product requiring an NCP Application (API Gateway Client ID/Secret). Current host papago.apigw.ntruss.com (legacy naveropenapi.apigw.ntruss.com/nmt/v1/translation).
Install or connect Papago Translation (NAVER Cloud); then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
PrivateGPT
Ask questions of your documents, 100% offline (RAG)
PrivateGPT is a production-ready RAG app that lets you query your own documents fully offline; no data leaves your machine. Ingests files into a local vector store and answers with citations via a local LLM. Run with Python/Poetry (or Docker); API + UI on :8001. A privacy-first alternative to cloud document Q&A.
Install or connect PrivateGPT; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
Tetrate Agent Router
Hosted LLM router; one OpenAI-compatible API (cloud)
Tetrate Agent Router Service (TARS) is a hosted LLM router: a single 100% OpenAI-compatible endpoint in front of proprietary and open-weight models (OpenAI, Anthropic, Google, …) with intelligent routing for cost/performance, per-environment keys and spend controls; point any OpenAI-SDK agent at it by changing base URL + key. Sign in at router.tetrate.ai (pay-as-you-go, model cost + 5% routing fee); Agent Router Enterprise runs in cloud/on-prem for data residency. FrontierStack checks it live; key valid and models listed; API key in Keychain. The self-hosted alternative is LiteLLM.
Tetrate Agent Router is hosted, so FrontierStack does not try to install it. Connect an API token to check reachability and published status beside the services you host yourself; FrontierStack alerts you if it stops answering. Credentials stay in Keychain.
TurboFieldfare
Gemma 4 26B-A4B on Apple Silicon in ~2 GB of RAM
TurboFieldfare is a custom Swift 6.2 + Metal 4 runtime that runs Google's Gemma 4 26B-A4B (instruction-tuned) on any Apple Silicon Mac; including 8 GB machines; in roughly 2 GB of RAM, by streaming the mixture-of-experts weights from disk instead of holding them resident. Useful when you want a 26B-class local model on a Mac that can't hold one in memory; for other Gemma sizes use the Ollama gemma3:* tags in Model Manager instead, since this runtime serves only the one model. Built from source: git clone, swift build -c release, then run .build/release/TurboFieldfareMac (the ~14.3 GB model streams down on first launch). TurboFieldfareCLI gives instruction chat and raw completions with --temperature/--top-k/--max-new. An experimental loopback OpenAI-compatible Chat Completions server on http://127.0.0.1:8080/v1 adds streaming and function-tool support; add that in AI Models as a network endpoint to use it in the assistant. Apache-2.0. Note it builds unsigned third-party source rather than installing a notarized binary, and :8080 is also llama.cpp's default; change one if you run both.
Install or connect TurboFieldfare; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply. Full page →
Upstage (Solar)
Korean Solar LLMs + document AI (OpenAI-compatible)
Upstage offers the Solar LLM family plus document AI (parse, OCR, extraction, classification) through api.upstage.ai, with OpenAI SDK compatibility (base_url https://api.upstage.ai/v1). Vector stores and file search use /v2. Keys are created with 'Create new key' in the console's API keys page.
Install or connect Upstage (Solar); then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
Vane (Perplexica)
Self-hosted AI answer engine; the open-source Perplexity alternative
Vane; formerly Perplexica, renamed in March 2026; is the most popular open-source, self-hosted alternative to Perplexity: ask a question, it searches the web and answers with citations. One container does everything (docker run -d -p 3000:3000 -v vane-data:/home/vane/data itzcrazykns1337/vane:latest): SearXNG runs inside it for private metasearch, and you bring your own model; an OpenAI/Anthropic/Gemini/Groq key, or fully local answers via Ollama on this Mac. Web UI on :3000. Focus modes cover academic search, YouTube, Reddit and writing; nothing you search is profiled or sold.
Install or connect Vane (Perplexica); then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
vLLM
High-throughput LLM inference server (OpenAI-compatible)
vLLM is a fast LLM inference and serving engine; PagedAttention and continuous batching give high throughput for production serving. pip install vllm, then vllm serve <model> exposes an OpenAI-compatible API on :8000 (/v1/chat/completions) you can point LibreChat, OpenCode or the Model Costs pane at. Best on CUDA GPUs; on Apple Silicon prefer MLX-LM. Models pull from Hugging Face.
Install or connect vLLM; then check its status, ports and certificate from FrontierStack. The same screen links to firewall checks and backups where they apply.
Run it from your Mac.
FrontierStack installs, monitors and secures services on this Mac and on linked servers; and keeps your passwords and keys out of any AI's sight.
Download FrontierStackApple notarized · Safe & secure · macOS 13 Ventura+
