When Apple discontinued macOS Server.app, Mac users lost the simple, point-and-click way to run websites and services. FrontierStack brings that back and goes much further: it installs, controls, secures and monitors Apache, PHP, MySQL, PostgreSQL, Nginx and more than a thousand other services — on the Mac in front of you and on every server you link to it — and it puts a tool-using AI administrator at the helm.

This manual is the complete reference for the desktop app. A companion Mobile Manual covers the iPhone and iPad app. Read this first chapter end to end; after that, dip into whichever part you need.

1.1What FrontierStack is

FrontierStack is a native macOS application — not a web panel, not a container you have to babysit. It manages real services on real machines using the tools they already ship with: Homebrew formulae, launchd and systemd units, Apache and Nginx config, certificates, firewalls and SSH. It is three things at once:

• A control panel. Install, start, stop and configure services with validation, live status and one-click fixes — the macOS Server replacement, modernised.
• A monitoring and alerting system. A continuous health sweep watches services, sites, certificates, disks and your whole fleet, and turns the first sign of trouble into a message on the channels you choose.
• An AI administrator. A guard-railed, tool-using assistant with a live, accurate picture of your setup. It can diagnose what is broken, propose a fix you approve, run it locally or across your fleet, and reshape the app to exactly what you run.

1.2An AI-first design

Most apps bolt a chat box onto the side. FrontierStack is built the other way around: the AI is a first-class operator of the same controls you use. Because it acts only through a fixed set of audited tools — read a log, run a diagnostic, restart a service, issue a certificate, write a file — everything it does is something you could do by hand, and everything that changes the system is shown to you for approval first.

Three principles make this safe enough to trust with production servers:

• Read-only by default. Out of the box the assistant can look but not touch. Changes and scripts are unlocked by explicit toggles, and every mutating action shows an approval card before it runs.
• Your secrets never reach the model. Passwords and API tokens live in a local .env vault. Scripts reference them by name; the app injects the real value at run time, locally or over SSH. The cloud model sees the name, never the value. Logs and files are secret-redacted before they are sent.
• You bring your own AI. Point it at a cloud model or a local one (Ollama, LM Studio). If you drive it from your own subscription tool over MCP — Claude Code, Cursor — there is no metered per-token bill at all.

1.3Who it is for

FrontierStack suits anyone running — or wanting to run — servers on a Mac:

• Developers who want a clean local stack instead of MAMP or XAMPP.
• Small businesses self-hosting POS, CRM, e-commerce, mail and helpdesk on a Mac mini.
• Admins keeping a vintage Xserve or Mac mini on macOS Server useful by managing it remotely.
• Anyone moving from metered SaaS to self-hosted, or running a fleet of Macs, Linux boxes, BSD servers and Raspberry Pis from one window.

1.4Key concepts and vocabulary

A handful of terms recur throughout this manual. Meet them once here.

1.5How this manual is organised

The manual follows the shape of the work:

• Part I — Getting Started: install, first run, and a tour of the interface.
• Part II — Running Services: the web stack, databases, the full catalog, and sites & domains.
• Part III — Fleet, Network & Security: linking servers, networking and the perimeter, hardening, and monitoring & alerts.
• Part IV — Automation & AI: scripts and schedules, the AI Administrator in full, MCP, and the AI stack.
• Part V — Business & Reference: vertical solutions, settings, shortcuts, licensing and troubleshooting.

If you are brand new, continue to Chapter 2, Installation & First Run. If you just want to understand the AI, jump ahead to Chapter 13 — but the safety model there assumes the concepts above.

FrontierStack installs like any other Mac app — drag it to /Applications and open it. What happens next is up to you: it can run entirely with the per-action password prompts macOS already gives you, or you can grant it a small set of permissions once and never be asked again. This chapter walks the whole first run, then shows how to migrate an existing local stack so you keep your sites and databases.

2.1System requirements

FrontierStack is a native, signed macOS application. You need:

Nothing in this list is unusual; if your Mac is recent enough to run a current macOS, it can run FrontierStack.

2.2Downloading and first launch

Download the app, move FrontierStack.app into your /Applications folder, and double-click it. Because the app is signed by Enfour, Inc. (Apple Developer team 6Y87KNMALP) and notarised, macOS opens it without the unidentified-developer warning.

1. Open FrontierStack. The main window appears with the service sidebar on the left and a health overview in the centre — the layout you met in Chapter 1.
2. If Homebrew isn't present yet, panes that need it will say so and offer to install it. You can do that now or later.
3. FrontierStack enforces a single running instance. If a copy is already open and you launch it again, the existing window is simply brought to the front rather than a second copy starting. This keeps two windows from fighting over the same services and config files.

The first launch makes no changes to your system. The app reads what is installed and shows you the state of things; nothing is started, stopped or reconfigured until you ask.

2.3The privileged helper — and the permissions macOS asks for

Starting and stopping services and editing system configuration (Apache, Nginx, MySQL, DNS, the firewall) are privileged operations. Out of the box, FrontierStack performs each one through the standard macOS authorisation prompt: you type your password every time. That is safe but quickly becomes tedious.

The optional helper removes that friction. It is a small privileged tool — an SMAppService daemon — that the app registers with macOS so it can carry out admin actions without re-entering your password each time. You install it deliberately: use the Install affordance at the foot of the sidebar, or accept the prompt when an action first needs it.

1. Click Install on the helper row at the bottom of the sidebar.
2. macOS shows its Login Items & Extensions approval (or asks you to enable the background item in System Settings ▸ General ▸ Login Items). Approve it — this is macOS asking whether to let the daemon run, not the app overriding anything.
3. The sidebar row flips to show the helper registered and running. From now on, service start/stop and config edits proceed without the per-action password prompt.

You can remove the helper at any time; FrontierStack then falls back to per-action admin prompts. Both modes work — the helper is purely a convenience.

2.4Installing Homebrew

Most of the catalog — web servers, databases, caches, mail, dozens of CLI tools — is installed as Homebrew formulae. Homebrew is the package manager FrontierStack uses to fetch and update those services, so if it isn't already on your Mac you'll want it early.

Open the Homebrew pane from the sidebar. If Homebrew is missing, the pane says "Homebrew isn't installed. It's the package manager this app uses to install services," with a Get Homebrew… button. Installation runs in the app's in-app console: it executes the official Homebrew installer and, at the point the installer needs administrator rights, you type your own password into the console. FrontierStack never captures or stores it — the prompt is Homebrew's own.

Once Homebrew is present, the pane becomes a full package manager: a searchable table of installed formulae and casks with their versions, an Install formula or cask field, an Update All button that shows the outstanding count, and per-row Update / Uninstall actions. The AI Administrator can drive the same flow with its install_homebrew and install_package tools, again through the in-app console where you enter your password.

2.5The Command-Line PATH pane

FrontierStack detects installed tools by resolving them the way your shell does. Occasionally a tool you've installed (via pip, pipx, a virtualenv, or a non-standard prefix) isn't on the PATH the app sees, so it reports the tool as missing. The Command-Line PATH pane fixes that.

• Pick your shell profile (for example ~/.zprofile). Directories you add are written to a clearly marked managed block in that file and prepended to your PATH — open a new terminal to pick up the change.
• Directories to add lets you add a path by typing it or with Choose…; the top entry is searched first. Current PATH (effective) shows your login shell's PATH, tagging entries the app manages.
• PATH Doctor diagnoses why a tool isn't detected: enter a command (e.g. mlx_lm.server, gcloud) and it reports whether the app finds it, whether only your shell finds it, or whether it isn't installed at all — with a one-click Add to managed PATH for any directory the app is missing.

2.6The Install Wizard

If you're starting from scratch, the Install Wizard (in the sidebar) is the fastest way to a working stack. Pick a target use-case — WordPress/PHP, Full-Stack JS, Python/Django, Rails, static site, database, or reverse proxy — and the wizard pre-selects the recommended components (web server, database, cache) for that goal. Adjust the selection to taste, then it installs everything through Homebrew.

The same pane shows the status of the native Apache + MySQL + PHP stack at a glance, and detects all-in-one bundles already on your Mac — XAMPP, MAMP, AMPPS, Local. If it finds one, it routes you to Migrate Setups rather than installing a parallel stack, so you don't end up running two of everything.

2.7Migrating an existing stack

If you already run sites locally under MAMP, XAMPP or Apple's Server.app, you don't have to rebuild them. The Migrate Setups pane detects those stacks and brings their pieces into FrontierStack's native Homebrew stack: web files, MySQL databases, PHP settings, and the Apache vhosts that define each site.

1. Open Migrate Setups. It scans for installed sources — MAMP / MAMP Pro, XAMPP and Apple Server — and reports what it found: the sites, the database, and whether PHP was detected. (The AI Administrator's detect_migration_sources tool runs the same read-only scan.)
2. Review the Sites to import list. Each candidate shows its server name and listening port; anything that can't be brought across cleanly appears under Skipped with the reason.
3. For settings, the Import All Settings action copies the source's php.ini, my.cnf and nginx.conf over the matching Homebrew config. A .pre-mamp.bak backup of each file is kept so you can compare or revert.
4. Choose Import Selected to run the migration. Live progress is shown in the pane as it installs the needed Homebrew services, copies web files, migrates the MySQL databases and imports the vhosts.
5. Review afterwards. Source configs often reference the old tool's paths and ports (for example MAMP's extension_dir, datadir, socket, or port 8889), so check the imported settings before relying on them.

The AI Administrator can drive this end-to-end with run_migration, but only after detecting sources and confirming with you — and only with changes enabled.

2.8Where your data lives

FrontierStack keeps its own state in predictable, standard places, so you always know what it touches and can back it up:

Keeping the app's Apache config in its own frontierstack/ directory means FrontierStack can manage vhosts safely without clobbering configuration you maintain by hand. The bundle identifier is jp.co.enfour.frontierstack (the helper and DDNS components use the .helper and .ddns suffixes).

With the app installed, the helper decided, Homebrew in place and any old stack migrated, you're ready to actually use it. Chapter 3, The Interface tours the window in detail: the sidebar, panes, search, the menu bar, and the controls you'll reach for every day.

FrontierStack manages an enormous range of services, yet it presents them through a single, familiar macOS window: a sidebar of things on the left, the screen for the selected thing in the centre, and a row of always-available tools in the toolbar. Learn this layout once and every pane in the book — from Apache to a Cloudflare zone — sits exactly where you expect it. This chapter walks the interface end to end so the rest of the manual can simply say "open the Sites pane" and trust that you know how.

3.1The main window

The window is a standard macOS NavigationSplitView with three regions. On the left is the service sidebar: a scrollable, collapsible list of every service and tool you have chosen to see, grouped into categories. In the centre is the active pane — the screen for whatever you selected in the sidebar, with its own header, tabs and controls. Running across the top is the toolbar, whose buttons are available no matter which pane you are in.

From right to left, the toolbar holds: Search (the magnifying glass, also ⌘F); the AI assistant (the wand, which opens the floating command palette); a Shell button that drops a terminal into the current working directory; a Dark mode toggle that forces dark even when macOS is in light; and, when the selected pane can be floated, a break-off button to pop it out as a palette. If you have unsaved changes to a Setup, an orange Unsaved menu appears here too.

The sidebar's width is yours to set — drag the divider and FrontierStack remembers it across launches. Your last-selected pane is restored when you reopen the app, so you return to where you left off.

3.2The service sidebar

The sidebar is organised into two tiers. At the top sits Overview — the app's own built-in tools: Local Health, Status Page, Alerts, Domains, Logs, Certificates, Notes, Scripts, Setups, Settings and more. Below it, one section per service category — Web Servers, Databases, Caching, Mail, Networking, Security Tools, Containers, AI and dozens more — each listing the services you have chosen to show from the catalog (Chapter 6). Every section header is a chevron you can click to collapse, and the app remembers which groups you leave folded.

Each service row shows a status dot when it is running. Right-click a service for two quick actions: Pin to Top of Section keeps a service you touch often at the top of its category, and Hide removes it from the sidebar. Hiding never uninstalls anything — the service keeps running and stays reachable from Search; it just stops cluttering the list.

Below the categories, the sidebar grows extra sections for the things you pin elsewhere in the app: linked servers (each with little category icons showing which services it runs), monitored Wi-Fi networks, Cloudflare zones and Kubernetes clusters. Type in the filter field at the top of the sidebar to narrow every section at once.

3.3Presets and Setups

Two features shape what the sidebar shows. The Presets menu (in the menu bar) is the fast way to load a whole use-case at once. Its Base — show services for… section swaps the sidebar to a curated set for a scenario (a web stack, a mail server, a shop), while Add to current view… layers an extra group onto what you already have. Show All reveals everything; it is the escape hatch when you cannot find something.

Setups go further: a Setup is a named snapshot of your whole working view — which services are hidden, which are pinned, and the floating palettes you have open and where they sit. Save one from Presets ▸ Save Current as Setup… (or from the Setups pane), then flip between Setups from the top of the Presets menu; the active one carries a check mark. The palette layout auto-saves into the active Setup as you move windows around, so a Setup always reflects how you really work. Change which services are shown and the Setup goes dirty; the toolbar shows an orange Unsaved badge with Update, Save as New and Revert.

A Setup can also be bound to a Location (Chapter 8). Bind your "Office" Setup to your office network and FrontierStack switches to it automatically when you arrive — the right services, the right palettes, for where you are.

3.4Floating palettes and mini-palettes

Sometimes you want to keep one screen visible while you work elsewhere. Any monitor-style pane can be broken off into a floating palette: a small, always-on-top utility window built on macOS NSPanel, so it stays above the main window and other apps. Use the break-off button in the toolbar (or ⌘⌥⇧W to close the front palette). Local Health, Containers, Cloud Servers, the Wi-Fi Analyzer and others all float.

For an even lighter footprint there are mini-palettes — compact, glanceable summaries rather than the full pane. Minis exist for Local Health, Cloudflare Zones, Wi-Fi, Printers, Router & Network, Containers, Scripts, Message Delivery and AWS. Each mini row has a button to open the matching pane in the main window when you need the detail. Open palettes are part of a Setup, so a saved view restores your floating windows exactly as you arranged them.

3.5The command and AI palette

The wand button in the toolbar opens the AI assistant palette — a draggable, always-on-top NSPanel with a single ask-anything field ("why is my site down?", "tune MySQL", "show only what I use"). It is the same AI Administrator covered in Chapter 13, surfaced as a floating window you can keep beside whatever you are doing. Like every floating palette it stays above the main window; unlike the others it talks back, proposes fixes and — with your approval — acts. It carries its own conversation history and an approval card for any mutating action, so nothing runs until you say so.

3.6The menu-bar item

FrontierStack can place a small item in the macOS menu bar (toggle Show in Menu Bar). It is the quickest way to act without bringing the whole window forward. From it you can Start or Stop your startup group, Restart any of the core servers (Apache, Nginx, PHP-FPM, MySQL, Postgres) or everything running at once, open localhost in your browser, refresh status, set Launch at Login, and open or quit the app.

3.7In-app Search

Search (the magnifying glass, ⌘F) is the fastest way to reach anything. Start typing "search services, tools & actions…" and the palette matches panes, tools, services and actions as you go — including services you have hidden from the sidebar. Pick a result and the app navigates straight to it. A Semantic mode lets you search by meaning ("block an IP", "free up disk") rather than exact names, so you can find a capability without knowing what FrontierStack calls it.

3.8The Help window

Open Help with ⌘? for a dedicated FrontierStack Help window. It is a real book of in-app documentation, organised to mirror the app. Help is more than reading: its pages carry Open in app links. Click one and FrontierStack jumps the main window to that exact pane or service — the Help window stays open beside it, so you can read a procedure and follow along step by step. Many panes have a small help button in their header that opens Help at the matching topic.

3.9The Notes pane

Notes is your private place for credentials, runbooks and per-service instructions. Each note has a title, a service tag and a body, and the whole thing is encrypted in the macOS Keychain — not stored in plain files. The defining feature is the per-note lock: every note carries a Share with AI Administrator toggle that is off by default. While a note is locked, the AI literally cannot read it. Unlock a note only when you want the assistant to use those credentials or steps; it then appears in the AI's list_notes and can be read on demand.

3.10The Status Page

The Status Page pane publishes a static index.html built from the same up/down data as Local Health, so customers or colleagues can see what is running without access to the app. Choose a destination — a folder inside one of your Apache sites' DocumentRoots, so it is served at a path like /status/ — set a page title, and decide whether to Include item details (leave detail lines off for a public page, since they can reveal internal hosts and ports). The page regenerates on an interval you set, or on demand with Generate Now. It is plain static HTML; there is nothing extra to run.

3.11Local Health and Locations at a glance

Two Overview panes anchor everyday use, and each has a full chapter of its own. Local Health is the board that answers "is everything OK?": every monitored service, device and subsystem with an UP or DOWN state and a one-line summary. You mark items as critical from their own panes to add them to the board, and it doubles as a floating palette and a mini. Its full treatment — the health sweep, thresholds and how it drives alerts — is Chapter 11.

Locations teach the app where your Mac is on the network right now — Home, Office, On VPN — matched from the Wi-Fi network or subnet. Locations make monitoring network-aware: while you are somewhere unrecognised the app treats you as Away and pauses location-scoped checks, so a café or hotel network never sets off false alarms. They also bind Setups, as we saw above. The complete model — defining locations, scoping alerts and reachability, the fleet — is Chapter 8.

With the layout learned, you are ready to put it to work. The next part of the manual opens the panes themselves, starting with Chapter 4, The Web Stack.

A web server is the heart of most self-hosted setups, so FrontierStack treats it as a first-class citizen. The Apache pane installs, starts, validates and configures a Homebrew Apache; the PHP, Localhost, Certificates and DNS panes round out a complete development and hosting stack — and every change goes through validation so a typo never takes a site down silently.

This chapter covers the panes that make and run web sites on the Mac in front of you. Connecting that stack to a real domain and the public internet is Chapter 7; tightening it for production is Chapter 10.

4.1The Apache pane

Open the Apache pane from the sidebar to see your web server’s state at a glance: version, whether it is running, and the ports it listens on. The buttons across the top are the controls you reach for daily — Start, Stop, Restart and Validate Config. Validation runs apachectl -t (a real syntax check) and shows the result in Last Result, so you can confirm a change is sound before you reload.

FrontierStack recommends a Homebrew Apache rather than the macOS system copy. The Homebrew server lives under /opt/homebrew (or /usr/local on Intel Macs), runs as you through brew services without a root prompt for every action, and is not locked down by System Integrity Protection — so a current mod_php loads cleanly and the whole stack stays on one toolchain. The system Apache under /etc/apache2 is wiped by OS updates and ties you to Apple’s bundled libphp. If Homebrew Apache is not installed, the pane offers Install Homebrew Apache… in one click.

If you keep several Apache builds — the Homebrew one, a legacy local build, or a remote server’s — the binary picker (see Add Homebrew Apache / Add Local Build…) lets you register and switch between them, each with its own version and ports shown.

4.2Modules, ports and the managed config

Lower in the Apache pane are the parts of the config you most often edit, each surfaced as a control rather than a file:

For anything not exposed as a control, the config rows give you Edit Managed Config… (the FrontierStack include) and Edit httpd.conf… (the main file) directly. Every save validates with apachectl -t, reloads gracefully, and rolls back if the new config does not parse — so a bad edit is caught before it can break the running server.

Toggling a module rewrites its LoadModule line and revalidates. If a change leaves httpd -t failing — the classic case is enabling proxy_fcgi without proxy, which the app now auto-enables for you — the Modules window shows the error and won’t close silently: you either correct it or choose Save anyway. When Apache is wedged and won’t start (the classic Homebrew Bootstrap failed: 5: Input/output error, from a service that was already loaded, or from sudo brew services leaving some httpd paths owned by root), the pane’s Recovery & reset section offers three escalating fixes: Restart cleanly (stops httpd in both the user and system launchd domains, then starts it once), Reset config to default (backs up your httpd.conf and restores Homebrew’s), and Remove & Reinstall (clears the root-owned paths and reinstalls the formula). All run in the in-app console so you see the sudo prompt and output. If Service Watchdog keep-alive is also on for Apache, it may restart httpd on its own — expected, but pause it if it fights a manual recovery.

4.3Choosing the PHP version

Open the PHP pane to manage the PHP runtimes installed on the Mac. Homebrew lets several major versions live side by side (for example [email protected] through the current release), and FrontierStack treats them as a set you switch between rather than a single global install.

There are two distinct choices:

• The Apache module. In the Apache pane’s PHP Module section, the Load for websites picker chooses which version’s libphp.so Apache loads. Changing it rewrites the managed config and reloads Apache.
• Per-site PHP. A local dev server (the Localhost pane, below) has its own PHP Version and PHP Binary fields, so one project can run on an older PHP while another runs on the latest — without changing what the rest of the machine uses. This is the usual way to keep a legacy app and a new one happy at the same time.

The Runtime Health pane gives the wider picture — PHP alongside Python, Node.js, Java, Go, Rust, Ruby, Perl and .NET — with a self-check that flags a missing or end-of-life runtime before it bites you.

The Runtimes section also carries a React pane for front-end work. It scaffolds a new app (Vite, TanStack Start or Next.js), points at any React project, and offers a curated companion-library list filed by the job each does — state management (Zustand, Jotai, Redux Toolkit), data fetching and server state (TanStack Query — the library formerly called React Query — and SWR), full-stack frameworks (TanStack Start, Next.js), routing, validation (ArkType, Zod), forms, and authentication (Clerk). Each row shows its docs, whether the selected project already uses it, and a one-click npm install into that project.

4.4Sites and virtual hosts

The Websites list in the Apache pane is where you manage virtual hosts. Each row shows the site’s name, the bind address and the port (for example :8080), and a Running toggle for its live status. The buttons let you Add Site…, edit, duplicate, disable, Push to Server… (copy a vhost to a linked server), and open the site in a browser. Import Sites… and Copy from… pull existing vhosts in from MAMP, XAMPP, Apple’s Server.app or another Apache, bound to localhost by default.

Adding or editing a site, you set the essentials a vhost needs: its ServerName (the domain), the document root (Choose… a folder), and the bind address and port. Enable the site and it appears in the list with a live status dot; FrontierStack writes the vhost into a .conf file under the managed tree, validates, and reloads. To create the vhost, certificate and a Cloudflare DNS record together, use New Site + DNS… — the one-flow wizard described in Chapter 7.

4.5TLS certificates for sites

The Certificates pane issues and manages the certificates that put the padlock on your sites. It offers three paths, suited to different needs:

Once issued, Deploy to Apache wires the certificate into the site’s vhost. The Expiry Monitoring section watches your certificates (and any domain you add with Watch) and alerts you well before they lapse — the deeper hardening story, including HSTS and modern cipher policy, is in Chapter 10.

4.6The .htaccess editor

Per-directory Apache rules — rewrites, redirects, access control — live in .htaccess files, and FrontierStack gives them a dedicated editor with line numbers and a syntax sanity check. Open a site’s document-root .htaccess with Use Site Root, or point it at a custom path. Insert Recipe drops in common, correct snippets (a front-controller rewrite, a force-HTTPS block) so you start from working rules rather than memory, and the editor flags obvious problems before you save. Export PHP Router… turns .htaccess rewrite rules into a router file for a built-in PHP dev server — useful when you move a site to the Localhost pane.

4.7Nginx and reverse proxies

FrontierStack manages Nginx as an alternative web server or a reverse proxy in front of an app. When both are installed, the Domains view groups your sites by server — Apache virtual hosts and Nginx server blocks side by side — reading each Nginx server block’s listen port, root and TLS state. A reverse proxy is the standard pattern for fronting an application that speaks HTTP on a high port (a Node, .NET or Python service): Nginx terminates TLS on 443 and forwards to the app. Because Apache and Nginx both want 80/443 by default, use the Apache pane’s Ports section to move one off those ports if you run them together.

4.8Localhost, DNS and dev stacks

Three more panes complete the local picture:

• Localhost — a manager for ad-hoc local dev servers. Add a New Local Server, pick a Runtime (PHP, Python/Django, Node/Vite…) and folder, set a port and an optional ServerName, and Start it — with per-server Environment Globals and a .test resolver so myapp.test resolves locally. A Free Port helper and a stop whatever is holding localhost action clear the usual conflicts.
• DNS — local name resolution and authoritative hosting. Run dnsmasq, unbound or nsd; import and export zone files; and flush the resolver when records change so the Mac stops serving a stale answer.
• LAMP / DevStacks — one-click stacks. The LAMP pane offers presets (Choose a Setup) that install and start a matched set of components together (Install & Start, Start All, Stop All); DevStacks detects the framework of a project folder and shows the right commands and npm scripts to run it.

An Localhost overview shows every local server you have running, side by side, so you can run many sites at once without a wall of Terminal windows to babysit.

A web stack is only as good as the data underneath it. FrontierStack treats databases as first-class services: it installs and runs them, browses what is inside, lets the AI Administrator inspect and repair a site's database in plain language, and — the part most setups forget — backs everything up on a verified schedule that you can actually restore from. This chapter covers the relational engines, the in-memory caches, the newer vector and analytics stores, and the backup machinery that protects them all.

5.1The MySQL / MariaDB pane

Open MySQL from the Databases section of the sidebar. The pane installs the server through Homebrew if it isn't present, then gives you the controls Server.app used to: Start, Stop and a live status light, the listening port (default 3306), and a list of the databases on the server. MariaDB — the drop-in MySQL-compatible fork — is managed the same way; the pane recognises whichever is installed.

Local development follows one deliberate convention: the local root account has no password. This is the MAMP/XAMPP norm and it is what makes one-click setup and the AI's SQL tools work without a credential prompt. It is safe precisely because MySQL on your Mac listens only on 127.0.0.1 and is reachable only from the machine itself. The moment a database needs to be reached from elsewhere — a passworded account, or a database on a linked server — you switch to the secret vault, described below.

5.2Database Ops and running SQL

The Database Ops pane is where databases stop being a black box. Its Replication section detects whether a configured server is a MySQL replica or a PostgreSQL standby and shows the lag: MySQL reads SHOW REPLICA STATUS (IO and SQL threads, Seconds_Behind_Source); PostgreSQL uses pg_is_in_recovery() and the replay timestamp. Stopped replication, or lag past your threshold, raises an alert on your channels — the basis of the Database Slave monitoring preset.

For inspecting or fixing data, the AI Administrator carries a run_sql tool. Ask it in plain language — "why are these orders showing the wrong total?", "delete the duplicate users older than the originals" — and it runs SQL against the local server and shows you the result as a table, much like MAMP's Claude integration. Because statements can modify data, run_sql requires changes enabled; read-only queries work the moment the assistant is on. It defaults to local root with no password and is local-only by design.

5.3Passworded and remote databases

The local-root shortcut stops exactly where the network begins. For a database that needs a password, or one on a remote or linked server, you never paste the password into a prompt or a tool. Instead you add it once to the AI Administrator's Script secrets (.env) bar as a named secret — DB_PASSWORD or MYSQL_PWD — and the assistant runs the mysql client through run_script, referencing the value as $DB_PASSWORD and running it over SSH against the named server. The real value stays on your Mac and is injected at run time; the cloud model sees the name, never the secret.

5.4PostgreSQL

Open Postgres for projects that want advanced SQL, rich JSON, full-text search or GIS. The pane installs and runs PostgreSQL (default port 5432) and sits alongside MySQL with the same start/stop/status controls. Choose PostgreSQL when your application or framework expects it; otherwise MySQL/MariaDB remains the easy default. Both engines are first-class targets for the backup jobs and AI tools described later, and PostgreSQL is the foundation for several of the specialised stores below — pgvector for embeddings, TimescaleDB for time-series, and PostgREST for an instant REST API over a Postgres database.

5.5Redis and other caches

Sessions, queues, rate limits and hot-path caching usually want an in-memory store rather than a relational table. FrontierStack runs the two standards from the Caching catalog: Redis (port 6379), the in-memory key-value store that also backs Redis Streams, and Memcached (port 11211) for a simple distributed cache. Install, start and monitor them like any other service. Most CMS and framework deployments pair Redis with MySQL or PostgreSQL; the Service Catalog (Chapter 6) lists every option.

5.6Vector and analytics databases

Two newer families of store get their own catalog categories. Vector databases hold embeddings for retrieval-augmented generation (RAG) and semantic search; they are how the AI stack (Chapter 15) gives a model access to your own documents. Analytics databases are columnar or time-series engines built for fast aggregation over large datasets rather than transactional reads and writes.

For purely in-process vector search there is also FAISS (a library, no port). Pick the engine your framework expects; the panes install and run each one through the same catalog machinery.

5.7Data pipelines & ETL

Moving and reshaping data between these stores is its own discipline. The Data Pipelines & ETL catalog category covers it at the overview level: Airbyte and Meltano for extract-load with hundreds of connectors, dbt for SQL-based transformation and modelling, and orchestrators such as Dagster, Prefect and Kestra for scheduling the work. These are full platforms; FrontierStack installs and runs them and surfaces their web UIs, but the pipeline logic lives in each tool. For programmatic data-pipeline DAGs there is also Apache Airflow in the Workflow Orchestration category.

5.8Backups: dumps, the overview pane, and CCC

A database with no tested backup is a future outage. FrontierStack approaches this from two directions. In Database Ops you define a verified dump schedule per server: name, engine (MySQL or PostgreSQL), host, port, user, database (blank = all) and a backup directory. Due jobs run automatically while the app is open, and Run Now triggers one immediately. Each run does more than dump — mysqldump runs with --single-transaction for a consistent InnoDB snapshot, then Verify checks gzip integrity and that the dump ends with its completion marker (catching truncated or half-written files), writes an .md5 sidecar, rotates to your keep count, and rsyncs the result offsite over key-based SSH.

The AI Administrator exposes the same operations as tools: db_dump dumps a database to a .sql file on the target (using mysqldump or pg_dump), and backup_now makes a timestamped tar.gz of configs, a site or a data folder. Both are mutating, so they require approval and changes enabled; for the DB password they reference a $NAME vault secret, never a pasted value.

The Backups overview pane rolls up every mechanism so staleness is obvious at a glance: Time Machine status, Carbon Copy Cloner tasks, app-managed Apache config snapshots, config version history in Git, and scheduled auto-exports. The Carbon Copy Cloner pane reads CCC's tasks and history through its ccc command-line tool and lets you Run a task on demand. Both Time Machine and CCC feed the Alerts engine — a backup that hasn't run within its threshold, or a failed CCC task, raises an alert on your channels (Chapter 11).

5.9Restore — and restore safety

Backups exist to be restored, and restore is the dangerous half. The AI's restore tool extracts a .tar.gz archive or imports a .sql dump into a database. Because importing a dump overwrites the live database and extracting an archive overwrites files in place, restore is mutating and potentially destructive: it always requires explicit approval with changes enabled, and SQL imports reference the DB password as a $NAME vault secret.

5.10Recovering a broken database

Sometimes a database is worse than down — it won't start, or a table is corrupt. FrontierStack gives the AI Administrator (Chapter 13) the same tools you would reach for by hand, behind the usual approval card. mysql_check_tables and pg_check_tables run mysqlcheck / pg_amcheck to find corrupt or crashed tables; with repair on they take a mysqldump/pg_dump backup first, then repair (--auto-repair / REINDEX), so a fix is reversible. mysql_service_control and pg_service_control start, stop, restart or check the server across systemd, launchd, Homebrew or service(8), local or over SSH.

For the hard cases there are guided skills — “Recover Broken MySQL” and “Recover Broken PostgreSQL” — that walk the whole runbook: read the error log, diagnose the fault, dump everything readable, rebuild into a fresh data directory, repair or recreate the damaged tables, and restart — always renaming the old data directory aside as a cold backup, never deleting it, and confirming before every destructive step. On a remote host, the fastest way in is the red Emergency button on the database's row in the server pane: it opens the assistant pre-loaded with the right recovery skill for that host.

5.11ORMs and app-native tooling: Drizzle and Convex

Finally, two tools meet your application where it defines its schema. The Drizzle ORM pane detects a Drizzle project (its tables, migrations and backend dialect) and runs drizzle-kit tasks for you — Generate SQL from your schema, Migrate, Push the schema directly, Pull (introspect) an existing database, and open Drizzle Studio to browse data. The Convex pane manages Convex, a reactive backend with a built-in database: run npx convex dev to watch and push functions, or stand up a self-hosted backend in Docker. Both need Node.js, which the panes will install via Homebrew if it is missing. These cover the application's own data layer; the dumps and backups above still protect whatever they write underneath.

FrontierStack ships a catalog of 1545+ services across 112 categories. Every entry can be discovered, installed, started, configured and monitored from the app — and the AI Administrator can act on any of them. This chapter is the field guide: each category, what lives in it, and a one-line description of every service. Service descriptions are kept in English so they match the labels in the app.

Each service also has its own page on the website at frontierstack.app/services/ with ports, default web UI and the Homebrew formula.

6.1Overview

6.2Setup & Migration

6.3Locations & Places

6.4Fleet & Remote

6.5Networking

6.6VPN

6.7Routers & Firewalls

6.8Web Servers

6.9DNS

6.10Runtimes

6.11Databases

6.12Vector Databases

6.13Analytics Databases

6.14Data Pipelines & ETL

6.15Caching

6.16Search

6.17Backup

6.18Storage & NAS

6.19Object Storage & S3-Compatible

6.20Mail

6.21Mailing Lists

6.22Email Deliverability & Reputation

6.23Message Queues & Streaming

6.24Push Services

6.25Chat & Collaboration

6.26SMS & iMessage Gateways

6.27Business Messaging

6.28Calendars

6.29Identity & Accounts

6.30Customer Identity

6.31Decentralized Identity

6.32Version Control

6.33CI/CD

6.34Build & Compile

6.35Internal Developer Portals

6.36Infrastructure as Code

6.37API Testing & Synthetic Monitoring

6.38Proxies & Load Balancers

6.39Proxy Gateways

6.40CDN & Edge

6.41Hosting

6.42AWS Control

6.43FinOps & Cloud Cost

6.44Containers

6.45Service Mesh (Kubernetes)

6.46Apps & CMS

6.47Knowledge & Memory

6.48E-Commerce

6.49QR & Link Management

6.50Student Information Systems

6.51Learning Management

6.52Digital Library

6.53Campus & School Ops

6.54Sourcing & Suppliers

6.55Print-on-Demand

6.56Shipping & Fulfilment

6.57Inventory Sync

6.58Profit Analytics

6.59Mesh Networking

6.60Anonymity Networks

6.61Secrets & Vaults

6.62Monitoring

6.63Security Tools

6.64Secrets Scanning & Supply Chain Security

6.65Policy, Compliance & Governance

6.66MDM & Device Management

6.67Logging & Observability

6.68Incident Management & On-Call

6.69Status Pages

6.70Mining

6.71AI / LLMs

6.72AI Clusters

6.73Agent Platforms

6.74ML Workbench

6.75AI Governance & Safety

6.76GPU Infrastructure

6.77Speech AI

6.78Image & Video AI

6.79MCP

6.80Self-Hosted Apps

6.81Web3

6.82Crypto

6.83RPC Services

6.84Tools

6.85Home Automation

6.86Cameras

6.87IoT

6.88MQTT Brokers

6.89POS Systems

6.90CRM & Loyalty

6.91Helpdesk & Ticketing

6.92IT Assets

6.93Project Management

6.94PMS

6.95Meeting Rooms & Desk Booking

6.96Coworking & Shared Spaces

6.97Facilities & Maintenance

6.98Accounting & ERP

6.99Healthcare

6.100Automation & Workflows

6.101Workflow Orchestration

6.102HRIS Integration

6.103VoIP & Telephony

6.104Contact Centre & IVR

6.105SEO & Marketing

6.106Conversion Rate Optimization

6.107Social Media

6.108Media Servers

6.109Download Automation

6.110AV & Stage

6.111Render Farm

6.112Tracking & Telematics

A working website is more than a running Apache. The domain has to resolve, the certificate has to be valid, the right machine has to answer, the CDN has to be in front of it, and the search engines have to know it exists. FrontierStack gathers all of that into the Sites pane and a small family of companion tools, so the whole chain — registrar, DNS, TLS, CDN, host, SEO — is visible and fixable from one window. This chapter covers the web above the stack; for the vhosts and certificates themselves see Chapter 4.

7.1The Sites pane and Domain Health

Open Sites from the sidebar. At the top sits the Domain Health section, a live dashboard that collects every hostname FrontierStack knows about: the local Apache and Nginx vhosts you have defined, the vhosts on linked servers, every Cloudflare zone on your token, and every domain in the registrar monitor. Each domain gets a row with a set of status badges; press Check on a row, or Check All in the header, and the app probes the domain end to end and colours the badges green or amber.

The point of the dashboard is to answer one question reliably: is this domain actually serving, and from where? Because the checks run against the public internet (a real DNS lookup, a real HTTP request, a real TLS handshake), they catch the failures a local config check never sees — a record that points at the wrong IP, an expired certificate, a Cloudflare zone stuck in "pending", a site that resolves but returns a 502.

7.2Proving domain ownership

Vendors verify that a domain is yours by asking you to publish a DNS TXT record only its owner could create — the same proof-by-DNS idea as ACME's DNS-01 challenge, but every vendor has its own record format. The Verify button on a Domain Health row keeps those formats in one place: pick the service — Google (Search Console / Workspace), Microsoft 365, Meta, Apple Business Manager, Atlassian, OpenAI, Stripe, GitHub organizations, or a custom record — paste the token from the vendor's console, and FrontierStack publishes the TXT record straight into the domain's Cloudflare zone. If the domain isn't on your Cloudflare token, it hands you the exact record to add at your DNS host instead. A bare token is completed automatically (paste abc123 for Google and the record becomes google-site-verification=abc123).

The sheet also shows the verification records already published on the apex — a quick audit of who this domain is currently proven to — and Check DNS queries public DNS (1.1.1.1) for the new record, so you know the proof is visible before you press "Verify" on the vendor's side.

For certificates, the Let's Encrypt service pane (Security & Secrets category) is a standing health check on the other kind of domain proof: it watches the ACME API, Let's Encrypt's status page and a domain's live certificate, and — because ACME clients renew when 30 days remain — alerts when a certificate is overdue for renewal, catching broken certbot/acme.sh automation weeks before visitors ever see an expiry warning.

7.3AI SEO and "Fix My Site"

Two of the badges on a domain row are not just diagnostics — they are buttons that hand the problem to the AI Administrator. Fix My Site takes a broken or misbehaving site and runs the assistant through a fixed playbook: reproduce the fault by curling the site locally with its Host header, read the relevant logs, check that Apache, PHP and the database are running, and then state the single most likely root cause with concrete next steps. With Allow changes off it only advises; with it on it may apply the reversible fixes (restart a service, correct a wrong value in a config file, repair the database). There is a WordPress-aware variant for the classic white screen, "Error establishing a database connection", broken permalinks and stuck maintenance mode.

The SEO action runs an on-device read-only audit: the assistant fetches the home page, robots.txt and the sitemap, scores the site out of 100 and writes a short report (titles, meta descriptions, headings, crawlability, structured data). The result opens in an SEO report sheet so you can read the findings and act on them.

7.4One-click CMS installs (Apps & CMS)

Installing a CMS is only the first day of its life; WordPress Sites covers the rest. Choose a host — This Mac or any fleet server — and scan for wp-config.php under the usual web roots, and every WordPress install on that machine appears with its core, plugin and theme versions and whatever is waiting to be updated. Update all exports the database before it touches anything, because a bad plugin update is recoverable and a lost database is not. Check integrity compares the core files against the checksums WordPress.org publishes, which is the quickest way to find a defaced or backdoored install; add a free WPScan token and the pane also checks your installed plugins and core against the vulnerability database, marking anything with no published fix. Multisite networks list their subsites. The remaining jobs are the ones that otherwise mean an SSH session and a half-remembered command: resetting the password of a locked-out administrator, search-replace to move a site between domains (WP-CLI rewrites URLs buried in serialized PHP, where a plain SQL replace corrupts them — so always dry-run first), cloning a site to staging, and exporting or optimising the database. A clone copies the files, gives the copy its own database, rewrites the URLs inside it and sets it to discourage search engines, so staging can never write to production or outrank it. Everything runs through WP-CLI over SSH, and the pane offers to install WP-CLI on any host that lacks it. Pin a site and Alerts will tell you about pending updates, a failed integrity check or a new vulnerability.

If your WordPress lives on managed hosting rather than your own machines, the hosting panes now list those sites too: WP Engine, Kinsta, Cloudways and Pressable each pull your installs through their API — environment, primary domain and status — and they roll up into Cloud Servers alongside your VPS instances, so one inventory covers everything you run. These are sites rather than machines, so they carry no power controls.

The Apps & CMS section installs a self-hosted CMS onto your Apache/PHP/MySQL stack in a single flow, each with its own pane: WordPress, Drupal, Joomla, Statamic, Grav, Kirby and Craft CMS. The installer downloads the app (or runs composer create-project for the Composer-based Statamic and Craft), extracts it into the document root, and — for the ones that need a database (WordPress, Drupal, Joomla, Craft) — creates the database and user and writes the config where the app expects it. The flat-file CMS (Grav, Kirby, Statamic) need no database at all. It then opens the app's setup or admin page so you can create your account (/wp-admin/install.php, /admin, /panel, /cp, /admin/install…), while FrontierStack wires the virtual host, PHP and trusted HTTPS around it. For the install folder you can pick a site Apache already manages — a menu prefills its domain, document root and port — or Choose… a new empty folder; a matching Apache vhost is created for a new folder, or updated when the domain is an existing site. If an install misbehaves, Fix My Site (AI) above debugs it.

7.5The Domain Registrar monitor

Sites tells you whether a domain works today; the Domain Registrars pane (in the DNS category) tells you whether it will keep working. Add a domain and FrontierStack watches three things on a schedule and raises an alert when any of them drifts:

• Expiry — the registration end date, registrar and nameservers, pulled over RDAP (with a whois fallback). Works for any domain, whatever the registrar.
• DNS — whether the domain still resolves (dig).
• SSL — the certificate expiry, checked directly with openssl.

A renewal warning fires when the expiry is within your reminder window, already past, or when auto-renew is off. For supported registrars the pane can import your domain list and auto-renew status by API — GoDaddy, Porkbun, Gandi, Value Domain, Route 53 (via the AWS CLI) and Cloudflare (reusing the Cloudflare token). Registrars without a usable list API (Namecheap, Squarespace, Onamae) you add by hand; RDAP still monitors them.

7.6Cloudflare: zones, DNS, cache and tunnels

The Cloudflare pane is where you paste an API token and choose which zones appear in the sidebar. Each zone you Pin to sidebar becomes its own pane with that zone's Cloudflare/DNS/HTTP/SSL health, DDoS and bot protection, WAF rules, Workers, CDN analytics, development mode and cache purging (everything, a list of URLs, or by prefix). Scope the token carefully: a read-only token lists zones, but DNS edits, certificate DNS-01 validation, Dev Mode, cache purge and zone creation each need their own permission, and creating a brand-new zone needs the account-level Zone · Zone · Edit permission. The pane links straight to Cloudflare's token templates, a Test Token button probes the live API and lists exactly which of those abilities the token has or lacks (reads are exercised directly; writes are probed with deliberately-invalid requests, so nothing is changed), and a Dashboard button deep-links into the Cloudflare web console for the current zone.

When a pinned zone shows a warning or its HTTP check has stopped answering, the pane's Diagnose button finds out why instead of leaving you with a coloured dot. It probes each layer of the serving path in order — the zone's state at Cloudflare, the public nameservers, DNS resolution, HTTPS through the edge (decoding Cloudflare's 52x errors: 521 origin down, 522 timeout, 524 the app or its database hanging, 525/526 origin TLS), the origin server's ports 80 and 443 directly, and finally the machine FrontierStack knows serves the domain: the local web server if that is this Mac, or a linked server's on-host monitor service states (a stopped mysql surfaces right in the report). When the public path is broken it also sweeps the fleet: which linked servers are up, which binds the origin IP, and which already serves this domain's vhost — so a host that went offline and took the sites' front-door IP with it is named directly, together with the server that could take over. Each layer is shown pass/fail, ending with a plain-language conclusion — which layer broke and what to do about it.

Cloudflare turns up across the app: the AI's dns_record tool writes records into a zone you manage and updates existing ones in place — including flipping a subdomain between proxied (orange cloud) and DNS-only — issue_certificate can validate over Cloudflare DNS-01, and email_auth_dns can publish SPF/DMARC for you (see Chapter 13). The same account also powers Cloudflare DDNS and the Cloudflare Tunnel for exposing a local site — both covered in Chapter 9.

7.7Hosting providers

Not everything you run lives on a Mac. The Hosting category brings VPS and managed providers into the same window. VPS providers — DigitalOcean, Vultr, Linode, Hetzner, Sakura and others — expose their servers through an API token: paste it, and the pane lists each server with plan, location, status and IP, and lets you power one on, off or restart with a confirmation. Dashboard-style providers (managed hosting such as Hostinger) store their token and surface quick links to the control panel, docs and sign-up. Either way the credential lives in the Keychain, and the servers you can reach over SSH become part of your fleet (Chapter 8).

7.8IndexNow: instant search-engine submission

When a page changes, you do not have to wait for a crawler to notice. IndexNow Manager (in SEO & Marketing) tells the participating engines — Bing, Yandex, Seznam, Naver and others — immediately. Pick one of your managed sites, press Install Key File, and FrontierStack writes the <key>.txt file into the site's document root so the engines can confirm you own the key; Verify checks it is being served. Then submit a single URL, fire off the pending queue, or feed it from a sitemap, and it can auto-submit on a schedule. Everything goes to api.indexnow.org, which fans out to all participating engines; the URL's host must match the selected site and its key.

7.9Site Transfer: copying sites between machines

Two tools move a site without a manual rsync dance. Copy Websites Between Apache handles the local case: when you have more than one Apache install on the Mac (say a Homebrew one and an old Server.app one), choose the source install, tick the sites you want, and FrontierStack copies their document roots and vhost definitions into the currently selected Apache. Sites with the same domain are overwritten, so it is also how you re-import after a migration.

To move a site to another machine, the AI's promote_site tool clones a local site to one named fleet host: it uploads the document root, writes the vhost (Apache or Nginx) on the far side, reloads the web server, and can add the Cloudflare A record pointing at the new host — the same steps as a manual fleet push, in one call.

7.10Provisioning a new web project

The pieces above combine into a single flow for standing up a project's infrastructure from nothing. The idea is a clean division of labour: your AI tool (over MCP) owns the code; FrontierStack owns the infrastructure. The built-in Provision Web Project skill walks the assistant through the recipe in order:

1. Discover — inventory the target, its privilege access, installed Apache/Nginx and connected DNS provider; identify or install missing prerequisites before making a public change.
2. Zone — cloudflare_zone_create makes the Cloudflare zone and returns the nameservers to set at the registrar (and tells you plainly if the token lacks account-level zone-create rights).
3. Vhost + folder — vhost_create defines the Apache/Nginx site. If you did not provide a document root, FrontierStack creates an empty platform-appropriate folder and reports its path. It does not design or generate site content.
4. DNS — dns_record points the name at the host.
5. TLS — issue_certificate issues a Let's Encrypt certificate.
6. Preview — a temporary share for a localhost test point (Chapter 9).
7. Promote — optionally promote_site clones it onto a fleet host.

Domain registration can join the front of this recipe through domain_register, so an agent can in principle buy the name, create the zone, stand up the vhost, issue the certificate and clone to the fleet — with every money-spending or system-changing step previewed for your approval first.

Everything so far has been about the Mac in front of you. This part of the manual widens the lens to the whole estate: the Linux boxes in a rack, a vintage Xserve still running macOS Server, a Raspberry Pi on a shelf, a cloud VPS, even a Windows host. You link each one once over SSH, and from then on FrontierStack treats them as a single fleet — running diagnostics, fanning commands out, deploying sites, and watching their health. It also keeps track of where this Mac is on the network, so a laptop that moves between home, office and a café doesn't fire false alarms about servers it simply can't see right now.

This chapter covers linking and running the fleet. Chapter 9 goes deeper on the network and perimeter, Chapter 10 on hardening and the security audits, and Chapter 11 on continuous monitoring and alerts. The AI Administrator can drive almost everything here over the same tools — see Chapter 13.

8.1Locations & Places

Open Locations & Places ▸ Locations. A location is simply a named network — Home, Office, On VPN — recognised from several signals: the Wi-Fi SSID, the router (gateway) MAC address, the Wi-Fi BSSID, the subnet prefix of your IP address (e.g. 192.168.1.), your public IP block, and whether a VPN is up. The header shows where you are right now (Now: Office), with the live snapshot beneath it: Wi-Fi name, IP, gateway and VPN state. Detection runs every 30 seconds.

When FrontierStack sees a network it doesn't recognise it offers a New network detected banner — click Add… to save it. You can also press Add Current Network as Location, then refine the criteria: a location matches when all the set criteria hold (with SSID and router MAC treated as any-of), so leaving a field blank simply ignores it. When several locations match, the most specific one wins rather than the first in the list. Use the Mode picker to pin a location manually instead of auto-detecting — handy when the signals are ambiguous.

Not all signals are equally trustworthy, and this matters more than it sounds. A subnet prefix is not an identity: home routers, cafés and hotels hand out 192.168.1.× everywhere, so a location defined only by its subnet would happily match a completely different building. The router MAC is the strongest signal — it is unique to one piece of hardware and readable without any special permission — which is why Update all criteria from current network is the one-click fix for a location that was saved before router matching existed.

8.12.1Pinning a place with GPS

For the awkward case — two sites that genuinely share a subnet and you would rather not fuss with MAC addresses — a location can also carry a GPS pin. Stand in the place, open the location and press Pin This Spot (GPS); a Radius slider sets how far counts as "here" (150 m by default, because indoor positioning is coarse and a site is a building, not a point).

The pin is a cross-reference, not a requirement. It does two things: it lets a weak match be confirmed, and — more usefully — it lets a wrong one be vetoed, so Home stops claiming a café that happens to use the same subnet. To keep that from ever making detection worse, a location identified only by shared criteria (a subnet, a VPN toggle) is no longer reported as confirmed when a position is available and contradicts it; you get Away / Unknown instead of a confident wrong answer.

Devices belong to Locations too. In Device Discovery, a monitored device's ⋯ menu ▸ Assign to Location places it at a specific site — or leave it Automatic, where it falls into whichever location owns its subnet. You can also set it from the device's own pane, and each location's row lists the devices assigned to it. When the Mac arrives at a recognised location, FrontierStack checks those devices (skipping any seen recently, e.g. over VPN) and shows each one's status: green reachable, red if it should be here but isn't answering, and orange when it's on a Wi-Fi segment this Mac can't reach — usually a different access point or band, such as a 2.4 GHz-only sensor while the Mac is on 5 GHz. Orange means “can't verify from here”, not “gone”.

The point of all this is the Location-Dependent Monitoring section. Mark a device, Wi-Fi network, Cloudflare zone or watch as critical in Overview ▸ Alerts, then scope it here to Only at Office. That check is paused everywhere else: your NAS won't ring as DOWN from a hotel, and your home Wi-Fi won't alert as lost while you're at work. Paused items resume silently when you return. When the location itself changes, FrontierStack can post a local notification, send a "📍 Now at: Office" alert through your Messaging Gateways, or both.

8.2Linking a server over SSH

A server joins the fleet through the Link a Server sheet — reachable from a device's pane, from Device Discovery, or from the Cloud Servers and Remote Tools panes. Enter a name, the host or IP, the port, an SSH username (root, ubuntu, ec2-user…) and the login password. That password is used exactly once: the app installs this Mac's managed SSH key into the server's authorized_keys, then connects key-only from then on. Nothing is stored except the server's address — the password is discarded.

For a key-only server with no password login, expand Can't use a password? Install the key manually, copy the shown public key into the server's authorized_keys yourself, then link. Tick Also install the monitoring helper to set up the host monitor in the same step (covered below).

Linked servers carry a small green or red status dot for reachability. If it goes red with Permission denied (publickey) — typically after a server is reinstalled, or the key was never installed — the AI tool repair_ssh_access clears the stale host key and re-installs the managed key, using a one-time login password you keep as a vault secret. The password is read locally and re-installs key access; on success the dot turns green again.

8.3The managed key and the saved sudo password

Key access lets the app connect, but many useful operations — reading /var/log, editing Apache config, reloading a firewall — need root on the server. FrontierStack handles this with a per-host sudo password, saved in the macOS Keychain (never in plain files). It is captured automatically when you link a server, and you can set or change it later in the host's pane or in the Remote Tools target section (the Sudo password field).

When a root-only command runs, the helper primes sudo's credential cache with that saved password over the SSH session, then runs the command. If no password is saved, root operations fall back to passwordless sudo -n and simply report "needs sudo" when that isn't configured. Because secrets are referenced by name, you can keep several — for example a distinct MySQL password per server in the .env vault.

8.4Fleet Run — one command across every node

Fleet & Remote ▸ Fleet Run fans a single operation out across all linked servers at once, or a selected subset, with a live per-host result. Pick your Targets (toggle hosts; the reachability dot shows which are up), choose an Operation, and press Run on N Server(s) — everything runs concurrently over SSH and each host reports back below, expandable to its full output.

Operations are OS-aware: a package update maps to apt, dnf, pacman, zypper, apk or brew depending on the host; a service restart tries systemd, then Homebrew. You never write per-OS commands by hand.

Fleet Run is the fastest way to spot drift: run Disk usage or Uptime & load to compare hosts at a glance, or use the file-checksum and health-check tools in Remote Tools (next) to find a config that's diverged across the fleet.

8.5Remote Tools — per-node diagnostics

Fleet & Remote ▸ Remote Tools is a suite of SSH-driven diagnostics, distilled from real fleet-debugging work. Choose the target first — a pinned server, or Other (IP / host)… for an ad-hoc address that isn't a saved server (its sudo password persists in the Keychain too) — then the tool and its parameters. Read-only checks need no root; ones that read protected logs or config use the saved sudo password through the helper, and a few are marked mutating.

Some tools work across several selected nodes at once — File Diff, Healthcheck, TLS Expiry and Serving Path Diagnosis (who's up, which public IPs each node binds, per-process CPU saturation and stopped services, cross-checked against every pinned Cloudflare zone's origin IP) are fleet-wide by design — which is exactly how you confirm every web node is serving the same content with a valid certificate. The bottom of the pane also offers safe over-SSH editing of /etc/hosts (root, through the helper) and the SSH user's ~/.ssh/known_hosts, each backed up to .fsbak first.

8.6Recovering a wedged server — and a safe power reset without SSH

A server can stop answering SSH while its services keep running — overloaded, not down. FrontierStack shows this: when SSH is unreachable but a service port still answers (say MySQL on 3306), the server's dot turns orange, not red, and its pane reads "SSH is wedged, but the server is still serving" and lists what's answering, with the database's version banner. The machine is alive — only its management channel is stuck — so don't assume it's dead or blindly cut the power.

Try a normal Reboot first. After Reboot, the dot flashes while FrontierStack waits out a grace period and then polls, refreshing the instant the host returns — no need to keep pressing Refresh.

Make a power reset safe — without SSH. When a database is still answering, the pane shows a red-outlined EMERGENCY box, Prepare databases for a safe power reset. Each step runs over the database's own connection (no SSH), using the credentials saved in Database Health, top to bottom: Stop accepting writes (reads still work, reversible), Quiesce & flush to disk, then Clean-shutdown. After a clean shutdown a hard power-cycle is safe, because nothing dirty is left in memory to corrupt. Coverage differs by engine — MySQL, Redis and MongoDB can be cleanly stopped over their protocol; PostgreSQL can be quiesced and flushed but not stopped over SQL, so you flush then power-cycle. The clean-shutdown button is red and confirms first.

Power-cycle, then bring it back. The hardware power controls (SwitchBot plug, smart powerboard/PDU, KVM) each offer Shut Down Databases First in their confirmation, so you can flush and stop the databases before cutting power. And if the shutdown relieved the pressure and SSH recovers, a Start button starts the database again over SSH — so the ideal path is often: clean-shutdown the runaway database, let the box recover, and Start it, with no power-cycle at all.

8.7The Host Monitor (remote agent)

SSH diagnostics run on demand. For continuous visibility, install the Host Monitor — a tiny, read-only Go helper (fsagent) that lives on the server and reports its own health, with no per-cycle SSH. From a linked host's pane, choose to install it (or tick the box in Link a Server); over the existing key the app pushes the right binary for the host's OS and architecture, sets up the platform service (systemd, launchd or BSD rc.d), and pins its TLS certificate. The AI tool install_monitor does the same on request.

Once enrolled, the monitor streams live metrics: CPU, memory, per-mount disk, network interfaces up, detected services, firewall and fail2ban state — and on GPU hosts, per-GPU temperature and utilisation, so a mining or ML rig surfaces thermals. list_monitors reports the fleet's helpers, their versions and latest metrics. Crucially, the monitor keeps watching and can alert even when the Mac app is offline, notifying directly through its own channels and replaying anything it buffered when the app returns. Logs windows prefer the monitor too — it runs as root on the server, so it reads privileged logs without needing the sudo password at all.

An enrolled host's pane also keeps a Behavior & History chart — CPU, memory and disk at five-minute resolution for up to 30 days, against a learned baseline that flags unusual behaviour. Vertical lines place incidents on the same timeline: a solid red line is a server reboot (derived from the reported uptime), a dashed orange line is a watchdog issue — a forced service restart, a database-corruption warning or a reboot escalation. If the OS crash reporter starts using CPU (a process crashing repeatedly), an orange Crashes curve joins the chart, so "MySQL was force-restarted twice last night and something was crash-looping" is visible at a glance.

The monitor is read-only by default. Turning on Allow actions in its Host Monitor pane unlocks a small, fixed set of control verbs — never arbitrary shell — driven by the gated monitor_action tool: restart/reload/start/stop an allowlisted service, flush DNS, reload the firewall or fail2ban, or reboot. Each action is logged on the server. Targets cover Linux (amd64/arm64/arm — covering every Raspberry Pi), macOS (including legacy Intel builds back to OS X 10.11 El Capitan), FreeBSD (pfSense/OPNsense/TrueNAS) and Windows.

Keeping a monitor healthy over time is handled for you. Rotate monitor credential replaces its bearer credential in place: the new secret is generated on the Mac, sent over your approved FS1 signing key, and activated on the server without ever being shown or handed to an AI. With Allow self-update enabled, pushing a newer build is safe by construction — the candidate binary's release signature is verified, it must pass a compatibility self-test before it is given root, and the previous binary is kept as a rollback; if the new one fails to answer its health check, FrontierStack restores the old one automatically. Pull-mode updates fetch the binary through a short-lived, single-use grant rather than a durable token. When a monitor is verifying FS1-signed requests, its pane shows a FS1 signed badge.

8.8Cloud Servers, Server Clone & Remote Apache

Alongside the compute panes, AWS ▸ DynamoDB lists the tables in the selected region with their status, item count, size and partition/sort key. It leans on the two things that actually cost you: the capacity mode capsule separates provisioned tables — which bill for their read and write units continuously, whether or not anything touches them — from on-demand tables that bill per request; and a no PITR flag marks any table without point-in-time recovery, which is off by default on every new table and is the only way to rewind after a bad write. Point-in-time recovery can be switched on or off from each row's menu. DynamoDB has no bulk describe call, so each table is queried individually and the list is capped at 40 per region.

Fleet & Remote ▸ Cloud Servers rolls up one inventory across every cloud provider that has a configured API token — DigitalOcean, Vultr, Linode, Hetzner, Sakura, Contabo — plus AWS EC2, Lightsail and RDS. It is a read-only roll-up (power control stays in each provider's own pane). Each running instance with an IP offers a Services button to discover what's listening, and a + Helper button that links the instance over SSH and installs the monitor in one move — just set the right username and key in the sheet.

Server Clone is a guided wizard that stands a fleet server up to match this Mac: it installs the detected Homebrew stack, copies configs and site files to their original paths, pushes your domain configs and reloads the web server, and can clone MySQL databases (local mysqldump piped into mysql on the target over SSH). Every step runs over key-based SSH and is safe to re-run; existing data isn't deleted.

When a single site is ready to go live, the Promote to Production push (from a domain's context menu) sends its vhost config to the same path it uses locally, optionally copies the document root and database, points DNS at the target via Cloudflare, and can issue a real Let's Encrypt certificate with certbot — files, database, vhost, DNS and HTTPS in one push.

You can also manage a remote server's Apache directly. The Apache pane gains a host selector (This Mac / each linked server) when any remote host exists; pick a server and FrontierStack discovers its Apache over SSH — version, config layout, and every active vhost with its real log paths — and lets you edit vhosts, modules, MIME types, ports and WebDAV, with each change validated (httpd -t) and gracefully reloaded before it sticks, rolling back if the config is invalid. It even understands the legacy macOS Server.app Apache tree. From the Sites list, Push to Server… renders a local site's vhost onto a monitor-linked server with its document root swapped in. Chapter 7 covers the local Apache and Sites workflow these mirror.

8.9KVM-over-IP: console and power, lights-out

SSH and Screen Sharing need the machine to be up and on the network. When it isn't — a kernel panic, a BIOS/firmware screen, a network stack that never came up — you need out-of-band access: a KVM-over-IP appliance that captures the real HDMI output and injects USB keyboard/mouse, independent of the operating system. The Remote KVM pane registers as many of these as you own — PiKVM, JetKVM, TinyPilot, NanoKVM, GL.iNet Comet, or any generic unit — each pinnable, each one click from its web console down to the BIOS.

Units with power control close the loop. A PiKVM with an ATX board, a network PDU, or GL.iNet's Comet Pro (GL-RM10) — which is a KVM and a smart power plug in one device — can switch mains power, so you can hard power-cycle a truly frozen box. Link such a unit as a server's power source in its Power section (beside SwitchBot, UPS and PDU commands) for On/Off/Cycle. The AI Administrator can drive it too: list_kvms, kvm_power, and server_power — the last reaches a host's linked plug/PDU/KVM to restart a hung server that won't answer reboot_host.

8.10Old Macs, Raspberry Pis & Windows hosts

Part of the point of a fleet is keeping older or odd hardware useful. A vintage Xserve or Mac mini still running macOS Server links like any other host; the Remote Apache discovery understands Server.app's own Apache tree and its internal ports, so its websites show up and can be edited, and the legacy Intel builds of the monitor run as far back as OS X 10.11 El Capitan. A shelf of Raspberry Pis links as ordinary Linux boxes — the monitor's arm build covers every Pi including the Zero — so you watch their CPU, disk and services right alongside everything else.

Windows hosts join over Microsoft's OpenSSH server: link them like a Linux box for SSH diagnostics, and install the monitor as a native Windows Service (because Windows services don't reliably inherit machine environment, its config is read from a file beside the executable). Windows monitor installs are done manually with the supplied PowerShell installer from the Host Monitor pane, rather than auto-pushed.

For interactive control beyond diagnostics, the catalog also carries the remote-access tools — RustDesk, MeshCentral, Apache Guacamole, Windows Remote Desktop and WinRM — but those are services you run, not the SSH fleet itself. The fleet's strength is uniform, scriptable, low-overhead management of everything from one window — and, with Chapter 13's run_script targeting any node by name, an AI that can diagnose and fix a remote server two steps at a time: a read-only test, then a minimal fix, re-run to confirm.

8.11Migration wizards: moving a stack onto a new device

Retiring an old server rarely means "reinstall everything by hand." FrontierStack ships guided migration wizards that lift a stack off a legacy machine and stand it up on a destination you choose — this Mac, another Mac, or a Linux box. The source is only ever read; everything is written to the target.

The Migrate Setups pane handles a MAMP, XAMPP or Apple Server.app web stack. It auto-detects the source, then lets you tick exactly what to carry: the Homebrew tools, web files, MySQL databases, the sites (vhosts), MIME overrides, Apache modules, PHP settings, WordPress wp-config.php fix-ups and Git repositories. A Set up on picker chooses the destination: leave it on This Mac for the full in-app migration, or pick a linked host to install the stack on that new device over SSH, copying the web files (rsync) and databases (dump over SSH) across — with a Linux toggle to use apt/dnf instead of Homebrew.

The Apple Server Migration pane is a one-click wizard for a whole Server.app machine. Point it at the source (this Mac or a remote one) and it inventories every service with serveradmin — Websites, Mail, Calendar and Contacts, Messages (XMPP), VPN, DNS, DHCP, NetInstall, Open Directory, File Sharing, Time Machine, Profile Manager and more — and anything else it finds running, such as a git server, databases, Docker or a standalone Nginx. Each service shows a status badge and a modern equivalent; where there's more than one sensible replacement (Calendar → Radicale / SOGo / Baïkal, VPN → WireGuard / strongSwan, DHCP → dnsmasq / Kea) you choose which one. Pick a target machine and OS, and the wizard installs the equivalents and prints a precise carry-over checklist. Services with dedicated importers — Websites (full vhost + file copy) and Open Directory (users and groups) — hand off to their own panes; a detected git server hands off to the Git Server migration.

8.12The Git Server: Gitea & repository migration

The Git Server pane stands up a self-hosted Gitea (behind Caddy for automatic HTTPS) on any target — this Mac, a linked server, a NAS, a Docker host or a Linux box — detecting the platform and installing the right way. Once it's running, connect with an access token for a live monitor: version, repository count and backup-staleness alerting when a mirror falls behind its source.

The same pane migrates or mirrors another Git server into Gitea. Point it at on-disk bare repos — the legacy macOS app Simple Git Server, Xcode Server, Gitolite, a plain git-daemon, or any folder — or at a running remote server via any git://, https:// or ssh:// clone base (enumerate the repo names over SSH, or paste them). Scan lists the repositories; Migrate mirror-clones each one with full history, every branch and tag, and pushes it into Gitea, creating the destination repo through the API. The copy runs on this Mac.

A Mac that runs services lives inside a network, and that network is where most real-world trouble starts: a router whose WAN link flapped, a forgotten device on the wrong subnet, a port you opened months ago and never closed. FrontierStack treats the network and its perimeter — the boundary between your LAN and the internet — as first-class objects. It reads your router directly, scans the LAN to build an inventory, draws a data-flow map, and gives you several disciplined ways to expose a service to the outside.

This chapter covers the network around your servers. For linking the servers themselves over SSH see Chapter 8; for the host firewall, fail2ban and the security audit see Chapter 10; and for Cloudflare DNS and TLS see Chapter 7.

9.1Routers & firewalls you can read and drive

FrontierStack talks to network gear through its own management API where one exists, and over SSH where it does not. The two tightest integrations are OPNsense (its full REST API) and Cloudflare (covered in Chapter 7); a wider set of routers and firewalls report live status. Add a device in the Router & Network pane with its address, vendor and API key, and the AI Administrator's router_info tool can then read its version and model, uptime, CPU and memory, WAN links and gateways (up or down), interfaces and client count — plus the raw API JSON for any field FrontierStack does not map. It is read-only: it queries the device's own API, never SSH.

For OPNsense and pfSense, an API key unlocks a continuous firewall monitor (the toggle in the pinned device's pane): every Alerts sweep polls the firewall itself for gateway state, interface carrier and service liveness — so a WAN line that dies while a multi-WAN router silently fails over (servers browse fine, inbound to your sites is dead) raises an alert naming the gateway and the port that lost carrier, instead of an unexplained sea of red site dots. Nothing is installed on the router: OPNsense's REST API is built in, and pfSense needs only its REST API v2 package. The same pane can also join an OPNsense router to your tailnet: sign in to Tailscale on your Mac (Google accounts work), mint a pre-auth key, paste it in the device pane's Tailscale section — FrontierStack installs the os-tailscale plugin if needed, saves the key and login server (Headscale too), and restarts the service over the API, no browser login on the router required.

Attack and failure signals. The same sweep that watches gateways also watches for trouble on the router itself, and a router in distress turns orange with a warning triangle in the sidebar — hover for the reason. On OPNsense with the Intrusion Detection plugin enabled, FrontierStack reads its recent Suricata alerts over the same API and shows the count and top signature (an attempted attack surfaced without opening the firewall UI; pfSense's REST API does not expose IDS uniformly, so this is OPNsense-only for now). Across every monitored router — OPNsense, pfSense, OpenWrt and the rest — each sweep also flags an unexpected reboot (uptime dropped sharply: a crash, a power event, or an attack), a brute-force spike on the router's admin/SSH (repeated failed logins, with the busiest source IP named from the router's own log), resource stress (memory or load dangerously high — a DoS, a runaway or compromised process), and an available firmware update (outdated firmware means known holes). Each raises an alert; signals a given platform cannot report are simply skipped, never a false all-clear. OPNsense and OpenWrt report the most; pfSense's API is more limited.

Driving and diagnosing a router. The AI Administrator can now do more than read: reboot_router restarts an appliance the same way it reboots a server (OPNsense/pfSense via API, OpenWrt over SSH), gated behind "Allow changes"; and when a task is genuinely better done in the router's own console, the assistant offers to open its Web UI with open_web_ui, resolving the address from the device pane rather than guessing. FrontierStack itself deliberately does not edit these appliances' firewall rules — OPNsense/pfSense/OpenWrt manage rules through their own config system, so the pane and the assistant point you at the native firewall page instead (right-click a router ▸ Open firewall rules… deep-links it). Two pane conveniences round this out: a Diagnostics button runs an API-free reachability check (ping, the common admin ports 80/443/8443/53, reverse DNS, gateway) so you can triage a router even without an API key; and for a router or firewall, Discover Services offers to open the Web UI rather than port-scanning it (a scan of your own perimeter device is rarely what you want). If SSH to an OPNsense or pfSense box is refused, SSH is either disabled or — more often — only allowed on certain interfaces (commonly the LAN, not the WAN): reaching it over the LAN, e.g. through a VPN, usually works, while from outside it is typically blocked. Enable SSH and its listening interfaces in the web UI (System ▸ Settings ▸ Administration ▸ Secure Shell on OPNsense). The SSH button itself is only hidden for consumer gateways that have no admin shell at all (NTT, some TP-Link/FRITZ!Box), where the Web UI is the only way in.

9.2Device Discovery: an inventory of your LAN

Open Device Discovery and press Find Monitors (or pick a subnet first). FrontierStack sweeps the chosen /24 using every method available at once, so routers, access points, switches, NAS boxes (Synology, QNAP, ZimaCube/ZimaOS…), printers, cameras and other servers all surface:

• Ping / ARP — an ICMP sweep, with ARP/MAC giving each host's vendor; turn on TCP Probe to find hosts that filter ICMP.
• Bonjour (mDNS) and SSDP / UPnP — advertised services and their friendly names.
• SNMP and LLDP / CDP — add model, port and neighbour details when net-snmp and lldpd are installed (buttons in the pane install them).
• Windows service ports — a machine that filters ICMP is still found by probing its distinctive ports (RDP, WinRM, SMB and friends), so Windows PCs no longer hide from the sweep.

The scan also notes how each host can be reached. An AnyDesk badge appears when AnyDesk is running locally or answering on its default port 7070, and a Tailscale badge — green when the peer is online — is built from this Mac's own tailscale status, matching mesh peers to discovered devices (only endpoints, hostnames and OS are kept; login names are never retained). On a directly-connected LAN, the Macs and devices macOS already knows through Bonjour appear before the slower IP sweep, so the list fills in quickly. A device's detail view carries matching AnyDesk and Tailscale rows with an Open app shortcut.

The AI Administrator's discover_devices tool runs the same scan and auto-pins the devices whose type it can identify confidently. Anything ambiguous is listed back rather than guessed at, so the assistant can ask you what it is and then pin it with pin_device. A pinned (monitored) device appears in its type's sidebar section; double-click it to open its pane, or use Open Web UI, Discover Services, Reclassify or Remove from its row. If you already know an address, the Add a device by IP field probes and pins it directly — no full scan needed.

9.3The Data Map

The Data Map pane draws a per-location data-flow diagram: where your data lives and how it moves across the devices and services at one site. FrontierStack serialises the location's inventory — the discovered devices, their roles, and (optionally) scanned ports — scrubs it of secrets, and hands it to your subscription Claude through the local claude CLI (not the metered API the in-app harness uses). The reply comes back as a Mermaid flowchart plus a short narrative, rendered offline in the pane; the last diagram for each location is kept, and you can save it to your Obsidian vault as a runbook. Locations themselves are explained in Chapter 8.

9.4Opening a port: UPnP and NAT-PMP/PCP

To reach a service from the internet the classic way, a port on the router has to forward to your Mac. Rather than make you log into the router by hand, FrontierStack can ask the router for a mapping using the standard protocols it advertises:

• UPnP IGD (via miniupnpc) — the common consumer-router method.
• NAT-PMP / PCP (via libnatpmp) — Apple's and the modern equivalent.
• OPNsense — a perimeter rule and port-forward through its API directly.

These tools are installed on demand the first time you use them. A mapping made for a temporary session (see Debug Share) is torn down automatically when the session ends.

9.5Dynamic DNS: following a changing IP

Home and small-office connections rarely have a static IP, so a hostname you point at today's address goes stale tomorrow. The Dynamic DNS pane keeps a hostname tracking your current public IP. Add an entry with a Name, a Hostname (for example myhost.duckdns.org) and the account data for your provider; press Update Now to publish immediately, or turn on Update automatically. You choose whether the updater runs only while FrontierStack is running or permanently as a system service, so the record stays current even when the app is closed. Cloudflare users get a tighter path: a Cloudflare DDNS A record that follows your public IP, driven by the same Cloudflare integration as Chapter 7; a generic interval-based updater covers other providers via an update URL.

9.6Tunnels and mesh: reaching in without opening up

The better way to reach a service from outside is to skip inbound ports entirely. A tunnel makes an outbound connection that an outside endpoint rides back in on, traversing NAT with nothing forwarded:

• Cloudflare Tunnel — a Quick Tunnel gives you a public *.trycloudflare.com URL with no account, or a named tunnel for a permanent hostname.
• Tailscale — Serve keeps a service private inside your tailnet; Funnel publishes it to the internet through your Tailscale node.

For ongoing connectivity between machines, a mesh or VPN network gives every node a stable private address wherever it roams. FrontierStack manages the common ones — by API on a router that supports it (OPNsense), or over SSH on a server:

9.11.1Tailnet: the whole-network view of Tailscale

The per-server Tailscale controls live inside each server's own pane and work over SSH: they answer “is the daemon up on this box”. The Tailnet pane answers the questions that span the whole network, which only Tailscale's own API can: paste an API access token (device read is enough — the pane never writes) and it lists every device with its addresses, OS, client version, owner, tags and last-seen time.

What earns the pane its place is the Needs attention list. It surfaces node keys that are about to expire — a scheduled outage, because when a key lapses the device silently drops off the tailnet until somebody re-authenticates it — devices waiting for approval, devices advertising routes nobody approved (the usual reason a subnet is unreachable even though the node is plainly up), and devices that stopped checking in weeks ago. Key expiry and pending approvals also raise Alerts.

9.11.2Running your own ZeroTier controller

ZeroTier's client is only the member side; networks are normally administered at my.zerotier.com. But every zerotier-one install can also be the controller for its own networks — fully self-hosted, no third party holding your membership list — and that path ships with no local administration UI at all. The ZeroTier Controller pane is that missing interface.

It shows the controller's identity and the networks it owns, and works the member-authorisation queue with one-click authorise/deauthorise. Members authorised but unseen for a fortnight are flagged, because an unused grant is a standing hole in the network. You can edit a network's name, privacy, routes and IP assignment pool, and write flow rules in ZeroTier's own rules language — compiled locally as you type, with templates for the common cases and a warning before you save a rule set that would lock everybody out.

Everything runs over SSH against the controller's own loopback API. The controller listens on 127.0.0.1:9993 and authenticates from a root-readable token, so FrontierStack never needs — and actively discourages — a publicly reachable controller API; the built-in audit flags it if one has been exposed anyway, along with loose file permissions and a missing backup. There is also a Prometheus export, a warm-standby backup/restore, and an assistant for migrating networks out of ZeroTier Central.

9.7UniFi: Cloud Gateways and Dream Machines

Ubiquiti's consoles get their own pane, because a UniFi gateway is doing more than a generic router probe can see. The UniFi pane speaks both of Ubiquiti's APIs, and they answer different questions. A Site Manager key — one key, created at unifi.ui.com — lists every console on your account with its model, firmware and online state. That is the only way to learn a remote site has lost power or its internet connection, because a console that is off the network cannot tell you anything locally. A local console key, created on the console itself under Settings ▸ Admins & Users, unlocks what the gateway is doing right now.

Locally the pane shows WAN links with their ISP, latency and throughput, and marks which link is actually carrying traffic. That last detail matters: when a primary line drops, UniFi fails over silently to the backup, which is usually slower and often metered, and the normal way people discover this is the following month's bill. It also lists adopted devices and their firmware, firewall rules in the order the gateway evaluates them, the client list, and every port forward. Each enabled forward is a deliberate hole through the firewall, so the pane flags the ones that accept connections from any source address — correct for a public web server, worth a second look for anything else.

Alerts cover a console dropping offline, a WAN link going down, running on the backup link, a device leaving the network, and internet-exposed port forwards. The pane is deliberately read-only: rule and network changes stay in the UniFi console, where their validation lives. PoE switch ports are handled in the next section, and plain console reachability still appears in Router & Network alongside the other vendors.

9.8Power over Ethernet: the power button for devices that have none

Access points, cameras, door controllers and desk phones have no power switch. Their only supply is the switch port they are plugged into, which means the recovery move for a wedged one — power-cycle it — normally involves a vendor web UI, or a walk to the cabinet.

The Power over Ethernet pane makes the port the button. Add a managed switch by IP and you get, per port: its PoE state (delivering power, searching, fault), powered-device class, priority and wattage where the switch reports it — plus power off/on and a proper power cycle. It also shows the switch's power budget: total watts, watts in use, and a warning when you cross the switch's own usage threshold, past which it starts shedding low priority ports — so port priority is editable here too.

This works broadly because PoE is standardised: RFC 3621's POWER-ETHERNET-MIB is implemented by essentially every managed PoE switch — Cisco, Aruba/HPE, Netgear, TP-Link/Omada, Ubiquiti, MikroTik, D-Link, Zyxel. Reading uses the SNMP community from Device Discovery. Switching a port needs a read-write community, added per switch and kept in your Keychain: cutting power to whatever is plugged in deserves a deliberate opt-in, and most switches keep that community separate and often disabled until you set it. With Monitor on, a port in fault state or a switch over its power budget raises an Alert — both otherwise stay invisible until someone notices a camera has gone dark.

9.9Internet Health and speed tests

The Internet Health pane is your view of the link itself. It continuously pings a list of Targets (Cloudflare's 1.1.1.1 and others, which you can edit), showing latency and packet loss and re-testing every 30 seconds; hosts that block ICMP fall back to a TCP-connect latency on :443. Live Traffic graphs real-time throughput across every active interface; Addresses & Networks lists each interface, its subnet (a network you can reach directly) and the gateway. A built-in traceroute shows the ISP and transit providers your traffic crosses. Press Test Now in the Speed Test section — or ask the AI with internet_speed — to measure download, upload, latency and responsiveness, using the Ookla speedtest CLI if installed and Apple's networkQuality otherwise.

9.10Debug Share: expose a localhost server, briefly

When you need to show a colleague or a phone a dev server running on localhost, the Debug Share pane exposes it for a session and then closes itself. Enter the port (or press Scan localhost to find a running server), choose how to Expose via, set an Auto-close time, and press Open debug session. Every share also closes automatically when FrontierStack quits.

The LAN forwarder has two extras. If the host firewall is on, the pane shows Open port in firewall to allow that port in the pf rules just for the session. And an Outside access picker can map a WAN port via UPnP, NAT-PMP or the OPNsense API and pair it with a DDNS host, so a LAN share becomes reachable from the public internet — the mapping is removed when the share closes. The AI Administrator can open, list and close sessions too (debug_share_open / _list / _close), including over MCP when "Allow changes" is on.

9.11Named tunnels: a permanent public hostname

A Quick Tunnel is deliberately throwaway. When you want a service on this Mac to be reachable permanently at a real address — a self-hosted app, an internal dashboard, a webhook receiver — use a named tunnel instead. It gives the service a stable hostname on one of your own Cloudflare zones (say app.example.com), survives reboots, and — like every tunnel — opens no inbound port: the Mac dials out to Cloudflare's edge, so there is nothing to forward and nothing for a port scan to find.

Create one in the Cloudflare pane's Named Tunnels section: New Tunnel…, then a hostname under one of your zones and the local port it fronts. FrontierStack drives the whole setup through the Cloudflare API using the token you already hold — it creates the tunnel, writes its ingress rule (hostname → http://localhost:PORT), adds a proxied CNAME pointing the hostname at <id>.cfargotunnel.com, and then installs the persistent daemon with cloudflared service install. That last step needs a one-time admin prompt (it writes a LaunchDaemon under /Library); cloudflared itself is fetched from Homebrew on first use. Untick Run on this Mac now if you only want the tunnel and DNS created, to run cloudflared on some other host. Delete unwinds all of it — daemon, DNS record and the account-side tunnel. The AI Administrator has the same three verbs (tunnel_create / tunnel_list / tunnel_delete), with creating one gated behind a red confirmation because it publishes a service to the internet.

A server is most exposed not when something attacks it, but when it quietly drifts: a firewall left open, a password login no one disabled, a certificate that expired last week, a mail server that will relay for anyone. FrontierStack treats hardening as a continuous, observable state rather than a one-time checklist. The Security pane is your console for the host in front of you; the AI Administrator (Chapter 13) carries the same checks across your whole fleet over SSH. Everything here is read-only until you ask for a change, and every change is one you could make by hand.

10.1The firewall: Application Firewall and pf

macOS ships two firewalls and FrontierStack reads both. The Application Firewall (ALF) is the per-app layer you set in System Settings; the Security pane reports its state — enabled, stealth mode, block-all-incoming, and how many app rules exist — so you can confirm at a glance that it is on. The pf packet filter is the lower, port-level layer, and this is the one FrontierStack manages directly.

App-managed pf rules live in their own file, ~/Library/Application Support/FrontierStack/pf-frontierstack.conf, loaded into a dedicated frontierstack anchor that is wired idempotently into the system /etc/pf.conf. Keeping rules in their own anchor means FrontierStack never edits Apple's base ruleset and your rules survive a reload without trampling anything. Adding or removing a rule reloads pf through an authenticated admin prompt.

From the AI side, the firewall_rule tool adds or removes a single allow/deny rule for a port, optionally scoped to a source address. It detects the target's operating system first: This Mac and remote macOS hosts get pf (the frontierstack anchor); Linux servers get ufw. It never sends a ufw command to a Mac or a pf rule to Linux.

10.2Intrusion prevention and the open-ports watch

An open port that has to stay open still needs a guard. The Security pane integrates two intrusion-prevention engines. fail2ban bans IP addresses that match abusive log patterns — SSH and Apache brute force are the classic cases; the pane shows whether it is installed and running, its jail count and how many addresses are currently banned, with a one-click Reload and a "write a starter jail.local" action that drops in a sane SSH jail without overwriting your overrides. CrowdSec adds a shared reputation network on top, showing its active decision count and a Metrics button. Both run as root services installed via Homebrew.

The Open Ports section lists every listener on the host. A companion watch — remote-control and screen-sharing exposure — specifically flags VNC/Screen Sharing (:5900), Apple Remote Desktop, AnyDesk, TeamViewer, Chrome Remote Desktop, Splashtop, RustDesk and Tailscale exit-node advertising, and turns red when someone is connected right now. The Recent Intrusions and Active Bans sections aggregate detections from fail2ban.log, CrowdSec, Suricata's eve.json and Wazuh's alerts.json, newest first; sources you do not run are simply skipped. A new ban, a new listener or an active inbound session can each raise an alert — configure delivery in Chapter 11.

10.3SSH hardening

SSH is the single most valuable door on most servers, so the Security pane includes an SSH Hardening Audit. It reads the effective configuration with sshd -T — not just the file on disk, but what the daemon actually applies after its drop-ins — and flags the settings that matter: PermitRootLogin, PasswordAuthentication, and the rest. When SSH is reachable from the network the pane says so plainly, because that is exactly where the findings matter.

The baseline for an internet-facing host is key-based authentication with PasswordAuthentication no. Apply fixes by editing /etc/ssh/sshd_config.d/ rather than the main file, so package updates leave your hardening intact. FrontierStack's own fleet channel already follows these rules: it uses Ed25519 keys, BatchMode=yes, IdentitiesOnly=yes and host-key trust-on-first-use recorded in known_hosts (Chapter 8). If a host's key access breaks, the repair_ssh_access tool reinstalls the managed key using a one-time login password held in your local vault — the password is read on your Mac and never sent to the AI.

10.4Server Security: the posture check for a remote host

Everything above describes the Security pane for this Mac. The Server Security pane is its counterpart for a linked server: the same posture check, read over SSH, for any host in your fleet. Open a server's detail and click Check… beside Security, or open the pane directly and pick a server from the host menu. A single scan reports the SSH hardening that matters (PermitRootLogin, PasswordAuthentication, the listening port), whether a firewall is present and enabled (ufw, firewalld, nftables or raw iptables), the intrusion-prevention engines running (fail2ban, CrowdSec), the count of pending security updates (apt, dnf, yum), every service listening on all interfaces, and host-integrity smells — world-writable directories missing the sticky bit, unexpected SUID binaries.

Each finding is scored High · Warning · Info · OK and rolled up into a one-line summary badge, so you can sweep a fleet and see at a glance which host needs attention. The pane never changes anything: a Fix it link takes you to the pane that actually makes the change — the Server Firewall pane to enable a firewall or edit rules, the Remote Tools SSH-hardening review, or the security tools that install fail2ban/CrowdSec. It reads deepest as root or with password-less sudo; without either it runs what it can and tells you so. Cross-links run both ways: the local Security pane links to macOS Firewall, Server Security links to Server Firewall, and a pinned server's panes link to both.

10.5The Malware Audit pane

The Malware Audit pane is a defender's dashboard for a single Mac, built in sections from the cheap, always-on checks down to the heavy forensics.

The same on-demand ClamAV scan is mirrored in the Security pane, and the AI's malware_scan tool runs ClamAV and/or YARA against any path on request — equally read-only.

10.6Deeper forensics tooling

When a quick scan is not enough, the heavier analysis lives as installable services in the Security Tools catalog category — reveal them with the catalog filter (Chapter 6). Volatility 3 is a memory-forensics framework for examining a RAM capture; capa identifies the capabilities hidden inside an executable; the CAPE and Cuckoo sandboxes detonate a suspicious sample in isolation and extract its configuration and behaviour. Alongside them sit Wazuh (host SIEM/XDR with file-integrity monitoring) and threat-intel feeds such as URLhaus, OpenPhish and OpenCTI to feed your YARA and sandbox rules. These are full tools, not built-in panes; FrontierStack installs and manages them so they are there when an incident demands them.

10.7The EDR Fleet board

If your endpoints already run a commercial EDR agent, the EDR Fleet pane gives you one read-only roll-up across every console. Choose Add Console…, pick a provider and paste its API credentials — stored in the Keychain, never in the AI's view. Endpoint and open-detection counts are pulled from each vendor's REST API for SentinelOne, Huntress, CrowdStrike, Sophos Central, Bitdefender GravityZone and Microsoft Defender for Endpoint; Jamf Protect and LimaCharlie connect for reachability, and any other console can be added as Generic (reachability only). A Fleet totals row sums the lot. The board refreshes every few minutes; a console going unreachable raises an alert under the Security group, with its open-detection count riding along in the alert detail. Open Console jumps you to the vendor's own dashboard for the deep work.

10.8Read-only security audits

The fastest way to learn where a host actually stands is the AI's security_audit tool, which runs a comprehensive, entirely read-only sweep on This Mac or any linked server over SSH. It self-detects the OS and checks pending security updates, disk encryption (FileVault or LUKS), the firewall (Application Firewall + pf on macOS, ufw / iptables on Linux), SSH hardening, listening ports, UID-0 and privileged accounts, passwordless-sudo (NOPASSWD) entries, intrusion prevention and recent failed logins. It uses passwordless sudo only where it is available and degrades gracefully where it is not. Nothing changes; you get a prioritised list of findings and specific fixes, which you can then apply yourself or hand back to the AI behind the usual approval card. Set target to a server's name to audit it remotely; omit it for the local Mac.

10.9The Security Audit pane: an exploitable-only auditor in a sandbox

The read-only security_audit sweep above tells you how a host is configured. The Security Audit pane asks a harder question: what here can an attacker actually exploit? It turns a coding agent into an adversarial auditor that maps the target, hunts from many angles — injection, broken access control, authentication and authorization, business logic, cryptography, exposed secrets, SSRF, misconfiguration and chained attacks — then tries to disprove each of its own findings, so what survives is only the issues with a concrete attack path. Every finding carries a scenario and a fix; there is no theoretical noise.

It runs in two modes. Codebase points the agent at a repository or web-project folder and audits the source. Infrastructure audits your own estate from the signals FrontierStack already gathers — Cloudflare zones and their TLS/WAF settings, domain and certificate health, each pinned server's OS, exposed services, remote-access and MDM/Activation Lock/FileVault posture, and the local firewall — assembled into a redacted inventory the agent then attacks.

The whole run happens inside a Sandcastle Docker sandbox, so an adversarial agent pointed at untrusted code never touches your host directly. The app provisions the sandbox image and runner the first time you run an audit (it needs Docker or Podman running); after that, runs start immediately. Findings render with severity badges and can be saved to Obsidian or raised as Alerts for anything critical or high. As everywhere else, only a redacted prompt leaves your Mac — secrets are stripped first — and the sandboxed agent authenticates with your Claude subscription using a token from claude setup-token (or an Anthropic API key).

10.10Strix: AI penetration testing

The audits so far are read-only: they inspect how a host is configured and where its code might be exploitable. Strix goes further — it runs an active penetration test. From the Strix AI Security Testing section of the Security pane you point it at a repository folder, a Git URL, a domain, a URL or an IP address (or pick one of your linked hosts), choose a depth of Quick, Standard or Deep, and it maps the attack surface, attempts real exploit paths, and writes a penetration-test report.

FrontierStack does this by driving the open-source Strix command-line agent locally on your Mac and reading the run artifacts it leaves on disk — the findings, the run metadata and the report — rather than scraping any hosted service. If the CLI is not installed the section says so and offers an Install Strix… link; Update Strix upgrades it through uv or pipx, never by piping a script into a shell. An AI provider key is optional; when you supply one it is stored in the macOS Keychain, handed only to the local Strix process, and never displayed or sent to any cloud model — the pane shows only whether a key is configured.

The tiles summarise the last run: Critical, High and Medium counts, duration, anything new in the last 24 hours, and any failed scans. A run in progress shows live phase text and a Stop button; when new critical or high findings appear you get a local notification unless Silence alerts is on. Open Findings and Export Report surface the detail, and Re-run Failed Scan and Restart Scan Engine recover from a bad run. Serious findings are treated as sensitive and are never sent to cloud AI. The AI Administrator can drive Strix as well, through strix_status, strix_scan and strix_control, all covered in Chapter 13.

10.11Certificate Sources and TLS

An expired certificate is a self-inflicted outage. The Certificate Sources dashboard gives you one row per issuance stack on the host — Let's Encrypt (via acme.sh), Certbot, Step CA, Traefik and Cloudflare SSL — with expiry read directly from the certificates themselves using openssl. Cloudflare uses the token and zone you configured for its analytics; sources that are not present on this Mac show dimmed. For Traefik you can point the dashboard at its acme.json; for Step CA, at its URL.

For issuing certificates, FrontierStack offers two paths. mkcert creates a locally-trusted certificate — ideal for development on .localhost or LAN names, where a public CA is irrelevant; the Certificates pane's Get action uses it when installed. For anything the public must reach, use Let's Encrypt: the AI's issue_certificate tool runs acme.sh or certbot on This Mac or a linked server, and for the DNS-Cloudflare method it reads a CF_TOKEN vault secret that is injected at run time, never sent to the model.

10.12Mail security

A mail server is two distinct risks: it can be spoofed (someone forges mail as your domain) and it can be abused (it relays mail for a stranger). FrontierStack addresses both, with tools that understand the old Apple Server case where Postfix sits underneath.

• Anti-spoofing DNS — email_auth_dns builds the SPF, DMARC and (optional) DKIM TXT records: SPF declares who may send for the domain, DMARC tells receivers how to treat failures, DKIM cryptographically signs outgoing mail. With publish=cloudflare it creates the records for you; otherwise it prints the exact records to paste into BIND, dnsmasq or your registrar.
• DKIM keys — generate_dkim_key runs opendkim-genkey on the server for a domain and selector and returns the public key as the TXT record to publish; the private key stays on the server for the OpenDKIM milter.
• Postfix hardening — harden_mail_postfix applies safe defaults over SSH with postconf: require SMTP AUTH to relay (this closes an open relay), reject unknown and unlisted sender domains, optionally enforce sender/login matching. It reloads Postfix and prints the effective settings.
• Open-relay test — test_open_relay runs a safe SMTP conversation (EHLO / MAIL FROM / RCPT TO to an external domain, then QUIT without sending) and reports whether the external recipient was rejected (good) or accepted (an open relay — bad).

10.13Who may change things from outside

Everything above hardens the machines FrontierStack looks after. This section is about hardening FrontierStack itself — because an app holding SSH keys, database passwords and cloud tokens for a whole fleet is a more attractive target than any single server it manages.

FrontierStack can be driven from outside through several doors: the MCP server for AI clients, the inbound HTTP control server for webhooks and the mobile panel, App Intents for Shortcuts and Siri, and — when you enable them — agent integrations. Each already carries its own token. Above all of them sit two settings in Remote Control & Lock, and it is worth being clear about which question each one answers.

The App Lock answers what survives a lock. Set a password, lock the app, and the window is covered while every external surface is either blocked outright or held to read-only.

External changes answers a different question: in an unlocked app, how much must an outside caller do before it can change anything? It applies to every external surface at once — the MCP server, the control panel, Shortcuts and the command line. Three levels. Full auto lets external tools act without asking — defensible only when you trust every connected client and the machine it runs on. Confirm each action, the default, puts an approval panel in front of every change, naming the caller. Password required demands your password before an external tool touches a server at all.

The confirm tier has a release valve that matters during an incident. Approving a change offers Allow for session, which grants five minutes — but only for that one caller and that one kind of action. Approving a service restart from an MCP client does not authorise a shell from that client, and does not authorise a restart requested through the mobile panel. The narrowness is the point: repeated prompting during an outage is exactly what drives people to switch to full auto permanently, and a grant that leaked sideways would be worse than the prompting it replaced.

Three properties hold at every level, and they are what make the setting trustworthy rather than merely configurable. Reading is never gated — a monitoring client keeps working whatever you choose, because a dashboard that goes dark during an incident is a liability. Credentials, shell access and FrontierStack’s own settings can never be changed from outside, at any level; those are refused rather than prompted, on the reasoning that a request arriving from a remote tool is simply the wrong origin for reading a password or running an arbitrary command. And session grants are never written to disk: quitting the app or locking it clears them, so a five-minute convenience cannot quietly become a standing permission you forgot you gave.

10.14Agent login with auth.md

Passwords are the worst credential to automate, so FrontierStack never asks you to paste a long-lived one for a third-party service. Instead it discovers auth.md — an open specification (originated by WorkOS) that a service can publish at https://<host>/auth.md to tell AI agents how to sign in on your behalf. A service's Connect ▸ Login Automation (auth.md) disclosure, and the AI's check_authmd tool, fetch that public file and summarise the advertised flows (agent-verified or user-claimed), scopes and OAuth endpoints — or report that none is published. The grant itself happens at the service and issues a short-lived, revocable, scoped token; FrontierStack stores no password. Cloudflare and Firecrawl publish an auth.md today, with more services appearing.

10.15The signed-request trust model (FS1)

FrontierStack's own control surfaces — the local control server, the MCP server, and the remote fsagent — are protected by a bearer token over TLS. A token alone, though, is a shared secret: replayable, and no proof of who is calling. The FS1 signed-request scheme adds a second factor on top. Every request is signed by an approved per-device Ed25519 key; the server keeps an allow-list of public keys and rejects anything signed by a key it has not explicitly approved. A leaked token then controls nothing on its own — this is the HTTP equivalent of the SSH keys your fleet already trusts.

Each request carries the device's key id, a timestamp, a random nonce and a signature over a canonical string. The server verifies the key is approved, the timestamp is within a 60-second window, the nonce is unseen (claimed only after the signature checks out, so a bad signature cannot burn nonces), and the signature is valid. The same scheme is implemented identically in the macOS app, the Go agent and the iOS client, pinned by a cross-language test.

You manage devices in the Paired Devices pane, in the Overview group of the sidebar (it used to live inside this Security pane). Each paired phone or tablet shows its label, key fingerprint, last-seen time, a scope picker and a Revoke button. Pairing is meant for one or two people with a couple of devices each — enrolment is capped at four — and each device carries a scope:

The same section holds the per-surface Require request signing switches for the control and MCP servers. Roll-out is deliberately gentle: the servers accept a token or a signature while signing is optional, with a bootstrap exception so you can enrol the first key with the token you already hold — turning on enforcement can never lock everyone out.

Two further per-device controls sit on each row. Allow AI Administrator (on by default, at any scope) governs whether that one phone may talk to the AI at all — you can let a trusted handset chat while blocking another, and the switch only ever restricts: even with it on, the AI still requires the app unlocked, a signed device, and a full-scope "Allow changes" for anything that mutates. Allow remote shell is offered only at full scope and is off by default; it is the explicit grant a phone needs to open a command-line shell and to install its own SSH key on your servers for direct access when this Mac is offline. If a handset is lost, Secure lost phone blocks the device and its older queued actions at once, closes its sessions, strips its direct-SSH keys from reachable servers and deactivates its push token; offline servers stay listed so you can finish the cleanup when they return.

Bluetooth pairing fallback. When a new phone cannot reach this Mac over Wi-Fi or Tailscale — a locked-down guest network, client isolation, a captive portal — enrolment can still complete over Bluetooth. Turn on Allow Bluetooth pairing (off by default) and the Mac advertises a short-lived GATT service only while the Pair dialog is open; the phone scans the same one-time QR code and runs the identical signed-enrolment handshake, just carried over BLE instead of HTTPS. Because the FS1 signature is transport-agnostic, the Bluetooth path inherits the same trust guarantees without a TLS channel.

10.16Hardening the headless Mac

A display-less server has its own security trade-offs, and the Headless Setup tool (covered fully in Chapter 8 and Chapter 17) surfaces the one that matters most here: its System Status section warns when FileVault is on, because an encrypted Mac with no keyboard or screen cannot be unlocked at boot — it will sit at the FileVault prompt instead of coming back online. For a headless box you generally choose between full-disk encryption and unattended restart, and FrontierStack makes that choice explicit rather than letting you discover it after the next power cut. Enabling Screen Sharing or Remote Login on such a host is exactly the kind of exposure the Open Ports watch will then keep an eye on.

A control panel that only lets you change things is half a tool. The other half is knowing when something has gone wrong — ideally before it becomes an outage. FrontierStack runs a continuous monitor sweep across everything you have told it to watch and turns the first sign of trouble into a message on whatever channel you choose. This chapter covers the Local Health board that summarises it all, the Alerts pane that raises and delivers warnings, the messaging gateways that carry them, and the specialised monitors for cloud services, project tools, power and storage.

11.1The Local Health board

Open Overview ▸ Local Health for the single screen that answers "is anything broken right now?" Every monitored service, device and subsystem appears as a row marked UP or DOWN, grouped by area — web and database services, the fleet, sites and certificates, disks, UPS and SNMP devices, connected cloud accounts. At the top sits a one-line summary in the form X up / Y down, so you can take in fleet-wide status in a glance without reading every row.

The board is the same data the AI Administrator reads through its get_server_health tool: ask it "what's down?" and it reports exactly what this screen shows, then offers to investigate before suggesting a fix (see Chapter 13). The sweep runs about every sixty seconds; each DOWN item is also attributed to the pane that owns it, which is why a red count capsule can appear on a sidebar row and on the Dock icon — you are told where the trouble is, not just that there is some.

11.2The Alerts pane: what is watched

The Local Health board shows status; the Alerts pane (Overview ▸ Alerts) decides what is worth a message and sends it. Switch on "Monitor critical services and alert me" and the sweep starts raising alerts. What it watches:

• Site & service health — HTTP/loopback checks, TCP ports, and deeper protocol probes (an SMTP/IMAP login, a database SELECT 1, an LDAP bind) so a service that is listening but actually broken still trips. The MySQL probe additionally reads the connection pool (current use vs max_connections and the "too many connections" refusal counter): it alerts when the pool passes 90% or when clients were refused between sweeps, and every sweep is charted as Connection Health History in the Database Health pane — pool % over time with red lines marking refusal bursts and outages.
• Linked servers — every fleet host's SSH endpoint is reachability-checked each sweep (a bare TCP connect, no login attempt), so a server that is completely off raises a plain "Server down" alert — and a "Server recovered" when it returns. This is separate from the SSH cool-down notice, which fires only when a host is up but rate-limiting connections.
• Certificates & domains — TLS expiry across the whole fleet, plus registrar / DNS / SSL checks that warn you days ahead rather than on the morning a domain lapses (Chapter 7).
• Disks filling up — a volume crossing its free-space threshold, surfaced from the Disk Health monitor below.
• Security exposure — a new open port or unexpected listener, a VPN tunnel drop, or a critical SSH-hardening failure while SSH is reachable from the network.
• Failed payments & billing — connected paid services (OpenAI, Anthropic, Cloudflare, Google Cloud) returning insufficient-quota, low-balance or auth errors — the symptom of a lapsed card.
• Connected cloud services — live monitors for Stripe, Shopify, Freshdesk, SendGrid and more, firing when a metric you chose crosses its threshold.

Most checks are threshold-based, so you decide what "trouble" means: alert when a certificate is within N days of expiry, when open tickets pass a number, when a disk crosses a percentage. Conservative thresholds turn a post-mortem into a heads-up. The pane also offers "Alert on intrusions" — new fail2ban/CrowdSec bans, high-severity Suricata signatures and notable Wazuh alerts (level ≥ 7) are messaged as they happen, and existing history is never re-alerted.

11.3Messaging gateways

An alert is only useful if it reaches you. The Alerts pane lets you enable several gateways at once and list multiple recipients per channel, so the right people are reached on the app they already have open. Paste a bot token or webhook URL, send a test, and you are live. The AI Administrator can enumerate the enabled gateways with get_messaging_channels and send through them with send_notification (Chapter 13).

11.11.1Paging the on-call (PagerDuty, Opsgenie, urgent ntfy)

The message gateways above are fire-and-forget text. The paging channels are different in kind: they are stateful. When a monitored item goes down, FrontierStack triggers an incident keyed to that item — PagerDuty (an Events API v2 routing key) or Opsgenie (a GenieKey) then runs your escalation policy, pushing, texting or phoning whoever is on call until someone acknowledges. When the item recovers, FrontierStack resolves the same incident automatically. Because the incident is keyed per item, a flapping service updates one incident rather than paging the rotation over and over, and nobody is woken for an outage that has already ended. Without an on-call service, the ntfy gateway's Page on down toggle sends "down" alerts at ntfy's maximum priority — a louder, repeating tone that overrides many phones' quiet settings — while recoveries and reports stay at normal priority. Each paging channel has a Send Test Page button that triggers a real incident and auto-resolves it about ten seconds later, proving the whole escalation path end to end.

Tokens and webhooks live in the macOS Keychain and never leave your Mac. You can also send a scheduled AI Evaluation Report: the AI Administrator composes a summary from your live monitored health (running read-only diagnostics) and emails it daily or weekly, or pushes it to all enabled channels — with an optional fuller report whenever two or more critical items are down at once.

11.4Delivery errors and how to fix them

A monitor is worthless if its alerts silently fail to send. FrontierStack verifies every send: when a gateway fails, it is recorded in the Delivery Errors section of the Alerts pane with plain-English fix-it guidance, the raw error underneath, and a Fix button that scrolls straight to the misconfigured channel's settings. A red badge appears on the sidebar's Alerts row and on the Dock icon so you actually notice, and it stays until you clear it. The AI Administrator reads the same list through get_alert_errors, so you can simply ask "why aren't my alerts arriving?"

The common causes are mundane and quick to fix:

11.5The on-host Service Watchdog & database health

The monitors above run on your Mac. For a critical server you also want a watchdog that lives on the box itself, so it keeps working when your Mac is asleep, offline, or simply not the machine that failed. FrontierStack's Service Watchdog is deployed with the host monitor (Chapter 8): it health-checks a service on a short interval, restarts it when it fails, and — as a last resort, and only when you have allowed it — reboots the machine, with a minimum-interval guard so a still-broken service can't cause a boot loop. Each watch has a check type: a process check, an HTTP or TCP probe, or a purpose-built MySQL or PostgreSQL check.

The database Heartbeats catch failures a process/port check misses. A MySQL Heartbeat asks for SELECT 1 without a password. Success or “access denied” proves that MySQL answered; a refused, timed-out or saturated connection is unhealthy. No MySQL password is stored and no tables are read. A blank probe name becomes the non-root fs_watchdog, which need not be a real account. A PostgreSQL Heartbeat uses pg_isready without a database username or password. With error-log early-warning switched on, the watchdog also tails the database's own error log and raises a critical event the first time it sees a corruption signature. Because all of this runs on the server, it can alert you autonomously over ntfy or a webhook even while your Mac is off; turn that on with Alert me directly from this server in the host's Service Watchdog panel.

To create a check-only account, replace your_database and the example password:

CREATE USER 'frontierstack_health'@'localhost' IDENTIFIED BY 'use-a-unique-random-password';
GRANT SELECT ON `your_database`.* TO 'frontierstack_health'@'localhost';

Save that login in Database Health. Leave auto-repair off until both a manual check and verified backup succeed. If a complete backup or repair needs another privilege, add only the schema-scoped privilege MySQL names; never use *.*. For a monitor connecting from another machine, replace localhost with that one exact private address.

The pinned server's Services row shows a green DB checker: On dot while scheduled table scans or server self-heal are active, Paused when a target exists but neither is running, and an orange Needs login instead of green when a selected MySQL scan has no database password. Database Health and the Heartbeat are complementary: the former checks tables on a slower schedule, while the latter detects outages quickly and can restart MySQL on the server.

When a database isn't down but is slow or stuck, the Why slow/stuck? button on each target in the Database Health pane answers it on demand: it reads the server's live activity (SHOW FULL PROCESSLIST on MySQL, pg_stat_activity on PostgreSQL), groups the running queries by shape so a burst of the same query collapses to one line with a count, and states the verdict — a single query hammered in a burst, a long-runner blocking others, connections waiting on a lock, oversized result sets streaming to clients (the bandwidth cost), or connections near the ceiling. A useful subtlety: if the load is bursty it may read idle between spikes, and the tool says so and tells you to re-run during one, rather than falsely reporting all-clear. Remote targets are probed over SSH, so no database port need be exposed.

11.11.2Pausing checks for maintenance

A watchdog that restarts things is exactly what you don't want while you are deliberately taking a service down. Rather than switching protection off — and relying on yourself to remember to switch it back on — use Pause Checks… in the Guardian's header, or the pause button on an individual service row. Choose a window from 15 minutes to a day.

While paused, the Guardian stops health-checking and stops restarting whatever you paused, so your work doesn't fight it. Nothing is disabled and no setting is lost. Keep Alive, Auto Recover, thresholds and intervals are all left exactly as they are, and checking resumes on its own when the window ends — or immediately, if you press Resume Now. A pause never survives its own window, even if you quit and reopen the app.

11.6Case study: the High Sierra Server.app serviceproxy wedge

Old Macs still running macOS Server (Server.app) on High Sierra have a well-known failure: the web front proxy — the launchd job com.apple.serviceproxy, which binds ports 80/443 in front of the real backend — occasionally wedges. It keeps accepting TCP connections but never answers them, so every website behind it goes dark while every process check still looks healthy. There is no fix from Apple; the stack is end-of-life. The Service Watchdog was built with exactly this case in mind.

How to set the watch up. On the server's pane, add a watchdog entry for the service name serviceproxy and — this is the important part — give it an HTTP check with the target http://127.0.0.1/, not a process check. The HTTP probe is the only check that sees the real failure (connections accepted, no answer); on failure the watchdog heals it with launchctl kickstart, which is precisely the manual fix an admin would type. Any HTTP status below 500 counts as alive (a 403/404 from the proxy still proves it is answering); each probe times out after 6 seconds, so a wedged proxy fails the check by timing out. The backend behind the proxy can be watched the same way as server-httpd.

Timings that work well. Check every 30 seconds; restart after 3 consecutive failed checks; try up to 3 restarts; leave the post-restart grace at its default (~20 s, generous enough for Apache on old hardware). That confirms a wedge for ~90 seconds before acting — long enough that one slow response or a momentary load spike never triggers a kickstart, short enough that sites are back about two minutes after a real wedge, hands-off. If the sites are critical, tighten to a 15–20 s interval with 2 failures (≈40 s to confirm) — going tighter than that mostly buys false restarts, because a genuinely slow old box can take a few seconds to answer under load. Leave “Reboot the server if recovery still fails” off unless the machine is truly unattended: a kickstart resolves the wedge in practice, and reboot-as-last-resort on a box like this mostly adds downtime (the anti-boot-loop guard enforces at least 30 minutes between watchdog reboots regardless).

Why not a process check? There is no process named serviceproxy — the job runs as httpd with a special config — and monitors older than v1.6.1 read a healthy proxy as permanently “down” because of that, restarting it over and over and, if reboot escalation was allowed, rebooting a healthy machine on a schedule. From v1.6.1 the monitor asks launchd for the job's real state instead, so a process check now tells the truth; the HTTP check remains the one that catches the actual wedge. If an older server of yours has been rebooting with no visible cause, check its monitor version first — and note that “Why did it reboot?” attributes watchdog-initiated reboots explicitly (with the reason, on v1.6+ monitors; older monitors leave a stamp that is still reported).

One guarantee closes the loop on delivery: an alert is never silent. On top of the messaging gateways, every state-change alert also raises a native macOS notification and push, independent of the gateways — so even with email off and every channel disabled, a service going down still reaches you on the Mac. Database recovery — running mysqlcheck/pg_amcheck and the AI-guided rebuild — is covered in Chapter 5.

11.7Incidents, on-call and status pages

For teams that already run formal incident response, FrontierStack connects to the tools you use rather than replacing them. Each has a live pane reached from its catalog entry: paste a read-only API key and it shows current state.

• Incident management & on-call — PagerDuty (open and high-urgency incidents, who is on-call, services), Opsgenie (open/unacknowledged alerts, with a US/EU region toggle), incident.io and Rootly (active incidents).
• Status pages — Statuspage (unresolved incidents and components that are down) and Better Stack (monitors up / down / paused).

Set up the key in each pane (for example PagerDuty ▸ Integrations ▸ API Access Keys) and the pane confirms the connection. These feed the same monitoring picture, so an open PagerDuty incident or a down status-page component shows alongside your own health.

11.8SaaS live monitors & project-management panes

FrontierStack does not stop at infrastructure. A config-driven SaaS live monitor watches connected cloud accounts and refreshes every few minutes. Each service has a credentials form, live metric tiles, an "Alert me" toggle and a threshold; alarmable services raise a DOWN alert when the metric crosses the line, and the rest are watched for reachability. Live monitors include Stripe (disputes needing response, balance), Shopify and WooCommerce (open orders), Freshdesk and Zammad (open/pending/overdue tickets), SendGrid, Mailgun and Postmark (bounces and blocks), GitHub Copilot (inactive seats), and identity providers Okta and Microsoft Entra ID. Pin a service to make it always-on-Local-Health regardless of its alert setting.

The same pattern gives project-management tools live token-login panes: Jira (open / unassigned / blocked / in-sprint counts), Linear, monday.com, OpenProject, Plane and Taiga. These are dashboards rather than alert sources, but they put your team's workload in the same window as the servers that run it. Email delivery has its own unified Email Delivery pane that combines transactional-provider metrics with SPF/DKIM/DMARC checks (Chapter 7).

11.9Queue Operations

Queue Operations is a read-only dashboard for RabbitMQ, Kafka, NATS/JetStream, Redpanda, AWS SQS, Azure Service Bus, Google Pub/Sub, Celery/Flower, Redis Streams, Apache Pulsar and Apache RocketMQ. Add one monitor for each broker, namespace or cloud account. FrontierStack shows queue or consumer-group names, ready and in-flight counts, consumer counts, lag, dead-letter counts and oldest-message age when the provider exposes them. It never reads or stores message bodies.

Client and administration ports are kept separate. For example, RabbitMQ clients use AMQP on 5672 while its management API normally uses 15672; NATS clients use 4222 while monitoring normally uses 8222; Pulsar clients use 6650 while its HTTP admin API uses 8080. Keep monitoring endpoints private. Use TLS and a monitoring-only account whenever the dashboard is not reached through the on-host FrontierStack monitor.

For a linked server, select its installed monitor and FrontierStack asks the credential-free helper for bounded summaries over the signed connection. Cloud providers use their normal local CLI profiles or a scoped credential saved in Keychain. Critical thresholds join the Alerts sweep, the same sanitised summaries are available in the iPhone/iPad app, and the AI Administrator can answer queue-health questions through the read-only get_server_health tool. The dashboard has no purge, delete, publish, acknowledge or replay action.

11.10UPS monitoring and SNMP devices

Power is the failure mode people forget until the lights go out. FrontierStack discovers UPS units three ways — macOS power sources (pmset -g ps), the NUT daemon (upsc), and apcupsd (apcaccess) — and shows charge, runtime, load and input in the UPS section of Power Control. Turn on alerts and you are paged when a unit goes on battery, hits a low-charge threshold you set, or loses comms with its controller. Pin a UPS to keep it on Local Health and on the Places map; one-click buttons install NUT or apcupsd if they are missing. Because a scan waits on those daemons, a Stop button appears while one is running — if a NUT server has wedged, stop the scan instead of waiting; readings already found are kept and automatic refresh pauses until you press Refresh.

For anything else that speaks SNMP — managed switches, printers, network UPSes, NAS units — the SNMP / OIDs section of a device's detail pane queries it directly. Pick a built-in template (System, Host Resources, Interfaces, Printer RFC 3805, UPS RFC 1628, Synology, QNAP, APC PowerNet) and press Query for a label-and-value readout, or fetch a single custom OID by hand. You can also watch an OID: set a comparison and threshold (greater-than, equals, contains, or simply "alert on any change") and FrontierStack polls it and raises an alert when the condition is met. It uses net-snmp, installable in one click.

11.11Disk, RAID and SMART health

Drives fail with warning if you are listening for it. The Disk Health pane (a built-in monitoring tool) combines several layers. Volume space and Time Machine status are always shown. With smartmontools installed (a one-click Install button), each physical drive is enriched with deep SMART attributes — health, temperature, power-on hours, reallocated and pending sectors, and predicted-failure flags — visible per drive with a Check button and a Details sheet for the raw report. A drive trips an alert on a SMART failure or on concerning attributes (reallocated/pending sectors or a predict-fail), not only on outright death.

Software AppleRAID sets (mirror, stripe, concat) appear in their own section with level, status and per-member state; an alert fires when a set is degraded or a member drops offline. Under Alerts & Thresholds, switch on disk alerts and they route to your channels exactly like every other monitor.

Taken together, these monitors give you one board to glance at, one pane to tune, and one set of channels to reach you on — and an AI Administrator (Chapter 13) that reads the same health and sends the same notifications on your behalf. The host monitors that feed fleet-wide CPU, memory and GPU metrics into this picture are covered in Chapter 8.

A good server administrator does the same handful of things over and over: run a backup, warm a cache, restart a backend the right way, check a vhost is actually serving what you think. FrontierStack lets you capture each of those as a saved script, run it with one click, and — when you trust it — hand it to cron so it runs unattended. From there it reaches outward, into Apple Shortcuts, Siri, the frontierstack:// URL scheme and scheduled actions the app performs on its own.

This chapter covers the mechanical side of automation: scripts, schedules and the macOS automation surfaces. The intelligent side — an AI that writes, saves and schedules these scripts for you — is Chapter 13. Everything here is something the AI Administrator can drive through its own audited tools, so the two chapters describe the same machinery from two ends.

12.1The Scripts & Cron pane

Open Scripts & Cron from the sidebar. It is your saved-script library and your scheduler in one place. The Saved Scripts section at the top lists every script you have kept, each with its name and the first line of its body. For each one you get four buttons: Run (execute it now), Schedule (hand it to cron, below), Edit (open the editor), and a trash icon to delete it. Press Add Script… to write a new one.

A saved script is a genuine executable file on disk — shell, PHP or Python — not a fragment the app interprets. Give it a clear name and a note describing its purpose, so that six months later (or the AI Administrator, scanning your library) can tell at a glance what it does and pick the right one. Because the saved file is exactly what runs, what you test by hand is exactly what runs on a schedule.

Each script can target This Mac by default, a local Docker container, or a linked server over SSH by name. Where a script needs a credential, it references a named secret from the local .env vault as an environment variable ($DB_PASSWORD, "$SSH_SUDO_PASS"); the app injects the real value at run time and never lets it reach a cloud model (Chapter 13).

12.2Scheduling with cron

The Cron Jobs section is a friendly front-end onto your real system crontab — not a separate scheduler the app invents, but the same cron that has run unattended work on Unix for decades. Each row shows a job's name, a plain-English description of its schedule (FrontierStack translates the five-field syntax for you), and the command it runs, with a switch to enable or disable it and buttons to Run it now, Edit it, or delete it.

Press Schedule on a saved script, or Add Job… for an arbitrary command, to open the schedule builder. Pick a cadence — hourly, daily at a time, weekly — and FrontierStack writes the crontab entry, tagging its own jobs with a # AC: comment so it can find and update them by name later. The same applies through the AI Administrator: list_cron reads your jobs, and schedule_cron creates or updates one (updating in place if a job of that name already exists).

For full control, expand Edit raw crontab at the bottom of the pane. This is the standard crontab in a text editor; FrontierStack-managed jobs appear as # AC: comments so you can see which lines it owns. Edit by hand and press Apply.

12.3The bundled sample scripts

FrontierStack ships a small read-only library of proven, battle-tested admin scripts. They are not your scripts — you cannot edit them in place — but you can read any one's full source, learn from it, and copy it into your own library to adapt. They encode real-world quirks of a Mac web stack (especially the old macOS Server backend), so reading one before writing your own saves you the surprises. The AI Administrator reads them first too, via list_sample_scripts and read_sample_script, so its scripts inherit the same hard-won knowledge.

12.4Apple Shortcuts, App Intents & Siri

FrontierStack ships native App Intents — actions that appear in the macOS Shortcuts app, in Spotlight, and (for several of them) as built-in Siri phrases you can speak. Drop them into a Shortcut you trigger from a hotkey, the menu bar or a Folder Action; they work from Automator too, since Automator can run Shortcuts and AppleScript. The shipped actions:

The catalogue is wider still — Get Server Status, List Servers, List Docker Containers, Check a Website and more are all available to build Shortcuts with, even where they carry no stock Siri phrase.

12.5Driving the app from a script

Because the App Intents are real Shortcuts actions, you can call them from a shell step with the macOS shortcuts command-line tool — useful inside a saved script, a cron job, an Automator “Run Shell Script” action or any launcher. Wrap a FrontierStack action in a one-line Shortcut, then invoke it with shortcuts run "My Shortcut", chaining it with anything else your script does.

The AI Administrator can do this assembly for you. Ask it for an automation script that drives the app through Shortcuts and it calls list_app_intents to see the available actions and the exact shortcuts run invocation for each, then authors the script with save_script (or schedules it with schedule_cron). You get a finished, runnable script rather than a hint.

12.6Scheduled app actions

A cron job runs a shell command. A scheduled app action is different: it is a task the app itself performs on a schedule, using its stored credentials and the same audited tools you and the AI Administrator use — not a raw shell command. Use an app action to reboot a linked server, start, stop or restart a local service the app controls, or flush DNS on a timetable. They are managed through schedule_app_action and listed by list_app_actions, and they appear under Scheduled App Actions in the pane. The natural way to create one is to ask the AI Administrator — “reboot web-03 every night at 3am” — and approve it.

There is one important difference from cron: an app action only runs while FrontierStack is open. It performs the action through the app, so it catches nothing that falls due while the app is closed. Cron, by contrast, runs at the system level whether the app is open or not. Choose cron for things that must happen no matter what, and app actions for things the app must do with its own credentials and tools.