Connecting Servers: Locations & the Fleet

Everything so far has been about the Mac in front of you. This part of the manual widens the lens to the whole estate: the Linux boxes in a rack, a vintage Xserve still running macOS Server, a Raspberry Pi on a shelf, a cloud VPS, even a Windows host. You link each one once over SSH, and from then on FrontierStack treats them as a single fleet — running diagnostics, fanning commands out, deploying sites, and watching their health. It also keeps track of where this Mac is on the network, so a laptop that moves between home, office and a café doesn't fire false alarms about servers it simply can't see right now.

This chapter covers linking and running the fleet. Chapter 9 goes deeper on the network and perimeter, Chapter 10 on hardening and the security audits, and Chapter 11 on continuous monitoring and alerts. The AI Administrator can drive almost everything here over the same tools — see Chapter 13.

8.1Locations & Places

Open Locations & Places ▸ Locations. A location is simply a named network — Home, Office, On VPN — recognised from several signals: the Wi-Fi SSID, the router (gateway) MAC address, the Wi-Fi BSSID, the subnet prefix of your IP address (e.g. 192.168.1.), your public IP block, and whether a VPN is up. The header shows where you are right now (Now: Office), with the live snapshot beneath it: Wi-Fi name, IP, gateway and VPN state. Detection runs every 30 seconds.

When FrontierStack sees a network it doesn't recognise it offers a New network detected banner — click Add… to save it. You can also press Add Current Network as Location, then refine the criteria: a location matches when all the set criteria hold (with SSID and router MAC treated as any-of), so leaving a field blank simply ignores it. When several locations match, the most specific one wins rather than the first in the list. Use the Mode picker to pin a location manually instead of auto-detecting — handy when the signals are ambiguous.

Not all signals are equally trustworthy, and this matters more than it sounds. A subnet prefix is not an identity: home routers, cafés and hotels hand out 192.168.1.× everywhere, so a location defined only by its subnet would happily match a completely different building. The router MAC is the strongest signal — it is unique to one piece of hardware and readable without any special permission — which is why Update all criteria from current network is the one-click fix for a location that was saved before router matching existed.

8.23.1Pinning a place with GPS

For the awkward case — two sites that genuinely share a subnet and you would rather not fuss with MAC addresses — a location can also carry a GPS pin. Stand in the place, open the location and press Pin This Spot (GPS); a Radius slider sets how far counts as "here" (150 m by default, because indoor positioning is coarse and a site is a building, not a point).

The pin is a cross-reference, not a requirement. It does two things: it lets a weak match be confirmed, and — more usefully — it lets a wrong one be vetoed, so Home stops claiming a café that happens to use the same subnet. To keep that from ever making detection worse, a location identified only by shared criteria (a subnet, a VPN toggle) is no longer reported as confirmed when a position is available and contradicts it; you get Away / Unknown instead of a confident wrong answer.

Devices belong to Locations too. In Device Discovery, a monitored device's ⋯ menu ▸ Assign to Location places it at a specific site — or leave it Automatic, where it falls into whichever location owns its subnet. You can also set it from the device's own pane, and each location's row lists the devices assigned to it. Private subnets repeat between sites, so a device pinned elsewhere can land in a location by subnet; press its × in that list (or choose No Location) to remove it, and a matching subnet won't put it back. When the Mac arrives at a recognised location, FrontierStack checks those devices (skipping any seen recently, e.g. over VPN) and shows each one's status: green reachable, red if it should be here but isn't answering, and orange when it's on a Wi-Fi segment this Mac can't reach — usually a different access point or band, such as a 2.4 GHz-only sensor while the Mac is on 5 GHz. Orange means “can't verify from here”, not “gone”.

Linked subnets. A location can also claim devices on subnets the Mac isn't on: list IPv4 CIDRs such as 10.20.0.0/24 in its Linked subnets field, or pick one from the Tailscale Routes menu, which lists the subnet routes on your tailnet. This is the way to give a remote site reached through a Tailscale subnet router its own devices. A linked subnet only places devices and outranks a look-alike subnet prefix on another location. It never makes the Mac “at” that location, because a routed subnet answers from anywhere. Tailnet 100.x addresses are scattered across the tailnet, so assign those devices by hand.

Same address, different hardware. Two sites can both have a router at 192.168.0.1, so FrontierStack identifies routers and devices by their hardware (MAC) address rather than trusting the address alone. A pinned router counts as up only when its MAC answers, and a router whose MAC is a location's router MAC belongs to that location. A location saved without a router MAC learns it automatically the next time it is identified by its access point (BSSID), or by its Wi-Fi name plus a GPS fix inside its pin. A Wi-Fi name on its own is not enough, because a travel router takes the same name everywhere. When a device's address changes, typically a new DHCP lease, a pinned device with a known MAC that stops answering on a trusted location's LAN is looked for by MAC across that subnet and its pin is moved to the new address (Followed by MAC from …). That check runs at most every five minutes per subnet and backs off to hourly while nothing turns up.

The point of all this is the Location-Dependent Monitoring section. Mark a device, Wi-Fi network, Cloudflare zone or watch as critical in Overview ▸ Alerts, then scope it here to Only at Office. That check is paused everywhere else: your NAS won't ring as DOWN from a hotel, and your home Wi-Fi won't alert as lost while you're at work. Paused items resume silently when you return. When the location itself changes, FrontierStack can post a local notification, send a "📍 Now at: Office" alert through your Messaging Gateways, or both.

8.2Linking a server over SSH

A server joins the fleet through the Link a Server sheet — reachable from a device's pane, from Device Discovery, or from the Cloud Servers and Remote Tools panes. Enter a name, the host or IP, the port, an SSH username (root, ubuntu, ec2-user…) and the login password. That password is used exactly once: the app installs this Mac's managed SSH key into the server's authorized_keys, then connects key-only from then on. Nothing is stored except the server's address — the password is discarded.

For a key-only server with no password login, expand Can't use a password? Install the key manually, copy the shown public key into the server's authorized_keys yourself, then link. Tick Also install the monitoring helper to set up the host monitor in the same step (covered below).

Windows and Tailscale servers. A Windows PC running OpenSSH links the same way: FrontierStack notices the Windows shell and installs the key with PowerShell — for an administrator account into C:\ProgramData\ssh\administrators_authorized_keys (with the permissions sshd requires), otherwise into the user's .ssh\authorized_keys. The server's SSH host key is recorded in FrontierStack's own trust store, so linking also works when your ~/.ssh/config discards host keys for some addresses (a Host 100.* rule with UserKnownHostsFile /dev/null for Tailscale is common).

Servers whose login shell isn’t sh. FreeBSD’s root account (csh or tcsh on FreeBSD 13 and earlier), OPNsense’s console menu shell and fish users link and run commands normally. Before the first command of each app run, FrontierStack asks the server for its login shell once; when it isn’t sh-compatible, every script is handed to /bin/sh intact instead of being parsed by that shell. sh, bash, zsh, ksh and dash servers receive commands exactly as before.

Linked servers carry a small green or red status dot for reachability. If it goes red with Permission denied (publickey) — typically after a server is reinstalled, or the key was never installed — the AI tool repair_ssh_access clears the stale host key and re-installs the managed key, using a one-time login password you keep as a vault secret. The password is read locally and re-installs key access; on success the dot turns green again.

The host's Status section also shows a Connection row: what actually carries the server's default route — the negotiated Ethernet rate (10 GbE, 2.5 GbE, 1 GbE), the Wi-Fi generation (Wi-Fi 7 = 802.11be, Wi-Fi 6E/6/5) with its channel width, or a VPN tunnel. It is read during the normal status probe on Linux, macOS and FreeBSD hosts, and reported by the host monitor agent (v1.17+) on agent-only hosts. The same row appears for This Mac, and on SNMP devices the Interfaces query now lists each port's negotiated link speed.

Each server also has a Connect via setting (in its Edit sheet) for interactive shells: plain SSH (default), mosh — which logs in over your SSH settings and then switches to its own UDP transport, so the session survives sleep, roaming and flaky links — or experimental SSH3 over QUIC/HTTP3, whose server can hide behind a secret URL on UDP 443. Open Shell and Open SSH honour the choice; probes and file transfers always stay on plain SSH. Both clients are in the Services catalogue (mosh via Homebrew, SSH3 via Go).

8.3The managed key and the saved sudo password

Key access lets the app connect, but many useful operations — reading /var/log, editing Apache config, reloading a firewall — need root on the server. FrontierStack handles this with a per-host sudo password, saved in the macOS Keychain (never in plain files). It is captured automatically when you link a server, and you can set or change it later in the host's pane or in the Remote Tools target section (the Sudo password field).

When a root-only command runs, the helper primes sudo's credential cache with that saved password over the SSH session, then runs the command. If no password is saved, root operations fall back to passwordless sudo -n and simply report "needs sudo" when that isn't configured. Because secrets are referenced by name, you can keep several — for example a distinct MySQL password per server in the .env vault.

8.4Fleet Run — one command across every node

Fleet & Remote ▸ Fleet Run fans a single operation out across all linked servers at once, or a selected subset, with a live per-host result. Pick your Targets (toggle hosts; the reachability dot shows which are up), choose an Operation, and press Run on N Server(s) — everything runs concurrently over SSH and each host reports back below, expandable to its full output.

Operations are OS-aware: a package update maps to apt, dnf, pacman, zypper, apk or brew depending on the host; a service restart tries systemd, then Homebrew. You never write per-OS commands by hand.

Fleet Run is the fastest way to spot drift: run Disk usage or Uptime & load to compare hosts at a glance, or use the file-checksum and health-check tools in Remote Tools (next) to find a config that's diverged across the fleet.

PHP versions & security fixes answers the question every PHP security release raises: which of my servers still need it? It lists every PHP on each host — Homebrew, MacPorts, MAMP, Apple’s own, distribution packages, and OPNsense or pfSense firmware over SSH — shows whether the host serves PHP to the web through php-fpm, php-cgi or mod_php, and compares each install with php.net’s latest release of its branch. Installs behind a security release, or on a branch that no longer gets fixes, are flagged in the results. It changes nothing. When something is flagged, Copy AI Prompt puts a prompt on the clipboard for Claude Code, the AI Administrator or another agent: it names the servers and versions found and asks the agent to update PHP from the source each server installed it from, one server at a time, checking with you before every change, then restart whatever serves PHP and confirm the new version.

8.5Creating a container, here or on a server

Everywhere FrontierStack lists containers there is a Create Container… button, and they all open the same sheet: the Docker pane for this Mac, the same pane with a linked server chosen in the host picker, and the Docker tab of the Server Activity window. What you build runs on whichever target you started from — locally through the docker binary, or on a server over the SSH connection you already have.

Start from a preset or from blank. The presets cover plain images — nginx, PostgreSQL, MySQL, Redis, a throwaway Ubuntu shell — and every self-hostable service the app can run as a single container, such as n8n, Flowise, Langflow, Vaultwarden and Checkmk. Choosing one fills in its image, port, volumes and environment, all of which you can change. Services that need a docker-compose stack are installed from their own pane on this Mac instead, and the sheet names them so nothing looks missing.

Before anything happens the sheet shows the exact docker run line with secrets replaced by dots, and lets you copy it. Creating pulls the image first (unless you turn that off) and then runs it, streaming the output back into the sheet.

8.23.2The Containers board as a monitor

The Containers pane is not only a list. It carries a status dot that rolls every runtime on this Mac into a single verdict, so a glance at the sidebar tells you whether anything needs attention. Green means everything installed is healthy; orange means something is stopped or still being probed; red means an installed engine is down, or a container is unhealthy, restarting, or exited with an error. A runtime you have never installed is not a fault — green therefore reads as “everything present is happy”, not “everything is running”.

The sections follow the same order as the services list: Apple Containers, Docker, Kubernetes, OrbStack, container machines, then Vagrant. Kubernetes appears here read-only, showing the clusters you pinned in Kubernetes Clusters; pinning and per-cluster actions stay in that pane, which watches whole clusters wherever they run rather than only what is on this Mac. Where a runtime is not installed, the button opens the pane that installs it instead of sending you to a download page.

The pin button in the header opens a small floating palette: a single status bar with the overall dot and a running count, above a short list of what is running, with anything set up but idle grouped under Waiting. It is deliberately tiny — something to leave in a corner of the screen while you work elsewhere. Clicking any row jumps the main window to it.

8.23.3Docker Swarm

Swarm is not a separate product to install — it is built into the Docker engine, which is why FrontierStack monitors it from the Docker pane rather than giving it a pane of its own. When the engine you are looking at belongs to a swarm, a Docker Swarm section appears listing the cluster's nodes (ready or down, whether one is draining, and which are managers) and its services with their replica counts — 3/3 when a service has every task it asked for, 1/3 when it does not. The Server Activity window's Docker tab carries the same thing as a Swarm segment, so you can read a cluster from any linked server.

The state comes from docker info, which the pane already calls for engine health, so a machine that is not in a swarm costs nothing extra to check. One rule of Swarm shapes what you see: only a manager may list the cluster. Ask a worker and its daemon refuses, so on a worker FrontierStack tells you that plainly rather than showing empty tables that look like a failure.

8.23.4Ready-made stacks

Some things are not one container but several that are useless apart. Install Stack…, on the Containers pane and in the Server Activity window's Docker tab, sets up a whole group at once. Four ship with the app: the *arr media stack (Prowlarr, which holds the indexer list the others read, plus Sonarr for series, Radarr for films, Lidarr for music, Readarr for books, Bazarr for subtitles and a download client), Monitoring (Prometheus, Grafana and node-exporter), a Media server (Jellyfin, optionally with Jellyseerr for requests) and Immich, a self-hosted photo library that genuinely needs four containers — server, machine learning, Redis and PostgreSQL.

You choose which members to include, where the config and media folders live, the user and group the containers write as, and a port offset that moves the whole stack clear of something already listening. Published ports bind to 127.0.0.1 unless you say otherwise. FrontierStack then writes a docker-compose.yml into the project folder and runs docker compose up -d. Compose is what makes a stack a stack: the members share a network and address each other by name, which is why Sonarr's indexer setting can simply be http://prowlarr:9696.

8.23.5Apple containers, and the GUI we recommend

On Apple silicon, macOS can run Linux containers natively through Apple's own container command, where each container gets its own lightweight virtual machine instead of sharing one big Linux VM. The Containers pane has an Apple Containers section for this: it lists what is running, starts and stops the container system, pulls images, and creates new containers, in the same shape as the Docker section above. Apple's tool is a command-line program, and FrontierStack installs it from its GitHub releases.

For day-to-day work with Apple containers we recommend Davit, an independent macOS app that puts a proper graphical front-end on that same container command — browsing containers and images, starting and stopping them and following logs, without having to remember subcommands. The Apple Containers section installs it for you through its Homebrew cask and opens it afterwards. Davit needs Apple silicon and macOS 15 or newer.

8.23.6Watchtower on your servers

Watchtower keeps Docker containers up to date by pulling new images and recreating the containers. When a linked server runs it, FrontierStack finds the container in the regular status check and reads its settings and log over SSH, at most hourly. The server pane, the Server Activity window’s Docker tab and the Watchtower pane under Containers show its last scan: how many containers it scanned, updated and failed to update. They also show its schedule and mode, and any errors since. Notification URLs and API tokens are never read. Alerts fire when Watchtower stops, when Docker refuses it, when updates fail, or when scans stop arriving.

8.6Remote Tools — per-node diagnostics

Fleet & Remote ▸ Remote Tools is a suite of SSH-driven diagnostics, distilled from real fleet-debugging work. Choose the target first — a pinned server, or Other (IP / host)… for an ad-hoc address that isn't a saved server (its sudo password persists in the Keychain too) — then the tool and its parameters. Read-only checks need no root; ones that read protected logs or config use the saved sudo password through the helper, and a few are marked mutating.

Some tools work across several selected nodes at once — File Diff, Healthcheck, TLS Expiry and Serving Path Diagnosis (who's up, which public IPs each node binds, per-process CPU saturation and stopped services, cross-checked against every pinned Cloudflare zone's origin IP) are fleet-wide by design — which is exactly how you confirm every web node is serving the same content with a valid certificate. The bottom of the pane also offers safe over-SSH editing of /etc/hosts (root, through the helper) and the SSH user's ~/.ssh/known_hosts, each backed up to .fsbak first.

8.7Recovering a wedged server — and a safe power reset without SSH

A server can stop answering SSH while its services keep running — overloaded, not down. FrontierStack shows this: when SSH is unreachable but a service port still answers (say MySQL on 3306), the server's dot turns orange, not red, and its pane reads "SSH is wedged, but the server is still serving" and lists what's answering, with the database's version banner. The machine is alive — only its management channel is stuck — so don't assume it's dead or blindly cut the power.

Try a normal Reboot first. After Reboot, the dot flashes while FrontierStack waits out a grace period and then polls, refreshing the instant the host returns — no need to keep pressing Refresh.

Make a power reset safe — without SSH. When a database is still answering, the pane shows a red-outlined EMERGENCY box, Prepare databases for a safe power reset. Each step runs over the database's own connection (no SSH), using the credentials saved in Database Health, top to bottom: Stop accepting writes (reads still work, reversible), Quiesce & flush to disk, then Clean-shutdown. After a clean shutdown a hard power-cycle is safe, because nothing dirty is left in memory to corrupt. Coverage differs by engine — MySQL, Redis and MongoDB can be cleanly stopped over their protocol; PostgreSQL can be quiesced and flushed but not stopped over SQL, so you flush then power-cycle. The clean-shutdown button is red and confirms first.

Power-cycle, then bring it back. The hardware power controls (SwitchBot plug, smart powerboard/PDU, KVM) each offer Shut Down Databases First in their confirmation, so you can flush and stop the databases before cutting power. And if the shutdown relieved the pressure and SSH recovers, a Start button starts the database again over SSH — so the ideal path is often: clean-shutdown the runaway database, let the box recover, and Start it, with no power-cycle at all.

8.8The Host Monitor (remote agent)

SSH diagnostics run on demand. For continuous visibility, install the Host Monitor — a tiny, read-only Go helper (fsagent) that lives on the server and reports its own health, with no per-cycle SSH. From a linked host's pane, choose to install it (or tick the box in Link a Server); over the existing key the app pushes the right binary for the host's OS and architecture, sets up the platform service (systemd, launchd or BSD rc.d), and pins its TLS certificate. The AI tool install_monitor does the same on request.

LAN and Tailscale at once. The monitor listens on the address this Mac used to install it, plus the server's own LAN address and its Tailscale address — never on every interface, and never on a public or container-bridge address. If it stops answering at one (say you left the office and now reach the server over Tailscale), FrontierStack switches to another address that presents the same pinned certificate. A monitor installed before version 1.22 answers on one address only: Detect monitor tells you when it is running but out of reach, and Install & Enroll upgrades it.

Once enrolled, the monitor streams live metrics: CPU, memory, per-mount disk, network interfaces up, detected services, firewall and fail2ban state — and on GPU hosts, per-GPU temperature and utilisation, so a mining or ML rig surfaces thermals. list_monitors reports the fleet's helpers, their versions and latest metrics. Crucially, the monitor keeps watching and can alert even when the Mac app is offline, notifying directly through its own channels and replaying anything it buffered when the app returns. Logs windows prefer the monitor too — it runs as root on the server, so it reads privileged logs without needing the sudo password at all.

An enrolled host's pane also keeps a Behavior & History chart — CPU, memory, disk and temperature at five-minute resolution for up to 30 days, against a learned baseline that flags unusual behaviour. Temperature uses its own degree-marked scale, so it remains readable beside percentage-based resource use. Vertical lines place incidents on the same timeline: a solid red line is a server reboot (derived from the reported uptime), a dashed orange line is a watchdog issue — a forced service restart, a database-corruption warning or a reboot escalation. If the OS crash reporter starts using CPU (a process crashing repeatedly), an orange Crashes curve joins the chart, so "MySQL was force-restarted twice last night and something was crash-looping" is visible at a glance. Open Crash Logs… under the chart opens the server’s log viewer on its crash, hang and panic reports (macOS DiagnosticReports), so you can see which process that curve was about.

The monitor is read-only by default. Turning on Allow actions in its Host Monitor pane unlocks a small, fixed set of control verbs — never arbitrary shell — driven by the gated monitor_action tool: restart/reload/start/stop an allowlisted service, flush DNS, reload the firewall or fail2ban, or reboot. Each action is logged on the server. Targets cover Linux (amd64/arm64/arm — covering every Raspberry Pi), macOS (including legacy Intel builds back to OS X 10.11 El Capitan), FreeBSD (pfSense/OPNsense/TrueNAS) and Windows.

Keeping a monitor healthy over time is handled for you. Rotate monitor credential replaces its bearer credential in place: the new secret is generated on the Mac, sent over your approved FS1 signing key, and activated on the server without ever being shown or handed to an AI. With Allow self-update enabled, pushing a newer build is safe by construction — the candidate binary's release signature is verified, it must pass a compatibility self-test before it is given root, and the previous binary is kept as a rollback; if the new one fails to answer its health check, FrontierStack restores the old one automatically. Pull-mode updates fetch the binary through a short-lived, single-use grant rather than a durable token. When a monitor is verifying FS1-signed requests, its pane shows a FS1 signed badge.

8.9When a server stops answering: what FrontierStack last saw

The server that has stopped answering is exactly the one whose details you need: its address, to try another route; its hardware, to find it in the rack; the services it was running, to know what is affected. So FrontierStack keeps a record of each linked server as it last saw it while the server was answering, and shows that record instead of a pane full of blank checking… fields.

When a server stops answering, its pane opens with an orange line — Not answering — details below are what FrontierStack last saw, followed by when that was — and the remembered values appear dimmed, each marked with a clock icon. Under Services, a Running when last seen list shows what was running at that moment. A dimmed value is not a live reading: it is what was true then, and the server may have changed since. As soon as the server answers again, the live values replace them.

Two rows in the Status section are new, and live whenever the server answers. IPv6 lists the stable IPv6 addresses. Scheduled reboot shows any restart the operating system has been set to perform on its own — the first thing to check when a server “mysteriously” reboots at the same time every week.

The AI Administrator sees the same remembered details, labelled as last seen and dated, so “what was the IP of the server that just went down?” has an answer even when the server can’t give one (Chapter 13).

8.10Lockup reboot: when the server is on but nothing gets in

The Service Watchdog (Chapter 11) heals one service at a time. Some failures take out the whole machine while every service still looks fine from inside: a Mac whose SSH and Screen Sharing stop responding, or a server that has quietly lost its network. Nothing outside can reach it to fix it, so the fix has to come from the server itself. Lockup reboot, in the server pane’s Service Watchdog panel, is that fix. It needs monitor 1.23.0 or later; the panel says so if the server’s monitor is older.

A check only arms after it has worked at least once since the monitor started. A server with no sshd, or with no gateway the monitor can find, is therefore never rebooted by it — the check simply keeps waiting. The server also refuses to reboot unless Allow actions and Allow reboot are both on in the pane’s Remote Actions section, and the panel warns you when they are not.

While Lockup reboot is on, the panel shows each check’s live state — answering · armed, waiting for a first good check, or failing since a given time — the gateway the network check is using, and the time of the last automatic reboot.

8.23.7The root reboot sentinel

The wedge that prompted this was stranger than a crash: the server’s ports still accepted connections, but sshd and Screen Sharing never answered. The machine could no longer start new processes — so a reboot that begins by starting a shutdown command could not run either. From monitor 1.24.0, a reboot sentinel is installed alongside the monitor on macOS and Linux: the same binary, running as root, with no network listener and no token.

The sentinel has no policy of its own. It acts only on the monitor’s own Lockup reboot settings and permissions, as a backstop: if the monitor’s reboot should have happened and hasn’t, the sentinel reboots the server 3 minutes later (13 minutes into a failure with the default 10). On that path it starts no programs and does not wait on the disk; it calls the kernel’s reboot itself, so it still works when the server cannot start a new process. Just before rebooting, it records whether a process could be started — evidence afterwards for “this server ran out of process slots”. The panel shows whether the sentinel is installed and reporting, when it would act, and its last reboot. If it says the sentinel isn’t installed, update the monitor.

8.23.8Process counts

Running out of process slots is how a server ends up accepting connections it cannot start sshd or Screen Sharing for. From 1.24.0 the monitor samples process counts every minute: the total against the kernel’s limit, the per-user limit, and the users running the most processes. The Processes rows in the Lockup reboot area show the current figures, a Last 24 hours chart with its peak, and an orange warning when the total, or one user’s count, nears its limit — the moment to find the runaway before it becomes a lockup.

8.11Cloud Servers, Server Clone & Remote Apache

Alongside the compute panes, AWS ▸ DynamoDB lists the tables in the selected region with their status, item count, size and partition/sort key. It leans on the two things that actually cost you: the capacity mode capsule separates provisioned tables — which bill for their read and write units continuously, whether or not anything touches them — from on-demand tables that bill per request; and a no PITR flag marks any table without point-in-time recovery, which is off by default on every new table and is the only way to rewind after a bad write. Point-in-time recovery can be switched on or off from each row's menu. DynamoDB has no bulk describe call, so each table is queried individually and the list is capped at 40 per region.

Fleet & Remote ▸ Cloud Servers rolls up one inventory across every cloud provider that has a configured API token — DigitalOcean, Vultr, Linode, Hetzner, Sakura, Contabo — plus AWS EC2, Lightsail and RDS. It is a read-only roll-up (power control stays in each provider's own pane). Each running instance with an IP offers a Services button to discover what's listening, and a + Helper button that links the instance over SSH and installs the monitor in one move — just set the right username and key in the sheet.

Google Cloud joins the same inventory through the Google Cloud CLI pane (Hosting). Installing there also runs gcloud auth login in your browser, and FrontierStack then uses your own login without storing a key. Choose a project or All projects, then work with Compute Engine VMs: SSH, start, stop and reset, with a confirmation before a stop or reset. The AI Administrator reads through diagnose and needs your approval for anything other than a list command.

Microsoft Azure joins it the same way through the Azure CLI pane (Hosting). Installing there also runs az login in your browser. Choose a subscription or All subscriptions, then SSH to Linux VMs, or start, stop and restart them, with a confirmation before a stop or restart. Stop deallocates the VM, because an Azure VM that is only powered off still costs compute. The AI Administrator reads through diagnose, limited to a fixed set of commands because many Azure list commands return secrets, and needs your approval for anything else.

Server Clone is a guided wizard that stands a fleet server up to match this Mac: it installs the detected Homebrew stack, copies configs and site files to their original paths, pushes your domain configs and reloads the web server, and can clone MySQL databases (local mysqldump piped into mysql on the target over SSH). Every step runs over key-based SSH and is safe to re-run; existing data isn't deleted.

When a single site is ready to go live, the Promote to Production push (from a domain's context menu) sends its vhost config to the same path it uses locally, optionally copies the document root and database, points DNS at the target via Cloudflare, and can issue a real Let's Encrypt certificate with certbot — files, database, vhost, DNS and HTTPS in one push.

You can also manage a remote server's Apache directly. The Apache pane gains a host selector (This Mac / each linked server) when any remote host exists; pick a server and FrontierStack discovers its Apache over SSH — version, config layout, and every active vhost with its real log paths — and lets you edit vhosts, modules, MIME types, ports and WebDAV, with each change validated (httpd -t) and gracefully reloaded before it sticks, rolling back if the config is invalid. It even understands the legacy macOS Server.app Apache tree. From the Sites list, Push to Server… renders a local site's vhost onto a monitor-linked server with its document root swapped in. Chapter 7 covers the local Apache and Sites workflow these mirror.

When a list has five or more sites, Test All appears beside it, for a linked server or for this Mac. It requests every site once from this Mac, connecting straight to that server, and lists what each answered, with problems first. To start or stop a web server, Control-click the server in the sidebar and choose Start Web Server or Stop Web Server. Stopping asks first.

8.12Screen sharing to a host

Each server's pane offers whatever remote-access method its probe detected — Apple Screen Sharing / VNC for a Mac, RDP for Windows, and AnyDesk, TeamViewer, RustDesk or NoMachine when their clients are present. When Apple Remote Desktop is installed on this Mac it registers the vnc:// handler and captures the click, so the pane offers a separate Screen Sharing (macOS) action that always opens Apple's built-in Screen Sharing client instead. If the host is reachable at more than one address — a LAN IP and a VPN or Tailscale address, say — that action becomes a menu so you pick which network to connect over; each choice dials the numeric IP directly, so it still works when the host's name isn't resolving (mDNS, split-horizon DNS, or a VPN quirk). A matching “— by IP” button appears for VNC and RDP whenever the pane would otherwise use a name that might not resolve.

8.23.9Your own RustDesk server

RustDesk is the open-source remote desktop you can run entirely yourself, and its pane (Fleet & Remote ▸ RustDesk) does more than open the client. It watches your self-hosted RustDesk server without credentials — a real rendezvous exchange with the ID server (hbbs, TCP 21115, so an unrelated open port can't pass), TCP 21116, and the relay (hbbr, TCP 21117) — and raises an Alert when it stops answering.

It also shows which server this Mac's RustDesk client uses, and warns when the client is still on RustDesk's public servers or holds a missing or different key. To bring clients across, paste each server's public key (id_ed25519.pub) and copy either the config string clients import under RustDesk ▸ Settings ▸ Network, or the file name to give the Windows installer so it configures itself. RustDesk Server has no macOS build, so Run your own RustDesk server gives a Docker compose file for a Linux server or VPS, with the ports mapped explicitly and hbbr -k _ so the relay serves only clients that hold your key.

8.13KVM-over-IP: console and power, lights-out

SSH and Screen Sharing need the machine to be up and on the network. When it isn't — a kernel panic, a BIOS/firmware screen, a network stack that never came up — you need out-of-band access: a KVM-over-IP appliance that captures the real HDMI output and injects USB keyboard/mouse, independent of the operating system. The Remote KVM pane registers as many of these as you own — PiKVM, JetKVM, TinyPilot, NanoKVM, GL.iNet Comet, or any generic unit — each pinnable, each one click from its web console down to the BIOS.

Units with power control close the loop. A PiKVM with an ATX board, a network PDU, or GL.iNet's Comet Pro (GL-RM10) — which is a KVM and a smart power plug in one device — can switch mains power, so you can hard power-cycle a truly frozen box. Link such a unit as a server's power source in its Power section (beside SwitchBot, UPS and PDU commands) for On/Off/Cycle. The AI Administrator can drive it too: list_kvms, kvm_power, and server_power — the last reaches a host's linked plug/PDU/KVM to restart a hung server that won't answer reboot_host.

8.14Console connections: serial and console-server access

A router or switch that has lost its network configuration can't be reached by SSH, its web UI or a KVM — but its console port still works. The Console Connections pane (Fleet & Remote) is a serial terminal built into FrontierStack. Plug in a USB-to-serial adapter (FTDI, Silicon Labs CP210x, WCH CH340/CH9102, Prolific), or connect the device's own USB console port (Cisco's USB mini-B or USB-C, Netgate firewalls), and it appears under Attached serial ports within a couple of seconds, with the adapter's chip named. Quick Connect opens it at a chosen speed without saving anything. Save as… stores a connection with a vendor's console settings: Cisco, Juniper, Arista, Aruba CX, ProCurve, Ruckus, Fortinet, Palo Alto, SonicWall, WatchGuard, pfSense/OPNsense, MikroTik, Ubiquiti, TP-Link Omada, Dell OS10, Allied Telesis, APC, Raspberry Pi, Linux, and board-level UARTs on OpenWrt and GL.iNet routers.

Every setting stays editable: speed (any rate the adapter supports, including 460800 and 921600), data bits, parity, stop bits and flow control, plus what Return sends (CR, CR LF or LF), Backspace as ^H, local echo, and LF → CR LF for devices that stair-step down the screen. If an adapter is moved to another USB socket and macOS renames it, FrontierStack finds it again by its USB serial number. A console server (Lantronix, Opengear, Digi, Avocent ACS, a Cisco terminal server's reverse telnet, or ser2net on a Raspberry Pi) is added as a telnet, raw TCP or SSH connection.

Each session opens in its own window. Break sends a line break — the signal Cisco ROMMON, Juniper and most boot loaders wait for during password recovery. DTR and RTS can be toggled. A paste delay sends a pasted configuration one line at a time, so a 9600-baud console doesn't drop characters. A session log saves everything the device sends to ~/Library/Logs/FrontierStack/Console, and Export saves or emails the screen. Link a connection to a pinned device and it appears in a Console section on that device's page; a console added from there starts with the platform's usual settings. The AI Administrator can list your consoles and attached adapters (kvm, action consoles), but it never opens a session or types into one.

8.15Boot Media: installing an operating system over the network

Reinstalling a server used to mean a trip to the rack with a USB stick. The Boot Media pane (Fleet & Remote) replaces the stick: you keep installer images in one library on this Mac, and FrontierStack presents the one you choose to the target machine as a virtual CD — through the server's management controller, a PiKVM, a Proxmox VE virtual machine's CD drive, or a network boot menu. Macs are handled differently, because Apple no longer lets them boot that way; see Macs below.

8.23.10The image library

Add ISO or Disk Image… records a local file; the file stays where it is and nothing is copied. Add Image URL… records an image that already lives on a web or file server (http, https, nfs or cifs) — a remote URL is handed straight to the target, so this Mac never has to carry the bytes. Compute SHA-256 records the file's checksum so you can compare it with the vendor's published one before installing anything from it. Each entry can be renamed, shown in Finder, removed from the library, or its served address copied with Copy Image URL. An entry whose file is no longer at its saved location is flagged as missing.

8.23.11The built-in media server

A management controller mounts an ISO by reading it over HTTP, piece by piece, for as long as the install runs. The pane's Media server serves the library's local images for exactly that: it supports byte-range requests and keep-alive, which virtual CDs need, and shows a log of recent requests and the bytes served per image, so you can see a controller actually reading. It starts automatically when a target needs a local image, or you can press Start Serving yourself.

8.23.12Server management controllers (Redfish)

Most rack servers carry a baseboard management controller that can present an image as a virtual CD. FrontierStack drives it through the standard DMTF Redfish API: Dell iDRAC 8 and 9, HPE iLO 4, 5 and 6, Lenovo XClarity Controller (XCC), Supermicro X12 and newer, and any other controller that implements standard Redfish virtual media.

1. Add Controller and enter its address, port, username and password. The password is kept in the Keychain and sent only to the controller. Optionally link the controller to a fleet server so the two are shown together. Use an account with the Virtual Media and Control privileges.
2. Read Controller discovers what it manages: each system with its model and power state, and each virtual CD slot with whatever is mounted in it now.
3. Choose the System, the Virtual CD and the Image. Leave Boot from it once on to boot the CD a single time, after which the server returns to its normal boot order.
4. Under Then, choose Don't restart, Graceful restart or Force restart, and press Mount (or Mount and Restart, which asks for confirmation first).
5. When the install is done, Eject the virtual CD.

8.23.13IPMI controllers and network boot once

Older boards have a controller with no Redfish, or with Redfish but no virtual media: Supermicro X10/X11, ASRock Rack, Gigabyte, Intel and iDRAC 7. Set such a controller's Protocol to IPMI over LAN. FrontierStack then drives it with ipmitool (Install ipmitool installs it with Homebrew in the in-app console). The password stays in Keychain and is passed to ipmitool in its environment, never on the command line. Read Controller shows the power state. IPMI has no standard virtual media, so the way to install an OS through it is the network: Network Boot Once sets the next boot to PXE (chassis bootdev pxe, in UEFI mode unless you tick Legacy BIOS boot), then optionally sends a hard reset, or powers the server on if it is off. IPMI has no graceful restart.

Redfish controllers have Network Boot Once too: a one-time Pxe boot override followed by your choice of restart. Combined with the built-in PXE server below, this boots a remote server into this Mac's ISO menu with no virtual media at all.

8.23.14PiKVM virtual media

A PiKVM can emulate a USB drive for the machine it is plugged into. Boot Media uses PiKVM's documented /api/msd interface, with the same PiKVM API account as its ATX power control (see KVM-over-IP). First copy an image to the PiKVM's own storage: a local file is streamed up with a progress bar, or the PiKVM downloads a remote URL itself. Free space is checked before the copy starts. Then connect a stored image As CD-ROM (for an ISO) or As Flash (for a raw .img), pick it in the machine's boot menu or use Power Cycle… (a hard ATX cycle, with confirmation), and open the console to follow the install. Disconnect Drive and delete tidy up afterwards.

8.23.15Proxmox VE virtual machines

For a virtual machine on Proxmox VE (connected as described in Proxmox VE below), choose the Node, the Virtual machine and the ISO storage, then the Source. Image from the library has Proxmox download the image into ISO storage itself (its download-url call), so a local image is fetched from this Mac's media server; ISO already on Proxmox uses one that is there. FrontierStack attaches the ISO to the VM's CD drive and can put the CD first in the boot order — the original order is remembered and restored when you eject. Finally choose to leave the VM as it is, start it, or reset it (a hard reset, which asks for confirmation).

The API token needs Datastore.AllocateTemplate and Sys.AccessNetwork for the download, and VM.Config.CDROM, VM.Config.Options and VM.PowerMgmt on the VM. The pane lists these beside the controls.

8.23.16Network boot (PXE / iPXE)

Machines with no controller and no KVM can still boot over the network. Turn on Serve an iPXE boot menu from this Mac and the media server also publishes an iPXE menu that SAN-boots any library ISO over HTTP, chains to netboot.xyz for its catalogue of installers, and offers an iPXE shell and boot from local disk. Download iPXE Boot Loaders fetches the official loaders from boot.ipxe.org (ipxe.efi, undionly.kpxe and the arm64 ipxe.efi) so UEFI HTTP Boot clients can load iPXE straight from this Mac.

What remains is telling your DHCP server where to send booting machines. Pick your DHCP server and press Copy Settings for a ready-made snippet:

8.23.17The built-in PXE server

With no dnsmasq on the network, and no change to its DHCP server, turn on Answer PXE clients from this Mac under the network-boot settings. The Mac then runs two small services while serving is on:

• ProxyDHCP (UDP 67 and 4011) replies only to machines asking to boot from the network (PXE firmware, UEFI HTTP Boot, iPXE), and only with a boot file. It never hands out an address, so the router's DHCP keeps working exactly as before.
• TFTP (UDP 69, read-only) serves the three iPXE loaders and nothing else. BIOS machines get undionly.kpxe, UEFI x64 machines get ipxe.efi, and ARM64 machines get the ARM64 ipxe.efi. UEFI HTTP Boot clients get the loader over HTTP instead. Once iPXE is running, it asks again and receives the menu.

Choose the Network it answers on (the services are bound to that one interface), and optionally list the MAC addresses of the only machines it should answer. Each request and transfer appears below the switch. The loaders are downloaded automatically the first time. If pf or the Application Firewall is on, the server section's firewall button now opens UDP 67, 69 and 4011 as well.

8.23.18Other KVM-over-IP units

JetKVM firmware. For a JetKVM, JetKVM Mini or Mini W, Remote KVM shows the installed firmware and whether an update is waiting. FrontierStack reads the version from the KVM's own web interface without a login, asks JetKVM's update server for the latest release, and re-checks every half hour. Firmware before 0.5.4 does not verify that a downloaded update really came from JetKVM (CVE-2026-32294) and is marked Update needed. If you turn on MQTT in the JetKVM and enter its base topic, FrontierStack reads the KVM's own update state instead, along with online status, CPU, temperature, memory and storage. A JetKVM normally installs updates by itself; one stays pending when automatic updates are off or someone is always connected to its console. Alert me on the KVM's row raises an alert when it stops answering, runs the vulnerable firmware, or has had an update waiting for more than a week.

KVM storage. Most compact KVMs run from an SD card or built-in eMMC, and a worn-out card is a common reason one stops booting. Each KVM's Storage section records what it runs from (microSD, SD, eMMC, USB or SSD), the capacity, the card's make and model, when it was fitted, and notes such as where the spare is kept. A removable card that has been in service for three years or more shows a wear warning. For a PiKVM, Read from PiKVM shows the free space on the card's image partition, using the same API account as ATX power; a JetKVM's storage figure appears once MQTT is on. FrontierStack never reads from or writes to the card itself.

JetKVM, GL.iNet Comet, NanoKVM, TinyPilot, Raritan, ATEN, Avocent, Lantronix and Adder units have no stable public API for virtual media, so FrontierStack does not pretend to drive it. Instead Copy URL starts serving and copies an image address the KVM can reach, and the pane shows where to paste it in that vendor's own console.

8.23.19Macs

A Mac cannot boot an installer ISO, and network boot is gone: Apple silicon has none, and Intel NetBoot lost its server in macOS Server 5.7 and is refused by T2 Macs at Full Security. So for Macs, Boot Media installs macOS the way Apple supports — by downloading Apple's full installer on the Mac itself over SSH and running it. Every remote Mac in the fleet gets a card (add Macs under Remote Servers first):

• Refresh reads its macOS version, whether it is Apple silicon, Intel or Intel with T2, the model, free space, the installers already present, and whether Content Caching is on.
• List asks Apple which full installers are available (softwareupdate --list-full-installers); Download on the Mac fetches one in the background (--fetch-full-installer) with progress.
• Install… runs startosinstall in one of two modes: Upgrade / reinstall (keeps data) or Erase and install. On Apple silicon only a volume owner may start an install, so you enter an administrator user and their password; the password is passed to startosinstall on standard input and never stored.
• Turn On Content Caching (on this Mac or any fleet Mac) keeps a copy of Apple's updates and installers on the LAN, so every other Mac downloads them from there rather than from Apple.
• For an Intel Mac with a linked KVM, Internet Recovery… restarts it into Apple's own network recovery, after confirmation. Recovery has no Screen Sharing or SSH, which is why a KVM is required to drive it.

8.23.20From the AI Administrator

The AI Administrator and MCP clients reach Boot Media through the existing kvm tool: media_list (read-only) reports the library, the media server's state, each controller's power and mounted image, and each PiKVM's drive; media_attach mounts an image on a target — a controller or PiKVM by name, or proxmox:<vmid> — and with boot=true also restarts into it; media_eject removes it. Attaching, ejecting and restarting need Allow changes and always ask you to confirm (Chapter 13).

8.23.21Serial-console installs

Appliances, small x86 boards such as the UP Squared, and servers wired only to a console port have no screen to install on, so the installer has to run on the serial line. Turn on Install over a serial console in Boot Media and choose the machine's port (ttyS0–ttyS7; a UP Squared's debug UART is ttyS4) and the speed — 9600, 19200, 38400, 57600 or 115200 baud, or 230400–921600 for UARTs and adapters that can take it. From then on the iPXE menu sets console=tty0 console=ttySn,baudn8 for every Linux installer it starts through netboot.xyz, and tells netboot.xyz not to send screen-clear codes.

An ISO's own boot menu can't be changed from outside, so the section shows the exact edit for each installer: the GRUB or ISOLINUX kernel line for Ubuntu, Debian (after the ---, so the setting carries into the installed system), Rocky/Alma/RHEL (with inst.text) and other Linux ISOs; Proxmox VE's Terminal UI, Serial Console entry (fixed at ttyS0, 115200 — edit it for another port or speed); pfSense's memstick and OPNsense's serial image, or the FreeBSD loader commands for any other speed; and, for Windows, which has no serial setup, the EMS commands for after installation. Copy GRUB Settings gives the /etc/default/grub lines that keep the installed system on the serial line. Open Console at n Baud opens a session on the USB adapter plugged into this Mac at the same speed, and saved consoles set to another speed can be switched in one click.

8.16Proxmox VE: live cluster status and guest power

A Proxmox VE host is a server full of servers, and SSH to the node alone does not tell you much about its guests. The Proxmox VE service page (Containers category; see Chapter 6) therefore talks to Proxmox's own API. Add Connection and enter a cluster or single node's address with an API token — the token ID in user@realm!name form and its secret, which is kept in the Keychain and sent only in the Authorization header. Proxmox's self-signed certificate is accepted only on a private address. A token with the PVEAuditor role is enough for status; power control and boot media need the extra permissions listed above.

Once connected, the page shows the cluster name, Proxmox version and quorum; for each node its CPU, memory, root disk and uptime; every VM and container with its state; and a usage bar for each storage. Each guest can be started, shut down or rebooted, and — after confirmation — hard-stopped or reset. Show in Server Activity and Boot an ISO… jump to the other two places Proxmox appears.

In the Server Activity window (Chapter 11) every node is a tile in the All Servers, Cluster and Everything layouts, with CPU, memory, disk and uptime sparklines and a line such as “5/7 VMs · 2/3 containers running”. A node's detail shows its KPIs, a CPU and memory chart, the guest list with the same power controls, its storage and the cluster facts, plus Open Web UI, Refresh and Boot an ISO….

Alerts watches each connection in the background, refreshing about every five minutes: API reachability, each node's online state, cluster quorum, and any storage more than 90% full. The AI Administrator has the same view through the kvm tool's proxmox_status action (read-only) and can change a guest's power with proxmox_power, which always asks for confirmation.

8.17Garage: object-storage clusters at a glance

Garage is a lightweight S3-compatible object store that spreads its data over several small nodes, often in different places. The Garage service page (Object Storage category) watches any number of Garage clusters through Garage's admin API. Add Cluster, then enter the admin endpoint of one node — 10.0.0.5:3903 is enough on a LAN, since port 3903 and plain HTTP are assumed — and the admin token. One node reports the state of the whole cluster; add a second endpoint on another node so the cluster stays visible when the first one is down. The token is kept in the Keychain and sent only in the Authorization header, and plain HTTP is used only for a private, Tailscale or .local address; anything else must be https://. Both Garage 2.x (admin API v2) and 0.9/1.x (v1) are understood.

The section's first line counts every cluster as healthy, needing attention, or down. Each cluster then shows its health — healthy, degraded (some data has fewer copies than it should) or unavailable (some data can't be read or written) — with the nodes connected, storage nodes up, partitions with quorum and fully replicated, the layout version, zones and assigned capacity. Below that comes every node, with down nodes and the fullest disks first: zone, capacity, Garage version (a node on a different version from the rest is flagged), draining, and how full its data and metadata disks are. Turn on Also read /metrics to count blocks that failed to resync on the listed nodes — a number that should always be zero.

Alerts checks each cluster about every five minutes: admin API reachability, cluster health, each node in the layout, data and metadata disks at or above the cluster's threshold (90% unless you change it), and errored blocks. The AI Administrator reads the same view with get_server_health area object_storage. Nothing here changes the cluster.

8.18Old Macs, Raspberry Pis & Windows hosts

Part of the point of a fleet is keeping older or odd hardware useful. A vintage Xserve or Mac mini still running macOS Server links like any other host; the Remote Apache discovery understands Server.app's own Apache tree and its internal ports, so its websites show up and can be edited, and the legacy Intel builds of the monitor run as far back as OS X 10.11 El Capitan. A shelf of Raspberry Pis links as ordinary Linux boxes — the monitor's arm build covers every Pi including the Zero — so you watch their CPU, disk and services right alongside everything else.

Pop!_OS and other Ubuntu-based systems are recognised by their distribution rather than by the word “Linux” in the OS name. Pop!_OS reports itself as “Pop!_OS 24.04 LTS”, which earlier releases didn't treat as Linux, so the monitor's self-update refused it (as it did some Ubuntu hosts). On apt systems, Check for Updates in the server's package tools (Manage in Server Activity…) also reports what apt alone doesn't: a newer kernel that's installed but not yet running, even where /var/run/reboot-required is never created, as on Pop!_OS and some Debian installs; the free space on the EFI system partition — Pop!_OS boots with systemd-boot and kernelstub copies every kernel and initrd there, so a nearly full partition is the usual reason a kernel update fails half-way; and how many Flatpak apps have updates.

Windows hosts join over Microsoft's OpenSSH server: link them like a Linux box for SSH diagnostics, and install the monitor as a native Windows Service (because Windows services don't reliably inherit machine environment, its config is read from a file beside the executable). Windows monitor installs are done manually with the supplied PowerShell installer from the Host Monitor pane, rather than auto-pushed.

For interactive control beyond diagnostics, the catalog also carries the remote-access tools — RustDesk, MeshCentral, Apache Guacamole, Windows Remote Desktop and WinRM — but those are services you run, not the SSH fleet itself. The fleet's strength is uniform, scriptable, low-overhead management of everything from one window — and, with Chapter 13's run_script targeting any node by name, an AI that can diagnose and fix a remote server two steps at a time: a read-only test, then a minimal fix, re-run to confirm.

UP Squared and other UP boards. AAEON's UP boards — UP Squared, UP Squared Pro, V2, 6000 and Pro 7000, UP 7000, UP Xtreme — are small x86 machines that often run headless as a firewall, a Proxmox node or a Linux box. They link like any Linux host, and the status check reads the board's firmware identity (DMI; serial numbers are never read), so the server's page names the model and shows where its 3.3 V debug UART is and which ttyS Linux gives it (ttyS4 on a UP Squared, ttyS5 on the first UP Xtreme), with the firmware keys (F7 boot menu, Del or Esc setup) and the BIOS path that turns on PXE. Console Connections has a matching UP Squared / UP board preset, Device Discovery recognises AAEON's MAC prefix, and Boot Media's serial-console install takes it from there. The same check names other hardware too — Dell, HPE or Supermicro servers, and Raspberry Pis by their device-tree model.

8.19Group Policy & domain trusts

For Windows domains, the Group Policy & Trusts pane (under Identity & Accounts, Full Fleet) reads a linked domain controller over SSH. A Windows Server DC needs the Group Policy Management feature and the AD PowerShell module (both are there on a normal DC); a Samba AD DC is read with samba-tool, and with the server's sudo password saved, ldbsearch adds where each GPO is linked. The pane has three tabs.

What it flags. For Group Policy: GPOs whose Active Directory and SYSVOL versions disagree (replication is broken if it lasts), SYSVOL still on FRS or half-way through migration to DFSR, a default policy that is missing or unlinked, and, as information, GPOs that are unlinked, empty or fully disabled. For trusts: a trust whose secure channel fails (nltest /sc_query on Windows, samba-tool domain trust validate on Samba), SID filtering turned off on an external trust, SID history allowed across a forest trust, TGT delegation across a trust, RC4-only trusts, down-level (NTLM) trusts, and two-way trusts that use forest-wide rather than selective authentication. Inbound-only trusts can only be tested from the other domain.

Actions (Windows DCs only): Report opens a GPO's full settings report (Get-GPOReport) in your browser; Back Up All GPOs… runs Backup-GPO into a new dated folder under ProgramData\FrontierStack\GPOBackups on the DC; Force Refresh on a Computer… runs Invoke-GPUpdate on one domain computer. FrontierStack never creates, edits, links or deletes a GPO or a trust.

Tick Watch in Alerts and the DC is checked every 30 minutes; a failing trust or a DC that stops answering raises an alert under Directory Services. The AI Administrator reads the same state with get_server_health and area=domain.

8.20OpenCore Legacy Patcher Macs: is it safe to update?

Some of those older Intel Macs will be running a macOS newer than Apple ever shipped for them, courtesy of OpenCore Legacy Patcher (OCLP). That changes what a reboot means. An ordinary server Mac boots power → macOS → FrontierStack agent; an OCLP Mac boots power → firmware → EFI/OpenCore → macOS → OCLP root patches → FrontierStack. Every extra link is a place where an unattended boot can stop: a macOS update replaces the sealed system volume and takes the root patches with it (Wi-Fi, graphics acceleration and Bluetooth vanish until they are re-applied), a major upgrade needs a matching OCLP release first, and if the firmware's startup entry ever stops pointing at OpenCore the machine sits at a boot picker until someone holds Option and chooses “EFI Boot” — considerably more troublesome when that Mac is in another building.

The OpenCore Patcher pane (Fleet & Remote) exists so you know this before you press Update. Pick This Mac or a linked Mac and it runs a read-only probe — the OCLP app version, OpenCore's NVRAM stamp (only present when the Mac actually booted through OpenCore), the root-patch receipt, bless --getBoot, SIP, Software Update settings, the real model behind OpenCore's spoofed one — and turns it into a Remote boot safety list: firmware boot entry → OpenCore, default boot volume set, FileVault's effect on unattended reboot, root patches present and current, OCLP release supports the installed macOS (checked against the newest GitHub release), pending macOS update, automatic macOS installs off, and an EFI/OpenCore backup. Above it sits one verdict: Safe to update macOS, Update with caution, or Not safe, with the reasons.

The pane deliberately does not administer OCLP — building OpenCore and applying root patches stay in OCLP's own app. It offers three fixed, reversible admin actions: turn off automatic macOS installs (so System Settings never updates an OCLP Mac unattended), back up EFI (an archive of the EFI/OC folder kept on that Mac), and re-bless OpenCore (the same bless --setBoot OCLP's “Install to disk” runs, refused unless OpenCore.efi is really there). The fleet status probe also notices OCLP on every pinned Mac: the host row and the Server Activity window show it, and Alerts raises a Security item when root patches are missing or the boot entry bypasses OpenCore.

8.21A Build Mac: Xcode builds on a server

Xcode’s Build button always builds on the Mac in front of you. To build somewhere else — a Mac mini in the rack with more cores, a stable signing setup and nobody’s editor open on it — give a linked Mac the Build Mac role, a Full Fleet feature. Use the toggle in its server pane or in the server editor, and the pane gains a Build Mac section.

The section shows what the Mac can build with. It is checked over SSH every 30 minutes, read-only: the installed Xcodes and the selected one, whether the licence is accepted and the first-launch components are installed, free disk, how much DerivedData, archives and simulators take, the simulator runtimes, and every code-signing certificate with its expiry date. Problems go to Alerts under Build Macs: no Xcode, licence not accepted, under 30 GB free (you can change the limit), or a certificate expiring within 14 days.

Projects are saved on the server: a git URL, branch, optional project or workspace, scheme and destination. Build, Test and Archive clone or update the repository into ~/fs-builds on the build Mac and run xcodebuild there in the console, so the whole log is in front of you and you can cancel it. The build Mac needs its own access to the repository. Archive unlocks the login keychain first, because signing over SSH can’t reach a locked keychain.

Two set-up steps make it a real build server. Set Up Gitea Runner installs act_runner, registers it with your Gitea and keeps it running with the label macos-xcode. You paste the registration token in the console, and FrontierStack never stores it. Then a workflow with runs-on: macos-xcode builds and tests on every push, the job Xcode Server used to do. Turn On Headless Xcode MCP enables Xcode 27’s xcrun mcp-server for the build folder, so coding agents can use Xcode’s tools on that Mac without an Xcode window.

8.22Migration wizards: moving a stack onto a new device

Retiring an old server rarely means "reinstall everything by hand." FrontierStack ships guided migration wizards that lift a stack off a legacy machine and stand it up on a destination you choose — this Mac, another Mac, or a Linux box. The source is only ever read; everything is written to the target.

Migrate Setups starts with Migrate from: a stack found on this Mac (MAMP, XAMPP, Pancake Stack), Apple Server (a whole Server.app machine, below), Windows, Web Servers (other web servers, dev tools and dev servers) or Other Tools (what isn't a website: databases in other apps, Git servers, hosts files and repos). Each runs in the order the work happens — what moves, where it goes, move it — and ends with Check everything, then retire. As with Retire Apple's PHP in Modernize Old Mac Server, nothing is ready to retire until it has been named and checked: every imported site's pages and files are fetched through the new Apache (pages by status and PHP source, plain files byte for byte against the source's copy) and every database table's row count is compared with the source. Scripts other than a folder's index page are never opened, because opening one can run it. Whatever differs or couldn't be checked is listed by name, and the pane tells you to keep the old setup until the check passes. A move to another device runs the same comparison for files (rsync checksums) and tables in the console, and copies made from other database apps are checked table by table too.

The Migrate Setups pane handles a MAMP, XAMPP or Apple Server.app web stack. It auto-detects the source, then lets you tick exactly what to carry: the Homebrew tools, web files, MySQL databases, the sites (vhosts), MIME overrides, Apache modules, PHP settings, WordPress wp-config.php fix-ups and Git repositories. A Set up on picker chooses the destination: leave it on This Mac for the full in-app migration, or pick a linked host to install the stack on that new device over SSH, copying the web files (rsync) and databases (dump over SSH) across — with a Linux toggle to use apt/dnf instead of Homebrew.

Other Tools ▸ Servers and settings also has Import from a Repo…: point it at a folder or git repo that holds an SSH config, an Ansible inventory or .env files, and it adds the servers to your fleet and the credentials to Script Secrets or the matching service, after you review them. FrontierStack AI can read other layouts without ever seeing a value (Ch 13).

Coming from another tool? Import from a Web Server or Dev Tool… (Migrate Setups ▸ Web Servers, and beside Import Sites… in the Apache pane) reads sites, settings and databases from Laravel Herd, Laravel Valet, ServBay, AMPPS, DesktopServer, Devilbox, DDEV, Docksal, Lando, Local and DevKinsta; from web servers' own config files — Caddy, nginx, Lighttpd, OpenLiteSpeed, H2O, Cherokee, Abyss, Tomcat, Jetty, WildFly/Undertow, CivetWeb and Mongoose; and from dev servers that are running right now (Python http.server, Uvicorn, Gunicorn, Waitress, Vite, webpack-dev-server, BrowserSync, http-server, live-server, miniserve, Node.js, Bun and Deno). Each site becomes an Apache site on ports 80/443 — serving its files, or forwarding to an app server that keeps running — or a Localhost project. Domains, aliases, HTTPS certificates, redirects and proxied paths carry over, and you can rename any domain before importing. The Databases tab finds every MySQL, MariaDB and PostgreSQL server running on this Mac — DBngin, Herd, ServBay, MAMP, Local and others — dumps the databases you tick with that server's own tools and loads them into FrontierStack's Homebrew MySQL or PostgreSQL, then compares every table's row count with the source (it's also on its own as Other Tools ▸ Copy Databases…). Docker-based projects (DDEV, Lando, Docksal, Devilbox, DevKinsta) are exported with their own command. An opt-in switch also points each copied site's wp-config.php or .env at FrontierStack's MySQL, keeping a .pre-migrate.bak copy.

Coming from Windows? Copy the whole C:\laragon folder to the Mac (USB drive, shared folder, a Parallels/VMware Windows disk or a ZIP) and use Migrate Setups ▸ Windows ▸ Import from a Windows Stack…. The same importer reads XAMPP for Windows, WampServer, AMPPS, Uniform Server and IIS from a copy of their folder (for IIS, also copy applicationHost.config and the site folders); folders without a vhost become <folder>.test, and ASP.NET sites come over for reference only. Every project in www and every vhost in etc/apache2/sites-enabled becomes an Apache site under Laragon's host name ({name}.test or your own template); Windows paths are mapped onto the copy. Laragon's data folder can't be reused on a Mac, so export each database on the PC first (HeidiSQL ▸ Export database as SQL) and save it in the Laragon folder or the site's folder — it is matched to its site and loaded on the Databases tab.

Migrate Setups ▸ Apple Server (formerly the Apple Server Migration pane) handles a whole Server.app machine. Point it at the source (this Mac or a remote one) and it inventories every service with serveradmin — Websites, Mail, Calendar and Contacts, Messages (XMPP), VPN, DNS, DHCP, NetInstall, Open Directory, File Sharing, Time Machine, Profile Manager and more — and anything else it finds running, such as a git server, databases, Docker or a standalone Nginx. Each service shows a status badge and a modern equivalent; where there's more than one sensible replacement (Calendar → Radicale / SOGo / Baïkal, VPN → WireGuard / strongSwan, DHCP → dnsmasq / Kea) you choose which one. Pick a target machine and OS, and the wizard installs the equivalents and prints a precise carry-over checklist. This Mac's own websites move in the same place (full vhost + file copy) and get the same check before anything is retired, and the DNS zone files the services' run copies are named and checked against the source. Open Directory (users and groups) hands off to the Accounts pane, and a detected git server to the Git Server migration. To keep the Mac rather than leave it, use Modernize Old Mac Server (Chapter 4).

Leaving a site builder? Migrate Setups ▸ Site Builders ▸ Import from STUDIO… moves a site off STUDIO (studio.design), which has no export of its own. Type the site's address and FrontierStack reads what the site shows every visitor — no STUDIO sign-in — then adds its pages, its CMS items (as posts, with dates, cover images, categories and tags) and its images to a new WordPress site on this Mac — FrontierStack downloads WordPress, creates the site's own database and MySQL user, adds an Apache site for its name and installs it — or to a site the WordPress pane already lists, with redirects from the old addresses. Newer STUDIO sites are read from their finished pages; older ones, which arrive as an empty page, are read from STUDIO's publish storage and CMS service, the same requests a visitor's browser makes. The design, animations, shared headers and footers, forms and fonts don't come over, and the import ends with a checklist of them. When the new site is ready, Publish to a Host… opens Site Clone set up to copy it and its database to a server or a hosting account under the STUDIO site's own address; a WordPress copy gets a wp-config.php for the new place, with its own database login there. Running the import needs a licence; looking at the site doesn't.

8.23The Git Server: Gitea & repository migration

The Git Server pane stands up a self-hosted Gitea (behind Caddy for automatic HTTPS) on any target — this Mac, a linked server, a NAS, a Docker host or a Linux box — detecting the platform and installing the right way. Once it's running, connect with an access token for a live monitor: version, repository count and backup-staleness alerting when a mirror falls behind its source.

The same pane migrates or mirrors another Git server into Gitea. Point it at on-disk bare repos — the legacy macOS app Simple Git Server, Xcode Server, Gitolite, a plain git-daemon, or any folder — or at a running remote server via any git://, https:// or ssh:// clone base (enumerate the repo names over SSH, or paste them). Scan lists the repositories; Migrate mirror-clones each one with full history, every branch and tag, and pushes it into Gitea, creating the destination repo through the API. The copy runs on this Mac.