ADR 0120: a roster fact carries its format as a template; rewrite to-be 29 around it
The facts mechanism formatted the roster in Go in the control plane — one formatter per fact, in the consumer's own configuration language. ADR 0120 makes a fact a path and a template: the mesh owns the data, the module owns the format, and the control plane holds no format at all. to-be 29 (operator accounts + what lives under a home) is rewritten to ride it: the ssh files become roster templates, the whole ~/.ssh is owned with a found/owned boundary that cannot lock the operator out, keys are mesh-owned through an SSH CA (existing keys adopted not regenerated, the operator's personal key signed not minted), and the ssh-agent is a user-scoped service.
This commit is contained in:
@@ -0,0 +1,127 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
date: 2026-09-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0112-a-module-definition-names-no-node-mesh-or-path.md
|
||||
---
|
||||
|
||||
# 120. A roster fact carries its format as a template: the mesh owns the data, the module owns the format
|
||||
|
||||
## Context
|
||||
|
||||
A **fact** is a thing only the mesh knows — which machines exist, what they are called, where they
|
||||
are — written into a file where a module asks for it. The mesh computes it from the graph; a module
|
||||
loads it, restarts on it, does what its software does with it. Facts replaced three modules that
|
||||
existed only because computed output needed somewhere to live and ran no software of their own
|
||||
([ADR 0040](0040-what-a-module-is.md)).
|
||||
|
||||
But the *format* lived in the control plane. A fact was a name from a closed list, and each name had
|
||||
a formatter written in Go beside the others: `node-names` wrote the roster as an `/etc/hosts` file,
|
||||
`node-zones` wrote it as a dnsmasq resolver's `local=`/`address=` lines. Adding a consumer meant
|
||||
adding a formatter — in the consumer's own configuration language — to the mesh.
|
||||
|
||||
The ssh work made the cost plain. An operator's `~/.ssh` wants three roster projections — a
|
||||
`known_hosts`, an ssh `config` of `Host` blocks, an `authorized_keys` — each in ssh's syntax. Under
|
||||
the closed list that is three more formatters in the control plane, teaching it ssh's configuration
|
||||
language. And it does not stop at ssh: every daemon that reads the roster in its own file format
|
||||
would put its grammar here. The control plane was accreting the configuration languages of software
|
||||
it does not run — the exact thing [ADR 0040](0040-what-a-module-is.md) says is a
|
||||
module's and not the mesh's.
|
||||
|
||||
The shape underneath is one shape. WireGuard's `[Peer]` blocks, `/etc/hosts`, dnsmasq's zones, an
|
||||
ssh `known_hosts` — all of them are *the roster, projected into a file*. Only the projection differs,
|
||||
and the projection belongs to whoever runs the software that reads it.
|
||||
|
||||
## Considered Options
|
||||
|
||||
**1. Keep the closed list; add a formatter per consumer.** Rejected. The control plane learns the
|
||||
configuration language of every daemon any module might run, without bound, and each format lives in
|
||||
the mesh rather than in the module that owns the file. A module cannot change how its own file is
|
||||
written without a control-plane change.
|
||||
|
||||
**2. A general placeholder vocabulary over `content`, like `${machine:address}` but for the
|
||||
roster.** Rejected. The mesh's other substitutions each resolve to *one* scalar — this machine's
|
||||
address, one provider's port. The roster is inherently a *repetition*: one block per machine. A flat
|
||||
`${…}` vocabulary cannot iterate, and a mechanism that could would be a template in all but name.
|
||||
|
||||
**3. The module gives a path and a template over the roster; the mesh renders it.** Chosen. The mesh
|
||||
owns the data — who exists, their names and addresses — and hands it to a Go `text/template` the
|
||||
module wrote. The mesh renders and reads neither the template's intent nor the file's meaning.
|
||||
|
||||
## Decision
|
||||
|
||||
**A fact is a path and a template.** In a module's manifest, `facts` maps a name the module chooses
|
||||
to a `{ path, template }`. The template is a Go `text/template` over a fixed **roster view**:
|
||||
|
||||
- `.Node` — this machine's bare name.
|
||||
- `.Suffix` — what a mesh name ends in (`internal`, or the operator's choice), as composed.
|
||||
- `.Names` — every name the mesh serves: the machines *and* the names it was told to route.
|
||||
- `.Machines` — only the machines that are nodes of this mesh.
|
||||
|
||||
Each of `.Names` and `.Machines` is a list of `{ Name, FQDN, Address }`. A machine the mesh has a
|
||||
record for but cannot yet place has no address and is left out of both — a name that resolves to
|
||||
nothing is a connection that hangs, so it is omitted rather than written (the same rule as before).
|
||||
|
||||
**The mesh owns the data; the module owns the format.** The control plane holds **no** formatter.
|
||||
The two built-in projections render through the same path any module uses:
|
||||
|
||||
- **`/etc/hosts`** is a template on the mesh's own network module. The mesh writes `/etc/hosts`
|
||||
because being on the private network is what gives a machine a name — but the *layout* is a
|
||||
template like any other, shipped with the control plane because that module ships with it, not
|
||||
because the control plane knows the hosts-file format.
|
||||
- **dnsmasq's zones** move into dnsmasq. The `local=`/`address=` grammar is dnsmasq's configuration
|
||||
language, and it now lives in dnsmasq's manifest, where the module that runs dnsmasq owns it.
|
||||
|
||||
**The names-vs-machines distinction is the template's choice** ([04-ISSUES/111](../04-ISSUES/111-the-resolver-is-told-names-the-mesh-serves-not-only-machines/00-report.md)):
|
||||
a container's hosts ranges `.Names`, so a routed name resolves to the machine serving it; a resolver
|
||||
told the mesh's suffix is its own ranges `.Machines`, or a routed name written there with the suffix
|
||||
appended is a name nobody will ever ask for.
|
||||
|
||||
**A template that will not render is refused at composition, not on a machine.** A template that does
|
||||
not parse, or reads a field the roster does not have, fails where the manifest is — the closed-list
|
||||
safety, moved from the fact's *name* to the roster's *shape*. A daemon that starts, reads a file the
|
||||
mesh could not render, and answers nothing is a much worse way to find out.
|
||||
|
||||
**WireGuard stays a computed generator, and that is the line.** Its `mesh0.conf` is not a pure roster
|
||||
projection — it carries topology the control plane decides: which peers are reachable, endpoints, hub
|
||||
forwarding, keepalive for a NAT'd node. And it is *foundational*: the overlay must be up before any
|
||||
module can be delivered, so the thing that writes it cannot itself be a delivered module. The line
|
||||
this draws: **the substrate that delivery rides on is the control plane's; everything layered on a
|
||||
working overlay is a roster template.** DNS, hosts, and ssh are layered; the overlay is the floor.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **ssh is two templates and no control-plane change.** Once the roster view carries a machine's ssh
|
||||
host key and its operator account ([to-be 29](../03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md)),
|
||||
`known_hosts`, the ssh `config`, and `authorized_keys` are templates on the ssh modules — the mesh
|
||||
gains no knowledge of ssh's syntax. This ADR is what makes that work land without touching the
|
||||
controller.
|
||||
- **A new roster projection never touches the control plane.** Any module that reads the roster in
|
||||
its own format ships its own template.
|
||||
- **A module can change how its own file is written** without a control-plane change — it is editing
|
||||
its own manifest.
|
||||
- **The schema changed and is not backward compatible.** A fact was a string (a path); it is now
|
||||
`{ path, template }`. The old string form has no template and cannot be auto-upgraded, because the
|
||||
format it implied was the formatter this ADR deletes. The controller and every catalogue module
|
||||
using facts — only dnsmasq — land together. A controller and a catalogue that disagree cannot
|
||||
compose the module: the running daemon on a machine is unaffected, but the mesh will not send it a
|
||||
new declaration until both sides agree.
|
||||
- **The output did not change.** The `/etc/hosts` and dnsmasq zones a machine receives are
|
||||
byte-for-byte what the deleted formatters wrote, pinned by tests that render the built-in template
|
||||
and compose the real dnsmasq manifest.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0040](0040-what-a-module-is.md): a module is software the mesh runs — a
|
||||
format the mesh knows for software it does not run was the accretion this stops
|
||||
- [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md): a module definition names no
|
||||
path; this is its sibling for content — a module definition names no format the mesh must know
|
||||
- [to-be 29](../03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md): the ssh consumer this
|
||||
unblocks, and the roster fields it will add
|
||||
- [04-ISSUES/111](../04-ISSUES/111-the-resolver-is-told-names-the-mesh-serves-not-only-machines/00-report.md): every served name is not a
|
||||
machine — now the template's choice of `.Names` or `.Machines`
|
||||
- mesh-controller `internal/catalogue/roster.go` (the mechanism), `internal/overlay/generator.go`
|
||||
(the built-in `/etc/hosts` template), `internal/catalogue/manifest.go` (`RosterFile`)
|
||||
- mesh-catalog `modules/dnsmasq/module.json` (the zones template, dnsmasq's own)
|
||||
@@ -5,7 +5,10 @@ code: []
|
||||
updated: 2026-09-27
|
||||
decisions:
|
||||
- 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md
|
||||
- 02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md
|
||||
- 02-DECISIONS/0051-shared-data-is-the-operators.md
|
||||
- 02-DECISIONS/0113-the-vault-makes-every-secret.md
|
||||
- 02-DECISIONS/0117-a-machines-uplink-is-a-seat.md
|
||||
---
|
||||
|
||||
# 29 — A node has operator accounts, and the mesh owns what lives under a home
|
||||
@@ -18,7 +21,7 @@ owned by, who a user service runs as, and — the case that surfaced this — wh
|
||||
person's `~/.ssh/config`, `~/.zshrc`, `~/.config`); the mesh, taking those over, kept the machine
|
||||
facts and dropped the human one.
|
||||
|
||||
Two things are missing, and they are one idea:
|
||||
Several things are missing, and they are one idea.
|
||||
|
||||
## 1. The account is a node fact
|
||||
|
||||
@@ -28,17 +31,14 @@ mesh already knows the node and its address, so `<account>@<node>` is then a com
|
||||
because it is exactly the fact that was silently lost — `ssh ace` failed to `ace` because nothing
|
||||
in the mesh said ace's account is `ace`.
|
||||
|
||||
It is **not** a credential. The account names a login; the key that authorises it is the
|
||||
operator's, placed as a secret or an operator-owned file, never minted by the mesh (ADR 0051).
|
||||
|
||||
## 2. A resource may live under a home, owned by its account
|
||||
|
||||
[ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) placed a
|
||||
module's *system* data — `<root>/<module>`, owned by the module. It has no analog for the other
|
||||
half of the filesystem: the things that belong under a person's home and are owned by that
|
||||
person. `~/.ssh/config`, `~/.ssh/config.d/mesh`, `~/.zshrc`, `~/.config/hal` — every one of these
|
||||
is a resource the mesh should be able to place and own, resolved against **the account's home**
|
||||
rather than a system root, and chowned to **the account** rather than to root or a module uid.
|
||||
person. `~/.ssh/config`, `~/.zshrc`, `~/.config/hal` — every one of these is a resource the mesh
|
||||
should be able to place and own, resolved against **the account's home** rather than a system
|
||||
root, and chowned to **the account** rather than to root or a module uid.
|
||||
|
||||
This is the same move as `${dir:…}`, one level over: a resource says `home: <account>` (or names
|
||||
an account requirement), and the mesh resolves the home directory and the owning uid on the node
|
||||
@@ -46,39 +46,133 @@ that account lives on. A module that writes operator config — the eventual rep
|
||||
`hal/terminal`, `hal/claude-code`, `hal/secrets` — declares its files this way and names no
|
||||
`/home/...` path, exactly as a system module now names no `/var/lib` path.
|
||||
|
||||
These are a **family**, not one module: an `ssh-client` module, a shell module, a `~/.config`
|
||||
module, each a *universal-tier* consumer of the account fact — assigned wherever a person logs in,
|
||||
which is every node, unlike the graphical stack that a capability gates.
|
||||
|
||||
## 3. The whole of `~/.ssh` is the mesh's — with one boundary drawn inside it
|
||||
|
||||
The predecessor owned a single file (`~/.ssh/config`) and left the rest alone; it drifted, because
|
||||
owning one file beside foreign ones is not owning anything. The mesh should own **the directory**:
|
||||
create `~/.ssh` at `0700`, chown it to the account, and own the files it places there —
|
||||
|
||||
- **`config`** (or the mesh's region of it): the `Host` blocks for every other node, composed
|
||||
from the roster;
|
||||
- **`known_hosts`**: authoritative, so the "Host key verification failed / accept-new" dance that
|
||||
cost real time during enrolment simply ends;
|
||||
- **`authorized_keys`**: who may log into this account, governed centrally rather than by whichever
|
||||
key happened to be pasted where.
|
||||
|
||||
**The boundary — and it is the reason this is safe:** `~/.ssh` is the one directory where a wrong
|
||||
declaration locks a person out of their own machine. So the mesh's *found-vs-owned* semantics
|
||||
([ADR 0118](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md),
|
||||
adoption) apply *inside* the home directory. The mesh **owns** the directory and the files above; it
|
||||
**holds as found — never rewrites, never removes** — the operator's own contents: their **private
|
||||
keys** and their **personal drop-ins** (`config.d/personal`, the personal `Host` aliases a
|
||||
workstation carries, exactly as `hosts.local` is the home the mesh never rewrites for `/etc/hosts`).
|
||||
Reconcile removing an unassigned `config.d/mesh` is fine; the same logic aimed at `id_ed25519` or an
|
||||
operator's own `authorized_keys` entry is a lockout. This is the login-channel cousin of the rule
|
||||
[ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md) draws for the uplink and the sshd
|
||||
module draws for the firewall: **the mesh must never be able to arrange the one failure that severs
|
||||
its own way back in.** The carve-out is not a convenience; it is that rule, in `~/.ssh`.
|
||||
|
||||
## 4. Keys are the mesh's to generate — through a CA, and existing keys are adopted, not replaced
|
||||
|
||||
Key *generation* is the mesh's, not each node's improvising its own. The clean form is an **SSH
|
||||
certificate authority as a seat**, the sibling of the TLS internal CA the mesh already runs:
|
||||
|
||||
- **Host certs.** The mesh signs each node's host key. Every node's `known_hosts` becomes one line
|
||||
— `@cert-authority *.<suffix> <mesh-CA-key>` — and nothing is distributed per node; a new node is
|
||||
trusted the instant its host key is signed.
|
||||
- **User certs.** The mesh signs a cert naming the principals (accounts) allowed. Every node's
|
||||
`authorized_keys` / sshd `TrustedUserCAKeys` becomes one trust line — no N×N key spraying — and
|
||||
short-lived certs give rotation for free
|
||||
([ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md)).
|
||||
- The **CA private key is the mesh's**, a secret the vault makes
|
||||
([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)).
|
||||
|
||||
**Three kinds of key, and only one is never minted.** Host keys (server identity) and pure
|
||||
machine-to-machine keys the mesh may generate end to end. The operator's **personal** private key —
|
||||
possibly on a hardware token, possibly used from an off-mesh laptop — the mesh **signs into a cert
|
||||
but never generates**; that, and only that, is the residue of
|
||||
[ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md). So "keys are mesh-owned" and
|
||||
"the operator's login key is the operator's" reconcile: the mesh owns the CA and the signing; it
|
||||
holds the human's private half, never mints it.
|
||||
|
||||
**Existing keys are not lost.** Taking ownership is *adoption*, not regeneration: a key already on a
|
||||
machine is recorded and signed, not overwritten. The mesh gains authority over `~/.ssh` — it does
|
||||
not clear it. An enrolling node's host key and the operator's existing key are carried forward; the
|
||||
found-vs-owned boundary of §3 is exactly what guarantees nothing already there is destroyed.
|
||||
|
||||
## 5. How it is distributed: the controller composes, the node applies
|
||||
|
||||
None of this needs a node to discover the mesh, and none of it needs a control-plane module of its
|
||||
own. The ssh files are **roster facts**
|
||||
([ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md)): once the
|
||||
roster view carries a node's **host key** and its **account** beside its name and address, the
|
||||
`ssh-client` module ships a template for `known_hosts`, `config` and `authorized_keys`, and the
|
||||
controller renders each node's copy from the full roster and pushes it. The mesh owns the data; the
|
||||
module owns ssh's format; the control plane gains no ssh syntax. It is the same act as composing a
|
||||
peer list or `/etc/hosts` — which is why there is **no novox-only "mesh-ssh" module**: the
|
||||
centralization is the controller's composition, not a module that runs somewhere. Only non-secret
|
||||
facts travel (names, addresses, accounts, host keys, the CA public key); the private key stays the
|
||||
operator's, placed as an operator-owned file, referenced by path.
|
||||
|
||||
## 6. The two modules, and the seat between them
|
||||
|
||||
- **`sshd`** (server, every node) — manages sshd, owns and **reports** its host key so the roster
|
||||
carries it, and trusts the user CA.
|
||||
- **`ssh-client`** (client, every node) — owns `~/.ssh` per §3, consumes the roster and the CA
|
||||
public key.
|
||||
- **`the-ssh-ca`** (a seat, held on the control node) — signs host and user certs.
|
||||
|
||||
They meet at the account and the CA, not at a bespoke module. The `sshd` server side already exists;
|
||||
the client/identity side and the CA are the open pieces.
|
||||
|
||||
## Why now, and why not yet
|
||||
|
||||
**Why it matters:** when HAL retires, the generators that keep `~/.ssh/config`, shell config and
|
||||
the operator's `~/.config/hal` current retire with it. Without this, adding a node stops adding
|
||||
its ssh alias, and a fresh machine has no operator dotfiles at all — the mesh would run every
|
||||
service and leave the human unable to work on the box. The account is also load-bearing for
|
||||
correctness already: `ssh <node>` (issue 122's cousin), user-scoped systemd units, and any file a
|
||||
person rather than a daemon must own.
|
||||
**Why it matters:** when HAL retires, the generators that keep `~/.ssh`, shell config and the
|
||||
operator's `~/.config/hal` current retire with it. Without this, adding a node stops adding its ssh
|
||||
alias and its trust, and a fresh machine has no operator dotfiles at all — the mesh would run every
|
||||
service and leave the human unable to work on the box.
|
||||
|
||||
**Why not build it reflexively:** it is a real addition to the node model and the resource model,
|
||||
and it must be gotten right, not smuggled in beside a firewall fix. Open questions to settle
|
||||
first:
|
||||
**Why not build it reflexively:** it is a real addition to the node model, the resource model, and
|
||||
the seat set, and must be gotten right. The mechanism half is now settled — ADR 0120 is what lets
|
||||
the ssh files be templates with no control-plane format — so what remains to decide here is the
|
||||
model:
|
||||
|
||||
- **One account or several per node?** A workstation has one human; a shared box might have more.
|
||||
The model should allow more than one without forcing the common case to name it.
|
||||
- **Where the login key lives.** An operator-owned file (ADR 0051) or an accepted secret — never
|
||||
minted. The account fact and the key that authorises it are separate, and only the first is the
|
||||
mesh's to generate.
|
||||
- **The boundary with `sshd`.** The `sshd` module (server side) already exists. This is the
|
||||
*client* and *identity* side: the account a node offers, and the home-scoped files an operator
|
||||
needs. They meet at the account but are not the same module.
|
||||
- **Multi-operator.** Today there is one human. The model should not assume it, but the first
|
||||
cut may serve one and leave the shape open.
|
||||
Allow more than one without forcing the common case to name it.
|
||||
- **The CA's shape.** Host-cert and user-cert principals, cert lifetime and renewal, where the CA
|
||||
runs (a seat on the control node). The one thing fixed: the operator's personal key is signed,
|
||||
never minted.
|
||||
- **Adoption of existing keys.** How an enrolling node's host key and an operator's existing key are
|
||||
recorded and signed rather than replaced — the found-vs-owned boundary, made concrete for keys.
|
||||
- **The `sshd` boundary.** Server side exists; this is the client, the identity, and the CA.
|
||||
- **The ssh-agent.** An agent is a *user-scoped service running as the account* — the first concrete
|
||||
case of the user services §2 anticipates. It holds the operator's private key in memory; the mesh
|
||||
declares the unit and sets `AddKeysToAgent`/`IdentityAgent` in `config`, and still never sees the
|
||||
private half. Agent *forwarding* wants a policy, not a default: with user certs it is largely
|
||||
unnecessary, and forwarding an agent into a node exposes the operator's keys to that node's root —
|
||||
so prefer certificates and `ProxyJump` over forwarding.
|
||||
|
||||
**Not urgent, not blocking.** ssh and dotfiles work today because HAL's generators still run as
|
||||
the substrate. This becomes load-bearing in the node-by-node retirement phase, not before — which
|
||||
is the right time to build it, once the account model is decided here.
|
||||
**Not urgent, not blocking.** ssh and dotfiles work today because HAL's generators still run as the
|
||||
substrate. This becomes load-bearing in the node-by-node retirement phase, not before — which is the
|
||||
right time to build it, once the account and CA model are decided here.
|
||||
|
||||
## References
|
||||
|
||||
- The gap was found generating `~/.ssh/config` from the *HAL* registry (`hal/terminal`'s
|
||||
postConfigure hook), which the nox mesh has no equivalent for.
|
||||
- [ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md) — the roster
|
||||
fact mechanism that renders the ssh files, format owned by the module.
|
||||
- [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) — the
|
||||
system-path placement this mirrors for home paths.
|
||||
- [ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md) — why the login key stays
|
||||
the operator's, never minted.
|
||||
- [ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md) — why the operator's personal
|
||||
key is signed, never minted.
|
||||
- [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md) — the CA key is a secret the
|
||||
vault makes; [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md)
|
||||
— short-lived certs as rotation.
|
||||
- [ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md),
|
||||
[ADR 0118](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md) —
|
||||
the never-sever-the-channel rule and the found-vs-owned semantics, applied here to `~/.ssh`.
|
||||
|
||||
Reference in New Issue
Block a user