novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

ADR 0120: a roster fact carries its format as a template; rewrite to-be 29 #144

Merged
jschoubben merged 2 commits from design/roster-fact-is-a-template into main 2026-09-26 23:51:21 +00:00
Conversation 0 Commits 2 Files Changed 2
+251 -30
2 changed files with 251 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 4d4012cdf6 - Show all commits
02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md
+127
View File
@@ -0,0 +1,127 @@
---
topic: what runs on it
status: accepted
date: 2026-09-27
deciders: jochen
reconstructed: false
extends: 0112-a-module-definition-names-no-node-mesh-or-path.md
---
# 120. A roster fact carries its format as a template: the mesh owns the data, the module owns the format
## Context
A **fact** is a thing only the mesh knows — which machines exist, what they are called, where they
are — written into a file where a module asks for it. The mesh computes it from the graph; a module
loads it, restarts on it, does what its software does with it. Facts replaced three modules that
existed only because computed output needed somewhere to live and ran no software of their own
([ADR 0040](0040-what-a-module-is.md)).
But the *format* lived in the control plane. A fact was a name from a closed list, and each name had
a formatter written in Go beside the others: `node-names` wrote the roster as an `/etc/hosts` file,
`node-zones` wrote it as a dnsmasq resolver's `local=`/`address=` lines. Adding a consumer meant
adding a formatter — in the consumer's own configuration language — to the mesh.
The ssh work made the cost plain. An operator's `~/.ssh` wants three roster projections — a
`known_hosts`, an ssh `config` of `Host` blocks, an `authorized_keys` — each in ssh's syntax. Under
the closed list that is three more formatters in the control plane, teaching it ssh's configuration
language. And it does not stop at ssh: every daemon that reads the roster in its own file format
would put its grammar here. The control plane was accreting the configuration languages of software
it does not run — the exact thing [ADR 0040](0040-what-a-module-is.md) says is a
module's and not the mesh's.
The shape underneath is one shape. WireGuard's `[Peer]` blocks, `/etc/hosts`, dnsmasq's zones, an
ssh `known_hosts` — all of them are *the roster, projected into a file*. Only the projection differs,
and the projection belongs to whoever runs the software that reads it.
## Considered Options
**1. Keep the closed list; add a formatter per consumer.** Rejected. The control plane learns the
configuration language of every daemon any module might run, without bound, and each format lives in
the mesh rather than in the module that owns the file. A module cannot change how its own file is
written without a control-plane change.
**2. A general placeholder vocabulary over `content`, like `${machine:address}` but for the
roster.** Rejected. The mesh's other substitutions each resolve to *one* scalar — this machine's
address, one provider's port. The roster is inherently a *repetition*: one block per machine. A flat
`${…}` vocabulary cannot iterate, and a mechanism that could would be a template in all but name.
**3. The module gives a path and a template over the roster; the mesh renders it.** Chosen. The mesh
owns the data — who exists, their names and addresses — and hands it to a Go `text/template` the
module wrote. The mesh renders and reads neither the template's intent nor the file's meaning.
## Decision
**A fact is a path and a template.** In a module's manifest, `facts` maps a name the module chooses
to a `{ path, template }`. The template is a Go `text/template` over a fixed **roster view**:
- `.Node` — this machine's bare name.
- `.Suffix` — what a mesh name ends in (`internal`, or the operator's choice), as composed.
- `.Names` — every name the mesh serves: the machines *and* the names it was told to route.
- `.Machines` — only the machines that are nodes of this mesh.
Each of `.Names` and `.Machines` is a list of `{ Name, FQDN, Address }`. A machine the mesh has a
record for but cannot yet place has no address and is left out of both — a name that resolves to
nothing is a connection that hangs, so it is omitted rather than written (the same rule as before).
**The mesh owns the data; the module owns the format.** The control plane holds **no** formatter.
The two built-in projections render through the same path any module uses:
- **`/etc/hosts`** is a template on the mesh's own network module. The mesh writes `/etc/hosts`
because being on the private network is what gives a machine a name — but the *layout* is a
template like any other, shipped with the control plane because that module ships with it, not
because the control plane knows the hosts-file format.
- **dnsmasq's zones** move into dnsmasq. The `local=`/`address=` grammar is dnsmasq's configuration
language, and it now lives in dnsmasq's manifest, where the module that runs dnsmasq owns it.
**The names-vs-machines distinction is the template's choice** ([04-ISSUES/111](../04-ISSUES/111-the-resolver-is-told-names-the-mesh-serves-not-only-machines/00-report.md)):
a container's hosts ranges `.Names`, so a routed name resolves to the machine serving it; a resolver
told the mesh's suffix is its own ranges `.Machines`, or a routed name written there with the suffix
appended is a name nobody will ever ask for.
**A template that will not render is refused at composition, not on a machine.** A template that does
not parse, or reads a field the roster does not have, fails where the manifest is — the closed-list
safety, moved from the fact's *name* to the roster's *shape*. A daemon that starts, reads a file the
mesh could not render, and answers nothing is a much worse way to find out.
**WireGuard stays a computed generator, and that is the line.** Its `mesh0.conf` is not a pure roster
projection — it carries topology the control plane decides: which peers are reachable, endpoints, hub
forwarding, keepalive for a NAT'd node. And it is *foundational*: the overlay must be up before any
module can be delivered, so the thing that writes it cannot itself be a delivered module. The line
this draws: **the substrate that delivery rides on is the control plane's; everything layered on a
working overlay is a roster template.** DNS, hosts, and ssh are layered; the overlay is the floor.
## Consequences
- **ssh is two templates and no control-plane change.** Once the roster view carries a machine's ssh
host key and its operator account ([to-be 29](../03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md)),
`known_hosts`, the ssh `config`, and `authorized_keys` are templates on the ssh modules — the mesh
gains no knowledge of ssh's syntax. This ADR is what makes that work land without touching the
controller.
- **A new roster projection never touches the control plane.** Any module that reads the roster in
its own format ships its own template.
- **A module can change how its own file is written** without a control-plane change — it is editing
its own manifest.
- **The schema changed and is not backward compatible.** A fact was a string (a path); it is now
`{ path, template }`. The old string form has no template and cannot be auto-upgraded, because the
format it implied was the formatter this ADR deletes. The controller and every catalogue module
using facts — only dnsmasq — land together. A controller and a catalogue that disagree cannot
compose the module: the running daemon on a machine is unaffected, but the mesh will not send it a
new declaration until both sides agree.
- **The output did not change.** The `/etc/hosts` and dnsmasq zones a machine receives are
byte-for-byte what the deleted formatters wrote, pinned by tests that render the built-in template
and compose the real dnsmasq manifest.
## References
- [ADR 0040](0040-what-a-module-is.md): a module is software the mesh runs — a
format the mesh knows for software it does not run was the accretion this stops
- [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md): a module definition names no
path; this is its sibling for content — a module definition names no format the mesh must know
- [to-be 29](../03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md): the ssh consumer this
unblocks, and the roster fields it will add
- [04-ISSUES/111](../04-ISSUES/111-the-resolver-is-told-names-the-mesh-serves-not-only-machines/00-report.md): every served name is not a
machine — now the template's choice of `.Names` or `.Machines`
- mesh-controller `internal/catalogue/roster.go` (the mechanism), `internal/overlay/generator.go`
(the built-in `/etc/hosts` template), `internal/catalogue/manifest.go` (`RosterFile`)
- mesh-catalog `modules/dnsmasq/module.json` (the zones template, dnsmasq's own)
03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md
+124 -30
View File
@@ -5,7 +5,10 @@ code: []
updated: 2026-09-27
decisions:
- 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md
- 02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md
- 02-DECISIONS/0051-shared-data-is-the-operators.md
- 02-DECISIONS/0113-the-vault-makes-every-secret.md
- 02-DECISIONS/0117-a-machines-uplink-is-a-seat.md
---
# 29 — A node has operator accounts, and the mesh owns what lives under a home
@@ -18,7 +21,7 @@ owned by, who a user service runs as, and — the case that surfaced this — wh
person's `~/.ssh/config`, `~/.zshrc`, `~/.config`); the mesh, taking those over, kept the machine
facts and dropped the human one.
Two things are missing, and they are one idea:
Several things are missing, and they are one idea.
## 1. The account is a node fact
@@ -28,17 +31,14 @@ mesh already knows the node and its address, so `<account>@<node>` is then a com
because it is exactly the fact that was silently lost — `ssh ace` failed to `ace` because nothing
in the mesh said ace's account is `ace`.
It is **not** a credential. The account names a login; the key that authorises it is the
operator's, placed as a secret or an operator-owned file, never minted by the mesh (ADR 0051).
## 2. A resource may live under a home, owned by its account
[ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) placed a
module's *system* data — `<root>/<module>`, owned by the module. It has no analog for the other
half of the filesystem: the things that belong under a person's home and are owned by that
person. `~/.ssh/config`, `~/.ssh/config.d/mesh`, `~/.zshrc`, `~/.config/hal` — every one of these
is a resource the mesh should be able to place and own, resolved against **the account's home**
rather than a system root, and chowned to **the account** rather than to root or a module uid.
person. `~/.ssh/config`, `~/.zshrc`, `~/.config/hal` — every one of these is a resource the mesh
should be able to place and own, resolved against **the account's home** rather than a system
root, and chowned to **the account** rather than to root or a module uid.
This is the same move as `${dir:…}`, one level over: a resource says `home: <account>` (or names
an account requirement), and the mesh resolves the home directory and the owning uid on the node
@@ -46,39 +46,133 @@ that account lives on. A module that writes operator config — the eventual rep
`hal/terminal`, `hal/claude-code`, `hal/secrets` — declares its files this way and names no
`/home/...` path, exactly as a system module now names no `/var/lib` path.
These are a **family**, not one module: an `ssh-client` module, a shell module, a `~/.config`
module, each a *universal-tier* consumer of the account fact — assigned wherever a person logs in,
which is every node, unlike the graphical stack that a capability gates.
## 3. The whole of `~/.ssh` is the mesh's — with one boundary drawn inside it
The predecessor owned a single file (`~/.ssh/config`) and left the rest alone; it drifted, because
owning one file beside foreign ones is not owning anything. The mesh should own **the directory**:
create `~/.ssh` at `0700`, chown it to the account, and own the files it places there —
- **`config`** (or the mesh's region of it): the `Host` blocks for every other node, composed
from the roster;
- **`known_hosts`**: authoritative, so the "Host key verification failed / accept-new" dance that
cost real time during enrolment simply ends;
- **`authorized_keys`**: who may log into this account, governed centrally rather than by whichever
key happened to be pasted where.
**The boundary — and it is the reason this is safe:** `~/.ssh` is the one directory where a wrong
declaration locks a person out of their own machine. So the mesh's *found-vs-owned* semantics
([ADR 0118](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md),
adoption) apply *inside* the home directory. The mesh **owns** the directory and the files above; it
**holds as found — never rewrites, never removes** — the operator's own contents: their **private
keys** and their **personal drop-ins** (`config.d/personal`, the personal `Host` aliases a
workstation carries, exactly as `hosts.local` is the home the mesh never rewrites for `/etc/hosts`).
Reconcile removing an unassigned `config.d/mesh` is fine; the same logic aimed at `id_ed25519` or an
operator's own `authorized_keys` entry is a lockout. This is the login-channel cousin of the rule
[ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md) draws for the uplink and the sshd
module draws for the firewall: **the mesh must never be able to arrange the one failure that severs
its own way back in.** The carve-out is not a convenience; it is that rule, in `~/.ssh`.
## 4. Keys are the mesh's to generate — through a CA, and existing keys are adopted, not replaced
Key *generation* is the mesh's, not each node's improvising its own. The clean form is an **SSH
certificate authority as a seat**, the sibling of the TLS internal CA the mesh already runs:
- **Host certs.** The mesh signs each node's host key. Every node's `known_hosts` becomes one line
— `@cert-authority *.<suffix> <mesh-CA-key>` — and nothing is distributed per node; a new node is
trusted the instant its host key is signed.
- **User certs.** The mesh signs a cert naming the principals (accounts) allowed. Every node's
`authorized_keys` / sshd `TrustedUserCAKeys` becomes one trust line — no N×N key spraying — and
short-lived certs give rotation for free
([ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md)).
- The **CA private key is the mesh's**, a secret the vault makes
([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)).
**Three kinds of key, and only one is never minted.** Host keys (server identity) and pure
machine-to-machine keys the mesh may generate end to end. The operator's **personal** private key —
possibly on a hardware token, possibly used from an off-mesh laptop — the mesh **signs into a cert
but never generates**; that, and only that, is the residue of
[ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md). So "keys are mesh-owned" and
"the operator's login key is the operator's" reconcile: the mesh owns the CA and the signing; it
holds the human's private half, never mints it.
**Existing keys are not lost.** Taking ownership is *adoption*, not regeneration: a key already on a
machine is recorded and signed, not overwritten. The mesh gains authority over `~/.ssh` — it does
not clear it. An enrolling node's host key and the operator's existing key are carried forward; the
found-vs-owned boundary of §3 is exactly what guarantees nothing already there is destroyed.
## 5. How it is distributed: the controller composes, the node applies
None of this needs a node to discover the mesh, and none of it needs a control-plane module of its
own. The ssh files are **roster facts**
([ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md)): once the
roster view carries a node's **host key** and its **account** beside its name and address, the
`ssh-client` module ships a template for `known_hosts`, `config` and `authorized_keys`, and the
controller renders each node's copy from the full roster and pushes it. The mesh owns the data; the
module owns ssh's format; the control plane gains no ssh syntax. It is the same act as composing a
peer list or `/etc/hosts` — which is why there is **no novox-only "mesh-ssh" module**: the
centralization is the controller's composition, not a module that runs somewhere. Only non-secret
facts travel (names, addresses, accounts, host keys, the CA public key); the private key stays the
operator's, placed as an operator-owned file, referenced by path.
## 6. The two modules, and the seat between them
- **`sshd`** (server, every node) — manages sshd, owns and **reports** its host key so the roster
carries it, and trusts the user CA.
- **`ssh-client`** (client, every node) — owns `~/.ssh` per §3, consumes the roster and the CA
public key.
- **`the-ssh-ca`** (a seat, held on the control node) — signs host and user certs.
They meet at the account and the CA, not at a bespoke module. The `sshd` server side already exists;
the client/identity side and the CA are the open pieces.
## Why now, and why not yet
**Why it matters:** when HAL retires, the generators that keep `~/.ssh/config`, shell config and
the operator's `~/.config/hal` current retire with it. Without this, adding a node stops adding
its ssh alias, and a fresh machine has no operator dotfiles at all — the mesh would run every
service and leave the human unable to work on the box. The account is also load-bearing for
correctness already: `ssh <node>` (issue 122's cousin), user-scoped systemd units, and any file a
person rather than a daemon must own.
**Why it matters:** when HAL retires, the generators that keep `~/.ssh`, shell config and the
operator's `~/.config/hal` current retire with it. Without this, adding a node stops adding its ssh
alias and its trust, and a fresh machine has no operator dotfiles at all — the mesh would run every
service and leave the human unable to work on the box.
**Why not build it reflexively:** it is a real addition to the node model and the resource model,
and it must be gotten right, not smuggled in beside a firewall fix. Open questions to settle
first:
**Why not build it reflexively:** it is a real addition to the node model, the resource model, and
the seat set, and must be gotten right. The mechanism half is now settled — ADR 0120 is what lets
the ssh files be templates with no control-plane format — so what remains to decide here is the
model:
- **One account or several per node?** A workstation has one human; a shared box might have more.
The model should allow more than one without forcing the common case to name it.
- **Where the login key lives.** An operator-owned file (ADR 0051) or an accepted secret — never
minted. The account fact and the key that authorises it are separate, and only the first is the
mesh's to generate.
- **The boundary with `sshd`.** The `sshd` module (server side) already exists. This is the
*client* and *identity* side: the account a node offers, and the home-scoped files an operator
needs. They meet at the account but are not the same module.
- **Multi-operator.** Today there is one human. The model should not assume it, but the first
cut may serve one and leave the shape open.
Allow more than one without forcing the common case to name it.
- **The CA's shape.** Host-cert and user-cert principals, cert lifetime and renewal, where the CA
runs (a seat on the control node). The one thing fixed: the operator's personal key is signed,
never minted.
- **Adoption of existing keys.** How an enrolling node's host key and an operator's existing key are
recorded and signed rather than replaced — the found-vs-owned boundary, made concrete for keys.
- **The `sshd` boundary.** Server side exists; this is the client, the identity, and the CA.
- **The ssh-agent.** An agent is a *user-scoped service running as the account* — the first concrete
case of the user services §2 anticipates. It holds the operator's private key in memory; the mesh
declares the unit and sets `AddKeysToAgent`/`IdentityAgent` in `config`, and still never sees the
private half. Agent *forwarding* wants a policy, not a default: with user certs it is largely
unnecessary, and forwarding an agent into a node exposes the operator's keys to that node's root —
so prefer certificates and `ProxyJump` over forwarding.
**Not urgent, not blocking.** ssh and dotfiles work today because HAL's generators still run as
the substrate. This becomes load-bearing in the node-by-node retirement phase, not before — which
is the right time to build it, once the account model is decided here.
**Not urgent, not blocking.** ssh and dotfiles work today because HAL's generators still run as the
substrate. This becomes load-bearing in the node-by-node retirement phase, not before — which is the
right time to build it, once the account and CA model are decided here.
## References
- The gap was found generating `~/.ssh/config` from the *HAL* registry (`hal/terminal`'s
postConfigure hook), which the nox mesh has no equivalent for.
- [ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md) — the roster
fact mechanism that renders the ssh files, format owned by the module.
- [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) — the
system-path placement this mirrors for home paths.
- [ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md) — why the login key stays
the operator's, never minted.
- [ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md) — why the operator's personal
key is signed, never minted.
- [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md) — the CA key is a secret the
vault makes; [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md)
— short-lived certs as rotation.
- [ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md),
[ADR 0118](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md) —
the never-sever-the-channel rule and the found-vs-owned semantics, applied here to `~/.ssh`.