Compare commits

...
Author SHA1 Message Date
jschoubben 17ca9a262b 0164: a setting names the file it lands in (issue 198's leak between one module's files); 190 notes the fourth machine now has the resolver 2026-10-02 12:23:28 +02:00
jschoubben 967c793eaa Merge remote-tracking branch 'origin/main' into decision/docker-module 2026-10-02 12:23:27 +02:00
mesh-admin 78351560f7 Merge pull request 'Issue 198: the home network's DNS server ran outside the mesh, and its filter closed it' (#283) from jschoubben/the-lans-dns-is-the-mesh into main 2026-10-02 10:22:51 +00:00
mesh-admin 62cc2f89c7 Merge pull request 'Issue 197: a physical link that is down is not filtered when it comes up' (#280) from jschoubben/every-physical-link-faces-outside into main 2026-10-02 10:22:40 +00:00
jschoubben 426f741ad0 Issue 198: the home network's DNS server ran outside the mesh, and its filter closed it 2026-10-02 12:22:31 +02:00
jschoubben 9bed54d3be Issue 197 resolved: the wired port is guarded before it is plugged in 2026-10-02 12:22:15 +02:00
jschoubben 413daf8ad5 Issue 197: a physical link that is down is not filtered when it comes up 2026-10-02 12:22:15 +02:00
mesh-admin 5c993c09b7 Merge pull request 'Issues 143 and 144 resolved: the live row of ADR 0168 read on the home server and the control node' (#282) from issues/143-144-resolved into main 2026-10-02 10:11:37 +00:00
jschoubben 7c3be48db2 Issues 143 and 144 resolved: the live row of ADR 0168 read on the home server and the control node 2026-10-02 12:11:17 +02:00
mesh-admin b665d06701 Merge pull request 'ADR 0168: a converged machine is filtered by the mesh alone, and the host says what else refuses (group 7)' (#281) from feat/one-thing-filters-a-converged-machine into main 2026-10-02 10:03:48 +00:00
jschoubben 1bd13446d4 ADR 0168: a converged machine is filtered by the mesh alone, and the host says what else refuses (group 7)
Designs 08 and 05 revised; 141 resolved by ADR 0140 and 084 by ADR 0102 and
issue 128, both by reading; 143 and 144 decided, built on the matching
branches in mesh-host and mesh-controller, resolved when the home server's
record names the predecessor's chain.
2026-10-02 11:58:18 +02:00
mesh-admin 14adaafa53 Merge pull request 'to-be 29: the account fact and home-scoped resources shipped; the ssh-client module, CA and ~/.ssh boundary did not' (#279) from design/29-what-shipped into main 2026-10-02 09:49:52 +00:00
mesh-admin bd673cc6ec Merge pull request 'Group 6 closed: 086, 090, 098, 099, 100, 101 resolved on the operator's decision' (#278) from issues/group-6-closed into main 2026-10-02 09:38:11 +00:00
jschoubben bd6c55d225 Group 6 closed: 086, 090, 098, 099, 100, 101 resolved on the operator's decision, each saying the live row was not run 2026-10-02 11:37:08 +02:00
mesh-admin 2bbbc56502 Merge pull request 'Issue 194 resolved: the fixed host forgot its former archive on all four machines' (#277) from issue/194-resolved into main 2026-10-02 09:31:49 +00:00
jschoubben c8935aceca Issue 194 resolved: the fixed host forgot its former archive on all four machines 2026-10-02 11:31:29 +02:00
mesh-admin 29b656f2c0 Merge pull request 'Issue 196: the hub relays the mesh only on the ports it publishes itself' (#276) from jschoubben/the-hub-relays-the-mesh into main 2026-10-02 09:23:55 +00:00
jschoubben d05ac367f1 Issue 196 resolved: the hub relays the mesh, re-swept live 2026-10-02 11:23:37 +02:00
jschoubben 1c0dafb918 Issue 196: the hub relays the mesh only on the ports it publishes itself 2026-10-02 11:17:11 +02:00
mesh-admin 018ee359ae Merge pull request 'Issue 194: the host's own former archive stops every machine applying anything' (#274) from issue/194-the-hosts-own-former-archive-stops-every-apply into main 2026-10-02 09:04:30 +00:00
mesh-admin c5535eeebd Merge pull request 'ADR 0163 built: the take digest, the minted-secret refusal, the networks setting, settings judged where stored, genesis raising the forge as declared' (#270) from feat/a-take-is-a-comparison-the-rest into main 2026-10-02 09:04:21 +00:00
mesh-admin dbb9d2bc16 Merge pull request 'Issue 191 and ADR 0167: a membership carries what its module receives, and who the mesh is' (#273) from jschoubben/an-internal-only-route into main 2026-10-02 07:49:27 +00:00
jschoubben 28d53dcc28 Issue 191 resolved: internal-only routes are served to the mesh, live on both proxies 2026-10-02 09:49:25 +02:00
jschoubben df667eb710 ADR 0167: a membership carries what its module receives, and who the mesh is
Issue 191's route proxy needs to know who the mesh is to serve an
internal name correctly, and the first fix had it work that out alone.
The membership on the bus now carries it, from the same list the filter
uses. ADR 0138 gains an insight that the proxy is where internal reach
is kept; designs 08 and 25 say how.
2026-10-02 09:49:19 +02:00
jschoubben 098a2ca485 Issue 191: a route with only an internal name is dropped as naming nothing 2026-10-02 09:49:19 +02:00
mesh-admin a34cedeb5d Merge pull request 'Issue 195: every assigned module is counted as a bus user without a credential' (#275) from jschoubben/issue-195 into main 2026-10-02 00:44:29 +00:00
jschoubben db5ff5a5ee Issue 195: every assigned module is counted as a bus user without a credential
The status warning names 49 users; 48 are modules that never speak on the
bus, and the one real fault, a declared broker secret filled with a
generated value, looked the same as the rest.
2026-10-02 02:44:24 +02:00
jschoubben 27c1db8a86 Review of 0164-0166 and 190: the mesh's own setting words stay settable; changing runtime verbs are not the console's wildcard; migration steps 1-2 are one push; dnsmasq's dns key dates from 09-23 2026-10-02 00:48:06 +02:00
jschoubben 3d54fcbb86 Merge remote-tracking branch 'origin/main' into decision/docker-module 2026-10-02 00:02:57 +02:00
jschoubben c4151e6bc4 ADR 0163 built: the take digest, the minted-secret refusal, the networks setting, settings judged where stored, genesis raising the forge as declared; issues 096, 097, 126 resolved
The record gets its built note; designs 05 and 09 the revisions; 086, 098,
099, 100 and 101 stay located because every machine is converged and the
record's live row — a take read on an adopted machine — has not been run;
090 is built in part, its network difference left for the take to say.
2026-10-01 23:45:57 +02:00
jochen 8c231102f8 to-be 29: the account fact and home-scoped resources shipped; the ssh-client module, CA and ~/.ssh boundary did not
The controller merged §1, §2 and the composed ssh config on 2026-09-27 while hq still
called the design proposed. Record what is on main, what is held on a branch, and what
has no code — and that the account fact shipped without a decision record, which is the
next thing to write. Also drop the real account and node names the public repository must
not carry, and correct ADR numbers that drifted in a renumbering.
2026-10-01 23:14:10 +02:00
jschoubben f1941304cc ADRs 0164-0166 and issue 190: the container runtime gets a module, a seat and declared settings
Proposed for the operator's review: settings declared with defaults and cost (0164),
container-runtime as a kernel capability (0165), node-container-runtime seat with the
host creating containers through its holder (0166), and the runtime's file written by
modules that are not its own (190).
2026-10-01 23:13:18 +02:00
37 changed files with 1541 additions and 52 deletions
@@ -124,6 +124,32 @@ This corrects a fact, not the decision: one statement per endpoint, three things
none of them deciding on its own, all stand. The table in the decision should be read with the filter none of them deciding on its own, all stand. The table in the decision should be read with the filter
column applying to an unrouted endpoint. column applying to an unrouted endpoint.
## Progressive insight — 2026-10-02, from issue 191
**For a routed endpoint, "the proxy serves the internal name" has to mean "serves it to the private
network", and only the proxy can make it mean that.** The decision says `internal` means the proxy
serves the internal name and not the public one. It does not say to whom, and the proxy answered
every name it routes to any request that carried it, on the same listeners as its public names. A
name being internal kept nobody out: a request from the internet only had to send it. While every
routed endpoint also had a public name, nothing showed it. Once an endpoint could be internal alone
([issue 191](../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md)), serving
its name to everyone would have published exactly what `internal` was chosen to keep private.
The earlier insight above says the port is not the path for a routed endpoint. This is its other
half: the proxy is the path, so the proxy is where `internal` is enforced. It serves an internal name
only to the machines of the mesh and to the machine itself
([ADR 0144](0144-anything-on-a-machine-may-call-anything-on-it.md)). Who the mesh is, it is told,
not left to work out: its membership carries the same list of machine addresses the filter's "from the
mesh" is rendered from ([ADR 0167](0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)).
To anyone else, the name is answered as one never routed, in the handshake and in the request, and not
listed among the names it serves. This holds for the internal name of a `both` endpoint too, whose
outsiders have its public name.
The decision, the options and the consequences stand: one statement per endpoint, three things
derived from it. Checked in the proxy's own tests: an internal-only name is served to a machine the
membership names and to loopback, and refused, unlisted and uncertified for any other request; until
the mesh is issued, it is served to the machine alone.
## Consequences ## Consequences
- **A manifest gains endpoint names, and a route contribution names an endpoint instead of a port.** - **A manifest gains endpoint names, and a route contribution names an endpoint instead of a port.**
@@ -131,6 +131,32 @@ the plan says it too.
| Genesis raises the forge as its module declares it | a genesis test that raises, assigns, and finds the module holding rather than raising a second | | Genesis raises the forge as its module declares it | a genesis test that raises, assigns, and finds the module holding rather than raising a second |
| Live | the next cutover on an adopted machine: `take` shows the comparison, refuses the downgrade if there is one, and the service keeps its configuration and its secret | | Live | the next cutover on an adopted machine: `take` shows the comparison, refuses the downgrade if there is one, and the service keeps its configuration and its secret |
## Built, 2026-10-02
> **Progressive insight — 2026-10-02.** The decision stands; these are the facts of its building.
Built across mesh-host 63 and 64 and mesh-controller 201, 202 and the pull request that followed
them. Rule 1: `take` previews every held thing's comparison and ends with a digest; `take --yes
<digest>` acts on exactly that preview, and a changed preview or an account older than the flip
allows is refused, as the flip's are. A published port's reach is said as the machine reported it,
behind the found firewall whose rules are not read. Rule 2: an older image, a differing file and a
minted, unaccepted secret for found data refuse, overridden by `--downgrade`, `--replace <path>` and
`--mint <name>`; the secrets a module holds on a machine are read with where each came from. Rule 3:
`secret accept --provider` reaches a required secret. Rule 4: the per-machine setting is `networks`,
a container id to the found networks it keeps; judged for an adopted machine only, joined by the host
after the container runs, part of the container's spec, named in the preview. Rule 5: the host's
facts, former targets and strays. Rule 6: one judgement, run where a setting is stored and where a
machine is composed; a module whose stored setting its definition can no longer compose is left out
of the declaration, the envelope says so, the host keeps that module's things, and `plan` and `push`
say it by name. A key that reaches nothing is refused where stored and said by `plan`, and never
costs a module. Rule 7: genesis raises the forge under the module's container name, with its image
digest and its data directory; the network is the one difference left, said by the take, because the
bootstrap forge reaches the store on the machine's loopback.
**Not yet proven live.** Every machine of this mesh is converged, so the table's last row — a take
on an adopted machine — waits for the next adoption. What is live is what the rows above it check.
Issues 086, 098, 099, 100 and 101 stay located until that row is read.
## References ## References
- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md), [ADR 0103](0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md), [ADR 0104](0104-a-provision-may-be-answered-by-an-adapter-to-the-predecessor.md), [ADR 0162](0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md) - [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md), [ADR 0103](0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md), [ADR 0104](0104-a-provision-may-be-answered-by-an-adapter-to-the-predecessor.md), [ADR 0162](0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md)
@@ -0,0 +1,146 @@
---
topic: what runs on it
status: proposed
date: 2026-10-01
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0046-a-module-configuration-is-its-assignments-not-its-manifest.md
---
# 164. A setting is declared with its default, its meaning and what changing it costs
## Context
The operator asked for one thing for every module, with the container runtime as the first case: **one
consistent default configuration for every machine, overridable per assignment, and easy to change
later.** The four machines' runtime configurations were each written by hand and differ — one keeps
running containers through a daemon restart and one does not, their log rotation differs, and each
names its resolver and its trusted registries in its own words.
Most of this was already decided.
[ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md) said the definition is
identity and **defaults**, the assignment's settings are the configuration, *unset is the default*, and
*an unknown setting is refused*. [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md) made
a setting an operator requirement whose contract is "a type and, optionally, a default", answered by
"the assignment's, or the requirement's default, or unresolved". An assignment is a module on a node,
so the node layer of a module's settings already *is* the per-assignment override, and the mesh-wide
layer is the one consistent default a person changes once.
What was built is narrower than what was decided, measured in the controller on the day of deciding:
- **Nothing declares which keys are settable.** A mergeable file's content is its defaults, and every
key of it — and every key not in it — is accepted. Nothing tells a person, or the console, what can be
set, of what type, or what it means.
- **The refusal of an unknown setting is not there for most modules.** The stray-setting report returns
nothing at all for a module with any mergeable file, because such a file "takes any key"
([issue 173](../04-ISSUES/173-a-modules-settings-reach-every-fact-it-contributes/00-report.md) left
files that way on purpose). It reports rather than refuses where it does run.
- **A value in a file that is not JSON can have no default.** `${setting:<key>}`
([ADR 0155](0155-a-definition-names-no-installation-and-how-that-is-checked.md)) is refused when no
layer sets it — right for a mail domain, where a default is the very literal 0155 removes, and wrong
for a tunable like the resolver's upstreams, which the resolver module therefore carries as literals
in its file.
- **A setting reaches every mergeable file its module owns.** The layers are one flat map per module,
laid over each such file. Adding a setting to the resolver module for its own configuration put the
key into the container runtime's file as well — the resolver writes into that file too — and the
runtime refuses keys it does not know. The plan showed it before any push; the runtime's file was
then made to take no settings at all ([issue 198](../04-ISSUES/198-the-lans-dns-server-ran-outside-the-mesh-and-its-filter-closed-it/00-report.md)). Issue 173 stopped settings leaking into
contributions and served facts; between one module's own files the leak remains.
- **What a change costs is said per file, not per key.** A service names the files it is reloaded or
restarted on. The runtime re-reads its trusted registries on a reload and its `dns` key only when it
starts; the resolver module declared a reload, so on two machines the key was written, reloaded,
and never read, and every container got a public resolver for weeks while everything read as
current ([issue 110](../04-ISSUES/110-a-container-on-the-runtimes-own-network-cannot-reach-the-resolver/01-resolution.md)).
## Considered Options
1. **Leave settings implicit; document each module's keys in its README.** Rejected: a key the mesh
does not know cannot be refused, typed, listed by the console or costed, and a README is a rule
enforced by nothing.
2. **A second mechanism for tunables beside settings** — defaults in a new block, settings untouched.
Rejected: two ways to state one person's value, and design 27 already retires six mechanisms
that grew that way.
3. **Settings declared in the definition, as 0112's operator requirement: a key, a type, a meaning,
optionally a default, and what a change costs.** Adopted.
## Decision
**A module declares every setting it takes.** Each declared setting has a name, a type, one sentence
of meaning, optionally a default, and what a change to it costs. The spelling is design 27's to settle
with the rest of the requirement form; this record decides the content.
**A setting with a default is a tunable; a setting without one is the operator's.** A tunable resolves
to its default when no layer sets it — wherever it is read, a mergeable file or `${setting:<key>}` in a
file of any format. A setting with no default is refused by name when nothing sets it, as 0155 decided;
0155's refusal is narrowed to exactly that case, not changed for it. Whether a value has a default is a
fact about the software (a log size does, a mail domain does not), and the definition states it once.
**The layers stay as they are, and every value says where it came from.** The definition's default,
then the mesh-wide layer, then the node's — later wins, objects merge, lists replace. One consistent
configuration for every machine is the default plus the mesh-wide layer; one machine that differs says
so in its own layer and nothing else. Asked for a module's configuration on a machine, the mesh lists
every declared setting with its effective value and its source: *default*, *mesh*, or *node*.
**Changing later is changing one of three places, and the plan shows its reach before anything moves.**
A new default ships with the module's next version and reaches every assignment that does not override
it; a mesh-wide setting reaches every assignment of the module; a node's reaches one. The plan of a
change names each assignment whose effective value moves.
**A declared setting says where it lands.** Each names the file or files of its module that read it,
and reaches no other: a module that owns two mergeable files no longer has one flat map laid over both.
A file that names no setting takes none.
**A declared setting is the only kind accepted.** Setting a key the module does not declare is refused
when it is set, naming the declared keys, rather than reported when the machine is planned. The mesh's
own words — where a port, a directory or an operator's data is placed, how far an endpoint reaches —
are the mesh's to validate as they are today, and no module declares them. A module
that declares no settings keeps today's behaviour until it does; a catalogue test lists those modules,
and the list shrinks to empty before the implicit form is removed — design 27's rule for every retired
mechanism.
**A setting says what it costs: nothing, a reload, or a restart.** When a file changes, the host
applies the strongest cost among the settings whose values moved in it, so a key the software reads
only at start can no longer be written and never read. A setting that reaches a container's environment
costs that container being recreated, which the host already does when a container's specification
changes; it needs no declaration. A service's `reload-on` and `restart-on` keep
naming the files that are not settings — a generated roster, a credential.
**The container runtime is the first module to declare its settings** and the model for the rest:
its log rotation, keeping containers through a daemon restart, and its resolver are tunables, and
its trusted registries are what the mesh tells it.
## Consequences
- The console can show a module's settings as a form: what can be set, of what type, its default,
and where the current value came from. That is the surface the operator wants for changing a
default later.
- `settings set` can refuse an unknown key, so ADR 0046's rule is enforced where it was only stated.
- The resolver's upstreams, the runtime's log rotation, and other literals a definition carries
because it could not give them a default become declared tunables.
- **What got harder:** every module that takes settings must list them, and a mergeable file no
longer silently accepts a key its author did not foresee. A person who needs one adds it to the
definition, which is a new module version, not a setting.
- Issue 173's open question — a consumer checks nothing against a contract — is unchanged; this record
is the operator half of design 27's contract, not the provider half.
- Not decided here: the spelling (design 27); whether a node's layer may be narrowed to a single key
rather than replaced whole, as `settings set` does today.
## How this is checked
| Rule | Checked by |
|---|---|
| Every setting a module takes is declared | A parser test refusing a setting declaration without a type or meaning; a catalogue test listing modules with mergeable files or `${setting:}` and no declarations, which must be empty before the implicit form is removed |
| A tunable resolves to its default; an operator value without one is refused | Resolution tests: an unset tunable in a JSON file and in a text file both take the default; an unset setting with no default is refused naming it (0155's existing test) |
| A setting reaches only the files it names | A resolution test: a module with two mergeable files and a setting declared for one; the other file's content is unchanged by it (the case of issue 198) |
| An undeclared key is refused when set | A controller test: `settings set` with an undeclared key fails naming the declared keys, and nothing is stored |
| Every effective value names its source | A test listing a module's configuration on a node with one key from each of default, mesh and node |
| A change's reach is shown before it moves | A plan test: changing a mesh-wide setting names every assignment whose effective value moves and no other |
| The strongest cost applies | A host test: a file where a reload-cost key and a restart-cost key both moved restarts; a file where only reload-cost keys moved reloads |
| Live | The container runtime's module lists its settings with their sources on every machine, and a mesh-wide change to its log rotation reaches all four at the next push |
## References
- [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md), [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0155](0155-a-definition-names-no-installation-and-how-that-is-checked.md), [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md)
- [Design 27 — a module requires, the mesh resolves](../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md)
- Issues [110](../04-ISSUES/110-a-container-on-the-runtimes-own-network-cannot-reach-the-resolver/00-report.md), [173](../04-ISSUES/173-a-modules-settings-reach-every-fact-it-contributes/00-report.md)
- mesh-controller `internal/catalogue/settings.go` (`settle`, `UnusedSettings`), `internal/catalogue/setting_into.go`
@@ -0,0 +1,105 @@
---
topic: what runs on it
status: proposed
date: 2026-10-01
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0161-what-deserves-a-seat.md
---
# 165. `container-runtime` is what a machine can run; that a runtime is running is its holder's health
## Context
A capability is a requirement a module places on a machine, detected by the host and renewed with
every report ([ADR 0161](0161-what-deserves-a-seat.md)). The host's `container-runtime` asks the
daemon for its version: *a running daemon, not an installed client*. It was made that way by
[issue 007](../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md), where an
installed package was believed to be a working service, and
[design 05](../03-DESIGN/01-to-be/05-the-node-host.md)'s table says the same: *a runtime is
running*. The installer's preflight borrows the same detector to wait for the runtime the
foundation bundle installs, so there is one answer to "is there a runtime here".
The mesh is now to have a module for the runtime itself — its packages, its configuration, its
service — on every machine ([ADR 0166](0166-the-container-runtime-is-a-node-seat-and-the-host-creates-containers-through-its-holder.md)).
That module cannot declare `container-runtime` as defined: it would require the very thing it
installs, the cycle [research 011](../01-RESEARCH/011-the-module-graph/cases.md)'s case 12 names
("something the mesh installs that then becomes a node capability"). The operator defined the word
for it: **`container-runtime` means the machine is able, at the kernel level, to install a runtime and
execute containers** — not that one is installed, and not that one is running.
The host already draws this line once. `seat` is hardware, a display server *could* run here;
`graphical-session` is state, one *is* running; the detector's own comment says "assignment needs the
first". A machine without a display has no seat however much software is installed, and a machine
with one has a seat before anything is.
Fifty-four catalogue modules declare `container-runtime` today, counted on the catalogue's main
branch on the day of deciding: every module that delivers a container. Each relies on the current
meaning to keep it off a machine with no running runtime.
## Considered Options
1. **Keep the meaning; let the runtime's module declare nothing.** Rejected: a module that installs
the runtime has requirements on the machine — the kernel features without which installing it is
pointless — and would state none of them. The cycle stays, only hidden.
2. **Two capabilities, "can run" and "is running".** Rejected: the second is made true by assigning a
module, so it is the module's state, not a fact of the machine; a capability the mesh itself
flips by its own assignment is case 12's cycle with an extra name.
3. **The capability is the kernel's; whether a runtime runs is the runtime module's health, and a
module that delivers a container needs the runtime's seat held.** Adopted.
## Decision
**`container-runtime` is detected from what the kernel offers**, as `seat` is: the namespaces a
container needs, a control-group hierarchy the runtime can manage, and an overlay filesystem the
running kernel has or can load. Present when all three are; absent naming the missing one. Nothing is
run and no runtime is asked. The verdict's detail names what was found, not a runtime's version.
**"A runtime is running and answers" is one probe, owned by the host and used twice:** by the
installer's preflight, which waits for the runtime the foundation installs, and as the runtime
module's health. It asks the daemon, as issue 007 requires. The preflight stops borrowing the
capability's detector, and there is still one answer to "is a runtime running here".
**The runtime's module declares `container-runtime`**, with `package-manager`, `service-manager` and
`privileged`, like any module that manages machine software.
**A module that delivers a container needs the runtime seat held on its machine**, and is refused
otherwise, naming the seat and the modules that could hold it — the refusal design 27 already lists
for an unheld seat. That requirement is derived from the container resource and needs no manifest
field ([ADR 0166](0166-the-container-runtime-is-a-node-seat-and-the-host-creates-containers-through-its-holder.md)).
The fifty-four existing declarations of the capability stay valid and become redundant; a catalogue
test lists them, and they retire when the list is empty.
**The order is fixed, not preferred.** The detector changes only once the seat requirement is
enforced. In between, a machine with the kernel and no running runtime would read as able to run
every containerised module, which is issue 007 again.
## Consequences
- Design 05's capability table changes its `container-runtime` row from *a runtime is running* to
*the kernel can run containers*, and names the runtime module's health as where "running" is now
asked.
- The node listing stops showing the runtime's version beside the capability. The version moves to
the runtime module's health and its seat's verbs.
- A fresh machine with no runtime reads as able to run one, so it can be assigned the runtime's
module, which is what makes the mesh able to install the runtime instead of the bootstrap alone.
- **What got harder:** "is this machine running containers" is no longer one glance at the profile;
it is the runtime seat's holder and its health. The node's listing should show both side by side.
## How this is checked
| Rule | Checked by |
|---|---|
| The capability is the kernel's | Host detector tests over a fixture `/proc` and `/sys`: all three present → present; each one missing → absent naming it; no runtime binary on the fixture machine changes nothing |
| One probe asks whether a runtime runs | A host test that the preflight and the runtime module's health call the same probe, and that the probe fails against a stopped daemon with an installed client (issue 007's shape) |
| A containerised module needs the runtime seat held | A resolution test: a module with a container resource on a machine whose runtime seat is unheld is refused, naming the seat and its candidate holders |
| The order holds | The host release that changes the detector is gated on the controller release that enforces the seat requirement — stated in both changes' descriptions and checked at review |
| Live | Every machine's profile shows `container-runtime` present with the kernel's features as its detail; a machine with no runtime installed reads present too |
## References
- [ADR 0161](0161-what-deserves-a-seat.md) — the profile renewed by every report; a capability that names a dialect
- [ADR 0166](0166-the-container-runtime-is-a-node-seat-and-the-host-creates-containers-through-its-holder.md) — the seat and its holder
- [Issue 007](../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md), [research 011](../01-RESEARCH/011-the-module-graph/cases.md) cases 12–13
- [Design 05 — the node host](../03-DESIGN/01-to-be/05-the-node-host.md)
- mesh-host `internal/profile/detectors.go` (the runtime detector), `internal/profile/seat.go` (the hardware/state split), `internal/bootstrap/preflight.go` (the preflight that borrows it)
@@ -0,0 +1,161 @@
---
topic: what runs on it
status: proposed
date: 2026-10-01
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0161-what-deserves-a-seat.md
---
# 166. The container runtime is a node seat, and the host creates containers through its holder
## Context
Every container the mesh runs on a machine is created by the host, which looks for a runtime
(`docker info`, then `podman info`) and drives that runtime's command line itself: run, inspect,
remove, exec. Research 012 called this "detected rather than declared": the host takes over whatever
runtime it finds. Nothing in the mesh owns the runtime. Its package came from the foundation bundle
or was already on the machine. Its configuration file was written by hand, differs on each of the
four machines, and is also written into by two modules that are not the runtime's
([issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md)).
Its service is declared by those same two.
The operator set the direction:
- a module for the runtime, on every machine, owning "what is needed to run containers here": its
packages, its configuration and its service;
- that module holds a node seat for the runtime, so a second runtime (podman) can later compete
for the seat;
- the host stops speaking to the runtime directly and uses the seat's holder. The host stays the one
that decides, and the holder becomes the one that executes;
- every container on the machine is in scope, not only the mesh's. A development environment started
by hand, or a test database a tool runs, is legitimate. The host already calls these *strays*: 3,
8 and 25 on three of the machines on the day of deciding;
- the runtime's events and verbs are subjects on the bus, and the mesh's own interface is built on
them. The third-party interface run until now was removed by hand.
[ADR 0161](0161-what-deserves-a-seat.md)'s test for a seat is whether the mesh's own code finds it by
name. Here it does: the host would look up the holder on its own machine. A singular role of a module
held once per machine is a `node-*` seat ([ADR 0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md)),
in the controller's seed.
The constraint that decides most of this record is a cycle. The bus runs in containers. On the
broker's machine, the broker's own container is created by the host. A holder's code served from a
container cannot create the container that runs it. On a first machine, before the controller exists,
nothing holds anything.
## Considered Options
1. **The host calls the holder's verbs over the bus.** Rejected: with the broker down, no machine can
create any container, including the broker's. The mesh would be unable to restart its own
transport.
2. **The holder picks a dialect that the host speaks itself, as with the uplink.** Rejected: the
host would still drive the runtime, and the module would drive it too for every other caller.
That is two programs speaking to one daemon, and they come to disagree about the same machine
(the installer's preflight already exists to avoid this).
3. **The holder's code runs as a supervised process on the machine and serves the seat's verbs
twice: locally to the host, on the bus to everyone else.** Adopted.
## Decision
**`node-container-runtime` is a seat of the mesh's own, node-scoped,** in the controller's seed under
this record. It delivers no provision; what it carries is its role's protocol: verbs its holder must
serve ([ADR 0132](0132-a-seat-carries-the-tools-its-holder-must-serve.md)) and events its holder emits
([ADR 0129](0129-a-seat-carries-the-protocol-of-its-role.md)). The runtime's module, `docker`, claims
it and is assigned to every machine. A podman module may claim it later; one machine runs one.
**The seat's verbs cover every container on the machine:** list, inspect, logs, stats, start, stop,
restart, create and remove. A mesh-held container is marked by the host's label and says which
assignment holds it. **A container the runtime runs can be root on the machine** — privileged, a host
path mounted, the host's network or process namespace, the runtime's own socket — so a caller other
than the host may not create one that is any of these; only a declaration the mesh composed may ask
for them. And the verbs that change anything are granted by name, never by a wildcard: a grant of
every tool (the console's today) reaches the reading verbs only. Issue 193 is what a verb that trusts
its caller costs. **Creating or removing a mesh-held container is the host's alone.** Any other
caller is refused naming the assignment, because the host would undo it at its next apply. Starting,
stopping or restarting one is allowed, and the answer says the host will restore what its
declaration says. A container the mesh does not hold is the caller's to do anything with.
**The seat's events are the runtime's own** — a container created, started, stopped, died, removed,
its health changed. They are emitted on the seat's subjects, so every holder emits the same events and
no reader depends on which runtime holds the seat. As
[ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)
decides, the subjects are issued by the controller, not composed by the module.
**The holder's code is a supervised process, not a container** ([ADR 0150](0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md)).
A runtime cannot be run by the thing it runs. The process serves the seat's verbs on the bus to the
console, to tools and to the mesh's interface. The same verbs are served on a local socket on the
machine, which only the host may use. **The host creates, inspects and removes its containers
through that socket and nothing else.** If the holder does not answer, the host creates nothing. It
says so in its report, naming the seat. It never falls back to the command line.
**A container needs the seat held on its machine.** An assignment that delivers a container on a
machine whose runtime seat is unheld is refused, naming the seat and its candidates
([ADR 0165](0165-container-runtime-is-what-a-machine-can-run-and-a-running-runtime-is-its-holders-health.md)).
Mounting the runtime's socket into a container is granted by the seat, not by the capability. The
socket's path is the holder's to state, because podman's is not docker's.
**The runtime module owns the runtime's configuration.** Its settings are declared with defaults
([ADR 0164](0164-a-setting-is-declared-with-its-default-its-meaning-and-what-changing-it-costs.md)):
the resolver containers use, the registries it trusts, log rotation, and keeping containers through a
daemon restart. The module is given the resolver's address and the mesh's registry as values; no other
module writes the runtime's file or declares its service.
**The first machine is bootstrapped with the holder, and adopted afterwards.** The foundation bundle
already installs the runtime's package and service. It also carries the holder's process, delivered as
a binary the way the host is ([ADR 0142](0142-the-mesh-delivers-its-own-components-as-binaries.md)).
When the runtime module is assigned, it adopts what the bundle made, as the store and broker modules
adopt theirs ([ADR 0078](0078-the-store-and-broker-are-modules.md)).
## Consequences
- **The migration on the running mesh has a fixed order:**
1. Each machine's hand-written configuration is read, because the module's defaults replace what
differs.
2. In one push per machine: the resolver module and the private network stop writing the
runtime's file ([issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md)),
and the runtime module is assigned and adopts the runtime, its file and its service. Split in
two, either the controller refuses two modules declaring one path, or a machine is left with
nothing setting `dns` and `live-restore`.
3. The controller seeds the seat and enforces the container requirement.
4. The host releases the version that uses the holder.
5. The host's command-line path is removed in the release after every machine's holder answers.
Until then, the host reports per machine which path it used.
- **Every container the host makes depends on the holder's process.** A crash-looping holder stops
new containers on its machine. Running containers are unaffected. The host's report names the cause.
- The process form of a module's own code must serve tools on the live mesh before this ships. Only the
showcase declares it, and [issue 117](../04-ISSUES/117-a-modules-own-code-is-a-container-and-a-process/01-diagnosis.md)
found the showcase's tools declared in a form nothing runs. The runtime module is the first whose
tools cannot fall back to a container.
- A user interface subscribing to events directly does not exist. Today a reader of events is a module
that consumes them. The mesh's container view is a module, or waits for that path.
- [ADR 0005](0005-the-node-host.md) ("a container runtime is detected, not chosen") and
[ADR 0006](0006-the-substrate-and-the-control-plane.md)'s matching line describe the mechanism this replaces: on
acceptance, each gets a dated note saying the runtime is now a seat's holder, as the decision
records' rule for a moved mechanism requires. Design 05 and design 26 are amended after acceptance.
- The operator's decision to remove the third-party interface by hand needs no mechanism. No
module-retires-module rule is introduced.
- **What got harder:** the host gains a dependency it did not have, and a first machine's bundle gains
a component. The direct path was simpler and is what makes a runtime a black box to the rest of the
mesh.
## How this is checked
| Rule | Checked by |
|---|---|
| The seat is the mesh's own, node-scoped, with its verbs and events | A catalogue test on the default seats; registration refuses a claimant that does not serve every verb (design 33's existing check) |
| Creating or removing a mesh-held container is the host's alone | A test of the runtime module's verbs: create or remove of a container carrying the host's label, from any caller but the host's socket, is refused naming the assignment; the same verbs on an unlabelled container succeed |
| No caller but the host creates a container that is root on the machine | A test of `create` from the bus: privileged, a host path, the host's namespaces and the runtime's socket are each refused; the same request on the host's socket is accepted. A broker test: a grant of every tool does not reach a changing verb |
| The host uses the holder and never the command line | A host test with a fake holder on the local socket: every container operation goes to it, and with the holder absent the apply creates nothing and reports the seat; after step 5, the host carries no command-line runtime code (checked by build: the package is gone) |
| A container needs the seat held | A resolution test refusing a containerised assignment on a machine with the seat unheld, naming the seat |
| Socket mounts are granted by the seat | A catalogue test: a module mounting the runtime's socket on a machine whose holder states a different path is refused |
| No other module writes the runtime's file | The existing collision check, once the private network's computed resources are inside it ([issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md)) |
| Live | `seats` lists `node-container-runtime` held on every machine; the node listing shows each machine's containers, strays included, from the seat's `list` verb; a container started by hand appears as an event on the bus |
## References
- [ADR 0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md), [ADR 0129](0129-a-seat-carries-the-protocol-of-its-role.md), [ADR 0132](0132-a-seat-carries-the-tools-its-holder-must-serve.md), [ADR 0159](0159-a-tool-call-names-the-machine-and-a-holder-serves-its-seats-verbs.md), [ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md), [ADR 0161](0161-what-deserves-a-seat.md)
- [ADR 0150](0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md), [ADR 0142](0142-the-mesh-delivers-its-own-components-as-binaries.md), [ADR 0078](0078-the-store-and-broker-are-modules.md), [ADR 0005](0005-the-node-host.md)
- [ADR 0164](0164-a-setting-is-declared-with-its-default-its-meaning-and-what-changing-it-costs.md), [ADR 0165](0165-container-runtime-is-what-a-machine-can-run-and-a-running-runtime-is-its-holders-health.md), [issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md)
- [Design 26 — the seats](../03-DESIGN/01-to-be/26-the-seats.md), [design 33 — the tools the mesh answers](../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md)
- mesh-host `internal/apply/apply.go` (the runtime lookup and the command line it drives)
@@ -0,0 +1,99 @@
---
topic: the mesh
status: accepted
date: 2026-10-02
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md
---
# 167. A membership carries what its module receives, and who the mesh is
## Context
A provider learns what it is given from a file. The controller composes every consumer's contribution
to a requirement, and the node's declaration writes them into the provider's received file. The route
proxy reads its routes that way: one JSON file, re-read every two seconds.
[Issue 191](../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md) showed what
that file leaves out. Since [ADR 0138](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md),
a route whose endpoint reaches only the private network carries an internal name and no public one.
The proxy dropped it. Serving it was not enough either: the proxy answers public and internal names on
the same listeners, so an internal name served to every request is public under a guessable name. To
serve it correctly the proxy needs a second fact, **who the mesh is**, and nothing gave it one.
The first attempt had the proxy work it out: the mesh's range from an environment variable written by
the catalogue, and the machine's container bridges read from its own interfaces. That is a second
definition of "the mesh", kept by one module, beside the one the packet filter already uses. The
controller resolves "from the mesh" to every machine's address on the private network, and the filter
is rendered from that list. Two definitions agree until one changes.
[ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)
already gives every assignment one document on the bus, its membership, read once at connect and
followed. It says what the assignment serves and reaches. It does not yet say what it is given.
## Considered Options
1. **Keep the file, add the mesh to it.** The proxy keeps polling a file, and the controller writes the
mesh's addresses beside the routes. It fixes the definition, but delivery stays a file re-read on a
timer, written by a separate path from the one every other fact a module is told now takes.
2. **Have the proxy work it out** from a range in its environment and the machine's interfaces. Rejected:
it is the second definition this record exists to remove.
3. **The membership carries it.** What each module receives, from the same composition its received
file is written from, and the mesh's addresses, from the same list the filter is rendered from. The
proxy follows its membership and serves exactly that.
## Decision
**Option 3.**
- **A membership carries what its module receives**, by requirement: the contributions every consumer
made, exactly as composed for its received file. A requirement nobody contributed to is an empty
list, never absent, for the reason the file is written empty: "nothing asked" and "never told" want
different responses.
- **A membership carries who the mesh is**: every machine's address on the private network, the list
a rule saying "from the mesh" resolves to. One list, two readers: the filter and any module that
must tell the mesh from the world.
- **The route proxy reads its routes and the mesh from its membership**, with the bus account every
module that speaks on the bus is given. It serves an internal name only to the machines the mesh
names and to the machine itself, and answers anyone else as it answers a name it never routed: in
the request, in the handshake, and in the list of names it serves.
- **The file stays until the bus has spoken.** While a proxy has read no membership that carries routes,
it serves the file, and an internal name only to its own machine: refused, never opened. A
membership from a controller that issues no routes changes nothing.
## Consequences
- Every membership grows two fields. A machine joining or leaving republishes every membership, which
a push already does.
- A provider that receives something is told it twice for now, in its file and on the bus. The file
goes when every provider reads its membership; that is its own change.
- The route proxy needs a bus account. It is issued like any module's, so a machine running the proxy
cannot be composed between the catalogue declaring the account and the operator issuing it. The
machine keeps what it runs meanwhile.
- The internal name of a route that also has a public one is now served to the mesh only. Outsiders
have the public name.
- A container on the same machine that calls that machine's own internal name arrives from its
container network, not from a mesh address, and is refused. Calls between machines are unaffected:
they leave by the machine's mesh address. Whether the mesh should also issue each machine's container
networks is left open, because the mesh does not record them today.
## How this is checked
| Rule | Checked by |
|---|---|
| What a provider receives on the bus is what its received file says, same-node port fix included | a controller test composing a provider and a consumer on one machine and comparing the two |
| An internal name is served to the machines the membership names and to loopback, and to nobody else | the proxy's tests: served from a named address and from loopback; refused, unlisted and uncertified from any other |
| A membership that carries no routes, or a mesh that cannot be read, changes nothing | the proxy's tests |
| Until the mesh is issued, an internal name is served to the machine alone | the proxy's tests |
| Live: an internal-only route answers over the mesh and is refused from outside | by hand, after the release |
## References
- [ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md) —
the membership this extends
- [ADR 0138](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md) — reach, and the
insight of 2026-10-02 that the proxy is where internal reach is kept
- [ADR 0144](0144-anything-on-a-machine-may-call-anything-on-it.md) — the machine itself is always inside
- [Issue 191](../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md) — what
found it
@@ -0,0 +1,120 @@
---
topic: the mesh
status: accepted
date: 2026-10-02
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
---
# 168. A converged machine is filtered by the mesh alone, and the host says what else refuses
## Context
[ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md) says what converging does to
the firewall a machine was found with: the mesh's derived filter is loaded in place of the
refusal-only guard, and the found firewall is retired — disabled, never flushed. Four issues from
the first two convergences are four ways that sentence was not the machine:
- the flip reported the found firewall retired and it was active two minutes later; fifty minutes
on, a reconcile found it disabled by hand and recorded that the mesh had done it
([143](../04-ISSUES/143-converging-does-not-retire-the-firewall-it-found/00-report.md));
- "the firewall found" named one front end, and what filtered the forwarded path on that machine
was a chain a predecessor had installed in the container runtime's user hook — invisible to the
mesh, refusing two ports the mesh declared open, and when it was removed, carrying an allowance
every module reaching another by the machine's own name had been relying on
([144](../04-ISSUES/144-the-predecessors-rules-outlive-the-firewall-it-was-found-as/00-report.md),
[145](../04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md));
- the forward chain listed address ranges that followed neither the modules nor the machine
([141](../04-ISSUES/141-the-forward-chain-does-not-follow-the-modules/00-report.md)), answered by
[ADR 0140](0140-the-filter-constrains-what-arrives-from-outside.md) before this record;
- the networking module wrote two machine-wide files whole, so taking it restarted every
container ([084](../04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md)),
answered by [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md) and the hosts
file's marked region ([issue 128](../04-ISSUES/128-the-hosts-file-is-written-whole/00-report.md)).
Read on the four machines of this mesh on 2026-10-02, after every one had converged: on both
machines that had a front end it is inactive, and the host's record says the mesh retired it on
both — true of one, false of the other. On the home server the predecessor's chain is still in
force on the forwarded path, in the legacy packet filter the mesh's reader of rules does not
consult once a machine is converged, so that machine is filtered by two things and the mesh says
one. The host's reader already knows how to tell a table that refuses traffic from the runtime's
own plumbing and from a ban list; it is asked once, at adoption, and only to refuse a machine
whose firewall nobody speaks. Nothing asks it afterwards, and nothing reports what it saw.
The group's exit is one sentence: *a converged machine has exactly one thing filtering it, and
the mesh says truthfully which.* The first half the mesh can enforce only for what it owns; the
second half it can always do, and it is the half that was missing.
## Decision
**1. Convergence is a state the host keeps, not a step it takes once.** Every apply of a converged
declaration reads whether the found firewall is in force. Active — enabled again by a package, a
boot, a hand — it is retired again and said. The record distinguishes *the mesh disabled it* from
*it was found inactive*, and a reconcile that finds it inactive never records that the mesh did
it. When the step is skipped because the apply had failures, the report says the found firewall
was left in force and why; a step that does nothing is never silent.
**2. The host reports what filters the machine, with every apply, adopted or converged.** Every
table of the packet filter, and every chain of the legacy filter, that refuses traffic — a drop or
a reject, or a base chain whose policy drops — with an owner: the *mesh's*, the *found firewall's*,
the *container runtime's own*, a *ban* (a refusal that names the sources it refuses, in a chain
that accepts nothing), or *other*. The runtime's own is its plumbing — its chains, the forward
policy it sets when it turns forwarding on, its guard against reaching a container's address from
off its bridge. The user chain the runtime leaves for an administrator is not the runtime's:
anything refusing in it is *other*, which is where both predecessors' chains lived. Each entry
says in one line what it refuses. The mesh removes none of it: a rule it did not write is the
operator's to remove, now that they can see it.
**3. The mesh says which.** `node show` lists the filters with their owners. `status` names every
converged machine that something other than the mesh's table, the runtime's plumbing and a ban
list filters, the way it names strays and untaken modules, and such a machine is not "all well".
The converge preview lists the filters found and the fate of each: the found firewall retired, the
runtime's and the bans left, *other* left and named — so a person knows before the flip that the
machine will not be filtered by the mesh alone until they remove it, and what they would be
removing. *A converged machine is filtered by the mesh alone* when its list holds nothing but the
mesh's, the runtime's own and bans.
**4. Adoption's threshold does not move.** A machine whose front end nobody speaks is still refused
adoption; a refusing rule in the runtime's user chain still does not refuse it — on both machines
of this mesh it would have, and the migration would not have happened. It is reported instead,
from the first report on.
**5. Two of the group's issues are settled by records already accepted.** The forward chain follows
the machine's outward links and says nothing about networks ([ADR 0140](0140-the-filter-constrains-what-arrives-from-outside.md)),
which answers 141 whole. The runtime's file is written into and reloaded, and the hosts file's
region is the mesh's alone ([ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md),
[issue 128](../04-ISSUES/128-the-hosts-file-is-written-whole/00-report.md)), which answers 084. One
machine-wide file the mesh still writes whole is its own filter, at the path the distribution's
packet filter reads; an operator's own rules at that path would be contested, and are held as
found until the filter module is taken ([ADR 0163](0163-taking-a-module-over-is-a-comparison.md)).
That is a difference a take shows, not a fault, and is decided when it bites.
## Consequences
- The host's report grows by the filters it found and, for a converged machine, the state of its
found firewall and who retired it; the controller keeps both on the node's record.
- `retireFirewall` runs on every converged apply and can disable the found firewall more than
once; the record's *disabled by the mesh* means exactly that.
- The reader of rules gains an owner per table and chain; what it refuses adoption for does not
change. A ban stays what it was: not a firewall.
- Issues 143 and 144 close on rules 1 to 3 once a machine's record names the predecessor's chain;
141 closes on ADR 0140 and 084 on ADR 0102, both by reading.
- Removing what is reported is the operator's act, by hand, with the preview's words in front of
them. The mesh never flushes and never deletes a rule it did not mark.
## How this is checked
| Rule | Checked by |
|---|---|
| Every refusing table and chain is classified, the user chain's refusals as *other* | host tests over rulesets captured from three machines of this mesh: a predecessor's chain in the legacy filter, a ban list and empty front-end chains beside the runtime's, a virtualisation host and an endpoint agent that refuse nothing |
| The found firewall active again on a converged machine is retired again and said; found inactive is recorded as found, not done; a skipped step is said | host tests over a fake front end |
| The report carries the filters and the found firewall's state for a converged machine | a host test reading the report |
| `node show` lists filters with owners; `status` names a converged machine something else filters and is not well; the preview lists filters and fates | controller tests over a fixture report |
| Live | the home server's record names the predecessor's chain in the runtime's user chain as *other*; `status` names the machine; after the operator removes the chain, the next report drops it and `status` is well |
## References
- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), [ADR 0103](0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md), [ADR 0140](0140-the-filter-constrains-what-arrives-from-outside.md), [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md), [ADR 0163](0163-taking-a-module-over-is-a-comparison.md)
- [Design 08 — Connectivity](../03-DESIGN/01-to-be/08-connectivity.md), [Design 05 — The node host](../03-DESIGN/01-to-be/05-the-node-host.md)
- Issues 084, 141, 143, 144, 145
+5
View File
@@ -177,6 +177,8 @@ python3 00-META/checks/index.py fail if stale
- **0161** — [What deserves a seat: a role of a module is a seat, a singular fact about machines is a placement with a capacity of one, and a holder's software is the machine's](0161-what-deserves-a-seat.md) - **0161** — [What deserves a seat: a role of a module is a seat, a singular fact about machines is a placement with a capacity of one, and a holder's software is the machine's](0161-what-deserves-a-seat.md)
- **0162** — [A merge produces a tiered plan the mesh keeps, and a module's dependencies are one relation in the catalogue](0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md) - **0162** — [A merge produces a tiered plan the mesh keeps, and a module's dependencies are one relation in the catalogue](0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md)
- **0163** — [Taking a module over is a comparison: what it compares, what it refuses, and what it carries](0163-taking-a-module-over-is-a-comparison.md) - **0163** — [Taking a module over is a comparison: what it compares, what it refuses, and what it carries](0163-taking-a-module-over-is-a-comparison.md)
- **0167** — [A membership carries what its module receives, and who the mesh is](0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)
- **0168** — [A converged machine is filtered by the mesh alone, and the host says what else refuses](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md)
### Its tiers, from the bottom up ### Its tiers, from the bottom up
@@ -267,6 +269,9 @@ python3 00-META/checks/index.py fail if stale
- **0150** — [A module's own code runs as supervised processes under the module's one account](0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md) - **0150** — [A module's own code runs as supervised processes under the module's one account](0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md)
- **0152** — [The operator's surface is a module the mesh assigns: the console](0152-the-operators-surface-is-a-module-the-console.md) - **0152** — [The operator's surface is a module the mesh assigns: the console](0152-the-operators-surface-is-a-module-the-console.md)
- **0155** — [A definition names no installation: how that is checked, and the three ways a value that did gets out](0155-a-definition-names-no-installation-and-how-that-is-checked.md) - **0155** — [A definition names no installation: how that is checked, and the three ways a value that did gets out](0155-a-definition-names-no-installation-and-how-that-is-checked.md)
- **0164** — [A setting is declared with its default, its meaning and what changing it costs](0164-a-setting-is-declared-with-its-default-its-meaning-and-what-changing-it-costs.md) *(proposed)*
- **0165** — [`container-runtime` is what a machine can run; that a runtime is running is its holder's health](0165-container-runtime-is-what-a-machine-can-run-and-a-running-runtime-is-its-holders-health.md) *(proposed)*
- **0166** — [The container runtime is a node seat, and the host creates containers through its holder](0166-the-container-runtime-is-a-node-seat-and-the-host-creates-containers-through-its-holder.md) *(proposed)*
### How it is built ### How it is built
+23 -1
View File
@@ -2,8 +2,9 @@
layer: to-be layer: to-be
status: in-progress status: in-progress
code: [mesh-host] code: [mesh-host]
updated: 2026-10-01 updated: 2026-10-02
decisions: decisions:
- 02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md
- 02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md - 02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md
- 02-DECISIONS/0141-the-host-delivers-its-own-successor.md - 02-DECISIONS/0141-the-host-delivers-its-own-successor.md
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md - 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
@@ -163,6 +164,27 @@ a resource's former targets, removes a container or file it wrote under a name t
longer names, never removes what was found, and reports what runs on the machine that it neither longer names, never removes what was found, and reports what runs on the machine that it neither
wrote nor holds. *How it is checked:* ADR 0163's table. wrote nor holds. *How it is checked:* ADR 0163's table.
**What the host joins, keeps and raises for a take** — revision, 2026-10-02
([ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 4, 6 and 7). A
container may name networks it also joins once it runs — the found network a per-machine setting keeps
for a taken container while a neighbour still resolves it there; joined after the run, part of the
container's spec, refused when it cannot be joined. A declaration may name the modules the mesh left
out of it because a stored setting cannot compose with the module's definition: the host keeps what it
wrote and holds for a left-out module and says so, where absence used to read as removal. And genesis
raises the bootstrap forge under the forge module's container name, with the module's image digest and
its data directory, so the module holds it by the found rule; the network is the one difference a take
has left to say. *How it is checked:* a host test joins a kept network and refuses one it cannot; a
host test keeps a left-out module's record and hold and removes an absent module's; a bootstrap test
holds the installer's constants to the module's manifest where the catalogue is checked out beside it.
**What filters the machine, and the found firewall kept retired** — revision, 2026-10-02
([ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md)). The host
reports, with every apply, every table and legacy chain that refuses traffic and whose it reads it as —
the mesh's, the found firewall's, the container runtime's own, a ban, or other — and, converged, whether
the firewall it was found with is in force and who retired it. It retires that firewall on every
converged apply, not once, records *found inactive* apart from *disabled by the mesh*, and says when the
step was skipped. *How it is checked:* ADR 0168's table.
**Found reaches every kind that can touch what the machine has** **Found reaches every kind that can touch what the machine has**
([ADR 0103](../../02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md)). For a module not yet taken, a directory present with no record ([ADR 0103](../../02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md)). For a module not yet taken, a directory present with no record
keeps its mode and owner, a unit present with no record keeps its state and boot setting, a keeps its mode and owner, a unit present with no record keeps its state and boot setting, a
+47 -1
View File
@@ -7,8 +7,10 @@ code:
- mesh-controller internal/identity/authority.go - mesh-controller internal/identity/authority.go
- mesh-host internal/identity/serving.go - mesh-host internal/identity/serving.go
- mesh-host internal/apply (the service that reflects a rule set) - mesh-host internal/apply (the service that reflects a rule set)
updated: 2026-09-30 updated: 2026-10-02
decisions: decisions:
- 02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md
- 02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md
- 02-DECISIONS/0148-the-meshs-names-are-resolved-not-copied-into-containers.md - 02-DECISIONS/0148-the-meshs-names-are-resolved-not-copied-into-containers.md
- 02-DECISIONS/0151-a-routes-internal-name-is-composed-under-the-node-that-serves-it.md - 02-DECISIONS/0151-a-routes-internal-name-is-composed-under-the-node-that-serves-it.md
- 02-DECISIONS/0147-a-module-anchors-the-meshs-authority.md - 02-DECISIONS/0147-a-module-anchors-the-meshs-authority.md
@@ -709,6 +711,38 @@ needs no new filter; a declared port is reachable from off the private network a
not; no address of a machine's own networks appears in a rendered filter, asserted on the text; and a not; no address of a machine's own networks appears in a rendered filter, asserted on the text; and a
machine reporting no outward link is refused in the control plane with its existing filter left alone. machine reporting no outward link is refused in the control plane with its existing filter left alone.
### A converged machine is filtered by the mesh alone, and the host says what else refuses
*2026-10-02, [ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md),
from [issues 143](../../04-ISSUES/143-converging-does-not-retire-the-firewall-it-found/00-report.md) and
[144](../../04-ISSUES/144-the-predecessors-rules-outlive-the-firewall-it-was-found-as/00-report.md).*
Retiring the found firewall was a step the flip took once, and said it had taken whatever happened;
on the first machine with one it did not take, and a hand's work fifty minutes later was recorded as
the mesh's. And "the firewall found" named one front end while a predecessor's chain in the container
runtime's user hook — legacy iptables on one machine, invisible to a reader of nftables — filtered the
forwarded path, refused ports the mesh declared open, and carried an allowance every module reaching
another by the machine's own name relied on.
**Convergence is a state the host keeps.** Every converged apply reads whether the found firewall is in
force; enabled again, it is retired again and said; the record says whether the mesh disabled it or
found it inactive, and a skipped step is said. **The host reports what filters the machine**, every
apply, adopted or converged: every table and legacy chain that refuses, with an owner — the mesh's,
the found firewall's, the runtime's own plumbing, a ban, or *other*, which is where the runtime's user
chain's refusals go. **The mesh says which:** `node show` lists them; `status` names a converged machine
anything *other* filters and is not well; the converge preview lists what filters the machine and the
fate of each — retired with the front end, left as the runtime's, left as a ban, or *left in force and
not the mesh's*. The mesh removes none of it; adoption's threshold does not move.
*How it is checked:* host tests over rulesets captured from three machines of this mesh classify every
refusing chain (a predecessor's chain in the legacy filter as *other*, a ban list reached through the
user chain as a ban, a leftover front-end chain as *other*); a fake front end enabled again on a
converged machine is retired again and said, found inactive is recorded as found; the report carries
the filters and the found firewall's state and a change in them is worth an unasked report; controller
tests over a fixture report check the recording, the preview's fates, the status JSON and the well
predicate. Live: the home server's record names the predecessor's chain as *other* and `status`
names the machine until the chain is removed by hand.
## 5 — Certificates ## 5 — Certificates
**Two authorities, kept separate on purpose.** **Two authorities, kept separate on purpose.**
@@ -842,6 +876,18 @@ One value, three readers:
| `public` | the machine port, to anywhere | the public name | the public authority | | `public` | the machine port, to anywhere | the public name | the public authority |
| `both` | the machine port, to anywhere | both names | each name's own authority | | `both` | the machine port, to anywhere | both names | each name's own authority |
*2026-10-02.* **The proxy serves an internal name to the private network only**
([ADR 0138](../../02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md),
its insight of this date). It answers public and internal names on the same listeners, so the name a
request carries is the request's own claim, not where the request came from. An internal name is
served to the machines of the mesh and to the machine itself; to anyone else it is answered as a name
never routed, in the handshake as well as the request. Without this, an endpoint with reach `internal`
would be public under a name that is easy to guess
([issue 191](../../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md)).
The proxy is told who the mesh is, and its routes, in its membership on the bus — the same machine
addresses the filter's "from the mesh" is rendered from
([ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)).
**An endpoint that is not routed is reached and never named.** No route contribution means no name is **An endpoint that is not routed is reached and never named.** No route contribution means no name is
composed and no certificate requested, while the filter still acts on it. That is the case the model composed and no certificate requested, while the filter still acts on it. That is the case the model
could not express at all, and it is the ordinary case for anything that is not HTTP. could not express at all, and it is the ordinary case for anything that is not HTTP.
+16 -1
View File
@@ -8,7 +8,7 @@ code:
- mesh-host packaging/nox-mesh-host-network.sh - mesh-host packaging/nox-mesh-host-network.sh
- mesh-controller internal/token - mesh-controller internal/token
- mesh-controller internal/inventory/nodes.go - mesh-controller internal/inventory/nodes.go
updated: 2026-10-01 updated: 2026-10-02
decisions: decisions:
- 02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md - 02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md - 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
@@ -323,6 +323,21 @@ network are said. `take --yes <digest>` cuts over what was previewed, as the fli
container may keep a found network by a per-machine setting while its neighbours are not yet taken. container may keep a found network by a per-machine setting while its neighbours are not yet taken.
*How it is checked:* ADR 0163's table. *How it is checked:* ADR 0163's table.
**A setting is judged where it is stored, and the take's words** — revision, 2026-10-02
([ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1, 2, 4 and 6).
The preview ends with a digest of what it said; `take --yes <digest>` acts on that preview and nothing
else, and a preview that has changed since, or an account of the machine older than the flip allows, is
refused as the flip's is. A module the machine holds nothing for has nothing to compare, and `--yes`
suffices. The overrides are `--downgrade`, `--replace <path>` and `--mint <name>`; the per-machine
setting that keeps a found network is `networks`, a container id to the networks it keeps, accepted
for an adopted machine only. Storing a setting composes it against the module's current definition and
refuses, naming node, module, layer and key, what cannot compose or reaches nothing. A definition that
later moves under a stored setting costs that module its place in the machine's declaration, said by
name in `plan`, `push` and the declaration itself, and the machine is told everything else; a stray
setting no longer refuses the machine where it is read. *How it is checked:* controller tests over the
one judgement — refused where stored, a module left out where composed, the envelope naming it — and
over a take's digest, staleness and secrets.
A candidate machine is not empty. It has a package manager, probably a container runtime, A candidate machine is not empty. It has a package manager, probably a container runtime,
configuration somebody chose. [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) configuration somebody chose. [ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
says the host never touches what it did not create — adoption is the deliberate act of taking says the host never touches what it did not create — adoption is the deliberate act of taking
+8 -1
View File
@@ -7,8 +7,9 @@ code:
- mesh-tools src/broker-amqp.ts (to be replaced) - mesh-tools src/broker-amqp.ts (to be replaced)
- mesh-catalog modules/nats (to be written) - mesh-catalog modules/nats (to be written)
- mesh-sdk src (the protocol's NATS binding, step 3) - mesh-sdk src (the protocol's NATS binding, step 3)
updated: 2026-10-01 updated: 2026-10-02
decisions: decisions:
- 02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md
- 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md - 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md
- 02-DECISIONS/0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md - 02-DECISIONS/0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md
- 02-DECISIONS/0106-the-bus-is-nats.md - 02-DECISIONS/0106-the-bus-is-nats.md
@@ -94,6 +95,12 @@ list; the account's grant is the same membership read the other way; the console
tool's subject. The one rule a runtime keeps is the membership's own subject, from the two names in its tool's subject. The one rule a runtime keeps is the membership's own subject, from the two names in its
credential. credential.
**Revised 2026-10-02** ([ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)):
a membership also carries **what its module receives**, by requirement — the contributions its
received file is written from, from the same composition — and **who the mesh is**, every machine's
address on the private network, the list the filter's "from the mesh" is rendered from. The route
proxy is the first reader of both.
**Revised 2026-09-27** ([ADR 0129](../../02-DECISIONS/0129-a-seat-carries-the-protocol-of-its-role.md)): **Revised 2026-09-27** ([ADR 0129](../../02-DECISIONS/0129-a-seat-carries-the-protocol-of-its-role.md)):
**`mesh.build.request`, `mesh.control.built` and the BUILDS stream are gone.** A build is work submitted to a role, and the **`mesh.build.request`, `mesh.control.built` and the BUILDS stream are gone.** A build is work submitted to a role, and the
mesh already has a shape for that — a seat's `accept` subjects, on a work queue with a queue group of mesh already has a shape for that — a seat's `accept` subjects, on a work queue with a queue group of
@@ -1,8 +1,11 @@
--- ---
layer: to-be layer: to-be
status: proposed status: in-progress
code: [] code:
updated: 2026-09-27 - mesh-controller internal/inventory
- mesh-controller internal/catalogue
- mesh-controller cmd/mesh-controller
updated: 2026-10-01
decisions: decisions:
- 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md - 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md
- 02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md - 02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md
@@ -14,10 +17,10 @@ decisions:
# 29 — A node has operator accounts, and the mesh owns what lives under a home # 29 — A node has operator accounts, and the mesh owns what lives under a home
**The mesh models machines but not the people on them.** A node record holds its name, its **The mesh models machines but not the people on them.** A node record holds its name, its
address, its mode — and nothing about *who a person is* on it: `jochens` on novox, `ace` on ace, address, its mode — and nothing about *who a person is* on it: one login name on the build node,
`jochen` on shanks and g14. That username is not incidental. It decides who a file under `~` is another on the home-server, a third on both workstations. That username is not incidental. It
owned by, who a user service runs as, and — the case that surfaced this — which account `ssh decides who a file under `~` is owned by, who a user service runs as, and — the case that surfaced
<node>` logs in as. The predecessor knew it (its per-node `user:`, and the modules that wrote a this — which account `ssh <node>` logs in as. The predecessor knew it (its per-node `user:`, and the modules that wrote a
person's `~/.ssh/config`, `~/.zshrc`, `~/.config`); the mesh, taking those over, kept the machine person's `~/.ssh/config`, `~/.zshrc`, `~/.config`); the mesh, taking those over, kept the machine
facts and dropped the human one. facts and dropped the human one.
@@ -28,8 +31,9 @@ Several things are missing, and they are one idea.
A node has one or more **operator accounts**: the human logins on it. At minimum a name; the A node has one or more **operator accounts**: the human logins on it. At minimum a name; the
mesh already knows the node and its address, so `<account>@<node>` is then a complete answer to mesh already knows the node and its address, so `<account>@<node>` is then a complete answer to
"who am I, where." It is the mesh's to hold because everything below is derived from it, and "who am I, where." It is the mesh's to hold because everything below is derived from it, and
because it is exactly the fact that was silently lost — `ssh ace` failed to `ace` because nothing because it is exactly the fact that was silently lost — `ssh home-server` logged in under the
in the mesh said ace's account is `ace`. workstation's own name, because nothing in the mesh said the home-server's account is a different
one.
## 2. A resource may live under a home, owned by its account ## 2. A resource may live under a home, owned by its account
@@ -65,14 +69,14 @@ create `~/.ssh` at `0700`, chown it to the account, and own the files it places
**The boundary — and it is the reason this is safe:** `~/.ssh` is the one directory where a wrong **The boundary — and it is the reason this is safe:** `~/.ssh` is the one directory where a wrong
declaration locks a person out of their own machine. So the mesh's *found-vs-owned* semantics declaration locks a person out of their own machine. So the mesh's *found-vs-owned* semantics
([ADR 0126](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md), ([ADR 0118](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md),
adoption) apply *inside* the home directory. The mesh **owns** the directory and the files above; it adoption) apply *inside* the home directory. The mesh **owns** the directory and the files above; it
**holds as found — never rewrites, never removes** — the operator's own contents: their **private **holds as found — never rewrites, never removes** — the operator's own contents: their **private
keys** and their **personal drop-ins** (`config.d/personal`, the personal `Host` aliases a keys** and their **personal drop-ins** (`config.d/personal`, the personal `Host` aliases a
workstation carries, exactly as `hosts.local` is the home the mesh never rewrites for `/etc/hosts`). workstation carries, exactly as `hosts.local` is the home the mesh never rewrites for `/etc/hosts`).
Reconcile removing an unassigned `config.d/mesh` is fine; the same logic aimed at `id_ed25519` or an Reconcile removing an unassigned `config.d/mesh` is fine; the same logic aimed at `id_ed25519` or an
operator's own `authorized_keys` entry is a lockout. This is the login-channel cousin of the rule operator's own `authorized_keys` entry is a lockout. This is the login-channel cousin of the rule
[ADR 0125](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md) draws for the uplink and the sshd [ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md) draws for the uplink and the sshd
module draws for the firewall: **the mesh must never be able to arrange the one failure that severs module draws for the firewall: **the mesh must never be able to arrange the one failure that severs
its own way back in.** The carve-out is not a convenience; it is that rule, in `~/.ssh`. its own way back in.** The carve-out is not a convenience; it is that rule, in `~/.ssh`.
@@ -108,12 +112,12 @@ found-vs-owned boundary of §3 is exactly what guarantees nothing already there
None of this needs a node to discover the mesh, and none of it needs a control-plane module of its None of this needs a node to discover the mesh, and none of it needs a control-plane module of its
own. The ssh files are **roster facts** own. The ssh files are **roster facts**
([ADR 0128](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md)): once the ([ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md)): once the
roster view carries a node's **host key** and its **account** beside its name and address, the roster view carries a node's **host key** and its **account** beside its name and address, the
`ssh-client` module ships a template for `known_hosts`, `config` and `authorized_keys`, and the `ssh-client` module ships a template for `known_hosts`, `config` and `authorized_keys`, and the
controller renders each node's copy from the full roster and pushes it. The mesh owns the data; the controller renders each node's copy from the full roster and pushes it. The mesh owns the data; the
module owns ssh's format; the control plane gains no ssh syntax. It is the same act as composing a module owns ssh's format; the control plane gains no ssh syntax. It is the same act as composing a
peer list or `/etc/hosts` — which is why there is **no novox-only "mesh-ssh" module**: the peer list or `/etc/hosts` — which is why there is **no control-node-only "mesh-ssh" module**: the
centralization is the controller's composition, not a module that runs somewhere. Only non-secret centralization is the controller's composition, not a module that runs somewhere. Only non-secret
facts travel (names, addresses, accounts, host keys, the CA public key); the private key stays the facts travel (names, addresses, accounts, host keys, the CA public key); the private key stays the
operator's, placed as an operator-owned file, referenced by path. operator's, placed as an operator-owned file, referenced by path.
@@ -129,6 +133,44 @@ operator's, placed as an operator-owned file, referenced by path.
They meet at the account and the CA, not at a bespoke module. The `sshd` server side already exists; They meet at the account and the CA, not at a bespoke module. The `sshd` server side already exists;
the client/identity side and the CA are the open pieces. the client/identity side and the CA are the open pieces.
## What has shipped, and what has not
*Recorded 2026-10-01 from the controller's main branch, not from intent.*
**Built (mesh-controller, merged 2026-09-27):**
- **§1, the account as a node fact.** A node record carries an operator account and, optionally,
its home. Empty is a real state — a freshly enrolled or headless machine has no operator account
known yet — and an empty home means *derive it* (the superuser's home for the superuser, the
conventional per-user home otherwise), so the common case needs no entry. The controller's node
command sets it. One account per node is what exists; "one or several" below is still open.
- **§2, resources under a home.** The account and its home are offered as machine facts, and a
resource's *path and owner* resolve placeholders exactly as its content does — so a module places
a file under a person's home, owned by that person, naming neither. A roster file may say it lives
under the home: it is rendered per node, placed under that node's account's home, chowned to the
account, and a node with no account gets none.
- **§5, the composed ssh config.** The roster rendering carries each node's account, so the
`ssh-client` template can emit a `Host` block per node with the right login name. Composed
end-to-end in the controller's tests.
**Written but not shipped:** the `ssh-client` catalogue module itself exists on a branch of the
module repository; its pull request was closed with a hold until this design is deployed, and
nothing has deployed it since. The predecessor's generator still writes every workstation's ssh
client blocks today — which is where [issue 172](../../04-ISSUES/172-the-ssh-client-block-matches-one-spelling-of-a-machine/00-report.md)
was found.
**Not built:** the SSH CA and certificates (§4), `known_hosts` and `authorized_keys` as roster files,
the found-vs-owned boundary inside `~/.ssh` (§3 — the controller has no rule yet that refuses to
rewrite a private key), adoption of existing keys, the ssh-agent as a user service, and user-scoped
services in general. The host vocabulary still has no user-scope unit at all; a workstation's
per-user daemons (a bar watchdog, a config reloader, an audio service masked per user) have no form
the mesh can send.
**A gap this surfaced:** §1 shipped as code before it had a decision record. The account as a node
fact, the home as a placement root, and what the mesh may and may not do under a home are each a
decision this document names but no record states. They are the next records to write, before the
family of §2 modules is built.
## Why now, and why not yet ## Why now, and why not yet
**Why it matters:** when HAL retires, the generators that keep `~/.ssh`, shell config and the **Why it matters:** when HAL retires, the generators that keep `~/.ssh`, shell config and the
@@ -137,7 +179,7 @@ alias and its trust, and a fresh machine has no operator dotfiles at all — the
service and leave the human unable to work on the box. service and leave the human unable to work on the box.
**Why not build it reflexively:** it is a real addition to the node model, the resource model, and **Why not build it reflexively:** it is a real addition to the node model, the resource model, and
the seat set, and must be gotten right. The mechanism half is now settled — ADR 0128 is what lets the seat set, and must be gotten right. The mechanism half is now settled — ADR 0120 is what lets
the ssh files be templates with no control-plane format — so what remains to decide here is the the ssh files be templates with no control-plane format — so what remains to decide here is the
model: model:
@@ -156,15 +198,18 @@ model:
unnecessary, and forwarding an agent into a node exposes the operator's keys to that node's root — unnecessary, and forwarding an agent into a node exposes the operator's keys to that node's root —
so prefer certificates and `ProxyJump` over forwarding. so prefer certificates and `ProxyJump` over forwarding.
**Not urgent, not blocking.** ssh and dotfiles work today because HAL's generators still run as the **Now load-bearing.** The migration of every node to the mesh is complete; what remains of the
substrate. This becomes load-bearing in the node-by-node retirement phase, not before — which is the predecessor is exactly the user environment this design covers — ssh config, dotfiles, the desktop
right time to build it, once the account and CA model are decided here. stack and the per-user services of the two workstations. Those generators are the last thing
keeping the predecessor running, so the model questions above are no longer deferred: the account
record, the home as a placement root, user-scoped services and the one-off steps a hook used to run
each need a decision before the modules that replace the generators can be written.
## References ## References
- The gap was found generating `~/.ssh/config` from the *HAL* registry (`hal/terminal`'s - The gap was found generating `~/.ssh/config` from the *HAL* registry (`hal/terminal`'s
postConfigure hook), which the nox mesh has no equivalent for. postConfigure hook), which the nox mesh has no equivalent for.
- [ADR 0128](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md) — the roster - [ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md) — the roster
fact mechanism that renders the ssh files, format owned by the module. fact mechanism that renders the ssh files, format owned by the module.
- [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) — the - [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) — the
system-path placement this mirrors for home paths. system-path placement this mirrors for home paths.
@@ -173,6 +218,6 @@ right time to build it, once the account and CA model are decided here.
- [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md) — the CA key is a secret the - [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md) — the CA key is a secret the
vault makes; [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md) vault makes; [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md)
— short-lived certs as rotation. — short-lived certs as rotation.
- [ADR 0125](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md), - [ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md),
[ADR 0126](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md) — [ADR 0118](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md) —
the never-sever-the-channel rule and the found-vs-owned semantics, applied here to `~/.ssh`. the never-sever-the-channel rule and the found-vs-owned semantics, applied here to `~/.ssh`.
+1 -1
View File
@@ -38,7 +38,7 @@ document is written and this one's status becomes `implemented`.
| [`26-the-seats.md`](26-the-seats.md) | **Proposed.** What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) | | [`26-the-seats.md`](26-the-seats.md) | **Proposed.** What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) |
| [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md), [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)) | | [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md), [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)) |
| [`28-building-the-bus.md`](28-building-the-bus.md) | **Proposed.** The five steps of the bus work in the order their dependencies allow, each ending at a bed — with the surface measured, so no step's size is a guess | [ADR 0116](../../02-DECISIONS/0116-the-bus-is-built-in-five-steps.md), [ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md), [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md) | | [`28-building-the-bus.md`](28-building-the-bus.md) | **Proposed.** The five steps of the bus work in the order their dependencies allow, each ending at a bed — with the surface measured, so no step's size is a guess | [ADR 0116](../../02-DECISIONS/0116-the-bus-is-built-in-five-steps.md), [ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md), [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md) |
| [`29-a-node-has-operator-accounts.md`](29-a-node-has-operator-accounts.md) | **Proposed.** The mesh models machines but not the humans on them: a node gains operator accounts, and a resource may live under a home owned by its account — what would own ~/.ssh, dotfiles and ~/.config when HAL retires | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md) | | [`29-a-node-has-operator-accounts.md`](29-a-node-has-operator-accounts.md) | **In progress.** A node has an operator account and a resource may live under its home — built in the controller; the ssh-client module, the SSH CA, the `~/.ssh` boundary and user-scoped services are not. The account fact still wants its decision record | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md) |
| [`32-what-a-module-declares.md`](32-what-a-module-declares.md) | **Proposed.** What a module declares and what the bus derives from it: three namespaces, subjects from local names, queues never declared, the five relationships, and the build-publish-deploy lifecycle on one bus | [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md), [ADR 0127](../../02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md), superseded by [ADR 0131](../../02-DECISIONS/0131-everything-on-the-mesh-speaks-to-the-broker-seat.md) (superseding [ADR 0125](../../02-DECISIONS/0125-the-bus-is-the-only-broker.md)), [ADR 0041](../../02-DECISIONS/0041-events-are-a-relationship.md) | | [`32-what-a-module-declares.md`](32-what-a-module-declares.md) | **Proposed.** What a module declares and what the bus derives from it: three namespaces, subjects from local names, queues never declared, the five relationships, and the build-publish-deploy lifecycle on one bus | [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md), [ADR 0127](../../02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md), superseded by [ADR 0131](../../02-DECISIONS/0131-everything-on-the-mesh-speaks-to-the-broker-seat.md) (superseding [ADR 0125](../../02-DECISIONS/0125-the-bus-is-the-only-broker.md)), [ADR 0041](../../02-DECISIONS/0041-events-are-a-relationship.md) |
@@ -1,8 +1,8 @@
--- ---
status: located status: resolved
opened: 2026-09-22 opened: 2026-09-22
located-in: [mesh-controller internal/overlay, mesh-host internal/apply] located-in: [mesh-controller internal/overlay, mesh-host internal/apply]
fixed-by: fixed-by: ADR 0102 (mesh-controller internal/overlay: the runtime file written into, reloaded), issue 128 (the hosts file as a region)
amended-design: 03-DESIGN/01-to-be/05-the-node-host.md amended-design: 03-DESIGN/01-to-be/05-the-node-host.md
--- ---
@@ -56,3 +56,12 @@ and nothing checks for it today.
- Should an adopted node that cannot trust the registry be refused a module that needs to pull? - Should an adopted node that cannot trust the registry be refused a module that needs to pull?
Or should the refusal come earlier, when the node joins? Or should the refusal come earlier, when the node joins?
## Resolved, 2026-10-02
The runtime's file is written into and the runtime reloaded, never restarted
([ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md), the
diagnosis above); the hosts file is a marked region the mesh owns alone
([issue 128](../128-the-hosts-file-is-written-whole/00-report.md)). Neither whole file remains. Read
into [ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md), rule 5,
which names the one whole machine-wide file the mesh still writes — its own filter at the
distribution's path — as a difference a take shows, not a fault.
@@ -1,8 +1,8 @@
--- ---
status: located status: resolved
opened: 2026-09-22 opened: 2026-09-22
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)] located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
fixed-by: fixed-by: mesh-controller 201 (the preview names the narrowing and the port's reach), 206 (`take --yes <digest>` acts on the preview read)
amended-design: amended-design:
--- ---
@@ -40,3 +40,13 @@ it changes before it changes it, and for taking a module this one does not.
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 2: the preview names a narrowing. Building follows, [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 2: the preview names a narrowing. Building follows,
host first, then the controller's `take`. host first, then the controller's `take`.
## Built, 2026-10-02
mesh-controller 201 and the pull request after it: the preview names it, and `take --yes <digest>`
acts on the preview that was read. Stays located until a take is read on an adopted machine — every
machine of this mesh is converged today, so the record's live row has not been run.
## Resolved, 2026-10-02
Closed on the operator's decision of 2026-10-02 with the built and tested code live on every machine (mesh-controller 206, mesh-host 64), not on a take read on an adopted machine: every machine of this mesh is converged, so none holds a found thing to compare, and the record's live row — ADR 0163's last — will be read at the next real adoption rather than staged. Said here so nobody later believes that row was run.
@@ -1,8 +1,8 @@
--- ---
status: located status: resolved
opened: 2026-09-22 opened: 2026-09-22
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)] located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
fixed-by: fixed-by: mesh-host 64 (genesis raises the forge as `gitea`, on the module's image digest, with the module's data directory at /data; a test holds the installer to the module's manifest)
amended-design: amended-design:
--- ---
@@ -53,3 +53,17 @@ network, or it is not a takeover.
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 7: genesis raises as the module declares. Building follows, [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 7: genesis raises as the module declares. Building follows,
host first, then the controller's `take`. host first, then the controller's `take`.
## Built in part, 2026-10-02
mesh-host 64: genesis raises the forge under the module's container name (`gitea`), pinned to the
module's image digest, with the module's data directory mounted at `/data` — so the module finds it,
holds it, and a take compares equal images and the same data. A test holds the installer's constants
to the module's manifest where the catalogue is checked out beside it. The network is the difference
left: the bootstrap forge runs on the machine's network to reach the store on its loopback, the module
runs bridged and publishes its ports, and the take says so. Closing waits for group 9's genesis test —
a mesh raised, the module assigned, and the module found holding rather than raising a second forge.
## Resolved, 2026-10-02
Closed on the operator's decision of 2026-10-02. Name, image and data directory align; the network does not — the bootstrap forge runs on the machine's network to reach the store on its loopback, the module runs bridged — and a take says so rather than hides it. Whether genesis should move the forge onto a bridge, and the test that raises a mesh and finds the module holding rather than raising a second forge, belong to group 9's genesis work and are not owed by this record any more.
@@ -1,8 +1,8 @@
--- ---
status: located status: resolved
opened: 2026-09-23 opened: 2026-09-23
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)] located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
fixed-by: fixed-by: mesh-controller (the pull request after 201: JudgeSettings, LeftOut), mesh-host 64 (left_out kept)
amended-design: amended-design:
--- ---
@@ -63,3 +63,12 @@ knowing the code.
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 6: judged where stored; an impossible statement costs a module. Building follows, [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 6: judged where stored; an impossible statement costs a module. Building follows,
host first, then the controller's `take`. host first, then the controller's `take`.
## Resolved, 2026-10-02
One judgement, in the catalogue, run where a setting is stored and where a machine is composed. Stored,
a setting that cannot compose with the module's current definition is refused naming the node, the
module, the layer and the key; a key that reaches nothing is refused there too. Composed, a definition
that moved under a stored setting leaves that module out of the machine's declaration — the envelope
names it, the host keeps what it holds and wrote for it, `plan` and `push` say it — and the machine is
told everything else. A stray setting no longer refuses the whole machine where it is read.
@@ -1,8 +1,8 @@
--- ---
status: located status: resolved
opened: 2026-09-23 opened: 2026-09-23
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)] located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
fixed-by: fixed-by: mesh-host 63 (former targets removed, strays reported), mesh-controller 201/202 (strays shown)
amended-design: amended-design:
--- ---
@@ -80,3 +80,11 @@ found, and so would be kept for ever on purpose.
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 5: former targets are removed and strays reported. Building follows, [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 5: former targets are removed and strays reported. Building follows,
host first, then the controller's `take`. host first, then the controller's `take`.
## Resolved, 2026-10-02
mesh-host 63: the host's record keeps a resource's former targets, removes a container or file it
wrote under a name the declaration no longer names, never what was found, and reports strays — what
runs on the machine that the mesh neither wrote nor holds. mesh-controller 201 and 202 show strays
on `node show` for an adopted and a converged machine alike; the live mesh reported four on the
control node the evening it rolled.
@@ -1,8 +1,8 @@
--- ---
status: located status: resolved
opened: 2026-09-23 opened: 2026-09-23
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)] located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
fixed-by: fixed-by: mesh-host 63 (the kept original's difference), mesh-controller 201 (shown; a differing file refuses unless `--replace <path>`)
amended-design: amended-design:
--- ---
@@ -68,3 +68,13 @@ written.
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 2: the difference is shown and a differing file refuses. Building follows, [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 2: the difference is shown and a differing file refuses. Building follows,
host first, then the controller's `take`. host first, then the controller's `take`.
## Built, 2026-10-02
mesh-host 63 reports the difference between the kept original and the declared content; mesh-controller
201 shows it in the preview and refuses a differing file unless `--replace <path>` names it, or the
module declares the file partially. Stays located until a take is read on an adopted machine.
## Resolved, 2026-10-02
Closed on the operator's decision of 2026-10-02 with the built and tested code live on every machine (mesh-controller 206, mesh-host 64), not on a take read on an adopted machine: every machine of this mesh is converged, so none holds a found thing to compare, and the record's live row — ADR 0163's last — will be read at the next real adoption rather than staged. Said here so nobody later believes that row was run.
@@ -1,8 +1,8 @@
--- ---
status: located status: resolved
opened: 2026-09-23 opened: 2026-09-23
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)] located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
fixed-by: fixed-by: mesh-host 63 (both images' creation dates), mesh-controller 201 (DOWNGRADE said; refused unless `--downgrade`)
amended-design: amended-design:
--- ---
@@ -65,3 +65,13 @@ expected rate.
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 2: the images are compared by age and a downgrade refuses. Building follows, [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 2: the images are compared by age and a downgrade refuses. Building follows,
host first, then the controller's `take`. host first, then the controller's `take`.
## Built, 2026-10-02
mesh-host 63 reports the found image and both images' creation dates; mesh-controller 201 says
DOWNGRADE and refuses unless `--downgrade` is said. Stays located until a take is read on an adopted
machine.
## Resolved, 2026-10-02
Closed on the operator's decision of 2026-10-02 with the built and tested code live on every machine (mesh-controller 206, mesh-host 64), not on a take read on an adopted machine: every machine of this mesh is converged, so none holds a found thing to compare, and the record's live row — ADR 0163's last — will be read at the next real adoption rather than staged. Said here so nobody later believes that row was run.
@@ -1,8 +1,8 @@
--- ---
status: located status: resolved
opened: 2026-09-23 opened: 2026-09-23
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)] located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
fixed-by: fixed-by: mesh-controller 201 (`secret accept --provider` reaches a required secret), 206 (a module's secrets listed with origin; a minted one for found data refuses unless `--mint <name>`)
amended-design: amended-design:
--- ---
@@ -70,3 +70,15 @@ the module can only be installed fresh.
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 2 and 3: a minted secret for found data refuses; secret accept reaches required secrets. Building follows, [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 2 and 3: a minted secret for found data refuses; secret accept reaches required secrets. Building follows,
host first, then the controller's `take`. host first, then the controller's `take`.
## Built, 2026-10-02
`secret accept <node> <module> <name> --provider <node>` reaches a required secret (mesh-controller 201).
The pull request after it reads every secret a module holds on a machine with its origin, and a take
of a module whose data was found refuses a minted, unaccepted one — naming the accept that carries
the existing value in, or `--mint <name>` to let the service take the new one. Stays located until
a take is read on an adopted machine.
## Resolved, 2026-10-02
Closed on the operator's decision of 2026-10-02 with the built and tested code live on every machine (mesh-controller 206, mesh-host 64), not on a take read on an adopted machine: every machine of this mesh is converged, so none holds a found thing to compare, and the record's live row — ADR 0163's last — will be read at the next real adoption rather than staged. Said here so nobody later believes that row was run.
@@ -1,8 +1,8 @@
--- ---
status: located status: resolved
opened: 2026-09-23 opened: 2026-09-23
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)] located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
fixed-by: fixed-by: mesh-controller 201 (the neighbours on a found network are named), 206 (the per-machine `networks` setting), mesh-host 64 (the taken container joins the kept network)
amended-design: amended-design:
--- ---
@@ -66,3 +66,14 @@ exercise.
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 4: the neighbours are named; a found network may be kept by a setting. Building follows, [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 4: the neighbours are named; a found network may be kept by a setting. Building follows,
host first, then the controller's `take`. host first, then the controller's `take`.
## Built, 2026-10-02
The preview names every neighbour on a found network (mesh-controller 201). The pull request after it
adds the per-machine setting `networks` — a container id to the found networks it keeps — judged for an
adopted machine only, and mesh-host 64 has the taken container join each once it runs. Stays located
until a take is read on an adopted machine.
## Resolved, 2026-10-02
Closed on the operator's decision of 2026-10-02 with the built and tested code live on every machine (mesh-controller 206, mesh-host 64), not on a take read on an adopted machine: every machine of this mesh is converged, so none holds a found thing to compare, and the record's live row — ADR 0163's last — will be read at the next real adoption rather than staged. Said here so nobody later believes that row was run.
@@ -1,7 +1,9 @@
--- ---
status: located status: resolved
opened: 2026-09-26 opened: 2026-09-26
located-in: [mesh-host internal/apply] located-in: [mesh-host internal/apply]
fixed-by: mesh-host 63 (every written field compared), mesh-controller 201 (build says the policy)
amended-design:
--- ---
# 126 — a volume path is not in the spec comparison, and a roll-out raced a data move # 126 — a volume path is not in the spec comparison, and a roll-out raced a data move
@@ -50,3 +52,9 @@ the install-page junk was discarded twice.
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 5 and 7: every field compared; build says the policy. Building follows, [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 5 and 7: every field compared; build says the policy. Building follows,
host first, then the controller's `take`. host first, then the controller's `take`.
## Resolved, 2026-10-02
mesh-host 63: every field the host writes is compared before a container is called current, volumes
and paths included. mesh-controller 201: `build` and the take-in say when a module's policy rolls a
result out at once; under ADR 0162 the plan says it too.
@@ -1,10 +1,10 @@
--- ---
status: located status: resolved
opened: 2026-09-28 opened: 2026-09-28
located-in: located-in:
- mesh-controller internal/catalogue/filtering.go - mesh-controller internal/catalogue/filtering.go
- mesh-host internal/apply - mesh-host internal/apply
fixed-by: fixed-by: ADR 0140 — mesh-controller (the filter around outward links; no network ranges anywhere)
amended-design: 03-DESIGN/01-to-be/08-connectivity.md amended-design: 03-DESIGN/01-to-be/08-connectivity.md
--- ---
@@ -86,3 +86,11 @@ supersedes both 0137 and the first attempt at answering this.
runtime, or left as the one constant? runtime, or left as the one constant?
- Should the preview say which of a machine's networks are the mesh's and which are not, so a range - Should the preview say which of a machine's networks are the mesh's and which are not, so a range
that exists to protect a leftover is visible as such? that exists to protect a leftover is visible as such?
## Resolved, 2026-10-02
By [ADR 0140](../../02-DECISIONS/0140-the-filter-constrains-what-arrives-from-outside.md), built and
live since 2026-09-29: the forward chain constrains what arrives on the machine's outward links and
says nothing about networks, so there is no list to derive and nothing for a preview to tell apart.
Read into [ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md),
rule 5, which closes it.
@@ -1,10 +1,10 @@
--- ---
status: located status: resolved
opened: 2026-09-29 opened: 2026-09-29
located-in: located-in:
- mesh-host internal/apply/opening.go (retireFirewall) - mesh-host internal/apply/opening.go (retireFirewall)
- mesh-host internal/apply/apply.go (the condition it is called under) - mesh-host internal/apply/apply.go (the condition it is called under)
fixed-by: fixed-by: mesh-host 67 (retire on every converged apply; found-inactive apart from disabled-by-mesh; a skipped step said), mesh-controller 211 (the found firewall's state on node show)
amended-design: amended-design:
--- ---
@@ -100,3 +100,21 @@ harmless, but the mesh's belief about which firewall is in force has been wrong
so". Should it? so". Should it?
- Why do the host's own detail lines not reach the journal? Everything it decided during the flip is - Why do the host's own detail lines not reach the journal? Everything it decided during the flip is
unrecoverable, which is why this account has candidates instead of a cause. unrecoverable, which is why this account has candidates instead of a cause.
## Decided, 2026-10-02
[ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md), rule 1:
convergence is a state the host keeps — the found firewall active again is retired again and said, a
reconcile that finds it inactive records *found so* and never *done by the mesh*, and a step skipped
after a failed apply is said. Built in mesh-host on `feat/one-thing-filters-a-converged-machine`; the
record of both machines of this mesh is corrected by the first report under it.
## Resolved, 2026-10-02
mesh-host 67 and mesh-controller 211, live on every machine at 10:10Z. The step now runs on every
converged apply and says what it did; a found firewall enabled again is retired again. The record's
one inherited lie stands as history: on the control node the machine's own record already said the
mesh had disabled the firewall, and the host trusts its record, so `node show` says "retired by the
mesh" there. From this build on, a reconcile that finds the firewall inactive records *found inactive*
and never the other thing. Whether the flip's step took on 2026-09-29 is not recoverable and is not
owed by this record any more.
@@ -1,10 +1,10 @@
--- ---
status: located status: resolved
opened: 2026-09-29 opened: 2026-09-29
located-in: located-in:
- mesh-host internal/apply/opening.go - mesh-host internal/apply/opening.go
- mesh-controller cmd/mesh-controller (the converge preview) - mesh-controller cmd/mesh-controller (the converge preview)
fixed-by: fixed-by: mesh-host 67 (every refusing table and legacy chain classified with an owner; the runtime's user chain is other), mesh-controller 211 (kept, shown on node show, named by status, previewed with fates)
amended-design: amended-design:
--- ---
@@ -82,3 +82,24 @@ everything reached from within.
- Is the bus and the registry being reachable from anywhere still what the mesh wants on a machine that - Is the bus and the registry being reachable from anywhere still what the mesh wants on a machine that
faces the internet? The design says yes, for enrolment. It deserves asking on its own rather than faces the internet? The design says yes, for enrolment. It deserves asking on its own rather than
being answered by a leftover. being answered by a leftover.
## Decided, 2026-10-02
[ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md), rules 2 and
3: the host reports every table and legacy chain that refuses, with an owner, and the runtime's user
chain's refusals as *other*; `node show`, `status` and the converge preview say it. Built on
`feat/one-thing-filters-a-converged-machine` in mesh-host and mesh-controller. On 2026-10-02 the home
server still carries the predecessor's chain in its legacy filter; the record's live row is reading it
there.
## Resolved, 2026-10-02
mesh-host 67 and mesh-controller 211, live at 10:10Z. The live row of
[ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md) was read the
same hour: the home server's record names the predecessor's chain in the legacy filter's user chain
as *other*, with what it refuses, beside two chains a retired front end left in the IPv6 legacy filter;
the control node's record names the same two leftovers; the laptop and the workstation read *the mesh
alone*. `status` names both machines and is not well until the operator removes what the mesh did not
write. The allowance the predecessor's chain carried is
[issue 145](../145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md)'s,
and that record is not closed by this one.
@@ -0,0 +1,85 @@
---
status: located
opened: 2026-10-01
located-in: [mesh-catalog modules/dnsmasq, mesh-controller internal/overlay/generator.go, mesh-controller internal/catalogue/resolve.go (checkResources)]
fixed-by:
amended-design:
---
# 190 — The container runtime's configuration is written by modules that are not the runtime's
## What was observed
The runtime's configuration file and its service are declared by two parties, neither of which is
the runtime:
- **The resolver module** writes the runtime's `dns` key (the machine's private address) and
`live-restore` into the runtime's file, written into rather than over
([ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md)). It also
declares the runtime's service, reloaded when that file changes. The `dns` key has been written
since the resolver module was converted from its predecessor on 2026-09-23; `live-restore` and the
service were added on 2026-09-30 while fixing
[issue 110](../110-a-container-on-the-runtimes-own-network-cannot-reach-the-resolver/00-report.md),
where containers silently resolved through a public resolver.
- **The private network** writes the runtime's `insecure-registries` into the same file, and declares
the same service reloaded on it, as [ADR 0082](../../02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md)
and ADR 0102 decided. The controller generates both resources per machine.
On the three machines that run the resolver module, both declare one path and one unit. Nothing refuses
it. The collision check compares the resources of catalogue modules. The private network is computed,
so its resources are produced when a machine's declaration is composed, and the check never sees them.
The machine without the resolver module shows the other half. Its runtime still has the predecessor's
resolver and `live-restore` off, because the only module that sets them is a DNS server. A machine
gets a correct container runtime only as a side effect of being given a resolver.
> **Later the same day, 2026-10-02.** The resolver module and its sibling for the resolver file were
> assigned to the fourth machine ([issue 198](../198-the-lans-dns-server-ran-outside-the-mesh-and-its-filter-closed-it/00-report.md)),
> so all four now have the resolver writing into the runtime's file, and the predecessor's
> `live-restore: false` there is gone. The same work made the runtime's file, as the resolver declares
> it, take no settings: a setting meant for the resolver's own configuration had reached it. The
> collision and the ownership question above are unchanged.
## Why this is here
The operator ruled it a defect, not a design: **a module does not write another software's
configuration.** The need behind each write is real. Containers must resolve the mesh's names
([ADR 0148](../../02-DECISIONS/0148-the-meshs-names-are-resolved-not-copied-into-containers.md) step 2).
A daemon restart must not stop every container. Every machine on the network must trust the mesh's
registry. But each of these is a fact the runtime must be *given*, and the module that gives it is the
runtime's own. With three writers, nobody can say what the file should contain. Two of the facts are
reloaded when one of them needs a restart (issue 110's first fault). And the moment a module for the
runtime exists, it is refused on every machine with the resolver, or, through the private network's
path, accepted without anyone noticing a collision.
## What resolves it
[ADR 0166](../../02-DECISIONS/0166-the-container-runtime-is-a-node-seat-and-the-host-creates-containers-through-its-holder.md)
gives the runtime a module that holds its seat and owns its file and service.
[ADR 0164](../../02-DECISIONS/0164-a-setting-is-declared-with-its-default-its-meaning-and-what-changing-it-costs.md)
gives that module declared settings with defaults. The fix, once both are accepted:
1. The resolver module drops its runtime file and runtime service. It knows nothing of the runtime.
2. The private network stops generating either resource. ADR 0082's decision stands — being on the
network is what grants the trust, and no module author is involved — and only *who writes it*
moves. The mesh gives the registry to the runtime module as a value. ADR 0082 and ADR 0102 each
get a dated note saying where their mechanism now lives.
3. The runtime module writes `dns`, `live-restore` and `insecure-registries`, each a declared
setting with its cost: `dns` costs a restart, which `live-restore` makes harmless.
4. Steps 1–3 land in one push. A runtime module declaring the file beside a resolver module still
declaring it is refused.
5. The collision check sees a computed module's resources as well, so a second writer cannot come
back through generated code.
## Open questions
- **How the resolver's address reaches the runtime.** Either the resolver seat (`node-dns-resolver`)
delivers an address its holder serves, or the runtime module reads a machine fact and the seat
being held is only a precondition. The first tracks a resolver moving off the private address. The
second needs nothing new.
- **What `dns` defaults to on a machine with no resolver seat held.** Nothing, leaving the runtime's
own behaviour, is the honest default. A public resolver hides exactly the failure issue 110 took a
day to find.
- **The adopted machine's predecessor values.** The runtime module adopting a file with a
hand-written `dns` and `live-restore: false` replaces both. That is intended, and is the one
restart the operator must make on that machine.
@@ -0,0 +1,80 @@
---
status: resolved
opened: 2026-10-01
located-in: [mesh-controller examples/route-proxy/main.go (routesFrom requires a route's public `name` and treats `internal-name` only as an alias of it; the handler serves every routed name to any source), mesh-controller internal/broker/membership.go (a membership says nothing of what its module receives or who the mesh is)]
fixed-by: mesh-controller PR 207 (the membership carries what a module receives and who the mesh is; the proxy follows it and serves internal names to the mesh only), mesh-catalog PR 211 (the proxy's bus account), mesh-controller PR 208 (the issue verb that delivers it), live 2026-10-02
amended-design: [03-DESIGN/01-to-be/08-connectivity.md, 03-DESIGN/01-to-be/25-the-bus-on-nats.md]
---
# 191 — A route with only an internal name is dropped as naming nothing
## What was observed
A module whose endpoint reaches only the private network could not be reached by its internal name.
The module ran and answered on its own port. Its route's internal name resolved to the serving node.
The request failed during the TLS handshake:
```
http: TLS handshake error from …: no public route for "unifi.home-server.internal" in this mesh,
so no certificate is asked for
```
The proxy's own log said why, every time it re-read its routes:
```
unifi on home-server asked for a route and named nothing; skipped
```
The route it skipped was not empty. The mesh had given it an endpoint, a port, a scheme and an internal
name, and no public name:
| route | `name` | `internal-name` | served |
|---|---|---|---|
| home-assistant | a public name | `home-assistant.home-server.internal` | under both |
| unifi | — | `unifi.home-server.internal` | under neither |
Three other modules on the same node were skipped with the same line on the same pass.
## Why it matters
**Reach is decided in one place, and the proxy reads the old shape of the decision.**
[ADR 0138](../../02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md)
made an endpoint's reach decide which names exist. The controller composes the public name, the
internal name, or both, and composes no name that nobody asked for
([issue 140](../140-an-endpoints-reach-is-not-declared/01-resolution.md)). An endpoint that reaches
only the private network is the ordinary case for anything that should not face the internet. It is
exactly the case the proxy drops.
**The failure is quiet and points the wrong way.** Nothing marks the module unhealthy. The handshake
error says *no public route*, which reads as a certificate fault on the reader's side. The line that
gives the real cause is one of four identical lines repeated every few seconds in a log nobody reads
until they already suspect the proxy.
**The opposite move is not a workaround.** Giving the endpoint public reach makes the proxy serve it.
It also publishes an administration interface to the internet to get a name on the private network.
## Open questions
- Should the proxy refuse a route it cannot serve in a way the controller or an operator sees, not
only in its own log? The same silent skip covers a route with no usable port or an unknown scheme.
- What checks that what the controller composes and what the proxy serves stay the same shape? ADR
0138 changed one side and nothing failed on the other.
## Resolved (2026-10-02)
Built as [ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)
decided, and live on both machines that run the proxy. Each logs that its routes now come from its
membership, and serves internal names to the four machines the mesh names. Checked by hand:
- the internal-only route answers through the proxy from the serving machine and from two other
machines of the mesh, over a certificate from the mesh's own authority that each verifies;
- the same name asked from an address outside the mesh is answered as a name never routed, over plain
HTTP, and refused in the TLS handshake; the list of served names it is shown leaves out every internal
name.
Two things the rollout found are their own records: the proxy's bus account could be issued only from
the controller's command line, until mesh-controller PR 208 added the `issue` verb, and the status line
counting every module as a bus user without a credential is
[issue 195](../195-every-assigned-module-is-counted-as-a-bus-user-without-a-credential/00-report.md).
The serving machine also lacked the certificate-trust module, so it could not verify the mesh's own
certificates until it was assigned there.
@@ -0,0 +1,75 @@
# Diagnosis
*2026-10-01.*
**Ruled out first: the module itself.** Its container was up and had not restarted. The controller's
status endpoint answered on its own port with `"up": true`. The tool wrapper beside it was serving
all its tools.
**Ruled out: name resolution.** The internal name resolved to the serving node's private-network
address, which is where the proxy listens. Plain HTTP to the name reached the proxy and got a 404.
HTTPS failed in the handshake, and the proxy logged that it had no route for the name.
**The route as the proxy received it.** The mesh-written route file held a complete contribution
for the module: endpoint `web`, port, scheme `https`, `insecure`, a label, and `internal-name`. It had
no `name`. That is what the controller composes for an endpoint whose reach stops at the private
network (ADR 0138, `composeName`). The contribution was correct.
**Located: `routesFrom` in the proxy.** It reads `name` first and skips the contribution if `name`
is empty. It reads `internal-name` only at the end, as a second host for a rule that already has a
public one. So the proxy can serve an internal name only next to a public one. That matched the
mesh before ADR 0138, when both names were always composed. It has been wrong since then.
The other half of the proxy already handles the case. Certificates for a host are split by whether it
is in the public set: hosts outside it go to the internal authority, and only hosts inside it are
eligible for ACME. A host that is only ever an internal name falls on the correct side of both checks
without change. For certificates, only reading the route was wrong; who may reach the route is the next section.
**The fix.** `routesFrom` takes a route that names either host, serves each name it carries, and
marks only the public one as public. It still skips a route that names neither, with the same log line.
A test proves an internal-only route is served, certified by the internal authority, and refused by
the public one. That test fails against the code before the change.
## The first fix would have made the name public — 2026-10-02, from review
Serving the dropped route was not enough. The proxy picks a route from the name a request carries
and never from where the request came from, and it answers public and internal names on the same
listeners. Its public names resolve to an address the internet reaches. So once the internal-only
route was served, any request from the internet carrying `unifi.home-server.internal` — a name of a
fixed, guessable shape — would have reached an administration interface that reach `internal` was
chosen to keep private. Before the fix the route was unreachable from everywhere. After it, it would
have been reachable from everywhere. Two more leaks came with it: the proxy's answer for an unrouted
name listed every name it serves, internal ones included, and the handshake handed a certificate
naming the internal host to any client.
Nothing showed this while every routed endpoint also had a public name: its internal name exposed
nothing the public one did not. It is a gap in the decision's wording, not only in the proxy — ADR
0138 says the proxy *serves* the internal name without saying to whom — so it is recorded there as a
progressive insight and in the to-be connectivity design.
**Where "inside" is decided: told, not worked out.** The first correction had the proxy work it out
for itself — the mesh's range from an environment variable the catalogue wrote, and the machine's
container bridges from its own interfaces. That was a second definition of "the mesh", kept by one
module beside the one the controller already has: it resolves "from the mesh" to every machine's
address on the private network, and the packet filter is rendered from that list. Reviewed, it was
replaced: [ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)
has every membership on the bus carry what its module receives and that list, and the proxy follows
its membership. One composition, read by the filter and by the proxy.
The proxy reads the source address, where the guard reads the interface, because it cannot see the
interface a request arrived on. A claimed source does not carry here: a connection needs its replies,
and replies to a mesh address leave by the tunnel.
**What changed with it.** The internal name of a route that also has a public one is now served to the
mesh only, like any other internal name. Outsiders have the public name, so nothing they could reach is
lost. A container calling its own machine's internal name arrives from its container network and is
refused; whether the mesh should issue those networks too is left open in ADR 0167.
**Order of release.**
1. The catalogue change, which gives the proxy a bus account. A machine running the proxy is not
composed until its account is issued, so the account is issued straight after
(`module issue route-proxy --node <machine>`), and then the machine is pushed.
2. The controller and proxy change. The push after it publishes memberships that carry the routes and
the mesh, and each proxy takes them. Until then, a proxy serves its file, and internal names to its
own machine alone.
@@ -1,8 +1,8 @@
--- ---
status: located status: resolved
opened: 2026-10-02 opened: 2026-10-02
located-in: [mesh-host internal/apply (removeOrphan: a former target of a kind with no removal was fatal), mesh-host internal/store (Record keeps a former target for every kind, the host's own archive included)] located-in: [mesh-host internal/apply (removeOrphan: a former target of a kind with no removal was fatal), mesh-host internal/store (Record keeps a former target for every kind, the host's own archive included)]
fixed-by: fixed-by: mesh-host 65 — a former target of a kind the host cannot remove is left in place, said and forgotten; a dropped archive still refuses
amended-design: amended-design:
--- ---
@@ -62,3 +62,10 @@ because the alternative was four machines that could apply nothing.
- Is there a bed that replaces a host under the current rules before the live mesh does - Is there a bed that replaces a host under the current rules before the live mesh does
(the proof row of [ADR 0141](../../02-DECISIONS/0141-the-host-delivers-its-own-successor.md) was (the proof row of [ADR 0141](../../02-DECISIONS/0141-the-host-delivers-its-own-successor.md) was
a single crossover, before former targets existed)? a single crossover, before former targets existed)?
## Resolved, 2026-10-02
mesh-host 65, merged 07:35Z. Recovered as the record above says: the operator dropped the one
`@former:` entry from each machine's host record and pushed; the fixed host then ran on all four and
its first apply said `forgotten mesh-host.next@former:… a former target left in place` and applied the
rest. The open questions stand as questions for the host's own versions, not as faults.
@@ -0,0 +1,62 @@
---
status: open
opened: 2026-10-02
located-in: []
fixed-by:
amended-design:
---
# 195 — Every assigned module is counted as a bus user without a credential, and the real gaps are lost in the count
## What was observed
`status`, and `plan` for any machine, open with one line before anything else:
```
the bus's user list leaves out 49 user(s) the mesh has minted no credential for: <node>.<module>, …
Each is a user that cannot connect until one is issued
```
The 49 are spread over four machines and name 26 distinct modules. Checked against the catalogue on
2026-10-02:
| what the module's definition says | modules |
|---|---|
| declares an own secret named `broker` | 1 — the route proxy, which needed a bus account for issue 191 |
| declares no `broker` secret, and emits, consumes and serves nothing on the bus | 17 — the packet filter, the intrusion filter, the ssh daemon, the resolver configuration, the certificate authority, the broker itself and others |
| declares no `broker` secret, and **emits events** | 1 |
| not in this catalogue, so not checked | 7 |
So the line counts every module assigned anywhere as a bus user. For almost all of them that is not a
missing credential. A module with no `broker` secret has nowhere to receive one, and the mesh already
says an account nothing reads is an orphan ([issue 078](../078-a-delivered-secret-is-accepted-under-any-name/00-report.md)).
Two real gaps sit inside the count and cannot be told from the noise:
- **A declared `broker` secret was filled with a value that is not an account.** Before its account was
issued, the route proxy's plan on both machines already carried a sealed `broker` file, while the same
status line said no credential had been minted for it. A push had made the declared secret the way it
makes any own secret. The module would have started with a credential the bus does not know, and
nothing would have said why. It was found only because the account was being issued by hand.
- **A module that emits events declares no way to reach the bus.** Its events can go nowhere, and no
check refuses that.
## Why it matters
**A warning that is always on is read as never on.** The line names 49 users on every `status` and every
`plan`. An operator, or an agent, learns to scroll past it. The one entry that was a real fault looked
exactly like the 48 that were not.
**The fault that was real is the silent kind.** A module whose broker credential is a generated value
starts, fails to authenticate, and reports that three layers away from the cause. That is the failure
the composition already refuses for a secret that was never made at all ("declared and not made"). Here
a value was made, so the refusal never fired.
## Open questions
- Should a bus user be composed for a module that declares no `broker` secret at all? If not, the line
shrinks to the modules that can actually use an account.
- Is a `broker` secret ever correctly made by the generic generator? If not, should composition refuse
a declared `broker` until it is issued, or should the mesh issue it as part of placing the module?
- Should a module that emits, consumes or serves on the bus be refused when it declares no `broker`
secret?
@@ -0,0 +1,54 @@
---
status: resolved
opened: 2026-10-02
located-in: [mesh-controller internal/catalogue/filtering.go (AsNftables: the forward chain has no rule for the mesh passing through, so a relayed packet is judged by this machine's own published ports)]
fixed-by: mesh-controller PR 209 (the forward chain relays what comes in and goes out on the tunnel), live 2026-10-02
amended-design: []
---
# 196 — The hub relays the mesh only on the ports it publishes for itself
## What was observed
A sweep of every listening port on every machine, from every other machine, on 2026-10-02. Two home
machines, neither of which can be dialled, reach a third home machine through the hub, as
[ADR 0007](../../02-DECISIONS/0007-connectivity.md) says every path between machines that are not
co-located does.
From either of the two, the third answered on **17 of its 55** listening ports over the mesh. The hub
itself, probing the same machine directly, reached all the ports that machine's rules open to the mesh.
The result was the same at 40 probes in parallel and at 4, so it was not load.
The 17 were not a property of the target. They were exactly the ports **the hub** publishes for its own
containers: ssh, the proxy's two, and the hub's own block of published ports. A capture on the target
during one probe to a port that answered and one that did not:
- the answering one: the SYN arrives on the tunnel, reaches the container, and the reply leaves by the
tunnel;
- the other: nothing arrives at all, on any interface.
## Why it matters
**ADR 0007's hub carries every path between machines that are not co-located, and the filter breaks
that path without saying so.** Whether one home machine can reach a service on another depends on
whether the hub happens to publish the same port number for something of its own. Adding or removing
a module on the hub silently opens or closes paths between two other machines that it has nothing to
do with.
It also hid behind another fault. A missing placement made the same pair look disconnected earlier the
same day, and that explanation fit well enough that the per-port pattern was not looked for.
## Open questions
- The relaying rule accepts what comes in on the tunnel and leaves on it, and leaves judging to the
machine it is for. Should the hub also restrict relayed traffic to what that machine opens to the
mesh? That would duplicate the target's rules on the hub.
- No test raises two machines behind a hub and checks a port between them that the hub does not
publish. The lab's beds have one machine per site.
## Resolved (2026-10-02)
Live on all four machines after one push each. The same sweep, from both home machines to the third
over the mesh: 45 of 55 ports answer, the same 45 the hub reaches directly. The 9 that do not are
ports the target opens to nobody on the mesh, and one is refused because it listens only on a LAN
address. Nothing answers that the target's rules do not open.
@@ -0,0 +1,27 @@
# Diagnosis
*2026-10-02.*
**Not the tunnel.** The route from either home machine to the target is the tunnel, and traffic to the
17 ports travels it in both directions. A placement fault would have stopped every port.
**Not the target's filter.** The target opens the failing ports to every address of the mesh in its
input chain and its forward chain, the hub reaches them directly, and the SYN for a failing port never
arrived at the target to be judged.
**The hub's forward chain.** A relayed packet comes in on the tunnel and leaves on it, so the hub's
forward hook judges it. The chain the controller renders (`AsNftables`) has a default of drop, accepts
established traffic, and accepts what did not arrive on an outward link or the tunnel. That last rule is
for the machine's own containers reaching outward. After that come the rules for this machine's own
published ports, each matching the **original destination port** of the connection. None of them names
an outgoing interface or a destination. So a relayed packet to another machine's port 20000 matched the
hub's own rule for its own port 20000 and passed. One to port 8080, which the hub does not publish,
matched nothing and was dropped.
**The fix.** One rule: in on the tunnel **and** out on the tunnel is accepted. That is the mesh passing
through to another of its machines, which filters it against its own rules. It does not widen anything
on the hub. A packet for the hub itself is the input chain's, and one for the hub's own containers
leaves by a bridge, not the tunnel. Both still meet their rules. WireGuard only accepts a packet from a
peer whose address that peer is allowed to use, so in-on-the-tunnel means from a machine of the mesh.
A controller test asserts the rule in the forward chain only, never in the input chain, and absent on a
machine with no tunnel. It fails without the fix, and the rendered set loads with `nft -c`.
@@ -0,0 +1,43 @@
---
status: resolved
opened: 2026-10-02
located-in: [mesh-host internal/outward (Links reported only the links carrying a default route)]
fixed-by: mesh-host PR 66 (a link backed by a physical device is named outward, up or down), live 2026-10-02
amended-design: []
---
# 197 — A physical link that is down is not filtered when it comes up
## What was observed
A sweep of every machine's filter on 2026-10-02. A laptop-class machine connected by its radio has a
wired port that was unplugged. Its filter guarded the radio and the tunnel, and accepted everything
arriving on any other link:
```
iifname != { "mesh0", "<radio>" } accept
```
The wired port was not in the list. Plugged in, everything arriving on it would have been accepted,
every port of the machine open to whatever network the cable reached. That would last until the
machine reported again and was pushed a new filter.
## Why it matters
**The filter's one rule about links fails open.** [ADR 0140](../../02-DECISIONS/0140-the-filter-constrains-what-arrives-from-outside.md)
has the filter constrain what arrives from outside, and has the machine say which links face outside.
Everything not named is treated as the machine's own, its containers and bridges. So a link the machine
fails to name is not filtered at all. The host named only the links carrying a default route at the
moment it reported. A cable plugged in later is the ordinary case for a laptop. A second wired network
that never carries the default route, such as a direct link to a storage box, is never named at all.
## Open questions
- A virtual link that faces outside (a VPN client's interface, a USB tether that appears as a virtual
device) has no physical device behind it. It is named only while it carries the default route. Is
that enough?
## Resolved (2026-10-02)
Live on the affected machine after the host was delivered and one more push: its filter now guards the
radio, the tunnel and the unplugged wired port, before anything is plugged into it.
@@ -0,0 +1,14 @@
# Diagnosis
*2026-10-02.*
**Located in `mesh-host` `internal/outward`.** `Links` read the kernel's routing tables and returned the
interfaces carrying a default route. An unplugged port carries none, so it was never reported, and the
controller rendered the filter around the links it was given.
**The fix.** A link faces outside if it carries a default route **or** has a physical device behind it.
The kernel lists every interface under `/sys/class/net`, with a `device` entry for one backed by
hardware. A bridge, a veth, the tunnel and the loopback have none, so they stay the machine's own. The
wired port is now reported up or down, and the filter guards it before anything is plugged in. Tested
with a radio carrying the default route and an unplugged wired port beside a bridge, a veth, the docker
bridge, the tunnel and the loopback: the two physical links are reported, nothing else.
@@ -0,0 +1,71 @@
---
status: resolved
opened: 2026-10-02
located-in: [mesh-catalog modules/dnsmasq (listens on loopback and the machine's mesh address only), the home-server's DNS (a predecessor's dnsmasq configuration the mesh did not own), the home network's DHCP (hands out the home-server as every device's DNS)]
fixed-by: mesh-catalog PR 214 (dnsmasq listens on addresses from a setting; docker's file takes no settings), mesh-controller PR 210 (the settings verb), mesh-catalog PR 215 (unifi network DNS tools), live 2026-10-02
amended-design: []
---
# 198 — The home network's DNS server ran outside the mesh, and the mesh's filter closed it
## What was observed
Every phone on the home Wi-Fi had no internet, while a laptop on the same Wi-Fi did. The router's
DHCP hands every device the home-server's LAN address as its DNS server. The home-server's DNS daemon
was listening on that address, and every query to it timed out. The router itself answered the same
query at once. The laptop worked because it resolves through its own local resolver, not through the
server DHCP names.
## Why it happened
The DNS daemon on the home-server was not the mesh's. It ran under a configuration file a predecessor
generated, listening on loopback, the mesh address and the LAN address. The mesh's `dnsmasq` module was
assigned to the other three machines and not to this one, so no module on the home-server declared
port 53. Its filter opens only what a module declares, so DNS from the LAN was dropped. It started when
the home-server applied the filter this morning, after nine hours of applying nothing
([issue 194](../194-the-hosts-own-former-archive-stops-every-apply/00-report.md)).
Nothing said so. The daemon reported running, the filter applied cleanly, and the mesh had no record
that the home network depended on a service it did not know.
## Why it matters
**A service the mesh does not know is closed by the mesh's filter, by design, and nothing asks whether
something depends on it.** That is the right default for an unknown port. It is the wrong outcome for
the one service a whole network was told to use. The gap is that a machine can run something
important outside the mesh with nothing to show it.
**The mesh's `dnsmasq` could not have served the LAN either.** It listened on loopback and the mesh
address only. The reach of its DNS endpoints opens the filter, but the daemon would not have been
listening on the LAN address anyway.
## Open questions
- Should a machine report the listening services the mesh does not own, the way it reports the links
that face outside? This one would have been visible before the filter closed it.
- The LAN address the home-server answers on is now a setting, beside the reach that opens the filter.
Two statements that must agree. Should reach `public` on a DNS endpoint imply listening beyond the
mesh?
## Resolved (2026-10-02)
The home network was pointed at the gateway for DNS while the fix was built, which got the phones back
within minutes. Then:
- the mesh's `dnsmasq` takes the addresses it listens on beside the machine's from a setting, with
loopback as the mesh-wide default, so no other machine changed;
- the home-server's layer adds its LAN address, and its DNS endpoints' reach is `public`. The router
forwards no DNS, so that means the LAN;
- the module and its sibling `resolv-conf` were assigned to the home-server, replacing the
predecessor's daemon and configuration, which were kept aside;
- the home network was pointed back at the home-server, through a new `unifi` tool.
Checked live: from another machine on the LAN, public names and mesh names both resolve through the
home-server's LAN address, and the mesh and the machine itself resolve as before.
**One fault found on the way, and caught before it reached any machine.** A module's settings are
merged into every mergeable file the module owns. The first attempt therefore put the new setting into
docker's `daemon.json` as well as into dnsmasq's config, and dockerd refuses keys it does not know. The
plan showed it before any push. The change was reverted and redone with docker's file declared to take
no settings. The general fault, a module's settings reaching files they were not meant for, is still
there for any module with more than one file.