Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
17ca9a262b | ||
|
|
967c793eaa | ||
|
|
78351560f7 | ||
|
|
62cc2f89c7 | ||
|
|
426f741ad0 | ||
|
|
9bed54d3be | ||
|
|
413daf8ad5 | ||
|
|
5c993c09b7 | ||
|
|
7c3be48db2 | ||
|
|
b665d06701 | ||
|
|
1bd13446d4 | ||
|
|
14adaafa53 | ||
|
|
bd673cc6ec | ||
|
|
bd6c55d225 | ||
|
|
2bbbc56502 | ||
|
|
c8935aceca | ||
|
|
29b656f2c0 | ||
|
|
d05ac367f1 | ||
|
|
1c0dafb918 | ||
|
|
018ee359ae | ||
|
|
c5535eeebd | ||
|
|
dbb9d2bc16 | ||
|
|
28d53dcc28 | ||
|
|
df667eb710 | ||
|
|
098a2ca485 | ||
|
|
a34cedeb5d | ||
|
|
db5ff5a5ee | ||
|
|
780c2b6e58 | ||
|
|
27c1db8a86 | ||
|
|
3d54fcbb86 | ||
|
|
c4151e6bc4 | ||
|
|
8c231102f8 | ||
|
|
f1941304cc |
@@ -124,6 +124,32 @@ This corrects a fact, not the decision: one statement per endpoint, three things
|
||||
none of them deciding on its own, all stand. The table in the decision should be read with the filter
|
||||
column applying to an unrouted endpoint.
|
||||
|
||||
## Progressive insight — 2026-10-02, from issue 191
|
||||
|
||||
**For a routed endpoint, "the proxy serves the internal name" has to mean "serves it to the private
|
||||
network", and only the proxy can make it mean that.** The decision says `internal` means the proxy
|
||||
serves the internal name and not the public one. It does not say to whom, and the proxy answered
|
||||
every name it routes to any request that carried it, on the same listeners as its public names. A
|
||||
name being internal kept nobody out: a request from the internet only had to send it. While every
|
||||
routed endpoint also had a public name, nothing showed it. Once an endpoint could be internal alone
|
||||
([issue 191](../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md)), serving
|
||||
its name to everyone would have published exactly what `internal` was chosen to keep private.
|
||||
|
||||
The earlier insight above says the port is not the path for a routed endpoint. This is its other
|
||||
half: the proxy is the path, so the proxy is where `internal` is enforced. It serves an internal name
|
||||
only to the machines of the mesh and to the machine itself
|
||||
([ADR 0144](0144-anything-on-a-machine-may-call-anything-on-it.md)). Who the mesh is, it is told,
|
||||
not left to work out: its membership carries the same list of machine addresses the filter's "from the
|
||||
mesh" is rendered from ([ADR 0167](0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)).
|
||||
To anyone else, the name is answered as one never routed, in the handshake and in the request, and not
|
||||
listed among the names it serves. This holds for the internal name of a `both` endpoint too, whose
|
||||
outsiders have its public name.
|
||||
|
||||
The decision, the options and the consequences stand: one statement per endpoint, three things
|
||||
derived from it. Checked in the proxy's own tests: an internal-only name is served to a machine the
|
||||
membership names and to loopback, and refused, unlisted and uncertified for any other request; until
|
||||
the mesh is issued, it is served to the machine alone.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **A manifest gains endpoint names, and a route contribution names an endpoint instead of a port.**
|
||||
|
||||
@@ -131,6 +131,32 @@ the plan says it too.
|
||||
| Genesis raises the forge as its module declares it | a genesis test that raises, assigns, and finds the module holding rather than raising a second |
|
||||
| Live | the next cutover on an adopted machine: `take` shows the comparison, refuses the downgrade if there is one, and the service keeps its configuration and its secret |
|
||||
|
||||
## Built, 2026-10-02
|
||||
|
||||
> **Progressive insight — 2026-10-02.** The decision stands; these are the facts of its building.
|
||||
|
||||
Built across mesh-host 63 and 64 and mesh-controller 201, 202 and the pull request that followed
|
||||
them. Rule 1: `take` previews every held thing's comparison and ends with a digest; `take --yes
|
||||
<digest>` acts on exactly that preview, and a changed preview or an account older than the flip
|
||||
allows is refused, as the flip's are. A published port's reach is said as the machine reported it,
|
||||
behind the found firewall whose rules are not read. Rule 2: an older image, a differing file and a
|
||||
minted, unaccepted secret for found data refuse, overridden by `--downgrade`, `--replace <path>` and
|
||||
`--mint <name>`; the secrets a module holds on a machine are read with where each came from. Rule 3:
|
||||
`secret accept --provider` reaches a required secret. Rule 4: the per-machine setting is `networks`,
|
||||
a container id to the found networks it keeps; judged for an adopted machine only, joined by the host
|
||||
after the container runs, part of the container's spec, named in the preview. Rule 5: the host's
|
||||
facts, former targets and strays. Rule 6: one judgement, run where a setting is stored and where a
|
||||
machine is composed; a module whose stored setting its definition can no longer compose is left out
|
||||
of the declaration, the envelope says so, the host keeps that module's things, and `plan` and `push`
|
||||
say it by name. A key that reaches nothing is refused where stored and said by `plan`, and never
|
||||
costs a module. Rule 7: genesis raises the forge under the module's container name, with its image
|
||||
digest and its data directory; the network is the one difference left, said by the take, because the
|
||||
bootstrap forge reaches the store on the machine's loopback.
|
||||
|
||||
**Not yet proven live.** Every machine of this mesh is converged, so the table's last row — a take
|
||||
on an adopted machine — waits for the next adoption. What is live is what the rows above it check.
|
||||
Issues 086, 098, 099, 100 and 101 stay located until that row is read.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md), [ADR 0103](0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md), [ADR 0104](0104-a-provision-may-be-answered-by-an-adapter-to-the-predecessor.md), [ADR 0162](0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md)
|
||||
|
||||
+146
@@ -0,0 +1,146 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: proposed
|
||||
date: 2026-10-01
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0046-a-module-configuration-is-its-assignments-not-its-manifest.md
|
||||
---
|
||||
|
||||
# 164. A setting is declared with its default, its meaning and what changing it costs
|
||||
|
||||
## Context
|
||||
|
||||
The operator asked for one thing for every module, with the container runtime as the first case: **one
|
||||
consistent default configuration for every machine, overridable per assignment, and easy to change
|
||||
later.** The four machines' runtime configurations were each written by hand and differ — one keeps
|
||||
running containers through a daemon restart and one does not, their log rotation differs, and each
|
||||
names its resolver and its trusted registries in its own words.
|
||||
|
||||
Most of this was already decided.
|
||||
[ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md) said the definition is
|
||||
identity and **defaults**, the assignment's settings are the configuration, *unset is the default*, and
|
||||
*an unknown setting is refused*. [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md) made
|
||||
a setting an operator requirement whose contract is "a type and, optionally, a default", answered by
|
||||
"the assignment's, or the requirement's default, or unresolved". An assignment is a module on a node,
|
||||
so the node layer of a module's settings already *is* the per-assignment override, and the mesh-wide
|
||||
layer is the one consistent default a person changes once.
|
||||
|
||||
What was built is narrower than what was decided, measured in the controller on the day of deciding:
|
||||
|
||||
- **Nothing declares which keys are settable.** A mergeable file's content is its defaults, and every
|
||||
key of it — and every key not in it — is accepted. Nothing tells a person, or the console, what can be
|
||||
set, of what type, or what it means.
|
||||
- **The refusal of an unknown setting is not there for most modules.** The stray-setting report returns
|
||||
nothing at all for a module with any mergeable file, because such a file "takes any key"
|
||||
([issue 173](../04-ISSUES/173-a-modules-settings-reach-every-fact-it-contributes/00-report.md) left
|
||||
files that way on purpose). It reports rather than refuses where it does run.
|
||||
- **A value in a file that is not JSON can have no default.** `${setting:<key>}`
|
||||
([ADR 0155](0155-a-definition-names-no-installation-and-how-that-is-checked.md)) is refused when no
|
||||
layer sets it — right for a mail domain, where a default is the very literal 0155 removes, and wrong
|
||||
for a tunable like the resolver's upstreams, which the resolver module therefore carries as literals
|
||||
in its file.
|
||||
- **A setting reaches every mergeable file its module owns.** The layers are one flat map per module,
|
||||
laid over each such file. Adding a setting to the resolver module for its own configuration put the
|
||||
key into the container runtime's file as well — the resolver writes into that file too — and the
|
||||
runtime refuses keys it does not know. The plan showed it before any push; the runtime's file was
|
||||
then made to take no settings at all ([issue 198](../04-ISSUES/198-the-lans-dns-server-ran-outside-the-mesh-and-its-filter-closed-it/00-report.md)). Issue 173 stopped settings leaking into
|
||||
contributions and served facts; between one module's own files the leak remains.
|
||||
- **What a change costs is said per file, not per key.** A service names the files it is reloaded or
|
||||
restarted on. The runtime re-reads its trusted registries on a reload and its `dns` key only when it
|
||||
starts; the resolver module declared a reload, so on two machines the key was written, reloaded,
|
||||
and never read, and every container got a public resolver for weeks while everything read as
|
||||
current ([issue 110](../04-ISSUES/110-a-container-on-the-runtimes-own-network-cannot-reach-the-resolver/01-resolution.md)).
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Leave settings implicit; document each module's keys in its README.** Rejected: a key the mesh
|
||||
does not know cannot be refused, typed, listed by the console or costed, and a README is a rule
|
||||
enforced by nothing.
|
||||
2. **A second mechanism for tunables beside settings** — defaults in a new block, settings untouched.
|
||||
Rejected: two ways to state one person's value, and design 27 already retires six mechanisms
|
||||
that grew that way.
|
||||
3. **Settings declared in the definition, as 0112's operator requirement: a key, a type, a meaning,
|
||||
optionally a default, and what a change costs.** Adopted.
|
||||
|
||||
## Decision
|
||||
|
||||
**A module declares every setting it takes.** Each declared setting has a name, a type, one sentence
|
||||
of meaning, optionally a default, and what a change to it costs. The spelling is design 27's to settle
|
||||
with the rest of the requirement form; this record decides the content.
|
||||
|
||||
**A setting with a default is a tunable; a setting without one is the operator's.** A tunable resolves
|
||||
to its default when no layer sets it — wherever it is read, a mergeable file or `${setting:<key>}` in a
|
||||
file of any format. A setting with no default is refused by name when nothing sets it, as 0155 decided;
|
||||
0155's refusal is narrowed to exactly that case, not changed for it. Whether a value has a default is a
|
||||
fact about the software (a log size does, a mail domain does not), and the definition states it once.
|
||||
|
||||
**The layers stay as they are, and every value says where it came from.** The definition's default,
|
||||
then the mesh-wide layer, then the node's — later wins, objects merge, lists replace. One consistent
|
||||
configuration for every machine is the default plus the mesh-wide layer; one machine that differs says
|
||||
so in its own layer and nothing else. Asked for a module's configuration on a machine, the mesh lists
|
||||
every declared setting with its effective value and its source: *default*, *mesh*, or *node*.
|
||||
|
||||
**Changing later is changing one of three places, and the plan shows its reach before anything moves.**
|
||||
A new default ships with the module's next version and reaches every assignment that does not override
|
||||
it; a mesh-wide setting reaches every assignment of the module; a node's reaches one. The plan of a
|
||||
change names each assignment whose effective value moves.
|
||||
|
||||
**A declared setting says where it lands.** Each names the file or files of its module that read it,
|
||||
and reaches no other: a module that owns two mergeable files no longer has one flat map laid over both.
|
||||
A file that names no setting takes none.
|
||||
|
||||
**A declared setting is the only kind accepted.** Setting a key the module does not declare is refused
|
||||
when it is set, naming the declared keys, rather than reported when the machine is planned. The mesh's
|
||||
own words — where a port, a directory or an operator's data is placed, how far an endpoint reaches —
|
||||
are the mesh's to validate as they are today, and no module declares them. A module
|
||||
that declares no settings keeps today's behaviour until it does; a catalogue test lists those modules,
|
||||
and the list shrinks to empty before the implicit form is removed — design 27's rule for every retired
|
||||
mechanism.
|
||||
|
||||
**A setting says what it costs: nothing, a reload, or a restart.** When a file changes, the host
|
||||
applies the strongest cost among the settings whose values moved in it, so a key the software reads
|
||||
only at start can no longer be written and never read. A setting that reaches a container's environment
|
||||
costs that container being recreated, which the host already does when a container's specification
|
||||
changes; it needs no declaration. A service's `reload-on` and `restart-on` keep
|
||||
naming the files that are not settings — a generated roster, a credential.
|
||||
|
||||
**The container runtime is the first module to declare its settings** and the model for the rest:
|
||||
its log rotation, keeping containers through a daemon restart, and its resolver are tunables, and
|
||||
its trusted registries are what the mesh tells it.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The console can show a module's settings as a form: what can be set, of what type, its default,
|
||||
and where the current value came from. That is the surface the operator wants for changing a
|
||||
default later.
|
||||
- `settings set` can refuse an unknown key, so ADR 0046's rule is enforced where it was only stated.
|
||||
- The resolver's upstreams, the runtime's log rotation, and other literals a definition carries
|
||||
because it could not give them a default become declared tunables.
|
||||
- **What got harder:** every module that takes settings must list them, and a mergeable file no
|
||||
longer silently accepts a key its author did not foresee. A person who needs one adds it to the
|
||||
definition, which is a new module version, not a setting.
|
||||
- Issue 173's open question — a consumer checks nothing against a contract — is unchanged; this record
|
||||
is the operator half of design 27's contract, not the provider half.
|
||||
- Not decided here: the spelling (design 27); whether a node's layer may be narrowed to a single key
|
||||
rather than replaced whole, as `settings set` does today.
|
||||
|
||||
## How this is checked
|
||||
|
||||
| Rule | Checked by |
|
||||
|---|---|
|
||||
| Every setting a module takes is declared | A parser test refusing a setting declaration without a type or meaning; a catalogue test listing modules with mergeable files or `${setting:}` and no declarations, which must be empty before the implicit form is removed |
|
||||
| A tunable resolves to its default; an operator value without one is refused | Resolution tests: an unset tunable in a JSON file and in a text file both take the default; an unset setting with no default is refused naming it (0155's existing test) |
|
||||
| A setting reaches only the files it names | A resolution test: a module with two mergeable files and a setting declared for one; the other file's content is unchanged by it (the case of issue 198) |
|
||||
| An undeclared key is refused when set | A controller test: `settings set` with an undeclared key fails naming the declared keys, and nothing is stored |
|
||||
| Every effective value names its source | A test listing a module's configuration on a node with one key from each of default, mesh and node |
|
||||
| A change's reach is shown before it moves | A plan test: changing a mesh-wide setting names every assignment whose effective value moves and no other |
|
||||
| The strongest cost applies | A host test: a file where a reload-cost key and a restart-cost key both moved restarts; a file where only reload-cost keys moved reloads |
|
||||
| Live | The container runtime's module lists its settings with their sources on every machine, and a mesh-wide change to its log rotation reaches all four at the next push |
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md), [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0155](0155-a-definition-names-no-installation-and-how-that-is-checked.md), [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md)
|
||||
- [Design 27 — a module requires, the mesh resolves](../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md)
|
||||
- Issues [110](../04-ISSUES/110-a-container-on-the-runtimes-own-network-cannot-reach-the-resolver/00-report.md), [173](../04-ISSUES/173-a-modules-settings-reach-every-fact-it-contributes/00-report.md)
|
||||
- mesh-controller `internal/catalogue/settings.go` (`settle`, `UnusedSettings`), `internal/catalogue/setting_into.go`
|
||||
+105
@@ -0,0 +1,105 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: proposed
|
||||
date: 2026-10-01
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0161-what-deserves-a-seat.md
|
||||
---
|
||||
|
||||
# 165. `container-runtime` is what a machine can run; that a runtime is running is its holder's health
|
||||
|
||||
## Context
|
||||
|
||||
A capability is a requirement a module places on a machine, detected by the host and renewed with
|
||||
every report ([ADR 0161](0161-what-deserves-a-seat.md)). The host's `container-runtime` asks the
|
||||
daemon for its version: *a running daemon, not an installed client*. It was made that way by
|
||||
[issue 007](../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md), where an
|
||||
installed package was believed to be a working service, and
|
||||
[design 05](../03-DESIGN/01-to-be/05-the-node-host.md)'s table says the same: *a runtime is
|
||||
running*. The installer's preflight borrows the same detector to wait for the runtime the
|
||||
foundation bundle installs, so there is one answer to "is there a runtime here".
|
||||
|
||||
The mesh is now to have a module for the runtime itself — its packages, its configuration, its
|
||||
service — on every machine ([ADR 0166](0166-the-container-runtime-is-a-node-seat-and-the-host-creates-containers-through-its-holder.md)).
|
||||
That module cannot declare `container-runtime` as defined: it would require the very thing it
|
||||
installs, the cycle [research 011](../01-RESEARCH/011-the-module-graph/cases.md)'s case 12 names
|
||||
("something the mesh installs that then becomes a node capability"). The operator defined the word
|
||||
for it: **`container-runtime` means the machine is able, at the kernel level, to install a runtime and
|
||||
execute containers** — not that one is installed, and not that one is running.
|
||||
|
||||
The host already draws this line once. `seat` is hardware, a display server *could* run here;
|
||||
`graphical-session` is state, one *is* running; the detector's own comment says "assignment needs the
|
||||
first". A machine without a display has no seat however much software is installed, and a machine
|
||||
with one has a seat before anything is.
|
||||
|
||||
Fifty-four catalogue modules declare `container-runtime` today, counted on the catalogue's main
|
||||
branch on the day of deciding: every module that delivers a container. Each relies on the current
|
||||
meaning to keep it off a machine with no running runtime.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Keep the meaning; let the runtime's module declare nothing.** Rejected: a module that installs
|
||||
the runtime has requirements on the machine — the kernel features without which installing it is
|
||||
pointless — and would state none of them. The cycle stays, only hidden.
|
||||
2. **Two capabilities, "can run" and "is running".** Rejected: the second is made true by assigning a
|
||||
module, so it is the module's state, not a fact of the machine; a capability the mesh itself
|
||||
flips by its own assignment is case 12's cycle with an extra name.
|
||||
3. **The capability is the kernel's; whether a runtime runs is the runtime module's health, and a
|
||||
module that delivers a container needs the runtime's seat held.** Adopted.
|
||||
|
||||
## Decision
|
||||
|
||||
**`container-runtime` is detected from what the kernel offers**, as `seat` is: the namespaces a
|
||||
container needs, a control-group hierarchy the runtime can manage, and an overlay filesystem the
|
||||
running kernel has or can load. Present when all three are; absent naming the missing one. Nothing is
|
||||
run and no runtime is asked. The verdict's detail names what was found, not a runtime's version.
|
||||
|
||||
**"A runtime is running and answers" is one probe, owned by the host and used twice:** by the
|
||||
installer's preflight, which waits for the runtime the foundation installs, and as the runtime
|
||||
module's health. It asks the daemon, as issue 007 requires. The preflight stops borrowing the
|
||||
capability's detector, and there is still one answer to "is a runtime running here".
|
||||
|
||||
**The runtime's module declares `container-runtime`**, with `package-manager`, `service-manager` and
|
||||
`privileged`, like any module that manages machine software.
|
||||
|
||||
**A module that delivers a container needs the runtime seat held on its machine**, and is refused
|
||||
otherwise, naming the seat and the modules that could hold it — the refusal design 27 already lists
|
||||
for an unheld seat. That requirement is derived from the container resource and needs no manifest
|
||||
field ([ADR 0166](0166-the-container-runtime-is-a-node-seat-and-the-host-creates-containers-through-its-holder.md)).
|
||||
The fifty-four existing declarations of the capability stay valid and become redundant; a catalogue
|
||||
test lists them, and they retire when the list is empty.
|
||||
|
||||
**The order is fixed, not preferred.** The detector changes only once the seat requirement is
|
||||
enforced. In between, a machine with the kernel and no running runtime would read as able to run
|
||||
every containerised module, which is issue 007 again.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Design 05's capability table changes its `container-runtime` row from *a runtime is running* to
|
||||
*the kernel can run containers*, and names the runtime module's health as where "running" is now
|
||||
asked.
|
||||
- The node listing stops showing the runtime's version beside the capability. The version moves to
|
||||
the runtime module's health and its seat's verbs.
|
||||
- A fresh machine with no runtime reads as able to run one, so it can be assigned the runtime's
|
||||
module, which is what makes the mesh able to install the runtime instead of the bootstrap alone.
|
||||
- **What got harder:** "is this machine running containers" is no longer one glance at the profile;
|
||||
it is the runtime seat's holder and its health. The node's listing should show both side by side.
|
||||
|
||||
## How this is checked
|
||||
|
||||
| Rule | Checked by |
|
||||
|---|---|
|
||||
| The capability is the kernel's | Host detector tests over a fixture `/proc` and `/sys`: all three present → present; each one missing → absent naming it; no runtime binary on the fixture machine changes nothing |
|
||||
| One probe asks whether a runtime runs | A host test that the preflight and the runtime module's health call the same probe, and that the probe fails against a stopped daemon with an installed client (issue 007's shape) |
|
||||
| A containerised module needs the runtime seat held | A resolution test: a module with a container resource on a machine whose runtime seat is unheld is refused, naming the seat and its candidate holders |
|
||||
| The order holds | The host release that changes the detector is gated on the controller release that enforces the seat requirement — stated in both changes' descriptions and checked at review |
|
||||
| Live | Every machine's profile shows `container-runtime` present with the kernel's features as its detail; a machine with no runtime installed reads present too |
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0161](0161-what-deserves-a-seat.md) — the profile renewed by every report; a capability that names a dialect
|
||||
- [ADR 0166](0166-the-container-runtime-is-a-node-seat-and-the-host-creates-containers-through-its-holder.md) — the seat and its holder
|
||||
- [Issue 007](../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md), [research 011](../01-RESEARCH/011-the-module-graph/cases.md) cases 12–13
|
||||
- [Design 05 — the node host](../03-DESIGN/01-to-be/05-the-node-host.md)
|
||||
- mesh-host `internal/profile/detectors.go` (the runtime detector), `internal/profile/seat.go` (the hardware/state split), `internal/bootstrap/preflight.go` (the preflight that borrows it)
|
||||
+161
@@ -0,0 +1,161 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: proposed
|
||||
date: 2026-10-01
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0161-what-deserves-a-seat.md
|
||||
---
|
||||
|
||||
# 166. The container runtime is a node seat, and the host creates containers through its holder
|
||||
|
||||
## Context
|
||||
|
||||
Every container the mesh runs on a machine is created by the host, which looks for a runtime
|
||||
(`docker info`, then `podman info`) and drives that runtime's command line itself: run, inspect,
|
||||
remove, exec. Research 012 called this "detected rather than declared": the host takes over whatever
|
||||
runtime it finds. Nothing in the mesh owns the runtime. Its package came from the foundation bundle
|
||||
or was already on the machine. Its configuration file was written by hand, differs on each of the
|
||||
four machines, and is also written into by two modules that are not the runtime's
|
||||
([issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md)).
|
||||
Its service is declared by those same two.
|
||||
|
||||
The operator set the direction:
|
||||
|
||||
- a module for the runtime, on every machine, owning "what is needed to run containers here": its
|
||||
packages, its configuration and its service;
|
||||
- that module holds a node seat for the runtime, so a second runtime (podman) can later compete
|
||||
for the seat;
|
||||
- the host stops speaking to the runtime directly and uses the seat's holder. The host stays the one
|
||||
that decides, and the holder becomes the one that executes;
|
||||
- every container on the machine is in scope, not only the mesh's. A development environment started
|
||||
by hand, or a test database a tool runs, is legitimate. The host already calls these *strays*: 3,
|
||||
8 and 25 on three of the machines on the day of deciding;
|
||||
- the runtime's events and verbs are subjects on the bus, and the mesh's own interface is built on
|
||||
them. The third-party interface run until now was removed by hand.
|
||||
|
||||
[ADR 0161](0161-what-deserves-a-seat.md)'s test for a seat is whether the mesh's own code finds it by
|
||||
name. Here it does: the host would look up the holder on its own machine. A singular role of a module
|
||||
held once per machine is a `node-*` seat ([ADR 0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md)),
|
||||
in the controller's seed.
|
||||
|
||||
The constraint that decides most of this record is a cycle. The bus runs in containers. On the
|
||||
broker's machine, the broker's own container is created by the host. A holder's code served from a
|
||||
container cannot create the container that runs it. On a first machine, before the controller exists,
|
||||
nothing holds anything.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **The host calls the holder's verbs over the bus.** Rejected: with the broker down, no machine can
|
||||
create any container, including the broker's. The mesh would be unable to restart its own
|
||||
transport.
|
||||
2. **The holder picks a dialect that the host speaks itself, as with the uplink.** Rejected: the
|
||||
host would still drive the runtime, and the module would drive it too for every other caller.
|
||||
That is two programs speaking to one daemon, and they come to disagree about the same machine
|
||||
(the installer's preflight already exists to avoid this).
|
||||
3. **The holder's code runs as a supervised process on the machine and serves the seat's verbs
|
||||
twice: locally to the host, on the bus to everyone else.** Adopted.
|
||||
|
||||
## Decision
|
||||
|
||||
**`node-container-runtime` is a seat of the mesh's own, node-scoped,** in the controller's seed under
|
||||
this record. It delivers no provision; what it carries is its role's protocol: verbs its holder must
|
||||
serve ([ADR 0132](0132-a-seat-carries-the-tools-its-holder-must-serve.md)) and events its holder emits
|
||||
([ADR 0129](0129-a-seat-carries-the-protocol-of-its-role.md)). The runtime's module, `docker`, claims
|
||||
it and is assigned to every machine. A podman module may claim it later; one machine runs one.
|
||||
|
||||
**The seat's verbs cover every container on the machine:** list, inspect, logs, stats, start, stop,
|
||||
restart, create and remove. A mesh-held container is marked by the host's label and says which
|
||||
assignment holds it. **A container the runtime runs can be root on the machine** — privileged, a host
|
||||
path mounted, the host's network or process namespace, the runtime's own socket — so a caller other
|
||||
than the host may not create one that is any of these; only a declaration the mesh composed may ask
|
||||
for them. And the verbs that change anything are granted by name, never by a wildcard: a grant of
|
||||
every tool (the console's today) reaches the reading verbs only. Issue 193 is what a verb that trusts
|
||||
its caller costs. **Creating or removing a mesh-held container is the host's alone.** Any other
|
||||
caller is refused naming the assignment, because the host would undo it at its next apply. Starting,
|
||||
stopping or restarting one is allowed, and the answer says the host will restore what its
|
||||
declaration says. A container the mesh does not hold is the caller's to do anything with.
|
||||
|
||||
**The seat's events are the runtime's own** — a container created, started, stopped, died, removed,
|
||||
its health changed. They are emitted on the seat's subjects, so every holder emits the same events and
|
||||
no reader depends on which runtime holds the seat. As
|
||||
[ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)
|
||||
decides, the subjects are issued by the controller, not composed by the module.
|
||||
|
||||
**The holder's code is a supervised process, not a container** ([ADR 0150](0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md)).
|
||||
A runtime cannot be run by the thing it runs. The process serves the seat's verbs on the bus to the
|
||||
console, to tools and to the mesh's interface. The same verbs are served on a local socket on the
|
||||
machine, which only the host may use. **The host creates, inspects and removes its containers
|
||||
through that socket and nothing else.** If the holder does not answer, the host creates nothing. It
|
||||
says so in its report, naming the seat. It never falls back to the command line.
|
||||
|
||||
**A container needs the seat held on its machine.** An assignment that delivers a container on a
|
||||
machine whose runtime seat is unheld is refused, naming the seat and its candidates
|
||||
([ADR 0165](0165-container-runtime-is-what-a-machine-can-run-and-a-running-runtime-is-its-holders-health.md)).
|
||||
Mounting the runtime's socket into a container is granted by the seat, not by the capability. The
|
||||
socket's path is the holder's to state, because podman's is not docker's.
|
||||
|
||||
**The runtime module owns the runtime's configuration.** Its settings are declared with defaults
|
||||
([ADR 0164](0164-a-setting-is-declared-with-its-default-its-meaning-and-what-changing-it-costs.md)):
|
||||
the resolver containers use, the registries it trusts, log rotation, and keeping containers through a
|
||||
daemon restart. The module is given the resolver's address and the mesh's registry as values; no other
|
||||
module writes the runtime's file or declares its service.
|
||||
|
||||
**The first machine is bootstrapped with the holder, and adopted afterwards.** The foundation bundle
|
||||
already installs the runtime's package and service. It also carries the holder's process, delivered as
|
||||
a binary the way the host is ([ADR 0142](0142-the-mesh-delivers-its-own-components-as-binaries.md)).
|
||||
When the runtime module is assigned, it adopts what the bundle made, as the store and broker modules
|
||||
adopt theirs ([ADR 0078](0078-the-store-and-broker-are-modules.md)).
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The migration on the running mesh has a fixed order:**
|
||||
1. Each machine's hand-written configuration is read, because the module's defaults replace what
|
||||
differs.
|
||||
2. In one push per machine: the resolver module and the private network stop writing the
|
||||
runtime's file ([issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md)),
|
||||
and the runtime module is assigned and adopts the runtime, its file and its service. Split in
|
||||
two, either the controller refuses two modules declaring one path, or a machine is left with
|
||||
nothing setting `dns` and `live-restore`.
|
||||
3. The controller seeds the seat and enforces the container requirement.
|
||||
4. The host releases the version that uses the holder.
|
||||
5. The host's command-line path is removed in the release after every machine's holder answers.
|
||||
Until then, the host reports per machine which path it used.
|
||||
- **Every container the host makes depends on the holder's process.** A crash-looping holder stops
|
||||
new containers on its machine. Running containers are unaffected. The host's report names the cause.
|
||||
- The process form of a module's own code must serve tools on the live mesh before this ships. Only the
|
||||
showcase declares it, and [issue 117](../04-ISSUES/117-a-modules-own-code-is-a-container-and-a-process/01-diagnosis.md)
|
||||
found the showcase's tools declared in a form nothing runs. The runtime module is the first whose
|
||||
tools cannot fall back to a container.
|
||||
- A user interface subscribing to events directly does not exist. Today a reader of events is a module
|
||||
that consumes them. The mesh's container view is a module, or waits for that path.
|
||||
- [ADR 0005](0005-the-node-host.md) ("a container runtime is detected, not chosen") and
|
||||
[ADR 0006](0006-the-substrate-and-the-control-plane.md)'s matching line describe the mechanism this replaces: on
|
||||
acceptance, each gets a dated note saying the runtime is now a seat's holder, as the decision
|
||||
records' rule for a moved mechanism requires. Design 05 and design 26 are amended after acceptance.
|
||||
- The operator's decision to remove the third-party interface by hand needs no mechanism. No
|
||||
module-retires-module rule is introduced.
|
||||
- **What got harder:** the host gains a dependency it did not have, and a first machine's bundle gains
|
||||
a component. The direct path was simpler and is what makes a runtime a black box to the rest of the
|
||||
mesh.
|
||||
|
||||
## How this is checked
|
||||
|
||||
| Rule | Checked by |
|
||||
|---|---|
|
||||
| The seat is the mesh's own, node-scoped, with its verbs and events | A catalogue test on the default seats; registration refuses a claimant that does not serve every verb (design 33's existing check) |
|
||||
| Creating or removing a mesh-held container is the host's alone | A test of the runtime module's verbs: create or remove of a container carrying the host's label, from any caller but the host's socket, is refused naming the assignment; the same verbs on an unlabelled container succeed |
|
||||
| No caller but the host creates a container that is root on the machine | A test of `create` from the bus: privileged, a host path, the host's namespaces and the runtime's socket are each refused; the same request on the host's socket is accepted. A broker test: a grant of every tool does not reach a changing verb |
|
||||
| The host uses the holder and never the command line | A host test with a fake holder on the local socket: every container operation goes to it, and with the holder absent the apply creates nothing and reports the seat; after step 5, the host carries no command-line runtime code (checked by build: the package is gone) |
|
||||
| A container needs the seat held | A resolution test refusing a containerised assignment on a machine with the seat unheld, naming the seat |
|
||||
| Socket mounts are granted by the seat | A catalogue test: a module mounting the runtime's socket on a machine whose holder states a different path is refused |
|
||||
| No other module writes the runtime's file | The existing collision check, once the private network's computed resources are inside it ([issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md)) |
|
||||
| Live | `seats` lists `node-container-runtime` held on every machine; the node listing shows each machine's containers, strays included, from the seat's `list` verb; a container started by hand appears as an event on the bus |
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md), [ADR 0129](0129-a-seat-carries-the-protocol-of-its-role.md), [ADR 0132](0132-a-seat-carries-the-tools-its-holder-must-serve.md), [ADR 0159](0159-a-tool-call-names-the-machine-and-a-holder-serves-its-seats-verbs.md), [ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md), [ADR 0161](0161-what-deserves-a-seat.md)
|
||||
- [ADR 0150](0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md), [ADR 0142](0142-the-mesh-delivers-its-own-components-as-binaries.md), [ADR 0078](0078-the-store-and-broker-are-modules.md), [ADR 0005](0005-the-node-host.md)
|
||||
- [ADR 0164](0164-a-setting-is-declared-with-its-default-its-meaning-and-what-changing-it-costs.md), [ADR 0165](0165-container-runtime-is-what-a-machine-can-run-and-a-running-runtime-is-its-holders-health.md), [issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md)
|
||||
- [Design 26 — the seats](../03-DESIGN/01-to-be/26-the-seats.md), [design 33 — the tools the mesh answers](../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md)
|
||||
- mesh-host `internal/apply/apply.go` (the runtime lookup and the command line it drives)
|
||||
+99
@@ -0,0 +1,99 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-10-02
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md
|
||||
---
|
||||
|
||||
# 167. A membership carries what its module receives, and who the mesh is
|
||||
|
||||
## Context
|
||||
|
||||
A provider learns what it is given from a file. The controller composes every consumer's contribution
|
||||
to a requirement, and the node's declaration writes them into the provider's received file. The route
|
||||
proxy reads its routes that way: one JSON file, re-read every two seconds.
|
||||
|
||||
[Issue 191](../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md) showed what
|
||||
that file leaves out. Since [ADR 0138](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md),
|
||||
a route whose endpoint reaches only the private network carries an internal name and no public one.
|
||||
The proxy dropped it. Serving it was not enough either: the proxy answers public and internal names on
|
||||
the same listeners, so an internal name served to every request is public under a guessable name. To
|
||||
serve it correctly the proxy needs a second fact, **who the mesh is**, and nothing gave it one.
|
||||
|
||||
The first attempt had the proxy work it out: the mesh's range from an environment variable written by
|
||||
the catalogue, and the machine's container bridges read from its own interfaces. That is a second
|
||||
definition of "the mesh", kept by one module, beside the one the packet filter already uses. The
|
||||
controller resolves "from the mesh" to every machine's address on the private network, and the filter
|
||||
is rendered from that list. Two definitions agree until one changes.
|
||||
|
||||
[ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)
|
||||
already gives every assignment one document on the bus, its membership, read once at connect and
|
||||
followed. It says what the assignment serves and reaches. It does not yet say what it is given.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Keep the file, add the mesh to it.** The proxy keeps polling a file, and the controller writes the
|
||||
mesh's addresses beside the routes. It fixes the definition, but delivery stays a file re-read on a
|
||||
timer, written by a separate path from the one every other fact a module is told now takes.
|
||||
2. **Have the proxy work it out** from a range in its environment and the machine's interfaces. Rejected:
|
||||
it is the second definition this record exists to remove.
|
||||
3. **The membership carries it.** What each module receives, from the same composition its received
|
||||
file is written from, and the mesh's addresses, from the same list the filter is rendered from. The
|
||||
proxy follows its membership and serves exactly that.
|
||||
|
||||
## Decision
|
||||
|
||||
**Option 3.**
|
||||
|
||||
- **A membership carries what its module receives**, by requirement: the contributions every consumer
|
||||
made, exactly as composed for its received file. A requirement nobody contributed to is an empty
|
||||
list, never absent, for the reason the file is written empty: "nothing asked" and "never told" want
|
||||
different responses.
|
||||
- **A membership carries who the mesh is**: every machine's address on the private network, the list
|
||||
a rule saying "from the mesh" resolves to. One list, two readers: the filter and any module that
|
||||
must tell the mesh from the world.
|
||||
- **The route proxy reads its routes and the mesh from its membership**, with the bus account every
|
||||
module that speaks on the bus is given. It serves an internal name only to the machines the mesh
|
||||
names and to the machine itself, and answers anyone else as it answers a name it never routed: in
|
||||
the request, in the handshake, and in the list of names it serves.
|
||||
- **The file stays until the bus has spoken.** While a proxy has read no membership that carries routes,
|
||||
it serves the file, and an internal name only to its own machine: refused, never opened. A
|
||||
membership from a controller that issues no routes changes nothing.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Every membership grows two fields. A machine joining or leaving republishes every membership, which
|
||||
a push already does.
|
||||
- A provider that receives something is told it twice for now, in its file and on the bus. The file
|
||||
goes when every provider reads its membership; that is its own change.
|
||||
- The route proxy needs a bus account. It is issued like any module's, so a machine running the proxy
|
||||
cannot be composed between the catalogue declaring the account and the operator issuing it. The
|
||||
machine keeps what it runs meanwhile.
|
||||
- The internal name of a route that also has a public one is now served to the mesh only. Outsiders
|
||||
have the public name.
|
||||
- A container on the same machine that calls that machine's own internal name arrives from its
|
||||
container network, not from a mesh address, and is refused. Calls between machines are unaffected:
|
||||
they leave by the machine's mesh address. Whether the mesh should also issue each machine's container
|
||||
networks is left open, because the mesh does not record them today.
|
||||
|
||||
## How this is checked
|
||||
|
||||
| Rule | Checked by |
|
||||
|---|---|
|
||||
| What a provider receives on the bus is what its received file says, same-node port fix included | a controller test composing a provider and a consumer on one machine and comparing the two |
|
||||
| An internal name is served to the machines the membership names and to loopback, and to nobody else | the proxy's tests: served from a named address and from loopback; refused, unlisted and uncertified from any other |
|
||||
| A membership that carries no routes, or a mesh that cannot be read, changes nothing | the proxy's tests |
|
||||
| Until the mesh is issued, an internal name is served to the machine alone | the proxy's tests |
|
||||
| Live: an internal-only route answers over the mesh and is refused from outside | by hand, after the release |
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md) —
|
||||
the membership this extends
|
||||
- [ADR 0138](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md) — reach, and the
|
||||
insight of 2026-10-02 that the proxy is where internal reach is kept
|
||||
- [ADR 0144](0144-anything-on-a-machine-may-call-anything-on-it.md) — the machine itself is always inside
|
||||
- [Issue 191](../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md) — what
|
||||
found it
|
||||
@@ -0,0 +1,120 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-10-02
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
---
|
||||
|
||||
# 168. A converged machine is filtered by the mesh alone, and the host says what else refuses
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md) says what converging does to
|
||||
the firewall a machine was found with: the mesh's derived filter is loaded in place of the
|
||||
refusal-only guard, and the found firewall is retired — disabled, never flushed. Four issues from
|
||||
the first two convergences are four ways that sentence was not the machine:
|
||||
|
||||
- the flip reported the found firewall retired and it was active two minutes later; fifty minutes
|
||||
on, a reconcile found it disabled by hand and recorded that the mesh had done it
|
||||
([143](../04-ISSUES/143-converging-does-not-retire-the-firewall-it-found/00-report.md));
|
||||
- "the firewall found" named one front end, and what filtered the forwarded path on that machine
|
||||
was a chain a predecessor had installed in the container runtime's user hook — invisible to the
|
||||
mesh, refusing two ports the mesh declared open, and when it was removed, carrying an allowance
|
||||
every module reaching another by the machine's own name had been relying on
|
||||
([144](../04-ISSUES/144-the-predecessors-rules-outlive-the-firewall-it-was-found-as/00-report.md),
|
||||
[145](../04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md));
|
||||
- the forward chain listed address ranges that followed neither the modules nor the machine
|
||||
([141](../04-ISSUES/141-the-forward-chain-does-not-follow-the-modules/00-report.md)), answered by
|
||||
[ADR 0140](0140-the-filter-constrains-what-arrives-from-outside.md) before this record;
|
||||
- the networking module wrote two machine-wide files whole, so taking it restarted every
|
||||
container ([084](../04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md)),
|
||||
answered by [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md) and the hosts
|
||||
file's marked region ([issue 128](../04-ISSUES/128-the-hosts-file-is-written-whole/00-report.md)).
|
||||
|
||||
Read on the four machines of this mesh on 2026-10-02, after every one had converged: on both
|
||||
machines that had a front end it is inactive, and the host's record says the mesh retired it on
|
||||
both — true of one, false of the other. On the home server the predecessor's chain is still in
|
||||
force on the forwarded path, in the legacy packet filter the mesh's reader of rules does not
|
||||
consult once a machine is converged, so that machine is filtered by two things and the mesh says
|
||||
one. The host's reader already knows how to tell a table that refuses traffic from the runtime's
|
||||
own plumbing and from a ban list; it is asked once, at adoption, and only to refuse a machine
|
||||
whose firewall nobody speaks. Nothing asks it afterwards, and nothing reports what it saw.
|
||||
|
||||
The group's exit is one sentence: *a converged machine has exactly one thing filtering it, and
|
||||
the mesh says truthfully which.* The first half the mesh can enforce only for what it owns; the
|
||||
second half it can always do, and it is the half that was missing.
|
||||
|
||||
## Decision
|
||||
|
||||
**1. Convergence is a state the host keeps, not a step it takes once.** Every apply of a converged
|
||||
declaration reads whether the found firewall is in force. Active — enabled again by a package, a
|
||||
boot, a hand — it is retired again and said. The record distinguishes *the mesh disabled it* from
|
||||
*it was found inactive*, and a reconcile that finds it inactive never records that the mesh did
|
||||
it. When the step is skipped because the apply had failures, the report says the found firewall
|
||||
was left in force and why; a step that does nothing is never silent.
|
||||
|
||||
**2. The host reports what filters the machine, with every apply, adopted or converged.** Every
|
||||
table of the packet filter, and every chain of the legacy filter, that refuses traffic — a drop or
|
||||
a reject, or a base chain whose policy drops — with an owner: the *mesh's*, the *found firewall's*,
|
||||
the *container runtime's own*, a *ban* (a refusal that names the sources it refuses, in a chain
|
||||
that accepts nothing), or *other*. The runtime's own is its plumbing — its chains, the forward
|
||||
policy it sets when it turns forwarding on, its guard against reaching a container's address from
|
||||
off its bridge. The user chain the runtime leaves for an administrator is not the runtime's:
|
||||
anything refusing in it is *other*, which is where both predecessors' chains lived. Each entry
|
||||
says in one line what it refuses. The mesh removes none of it: a rule it did not write is the
|
||||
operator's to remove, now that they can see it.
|
||||
|
||||
**3. The mesh says which.** `node show` lists the filters with their owners. `status` names every
|
||||
converged machine that something other than the mesh's table, the runtime's plumbing and a ban
|
||||
list filters, the way it names strays and untaken modules, and such a machine is not "all well".
|
||||
The converge preview lists the filters found and the fate of each: the found firewall retired, the
|
||||
runtime's and the bans left, *other* left and named — so a person knows before the flip that the
|
||||
machine will not be filtered by the mesh alone until they remove it, and what they would be
|
||||
removing. *A converged machine is filtered by the mesh alone* when its list holds nothing but the
|
||||
mesh's, the runtime's own and bans.
|
||||
|
||||
**4. Adoption's threshold does not move.** A machine whose front end nobody speaks is still refused
|
||||
adoption; a refusing rule in the runtime's user chain still does not refuse it — on both machines
|
||||
of this mesh it would have, and the migration would not have happened. It is reported instead,
|
||||
from the first report on.
|
||||
|
||||
**5. Two of the group's issues are settled by records already accepted.** The forward chain follows
|
||||
the machine's outward links and says nothing about networks ([ADR 0140](0140-the-filter-constrains-what-arrives-from-outside.md)),
|
||||
which answers 141 whole. The runtime's file is written into and reloaded, and the hosts file's
|
||||
region is the mesh's alone ([ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md),
|
||||
[issue 128](../04-ISSUES/128-the-hosts-file-is-written-whole/00-report.md)), which answers 084. One
|
||||
machine-wide file the mesh still writes whole is its own filter, at the path the distribution's
|
||||
packet filter reads; an operator's own rules at that path would be contested, and are held as
|
||||
found until the filter module is taken ([ADR 0163](0163-taking-a-module-over-is-a-comparison.md)).
|
||||
That is a difference a take shows, not a fault, and is decided when it bites.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The host's report grows by the filters it found and, for a converged machine, the state of its
|
||||
found firewall and who retired it; the controller keeps both on the node's record.
|
||||
- `retireFirewall` runs on every converged apply and can disable the found firewall more than
|
||||
once; the record's *disabled by the mesh* means exactly that.
|
||||
- The reader of rules gains an owner per table and chain; what it refuses adoption for does not
|
||||
change. A ban stays what it was: not a firewall.
|
||||
- Issues 143 and 144 close on rules 1 to 3 once a machine's record names the predecessor's chain;
|
||||
141 closes on ADR 0140 and 084 on ADR 0102, both by reading.
|
||||
- Removing what is reported is the operator's act, by hand, with the preview's words in front of
|
||||
them. The mesh never flushes and never deletes a rule it did not mark.
|
||||
|
||||
## How this is checked
|
||||
|
||||
| Rule | Checked by |
|
||||
|---|---|
|
||||
| Every refusing table and chain is classified, the user chain's refusals as *other* | host tests over rulesets captured from three machines of this mesh: a predecessor's chain in the legacy filter, a ban list and empty front-end chains beside the runtime's, a virtualisation host and an endpoint agent that refuse nothing |
|
||||
| The found firewall active again on a converged machine is retired again and said; found inactive is recorded as found, not done; a skipped step is said | host tests over a fake front end |
|
||||
| The report carries the filters and the found firewall's state for a converged machine | a host test reading the report |
|
||||
| `node show` lists filters with owners; `status` names a converged machine something else filters and is not well; the preview lists filters and fates | controller tests over a fixture report |
|
||||
| Live | the home server's record names the predecessor's chain in the runtime's user chain as *other*; `status` names the machine; after the operator removes the chain, the next report drops it and `status` is well |
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), [ADR 0103](0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md), [ADR 0140](0140-the-filter-constrains-what-arrives-from-outside.md), [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md), [ADR 0163](0163-taking-a-module-over-is-a-comparison.md)
|
||||
- [Design 08 — Connectivity](../03-DESIGN/01-to-be/08-connectivity.md), [Design 05 — The node host](../03-DESIGN/01-to-be/05-the-node-host.md)
|
||||
- Issues 084, 141, 143, 144, 145
|
||||
@@ -177,6 +177,8 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0161** — [What deserves a seat: a role of a module is a seat, a singular fact about machines is a placement with a capacity of one, and a holder's software is the machine's](0161-what-deserves-a-seat.md)
|
||||
- **0162** — [A merge produces a tiered plan the mesh keeps, and a module's dependencies are one relation in the catalogue](0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md)
|
||||
- **0163** — [Taking a module over is a comparison: what it compares, what it refuses, and what it carries](0163-taking-a-module-over-is-a-comparison.md)
|
||||
- **0167** — [A membership carries what its module receives, and who the mesh is](0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)
|
||||
- **0168** — [A converged machine is filtered by the mesh alone, and the host says what else refuses](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md)
|
||||
|
||||
### Its tiers, from the bottom up
|
||||
|
||||
@@ -267,6 +269,9 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0150** — [A module's own code runs as supervised processes under the module's one account](0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md)
|
||||
- **0152** — [The operator's surface is a module the mesh assigns: the console](0152-the-operators-surface-is-a-module-the-console.md)
|
||||
- **0155** — [A definition names no installation: how that is checked, and the three ways a value that did gets out](0155-a-definition-names-no-installation-and-how-that-is-checked.md)
|
||||
- **0164** — [A setting is declared with its default, its meaning and what changing it costs](0164-a-setting-is-declared-with-its-default-its-meaning-and-what-changing-it-costs.md) *(proposed)*
|
||||
- **0165** — [`container-runtime` is what a machine can run; that a runtime is running is its holder's health](0165-container-runtime-is-what-a-machine-can-run-and-a-running-runtime-is-its-holders-health.md) *(proposed)*
|
||||
- **0166** — [The container runtime is a node seat, and the host creates containers through its holder](0166-the-container-runtime-is-a-node-seat-and-the-host-creates-containers-through-its-holder.md) *(proposed)*
|
||||
|
||||
### How it is built
|
||||
|
||||
|
||||
@@ -2,8 +2,9 @@
|
||||
layer: to-be
|
||||
status: in-progress
|
||||
code: [mesh-host]
|
||||
updated: 2026-10-01
|
||||
updated: 2026-10-02
|
||||
decisions:
|
||||
- 02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md
|
||||
- 02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md
|
||||
- 02-DECISIONS/0141-the-host-delivers-its-own-successor.md
|
||||
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
@@ -163,6 +164,27 @@ a resource's former targets, removes a container or file it wrote under a name t
|
||||
longer names, never removes what was found, and reports what runs on the machine that it neither
|
||||
wrote nor holds. *How it is checked:* ADR 0163's table.
|
||||
|
||||
**What the host joins, keeps and raises for a take** — revision, 2026-10-02
|
||||
([ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 4, 6 and 7). A
|
||||
container may name networks it also joins once it runs — the found network a per-machine setting keeps
|
||||
for a taken container while a neighbour still resolves it there; joined after the run, part of the
|
||||
container's spec, refused when it cannot be joined. A declaration may name the modules the mesh left
|
||||
out of it because a stored setting cannot compose with the module's definition: the host keeps what it
|
||||
wrote and holds for a left-out module and says so, where absence used to read as removal. And genesis
|
||||
raises the bootstrap forge under the forge module's container name, with the module's image digest and
|
||||
its data directory, so the module holds it by the found rule; the network is the one difference a take
|
||||
has left to say. *How it is checked:* a host test joins a kept network and refuses one it cannot; a
|
||||
host test keeps a left-out module's record and hold and removes an absent module's; a bootstrap test
|
||||
holds the installer's constants to the module's manifest where the catalogue is checked out beside it.
|
||||
|
||||
**What filters the machine, and the found firewall kept retired** — revision, 2026-10-02
|
||||
([ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md)). The host
|
||||
reports, with every apply, every table and legacy chain that refuses traffic and whose it reads it as —
|
||||
the mesh's, the found firewall's, the container runtime's own, a ban, or other — and, converged, whether
|
||||
the firewall it was found with is in force and who retired it. It retires that firewall on every
|
||||
converged apply, not once, records *found inactive* apart from *disabled by the mesh*, and says when the
|
||||
step was skipped. *How it is checked:* ADR 0168's table.
|
||||
|
||||
**Found reaches every kind that can touch what the machine has**
|
||||
([ADR 0103](../../02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md)). For a module not yet taken, a directory present with no record
|
||||
keeps its mode and owner, a unit present with no record keeps its state and boot setting, a
|
||||
|
||||
@@ -7,8 +7,10 @@ code:
|
||||
- mesh-controller internal/identity/authority.go
|
||||
- mesh-host internal/identity/serving.go
|
||||
- mesh-host internal/apply (the service that reflects a rule set)
|
||||
updated: 2026-09-30
|
||||
updated: 2026-10-02
|
||||
decisions:
|
||||
- 02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md
|
||||
- 02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md
|
||||
- 02-DECISIONS/0148-the-meshs-names-are-resolved-not-copied-into-containers.md
|
||||
- 02-DECISIONS/0151-a-routes-internal-name-is-composed-under-the-node-that-serves-it.md
|
||||
- 02-DECISIONS/0147-a-module-anchors-the-meshs-authority.md
|
||||
@@ -709,6 +711,38 @@ needs no new filter; a declared port is reachable from off the private network a
|
||||
not; no address of a machine's own networks appears in a rendered filter, asserted on the text; and a
|
||||
machine reporting no outward link is refused in the control plane with its existing filter left alone.
|
||||
|
||||
### A converged machine is filtered by the mesh alone, and the host says what else refuses
|
||||
|
||||
*2026-10-02, [ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md),
|
||||
from [issues 143](../../04-ISSUES/143-converging-does-not-retire-the-firewall-it-found/00-report.md) and
|
||||
[144](../../04-ISSUES/144-the-predecessors-rules-outlive-the-firewall-it-was-found-as/00-report.md).*
|
||||
|
||||
Retiring the found firewall was a step the flip took once, and said it had taken whatever happened;
|
||||
on the first machine with one it did not take, and a hand's work fifty minutes later was recorded as
|
||||
the mesh's. And "the firewall found" named one front end while a predecessor's chain in the container
|
||||
runtime's user hook — legacy iptables on one machine, invisible to a reader of nftables — filtered the
|
||||
forwarded path, refused ports the mesh declared open, and carried an allowance every module reaching
|
||||
another by the machine's own name relied on.
|
||||
|
||||
**Convergence is a state the host keeps.** Every converged apply reads whether the found firewall is in
|
||||
force; enabled again, it is retired again and said; the record says whether the mesh disabled it or
|
||||
found it inactive, and a skipped step is said. **The host reports what filters the machine**, every
|
||||
apply, adopted or converged: every table and legacy chain that refuses, with an owner — the mesh's,
|
||||
the found firewall's, the runtime's own plumbing, a ban, or *other*, which is where the runtime's user
|
||||
chain's refusals go. **The mesh says which:** `node show` lists them; `status` names a converged machine
|
||||
anything *other* filters and is not well; the converge preview lists what filters the machine and the
|
||||
fate of each — retired with the front end, left as the runtime's, left as a ban, or *left in force and
|
||||
not the mesh's*. The mesh removes none of it; adoption's threshold does not move.
|
||||
|
||||
*How it is checked:* host tests over rulesets captured from three machines of this mesh classify every
|
||||
refusing chain (a predecessor's chain in the legacy filter as *other*, a ban list reached through the
|
||||
user chain as a ban, a leftover front-end chain as *other*); a fake front end enabled again on a
|
||||
converged machine is retired again and said, found inactive is recorded as found; the report carries
|
||||
the filters and the found firewall's state and a change in them is worth an unasked report; controller
|
||||
tests over a fixture report check the recording, the preview's fates, the status JSON and the well
|
||||
predicate. Live: the home server's record names the predecessor's chain as *other* and `status`
|
||||
names the machine until the chain is removed by hand.
|
||||
|
||||
## 5 — Certificates
|
||||
|
||||
**Two authorities, kept separate on purpose.**
|
||||
@@ -842,6 +876,18 @@ One value, three readers:
|
||||
| `public` | the machine port, to anywhere | the public name | the public authority |
|
||||
| `both` | the machine port, to anywhere | both names | each name's own authority |
|
||||
|
||||
*2026-10-02.* **The proxy serves an internal name to the private network only**
|
||||
([ADR 0138](../../02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md),
|
||||
its insight of this date). It answers public and internal names on the same listeners, so the name a
|
||||
request carries is the request's own claim, not where the request came from. An internal name is
|
||||
served to the machines of the mesh and to the machine itself; to anyone else it is answered as a name
|
||||
never routed, in the handshake as well as the request. Without this, an endpoint with reach `internal`
|
||||
would be public under a name that is easy to guess
|
||||
([issue 191](../../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md)).
|
||||
The proxy is told who the mesh is, and its routes, in its membership on the bus — the same machine
|
||||
addresses the filter's "from the mesh" is rendered from
|
||||
([ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)).
|
||||
|
||||
**An endpoint that is not routed is reached and never named.** No route contribution means no name is
|
||||
composed and no certificate requested, while the filter still acts on it. That is the case the model
|
||||
could not express at all, and it is the ordinary case for anything that is not HTTP.
|
||||
|
||||
@@ -8,7 +8,7 @@ code:
|
||||
- mesh-host packaging/nox-mesh-host-network.sh
|
||||
- mesh-controller internal/token
|
||||
- mesh-controller internal/inventory/nodes.go
|
||||
updated: 2026-10-01
|
||||
updated: 2026-10-02
|
||||
decisions:
|
||||
- 02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md
|
||||
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
@@ -323,6 +323,21 @@ network are said. `take --yes <digest>` cuts over what was previewed, as the fli
|
||||
container may keep a found network by a per-machine setting while its neighbours are not yet taken.
|
||||
*How it is checked:* ADR 0163's table.
|
||||
|
||||
**A setting is judged where it is stored, and the take's words** — revision, 2026-10-02
|
||||
([ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1, 2, 4 and 6).
|
||||
The preview ends with a digest of what it said; `take --yes <digest>` acts on that preview and nothing
|
||||
else, and a preview that has changed since, or an account of the machine older than the flip allows, is
|
||||
refused as the flip's is. A module the machine holds nothing for has nothing to compare, and `--yes`
|
||||
suffices. The overrides are `--downgrade`, `--replace <path>` and `--mint <name>`; the per-machine
|
||||
setting that keeps a found network is `networks`, a container id to the networks it keeps, accepted
|
||||
for an adopted machine only. Storing a setting composes it against the module's current definition and
|
||||
refuses, naming node, module, layer and key, what cannot compose or reaches nothing. A definition that
|
||||
later moves under a stored setting costs that module its place in the machine's declaration, said by
|
||||
name in `plan`, `push` and the declaration itself, and the machine is told everything else; a stray
|
||||
setting no longer refuses the machine where it is read. *How it is checked:* controller tests over the
|
||||
one judgement — refused where stored, a module left out where composed, the envelope naming it — and
|
||||
over a take's digest, staleness and secrets.
|
||||
|
||||
A candidate machine is not empty. It has a package manager, probably a container runtime,
|
||||
configuration somebody chose. [ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
|
||||
says the host never touches what it did not create — adoption is the deliberate act of taking
|
||||
|
||||
@@ -7,8 +7,9 @@ code:
|
||||
- mesh-tools src/broker-amqp.ts (to be replaced)
|
||||
- mesh-catalog modules/nats (to be written)
|
||||
- mesh-sdk src (the protocol's NATS binding, step 3)
|
||||
updated: 2026-10-01
|
||||
updated: 2026-10-02
|
||||
decisions:
|
||||
- 02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md
|
||||
- 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md
|
||||
- 02-DECISIONS/0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md
|
||||
- 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
@@ -94,6 +95,12 @@ list; the account's grant is the same membership read the other way; the console
|
||||
tool's subject. The one rule a runtime keeps is the membership's own subject, from the two names in its
|
||||
credential.
|
||||
|
||||
**Revised 2026-10-02** ([ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)):
|
||||
a membership also carries **what its module receives**, by requirement — the contributions its
|
||||
received file is written from, from the same composition — and **who the mesh is**, every machine's
|
||||
address on the private network, the list the filter's "from the mesh" is rendered from. The route
|
||||
proxy is the first reader of both.
|
||||
|
||||
**Revised 2026-09-27** ([ADR 0129](../../02-DECISIONS/0129-a-seat-carries-the-protocol-of-its-role.md)):
|
||||
**`mesh.build.request`, `mesh.control.built` and the BUILDS stream are gone.** A build is work submitted to a role, and the
|
||||
mesh already has a shape for that — a seat's `accept` subjects, on a work queue with a queue group of
|
||||
|
||||
@@ -1,8 +1,11 @@
|
||||
---
|
||||
layer: to-be
|
||||
status: proposed
|
||||
code: []
|
||||
updated: 2026-09-27
|
||||
status: in-progress
|
||||
code:
|
||||
- mesh-controller internal/inventory
|
||||
- mesh-controller internal/catalogue
|
||||
- mesh-controller cmd/mesh-controller
|
||||
updated: 2026-10-01
|
||||
decisions:
|
||||
- 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md
|
||||
- 02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md
|
||||
@@ -14,10 +17,10 @@ decisions:
|
||||
# 29 — A node has operator accounts, and the mesh owns what lives under a home
|
||||
|
||||
**The mesh models machines but not the people on them.** A node record holds its name, its
|
||||
address, its mode — and nothing about *who a person is* on it: `jochens` on novox, `ace` on ace,
|
||||
`jochen` on shanks and g14. That username is not incidental. It decides who a file under `~` is
|
||||
owned by, who a user service runs as, and — the case that surfaced this — which account `ssh
|
||||
<node>` logs in as. The predecessor knew it (its per-node `user:`, and the modules that wrote a
|
||||
address, its mode — and nothing about *who a person is* on it: one login name on the build node,
|
||||
another on the home-server, a third on both workstations. That username is not incidental. It
|
||||
decides who a file under `~` is owned by, who a user service runs as, and — the case that surfaced
|
||||
this — which account `ssh <node>` logs in as. The predecessor knew it (its per-node `user:`, and the modules that wrote a
|
||||
person's `~/.ssh/config`, `~/.zshrc`, `~/.config`); the mesh, taking those over, kept the machine
|
||||
facts and dropped the human one.
|
||||
|
||||
@@ -28,8 +31,9 @@ Several things are missing, and they are one idea.
|
||||
A node has one or more **operator accounts**: the human logins on it. At minimum a name; the
|
||||
mesh already knows the node and its address, so `<account>@<node>` is then a complete answer to
|
||||
"who am I, where." It is the mesh's to hold because everything below is derived from it, and
|
||||
because it is exactly the fact that was silently lost — `ssh ace` failed to `ace` because nothing
|
||||
in the mesh said ace's account is `ace`.
|
||||
because it is exactly the fact that was silently lost — `ssh home-server` logged in under the
|
||||
workstation's own name, because nothing in the mesh said the home-server's account is a different
|
||||
one.
|
||||
|
||||
## 2. A resource may live under a home, owned by its account
|
||||
|
||||
@@ -65,14 +69,14 @@ create `~/.ssh` at `0700`, chown it to the account, and own the files it places
|
||||
|
||||
**The boundary — and it is the reason this is safe:** `~/.ssh` is the one directory where a wrong
|
||||
declaration locks a person out of their own machine. So the mesh's *found-vs-owned* semantics
|
||||
([ADR 0126](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md),
|
||||
([ADR 0118](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md),
|
||||
adoption) apply *inside* the home directory. The mesh **owns** the directory and the files above; it
|
||||
**holds as found — never rewrites, never removes** — the operator's own contents: their **private
|
||||
keys** and their **personal drop-ins** (`config.d/personal`, the personal `Host` aliases a
|
||||
workstation carries, exactly as `hosts.local` is the home the mesh never rewrites for `/etc/hosts`).
|
||||
Reconcile removing an unassigned `config.d/mesh` is fine; the same logic aimed at `id_ed25519` or an
|
||||
operator's own `authorized_keys` entry is a lockout. This is the login-channel cousin of the rule
|
||||
[ADR 0125](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md) draws for the uplink and the sshd
|
||||
[ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md) draws for the uplink and the sshd
|
||||
module draws for the firewall: **the mesh must never be able to arrange the one failure that severs
|
||||
its own way back in.** The carve-out is not a convenience; it is that rule, in `~/.ssh`.
|
||||
|
||||
@@ -108,12 +112,12 @@ found-vs-owned boundary of §3 is exactly what guarantees nothing already there
|
||||
|
||||
None of this needs a node to discover the mesh, and none of it needs a control-plane module of its
|
||||
own. The ssh files are **roster facts**
|
||||
([ADR 0128](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md)): once the
|
||||
([ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md)): once the
|
||||
roster view carries a node's **host key** and its **account** beside its name and address, the
|
||||
`ssh-client` module ships a template for `known_hosts`, `config` and `authorized_keys`, and the
|
||||
controller renders each node's copy from the full roster and pushes it. The mesh owns the data; the
|
||||
module owns ssh's format; the control plane gains no ssh syntax. It is the same act as composing a
|
||||
peer list or `/etc/hosts` — which is why there is **no novox-only "mesh-ssh" module**: the
|
||||
peer list or `/etc/hosts` — which is why there is **no control-node-only "mesh-ssh" module**: the
|
||||
centralization is the controller's composition, not a module that runs somewhere. Only non-secret
|
||||
facts travel (names, addresses, accounts, host keys, the CA public key); the private key stays the
|
||||
operator's, placed as an operator-owned file, referenced by path.
|
||||
@@ -129,6 +133,44 @@ operator's, placed as an operator-owned file, referenced by path.
|
||||
They meet at the account and the CA, not at a bespoke module. The `sshd` server side already exists;
|
||||
the client/identity side and the CA are the open pieces.
|
||||
|
||||
## What has shipped, and what has not
|
||||
|
||||
*Recorded 2026-10-01 from the controller's main branch, not from intent.*
|
||||
|
||||
**Built (mesh-controller, merged 2026-09-27):**
|
||||
|
||||
- **§1, the account as a node fact.** A node record carries an operator account and, optionally,
|
||||
its home. Empty is a real state — a freshly enrolled or headless machine has no operator account
|
||||
known yet — and an empty home means *derive it* (the superuser's home for the superuser, the
|
||||
conventional per-user home otherwise), so the common case needs no entry. The controller's node
|
||||
command sets it. One account per node is what exists; "one or several" below is still open.
|
||||
- **§2, resources under a home.** The account and its home are offered as machine facts, and a
|
||||
resource's *path and owner* resolve placeholders exactly as its content does — so a module places
|
||||
a file under a person's home, owned by that person, naming neither. A roster file may say it lives
|
||||
under the home: it is rendered per node, placed under that node's account's home, chowned to the
|
||||
account, and a node with no account gets none.
|
||||
- **§5, the composed ssh config.** The roster rendering carries each node's account, so the
|
||||
`ssh-client` template can emit a `Host` block per node with the right login name. Composed
|
||||
end-to-end in the controller's tests.
|
||||
|
||||
**Written but not shipped:** the `ssh-client` catalogue module itself exists on a branch of the
|
||||
module repository; its pull request was closed with a hold until this design is deployed, and
|
||||
nothing has deployed it since. The predecessor's generator still writes every workstation's ssh
|
||||
client blocks today — which is where [issue 172](../../04-ISSUES/172-the-ssh-client-block-matches-one-spelling-of-a-machine/00-report.md)
|
||||
was found.
|
||||
|
||||
**Not built:** the SSH CA and certificates (§4), `known_hosts` and `authorized_keys` as roster files,
|
||||
the found-vs-owned boundary inside `~/.ssh` (§3 — the controller has no rule yet that refuses to
|
||||
rewrite a private key), adoption of existing keys, the ssh-agent as a user service, and user-scoped
|
||||
services in general. The host vocabulary still has no user-scope unit at all; a workstation's
|
||||
per-user daemons (a bar watchdog, a config reloader, an audio service masked per user) have no form
|
||||
the mesh can send.
|
||||
|
||||
**A gap this surfaced:** §1 shipped as code before it had a decision record. The account as a node
|
||||
fact, the home as a placement root, and what the mesh may and may not do under a home are each a
|
||||
decision this document names but no record states. They are the next records to write, before the
|
||||
family of §2 modules is built.
|
||||
|
||||
## Why now, and why not yet
|
||||
|
||||
**Why it matters:** when HAL retires, the generators that keep `~/.ssh`, shell config and the
|
||||
@@ -137,7 +179,7 @@ alias and its trust, and a fresh machine has no operator dotfiles at all — the
|
||||
service and leave the human unable to work on the box.
|
||||
|
||||
**Why not build it reflexively:** it is a real addition to the node model, the resource model, and
|
||||
the seat set, and must be gotten right. The mechanism half is now settled — ADR 0128 is what lets
|
||||
the seat set, and must be gotten right. The mechanism half is now settled — ADR 0120 is what lets
|
||||
the ssh files be templates with no control-plane format — so what remains to decide here is the
|
||||
model:
|
||||
|
||||
@@ -156,15 +198,18 @@ model:
|
||||
unnecessary, and forwarding an agent into a node exposes the operator's keys to that node's root —
|
||||
so prefer certificates and `ProxyJump` over forwarding.
|
||||
|
||||
**Not urgent, not blocking.** ssh and dotfiles work today because HAL's generators still run as the
|
||||
substrate. This becomes load-bearing in the node-by-node retirement phase, not before — which is the
|
||||
right time to build it, once the account and CA model are decided here.
|
||||
**Now load-bearing.** The migration of every node to the mesh is complete; what remains of the
|
||||
predecessor is exactly the user environment this design covers — ssh config, dotfiles, the desktop
|
||||
stack and the per-user services of the two workstations. Those generators are the last thing
|
||||
keeping the predecessor running, so the model questions above are no longer deferred: the account
|
||||
record, the home as a placement root, user-scoped services and the one-off steps a hook used to run
|
||||
each need a decision before the modules that replace the generators can be written.
|
||||
|
||||
## References
|
||||
|
||||
- The gap was found generating `~/.ssh/config` from the *HAL* registry (`hal/terminal`'s
|
||||
postConfigure hook), which the nox mesh has no equivalent for.
|
||||
- [ADR 0128](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md) — the roster
|
||||
- [ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md) — the roster
|
||||
fact mechanism that renders the ssh files, format owned by the module.
|
||||
- [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) — the
|
||||
system-path placement this mirrors for home paths.
|
||||
@@ -173,6 +218,6 @@ right time to build it, once the account and CA model are decided here.
|
||||
- [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md) — the CA key is a secret the
|
||||
vault makes; [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md)
|
||||
— short-lived certs as rotation.
|
||||
- [ADR 0125](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md),
|
||||
[ADR 0126](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md) —
|
||||
- [ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md),
|
||||
[ADR 0118](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md) —
|
||||
the never-sever-the-channel rule and the found-vs-owned semantics, applied here to `~/.ssh`.
|
||||
|
||||
@@ -38,7 +38,7 @@ document is written and this one's status becomes `implemented`.
|
||||
| [`26-the-seats.md`](26-the-seats.md) | **Proposed.** What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) |
|
||||
| [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md), [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)) |
|
||||
| [`28-building-the-bus.md`](28-building-the-bus.md) | **Proposed.** The five steps of the bus work in the order their dependencies allow, each ending at a bed — with the surface measured, so no step's size is a guess | [ADR 0116](../../02-DECISIONS/0116-the-bus-is-built-in-five-steps.md), [ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md), [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md) |
|
||||
| [`29-a-node-has-operator-accounts.md`](29-a-node-has-operator-accounts.md) | **Proposed.** The mesh models machines but not the humans on them: a node gains operator accounts, and a resource may live under a home owned by its account — what would own ~/.ssh, dotfiles and ~/.config when HAL retires | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md) |
|
||||
| [`29-a-node-has-operator-accounts.md`](29-a-node-has-operator-accounts.md) | **In progress.** A node has an operator account and a resource may live under its home — built in the controller; the ssh-client module, the SSH CA, the `~/.ssh` boundary and user-scoped services are not. The account fact still wants its decision record | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md) |
|
||||
|
||||
| [`32-what-a-module-declares.md`](32-what-a-module-declares.md) | **Proposed.** What a module declares and what the bus derives from it: three namespaces, subjects from local names, queues never declared, the five relationships, and the build-publish-deploy lifecycle on one bus | [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md), [ADR 0127](../../02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md), superseded by [ADR 0131](../../02-DECISIONS/0131-everything-on-the-mesh-speaks-to-the-broker-seat.md) (superseding [ADR 0125](../../02-DECISIONS/0125-the-bus-is-the-only-broker.md)), [ADR 0041](../../02-DECISIONS/0041-events-are-a-relationship.md) |
|
||||
|
||||
|
||||
+11
-2
@@ -1,8 +1,8 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-22
|
||||
located-in: [mesh-controller internal/overlay, mesh-host internal/apply]
|
||||
fixed-by:
|
||||
fixed-by: ADR 0102 (mesh-controller internal/overlay: the runtime file written into, reloaded), issue 128 (the hosts file as a region)
|
||||
amended-design: 03-DESIGN/01-to-be/05-the-node-host.md
|
||||
---
|
||||
|
||||
@@ -56,3 +56,12 @@ and nothing checks for it today.
|
||||
- Should an adopted node that cannot trust the registry be refused a module that needs to pull?
|
||||
Or should the refusal come earlier, when the node joins?
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
The runtime's file is written into and the runtime reloaded, never restarted
|
||||
([ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md), the
|
||||
diagnosis above); the hosts file is a marked region the mesh owns alone
|
||||
([issue 128](../128-the-hosts-file-is-written-whole/00-report.md)). Neither whole file remains. Read
|
||||
into [ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md), rule 5,
|
||||
which names the one whole machine-wide file the mesh still writes — its own filter at the
|
||||
distribution's path — as a difference a take shows, not a fault.
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-22
|
||||
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
|
||||
fixed-by:
|
||||
fixed-by: mesh-controller 201 (the preview names the narrowing and the port's reach), 206 (`take --yes <digest>` acts on the preview read)
|
||||
amended-design:
|
||||
---
|
||||
|
||||
@@ -40,3 +40,13 @@ it changes before it changes it, and for taking a module this one does not.
|
||||
|
||||
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 2: the preview names a narrowing. Building follows,
|
||||
host first, then the controller's `take`.
|
||||
|
||||
## Built, 2026-10-02
|
||||
|
||||
mesh-controller 201 and the pull request after it: the preview names it, and `take --yes <digest>`
|
||||
acts on the preview that was read. Stays located until a take is read on an adopted machine — every
|
||||
machine of this mesh is converged today, so the record's live row has not been run.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
Closed on the operator's decision of 2026-10-02 with the built and tested code live on every machine (mesh-controller 206, mesh-host 64), not on a take read on an adopted machine: every machine of this mesh is converged, so none holds a found thing to compare, and the record's live row — ADR 0163's last — will be read at the next real adoption rather than staged. Said here so nobody later believes that row was run.
|
||||
|
||||
+16
-2
@@ -1,8 +1,8 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-22
|
||||
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
|
||||
fixed-by:
|
||||
fixed-by: mesh-host 64 (genesis raises the forge as `gitea`, on the module's image digest, with the module's data directory at /data; a test holds the installer to the module's manifest)
|
||||
amended-design:
|
||||
---
|
||||
|
||||
@@ -53,3 +53,17 @@ network, or it is not a takeover.
|
||||
|
||||
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 7: genesis raises as the module declares. Building follows,
|
||||
host first, then the controller's `take`.
|
||||
|
||||
## Built in part, 2026-10-02
|
||||
|
||||
mesh-host 64: genesis raises the forge under the module's container name (`gitea`), pinned to the
|
||||
module's image digest, with the module's data directory mounted at `/data` — so the module finds it,
|
||||
holds it, and a take compares equal images and the same data. A test holds the installer's constants
|
||||
to the module's manifest where the catalogue is checked out beside it. The network is the difference
|
||||
left: the bootstrap forge runs on the machine's network to reach the store on its loopback, the module
|
||||
runs bridged and publishes its ports, and the take says so. Closing waits for group 9's genesis test —
|
||||
a mesh raised, the module assigned, and the module found holding rather than raising a second forge.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
Closed on the operator's decision of 2026-10-02. Name, image and data directory align; the network does not — the bootstrap forge runs on the machine's network to reach the store on its loopback, the module runs bridged — and a take says so rather than hides it. Whether genesis should move the forge onto a bridge, and the test that raises a mesh and finds the module holding rather than raising a second forge, belong to group 9's genesis work and are not owed by this record any more.
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-23
|
||||
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
|
||||
fixed-by:
|
||||
fixed-by: mesh-controller (the pull request after 201: JudgeSettings, LeftOut), mesh-host 64 (left_out kept)
|
||||
amended-design:
|
||||
---
|
||||
|
||||
@@ -63,3 +63,12 @@ knowing the code.
|
||||
|
||||
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 6: judged where stored; an impossible statement costs a module. Building follows,
|
||||
host first, then the controller's `take`.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
One judgement, in the catalogue, run where a setting is stored and where a machine is composed. Stored,
|
||||
a setting that cannot compose with the module's current definition is refused naming the node, the
|
||||
module, the layer and the key; a key that reaches nothing is refused there too. Composed, a definition
|
||||
that moved under a stored setting leaves that module out of the machine's declaration — the envelope
|
||||
names it, the host keeps what it holds and wrote for it, `plan` and `push` say it — and the machine is
|
||||
told everything else. A stray setting no longer refuses the whole machine where it is read.
|
||||
|
||||
+10
-2
@@ -1,8 +1,8 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-23
|
||||
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
|
||||
fixed-by:
|
||||
fixed-by: mesh-host 63 (former targets removed, strays reported), mesh-controller 201/202 (strays shown)
|
||||
amended-design:
|
||||
---
|
||||
|
||||
@@ -80,3 +80,11 @@ found, and so would be kept for ever on purpose.
|
||||
|
||||
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 5: former targets are removed and strays reported. Building follows,
|
||||
host first, then the controller's `take`.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
mesh-host 63: the host's record keeps a resource's former targets, removes a container or file it
|
||||
wrote under a name the declaration no longer names, never what was found, and reports strays — what
|
||||
runs on the machine that the mesh neither wrote nor holds. mesh-controller 201 and 202 show strays
|
||||
on `node show` for an adopted and a converged machine alike; the live mesh reported four on the
|
||||
control node the evening it rolled.
|
||||
|
||||
+12
-2
@@ -1,8 +1,8 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-23
|
||||
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
|
||||
fixed-by:
|
||||
fixed-by: mesh-host 63 (the kept original's difference), mesh-controller 201 (shown; a differing file refuses unless `--replace <path>`)
|
||||
amended-design:
|
||||
---
|
||||
|
||||
@@ -68,3 +68,13 @@ written.
|
||||
|
||||
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 2: the difference is shown and a differing file refuses. Building follows,
|
||||
host first, then the controller's `take`.
|
||||
|
||||
## Built, 2026-10-02
|
||||
|
||||
mesh-host 63 reports the difference between the kept original and the declared content; mesh-controller
|
||||
201 shows it in the preview and refuses a differing file unless `--replace <path>` names it, or the
|
||||
module declares the file partially. Stays located until a take is read on an adopted machine.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
Closed on the operator's decision of 2026-10-02 with the built and tested code live on every machine (mesh-controller 206, mesh-host 64), not on a take read on an adopted machine: every machine of this mesh is converged, so none holds a found thing to compare, and the record's live row — ADR 0163's last — will be read at the next real adoption rather than staged. Said here so nobody later believes that row was run.
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-23
|
||||
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
|
||||
fixed-by:
|
||||
fixed-by: mesh-host 63 (both images' creation dates), mesh-controller 201 (DOWNGRADE said; refused unless `--downgrade`)
|
||||
amended-design:
|
||||
---
|
||||
|
||||
@@ -65,3 +65,13 @@ expected rate.
|
||||
|
||||
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 2: the images are compared by age and a downgrade refuses. Building follows,
|
||||
host first, then the controller's `take`.
|
||||
|
||||
## Built, 2026-10-02
|
||||
|
||||
mesh-host 63 reports the found image and both images' creation dates; mesh-controller 201 says
|
||||
DOWNGRADE and refuses unless `--downgrade` is said. Stays located until a take is read on an adopted
|
||||
machine.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
Closed on the operator's decision of 2026-10-02 with the built and tested code live on every machine (mesh-controller 206, mesh-host 64), not on a take read on an adopted machine: every machine of this mesh is converged, so none holds a found thing to compare, and the record's live row — ADR 0163's last — will be read at the next real adoption rather than staged. Said here so nobody later believes that row was run.
|
||||
|
||||
+14
-2
@@ -1,8 +1,8 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-23
|
||||
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
|
||||
fixed-by:
|
||||
fixed-by: mesh-controller 201 (`secret accept --provider` reaches a required secret), 206 (a module's secrets listed with origin; a minted one for found data refuses unless `--mint <name>`)
|
||||
amended-design:
|
||||
---
|
||||
|
||||
@@ -70,3 +70,15 @@ the module can only be installed fresh.
|
||||
|
||||
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 2 and 3: a minted secret for found data refuses; secret accept reaches required secrets. Building follows,
|
||||
host first, then the controller's `take`.
|
||||
|
||||
## Built, 2026-10-02
|
||||
|
||||
`secret accept <node> <module> <name> --provider <node>` reaches a required secret (mesh-controller 201).
|
||||
The pull request after it reads every secret a module holds on a machine with its origin, and a take
|
||||
of a module whose data was found refuses a minted, unaccepted one — naming the accept that carries
|
||||
the existing value in, or `--mint <name>` to let the service take the new one. Stays located until
|
||||
a take is read on an adopted machine.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
Closed on the operator's decision of 2026-10-02 with the built and tested code live on every machine (mesh-controller 206, mesh-host 64), not on a take read on an adopted machine: every machine of this mesh is converged, so none holds a found thing to compare, and the record's live row — ADR 0163's last — will be read at the next real adoption rather than staged. Said here so nobody later believes that row was run.
|
||||
|
||||
+13
-2
@@ -1,8 +1,8 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-23
|
||||
located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
|
||||
fixed-by:
|
||||
fixed-by: mesh-controller 201 (the neighbours on a found network are named), 206 (the per-machine `networks` setting), mesh-host 64 (the taken container joins the kept network)
|
||||
amended-design:
|
||||
---
|
||||
|
||||
@@ -66,3 +66,14 @@ exercise.
|
||||
|
||||
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 4: the neighbours are named; a found network may be kept by a setting. Building follows,
|
||||
host first, then the controller's `take`.
|
||||
|
||||
## Built, 2026-10-02
|
||||
|
||||
The preview names every neighbour on a found network (mesh-controller 201). The pull request after it
|
||||
adds the per-machine setting `networks` — a container id to the found networks it keeps — judged for an
|
||||
adopted machine only, and mesh-host 64 has the taken container join each once it runs. Stays located
|
||||
until a take is read on an adopted machine.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
Closed on the operator's decision of 2026-10-02 with the built and tested code live on every machine (mesh-controller 206, mesh-host 64), not on a take read on an adopted machine: every machine of this mesh is converged, so none holds a found thing to compare, and the record's live row — ADR 0163's last — will be read at the next real adoption rather than staged. Said here so nobody later believes that row was run.
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-26
|
||||
located-in: [mesh-host internal/apply]
|
||||
fixed-by: mesh-host 63 (every written field compared), mesh-controller 201 (build says the policy)
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 126 — a volume path is not in the spec comparison, and a roll-out raced a data move
|
||||
@@ -50,3 +52,9 @@ the install-page junk was discarded twice.
|
||||
|
||||
[ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 5 and 7: every field compared; build says the policy. Building follows,
|
||||
host first, then the controller's `take`.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
mesh-host 63: every field the host writes is compared before a container is called current, volumes
|
||||
and paths included. mesh-controller 201: `build` and the take-in say when a module's policy rolls a
|
||||
result out at once; under ADR 0162 the plan says it too.
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-28
|
||||
located-in:
|
||||
- mesh-controller internal/catalogue/filtering.go
|
||||
- mesh-host internal/apply
|
||||
fixed-by:
|
||||
fixed-by: ADR 0140 — mesh-controller (the filter around outward links; no network ranges anywhere)
|
||||
amended-design: 03-DESIGN/01-to-be/08-connectivity.md
|
||||
---
|
||||
|
||||
@@ -86,3 +86,11 @@ supersedes both 0137 and the first attempt at answering this.
|
||||
runtime, or left as the one constant?
|
||||
- Should the preview say which of a machine's networks are the mesh's and which are not, so a range
|
||||
that exists to protect a leftover is visible as such?
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
By [ADR 0140](../../02-DECISIONS/0140-the-filter-constrains-what-arrives-from-outside.md), built and
|
||||
live since 2026-09-29: the forward chain constrains what arrives on the machine's outward links and
|
||||
says nothing about networks, so there is no list to derive and nothing for a preview to tell apart.
|
||||
Read into [ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md),
|
||||
rule 5, which closes it.
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-29
|
||||
located-in:
|
||||
- mesh-host internal/apply/opening.go (retireFirewall)
|
||||
- mesh-host internal/apply/apply.go (the condition it is called under)
|
||||
fixed-by:
|
||||
fixed-by: mesh-host 67 (retire on every converged apply; found-inactive apart from disabled-by-mesh; a skipped step said), mesh-controller 211 (the found firewall's state on node show)
|
||||
amended-design:
|
||||
---
|
||||
|
||||
@@ -100,3 +100,21 @@ harmless, but the mesh's belief about which firewall is in force has been wrong
|
||||
so". Should it?
|
||||
- Why do the host's own detail lines not reach the journal? Everything it decided during the flip is
|
||||
unrecoverable, which is why this account has candidates instead of a cause.
|
||||
|
||||
## Decided, 2026-10-02
|
||||
|
||||
[ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md), rule 1:
|
||||
convergence is a state the host keeps — the found firewall active again is retired again and said, a
|
||||
reconcile that finds it inactive records *found so* and never *done by the mesh*, and a step skipped
|
||||
after a failed apply is said. Built in mesh-host on `feat/one-thing-filters-a-converged-machine`; the
|
||||
record of both machines of this mesh is corrected by the first report under it.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
mesh-host 67 and mesh-controller 211, live on every machine at 10:10Z. The step now runs on every
|
||||
converged apply and says what it did; a found firewall enabled again is retired again. The record's
|
||||
one inherited lie stands as history: on the control node the machine's own record already said the
|
||||
mesh had disabled the firewall, and the host trusts its record, so `node show` says "retired by the
|
||||
mesh" there. From this build on, a reconcile that finds the firewall inactive records *found inactive*
|
||||
and never the other thing. Whether the flip's step took on 2026-09-29 is not recoverable and is not
|
||||
owed by this record any more.
|
||||
|
||||
+23
-2
@@ -1,10 +1,10 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-29
|
||||
located-in:
|
||||
- mesh-host internal/apply/opening.go
|
||||
- mesh-controller cmd/mesh-controller (the converge preview)
|
||||
fixed-by:
|
||||
fixed-by: mesh-host 67 (every refusing table and legacy chain classified with an owner; the runtime's user chain is other), mesh-controller 211 (kept, shown on node show, named by status, previewed with fates)
|
||||
amended-design:
|
||||
---
|
||||
|
||||
@@ -82,3 +82,24 @@ everything reached from within.
|
||||
- Is the bus and the registry being reachable from anywhere still what the mesh wants on a machine that
|
||||
faces the internet? The design says yes, for enrolment. It deserves asking on its own rather than
|
||||
being answered by a leftover.
|
||||
|
||||
## Decided, 2026-10-02
|
||||
|
||||
[ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md), rules 2 and
|
||||
3: the host reports every table and legacy chain that refuses, with an owner, and the runtime's user
|
||||
chain's refusals as *other*; `node show`, `status` and the converge preview say it. Built on
|
||||
`feat/one-thing-filters-a-converged-machine` in mesh-host and mesh-controller. On 2026-10-02 the home
|
||||
server still carries the predecessor's chain in its legacy filter; the record's live row is reading it
|
||||
there.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
mesh-host 67 and mesh-controller 211, live at 10:10Z. The live row of
|
||||
[ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md) was read the
|
||||
same hour: the home server's record names the predecessor's chain in the legacy filter's user chain
|
||||
as *other*, with what it refuses, beside two chains a retired front end left in the IPv6 legacy filter;
|
||||
the control node's record names the same two leftovers; the laptop and the workstation read *the mesh
|
||||
alone*. `status` names both machines and is not well until the operator removes what the mesh did not
|
||||
write. The allowance the predecessor's chain carried is
|
||||
[issue 145](../145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md)'s,
|
||||
and that record is not closed by this one.
|
||||
|
||||
+85
@@ -0,0 +1,85 @@
|
||||
---
|
||||
status: located
|
||||
opened: 2026-10-01
|
||||
located-in: [mesh-catalog modules/dnsmasq, mesh-controller internal/overlay/generator.go, mesh-controller internal/catalogue/resolve.go (checkResources)]
|
||||
fixed-by:
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 190 — The container runtime's configuration is written by modules that are not the runtime's
|
||||
|
||||
## What was observed
|
||||
|
||||
The runtime's configuration file and its service are declared by two parties, neither of which is
|
||||
the runtime:
|
||||
|
||||
- **The resolver module** writes the runtime's `dns` key (the machine's private address) and
|
||||
`live-restore` into the runtime's file, written into rather than over
|
||||
([ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md)). It also
|
||||
declares the runtime's service, reloaded when that file changes. The `dns` key has been written
|
||||
since the resolver module was converted from its predecessor on 2026-09-23; `live-restore` and the
|
||||
service were added on 2026-09-30 while fixing
|
||||
[issue 110](../110-a-container-on-the-runtimes-own-network-cannot-reach-the-resolver/00-report.md),
|
||||
where containers silently resolved through a public resolver.
|
||||
- **The private network** writes the runtime's `insecure-registries` into the same file, and declares
|
||||
the same service reloaded on it, as [ADR 0082](../../02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md)
|
||||
and ADR 0102 decided. The controller generates both resources per machine.
|
||||
|
||||
On the three machines that run the resolver module, both declare one path and one unit. Nothing refuses
|
||||
it. The collision check compares the resources of catalogue modules. The private network is computed,
|
||||
so its resources are produced when a machine's declaration is composed, and the check never sees them.
|
||||
|
||||
The machine without the resolver module shows the other half. Its runtime still has the predecessor's
|
||||
resolver and `live-restore` off, because the only module that sets them is a DNS server. A machine
|
||||
gets a correct container runtime only as a side effect of being given a resolver.
|
||||
|
||||
> **Later the same day, 2026-10-02.** The resolver module and its sibling for the resolver file were
|
||||
> assigned to the fourth machine ([issue 198](../198-the-lans-dns-server-ran-outside-the-mesh-and-its-filter-closed-it/00-report.md)),
|
||||
> so all four now have the resolver writing into the runtime's file, and the predecessor's
|
||||
> `live-restore: false` there is gone. The same work made the runtime's file, as the resolver declares
|
||||
> it, take no settings: a setting meant for the resolver's own configuration had reached it. The
|
||||
> collision and the ownership question above are unchanged.
|
||||
|
||||
## Why this is here
|
||||
|
||||
The operator ruled it a defect, not a design: **a module does not write another software's
|
||||
configuration.** The need behind each write is real. Containers must resolve the mesh's names
|
||||
([ADR 0148](../../02-DECISIONS/0148-the-meshs-names-are-resolved-not-copied-into-containers.md) step 2).
|
||||
A daemon restart must not stop every container. Every machine on the network must trust the mesh's
|
||||
registry. But each of these is a fact the runtime must be *given*, and the module that gives it is the
|
||||
runtime's own. With three writers, nobody can say what the file should contain. Two of the facts are
|
||||
reloaded when one of them needs a restart (issue 110's first fault). And the moment a module for the
|
||||
runtime exists, it is refused on every machine with the resolver, or, through the private network's
|
||||
path, accepted without anyone noticing a collision.
|
||||
|
||||
## What resolves it
|
||||
|
||||
[ADR 0166](../../02-DECISIONS/0166-the-container-runtime-is-a-node-seat-and-the-host-creates-containers-through-its-holder.md)
|
||||
gives the runtime a module that holds its seat and owns its file and service.
|
||||
[ADR 0164](../../02-DECISIONS/0164-a-setting-is-declared-with-its-default-its-meaning-and-what-changing-it-costs.md)
|
||||
gives that module declared settings with defaults. The fix, once both are accepted:
|
||||
|
||||
1. The resolver module drops its runtime file and runtime service. It knows nothing of the runtime.
|
||||
2. The private network stops generating either resource. ADR 0082's decision stands — being on the
|
||||
network is what grants the trust, and no module author is involved — and only *who writes it*
|
||||
moves. The mesh gives the registry to the runtime module as a value. ADR 0082 and ADR 0102 each
|
||||
get a dated note saying where their mechanism now lives.
|
||||
3. The runtime module writes `dns`, `live-restore` and `insecure-registries`, each a declared
|
||||
setting with its cost: `dns` costs a restart, which `live-restore` makes harmless.
|
||||
4. Steps 1–3 land in one push. A runtime module declaring the file beside a resolver module still
|
||||
declaring it is refused.
|
||||
5. The collision check sees a computed module's resources as well, so a second writer cannot come
|
||||
back through generated code.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **How the resolver's address reaches the runtime.** Either the resolver seat (`node-dns-resolver`)
|
||||
delivers an address its holder serves, or the runtime module reads a machine fact and the seat
|
||||
being held is only a precondition. The first tracks a resolver moving off the private address. The
|
||||
second needs nothing new.
|
||||
- **What `dns` defaults to on a machine with no resolver seat held.** Nothing, leaving the runtime's
|
||||
own behaviour, is the honest default. A public resolver hides exactly the failure issue 110 took a
|
||||
day to find.
|
||||
- **The adopted machine's predecessor values.** The runtime module adopting a file with a
|
||||
hand-written `dns` and `live-restore: false` replaces both. That is intended, and is the one
|
||||
restart the operator must make on that machine.
|
||||
@@ -0,0 +1,80 @@
|
||||
---
|
||||
status: resolved
|
||||
opened: 2026-10-01
|
||||
located-in: [mesh-controller examples/route-proxy/main.go (routesFrom requires a route's public `name` and treats `internal-name` only as an alias of it; the handler serves every routed name to any source), mesh-controller internal/broker/membership.go (a membership says nothing of what its module receives or who the mesh is)]
|
||||
fixed-by: mesh-controller PR 207 (the membership carries what a module receives and who the mesh is; the proxy follows it and serves internal names to the mesh only), mesh-catalog PR 211 (the proxy's bus account), mesh-controller PR 208 (the issue verb that delivers it), live 2026-10-02
|
||||
amended-design: [03-DESIGN/01-to-be/08-connectivity.md, 03-DESIGN/01-to-be/25-the-bus-on-nats.md]
|
||||
---
|
||||
|
||||
# 191 — A route with only an internal name is dropped as naming nothing
|
||||
|
||||
## What was observed
|
||||
|
||||
A module whose endpoint reaches only the private network could not be reached by its internal name.
|
||||
The module ran and answered on its own port. Its route's internal name resolved to the serving node.
|
||||
The request failed during the TLS handshake:
|
||||
|
||||
```
|
||||
http: TLS handshake error from …: no public route for "unifi.home-server.internal" in this mesh,
|
||||
so no certificate is asked for
|
||||
```
|
||||
|
||||
The proxy's own log said why, every time it re-read its routes:
|
||||
|
||||
```
|
||||
unifi on home-server asked for a route and named nothing; skipped
|
||||
```
|
||||
|
||||
The route it skipped was not empty. The mesh had given it an endpoint, a port, a scheme and an internal
|
||||
name, and no public name:
|
||||
|
||||
| route | `name` | `internal-name` | served |
|
||||
|---|---|---|---|
|
||||
| home-assistant | a public name | `home-assistant.home-server.internal` | under both |
|
||||
| unifi | — | `unifi.home-server.internal` | under neither |
|
||||
|
||||
Three other modules on the same node were skipped with the same line on the same pass.
|
||||
|
||||
## Why it matters
|
||||
|
||||
**Reach is decided in one place, and the proxy reads the old shape of the decision.**
|
||||
[ADR 0138](../../02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md)
|
||||
made an endpoint's reach decide which names exist. The controller composes the public name, the
|
||||
internal name, or both, and composes no name that nobody asked for
|
||||
([issue 140](../140-an-endpoints-reach-is-not-declared/01-resolution.md)). An endpoint that reaches
|
||||
only the private network is the ordinary case for anything that should not face the internet. It is
|
||||
exactly the case the proxy drops.
|
||||
|
||||
**The failure is quiet and points the wrong way.** Nothing marks the module unhealthy. The handshake
|
||||
error says *no public route*, which reads as a certificate fault on the reader's side. The line that
|
||||
gives the real cause is one of four identical lines repeated every few seconds in a log nobody reads
|
||||
until they already suspect the proxy.
|
||||
|
||||
**The opposite move is not a workaround.** Giving the endpoint public reach makes the proxy serve it.
|
||||
It also publishes an administration interface to the internet to get a name on the private network.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should the proxy refuse a route it cannot serve in a way the controller or an operator sees, not
|
||||
only in its own log? The same silent skip covers a route with no usable port or an unknown scheme.
|
||||
- What checks that what the controller composes and what the proxy serves stay the same shape? ADR
|
||||
0138 changed one side and nothing failed on the other.
|
||||
|
||||
## Resolved (2026-10-02)
|
||||
|
||||
Built as [ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)
|
||||
decided, and live on both machines that run the proxy. Each logs that its routes now come from its
|
||||
membership, and serves internal names to the four machines the mesh names. Checked by hand:
|
||||
|
||||
- the internal-only route answers through the proxy from the serving machine and from two other
|
||||
machines of the mesh, over a certificate from the mesh's own authority that each verifies;
|
||||
- the same name asked from an address outside the mesh is answered as a name never routed, over plain
|
||||
HTTP, and refused in the TLS handshake; the list of served names it is shown leaves out every internal
|
||||
name.
|
||||
|
||||
Two things the rollout found are their own records: the proxy's bus account could be issued only from
|
||||
the controller's command line, until mesh-controller PR 208 added the `issue` verb, and the status line
|
||||
counting every module as a bus user without a credential is
|
||||
[issue 195](../195-every-assigned-module-is-counted-as-a-bus-user-without-a-credential/00-report.md).
|
||||
The serving machine also lacked the certificate-trust module, so it could not verify the mesh's own
|
||||
certificates until it was assigned there.
|
||||
@@ -0,0 +1,75 @@
|
||||
# Diagnosis
|
||||
|
||||
*2026-10-01.*
|
||||
|
||||
**Ruled out first: the module itself.** Its container was up and had not restarted. The controller's
|
||||
status endpoint answered on its own port with `"up": true`. The tool wrapper beside it was serving
|
||||
all its tools.
|
||||
|
||||
**Ruled out: name resolution.** The internal name resolved to the serving node's private-network
|
||||
address, which is where the proxy listens. Plain HTTP to the name reached the proxy and got a 404.
|
||||
HTTPS failed in the handshake, and the proxy logged that it had no route for the name.
|
||||
|
||||
**The route as the proxy received it.** The mesh-written route file held a complete contribution
|
||||
for the module: endpoint `web`, port, scheme `https`, `insecure`, a label, and `internal-name`. It had
|
||||
no `name`. That is what the controller composes for an endpoint whose reach stops at the private
|
||||
network (ADR 0138, `composeName`). The contribution was correct.
|
||||
|
||||
**Located: `routesFrom` in the proxy.** It reads `name` first and skips the contribution if `name`
|
||||
is empty. It reads `internal-name` only at the end, as a second host for a rule that already has a
|
||||
public one. So the proxy can serve an internal name only next to a public one. That matched the
|
||||
mesh before ADR 0138, when both names were always composed. It has been wrong since then.
|
||||
|
||||
The other half of the proxy already handles the case. Certificates for a host are split by whether it
|
||||
is in the public set: hosts outside it go to the internal authority, and only hosts inside it are
|
||||
eligible for ACME. A host that is only ever an internal name falls on the correct side of both checks
|
||||
without change. For certificates, only reading the route was wrong; who may reach the route is the next section.
|
||||
|
||||
**The fix.** `routesFrom` takes a route that names either host, serves each name it carries, and
|
||||
marks only the public one as public. It still skips a route that names neither, with the same log line.
|
||||
A test proves an internal-only route is served, certified by the internal authority, and refused by
|
||||
the public one. That test fails against the code before the change.
|
||||
|
||||
## The first fix would have made the name public — 2026-10-02, from review
|
||||
|
||||
Serving the dropped route was not enough. The proxy picks a route from the name a request carries
|
||||
and never from where the request came from, and it answers public and internal names on the same
|
||||
listeners. Its public names resolve to an address the internet reaches. So once the internal-only
|
||||
route was served, any request from the internet carrying `unifi.home-server.internal` — a name of a
|
||||
fixed, guessable shape — would have reached an administration interface that reach `internal` was
|
||||
chosen to keep private. Before the fix the route was unreachable from everywhere. After it, it would
|
||||
have been reachable from everywhere. Two more leaks came with it: the proxy's answer for an unrouted
|
||||
name listed every name it serves, internal ones included, and the handshake handed a certificate
|
||||
naming the internal host to any client.
|
||||
|
||||
Nothing showed this while every routed endpoint also had a public name: its internal name exposed
|
||||
nothing the public one did not. It is a gap in the decision's wording, not only in the proxy — ADR
|
||||
0138 says the proxy *serves* the internal name without saying to whom — so it is recorded there as a
|
||||
progressive insight and in the to-be connectivity design.
|
||||
|
||||
**Where "inside" is decided: told, not worked out.** The first correction had the proxy work it out
|
||||
for itself — the mesh's range from an environment variable the catalogue wrote, and the machine's
|
||||
container bridges from its own interfaces. That was a second definition of "the mesh", kept by one
|
||||
module beside the one the controller already has: it resolves "from the mesh" to every machine's
|
||||
address on the private network, and the packet filter is rendered from that list. Reviewed, it was
|
||||
replaced: [ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)
|
||||
has every membership on the bus carry what its module receives and that list, and the proxy follows
|
||||
its membership. One composition, read by the filter and by the proxy.
|
||||
|
||||
The proxy reads the source address, where the guard reads the interface, because it cannot see the
|
||||
interface a request arrived on. A claimed source does not carry here: a connection needs its replies,
|
||||
and replies to a mesh address leave by the tunnel.
|
||||
|
||||
**What changed with it.** The internal name of a route that also has a public one is now served to the
|
||||
mesh only, like any other internal name. Outsiders have the public name, so nothing they could reach is
|
||||
lost. A container calling its own machine's internal name arrives from its container network and is
|
||||
refused; whether the mesh should issue those networks too is left open in ADR 0167.
|
||||
|
||||
**Order of release.**
|
||||
|
||||
1. The catalogue change, which gives the proxy a bus account. A machine running the proxy is not
|
||||
composed until its account is issued, so the account is issued straight after
|
||||
(`module issue route-proxy --node <machine>`), and then the machine is pushed.
|
||||
2. The controller and proxy change. The push after it publishes memberships that carry the routes and
|
||||
the mesh, and each proxy takes them. Until then, a proxy serves its file, and internal names to its
|
||||
own machine alone.
|
||||
@@ -0,0 +1,71 @@
|
||||
---
|
||||
status: resolved
|
||||
opened: 2026-10-02
|
||||
located-in: [mesh-host internal/apply (removeOrphan: a former target of a kind with no removal was fatal), mesh-host internal/store (Record keeps a former target for every kind, the host's own archive included)]
|
||||
fixed-by: mesh-host 65 — a former target of a kind the host cannot remove is left in place, said and forgotten; a dropped archive still refuses
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 194 — The host's own former archive stops every machine applying anything
|
||||
|
||||
## What was observed
|
||||
|
||||
2026-10-02, 00:34Z, on all four machines of this mesh, the first time a host carrying former
|
||||
targets ([ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 5,
|
||||
built in mesh-host 63) replaced itself with a newer host (mesh-host 64).
|
||||
|
||||
The host delivers its own successor as an archive whose target is a versioned directory
|
||||
([ADR 0141](../../02-DECISIONS/0141-the-host-delivers-its-own-successor.md)): every new version
|
||||
is the same resource with a new target. Since mesh-host 63 the record keeps a resource's former
|
||||
target so the next apply removes what the host wrote under it
|
||||
([issue 097](../097-a-resource-that-changes-target-leaves-the-old-one-behind/00-report.md)). So
|
||||
the new host's first apply found the previous version's directory as a former target of its own
|
||||
archive, and asked the removal for an archive — which does not exist
|
||||
([issue 162](../162-an-archive-cannot-be-undeclared/00-report.md)):
|
||||
|
||||
```
|
||||
applying "mesh-host.next@former:/usr/lib/nox-mesh-host/versions/3c906749ad27": no way to remove a "archive"
|
||||
0 resource(s) were applied and remain
|
||||
```
|
||||
|
||||
Orphans are removed before any resource is applied on a converged machine, so the refusal ended
|
||||
every apply at its first step. Every machine reported `failed`, applied nothing, and would have
|
||||
gone on doing so: a host fix is itself an archive the same apply would have to write, and the apply
|
||||
never reached it. The machines kept running what they had; nothing new from the mesh could land.
|
||||
|
||||
## Why it matters beyond this instance
|
||||
|
||||
Two rules that are each right met in the one resource the host cannot afford to stop on. Rule 5
|
||||
says a former target is removed and said; issue 162 says an archive has no removal, deliberately,
|
||||
so an unassignment nothing can undo is never reported as done. Neither rule was wrong; their
|
||||
meeting was never tested, because the bed that would have found it is a host replacing itself
|
||||
under the new rule, and the first such replacement was the live one. The fix is narrow: a former
|
||||
target of a kind the host cannot remove is left in place, said, and forgotten — never fatal,
|
||||
because nobody dropped it. An archive the declaration dropped still refuses, as 162 has it.
|
||||
|
||||
## What it took to recover
|
||||
|
||||
The broken host cannot apply its own fix: the fix is delivered as an archive, and the apply fails
|
||||
before writing anything. On each machine the host's record (`/var/lib/mesh-host/state.json`) had to
|
||||
lose the one `@former:` entry by hand, once, so that the next push could write the fixed archive and
|
||||
stand aside for it. A manual edit of the host's record is otherwise never done; it is written here
|
||||
because the alternative was four machines that could apply nothing.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should `Record` keep a former target for a kind the host cannot remove at all? The trace is
|
||||
useful; the removal it implies is not. Keeping it and letting the apply forget it is what the fix
|
||||
does; not recording it would be quieter.
|
||||
- Should the host's own versions directory be cleaned by the launcher rather than by the apply —
|
||||
the one archive whose former targets are genuinely removable, by the thing that knows which one
|
||||
runs?
|
||||
- Is there a bed that replaces a host under the current rules before the live mesh does
|
||||
(the proof row of [ADR 0141](../../02-DECISIONS/0141-the-host-delivers-its-own-successor.md) was
|
||||
a single crossover, before former targets existed)?
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
mesh-host 65, merged 07:35Z. Recovered as the record above says: the operator dropped the one
|
||||
`@former:` entry from each machine's host record and pushed; the fixed host then ran on all four and
|
||||
its first apply said `forgotten mesh-host.next@former:… a former target left in place` and applied the
|
||||
rest. The open questions stand as questions for the host's own versions, not as faults.
|
||||
+62
@@ -0,0 +1,62 @@
|
||||
---
|
||||
status: open
|
||||
opened: 2026-10-02
|
||||
located-in: []
|
||||
fixed-by:
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 195 — Every assigned module is counted as a bus user without a credential, and the real gaps are lost in the count
|
||||
|
||||
## What was observed
|
||||
|
||||
`status`, and `plan` for any machine, open with one line before anything else:
|
||||
|
||||
```
|
||||
the bus's user list leaves out 49 user(s) the mesh has minted no credential for: <node>.<module>, …
|
||||
Each is a user that cannot connect until one is issued
|
||||
```
|
||||
|
||||
The 49 are spread over four machines and name 26 distinct modules. Checked against the catalogue on
|
||||
2026-10-02:
|
||||
|
||||
| what the module's definition says | modules |
|
||||
|---|---|
|
||||
| declares an own secret named `broker` | 1 — the route proxy, which needed a bus account for issue 191 |
|
||||
| declares no `broker` secret, and emits, consumes and serves nothing on the bus | 17 — the packet filter, the intrusion filter, the ssh daemon, the resolver configuration, the certificate authority, the broker itself and others |
|
||||
| declares no `broker` secret, and **emits events** | 1 |
|
||||
| not in this catalogue, so not checked | 7 |
|
||||
|
||||
So the line counts every module assigned anywhere as a bus user. For almost all of them that is not a
|
||||
missing credential. A module with no `broker` secret has nowhere to receive one, and the mesh already
|
||||
says an account nothing reads is an orphan ([issue 078](../078-a-delivered-secret-is-accepted-under-any-name/00-report.md)).
|
||||
|
||||
Two real gaps sit inside the count and cannot be told from the noise:
|
||||
|
||||
- **A declared `broker` secret was filled with a value that is not an account.** Before its account was
|
||||
issued, the route proxy's plan on both machines already carried a sealed `broker` file, while the same
|
||||
status line said no credential had been minted for it. A push had made the declared secret the way it
|
||||
makes any own secret. The module would have started with a credential the bus does not know, and
|
||||
nothing would have said why. It was found only because the account was being issued by hand.
|
||||
- **A module that emits events declares no way to reach the bus.** Its events can go nowhere, and no
|
||||
check refuses that.
|
||||
|
||||
## Why it matters
|
||||
|
||||
**A warning that is always on is read as never on.** The line names 49 users on every `status` and every
|
||||
`plan`. An operator, or an agent, learns to scroll past it. The one entry that was a real fault looked
|
||||
exactly like the 48 that were not.
|
||||
|
||||
**The fault that was real is the silent kind.** A module whose broker credential is a generated value
|
||||
starts, fails to authenticate, and reports that three layers away from the cause. That is the failure
|
||||
the composition already refuses for a secret that was never made at all ("declared and not made"). Here
|
||||
a value was made, so the refusal never fired.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should a bus user be composed for a module that declares no `broker` secret at all? If not, the line
|
||||
shrinks to the modules that can actually use an account.
|
||||
- Is a `broker` secret ever correctly made by the generic generator? If not, should composition refuse
|
||||
a declared `broker` until it is issued, or should the mesh issue it as part of placing the module?
|
||||
- Should a module that emits, consumes or serves on the bus be refused when it declares no `broker`
|
||||
secret?
|
||||
@@ -0,0 +1,54 @@
|
||||
---
|
||||
status: resolved
|
||||
opened: 2026-10-02
|
||||
located-in: [mesh-controller internal/catalogue/filtering.go (AsNftables: the forward chain has no rule for the mesh passing through, so a relayed packet is judged by this machine's own published ports)]
|
||||
fixed-by: mesh-controller PR 209 (the forward chain relays what comes in and goes out on the tunnel), live 2026-10-02
|
||||
amended-design: []
|
||||
---
|
||||
|
||||
# 196 — The hub relays the mesh only on the ports it publishes for itself
|
||||
|
||||
## What was observed
|
||||
|
||||
A sweep of every listening port on every machine, from every other machine, on 2026-10-02. Two home
|
||||
machines, neither of which can be dialled, reach a third home machine through the hub, as
|
||||
[ADR 0007](../../02-DECISIONS/0007-connectivity.md) says every path between machines that are not
|
||||
co-located does.
|
||||
|
||||
From either of the two, the third answered on **17 of its 55** listening ports over the mesh. The hub
|
||||
itself, probing the same machine directly, reached all the ports that machine's rules open to the mesh.
|
||||
The result was the same at 40 probes in parallel and at 4, so it was not load.
|
||||
|
||||
The 17 were not a property of the target. They were exactly the ports **the hub** publishes for its own
|
||||
containers: ssh, the proxy's two, and the hub's own block of published ports. A capture on the target
|
||||
during one probe to a port that answered and one that did not:
|
||||
|
||||
- the answering one: the SYN arrives on the tunnel, reaches the container, and the reply leaves by the
|
||||
tunnel;
|
||||
- the other: nothing arrives at all, on any interface.
|
||||
|
||||
## Why it matters
|
||||
|
||||
**ADR 0007's hub carries every path between machines that are not co-located, and the filter breaks
|
||||
that path without saying so.** Whether one home machine can reach a service on another depends on
|
||||
whether the hub happens to publish the same port number for something of its own. Adding or removing
|
||||
a module on the hub silently opens or closes paths between two other machines that it has nothing to
|
||||
do with.
|
||||
|
||||
It also hid behind another fault. A missing placement made the same pair look disconnected earlier the
|
||||
same day, and that explanation fit well enough that the per-port pattern was not looked for.
|
||||
|
||||
## Open questions
|
||||
|
||||
- The relaying rule accepts what comes in on the tunnel and leaves on it, and leaves judging to the
|
||||
machine it is for. Should the hub also restrict relayed traffic to what that machine opens to the
|
||||
mesh? That would duplicate the target's rules on the hub.
|
||||
- No test raises two machines behind a hub and checks a port between them that the hub does not
|
||||
publish. The lab's beds have one machine per site.
|
||||
|
||||
## Resolved (2026-10-02)
|
||||
|
||||
Live on all four machines after one push each. The same sweep, from both home machines to the third
|
||||
over the mesh: 45 of 55 ports answer, the same 45 the hub reaches directly. The 9 that do not are
|
||||
ports the target opens to nobody on the mesh, and one is refused because it listens only on a LAN
|
||||
address. Nothing answers that the target's rules do not open.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Diagnosis
|
||||
|
||||
*2026-10-02.*
|
||||
|
||||
**Not the tunnel.** The route from either home machine to the target is the tunnel, and traffic to the
|
||||
17 ports travels it in both directions. A placement fault would have stopped every port.
|
||||
|
||||
**Not the target's filter.** The target opens the failing ports to every address of the mesh in its
|
||||
input chain and its forward chain, the hub reaches them directly, and the SYN for a failing port never
|
||||
arrived at the target to be judged.
|
||||
|
||||
**The hub's forward chain.** A relayed packet comes in on the tunnel and leaves on it, so the hub's
|
||||
forward hook judges it. The chain the controller renders (`AsNftables`) has a default of drop, accepts
|
||||
established traffic, and accepts what did not arrive on an outward link or the tunnel. That last rule is
|
||||
for the machine's own containers reaching outward. After that come the rules for this machine's own
|
||||
published ports, each matching the **original destination port** of the connection. None of them names
|
||||
an outgoing interface or a destination. So a relayed packet to another machine's port 20000 matched the
|
||||
hub's own rule for its own port 20000 and passed. One to port 8080, which the hub does not publish,
|
||||
matched nothing and was dropped.
|
||||
|
||||
**The fix.** One rule: in on the tunnel **and** out on the tunnel is accepted. That is the mesh passing
|
||||
through to another of its machines, which filters it against its own rules. It does not widen anything
|
||||
on the hub. A packet for the hub itself is the input chain's, and one for the hub's own containers
|
||||
leaves by a bridge, not the tunnel. Both still meet their rules. WireGuard only accepts a packet from a
|
||||
peer whose address that peer is allowed to use, so in-on-the-tunnel means from a machine of the mesh.
|
||||
A controller test asserts the rule in the forward chain only, never in the input chain, and absent on a
|
||||
machine with no tunnel. It fails without the fix, and the rendered set loads with `nft -c`.
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
---
|
||||
status: resolved
|
||||
opened: 2026-10-02
|
||||
located-in: [mesh-host internal/outward (Links reported only the links carrying a default route)]
|
||||
fixed-by: mesh-host PR 66 (a link backed by a physical device is named outward, up or down), live 2026-10-02
|
||||
amended-design: []
|
||||
---
|
||||
|
||||
# 197 — A physical link that is down is not filtered when it comes up
|
||||
|
||||
## What was observed
|
||||
|
||||
A sweep of every machine's filter on 2026-10-02. A laptop-class machine connected by its radio has a
|
||||
wired port that was unplugged. Its filter guarded the radio and the tunnel, and accepted everything
|
||||
arriving on any other link:
|
||||
|
||||
```
|
||||
iifname != { "mesh0", "<radio>" } accept
|
||||
```
|
||||
|
||||
The wired port was not in the list. Plugged in, everything arriving on it would have been accepted,
|
||||
every port of the machine open to whatever network the cable reached. That would last until the
|
||||
machine reported again and was pushed a new filter.
|
||||
|
||||
## Why it matters
|
||||
|
||||
**The filter's one rule about links fails open.** [ADR 0140](../../02-DECISIONS/0140-the-filter-constrains-what-arrives-from-outside.md)
|
||||
has the filter constrain what arrives from outside, and has the machine say which links face outside.
|
||||
Everything not named is treated as the machine's own, its containers and bridges. So a link the machine
|
||||
fails to name is not filtered at all. The host named only the links carrying a default route at the
|
||||
moment it reported. A cable plugged in later is the ordinary case for a laptop. A second wired network
|
||||
that never carries the default route, such as a direct link to a storage box, is never named at all.
|
||||
|
||||
## Open questions
|
||||
|
||||
- A virtual link that faces outside (a VPN client's interface, a USB tether that appears as a virtual
|
||||
device) has no physical device behind it. It is named only while it carries the default route. Is
|
||||
that enough?
|
||||
|
||||
## Resolved (2026-10-02)
|
||||
|
||||
Live on the affected machine after the host was delivered and one more push: its filter now guards the
|
||||
radio, the tunnel and the unplugged wired port, before anything is plugged into it.
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
# Diagnosis
|
||||
|
||||
*2026-10-02.*
|
||||
|
||||
**Located in `mesh-host` `internal/outward`.** `Links` read the kernel's routing tables and returned the
|
||||
interfaces carrying a default route. An unplugged port carries none, so it was never reported, and the
|
||||
controller rendered the filter around the links it was given.
|
||||
|
||||
**The fix.** A link faces outside if it carries a default route **or** has a physical device behind it.
|
||||
The kernel lists every interface under `/sys/class/net`, with a `device` entry for one backed by
|
||||
hardware. A bridge, a veth, the tunnel and the loopback have none, so they stay the machine's own. The
|
||||
wired port is now reported up or down, and the filter guards it before anything is plugged in. Tested
|
||||
with a radio carrying the default route and an unplugged wired port beside a bridge, a veth, the docker
|
||||
bridge, the tunnel and the loopback: the two physical links are reported, nothing else.
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
---
|
||||
status: resolved
|
||||
opened: 2026-10-02
|
||||
located-in: [mesh-catalog modules/dnsmasq (listens on loopback and the machine's mesh address only), the home-server's DNS (a predecessor's dnsmasq configuration the mesh did not own), the home network's DHCP (hands out the home-server as every device's DNS)]
|
||||
fixed-by: mesh-catalog PR 214 (dnsmasq listens on addresses from a setting; docker's file takes no settings), mesh-controller PR 210 (the settings verb), mesh-catalog PR 215 (unifi network DNS tools), live 2026-10-02
|
||||
amended-design: []
|
||||
---
|
||||
|
||||
# 198 — The home network's DNS server ran outside the mesh, and the mesh's filter closed it
|
||||
|
||||
## What was observed
|
||||
|
||||
Every phone on the home Wi-Fi had no internet, while a laptop on the same Wi-Fi did. The router's
|
||||
DHCP hands every device the home-server's LAN address as its DNS server. The home-server's DNS daemon
|
||||
was listening on that address, and every query to it timed out. The router itself answered the same
|
||||
query at once. The laptop worked because it resolves through its own local resolver, not through the
|
||||
server DHCP names.
|
||||
|
||||
## Why it happened
|
||||
|
||||
The DNS daemon on the home-server was not the mesh's. It ran under a configuration file a predecessor
|
||||
generated, listening on loopback, the mesh address and the LAN address. The mesh's `dnsmasq` module was
|
||||
assigned to the other three machines and not to this one, so no module on the home-server declared
|
||||
port 53. Its filter opens only what a module declares, so DNS from the LAN was dropped. It started when
|
||||
the home-server applied the filter this morning, after nine hours of applying nothing
|
||||
([issue 194](../194-the-hosts-own-former-archive-stops-every-apply/00-report.md)).
|
||||
|
||||
Nothing said so. The daemon reported running, the filter applied cleanly, and the mesh had no record
|
||||
that the home network depended on a service it did not know.
|
||||
|
||||
## Why it matters
|
||||
|
||||
**A service the mesh does not know is closed by the mesh's filter, by design, and nothing asks whether
|
||||
something depends on it.** That is the right default for an unknown port. It is the wrong outcome for
|
||||
the one service a whole network was told to use. The gap is that a machine can run something
|
||||
important outside the mesh with nothing to show it.
|
||||
|
||||
**The mesh's `dnsmasq` could not have served the LAN either.** It listened on loopback and the mesh
|
||||
address only. The reach of its DNS endpoints opens the filter, but the daemon would not have been
|
||||
listening on the LAN address anyway.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should a machine report the listening services the mesh does not own, the way it reports the links
|
||||
that face outside? This one would have been visible before the filter closed it.
|
||||
- The LAN address the home-server answers on is now a setting, beside the reach that opens the filter.
|
||||
Two statements that must agree. Should reach `public` on a DNS endpoint imply listening beyond the
|
||||
mesh?
|
||||
|
||||
## Resolved (2026-10-02)
|
||||
|
||||
The home network was pointed at the gateway for DNS while the fix was built, which got the phones back
|
||||
within minutes. Then:
|
||||
|
||||
- the mesh's `dnsmasq` takes the addresses it listens on beside the machine's from a setting, with
|
||||
loopback as the mesh-wide default, so no other machine changed;
|
||||
- the home-server's layer adds its LAN address, and its DNS endpoints' reach is `public`. The router
|
||||
forwards no DNS, so that means the LAN;
|
||||
- the module and its sibling `resolv-conf` were assigned to the home-server, replacing the
|
||||
predecessor's daemon and configuration, which were kept aside;
|
||||
- the home network was pointed back at the home-server, through a new `unifi` tool.
|
||||
|
||||
Checked live: from another machine on the LAN, public names and mesh names both resolve through the
|
||||
home-server's LAN address, and the mesh and the machine itself resolve as before.
|
||||
|
||||
**One fault found on the way, and caught before it reached any machine.** A module's settings are
|
||||
merged into every mergeable file the module owns. The first attempt therefore put the new setting into
|
||||
docker's `daemon.json` as well as into dnsmasq's config, and dockerd refuses keys it does not know. The
|
||||
plan showed it before any push. The change was reverted and redone with docker's file declared to take
|
||||
no settings. The general fault, a module's settings reaching files they were not meant for, is still
|
||||
there for any module with more than one file.
|
||||
Reference in New Issue
Block a user