Building the bus: the decisions the work needed, and what it taught back #150
+3
-3
@@ -34,13 +34,13 @@ another — and a mesh you cannot name precisely is a mesh two people describe d
|
||||
- **bus** — the mesh's own nervous system: NATS, one per mesh, carrying every link the mesh has —
|
||||
control, declarations, builds, events, tool calls
|
||||
([ADR 0106](../02-DECISIONS/0106-the-bus-is-nats.md)). A module reaches it by requiring
|
||||
`mesh-bus` ([ADR 0120](../02-DECISIONS/0120-the-mesh-bus-is-required-not-ambient.md)); one that
|
||||
`mesh-bus` ([ADR 0128](../02-DECISIONS/0128-the-mesh-bus-is-required-not-ambient.md)); one that
|
||||
does not require it has no account on it. Held by the `mesh-broker` seat, which is named after
|
||||
the *role* rather than the server, so the server can change without the seat doing so.
|
||||
- **the deprecated broker** — the lavinmq module. It was the mesh's bus and is not any more. It
|
||||
keeps running as an **ordinary provider** of the `amqp` provision, for modules that need a
|
||||
message broker of their own the way something needs a database
|
||||
([ADR 0119](../02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md)) — no seat, not foundation,
|
||||
([ADR 0127](../02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md)) — no seat, not foundation,
|
||||
never raised at genesis, and a mesh that never installs it is complete.
|
||||
|
||||
Say *the deprecated broker*, not "the compatibility broker" (it serves the mesh's own modules,
|
||||
@@ -61,7 +61,7 @@ another — and a mesh you cannot name precisely is a mesh two people describe d
|
||||
- **seat** — a named role at a scope (node / site / mesh), held by a module assignment, from a
|
||||
**closed set** the mesh defines: a claim naming a seat outside the set is refused. A seat may
|
||||
**deliver a provision**, and its holder is then the mesh's answer for it when several modules
|
||||
provide it ([ADR 0118](../02-DECISIONS/0118-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md))).
|
||||
provide it ([ADR 0126](../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md))).
|
||||
The set, with who holds each seat, is the overview of what a mesh has
|
||||
([26 — The seats](../03-DESIGN/01-to-be/26-the-seats.md)). A seat has a **capacity**: a
|
||||
capacity-1 seat is exclusive (one holder); a higher-capacity seat is a **bench** (several holders
|
||||
|
||||
@@ -89,7 +89,7 @@ three relationships, one broker, one runtime, all declared on the manifest.
|
||||
>
|
||||
> The decision is untouched — events are declared on both sides, 1:many, credential-free, and
|
||||
> still provisioning's lighter sibling; the lightness is now relative rather than absolute.
|
||||
> [ADR 0118](0118-a-module-declares-its-own-seats.md) adds the relationship this record's two
|
||||
> [ADR 0126](0126-a-module-declares-its-own-seats.md) adds the relationship this record's two
|
||||
> columns had no room for: work addressed to a role, where exactly one holder must act.
|
||||
|
||||
## References
|
||||
|
||||
@@ -83,16 +83,16 @@ compatibility broker beside it moves its bus in one rollout with every node repo
|
||||
> ["amqp"]` grant its provisioner answered with a private vhost — `amqp-ping` and
|
||||
> `amqp-email-forwarder`. On the retirement condition below, both would have been left requiring
|
||||
> something no provider answers.
|
||||
> [ADR 0117](0117-the-bus-is-the-only-broker.md) resolves it by moving them onto the bus and
|
||||
> [ADR 0125](0125-the-bus-is-the-only-broker.md) resolves it by moving them onto the bus and
|
||||
> retiring the interface, which makes this record's sentence true rather than merely intended. The
|
||||
> decision — the bus is NATS, the AMQP broker becomes the predecessor's compatibility broker and
|
||||
> retires with the last of them — is unchanged.
|
||||
|
||||
> **Progressive insight — 2026-09-26, correcting the one above.** *The broker is not a
|
||||
> compatibility module at all, and the sentence does not become true.* The insight above said
|
||||
> [ADR 0117](0117-the-bus-is-the-only-broker.md) would make "one purpose — the predecessor's
|
||||
> [ADR 0125](0125-the-bus-is-the-only-broker.md) would make "one purpose — the predecessor's
|
||||
> clients" true by moving the mesh's own modules off it.
|
||||
> [ADR 0119](0119-amqp-is-a-provision-not-the-bus.md) supersedes that: nothing moves off, because
|
||||
> [ADR 0127](0127-amqp-is-a-provision-not-the-bus.md) supersedes that: nothing moves off, because
|
||||
> a module may legitimately need an AMQP broker as a backing service the way it needs a database.
|
||||
> The broker becomes **an ordinary provider module** — no seat, not foundation, not raised at
|
||||
> genesis, and with no retirement condition, because the day its last client disappears is not a
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: superseded
|
||||
superseded-by: 02-DECISIONS/0118-a-module-declares-its-own-seats.md
|
||||
status: accepted
|
||||
date: 2026-09-25
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
@@ -10,6 +9,17 @@ extends: 0009-modules-and-the-graph.md
|
||||
|
||||
# 110. A seat is held by one assignment, from a closed set, and it may deliver a provision
|
||||
|
||||
> **Narrowed, not replaced — 2026-09-27, on merging two lines of work.** This was marked superseded by
|
||||
> [ADR 0126](0126-a-module-declares-its-own-seats.md), and that overstated it: 0126 says in as many
|
||||
> words that *"everything 0110 decided about what a seat is stands untouched"*. What moved is where the
|
||||
> set lives and who may add to it —
|
||||
> [0126](0126-a-module-declares-its-own-seats.md) lets a module declare one and makes the set derived,
|
||||
> [0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md) names the mesh's own
|
||||
> for their scope, and [0122](0122-a-seat-is-data-a-rename-is-a-database-update.md) moves them out of
|
||||
> code into a table. **What a seat *is* — one holder at its scope, a definition saying what a module
|
||||
> can hold against an assignment saying what it does, a role made singular rather than a module — is
|
||||
> this record and still current**, which is why those three rest on it.
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0009](0009-modules-and-the-graph.md) introduced claims: a module declares something
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
date: 2026-09-26
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0110-a-seat-is-a-module-assignment-from-a-closed-set.md
|
||||
---
|
||||
|
||||
# 117. A machine's uplink is a seat: the mesh configures the manager, never the link
|
||||
|
||||
## Context
|
||||
|
||||
The mesh installs on top of a machine's own networking. The private network's generator says
|
||||
so in as many words: a machine has an address and a route to the broker *before* the mesh
|
||||
exists, the broker's address travels in the enrolment token rather than being resolved, and the
|
||||
private network is something the mesh installs on top, like anything else. Nothing in the mesh
|
||||
says who manages that uplink, or what the mesh needs from whoever does.
|
||||
|
||||
Adopting the first workstations showed that the mesh does need something from it, and gets it
|
||||
by accident:
|
||||
|
||||
- **The resolver the mesh owns depends on a file the mesh does not.** `resolv-conf` writes
|
||||
`/etc/resolv.conf` and names the mesh's resolver. On a machine running NetworkManager, the
|
||||
manager rewrites that file on every connectivity change unless it is told `dns=none`; on one
|
||||
running dhcpcd, every lease renewal rewrites it unless it is told `nohook resolv.conf`. On
|
||||
the adopted machines both settings exist only because the predecessor wrote them. No module
|
||||
declares them. Remove the predecessor's file and the mesh's resolver is silently replaced the
|
||||
next time a laptop changes network, while every surface of the mesh still reads green.
|
||||
- **`resolv-conf` cannot declare them itself.** Which setting is needed depends on which
|
||||
manager runs, and a `service` resource for a manager that is not installed fails the
|
||||
declaration. A resolver module that knew about network managers would be the wrong module
|
||||
knowing the wrong thing.
|
||||
- **The private network's interface is exposed to the manager.** A manager that considers
|
||||
every interface its own may try to configure `mesh0`, or tear it down on a profile change.
|
||||
Nothing tells it not to.
|
||||
- **Two managers on one machine go unnoticed.** Among the machines adopted so far, one runs
|
||||
NetworkManager *and* dhcpcd at once: two programs that each believe they own the machine's
|
||||
addresses and its resolver file.
|
||||
Nothing detected it, because nothing in the mesh knows the role exists.
|
||||
|
||||
The machines differ in a way that matters: servers are wired and never move, while
|
||||
workstations join wireless networks, captive portals and phone hotspots wherever they are.
|
||||
|
||||
## Considered Options
|
||||
|
||||
**1. The mesh manages the uplink: links, addressing, wireless networks and their
|
||||
credentials.** Rejected. The mesh reaches a machine only over that link. A declaration that
|
||||
gets it wrong — a mistyped network, a stale credential, a manager that fails to start — takes
|
||||
the machine off the network, and with it the only channel a fix could arrive on. That is the
|
||||
one failure the sshd module's `listens` rule forbids the firewall to arrange; a mesh that owned
|
||||
the link could arrange it with any push. And a wireless network is joined at the machine, by
|
||||
the person using it, in the moment. A declaration composed elsewhere cannot answer a captive
|
||||
portal.
|
||||
|
||||
**2. Leave the uplink unmanaged; accept the implicit dependency.** Rejected. It keeps the
|
||||
resolver working only for as long as a predecessor's file survives, and it leaves two managers
|
||||
on one machine undetectable.
|
||||
|
||||
**3. The uplink is a seat. The module holding it configures the manager's relationship to the
|
||||
mesh, and never the link.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**`the-uplink` is a node-scoped seat** in the closed set ([ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)).
|
||||
It delivers no provision. It is held by the module for the program that manages the machine's
|
||||
own network, one per manager: `networkmanager`, `systemd-networkd`, and `dhcpcd` for a machine
|
||||
with nothing more. Assigning a second is refused, naming the first.
|
||||
|
||||
**What a holder declares** — only what keeps the manager and the mesh from contradicting each
|
||||
other:
|
||||
|
||||
- the manager's package, present — and its service **with no state**: the manager's lifecycle is
|
||||
the machine's. The mesh never starts, stops, enables or disables it, because stopping it takes
|
||||
the link down, and a holder unassigned by mistake — or the wrong holder assigned — must not be
|
||||
able to do that, nor start a second manager beside the one the machine runs. The service is
|
||||
declared only so a change to the holder's settings reaches a *running* manager;
|
||||
- the manager's own configuration that leaves the resolver file to the mesh (`dns=none` for
|
||||
NetworkManager, `nohook resolv.conf` for dhcpcd, and nothing for systemd-networkd, which
|
||||
never writes the resolver file);
|
||||
- the manager's own configuration that leaves the private network's interface alone
|
||||
(NetworkManager's `unmanaged-devices` naming `mesh0`; dhcpcd's `denyinterfaces mesh0`; for
|
||||
systemd-networkd a network file of the module's matching `mesh0` as `Unmanaged=yes`);
|
||||
- each as a drop-in beside the manager's main file where the manager reads one, and written
|
||||
*into* a shared file otherwise, as a marked region the host owns
|
||||
([ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md)'s idea for text files),
|
||||
placed where the manager reads it as global — at the start of `dhcpcd.conf`, above any
|
||||
`interface` line, because every line after one belongs to that interface;
|
||||
- the service **reloaded** when a drop-in changes, never restarted — a restart drops the link,
|
||||
and the link is the mesh's own channel to the machine. **A manager that cannot reload is not
|
||||
restarted instead:** its setting takes effect at the manager's next start. Measured on the
|
||||
adopted machines: NetworkManager (1.58) and systemd-networkd (systemd 261) both report
|
||||
`CanReload=yes`; dhcpcd (10.3) reports `CanReload=no`, so its module declares no trigger at
|
||||
all. Whether each setting is actually *applied* by a reload is confirmed on a machine before
|
||||
the module is taken there, not assumed.
|
||||
|
||||
**What a holder never declares:** a link, an address, a route, a connection profile, a
|
||||
wireless network or its credentials. Those are the operator's, in the sense of
|
||||
[ADR 0051](0051-shared-data-is-the-operators.md): the mesh does not create, change or delete
|
||||
them, and the module's `access`, if it needs one, is read-only.
|
||||
|
||||
## Consequences
|
||||
|
||||
- `resolv-conf` stays generic. The condition it could not express — "only if NetworkManager
|
||||
runs" — is expressed by assigning the module for the manager that does.
|
||||
- The dependency on the predecessor's `dns=none` file becomes a declared resource. On an
|
||||
adopted node the holder's drop-in arrives beside the predecessor's; both say the same thing,
|
||||
and the predecessor's is retired by hand after the take, like any other file the mesh
|
||||
replaced under another name.
|
||||
- A setting a manager reads only at its start is not in force until then. On an adopted machine
|
||||
the predecessor's identical line normally already is; on a machine that was not adopted,
|
||||
dhcpcd's resolver hook keeps rewriting the resolver file until dhcpcd next starts, and the
|
||||
operator restarts it once, in a window of their choosing.
|
||||
- A machine running two managers is found at assignment: the second holder is refused, and the
|
||||
operator decides which manager the machine keeps before either module is taken.
|
||||
- Workstations keep joining networks the way they always have. Under NetworkManager and
|
||||
systemd-networkd the host already cooperates with the manager — its dispatcher hook wakes it
|
||||
on every connectivity change — and nothing here changes that. A dhcpcd-only machine has no
|
||||
such hook, and nothing here adds one.
|
||||
- The seat table gains one entry: `the-uplink`, node scope, delivering nothing, decided here.
|
||||
- **Not decided here:** whether the mesh should ever *offer* known networks to a machine — a
|
||||
sealed, add-only list the operator curates once for all workstations. That is a different
|
||||
question (the mesh holding credentials for links it must never be able to break) and gets its
|
||||
own record if it is wanted.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md): the closed set this seat joins;
|
||||
[to-be 26](../03-DESIGN/01-to-be/26-the-seats.md): the seat table
|
||||
- [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md): written into, never over
|
||||
- [ADR 0051](0051-shared-data-is-the-operators.md): what is the operator's stays the operator's
|
||||
- mesh-controller `internal/catalogue/seats.go` (the seat), `internal/overlay/generator.go` (the
|
||||
mesh installs on top of the machine's own networking)
|
||||
- mesh-catalog `modules/networkmanager`, `modules/systemd-networkd`, `modules/dhcpcd`
|
||||
- mesh-host `internal/apply/block.go` (a file written into a marked region, `at` start or end)
|
||||
@@ -0,0 +1,119 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
date: 2026-09-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0102-the-mesh-writes-into-a-shared-file-never-over-it.md
|
||||
---
|
||||
|
||||
# 118. Undeclaring removes what the mesh made, and gives a unit back the state it was found in
|
||||
|
||||
## Context
|
||||
|
||||
When a resource stops being declared — its module unassigned, the node sent a
|
||||
deliberately-empty declaration ([issue 127](../04-ISSUES/127-a-declaration-that-shrinks-to-empty-is-skipped-not-sent/00-report.md)),
|
||||
or a new catalogue version renaming its id — the host undoes it. The host's own code states
|
||||
the rule it means to follow: **it removes what it made and leaves what it merely configured.**
|
||||
For almost every resource it does exactly that:
|
||||
|
||||
- a container, a network, a process's unit, a directory it created: removed;
|
||||
- a file it created: removed; a file it replaced: its kept original put back
|
||||
([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md));
|
||||
- keys and list members it wrote into a shared file: given back as they were
|
||||
([ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md));
|
||||
- a package: left installed — the host cannot know it is unused;
|
||||
- an operator's path it was given access to: never touched
|
||||
([ADR 0051](0051-shared-data-is-the-operators.md)).
|
||||
|
||||
**A service is the exception.** A `service` resource never installs a unit: it puts one that
|
||||
already exists — the distribution's, the operator's — into a state. Undeclared, the host stops
|
||||
it. That contradicts the rule above, and in practice it is the most dangerous thing an
|
||||
undeclare can do. Found reviewing the uplink modules
|
||||
([issue 130](../04-ISSUES/130-undeclaring-a-service-stops-it/00-report.md)):
|
||||
|
||||
- the private network declares the container runtime's unit only so a change to the registry
|
||||
trust reloads it — unassigning the private network stops the runtime, and every container on
|
||||
the machine, the mesh's and not;
|
||||
- the sshd module declares the ssh daemon — unassigning it stops ssh, the lockout that module's
|
||||
own `listens` rule forbids;
|
||||
- the uplink modules would have stopped the network manager, taking the machine off the only
|
||||
link the mesh reaches it by.
|
||||
|
||||
[ADR 0125](0117-a-machines-uplink-is-a-seat.md) answered that for its own modules with a
|
||||
service declared with no `state`. Every other module that declares a unit it did not make is
|
||||
exposed in the same way, and relying on each author to remember an opt-out is how the next one
|
||||
is missed.
|
||||
|
||||
## Considered Options
|
||||
|
||||
**1. Undeclaring touches nothing on the machine.** Rejected. What the mesh made would outlive
|
||||
the module that made it: a container nobody manages keeps serving and stops being patched; a
|
||||
unit the mesh wrote keeps running a bundle nothing updates; a name collides when the module
|
||||
is assigned again. An undeclare that leaves the mesh's own work behind is an orphan factory.
|
||||
|
||||
**2. Keep stopping services; make "leave it running" an opt-in per resource.** Rejected. It
|
||||
keeps the dangerous behaviour as the default for exactly the units that matter most — the
|
||||
runtime, the ssh daemon, the network — and each new module is one forgotten field away from a
|
||||
machine that goes dark when it is unassigned.
|
||||
|
||||
**3. Never stop a unit the mesh did not create.** Rejected, found while implementing it. The
|
||||
mesh's packet filter is a unit the distribution installed and the mesh started at converge;
|
||||
returning a node to adopted ([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md))
|
||||
unloads it by undeclaring it. Never stopping it would leave the mesh's filter loaded beside the
|
||||
predecessor's firewall re-enabled — the one rollback a converge promises, broken. Who wrote the
|
||||
unit file is not the line; what the mesh *did* to the unit is.
|
||||
|
||||
**4. Give the unit back the state it was found in.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**Undeclaring removes what the mesh made and gives back what it changed.** For a unit the mesh
|
||||
did not create, what it changed is the unit's state, so that is what is given back: **the host
|
||||
records the state it first found the unit in, and undeclaring returns the unit to it.**
|
||||
|
||||
- **Recorded once**, the first time the host applies the service — whether it was running, and,
|
||||
where the declaration sets it, whether it was enabled at boot — and carried in the host's
|
||||
record from then on. Later applies never overwrite it: by then the unit's state is the mesh's
|
||||
doing.
|
||||
- **A unit found running is left running.** The container runtime, the ssh daemon, a network
|
||||
manager: running before the mesh arrived, running after it leaves.
|
||||
- **A unit the mesh started is stopped again**, and one it enabled is disabled again — the packet
|
||||
filter a converge loaded, which returning to adopted unloads.
|
||||
- **Never started on the way out.** A unit the mesh stopped is not started again when its
|
||||
declaration goes; starting something is a decision, and the operator makes it.
|
||||
- **Unknown is left alone.** A record written before the host kept what it found says nothing
|
||||
about the unit before the mesh; the unit is left exactly as it is. A unit left running can be
|
||||
stopped by the operator; one stopped by mistake may be the link the operator needed to do it.
|
||||
- A unit the mesh *did* create — a `process` resource's unit and bundle — is stopped and removed
|
||||
with its declaration. That is the mesh's own code. (Before this record there was no way to
|
||||
remove one at all: an undeclared process failed every apply on its node.)
|
||||
- The service's settings the mesh wrote are given back by their own resources (a kept original
|
||||
restored, a region or keys removed). A running service keeps running on what it read until it
|
||||
next reads its configuration; the mesh does not restart it to make it notice.
|
||||
- A service declared with no `state` (ADR 0125) remains the way to say the mesh must not
|
||||
**start** a unit either; undeclared, it is forgotten.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Unassigning the private network no longer stops the container runtime; unassigning sshd no
|
||||
longer stops ssh; no uplink module can take a machine's network down on its way out.
|
||||
- The host's removal report says what it gave back — "restored: stopped again, as the host
|
||||
found it" — or "forgotten: it was running before the mesh; left as it is" where it used to say
|
||||
"stopped". Its plan names each unit an undeclare will stop, before it does.
|
||||
- On a fresh machine where the mesh installed and started a service, unassigning its module
|
||||
stops it again — the mesh gave, the mesh takes back. An operator who wants it kept declares it
|
||||
in a module of their own, or starts it themselves after.
|
||||
- A daemon can keep running after its module is gone, on configuration that was taken back from
|
||||
under it. That is a visible, running process the operator can see and stop; the alternative
|
||||
was an invisible outage.
|
||||
- **Not decided here:** an unassign preview that lists what an undeclare will remove and what it
|
||||
will leave running. Issue 130 asks for it; it is the controller's to build.
|
||||
|
||||
## References
|
||||
|
||||
- [issue 130](../04-ISSUES/130-undeclaring-a-service-stops-it/00-report.md): the finding
|
||||
- [ADR 0125](0117-a-machines-uplink-is-a-seat.md): the uplink modules, and a service with no state
|
||||
- [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md), [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md):
|
||||
what is given back, and how
|
||||
- mesh-host `internal/apply/apply.go` (`remove`, the service case)
|
||||
@@ -0,0 +1,87 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-09-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0105-the-mesh-adopts-the-predecessors-tunnel-in-place.md
|
||||
---
|
||||
|
||||
# 119. A taken tunnel's predecessor is retired once the take is proven
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0105](0105-the-mesh-adopts-the-predecessors-tunnel-in-place.md) has the private network take
|
||||
over the tunnel it finds: the found unit stopped and disabled, never flushed, and **its
|
||||
configuration left on disk, kept like any held file.** That was the right caution for the take
|
||||
itself — if the mesh's interface failed to come up, the host starts the found unit again and the
|
||||
peers never notice — and every apply since stops the found unit again should anyone start it.
|
||||
|
||||
What it leaves is a predecessor that never finishes leaving. On every machine that has enrolled,
|
||||
the tunnel is the mesh's and has been proven so — its interface up with the found key, the peers
|
||||
handshaking, the machines resolving and reaching each other over it — and still the predecessor's
|
||||
configuration sits where its unit reads it, held for a module that has long since replaced it.
|
||||
The predecessor itself is being deprecated. A tunnel that can be started again by one command, with
|
||||
a configuration nothing maintains any more, is not a rollback path; it is a second way onto the
|
||||
network that nobody is watching. And the hold never ends, so every node report keeps listing it.
|
||||
|
||||
## Considered Options
|
||||
|
||||
**1. Keep it, as 0105 says.** Rejected: the caution it bought is spent once the take is proven, and
|
||||
what remains is a live, unmaintained way back onto the network.
|
||||
|
||||
**2. Delete it at the take.** Rejected: the take is exactly the moment the fallback is needed. If
|
||||
the mesh's interface does not come up, the host must still be able to raise the found one.
|
||||
|
||||
**3. Retire it once the take is proven.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**Once the mesh's interface has proven it carries the tunnel, the found interface's configuration
|
||||
is removed from where its unit reads it.**
|
||||
|
||||
- **Proven means:** the tunnel's state is *taken* — the found unit down and disabled, the mesh's
|
||||
interface up with the found key — and the mesh's interface has completed a handshake with at
|
||||
least one peer. Not before: until then, a failed take still falls back to the found unit.
|
||||
- **Retired means:** the configuration file the found unit reads is removed. Its original was
|
||||
already kept, before anything happened to it
|
||||
([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md)), and stays kept; that copy
|
||||
is the record of what the predecessor was, and a person's way back if one is ever wanted.
|
||||
- The found unit stays disabled. Without its configuration it cannot raise the interface, so the
|
||||
every-apply stop that guarded against it becomes a check that finds nothing to do.
|
||||
- **The hold ends.** What was held for the private network has been replaced; the node stops
|
||||
reporting it.
|
||||
- **The mesh never brings it back.** Undeclaring the private network does not restore the found
|
||||
tunnel: the mesh stopped it, and nothing is started on the way out
|
||||
([ADR 0126](0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md)). A machine whose
|
||||
private network is unassigned has no tunnel until it is assigned again — which is what
|
||||
unassigning it means.
|
||||
|
||||
## Consequences
|
||||
|
||||
- On every machine that took a tunnel, the predecessor's tunnel configuration disappears at the
|
||||
first apply after the take is proven. Nothing a peer sees changes; the mesh's interface already
|
||||
carries the same key, port, address and peers.
|
||||
- A take that is never proven — no peer ever handshakes — keeps the found configuration, and the
|
||||
node says so, so a broken take is visible rather than silently retired.
|
||||
- Rolling back to the predecessor's tunnel becomes a deliberate act, in this order: **unassign the
|
||||
private network first**, then copy the kept original back and start its unit. The mesh does
|
||||
neither. While the private network is still assigned, the tunnel is the mesh's: a restored
|
||||
configuration is held and retired again at the next proven apply, and the found unit cannot
|
||||
bind the port the mesh's interface holds. The node says so when it happens.
|
||||
- A configuration something keeps writing back — the predecessor's own tooling, say — is retired
|
||||
again each time it appears, but the first original stays the one kept; a different content is
|
||||
kept once beside it, and the node reports that the configuration came back.
|
||||
- The host retires only the found interface's own configuration file (`/etc/wireguard/<iface>.conf`),
|
||||
never a path the mesh writes, and never a link: a configuration that is a link to somewhere else
|
||||
is left, with its target, for a person to retire.
|
||||
- 0105's "its configuration stays on disk, kept like any held file" holds until the take is proven,
|
||||
and not after.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0105](0105-the-mesh-adopts-the-predecessors-tunnel-in-place.md): the take, and why it keeps
|
||||
the found configuration during it
|
||||
- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md): kept originals
|
||||
- [ADR 0126](0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md): nothing is started on the way out
|
||||
- mesh-host `internal/apply/takeover.go`
|
||||
@@ -0,0 +1,136 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
date: 2026-09-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0112-a-module-definition-names-no-node-mesh-or-path.md
|
||||
---
|
||||
|
||||
# 120. A roster fact carries its format as a template: the mesh owns the data, the module owns the format
|
||||
|
||||
## Context
|
||||
|
||||
A **fact** is a thing only the mesh knows — which machines exist, what they are called, where they
|
||||
are — written into a file where a module asks for it. The mesh computes it from the graph; a module
|
||||
loads it, restarts on it, does what its software does with it. Facts replaced three modules that
|
||||
existed only because computed output needed somewhere to live and ran no software of their own
|
||||
([ADR 0040](0040-what-a-module-is.md)).
|
||||
|
||||
But the *format* lived in the control plane. A fact was a name from a closed list, and each name had
|
||||
a formatter written in Go beside the others: `node-names` wrote the roster as an `/etc/hosts` file,
|
||||
`node-zones` wrote it as a dnsmasq resolver's `local=`/`address=` lines. Adding a consumer meant
|
||||
adding a formatter — in the consumer's own configuration language — to the mesh.
|
||||
|
||||
The ssh work made the cost plain. An operator's `~/.ssh` wants three roster projections — a
|
||||
`known_hosts`, an ssh `config` of `Host` blocks, an `authorized_keys` — each in ssh's syntax. Under
|
||||
the closed list that is three more formatters in the control plane, teaching it ssh's configuration
|
||||
language. And it does not stop at ssh: every daemon that reads the roster in its own file format
|
||||
would put its grammar here. The control plane was accreting the configuration languages of software
|
||||
it does not run — the exact thing [ADR 0040](0040-what-a-module-is.md) says is a
|
||||
module's and not the mesh's.
|
||||
|
||||
The shape underneath is one shape. WireGuard's `[Peer]` blocks, `/etc/hosts`, dnsmasq's zones, an
|
||||
ssh `known_hosts` — all of them are *the roster, projected into a file*. Only the projection differs,
|
||||
and the projection belongs to whoever runs the software that reads it.
|
||||
|
||||
## Considered Options
|
||||
|
||||
**1. Keep the closed list; add a formatter per consumer.** Rejected. The control plane learns the
|
||||
configuration language of every daemon any module might run, without bound, and each format lives in
|
||||
the mesh rather than in the module that owns the file. A module cannot change how its own file is
|
||||
written without a control-plane change.
|
||||
|
||||
**2. A general placeholder vocabulary over `content`, like `${machine:address}` but for the
|
||||
roster.** Rejected. The mesh's other substitutions each resolve to *one* scalar — this machine's
|
||||
address, one provider's port. The roster is inherently a *repetition*: one block per machine. A flat
|
||||
`${…}` vocabulary cannot iterate, and a mechanism that could would be a template in all but name.
|
||||
|
||||
**3. The module gives a path and a template over the roster; the mesh renders it.** Chosen. The mesh
|
||||
owns the data — who exists, their names and addresses — and hands it to a Go `text/template` the
|
||||
module wrote. The mesh renders and reads neither the template's intent nor the file's meaning.
|
||||
|
||||
## Decision
|
||||
|
||||
**A fact is a path and a template.** In a module's manifest, `facts` maps a name the module chooses
|
||||
to a `{ path, template }`. The template is a Go `text/template` over a fixed **roster view**:
|
||||
|
||||
- `.Node` — this machine's bare name.
|
||||
- `.Suffix` — what a mesh name ends in (`internal`, or the operator's choice), as composed.
|
||||
- `.Names` — every name the mesh serves: the machines *and* the names it was told to route.
|
||||
- `.Machines` — only the machines that are nodes of this mesh.
|
||||
|
||||
Each of `.Names` and `.Machines` is a list of `{ Name, FQDN, Address }`. A machine the mesh has a
|
||||
record for but cannot yet place has no address and is left out of both — a name that resolves to
|
||||
nothing is a connection that hangs, so it is omitted rather than written (the same rule as before).
|
||||
|
||||
**The mesh owns the data; the module owns the format.** The control plane holds **no** formatter.
|
||||
The two built-in projections render through the same path any module uses:
|
||||
|
||||
- **`/etc/hosts`** is a template on the mesh's own network module. The mesh writes `/etc/hosts`
|
||||
because being on the private network is what gives a machine a name — but the *layout* is a
|
||||
template like any other, shipped with the control plane because that module ships with it, not
|
||||
because the control plane knows the hosts-file format.
|
||||
- **dnsmasq's zones** move into dnsmasq. The `local=`/`address=` grammar is dnsmasq's configuration
|
||||
language, and it now lives in dnsmasq's manifest, where the module that runs dnsmasq owns it.
|
||||
|
||||
**A fact also says whether its file is the mesh's whole or a region of the machine's.** A hosts file
|
||||
is the machine's — its `localhost`, the operator's lines, another tool's marked blocks — so
|
||||
`node-names` is `shared`: the mesh owns only its region and keeps the rest byte for byte, the host
|
||||
laying it down `into: block`
|
||||
([ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md), hq issue 128). A resolver's
|
||||
zones file is the mesh's whole, and is not shared. The template renders the content either way;
|
||||
`shared` decides how the host writes it. This composes with hq 128 rather than replacing it: the
|
||||
region *mechanism* is the host's, the region's *format* is the module's template.
|
||||
|
||||
**The names-vs-machines distinction is the template's choice** ([04-ISSUES/111](../04-ISSUES/111-the-resolver-is-told-names-the-mesh-serves-not-only-machines/00-report.md)):
|
||||
a container's hosts ranges `.Names`, so a routed name resolves to the machine serving it; a resolver
|
||||
told the mesh's suffix is its own ranges `.Machines`, or a routed name written there with the suffix
|
||||
appended is a name nobody will ever ask for.
|
||||
|
||||
**A template that will not render is refused at composition, not on a machine.** A template that does
|
||||
not parse, or reads a field the roster does not have, fails where the manifest is — the closed-list
|
||||
safety, moved from the fact's *name* to the roster's *shape*. A daemon that starts, reads a file the
|
||||
mesh could not render, and answers nothing is a much worse way to find out.
|
||||
|
||||
**WireGuard stays a computed generator, and that is the line.** Its `mesh0.conf` is not a pure roster
|
||||
projection — it carries topology the control plane decides: which peers are reachable, endpoints, hub
|
||||
forwarding, keepalive for a NAT'd node. And it is *foundational*: the overlay must be up before any
|
||||
module can be delivered, so the thing that writes it cannot itself be a delivered module. The line
|
||||
this draws: **the substrate that delivery rides on is the control plane's; everything layered on a
|
||||
working overlay is a roster template.** DNS, hosts, and ssh are layered; the overlay is the floor.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **ssh is two templates and no control-plane change.** Once the roster view carries a machine's ssh
|
||||
host key and its operator account ([to-be 29](../03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md)),
|
||||
`known_hosts`, the ssh `config`, and `authorized_keys` are templates on the ssh modules — the mesh
|
||||
gains no knowledge of ssh's syntax. This ADR is what makes that work land without touching the
|
||||
controller.
|
||||
- **A new roster projection never touches the control plane.** Any module that reads the roster in
|
||||
its own format ships its own template.
|
||||
- **A module can change how its own file is written** without a control-plane change — it is editing
|
||||
its own manifest.
|
||||
- **The schema changed and is not backward compatible.** A fact was a string (a path); it is now
|
||||
`{ path, template }`. The old string form has no template and cannot be auto-upgraded, because the
|
||||
format it implied was the formatter this ADR deletes. The controller and every catalogue module
|
||||
using facts — only dnsmasq — land together. A controller and a catalogue that disagree cannot
|
||||
compose the module: the running daemon on a machine is unaffected, but the mesh will not send it a
|
||||
new declaration until both sides agree.
|
||||
- **The output did not change.** The `/etc/hosts` and dnsmasq zones a machine receives are
|
||||
byte-for-byte what the deleted formatters wrote, pinned by tests that render the built-in template
|
||||
and compose the real dnsmasq manifest.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0040](0040-what-a-module-is.md): a module is software the mesh runs — a
|
||||
format the mesh knows for software it does not run was the accretion this stops
|
||||
- [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md): a module definition names no
|
||||
path; this is its sibling for content — a module definition names no format the mesh must know
|
||||
- [to-be 29](../03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md): the ssh consumer this
|
||||
unblocks, and the roster fields it will add
|
||||
- [04-ISSUES/111](../04-ISSUES/111-the-resolver-is-told-names-the-mesh-serves-not-only-machines/00-report.md): every served name is not a
|
||||
machine — now the template's choice of `.Names` or `.Machines`
|
||||
- mesh-controller `internal/catalogue/roster.go` (the mechanism), `internal/overlay/generator.go`
|
||||
(the built-in `/etc/hosts` template), `internal/catalogue/manifest.go` (`RosterFile`)
|
||||
- mesh-catalog `modules/dnsmasq/module.json` (the zones template, dnsmasq's own)
|
||||
+132
@@ -0,0 +1,132 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
date: 2026-09-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0110-a-seat-is-a-module-assignment-from-a-closed-set.md
|
||||
---
|
||||
|
||||
# 121. A system seat is named for its scope, and a module may define its own
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) made seats a closed set the
|
||||
control plane defines: a well-formed name no longer becomes a seat by being claimed, so a person can
|
||||
read what a mesh can have and who fills each role. It left two things unsettled that the growing set
|
||||
now exposes:
|
||||
|
||||
- **The names carry no rule.** `mesh-controller`, `mesh-store`, `mesh-broker` are named for the mesh;
|
||||
beside them sit `the-artifact-store`, `the-build-machine`, `the-dns-port`, `the-showcase`,
|
||||
`the-uplink` — a second naming style with no principle behind it. A reader cannot tell a seat's
|
||||
scope from its name, and the mesh's own roles do not look like the mesh's.
|
||||
- **The set is the *only* place a seat may be defined.** A module claiming any name not in the
|
||||
control plane's set is refused. That is right for *system* roles — one broker, one packet filter
|
||||
per node — but it means a module can never define a role of its own: a demo module's
|
||||
`the-showcase`, a future application's coordination role, must be smuggled into the control plane's
|
||||
set or not exist. The control plane ends up holding roles that are not the mesh's to define.
|
||||
|
||||
Reviewing the set against these also found seats whose *scope* or *membership* is wrong, not just
|
||||
their name — the review is the occasion to fix those too.
|
||||
|
||||
## Decision
|
||||
|
||||
**A system seat — one the control plane defines — is named for its scope:**
|
||||
|
||||
- **`mesh-*`** for a mesh-scoped seat: one holder in the whole mesh, a role the mesh has once
|
||||
(`mesh-controller`, `mesh-store`, `mesh-broker`, `mesh-git`, …). A `mesh-*` seat is always held by
|
||||
a module **on a named node** — `mesh-git` is gitea *on novox*, not "gitea"; another node running
|
||||
gitea does not hold `mesh-git` unless it is the holder. The seat is the mesh's single answer for
|
||||
the role, and which node answers is part of what the seat records.
|
||||
- **`node-*`** for a node-scoped seat: one holder per node, a role each machine has at most once
|
||||
(`node-packet-filter`, `node-intrusion-prevention`, `node-uplink`, …).
|
||||
|
||||
The three already-`mesh-*` seats keep their names; the rest are renamed by this rule. The scope a
|
||||
name declares must match the seat's actual scope — a `mesh-*` seat at node scope, or the reverse, is
|
||||
a contradiction the reader is entitled to trust is impossible.
|
||||
|
||||
**The control plane defines only system seats. A module may define its own.** A seat named `mesh-*`
|
||||
or `node-*` is the control plane's, and claiming one the control plane does not define is refused as
|
||||
before. Any *other* name is a **module-defined seat**: valid when the module declaring the claim also
|
||||
declares the seat (its name, scope, and — if any — the protocol its holder speaks). The control plane
|
||||
enforces one-holder-per-scope for it exactly as for its own, but does not otherwise know what it
|
||||
means. So an application can coordinate its own instances through a seat of its own, and the mesh's
|
||||
closed set stays what its name says it is: the *system's* roles, not everyone's.
|
||||
|
||||
**Specific seats this settles:**
|
||||
|
||||
- **`the-build-machine` → `mesh-build-machine`, and its scope becomes mesh.** There is one build
|
||||
machine in the mesh (the builder on novox), not one per node. Node scope said the opposite. It
|
||||
delivers no provision; it is the mesh's single build machine.
|
||||
- **`the-private-network` → `mesh-private-network`, held by the network *server* on one node.** Today
|
||||
it is node-scoped and held on every node, with a stated (untested) story that a different VPN could
|
||||
hold it per machine — which would force every provider module to independently implement receiving
|
||||
and applying the controller-composed configuration. The mesh does not work that way and should not
|
||||
pretend to: **one mesh decides one private network.** The seat is mesh-scoped, held by the server
|
||||
module (WireGuard on the hub, novox). A machine that joins is given a **client module** that
|
||||
receives the composed configuration and applies it; when a node joins, the mesh emits each node's
|
||||
configuration so all of them know each other at once. This drops per-node VPN choice deliberately —
|
||||
the private network is nox-mesh's own, and it defines the nodes' configuration rather than being
|
||||
assembled from each node's opinion. (Implementation: the overlay generator's per-node computation
|
||||
is unchanged; what changes is the seat's scope and the server/client split of the module.)
|
||||
- **`the-showcase` → removed from the set; it becomes a module-defined seat.** It is a demo module's
|
||||
own coordination role, claimed by nothing else and held nowhere. It is the first module-defined
|
||||
seat, and the reason the rule above is needed rather than hypothetical.
|
||||
- **`the-dns-port` → `node-dns-resolver`** (the daemon that binds `:53`), kept distinct from
|
||||
**`the-resolver-configuration` → `node-resolver-config`** (what writes `resolv.conf`). Two roles,
|
||||
two seats; the rename must not blur them.
|
||||
- **`the-packet-filter` → `node-packet-filter`** and **`the-intrusion-prevention` →
|
||||
`node-intrusion-prevention`** — names kept as-is but for the prefix. "Packet filter" stays distinct
|
||||
from "firewall", which would swallow intrusion-prevention too.
|
||||
- **`the-uplink` → `node-uplink`** ([ADR 0125](0117-a-machines-uplink-is-a-seat.md)). Unheld, so it
|
||||
renames with no migration.
|
||||
- **The registry seats — `the-artifact-store`, `npm-package-registry` (→ `mesh-artifact-store`,
|
||||
`mesh-npm-package-registry`) — and `git` (→ `mesh-git`) — are decided but deferred.** They each
|
||||
*deliver* a provision, so renaming them is a delivering-seat migration: a holder that stops
|
||||
resolving mid-flight takes a provision away from every consumer. That risk is not worth carrying in
|
||||
the same pass as the node-* renames, so they keep their names until done deliberately.
|
||||
|
||||
**`distribution` stays the mesh's registry; only `verdaccio` is retired.** An earlier draft of this
|
||||
record had the registry consolidating onto gitea and `distribution` retired — that was reversed:
|
||||
`distribution` is the standalone OCI registry serving every `artifact-store://…@sha256` image (the
|
||||
control plane's own included), and the mesh keeps it. `verdaccio` was a *second* npm registry;
|
||||
gitea already provides `npm-package-registry`, so verdaccio is redundant and is removed. It is only
|
||||
in the catalogue (never registered in the running mesh), so removing it is deleting the module — no
|
||||
migration, nothing to strand.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **A reader learns a seat's scope from its name.** `mesh-*` is mesh-wide and one; `node-*` is
|
||||
per-machine. The mesh's own roles finally look like the mesh's.
|
||||
- **Applications get their own seats** without the control plane learning their meaning. The closed
|
||||
set shrinks to what it should be — the system's roles — and stops being where unrelated roles hide.
|
||||
- **The renames are a coordinated migration, not a rename.** A held seat's name lives in three places
|
||||
that must move together: the control plane's set (`seats.go`), every claiming manifest, and what
|
||||
each node reports it holds (re-derived by re-registering the manifest and re-pushing). A seat
|
||||
renamed in one place and not the others stops resolving to its holder — and for a *delivering* seat
|
||||
(`mesh-store`→postgres, `mesh-broker`→amqp, the registry seats) that is a mesh-wide provision
|
||||
outage, the same failure mode as a schema change hitting an old manifest. So: the non-delivering
|
||||
`node-*` seats and `mesh-build-machine` migrate as one tested controller+catalogue change;
|
||||
`node-uplink` is free (unheld); the delivering registry seats are deferred to their own pass.
|
||||
- **The node-* migration was done as one controlled step, and it froze briefly.** Deploying the new
|
||||
controller made it reject the still-old-named claims in the stored manifests, so composition stopped
|
||||
for the affected nodes until each manifest was re-registered under its new name; running services
|
||||
were untouched, and the window was seconds. This is the coordinated-migration cost named above,
|
||||
paid once — and the reason the *delivering* registry seats, whose freeze would be a provision
|
||||
outage rather than a compose pause, are not folded into the same pass.
|
||||
- **`distribution` is not retired.** It stays as the registry; only `verdaccio` (a redundant second
|
||||
npm registry) is removed. The mesh keeps one OCI registry (`distribution`) and gitea for npm/git —
|
||||
the "one registry, on gitea" idea was considered and dropped.
|
||||
- **The private network stops pretending to be swappable per node.** The gain is a coherent
|
||||
server/client model matching how the controller already composes configuration; the cost is that
|
||||
choosing a different VPN is now a mesh-wide change, not a per-node one — accepted.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) — the closed set this refines
|
||||
- [ADR 0125](0117-a-machines-uplink-is-a-seat.md) — `the-uplink`, renamed here to `node-uplink`
|
||||
- [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md) — the original `mesh-*` seats
|
||||
whose naming this generalises
|
||||
- [to-be 26](../03-DESIGN/01-to-be/26-the-seats.md) — the seat table, updated by this
|
||||
- mesh-controller `internal/catalogue/seats.go` (the set and claim validation),
|
||||
`internal/overlay/generator.go` (the private network as server + client)
|
||||
@@ -0,0 +1,107 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
date: 2026-09-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
supersedes-in-part:
|
||||
- 0110-a-seat-is-a-module-assignment-from-a-closed-set.md
|
||||
- 0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md
|
||||
---
|
||||
|
||||
# 122. A seat is data the controller owns, and a rename is a database update
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) made the seats a closed set the
|
||||
control plane defines, and [ADR 0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md)
|
||||
named them by scope. Both were right about *what* a seat is. Both left it defined the wrong *way*:
|
||||
**the set is a hardcoded Go slice compiled into the controller, and everything references a seat by
|
||||
its name as a string literal.** Renaming `the-packet-filter` to `node-packet-filter` this session
|
||||
took, in one pass:
|
||||
|
||||
- an edit to the Go slice in `internal/catalogue/seats.go`, recompiled into a new controller image;
|
||||
- an edit to a `const gitSeat = "git"` in *production* control-plane code (`source.go`), because a
|
||||
seat's name was hardcoded where a repository's home is resolved;
|
||||
- edits to every claiming manifest in the catalogue, each re-registered;
|
||||
- a controller **rebuild and redeploy**, which — because the running controller then refused the
|
||||
still-old-named claims in stored manifests — **froze composition** for the affected nodes until
|
||||
each manifest was re-registered under its new name;
|
||||
- the same coupling in the **build machine**, which embeds the same seat set and refused to build
|
||||
anything claiming a name it did not yet know;
|
||||
- a **deadlock** when the build machine's own seat was renamed, since the old builder could not
|
||||
build the new builder whose manifest claimed a name it rejected.
|
||||
|
||||
None of that is what a rename should cost. A rename is the operator changing a label. It should be a
|
||||
single write, and nothing should have to be rebuilt, refused, or unfrozen. The set being *closed*
|
||||
(0110) and *named by scope* (0121) are good rules; **the set being code is the mistake.** When
|
||||
adhering to the design means twenty steps and a `const` in the resolver, the design is what to fix.
|
||||
|
||||
## Decision
|
||||
|
||||
**The seat set is data the control plane owns, not code it is compiled from.** The seats live in a
|
||||
table in the controller's store — one row per seat: a **stable id**, a `name`, a `scope`, what it
|
||||
`delivers` (a provision, or nothing), and the record that decided it. The rows are seeded by a
|
||||
migration (the closed set 0110 defines still ships with the mesh), and thereafter they are ordinary
|
||||
data the control plane reads and writes.
|
||||
|
||||
**A seat is referenced by its stable id, never by its name.** A claim, a held-seat record, and any
|
||||
control-plane code that must name a seat (the git-seat resolver, the artifact-store guard) hold the
|
||||
**id**. The `name` is a label for people and for what a manifest writes; it is resolved to an id
|
||||
once, when a claim is registered. So:
|
||||
|
||||
- **A rename is one `UPDATE seats set name = … where id = …`.** Nothing is recompiled, nothing is
|
||||
re-registered, nothing is refused, nothing freezes. Held records and claims already point at the
|
||||
id, so they follow the rename for free. The build machine is not involved, because the build
|
||||
machine validates a claim against the set it reads from the mesh, not one baked into its image.
|
||||
- **Adding or removing a seat is an `INSERT`/`DELETE`** (within the closed-set discipline: a change
|
||||
to the set is still a decision with a record — the record is now a row's `decided` column and an
|
||||
ADR, not a line of Go). No controller release is needed to change the roster of roles.
|
||||
- **Production code stops hardcoding names.** `const gitSeat = "git"` becomes a lookup of the seat
|
||||
that delivers the `git` provision (or a well-known id), so renaming its label cannot break the
|
||||
code that finds a repository's forge.
|
||||
|
||||
**What does not change** (0110 and 0121 still hold): a seat is still a module assignment from a
|
||||
closed set; there is still one holder per scope; a delivering seat is still the single answer for
|
||||
its provision; system seats are still `mesh-*`/`node-*` and a module may still define its own. Only
|
||||
their *storage and reference* change — from a compiled slice keyed by name to a table keyed by id.
|
||||
|
||||
**A manifest still claims by name, and that is fine.** A manifest is written by a person and names
|
||||
the seat in words; the mesh resolves the name to an id at registration and stores the id. If a
|
||||
seat's name changes, manifests written against the old name are updated in the catalogue like any
|
||||
other edit (and the mesh can keep the old name as an alias row during a transition so nothing breaks
|
||||
in the window) — but the *control plane* never has to change or redeploy for it, which is the whole
|
||||
point. The heavy, mesh-wide, freeze-prone half of a rename disappears; only the ordinary catalogue
|
||||
edit remains.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **A rename, and a set change, become operations, not releases.** The pain this session paid —
|
||||
three freezes, a builder deadlock, hand-resolved manifests — is designed out. The seat migrations
|
||||
still outstanding (the delivering registry seats, and the private network's scope change) should
|
||||
wait for this: done as data, each is a write, not a coupled multi-repo deploy.
|
||||
- **The controller gains a small table and a seed migration**, and its seat lookups change from
|
||||
slice scans to id-keyed reads. `SeatNamed`, `SeatDelivering`, `claimProblems` read the table.
|
||||
- **The build machine reads the set from the mesh** (it already talks to the control plane), rather
|
||||
than embedding it — which removes the controller/builder seat coupling that made every breaking
|
||||
seat change a two-sided deadlock (see [to-be 30](../03-DESIGN/01-to-be/30-the-mesh-updates-itself-on-a-push.md)).
|
||||
- **The closed set is still closed.** Data being editable is not the set being open: changing it is
|
||||
still a decision, still recorded. What changes is that recording it no longer means shipping a
|
||||
binary.
|
||||
- **This is a real refactor**, touching the store schema, the seat lookups, claim registration
|
||||
(name→id resolution), and the held-seat records. It is worth its own build; until it lands, the
|
||||
current compiled set stands and further renames are held rather than forced through the heavy path.
|
||||
- **Config on a seat is still the module's** (the question that surfaced this): a seat row carries
|
||||
the seat's own metadata (scope, delivers, protocol), not a module's configuration — that stays in
|
||||
the holding module's manifest ([ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md)). Making
|
||||
seats data does not make them a config store.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md),
|
||||
[ADR 0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md) — the seat
|
||||
rules this keeps, whose *storage* it changes
|
||||
- [to-be 30](../03-DESIGN/01-to-be/30-the-mesh-updates-itself-on-a-push.md) — the controller/builder
|
||||
seat coupling and the breaking-change freeze this removes for seat changes
|
||||
- mesh-controller `internal/catalogue/seats.go` (the compiled slice this replaces),
|
||||
`cmd/mesh-controller/source.go` (`const gitSeat`, the hardcoded name this removes)
|
||||
+2
-2
@@ -1,14 +1,14 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: superseded
|
||||
superseded-by: 02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md
|
||||
superseded-by: 02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md
|
||||
date: 2026-09-26
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
---
|
||||
|
||||
# 117. The bus is the only broker
|
||||
# 125. The bus is the only broker
|
||||
|
||||
## Context
|
||||
|
||||
+4
-4
@@ -4,10 +4,10 @@ status: accepted
|
||||
date: 2026-09-26
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
supersedes: 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md
|
||||
extends: 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md
|
||||
---
|
||||
|
||||
# 118. A module declares its own seats; the mesh reserves its own
|
||||
# 126. A module declares its own seats; the mesh reserves its own
|
||||
|
||||
## Context
|
||||
|
||||
@@ -18,7 +18,7 @@ controller's own code, and when that enumeration was done by hand while writing
|
||||
reported eleven claims where there were thirteen.** The fix was a table in the controller, and
|
||||
adding a seat became a decision.
|
||||
|
||||
What that table cannot express is the architecture [ADR 0117](0117-the-bus-is-the-only-broker.md)
|
||||
What that table cannot express is the architecture [ADR 0125](0125-the-bus-is-the-only-broker.md)
|
||||
opened. With one bus and no private brokers, a module offering a service to other modules offers
|
||||
it as **a role on the bus**: a set of subjects, exactly one holder, addressed by what it does
|
||||
rather than by which module or node provides it. A telegram sender, a licensing master, anything
|
||||
@@ -139,7 +139,7 @@ protocols is the failure nobody could diagnose afterwards.
|
||||
|
||||
- [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) — superseded here; its
|
||||
requirement is kept and only its mechanism replaced.
|
||||
- [ADR 0117](0117-the-bus-is-the-only-broker.md) — one bus, which is what makes a role addressable.
|
||||
- [ADR 0125](0125-the-bus-is-the-only-broker.md) — one bus, which is what makes a role addressable.
|
||||
- [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md) — the naming convention
|
||||
the reserved prefix restores.
|
||||
- [ADR 0041](0041-events-are-a-relationship.md) — the event half of the boundary drawn here.
|
||||
+5
-5
@@ -4,14 +4,14 @@ status: accepted
|
||||
date: 2026-09-26
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
supersedes: 02-DECISIONS/0117-the-bus-is-the-only-broker.md
|
||||
supersedes: 02-DECISIONS/0125-the-bus-is-the-only-broker.md
|
||||
---
|
||||
|
||||
# 119. AMQP is a provision, not the bus
|
||||
# 127. AMQP is a provision, not the bus
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0117](0117-the-bus-is-the-only-broker.md) decided that the bus is the only broker, and went
|
||||
[ADR 0125](0125-the-bus-is-the-only-broker.md) decided that the bus is the only broker, and went
|
||||
one step further than it had grounds for: it also decided that the `amqp` **interface** — a module
|
||||
requiring a message broker of its own — "is not carried forward" and "retires with the
|
||||
compatibility broker rather than gaining a successor", with the two modules declaring it converted
|
||||
@@ -95,9 +95,9 @@ it is removed. They require a backing service and a provider answers.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0117](0117-the-bus-is-the-only-broker.md) — superseded; its ruling on the bus is kept
|
||||
- [ADR 0125](0125-the-bus-is-the-only-broker.md) — superseded; its ruling on the bus is kept
|
||||
whole and only its ruling on the interface is reversed.
|
||||
- [ADR 0106](0106-the-bus-is-nats.md) — the bus is NATS; its compatibility-broker framing is
|
||||
corrected here.
|
||||
- [ADR 0118](0118-a-module-declares-its-own-seats.md) — seats, including the one the NATS server
|
||||
- [ADR 0126](0126-a-module-declares-its-own-seats.md) — seats, including the one the NATS server
|
||||
now holds alone.
|
||||
+9
-9
@@ -4,14 +4,14 @@ status: accepted
|
||||
date: 2026-09-26
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md
|
||||
extends: 02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md
|
||||
---
|
||||
|
||||
# 120. The mesh bus is required, not ambient
|
||||
# 128. The mesh bus is required, not ambient
|
||||
|
||||
## Context
|
||||
|
||||
[Design 29](../03-DESIGN/01-to-be/29-what-a-module-declares.md) opened by saying the bus is
|
||||
[Design 29](../03-DESIGN/01-to-be/32-what-a-module-declares.md) opened by saying the bus is
|
||||
*ambient*: "No module requires it, the way no module requires a filesystem. Every module gets a
|
||||
connection and an identity whether it asks or not."
|
||||
|
||||
@@ -26,7 +26,7 @@ provisioning already does well: a consumer names where a credential lands, the m
|
||||
there, and rotation and removal follow the same path as every other credential.
|
||||
|
||||
**The argument that made the bus ambient was narrower than it looked.**
|
||||
[ADR 0117](0117-the-bus-is-the-only-broker.md) reasoned that bus accounts cannot be provisioned
|
||||
[ADR 0125](0125-the-bus-is-the-only-broker.md) reasoned that bus accounts cannot be provisioned
|
||||
because a provisioner is itself a module that needs an account before it can run. That is true of
|
||||
a **provisioner process**, and it is not true of a provision: the mesh's bus accounts are composed
|
||||
by the *controller*, into configuration, and the controller is not waiting on a bus account to
|
||||
@@ -65,14 +65,14 @@ grants no subject; declaring a subject without requiring the bus is refused at r
|
||||
incoherent.
|
||||
|
||||
**The `mesh-bus` provision is answered by the controller, not by a provisioner.** This is the
|
||||
surviving kernel of ADR 0117's bootstrap argument, narrowed to what it actually supports: the
|
||||
surviving kernel of ADR 0125's bootstrap argument, narrowed to what it actually supports: the
|
||||
bus's accounts are configuration the controller composes and the server reloads
|
||||
([ADR 0106](0106-the-bus-is-nats.md) — never through a management API), so there is no provisioner
|
||||
process in the path and nothing waiting on a bus account to create bus accounts. It is a provision
|
||||
whose provider is the mesh itself.
|
||||
|
||||
**A module may also provide a NATS server of its own, and that is a different interface.** Exactly
|
||||
as the AMQP broker provides `amqp` ([ADR 0119](0119-amqp-is-a-provision-not-the-bus.md)), a module
|
||||
as the AMQP broker provides `amqp` ([ADR 0127](0127-amqp-is-a-provision-not-the-bus.md)), a module
|
||||
may run its own NATS and offer it as a backing service. That interface is **`nats`**; the mesh's
|
||||
own bus is **`mesh-bus`**; the two are never the same name, because a manifest that said `nats`
|
||||
could mean either and the difference is the whole architecture. The rule from 0119 decides which
|
||||
@@ -83,7 +83,7 @@ is legitimate: a private bus is a backing service, never a channel to another mo
|
||||
- **Design 29's opening is reversed.** The bus is not ambient; it is required, and the document's
|
||||
first paragraph says the opposite of this.
|
||||
- **The seat's `Delivers` is `mesh-bus`** — corrected twice in one day, which is worth recording
|
||||
rather than tidying: it read `amqp`, which was the old broker's interface; ADR 0117 emptied it,
|
||||
rather than tidying: it read `amqp`, which was the old broker's interface; ADR 0125 emptied it,
|
||||
on the reasoning that a bus cannot be provisioned; and it is neither. The seat delivers the
|
||||
mesh's bus.
|
||||
- **`own-secrets: { broker: ... }` is retired** in favour of the provision's own delivery, across
|
||||
@@ -107,9 +107,9 @@ is legitimate: a private bus is a backing service, never a channel to another mo
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0117](0117-the-bus-is-the-only-broker.md) — superseded by 0119; its bootstrap argument is
|
||||
- [ADR 0125](0125-the-bus-is-the-only-broker.md) — superseded by 0119; its bootstrap argument is
|
||||
narrowed here to the case it supports.
|
||||
- [ADR 0119](0119-amqp-is-a-provision-not-the-bus.md) — a broker as a backing service; this
|
||||
- [ADR 0127](0127-amqp-is-a-provision-not-the-bus.md) — a broker as a backing service; this
|
||||
applies the same shape to the mesh's own bus and separates the two names.
|
||||
- [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md) — authority from
|
||||
declarations, which this leaves untouched.
|
||||
+14
-3
@@ -4,14 +4,14 @@ status: accepted
|
||||
date: 2026-09-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0118-a-module-declares-its-own-seats.md
|
||||
extends: 02-DECISIONS/0122-a-seat-is-data-a-rename-is-a-database-update.md
|
||||
---
|
||||
|
||||
# 121. A seat carries the protocol of its role
|
||||
# 129. A seat carries the protocol of its role
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0118](0118-a-module-declares-its-own-seats.md) let a module declare a seat with its protocol:
|
||||
[ADR 0126](0126-a-module-declares-its-own-seats.md) let a module declare a seat with its protocol:
|
||||
what work the role accepts, what it emits, what it serves. A module's own seats work that way today.
|
||||
**The mesh's own seats — the `mesh-*` set — carry no protocol at all**, only a name, a scope and the
|
||||
provision they deliver. They say who does a job and nothing about what may be said to them or by
|
||||
@@ -71,6 +71,17 @@ the mesh has a word for a role.
|
||||
any asker needs a grant across the whole inbox space, which is the one grant design 25 §4 refuses by
|
||||
name. The seat's event costs the asker a filter and costs the mesh nothing.
|
||||
|
||||
## Reconciled with 0122, which landed in parallel
|
||||
|
||||
*Added 2026-09-27, on merging.* [ADR 0122](0122-a-seat-is-data-a-rename-is-a-database-update.md) moved
|
||||
the seat set out of compiled code and into a table the controller owns. This record was written against
|
||||
the slice, and says the `mesh-*` set "gains the same three fields a declared seat has".
|
||||
|
||||
**The decision is unaffected and the mechanism is better for it.** What a seat accepts, emits and serves
|
||||
becomes three columns beside its name and scope, so giving a role a protocol is a write rather than a
|
||||
rebuild — which is the whole argument of 0122 applied to the thing this record adds. Where this text
|
||||
says the set gains fields, read: the table gains columns.
|
||||
|
||||
## Consequences
|
||||
|
||||
**A seat is now the mesh's unit of "a role that talks".** A role that accepts work, announces outcomes
|
||||
+4
-4
@@ -4,14 +4,14 @@ status: accepted
|
||||
date: 2026-09-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md
|
||||
extends: 02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md
|
||||
---
|
||||
|
||||
# 122. The predecessor is ending, and its broker goes with it
|
||||
# 130. The predecessor is ending, and its broker goes with it
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0119](0119-amqp-is-a-provision-not-the-bus.md) settled that the old broker is an ordinary
|
||||
[ADR 0127](0127-amqp-is-a-provision-not-the-bus.md) settled that the old broker is an ordinary
|
||||
provider of the `amqp` provision rather than a compatibility module with an end date. It rejected
|
||||
giving it a retirement condition, and said why: *"its clients are not only the predecessor's, so the
|
||||
retirement condition describes a day that will not come."*
|
||||
@@ -28,7 +28,7 @@ because three documents reason from the premise it overturns:
|
||||
## Decision
|
||||
|
||||
**The predecessor's broker retires when nothing requires `amqp`, by being unassigned like any other
|
||||
provider.** No retirement condition, no end-date machinery, no special case — which is ADR 0119 being
|
||||
provider.** No retirement condition, no end-date machinery, no special case — which is ADR 0127 being
|
||||
paid off rather than revised. Because that record made the broker an ordinary provider, ending it
|
||||
needs nothing that does not already exist: a provision with no consumers has its provider unassigned,
|
||||
and the module system has done that since it existed.
|
||||
+13
-7
@@ -133,11 +133,12 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0105** — [The mesh adopts the predecessor's tunnel in place](0105-the-mesh-adopts-the-predecessors-tunnel-in-place.md)
|
||||
- **0106** — [The bus is NATS](0106-the-bus-is-nats.md)
|
||||
- **0116** — [The bus is built in five steps, and the protocol moves with it](0116-the-bus-is-built-in-five-steps.md)
|
||||
- **0117** — [The bus is the only broker](0117-the-bus-is-the-only-broker.md) *(superseded)*
|
||||
- **0119** — [AMQP is a provision, not the bus](0119-amqp-is-a-provision-not-the-bus.md)
|
||||
- **0120** — [The mesh bus is required, not ambient](0120-the-mesh-bus-is-required-not-ambient.md)
|
||||
- **0121** — [A seat carries the protocol of its role](0121-a-seat-carries-the-protocol-of-its-role.md)
|
||||
- **0122** — [The predecessor is ending, and its broker goes with it](0122-the-predecessor-is-ending-and-its-broker-goes-with-it.md)
|
||||
- **0119** — [A taken tunnel's predecessor is retired once the take is proven](0119-a-taken-tunnels-predecessor-is-retired.md)
|
||||
- **0125** — [The bus is the only broker](0125-the-bus-is-the-only-broker.md) *(superseded)*
|
||||
- **0127** — [AMQP is a provision, not the bus](0127-amqp-is-a-provision-not-the-bus.md)
|
||||
- **0128** — [The mesh bus is required, not ambient](0128-the-mesh-bus-is-required-not-ambient.md)
|
||||
- **0129** — [A seat carries the protocol of its role](0129-a-seat-carries-the-protocol-of-its-role.md)
|
||||
- **0130** — [The predecessor is ending, and its broker goes with it](0130-the-predecessor-is-ending-and-its-broker-goes-with-it.md)
|
||||
|
||||
### Its tiers, from the bottom up
|
||||
|
||||
@@ -168,7 +169,7 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0098** — [A fact a provider makes at first start is fetched from it, not carried in its manifest](0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md)
|
||||
- **0108** — [A route carries the policy applied to a request, and names a secret rather than holding one](0108-a-route-carries-the-policy-applied-to-a-request.md)
|
||||
- **0109** — [A package registry seat is one per ecosystem, not one for all of them](0109-a-package-registry-seat-is-one-per-ecosystem.md)
|
||||
- **0118** — [A module declares its own seats; the mesh reserves its own](0118-a-module-declares-its-own-seats.md)
|
||||
- **0126** — [A module declares its own seats; the mesh reserves its own](0126-a-module-declares-its-own-seats.md)
|
||||
|
||||
### What runs on them, and how it gets there
|
||||
|
||||
@@ -200,11 +201,16 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0087** — [A seeded file is created once, and what grows in it is not the mesh's](0087-a-seeded-file-is-created-once.md)
|
||||
- **0091** — [A mount is declared, and there are three things it can be](0091-a-mount-is-declared-three-ways.md)
|
||||
- **0099** — [A step that runs once names what it reads, and runs again when it changed](0099-a-step-that-runs-once-names-what-it-reads.md)
|
||||
- **0110** — [A seat is held by one assignment, from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) *(superseded)*
|
||||
- **0110** — [A seat is held by one assignment, from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)
|
||||
- **0112** — [A module definition names no node, no mesh and no path: everything it needs is a requirement the mesh resolves](0112-a-module-definition-names-no-node-mesh-or-path.md) *(proposed)*
|
||||
- **0113** — [The vault makes every shared secret, a provider makes resources and data, and the mesh carries both](0113-the-vault-makes-every-secret.md) *(proposed)*
|
||||
- **0114** — [A credential two parties hold rotates over two credentials; one a single party holds rotates in place, staged; and retiring a credential never removes what it reached](0114-a-shared-credential-rotates-over-two-credentials.md) *(proposed)*
|
||||
- **0115** — [One assignment of a module per node: the module's name is the assignment's identity](0115-one-assignment-of-a-module-per-node.md) *(proposed)*
|
||||
- **0117** — [A machine's uplink is a seat: the mesh configures the manager, never the link](0117-a-machines-uplink-is-a-seat.md)
|
||||
- **0118** — [Undeclaring removes what the mesh made, and gives a unit back the state it was found in](0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md)
|
||||
- **0120** — [A roster fact carries its format as a template: the mesh owns the data, the module owns the format](0120-a-roster-fact-carries-its-format-as-a-template.md)
|
||||
- **0121** — [A system seat is named for its scope, and a module may define its own](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md)
|
||||
- **0122** — [A seat is data the controller owns, and a rename is a database update](0122-a-seat-is-data-a-rename-is-a-database-update.md)
|
||||
|
||||
### How it is built
|
||||
|
||||
|
||||
@@ -7,11 +7,12 @@ code:
|
||||
- mesh-controller internal/identity/authority.go
|
||||
- mesh-host internal/identity/serving.go
|
||||
- mesh-host internal/apply (the service that reflects a rule set)
|
||||
updated: 2026-09-25
|
||||
updated: 2026-09-27
|
||||
decisions:
|
||||
- 02-DECISIONS/0104-a-provision-may-be-answered-by-an-adapter-to-the-predecessor.md
|
||||
- 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
- 02-DECISIONS/0105-the-mesh-adopts-the-predecessors-tunnel-in-place.md
|
||||
- 02-DECISIONS/0119-a-taken-tunnels-predecessor-is-retired.md
|
||||
- 02-DECISIONS/0108-a-route-carries-the-policy-applied-to-a-request.md
|
||||
- 02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md
|
||||
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
@@ -768,6 +769,12 @@ where a found tunnel is left running beside the mesh's; where it is adopted ther
|
||||
The guard admits the mesh's ports from that one interface, and the predecessor's peers arrive on
|
||||
it.
|
||||
|
||||
*2026-09-27, [ADR 0127](../../02-DECISIONS/0119-a-taken-tunnels-predecessor-is-retired.md).* The
|
||||
found configuration is kept only until the take is proven — the found unit down, the mesh's
|
||||
interface up and handshaking with a peer. Then it is removed from where the found unit reads it
|
||||
(its original stays kept), the hold ends, and the predecessor's tunnel cannot be raised again by
|
||||
anything but a person restoring it by hand. Undeclaring the private network does not bring it back.
|
||||
|
||||
## The bus is NATS
|
||||
|
||||
*2026-09-23, [ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md). Architecture to be written
|
||||
|
||||
@@ -9,7 +9,7 @@ updated: 2026-09-26
|
||||
decisions:
|
||||
- 02-DECISIONS/0095-the-control-plane-is-the-way-to-ask-a-module.md
|
||||
- 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
- 02-DECISIONS/0120-the-mesh-bus-is-required-not-ambient.md
|
||||
- 02-DECISIONS/0128-the-mesh-bus-is-required-not-ambient.md
|
||||
- 02-DECISIONS/0116-the-bus-is-built-in-five-steps.md
|
||||
- 02-DECISIONS/0074-the-wire-is-specified-not-the-types.md
|
||||
- 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md
|
||||
@@ -78,7 +78,7 @@ which is what the identity rule below exists to prevent.
|
||||
|
||||
The credential itself is fetched, never carried in a declaration: a declaration is persisted as
|
||||
state and a sealed secret in a stream is an archive rather than a moment
|
||||
([design 29](29-what-a-module-declares.md) §10).
|
||||
([design 29](32-what-a-module-declares.md) §10).
|
||||
|
||||
### Connecting
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ updated: 2026-09-25
|
||||
decisions:
|
||||
- 02-DECISIONS/0084-which-provider-serves-a-consumer.md
|
||||
- 02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md
|
||||
- 02-DECISIONS/0118-a-module-declares-its-own-seats.md
|
||||
- 02-DECISIONS/0126-a-module-declares-its-own-seats.md
|
||||
---
|
||||
|
||||
# 23 — Choosing a provider
|
||||
@@ -54,7 +54,7 @@ to that provider and not to whichever one is nearest.
|
||||
**A seat names the mesh's one provider of a kind.** Where a seat delivers the provision, its holder
|
||||
answers for it when several providers exist and the consumer named none. That is not picking: the
|
||||
choice was made once, mesh-wide, by assigning the holder, rather than once per consumer by naming it
|
||||
([ADR 0118](../../02-DECISIONS/0118-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)),
|
||||
([ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)),
|
||||
[26 — The seats](26-the-seats.md)). A named provider still wins over the seat, because a consumer
|
||||
coupled to particular contents has said so.
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ code:
|
||||
updated: 2026-09-27
|
||||
decisions:
|
||||
- 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
- 02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md
|
||||
- 02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md
|
||||
- 02-DECISIONS/0116-the-bus-is-built-in-five-steps.md
|
||||
- 02-DECISIONS/0074-the-wire-is-specified-not-the-types.md
|
||||
- 02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md
|
||||
@@ -18,8 +18,8 @@ decisions:
|
||||
- 02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md
|
||||
- 02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md
|
||||
- 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md
|
||||
- 02-DECISIONS/0121-a-seat-carries-the-protocol-of-its-role.md
|
||||
- 02-DECISIONS/0122-the-predecessor-is-ending-and-its-broker-goes-with-it.md
|
||||
- 02-DECISIONS/0129-a-seat-carries-the-protocol-of-its-role.md
|
||||
- 02-DECISIONS/0130-the-predecessor-is-ending-and-its-broker-goes-with-it.md
|
||||
---
|
||||
|
||||
# 25. The bus on NATS
|
||||
@@ -49,7 +49,7 @@ mesh's own state lives, and where what a module may say is decided by what it de
|
||||
The last two rows are the ones worth dwelling on, because they are not messaging in the sense of
|
||||
carrying bytes from A to B. **A role is addressable**, so a caller names the capability and never
|
||||
the module or the node — and the implementation can be replaced under it without a caller
|
||||
changing ([ADR 0118](../../02-DECISIONS/0118-a-module-declares-its-own-seats.md)). That is a
|
||||
changing ([ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md)). That is a
|
||||
property of the mesh's architecture that happens to be expressed in subjects.
|
||||
|
||||
And more of the mesh lands here as it is built: conditions and observed state in key-value
|
||||
@@ -60,7 +60,7 @@ is a message being moved; all of it is the bus being the mesh's centre.
|
||||
|
||||
**What a module sees of it is small and derived.** It declares what it emits, consumes, serves
|
||||
and uses, and the subjects, streams, consumers and permissions all follow from that
|
||||
([design 29](29-what-a-module-declares.md)). The sdk's contract — `request`, `handle`, `publish`,
|
||||
([design 29](32-what-a-module-declares.md)). The sdk's contract — `request`, `handle`, `publish`,
|
||||
`subscribe`, `close` — is the whole surface, and it does not change
|
||||
([ADR 0039](../../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md)).
|
||||
|
||||
@@ -82,7 +82,7 @@ mesh.seat.<seat>.tool.<verb> a role's tool (core request/rep
|
||||
mesh.ask.<node>.<command> the controller's command api (core request/reply)
|
||||
```
|
||||
|
||||
**Revised 2026-09-27** ([ADR 0121](../../02-DECISIONS/0121-a-seat-carries-the-protocol-of-its-role.md)):
|
||||
**Revised 2026-09-27** ([ADR 0129](../../02-DECISIONS/0129-a-seat-carries-the-protocol-of-its-role.md)):
|
||||
**`mesh.build.request`, `mesh.control.built` and the BUILDS stream are gone.** A build is work submitted to a role, and the
|
||||
mesh already has a shape for that — a seat's `accept` subjects, on a work queue with a queue group of
|
||||
holders, which is what a build queue shared by several machines *is*. Keeping a second mechanism for it
|
||||
@@ -91,7 +91,7 @@ seat's own event — which is why the control branch loses its copy too: one pub
|
||||
asked, the controller that records it and the catalogue that places it — the fan-out a shared exchange gave for free, as a derived subject rather than a
|
||||
configured topology.
|
||||
|
||||
**Revised 2026-09-26** ([design 29](29-what-a-module-declares.md)): a module's events and tools
|
||||
**Revised 2026-09-26** ([design 29](32-what-a-module-declares.md)): a module's events and tools
|
||||
moved from `mesh.events.*` / `mesh.tools.*` into one namespace per module, `mesh.mod.<module>.>`,
|
||||
so a module's authority over its own name is a single subject pattern the server enforces — and
|
||||
each carries a **kind token**, without which an events stream's filter would capture tool calls.
|
||||
@@ -303,7 +303,7 @@ the private network. It is raised at genesis like the store, adopted as a module
|
||||
phase.
|
||||
|
||||
**The deprecated broker is an ordinary module, not a compatibility layer.** Revision, second review
|
||||
([ADR 0119](../../02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md)): earlier text here, and
|
||||
([ADR 0127](../../02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md)): earlier text here, and
|
||||
ADR 0106 before it, called it `lavinmq-compat` — one purpose, the predecessor's clients, and a
|
||||
retirement condition of no client connected for a period the operator sets. It is none of those.
|
||||
A module may legitimately need an AMQP broker as a **backing service**, the way it needs a
|
||||
@@ -312,11 +312,11 @@ foundation, never raised at genesis, installed when something wants it and absen
|
||||
that does not. There is no retirement condition, because the day its last client disappears is
|
||||
not a day anything is waiting for.
|
||||
|
||||
**Revised 2026-09-27** ([ADR 0122](../../02-DECISIONS/0122-the-predecessor-is-ending-and-its-broker-goes-with-it.md)):
|
||||
**Revised 2026-09-27** ([ADR 0130](../../02-DECISIONS/0130-the-predecessor-is-ending-and-its-broker-goes-with-it.md)):
|
||||
**that day is coming.** The predecessor is deprecated — some of it still running, none of it being
|
||||
migrated, left to stop rather than moved — so the broker retires once nothing requires `amqp`. Still no
|
||||
retirement *condition* and no end-date machinery: a provision with no consumers has its provider
|
||||
unassigned, which is the ordinary mechanism and is ADR 0119 being paid off rather than revised. What
|
||||
unassigned, which is the ordinary mechanism and is ADR 0127 being paid off rather than revised. What
|
||||
also goes with it is the tooling that reaches this installation's machines remotely, because the
|
||||
predecessor's own mesh talks over that broker — so the rollout is driven from the node, or before the
|
||||
broker stops.
|
||||
@@ -534,7 +534,7 @@ find what changed and why.
|
||||
**Still open:**
|
||||
|
||||
- ~~Whether EVENTS should be one stream or one per emitting module.~~ **Closed**
|
||||
([ADR 0119](../../02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md) (superseding [ADR 0117](../../02-DECISIONS/0117-the-bus-is-the-only-broker.md))): one stream, and not as a
|
||||
([ADR 0127](../../02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md) (superseding [ADR 0125](../../02-DECISIONS/0125-the-bus-is-the-only-broker.md))): one stream, and not as a
|
||||
preference — the streams the bus is made of are composed as configuration before any module
|
||||
runs, because a provisioner is itself a module that needs a bus account to start. Bootstrapping
|
||||
decides it.
|
||||
|
||||
@@ -8,12 +8,13 @@ code:
|
||||
- mesh-controller cmd/mesh-controller/source.go
|
||||
- mesh-controller internal/inventory/migrations/0032-a-source-may-live-on-a-seat.sql
|
||||
- mesh-catalog modules/gitea/module.json
|
||||
updated: 2026-09-26
|
||||
updated: 2026-09-27
|
||||
decisions:
|
||||
- 02-DECISIONS/0118-a-module-declares-its-own-seats.md
|
||||
- 02-DECISIONS/0120-the-mesh-bus-is-required-not-ambient.md
|
||||
- 02-DECISIONS/0126-a-module-declares-its-own-seats.md
|
||||
- 02-DECISIONS/0128-the-mesh-bus-is-required-not-ambient.md
|
||||
- 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md
|
||||
- 02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md
|
||||
- 02-DECISIONS/0117-a-machines-uplink-is-a-seat.md
|
||||
---
|
||||
|
||||
# 26 — The seats
|
||||
@@ -49,13 +50,26 @@ nobody argued for is an entry nobody can explain.
|
||||
## The set
|
||||
|
||||
**The set is derived, and only the mesh's half is written here.** Revision, 2026-09-26
|
||||
([ADR 0118](../../02-DECISIONS/0118-a-module-declares-its-own-seats.md), superseding
|
||||
([ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md), superseding
|
||||
[ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)): a module
|
||||
declares its own seats with their protocols, so the seats a mesh has are the mesh's own **plus
|
||||
every registered module's**. The set is still closed — a seat named nowhere is refused — but it is
|
||||
computed from the catalogue rather than maintained by hand, which is the property 0110 actually
|
||||
needed and the table could not keep.
|
||||
|
||||
**And the mesh's own half is data, named for its scope.** Revision, 2026-09-27, reconciling two
|
||||
records made in parallel: [ADR 0121](../../02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md)
|
||||
names a system seat for the scope it is held at — `mesh-*` for one per mesh, `node-*` for one per
|
||||
machine — and [ADR 0122](../../02-DECISIONS/0122-a-seat-is-data-a-rename-is-a-database-update.md) moves
|
||||
the set out of compiled code into a table the controller owns, so a rename is one write rather than a
|
||||
rebuild of everything that names one.
|
||||
|
||||
So the set has two halves and neither is written out here: the mesh's own, which the controller holds
|
||||
as rows, and every registered module's, which is computed from the catalogue. What this document keeps
|
||||
is what a seat *is* — the rest would be a third copy, stale the first time somebody renamed one, which
|
||||
is the fault ADR 0122 exists about.
|
||||
|
||||
|
||||
**Every seat below is named `mesh-*`, and the prefix is the reservation rule**: a module declaring
|
||||
any `mesh-*` name is refused at registration, so there is no reserved-names list to drift. Ten of
|
||||
these are renamed to restore [ADR 0079](../../02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md)'s
|
||||
@@ -82,7 +96,7 @@ convention, which later seats departed from.
|
||||
The controller holds **the mesh's own** entries in code, and a test asserts their size and that
|
||||
every one names the record that made it a seat. A module's seats are not here and never will be —
|
||||
they are read from the catalogue. **This table and
|
||||
[ADR 0118](../../02-DECISIONS/0118-a-module-declares-its-own-seats.md) govern, and code that
|
||||
[ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) govern, and code that
|
||||
disagrees is what is wrong.** The implementation in progress predates several
|
||||
things here: seats held by assignments rather than claimed by definitions, the `mesh-vault` seat and
|
||||
its reservation, and the foundation's seats delivering nothing. It is brought to this table before it
|
||||
@@ -171,7 +185,7 @@ still to take.
|
||||
|
||||
## How it is checked
|
||||
|
||||
The rules here are [ADR 0118](../../02-DECISIONS/0118-a-module-declares-its-own-seats.md)'s —
|
||||
The rules here are [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md)'s —
|
||||
which supersedes [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)
|
||||
and keeps every rule below except how the set is formed — and
|
||||
[ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md)'s. Each is
|
||||
|
||||
@@ -8,7 +8,7 @@ decisions:
|
||||
- 02-DECISIONS/0115-one-assignment-of-a-module-per-node.md
|
||||
- 02-DECISIONS/0113-the-vault-makes-every-secret.md
|
||||
- 02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md
|
||||
- 02-DECISIONS/0118-a-module-declares-its-own-seats.md
|
||||
- 02-DECISIONS/0126-a-module-declares-its-own-seats.md
|
||||
- 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md
|
||||
- 02-DECISIONS/0084-which-provider-serves-a-consumer.md
|
||||
- 02-DECISIONS/0046-a-module-configuration-is-its-assignments-not-its-manifest.md
|
||||
@@ -64,7 +64,7 @@ Which module answers, in order:
|
||||
1. **the holder of the seat the requirement names.** A requirement may name a seat instead of leaving
|
||||
the provider open. It asks for *the mesh's* one, and the mesh answers with whichever assignment
|
||||
holds that seat, with nothing asked of anyone. Unheld, the requirement is refused, naming the seat
|
||||
([ADR 0118](../../02-DECISIONS/0118-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)),
|
||||
([ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)),
|
||||
[26 — The seats](26-the-seats.md)). Only a seat that delivers a provision can be named; naming a
|
||||
foundation seat is refused, because it delivers nothing. A `secret` requirement always names
|
||||
`mesh-vault`, because that provision is reserved;
|
||||
@@ -199,7 +199,7 @@ containers, login, broker account and settings are keyed by it, as today, and a
|
||||
tightest backend ([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)).
|
||||
|
||||
**A module may run on many nodes, and one assignment may hold a seat**
|
||||
([ADR 0118](../../02-DECISIONS/0118-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md))). The definition
|
||||
([ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md))). The definition
|
||||
says which seats the module can hold; the assignment says which it does. So the store module can run
|
||||
on every node, one of those assignments holds `mesh-store`, and moving that role changes an
|
||||
assignment, not a definition.
|
||||
|
||||
@@ -9,13 +9,13 @@ updated: 2026-09-27
|
||||
decisions:
|
||||
- 02-DECISIONS/0116-the-bus-is-built-in-five-steps.md
|
||||
- 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
- 02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md
|
||||
- 02-DECISIONS/0118-a-module-declares-its-own-seats.md
|
||||
- 02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md
|
||||
- 02-DECISIONS/0126-a-module-declares-its-own-seats.md
|
||||
- 02-DECISIONS/0074-the-wire-is-specified-not-the-types.md
|
||||
- 02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md
|
||||
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
- 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md
|
||||
- 02-DECISIONS/0122-the-predecessor-is-ending-and-its-broker-goes-with-it.md
|
||||
- 02-DECISIONS/0130-the-predecessor-is-ending-and-its-broker-goes-with-it.md
|
||||
---
|
||||
|
||||
# 28. Building the bus
|
||||
@@ -117,8 +117,8 @@ step 5 the rollout
|
||||
|
||||
## Step 1 — the module, and genesis raises it
|
||||
|
||||
> **Revised 2026-09-26** ([ADR 0118](../../02-DECISIONS/0118-a-module-declares-its-own-seats.md),
|
||||
> [design 29](29-what-a-module-declares.md)). Tasks 1.3 and 1.4 said the controller composes every
|
||||
> **Revised 2026-09-26** ([ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md),
|
||||
> [design 29](32-what-a-module-declares.md)). Tasks 1.3 and 1.4 said the controller composes every
|
||||
> account and creates *the four streams* at genesis, from a fixed set. That is only the mesh's own
|
||||
> half. A module declares seats with their protocols, so streams are created **at registration**
|
||||
> and durable consumers **at assignment** — neither of which has happened at genesis. The fixed
|
||||
@@ -139,7 +139,7 @@ paper is wrong until there is a second mesh to find out.
|
||||
because a container has no reload and a recreate would drop every connection the mesh has
|
||||
- [x] 1.3 the controller composes that file: accounts, permissions, TLS, JetStream — a user's
|
||||
permissions derived from its declaration and nothing else, over the three namespaces of
|
||||
[design 29](29-what-a-module-declares.md) §2, plus its own ack subject and its own inbox
|
||||
[design 29](32-what-a-module-declares.md) §2, plus its own ack subject and its own inbox
|
||||
prefix (design 25 §4)
|
||||
- [x] 1.4 the mesh's own streams, created at genesis and asserted idempotently on start, by the
|
||||
controller as their only writer — **the mesh's own, not all of them**: a seat's streams are
|
||||
@@ -444,7 +444,7 @@ pays for itself furthest away.
|
||||
connection is refused by a library error rather than by anything the mesh says.
|
||||
- [x] 3.7 the sdk's three stale comments, and nothing else in it — three lines, which is the
|
||||
whole of the sdk's diff for the bus change, and the measurement that predicted it
|
||||
- [x] 3.8 **the declaration model** of [design 29](29-what-a-module-declares.md): local names
|
||||
- [x] 3.8 **the declaration model** of [design 29](32-what-a-module-declares.md): local names
|
||||
derived to subjects, the three namespaces, permissions computed from a declaration, and a
|
||||
manifest that contains no subject. Done in the controller's composer (permissions, streams,
|
||||
consumers), in the runtime's client (subjects derived from the credential, never named by a
|
||||
@@ -469,7 +469,7 @@ pays for itself furthest away.
|
||||
- [x] 3.10 **the ten seat renames** — done in the controller's table, the ten manifests that
|
||||
claim them, the controller's own shipped manifests, and every test. Not a migration after
|
||||
all: a holding is derived at resolution, never stored, so nothing recorded points at an old
|
||||
name (recorded as a progressive insight on ADR 0118). A **kept** rename table tells a
|
||||
name (recorded as a progressive insight on ADR 0126). A **kept** rename table tells a
|
||||
manifest written against an old name what it became, because a module lives in its own
|
||||
repository and may be registered long after the catalogue stopped using one.
|
||||
|
||||
@@ -520,7 +520,7 @@ it, and the beds that need a mesh living on NATS can finally run.
|
||||
is where the code stops and the lab starts
|
||||
- [x] 4.2 a build source's change reaches the builder over the bus, and the build that follows is
|
||||
the one the change asked for — **a build is work submitted to a role now**
|
||||
([ADR 0121](../../02-DECISIONS/0121-a-seat-carries-the-protocol-of-its-role.md)). Both sides
|
||||
([ADR 0129](../../02-DECISIONS/0129-a-seat-carries-the-protocol-of-its-role.md)). Both sides
|
||||
are behind a seam with an implementation per bus, and on the bus being built one publish does
|
||||
what two did: the outcome is the role's own event, so the asker matches it by the id its
|
||||
request carried, the controller records it and the catalogue places it in the graph. A build
|
||||
@@ -652,7 +652,7 @@ reserves them for after the move, and a flow built ahead of its design would be
|
||||
|
||||
> **What this costs if it goes wrong, measured rather than assumed.** On the installation this is
|
||||
> for, the old broker is also what a whole automation layer outside the mesh connects to — so it
|
||||
> stays, as an ordinary provider of `amqp` ([ADR 0119](../../02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md)),
|
||||
> stays, as an ordinary provider of `amqp` ([ADR 0127](../../02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md)),
|
||||
> and this step is not its retirement. Nothing in a served request's path goes over the mesh's own
|
||||
> bus: modules serve from their own containers. What a failed move costs is the mesh's ability to
|
||||
> *change* anything — pushes, tool calls, new provisioning — until it is finished or undone. That
|
||||
@@ -660,10 +660,10 @@ reserves them for after the move, and a flow built ahead of its design would be
|
||||
> keep running" is a reasonable position rather than a gamble.
|
||||
- [ ] 5.3 the mesh's own accounts removed from the deprecated broker, and then the broker itself:
|
||||
after the rollout nothing of the mesh speaks to it, and an account nothing uses is one nobody
|
||||
rotates. **It finishes now** ([ADR 0122](../../02-DECISIONS/0122-the-predecessor-is-ending-and-its-broker-goes-with-it.md)):
|
||||
rotates. **It finishes now** ([ADR 0130](../../02-DECISIONS/0130-the-predecessor-is-ending-and-its-broker-goes-with-it.md)):
|
||||
the predecessor is deprecated rather than kept, so once its remnants have stopped the module is
|
||||
unassigned and the port is free. No retirement machinery — a provision with no consumers has its
|
||||
provider unassigned, which is ADR 0119 being paid off rather than revised.
|
||||
provider unassigned, which is ADR 0127 being paid off rather than revised.
|
||||
|
||||
Retiring with it: the build outcome's second announcement under the module's own name, which
|
||||
exists only so a catalogue deployed before the rename and one deployed after both hear it.
|
||||
@@ -673,10 +673,10 @@ reserves them for after the move, and a flow built ahead of its design would be
|
||||
> The rollout has to be driven from the node, or driven before the broker stops — which is a
|
||||
> sequencing constraint on 5.2 and not an afterthought.
|
||||
|
||||
> **5.4 is gone, and was wrong from ADR 0119 onward.** It read "the deprecated broker retires
|
||||
> **5.4 is gone, and was wrong from ADR 0127 onward.** It read "the deprecated broker retires
|
||||
> when its condition holds — no client connected for the period the operator sets", which is
|
||||
> ADR 0106's framing of it as a compatibility module with an end date.
|
||||
> [ADR 0119](../../02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md) settled that it is an
|
||||
> [ADR 0127](../../02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md) settled that it is an
|
||||
> **ordinary provider** of the `amqp` provision, like any module answering a backing service:
|
||||
> no seat, not foundation, and **no retirement condition**, because the day its last client
|
||||
> disappears is not a day anything is waiting for. Removed rather than reworded — a step that
|
||||
@@ -699,7 +699,7 @@ itself moves once, at the end, on one day.
|
||||
- **Leaf nodes** — design 25 §11 keeps this out of scope and says so; a leaf per machine is a later
|
||||
question, noted so it is not forgotten.
|
||||
- **The predecessor's world.** It is AMQP and it is not moving —
|
||||
[ADR 0122](../../02-DECISIONS/0122-the-predecessor-is-ending-and-its-broker-goes-with-it.md): it is
|
||||
[ADR 0130](../../02-DECISIONS/0130-the-predecessor-is-ending-and-its-broker-goes-with-it.md): it is
|
||||
deprecated, some of it is still running, and it is being left to stop rather than migrated. Its
|
||||
broker goes with it, unassigned like any provider whose provision nothing requires.
|
||||
|
||||
|
||||
@@ -0,0 +1,178 @@
|
||||
---
|
||||
layer: to-be
|
||||
status: proposed
|
||||
code: []
|
||||
updated: 2026-09-27
|
||||
decisions:
|
||||
- 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md
|
||||
- 02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md
|
||||
- 02-DECISIONS/0051-shared-data-is-the-operators.md
|
||||
- 02-DECISIONS/0113-the-vault-makes-every-secret.md
|
||||
- 02-DECISIONS/0117-a-machines-uplink-is-a-seat.md
|
||||
---
|
||||
|
||||
# 29 — A node has operator accounts, and the mesh owns what lives under a home
|
||||
|
||||
**The mesh models machines but not the people on them.** A node record holds its name, its
|
||||
address, its mode — and nothing about *who a person is* on it: `jochens` on novox, `ace` on ace,
|
||||
`jochen` on shanks and g14. That username is not incidental. It decides who a file under `~` is
|
||||
owned by, who a user service runs as, and — the case that surfaced this — which account `ssh
|
||||
<node>` logs in as. The predecessor knew it (its per-node `user:`, and the modules that wrote a
|
||||
person's `~/.ssh/config`, `~/.zshrc`, `~/.config`); the mesh, taking those over, kept the machine
|
||||
facts and dropped the human one.
|
||||
|
||||
Several things are missing, and they are one idea.
|
||||
|
||||
## 1. The account is a node fact
|
||||
|
||||
A node has one or more **operator accounts**: the human logins on it. At minimum a name; the
|
||||
mesh already knows the node and its address, so `<account>@<node>` is then a complete answer to
|
||||
"who am I, where." It is the mesh's to hold because everything below is derived from it, and
|
||||
because it is exactly the fact that was silently lost — `ssh ace` failed to `ace` because nothing
|
||||
in the mesh said ace's account is `ace`.
|
||||
|
||||
## 2. A resource may live under a home, owned by its account
|
||||
|
||||
[ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) placed a
|
||||
module's *system* data — `<root>/<module>`, owned by the module. It has no analog for the other
|
||||
half of the filesystem: the things that belong under a person's home and are owned by that
|
||||
person. `~/.ssh/config`, `~/.zshrc`, `~/.config/hal` — every one of these is a resource the mesh
|
||||
should be able to place and own, resolved against **the account's home** rather than a system
|
||||
root, and chowned to **the account** rather than to root or a module uid.
|
||||
|
||||
This is the same move as `${dir:…}`, one level over: a resource says `home: <account>` (or names
|
||||
an account requirement), and the mesh resolves the home directory and the owning uid on the node
|
||||
that account lives on. A module that writes operator config — the eventual replacements for
|
||||
`hal/terminal`, `hal/claude-code`, `hal/secrets` — declares its files this way and names no
|
||||
`/home/...` path, exactly as a system module now names no `/var/lib` path.
|
||||
|
||||
These are a **family**, not one module: an `ssh-client` module, a shell module, a `~/.config`
|
||||
module, each a *universal-tier* consumer of the account fact — assigned wherever a person logs in,
|
||||
which is every node, unlike the graphical stack that a capability gates.
|
||||
|
||||
## 3. The whole of `~/.ssh` is the mesh's — with one boundary drawn inside it
|
||||
|
||||
The predecessor owned a single file (`~/.ssh/config`) and left the rest alone; it drifted, because
|
||||
owning one file beside foreign ones is not owning anything. The mesh should own **the directory**:
|
||||
create `~/.ssh` at `0700`, chown it to the account, and own the files it places there —
|
||||
|
||||
- **`config`** (or the mesh's region of it): the `Host` blocks for every other node, composed
|
||||
from the roster;
|
||||
- **`known_hosts`**: authoritative, so the "Host key verification failed / accept-new" dance that
|
||||
cost real time during enrolment simply ends;
|
||||
- **`authorized_keys`**: who may log into this account, governed centrally rather than by whichever
|
||||
key happened to be pasted where.
|
||||
|
||||
**The boundary — and it is the reason this is safe:** `~/.ssh` is the one directory where a wrong
|
||||
declaration locks a person out of their own machine. So the mesh's *found-vs-owned* semantics
|
||||
([ADR 0126](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md),
|
||||
adoption) apply *inside* the home directory. The mesh **owns** the directory and the files above; it
|
||||
**holds as found — never rewrites, never removes** — the operator's own contents: their **private
|
||||
keys** and their **personal drop-ins** (`config.d/personal`, the personal `Host` aliases a
|
||||
workstation carries, exactly as `hosts.local` is the home the mesh never rewrites for `/etc/hosts`).
|
||||
Reconcile removing an unassigned `config.d/mesh` is fine; the same logic aimed at `id_ed25519` or an
|
||||
operator's own `authorized_keys` entry is a lockout. This is the login-channel cousin of the rule
|
||||
[ADR 0125](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md) draws for the uplink and the sshd
|
||||
module draws for the firewall: **the mesh must never be able to arrange the one failure that severs
|
||||
its own way back in.** The carve-out is not a convenience; it is that rule, in `~/.ssh`.
|
||||
|
||||
## 4. Keys are the mesh's to generate — through a CA, and existing keys are adopted, not replaced
|
||||
|
||||
Key *generation* is the mesh's, not each node's improvising its own. The clean form is an **SSH
|
||||
certificate authority as a seat**, the sibling of the TLS internal CA the mesh already runs:
|
||||
|
||||
- **Host certs.** The mesh signs each node's host key. Every node's `known_hosts` becomes one line
|
||||
— `@cert-authority *.<suffix> <mesh-CA-key>` — and nothing is distributed per node; a new node is
|
||||
trusted the instant its host key is signed.
|
||||
- **User certs.** The mesh signs a cert naming the principals (accounts) allowed. Every node's
|
||||
`authorized_keys` / sshd `TrustedUserCAKeys` becomes one trust line — no N×N key spraying — and
|
||||
short-lived certs give rotation for free
|
||||
([ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md)).
|
||||
- The **CA private key is the mesh's**, a secret the vault makes
|
||||
([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)).
|
||||
|
||||
**Three kinds of key, and only one is never minted.** Host keys (server identity) and pure
|
||||
machine-to-machine keys the mesh may generate end to end. The operator's **personal** private key —
|
||||
possibly on a hardware token, possibly used from an off-mesh laptop — the mesh **signs into a cert
|
||||
but never generates**; that, and only that, is the residue of
|
||||
[ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md). So "keys are mesh-owned" and
|
||||
"the operator's login key is the operator's" reconcile: the mesh owns the CA and the signing; it
|
||||
holds the human's private half, never mints it.
|
||||
|
||||
**Existing keys are not lost.** Taking ownership is *adoption*, not regeneration: a key already on a
|
||||
machine is recorded and signed, not overwritten. The mesh gains authority over `~/.ssh` — it does
|
||||
not clear it. An enrolling node's host key and the operator's existing key are carried forward; the
|
||||
found-vs-owned boundary of §3 is exactly what guarantees nothing already there is destroyed.
|
||||
|
||||
## 5. How it is distributed: the controller composes, the node applies
|
||||
|
||||
None of this needs a node to discover the mesh, and none of it needs a control-plane module of its
|
||||
own. The ssh files are **roster facts**
|
||||
([ADR 0128](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md)): once the
|
||||
roster view carries a node's **host key** and its **account** beside its name and address, the
|
||||
`ssh-client` module ships a template for `known_hosts`, `config` and `authorized_keys`, and the
|
||||
controller renders each node's copy from the full roster and pushes it. The mesh owns the data; the
|
||||
module owns ssh's format; the control plane gains no ssh syntax. It is the same act as composing a
|
||||
peer list or `/etc/hosts` — which is why there is **no novox-only "mesh-ssh" module**: the
|
||||
centralization is the controller's composition, not a module that runs somewhere. Only non-secret
|
||||
facts travel (names, addresses, accounts, host keys, the CA public key); the private key stays the
|
||||
operator's, placed as an operator-owned file, referenced by path.
|
||||
|
||||
## 6. The two modules, and the seat between them
|
||||
|
||||
- **`sshd`** (server, every node) — manages sshd, owns and **reports** its host key so the roster
|
||||
carries it, and trusts the user CA.
|
||||
- **`ssh-client`** (client, every node) — owns `~/.ssh` per §3, consumes the roster and the CA
|
||||
public key.
|
||||
- **`the-ssh-ca`** (a seat, held on the control node) — signs host and user certs.
|
||||
|
||||
They meet at the account and the CA, not at a bespoke module. The `sshd` server side already exists;
|
||||
the client/identity side and the CA are the open pieces.
|
||||
|
||||
## Why now, and why not yet
|
||||
|
||||
**Why it matters:** when HAL retires, the generators that keep `~/.ssh`, shell config and the
|
||||
operator's `~/.config/hal` current retire with it. Without this, adding a node stops adding its ssh
|
||||
alias and its trust, and a fresh machine has no operator dotfiles at all — the mesh would run every
|
||||
service and leave the human unable to work on the box.
|
||||
|
||||
**Why not build it reflexively:** it is a real addition to the node model, the resource model, and
|
||||
the seat set, and must be gotten right. The mechanism half is now settled — ADR 0128 is what lets
|
||||
the ssh files be templates with no control-plane format — so what remains to decide here is the
|
||||
model:
|
||||
|
||||
- **One account or several per node?** A workstation has one human; a shared box might have more.
|
||||
Allow more than one without forcing the common case to name it.
|
||||
- **The CA's shape.** Host-cert and user-cert principals, cert lifetime and renewal, where the CA
|
||||
runs (a seat on the control node). The one thing fixed: the operator's personal key is signed,
|
||||
never minted.
|
||||
- **Adoption of existing keys.** How an enrolling node's host key and an operator's existing key are
|
||||
recorded and signed rather than replaced — the found-vs-owned boundary, made concrete for keys.
|
||||
- **The `sshd` boundary.** Server side exists; this is the client, the identity, and the CA.
|
||||
- **The ssh-agent.** An agent is a *user-scoped service running as the account* — the first concrete
|
||||
case of the user services §2 anticipates. It holds the operator's private key in memory; the mesh
|
||||
declares the unit and sets `AddKeysToAgent`/`IdentityAgent` in `config`, and still never sees the
|
||||
private half. Agent *forwarding* wants a policy, not a default: with user certs it is largely
|
||||
unnecessary, and forwarding an agent into a node exposes the operator's keys to that node's root —
|
||||
so prefer certificates and `ProxyJump` over forwarding.
|
||||
|
||||
**Not urgent, not blocking.** ssh and dotfiles work today because HAL's generators still run as the
|
||||
substrate. This becomes load-bearing in the node-by-node retirement phase, not before — which is the
|
||||
right time to build it, once the account and CA model are decided here.
|
||||
|
||||
## References
|
||||
|
||||
- The gap was found generating `~/.ssh/config` from the *HAL* registry (`hal/terminal`'s
|
||||
postConfigure hook), which the nox mesh has no equivalent for.
|
||||
- [ADR 0128](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md) — the roster
|
||||
fact mechanism that renders the ssh files, format owned by the module.
|
||||
- [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) — the
|
||||
system-path placement this mirrors for home paths.
|
||||
- [ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md) — why the operator's personal
|
||||
key is signed, never minted.
|
||||
- [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md) — the CA key is a secret the
|
||||
vault makes; [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md)
|
||||
— short-lived certs as rotation.
|
||||
- [ADR 0125](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md),
|
||||
[ADR 0126](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md) —
|
||||
the never-sever-the-channel rule and the found-vs-owned semantics, applied here to `~/.ssh`.
|
||||
@@ -0,0 +1,137 @@
|
||||
---
|
||||
layer: to-be
|
||||
status: proposed
|
||||
code: []
|
||||
updated: 2026-09-27
|
||||
decisions:
|
||||
- 02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md
|
||||
- 02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md
|
||||
---
|
||||
|
||||
# 30 — The mesh updates itself on a push
|
||||
|
||||
**Today the mesh does not update itself; a person drives the pipeline by hand, and one class of
|
||||
change freezes it.** A code change lands in `mesh-controller` or `mesh-catalog`, and getting it onto
|
||||
the machines is a sequence somebody types. The predecessor's pipelines rebuilt and redeployed on a
|
||||
push without anyone watching; the successor should too. This records the process as it is done by
|
||||
hand now — so it can be read, and then coded — and the two things that make it more than "add a
|
||||
webhook".
|
||||
|
||||
## The process, as done by hand
|
||||
|
||||
**An ordinary (non-breaking) change** — new module code, a bug fix, a manifest tweak that changes no
|
||||
seat or schema:
|
||||
|
||||
1. `module moved <module> <commit>` — tell the mesh its source advanced (the controller repo has no
|
||||
trigger, so this is manual; the catalogue's webhook does it automatically — see below).
|
||||
2. `build --behind` (or `build <repo> [--ref] [--path <subdir>]`) — the build machine rebuilds and
|
||||
records the new image.
|
||||
3. The mesh **reconciles on its own**: the module's declaration now names the new image, the next
|
||||
push/heartbeat sends it, and the host swaps the container. For the control plane this is a
|
||||
self-upgrade — the running controller composes its own new image and the host replaces it. No
|
||||
restart is typed.
|
||||
|
||||
**A breaking change** — a manifest schema the controller parses differently (a fact's shape, a
|
||||
seat's name), where the new control plane cannot read the manifests the old one stored:
|
||||
|
||||
4. Land the code (controller + catalogue together — they are one change).
|
||||
5. Rebuild + deploy the new controller (steps 1–3). **The moment it is live it refuses the
|
||||
still-old-shape stored manifests, and composition freezes for every node that runs an affected
|
||||
module.** Running services are untouched; only new declarations stop.
|
||||
6. **Re-register each affected manifest under the new shape**, which the *new* controller accepts —
|
||||
`module add <file> -source <repo> -ref <ref> -commit <commit>`. This writes the manifest to the
|
||||
store without a build, so it is the fast way to lift the freeze. (The controller container is
|
||||
distroless: `docker cp` the file to the container root `/x.json`; `/tmp` does not exist; the
|
||||
root filesystem is writable. The file is lost when the container is recreated on the next image
|
||||
swap, so copy it *after* the swap.)
|
||||
7. `push --behind`, then verify `status` is clean and `seats` (or the relevant surface) shows the
|
||||
new shape held by the right holders.
|
||||
|
||||
The freeze in a breaking change has been paid three times in one session (a fact-shape change, the
|
||||
`/etc/hosts` region, a seat rename); each time it lasted seconds and no service dropped. It is
|
||||
recoverable, but it is not something a push should trigger unwatched — which is the crux of what
|
||||
automating this must solve.
|
||||
|
||||
## Why it is more than "add a webhook"
|
||||
|
||||
### 1. The trigger today is HAL's, not the mesh's
|
||||
|
||||
Build-on-push works for the catalogue because its repository has a Gitea webhook pointing at
|
||||
`http://host.docker.internal:9877/webhook/gitea` — and **that receiver is `hal-gitea-tools.service`**
|
||||
(`~/.hal/modules/hal/gitea/tools/server.js`), a *predecessor* component. The nox builder consumes
|
||||
build work; it does not receive Git events. So the mesh's own build pipeline currently rides on a
|
||||
HAL service, and:
|
||||
|
||||
- the `mesh-controller` repository was never wired to it, which is why the control plane is the one
|
||||
thing that does **not** self-update — every controller deploy this session was `module moved` +
|
||||
`build` by hand;
|
||||
- when HAL is retired, build-on-push stops for the whole mesh.
|
||||
|
||||
**The mesh needs its own forge-webhook→build trigger**, a nox component (a module, and likely a
|
||||
seat — `mesh-forge-trigger` or folded into the git seat's holder) that receives Git events and turns
|
||||
them into build work over the broker, for **every** repository including `mesh-controller`. Replacing
|
||||
`hal-gitea-tools` is the concrete first build. Its logic already exists to copy: match the pushed
|
||||
repository (and changed paths, for a monorepo like the catalogue) against the build-context
|
||||
repository of every registered module, and rebuild the matches.
|
||||
|
||||
### 2. The builder validates too — and a breaking change deadlocks it
|
||||
|
||||
The build machine embeds the same catalogue package the controller does, so **it validates a
|
||||
manifest against its own compiled-in seat/schema set**. A breaking change therefore couples *four*
|
||||
things, not two: the controller, the **builder**, every affected manifest, and every node's host.
|
||||
This session's seat rename rebuilt the controller but not the builder, and the stale builder then
|
||||
refused every manifest claiming a renamed seat.
|
||||
|
||||
Worse, one rename **deadlocked** the builder: the build machine's own seat was renamed
|
||||
(`the-build-machine` → `mesh-build-machine`). To refresh the builder you must build it; to build it
|
||||
the *running* (old) builder must accept the new builder's manifest — which claims the new name it
|
||||
does not know. The old builder cannot build the new builder. Escapes:
|
||||
|
||||
- **Never rename a seat whose holder validates manifests** in an ordinary pass — the build machine's
|
||||
seat belongs with the deferred delivering seats (ADR 0121). Reverting `mesh-build-machine` to
|
||||
`the-build-machine` (deferred) lets the old builder build the new builder, which then knows the
|
||||
new names.
|
||||
- Or bootstrap a new builder image **out of band** (build locally, publish to the registry, register
|
||||
the module at that digest), the way genesis loads the first builder — bypassing the old builder's
|
||||
validation once.
|
||||
|
||||
Either way, self-update for breaking changes needs a **transition discipline** so a push does not
|
||||
auto-freeze: the new control plane (and builder) should accept the *old and new* shape together for
|
||||
one release — deprecated aliases in the seat set, a schema that reads both — then a later release
|
||||
drops the old. With that, a breaking change rolls out on a push like any other: everything reads
|
||||
both, the manifests migrate, the compatibility is removed. Without it, self-update would simply
|
||||
automate the freeze.
|
||||
|
||||
## What to build
|
||||
|
||||
- **A nox forge-webhook trigger** (replaces `hal-gitea-tools`): receives Git events for every mesh
|
||||
repository, dispatches build work to the builder over the broker, and records `module moved`
|
||||
automatically. Wire `mesh-controller` to it so the control plane self-updates like everything else.
|
||||
- **A transition discipline for breaking changes**: the control plane and builder accept old+new for
|
||||
one release; the tooling that lands a schema/seat change emits the compatibility shim and the
|
||||
follow-up that removes it. This is what makes step 4–7 above safe to trigger unwatched.
|
||||
- **Config/package modules need no builder** — `module add` registers their manifest directly
|
||||
(this is how the uplink managers and the re-registrations above were done). Only image-bearing
|
||||
modules need the build machine, which narrows what the deadlock above can block.
|
||||
|
||||
## Why now, and why not yet
|
||||
|
||||
**Why it matters:** self-update is the difference between a mesh a person maintains by typing
|
||||
pipeline steps and one that maintains itself, and it is a stated goal (parity with the predecessor's
|
||||
pipelines). The HAL trigger dependency also makes it a retirement blocker: build-on-push dies with
|
||||
HAL.
|
||||
|
||||
**Why not reflexively:** the trigger is a new component with the broker and forge in its blast
|
||||
radius, and the transition discipline changes how every breaking change is written. Both should be
|
||||
designed, not bolted on beside a freeze. The manual process above is the interim, and it works.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0121](../../02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md)
|
||||
— the seat rename whose migration and builder deadlock this record is drawn from
|
||||
- [ADR 0128](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md) — the
|
||||
fact-shape change that first showed the breaking-change freeze
|
||||
- `hal-gitea-tools.service` (`~/.hal/modules/hal/gitea/tools/server.js`) — the predecessor webhook
|
||||
receiver on `:9877` the mesh currently rides on
|
||||
- mesh-controller `cmd/mesh-builder` (the build machine), `internal/catalogue` (the seat/schema
|
||||
validation the builder shares with the controller)
|
||||
@@ -0,0 +1,65 @@
|
||||
---
|
||||
layer: to-be
|
||||
status: proposed
|
||||
code: []
|
||||
updated: 2026-09-27
|
||||
decisions:
|
||||
- 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md
|
||||
---
|
||||
|
||||
# 31 — A module declares its fail2ban jail, and the mesh composes them per node
|
||||
|
||||
**A node's intrusion filter should be composed from the modules it runs, the same way its firewall
|
||||
is.** The mesh already derives a node's nftables ruleset from every assigned module's `listens` and
|
||||
`guards` (the `Filtering` mechanism). fail2ban is the same shape and is not modelled: a module that
|
||||
runs an authenticating service — postgres, mssql, mailu — has a jail (a filter that reads its log
|
||||
and a jail stanza that bans on it), and which jails a node's fail2ban runs should be exactly the
|
||||
jails of the modules assigned to that node.
|
||||
|
||||
The predecessor did this with per-module files: `postgres` shipped `postgres-auth.conf`, `mssql`
|
||||
shipped `mssql-auth.conf`, `mailu` shipped `mailu.conf`, and the node's fail2ban read whichever were
|
||||
present. When HAL retired on novox those became dangling symlinks — fail2ban ran the jails only from
|
||||
memory, and a restart would have dropped them. The base was salvaged (the fail2ban module now ships
|
||||
`sshd`, `recidive`, and the `ignoreip` that spares the mesh's own range), but the **service jails
|
||||
are gone**, because no nox module declares one yet.
|
||||
|
||||
## The shape
|
||||
|
||||
- **A module declares its jail in its manifest**, naming no node and no path (ADR 0112): the filter
|
||||
(the failregex, or a stock filter it uses) and the jail stanza (port, logpath, maxretry, bantime).
|
||||
The `postgres` module says what a postgres brute-force looks like and how to ban it; it does not
|
||||
say on which machine, because it does not know.
|
||||
- **The mesh composes them per node.** For each node, the jails of its assigned modules are gathered
|
||||
and written into the fail2ban holder's `jail.d/` (and filters into `filter.d/`), exactly as
|
||||
`listens`/`guards` are gathered into the node's firewall. So a node running postgres gets the
|
||||
postgres jail; a node not running it does not. The `node-intrusion-prevention` holder receives
|
||||
them the way a provider receives its consumers' contributions.
|
||||
- **The base stays the fail2ban module's**: `sshd`, `recidive`, and the `ignoreip` naming
|
||||
`${machine:mesh-range}` so a tunnel peer is never banned.
|
||||
|
||||
## Why this, and not the module writing the file itself
|
||||
|
||||
A module could declare a `file` resource at `/etc/fail2ban/jail.d/<x>.conf` directly. Rejected: the
|
||||
path is the fail2ban holder's to own (one module owns `jail.d`, as one module owns the firewall
|
||||
table), the jail's logpath and defaults want the mesh's composition (the `ignoreip`, the ban action
|
||||
the node uses), and two modules writing into one directory is the collision the seat/holder model
|
||||
exists to prevent. The module declares *what its jail is*; the holder's composition decides *how it
|
||||
lands* — the same split as `listens` (the module says the port; the mesh says the rule).
|
||||
|
||||
## Why now
|
||||
|
||||
fail2ban on novox currently runs the service jails from memory only; the next restart drops them
|
||||
(the `ignoreip` is safe on disk, so the mesh-partition risk is closed, but postgres/mssql/mailu
|
||||
auth-banning would be lost). This is the mechanism that restores them properly, and it is needed as
|
||||
each of those modules migrates to the other nodes — ace running postgres should get the postgres
|
||||
jail, composed from the postgres module's manifest, without anyone editing a node.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) — a module
|
||||
names no node or path; its jail is declared the same way its `listens` are
|
||||
- mesh-controller `internal/catalogue/adoption.go` (`Filtering` — the firewall composition this
|
||||
mirrors), `internal/catalogue/manifest.go` (`Listens`/`Guards`, the fields a jail field sits
|
||||
beside)
|
||||
- mesh-catalog `modules/fail2ban` (the base: sshd, recidive, ignoreip); the service modules
|
||||
(`postgres`, `mssql`, `mailu`) that will declare jails
|
||||
+10
-10
@@ -4,20 +4,20 @@ status: proposed
|
||||
code: []
|
||||
updated: 2026-09-27
|
||||
decisions:
|
||||
- 02-DECISIONS/0118-a-module-declares-its-own-seats.md
|
||||
- 02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md
|
||||
- 02-DECISIONS/0120-the-mesh-bus-is-required-not-ambient.md
|
||||
- 02-DECISIONS/0126-a-module-declares-its-own-seats.md
|
||||
- 02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md
|
||||
- 02-DECISIONS/0128-the-mesh-bus-is-required-not-ambient.md
|
||||
- 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
- 02-DECISIONS/0041-events-are-a-relationship.md
|
||||
- 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md
|
||||
- 02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md
|
||||
- 02-DECISIONS/0121-a-seat-carries-the-protocol-of-its-role.md
|
||||
- 02-DECISIONS/0129-a-seat-carries-the-protocol-of-its-role.md
|
||||
---
|
||||
|
||||
# 29. What a module declares, and what the bus makes of it
|
||||
# 32. What a module declares, and what the bus makes of it
|
||||
|
||||
**A module that speaks to the mesh requires the bus, and receives what it needs to connect**
|
||||
([ADR 0120](../../02-DECISIONS/0120-the-mesh-bus-is-required-not-ambient.md)). What a module
|
||||
([ADR 0128](../../02-DECISIONS/0128-the-mesh-bus-is-required-not-ambient.md)). What a module
|
||||
declares are *relationships*; subjects, streams, consumers and permissions are all derived from
|
||||
those, and a manifest never contains one.
|
||||
|
||||
@@ -132,7 +132,7 @@ and the controller stays the only writer of stream and consumer definitions
|
||||
([design 25](25-the-bus-on-nats.md) §3).
|
||||
|
||||
**The mesh's own seats carry protocol too.** *Added 2026-09-27,
|
||||
[ADR 0121](../../02-DECISIONS/0121-a-seat-carries-the-protocol-of-its-role.md).* A seat declared by a
|
||||
[ADR 0129](../../02-DECISIONS/0129-a-seat-carries-the-protocol-of-its-role.md).* A seat declared by a
|
||||
module says what it accepts, emits and serves; the `mesh-*` set said only who does a job. So the mesh
|
||||
had roles it could not describe — a build machine with three audiences for one outcome and no way to
|
||||
derive a grant for any of them, and an event genuinely about a role with nowhere to live but the
|
||||
@@ -219,7 +219,7 @@ That is the wire-level answer to
|
||||
## 5. Seats
|
||||
|
||||
A module declares a seat with its protocol, and the mesh enforces one holder at its scope
|
||||
([ADR 0118](../../02-DECISIONS/0118-a-module-declares-its-own-seats.md)). A caller declares that
|
||||
([ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md)). A caller declares that
|
||||
it uses the *seat*, never the module, so the implementation can be replaced under it.
|
||||
|
||||
- The set of seats is **derived** — the mesh's own, plus every registered module's — so it is both
|
||||
@@ -357,7 +357,7 @@ controller — because the bus's accounts are configuration rather than somethin
|
||||
creates, so there is no provisioner process in the path and nothing waiting on a bus account in
|
||||
order to make bus accounts. A module that runs a NATS server of its own and offers it as a
|
||||
backing service provides **`nats`**, exactly as the deprecated broker provides `amqp`
|
||||
([ADR 0119](../../02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md)).
|
||||
([ADR 0127](../../02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md)).
|
||||
|
||||
They are never the same name. A manifest saying `nats` could otherwise mean either the mesh's
|
||||
nervous system or a private queue, and the difference between those is the whole architecture.
|
||||
@@ -427,7 +427,7 @@ it as an ordinary module once the registry exists.
|
||||
|
||||
So there are exactly two things the normal path cannot make, both at genesis, both ending the
|
||||
moment the mesh can mint for itself: **the bus's own accounts** (§the bootstrap argument in
|
||||
[ADR 0119](../../02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md) (superseding [ADR 0117](../../02-DECISIONS/0117-the-bus-is-the-only-broker.md)) — a provisioner is a module and
|
||||
[ADR 0127](../../02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md) (superseding [ADR 0125](../../02-DECISIONS/0125-the-bus-is-the-only-broker.md)) — a provisioner is a module and
|
||||
needs an account before it can run) and **the vault's own credential**. Any third exception is a
|
||||
design failure, and naming these two is what makes a third one visible.
|
||||
|
||||
@@ -35,11 +35,12 @@ document is written and this one's status becomes `implemented`.
|
||||
| [`23-choosing-a-provider.md`](23-choosing-a-provider.md) | Which of several providers of a kind serves a consumer, and when a module carries its own instead | [ADR 0084](../../02-DECISIONS/0084-which-provider-serves-a-consumer.md), [ADR 0027](../../02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md) |
|
||||
| [`24-the-secrets-vault.md`](24-the-secrets-vault.md) | The module that owns a secret — a `secret` provision, and the boundary of what it owns | [ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md), [ADR 0031](../../02-DECISIONS/0031-the-control-plane-authenticates-nobody.md), [ADR 0048](../../02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md) |
|
||||
| [`25-the-bus-on-nats.md`](25-the-bus-on-nats.md) | **Proposed.** The architecture of the mesh's bus on NATS: what rides which subject under which guarantee and whose account, how a node joins, how a person reaches a tool, and how the mesh moves from the bus it has | [ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md), [ADR 0116](../../02-DECISIONS/0116-the-bus-is-built-in-five-steps.md), [ADR 0043](../../02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md), [ADR 0083](../../02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md) |
|
||||
| [`26-the-seats.md`](26-the-seats.md) | **Proposed.** What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0118](../../02-DECISIONS/0118-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) |
|
||||
| [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md), [ADR 0118](../../02-DECISIONS/0118-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)) |
|
||||
| [`26-the-seats.md`](26-the-seats.md) | **Proposed.** What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) |
|
||||
| [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md), [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)) |
|
||||
| [`28-building-the-bus.md`](28-building-the-bus.md) | **Proposed.** The five steps of the bus work in the order their dependencies allow, each ending at a bed — with the surface measured, so no step's size is a guess | [ADR 0116](../../02-DECISIONS/0116-the-bus-is-built-in-five-steps.md), [ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md), [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md) |
|
||||
| [`29-a-node-has-operator-accounts.md`](29-a-node-has-operator-accounts.md) | **Proposed.** The mesh models machines but not the humans on them: a node gains operator accounts, and a resource may live under a home owned by its account — what would own ~/.ssh, dotfiles and ~/.config when HAL retires | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md) |
|
||||
|
||||
| [`29-what-a-module-declares.md`](29-what-a-module-declares.md) | **Proposed.** What a module declares and what the bus derives from it: three namespaces, subjects from local names, queues never declared, the five relationships, and the build-publish-deploy lifecycle on one bus | [ADR 0118](../../02-DECISIONS/0118-a-module-declares-its-own-seats.md), [ADR 0119](../../02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md) (superseding [ADR 0117](../../02-DECISIONS/0117-the-bus-is-the-only-broker.md)), [ADR 0041](../../02-DECISIONS/0041-events-are-a-relationship.md) |
|
||||
| [`32-what-a-module-declares.md`](32-what-a-module-declares.md) | **Proposed.** What a module declares and what the bus derives from it: three namespaces, subjects from local names, queues never declared, the five relationships, and the build-publish-deploy lifecycle on one bus | [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md), [ADR 0127](../../02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md) (superseding [ADR 0125](../../02-DECISIONS/0125-the-bus-is-the-only-broker.md)), [ADR 0041](../../02-DECISIONS/0041-events-are-a-relationship.md) |
|
||||
|
||||
## Not yet written
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
---
|
||||
status: located
|
||||
status: fixed
|
||||
opened: 2026-09-24
|
||||
located-in: [mesh-controller internal/inventory, mesh-controller internal/catalogue, mesh-controller cmd/mesh-controller, mesh-catalog modules/dnsmasq]
|
||||
fixed-by:
|
||||
fixed-by: [mesh-controller#73 overlay-name + namesInTheMesh, mesh-catalog#106 dnsmasq daemon.json merge]
|
||||
amended-design:
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
---
|
||||
status: located
|
||||
opened: 2026-09-27
|
||||
located-in: [mesh-controller cmd/mesh-controller/push.go]
|
||||
---
|
||||
|
||||
# A declaration that shrinks to empty is skipped, so the node keeps what it should drop
|
||||
|
||||
## What was observed
|
||||
|
||||
Fixing the broker-opening leak (the foundation port scoped to the broker's host) made ace's
|
||||
declaration compose to **zero resources** — ace is adopted with nothing assigned, and the
|
||||
stray opening was its only resource. `push ace` then printed `ace is assigned nothing —
|
||||
skipped` and sent nothing. ace goes on holding `adoption.opening-tcp-5671-incoming` in its
|
||||
ufw, because it was never told the resource is gone.
|
||||
|
||||
`composeEach` (push.go) skips any node whose composed declaration has no resources. That is
|
||||
right for a node that never had anything. It is wrong for a node that **had** resources and
|
||||
now composes to none: the empty declaration is the correction, and skipping it leaves the last
|
||||
non-empty one in force forever.
|
||||
|
||||
## Why it matters
|
||||
|
||||
Any adopted node whose openings (or other baseline resources) are all removed keeps the stale
|
||||
ones until something else pushes a non-empty declaration to it. Converge is unaffected — it
|
||||
composes the full ruleset fresh — so this is an incremental-push gap, not a firewall-safety
|
||||
one. But "the mesh cannot tell a node to drop its last resource" is a real hole in reconcile.
|
||||
|
||||
## The fix, roughly
|
||||
|
||||
Send the empty declaration when the node's last-sent declaration was non-empty — i.e. skip
|
||||
only when empty-and-was-already-empty. Requires push to know (or the host to be told) that the
|
||||
node held something. Simplest: always send to a placed, enrolled node; let an empty declaration
|
||||
mean "own nothing", which the host already applies correctly when it receives one.
|
||||
|
||||
## Workaround used
|
||||
|
||||
On ace, one command drops it permanently (the corrected controller never re-composes it):
|
||||
`sudo ufw delete allow 5671`. At ace's converge it would clear on its own.
|
||||
@@ -3,14 +3,14 @@ status: resolved
|
||||
opened: 2026-09-27
|
||||
located-in: [mesh-catalog modules, mesh-control internal/catalogue, mesh-tools src]
|
||||
fixed-by: mesh-catalog 7b06a7a, mesh-tools fbeb373, mesh-control 05ff606
|
||||
amended-design: 03-DESIGN/01-to-be/29-what-a-module-declares.md
|
||||
amended-design: 03-DESIGN/01-to-be/32-what-a-module-declares.md
|
||||
---
|
||||
|
||||
# 127 — A module's event derives a subject nothing publishes
|
||||
|
||||
## What was observed
|
||||
|
||||
[Design 29](../../03-DESIGN/01-to-be/29-what-a-module-declares.md) §1 says a module names an event
|
||||
[Design 29](../../03-DESIGN/01-to-be/32-what-a-module-declares.md) §1 says a module names an event
|
||||
locally and the mesh derives the subject: `emits: order.placed` becomes
|
||||
`mesh.mod.<module>.event.order.placed`, and a consumer declaring `consumes: shop.order.placed`
|
||||
subscribes the emitter's own subject. That derivation is built and tested.
|
||||
@@ -37,7 +37,7 @@ Two further consequences of the same cause, found in the same check:
|
||||
and the module cannot be assigned.
|
||||
- One module emits under a name that is not its own — it declares `module.<other>.image.pushed`
|
||||
while being a differently named module — which the derivation puts inside *its* namespace. Whether
|
||||
that is legitimate is a design question: design 29 §2 makes an event's source a fact the server
|
||||
that is legitimate is a design question: design 32 §2 makes an event's source a fact the server
|
||||
enforces, and this is a module claiming another's name in its own event.
|
||||
|
||||
None of it fails on the bus the mesh runs on today, where a routing key is matched literally and
|
||||
@@ -47,7 +47,7 @@ the first mesh raised on the new bus, and not before.
|
||||
Evidence: run against the controller's own `PermissionsFor` on the current feature branch, with the
|
||||
declarations read from the catalogue's manifests. Found while wiring the controller's consume side
|
||||
(design 28 step 3.4), when the controller's own subscription had to be written and the subject it
|
||||
would have to name turned out not to be the one design 29 specifies.
|
||||
would have to name turned out not to be the one design 32 specifies.
|
||||
|
||||
## Why it matters beyond this instance
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ Three places, and only one of them is a bug in code.
|
||||
|
||||
**The manifests, in the module catalogue.** Thirty-seven declare events, and every one of them
|
||||
spells an event the way a routing key on the bus the mesh runs on today is spelled —
|
||||
`module.<module>.<verb>`. [Design 29](../../03-DESIGN/01-to-be/29-what-a-module-declares.md) §1 says
|
||||
`module.<module>.<verb>`. [Design 29](../../03-DESIGN/01-to-be/32-what-a-module-declares.md) §1 says
|
||||
a module names an event **locally and bare** (`emits: order.placed`) and a consumer names
|
||||
`<emitter>.<event>` (`consumes: billing.order.placed`). So the manifests are stale against a rule
|
||||
that was already decided, not wrong against an undecided one. **This is the whole of the reported
|
||||
@@ -27,7 +27,7 @@ first thing that notices is a subscription that never fires.
|
||||
## What was ruled out
|
||||
|
||||
**The derivation is not wrong.** Asked directly, with the module names and declarations the
|
||||
catalogue holds, `PermissionsFor` produces exactly what design 29 §1 specifies for the input it is
|
||||
catalogue holds, `PermissionsFor` produces exactly what design 32 §1 specifies for the input it is
|
||||
given: it reads a consumer's `<emitter>.<event>` and builds the emitter's subject. Given
|
||||
`module.builder.built` it reads the emitter as `module`, which is a correct reading of an incorrect
|
||||
declaration.
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
---
|
||||
status: located
|
||||
opened: 2026-09-26
|
||||
located-in: [mesh-controller internal/catalogue/facts.go, mesh-host internal/apply]
|
||||
---
|
||||
|
||||
# 128 — the machine's hosts file is written whole, and on a workstation it is shared
|
||||
|
||||
## What was observed
|
||||
|
||||
The private network asks for the `node-names` fact, and the mesh delivers it as
|
||||
`/etc/hosts`. `nodeNames` composes a **complete** file — its own header, `localhost`, the
|
||||
machine's own name, and every name in the mesh — and the host writes it over whatever is there.
|
||||
|
||||
On an adopted workstation the file the mesh holds contains, besides the predecessor's block of
|
||||
mesh names:
|
||||
|
||||
- the distribution's own lines (`localhost`, the machine's `.localdomain` name);
|
||||
- two marked blocks (`# BEGIN … # END …`) maintained by a local-development tool, pointing a
|
||||
dozen development hostnames at `127.0.0.1` — rewritten by that tool whenever its project
|
||||
list changes;
|
||||
- hand-added entries of the operator's.
|
||||
|
||||
Today the file is only **held** ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)):
|
||||
the private network was assigned, not yet taken, so nothing was lost. Taking it — or
|
||||
converging the node, which takes everything — replaces the file. The development tool's
|
||||
entries disappear, its projects stop resolving, and every later write it makes is overwritten
|
||||
at the next change to the mesh's names (a machine joins, a route is contributed), silently and
|
||||
without a failure anywhere: the development tool thinks it wrote its block, and the mesh thinks
|
||||
it owns the file.
|
||||
|
||||
This is [ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md)'s
|
||||
failure exactly — a file the mesh shares with software it did not install, written over — in a
|
||||
file 0102 did not name, because its merge verb is structured (`into: json`) and a hosts file is
|
||||
not JSON.
|
||||
|
||||
A second, smaller finding from the same reading: the fact's contents depend on which machines
|
||||
hold the private network. A machine that is enrolled but not yet assigned the private network
|
||||
is in neither `node-names` nor `node-zones`; its name resolves on the others only for as long
|
||||
as a predecessor's hosts block survives. Taking the hosts file before every machine is on the
|
||||
private network loses that name too.
|
||||
|
||||
## What would have prevented it
|
||||
|
||||
- A **marked-region** merge in the host's vocabulary: `into: "block"` (or similar) — the host
|
||||
owns only the lines between its own begin and end markers, keeps everything outside them
|
||||
byte for byte, records what the region held before, and on undeclare removes the region and
|
||||
nothing else. The shape local tools already use for this very file.
|
||||
- The `node-names` fact written as that region — no header of its own, no `localhost`, no
|
||||
machine name — so the distribution's lines and every other tool's stay where they are.
|
||||
- A converge preview that names a held file the take would replace *whole*, with its line
|
||||
count before and after, so a person sees "hosts: 31 lines → 12" before the flip.
|
||||
|
||||
## The fix, as built (in review)
|
||||
|
||||
- **Host:** a file resource may say `"into": "block"`. The host owns only the lines between
|
||||
`# BEGIN mesh <id>` and `# END mesh <id>` and keeps everything outside them byte for byte. A
|
||||
new region goes at the `end` by default, or at the `start` (`"at": "start"`) for files where a
|
||||
line's meaning depends on what stands above it; a region already present is never moved.
|
||||
Undeclared, what the region held before is put back, or the region is removed and nothing
|
||||
else. Replacing nothing, it is written on an adopted node without being held — so a machine
|
||||
gets the mesh's names before its private network is taken.
|
||||
- **Controller:** `node-names` is a fact written into a shared file, emitted as that region: the
|
||||
mesh's names only, no header, no `localhost`, no `127.0.1.1` line.
|
||||
- **Order:** a host older than the block mode refuses the whole declaration on an unknown
|
||||
`into`, so hosts are upgraded before the controller that emits it.
|
||||
|
||||
## Evidence to carry into diagnosis
|
||||
|
||||
- `internal/catalogue/facts.go`, `nodeNames`: the complete file is built here.
|
||||
- The host's file resource supports `into: "json"` only; anything else is a whole write.
|
||||
- `node show <node>` on the adopted workstation: `holds file /etc/hosts
|
||||
mesh-wireguard.fact-node-names`, original kept.
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
status: open
|
||||
opened: 2026-09-26
|
||||
located-in: [mesh-controller, mesh-catalog step-ca]
|
||||
---
|
||||
|
||||
# 129 — nothing makes a machine trust the mesh's own certificate authority
|
||||
|
||||
## What was observed
|
||||
|
||||
On an enrolled, adopted workstation — on the private network, resolving the mesh's names
|
||||
through the mesh's resolver — every HTTPS name the mesh serves internally fails verification:
|
||||
|
||||
```
|
||||
curl https://<a name the mesh routes internally>/
|
||||
curl: (60) SSL certificate OpenSSL verify result: unable to get local issuer certificate (20)
|
||||
```
|
||||
|
||||
The route proxy presents a certificate issued by the mesh's internal authority (step-ca, the
|
||||
`internal-acme-ca` provision). The machine's trust store holds the **predecessor's** authority
|
||||
and a developer tool's local root, and nothing of the mesh's. No module installs the mesh's
|
||||
root, and no fact carries it: step-ca's only consumers are proxies, which obtain certificates
|
||||
over ACME and never need the root on the machine they run on.
|
||||
|
||||
[Issue 048](../048-nothing-makes-a-machine-trust-the-mesh-registry/00-report.md) found the same
|
||||
shape for the mesh's registry and resolved it by treating the private network as the transport
|
||||
security ([ADR 0082](../../02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md)):
|
||||
the runtime pulls in the clear, over the tunnel. That answer does not carry over. A browser, git
|
||||
over HTTPS, a package manager and every TLS client a person or a module uses verify the
|
||||
certificate chain, and there is no "insecure registries" for them — nor should there be.
|
||||
|
||||
Consequences today, all silent until someone tries:
|
||||
|
||||
- a person on a workstation cannot open any internal HTTPS name without a warning;
|
||||
- git over HTTPS to the mesh's forge fails, so the working clone URL is ssh-only;
|
||||
- a module on a non-hub machine that calls another module's internal HTTPS name fails
|
||||
verification unless its image happens to carry the root;
|
||||
- the predecessor's authority cannot be retired from any machine while anything there still
|
||||
speaks TLS to a mesh name, because it is the only authority those machines trust.
|
||||
|
||||
## What would have prevented it
|
||||
|
||||
- A **mesh fact carrying the internal authority's root** (public material; the controller or
|
||||
the step-ca module is its source), written onto every machine on the private network — the
|
||||
same reasoning that has the private network write the registry trust and the names: being on
|
||||
the network is what makes a machine one that speaks to the mesh's names.
|
||||
- A resource that puts it where the machine's TLS clients look — on Arch,
|
||||
`/etc/ca-certificates/trust-source/anchors/` — and **refreshes the extracted bundles**
|
||||
(`update-ca-trust`). The refresh is the open design question: it is a command, and the link
|
||||
may not carry an action ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). A declared
|
||||
one-shot unit, or a host primitive for "trust this anchor", are the obvious candidates.
|
||||
- Removal symmetric to arrival: undeclared, the anchor goes and the bundles are refreshed again,
|
||||
so a machine leaving the mesh stops trusting it.
|
||||
|
||||
## Evidence to carry into diagnosis
|
||||
|
||||
- `step-ca` module: provides `acme-ca` / `internal-acme-ca`, listens on 9000 for proxies; no
|
||||
resource writes its root anywhere but its own state directory.
|
||||
- The private network's generator writes `/etc/hosts` and the registry trust, and nothing
|
||||
about certificates.
|
||||
- On the workstation, the trust anchors present are the predecessor's authority and a local
|
||||
development root; `trust list` shows no entry for the mesh.
|
||||
@@ -0,0 +1,66 @@
|
||||
---
|
||||
status: located
|
||||
opened: 2026-09-27
|
||||
located-in: [mesh-host internal/apply/apply.go (remove)]
|
||||
amended-design: 02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md
|
||||
---
|
||||
|
||||
# 130 — undeclaring a service stops it, even one the mesh only reloads or only keeps running
|
||||
|
||||
## What was observed
|
||||
|
||||
Reviewing the uplink modules ([ADR 0125](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md))
|
||||
found that the host's `remove` path stops every `service` resource that is no longer declared:
|
||||
`SetServiceState(..., "stopped")`, reported as "stopped; the unit file is not the host's to
|
||||
delete". `store.Orphans` matches by id alone. So any of these stops the unit:
|
||||
|
||||
- the module is unassigned — by mistake, or to switch it for another;
|
||||
- the node is sent a deliberately-empty declaration ([issue 127](../127-a-declaration-that-shrinks-to-empty-is-skipped-not-sent/00-report.md));
|
||||
- a later catalogue version renames the resource's `id`.
|
||||
|
||||
That is right for a service the mesh brought into being. It is wrong for a unit the mesh
|
||||
declares only to act on — and the catalogue already has two:
|
||||
|
||||
- **The private network declares `docker.service`** (`registry-trust-reload`, state `running`)
|
||||
so that a change to the registry trust reloads the runtime ([ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md)).
|
||||
Unassigning the private network stops the container runtime, and every container on the
|
||||
machine with it — including ones the mesh does not manage.
|
||||
- **The sshd module declares `sshd.service`.** Unassigning it stops the machine's ssh daemon:
|
||||
the lockout the same module's `listens` rule says a firewall must never arrange.
|
||||
|
||||
The uplink modules would have added a third and a fourth: unassigning the network manager's
|
||||
module would have stopped the network manager, taking the machine off the only link the mesh
|
||||
reaches it by.
|
||||
|
||||
## What would have prevented it
|
||||
|
||||
- A service resource that says the unit's **lifecycle is the machine's**: declared with no
|
||||
`state`, the mesh never starts, stops, enables or disables it; it only reloads or restarts a
|
||||
*running* unit when a trigger changes; undeclared, it is left exactly as it is. (Being built
|
||||
on mesh-host `feat/a-file-written-into-a-marked-block` for the uplink modules.)
|
||||
- Then: `registry-trust-reload` declared that way (the runtime is the machine's), and the sshd
|
||||
module's service too — a machine's ssh daemon outlives any module that configures it.
|
||||
- A plan or unassign preview that names every unit an undeclare will stop, so the consequence
|
||||
is read before it happens.
|
||||
|
||||
## Resolution
|
||||
|
||||
[ADR 0126](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md):
|
||||
undeclaring removes what the mesh made and gives back what it changed. The host records the state
|
||||
it first found a unit in, and undeclaring returns the unit to it — a unit found running (the
|
||||
container runtime, sshd, a network manager) is left running; one the mesh started (the packet
|
||||
filter a converge loaded) is stopped again; nothing is started on the way out; a record from
|
||||
before the host kept what it found leaves the unit alone. That covers the runtime, sshd and the
|
||||
uplink modules at once, without each module opting out; the private network and the sshd module
|
||||
need no change.
|
||||
|
||||
A first draft — never stop a unit the mesh did not create — was rejected while implementing it:
|
||||
returning a converged node to adopted unloads the mesh's filter by exactly this path.
|
||||
|
||||
Found on the way: an undeclared `process` failed every apply on its node (`remove` had no case
|
||||
for it). Now removed with its unit, timer and bundle — the mesh's own code. `user` and `archive`
|
||||
have the same gap and are left for their own decisions: removing a login or unpacked files is not
|
||||
something to settle in passing.
|
||||
|
||||
The unassign preview is partly answered — the host's plan names each unit it will stop — and the
|
||||
controller's side is left open.
|
||||
Reference in New Issue
Block a user