Compare commits

...
Author SHA1 Message Date
mesh-admin e11bf320c9 Merge pull request 'ADR 0188: a module's own code is bundles in any language, and a tools bundle speaks MCP to the runtime; design 38 gains WP1b' (#300) from decision/0187-a-modules-own-code-is-bundles-in-any-language into main 2026-10-02 19:27:39 +00:00
jschoubben da8b4b4ee4 Renumber to ADR 0188: 0187 landed on main first, as the dead-tracker record
Two records shared 0187 (issue 155's collision); the branch landing last
renumbers, and this is it. Only the number changes.
2026-10-02 21:01:02 +02:00
jschoubben c026d5221e Merge remote-tracking branch 'origin/main' into renumber-0187 2026-10-02 21:00:45 +02:00
mesh-admin 983fd412c6 Merge pull request 'ADRs 0180, 0186 and 0187: their live rows, done' (#302) from docs/the-four-fixes-proven-live into main 2026-10-02 18:46:49 +00:00
jschoubben 9ac2493e2c ADRs 0180, 0186 and 0187: their live rows, done — all four machines filtered by the mesh alone, both front ends removed, no machine wrong or behind 2026-10-02 20:46:35 +02:00
mesh-admin 560f25c2c7 Merge pull request 'ADR 0187: a dead tracker is not the machine's failure' (#301) from fix/a-dead-tracker-is-not-the-meshs-failure into main 2026-10-02 18:42:01 +00:00
jschoubben 9e0288128b ADR 0187: a dead tracker is not the machine's failure; design 32 2026-10-02 20:41:30 +02:00
jochen 709240ec1f ADR 0187: a module's own code is bundles in any language, and a tools bundle speaks MCP to the runtime; design 38 gains WP1b
The operator's direction, absent from every record until now: the SDK must not limit who writes a
module; tools and services may be written in any language; one module may ship several bundles
(tools, a seat's implementation, a daemon); skeleton first, a full implementation when the work
requires it. ADR 0175 had the runtime import a bundle, which only JavaScript can be.

0187 makes a tools bundle a process the node's runtime launches and speaks MCP over stdio to —
the vocabulary the runtime already speaks outward — so any language with an MCP library can write
one today and the mesh's SDK per language is thin; the transport stays in the runtime (0039's
refusal, kept). Importing a TypeScript bundle is the shortcut, not the contract. Notes in 0175,
0039 and 0150 say where their mechanism moved; design 38 records WP1 as built and adds WP1b (the
launcher and the skeleton SDKs); the glossary's bundle widens.
2026-10-02 18:56:57 +02:00
mesh-admin d57289e049 Merge pull request 'ADR 0186: a ban list never holds a neighbour, and the mesh's own bans are its own wherever they hang' (#299) from fix/a-ban-list-never-holds-a-neighbour into main 2026-10-02 16:43:32 +00:00
jschoubben d4a2f99ab5 ADR 0186: a ban list never holds a neighbour, and the mesh's own bans are its own wherever they hang; design 31 2026-10-02 18:42:16 +02:00
mesh-admin a9f91fdd0c Merge pull request 'ADRs 0184 and 0185: a service is still running a moment later; a control plane behind its row serves what it can' (#298) from fix/a-service-asked-to-run-is-still-running into main 2026-10-02 16:25:44 +00:00
jschoubben 131a5e4714 Issue 201: what was established about the race while closing the outage half 2026-10-02 18:19:12 +02:00
jschoubben 329a24fdae ADRs 0184 and 0185: a service is still running a moment later; a control plane behind its row serves what it can; issue 201 half closed 2026-10-02 18:16:24 +02:00
mesh-admin 7f72f3b79a Merge pull request 'ADR 0179: the intrusion seat serves its verbs, a container may log to the journal, and every door declares its jail' (#295) from feat/the-intrusion-seat-serves-its-verbs into main 2026-10-02 15:28:48 +00:00
jschoubben 114a71f36f ADR 0179: built and proven live; the one fault the machine found, and the check that refuses it 2026-10-02 17:28:38 +02:00
jschoubben 0f407417f3 Merge main: the ufw record renumbered to 0180, and ADR 0175 retires the per-module tool runtime this one ships 2026-10-02 17:28:23 +02:00
jschoubben d0d5799884 Issue 201: a push recreated the controller at a digest older than the seat row its successor wrote 2026-10-02 17:15:22 +02:00
jschoubben 9ba4de5557 ADR 0179: the intrusion seat serves its verbs, a container may log to the journal, and every door declares its jail; designs 31 and 33 2026-10-02 17:02:49 +02:00
18 changed files with 770 additions and 8 deletions
+1 -1
View File
@@ -108,7 +108,7 @@ term retired here may still appear there, and the mapping above is how to read i
memberships issue; its serving mode on loopback is what was called **the console**
([ADR 0175](../02-DECISIONS/0175-one-tool-runtime-per-node-serves-every-modules-tools-on-the-host-side.md)).
Replaces **"console"** as the module's name; *console* remains the word for the person's end of it.
- **bundle** — the artifact a module's tools are built into, interpreted or compiled; never an image.
- **bundle** — the artifact a module's own code is built into — its tools, a seat's implementation, a daemon — in any language the mesh has a toolchain for, interpreted or compiled; never an image. One module may declare several ([ADR 0188](../02-DECISIONS/0188-a-modules-own-code-is-bundles-in-any-language-and-a-tools-bundle-speaks-mcp-to-the-runtime.md)).
- **kept region** — a marked block in a managed file the mesh writes *into*, where the operator's own
lines survive every push and are given back when the module goes
([ADR 0174](../02-DECISIONS/0174-a-node-varies-a-module-through-settings-and-kept-regions-never-an-edit.md)).
@@ -8,6 +8,8 @@ reconstructed: false
# 39. What the SDK holds, and what it refuses
> **The mechanism changed — 2026-10-02, by [ADR 0188](0188-a-modules-own-code-is-bundles-in-any-language-and-a-tools-bundle-speaks-mcp-to-the-runtime.md).** The test of this record — frequent *and* cascading does not belong — stands and now applies to one SDK per language. Where *what it holds* names the broker client and the event consumer, read: the local protocol a tools bundle speaks to the node's runtime; the transport lives in the runtime and in no SDK, which is what keeps a bus change from rebuilding any module in any language.
_Reconciliation note (2026-09-05): supersedes the earlier "repository structure" decision, which the consolidation folded; no standalone record remains to point at, so body references to it now point at the nearest surviving record, [ADR 0015](0015-applications-live-in-their-own-repository.md)._
## Context
@@ -9,6 +9,8 @@ extends: 02-DECISIONS/0047-a-module-runs-its-code-as-its-own-process-with-its-ow
# 150. A module's own code runs as supervised processes under the module's one account
> **Widened — 2026-10-02, by [ADR 0188](0188-a-modules-own-code-is-bundles-in-any-language-and-a-tools-bundle-speaks-mcp-to-the-runtime.md).** A module's long-lived process is a bundle in any language the mesh has a toolchain for, run as a unit the host writes; this record never said one language and never meant one, and 0188 says so as the rule.
> **The mechanism changed — 2026-10-02, by [ADR 0175](0175-one-tool-runtime-per-node-serves-every-modules-tools-on-the-host-side.md).** For a module's *tools*, read that record: one runtime per node, the node's one account, bundles loaded from the memberships. This record still governs a module's long-lived processes — a daemon, a provisioner, a scheduled ingest — and the account invariant for them.
## Context
@@ -9,6 +9,8 @@ extends: 02-DECISIONS/0150-a-modules-own-code-runs-as-supervised-processes-under
# 175. One tool runtime per node serves every module's tools, on the host side
> **The mechanism changed — 2026-10-02, by [ADR 0188](0188-a-modules-own-code-is-bundles-in-any-language-and-a-tools-bundle-speaks-mcp-to-the-runtime.md).** Everything decided here stands: one runtime per node, host-side, every module's tools and every held seat's verbs on the memberships' subjects, root the module's concern, any node calls any tool, the console its serving mode. What moved is how the runtime brings a bundle to life. Decision 3 and the consequence *the node tools runtime needs an interpreter on the machine* read as though a bundle were always interpreted code the runtime imports; a tools bundle is now a process in any language that speaks MCP over stdio to the runtime, and importing a TypeScript bundle is the shortcut, not the contract.
## Context
A module's tools are code the module wrote, one function behind each verb, served on the subjects
@@ -0,0 +1,132 @@
---
topic: the mesh
status: accepted
date: 2026-10-02
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0170-the-firewall-seat-serves-its-verbs.md
---
# 179. The intrusion seat serves its verbs, a container may log to the journal, and every door declares its jail
## Context
Read on the control node on 2026-10-02, the day the machines were confirmed filtered by the mesh
alone ([ADR 0168](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md)): the intrusion
prevention watched one door. Its two jails read the ssh daemon's journal and its own log, banned
five failures in ten minutes for ten minutes, and in a day had seen twelve thousand failed logins
from three hundred addresses and banned none of the busiest, which paced themselves at one try every
ten minutes. The mail submission port took a hundred and sixty password guesses in the same day from
thirty-eight addresses with no jail reading it at all; the forge and the public proxy had no jail
either, and the proxy logged nothing a jail could read. Nobody could see the jails without a shell:
the module's three tools existed in code and were served by nothing, and the seat it holds declared
no verbs.
Three things were missing and they are three shapes the mesh already has. The packet filter's seat
serves verbs every holder owes ([ADR 0170](0170-the-firewall-seat-serves-its-verbs.md)); the
intrusion seat serves none. A module's `listens` compose into the machine's filter, and [to-be 31](../03-DESIGN/01-to-be/31-a-module-declares-its-fail2ban-jail.md)
says a module's `jails` compose into the machine's intrusion prevention the same way — the controller
composes them, and no module declares one. And a jail reads a log; a container's output goes to a
file of the runtime's own under a path that changes when the container is recreated, which is why
no jail could read the mail front end, the forge or the proxy, however they logged.
## Decision
**1. The `node-intrusion-prevention` seat serves four verbs**, and a module that claims it serves
all four or is refused the claim, as with every seat:
- `status` — every jail with what it watches, how many addresses it is counting failures against and
holding now, and the totals since it started; one jail's detail when named. Read-only.
- `banned` — every address banned now, with the jail holding it, when it was banned and when the ban
ends. Read-only.
- `ban` — ban one address in one jail now, for that jail's ban time. An operator's act on the live
ban list, which the mesh composes the rules for and never writes itself.
- `unban` — let one address go, from one jail or from every jail.
A holder may serve its own tools beside these; the fail2ban module reads one jail's effective
settings as its own.
**2. A container may log to the journal.** `logging: journald` on a container has the host run it
with the journal as its log driver; the journal keeps the container's name on every line, and
`docker logs` keeps working. Where a container logs is part of its spec, so moving it recreates the
container, and the only place besides the runtime's own file is the journal: a machine's intrusion
prevention reads the journal already, for the ssh daemon, and a container that logs there is read
the same way, by the container's name, whatever the container is called by the runtime this time.
**3. A module with a door declares its jail, and the holder composes them.** What to-be 31 designed
is now the rule: a module whose service authenticates from outside — the mail front end, the forge,
the public proxy — declares in its manifest what a failed attempt looks like in its log and how to
ban on it, naming no node and no path; the module that holds the intrusion seat declares where the
composed jails and filters land, and the mesh writes them on every machine that runs both. A machine
not running the module has no such jail. The holder restarts its daemon on the composed file.
**4. The base is strict, and the mesh's own range is never banned.** Three failures in a day ban for
a day, on every jail unless the jail says otherwise; banned twice in two weeks, by any jail, is
banned for four. The attackers this mesh sees pace themselves under any ten-minute window; a day's
window counts them. A person who mistypes three times from one address is out for a day from that
address, and never from a machine of the mesh, whose range stays in the never-banned list the module
has carried since [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md). The operator
chose this knowing it.
**5. The proxy says a refused name in its log.** A request for a name this mesh does not serve, from
outside, is what a scanner does; the proxy already logged a certificate refused for such a name, and
now logs the plain request too, with the asking address last, as its own jail's filter expects it.
## Consequences
- The seat's row gains four verbs; a mesh that already runs widens its row at the next controller
start. The fail2ban module claims them and gains a runtime — a tool server whose image carries the
fail2ban client, with the daemon's socket shared in from the machine, and nothing else of the
machine. The daemon stays the machine's; what runs in the container is only the client.
- **That runtime is the shape the catalogue has today, and it is on its way out.**
[ADR 0175](0175-one-tool-runtime-per-node-serves-every-modules-tools-on-the-host-side.md),
accepted the same day as this record, replaces a tool container per module with one tool runtime
per node on the host side, taking each module's tools as a bundle. Nothing here depends on the
container: the verbs, the client that speaks to the daemon over its socket, and the jails are the
same code under either. This module converts with the packet filter's, whose runtime that record
names, and the socket it needs becomes the node runtime's to reach rather than a mount of its own.
- The host's container vocabulary grows by `logging`; an older host refuses a declaration that carries
it, so the host rolls before the modules. Three containers are recreated once, when their modules
are pushed with the field: the mail front end, the forge and the proxy — each a moment's outage.
- The fail2ban module declares where jails compose (`jailing`) and the directory the filters go in;
the mail, forge and proxy modules each declare one jail reading the journal by their container's
name. The composed jail file is the one resource the daemon restarts on when a module arrives or
leaves a machine.
- The two base jails and the composed ones take the day's window; the ssh jail's ten minutes are
gone. An address banned on the first day of this record stays banned for the day.
- The module's three old tools, served by nothing, are replaced by the seat's four verbs and one
own tool; `fail2ban_status` as a name is gone.
## How this is checked
| Rule | Checked by |
|---|---|
| The seat declares the four verbs; a claim that serves fewer is refused by name | the catalogue's seat tests |
| `status`, `banned`, `ban` and `unban` read and steer the daemon through its client, with the shapes fail2ban 1.1.0 printed live; a non-address and a non-name are refused before anything runs | the module's tests over a fake command runner |
| A container's `logging` reaches the runtime's arguments and its spec; a place other than the journal is refused | host tests |
| A module's jails compose into the holder's file and a filter per jail, and the file is written empty when none is declared | the controller's composition tests (to-be 31) |
| The proxy logs a refused name with the address last | the proxy's tests |
| A jail's pattern names `<HOST>` once per shape, since two is a duplicate capture group and costs the machine every ban | the catalogue's manifest tests |
| Live | done 2026-10-02: `status` and `banned` answered on both servers through the console; the proxy's jail counted seven refusals on the home server; a documentation address banned in the ssh jail came back with its end time and was released |
## Built and proven live, 2026-10-02
All five rules are in the mesh. The host carries `logging`; the controller's seat row carries the four
verbs and the proxy says a refused name in its log; the fail2ban module holds the seat from a runtime
with the daemon's socket shared in, composes the jails, and the mail front end, the forge and the
proxy each declare one. Through the console on the control node: `status` listed five jails with what
each watches, `banned` listed the nine the long jail holds, and a documentation address banned in the
ssh jail came back with its ban's end time and was released again. On the home server the proxy's jail
had counted seven refusals within minutes of starting.
**One fault, found by the machine and not by a test.** The proxy's pattern matched two shapes of
refusal in one expression and so named `<HOST>` twice. fail2ban expands that placeholder into a named
capture group; two of them is a duplicate group name, and the daemon refuses *its whole configuration*
and exits — both servers kept no bans at all for about ten minutes, every jail and not the one at
fault. The pattern is now one per shape. A manifest check refuses the mistake at merge time, naming
what it would cost, which is the only reason this record can claim the rule rather than the instance.
## References
- [ADR 0170](0170-the-firewall-seat-serves-its-verbs.md), [ADR 0168](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md), [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0159](0159-a-tool-call-names-the-machine-and-a-holder-serves-its-seats-verbs.md), [ADR 0175](0175-one-tool-runtime-per-node-serves-every-modules-tools-on-the-host-side.md)
- [Design 31 — A module declares its fail2ban jail](../03-DESIGN/01-to-be/31-a-module-declares-its-fail2ban-jail.md), [Design 33 — The tools the mesh answers](../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md), [Design 08 — Connectivity](../03-DESIGN/01-to-be/08-connectivity.md)
@@ -60,7 +60,7 @@ configuration file; the rollback path it described is given up on purpose.
|---|---|
| An absent package is removed when present, left when not, and read back | host tests over a fake package manager |
| An uninstalled front end is recorded as removed and nothing is asked of it | a host test with ufw missing on a converged apply |
| Live | the two machines report ufw gone: `pacman -Q ufw` has no answer, `node show` says removed, `status` is well |
| Live | done 2026-10-02: both machines report ufw gone — `pacman -Q ufw` has no answer, `node show` says *removed*, `status` is well. The home server said *retired* for six hours after the package went, because this record's step runs only after a clean apply ([ADR 0168](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md)) and one dead tracker was failing its applies ([ADR 0187](0187-a-dead-tracker-is-not-the-machines-failure.md)) |
## References
@@ -0,0 +1,75 @@
---
topic: the mesh
status: accepted
date: 2026-10-02
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0005-the-node-host.md
---
# 184. A service the mesh asked to run is still running a moment later
## Context
The host already refuses to take a service manager's word for it. Three places in one function read
a unit back after acting on it, each with a comment saying why: *a service manager accepting a
command says the transaction was accepted, not that the unit is running — one that starts and
immediately dies satisfies it.* The intent was right and the implementation did not reach it.
On 2026-10-02 the mesh composed a fail2ban jail whose pattern the daemon refused. The host wrote the
files, restarted the service, read the unit back and reported *restarted*. The unit was `active` at
that instant and `failed` 221 milliseconds later, which the unit's own record states. Both public
machines then kept no bans at all — every jail, not the one at fault — and nothing in the mesh said
so. The fault was found by calling a tool that needed the daemon, not by the mesh noticing.
The read-back races the failure. A service manager returns when it has started the process; a daemon
that reads its configuration, refuses it and exits does so a fraction of a second afterwards. One
look sees `activating` or `active` whatever the process is about to do, and *the host reports success
for a machine that is already wrong* — the one shape of failure this host exists to refuse
([ADR 0005](0005-the-node-host.md)).
A command the module declares — *test the configuration before restarting* — was considered and
rejected. The link carries no actions ([ADR 0005](0005-the-node-host.md)), and a verification
command is a command: a declaration that carried one would be remote execution over the bus,
arriving as root on every machine, which is a far larger door than the fault it closes. The host
does not need one. It already knows what it asked for.
## Decision
**1. A unit the host has just asked to run is read twice**, with a pause between the reads long
enough for a daemon that refuses its configuration to have exited. Not running at the second look is
a failure of that resource, named with the unit and the state it is in — the same failure the single
read was always meant to catch.
**2. It is never a wait for a unit to come up.** A unit still starting reads as running at both
looks and is accepted, exactly as before. What the second look catches is a unit that *was* running
and is not any more. A service asked to be stopped is not waited on at all.
**3. The host tests nothing and runs nothing of a module's.** The second look is the host checking
the state it was told to establish, which is its whole job; the declaration gains no vocabulary, and
no command reaches a machine that did not already come from a built artifact.
## Consequences
- Every apply that starts, restarts or reloads a service spends a moment confirming it. The cost is
bounded by the number of services that changed in that apply, which is usually none.
- A module whose configuration the mesh composes — the packet filter, the intrusion prevention, the
resolver — now fails its apply when the composition is bad, instead of reporting success onto a
dead daemon. `status` names the machine, which is how the operator finds out.
- It does not prevent the bad composition. [ADR 0179](0179-the-intrusion-seat-serves-its-verbs-and-every-door-declares-its-jail.md)'s
manifest check is what refuses the one that caused this, at merge time; this record is what makes
the *next* one visible within a minute rather than invisible until something asks the daemon a
question.
## How this is checked
| Rule | Checked by |
|---|---|
| A unit that is running at the first look and dead at the second fails the apply, naming the unit and its state | a host test over a service manager that answers as systemd does |
| A unit still starting is accepted at both looks | a host test |
| A service asked to be stopped is not waited on | a host test |
## References
- [ADR 0005](0005-the-node-host.md), [ADR 0179](0179-the-intrusion-seat-serves-its-verbs-and-every-door-declares-its-jail.md)
- [Design 05 — The node host](../03-DESIGN/01-to-be/05-the-node-host.md)
@@ -0,0 +1,73 @@
---
topic: the mesh
status: accepted
date: 2026-10-02
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0154-the-meshs-own-verbs-are-the-controller-seats-tools.md
---
# 185. A control plane behind its seat's row serves what it can
## Context
The mesh's own verbs are the controller seat's tools, and the seat's row is the store's
([ADR 0154](0154-the-meshs-own-verbs-are-the-controller-seats-tools.md)). A control plane reads the
row at start and installs a handler per verb; a verb the row carries that the binary cannot run was
refused at start rather than at the first call, so that a disagreement between the row and the
binary was said early. The refusal aborted the start.
On 2026-10-02 a merge added one verb. The new control plane started, widened the row, and ran. A
push a few seconds later recreated its container at the previous image — a stale declaration from
an overlapping wave, [issue 201](../04-ISSUES/201-a-push-recreated-the-controller-behind-the-row-its-successor-wrote/00-report.md) —
and the older binary read a row naming a word it had never heard. It refused to start, and kept
refusing. The mesh had no voice for ten minutes: no verb answered, no node could be pushed, no build
was dispatched, and `status` said nothing because `status` is one of the verbs that had stopped
being served. The way back was a person running the binary by hand outside its service, because the
push that would have replaced it is itself a verb of the control plane that was down.
The check was right about the fact and wrong about the cost. A row ahead of a binary is the ordinary
state of a roll-out: the row is widened by whichever control plane starts first, and a mesh with one
control plane sees that gap on every merge that adds a verb. Making it fatal turned a transient into
an outage with no path out that did not need a human.
## Decision
**1. A control plane serves the verbs it can run and does not refuse to start for the ones it
cannot.** The row remains the authority on what the seat serves; this is only about what this binary
does when it is behind the row.
**2. A verb it cannot run answers the reason.** Not silence and not a missing subject: a caller gets
a sentence naming the verb, saying this control plane cannot run it and that it is a verb of a newer
build. A verb that is simply absent from the row is still not served at all — that is the row
deciding, which is unchanged.
**3. It says so once at start**, naming every verb of the row it cannot run, so the gap is visible
in the log of the thing that has it rather than only at the moment somebody calls one.
**4. A mesh with no controller seat at all is still a refusal.** That is not a version gap, it is a
mesh that has not been seeded, and nothing this control plane does would be meaningful.
## Consequences
- An overlapping roll-out costs the verbs the newer build added, for as long as the older binary is
in place. Everything else — every push, every build, every read — keeps working, and the ordinary
machinery that notices a machine is behind is what puts the newer binary back.
- The log gains one line on a control plane that is behind, and nothing on one that is not.
- Issue 201's other half remains: the push that sent a stale declaration is a race worth closing on
its own terms. This record makes that race survivable rather than fatal, which is the difference
between a transient and an outage, and is deliberately the cheaper half.
## How this is checked
| Rule | Checked by |
|---|---|
| A row carrying a verb this build cannot run still serves every verb it can, and names the one it cannot | a controller test over a widened row |
| The unknown verb answers a sentence naming itself and saying this build is behind | the same test |
| A mesh with no controller seat is refused | the existing start-up path |
## References
- [ADR 0154](0154-the-meshs-own-verbs-are-the-controller-seats-tools.md), [ADR 0162](0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md)
- [Issue 201](../04-ISSUES/201-a-push-recreated-the-controller-behind-the-row-its-successor-wrote/00-report.md)
- [Design 33 — The tools the mesh answers](../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md)
@@ -0,0 +1,81 @@
---
topic: the mesh
status: accepted
date: 2026-10-02
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0179-the-intrusion-seat-serves-its-verbs-and-every-door-declares-its-jail.md
---
# 186. A ban list never holds a neighbour, and the mesh's own bans are its own wherever they hang
## Context
[ADR 0179](0179-the-intrusion-seat-serves-its-verbs-and-every-door-declares-its-jail.md) gave the
public proxy a jail. Within the hour the home server's ban list held `192.168.1.1` — the house's own
router. The router reflects local traffic, so every client in the building reaches that machine as
the gateway's address; one local request for a name the mesh does not serve, three times in a day,
and the whole house is refused by the machine it was asking. The jails inherited an `ignoreip` of
the loopback and the mesh's own range, which was right when the only jail read the ssh daemon and
the only clients were the mesh's; a jail on a public front door sees the neighbours too.
The same jail broke the other half of [ADR 0168](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md).
The home server began reading *NOT the mesh alone: 1 rule set the mesh did not write refuses traffic
here*, and the rule set named was the mesh's own ban chain, written by the mesh's own intrusion
prevention minutes earlier. The host's reader of the legacy filter required every path into a chain
of refusals to come from a built-in chain whose policy accepts, before it would call that chain a
ban. On that machine the chain hangs off the container runtime's user chain as well as the input
chain, and the runtime had set the forward policy to DROP — so the mesh reported its own work as a
foreigner's, on the one machine where the group's exit condition was supposed to hold.
Both faults are one mistake in two places: a rule written about the public internet, applied to
everything that arrives.
## Decision
**1. A ban list never holds a neighbour.** The jails the mesh composes never ban a source on a
private range — the mesh's own range, which was already named rather than written
([ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md)), and every address space
reserved for private use beside it, in both families. A machine behind a router that reflects local
traffic sees its whole building as one address; a ban there is a self-inflicted outage, and the
sources worth banning are not on those ranges in the first place.
**2. The mesh's own bans are its own wherever they hang.** A chain of refusals is a ban list when
every refusal names the sources it refuses and the chain accepts nothing — the rule the host already
applied to the packet filter's own tables, now applied to the legacy filter too, and nothing more.
The policy of the chains that jump into it says nothing about what it is: that policy is already
classified where it belongs, as the container runtime's, and requiring it here counted it twice.
**3. A chain that accepts anything is still not a ban.** That is what keeps a predecessor's
allow-these-and-drop-the-rest chain classified as something an operator must look at, which is the
distinction [ADR 0168](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md) exists to draw.
## Consequences
- The composed jails gain the private ranges in their never-ban list. An address already banned
stays banned until it is released; the house's router was released by hand the moment it was found.
- The home server reads *the mesh alone* again, which is group 7's exit condition and was false for
about an hour.
- A machine whose apply fails for an unrelated reason does not revisit its found firewall's record
at all — the step runs only after a clean apply ([ADR 0168](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md)).
The home server's record therefore still reads *retired by the mesh* although the front end is
uninstalled, and will correct itself once that machine's own stuck module is fixed. It is a stale
record, not a wrong machine.
- The record number the front end's removal was given moved under it: another session took 0175
while that record was in review, and it is now
[ADR 0180](0180-the-found-front-end-is-uninstalled-once-a-machine-is-converged.md). The citations
the host and the control plane print were pointing at an unrelated record and are corrected here.
## How this is checked
| Rule | Checked by |
|---|---|
| A private source is never banned | the module's jail configuration, read back by `fail2ban.fail2ban_settings` on a machine |
| The mesh's own ban chain reads as a ban behind a dropping forward policy | a host test over the home server's own captured rule set |
| A chain that accepts anything is not a ban | a host test |
| Live | done 2026-10-02: all four machines read *the mesh alone*, the home server counting its own ban chain as a ban; no ban held anywhere is a private address |
## References
- [ADR 0179](0179-the-intrusion-seat-serves-its-verbs-and-every-door-declares-its-jail.md), [ADR 0168](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md), [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0180](0180-the-found-front-end-is-uninstalled-once-a-machine-is-converged.md)
- [Design 08 — Connectivity](../03-DESIGN/01-to-be/08-connectivity.md), [Design 31 — A module declares its fail2ban jail](../03-DESIGN/01-to-be/31-a-module-declares-its-fail2ban-jail.md)
@@ -0,0 +1,73 @@
---
topic: the mesh
status: accepted
date: 2026-10-02
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0136-a-step-gates-its-module-not-the-machine.md
---
# 187. A dead tracker is not the machine's failure
## Context
The home server had not applied a declaration cleanly since midday. One run-once step — the one
that writes a media app's download clients and indexers through the app's own API — exited
non-zero, forty-nine times over six hours, for one public tracker that had stopped answering. The
step's own words: the entry was *written*, and the app's test of it then failed with a 400 from the
indexer proxy. The machine reported *not doing what it was told* for the rest of the day.
What that gated matters more than the step. A converged machine retires the firewall it was found
with only after a clean apply ([ADR 0168](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md)),
so that machine went on recording its found front end as merely *retired* long after the package
had been uninstalled ([ADR 0180](0180-the-found-front-end-is-uninstalled-once-a-machine-is-converged.md)).
A dead public tracker was holding a firewall record hostage, which is not a connection anybody
would design.
The step already knew this was not its business. It had a rule for exactly this: an entry the mesh
only *found and re-pointed*, rather than one it was told to make, whose feed is gone, is said and
left as found — *failing the node's apply on every heartbeat for it reports the mesh as wrong about
a tracker*. The rule was there and matched one shape of the fault. An app can refuse to save such an
entry, and it can save it and then fail its own test; saving validates settings, and the test runs a
live search. The rule caught the first and let the second through.
## Decision
**1. An entry the mesh only found is never the machine's failure.** Whatever shape the app's
refusal takes — it would not save it, or it saved it and its own test fails — an indexer the mesh
found and re-pointed is reported as a notice and left as found. What decides is whose entry it is,
not which sentence the app returned.
**2. What the mesh is answerable for is the plumbing.** That the entry exists, points at this
mesh's indexer proxy, and carries the credential the mesh delivered — which was checked against the
proxy before anything was written. Whether a public tracker answers today is not the mesh's to
promise, and a machine that reports itself broken because one did is lying about itself.
**3. An entry the operator listed is theirs to insist on.** An indexer named in the step's settings
is one the mesh was told to make, and it still fails the step when it cannot be made to work. The
notice says so, and says that listing the indexer is how to turn it back into a failure.
## Consequences
- The home server applies cleanly again, and everything a clean apply gates — its found firewall's
record among it — follows.
- A tracker that dies is a line in a report rather than a machine that reads as broken. An operator
who wants it gone removes the entry or repairs the feed; the mesh says which, every time it runs.
- The four Servarr modules carry one byte-identical copy of this step each
([ADR 0069](0069-a-module-is-a-repository-and-a-path.md)), so the change lands in four places and
a test refuses any drift between them.
- It does not widen to a download client: one the mesh was told to write and cannot is still a
failure, because the mesh chose it and nothing else will fix it.
## How this is checked
| Rule | Checked by |
|---|---|
| A found feed whose tracker answers an error after the entry was written is a notice | the step's tests, with the home server's own message and the app's two validations modelled apart |
| An indexer the settings list is still a failure | the same test |
| The four copies of the step do not drift | the step's own sameness test |
| Live | done 2026-10-02: the home server applies cleanly after six hours of failing, `status` holds no machine wrong or behind, and its found firewall reads *removed* |
## References
- [ADR 0136](0136-a-step-gates-its-module-not-the-machine.md), [ADR 0168](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md), [ADR 0180](0180-the-found-front-end-is-uninstalled-once-a-machine-is-converged.md), [ADR 0069](0069-a-module-is-a-repository-and-a-path.md)
@@ -0,0 +1,127 @@
---
topic: what runs on it
status: accepted
date: 2026-10-02
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0175-one-tool-runtime-per-node-serves-every-modules-tools-on-the-host-side.md
---
# 188. A module's own code is bundles in any language, and a tools bundle speaks MCP to the runtime
## Context
[ADR 0175](0175-one-tool-runtime-per-node-serves-every-modules-tools-on-the-host-side.md) put one
tool runtime on every node and said a module brings its tools as a bundle. The runtime that exists
is written in TypeScript and brings a bundle to life by **importing it into its own process**, which
only JavaScript can be. The SDK ([ADR 0039](0039-what-the-sdk-holds-and-refuses.md)) is one
TypeScript package. The builder knows three toolchains — TypeScript, Go, Python — and every one of
the 35 catalogue modules with tools wraps them in a container on the runtime's TypeScript image.
Nothing in the records says a module's code may be written in anything else, and nothing refuses a
module that wraps its own code in an image to get around that.
The operator's direction, stated on 2026-10-02 and repeated: *the SDK is the most important part;
we must not limit developers; tools can be written in any possible language — Rust, C, Go,
JavaScript. A service in Go or Rust as a systemd unit must be possible too. One module can deliver
all kinds of bundles: one for its tools, one for a seat's implementation, one for a daemon. Support
the bare minimum first, as a skeleton; a full implementation comes when the work requires it.*
Measured against that: the `bundle` artifact kind already names a language and the `process`
resource already runs a command from an unpacked bundle as a unit the host writes
([ADR 0150](0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md)), so a Go
daemon as a native service is possible today and one module in the catalogue does it. What is not
possible is a tool in any language but one, and what is not written is that any of this is the
rule.
## Considered Options
1. **One SDK, one language, as now.** Rejected: it limits who can write a module to one
ecosystem, which the operator declines, and it is what made every module's tools a container
on one image.
2. **A full bus client per language.** Each SDK speaks the bus itself; the runtime only
supervises. Rejected: a transport in every SDK is what ADR 0039 refuses, and a bus change
would then rebuild every module in every language — the cascade, multiplied.
3. **A tools bundle is a process the runtime launches and speaks a small local protocol to,
and that protocol is MCP over stdio.** Chosen. The runtime already speaks MCP outward (the
console); speaking it inward to a child process is the same vocabulary. Every language that
has an MCP server library can write a tools bundle today with no mesh SDK at all, and the
mesh's own SDK for a language is a thin convenience over it. The transport stays in the
runtime, so a bus change rebuilds nothing.
4. **A protocol of the mesh's own design.** Rejected: a second way to describe a tool, its
schema and its call, inventing what MCP already settled, for no gain.
## Decision
**1. A module's own code is bundles, in any language the mesh has a toolchain for, and never an
image.** A `bundle` names its language and what it is for. Images are for third-party software a
module installs — a database, a forge — never for code the module wrote. One module may declare
several bundles: its tools, its implementation of a seat's verbs, a daemon, a step. Each is built
alone and delivered alone, as [ADR 0156](0156-an-artifact-is-what-a-build-produces-and-the-store-is-named-for-its-scope.md)
already has it.
**2. A bundle the runtime serves is a process that speaks MCP over stdio.** The node's runtime
launches it as the bundle names it — an interpreter and a file, or a binary — with the runtime's
environment, asks `tools/list`, and answers each call on the bus by `tools/call`. A tool whose name
is `<seat>.<verb>` is the module's implementation of that seat's verb; any other name is the
module's own tool. Everything the runtime does with what it is told — subjects from the membership,
a held seat's verbs, the `tools` answer, a bundle that fails named and the others serving — stays as
[ADR 0175](0175-one-tool-runtime-per-node-serves-every-modules-tools-on-the-host-side.md) and
[ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)
have it. A TypeScript bundle may still be imported into the runtime's own process; that is a
shortcut over the same contract, not a second contract, and a TypeScript bundle written against
the protocol is served the same way as any other.
**3. A bundle that is a service is a `process`**, run by the host as a unit, in whatever language it
is compiled from, exactly as the host's own bundle already is. Nothing new is decided here; it is
said so that it is the rule and not an example.
**4. One thin SDK per language, and the test of ADR 0039 applies to each.** An SDK for a language
holds the MCP-over-stdio loop, the tool-definition type and the few primitives a module's code
needs; it holds no transport, no module's client and nothing volatile. Where a language has a
sound MCP library, the SDK wraps it rather than re-implementing it. The languages are those that
make sense to write a module in; the first set is TypeScript, Go, Python, Rust and C, and the set
grows when a module needs one, not before.
**5. Skeleton first.** Each piece — a toolchain, a launcher, an SDK — exists at the bare minimum
that lets one bundle in that language be built, delivered and answer one tool on the live mesh.
Anything beyond that is added when a module needs it. A skeleton that is not proven by one bundle
answering is not a skeleton; it is a promise.
## Consequences
- The runtime gains a launcher beside its loader. The loader, the memberships, the seats and the
failure handling built for ADR 0175 stand; the launcher is the one new step.
- The builder gains a toolchain per language, each at the skeleton: compile, pack, name the
entrypoint. Rust and C are new; a language that compiles to a binary says its operating system
as a Go bundle already does.
- An existing MCP server in any language is already a valid tools bundle. What the mesh adds is
the subjects, the seats and the memberships around it.
- The gate [to-be 38](../03-DESIGN/01-to-be/38-building-the-operators-machine.md) WP2 adds —
refusing a tools container built on the runtime's image — widens: a module whose own code is
an image artifact is refused at registration, naming this record.
- What got harder: a tools bundle is now a process per module on the node rather than code in
one process, so the runtime supervises children and restarts one that dies. The one-process
shape ADR 0175 counted on for the TypeScript shortcut remains available for it.
- ADR 0039's "what the SDK holds" now reads per language; its refusals are unchanged and are the
reason option 2 was rejected.
## How it is checked
| Rule | Checked by |
|---|---|
| A module's own code is never an image | the catalogue's registration check: a manifest with a `bundle` kind of own code *and* an image artifact built from the module's own directory is refused, naming this record |
| A tools bundle in a language other than TypeScript answers on the bus | the runtime's tests: a bundle written against the protocol in a second language, launched, its tool called over a real bus |
| A TypeScript bundle written against the protocol is served like any other | the same tests, with the TypeScript shortcut off |
| Each SDK is thin | each SDK's own README states what it holds under ADR 0039's test, and its size is in the mesh's records |
| Live | a tool in a compiled language answers from the node's runtime on one machine |
## References
- [ADR 0175](0175-one-tool-runtime-per-node-serves-every-modules-tools-on-the-host-side.md),
[ADR 0039](0039-what-the-sdk-holds-and-refuses.md),
[ADR 0150](0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md),
[ADR 0156](0156-an-artifact-is-what-a-build-produces-and-the-store-is-named-for-its-scope.md),
[ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)
- [To-be 38](../03-DESIGN/01-to-be/38-building-the-operators-machine.md) — the work packages this
record widens
- The Model Context Protocol's stdio transport — the local protocol a tools bundle speaks
+6
View File
@@ -182,7 +182,12 @@ python3 00-META/checks/index.py fail if stale
- **0169** — [A machine joins through the tunnel, and the bus is never public](0169-a-machine-joins-through-the-tunnel-and-the-bus-is-never-public.md)
- **0170** — [The firewall seat serves its verbs, and a foreign rule set is removed through one of them](0170-the-firewall-seat-serves-its-verbs.md)
- **0172** — [The lab is a module, and runs a bed when the mesh asks](0172-the-lab-is-a-module-and-runs-a-bed-when-the-mesh-asks.md)
- **0179** — [The intrusion seat serves its verbs, a container may log to the journal, and every door declares its jail](0179-the-intrusion-seat-serves-its-verbs-and-every-door-declares-its-jail.md)
- **0180** — [The found front end is uninstalled once a machine is converged](0180-the-found-front-end-is-uninstalled-once-a-machine-is-converged.md)
- **0184** — [A service the mesh asked to run is still running a moment later](0184-a-service-the-mesh-asked-to-run-is-still-running-a-moment-later.md)
- **0185** — [A control plane behind its seat's row serves what it can](0185-a-control-plane-behind-its-seats-row-serves-what-it-can.md)
- **0186** — [A ban list never holds a neighbour, and the mesh's own bans are its own wherever they hang](0186-a-ban-list-never-holds-a-neighbour.md)
- **0187** — [A dead tracker is not the machine's failure](0187-a-dead-tracker-is-not-the-machines-failure.md)
### Its tiers, from the bottom up
@@ -280,6 +285,7 @@ python3 00-META/checks/index.py fail if stale
- **0181** — [The operator account is a node fact, and a home is a placement root](0181-the-operator-account-is-a-node-fact-and-a-home-is-a-placement-root.md)
- **0182** — [Inside a home, the mesh owns the directory and the files it places, writes into the tool's own files, and holds everything else as found](0182-inside-a-home-the-mesh-owns-what-it-places-and-holds-the-rest-as-found.md)
- **0183** — [The Anthropic licence manager is a module holding a seat; it hands each node's agent its token over the bus, sealed; the controller and the host have no part](0183-the-anthropic-licence-manager-is-a-module-and-hands-tokens-to-the-agent-over-the-bus.md)
- **0188** — [A module's own code is bundles in any language, and a tools bundle speaks MCP to the runtime](0188-a-modules-own-code-is-bundles-in-any-language-and-a-tools-bundle-speaks-mcp-to-the-runtime.md)
### How it is built
+15
View File
@@ -4,6 +4,7 @@ status: in-progress
code: [mesh-host]
updated: 2026-10-02
decisions:
- 02-DECISIONS/0184-a-service-the-mesh-asked-to-run-is-still-running-a-moment-later.md
- 02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md
- 02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md
- 02-DECISIONS/0141-the-host-delivers-its-own-successor.md
@@ -498,3 +499,17 @@ run and reported; the exit follows an in-flight apply rather than interrupting i
not start is rolled back once and the second failure halts; a completed reconcile retires what is older
than the predecessor and never the predecessor; and the newest of two delivered versions is the one
that runs.
## A service is still running a moment later, 2026-10-02
[ADR 0184](../../02-DECISIONS/0184-a-service-the-mesh-asked-to-run-is-still-running-a-moment-later.md).
The host has always read a unit back after acting on it, because a service manager accepting a
command says the transaction was accepted and nothing about the process. The read raced the failure:
a daemon that refuses the configuration the mesh just wrote exits a fraction of a second after the
manager returns, and one look sees it alive. So the host looks twice, with a pause between, and a
unit that was running and is not any more fails its resource by name. A unit still coming up reads
as running at both looks and is accepted; a service asked to stop is not waited on.
No command for this reaches a machine. A module declaring *how to test my configuration* was weighed
and refused: the link carries no actions, and a verification command is one. The host is checking
the state it was told to establish, which is what it is for. *How it is checked:* ADR 0184's table.
@@ -1,10 +1,15 @@
---
layer: to-be
status: proposed
code: []
updated: 2026-09-27
status: in-progress
code:
- mesh-controller: internal/catalogue/jails_into.go, internal/catalogue/manifest.go (Jail, Jailing)
- mesh-catalog: modules/fail2ban (jailing, the base and the seat's verbs), modules/mailu, modules/route-proxy, modules/gitea (jails)
- mesh-host: internal/declaration/declaration.go (a container's logging)
updated: 2026-10-02
decisions:
- 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md
- 02-DECISIONS/0179-the-intrusion-seat-serves-its-verbs-and-every-door-declares-its-jail.md
- 02-DECISIONS/0186-a-ban-list-never-holds-a-neighbour.md
---
# 31 — A module declares its fail2ban jail, and the mesh composes them per node
@@ -63,3 +68,38 @@ jail, composed from the postgres module's manifest, without anyone editing a nod
beside)
- mesh-catalog `modules/fail2ban` (the base: sshd, recidive, ignoreip); the service modules
(`postgres`, `mssql`, `mailu`) that will declare jails
## Decided and built, 2026-10-02
[ADR 0179](../../02-DECISIONS/0179-the-intrusion-seat-serves-its-verbs-and-every-door-declares-its-jail.md)
made this the rule and built it. A module declares `jails` — each a name, the `failregex` of a
failed attempt in its log, and the stanza's own keys — and the fail2ban module declares `jailing`:
the one file the stanzas compose into and the directory each filter lands in. The controller gathers
every assigned module's jails per node into those; the holder's daemon restarts on the composed file.
What made it workable was the log. A container's output went to a file of the runtime's own, under
a path that changes when the container is recreated, so no jail could read a container's service
however it logged. A container now declares `logging: journald`, the host runs it with the journal as
its driver, and a jail reads it with `backend = systemd` and a `journalmatch` on the container's
name — the same way the base's ssh jail has always read the ssh daemon. The first three doors: the
mail front end (every login failure on its proxying ports), the forge (a failed authentication
attempt) and the public proxy (a certificate or request for a name the mesh does not serve, which
the proxy now says in its log). The base is strict — three in a day for a day; twice banned in two
weeks for four — and the mesh's own range stays never banned.
The seat the module holds serves `status`, `banned`, `ban` and `unban`, from a runtime that carries
only the fail2ban client with the daemon's socket shared in; the jails are composed, the ban list is
the daemon's, and both are read through the console.
*How it is checked:* ADR 0179's table.
## What the first jails taught, 2026-10-02
[ADR 0186](../../02-DECISIONS/0186-a-ban-list-never-holds-a-neighbour.md). Within an hour of the
first public jail the home server had banned the house's own router: the router reflects local
traffic, so every client in the building arrives as the gateway's address. The never-ban list now
holds every private range as well as the mesh's own. And the mesh read its own ban chain as a
foreign rule set on that machine, because the chain hangs off the container runtime's user chain and
that machine's forward policy is the runtime's DROP — the reader now calls a chain of source-named
refusals a ban wherever it hangs, as it already did for the packet filter's own tables.
@@ -11,7 +11,7 @@ code:
- mesh-host internal/apply/apply.go
- mesh-tools src/main.ts
- mesh-catalog modules/mesh-catalog
updated: 2026-09-28
updated: 2026-10-02
decisions:
- 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md
- 02-DECISIONS/0126-a-module-declares-its-own-seats.md
@@ -24,6 +24,7 @@ decisions:
- 02-DECISIONS/0129-a-seat-carries-the-protocol-of-its-role.md
- 02-DECISIONS/0135-a-module-version-prepares-its-state-before-it-runs.md
- 02-DECISIONS/0136-a-step-gates-its-module-not-the-machine.md
- 02-DECISIONS/0187-a-dead-tracker-is-not-the-machines-failure.md
- 02-DECISIONS/0134-the-mesh-says-what-it-applied.md
---
@@ -470,6 +471,21 @@ moment the mesh can mint for itself: **the bus's own accounts** (§the bootstrap
needs an account before it can run) and **the vault's own credential**. Any third exception is a
design failure, and naming these two is what makes a third one visible.
## What a step is answerable for, 2026-10-02
[ADR 0187](../../02-DECISIONS/0187-a-dead-tracker-is-not-the-machines-failure.md). A step gates its
module and not the machine ([ADR 0136](../../02-DECISIONS/0136-a-step-gates-its-module-not-the-machine.md)),
but a step that exits non-zero still leaves the machine reporting that it is not doing what it was
told — and a clean apply gates other things entirely, the found firewall's retirement among them. So
what a step calls a failure matters beyond the step.
The rule the media step now follows, and the one to copy: a step fails for what the mesh chose and
can fix, and reports what it merely found and cannot. An indexer entry the mesh re-pointed at this
mesh's proxy is plumbing the mesh is answerable for; whether the public tracker behind it answers
today is not. An entry the operator listed is the operator's to insist on, and still fails. Six
hours of a machine reading as broken, for one tracker that had died, is what the distinction costs
when it is missing.
## 11. Open
**Semantic change has no mechanical defence** (§8). Recorded as open rather than solved, because
@@ -5,6 +5,7 @@ code: [mesh-controller, mesh-tools]
updated: 2026-10-02
decisions:
- 02-DECISIONS/0170-the-firewall-seat-serves-its-verbs.md
- 02-DECISIONS/0179-the-intrusion-seat-serves-its-verbs-and-every-door-declares-its-jail.md
- 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md
- 02-DECISIONS/0159-a-tool-call-names-the-machine-and-a-holder-serves-its-seats-verbs.md
- 02-DECISIONS/0154-the-meshs-own-verbs-are-the-controller-seats-tools.md
@@ -184,6 +185,18 @@ container to declare a capability. Removing a predecessor's rule set is an opera
through the seat, recorded on the bus, instead of a shell on the machine. *How it is checked:* ADR
0169's table.
## The intrusion seat's verbs, 2026-10-02
[ADR 0179](../../02-DECISIONS/0179-the-intrusion-seat-serves-its-verbs-and-every-door-declares-its-jail.md).
The second node-scoped seat to carry verbs: `node-intrusion-prevention` serves `status` (every jail
with what it watches and holds), `banned` (every address held now, with its jail and when the ban
ends), `ban` and `unban` (an operator's act on the live ban list). The fail2ban module serves them
from a runtime that carries only the daemon's client, the socket shared in from the machine — no
capability, no machine network, since the daemon on the machine does the banning. That runtime is the
per-module container [ADR 0175](../../02-DECISIONS/0175-one-tool-runtime-per-node-serves-every-modules-tools-on-the-host-side.md)
retires; the verbs and the client are the same code once the node's own runtime loads them as a bundle.
The module's own tool beside them reads one jail's effective settings. *How it is checked:* ADR 0179's table.
## What this does not settle
- Which verbs each seat should serve. That is a decision per seat, and the reason to do it slowly: a
@@ -11,6 +11,7 @@ decisions:
- 02-DECISIONS/0177-a-unit-may-be-user-scoped-and-the-service-manager-is-a-node-seat.md
- 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md
- 02-DECISIONS/0149-the-live-mesh-is-the-test-bed.md
- 02-DECISIONS/0188-a-modules-own-code-is-bundles-in-any-language-and-a-tools-bundle-speaks-mcp-to-the-runtime.md
---
# 38. Building the operator's machine
@@ -97,6 +98,19 @@ what `tools` answers, and the others serve. The runtime reads `MESH_OPERATOR_ACC
five tools and two seat verbs answer on their subjects; `tools` names the failed bundle; a
membership republished mid-run re-subscribes without a restart.
*Built and proven 2026-10-02* (mesh-tools, branch `feat/the-operators-machine`, commit `6390d1d`).
**WP1b — the launcher beside the loader** ([ADR 0188](../../02-DECISIONS/0188-a-modules-own-code-is-bundles-in-any-language-and-a-tools-bundle-speaks-mcp-to-the-runtime.md)).
*mesh-tools, mesh-sdk. A day for the skeleton.* A bundle whose entry is not JavaScript is launched
as a child process with the runtime's environment and spoken to over MCP on stdio: `tools/list`
once, `tools/call` per call; a tool named `<seat>.<verb>` is the seat's implementation. A child
that exits is named as a failed bundle and restarted on the next call. The TypeScript import stays
as the shortcut. Beside it, one skeleton SDK per language of the first set — the stdio loop and the
tool-definition type, nothing else — each proven by one bundle in that language answering one tool
in the runtime's test. **Proof.** The runtime's test: a bundle in a second language, launched, its
tool answering on its subject over a real bus; the TypeScript fixture served through the protocol
with the shortcut off answers the same.
## WP2 — The controller composes one runtime per node
*mesh-controller. Two to three days; the largest package.*
@@ -111,12 +125,15 @@ membership republished mid-run re-subscribes without a restart.
declaration gains an `archive` placed under a directory the controller derives, so the host
fetches and unpacks it as it does any artifact. The bundle's digest is what the build recorded.
3. **The runtime's process.** One `process` per node running the runtime from its own bundle
(WP3), `MESH_TOOL_MODULES` composed from the unpacked entrypoints, `MESH_OPERATOR_ACCOUNT` and
(WP3), `MESH_TOOL_MODULES` composed from the unpacked entrypoints — each as
`<module>=<path>`, and the runtime decides from the file whether it is loaded or launched
(WP1b) — `MESH_OPERATOR_ACCOUNT` and
`MESH_OPERATOR_HOME` from the account fact, `restart-on` naming every bundle so a push that
changes one restarts it. A node with no account composes the runtime without the two words.
4. **The gate.** A manifest declaring `tools` and a container built on the runtime's base image is
refused at registration once the runtime module is registered, naming this record. It is the
mechanism that keeps the old pattern from returning by habit.
mechanism that keeps the old pattern from returning by habit. ADR 0188 widens it, after WP4:
a module whose own code is an image artifact is refused, whatever image it is built on.
**Proof.** Composition tests: a node with three assigned modules, one holding a seat, yields one
process, three archives, one node principal whose grants are the union, and the same three
@@ -0,0 +1,88 @@
---
status: open
opened: 2026-10-02
located-in:
- mesh-controller
fixed-by: 02-DECISIONS/0185-a-control-plane-behind-its-seats-row-serves-what-it-can.md
amended-design:
---
# 201 — A push recreated the controller at a digest older than the seat row its successor had written
## What was observed
2026-10-02, two merges a minute apart on the control node: one to the controller, adding a verb to the
controller seat's row; one to the host, adding a container field. Each made a plan. The controller's plan
built and rolled the new controller, which started, widened its seat row with the new verb, and ran. The
host's plan then pushed the control node with the controller digest it had recorded when it was made —
the previous build — and recreated the controller container on it. The older binary read the row, found
a verb it could not run, and refused to start:
```
mesh-controller: the mesh-controller seat's row declares "command", which this control plane
cannot run: "command" is not a verb the mesh-controller seat serves
```
A crash loop followed for ten minutes: nothing answered on the bus, and no build was dispatched, since
the controller is what fills the builder's queue. Recovery was the mesh's own binary run once from the
newer image, outside the service, to push the control node again; the push sent the newer digest and
the controller came up.
## Why it matters beyond this instance
The row is the store's and the binary follows it ([ADR 0154](../../02-DECISIONS/0154-the-meshs-own-verbs-are-the-controller-seats-tools.md));
a start-up check that refuses a row the binary cannot serve is right, and was built after the outage of
2026-09-27 for exactly this reason. What is wrong is a plan sending a controller older than the one that
wrote the row. A plan is made at a moment and sends what it recorded ([ADR 0162](../../02-DECISIONS/0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md));
for every other module an older digest is a brief regression a later push corrects. For the controller
it is the mesh losing its voice, and the correction needs a hand, because the thing that would correct
it is the thing that is down. Two plans that overlap will happen again whenever two people merge within
a minute.
## What a fix would have to do
Either of two, and the first is the smaller:
- A push never sends a controller digest older than the one the running controller is — the controller
knows its own digest and refuses to downgrade itself through a plan, saying so in the plan's words.
- Or the start-up check tolerates a row wider than the binary while a roll-out is in flight, and serves
what it can. Weaker: it makes the row and the binary disagree on purpose, which is what the check
exists to refuse.
Until one is built: do not merge a controller change while another plan is rolling, and after merging
one, wait for `node show` on the control node to report the new controller before merging anything else.
## References
- [ADR 0154](../../02-DECISIONS/0154-the-meshs-own-verbs-are-the-controller-seats-tools.md), [ADR 0162](../../02-DECISIONS/0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md)
- mesh-controller `cmd/mesh-controller/seatverbs.go` (`seatToolHandlers`, the start-up check), `cmd/mesh-controller/push.go`
## Half of it is closed, 2026-10-02
[ADR 0185](../../02-DECISIONS/0185-a-control-plane-behind-its-seats-row-serves-what-it-can.md) takes
the outage out of it: a control plane behind its seat's row now serves every verb it can run, says
which it cannot, and answers the reason when one of those is called. The same race today would cost
the verbs the newer build added, for as long as the older binary is in place, and the ordinary
"this machine is behind" machinery would put the newer one back without a hand.
**The race itself is still open**, and this report stays open for it. What was established while
closing the other half, so the next reader does not redo it:
- Composing and sending are serialised per machine by a session advisory lock in the store, so two
control planes cannot compose one machine's declaration at the same time. The stale content did
not come from two concurrent composes.
- A container's image is resolved into the module's manifest when it is *built*, and a push composes
from the catalogue as it is at that moment, under the hold. So a compose that ran after the build
was taken in could not have named the older image.
- The declaration's sequence orders arrival and nothing else (the numbering of
[issue 107](../107-a-declaration-carries-no-order/00-report.md)); it cannot tell a later send
carrying earlier content from a later send carrying later content. The host refuses a declaration
numbered below the last it applied, and both of these were above it.
- The machine's own journal shows the two applies ten seconds apart and which replaced what; it does
not record which image each declaration named, which is the one fact that would settle it. A host
that recorded the digest it was told, per apply, would have answered this in a minute.
So the trigger is not yet pinned, and guessing at the push path is the most expensive place in the
mesh to guess. The fix the report first suggested — a push never sending a control plane a digest
older than the one that machine reports running — closes the class without needing the trigger, and
is now a correctness nicety rather than the difference between a working mesh and a dead one.