Issue 143: correct the diagnosis — the step exists and did not fire

The first account said the declaration carries no resource that would disable the
found firewall and that nothing implemented the sentence the flip prints. Wrong:
the mechanism is a step in the host's own apply, retireFirewall, and it is careful
— it reads back that the mesh's table is loaded before retiring anything, records
the forward policies first, and verifies ufw reports inactive afterwards.

What is established: ufw was active and enabled two minutes after a flip that
reported the node converged with nothing failed; and the machine's record now
reads disabled_by_mesh: true, written by a reconcile fifty minutes later that
found ufw already inactive because an operator had disabled it by hand.

So the step did not take effect and the record says it did. The candidates are
named rather than chosen — the step is skipped silently when the apply has any
failure, the mesh's table is loaded by a service in the same apply so ordering is
open, and the host's detail lines do not reach the journal, which is why this has
candidates instead of a cause.

02-DECISIONS/0142-the-mesh-delivers-its-own-components-as-binaries.md

@@ -0,0 +1,158 @@
+---
+topic: the mesh
+status: accepted
+date: 2026-09-29
+deciders: jochen
+reconstructed: false
+extends: 02-DECISIONS/0141-the-host-delivers-its-own-successor.md
+---
+
+# 142. The mesh delivers its own components as binaries, not as container images
+
+## Context
+
+Measured on the control-node, 2026-09-29:
+
+| what | how it runs | publishes |
+|---|---|---|
+| host | a binary on the machine | — |
+| controller, catalogue, builder, vault | containers | nothing |
+| store, registry, broker | containers | ports |
+
+**The mesh's own software is delivered two ways, and the difference is not a property of the
+software.** The host and the controller are both written in the same language, both the mesh's own,
+both doing the mesh's own work. One is an image fetched from a registry. The other is a file somebody
+copied to four machines, owned by no package, built by nothing
+([issue 142](../04-ISSUES/142-the-host-is-the-one-thing-the-mesh-does-not-deliver/00-report.md)).
+
+**The reason is not a judgement about either, it is that images are the only delivery that works.**
+There is no way to put a binary on a machine. The host is hand-copied because of that, and the
+controller is an image because of that. Neither was chosen on its merits.
+
+What it costs, all of it measured rather than argued:
+
+- **Genesis must raise a container runtime before the control plane can exist.** The bundle carries
+  three images and one of them is the controller, *"in the bundle for the same reason they are: there
+  is nothing to fetch it with yet"*
+  ([design 07](../03-DESIGN/01-to-be/07-the-foundation.md)). So the hardest moment in the mesh's life
+  has a prerequisite that the thing being started does not need.
+- **Updating the control plane depends on the control plane.** Its image is fetched from the registry,
+  which is a container the controller manages.
+- **A change to the host cannot be rolled out at all.** Every machine here runs a byte-identical
+  hand-copied binary. A change merged yesterday reached none of them.
+- **Compiling the language the mesh is written in is not a capability of the builder.** The bundle
+  toolchains are typescript — real, with a registered base module — and python, which is named in the
+  list and absent from the catalogue. The controller is built as an image from a Dockerfile, which is
+  the per-repository incantation the bundle toolchain exists to abolish
+  ([design 18](../03-DESIGN/01-to-be/18-building-a-module.md)).
+
+The half that *receives* a binary safely is already built and tested
+([ADR 0141](0141-the-host-delivers-its-own-successor.md)): versions side by side in directories named
+for them, the newest run, the running one standing aside between reconciles, retirement keeping the
+predecessor, and a rollback that chooses a directory. What is missing is everything that puts one
+there.
+
+## Considered Options
+
+1. **Leave it as it is.** Rejected: it is not a design, it is the reach of one mechanism. And it is
+   what makes a host change undeliverable.
+2. **Containerise the host too**, so everything is delivered one way. Rejected: the host is what
+   starts the container runtime and what applies containers. A host in a container is the bootstrap
+   problem made total, and the machine would have no way back from a bad one.
+3. **Deliver the mesh's components as operating-system packages.** Rejected for the reason
+   [ADR 0141](0141-the-host-delivers-its-own-successor.md) rejected it for the host: a package and a
+   trusted repository per operating system, three of each, and the `package` resource asserts presence
+   and deliberately never a version.
+4. **Binaries for the mesh's own components, containers for third-party software.** Adopted.
+
+## Decision
+
+**The mesh's own components are delivered as binaries on the machine.** The host, the controller, the
+catalogue, the builder, the vault — the software this project writes. They are delivered by the
+mechanism [ADR 0141](0141-the-host-delivers-its-own-successor.md) built: an archive, fetched by
+digest, unpacked into a directory named for its version, with the running one standing aside between
+reconciles and a rollback that chooses the predecessor.
+
+**Third-party software stays a container.** The store, the registry, the broker. They are somebody
+else's build, they are already adopted as modules
+([ADR 0078](0078-the-store-and-broker-are-modules.md)), and an image is the right way to carry
+somebody else's software. **The container runtime remains required** — modules use it — so this
+removes a dependency from the control plane, not from the machine.
+
+**The builder compiles the languages the mesh is written in.** A toolchain for Go, with a base module
+providing the compiler, exactly as typescript has. The obligation the toolchain list warns about — an
+SDK carrying the broker client, the envelope and tool serving — attaches to a *module* written in a
+language, not to the language being compilable. None of these components is a module in that sense;
+the host is what applies modules.
+
+**An artifact says what it targets.** A compiled binary is per operating system, pinned at link time
+([ADR 0005](0005-the-node-host.md)), and a toolchain deliberately takes nothing from the module,
+because anything a module could override there it would be writing a Dockerfile to override. So the
+target is a property of the artifact rather than of the recipe, and one artifact declared per target
+is one build each.
+
+**A component's version comes from where it sits, not from its linker.** It is unpacked into a
+directory named for its version, so it can read its own version from its path. The stamp goes, and
+with it the need for a build to know what it will be called.
+
+**Genesis carries a binary reference where it carried an image reference.** The principle does not
+change — the bundle names a thing by digest and the host fetches it, pinned because nothing can
+resolve a version when no mesh exists — and the container runtime stops being a prerequisite for the
+control plane. It stays a prerequisite for the store and the broker, which is where it belongs.
+
+**The order is staged, and each step stands alone.** Compiling Go; an artifact naming its target;
+delivering a binary; the host as the first component delivered; the controller, catalogue, builder and
+vault out of their containers; genesis last. Genesis is last for the reason it is always last: it
+matters for a machine nobody has yet, and every earlier step is provable on a mesh that exists.
+
+## Consequences
+
+- **One delivery for the mesh's own software**, so a change to the host ships the way a change to the
+  controller does, and neither is copied by hand.
+- **The control plane stops depending on a container runtime and on its own registry.** Both remain on
+  the machine for other reasons; neither gates the control plane's own life any more.
+- **`Replaced()`, the known-good record and the launcher's rollback stop being dead code.** They were
+  written for this and have been called by nothing but their tests.
+- **Four more components gain a rollback they do not have.** Today a bad controller image is recovered
+  by an operator; under this it is recovered the way a bad host is.
+- **Two versions of each component occupy disk.** Around nine megabytes each. The predecessor is what a
+  rollback needs.
+- **Genesis gets smaller, not larger.** One fewer image to carry and one fewer runtime to raise before
+  the control plane.
+- **This does not make the components smaller or simpler.** They are the same programs; what changes is
+  how they arrive. A reader expecting the containers to have been hiding complexity will not find any.
+- **What got harder:** the builder gains a language, artifacts gain a target, and the mesh gains a
+  second kind of thing it must deliver correctly — one where getting it wrong takes the control plane
+  down rather than a module. That is why the host is first: it is the component whose recovery is
+  already built and tested.
+
+## How it is checked
+
+- **A component is delivered and runs, with nothing copied by hand.** A bed builds the host from its
+  repository, delivers it to a machine running an older one, and the machine reports the new version.
+  This fails today at the first step, because nothing builds it.
+- **Each target is built once and only the matching one is delivered.** Asserted by declaring an
+  artifact per operating system and checking that a machine is offered the one it can run — a host
+  built for another is what ADR 0005's link-time pin exists to refuse.
+- **A component reads its version from its path**, asserted by unpacking the same bytes into two
+  differently named directories and seeing each report its own.
+- **A bad component is rolled back without an operator**, for the host first: a version that will not
+  start is replaced by its predecessor once, and the second failure halts naming the machine.
+- **The control plane comes up with no registry reachable**, which is the dependency this removes —
+  asserted by raising it with the registry stopped.
+- **Genesis raises a control plane with no container runtime running**, and raises the store and the
+  broker afterwards. Last, and on a machine with nothing on it.
+- **A published port count that does not change.** The mesh's own components publish nothing today, so
+  moving them out of containers must not open anything — asserted on the machine's reachable set before
+  and after, which the converge preview already reads.
+
+## References
+
+- [ADR 0141](0141-the-host-delivers-its-own-successor.md) — the receiving half, already built
+- [ADR 0005](0005-the-node-host.md) — the host, its supervision, and one binary per operating system
+- [ADR 0078](0078-the-store-and-broker-are-modules.md) — why third-party software stays a container
+- [ADR 0006](0006-the-substrate-and-the-control-plane.md) — what genesis must raise, and in what order
+- [issue 142](../04-ISSUES/142-the-host-is-the-one-thing-the-mesh-does-not-deliver/00-report.md) — the
+  measurement that started this
+- [design 07](../03-DESIGN/01-to-be/07-the-foundation.md) — the bundle's three images, one of them the
+  controller

02-DECISIONS/README.md

@@ -142,6 +142,7 @@ python3 00-META/checks/index.py            fail if stale
 - **0131** — [Everything on the mesh speaks to the broker seat, and AMQP is not a provision](0131-everything-on-the-mesh-speaks-to-the-broker-seat.md)
 - **0132** — [A seat carries the tools its holder must serve](0132-a-seat-carries-the-tools-its-holder-must-serve.md)
 - **0134** — [The mesh says what it applied](0134-the-mesh-says-what-it-applied.md)
+- **0142** — [The mesh delivers its own components as binaries, not as container images](0142-the-mesh-delivers-its-own-components-as-binaries.md)
 
 ### Its tiers, from the bottom up
 

03-DESIGN/01-to-be/07-the-foundation.md

@@ -11,8 +11,9 @@ code:
   - mesh-catalog modules/postgres
   - mesh-catalog modules/lavinmq
   - mesh-lab test/integration/mesh.test.ts (a bare machine becomes a mesh)
-updated: 2026-09-22
+updated: 2026-09-29
 decisions:
+  - 02-DECISIONS/0142-the-mesh-delivers-its-own-components-as-binaries.md
   - 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
   - 02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md
   - 02-DECISIONS/0004-a-node-and-how-it-joins.md
@@ -314,4 +315,24 @@ of a database and pushed to over the broker. What arrived and what did not is th
 
 **One fault, and it was in the joining.** The token did not say what the mesh calls the machine,
 so enrolment needed a flag its own help said it did not — and failed at the broker with an empty
-username. Recorded in ADR 0004 as the fifth thing a token carries.
+username. Recorded in ADR 0004 as the fifth thing a token carries.
+
+## The mesh's own components arrive as binaries
+
+*2026-09-29 —
+[ADR 0142](../../02-DECISIONS/0142-the-mesh-delivers-its-own-components-as-binaries.md).*
+
+The bundle carries three images and one of them is the controller, *because there is nothing to fetch
+it with yet*. That reasoning holds and its conclusion changes: the controller is carried as a **binary**
+reference rather than an image reference, pinned by digest exactly as before. Nothing about the bundle's
+shape moves — it names a thing and the host fetches it — and the container runtime stops being something
+genesis must raise before the control plane can exist. It still raises one, for the store and the broker,
+which is where somebody else's software belongs.
+
+The mesh's own components — the host, the controller, the catalogue, the builder, the vault — are
+delivered as binaries into directories named for their versions, by the mechanism
+[ADR 0141](../../02-DECISIONS/0141-the-host-delivers-its-own-successor.md) describes. Third-party
+software stays a container. The split is not about isolation; it is about who built the thing.
+
+Measured before deciding it: the mesh's own components publish no ports at all, so this opens nothing.
+Only the store, the registry and the broker publish, and they are staying as they are.

03-DESIGN/01-to-be/18-building-a-module.md

@@ -5,8 +5,9 @@ code:
   - mesh-controller cmd/mesh-builder
   - mesh-controller internal/builder
   - mesh-catalog modules/builder
-updated: 2026-09-25
+updated: 2026-09-29
 decisions:
+  - 02-DECISIONS/0142-the-mesh-delivers-its-own-components-as-binaries.md
   - 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md
   - 02-DECISIONS/0097-a-vendor-image-is-a-declared-build-input.md
   - 02-DECISIONS/0096-an-upstream-image-is-copied-between-registries.md
@@ -263,3 +264,27 @@ ships one and wrong for code the mesh built, which has no unit until the mesh wr
 
 **Tools, hooks and consumers are not further modes**, which is the test of whether three is the
 right number: they are loaded by a tool host, and a tool host is a process that stays up.
+
+## The builder compiles the languages the mesh is written in
+
+*2026-09-29 —
+[ADR 0142](../../02-DECISIONS/0142-the-mesh-delivers-its-own-components-as-binaries.md).*
+
+The toolchain list was typescript and python, and only typescript had a base module in the catalogue.
+Meanwhile the control plane — written in the language this project is mostly written in — was built as
+an image from a hand-written Dockerfile, which is the per-repository incantation this whole mechanism
+exists to abolish.
+
+So the list gains Go, with a base module providing the compiler exactly as typescript has one. The
+obligation the list's own comment warns about — an SDK carrying the broker client, the event envelope
+and tool serving — attaches to a **module** written in a language, not to the language being
+compilable. The mesh's own components are not modules in that sense; the host is what applies modules.
+
+**And an artifact says what it targets.** A compiled binary is per operating system, pinned at link
+time, and a toolchain deliberately accepts nothing from the module — anything a module could override
+there it would be writing a Dockerfile to override. The target is therefore a property of the artifact,
+not of the recipe: one artifact declared per target, one build each.
+
+A component's version stops being stamped in at link time. It is unpacked into a directory named for
+its version, so it reads its version from its own path, and a build no longer has to know what it will
+be called.

04-ISSUES/143-converging-does-not-retire-the-firewall-it-found/00-report.md

@@ -0,0 +1,102 @@
+---
+status: located
+opened: 2026-09-29
+located-in:
+  - mesh-host internal/apply/opening.go (retireFirewall)
+  - mesh-host internal/apply/apply.go (the condition it is called under)
+fixed-by:
+amended-design:
+---
+
+# 143 — Converging a machine does not retire the firewall it found, and says it does
+
+## What was observed
+
+The control-node was converged on 2026-09-29, the first machine with a found firewall to be flipped —
+the two converged before it had none.
+
+The preview said, and the flip repeated:
+
+```
+the found firewall (ufw) is disabled, never flushed: its configuration stays on disk
+...
+sent: the host loads the mesh's filter and disables the firewall it found
+```
+
+The mesh then reported the node `converged`, 372 resources applied, nothing failed. Afterwards, on the
+machine:
+
+```
+systemctl is-enabled ufw   -> enabled
+systemctl is-active  ufw   -> active
+```
+
+*Corrected 2026-09-29, an hour later, from reading the host rather than the declaration.* **The first
+account of this was wrong.** It said the declaration carries no resource that would disable the found
+firewall, and that the sentence was printed by the command with nothing implementing it. The
+declaration indeed carries no such resource — but the mechanism was never meant to be one. It is a
+step in the host's own apply, `retireFirewall`, and it exists, is careful, and is strict: it refuses to
+retire anything until it has read back from the machine that the mesh's own table is loaded, it records
+the forward policies first so a half-done retirement can be retried, and it verifies ufw reports
+inactive afterwards.
+
+What is established is narrower and stranger than "nothing implements it":
+
+- ufw was **active and enabled two minutes after the flip**, and the flip had reported the node
+  converged with 372 resources applied and nothing failed.
+- The machine's own record now reads `disabled_by_mesh: true` — but it was written by a reconcile
+  *after* an operator disabled ufw by hand, roughly fifty minutes later. A reconcile found ufw already
+  inactive, asked it to be inactive, read that back, and recorded that the mesh had done it.
+- So the step did not take effect at the flip, and the machine's record now says it did.
+
+The candidates are named rather than chosen, because the evidence does not separate them: the step is
+called only when the apply had no failures, and a skipped step is silent; the mesh's table is loaded by
+a service in the same apply, so whether it was loaded *at the moment the step asked* is an ordering
+question; and the host's own detail lines do not reach the journal, so what it decided is not
+recoverable after the fact.
+
+## Why it matters beyond this instance
+
+**It is a stated behaviour that does not happen, reported as success** — the fault this repository
+exists to catch, and
+[ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md) states it as
+part of what the flip *is*: "loads the mesh's derived filter in place of its refusal-only table, and
+retires the found firewall by disabling it, never by flushing".
+
+**It could only be found on the first machine that had one.** The two machines converged before this
+had no firewall to retire, so the step had never run, and nothing reported that it had not. That is
+the same shape as [issue 136](../136-a-module-may-name-a-program-the-machine-does-not-have/00-report.md):
+a step that is silent when it does nothing.
+
+**The machine is left doubly filtered, which is not what either firewall describes.** Every base chain
+at a hook runs and a drop in any is final, so the machine now enforces the *intersection* of the mesh's
+derived filter and a rule set left by the system being replaced. Nothing is broken by that today —
+measured from outside, mail, the proxy and git-over-ssh answer and the databases and admin interfaces
+are refused — but the machine's behaviour is described by neither of the two things claiming to
+describe it, and the stale set includes a rule for a broker that no longer exists.
+
+**And returning the node to adopted would be wrong in the other direction.** ADR 0100 says that
+restores the found firewall by enabling it again; enabling something that was never disabled is
+harmless, but the mesh's belief about which firewall is in force has been wrong in both modes.
+
+## Open questions
+
+- Which side owns retiring it — a resource in the declaration, so it is applied and reported like
+  everything else, or the flip as an act? A resource seems right: the flip is otherwise entirely
+  expressed as one, and an act that only the command performs cannot be re-checked on a later
+  reconcile.
+- What should a reconcile do if the found firewall is enabled again by hand, or by a package update?
+  Convergence is a state, so presumably re-disable it and say so.
+- Should the preview say what it *will* do rather than what it does, until a step exists that does it?
+  The wording was read as evidence twice in one session.
+- Is there a check that a sentence the mesh prints corresponds to something that happened? This is the
+  second time in one session that a printed claim and the machine disagreed.
+- **Why did the step not take effect?** It is called only when the apply had no failures, and being
+  skipped is silent. The mesh's table is loaded by a service in the same apply, so whether it was
+  loaded when the step asked is an ordering question — and ADR 0100 makes loading it first a
+  precondition rather than an expectation.
+- **A step that records the mesh as having done what an operator did is worse than the omission.** The
+  record now says the mesh disabled ufw. Nothing distinguishes "we did this" from "we found it already
+  so". Should it?
+- Why do the host's own detail lines not reach the journal? Everything it decided during the flip is
+  unrecoverable, which is why this account has candidates instead of a cause.

04-ISSUES/144-the-predecessors-rules-outlive-the-firewall-it-was-found-as/00-report.md

@@ -0,0 +1,70 @@
+---
+status: located
+opened: 2026-09-29
+located-in:
+  - mesh-host internal/apply/opening.go
+  - mesh-controller cmd/mesh-controller (the converge preview)
+fixed-by:
+amended-design:
+---
+
+# 144 — A predecessor's rules outlive the firewall the mesh found, and the mesh cannot see them
+
+## What was observed
+
+The mesh reports one thing about a machine's existing filtering: `firewall found: ufw`. On the
+control-node, ufw was never what filtered the traffic that mattered.
+
+Measured on 2026-09-29, before the machine was converged:
+
+- ufw filters connections *to the machine*. It does not filter connections to a container's published
+  port, which arrive on the forwarded path where the container runtime accepts them before ufw's
+  forward chains are reached. Around thirty ports were published that way.
+- Every one of the mesh's own forwarded openings, converged through ufw, had matched **zero packets** —
+  fifty rules in that chain, none ever matched, while the chain itself had passed 1.6 million
+  established packets. The restrictions read as applied and were inert.
+- What actually kept those ports off the internet was a chain the predecessor installed in the
+  container runtime's own pre-accept hook, allowing the deliberately public ports and the private
+  ranges and dropping the rest on the outward link. Confirmed from outside: the proxy answered, the
+  container manager did not.
+- That chain exists only in the running kernel. The persisted rule file is the distribution's empty
+  default, and nothing on disk recreates the chain.
+
+After the flip, the mesh's own filter is loaded and does cover the forwarded path, so the machine no
+longer depends on that chain. But **the chain is still there**, and it is now the only thing refusing
+two ports the mesh believes are open: the bus and the registry, which
+[ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md) requires be
+reachable from anywhere so a machine can enrol and pull before it has a private-network address. The
+mesh's rendered filter accepts both from anywhere. From outside, both are refused.
+
+## Why it matters beyond this instance
+
+**"The firewall found" is a kind, and filtering is not all in one place.** The host identifies one
+front-end and reports it. A machine can carry rules from several sources — the front-end's own, the
+container runtime's, an intrusion-prevention chain, and whatever a predecessor installed directly —
+and the mesh's account of what filters the machine names exactly one of them.
+
+**So adoption's central promise was half-true in both directions.** What the mesh converged through
+the found firewall on the forwarded path did nothing at all, and what did the work was invisible to it.
+A machine was reported as filtered by a mechanism that was not filtering.
+
+**And convergence cannot retire what it cannot see.** Even once
+[issue 143](../143-converging-does-not-retire-the-firewall-it-found/00-report.md) is fixed and the found
+firewall is disabled, this chain remains, silently narrowing the machine below what the mesh's own
+filter says. A rule the mesh did not write, cannot list, and will not remove — which today breaks the
+enrolment path the design guarantees.
+
+**The safe direction is not the same as the correct one.** Being more closed than intended broke nothing
+visible, which is exactly why it went unnoticed for as long as the mesh has been on this machine.
+
+## Open questions
+
+- Should the host report every place the machine filters from, rather than one kind — the front-end,
+  the runtime's hooks, and any chain it does not recognise, named so a person can look?
+- What should the mesh do about rules it did not write and does not understand? Reporting them seems
+  right; removing them cannot be, and leaving them silent is what produced this.
+- Does an opening converged through a found firewall need a check that it can actually take effect?
+  Fifty rules matching nothing would have been visible from the counters at any point.
+- Is the bus and the registry being reachable from anywhere still what the mesh wants on a machine that
+  faces the internet? The design says yes, for enrolment. It deserves asking on its own rather than
+  being answered by a leftover.