diff --git a/02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md b/02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md index 81366a6..4cac697 100644 --- a/02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md +++ b/02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md @@ -51,6 +51,14 @@ the builder) — cost with no new property. restarts the runtime when that file changes — the `/etc/hosts` pattern for the content, the nftables pattern for the reload. No module author is involved; being on the network is what grants the trust, because being on the network is what the trust *is*. + + > **The mechanism changed — 2026-10-05, by [ADR 0222](0222-a-module-is-told-where-a-mesh-seats-holder-is-reached-and-the-controller-writes-no-file-a-seats-holder-owns.md).** + > What stands: being on the network grants the trust, the overlay is the transport security, no + > module author chooses it, and the trust is written into the runtime's file rather than over it + > and reloaded rather than restarted (ADR 0102). What moved: the controller no longer injects the + > file or the service. The container runtime's own module writes `insecure-registries`, told where + > this machine reaches the store by `${seat:mesh-artifact-store:reach}`, because the controller + > writes no file a seat's holder owns ([issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md)). 3. **No accounts (issue 042), recorded as the position it always was.** Reading and pushing require presence on the overlay and nothing else. The boundary is enforced, not assumed: the registry's `listens` is `from: mesh`, the firewall derives from it, and the overlay admits only diff --git a/02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md b/02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md index 8731e5a..9be072a 100644 --- a/02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md +++ b/02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md @@ -64,6 +64,15 @@ node therefore trusts the mesh's registry as soon as it is on the private networ networking no longer touches the runtime. The hosts file networking writes is still written whole and stays held until networking is taken; a converge preview names it among the files it replaces. +> **The mechanism changed — 2026-10-05, by [ADR 0222](0222-a-module-is-told-where-a-mesh-seats-holder-is-reached-and-the-controller-writes-no-file-a-seats-holder-owns.md).** +> Writing into a shared file, adding to a list and reloading rather than restarting all stand. What +> moved is the writer: the networking module no longer writes the runtime's trust or declares its +> service. The container runtime's own module writes it into its own file and reloads its own +> service ([issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md)). +> The controller test named under *How it is checked* below, which held the networking module to +> declaring the runtime's file, is replaced by one holding the networking module to declaring neither, +> and by one holding the runtime's module to the trust. + ## Consequences - The runtime's file on a machine in use keeps its data directory, its logging settings and diff --git a/02-DECISIONS/0222-a-module-is-told-where-a-mesh-seats-holder-is-reached-and-the-controller-writes-no-file-a-seats-holder-owns.md b/02-DECISIONS/0222-a-module-is-told-where-a-mesh-seats-holder-is-reached-and-the-controller-writes-no-file-a-seats-holder-owns.md new file mode 100644 index 0000000..af65c3d --- /dev/null +++ b/02-DECISIONS/0222-a-module-is-told-where-a-mesh-seats-holder-is-reached-and-the-controller-writes-no-file-a-seats-holder-owns.md @@ -0,0 +1,150 @@ +--- +topic: the mesh +status: accepted +date: 2026-10-05 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md +--- + +# 222. A module is told where a mesh seat's holder is reached, and the controller writes no file a seat's holder owns + +## Context + +[Issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md) +found the container runtime's configuration file written by modules that are not the runtime's. The +first half is closed: the resolver module no longer writes into that file, and the catalogue's runtime +module (the holder of `node-container-runtime`) writes `live-restore` and reloads its own service. The +second half stands. The controller's private network still generates two resources on every machine +on the network: one writes `insecure-registries`, naming the mesh's artifact store, into the runtime's +file ([ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md) §2, +[ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md)), and the other declares the +runtime's service, reloaded on that file. So two parties declare one path and one unit on every +machine, and nothing refuses it, because the collision check runs over catalogue manifests and the +private network's resources only exist once its generator has answered for a machine. + +On 2026-10-05 the operator made the rule general: **the controller never writes a file a seat's holder +owns; it tells the owner.** The hosts file had already moved the same way +([ADR 0199](0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md)). + +What the runtime's module needs is one fact: the address this machine reaches the mesh's artifact +store at. The controller already composes that address for every machine at every push. It puts it +into every image and archive reference the mesh built, and never stores it. Today a module has no way +to ask for it. A binding gives a consumer the provider's address and a credential. The `${seat:…:}` +placeholder gives only a port, and only for the store and the broker the controller itself dials. + +Two facts about the move itself, read from the host's code: + +- The private network writes one member into a list. The host adds a list member and records it per + resource, so between the two writers declaring it and one of them going, the member is owned by + whichever record added it first. +- The host removes every resource no longer declared before it applies anything, so in the apply + where the private network's record goes and the runtime module's member arrives, the member leaves + and comes back with no reload in between. + +## Considered Options + +1. **Keep the private network writing the trust.** Rejected: it is the defect issue 190 names. Three + writers became two, and the second is computed code no check can see. +2. **An operator setting on the runtime module naming the registry**, as issue 190 first proposed + under [ADR 0164](0164-a-setting-is-declared-with-its-default-its-meaning-and-what-changing-it-costs.md). + Rejected: where the store is, is a fact the mesh holds, not a choice an operator makes. A setting + would be a copy that goes wrong the day the store moves. +3. **The runtime module requires `artifact-store` and reads the address from its binding + (`${bound:…}`).** Rejected. A binding makes the module the store's consumer, and that mints a + credential for every machine's runtime, which needs none: reading from the store needs presence on + the private network and nothing else (ADR 0082 §3). It also makes a cycle: the store runs as a + container in the runtime, and the runtime would require the store before it could be assigned. +4. **A placeholder that answers where a mesh seat's holder is reached, with no binding.** Chosen. It + is the reasoning of `${seat:…:}` taken one step further: nothing is required, nothing is + granted, and the answer is an address the mesh already holds in the clear. + +## Decision + +**1. `${seat::reach}` is where this machine reaches the holder of a seat the mesh holds once, +as host:port.** It is filled where every placeholder is, in a file's content and in an environment +value, from the address the controller composes for that machine at that push. It requires nothing +and grants nothing, and no credential comes with it. + +- **Only `mesh-artifact-store` is answered.** Another seat is refused by name, never answered with + nothing, so a module asking a question the mesh does not answer learns that at composition. A seat + is added to what is answered by a record saying why its address is needed. +- **The answer may be empty**: no machine on the private network holds the seat yet, which is how + genesis begins. In a file written into as JSON, an empty member is dropped from its list, and a list + left with no members is dropped, so the software is never told an empty name. + +**2. The runtime's module states the registry's trust.** Its `daemon` resource writes +`insecure-registries`, naming `${seat:mesh-artifact-store:reach}`, beside `live-restore`, and it +reloads its own service. ADR 0082's decision stands: being on the private network is what grants the +trust, the private network is the transport security, and no module author chooses it. Only who writes +it moves, from the private network to the runtime's module. + +**3. The controller writes no file a seat's holder owns.** The private network generates neither the +runtime's file nor its service. + +**4. What the mesh computes is held to the collision check.** At composition, each computed module's +resources, as its generator answers for that machine, are checked beside the other modules' resources. +The check is the same one: no two modules on a machine declare one path, unit, name or package. A +collision is refused by name. The generator is asked once, and what is checked is exactly what is +declared. + +**5. A unit still held is not given back when another record of it goes.** When the host removes a +service resource whose unit another declared resource still gives a state, it forgets that record and +leaves the unit as it is. What the record going had found is not the machine's to restore while +another resource holds the unit, and restoring it could stop the runtime only for the remaining +resource to start it again in the same apply. + +## Consequences + +- **Rollout has a fixed order, in three steps.** + 1. The controller learns the placeholder. Nothing uses it yet. + 2. The catalogue's runtime module uses it. A controller that does not know the placeholder would + send it through unfilled, so this waits until step 1 is deployed. + 3. The controller stops generating the private network's two resources and checks generated + resources for collisions. With step 3 before step 2, machines would lose the trust. The host + change in Decision 5 is deployed before step 3. + + This replaces issue 190's single push. That was needed when two writers of one scalar key would + have been refused. A list member written by two records is tolerated by the host. +- **In the apply of step 3**, the private network's records go first: the member leaves the list and + the service record is forgotten. Then the runtime module's file is applied, and the member is added + back, recorded as the runtime module's. The runtime is reloaded once. The address is unchanged, so + the trust never lapses for a pull. +- **A machine holding the store, before any network exists**, is answered with the loopback address + it reaches the store at. A runtime already trusts loopback, so this adds a redundant member, not a + wrong one. +- **A second writer cannot return through generated code.** It is refused at composition, naming + both modules and what they share. +- **A module can now learn where the artifact store is without being its consumer.** That is a + capability, and it is bounded: one seat today, and widened only by a record. + +## How it is checked + +| Rule | Checked by | +|---|---| +| `${seat:mesh-artifact-store:reach}` is filled with this machine's address for the store, in a file and in an environment value | mesh-controller `TestASeatsReachIsWhereThisMachineReachesItsHolder`, and through the whole composition `TestTheRuntimesTrustIsComposedFromTheSeatsReach` | +| An empty answer adds nothing to a list, and other members stay | mesh-controller `TestAnUnansweredReachAddsNothingToAList` | +| Any other seat is refused by name | mesh-controller `TestAReachTheMeshDoesNotAnswerIsRefused` | +| The catalogue's runtime module writes `live-restore` and the trust, and writes no trust when no store is reachable | mesh-controller `TestTheRuntimesModuleTrustsTheMeshsRegistry`, which composes the catalogue's manifest beside it | +| The private network declares neither the runtime's file nor its service | mesh-controller `TestTheNetworkWritesNothingOfTheRuntimes` | +| A generated resource colliding with a module's is refused by name; disjoint ones compose; the generator is asked once | mesh-controller `TestAGeneratedResourceCollidingWithAModulesIsRefused`, `TestAGeneratedResourceBesideAModulesOwnIsComposed`, `TestAComputedModuleIsAskedAboutTheNodeItIsFor` | +| The member moves between records in one apply without leaving the list; one reload; a unit still held is forgotten, never stopped or disabled; the plan says so | mesh-host `TestTheRegistryMovesToTheRuntimesModuleWithoutLeavingTheList`, `TestThePlanForgetsARecordOfAUnitStillHeld` | +| After rollout, every machine's runtime trusts the store's address | the runtime module's `docker_daemon_config` tool on each machine, read before step 3 and after it | + +## References + +- [Issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md), + steps 2 and 5. +- [ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md): the trust and why it + is plain HTTP; its mechanism moves here. +- [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md): writing into a shared file; + its writer of the runtime's trust moves here. +- [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md): a mesh seat names the + server, which is what lets a placeholder ask about it. +- [ADR 0166](0166-the-container-runtime-is-a-node-seat-and-the-host-creates-containers-through-its-holder.md): + the runtime module given the registry as a value. +- [ADR 0199](0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md): + the hosts file left the controller the same way. +- mesh-controller `internal/catalogue/seat_into.go`, `internal/catalogue/declaration.go` + (`generatedHere`), `internal/overlay/generator.go`; mesh-catalog `modules/docker`; mesh-host + `internal/apply/apply.go` (orphan removal). diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 4072782..f7c7f0f 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -196,6 +196,7 @@ python3 00-META/checks/index.py fail if stale - **0212** — [A seat says what it receives, and the machine's hotkeys are a seat](0212-a-seat-says-what-it-receives-and-the-machines-hotkeys-are-a-seat.md) - **0218** — [A plan sends grants before code, rolls a module out one machine first, and a newer merge takes over an older plan](0218-a-plan-sends-grants-before-code-rolls-out-one-machine-first-and-a-newer-merge-takes-over-an-older-plan.md) - **0219** — [The build queue is controlled through the controller and the build seat](0219-the-build-queue-is-controlled-through-the-controller-and-the-build-seat.md) +- **0222** — [A module is told where a mesh seat's holder is reached, and the controller writes no file a seat's holder owns](0222-a-module-is-told-where-a-mesh-seats-holder-is-reached-and-the-controller-writes-no-file-a-seats-holder-owns.md) ### Its tiers, from the bottom up diff --git a/04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md b/04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md index c8769e4..5f884e0 100644 --- a/04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md +++ b/04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md @@ -1,7 +1,7 @@ --- status: located opened: 2026-10-01 -located-in: [mesh-catalog modules/dnsmasq, mesh-controller internal/overlay/generator.go, mesh-controller internal/catalogue/resolve.go (checkResources)] +located-in: [mesh-catalog modules/dnsmasq, mesh-catalog modules/docker, mesh-controller internal/overlay/generator.go, mesh-controller internal/catalogue/resolve.go (checkResources), mesh-controller internal/catalogue/seat_into.go, mesh-host internal/apply/apply.go (orphan removal)] fixed-by: amended-design: --- @@ -71,6 +71,32 @@ gives that module declared settings with defaults. The fix, once both are accept 5. The collision check sees a computed module's resources as well, so a second writer cannot come back through generated code. +> **Where it stands, 2026-10-05.** Step 1 is done: the resolver module writes nothing into the +> runtime's file, and the runtime's module (the holder of `node-container-runtime`) writes +> `live-restore` and reloads its own service. No module writes `dns` +> ([ADR 0196](../../02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md)). +> That answers the first two open questions below. +> +> Steps 2, 3 and 5 are decided in +> [ADR 0222](../../02-DECISIONS/0222-a-module-is-told-where-a-mesh-seats-holder-is-reached-and-the-controller-writes-no-file-a-seats-holder-owns.md), +> and the operator's rule is general: the controller never writes a file a seat's holder owns; it +> tells the owner. The registry reaches the runtime's module as a value the mesh holds, not as a +> setting: `${seat:mesh-artifact-store:reach}`, with no binding. The ADR 0082 and ADR 0102 notes +> step 2 asks for are written. +> +> Step 4's single push is replaced by an order. A list member written by two records is tolerated +> by the host, which a scalar key in two modules was not. Four pull requests, merged in this order: +> +> 1. The controller learns the placeholder (mesh-controller #62). +> 2. The runtime's module states `insecure-registries` with it (mesh-catalog #72). +> 3. The host leaves a unit alone when another declared service still holds it. It is deployed +> before 4. Without it, removing the private network's record of the runtime's service gives back +> what that record found (mesh-host #26). +> 4. The controller stops generating the private network's two resources, and checks generated +> resources for collisions (mesh-controller #63). +> +> `fixed-by:` is filled when they merge. + ## Open questions - **How the resolver's address reaches the runtime.** Either the resolver seat (`node-dns-resolver`)