Merge pull request 'ADR 0222: a module is told where a mesh seat's holder is reached; the controller writes no file a seat's holder owns (issue 190)' (#113) from issues/190-controller-writes-no-owned-file into main
This commit was merged in pull request #113.
This commit is contained in:
@@ -51,6 +51,14 @@ the builder) — cost with no new property.
|
||||
restarts the runtime when that file changes — the `/etc/hosts` pattern for the content, the
|
||||
nftables pattern for the reload. No module author is involved; being on the network is what
|
||||
grants the trust, because being on the network is what the trust *is*.
|
||||
|
||||
> **The mechanism changed — 2026-10-05, by [ADR 0222](0222-a-module-is-told-where-a-mesh-seats-holder-is-reached-and-the-controller-writes-no-file-a-seats-holder-owns.md).**
|
||||
> What stands: being on the network grants the trust, the overlay is the transport security, no
|
||||
> module author chooses it, and the trust is written into the runtime's file rather than over it
|
||||
> and reloaded rather than restarted (ADR 0102). What moved: the controller no longer injects the
|
||||
> file or the service. The container runtime's own module writes `insecure-registries`, told where
|
||||
> this machine reaches the store by `${seat:mesh-artifact-store:reach}`, because the controller
|
||||
> writes no file a seat's holder owns ([issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md)).
|
||||
3. **No accounts (issue 042), recorded as the position it always was.** Reading and pushing
|
||||
require presence on the overlay and nothing else. The boundary is enforced, not assumed: the
|
||||
registry's `listens` is `from: mesh`, the firewall derives from it, and the overlay admits only
|
||||
|
||||
@@ -64,6 +64,15 @@ node therefore trusts the mesh's registry as soon as it is on the private networ
|
||||
networking no longer touches the runtime. The hosts file networking writes is still written whole
|
||||
and stays held until networking is taken; a converge preview names it among the files it replaces.
|
||||
|
||||
> **The mechanism changed — 2026-10-05, by [ADR 0222](0222-a-module-is-told-where-a-mesh-seats-holder-is-reached-and-the-controller-writes-no-file-a-seats-holder-owns.md).**
|
||||
> Writing into a shared file, adding to a list and reloading rather than restarting all stand. What
|
||||
> moved is the writer: the networking module no longer writes the runtime's trust or declares its
|
||||
> service. The container runtime's own module writes it into its own file and reloads its own
|
||||
> service ([issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md)).
|
||||
> The controller test named under *How it is checked* below, which held the networking module to
|
||||
> declaring the runtime's file, is replaced by one holding the networking module to declaring neither,
|
||||
> and by one holding the runtime's module to the trust.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The runtime's file on a machine in use keeps its data directory, its logging settings and
|
||||
|
||||
+150
@@ -0,0 +1,150 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-10-05
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md
|
||||
---
|
||||
|
||||
# 222. A module is told where a mesh seat's holder is reached, and the controller writes no file a seat's holder owns
|
||||
|
||||
## Context
|
||||
|
||||
[Issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md)
|
||||
found the container runtime's configuration file written by modules that are not the runtime's. The
|
||||
first half is closed: the resolver module no longer writes into that file, and the catalogue's runtime
|
||||
module (the holder of `node-container-runtime`) writes `live-restore` and reloads its own service. The
|
||||
second half stands. The controller's private network still generates two resources on every machine
|
||||
on the network: one writes `insecure-registries`, naming the mesh's artifact store, into the runtime's
|
||||
file ([ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md) §2,
|
||||
[ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md)), and the other declares the
|
||||
runtime's service, reloaded on that file. So two parties declare one path and one unit on every
|
||||
machine, and nothing refuses it, because the collision check runs over catalogue manifests and the
|
||||
private network's resources only exist once its generator has answered for a machine.
|
||||
|
||||
On 2026-10-05 the operator made the rule general: **the controller never writes a file a seat's holder
|
||||
owns; it tells the owner.** The hosts file had already moved the same way
|
||||
([ADR 0199](0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md)).
|
||||
|
||||
What the runtime's module needs is one fact: the address this machine reaches the mesh's artifact
|
||||
store at. The controller already composes that address for every machine at every push. It puts it
|
||||
into every image and archive reference the mesh built, and never stores it. Today a module has no way
|
||||
to ask for it. A binding gives a consumer the provider's address and a credential. The `${seat:…:<port>}`
|
||||
placeholder gives only a port, and only for the store and the broker the controller itself dials.
|
||||
|
||||
Two facts about the move itself, read from the host's code:
|
||||
|
||||
- The private network writes one member into a list. The host adds a list member and records it per
|
||||
resource, so between the two writers declaring it and one of them going, the member is owned by
|
||||
whichever record added it first.
|
||||
- The host removes every resource no longer declared before it applies anything, so in the apply
|
||||
where the private network's record goes and the runtime module's member arrives, the member leaves
|
||||
and comes back with no reload in between.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Keep the private network writing the trust.** Rejected: it is the defect issue 190 names. Three
|
||||
writers became two, and the second is computed code no check can see.
|
||||
2. **An operator setting on the runtime module naming the registry**, as issue 190 first proposed
|
||||
under [ADR 0164](0164-a-setting-is-declared-with-its-default-its-meaning-and-what-changing-it-costs.md).
|
||||
Rejected: where the store is, is a fact the mesh holds, not a choice an operator makes. A setting
|
||||
would be a copy that goes wrong the day the store moves.
|
||||
3. **The runtime module requires `artifact-store` and reads the address from its binding
|
||||
(`${bound:…}`).** Rejected. A binding makes the module the store's consumer, and that mints a
|
||||
credential for every machine's runtime, which needs none: reading from the store needs presence on
|
||||
the private network and nothing else (ADR 0082 §3). It also makes a cycle: the store runs as a
|
||||
container in the runtime, and the runtime would require the store before it could be assigned.
|
||||
4. **A placeholder that answers where a mesh seat's holder is reached, with no binding.** Chosen. It
|
||||
is the reasoning of `${seat:…:<port>}` taken one step further: nothing is required, nothing is
|
||||
granted, and the answer is an address the mesh already holds in the clear.
|
||||
|
||||
## Decision
|
||||
|
||||
**1. `${seat:<mesh seat>:reach}` is where this machine reaches the holder of a seat the mesh holds once,
|
||||
as host:port.** It is filled where every placeholder is, in a file's content and in an environment
|
||||
value, from the address the controller composes for that machine at that push. It requires nothing
|
||||
and grants nothing, and no credential comes with it.
|
||||
|
||||
- **Only `mesh-artifact-store` is answered.** Another seat is refused by name, never answered with
|
||||
nothing, so a module asking a question the mesh does not answer learns that at composition. A seat
|
||||
is added to what is answered by a record saying why its address is needed.
|
||||
- **The answer may be empty**: no machine on the private network holds the seat yet, which is how
|
||||
genesis begins. In a file written into as JSON, an empty member is dropped from its list, and a list
|
||||
left with no members is dropped, so the software is never told an empty name.
|
||||
|
||||
**2. The runtime's module states the registry's trust.** Its `daemon` resource writes
|
||||
`insecure-registries`, naming `${seat:mesh-artifact-store:reach}`, beside `live-restore`, and it
|
||||
reloads its own service. ADR 0082's decision stands: being on the private network is what grants the
|
||||
trust, the private network is the transport security, and no module author chooses it. Only who writes
|
||||
it moves, from the private network to the runtime's module.
|
||||
|
||||
**3. The controller writes no file a seat's holder owns.** The private network generates neither the
|
||||
runtime's file nor its service.
|
||||
|
||||
**4. What the mesh computes is held to the collision check.** At composition, each computed module's
|
||||
resources, as its generator answers for that machine, are checked beside the other modules' resources.
|
||||
The check is the same one: no two modules on a machine declare one path, unit, name or package. A
|
||||
collision is refused by name. The generator is asked once, and what is checked is exactly what is
|
||||
declared.
|
||||
|
||||
**5. A unit still held is not given back when another record of it goes.** When the host removes a
|
||||
service resource whose unit another declared resource still gives a state, it forgets that record and
|
||||
leaves the unit as it is. What the record going had found is not the machine's to restore while
|
||||
another resource holds the unit, and restoring it could stop the runtime only for the remaining
|
||||
resource to start it again in the same apply.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Rollout has a fixed order, in three steps.**
|
||||
1. The controller learns the placeholder. Nothing uses it yet.
|
||||
2. The catalogue's runtime module uses it. A controller that does not know the placeholder would
|
||||
send it through unfilled, so this waits until step 1 is deployed.
|
||||
3. The controller stops generating the private network's two resources and checks generated
|
||||
resources for collisions. With step 3 before step 2, machines would lose the trust. The host
|
||||
change in Decision 5 is deployed before step 3.
|
||||
|
||||
This replaces issue 190's single push. That was needed when two writers of one scalar key would
|
||||
have been refused. A list member written by two records is tolerated by the host.
|
||||
- **In the apply of step 3**, the private network's records go first: the member leaves the list and
|
||||
the service record is forgotten. Then the runtime module's file is applied, and the member is added
|
||||
back, recorded as the runtime module's. The runtime is reloaded once. The address is unchanged, so
|
||||
the trust never lapses for a pull.
|
||||
- **A machine holding the store, before any network exists**, is answered with the loopback address
|
||||
it reaches the store at. A runtime already trusts loopback, so this adds a redundant member, not a
|
||||
wrong one.
|
||||
- **A second writer cannot return through generated code.** It is refused at composition, naming
|
||||
both modules and what they share.
|
||||
- **A module can now learn where the artifact store is without being its consumer.** That is a
|
||||
capability, and it is bounded: one seat today, and widened only by a record.
|
||||
|
||||
## How it is checked
|
||||
|
||||
| Rule | Checked by |
|
||||
|---|---|
|
||||
| `${seat:mesh-artifact-store:reach}` is filled with this machine's address for the store, in a file and in an environment value | mesh-controller `TestASeatsReachIsWhereThisMachineReachesItsHolder`, and through the whole composition `TestTheRuntimesTrustIsComposedFromTheSeatsReach` |
|
||||
| An empty answer adds nothing to a list, and other members stay | mesh-controller `TestAnUnansweredReachAddsNothingToAList` |
|
||||
| Any other seat is refused by name | mesh-controller `TestAReachTheMeshDoesNotAnswerIsRefused` |
|
||||
| The catalogue's runtime module writes `live-restore` and the trust, and writes no trust when no store is reachable | mesh-controller `TestTheRuntimesModuleTrustsTheMeshsRegistry`, which composes the catalogue's manifest beside it |
|
||||
| The private network declares neither the runtime's file nor its service | mesh-controller `TestTheNetworkWritesNothingOfTheRuntimes` |
|
||||
| A generated resource colliding with a module's is refused by name; disjoint ones compose; the generator is asked once | mesh-controller `TestAGeneratedResourceCollidingWithAModulesIsRefused`, `TestAGeneratedResourceBesideAModulesOwnIsComposed`, `TestAComputedModuleIsAskedAboutTheNodeItIsFor` |
|
||||
| The member moves between records in one apply without leaving the list; one reload; a unit still held is forgotten, never stopped or disabled; the plan says so | mesh-host `TestTheRegistryMovesToTheRuntimesModuleWithoutLeavingTheList`, `TestThePlanForgetsARecordOfAUnitStillHeld` |
|
||||
| After rollout, every machine's runtime trusts the store's address | the runtime module's `docker_daemon_config` tool on each machine, read before step 3 and after it |
|
||||
|
||||
## References
|
||||
|
||||
- [Issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md),
|
||||
steps 2 and 5.
|
||||
- [ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md): the trust and why it
|
||||
is plain HTTP; its mechanism moves here.
|
||||
- [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md): writing into a shared file;
|
||||
its writer of the runtime's trust moves here.
|
||||
- [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md): a mesh seat names the
|
||||
server, which is what lets a placeholder ask about it.
|
||||
- [ADR 0166](0166-the-container-runtime-is-a-node-seat-and-the-host-creates-containers-through-its-holder.md):
|
||||
the runtime module given the registry as a value.
|
||||
- [ADR 0199](0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md):
|
||||
the hosts file left the controller the same way.
|
||||
- mesh-controller `internal/catalogue/seat_into.go`, `internal/catalogue/declaration.go`
|
||||
(`generatedHere`), `internal/overlay/generator.go`; mesh-catalog `modules/docker`; mesh-host
|
||||
`internal/apply/apply.go` (orphan removal).
|
||||
@@ -197,6 +197,7 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0218** — [A plan sends grants before code, rolls a module out one machine first, and a newer merge takes over an older plan](0218-a-plan-sends-grants-before-code-rolls-out-one-machine-first-and-a-newer-merge-takes-over-an-older-plan.md)
|
||||
- **0219** — [The build queue is controlled through the controller and the build seat](0219-the-build-queue-is-controlled-through-the-controller-and-the-build-seat.md)
|
||||
- **0221** — [A push sends no build a policy or a plan holds back, except to the machine it names](0221-a-push-sends-no-build-a-policy-or-a-plan-holds-back-except-to-the-machine-it-names.md)
|
||||
- **0222** — [A module is told where a mesh seat's holder is reached, and the controller writes no file a seat's holder owns](0222-a-module-is-told-where-a-mesh-seats-holder-is-reached-and-the-controller-writes-no-file-a-seats-holder-owns.md)
|
||||
|
||||
### Its tiers, from the bottom up
|
||||
|
||||
|
||||
+27
-1
@@ -1,7 +1,7 @@
|
||||
---
|
||||
status: located
|
||||
opened: 2026-10-01
|
||||
located-in: [mesh-catalog modules/dnsmasq, mesh-controller internal/overlay/generator.go, mesh-controller internal/catalogue/resolve.go (checkResources)]
|
||||
located-in: [mesh-catalog modules/dnsmasq, mesh-catalog modules/docker, mesh-controller internal/overlay/generator.go, mesh-controller internal/catalogue/resolve.go (checkResources), mesh-controller internal/catalogue/seat_into.go, mesh-host internal/apply/apply.go (orphan removal)]
|
||||
fixed-by:
|
||||
amended-design:
|
||||
---
|
||||
@@ -71,6 +71,32 @@ gives that module declared settings with defaults. The fix, once both are accept
|
||||
5. The collision check sees a computed module's resources as well, so a second writer cannot come
|
||||
back through generated code.
|
||||
|
||||
> **Where it stands, 2026-10-05.** Step 1 is done: the resolver module writes nothing into the
|
||||
> runtime's file, and the runtime's module (the holder of `node-container-runtime`) writes
|
||||
> `live-restore` and reloads its own service. No module writes `dns`
|
||||
> ([ADR 0196](../../02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md)).
|
||||
> That answers the first two open questions below.
|
||||
>
|
||||
> Steps 2, 3 and 5 are decided in
|
||||
> [ADR 0222](../../02-DECISIONS/0222-a-module-is-told-where-a-mesh-seats-holder-is-reached-and-the-controller-writes-no-file-a-seats-holder-owns.md),
|
||||
> and the operator's rule is general: the controller never writes a file a seat's holder owns; it
|
||||
> tells the owner. The registry reaches the runtime's module as a value the mesh holds, not as a
|
||||
> setting: `${seat:mesh-artifact-store:reach}`, with no binding. The ADR 0082 and ADR 0102 notes
|
||||
> step 2 asks for are written.
|
||||
>
|
||||
> Step 4's single push is replaced by an order. A list member written by two records is tolerated
|
||||
> by the host, which a scalar key in two modules was not. Four pull requests, merged in this order:
|
||||
>
|
||||
> 1. The controller learns the placeholder (mesh-controller #62).
|
||||
> 2. The runtime's module states `insecure-registries` with it (mesh-catalog #72).
|
||||
> 3. The host leaves a unit alone when another declared service still holds it. It is deployed
|
||||
> before 4. Without it, removing the private network's record of the runtime's service gives back
|
||||
> what that record found (mesh-host #26).
|
||||
> 4. The controller stops generating the private network's two resources, and checks generated
|
||||
> resources for collisions (mesh-controller #63).
|
||||
>
|
||||
> `fixed-by:` is filled when they merge.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **How the resolver's address reaches the runtime.** Either the resolver seat (`node-dns-resolver`)
|
||||
|
||||
Reference in New Issue
Block a user