|
|
|
@@ -0,0 +1,150 @@
|
|
|
|
|
---
|
|
|
|
|
topic: the mesh
|
|
|
|
|
status: accepted
|
|
|
|
|
date: 2026-10-05
|
|
|
|
|
deciders: jochen
|
|
|
|
|
reconstructed: false
|
|
|
|
|
extends: 02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
# 222. A module is told where a mesh seat's holder is reached, and the controller writes no file a seat's holder owns
|
|
|
|
|
|
|
|
|
|
## Context
|
|
|
|
|
|
|
|
|
|
[Issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md)
|
|
|
|
|
found the container runtime's configuration file written by modules that are not the runtime's. The
|
|
|
|
|
first half is closed: the resolver module no longer writes into that file, and the catalogue's runtime
|
|
|
|
|
module (the holder of `node-container-runtime`) writes `live-restore` and reloads its own service. The
|
|
|
|
|
second half stands. The controller's private network still generates two resources on every machine
|
|
|
|
|
on the network: one writes `insecure-registries`, naming the mesh's artifact store, into the runtime's
|
|
|
|
|
file ([ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md) §2,
|
|
|
|
|
[ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md)), and the other declares the
|
|
|
|
|
runtime's service, reloaded on that file. So two parties declare one path and one unit on every
|
|
|
|
|
machine, and nothing refuses it, because the collision check runs over catalogue manifests and the
|
|
|
|
|
private network's resources only exist once its generator has answered for a machine.
|
|
|
|
|
|
|
|
|
|
On 2026-10-05 the operator made the rule general: **the controller never writes a file a seat's holder
|
|
|
|
|
owns; it tells the owner.** The hosts file had already moved the same way
|
|
|
|
|
([ADR 0199](0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md)).
|
|
|
|
|
|
|
|
|
|
What the runtime's module needs is one fact: the address this machine reaches the mesh's artifact
|
|
|
|
|
store at. The controller already composes that address for every machine at every push. It puts it
|
|
|
|
|
into every image and archive reference the mesh built, and never stores it. Today a module has no way
|
|
|
|
|
to ask for it. A binding gives a consumer the provider's address and a credential. The `${seat:…:<port>}`
|
|
|
|
|
placeholder gives only a port, and only for the store and the broker the controller itself dials.
|
|
|
|
|
|
|
|
|
|
Two facts about the move itself, read from the host's code:
|
|
|
|
|
|
|
|
|
|
- The private network writes one member into a list. The host adds a list member and records it per
|
|
|
|
|
resource, so between the two writers declaring it and one of them going, the member is owned by
|
|
|
|
|
whichever record added it first.
|
|
|
|
|
- The host removes every resource no longer declared before it applies anything, so in the apply
|
|
|
|
|
where the private network's record goes and the runtime module's member arrives, the member leaves
|
|
|
|
|
and comes back with no reload in between.
|
|
|
|
|
|
|
|
|
|
## Considered Options
|
|
|
|
|
|
|
|
|
|
1. **Keep the private network writing the trust.** Rejected: it is the defect issue 190 names. Three
|
|
|
|
|
writers became two, and the second is computed code no check can see.
|
|
|
|
|
2. **An operator setting on the runtime module naming the registry**, as issue 190 first proposed
|
|
|
|
|
under [ADR 0164](0164-a-setting-is-declared-with-its-default-its-meaning-and-what-changing-it-costs.md).
|
|
|
|
|
Rejected: where the store is, is a fact the mesh holds, not a choice an operator makes. A setting
|
|
|
|
|
would be a copy that goes wrong the day the store moves.
|
|
|
|
|
3. **The runtime module requires `artifact-store` and reads the address from its binding
|
|
|
|
|
(`${bound:…}`).** Rejected. A binding makes the module the store's consumer, and that mints a
|
|
|
|
|
credential for every machine's runtime, which needs none: reading from the store needs presence on
|
|
|
|
|
the private network and nothing else (ADR 0082 §3). It also makes a cycle: the store runs as a
|
|
|
|
|
container in the runtime, and the runtime would require the store before it could be assigned.
|
|
|
|
|
4. **A placeholder that answers where a mesh seat's holder is reached, with no binding.** Chosen. It
|
|
|
|
|
is the reasoning of `${seat:…:<port>}` taken one step further: nothing is required, nothing is
|
|
|
|
|
granted, and the answer is an address the mesh already holds in the clear.
|
|
|
|
|
|
|
|
|
|
## Decision
|
|
|
|
|
|
|
|
|
|
**1. `${seat:<mesh seat>:reach}` is where this machine reaches the holder of a seat the mesh holds once,
|
|
|
|
|
as host:port.** It is filled where every placeholder is, in a file's content and in an environment
|
|
|
|
|
value, from the address the controller composes for that machine at that push. It requires nothing
|
|
|
|
|
and grants nothing, and no credential comes with it.
|
|
|
|
|
|
|
|
|
|
- **Only `mesh-artifact-store` is answered.** Another seat is refused by name, never answered with
|
|
|
|
|
nothing, so a module asking a question the mesh does not answer learns that at composition. A seat
|
|
|
|
|
is added to what is answered by a record saying why its address is needed.
|
|
|
|
|
- **The answer may be empty**: no machine on the private network holds the seat yet, which is how
|
|
|
|
|
genesis begins. In a file written into as JSON, an empty member is dropped from its list, and a list
|
|
|
|
|
left with no members is dropped, so the software is never told an empty name.
|
|
|
|
|
|
|
|
|
|
**2. The runtime's module states the registry's trust.** Its `daemon` resource writes
|
|
|
|
|
`insecure-registries`, naming `${seat:mesh-artifact-store:reach}`, beside `live-restore`, and it
|
|
|
|
|
reloads its own service. ADR 0082's decision stands: being on the private network is what grants the
|
|
|
|
|
trust, the private network is the transport security, and no module author chooses it. Only who writes
|
|
|
|
|
it moves, from the private network to the runtime's module.
|
|
|
|
|
|
|
|
|
|
**3. The controller writes no file a seat's holder owns.** The private network generates neither the
|
|
|
|
|
runtime's file nor its service.
|
|
|
|
|
|
|
|
|
|
**4. What the mesh computes is held to the collision check.** At composition, each computed module's
|
|
|
|
|
resources, as its generator answers for that machine, are checked beside the other modules' resources.
|
|
|
|
|
The check is the same one: no two modules on a machine declare one path, unit, name or package. A
|
|
|
|
|
collision is refused by name. The generator is asked once, and what is checked is exactly what is
|
|
|
|
|
declared.
|
|
|
|
|
|
|
|
|
|
**5. A unit still held is not given back when another record of it goes.** When the host removes a
|
|
|
|
|
service resource whose unit another declared resource still gives a state, it forgets that record and
|
|
|
|
|
leaves the unit as it is. What the record going had found is not the machine's to restore while
|
|
|
|
|
another resource holds the unit, and restoring it could stop the runtime only for the remaining
|
|
|
|
|
resource to start it again in the same apply.
|
|
|
|
|
|
|
|
|
|
## Consequences
|
|
|
|
|
|
|
|
|
|
- **Rollout has a fixed order, in three steps.**
|
|
|
|
|
1. The controller learns the placeholder. Nothing uses it yet.
|
|
|
|
|
2. The catalogue's runtime module uses it. A controller that does not know the placeholder would
|
|
|
|
|
send it through unfilled, so this waits until step 1 is deployed.
|
|
|
|
|
3. The controller stops generating the private network's two resources and checks generated
|
|
|
|
|
resources for collisions. With step 3 before step 2, machines would lose the trust. The host
|
|
|
|
|
change in Decision 5 is deployed before step 3.
|
|
|
|
|
|
|
|
|
|
This replaces issue 190's single push. That was needed when two writers of one scalar key would
|
|
|
|
|
have been refused. A list member written by two records is tolerated by the host.
|
|
|
|
|
- **In the apply of step 3**, the private network's records go first: the member leaves the list and
|
|
|
|
|
the service record is forgotten. Then the runtime module's file is applied, and the member is added
|
|
|
|
|
back, recorded as the runtime module's. The runtime is reloaded once. The address is unchanged, so
|
|
|
|
|
the trust never lapses for a pull.
|
|
|
|
|
- **A machine holding the store, before any network exists**, is answered with the loopback address
|
|
|
|
|
it reaches the store at. A runtime already trusts loopback, so this adds a redundant member, not a
|
|
|
|
|
wrong one.
|
|
|
|
|
- **A second writer cannot return through generated code.** It is refused at composition, naming
|
|
|
|
|
both modules and what they share.
|
|
|
|
|
- **A module can now learn where the artifact store is without being its consumer.** That is a
|
|
|
|
|
capability, and it is bounded: one seat today, and widened only by a record.
|
|
|
|
|
|
|
|
|
|
## How it is checked
|
|
|
|
|
|
|
|
|
|
| Rule | Checked by |
|
|
|
|
|
|---|---|
|
|
|
|
|
| `${seat:mesh-artifact-store:reach}` is filled with this machine's address for the store, in a file and in an environment value | mesh-controller `TestASeatsReachIsWhereThisMachineReachesItsHolder`, and through the whole composition `TestTheRuntimesTrustIsComposedFromTheSeatsReach` |
|
|
|
|
|
| An empty answer adds nothing to a list, and other members stay | mesh-controller `TestAnUnansweredReachAddsNothingToAList` |
|
|
|
|
|
| Any other seat is refused by name | mesh-controller `TestAReachTheMeshDoesNotAnswerIsRefused` |
|
|
|
|
|
| The catalogue's runtime module writes `live-restore` and the trust, and writes no trust when no store is reachable | mesh-controller `TestTheRuntimesModuleTrustsTheMeshsRegistry`, which composes the catalogue's manifest beside it |
|
|
|
|
|
| The private network declares neither the runtime's file nor its service | mesh-controller `TestTheNetworkWritesNothingOfTheRuntimes` |
|
|
|
|
|
| A generated resource colliding with a module's is refused by name; disjoint ones compose; the generator is asked once | mesh-controller `TestAGeneratedResourceCollidingWithAModulesIsRefused`, `TestAGeneratedResourceBesideAModulesOwnIsComposed`, `TestAComputedModuleIsAskedAboutTheNodeItIsFor` |
|
|
|
|
|
| The member moves between records in one apply without leaving the list; one reload; a unit still held is forgotten, never stopped or disabled; the plan says so | mesh-host `TestTheRegistryMovesToTheRuntimesModuleWithoutLeavingTheList`, `TestThePlanForgetsARecordOfAUnitStillHeld` |
|
|
|
|
|
| After rollout, every machine's runtime trusts the store's address | the runtime module's `docker_daemon_config` tool on each machine, read before step 3 and after it |
|
|
|
|
|
|
|
|
|
|
## References
|
|
|
|
|
|
|
|
|
|
- [Issue 190](../04-ISSUES/190-the-runtimes-configuration-is-written-by-modules-that-are-not-the-runtime/00-report.md),
|
|
|
|
|
steps 2 and 5.
|
|
|
|
|
- [ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md): the trust and why it
|
|
|
|
|
is plain HTTP; its mechanism moves here.
|
|
|
|
|
- [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md): writing into a shared file;
|
|
|
|
|
its writer of the runtime's trust moves here.
|
|
|
|
|
- [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md): a mesh seat names the
|
|
|
|
|
server, which is what lets a placeholder ask about it.
|
|
|
|
|
- [ADR 0166](0166-the-container-runtime-is-a-node-seat-and-the-host-creates-containers-through-its-holder.md):
|
|
|
|
|
the runtime module given the registry as a value.
|
|
|
|
|
- [ADR 0199](0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md):
|
|
|
|
|
the hosts file left the controller the same way.
|
|
|
|
|
- mesh-controller `internal/catalogue/seat_into.go`, `internal/catalogue/declaration.go`
|
|
|
|
|
(`generatedHere`), `internal/overlay/generator.go`; mesh-catalog `modules/docker`; mesh-host
|
|
|
|
|
`internal/apply/apply.go` (orphan removal).
|