Compare commits
8
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
709c095387 | ||
|
|
c262de3833 | ||
|
|
a815433214 | ||
|
|
61dc90e508 | ||
|
|
a7cf5c0c1b | ||
|
|
c6f86ae935 | ||
|
|
1aeb4fe8d8 | ||
|
|
08108b569d |
@@ -164,6 +164,12 @@ def check_rests_on(failures, records):
|
||||
# decision is exactly what as-is is for."
|
||||
if rel(path).startswith("03-DESIGN/00-as-is/"):
|
||||
continue
|
||||
# A withdrawn record's citations are history. It instructs nobody -- every reader
|
||||
# is sent to its superseder -- so what it was built on may itself be withdrawn.
|
||||
# Refusing that would mean rewriting the lineage of a record whose reasoning is
|
||||
# the thing the immutability rule protects.
|
||||
if frontmatter(read(path)).get("status") == "superseded":
|
||||
continue
|
||||
# An extension that supersedes legitimately names what it replaced.
|
||||
this = ADR_FILE.match(os.path.basename(path))
|
||||
supersedes = records[number]["front"].get("superseded-by", "")
|
||||
@@ -241,13 +247,20 @@ def check_supersession_symmetry(failures, records):
|
||||
failures.add("supersession", rel(record["path"]), f"superseder does not exist: {by}")
|
||||
continue
|
||||
other = records[match.group(1)]
|
||||
claims = os.path.basename(str(other["front"].get("supersedes", "")))
|
||||
if claims != record["name"]:
|
||||
# `supersedes:` may name one record or several. One decision replacing two is a real
|
||||
# situation -- two records that built and refined the same wrong mechanism are withdrawn
|
||||
# by the one record that removes it -- and a check that allows only one would force
|
||||
# either a chain of pro-forma records or an unmarked supersession.
|
||||
claimed = other["front"].get("supersedes", "")
|
||||
if isinstance(claimed, str):
|
||||
claimed = [claimed] if claimed else []
|
||||
claims = [os.path.basename(str(entry)) for entry in claimed]
|
||||
if record["name"] not in claims:
|
||||
failures.add(
|
||||
"supersession",
|
||||
rel(other["path"]),
|
||||
f"ADR {number} says this supersedes it; this record does not say so "
|
||||
f"(supersedes: {claims or 'absent'})",
|
||||
f"(supersedes: {', '.join(claims) or 'absent'})",
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
status: superseded
|
||||
date: 2026-09-28
|
||||
deciders: jochen
|
||||
extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
reconstructed: false
|
||||
superseded-by: 02-DECISIONS/0140-the-filter-constrains-what-arrives-from-outside.md
|
||||
---
|
||||
|
||||
# 137. A machine says which networks it routes
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
status: superseded
|
||||
date: 2026-09-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0137-a-machine-says-which-networks-it-routes.md
|
||||
superseded-by: 02-DECISIONS/0140-the-filter-constrains-what-arrives-from-outside.md
|
||||
---
|
||||
|
||||
# 139. A network is forwarded because a module declared it
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
date: 2026-09-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0045-a-machine-firewall-is-the-sum-of-what-it-listens-on.md
|
||||
supersedes:
|
||||
- 02-DECISIONS/0137-a-machine-says-which-networks-it-routes.md
|
||||
- 02-DECISIONS/0139-a-network-is-forwarded-because-a-module-declared-it.md
|
||||
---
|
||||
|
||||
# 140. The filter constrains what arrives from outside, and says nothing about a machine's own guests
|
||||
|
||||
## Context
|
||||
|
||||
The filter the mesh derives blocks traffic passing *through* a machine unless something allows it,
|
||||
because a container's published port is traffic passing through rather than traffic arriving at the
|
||||
machine itself ([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md),
|
||||
[issue 047](../04-ISSUES/047-the-firewall-does-not-cover-published-container-ports/00-report.md)).
|
||||
Having blocked all of it, the filter then had to let the machine's own containers reach outward again.
|
||||
It does that by listing the address ranges those containers sit on.
|
||||
|
||||
As rendered on a converged workstation today:
|
||||
|
||||
```
|
||||
policy drop
|
||||
ct state established,related accept
|
||||
ip saddr 172.16.0.0/12 accept
|
||||
ip saddr 192.168.128.0/17 accept
|
||||
ip saddr 10.0.0.0/8 accept
|
||||
ip saddr 192.168.16.0/20 accept
|
||||
... four more
|
||||
```
|
||||
|
||||
Two of those ranges were constants in the control plane's source. The rest were typed by the operator
|
||||
after [ADR 0137](0137-a-machine-says-which-networks-it-routes.md), which existed to make the typing
|
||||
possible, because converging that workstation had cut every one of its containers off from the
|
||||
internet and nothing reported a fault
|
||||
([issue 137](../04-ISSUES/137-converging-a-machine-cut-off-its-own-guests/00-report.md)).
|
||||
|
||||
**The list is the mistake, not its contents.** Every attempt to make it correct fails the same way.
|
||||
A constant describes one machine. A typed range goes stale, and cannot tell a network the mesh made
|
||||
from one a predecessor left behind — measured on the control-node, where six such ranges fall outside
|
||||
the constants and two of the six belong to services the mesh does not run
|
||||
([issue 141](../04-ISSUES/141-the-forward-chain-does-not-follow-the-modules/00-report.md)).
|
||||
[ADR 0139](0139-a-network-is-forwarded-because-a-module-declared-it.md) tried to generate the same
|
||||
list from the modules and put half the rule set on the machine to do it. Three records, one list, and
|
||||
the list should not exist.
|
||||
|
||||
**Because the mesh has no policy about a container reaching outward.** What the filter is for is
|
||||
stated in [ADR 0045](0045-a-machine-firewall-is-the-sum-of-what-it-listens-on.md): which port is open,
|
||||
and to whom. That is about what arrives. A container of this machine's own opening a connection to
|
||||
something else is not a port being opened to anybody, and enumerating the addresses it might do so
|
||||
from is bookkeeping about the machine's internal plumbing, which the mesh neither owns nor can know.
|
||||
|
||||
**The system being replaced never had this fault, and its rule says why.** The chain still protecting
|
||||
the control-node applies only to traffic arriving on that machine's outward link, and leaves
|
||||
everything else alone. The mesh's filter dropped that distinction and replaced it with a list of
|
||||
addresses.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Keep the list and generate it better** — from the modules' declared networks, or from what the
|
||||
machine reports. Rejected: [ADR 0139](0139-a-network-is-forwarded-because-a-module-declared-it.md)
|
||||
is that, and it puts part of the rule set on the machine, which makes the rule set partly the
|
||||
machine's and the derivation advisory.
|
||||
2. **Name the guest links instead of their addresses, and allow only those.** Rejected as more than is
|
||||
needed: it fails in the safe direction, but it is still a list that has to keep up with the
|
||||
machine, and the thing it protects against — a container reaching outward — is not a thing the mesh
|
||||
has a position on.
|
||||
3. **Do not block traffic passing through at all.** Rejected: that is
|
||||
[issue 047](../04-ISSUES/047-the-firewall-does-not-cover-published-container-ports/00-report.md),
|
||||
where a published port was reachable from anywhere because no rule mentioned it.
|
||||
4. **Constrain what arrives from outside, and nothing else.** Adopted.
|
||||
|
||||
## Decision
|
||||
|
||||
**The filter constrains traffic arriving from outside the machine, and says nothing about traffic that
|
||||
did not.** Traffic passing through the machine is allowed unless it arrived on one of the machine's
|
||||
outward links, in which case it is allowed only where a declared endpoint's reach admits it
|
||||
([ADR 0138](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md)). A container of this
|
||||
machine's own reaching anywhere is not filtered, because the mesh has no position on it.
|
||||
|
||||
**A machine says which of its links face outside.** One node-level fact, reported by the machine the
|
||||
way it already reports the kind of firewall it found and the tunnel it carries — not a setting, not a
|
||||
list of addresses, and not something anybody types. It does not change when a module is added or
|
||||
removed, which is what separates it from the list it replaces.
|
||||
|
||||
**A machine that has reported no outward link is sent no filter.** Rendering a rule around a link
|
||||
whose name is not known produces a rule set that does not load, which is a machine filtering nothing
|
||||
while its unit reports success. The refusal happens in the control plane, where a person reads it, and
|
||||
the machine keeps the filter it already has.
|
||||
|
||||
**No addresses of the machine's own networks appear in the filter.** The two constants are removed and
|
||||
`node networks` is removed with them, along with everything any machine was told to say through it.
|
||||
Ports continue to follow the modules exactly as before: a module assigned to a machine opens the port
|
||||
its assignment says it reaches on, and nothing about a network is said anywhere.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Three records collapse into one rule.** 0137 and 0139 are superseded. What 0137 was right about —
|
||||
that converging a machine had silently cut off its own containers, and that nothing previewed it — is
|
||||
answered by removing the cause rather than by giving the operator a way to compensate for it.
|
||||
- **Every machine already converged loses its declared ranges and keeps working**, because the traffic
|
||||
those ranges allowed is now allowed by not having arrived from outside. The workstation's five ranges
|
||||
and the laptop's one are deleted rather than migrated.
|
||||
- **A machine's test beds stop being a special case.** A bed's network is created while the machine
|
||||
runs and was the case no list could cover; it is now covered by not being mentioned.
|
||||
- **A new fact travels in the report**, and the control plane refuses to compose a filter without it,
|
||||
so the order of the roll-out matters: the machines report before the control plane depends on it.
|
||||
- **A machine with more than one outward link says so**, and a machine that acquires one while the mesh
|
||||
is not looking is treated as internal until its next report. That window is the cost of this shape;
|
||||
it is bounded by the report interval, and it exists on machines whose outward link changes, which
|
||||
are the machines with nothing published to the outside.
|
||||
- **What got harder:** nothing in the declaration, and one more thing a machine must be able to work
|
||||
out about itself. A machine that cannot say which link faces outside cannot be given a filter.
|
||||
|
||||
## How it is checked
|
||||
|
||||
- **A machine's own container reaches outward with no network named anywhere.** A bed converges a
|
||||
machine carrying containers on several networks, none of them mentioned in any setting, and each
|
||||
reaches out afterwards. This fails against the previous behaviour, where the same flip cut them off,
|
||||
and that is how it is written.
|
||||
- **A port declared reachable from outside is reachable; one that is not, is not.** Probed from off the
|
||||
machine's private network, for a published port and for an undeclared one, before and after the flip.
|
||||
- **A network created after the filter was composed needs no new filter.** A network is made on the
|
||||
machine after its last declaration and a container on it reaches out, with nothing re-sent.
|
||||
- **No address of a machine's own networks appears in a rendered filter**, asserted on the text so a
|
||||
range cannot creep back in.
|
||||
- **A machine that reports no outward link is sent no filter, and the refusal names it** — asserted in
|
||||
the control plane, and that the machine's existing filter is left alone.
|
||||
- **A machine reporting two outward links has both constrained**, asserted per chain body so a rule
|
||||
covering one and not the other cannot pass.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0045](0045-a-machine-firewall-is-the-sum-of-what-it-listens-on.md) — what the filter is for
|
||||
- [ADR 0138](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md) — what admits traffic
|
||||
arriving from outside
|
||||
- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md) — why traffic passing through is
|
||||
filtered at all
|
||||
- [ADR 0137](0137-a-machine-says-which-networks-it-routes.md),
|
||||
[ADR 0139](0139-a-network-is-forwarded-because-a-module-declared-it.md) — superseded here
|
||||
- [issue 137](../04-ISSUES/137-converging-a-machine-cut-off-its-own-guests/00-report.md),
|
||||
[issue 141](../04-ISSUES/141-the-forward-chain-does-not-follow-the-modules/00-report.md)
|
||||
@@ -0,0 +1,154 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
date: 2026-09-29
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0005-the-node-host.md
|
||||
---
|
||||
|
||||
# 141. The host delivers its own successor, and versions live side by side
|
||||
|
||||
## Context
|
||||
|
||||
[Issue 142](../04-ISSUES/142-the-host-is-the-one-thing-the-mesh-does-not-deliver/00-report.md). A
|
||||
merge builds every changed module and the control plane — which is itself a module — and the result
|
||||
reaches the machines running it with nobody asking. The host is the exception: it is not a build
|
||||
target, no declaration delivers it, and every machine in this mesh runs a byte-identical binary that
|
||||
somebody built on a workstation and copied out.
|
||||
|
||||
The half that *recovers* from a bad host exists. `internal/upgrade` can tell that the executable this
|
||||
process started from was replaced on disk, and it records which version last completed a reconcile.
|
||||
The launcher counts consecutive failed starts, calls a rollback at the limit, and treats a clean exit
|
||||
as the host standing aside so that the next loop runs whatever is on disk now. That supervision is
|
||||
complete and correct.
|
||||
|
||||
Two things make it dead code:
|
||||
|
||||
- **`Replaced()` is called by nothing but its own tests.** Nothing tells the running host that a
|
||||
successor is waiting.
|
||||
- **The rollback resolves a version through the machine's package manager** — `pacman -U` from the
|
||||
package cache. No machine here has the host installed as a package, so the recovery cannot run on
|
||||
any of them; and being written in one package manager's terms, it cannot run on two of the three
|
||||
operating systems the host is built for — [ADR 0005](0005-the-node-host.md) builds one binary per
|
||||
operating system, pinned at link time.
|
||||
|
||||
**The record already points at the answer.** What is kept is a *version*, not a path. Keeping a
|
||||
version is only useful to something that can choose between versions present on the machine, which is
|
||||
what the package manager was being asked to do. The versions can simply be on disk.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Deliver the host as a package, as the rollback assumes.** Rejected: it needs a package built and
|
||||
a repository trusted per operating system, three of each, and the existing `package` resource
|
||||
asserts presence and deliberately never a version — "version is the package manager's business and
|
||||
the mesh does not hold a second opinion about it" — so it cannot ask for a particular host anyway.
|
||||
Heaviest of the three and the only one that is different on every machine.
|
||||
2. **Write the new binary over the running one.** Rejected on a fact: a running executable cannot be
|
||||
truncated, and `archive` opens what it unpacks with `O_TRUNC`. It could be made to write and
|
||||
rename, which is better hygiene and worth doing for its own sake, but it buys nothing here that
|
||||
option 3 does not, and it leaves rollback with nowhere to go back to.
|
||||
3. **Versions side by side; the newest retires the old.** Adopted.
|
||||
|
||||
## Decision
|
||||
|
||||
**A host version is delivered as an archive into a directory named for it, and never over a running
|
||||
one.** The declaration names it like any other archive — fetched by digest, the digest checked before
|
||||
anything is unpacked. Nothing new travels, no new resource kind, and no change to how archives are
|
||||
applied, because the path being written is not the path being executed.
|
||||
|
||||
**The launcher starts the most recently delivered version.** That is what "the newest" means: the
|
||||
version whose directory arrived last. It reads no pointer and follows no link — the mesh creates no
|
||||
links ([ADR 0012](0012-the-mesh-creates-no-symlinks.md)) — and the version is in the path, so nothing
|
||||
has to be told what is running.
|
||||
|
||||
**The running host stands aside for a successor, and only between reconciles.** Finding a newer
|
||||
version delivered, it finishes the reconcile it is in and exits cleanly. The launcher already reads a
|
||||
clean exit as exactly this and starts what is on disk now. A host that stood aside mid-apply is the
|
||||
half-configured machine this project exists to prevent, so the check happens at the boundary and
|
||||
nowhere else.
|
||||
|
||||
**A version that completes a reconcile records itself, and retires what came before it.** The
|
||||
known-good record is written as it is today. Then versions older than the one before the running one
|
||||
are removed: the running version and its predecessor are kept, which is exactly what a rollback
|
||||
needs, and nothing else accumulates.
|
||||
|
||||
**Rollback starts the previous version instead of reinstalling a package.** At the failure limit the
|
||||
launcher pins the known-good version and starts that, once. The second failure is still a different
|
||||
diagnosis — the previously working version does not run either, so it is the machine and not the
|
||||
binary — and the halt is unchanged. No package manager, no package cache, and the same script on every
|
||||
operating system.
|
||||
|
||||
**A machine says which host version it is running,** on the report it already sends, beside the other
|
||||
facts it states about itself. Without it nothing can say a machine is behind, so "every machine
|
||||
current with its source" cannot include the host.
|
||||
|
||||
## Progressive insight — 2026-09-29, the same day
|
||||
|
||||
**The delivery is not "nothing new", and this record said it was.** The decision above stands and is
|
||||
built: versions side by side, the newest runs, the running host stands aside between reconciles, a
|
||||
completed reconcile retires what is older than the predecessor, rollback picks a directory. What was
|
||||
wrong was a claim about how a version reaches a machine. The paragraph on delivery said the
|
||||
declaration "names it like any other archive… nothing new travels, no new resource kind"; the second
|
||||
half is true and the first is not, because two things the delivery needs do not exist:
|
||||
|
||||
- **Nothing can compile it.** A `bundle` artifact is compiled by a closed list of toolchains —
|
||||
typescript and python — whose own comment says adding a language is a decision, because a language
|
||||
used by *modules* needs an SDK carrying the broker client, the event envelope and tool serving. The
|
||||
host uses none of that: it is what applies modules, not one of them. So the obligation that list
|
||||
warns about attaches to a module written in a language, not to the language being buildable, and
|
||||
the control plane — also written in Go — is built as an image from a Dockerfile rather than through
|
||||
a toolchain at all.
|
||||
- **A version cannot reach the path.** An `archive` resource names a fixed path in the manifest, and
|
||||
nothing interpolates the built version into it, so nothing can ask for
|
||||
`…/versions/<version>/`.
|
||||
|
||||
Neither changes what was decided, which options were weighed, or any consequence: the shape is
|
||||
unaffected and the host half is merged and tested. What it changes is the cost, which this record
|
||||
understated as none. The remaining work is a way to build the host and a way to name a version in a
|
||||
path, and until both exist nothing delivers a version and every machine takes the fallback — which is
|
||||
what every machine does today.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The host becomes a build target and a module** — a module whose resource is the next host, applied
|
||||
by the host that is running. The bootstrap is not circular because the two are different versions in
|
||||
different directories.
|
||||
- **Rollback becomes usable on every machine**, having been usable on none. It also stops being
|
||||
written in one operating system's terms.
|
||||
- **One copy by hand remains, once.** The first host that understands versioned directories cannot be
|
||||
fetched by a host that does not. That copy is the last, and it is the honest cost of the change
|
||||
rather than a step in the design.
|
||||
- **Two versions occupy disk instead of one.** About nine megabytes. The predecessor is the price of a
|
||||
rollback that does not depend on a cache somebody else may clean.
|
||||
- **What got harder:** a host must now be able to find its own successor and to judge when it is safe
|
||||
to stand aside. Both are between reconciles, which is the only moment the host is not mid-change.
|
||||
- **A machine that is never told a newer version keeps running what it has**, indefinitely and
|
||||
visibly, because its report says which version that is.
|
||||
|
||||
## How it is checked
|
||||
|
||||
- **A delivered version is run, and the old one is not.** A bed delivers a second version to a machine
|
||||
running the first; the host exits between reconciles, the launcher starts the new one, and the
|
||||
machine reports the new version. This fails against the previous behaviour, where nothing notices a
|
||||
delivered version at all.
|
||||
- **It stands aside between reconciles and never inside one.** Asserted by delivering a version while
|
||||
an apply is in flight: the apply completes, and the exit follows it.
|
||||
- **A version that will not start is rolled back to its predecessor, once**, and the second failure
|
||||
halts with the machine named rather than the binary — asserted with no package manager involved.
|
||||
- **A completed reconcile retires what is older than the predecessor**, and never the predecessor
|
||||
itself, because that is what a rollback needs. Asserted on the directory afterwards.
|
||||
- **The report names the running version**, asserted end to end rather than on the function that reads
|
||||
it, since the point is that the control plane can tell a machine is behind.
|
||||
- **The launcher picks the newest delivered version** with no pointer file and no link, asserted by
|
||||
delivering two and checking which runs.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0005](0005-the-node-host.md) — the host, and what its supervision is for
|
||||
- [ADR 0010](0010-delivery.md) — a declaration is owned resources; this adds no kind to it
|
||||
- [ADR 0012](0012-the-mesh-creates-no-symlinks.md) — why the version is in the path
|
||||
- [ADR 0005](0005-the-node-host.md), *it is built per operating system* — why a rollback written in
|
||||
one package manager's terms was wrong for two of three
|
||||
- [issue 142](../04-ISSUES/142-the-host-is-the-one-thing-the-mesh-does-not-deliver/00-report.md) — the
|
||||
measurement
|
||||
@@ -217,9 +217,11 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0133** — [A module owns its migrations, and the mesh owns when they run](0133-a-module-owns-its-migrations-and-the-mesh-owns-when-they-run.md) *(superseded)*
|
||||
- **0135** — [A module version prepares its state before it runs](0135-a-module-version-prepares-its-state-before-it-runs.md)
|
||||
- **0136** — [A step gates its module, not the machine](0136-a-step-gates-its-module-not-the-machine.md)
|
||||
- **0137** — [A machine says which networks it routes](0137-a-machine-says-which-networks-it-routes.md)
|
||||
- **0137** — [A machine says which networks it routes](0137-a-machine-says-which-networks-it-routes.md) *(superseded)*
|
||||
- **0138** — [An assignment binds an endpoint and says how far it reaches](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md)
|
||||
- **0139** — [A network is forwarded because a module declared it](0139-a-network-is-forwarded-because-a-module-declared-it.md)
|
||||
- **0139** — [A network is forwarded because a module declared it](0139-a-network-is-forwarded-because-a-module-declared-it.md) *(superseded)*
|
||||
- **0140** — [The filter constrains what arrives from outside, and says nothing about a machine's own guests](0140-the-filter-constrains-what-arrives-from-outside.md)
|
||||
- **0141** — [The host delivers its own successor, and versions live side by side](0141-the-host-delivers-its-own-successor.md)
|
||||
|
||||
### How it is built
|
||||
|
||||
|
||||
@@ -2,8 +2,9 @@
|
||||
layer: to-be
|
||||
status: in-progress
|
||||
code: [mesh-host]
|
||||
updated: 2026-09-22
|
||||
updated: 2026-09-29
|
||||
decisions:
|
||||
- 02-DECISIONS/0141-the-host-delivers-its-own-successor.md
|
||||
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
- 02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md
|
||||
- 02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md
|
||||
@@ -423,3 +424,45 @@ ignored an instruction and "applied" would be a lie. Applying stays one at a tim
|
||||
is not. **Checked** by the link's unit tests on the drain, and by the genesis bed's settle wait,
|
||||
which counts on a node catching up to the newest declaration rather than the oldest.
|
||||
|
||||
## The host delivers its own successor
|
||||
|
||||
*2026-09-29, from a change to the host that could reach no machine —
|
||||
[issue 142](../../04-ISSUES/142-the-host-is-the-one-thing-the-mesh-does-not-deliver/00-report.md),
|
||||
settled by [ADR 0141](../../02-DECISIONS/0141-the-host-delivers-its-own-successor.md).*
|
||||
|
||||
A merge builds every changed module and the control plane, and the result reaches the machines running
|
||||
it with nobody asking. The host was the exception: not a build target, named by no declaration, and
|
||||
identical on every machine because somebody had copied it there.
|
||||
|
||||
The supervision needed for this was already right. A clean exit from the host means it has stood aside,
|
||||
and the launcher's next turn runs whatever is on disk. Consecutive failed starts are counted, a
|
||||
rollback happens at the limit, and a second failure halts with the machine named rather than the binary.
|
||||
What was missing was smaller than it looked: nothing told the running host a successor was waiting, and
|
||||
the rollback resolved its known-good *version* through one operating system's package manager, which no
|
||||
machine here used.
|
||||
|
||||
Keeping a version rather than a path was the clue. That is only useful to something that can choose
|
||||
between versions present on the machine — so the versions live side by side:
|
||||
|
||||
- **A version arrives as an archive, in a directory named for it.** The ordinary resource, fetched by
|
||||
digest and checked before anything is unpacked. The path written is never the path being executed, so
|
||||
replacing a running binary — which the kernel refuses — never comes up.
|
||||
- **The launcher starts the most recently delivered version**, reading no pointer and following no
|
||||
link, because the version is in the path.
|
||||
- **The running host stands aside between reconciles and never inside one.** Standing aside mid-apply is
|
||||
the half-configured machine this document exists to prevent.
|
||||
- **A version that completes a reconcile records itself and retires what is older than its
|
||||
predecessor.** The predecessor stays, because that is what a rollback needs.
|
||||
- **Rollback starts that predecessor** instead of reinstalling a package: no package manager, no cache
|
||||
somebody else may clean, and the same script on every operating system.
|
||||
- **A machine says which host version it runs**, on the report it already sends, so being behind is
|
||||
answerable at all.
|
||||
|
||||
One copy by hand remains, once: the first host that understands versioned directories cannot be fetched
|
||||
by a host that does not.
|
||||
|
||||
*How it is checked* is stated with the decision — a second version delivered to a running machine is
|
||||
run and reported; the exit follows an in-flight apply rather than interrupting it; a version that will
|
||||
not start is rolled back once and the second failure halts; a completed reconcile retires what is older
|
||||
than the predecessor and never the predecessor; and the newest of two delivered versions is the one
|
||||
that runs.
|
||||
|
||||
@@ -10,7 +10,7 @@ code:
|
||||
updated: 2026-09-28
|
||||
decisions:
|
||||
- 02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md
|
||||
- 02-DECISIONS/0139-a-network-is-forwarded-because-a-module-declared-it.md
|
||||
- 02-DECISIONS/0140-the-filter-constrains-what-arrives-from-outside.md
|
||||
- 02-DECISIONS/0104-a-provision-may-be-answered-by-an-adapter-to-the-predecessor.md
|
||||
- 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
- 02-DECISIONS/0105-the-mesh-adopts-the-predecessors-tunnel-in-place.md
|
||||
@@ -627,41 +627,49 @@ the found firewall reloads and reachable from a container on the node, that a ma
|
||||
enrols through the openings before and after a reload and a reboot, and that after the flip the
|
||||
declared port is open and the undeclared one closed.
|
||||
|
||||
### The networks it forwards are the ones its modules declared
|
||||
### It filters what arrives from outside, and not what the machine's own guests send
|
||||
|
||||
*2026-09-28, preparing the control-node's convergence —
|
||||
[issue 141](../../04-ISSUES/141-the-forward-chain-does-not-follow-the-modules/00-report.md), settled by
|
||||
[ADR 0140](../../02-DECISIONS/0140-the-filter-constrains-what-arrives-from-outside.md), which replaces
|
||||
[ADR 0137](../../02-DECISIONS/0137-a-machine-says-which-networks-it-routes.md) and
|
||||
[ADR 0139](../../02-DECISIONS/0139-a-network-is-forwarded-because-a-module-declared-it.md).*
|
||||
|
||||
The forward chain denies by default, so a machine's own guests have to be allowed back in. Until now
|
||||
that was two ranges named in the control plane and, since
|
||||
[ADR 0137](../../02-DECISIONS/0137-a-machine-says-which-networks-it-routes.md), a list a machine could
|
||||
add to. On the machine measured here, six of its container networks fell outside those ranges — and
|
||||
four of the six were networks the mesh's own modules had declared and the host had created, while two
|
||||
were the predecessor's leftovers. A range wide enough to keep the four keeps the two: a filter widened
|
||||
by hand to protect what should not be there.
|
||||
Traffic passing through a machine is filtered, because a container's published port is traffic passing
|
||||
through rather than traffic arriving at the machine itself. Having blocked it, the filter then had to
|
||||
let the machine's own containers reach outward again — and it did that by listing the address ranges
|
||||
they sit on. Two of those ranges were constants in this repository's code, and the rest were typed by an
|
||||
operator after the flip had already cut a workstation's containers off from everything.
|
||||
|
||||
**A network is forwarded because a module declared it.** The forward chain forwards the networks of the
|
||||
modules assigned to that node and, by default, nothing else; a module unassigned stops being forwarded
|
||||
at the next reconcile. The control plane says which networks, by name, and the host — which created
|
||||
them — renders their addresses, because the runtime allocates the subnet and the host is what knows it.
|
||||
That keeps §4's shape and this document's: the mesh decides, the host applies.
|
||||
**The list was the mistake, not its contents.** A constant describes one machine. A typed range goes
|
||||
stale and cannot tell a network the mesh made from one a predecessor left behind — on the control-node,
|
||||
six ranges fall outside the constants and two of the six belong to services the mesh does not run. The
|
||||
attempt to generate the list from the modules put half the rule set on the machine and made the
|
||||
derivation advisory. Three records, one list.
|
||||
|
||||
**This is derivation from the declaration, not from the machine.** 0137 rejected reading the machine,
|
||||
for two reasons that do not apply here: a network created between two declarations is already named in
|
||||
the one that asked for it, and a network nobody declared is never forwarded however it appeared. What
|
||||
0137's mechanism keeps is the case it was right for — guests no module declares, a test bed's range —
|
||||
added to the derived set and never replacing it.
|
||||
**And the mesh has no position on a container reaching outward.** §4 exists to say which port is open
|
||||
and to whom, which is about what arrives. A container of this machine's own opening a connection
|
||||
somewhere is not a port opened to anybody, and the addresses it might do that from are the machine's
|
||||
internal plumbing, which the mesh neither owns nor can know.
|
||||
|
||||
The runtime's own default bridge, which a container attaches to when it names no module network, is the
|
||||
runtime's and not a module's, so the host renders it from what the runtime reports. With that, no range
|
||||
is named in the control plane at all.
|
||||
So the filter constrains what arrives from **outside** the machine and says nothing about what did not.
|
||||
Traffic passing through is allowed unless it came in on one of the machine's outward links, and then
|
||||
only where a declared endpoint's reach admits it (§6). The machine says which of its links face
|
||||
outside — one fact it reports, like the kind of firewall it found and the tunnel it carries, not a
|
||||
setting and not a list of addresses. It does not change when a module is added or removed, which is the
|
||||
whole difference from what it replaces. A machine that has reported no outward link is sent no filter
|
||||
at all, and keeps the one it has, because a rule written around a link with no name is a rule set that
|
||||
does not load — a machine filtering nothing while its unit reports success.
|
||||
|
||||
*How it is checked:* a node with two modules declaring networks renders rules for exactly those two and
|
||||
none for a third present on the machine that nothing declared — which fails against forwarding by
|
||||
range, and is how it was written; unassigning one removes its rule at the next reconcile; the default
|
||||
bridge is asserted for a runtime whose bridge is somewhere other than the old constant named; and the
|
||||
guests of a declared network keep address and name service, per chain body.
|
||||
Ports go on following the modules exactly as before: assign a module to a machine and the port its
|
||||
assignment says it reaches on opens. Nothing about a network is said anywhere, by anybody.
|
||||
|
||||
*How it is checked:* a bed converges a machine carrying containers on several networks, none of them
|
||||
named in any setting, and each reaches outward afterwards — which fails against the previous behaviour,
|
||||
where the same flip cut them off, and is how it was written; a network made *after* the last declaration
|
||||
needs no new filter; a declared port is reachable from off the private network and an undeclared one is
|
||||
not; no address of a machine's own networks appears in a rendered filter, asserted on the text; and a
|
||||
machine reporting no outward link is refused in the control plane with its existing filter left alone.
|
||||
|
||||
## 5 — Certificates
|
||||
|
||||
|
||||
@@ -61,6 +61,18 @@ module declares — a test bed's pool — which is a much smaller residue than t
|
||||
has to know the runtime's allocations to tell whether that line is sufficient. On the machine
|
||||
measured here it read as though nothing needed saying.
|
||||
|
||||
## What was decided
|
||||
|
||||
*2026-09-28, later the same day.* The answer is not a better list. The question in the first open
|
||||
item below — should the chain be derived from the networks the modules declare — was answered *no*,
|
||||
after a converged machine's rendered rules were read: the chain blocks everything passing through the
|
||||
machine and then allows its own guests back by listing their addresses. Every route to a correct list
|
||||
fails, because the mesh has no position on a container reaching outward in the first place. The filter
|
||||
now constrains what arrives from **outside** the machine and says nothing about what did not, and a
|
||||
machine says which of its links face outside — one reported fact instead of a list. See
|
||||
[ADR 0140](../../02-DECISIONS/0140-the-filter-constrains-what-arrives-from-outside.md), which
|
||||
supersedes both 0137 and the first attempt at answering this.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should the forward chain be derived from the network resources the node's modules declare, with the
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
---
|
||||
status: located
|
||||
opened: 2026-09-29
|
||||
located-in:
|
||||
- mesh-host internal/upgrade
|
||||
- mesh-host cmd/mesh-host
|
||||
- mesh-controller (no build source for the host; no resource delivers it)
|
||||
fixed-by:
|
||||
amended-design: 03-DESIGN/01-to-be/05-the-node-host.md
|
||||
---
|
||||
|
||||
# 142 — The host is the one thing the mesh does not deliver
|
||||
|
||||
## What was observed
|
||||
|
||||
A change to the host was merged and could not reach any machine without a person copying a file.
|
||||
|
||||
Checked on the mesh of four machines, 2026-09-29:
|
||||
|
||||
- **The host is not a build target.** Asked what had been built for it, the control plane answered
|
||||
`nothing has been built for mesh-host`. A merge on the forge builds every changed module and the
|
||||
control plane itself, because the control plane is a module. The host is not one, and nothing
|
||||
builds it.
|
||||
- **No declaration delivers it.** No resource kind names an executable to place on a machine, and
|
||||
nothing on a machine fetches one.
|
||||
- **The half that recovers from a bad host exists and is unused.** `internal/upgrade` can report that
|
||||
the executable this process started from has been replaced on disk, and records which version last
|
||||
completed a reconcile so a shell script can roll back a host that will not start. The launcher reads
|
||||
that record and rolls back. But `Replaced()` is called by nothing except its own tests — the
|
||||
recovery is wired and the delivery was never built.
|
||||
- **Every machine runs a byte-identical binary, stamped by hand.** All four carry the same size and
|
||||
the same timestamp, from the last time somebody built it on a workstation and copied it out. No
|
||||
package owns the file.
|
||||
|
||||
## Why it matters beyond this instance
|
||||
|
||||
**The component that implements updating is the one thing not updated.** The mesh's stated shape is
|
||||
that a push produces the right builds and they reach the machines running them with nobody asking. It
|
||||
is true of every module and of the control plane. It is false for the host, which is what applies all
|
||||
of them.
|
||||
|
||||
**It is a bootstrap problem being answered by a person.** The host cannot be an ordinary module
|
||||
because the host is what applies modules; a module that replaces the thing applying it has to survive
|
||||
its own replacement. That is a real difficulty, and the work already done — noticing that the
|
||||
executable changed, recording a known-good version, a launcher that rolls back — is the hard half of
|
||||
solving it. What is missing is the easy half, and its absence makes the hard half dead code.
|
||||
|
||||
**A hand-copied binary has no record anywhere.** Nothing says which version a machine runs, so
|
||||
nothing can say a machine is behind, and the mesh's own account of itself — every machine current with
|
||||
its source — cannot include the host. Four machines agreeing today is luck, not a property.
|
||||
|
||||
**And it silently gates any change that starts in the host.** A change that needs the host to report
|
||||
something new cannot be rolled out by merging it: the control plane must wait for a person, and until
|
||||
then it either refuses what depends on the new report or renders something wrong. That cost is paid by
|
||||
every future change of this shape, and it was paid today.
|
||||
|
||||
## Open questions
|
||||
|
||||
- How is the host delivered without being applied by itself? A candidate shape: the host is built like
|
||||
anything else, published as an artifact, and the *running* host fetches and stages the next one, then
|
||||
stands aside — which is what `Replaced()` was written for and what the launcher's rollback already
|
||||
covers.
|
||||
- **Should this ride the bus, rather than becoming a mechanism of its own?** Everything else that
|
||||
reaches a machine already does: a declaration is sent over it, a report comes back over it, and a
|
||||
build announces what it produced on it, which is how a module's new version reaches the machines
|
||||
running it. A host build announcing itself the same way, consumed by the host already running,
|
||||
would make this the existing mechanism pointed at one more artifact rather than a second way of
|
||||
delivering things. It would also give the machine somewhere to say which host it is running, on the
|
||||
report it already sends.
|
||||
- What records which version of the host a machine runs, so "behind" is answerable? Nothing does now.
|
||||
- Does the host's version belong in its report, beside the other facts a machine states about itself?
|
||||
- Who decides when a machine takes a new host — the mesh, on a build, or an operator per machine as
|
||||
with converging? The rollback path means a bad host costs a reconcile rather than a machine, which
|
||||
argues for the former.
|
||||
- Does the same gap apply to the launcher and the units beside the binary, which are also files no
|
||||
declaration names?
|
||||
Reference in New Issue
Block a user