Compare commits
10
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
05039c4f10 | ||
|
|
b7bf601ca4 | ||
|
|
bff32e3370 | ||
|
|
d6b62387f2 | ||
|
|
4789624857 | ||
|
|
e00862e317 | ||
|
|
3a92e4b80c | ||
|
|
bfddf78bf3 | ||
|
|
0b291d89f3 | ||
|
|
a924efcc28 |
@@ -1,6 +1,6 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: proposed
|
||||
status: accepted
|
||||
date: 2026-09-25
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
date: 2026-09-28
|
||||
deciders: jochen
|
||||
extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 137. A machine says which networks it routes
|
||||
|
||||
## Context
|
||||
|
||||
The filter the mesh derives denies forwarding by default, because without a forward chain it says
|
||||
nothing about a container's published port
|
||||
([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), [issue 047](../04-ISSUES/047-the-firewall-does-not-cover-published-container-ports/00-report.md)).
|
||||
To keep a machine's own containers working it then allows two ranges: the container runtime's
|
||||
default bridge pool, and the pool its compose files are given. Those two are named in the
|
||||
controller's code, with a comment saying what the gap is:
|
||||
|
||||
> A machine whose runtime is configured with something else needs this to say so — which is a thing
|
||||
> the mesh cannot derive and a reason this list is named here rather than computed.
|
||||
|
||||
**There was no way to say so.** The list was a constant. A machine whose guests live anywhere else
|
||||
was filtered by a rule that looked deliberate and was a guess.
|
||||
|
||||
**Measured, on the day a workstation was converged.** Flipping it cut egress for five of its
|
||||
container networks at once, and for every network its test beds create — the beds allocate a fresh
|
||||
range per run, from a pool neither default covers. Nothing reported a fault. The containers could
|
||||
not reach anything, the machine went on reporting that it had applied what it was told, and the
|
||||
converge preview had said nothing about it either, because the preview lists what *listens* and
|
||||
routing is not a listener.
|
||||
|
||||
**And two questions, not one.** A guest also asks its host for an address and for names. Both arrive
|
||||
at the input chain, where nothing declared them, so denying by default left the guests of a routed
|
||||
network with no address and no resolution — which is not a closed port but a network that does not
|
||||
function, asked for by this machine's own guest.
|
||||
|
||||
**Why the machine cannot simply be read.** A test bed creates its bridge while it runs, between one
|
||||
declaration and the next, so a filter derived from what the machine last reported would be correct
|
||||
only for the networks that already existed when it was composed. A declared range covers the ones
|
||||
that do not exist yet.
|
||||
|
||||
## Decision
|
||||
|
||||
**A machine says which networks it routes for what it hosts, and the filter forwards them.** A
|
||||
node-level fact, beside the node's public domain
|
||||
([ADR 0066](0066-public-routing-is-name-agnostic.md)) and for the same reason: the
|
||||
machine routes them, and the module that loads the filter holds a seat and may be replaced.
|
||||
|
||||
**Added to the runtime's defaults, never replacing them.** A machine that names one range has not
|
||||
stopped hosting whatever was already on the runtime's own pools, and replacing would trade one
|
||||
silent breakage for another.
|
||||
|
||||
**Their guests keep address and name service.** For a network that was named, the input chain admits
|
||||
that network's own DHCP and DNS, and nothing else: everything else a guest might want from its host
|
||||
is a port somebody declares, like every other port on this machine.
|
||||
|
||||
**Said in CIDR form and checked when it is said.** An entry that does not parse is a line nftables
|
||||
refuses, and a refused ruleset is a machine filtering nothing while its unit reports a fault — so
|
||||
the refusal happens where a person can read it, not on the machine.
|
||||
|
||||
**A machine that says nothing is filtered exactly as before.** Every machine already converged is
|
||||
untouched by this.
|
||||
|
||||
## Options considered
|
||||
|
||||
1. **Leave it constant and edit the code per installation.** Rejected: the value is a property of
|
||||
one machine, the code is the whole mesh's, and the two ranges as they stand describe a machine
|
||||
whose runtime was left at its defaults. It is also how this got here.
|
||||
2. **Derive it from what the machine reports.** Rejected as insufficient, not as wrong: it cannot
|
||||
cover a network created between two declarations, which is precisely the case that was broken. It
|
||||
would also make the filter follow whatever appeared on the machine, which is a firewall that
|
||||
widens itself.
|
||||
3. **A per-node setting on the module that loads the filter.** Rejected: the machine routes the
|
||||
networks. The filter module holds a node-scoped seat and is meant to be replaceable, and a
|
||||
replacement must not lose the machine's own truth.
|
||||
4. **Replace the defaults with what is said.** Rejected: see the decision. The first machine to name
|
||||
its bed range would lose its containers.
|
||||
5. **Admit all input from a routed network, not only address and name service.** Rejected: that is
|
||||
every port on the machine open to anything it hosts, which is the derivation abandoned.
|
||||
|
||||
## Consequences
|
||||
|
||||
**The converge preview says what a machine routes**, including when it routes nothing but the
|
||||
defaults, with the command that changes it. The preview's own sentence about traffic it cannot
|
||||
preview stays, because a tunnel and the found firewall's NAT are still not previewable.
|
||||
|
||||
**A machine whose guests are already broken by an earlier flip is fixed by saying its networks and
|
||||
pushing**, with no flip to undo.
|
||||
|
||||
**The list is one more thing that can be wrong and stale.** A range removed from the machine and
|
||||
left here keeps forwarding for a network that no longer exists, which admits nothing, because there
|
||||
is no guest on it to admit. That is the safe direction of being out of date.
|
||||
|
||||
## How this is checked
|
||||
|
||||
- **What a machine says it routes is forwarded, and its guests keep address and name service.** A
|
||||
test renders a ruleset for a machine that names one range and asserts both chains, per chain body
|
||||
so a line in the wrong chain cannot pass it. It fails against the previous behaviour, which is how
|
||||
it was written.
|
||||
- **The runtime's own defaults survive naming a range.** Asserted in the same test.
|
||||
- **A machine that names nothing renders byte-identically to one that names nil**, so every machine
|
||||
already behind this filter is untouched.
|
||||
- **Each family is matched in its own syntax.** A test with one v4 and one v6 network asserts
|
||||
`ip saddr` and `ip6 saddr`, because one set holding both is a syntax error and a ruleset that does
|
||||
not load is a machine filtering nothing.
|
||||
- **An entry that is not a network is refused where it is said**, by the parse in the setter.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md) — the derived filter this completes
|
||||
- [ADR 0066](0066-public-routing-is-name-agnostic.md) — the precedent for a node-level fact
|
||||
- [issue 047](../04-ISSUES/047-the-firewall-does-not-cover-published-container-ports/00-report.md) — why there is a forward chain at all
|
||||
- [issue 137](../04-ISSUES/137-converging-a-machine-cut-off-its-own-guests/00-report.md) — the measurement that produced this
|
||||
- mesh-controller `internal/catalogue/filtering.go` — the constant whose own comment named this gap
|
||||
@@ -205,7 +205,7 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0091** — [A mount is declared, and there are three things it can be](0091-a-mount-is-declared-three-ways.md)
|
||||
- **0099** — [A step that runs once names what it reads, and runs again when it changed](0099-a-step-that-runs-once-names-what-it-reads.md)
|
||||
- **0110** — [A seat is held by one assignment, from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)
|
||||
- **0112** — [A module definition names no node, no mesh and no path: everything it needs is a requirement the mesh resolves](0112-a-module-definition-names-no-node-mesh-or-path.md) *(proposed)*
|
||||
- **0112** — [A module definition names no node, no mesh and no path: everything it needs is a requirement the mesh resolves](0112-a-module-definition-names-no-node-mesh-or-path.md)
|
||||
- **0113** — [The vault makes every shared secret, a provider makes resources and data, and the mesh carries both](0113-the-vault-makes-every-secret.md) *(proposed)*
|
||||
- **0114** — [A credential two parties hold rotates over two credentials; one a single party holds rotates in place, staged; and retiring a credential never removes what it reached](0114-a-shared-credential-rotates-over-two-credentials.md) *(proposed)*
|
||||
- **0115** — [One assignment of a module per node: the module's name is the assignment's identity](0115-one-assignment-of-a-module-per-node.md) *(proposed)*
|
||||
@@ -217,6 +217,7 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0133** — [A module owns its migrations, and the mesh owns when they run](0133-a-module-owns-its-migrations-and-the-mesh-owns-when-they-run.md) *(superseded)*
|
||||
- **0135** — [A module version prepares its state before it runs](0135-a-module-version-prepares-its-state-before-it-runs.md)
|
||||
- **0136** — [A step gates its module, not the machine](0136-a-step-gates-its-module-not-the-machine.md)
|
||||
- **0137** — [A machine says which networks it routes](0137-a-machine-says-which-networks-it-routes.md)
|
||||
|
||||
### How it is built
|
||||
|
||||
|
||||
@@ -1,11 +1,20 @@
|
||||
---
|
||||
layer: to-be
|
||||
status: proposed
|
||||
code: []
|
||||
status: in-progress
|
||||
code:
|
||||
- mesh-controller internal/catalogue/declaration.go
|
||||
- mesh-controller internal/catalogue/manifest.go
|
||||
- mesh-controller internal/link/serve.go
|
||||
- mesh-controller internal/link/bus.go
|
||||
- mesh-controller internal/broker/nats.go
|
||||
- mesh-controller internal/inventory/nodes.go
|
||||
- mesh-host internal/apply/apply.go
|
||||
- mesh-tools src/main.ts
|
||||
- mesh-catalog modules/mesh-catalog
|
||||
updated: 2026-09-28
|
||||
decisions:
|
||||
- 02-DECISIONS/0126-a-module-declares-its-own-seats.md
|
||||
- 02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md
|
||||
- 02-DECISIONS/0131-everything-on-the-mesh-speaks-to-the-broker-seat.md
|
||||
- 02-DECISIONS/0128-the-mesh-bus-is-required-not-ambient.md
|
||||
- 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
- 02-DECISIONS/0041-events-are-a-relationship.md
|
||||
@@ -261,8 +270,8 @@ queue.
|
||||
and publishes it last-per-subject. A node that was away gets exactly the current one, never a
|
||||
queue of superseded ones, and a replayed older one is refused by sequence.
|
||||
|
||||
**A version prepares its state before it runs.** A module version may declare an entrypoint that brings
|
||||
its state to the shape that version needs — the same vocabulary as the entrypoints it declares for its
|
||||
**A version prepares its state before it runs.** *Built 2026-09-28.* A module version may declare an
|
||||
entrypoint that brings its state to the shape that version needs — the same vocabulary as the entrypoints it declares for its
|
||||
tools and its provisioner, and nothing about how a machine runs it. The mesh runs that entrypoint as it
|
||||
runs the module's own code, to completion, in the module's own context, and a version whose preparation
|
||||
did not succeed does not run: the step gates that module and nothing else on the machine
|
||||
@@ -277,7 +286,7 @@ mesh provisions is per consumer and preparation is too. No level to choose, and
|
||||
not to an address it was given at genesis. Held and retried while the store restarts
|
||||
([ADR 0083](../../02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md)).
|
||||
|
||||
**And the mesh says what it applied.** A report is control traffic only the control plane reads, so the
|
||||
**And the mesh says what it applied.** *Built 2026-09-28.* A report is control traffic only the control plane reads, so the
|
||||
chain above went dark at the moment it touched a machine: nothing said which version a machine now runs,
|
||||
or that it refused to. The control plane states those as facts under its own seat's namespace, when what
|
||||
a machine runs changes rather than on every convergence pass, and anything that cares subscribes the way
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
---
|
||||
status: open
|
||||
opened: 2026-09-28
|
||||
located-in: [mesh-catalog, mesh-controller internal/catalogue]
|
||||
fixed-by:
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 134 — A definition may still name the mesh, and the check that would say so does not exist
|
||||
|
||||
## What was observed
|
||||
|
||||
[ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) says a module
|
||||
definition names no node, no mesh and no host path, and states how that is checked:
|
||||
|
||||
> A catalogue test finds no domain name in any definition value.
|
||||
|
||||
There is no such test. Run by hand on 2026-09-28, across the 72 manifests in the catalogue, the
|
||||
question it asks has 15 answers. They are not all the same kind of thing, and the difference matters
|
||||
more than the count:
|
||||
|
||||
**Values the mesh acts on** — seven:
|
||||
|
||||
| module | where | what it names |
|
||||
|---|---|---|
|
||||
| keycloak | `env.KC_HOSTNAME` | this installation's public name for itself |
|
||||
| minio | `env.MINIO_BROWSER_REDIRECT_URL` | the same, for its console |
|
||||
| invoicing | a resource's `image` | a named registry rather than the mesh's artifact store |
|
||||
| builder | `build.artifacts[].context.repository` | the forge, by URL |
|
||||
| route-proxy | `build.artifacts[].context.repository` | the forge, by URL |
|
||||
| route-adapter | a resource's `content` | a proxy's dynamic configuration |
|
||||
| novox.be | `module` | the module is named after the domain it serves |
|
||||
|
||||
**Prose** — eight, in `listens[].why`: de-spiegel, mailu, n8n, only-office, photos, photos-eef,
|
||||
photos-filip, portainer. Each explains what a port is for and mentions the public name it is reached
|
||||
by. Nothing reads these; a check written as a string search would report them, and reporting them as
|
||||
violations of the same rule would be wrong.
|
||||
|
||||
## Why it matters beyond this instance
|
||||
|
||||
**An unenforced rule is indistinguishable from a wrong one, and costs more, because people believe
|
||||
it.** The record says the mesh is name-agnostic, four design documents rest on that, and a reader
|
||||
checking whether it holds finds that it does not — in the places that matter most. The two forge URLs
|
||||
are what a build reaches into for its source; the two hostnames are what a service tells a browser
|
||||
about itself.
|
||||
|
||||
**It is the difference between a mesh and this mesh.** A definition carrying `novox.be` is a
|
||||
definition that can only be installed here. The whole point of the rule is that the same catalogue
|
||||
raises a different mesh with a different name, and today seven modules would need editing to do it.
|
||||
|
||||
**And the shape of the fix is not the same for each.** A public name is an operator's choice about an
|
||||
assignment, which ADR 0112 already provides for; a forge URL should be a path on the git seat
|
||||
([ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md)); an image from a named
|
||||
registry is a question about the artifact store, not about naming. Counting them together would hide
|
||||
that.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Does a domain in a `why` string break the rule? It is documentation the mesh never reads, and a
|
||||
check that cannot tell the two apart will either pass things it should catch or fail things nobody
|
||||
should change.
|
||||
- Where does a service's public name live, concretely — a setting on the assignment, or a fact the
|
||||
mesh composes from the node's domain? ADR 0112 says a requirement the mesh resolves; the two
|
||||
hostnames above are the first real cases.
|
||||
- Should a build context name a repository on the git seat rather than by URL, and if so, what does
|
||||
that mean for a context in *another* mesh's forge?
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
status: resolved
|
||||
opened: 2026-09-28
|
||||
located-in: [mesh-host internal/apply]
|
||||
fixed-by: mesh-host — a container's mesh names are part of the spec digest the host compares, sorted so the digest does not move for a reordering. A container whose names moved is now recreated like a container whose image moved, and the test fails against the previous behaviour.
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 135 — A container's mesh names are not compared, so a moved address is never noticed
|
||||
|
||||
## What was observed
|
||||
|
||||
One container on this mesh had been restarting every thirty seconds for five days — 2286 times — and
|
||||
the mesh reported the machine as doing what it was told.
|
||||
|
||||
Its logs said its database connected and then a query timed out. The database was reachable: the same
|
||||
query from the same network, with the same credential, answered in milliseconds. What differed was the
|
||||
name. Inside that container, `novox.internal` resolved to `10.42.0.1`; in every other container on the
|
||||
machine it resolved to `10.10.0.1`. The mesh's overlay range had moved, and this container still held
|
||||
the old one:
|
||||
|
||||
```
|
||||
umami created 2026-09-23 novox.internal:10.42.0.1
|
||||
mesh-catalog created today novox.internal:10.10.0.1
|
||||
```
|
||||
|
||||
A container resolves other machines and public names through the entries the mesh gives it when it is
|
||||
created, and nothing re-reads them afterwards. The host compares a container against what was declared
|
||||
by a digest of its spec — image, name, environment, ports, volumes, arguments, resolver, address, and
|
||||
what it reads — and **the mesh's names were not in it**. So this container matched what was declared,
|
||||
was left alone, and kept an address that had not existed for five days.
|
||||
|
||||
Forty-eight other containers had current names. Not because anything corrected them: each had been
|
||||
recreated for some other reason — a new image, a changed file — and picked up the current roster on the
|
||||
way. This one's image is an upstream release that had not moved, and nothing else about it changed, so
|
||||
nothing ever recreated it.
|
||||
|
||||
## Why it matters beyond this instance
|
||||
|
||||
**It is the exact fault [issue 045](../045-a-container-keeps-the-values-it-started-with/00-report.md)
|
||||
named, in the one field that was left out.** That issue is why the digest carries what a container
|
||||
reads: "a container whose configuration had since been rewritten compared equal and was left alone —
|
||||
running values the machine no longer holds, while every check reported success." The same sentence
|
||||
describes this, with *names* in place of *files*.
|
||||
|
||||
**The failure is invisible in exactly the way that matters.** The container runs, so the machine
|
||||
reports it applied. It restarts, but a restarting container is a normal sight during an upgrade. The
|
||||
only account of the fault is inside the container's own log, in the words of the application rather
|
||||
than of the mesh — and what it says is that a query timed out, which points at the database.
|
||||
|
||||
**And it is most likely to bite what changes least.** Every container that is rebuilt often repairs
|
||||
itself by accident. The victim is the module whose image is stable — which is to say, the module that
|
||||
was working fine.
|
||||
|
||||
## What was done
|
||||
|
||||
The mesh's names are part of the digest, sorted so the digest does not move for a reordering nobody
|
||||
made. A container whose names moved is now recreated exactly as one whose image moved.
|
||||
|
||||
The first apply after this recreates every container that carries mesh names — one restart each,
|
||||
already the price the mesh pays for any image update — because their recorded digests predate the
|
||||
field.
|
||||
|
||||
## What is still true
|
||||
|
||||
The mesh gives a container its names at creation and has no way to change them in place. That is the
|
||||
container runtime's shape, not a choice; the answer is to recreate, which is what this does. A module
|
||||
that would rather re-read a roster from a file can already ask for one as a fact
|
||||
([ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md)) and restart on
|
||||
it.
|
||||
@@ -0,0 +1,92 @@
|
||||
---
|
||||
status: resolved
|
||||
opened: 2026-09-28
|
||||
located-in: [mesh-catalog modules/fail2ban]
|
||||
fixed-by: mesh-catalog — the intrusion-prevention module bans through an action it ships itself, already in use on every machine, instead of naming a firewall front-end two of them do not have. The instance is closed; the class in "What is still true" is not.
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 136 — A module may name a program the machine does not have, and everything reports success
|
||||
|
||||
## What was observed
|
||||
|
||||
Two machines were given the intrusion-prevention module on 2026-09-28. Both refused to start it:
|
||||
|
||||
```
|
||||
ERROR Failed during configuration: Have not found any log file for 'recidive' jail.
|
||||
ERROR Async configuration of server failed
|
||||
fail2ban.service: Main process exited, code=exited, status=255/EXCEPTION
|
||||
```
|
||||
|
||||
The jail that bans whoever keeps coming back reads the service's *own* log, and the service checks
|
||||
every jail's log file while it configures itself — before it has created that log. The module
|
||||
declared the jail and shipped the rotation for that log, and never declared the log. On the two
|
||||
machines where it had run for years the file was simply there, so nothing had ever noticed.
|
||||
|
||||
That failure was loud. Fixing it uncovered a second one in the same module that is not.
|
||||
|
||||
The module's defaults named `ufw` as the way to ban an address. Two of these four machines have no
|
||||
`ufw` — they filter with nftables — and nothing checks that until an address is banned. Asked to ban
|
||||
a documentation address on such a machine, the service accepted the instruction, counted it, ran the
|
||||
command, and wrote this to a log nobody reads:
|
||||
|
||||
```
|
||||
ERROR ... -- stderr: '/bin/sh: line 5: ufw: command not found'
|
||||
ERROR ... -- returned 127
|
||||
ERROR Failed to execute ban jail 'sshd' action 'ufw' ... Error banning 192.0.2.99
|
||||
```
|
||||
|
||||
No rule existed afterwards. Throughout, the unit was `active`, the module was applied, and the
|
||||
machine's report said so. **A machine had been added to the mesh's intrusion prevention, reported as
|
||||
protected, and was banning nobody.**
|
||||
|
||||
## Why it matters beyond this instance
|
||||
|
||||
**The two faults are the same mistake with opposite symptoms.** Both are the module assuming
|
||||
something about the machine — a file that happens to exist, a program that happens to be installed.
|
||||
One stopped the service, which anybody notices. The other left it running and empty, which nobody
|
||||
does. A mesh that only catches the loud one is a mesh whose coverage is unknown.
|
||||
|
||||
**"The unit is running" was taken for "the module is doing its job".** That is the only health a
|
||||
service resource has. It is the right answer for most modules and it is silent for any module whose
|
||||
work happens later, on an event — a ban, a renewal, a backup, a notification. The report cannot
|
||||
distinguish "protecting this machine" from "installed and inert".
|
||||
|
||||
**And it is exactly the naming rule, one level down.**
|
||||
[ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) says a
|
||||
definition names no node, no mesh and no host path, because the same definition has to raise a
|
||||
different mesh. `ufw` is not a node name, but it is the same class of assumption: a value the module
|
||||
cannot know, true on some machines and false on others, written as though it were a constant. The
|
||||
module already knew how to do better a few lines away — the mesh's own address range is named there
|
||||
as something the machine fills in.
|
||||
|
||||
## What was done
|
||||
|
||||
The module declares the log its own jail reads, created once and never touched again, since what
|
||||
grows in it is the service's and the rotation the module already ships is what keeps it small. And
|
||||
it bans through the action it ships itself, which every machine here can run, which was already in
|
||||
use by the other jail on all four, and which covers a container's published port as well as the
|
||||
host's own.
|
||||
|
||||
All four machines now run it, with both jails, and a ban lands on each — verified by banning and
|
||||
unbanning a documentation address on every one.
|
||||
|
||||
## What is still true
|
||||
|
||||
**Nothing would have caught either fault before it shipped.** The control plane reads a manifest, not
|
||||
a machine; `ufw` and `/var/log/…` are strings in a file it has no way to evaluate. The host could in
|
||||
principle be asked whether a declared program exists, but no resource says "this file names a command
|
||||
that must be there", so there is nothing to check.
|
||||
|
||||
**Two machines' bans from before this are stale rules in the old front-end**, which the service no
|
||||
longer knows about and will never lift. They reject two addresses for ever. Harmless, and a reminder
|
||||
that changing how a module enforces something leaves what it already enforced behind.
|
||||
|
||||
## Open questions
|
||||
|
||||
- What does a service resource's health mean for a module whose work is event-driven? A unit being
|
||||
active is the weakest claim available, and four of this mesh's modules are of that kind.
|
||||
- Should a declaration be able to say that a resource depends on a program, so the machine can refuse
|
||||
what it cannot carry out rather than reporting success?
|
||||
- Where should the packet filter a module bans through come from — the module's own choice, as now,
|
||||
or the seat that owns the machine's filtering?
|
||||
@@ -0,0 +1,74 @@
|
||||
---
|
||||
status: resolved
|
||||
opened: 2026-09-28
|
||||
located-in: [mesh-controller internal/catalogue]
|
||||
fixed-by: mesh-controller — a machine says which networks it routes and the derived filter forwards them, their guests keeping address and name service ([ADR 0137](../../02-DECISIONS/0137-a-machine-says-which-networks-it-routes.md)).
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 137 — Converging a machine cut off its own guests, and nothing said so
|
||||
|
||||
## What was observed
|
||||
|
||||
A workstation was flipped from adopted to converged, so the mesh's derived filter replaced what was
|
||||
there. The flip reported success, the machine reported that it had applied its declaration, and every
|
||||
surface of the mesh read green.
|
||||
|
||||
A container on one of that machine's networks could no longer reach anything:
|
||||
|
||||
```
|
||||
192.168.64.2/20
|
||||
OUTBOUND BLOCKED
|
||||
```
|
||||
|
||||
Five of the machine's container networks were affected, and every network its test beds create. The
|
||||
reason is in the filter's forward chain, which denies by default and then allows two ranges:
|
||||
|
||||
```
|
||||
ip saddr 172.16.0.0/12 accept # the container runtime's bridge networks
|
||||
ip saddr 192.168.128.0/17 accept # the networks its compose files are given
|
||||
```
|
||||
|
||||
Those two are constants in the controller. The machine's guests were allocated from neither: its
|
||||
compose networks from other parts of `192.168/16`, and each test bed a fresh `10.x/24`. So the rules
|
||||
were correct for a machine whose runtime was left at its defaults, and a guess on this one.
|
||||
|
||||
Two further things were closed by the same flip, and for the same reason nobody saw them: a guest asks
|
||||
its host for an address over DHCP and for names over DNS, both of which arrive at the input chain,
|
||||
where no module had declared them.
|
||||
|
||||
## Why it matters beyond this instance
|
||||
|
||||
**The preview could not have warned.** It lists what the machine reported as *listening*, and says so
|
||||
honestly: it ends with a line that traffic the machine routes is "not previewed". What it did not say
|
||||
is that routing was about to be denied by default, or which ranges would survive. An operator reading
|
||||
a 350-line preview approves what it shows.
|
||||
|
||||
**It is the second time today that a constant stood in for something the mesh cannot know.** The
|
||||
intrusion-prevention module named a firewall front-end two machines do not have
|
||||
([issue 136](../136-a-module-may-name-a-program-the-machine-does-not-have/00-report.md)), and the
|
||||
filter names the address ranges one runtime happens to use. Both were true where they were written and
|
||||
silently false elsewhere.
|
||||
|
||||
**And the code already knew.** The comment above those two lines says a machine configured otherwise
|
||||
"needs this to say so — which is a thing the mesh cannot derive and a reason this list is named here
|
||||
rather than computed". The gap was documented at the point where it was introduced, and the way to say
|
||||
it was never built. A comment naming a missing mechanism is a rule that is not enforced.
|
||||
|
||||
## What was done
|
||||
|
||||
A machine says which networks it routes; the filter forwards them and admits their guests' address and
|
||||
name service. Added to the runtime's defaults rather than replacing them, so a machine that names one
|
||||
range keeps the others. Node-level, because the machine routes them and the module that loads the
|
||||
filter may be replaced. The converge preview now says what a machine routes, and what it will keep
|
||||
forwarding if it says nothing.
|
||||
|
||||
## What is still true
|
||||
|
||||
**The flip is still the moment a machine's unmanaged services close.** That is what converging means
|
||||
and the preview names each one. This issue is not about the ports that were meant to close; it is about
|
||||
the ones nothing could name.
|
||||
|
||||
**Egress is still not previewed per network.** The preview says which ranges will be forwarded, not
|
||||
which of the machine's guests sit inside them. Deriving that would need the machine to report its
|
||||
bridges, and a bed's bridge does not exist until the bed runs.
|
||||
Reference in New Issue
Block a user