Compare commits
23
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
116b2d1793 | ||
|
|
68a14493c9 | ||
|
|
98eb3aa76f | ||
|
|
91bbe648a8 | ||
|
|
0e7b85f184 | ||
|
|
dac49de6e7 | ||
|
|
0d9208dbbf | ||
|
|
bf0ee7cb25 | ||
|
|
4567e13071 | ||
|
|
aa5d9f1045 | ||
|
|
79642251a1 | ||
|
|
331cb94c6e | ||
|
|
78351560f7 | ||
|
|
62cc2f89c7 | ||
|
|
426f741ad0 | ||
|
|
9bed54d3be | ||
|
|
413daf8ad5 | ||
|
|
5c993c09b7 | ||
|
|
7c3be48db2 | ||
|
|
b665d06701 | ||
|
|
1bd13446d4 | ||
|
|
14adaafa53 | ||
|
|
8c231102f8 |
@@ -131,6 +131,22 @@ def main():
|
||||
else:
|
||||
seen[number] = name
|
||||
|
||||
# And decision records, which 155's fix left out: on 2026-10-02 two ADRs numbered 0169 landed
|
||||
# on main from two sessions within the hour, and every check passed.
|
||||
seen_records = {}
|
||||
for path in sorted(glob.glob(os.path.join(ROOT, "02-DECISIONS", "[0-9]*.md"))):
|
||||
name = os.path.basename(path)
|
||||
number = name.split("-", 1)[0]
|
||||
if not number.isdigit():
|
||||
continue
|
||||
if number in seen_records:
|
||||
bad(os.path.join("02-DECISIONS", name),
|
||||
"is numbered %s, and so is %s -- a record's number is how it is cited. Take the next "
|
||||
"free number across main AND every open pull request; the branch that lands last "
|
||||
"renumbers" % (number, seen_records[number]))
|
||||
else:
|
||||
seen_records[number] = name
|
||||
|
||||
for path in sorted(glob.glob(os.path.join(ROOT, "04-ISSUES", "*", "00-report.md"))):
|
||||
front = frontmatter(path)
|
||||
if front is None:
|
||||
|
||||
@@ -113,6 +113,22 @@ That is a difference a take shows, not a fault, and is decided when it bites.
|
||||
| `node show` lists filters with owners; `status` names a converged machine something else filters and is not well; the preview lists filters and fates | controller tests over a fixture report |
|
||||
| Live | the home server's record names the predecessor's chain in the runtime's user chain as *other*; `status` names the machine; after the operator removes the chain, the next report drops it and `status` is well |
|
||||
|
||||
## Built and proven live, 2026-10-02
|
||||
|
||||
> **Progressive insight — 2026-10-02.** The decision stands; these are the facts of its building.
|
||||
|
||||
Built in mesh-host 67 (every refusing table and legacy chain classified with an owner, reported with
|
||||
every apply; the found firewall retired on every converged apply, *found inactive* kept apart from
|
||||
*disabled by the mesh*, a skipped step said) and mesh-controller 211 (kept per node, shown on `node
|
||||
show`, named by `status` and not well, previewed with fates). The live row was read at 10:10Z: the home
|
||||
server's record named the predecessor's chain in the legacy filter's user chain as *other*, beside two
|
||||
chains a retired front end left in the IPv6 legacy filter; the control node's record named the same two
|
||||
leftovers; the laptop and the workstation read *the mesh alone*; `status` named both machines. The five
|
||||
rule sets were removed at 12:46Z through the packet filter seat's `remove` verb
|
||||
([ADR 0170](0170-the-firewall-seat-serves-its-verbs.md)), and the next report read *the mesh alone* on
|
||||
all four machines. The control node's record still says the mesh retired its front end, which issue 143
|
||||
records as a hand's work: the host trusts its record, and from this build on the distinction is kept.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), [ADR 0103](0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md), [ADR 0140](0140-the-filter-constrains-what-arrives-from-outside.md), [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md), [ADR 0163](0163-taking-a-module-over-is-a-comparison.md)
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-10-02
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
||||
---
|
||||
|
||||
# 169. A machine joins through the tunnel, and the bus is never public
|
||||
|
||||
## Context
|
||||
|
||||
The bus is the one channel every machine depends on: enrolment, every declaration, every tool. The
|
||||
`nats` module declares it reachable from the mesh only. The controller still opens it to the whole
|
||||
internet on the machine that runs it, as a *foundation* port that no module declares and nothing may
|
||||
close ([issue 051](../04-ISSUES/051-the-mesh-cannot-update-what-it-depends-on/00-report.md)).
|
||||
The reason is joining. [ADR 0004](0004-a-node-and-how-it-joins.md) has a new machine enrol over the bus
|
||||
**before** it has a tunnel. [ADR 0007](0007-connectivity.md) states it as a requirement: the node
|
||||
running the broker must be reachable from wherever nodes are, at a stable address.
|
||||
|
||||
So the bus listens on the internet permanently, for an event that happens a few times a year. A
|
||||
sweep of every machine on 2026-10-02 found no client using the public path. Every connection arrives
|
||||
over the tunnel or from the machine itself. The join token does not use it either: it carries the
|
||||
controller's configured broker address, a mesh name with the old broker's port.
|
||||
|
||||
ADR 0004 already says what a joining machine needs: *an identity, an address, and one peer to reach*.
|
||||
The tunnel can be that peer, if the hub knows the new machine's key before the machine first knocks.
|
||||
WireGuard answers nothing to a key it does not know, which is why the tunnel's own port is safe to
|
||||
leave open where the bus's is not.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Keep the bus public.** It is authenticated and encrypted, but every exposure of it, and of the
|
||||
server behind it, is exposure of the one thing everything depends on.
|
||||
2. **Open the bus publicly only while a join token is live.** Small, and the hub is open only during a
|
||||
join window. But the window is real, the rule is about time rather than about who may reach the
|
||||
bus, and the opening and closing are pushes that can fail between them.
|
||||
3. **The controller makes the new machine's tunnel key and puts it in the token.** One step for the
|
||||
operator, but the private half leaves a machine it does not belong to. ADR 0004 refuses that for
|
||||
every key a node holds.
|
||||
4. **The machine makes its key first, and the token is issued for it.** The machine prints the public
|
||||
half of its tunnel key. The operator issues the token for that key. The controller gives the
|
||||
machine its address and adds it as a peer on the hub. The token carries the hub's tunnel endpoint
|
||||
and key, the machine's address, and the bus's address on the private network. The machine brings
|
||||
up its tunnel and enrols over it.
|
||||
|
||||
## Decision
|
||||
|
||||
**Option 4.**
|
||||
|
||||
- **A machine makes its own tunnel key before it has a token**, and prints the public half. The private
|
||||
half never leaves it, as ADR 0004 says of every key a node holds.
|
||||
- **A token is issued for a tunnel key.** Issuing it assigns the machine's address on the private
|
||||
network, records the key, and makes the machine a peer of the hub. The hub is sent that before the
|
||||
token is shown, so the tunnel answers the moment the machine first uses it.
|
||||
- **The token carries the one peer.** It adds the hub's tunnel endpoint and public key and the
|
||||
machine's own address. **Where** becomes the bus's address on the private network, which needs no
|
||||
name resolution.
|
||||
- **The machine joins through the tunnel.** It brings the tunnel up from the token alone, then enrols
|
||||
over it exactly as before. The enrolment checks that the key it is offered is the one the token was
|
||||
issued for.
|
||||
- **The bus is never public.** It is no longer a foundation port. Its reach is what the `nats` module
|
||||
declares: the mesh. The tunnel's port stays open, as the one way in.
|
||||
|
||||
This changes three things earlier records say. ADR 0004's *where* is the bus's private address, and the
|
||||
token carries the peer. ADR 0007's requirement that the broker be reachable from wherever nodes are
|
||||
becomes: **the hub's tunnel is**. Issue 051's broker port stops being a foundation port.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Joining is two commands on the new machine, with the token issued between them. A token issued for
|
||||
the wrong key gives a tunnel that never answers, and the machine says so rather than timing out at
|
||||
the bus.
|
||||
- An unused token leaves a peer on the hub until it expires. Expiry removes it, the same way it voids
|
||||
the secret.
|
||||
- A machine already in the mesh is unaffected: it reaches the bus over its tunnel today.
|
||||
- The genesis machine, the first one, raises the bus on itself and needs no tunnel to reach it.
|
||||
|
||||
## How this is checked
|
||||
|
||||
| Rule | Checked by |
|
||||
|---|---|
|
||||
| A token is refused without a tunnel key, and carries the hub's peer and the machine's address | a controller test |
|
||||
| Issuing a token makes the machine a peer of the hub before the token is shown | a controller test over the hub's composed tunnel |
|
||||
| An expired, unused token's peer is gone from the hub | a controller test |
|
||||
| Enrolment refuses a tunnel key other than the one the token was issued for | a controller test |
|
||||
| No machine's filter opens the bus to anywhere | a controller test over the composed filter, and the live sweep from outside the mesh |
|
||||
| A new machine joins from outside the hub's network with the bus closed to it | the lab, then by hand |
|
||||
@@ -0,0 +1,103 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-10-02
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md
|
||||
---
|
||||
|
||||
# 170. The firewall seat serves its verbs, and a foreign rule set is removed through one of them
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0168](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md) made the mesh say truthfully
|
||||
what filters a converged machine, and left the removal of what it did not write to the operator's
|
||||
hand. The first time that hand was needed — two machines, five rule sets a predecessor and a
|
||||
retired front end had left — there was no mesh way to lend it: the packet filter is a seat
|
||||
([ADR 0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md)), a seat's
|
||||
holder serves its verbs ([ADR 0154](0154-the-meshs-own-verbs-are-the-controller-seats-tools.md),
|
||||
[ADR 0159](0159-a-tool-call-names-the-machine-and-a-holder-serves-its-seats-verbs.md)), and the
|
||||
firewall seat declared none. The only remaining path was a shell on the machine, which is the path
|
||||
the mesh exists to replace, and which the operator's own tooling rightly refused to an agent.
|
||||
|
||||
A seat's verbs are the contract every holder implements, whatever filter it speaks. What a person
|
||||
asks a machine's packet filter is the same whether nftables, a front end or a legacy filter answers:
|
||||
what are the rules, reload the mesh's own, remove this thing the mesh did not write. What differs by
|
||||
filter is the holder's own business and may be its own tools beside the seat's.
|
||||
|
||||
## Decision
|
||||
|
||||
**1. The `node-packet-filter` seat serves three verbs**, and a module that claims it serves all
|
||||
three or is refused the claim, as with every seat:
|
||||
|
||||
- `rules` — the packet filter as the machine enforces it now: the nftables ruleset, and the legacy
|
||||
filter's listings where that tool exists; narrowed to one table or chain when asked. Read-only.
|
||||
- `reload` — load the mesh's own filter again from the file the mesh writes, and answer with the
|
||||
mesh's table as loaded. The holder's own act on the holder's own rules.
|
||||
- `remove` — remove one rule set the mesh did not write, named exactly as the host reports it under
|
||||
ADR 0168 (`chain HAL-MESH-ONLY (iptables-legacy)`, `table ip6 filter, chain DOCKER-USER`), and
|
||||
answer with what was done. It refuses the mesh's own tables, the container runtime's own chains,
|
||||
a built-in chain other than the runtime's user chain, and any chain of a found firewall that is
|
||||
in force. The runtime's user chain is emptied back to its one return; another chain loses the
|
||||
jumps into it, is flushed and deleted; a table of the machine's own is deleted whole. Each is an
|
||||
operator's act, by name, on one thing the mesh reported — never a flush, never a rule the mesh
|
||||
itself marked.
|
||||
|
||||
**2. A holder may serve its own tools beside the seat's.** The nftables module keeps its reading of
|
||||
the mesh's table as its own tool, and a holder speaking a filter with specifics of its own may add
|
||||
tools for them; the seat's three are what every holder owes.
|
||||
|
||||
**3. A container may ask for a capability.** Serving `remove` and `reload` needs the machine's
|
||||
network namespace and the right to change its packet filter; a holder's runtime declares
|
||||
`capabilities: ["NET_ADMIN"]` on its container and runs on the machine's network. The host grants
|
||||
exactly the capabilities declared, names them in the container's spec so a change recreates it, and
|
||||
refuses a name that is not a capability's. A privileged container stays undeclarable.
|
||||
|
||||
**4. ADR 0168's "by hand" is read as "by the operator, through the seat".** Removing what the mesh
|
||||
reports as *other* is still the operator's act and is still never the mesh's own doing; the verb is
|
||||
how the act reaches the machine, recorded on the bus like every other, instead of a shell.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The seat's row gains the three verbs; a mesh that already runs widens its row at the next
|
||||
controller start. The nftables module claims them and gains a runtime — a tool server with the
|
||||
packet filter's tools in its image, on the machine's network, with `NET_ADMIN`.
|
||||
- The host's container vocabulary grows by `capabilities`; an older host refuses a declaration that
|
||||
carries it, so the host rolls before the module.
|
||||
- The two machines of this mesh that ADR 0168 found not filtered by the mesh alone are cleaned
|
||||
through `remove`, and read *the mesh alone* afterwards; `status` returns to well without a hand on
|
||||
either machine.
|
||||
|
||||
## How this is checked
|
||||
|
||||
| Rule | Checked by |
|
||||
|---|---|
|
||||
| The seat declares the three verbs; a claim that serves fewer is refused by name | the catalogue's seat tests |
|
||||
| `remove` refuses the mesh's tables, the runtime's chains, a built-in chain and an active front end's chains, and removes a user chain with its jumps, empties the user chain, deletes an own table | the module's tests over a fake command runner, with the shapes the host reported live |
|
||||
| A container's capabilities reach the runtime and its spec; an unknown name is refused | host tests |
|
||||
| Live | `node-packet-filter.remove@<node>` on the home server and the control node; `node show` reads *the mesh alone* on both; `status` is well |
|
||||
|
||||
## Built and proven live, 2026-10-02
|
||||
|
||||
> **Progressive insight — 2026-10-02.** The decision stands; these are the facts of its building.
|
||||
> Written as 0169 for three hours and renumbered to 0170: another record took 0169 on main first,
|
||||
> and the check that refuses a shared number covered issues only (now records too).
|
||||
|
||||
Built in mesh-host 68 (`capabilities` on a container), mesh-controller 212 (the seat's three verbs)
|
||||
and 213 (the filter file a module names under `filtering.into` counts as declared for a mount — the
|
||||
module's first build was refused without it), mesh-catalog 216 (the nftables module's runtime and
|
||||
verbs) and mesh-tools 27 (the console lists a node-scoped seat's verbs with their scope and carries the
|
||||
machine; before it, the verbs were live on four machines and unreachable from the console —
|
||||
[issue 199](../04-ISSUES/199-a-node-scoped-seats-verb-could-not-be-called-through-the-console/00-report.md)).
|
||||
Each machine's holder was issued its bus account with `mesh-controller.issue`, the broker node pushed
|
||||
first. At 12:46Z the five rule sets ADR 0168 had named were removed through
|
||||
`node-packet-filter.remove`, three on the home server and two on the control node, each answering
|
||||
with the commands it ran; the next report read *the mesh alone* on all four machines and `status`
|
||||
listed nothing under `filtered`. The live row is read. What it cost on the way is
|
||||
[issue 200](../04-ISSUES/200-the-controllers-answer-to-the-console-is-refused-by-the-bus/00-report.md).
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0168](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md), [ADR 0154](0154-the-meshs-own-verbs-are-the-controller-seats-tools.md), [ADR 0159](0159-a-tool-call-names-the-machine-and-a-holder-serves-its-seats-verbs.md), [ADR 0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md)
|
||||
- [Design 33 — The tools the mesh answers](../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md), [Design 08 — Connectivity](../03-DESIGN/01-to-be/08-connectivity.md)
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-10-02
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0016-the-lab.md
|
||||
---
|
||||
|
||||
# 172. The lab is a module, and runs a bed when the mesh asks
|
||||
|
||||
## Context
|
||||
|
||||
The lab raises virtual machines and runs the mesh on them, end to end, before a change reaches a real
|
||||
machine ([ADR 0016](0016-the-lab.md)). It runs on one machine of the mesh, the one with the
|
||||
virtualisation it needs. Until now the only way to start a bed there was to sign in to that machine and
|
||||
run the lab's command line by hand, with a dozen environment variables pointing at sibling checkouts.
|
||||
|
||||
Nothing in the mesh could ask for it. An agent working through the mesh's own tools could build,
|
||||
merge and push a change, and could not prove it in the lab first. The operator's direction on
|
||||
2026-10-02: work on another machine goes through a mesh tool, not a shell on it.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Keep the lab a command line on one machine.** Every run is a person, or an agent with a shell on
|
||||
that machine, outside the mesh.
|
||||
2. **The lab is a module.** Assigned to the machine that can run it, serving tools that run a bed
|
||||
against named branches and say how it went.
|
||||
|
||||
## Decision
|
||||
|
||||
**Option 2.**
|
||||
|
||||
- **A `lab` module, assigned where the lab can run**, serves five tools: whether this machine can run
|
||||
beds, run beds against a branch per repository, a run's state, its log, and stopping it.
|
||||
- **A run is the lab's own suite**, against fresh checkouts of the named branches from the mesh's forge,
|
||||
side by side as the lab expects them. It builds what the beds place from those checkouts, as the suite
|
||||
already does. It answers at once with an id, like a build: a bed takes minutes, and a call does not.
|
||||
- **Only branches on the forge are run**, never code handed to the tool. What a run tested is what the
|
||||
forge holds at the commit it names.
|
||||
- **The lab is reached over the mesh only.** Its tools travel the bus, and the module opens no port.
|
||||
- **No grant beyond the mesh's own.** Running a bed is root on the lab's machine, but anyone who can call
|
||||
the mesh's tools can already do worse. The operator's judgement on 2026-10-02.
|
||||
|
||||
## Consequences
|
||||
|
||||
- An agent proves a change in the lab through the mesh, the same way it builds and pushes one.
|
||||
- The lab's machine carries a module whose runtime holds the virtualisation's and the container
|
||||
runtime's sockets, and a toolchain to build the mesh with.
|
||||
- A run's checkouts are its own, so two runs never build from each other's tree. Old ones are removed
|
||||
when their run ends.
|
||||
|
||||
## How this is checked
|
||||
|
||||
| Rule | Checked by |
|
||||
|---|---|
|
||||
| A run checks out exactly the named branches, and reports the commits it tested | the module's tests over a forge fixture, and each run's answer |
|
||||
| A run answers at once, and its state and log follow it to the end | by hand, the first run |
|
||||
| The module opens no port | the composed filter of the lab's machine |
|
||||
@@ -179,6 +179,9 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0163** — [Taking a module over is a comparison: what it compares, what it refuses, and what it carries](0163-taking-a-module-over-is-a-comparison.md)
|
||||
- **0167** — [A membership carries what its module receives, and who the mesh is](0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)
|
||||
- **0168** — [A converged machine is filtered by the mesh alone, and the host says what else refuses](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md)
|
||||
- **0169** — [A machine joins through the tunnel, and the bus is never public](0169-a-machine-joins-through-the-tunnel-and-the-bus-is-never-public.md)
|
||||
- **0170** — [The firewall seat serves its verbs, and a foreign rule set is removed through one of them](0170-the-firewall-seat-serves-its-verbs.md)
|
||||
- **0172** — [The lab is a module, and runs a bed when the mesh asks](0172-the-lab-is-a-module-and-runs-a-bed-when-the-mesh-asks.md)
|
||||
|
||||
### Its tiers, from the bottom up
|
||||
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
---
|
||||
layer: to-be
|
||||
status: in-progress
|
||||
code: [mesh-lab]
|
||||
updated: 2026-09-11
|
||||
code: [mesh-lab, mesh-catalog modules/lab]
|
||||
updated: 2026-10-02
|
||||
decisions:
|
||||
- 02-DECISIONS/0172-the-lab-is-a-module-and-runs-a-bed-when-the-mesh-asks.md
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
- 02-DECISIONS/0010-delivery.md
|
||||
---
|
||||
@@ -119,7 +120,25 @@ In order, on a machine with nothing:
|
||||
|
||||
6. **Verification**, as above, before anything is raised.
|
||||
|
||||
## Open
|
||||
## The lab answers the mesh
|
||||
|
||||
*2026-10-02* ([ADR 0172](../../02-DECISIONS/0172-the-lab-is-a-module-and-runs-a-bed-when-the-mesh-asks.md)).
|
||||
Once installed, the lab is also a module: `lab`, assigned to the machine that passed `check`. Its
|
||||
tools run there and nowhere else:
|
||||
|
||||
| tool | does |
|
||||
|---|---|
|
||||
| `lab_check` | the lab's `check`, on this machine |
|
||||
| `lab_run` | fresh checkouts of the named branches from the forge, side by side, then the suite on the named beds; answers with an id |
|
||||
| `lab_status` | where a run is, and how it ended: the commits it tested, passed and failed |
|
||||
| `lab_log` | the run's output so far |
|
||||
| `lab_stop` | ends a run |
|
||||
|
||||
The runtime is a container holding the toolchain the suite builds with. It reaches the
|
||||
virtualisation daemon and the container runtime through their sockets on the machine, so what it
|
||||
raises is what a hand run raises. The prerequisites above stay installed by hand. The module uses
|
||||
them, and never installs them.
|
||||
|
||||
|
||||
- **Whether the lab's bootstrap may install packages at all**, given that the mesh's rules
|
||||
forbid installing by hand. The resolution is probably that the lab's bootstrap *is* the
|
||||
|
||||
@@ -9,6 +9,8 @@ code:
|
||||
- mesh-host internal/apply (the service that reflects a rule set)
|
||||
updated: 2026-10-02
|
||||
decisions:
|
||||
- 02-DECISIONS/0170-the-firewall-seat-serves-its-verbs.md
|
||||
- 02-DECISIONS/0169-a-machine-joins-through-the-tunnel-and-the-bus-is-never-public.md
|
||||
- 02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md
|
||||
- 02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md
|
||||
- 02-DECISIONS/0148-the-meshs-names-are-resolved-not-copied-into-containers.md
|
||||
@@ -173,6 +175,28 @@ the broker's node must be dialable by every node, at a stable address, and so mu
|
||||
reachable; on one network it does not. A mesh whose nodes are all behind NAT cannot be raised, and
|
||||
a broker node whose address moves invalidates every token issued for it.
|
||||
|
||||
*2026-10-02.* **The order changes at step 1: the tunnel comes first, from the token**
|
||||
([ADR 0169](../../02-DECISIONS/0169-a-machine-joins-through-the-tunnel-and-the-bus-is-never-public.md)).
|
||||
The circularity above is real, and it is broken differently. The overlay is configured by the mesh,
|
||||
except for the one peer a joining machine needs, and the token carries that peer. So the sequence
|
||||
becomes:
|
||||
|
||||
```
|
||||
0 the node has an underlay address the machine's own
|
||||
1 the node makes its tunnel key before any token; it prints the public half
|
||||
2 a token is issued for that key its address assigned, and the hub sent it as a peer
|
||||
3 the tunnel comes up to the hub from the token alone: the hub's endpoint and key, its address
|
||||
4 the node dials the bus OVER THE TUNNEL, at the bus's private address
|
||||
5 it proves itself, and is proved to enrolment, checking the key is the one the token named
|
||||
6 the rest of the overlay the whole peer set, delivered as files
|
||||
7 names, filtering, routes as before
|
||||
```
|
||||
|
||||
The link no longer stays on the underlay. The bus is reached over the tunnel by every machine,
|
||||
including one that is joining, so it is never opened to the internet. The precondition becomes: **the
|
||||
hub's tunnel must be dialable by every node, at a stable address.** That port answers nothing to a
|
||||
key it does not know.
|
||||
|
||||
**Whether the link should later move onto the overlay, with the underlay as fallback, is
|
||||
[open](../../02-DECISIONS/0007-connectivity.md).** It is a decision rather than a derivation: the
|
||||
gain is which network carries bytes, not what an attacker can reach, since the link is already
|
||||
@@ -743,6 +767,11 @@ tests over a fixture report check the recording, the preview's fates, the status
|
||||
predicate. Live: the home server's record names the predecessor's chain as *other* and `status`
|
||||
names the machine until the chain is removed by hand.
|
||||
|
||||
*2026-10-02, [ADR 0170](../../02-DECISIONS/0170-the-firewall-seat-serves-its-verbs.md):* removing what
|
||||
the host reports as *other* is reached through the packet filter seat's `remove` verb, an operator's act
|
||||
by name on the bus; the seat also serves `rules` and `reload`, and its holder's runtime declares the
|
||||
`NET_ADMIN` capability on the machine's network. See design 33.
|
||||
|
||||
## 5 — Certificates
|
||||
|
||||
**Two authorities, kept separate on purpose.**
|
||||
|
||||
@@ -1,8 +1,11 @@
|
||||
---
|
||||
layer: to-be
|
||||
status: proposed
|
||||
code: []
|
||||
updated: 2026-09-27
|
||||
status: in-progress
|
||||
code:
|
||||
- mesh-controller internal/inventory
|
||||
- mesh-controller internal/catalogue
|
||||
- mesh-controller cmd/mesh-controller
|
||||
updated: 2026-10-01
|
||||
decisions:
|
||||
- 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md
|
||||
- 02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md
|
||||
@@ -14,10 +17,10 @@ decisions:
|
||||
# 29 — A node has operator accounts, and the mesh owns what lives under a home
|
||||
|
||||
**The mesh models machines but not the people on them.** A node record holds its name, its
|
||||
address, its mode — and nothing about *who a person is* on it: `jochens` on novox, `ace` on ace,
|
||||
`jochen` on shanks and g14. That username is not incidental. It decides who a file under `~` is
|
||||
owned by, who a user service runs as, and — the case that surfaced this — which account `ssh
|
||||
<node>` logs in as. The predecessor knew it (its per-node `user:`, and the modules that wrote a
|
||||
address, its mode — and nothing about *who a person is* on it: one login name on the build node,
|
||||
another on the home-server, a third on both workstations. That username is not incidental. It
|
||||
decides who a file under `~` is owned by, who a user service runs as, and — the case that surfaced
|
||||
this — which account `ssh <node>` logs in as. The predecessor knew it (its per-node `user:`, and the modules that wrote a
|
||||
person's `~/.ssh/config`, `~/.zshrc`, `~/.config`); the mesh, taking those over, kept the machine
|
||||
facts and dropped the human one.
|
||||
|
||||
@@ -28,8 +31,9 @@ Several things are missing, and they are one idea.
|
||||
A node has one or more **operator accounts**: the human logins on it. At minimum a name; the
|
||||
mesh already knows the node and its address, so `<account>@<node>` is then a complete answer to
|
||||
"who am I, where." It is the mesh's to hold because everything below is derived from it, and
|
||||
because it is exactly the fact that was silently lost — `ssh ace` failed to `ace` because nothing
|
||||
in the mesh said ace's account is `ace`.
|
||||
because it is exactly the fact that was silently lost — `ssh home-server` logged in under the
|
||||
workstation's own name, because nothing in the mesh said the home-server's account is a different
|
||||
one.
|
||||
|
||||
## 2. A resource may live under a home, owned by its account
|
||||
|
||||
@@ -65,14 +69,14 @@ create `~/.ssh` at `0700`, chown it to the account, and own the files it places
|
||||
|
||||
**The boundary — and it is the reason this is safe:** `~/.ssh` is the one directory where a wrong
|
||||
declaration locks a person out of their own machine. So the mesh's *found-vs-owned* semantics
|
||||
([ADR 0126](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md),
|
||||
([ADR 0118](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md),
|
||||
adoption) apply *inside* the home directory. The mesh **owns** the directory and the files above; it
|
||||
**holds as found — never rewrites, never removes** — the operator's own contents: their **private
|
||||
keys** and their **personal drop-ins** (`config.d/personal`, the personal `Host` aliases a
|
||||
workstation carries, exactly as `hosts.local` is the home the mesh never rewrites for `/etc/hosts`).
|
||||
Reconcile removing an unassigned `config.d/mesh` is fine; the same logic aimed at `id_ed25519` or an
|
||||
operator's own `authorized_keys` entry is a lockout. This is the login-channel cousin of the rule
|
||||
[ADR 0125](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md) draws for the uplink and the sshd
|
||||
[ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md) draws for the uplink and the sshd
|
||||
module draws for the firewall: **the mesh must never be able to arrange the one failure that severs
|
||||
its own way back in.** The carve-out is not a convenience; it is that rule, in `~/.ssh`.
|
||||
|
||||
@@ -108,12 +112,12 @@ found-vs-owned boundary of §3 is exactly what guarantees nothing already there
|
||||
|
||||
None of this needs a node to discover the mesh, and none of it needs a control-plane module of its
|
||||
own. The ssh files are **roster facts**
|
||||
([ADR 0128](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md)): once the
|
||||
([ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md)): once the
|
||||
roster view carries a node's **host key** and its **account** beside its name and address, the
|
||||
`ssh-client` module ships a template for `known_hosts`, `config` and `authorized_keys`, and the
|
||||
controller renders each node's copy from the full roster and pushes it. The mesh owns the data; the
|
||||
module owns ssh's format; the control plane gains no ssh syntax. It is the same act as composing a
|
||||
peer list or `/etc/hosts` — which is why there is **no novox-only "mesh-ssh" module**: the
|
||||
peer list or `/etc/hosts` — which is why there is **no control-node-only "mesh-ssh" module**: the
|
||||
centralization is the controller's composition, not a module that runs somewhere. Only non-secret
|
||||
facts travel (names, addresses, accounts, host keys, the CA public key); the private key stays the
|
||||
operator's, placed as an operator-owned file, referenced by path.
|
||||
@@ -129,6 +133,44 @@ operator's, placed as an operator-owned file, referenced by path.
|
||||
They meet at the account and the CA, not at a bespoke module. The `sshd` server side already exists;
|
||||
the client/identity side and the CA are the open pieces.
|
||||
|
||||
## What has shipped, and what has not
|
||||
|
||||
*Recorded 2026-10-01 from the controller's main branch, not from intent.*
|
||||
|
||||
**Built (mesh-controller, merged 2026-09-27):**
|
||||
|
||||
- **§1, the account as a node fact.** A node record carries an operator account and, optionally,
|
||||
its home. Empty is a real state — a freshly enrolled or headless machine has no operator account
|
||||
known yet — and an empty home means *derive it* (the superuser's home for the superuser, the
|
||||
conventional per-user home otherwise), so the common case needs no entry. The controller's node
|
||||
command sets it. One account per node is what exists; "one or several" below is still open.
|
||||
- **§2, resources under a home.** The account and its home are offered as machine facts, and a
|
||||
resource's *path and owner* resolve placeholders exactly as its content does — so a module places
|
||||
a file under a person's home, owned by that person, naming neither. A roster file may say it lives
|
||||
under the home: it is rendered per node, placed under that node's account's home, chowned to the
|
||||
account, and a node with no account gets none.
|
||||
- **§5, the composed ssh config.** The roster rendering carries each node's account, so the
|
||||
`ssh-client` template can emit a `Host` block per node with the right login name. Composed
|
||||
end-to-end in the controller's tests.
|
||||
|
||||
**Written but not shipped:** the `ssh-client` catalogue module itself exists on a branch of the
|
||||
module repository; its pull request was closed with a hold until this design is deployed, and
|
||||
nothing has deployed it since. The predecessor's generator still writes every workstation's ssh
|
||||
client blocks today — which is where [issue 172](../../04-ISSUES/172-the-ssh-client-block-matches-one-spelling-of-a-machine/00-report.md)
|
||||
was found.
|
||||
|
||||
**Not built:** the SSH CA and certificates (§4), `known_hosts` and `authorized_keys` as roster files,
|
||||
the found-vs-owned boundary inside `~/.ssh` (§3 — the controller has no rule yet that refuses to
|
||||
rewrite a private key), adoption of existing keys, the ssh-agent as a user service, and user-scoped
|
||||
services in general. The host vocabulary still has no user-scope unit at all; a workstation's
|
||||
per-user daemons (a bar watchdog, a config reloader, an audio service masked per user) have no form
|
||||
the mesh can send.
|
||||
|
||||
**A gap this surfaced:** §1 shipped as code before it had a decision record. The account as a node
|
||||
fact, the home as a placement root, and what the mesh may and may not do under a home are each a
|
||||
decision this document names but no record states. They are the next records to write, before the
|
||||
family of §2 modules is built.
|
||||
|
||||
## Why now, and why not yet
|
||||
|
||||
**Why it matters:** when HAL retires, the generators that keep `~/.ssh`, shell config and the
|
||||
@@ -137,7 +179,7 @@ alias and its trust, and a fresh machine has no operator dotfiles at all — the
|
||||
service and leave the human unable to work on the box.
|
||||
|
||||
**Why not build it reflexively:** it is a real addition to the node model, the resource model, and
|
||||
the seat set, and must be gotten right. The mechanism half is now settled — ADR 0128 is what lets
|
||||
the seat set, and must be gotten right. The mechanism half is now settled — ADR 0120 is what lets
|
||||
the ssh files be templates with no control-plane format — so what remains to decide here is the
|
||||
model:
|
||||
|
||||
@@ -156,15 +198,18 @@ model:
|
||||
unnecessary, and forwarding an agent into a node exposes the operator's keys to that node's root —
|
||||
so prefer certificates and `ProxyJump` over forwarding.
|
||||
|
||||
**Not urgent, not blocking.** ssh and dotfiles work today because HAL's generators still run as the
|
||||
substrate. This becomes load-bearing in the node-by-node retirement phase, not before — which is the
|
||||
right time to build it, once the account and CA model are decided here.
|
||||
**Now load-bearing.** The migration of every node to the mesh is complete; what remains of the
|
||||
predecessor is exactly the user environment this design covers — ssh config, dotfiles, the desktop
|
||||
stack and the per-user services of the two workstations. Those generators are the last thing
|
||||
keeping the predecessor running, so the model questions above are no longer deferred: the account
|
||||
record, the home as a placement root, user-scoped services and the one-off steps a hook used to run
|
||||
each need a decision before the modules that replace the generators can be written.
|
||||
|
||||
## References
|
||||
|
||||
- The gap was found generating `~/.ssh/config` from the *HAL* registry (`hal/terminal`'s
|
||||
postConfigure hook), which the nox mesh has no equivalent for.
|
||||
- [ADR 0128](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md) — the roster
|
||||
- [ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md) — the roster
|
||||
fact mechanism that renders the ssh files, format owned by the module.
|
||||
- [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) — the
|
||||
system-path placement this mirrors for home paths.
|
||||
@@ -173,6 +218,6 @@ right time to build it, once the account and CA model are decided here.
|
||||
- [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md) — the CA key is a secret the
|
||||
vault makes; [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md)
|
||||
— short-lived certs as rotation.
|
||||
- [ADR 0125](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md),
|
||||
[ADR 0126](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md) —
|
||||
- [ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md),
|
||||
[ADR 0118](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md) —
|
||||
the never-sever-the-channel rule and the found-vs-owned semantics, applied here to `~/.ssh`.
|
||||
|
||||
@@ -2,8 +2,9 @@
|
||||
layer: to-be
|
||||
status: implemented
|
||||
code: [mesh-controller, mesh-tools]
|
||||
updated: 2026-10-01
|
||||
updated: 2026-10-02
|
||||
decisions:
|
||||
- 02-DECISIONS/0170-the-firewall-seat-serves-its-verbs.md
|
||||
- 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md
|
||||
- 02-DECISIONS/0159-a-tool-call-names-the-machine-and-a-holder-serves-its-seats-verbs.md
|
||||
- 02-DECISIONS/0154-the-meshs-own-verbs-are-the-controller-seats-tools.md
|
||||
@@ -169,6 +170,18 @@ either way.
|
||||
What stays as designed and not built: which verbs any *other* seat serves, and §3 for module-declared
|
||||
seats' schemas beyond the names their manifests already list.
|
||||
|
||||
## The firewall seat's verbs, 2026-10-02
|
||||
|
||||
[ADR 0170](../../02-DECISIONS/0170-the-firewall-seat-serves-its-verbs.md). The first node-scoped seat
|
||||
to carry verbs: `node-packet-filter` serves `rules` (the filter as the machine enforces it, nftables
|
||||
and legacy), `reload` (the mesh's own filter from its file) and `remove` (one rule set the mesh did
|
||||
not write, named as the host reports it under ADR 0168; refusing the mesh's tables, the runtime's
|
||||
own chains, a built-in chain and an active found firewall's). Every holder serves all three; the
|
||||
nftables module does so from a runtime on the machine's network with `NET_ADMIN`, which is the first
|
||||
container to declare a capability. Removing a predecessor's rule set is an operator's act reached
|
||||
through the seat, recorded on the bus, instead of a shell on the machine. *How it is checked:* ADR
|
||||
0169's table.
|
||||
|
||||
## What this does not settle
|
||||
|
||||
- Which verbs each seat should serve. That is a decision per seat, and the reason to do it slowly: a
|
||||
|
||||
@@ -38,7 +38,7 @@ document is written and this one's status becomes `implemented`.
|
||||
| [`26-the-seats.md`](26-the-seats.md) | **Proposed.** What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) |
|
||||
| [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md), [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)) |
|
||||
| [`28-building-the-bus.md`](28-building-the-bus.md) | **Proposed.** The five steps of the bus work in the order their dependencies allow, each ending at a bed — with the surface measured, so no step's size is a guess | [ADR 0116](../../02-DECISIONS/0116-the-bus-is-built-in-five-steps.md), [ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md), [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md) |
|
||||
| [`29-a-node-has-operator-accounts.md`](29-a-node-has-operator-accounts.md) | **Proposed.** The mesh models machines but not the humans on them: a node gains operator accounts, and a resource may live under a home owned by its account — what would own ~/.ssh, dotfiles and ~/.config when HAL retires | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md) |
|
||||
| [`29-a-node-has-operator-accounts.md`](29-a-node-has-operator-accounts.md) | **In progress.** A node has an operator account and a resource may live under its home — built in the controller; the ssh-client module, the SSH CA, the `~/.ssh` boundary and user-scoped services are not. The account fact still wants its decision record | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md) |
|
||||
|
||||
| [`32-what-a-module-declares.md`](32-what-a-module-declares.md) | **Proposed.** What a module declares and what the bus derives from it: three namespaces, subjects from local names, queues never declared, the five relationships, and the build-publish-deploy lifecycle on one bus | [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md), [ADR 0127](../../02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md), superseded by [ADR 0131](../../02-DECISIONS/0131-everything-on-the-mesh-speaks-to-the-broker-seat.md) (superseding [ADR 0125](../../02-DECISIONS/0125-the-bus-is-the-only-broker.md)), [ADR 0041](../../02-DECISIONS/0041-events-are-a-relationship.md) |
|
||||
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-29
|
||||
located-in:
|
||||
- mesh-host internal/apply/opening.go (retireFirewall)
|
||||
- mesh-host internal/apply/apply.go (the condition it is called under)
|
||||
fixed-by:
|
||||
fixed-by: mesh-host 67 (retire on every converged apply; found-inactive apart from disabled-by-mesh; a skipped step said), mesh-controller 211 (the found firewall's state on node show)
|
||||
amended-design:
|
||||
---
|
||||
|
||||
@@ -108,3 +108,13 @@ convergence is a state the host keeps — the found firewall active again is ret
|
||||
reconcile that finds it inactive records *found so* and never *done by the mesh*, and a step skipped
|
||||
after a failed apply is said. Built in mesh-host on `feat/one-thing-filters-a-converged-machine`; the
|
||||
record of both machines of this mesh is corrected by the first report under it.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
mesh-host 67 and mesh-controller 211, live on every machine at 10:10Z. The step now runs on every
|
||||
converged apply and says what it did; a found firewall enabled again is retired again. The record's
|
||||
one inherited lie stands as history: on the control node the machine's own record already said the
|
||||
mesh had disabled the firewall, and the host trusts its record, so `node show` says "retired by the
|
||||
mesh" there. From this build on, a reconcile that finds the firewall inactive records *found inactive*
|
||||
and never the other thing. Whether the flip's step took on 2026-09-29 is not recoverable and is not
|
||||
owed by this record any more.
|
||||
|
||||
+14
-2
@@ -1,10 +1,10 @@
|
||||
---
|
||||
status: located
|
||||
status: resolved
|
||||
opened: 2026-09-29
|
||||
located-in:
|
||||
- mesh-host internal/apply/opening.go
|
||||
- mesh-controller cmd/mesh-controller (the converge preview)
|
||||
fixed-by:
|
||||
fixed-by: mesh-host 67 (every refusing table and legacy chain classified with an owner; the runtime's user chain is other), mesh-controller 211 (kept, shown on node show, named by status, previewed with fates)
|
||||
amended-design:
|
||||
---
|
||||
|
||||
@@ -91,3 +91,15 @@ chain's refusals as *other*; `node show`, `status` and the converge preview say
|
||||
`feat/one-thing-filters-a-converged-machine` in mesh-host and mesh-controller. On 2026-10-02 the home
|
||||
server still carries the predecessor's chain in its legacy filter; the record's live row is reading it
|
||||
there.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
mesh-host 67 and mesh-controller 211, live at 10:10Z. The live row of
|
||||
[ADR 0168](../../02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md) was read the
|
||||
same hour: the home server's record names the predecessor's chain in the legacy filter's user chain
|
||||
as *other*, with what it refuses, beside two chains a retired front end left in the IPv6 legacy filter;
|
||||
the control node's record names the same two leftovers; the laptop and the workstation read *the mesh
|
||||
alone*. `status` names both machines and is not well until the operator removes what the mesh did not
|
||||
write. The allowance the predecessor's chain carried is
|
||||
[issue 145](../145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md)'s,
|
||||
and that record is not closed by this one.
|
||||
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
---
|
||||
status: resolved
|
||||
opened: 2026-10-02
|
||||
located-in: [mesh-host internal/outward (Links reported only the links carrying a default route)]
|
||||
fixed-by: mesh-host PR 66 (a link backed by a physical device is named outward, up or down), live 2026-10-02
|
||||
amended-design: []
|
||||
---
|
||||
|
||||
# 197 — A physical link that is down is not filtered when it comes up
|
||||
|
||||
## What was observed
|
||||
|
||||
A sweep of every machine's filter on 2026-10-02. A laptop-class machine connected by its radio has a
|
||||
wired port that was unplugged. Its filter guarded the radio and the tunnel, and accepted everything
|
||||
arriving on any other link:
|
||||
|
||||
```
|
||||
iifname != { "mesh0", "<radio>" } accept
|
||||
```
|
||||
|
||||
The wired port was not in the list. Plugged in, everything arriving on it would have been accepted,
|
||||
every port of the machine open to whatever network the cable reached. That would last until the
|
||||
machine reported again and was pushed a new filter.
|
||||
|
||||
## Why it matters
|
||||
|
||||
**The filter's one rule about links fails open.** [ADR 0140](../../02-DECISIONS/0140-the-filter-constrains-what-arrives-from-outside.md)
|
||||
has the filter constrain what arrives from outside, and has the machine say which links face outside.
|
||||
Everything not named is treated as the machine's own, its containers and bridges. So a link the machine
|
||||
fails to name is not filtered at all. The host named only the links carrying a default route at the
|
||||
moment it reported. A cable plugged in later is the ordinary case for a laptop. A second wired network
|
||||
that never carries the default route, such as a direct link to a storage box, is never named at all.
|
||||
|
||||
## Open questions
|
||||
|
||||
- A virtual link that faces outside (a VPN client's interface, a USB tether that appears as a virtual
|
||||
device) has no physical device behind it. It is named only while it carries the default route. Is
|
||||
that enough?
|
||||
|
||||
## Resolved (2026-10-02)
|
||||
|
||||
Live on the affected machine after the host was delivered and one more push: its filter now guards the
|
||||
radio, the tunnel and the unplugged wired port, before anything is plugged into it.
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
# Diagnosis
|
||||
|
||||
*2026-10-02.*
|
||||
|
||||
**Located in `mesh-host` `internal/outward`.** `Links` read the kernel's routing tables and returned the
|
||||
interfaces carrying a default route. An unplugged port carries none, so it was never reported, and the
|
||||
controller rendered the filter around the links it was given.
|
||||
|
||||
**The fix.** A link faces outside if it carries a default route **or** has a physical device behind it.
|
||||
The kernel lists every interface under `/sys/class/net`, with a `device` entry for one backed by
|
||||
hardware. A bridge, a veth, the tunnel and the loopback have none, so they stay the machine's own. The
|
||||
wired port is now reported up or down, and the filter guards it before anything is plugged in. Tested
|
||||
with a radio carrying the default route and an unplugged wired port beside a bridge, a veth, the docker
|
||||
bridge, the tunnel and the loopback: the two physical links are reported, nothing else.
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
---
|
||||
status: resolved
|
||||
opened: 2026-10-02
|
||||
located-in: [mesh-catalog modules/dnsmasq (listens on loopback and the machine's mesh address only), the home-server's DNS (a predecessor's dnsmasq configuration the mesh did not own), the home network's DHCP (hands out the home-server as every device's DNS)]
|
||||
fixed-by: mesh-catalog PR 214 (dnsmasq listens on addresses from a setting; docker's file takes no settings), mesh-controller PR 210 (the settings verb), mesh-catalog PR 215 (unifi network DNS tools), live 2026-10-02
|
||||
amended-design: []
|
||||
---
|
||||
|
||||
# 198 — The home network's DNS server ran outside the mesh, and the mesh's filter closed it
|
||||
|
||||
## What was observed
|
||||
|
||||
Every phone on the home Wi-Fi had no internet, while a laptop on the same Wi-Fi did. The router's
|
||||
DHCP hands every device the home-server's LAN address as its DNS server. The home-server's DNS daemon
|
||||
was listening on that address, and every query to it timed out. The router itself answered the same
|
||||
query at once. The laptop worked because it resolves through its own local resolver, not through the
|
||||
server DHCP names.
|
||||
|
||||
## Why it happened
|
||||
|
||||
The DNS daemon on the home-server was not the mesh's. It ran under a configuration file a predecessor
|
||||
generated, listening on loopback, the mesh address and the LAN address. The mesh's `dnsmasq` module was
|
||||
assigned to the other three machines and not to this one, so no module on the home-server declared
|
||||
port 53. Its filter opens only what a module declares, so DNS from the LAN was dropped. It started when
|
||||
the home-server applied the filter this morning, after nine hours of applying nothing
|
||||
([issue 194](../194-the-hosts-own-former-archive-stops-every-apply/00-report.md)).
|
||||
|
||||
Nothing said so. The daemon reported running, the filter applied cleanly, and the mesh had no record
|
||||
that the home network depended on a service it did not know.
|
||||
|
||||
## Why it matters
|
||||
|
||||
**A service the mesh does not know is closed by the mesh's filter, by design, and nothing asks whether
|
||||
something depends on it.** That is the right default for an unknown port. It is the wrong outcome for
|
||||
the one service a whole network was told to use. The gap is that a machine can run something
|
||||
important outside the mesh with nothing to show it.
|
||||
|
||||
**The mesh's `dnsmasq` could not have served the LAN either.** It listened on loopback and the mesh
|
||||
address only. The reach of its DNS endpoints opens the filter, but the daemon would not have been
|
||||
listening on the LAN address anyway.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should a machine report the listening services the mesh does not own, the way it reports the links
|
||||
that face outside? This one would have been visible before the filter closed it.
|
||||
- The LAN address the home-server answers on is now a setting, beside the reach that opens the filter.
|
||||
Two statements that must agree. Should reach `public` on a DNS endpoint imply listening beyond the
|
||||
mesh?
|
||||
|
||||
## Resolved (2026-10-02)
|
||||
|
||||
The home network was pointed at the gateway for DNS while the fix was built, which got the phones back
|
||||
within minutes. Then:
|
||||
|
||||
- the mesh's `dnsmasq` takes the addresses it listens on beside the machine's from a setting, with
|
||||
loopback as the mesh-wide default, so no other machine changed;
|
||||
- the home-server's layer adds its LAN address, and its DNS endpoints' reach is `public`. The router
|
||||
forwards no DNS, so that means the LAN;
|
||||
- the module and its sibling `resolv-conf` were assigned to the home-server, replacing the
|
||||
predecessor's daemon and configuration, which were kept aside;
|
||||
- the home network was pointed back at the home-server, through a new `unifi` tool.
|
||||
|
||||
Checked live: from another machine on the LAN, public names and mesh names both resolve through the
|
||||
home-server's LAN address, and the mesh and the machine itself resolve as before.
|
||||
|
||||
**One fault found on the way, and caught before it reached any machine.** A module's settings are
|
||||
merged into every mergeable file the module owns. The first attempt therefore put the new setting into
|
||||
docker's `daemon.json` as well as into dnsmasq's config, and dockerd refuses keys it does not know. The
|
||||
plan showed it before any push. The change was reverted and redone with docker's file declared to take
|
||||
no settings. The general fault, a module's settings reaching files they were not meant for, is still
|
||||
there for any module with more than one file.
|
||||
+35
@@ -0,0 +1,35 @@
|
||||
---
|
||||
status: resolved
|
||||
opened: 2026-10-02
|
||||
located-in: [mesh-tools src/client.ts (toolsOn left node-scoped seats out of the listing), mesh-tools src/mcp.ts (the call resolved the key against that listing)]
|
||||
fixed-by: mesh-tools 27 — node-scoped seats are listed with their scope, the verb requires the machine, and the call carries it in the subject
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 199 — A node-scoped seat's verb could not be called through the console
|
||||
|
||||
## What was observed
|
||||
|
||||
2026-10-02, the first time a node-scoped seat declared verbs
|
||||
([ADR 0170](../../02-DECISIONS/0170-the-firewall-seat-serves-its-verbs.md)). The packet filter's
|
||||
holder on every machine served `rules`, `reload` and `remove` on the seat's per-machine subjects, and
|
||||
the bus admitted them. The console answered every call with *nothing serves
|
||||
node-packet-filter.rules@<machine>*.
|
||||
|
||||
[Design 33 §4](../../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md) says a node-scoped seat's
|
||||
tool carries the node it is asked of, as `<seat>.<verb>@<node>`. The console's listing left
|
||||
node-scoped seats out — the comment said they *wait for a caller naming the node* — but the roles map
|
||||
the console resolves a name against is built from that same listing. So the name never resolved as a
|
||||
seat's verb, fell through to a module's subject nobody served, and the refusal named the wrong cause.
|
||||
|
||||
## Why it matters beyond this instance
|
||||
|
||||
A stated behaviour that did not happen, with a refusal that pointed elsewhere: the seat's verbs were
|
||||
live on four machines and unreachable from the one surface a person uses. It could only be found by a
|
||||
node-scoped seat declaring verbs, which none had.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
mesh-tools 27: node-scoped seats are listed with their scope, their verbs take a required `node`, the
|
||||
call carries it in the subject, and a call without one is refused in words. Tested with a round trip
|
||||
asking one machine's holder and being refused without a machine.
|
||||
+36
@@ -0,0 +1,36 @@
|
||||
---
|
||||
status: open
|
||||
opened: 2026-10-02
|
||||
located-in: []
|
||||
fixed-by:
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 200 — The controller's answer to a long console call is refused by the bus
|
||||
|
||||
## What was observed
|
||||
|
||||
2026-10-02. A `push` of the control node asked through the console came back as *mesh-controller.push
|
||||
did not answer in time. Something is serving it, so this is the tool being slow rather than absent.*
|
||||
The push had run; the machine applied. The bus's log on the control node, at the same moment:
|
||||
|
||||
```
|
||||
[ERR] 10.10.0.1:56030 - cid:4015 - Publish Violation - User "controller",
|
||||
Subject "_INBOX.shanks.mesh-console.WRNO5V9IJ1FX35AU5NRBEB.WRNO5V9IJ1FX35AU5PN1UW"
|
||||
```
|
||||
|
||||
The controller's reply to the console's request was refused: the controller's bus account may not
|
||||
publish to the console's reply inbox. Shorter calls (`status`, `node`, `nodes`) answer; the long ones
|
||||
(`push` of a large machine, `issue`) time out on the console's side although they succeed.
|
||||
|
||||
## Why it matters beyond this instance
|
||||
|
||||
A call that succeeds and is reported as a timeout sends a person to retry what already happened — a
|
||||
second push, a second issue — and reads as the mesh being slow when it is the mesh refusing itself.
|
||||
Whether the inbox prefix the console uses is the one the controller's account is allowed to answer, or
|
||||
the request outlives the inbox subscription, is what a diagnosis has to tell apart.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Which reply inboxes may the controller's account publish to, and which does the console request on?
|
||||
- Does a reply after the requester's timeout count as a violation, or is the prefix itself refused?
|
||||
Reference in New Issue
Block a user