Compare commits
16
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
248c99ca6c | ||
|
|
13208f0f48 | ||
|
|
ebd19c4c6c | ||
|
|
dd4cbabffb | ||
|
|
5b76a09da6 | ||
|
|
a363a605cb | ||
|
|
24835ab710 | ||
|
|
ba397d4cbe | ||
|
|
df4a3538c3 | ||
|
|
90b8eeff6b | ||
|
|
a865fc7d79 | ||
|
|
c3313f6e17 | ||
|
|
60a53f9b18 | ||
|
|
ee31f9f761 | ||
|
|
0e066473b3 | ||
|
|
504adef221 |
@@ -0,0 +1,135 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
date: 2026-09-26
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0110-a-seat-is-a-module-assignment-from-a-closed-set.md
|
||||
---
|
||||
|
||||
# 117. A machine's uplink is a seat: the mesh configures the manager, never the link
|
||||
|
||||
## Context
|
||||
|
||||
The mesh installs on top of a machine's own networking. The private network's generator says
|
||||
so in as many words: a machine has an address and a route to the broker *before* the mesh
|
||||
exists, the broker's address travels in the enrolment token rather than being resolved, and the
|
||||
private network is something the mesh installs on top, like anything else. Nothing in the mesh
|
||||
says who manages that uplink, or what the mesh needs from whoever does.
|
||||
|
||||
Adopting the first workstations showed that the mesh does need something from it, and gets it
|
||||
by accident:
|
||||
|
||||
- **The resolver the mesh owns depends on a file the mesh does not.** `resolv-conf` writes
|
||||
`/etc/resolv.conf` and names the mesh's resolver. On a machine running NetworkManager, the
|
||||
manager rewrites that file on every connectivity change unless it is told `dns=none`; on one
|
||||
running dhcpcd, every lease renewal rewrites it unless it is told `nohook resolv.conf`. On
|
||||
the adopted machines both settings exist only because the predecessor wrote them. No module
|
||||
declares them. Remove the predecessor's file and the mesh's resolver is silently replaced the
|
||||
next time a laptop changes network, while every surface of the mesh still reads green.
|
||||
- **`resolv-conf` cannot declare them itself.** Which setting is needed depends on which
|
||||
manager runs, and a `service` resource for a manager that is not installed fails the
|
||||
declaration. A resolver module that knew about network managers would be the wrong module
|
||||
knowing the wrong thing.
|
||||
- **The private network's interface is exposed to the manager.** A manager that considers
|
||||
every interface its own may try to configure `mesh0`, or tear it down on a profile change.
|
||||
Nothing tells it not to.
|
||||
- **Two managers on one machine go unnoticed.** Among the machines adopted so far, one runs
|
||||
NetworkManager *and* dhcpcd at once: two programs that each believe they own the machine's
|
||||
addresses and its resolver file.
|
||||
Nothing detected it, because nothing in the mesh knows the role exists.
|
||||
|
||||
The machines differ in a way that matters: servers are wired and never move, while
|
||||
workstations join wireless networks, captive portals and phone hotspots wherever they are.
|
||||
|
||||
## Considered Options
|
||||
|
||||
**1. The mesh manages the uplink: links, addressing, wireless networks and their
|
||||
credentials.** Rejected. The mesh reaches a machine only over that link. A declaration that
|
||||
gets it wrong — a mistyped network, a stale credential, a manager that fails to start — takes
|
||||
the machine off the network, and with it the only channel a fix could arrive on. That is the
|
||||
one failure the sshd module's `listens` rule forbids the firewall to arrange; a mesh that owned
|
||||
the link could arrange it with any push. And a wireless network is joined at the machine, by
|
||||
the person using it, in the moment. A declaration composed elsewhere cannot answer a captive
|
||||
portal.
|
||||
|
||||
**2. Leave the uplink unmanaged; accept the implicit dependency.** Rejected. It keeps the
|
||||
resolver working only for as long as a predecessor's file survives, and it leaves two managers
|
||||
on one machine undetectable.
|
||||
|
||||
**3. The uplink is a seat. The module holding it configures the manager's relationship to the
|
||||
mesh, and never the link.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**`the-uplink` is a node-scoped seat** in the closed set ([ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)).
|
||||
It delivers no provision. It is held by the module for the program that manages the machine's
|
||||
own network, one per manager: `networkmanager`, `systemd-networkd`, and `dhcpcd` for a machine
|
||||
with nothing more. Assigning a second is refused, naming the first.
|
||||
|
||||
**What a holder declares** — only what keeps the manager and the mesh from contradicting each
|
||||
other:
|
||||
|
||||
- the manager's package, present — and its service **with no state**: the manager's lifecycle is
|
||||
the machine's. The mesh never starts, stops, enables or disables it, because stopping it takes
|
||||
the link down, and a holder unassigned by mistake — or the wrong holder assigned — must not be
|
||||
able to do that, nor start a second manager beside the one the machine runs. The service is
|
||||
declared only so a change to the holder's settings reaches a *running* manager;
|
||||
- the manager's own configuration that leaves the resolver file to the mesh (`dns=none` for
|
||||
NetworkManager, `nohook resolv.conf` for dhcpcd, and nothing for systemd-networkd, which
|
||||
never writes the resolver file);
|
||||
- the manager's own configuration that leaves the private network's interface alone
|
||||
(NetworkManager's `unmanaged-devices` naming `mesh0`; dhcpcd's `denyinterfaces mesh0`; for
|
||||
systemd-networkd a network file of the module's matching `mesh0` as `Unmanaged=yes`);
|
||||
- each as a drop-in beside the manager's main file where the manager reads one, and written
|
||||
*into* a shared file otherwise, as a marked region the host owns
|
||||
([ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md)'s idea for text files),
|
||||
placed where the manager reads it as global — at the start of `dhcpcd.conf`, above any
|
||||
`interface` line, because every line after one belongs to that interface;
|
||||
- the service **reloaded** when a drop-in changes, never restarted — a restart drops the link,
|
||||
and the link is the mesh's own channel to the machine. **A manager that cannot reload is not
|
||||
restarted instead:** its setting takes effect at the manager's next start. Measured on the
|
||||
adopted machines: NetworkManager (1.58) and systemd-networkd (systemd 261) both report
|
||||
`CanReload=yes`; dhcpcd (10.3) reports `CanReload=no`, so its module declares no trigger at
|
||||
all. Whether each setting is actually *applied* by a reload is confirmed on a machine before
|
||||
the module is taken there, not assumed.
|
||||
|
||||
**What a holder never declares:** a link, an address, a route, a connection profile, a
|
||||
wireless network or its credentials. Those are the operator's, in the sense of
|
||||
[ADR 0051](0051-shared-data-is-the-operators.md): the mesh does not create, change or delete
|
||||
them, and the module's `access`, if it needs one, is read-only.
|
||||
|
||||
## Consequences
|
||||
|
||||
- `resolv-conf` stays generic. The condition it could not express — "only if NetworkManager
|
||||
runs" — is expressed by assigning the module for the manager that does.
|
||||
- The dependency on the predecessor's `dns=none` file becomes a declared resource. On an
|
||||
adopted node the holder's drop-in arrives beside the predecessor's; both say the same thing,
|
||||
and the predecessor's is retired by hand after the take, like any other file the mesh
|
||||
replaced under another name.
|
||||
- A setting a manager reads only at its start is not in force until then. On an adopted machine
|
||||
the predecessor's identical line normally already is; on a machine that was not adopted,
|
||||
dhcpcd's resolver hook keeps rewriting the resolver file until dhcpcd next starts, and the
|
||||
operator restarts it once, in a window of their choosing.
|
||||
- A machine running two managers is found at assignment: the second holder is refused, and the
|
||||
operator decides which manager the machine keeps before either module is taken.
|
||||
- Workstations keep joining networks the way they always have. Under NetworkManager and
|
||||
systemd-networkd the host already cooperates with the manager — its dispatcher hook wakes it
|
||||
on every connectivity change — and nothing here changes that. A dhcpcd-only machine has no
|
||||
such hook, and nothing here adds one.
|
||||
- The seat table gains one entry: `the-uplink`, node scope, delivering nothing, decided here.
|
||||
- **Not decided here:** whether the mesh should ever *offer* known networks to a machine — a
|
||||
sealed, add-only list the operator curates once for all workstations. That is a different
|
||||
question (the mesh holding credentials for links it must never be able to break) and gets its
|
||||
own record if it is wanted.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md): the closed set this seat joins;
|
||||
[to-be 26](../03-DESIGN/01-to-be/26-the-seats.md): the seat table
|
||||
- [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md): written into, never over
|
||||
- [ADR 0051](0051-shared-data-is-the-operators.md): what is the operator's stays the operator's
|
||||
- mesh-controller `internal/catalogue/seats.go` (the seat), `internal/overlay/generator.go` (the
|
||||
mesh installs on top of the machine's own networking)
|
||||
- mesh-catalog `modules/networkmanager`, `modules/systemd-networkd`, `modules/dhcpcd`
|
||||
- mesh-host `internal/apply/block.go` (a file written into a marked region, `at` start or end)
|
||||
@@ -40,7 +40,7 @@ undeclare can do. Found reviewing the uplink modules
|
||||
- the uplink modules would have stopped the network manager, taking the machine off the only
|
||||
link the mesh reaches it by.
|
||||
|
||||
ADR 0117 (in review) answered that for its own modules with a
|
||||
[ADR 0117](0117-a-machines-uplink-is-a-seat.md) answered that for its own modules with a
|
||||
service declared with no `state`. Every other module that declares a unit it did not make is
|
||||
exposed in the same way, and relying on each author to remember an opt-out is how the next one
|
||||
is missed.
|
||||
@@ -113,7 +113,7 @@ records the state it first found the unit in, and undeclaring returns the unit t
|
||||
## References
|
||||
|
||||
- [issue 130](../04-ISSUES/130-undeclaring-a-service-stops-it/00-report.md): the finding
|
||||
- ADR 0117 (in review): the uplink modules, and a service with no state
|
||||
- [ADR 0117](0117-a-machines-uplink-is-a-seat.md): the uplink modules, and a service with no state
|
||||
- [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md), [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md):
|
||||
what is given back, and how
|
||||
- mesh-host `internal/apply/apply.go` (`remove`, the service case)
|
||||
|
||||
@@ -199,6 +199,7 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0113** — [The vault makes every shared secret, a provider makes resources and data, and the mesh carries both](0113-the-vault-makes-every-secret.md) *(proposed)*
|
||||
- **0114** — [A credential two parties hold rotates over two credentials; one a single party holds rotates in place, staged; and retiring a credential never removes what it reached](0114-a-shared-credential-rotates-over-two-credentials.md) *(proposed)*
|
||||
- **0115** — [One assignment of a module per node: the module's name is the assignment's identity](0115-one-assignment-of-a-module-per-node.md) *(proposed)*
|
||||
- **0117** — [A machine's uplink is a seat: the mesh configures the manager, never the link](0117-a-machines-uplink-is-a-seat.md)
|
||||
- **0118** — [Undeclaring removes what the mesh made, and gives a unit back the state it was found in](0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md)
|
||||
|
||||
### How it is built
|
||||
|
||||
@@ -8,11 +8,12 @@ code:
|
||||
- mesh-controller cmd/mesh-controller/source.go
|
||||
- mesh-controller internal/inventory/migrations/0032-a-source-may-live-on-a-seat.sql
|
||||
- mesh-catalog modules/gitea/module.json
|
||||
updated: 2026-09-26
|
||||
updated: 2026-09-27
|
||||
decisions:
|
||||
- 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md
|
||||
- 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md
|
||||
- 02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md
|
||||
- 02-DECISIONS/0117-a-machines-uplink-is-a-seat.md
|
||||
---
|
||||
|
||||
# 26 — The seats
|
||||
@@ -64,6 +65,7 @@ nobody argued for is an entry nobody can explain.
|
||||
| `the-private-network` | node | — | the private network the mesh runs over |
|
||||
| `the-resolver-configuration` | node | — | whichever of the alternative resolver configurations is chosen |
|
||||
| `the-showcase` | node | — | the showcase module |
|
||||
| `the-uplink` | node | — | the program that manages the machine's own network ([ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md)) |
|
||||
|
||||
The controller holds this set in code, and a test asserts both its size and that every entry names
|
||||
the record that made it a seat. **This table and [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
---
|
||||
status: located
|
||||
opened: 2026-09-26
|
||||
located-in: [mesh-controller internal/catalogue/facts.go, mesh-host internal/apply]
|
||||
---
|
||||
|
||||
# 128 — the machine's hosts file is written whole, and on a workstation it is shared
|
||||
|
||||
## What was observed
|
||||
|
||||
The private network asks for the `node-names` fact, and the mesh delivers it as
|
||||
`/etc/hosts`. `nodeNames` composes a **complete** file — its own header, `localhost`, the
|
||||
machine's own name, and every name in the mesh — and the host writes it over whatever is there.
|
||||
|
||||
On an adopted workstation the file the mesh holds contains, besides the predecessor's block of
|
||||
mesh names:
|
||||
|
||||
- the distribution's own lines (`localhost`, the machine's `.localdomain` name);
|
||||
- two marked blocks (`# BEGIN … # END …`) maintained by a local-development tool, pointing a
|
||||
dozen development hostnames at `127.0.0.1` — rewritten by that tool whenever its project
|
||||
list changes;
|
||||
- hand-added entries of the operator's.
|
||||
|
||||
Today the file is only **held** ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)):
|
||||
the private network was assigned, not yet taken, so nothing was lost. Taking it — or
|
||||
converging the node, which takes everything — replaces the file. The development tool's
|
||||
entries disappear, its projects stop resolving, and every later write it makes is overwritten
|
||||
at the next change to the mesh's names (a machine joins, a route is contributed), silently and
|
||||
without a failure anywhere: the development tool thinks it wrote its block, and the mesh thinks
|
||||
it owns the file.
|
||||
|
||||
This is [ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md)'s
|
||||
failure exactly — a file the mesh shares with software it did not install, written over — in a
|
||||
file 0102 did not name, because its merge verb is structured (`into: json`) and a hosts file is
|
||||
not JSON.
|
||||
|
||||
A second, smaller finding from the same reading: the fact's contents depend on which machines
|
||||
hold the private network. A machine that is enrolled but not yet assigned the private network
|
||||
is in neither `node-names` nor `node-zones`; its name resolves on the others only for as long
|
||||
as a predecessor's hosts block survives. Taking the hosts file before every machine is on the
|
||||
private network loses that name too.
|
||||
|
||||
## What would have prevented it
|
||||
|
||||
- A **marked-region** merge in the host's vocabulary: `into: "block"` (or similar) — the host
|
||||
owns only the lines between its own begin and end markers, keeps everything outside them
|
||||
byte for byte, records what the region held before, and on undeclare removes the region and
|
||||
nothing else. The shape local tools already use for this very file.
|
||||
- The `node-names` fact written as that region — no header of its own, no `localhost`, no
|
||||
machine name — so the distribution's lines and every other tool's stay where they are.
|
||||
- A converge preview that names a held file the take would replace *whole*, with its line
|
||||
count before and after, so a person sees "hosts: 31 lines → 12" before the flip.
|
||||
|
||||
## The fix, as built (in review)
|
||||
|
||||
- **Host:** a file resource may say `"into": "block"`. The host owns only the lines between
|
||||
`# BEGIN mesh <id>` and `# END mesh <id>` and keeps everything outside them byte for byte. A
|
||||
new region goes at the `end` by default, or at the `start` (`"at": "start"`) for files where a
|
||||
line's meaning depends on what stands above it; a region already present is never moved.
|
||||
Undeclared, what the region held before is put back, or the region is removed and nothing
|
||||
else. Replacing nothing, it is written on an adopted node without being held — so a machine
|
||||
gets the mesh's names before its private network is taken.
|
||||
- **Controller:** `node-names` is a fact written into a shared file, emitted as that region: the
|
||||
mesh's names only, no header, no `localhost`, no `127.0.1.1` line.
|
||||
- **Order:** a host older than the block mode refuses the whole declaration on an unknown
|
||||
`into`, so hosts are upgraded before the controller that emits it.
|
||||
|
||||
## Evidence to carry into diagnosis
|
||||
|
||||
- `internal/catalogue/facts.go`, `nodeNames`: the complete file is built here.
|
||||
- The host's file resource supports `into: "json"` only; anything else is a whole write.
|
||||
- `node show <node>` on the adopted workstation: `holds file /etc/hosts
|
||||
mesh-wireguard.fact-node-names`, original kept.
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
status: open
|
||||
opened: 2026-09-26
|
||||
located-in: [mesh-controller, mesh-catalog step-ca]
|
||||
---
|
||||
|
||||
# 129 — nothing makes a machine trust the mesh's own certificate authority
|
||||
|
||||
## What was observed
|
||||
|
||||
On an enrolled, adopted workstation — on the private network, resolving the mesh's names
|
||||
through the mesh's resolver — every HTTPS name the mesh serves internally fails verification:
|
||||
|
||||
```
|
||||
curl https://<a name the mesh routes internally>/
|
||||
curl: (60) SSL certificate OpenSSL verify result: unable to get local issuer certificate (20)
|
||||
```
|
||||
|
||||
The route proxy presents a certificate issued by the mesh's internal authority (step-ca, the
|
||||
`internal-acme-ca` provision). The machine's trust store holds the **predecessor's** authority
|
||||
and a developer tool's local root, and nothing of the mesh's. No module installs the mesh's
|
||||
root, and no fact carries it: step-ca's only consumers are proxies, which obtain certificates
|
||||
over ACME and never need the root on the machine they run on.
|
||||
|
||||
[Issue 048](../048-nothing-makes-a-machine-trust-the-mesh-registry/00-report.md) found the same
|
||||
shape for the mesh's registry and resolved it by treating the private network as the transport
|
||||
security ([ADR 0082](../../02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md)):
|
||||
the runtime pulls in the clear, over the tunnel. That answer does not carry over. A browser, git
|
||||
over HTTPS, a package manager and every TLS client a person or a module uses verify the
|
||||
certificate chain, and there is no "insecure registries" for them — nor should there be.
|
||||
|
||||
Consequences today, all silent until someone tries:
|
||||
|
||||
- a person on a workstation cannot open any internal HTTPS name without a warning;
|
||||
- git over HTTPS to the mesh's forge fails, so the working clone URL is ssh-only;
|
||||
- a module on a non-hub machine that calls another module's internal HTTPS name fails
|
||||
verification unless its image happens to carry the root;
|
||||
- the predecessor's authority cannot be retired from any machine while anything there still
|
||||
speaks TLS to a mesh name, because it is the only authority those machines trust.
|
||||
|
||||
## What would have prevented it
|
||||
|
||||
- A **mesh fact carrying the internal authority's root** (public material; the controller or
|
||||
the step-ca module is its source), written onto every machine on the private network — the
|
||||
same reasoning that has the private network write the registry trust and the names: being on
|
||||
the network is what makes a machine one that speaks to the mesh's names.
|
||||
- A resource that puts it where the machine's TLS clients look — on Arch,
|
||||
`/etc/ca-certificates/trust-source/anchors/` — and **refreshes the extracted bundles**
|
||||
(`update-ca-trust`). The refresh is the open design question: it is a command, and the link
|
||||
may not carry an action ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). A declared
|
||||
one-shot unit, or a host primitive for "trust this anchor", are the obvious candidates.
|
||||
- Removal symmetric to arrival: undeclared, the anchor goes and the bundles are refreshed again,
|
||||
so a machine leaving the mesh stops trusting it.
|
||||
|
||||
## Evidence to carry into diagnosis
|
||||
|
||||
- `step-ca` module: provides `acme-ca` / `internal-acme-ca`, listens on 9000 for proxies; no
|
||||
resource writes its root anywhere but its own state directory.
|
||||
- The private network's generator writes `/etc/hosts` and the registry trust, and nothing
|
||||
about certificates.
|
||||
- On the workstation, the trust anchors present are the predecessor's authority and a local
|
||||
development root; `trust list` shows no entry for the mesh.
|
||||
@@ -9,7 +9,7 @@ amended-design: 02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was
|
||||
|
||||
## What was observed
|
||||
|
||||
Reviewing the uplink modules (ADR 0117, in review)
|
||||
Reviewing the uplink modules ([ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md))
|
||||
found that the host's `remove` path stops every `service` resource that is no longer declared:
|
||||
`SetServiceState(..., "stopped")`, reported as "stopped; the unit file is not the host's to
|
||||
delete". `store.Orphans` matches by id alone. So any of these stops the unit:
|
||||
|
||||
Reference in New Issue
Block a user