ADR 0169 (proposed): a machine joins through the tunnel, and the bus is never public
The bus was public only so a new machine could enrol before it had a tunnel. The machine now makes its tunnel key first, the token is issued for it and makes it a peer of the hub, and enrolment happens over the tunnel.
This commit is contained in:
@@ -0,0 +1,89 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: proposed
|
||||
date: 2026-10-02
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
||||
---
|
||||
|
||||
# 169. A machine joins through the tunnel, and the bus is never public
|
||||
|
||||
## Context
|
||||
|
||||
The bus is the one channel every machine depends on: enrolment, every declaration, every tool. The
|
||||
`nats` module declares it reachable from the mesh only. The controller still opens it to the whole
|
||||
internet on the machine that runs it, as a *foundation* port that no module declares and nothing may
|
||||
close ([issue 051](../04-ISSUES/051-the-mesh-cannot-update-what-it-depends-on/00-report.md)).
|
||||
The reason is joining. [ADR 0004](0004-a-node-and-how-it-joins.md) has a new machine enrol over the bus
|
||||
**before** it has a tunnel. [ADR 0007](0007-connectivity.md) states it as a requirement: the node
|
||||
running the broker must be reachable from wherever nodes are, at a stable address.
|
||||
|
||||
So the bus listens on the internet permanently, for an event that happens a few times a year. A
|
||||
sweep of every machine on 2026-10-02 found no client using the public path. Every connection arrives
|
||||
over the tunnel or from the machine itself. The join token does not use it either: it carries the
|
||||
controller's configured broker address, a mesh name with the old broker's port.
|
||||
|
||||
ADR 0004 already says what a joining machine needs: *an identity, an address, and one peer to reach*.
|
||||
The tunnel can be that peer, if the hub knows the new machine's key before the machine first knocks.
|
||||
WireGuard answers nothing to a key it does not know, which is why the tunnel's own port is safe to
|
||||
leave open where the bus's is not.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Keep the bus public.** It is authenticated and encrypted, but every exposure of it, and of the
|
||||
server behind it, is exposure of the one thing everything depends on.
|
||||
2. **Open the bus publicly only while a join token is live.** Small, and the hub is open only during a
|
||||
join window. But the window is real, the rule is about time rather than about who may reach the
|
||||
bus, and the opening and closing are pushes that can fail between them.
|
||||
3. **The controller makes the new machine's tunnel key and puts it in the token.** One step for the
|
||||
operator, but the private half leaves a machine it does not belong to. ADR 0004 refuses that for
|
||||
every key a node holds.
|
||||
4. **The machine makes its key first, and the token is issued for it.** The machine prints the public
|
||||
half of its tunnel key. The operator issues the token for that key. The controller gives the
|
||||
machine its address and adds it as a peer on the hub. The token carries the hub's tunnel endpoint
|
||||
and key, the machine's address, and the bus's address on the private network. The machine brings
|
||||
up its tunnel and enrols over it.
|
||||
|
||||
## Decision
|
||||
|
||||
**Option 4.**
|
||||
|
||||
- **A machine makes its own tunnel key before it has a token**, and prints the public half. The private
|
||||
half never leaves it, as ADR 0004 says of every key a node holds.
|
||||
- **A token is issued for a tunnel key.** Issuing it assigns the machine's address on the private
|
||||
network, records the key, and makes the machine a peer of the hub. The hub is sent that before the
|
||||
token is shown, so the tunnel answers the moment the machine first uses it.
|
||||
- **The token carries the one peer.** It adds the hub's tunnel endpoint and public key and the
|
||||
machine's own address. **Where** becomes the bus's address on the private network, which needs no
|
||||
name resolution.
|
||||
- **The machine joins through the tunnel.** It brings the tunnel up from the token alone, then enrols
|
||||
over it exactly as before. The enrolment checks that the key it is offered is the one the token was
|
||||
issued for.
|
||||
- **The bus is never public.** It is no longer a foundation port. Its reach is what the `nats` module
|
||||
declares: the mesh. The tunnel's port stays open, as the one way in.
|
||||
|
||||
This changes three things earlier records say. ADR 0004's *where* is the bus's private address, and the
|
||||
token carries the peer. ADR 0007's requirement that the broker be reachable from wherever nodes are
|
||||
becomes: **the hub's tunnel is**. Issue 051's broker port stops being a foundation port.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Joining is two commands on the new machine, with the token issued between them. A token issued for
|
||||
the wrong key gives a tunnel that never answers, and the machine says so rather than timing out at
|
||||
the bus.
|
||||
- An unused token leaves a peer on the hub until it expires. Expiry removes it, the same way it voids
|
||||
the secret.
|
||||
- A machine already in the mesh is unaffected: it reaches the bus over its tunnel today.
|
||||
- The genesis machine, the first one, raises the bus on itself and needs no tunnel to reach it.
|
||||
|
||||
## How this is checked
|
||||
|
||||
| Rule | Checked by |
|
||||
|---|---|
|
||||
| A token is refused without a tunnel key, and carries the hub's peer and the machine's address | a controller test |
|
||||
| Issuing a token makes the machine a peer of the hub before the token is shown | a controller test over the hub's composed tunnel |
|
||||
| An expired, unused token's peer is gone from the hub | a controller test |
|
||||
| Enrolment refuses a tunnel key other than the one the token was issued for | a controller test |
|
||||
| No machine's filter opens the bus to anywhere | a controller test over the composed filter, and the live sweep from outside the mesh |
|
||||
| A new machine joins from outside the hub's network with the bus closed to it | the lab, then by hand |
|
||||
@@ -179,6 +179,7 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0163** — [Taking a module over is a comparison: what it compares, what it refuses, and what it carries](0163-taking-a-module-over-is-a-comparison.md)
|
||||
- **0167** — [A membership carries what its module receives, and who the mesh is](0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)
|
||||
- **0168** — [A converged machine is filtered by the mesh alone, and the host says what else refuses](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md)
|
||||
- **0169** — [A machine joins through the tunnel, and the bus is never public](0169-a-machine-joins-through-the-tunnel-and-the-bus-is-never-public.md) *(proposed)*
|
||||
|
||||
### Its tiers, from the bottom up
|
||||
|
||||
|
||||
Reference in New Issue
Block a user