I have been treating "what a node presents to prove it is that node" as an undecided design question for weeks, and blocking on it. It was decided. 08-connectivity says of the overlay keys: each node generates its own keypair, the private key never leaves the machine, the public key is published to the mesh -- and says explicitly that this IS ADR 0004's "a node holds its own identity", applied. Nobody had applied it to the thing 0004 is actually about. What caused it was a word. The lifecycle said a joining node receives its own durable identity, which reads as the mesh issuing something, and then the question is what. The mesh issues nothing. A node arrives holding its identity; what it receives is being known. That line now says what happens: it presents the one-time secret and its own public key, which the mesh records. The rule above it then holds literally rather than aspirationally. The mesh stores a public key, so a copy of the mesh's database grants nothing, and compromise of a node really is compromise of only that node. Also recorded, since it was asked directly: same principle as SSH, own key, not the machine's SSH host key. Host keys are regenerated by reinstalls and image clones, which would silently un-enrol a node; their lifecycle belongs to sshd rather than the mesh; and a partial host has no SSH daemon at all, so an identity scheme resting on one excludes a supported kind of node. The good half of that idea is kept: the mesh knows every node, so it can distribute host keys the way it distributes authorised keys, and node-to-node SSH stops depending on trust-on-first-use.
220 lines
12 KiB
Markdown
220 lines
12 KiB
Markdown
---
|
|
topic: the tiers
|
|
status: accepted
|
|
date: 2026-08-28
|
|
deciders: jochen
|
|
reconstructed: false
|
|
---
|
|
|
|
# 4. A node, and how it joins
|
|
|
|
*Consolidated 2026-08-28 from four records.*
|
|
|
|
## What a node is
|
|
|
|
**A managed machine inside the mesh.** Not a device that is merely known about, not an
|
|
unprivileged something. If the mesh does not manage it, it is not a node — it is a client, a
|
|
peer, or a thing on the network, and those want their own names rather than a weakened version of
|
|
this one.
|
|
|
|
**A disconnected node is still a node, in a different situation.** Reachability is **state, not
|
|
class**. A machine switched off, roaming, or behind a connection that has dropped has not become
|
|
a lesser kind of thing; it has a last-known state and a pending set of declarations.
|
|
|
|
The distinction people reach for is real, but it is **capability** — what this machine can be
|
|
asked to do — and that belongs in the host's profile rather than in the definition of a node.
|
|
|
|
**This is the rule that does the most work elsewhere.** A single control plane is tolerable
|
|
because its absence is every node in the ordinary disconnected situation at once. An episodic
|
|
host on a phone is that situation more often. Neither needed a new mechanism.
|
|
|
|
### A node runs one agent session
|
|
|
|
*Written 2026-08-29. It runs on every node today and appeared in no record, which is how something
|
|
deliberate comes to look accidental.*
|
|
|
|
**A node is a machine. The session is a feature of it** — one of the things running there, like the
|
|
host, like any workload. The node does not think; something on the node does. Which is why
|
|
[ADR 0001](0001-mesh-brokers-nodes-host-agents-think.md)'s *a node does not authenticate to a model
|
|
provider, agents do* holds unchanged: the session authenticates, and it is not the machine.
|
|
|
|
**It is permanent, and it remembers.** Anything in the mesh can send it a message; it replies; and
|
|
what it was asked ten minutes ago is still there next week, alongside what everything else asked in
|
|
between — the same way both sides of any conversation remember it.
|
|
|
|
Its system prompt is the node's **engram** — what makes one node's replies recognisably its own
|
|
rather than generic.
|
|
|
|
**It has its own tools**, and fewer than a session a person is driving directly. So a question can
|
|
be answered by going and looking: *what is in our forge*, not only *what is your battery*.
|
|
|
|
**Messages travel the broker like everything else** ([ADR 0002](0002-nodes-communicate-over-a-broker.md)).
|
|
There is no second transport and nothing is dialled.
|
|
|
|
**Any node can message any node, and this is the one part of the system that is genuinely a mesh**
|
|
— symmetric, with no centre. A node that is asked something it does not have can ask another, and
|
|
how it passes the question on is its own business: it may say who wants to know, or simply ask. A
|
|
person relaying a question makes the same choice, and it follows from the engram rather than from a
|
|
message format.
|
|
|
|
**There is no authorisation between nodes.** Every node is the operator's own, so a message from
|
|
one is a message from them, and asking a node something is asking a colleague rather than
|
|
presenting credentials. Stated once so it is not discovered later: **the mesh boundary is therefore
|
|
the security boundary** — anything inside can reach what any node can reach, which is what puts the
|
|
whole perimeter on the token and the overlay
|
|
([ADR 0007](0007-connectivity.md)).
|
|
|
|
**It can be switched off, and switched off it still answers.** A node whose session is disabled
|
|
replies saying so, at once, with no model involved — the queue is still read, and the state is the
|
|
reply. That is deliberate and it is the same rule the host follows about a service that does not
|
|
exist: **absence must never be indistinguishable from a failure to answer.** A node with nothing
|
|
there is a silence somebody has to go and diagnose; a node that says *I am switched off* is not.
|
|
|
|
**One per node, always, and it cannot be moved to another machine.** Two and nothing decides which
|
|
replies; none and the node is mute; moved, and one machine is answering as another.
|
|
|
|
**It is not an employee** ([ADR 0003](0003-agents-are-persistent-employees.md)). Nobody hires it,
|
|
it holds no tasks, it drains nothing and it is never reassigned — that vocabulary was written for
|
|
workers and does not describe this. It exists because the node does, and it is gone when the node
|
|
leaves.
|
|
|
|
## How it joins
|
|
|
|
**The host has one behaviour and two sources of declaration.** What differs between the first
|
|
node and the fiftieth is not what the host does but where the declaration comes from — and, as
|
|
above, that is a situation rather than a class.
|
|
|
|
| | declaration comes from |
|
|
|---|---|
|
|
| no mesh reachable | the pinned bundle the host carries |
|
|
| mesh reachable | the control plane, over the link |
|
|
|
|
**The first node is not a different kind of node.** It is a node whose mesh is not up yet. It
|
|
applies the bundle it carries, the control plane comes up on top of it, and from that moment it
|
|
takes declarations like everything else. **Its specialness is temporary and self-erasing**, which
|
|
is what the hand-run bootstrap scripts never were.
|
|
|
|
**A joining node does the minimum to be reachable and nothing else** — an identity, an address,
|
|
and one peer to reach. It does **not** compute the overlay: the whole peer set is derived
|
|
centrally and pushed down.
|
|
|
|
That is also why the migration is smaller than it looked. The hard part of the overlay — every
|
|
node's key, address, site and reachability — is only needed to compute the *whole* mesh, and a
|
|
joining node needs one peer.
|
|
|
|
## The link is the security boundary
|
|
|
|
**Everything reaching a node arrives one way**, and four properties make that a boundary rather
|
|
than a pipe.
|
|
|
|
**It is outbound and node-initiated.** The node dials the control plane; nothing dials a node. Not
|
|
only defensive — most nodes sit behind a household connection with no forwarded port, so an
|
|
inbound control channel would work for one node and not the rest, and the difference would be
|
|
invisible until it mattered. **A node has no listening control surface at all.**
|
|
|
|
**A node holds its own identity and nothing else.** No shared secret, no credential to anything it
|
|
does not own. **Compromise of a node is compromise of that node** — which the current arrangement
|
|
does not have, because every node permanently holds the same database and object-store
|
|
credentials, and there is no mechanism that rotates one and informs everything holding it.
|
|
|
|
### What that identity is: a keypair the node generates
|
|
|
|
*Written 2026-08-29. This is the same rule as the sentence above, and it had been treated as an
|
|
open question for weeks because of a word.*
|
|
|
|
**The node generates a keypair. The private half never leaves the machine. The mesh records the
|
|
public half.** Ed25519, the same as the control plane's signing key, in the other direction:
|
|
the mesh proves itself to a node by signing, and a node proves itself to the mesh by signing.
|
|
|
|
**This was never open.** [`08-connectivity.md`](../03-DESIGN/01-to-be/08-connectivity.md) already
|
|
says it of the overlay keys, in these words: *each node generates its own keypair, the private key
|
|
never leaves the machine, the public key is published to the mesh* — and adds that this **is**
|
|
ADR 0004's *a node holds its own identity*, applied. What was missing was applying it to the thing
|
|
this record is about.
|
|
|
|
**The word that caused it:** the lifecycle says a joining node *receives* its own durable identity,
|
|
which reads as the mesh issuing something, and then the question becomes *issuing what*. It does
|
|
not issue anything. The node arrives holding its identity; what it receives is **being known**.
|
|
Enrolment is the moment the mesh writes down a public key it will believe, and the one-time secret
|
|
is what buys the right to have it written down.
|
|
|
|
**Everything above then holds literally.** Nothing is stored that could be stolen and replayed: the
|
|
mesh's copy is a public key, so a copy of the mesh's database grants nothing. *Compromise of a node
|
|
is compromise of that node* becomes true rather than aspirational, because the only secret on a
|
|
machine is the one that identifies it.
|
|
|
|
### Its own key, not the machine's SSH host key
|
|
|
|
Reusing the host key is the obvious economy and it is refused, for reasons that are operational
|
|
rather than fastidious:
|
|
|
|
- **It is regenerated by ordinary events.** A reinstall, an image cloned, `ssh-keygen -A` on a
|
|
rebuild — each silently un-enrols the node, and the failure appears as an authentication problem
|
|
with no cause anybody changed.
|
|
- **It is managed by something else.** Its lifecycle belongs to the machine's SSH daemon, and an
|
|
identity the mesh depends on should not rotate on a schedule the mesh does not know about.
|
|
- **Not every node has one.** A partial host has no SSH daemon
|
|
([ADR 0005](0005-the-node-host.md)), and an identity scheme that excludes a supported kind of
|
|
node is not one.
|
|
|
|
**The mesh should still know the host key** — it knows every node, so it can distribute host keys
|
|
the same way it distributes authorised keys
|
|
([ADR 0006](0006-the-substrate-and-the-control-plane.md)), and node-to-node SSH stops depending on
|
|
trust-on-first-use. That is the good half of the idea, kept.
|
|
|
|
**Authority is mutual.** The node proves it may join, and the control plane proves it is the
|
|
mesh. One-way is not enough: the host applies whatever the link delivers, so a node that cannot
|
|
tell the mesh from something impersonating it will apply that something's declarations.
|
|
|
|
**What may be pushed is bounded by form, not by trust.** Declarations of known shape, never a
|
|
command to run. Stated honestly, **this bounds form and not impact**: a compromised control plane
|
|
can declare harmful state and the host will apply it faithfully, because that is what it is for.
|
|
What the property buys is that the blast radius is *describable* — exactly what the declaration
|
|
language can express, which can be reviewed. An arbitrary command channel has no such bound.
|
|
|
|
## The enrolment token carries the mesh
|
|
|
|
Mutual authority needs the node to verify something before it trusts anything, and that is a
|
|
circle: verifying the mesh needs the mesh's certificate authority, and obtaining one means
|
|
trusting whoever hands it over. There is a second circle beside it — a node must reach the mesh
|
|
before the mesh has configured it, so it can resolve no mesh name.
|
|
|
|
**Both are the same shape: a node needs a fact about the mesh before it has any trustworthy way
|
|
to obtain one.** So that fact arrives by a path other than the mesh.
|
|
|
|
**The token carries four things**, and it is the only thing a joining node needs:
|
|
|
|
| | |
|
|
|---|---|
|
|
| **where** | the broker's **address**, not a name — there is no resolution yet, and this is why none is needed |
|
|
| **what it is connecting to** | the fingerprint of the broker's certificate |
|
|
| **who it will believe** | the control plane's signing identity |
|
|
| **the right to join** | a one-time secret, useless once used and useless after it expires |
|
|
|
|
**Carried out of band**, by the person adopting the machine. That is what breaks both circles:
|
|
its authenticity comes from the channel it travelled, not from anything the node can check
|
|
afterwards. **Trust on first use, with the first use moved out of band** — the difference between
|
|
a pin and a guess.
|
|
|
|
**The endpoint and the authority are two identities.** A node connects to the broker and takes
|
|
instruction from the control plane behind it. Pinning only the broker would make the control
|
|
plane's authority *transitive*, and a compromised broker could then forge declarations — which,
|
|
since the host applies whatever the link delivers, is the whole machine. So the transport is
|
|
verified once at connect, and **each declaration is verified by its signature, every time**.
|
|
|
|
**What this settles:** the mesh's certificate authority is not a bootstrap concern — it certifies
|
|
internal names once a node is a member. Nothing needs name resolution before the link. And
|
|
nothing is placed on disk beforehand except the token, which is the first moment *a node holds
|
|
only its own identity* becomes true rather than aspirational.
|
|
|
|
## Consequences
|
|
|
|
- **Declarations must be signed**, and the host must tell *this is not from the mesh I joined*
|
|
apart from *this is malformed*. Rotating the signing identity is a fleet-wide operation with an
|
|
overlapping rollover, and that is the cost of not trusting the broker.
|
|
- **The token becomes security-critical**, because it carries the pin. Tampering substitutes the
|
|
mesh — which is strictly better than the alternative, where there is nothing to tamper with and
|
|
the node trusts the first answer unconditionally.
|
|
- **A rejoining node is ordinary.** There is no long-lived secret to recover, so a node that lost
|
|
its identity gets a new token.
|