Files
hq/02-DECISIONS/0004-a-node-and-how-it-joins.md
T
jschoubben 02afb7516b What connecting to the mesh is, and what a node presents
Two things this record never said, both asked directly.

Connecting to the mesh is one outbound AMQP connection from the node to the
broker, held open. There is no second connection and nothing is ever dialled at
a node. Being in the mesh means that connection is up.

Two different things ride on it and conflating them is what made this murky. An
AMQP account, which the mesh issues per node at enrolment, answers whether the
connection is accepted at all -- per node rather than shared, because a shared
one lets any node consume another's queue, which is the shared-credential fault
this record exists to remove reappearing at the transport.

The node's own keypair answers which node is speaking, on every message. It is
not made redundant by the account: with only an account the control plane knows
who is speaking because the broker says so, and that is the same transitive
authority this record already refuses in the other direction. A compromised
broker could attribute reports to whichever node it liked.

So a node holds two things after enrolment -- a credential the mesh issued for
reaching the broker, and a key it generated that the mesh only sees the public
half of. Both are its own, neither reaches anything else.
2026-08-29 15:25:18 +02:00

14 KiB

topic, status, date, deciders, reconstructed
topic status date deciders reconstructed
the tiers accepted 2026-08-28 jochen false

4. A node, and how it joins

Consolidated 2026-08-28 from four records.

What a node is

A managed machine inside the mesh. Not a device that is merely known about, not an unprivileged something. If the mesh does not manage it, it is not a node — it is a client, a peer, or a thing on the network, and those want their own names rather than a weakened version of this one.

A disconnected node is still a node, in a different situation. Reachability is state, not class. A machine switched off, roaming, or behind a connection that has dropped has not become a lesser kind of thing; it has a last-known state and a pending set of declarations.

The distinction people reach for is real, but it is capability — what this machine can be asked to do — and that belongs in the host's profile rather than in the definition of a node.

This is the rule that does the most work elsewhere. A single control plane is tolerable because its absence is every node in the ordinary disconnected situation at once. An episodic host on a phone is that situation more often. Neither needed a new mechanism.

A node runs one agent session

Written 2026-08-29. It runs on every node today and appeared in no record, which is how something deliberate comes to look accidental.

A node is a machine. The session is a feature of it — one of the things running there, like the host, like any workload. The node does not think; something on the node does. Which is why ADR 0001's a node does not authenticate to a model provider, agents do holds unchanged: the session authenticates, and it is not the machine.

It is permanent, and it remembers. Anything in the mesh can send it a message; it replies; and what it was asked ten minutes ago is still there next week, alongside what everything else asked in between — the same way both sides of any conversation remember it.

Its system prompt is the node's engram — what makes one node's replies recognisably its own rather than generic.

It has its own tools, and fewer than a session a person is driving directly. So a question can be answered by going and looking: what is in our forge, not only what is your battery.

Messages travel the broker like everything else (ADR 0002). There is no second transport and nothing is dialled.

Any node can message any node, and this is the one part of the system that is genuinely a mesh — symmetric, with no centre. A node that is asked something it does not have can ask another, and how it passes the question on is its own business: it may say who wants to know, or simply ask. A person relaying a question makes the same choice, and it follows from the engram rather than from a message format.

There is no authorisation between nodes. Every node is the operator's own, so a message from one is a message from them, and asking a node something is asking a colleague rather than presenting credentials. Stated once so it is not discovered later: the mesh boundary is therefore the security boundary — anything inside can reach what any node can reach, which is what puts the whole perimeter on the token and the overlay (ADR 0007).

It can be switched off, and switched off it still answers. A node whose session is disabled replies saying so, at once, with no model involved — the queue is still read, and the state is the reply. That is deliberate and it is the same rule the host follows about a service that does not exist: absence must never be indistinguishable from a failure to answer. A node with nothing there is a silence somebody has to go and diagnose; a node that says I am switched off is not.

One per node, always, and it cannot be moved to another machine. Two and nothing decides which replies; none and the node is mute; moved, and one machine is answering as another.

It is not an employee (ADR 0003). Nobody hires it, it holds no tasks, it drains nothing and it is never reassigned — that vocabulary was written for workers and does not describe this. It exists because the node does, and it is gone when the node leaves.

How it joins

The host has one behaviour and two sources of declaration. What differs between the first node and the fiftieth is not what the host does but where the declaration comes from — and, as above, that is a situation rather than a class.

declaration comes from
no mesh reachable the pinned bundle the host carries
mesh reachable the control plane, over the link

The first node is not a different kind of node. It is a node whose mesh is not up yet. It applies the bundle it carries, the control plane comes up on top of it, and from that moment it takes declarations like everything else. Its specialness is temporary and self-erasing, which is what the hand-run bootstrap scripts never were.

A joining node does the minimum to be reachable and nothing else — an identity, an address, and one peer to reach. It does not compute the overlay: the whole peer set is derived centrally and pushed down.

That is also why the migration is smaller than it looked. The hard part of the overlay — every node's key, address, site and reachability — is only needed to compute the whole mesh, and a joining node needs one peer.

Everything reaching a node arrives one way, and four properties make that a boundary rather than a pipe.

It is outbound and node-initiated. The node dials the control plane; nothing dials a node. Not only defensive — most nodes sit behind a household connection with no forwarded port, so an inbound control channel would work for one node and not the rest, and the difference would be invisible until it mattered. A node has no listening control surface at all.

A node holds its own identity and nothing else. No shared secret, no credential to anything it does not own. Compromise of a node is compromise of that node — which the current arrangement does not have, because every node permanently holds the same database and object-store credentials, and there is no mechanism that rotates one and informs everything holding it.

What that identity is: a keypair the node generates

Written 2026-08-29. This is the same rule as the sentence above, and it had been treated as an open question for weeks because of a word.

The node generates a keypair. The private half never leaves the machine. The mesh records the public half. Ed25519, the same as the control plane's signing key, in the other direction: the mesh proves itself to a node by signing, and a node proves itself to the mesh by signing.

This was never open. 08-connectivity.md already says it of the overlay keys, in these words: each node generates its own keypair, the private key never leaves the machine, the public key is published to the mesh — and adds that this is ADR 0004's a node holds its own identity, applied. What was missing was applying it to the thing this record is about.

The word that caused it: the lifecycle says a joining node receives its own durable identity, which reads as the mesh issuing something, and then the question becomes issuing what. It does not issue anything. The node arrives holding its identity; what it receives is being known. Enrolment is the moment the mesh writes down a public key it will believe, and the one-time secret is what buys the right to have it written down.

Everything above then holds literally. Nothing is stored that could be stolen and replayed: the mesh's copy is a public key, so a copy of the mesh's database grants nothing. Compromise of a node is compromise of that node becomes true rather than aspirational, because the only secret on a machine is the one that identifies it.

What "connecting to the mesh" is, concretely

Written 2026-08-29, because it was asked and this record had never said it.

One outbound AMQP connection from the node to the broker, held open. That is all of it. There is no second connection and nothing is ever dialled at a node. Being in the mesh, operationally, means that connection is up; being disconnected means it is not (09-the-node-lifecycle.md).

Two different things ride on it, and conflating them is what made this confusing:

what it answers who issues it
an AMQP account may this connection be accepted at all the mesh, at enrolment
the node's keypair which node is speaking, on every message the node, above

The account is the mesh's to issue, and per node. The broker has to authenticate somebody before a connection exists, and a shared account would let any node consume another's queue — which is the shared-credential fault this record exists to remove, reappearing at the transport. So enrolment creates that node's account and hands it over, and it is rotatable without touching the node's identity.

The keypair is not made redundant by it. With only an account, the control plane would know which node is speaking because the broker says so — and that is the same transitive authority this record refuses in the other direction. A compromised broker could then attribute reports to whichever node it liked, and the control plane would act on them. Signing is what removes the broker from the question in both directions.

So a node holds two things after enrolment: a credential the mesh issued for reaching the broker, and a key it generated itself that the mesh only ever sees the public half of. Both are its own, neither reaches anything else, and compromise of a node is compromise of that node still holds.

Its own key, not the machine's SSH host key

Reusing the host key is the obvious economy and it is refused, for reasons that are operational rather than fastidious:

  • It is regenerated by ordinary events. A reinstall, an image cloned, ssh-keygen -A on a rebuild — each silently un-enrols the node, and the failure appears as an authentication problem with no cause anybody changed.
  • It is managed by something else. Its lifecycle belongs to the machine's SSH daemon, and an identity the mesh depends on should not rotate on a schedule the mesh does not know about.
  • Not every node has one. A partial host has no SSH daemon (ADR 0005), and an identity scheme that excludes a supported kind of node is not one.

The mesh should still know the host key — it knows every node, so it can distribute host keys the same way it distributes authorised keys (ADR 0006), and node-to-node SSH stops depending on trust-on-first-use. That is the good half of the idea, kept.

Authority is mutual. The node proves it may join, and the control plane proves it is the mesh. One-way is not enough: the host applies whatever the link delivers, so a node that cannot tell the mesh from something impersonating it will apply that something's declarations.

What may be pushed is bounded by form, not by trust. Declarations of known shape, never a command to run. Stated honestly, this bounds form and not impact: a compromised control plane can declare harmful state and the host will apply it faithfully, because that is what it is for. What the property buys is that the blast radius is describable — exactly what the declaration language can express, which can be reviewed. An arbitrary command channel has no such bound.

The enrolment token carries the mesh

Mutual authority needs the node to verify something before it trusts anything, and that is a circle: verifying the mesh needs the mesh's certificate authority, and obtaining one means trusting whoever hands it over. There is a second circle beside it — a node must reach the mesh before the mesh has configured it, so it can resolve no mesh name.

Both are the same shape: a node needs a fact about the mesh before it has any trustworthy way to obtain one. So that fact arrives by a path other than the mesh.

The token carries four things, and it is the only thing a joining node needs:

where the broker's address, not a name — there is no resolution yet, and this is why none is needed
what it is connecting to the fingerprint of the broker's certificate
who it will believe the control plane's signing identity
the right to join a one-time secret, useless once used and useless after it expires

Carried out of band, by the person adopting the machine. That is what breaks both circles: its authenticity comes from the channel it travelled, not from anything the node can check afterwards. Trust on first use, with the first use moved out of band — the difference between a pin and a guess.

The endpoint and the authority are two identities. A node connects to the broker and takes instruction from the control plane behind it. Pinning only the broker would make the control plane's authority transitive, and a compromised broker could then forge declarations — which, since the host applies whatever the link delivers, is the whole machine. So the transport is verified once at connect, and each declaration is verified by its signature, every time.

What this settles: the mesh's certificate authority is not a bootstrap concern — it certifies internal names once a node is a member. Nothing needs name resolution before the link. And nothing is placed on disk beforehand except the token, which is the first moment a node holds only its own identity becomes true rather than aspirational.

Consequences

  • Declarations must be signed, and the host must tell this is not from the mesh I joined apart from this is malformed. Rotating the signing identity is a fleet-wide operation with an overlapping rollover, and that is the cost of not trusting the broker.
  • The token becomes security-critical, because it carries the pin. Tampering substitutes the mesh — which is strictly better than the alternative, where there is nothing to tamper with and the node trusts the first answer unconditionally.
  • A rejoining node is ordinary. There is no long-lived secret to recover, so a node that lost its identity gets a new token.