jschoubben
bffd2af40c
Raising a first node hits them in order: the bundle's bus image named for a registry that is gone; the certificate made by openssl in an image that has none; enrolment dialling TLS at a bus that speaks first, then refusing its own token for the empty bus. Each was right until the bus changed and nothing has raised a foundation since. The fourth is not a patch: the installer carries the bus's first user list and the controller composes the rest through the bus being a module, and at genesis there is no module — so the first node cannot be let onto the bus it just raised.
74 lines
3.7 KiB
Markdown
74 lines
3.7 KiB
Markdown
---
|
|
status: located
|
|
opened: 2026-09-29
|
|
located-in: [mesh-host examples + internal/link, mesh-controller internal/broker]
|
|
---
|
|
|
|
# 146 — the foundation cannot be raised on the bus the mesh runs on
|
|
|
|
## What was observed
|
|
|
|
Raising a first node in the lab, to check a module against a real mesh, fails before any module is
|
|
reached. Two separate faults, in the two bundles that exist:
|
|
|
|
**The older bundle raises a control plane that cannot start.** It brings up the previous broker,
|
|
and the control plane it then starts says, once every few seconds, for ever:
|
|
|
|
```
|
|
mesh-controller: this control plane has no MESH_BUS_NATS, so it cannot reach the mesh's bus
|
|
```
|
|
|
|
That is the control plane being right. The mesh moved to one bus
|
|
([ADR 0131](../../02-DECISIONS/0131-everything-on-the-mesh-speaks-to-the-broker-seat.md)) and the
|
|
bundle did not. Every bed that raises a foundation raises this one, so every bed is in this state.
|
|
|
|
**The newer bundle, written for the new bus, stops one step earlier.** Its certificate step asks a
|
|
container to make the broker's certificate:
|
|
|
|
```
|
|
docker run --rm --entrypoint sh -v <the broker's tls volume>:/tls <the bus image> \
|
|
-c "test -f /tls/tls.crt || (openssl req -x509 ... )"
|
|
...
|
|
failed bus-certificate: running the action: docker exited 127
|
|
```
|
|
|
|
127 is *command not found*. The bus's image has a shell and no `openssl`; the previous broker's
|
|
image had both, which is why the step worked when it was written against that one. Substituting the
|
|
store's image — the only other image the bundle carries — does not help: it has no `openssl`
|
|
either. So the step as written cannot succeed with anything the bundle names, and the fault is not
|
|
one image's: **the bundle asks for a certificate to be made by a tool it never says must be there.**
|
|
|
|
Measured 2026-09-29 on a fresh lab machine, both bundles, from bare.
|
|
|
|
## Why this is here and not a note in the knowledge base
|
|
|
|
The mesh's own foundation is the one thing it cannot raise. Nothing reports that: the bundles are
|
|
files in a repository, nothing applies them but a person raising a node, and the last thing that
|
|
did was the hand-driven cut-over
|
|
([ADR 0131](../../02-DECISIONS/0131-everything-on-the-mesh-speaks-to-the-broker-seat.md), whose
|
|
work was done on the machines rather than from a bundle). So the state where the mesh cannot make
|
|
another one of itself is reachable, and was reached, without anything saying so.
|
|
|
|
It is also load-bearing for everything else: a lab bed proves a claim by raising a mesh, so while
|
|
this holds, **no bed can run**, and every "checked in the lab" written from now on is a promise
|
|
against a suite nobody can execute.
|
|
|
|
## What would have prevented it
|
|
|
|
- **Something raising the foundation on a schedule, from the bundle, as it is written** — the
|
|
bundle is the mesh's own installer and nothing installs from it. A bed that raises a first node
|
|
is exactly that check, and it is the bed that cannot run.
|
|
- **A step naming what it needs.** The certificate step names an image and assumes a program inside
|
|
it. An action that said which tool it requires would have failed at the declaration rather than
|
|
at 127 on a machine.
|
|
|
|
## Evidence to carry into diagnosis
|
|
|
|
- `mesh-host examples/foundation-first-node.lock` — the previous broker, no `MESH_BUS_NATS`.
|
|
- `mesh-host examples/foundation-first-node-nats.lock` — the new bus; `bus-certificate` and its
|
|
`verify` both run `openssl` in the bus's image.
|
|
- The bus image the bundle pins has `sh` and no `openssl`; the store's image likewise.
|
|
- The lab rewrites a bundle's registry-prefixed third-party references to upstream ones for a
|
|
machine with an uplink (`test/integration/harness.ts`); the new bundle's bus reference needed
|
|
that rule added, which is done and is not this issue.
|