jschoubben
5e83ac2c22
Jochen: a normal application has 3-5 ADRs, maybe 10 for a large one, and we are at 65. Fair, and the cause is mine -- I recorded every FINDING as a decision rather than every fork in the road. Two merges, both cases where one decision had been split across many records because it was taken over several days rather than at once. 0019 absorbs ten records about how this repository works: what it is and that it is public, the folder flow, the two design layers, the issue front door, status in frontmatter, playbooks, the naming rule, the product name. Those were never ten decisions -- they were one, seen from ten angles as the repository took shape. 0016 absorbs the five about the lab: a node is a virtual machine, a router is scenery, a scenario declares the underlay, a scenario is a closed address space, and the two scenario classes. Same pattern -- one design, split by the order it was worked out in. The consolidated 0019 also raises the bar for what earns a record, since that is what produced 65: a record is warranted when there is a genuine fork -- a direction reversed, an alternative that will be proposed again, something contested. A finding is not a decision, and a bug is certainly not. Everything else belongs in the design document where the reasoning is actually read. The checker earned its place here. Deleting nine records left 13 dangling links across the repository and it named every one, including in AGENTS.md. Nothing was found by reading. Remaining clusters worth the same treatment: the host (8 records), delivery (5), modules (6), connectivity (4), substrate and control plane (4). That would be 52 down to roughly 30.
117 lines
6.2 KiB
Markdown
117 lines
6.2 KiB
Markdown
---
|
|
layer: to-be
|
|
status: designed
|
|
code: [mesh-lab]
|
|
updated: 2026-08-24
|
|
decisions:
|
|
- 02-DECISIONS/0016-the-lab.md
|
|
- 02-DECISIONS/0008-a-failed-step-fails-the-job.md
|
|
---
|
|
|
|
# Installing the lab on a clean machine
|
|
|
|
The lab has prerequisites — a virtualisation daemon, copy-on-write storage, a pool, an identity
|
|
permitted to talk to it — and it cannot get them from the mesh, because it is where the mesh is
|
|
built ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)).
|
|
|
|
So the lab needs an install path of its own. This describes it, and the shape it has to take is
|
|
determined by two failures observed while measuring
|
|
([research 010](../../01-RESEARCH/010-lab-inner-loop-cost/measurements.md)).
|
|
|
|
## The two failures that shape this
|
|
|
|
**One: installed is not available.** The virtualisation package was present and explicitly
|
|
installed. Both its units were disabled, the operator was in no group, and the client reported
|
|
the server unreachable. Nothing had failed — the declaration was satisfied exactly as written
|
|
([`04-ISSUES/007`](../../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md)).
|
|
|
|
**Two, and worse: available is not adequate.** With the storage tooling absent, the daemon
|
|
offered one driver, everything worked, and snapshots took **seventy-six times longer** than they
|
|
needed to. Nothing failed. Nothing warned. A lab in that state runs correctly and is simply too
|
|
slow to use — and the inner loop it exists to provide quietly does not exist.
|
|
|
|
The second is the more dangerous shape and it is a variant the mesh has not catalogued before.
|
|
Its usual failure is *reported success and did nothing*. This is **reported success and did it
|
|
seventy-six times slower**, which no error surface catches because nothing is wrong.
|
|
|
|
## What follows: the lab verifies capability, never installation
|
|
|
|
The install path may differ. **The verification does not.**
|
|
|
|
Before the lab will raise anything, it asserts the outcomes it needs — not that packages are
|
|
present, but that the machine can actually do the work:
|
|
|
|
| Assertion | Failing means |
|
|
|---|---|
|
|
| the daemon answers **as the invoking user**, not as root | a group membership that was granted but never took effect |
|
|
| a copy-on-write storage driver is offered | the userspace tooling is missing; snapshots will be full copies |
|
|
| **the pool the lab will use is on that driver** | a pool exists, and is the slow kind — the failure that has no symptom |
|
|
| hardware virtualisation is present | machines will be emulated and unusably slow |
|
|
| an image can be fetched or is cached | the first raise will fail late instead of early |
|
|
|
|
Each check states **why it matters**, in the terms of what it costs. *"The pool uses the `dir`
|
|
driver"* means nothing to someone who does not already know it means seventy-six times slower
|
|
and unbounded at worst.
|
|
|
|
**The lab refuses to run degraded.** It does not warn and continue: a warning about a slow inner
|
|
loop is read once and ignored forever, and the loop stays slow. This is
|
|
[ADR 0008](../../02-DECISIONS/0008-a-failed-step-fails-the-job.md) applied where the failure is
|
|
performance rather than an error.
|
|
|
|
## Two ways the prerequisites arrive
|
|
|
|
**On a machine the mesh manages** — a module declares them, and a hook turns them into
|
|
capabilities: the units enabled, the group granted, the pool created on the right driver. This
|
|
already works; it is what was done for the virtualisation daemon itself.
|
|
|
|
**On a machine the mesh does not manage** — the lab's own bootstrap does it. One command, on a
|
|
clean machine, that installs what is missing and configures it.
|
|
|
|
The second path is not a convenience. It is **required**, because the lab must work before the
|
|
mesh does, and a lab that could only be installed by a mesh would be unable to host the
|
|
development of the mesh that installs it.
|
|
|
|
Both paths end at the same verification. Whoever satisfied the prerequisites, the lab checks
|
|
them itself — because the lesson of the first failure is precisely that *something else said it
|
|
was done* is not evidence.
|
|
|
|
## The lab is the second thing installed by hand
|
|
|
|
Worth stating, because it looks like an exception and is not.
|
|
|
|
The node host is the one thing installed by hand on a machine
|
|
([research 006](../../01-RESEARCH/006-mesh-from-scratch/code-skeleton.md)): everything else
|
|
arrives through it. The lab is the same shape on a workstation — installed once, by hand, and
|
|
then everything about the mesh is developed inside it.
|
|
|
|
Two bootstraps, at two levels, for the same reason: **something has to be first, and pretending
|
|
otherwise produces a circularity that gets papered over with a script nobody exercises.**
|
|
|
|
## What a clean install actually needs
|
|
|
|
In order, on a machine with nothing:
|
|
|
|
1. **A virtualisation daemon**, running, with its socket enabled.
|
|
2. **Copy-on-write storage tooling** — the kernel side is usually already present; it is the
|
|
userspace half that is missing and that decides whether the driver is offered at all.
|
|
3. **A pool on that driver.** A loop-backed file is sufficient and needs no partitioning, which
|
|
matters: requiring a dedicated filesystem would make the lab uninstallable on a machine
|
|
already in use.
|
|
4. **Group membership** for the operator — which does **not** apply to sessions that already
|
|
existed. Observed directly: a shell whose process tree predated the grant could not reach
|
|
the daemon while a fresh lookup showed the membership present. The bootstrap has to say so,
|
|
or the first thing a person meets is a permission error that looks like a broken install.
|
|
5. **Verification**, as above, before anything is raised.
|
|
|
|
## Open
|
|
|
|
- **Whether the lab's bootstrap may install packages at all**, given that the mesh's rules
|
|
forbid installing by hand. The resolution is probably that the lab's bootstrap *is* the
|
|
sanctioned mechanism on an unmanaged machine, in the way the mesh's own first-node script is —
|
|
but that is an argument to record, not to assume.
|
|
- **What "adequate" means numerically.** The checks above are qualitative. A snapshot-time
|
|
threshold would catch a copy-on-write pool that is slow for some other reason, and would be a
|
|
real assertion rather than a proxy.
|
|
- **Whether the lab should own its pool** rather than using an existing one. Owning it makes the
|
|
driver guaranteed; sharing it avoids duplicating storage on a machine that already has a pool.
|