The mesh runs its own registry, certifies its own names, and computes its own filtering

Issue 003 is answered in both halves: manifests are parsed strictly, and a
module says what it listens on and from where rather than carrying a key
nothing reads. The design records what was built and how each part is checked.

Issue 013 is new, found by reading while writing the first module that has
both a computed file and a service that needs it. The file arrived second.
It failed, then the next reconcile fixed it, which is why nothing caught it.
This commit is contained in:
2026-08-31 00:37:34 +02:00
parent ba14e2b629
commit 778efaba8b
4 changed files with 213 additions and 7 deletions
+48 -2
View File
@@ -1,8 +1,10 @@
--- ---
layer: to-be layer: to-be
status: designed status: designed
code: [] code:
updated: 2026-08-29 - mesh-control internal/catalogue/filtering.go
- mesh-host internal/apply (the service that reflects it)
updated: 2026-08-31
decisions: decisions:
- 02-DECISIONS/0005-the-node-host.md - 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md - 02-DECISIONS/0004-a-node-and-how-it-joins.md
@@ -275,6 +277,50 @@ from a wrong one, and costs more, because people believe it.*
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)), and ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)), and
the one manifests lack. `scope:` survived because nothing rejected it. the one manifests lack. `scope:` survived because nothing rejected it.
### What was built
*2026-08-31. Everything above was the intention; this is what exists, and how each part is
checked. [04-ISSUES/003](../../04-ISSUES/003-firewall-scope-is-read-by-no-code/00-report.md) is
resolved by it.*
**A module says what it listens on**, as a port, a protocol and a source — `mesh`, `anywhere`, or
`machine`. The source is required and there is no default, which is the whole of *a rule names its
source*: a manifest that omitted it would read as a restriction and be none. *Checked by a manifest
with a port and no source being refused, and by one naming a source the mesh cannot render being
refused as well — the second is what stops a source becoming a comment.*
**The set is derived per node**, from every module assigned to it, not from the module asking for
it. Where two modules want the same port, the wider source wins and both are still named, because
removing one of them must not read as a reason to close a port the other needs. *Checked by
rendering a node whose firewall module has no ports of its own and asserting another module's port
is in the result; and by giving one port two modules and one source each, and asserting the
narrower rule disappears while both names survive.*
**What is not declared is closed.** The rule set drops by default. *Checked by naming the input
chain in the assertion rather than the policy alone — the first version of that test passed while
input accepted everything, because another chain in the same file also said `policy drop`.*
**From the mesh means the machines the mesh has**, as their addresses on the private network, not
as a subnet. A subnet is a guess that stays wrong quietly; the address set shrinks when a node
leaves and nobody edits anything. A machine that asks for `mesh` where the mesh knows no addresses
is **closed and told so in the file** — widening it would open a port nobody asked to open, and
dropping it silently would close one somebody did.
**Three things it deliberately does not do**, each of which looked right and would have broken
something:
| | why not |
|---|---|
| decide what the machine **forwards** | the container runtime writes its own forwarding rules and a second policy is consulted alongside them, so a drop here stops every container on the node — the control plane included. Nothing in a manifest says what a machine routes, so there is nothing to derive it from either |
| **flush the ruleset** when loading | that empties every table on the machine, the runtime's among them. Only the mesh's own table is replaced, and it is declared empty first so the replacement works on a machine loading one for the first time |
| carry a **command to load itself** | the link may not carry an action ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). A service is declared to reflect the file instead, so replacing it restarts what loads it — the shape that rule leaves, used here for the first time for its real purpose |
**And it is enforced, which is what separates this from `scope:`.** Proven on two real machines:
two ports opened, one declared, and from the other machine the declared one answers and the
undeclared one does not — then the module is removed and the port closes with nobody editing a
rule. *A rule set that is written but never loaded passes every check that reads the file, which
is why the check reads packets.*
## 5 — Certificates ## 5 — Certificates
**Two authorities, kept separate on purpose.** **Two authorities, kept separate on purpose.**
+79 -1
View File
@@ -4,7 +4,9 @@ status: designed
code: code:
- mesh-control internal/builder - mesh-control internal/builder
- mesh-control internal/catalogue/build.go - mesh-control internal/catalogue/build.go
updated: 2026-08-30 - mesh-control internal/inventory/secrets.go
- mesh-control cmd/mesh-builder
updated: 2026-08-31
decisions: decisions:
- 02-DECISIONS/0009-modules-and-the-graph.md - 02-DECISIONS/0009-modules-and-the-graph.md
- 02-DECISIONS/0010-delivery.md - 02-DECISIONS/0010-delivery.md
@@ -108,6 +110,34 @@ Three properties of the builder that are decisions:
is not running, and those want completely different responses — the same rule the host follows is not running, and those want completely different responses — the same rule the host follows
about a service that does not exist about a service that does not exist
### And it is a module the mesh assigns
*2026-08-31. Written after `builder issue --node`, which is the part that makes the sentence
"holding its own credential" true rather than aspirational.*
A build machine is a machine that runs the builder, and there is exactly one honest way to say
which machines those are: **assign it**. So the builder is a module like any other — an image, a
container, a working directory, and a claim so a machine does not end up running two.
The one thing that could not be a module in the ordinary way is the credential. It is not
generated, because the broker has to have been told about it, and it is not written in a manifest,
because a manifest is public and the same file goes to every machine that ever runs it. So the
mesh **creates the account, seals the URL to the machine that will use it, and discards the
plaintext** — the "given, not generated" case above, and its first user.
Nothing is printed. A credential shown on a terminal is a credential in a scrollback buffer, and
the copy that matters would then exist in two places, one of which nobody is guarding.
**What this replaces:** a builder started by hand with whatever credential was to hand, which in
practice meant the broker's administrative account. *A program documented as holding its own
credential and given somebody else's is worse than one with no story at all* — the documentation
is what stops anybody checking.
*Checked in the lab by assigning it and then asking the mesh to build a module: the credential
file arrives readable only by that machine, names the scoped account rather than the broker's own,
and the build completes — which is the only proof the credential authenticates, because a
container that is up holding a credential it cannot use looks identical from outside.*
## What is kept ## What is kept
**Every result, including the failures.** A failed build that leaves no trace is indistinguishable **Every result, including the failures.** A failed build that leaves no trace is indistinguishable
@@ -167,6 +197,35 @@ Remade when the machine's sealing key changes. **Declared and not made is refuse
module whose own credential is silently absent starts, fails to authenticate, and the reason is module whose own credential is silently absent starts, fails to authenticate, and the reason is
three layers from the machine reporting it. three layers from the machine reporting it.
### Some of them the mesh cannot make
*2026-08-31, from making the builder a module — the first thing to hold one.*
A generated secret is the mesh's, and remaking it costs nothing: **nothing else ever knew the old
one.** That is the assumption the paragraph above rests on, and it is not true of every secret a
module needs.
A broker account's password exists because **the broker was told about it**. A licence key exists
because somebody bought it. The mesh's job with these is to carry the value to the machine that
will use it and then be unable to read it — the same sealing, from the other direction: **given,
not generated.**
Treating the two alike is wrong in exactly one place, and it is the place nobody looks. When a
machine rejoins it has a new sealing key, and everything sealed to the old one is remade. Remaking
a *given* secret puts thirty-two random bytes where a working credential was, and every visible
signal says it worked: the mesh sealed a secret, the machine applied it, the file is there with
the right permissions. What fails is a program authenticating to something else, hours later,
with an error that names neither the mesh nor the secret.
So **where the value came from is recorded, and a given secret is never regenerated.** A rejoined
machine asking for one is refused, naming the remedy — issue it again — because the remedy is a
command somebody runs and no amount of pushing will produce a password the broker has never heard
of.
*Checked by taking a given secret, changing the machine's sealing key, and asserting the mesh
refuses rather than answers; and by asserting that two ordinary pushes hand back the same value,
without which the refusal would be a secret that never survives at all.*
## What one assignment gets you ## What one assignment gets you
A database module, written to see whether it could be: A database module, written to see whether it could be:
@@ -203,3 +262,22 @@ access, or are not build output. None of that describes a digest-pinned archive,
second service for one kind of immutable blob is two things to run, two to back up, and two ways second service for one kind of immutable blob is two things to run, two to back up, and two ways
for an artifact to be missing. **Overturnable without touching anything else**: a manifest carries for an artifact to be missing. **Overturnable without touching anything else**: a manifest carries
a URL and a digest, and neither says what served it. a URL and a digest, and neither says what served it.
### And the mesh runs it
*2026-08-31.* Which registry is a **provision**, mesh-scoped: a build machine requires
`artifact-store` and is told where it is, the same way an application is told where its database
is. Nothing is configured with an address.
This closes the last thing the mesh depended on and did not run. The registry a bootstrap pulls
from belongs to whoever raised the machine; from the moment the mesh has one of its own, an
artifact's home is somewhere the mesh can move, replace and back up.
**The chicken and egg is the bootstrap's, resolved the same way.** A registry module is an
`upstream` artifact — mirrored from a registry that already exists into the one being started. The
first copy comes from outside, exactly once, and every copy after it is the mesh's.
*Checked in the lab by assigning it and then asking for `/v2/` — on the machine, and from a second
machine across the private network, because a mesh-scoped provision that only answers locally is
not one. A container that is running is not a registry that replies, and this project has paid for
that distinction once already.*
@@ -1,9 +1,9 @@
--- ---
status: open status: resolved
opened: 2026-08-22 opened: 2026-08-22
located-in: [] located-in: [mesh-control]
fixed-by: fixed-by: mesh-control — a machine's filtering is computed from what it was assigned
amended-design: amended-design: 03-DESIGN/01-to-be/08-connectivity.md
--- ---
# 003 — A firewall rule's `scope:` is read by no code # 003 — A firewall rule's `scope:` is read by no code
@@ -38,3 +38,29 @@ any check.
- Were the five declarations intended to restrict something that is currently open? Each needs - Were the five declarations intended to restrict something that is currently open? Each needs
checking against what the node actually exposes — the declaration cannot be trusted either checking against what the node actually exposes — the declaration cannot be trusted either
way. way.
## How it is answered
*2026-08-31.* Both halves, in Novox Mesh. HAL keeps the fault until its provisioning is switched
off, which is what this issue is now waiting on rather than a fix of its own — patching `scope:`
into something that works would mean implementing it twice, in the system being replaced.
**The unknown key.** A manifest is parsed strictly: an unknown key is refused with the key named,
the discipline the host's declaration parser has always had. `scope:` would not survive being
written today, and neither would a misspelling of anything else. This is the general fix — the
issue's own observation was that *any* invented key behaved this way, and that the one instance
was found by reading rather than by any check.
**The rule that restricts nothing.** `scope:` is not reimplemented. A module says what it listens
on and **who may reach it**, and saying from where is required rather than defaulted: a rule with
no source is open, and must say so rather than appear to restrict something. A machine's whole
rule set is then derived from every module assigned to it — so there is no second list to keep in
step, which is the condition that let the first one drift out of use unnoticed.
**And it is enforced, which is the part that makes this different from before.** The mesh renders
the rule set; a service on the node is declared to reflect that file, so replacing it restarts
what loads it. Proven in the lab against two real ports on a real machine: the declared one
answers from another machine, the undeclared one does not, and removing the module that wanted the
port closes it with nobody editing a rule.
The design is [`03-DESIGN/01-to-be/08-connectivity.md`](../../03-DESIGN/01-to-be/08-connectivity.md) §4.
@@ -0,0 +1,56 @@
---
status: resolved
opened: 2026-08-31
located-in: [mesh-control]
fixed-by: mesh-control — what the mesh computes is applied before what the module declared
amended-design:
---
# 013 — A file the mesh computes arrives after the service that needs it
## Symptom
Everything the control plane computes for a module — a certificate, a sealed credential, a bound
file, a rule set — was placed **after** that module's own resources in the declaration. The host
applies resources in the order it is given and
[does not sort](../../02-DECISIONS/0005-the-node-host.md), so a service or container declared in a
manifest was applied **before** the file it depends on existed.
On the first apply the service starts against a missing file and fails. The next reconcile finds
the file there and starts it.
## Why this matters
**It repairs itself, which is why nothing caught it.** A fault that is gone by the second attempt
is worse than one that persists: what gets remembered is that the thing works, and the failed
first apply is read as a machine that was briefly slow. The mesh reports a failure, then reports
success, and nobody looks again.
It was also invisible to every test that existed, because none of them combined the two halves.
Modules with computed files declared no service; modules with a service needed no computed file.
The fault lived exactly in the gap between two repositories' assumptions — the control plane
deciding an order, the host promising not to change it — which is the shape this folder exists for.
Found by reading, while writing the first module that has both: a firewall whose service must
reflect a rule set the mesh computes.
## Evidence
`internal/catalogue/declaration.go` built each module's resource list as
`append(module's own, computed...)` in six places — certificate, authority, needs, secrets, grants,
bindings. `internal/apply/apply.go` iterates `d.Resources` in order, and
`internal/declaration/declaration_test.go` states the rule directly: *order is stated, not derived.
The host must not sort.*
## What was done
The computed resources are assembled first and the module's own resources follow. Nothing the mesh
computes is derived from a module's resources, so the order is unconditionally right rather than a
heuristic — there is no case where a module's resource must precede a file the mesh made for it.
Merged after the computed-resources branch, which replaces a module's resources wholesale and
would otherwise discard everything the mesh had made for it.
*Checked by a module declaring a service that reflects a rule set, asserting the rule set is first;
and by a module whose resources are computed elsewhere, asserting its credential survives and is
still first — the case the merge point exists for.*