The mesh runs its own registry, certifies its own names, and computes its own filtering
Issue 003 is answered in both halves: manifests are parsed strictly, and a module says what it listens on and from where rather than carrying a key nothing reads. The design records what was built and how each part is checked. Issue 013 is new, found by reading while writing the first module that has both a computed file and a service that needs it. The file arrived second. It failed, then the next reconcile fixed it, which is why nothing caught it.
This commit is contained in:
@@ -1,8 +1,10 @@
|
|||||||
---
|
---
|
||||||
layer: to-be
|
layer: to-be
|
||||||
status: designed
|
status: designed
|
||||||
code: []
|
code:
|
||||||
updated: 2026-08-29
|
- mesh-control internal/catalogue/filtering.go
|
||||||
|
- mesh-host internal/apply (the service that reflects it)
|
||||||
|
updated: 2026-08-31
|
||||||
decisions:
|
decisions:
|
||||||
- 02-DECISIONS/0005-the-node-host.md
|
- 02-DECISIONS/0005-the-node-host.md
|
||||||
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
||||||
@@ -275,6 +277,50 @@ from a wrong one, and costs more, because people believe it.*
|
|||||||
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)), and
|
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)), and
|
||||||
the one manifests lack. `scope:` survived because nothing rejected it.
|
the one manifests lack. `scope:` survived because nothing rejected it.
|
||||||
|
|
||||||
|
### What was built
|
||||||
|
|
||||||
|
*2026-08-31. Everything above was the intention; this is what exists, and how each part is
|
||||||
|
checked. [04-ISSUES/003](../../04-ISSUES/003-firewall-scope-is-read-by-no-code/00-report.md) is
|
||||||
|
resolved by it.*
|
||||||
|
|
||||||
|
**A module says what it listens on**, as a port, a protocol and a source — `mesh`, `anywhere`, or
|
||||||
|
`machine`. The source is required and there is no default, which is the whole of *a rule names its
|
||||||
|
source*: a manifest that omitted it would read as a restriction and be none. *Checked by a manifest
|
||||||
|
with a port and no source being refused, and by one naming a source the mesh cannot render being
|
||||||
|
refused as well — the second is what stops a source becoming a comment.*
|
||||||
|
|
||||||
|
**The set is derived per node**, from every module assigned to it, not from the module asking for
|
||||||
|
it. Where two modules want the same port, the wider source wins and both are still named, because
|
||||||
|
removing one of them must not read as a reason to close a port the other needs. *Checked by
|
||||||
|
rendering a node whose firewall module has no ports of its own and asserting another module's port
|
||||||
|
is in the result; and by giving one port two modules and one source each, and asserting the
|
||||||
|
narrower rule disappears while both names survive.*
|
||||||
|
|
||||||
|
**What is not declared is closed.** The rule set drops by default. *Checked by naming the input
|
||||||
|
chain in the assertion rather than the policy alone — the first version of that test passed while
|
||||||
|
input accepted everything, because another chain in the same file also said `policy drop`.*
|
||||||
|
|
||||||
|
**From the mesh means the machines the mesh has**, as their addresses on the private network, not
|
||||||
|
as a subnet. A subnet is a guess that stays wrong quietly; the address set shrinks when a node
|
||||||
|
leaves and nobody edits anything. A machine that asks for `mesh` where the mesh knows no addresses
|
||||||
|
is **closed and told so in the file** — widening it would open a port nobody asked to open, and
|
||||||
|
dropping it silently would close one somebody did.
|
||||||
|
|
||||||
|
**Three things it deliberately does not do**, each of which looked right and would have broken
|
||||||
|
something:
|
||||||
|
|
||||||
|
| | why not |
|
||||||
|
|---|---|
|
||||||
|
| decide what the machine **forwards** | the container runtime writes its own forwarding rules and a second policy is consulted alongside them, so a drop here stops every container on the node — the control plane included. Nothing in a manifest says what a machine routes, so there is nothing to derive it from either |
|
||||||
|
| **flush the ruleset** when loading | that empties every table on the machine, the runtime's among them. Only the mesh's own table is replaced, and it is declared empty first so the replacement works on a machine loading one for the first time |
|
||||||
|
| carry a **command to load itself** | the link may not carry an action ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). A service is declared to reflect the file instead, so replacing it restarts what loads it — the shape that rule leaves, used here for the first time for its real purpose |
|
||||||
|
|
||||||
|
**And it is enforced, which is what separates this from `scope:`.** Proven on two real machines:
|
||||||
|
two ports opened, one declared, and from the other machine the declared one answers and the
|
||||||
|
undeclared one does not — then the module is removed and the port closes with nobody editing a
|
||||||
|
rule. *A rule set that is written but never loaded passes every check that reads the file, which
|
||||||
|
is why the check reads packets.*
|
||||||
|
|
||||||
## 5 — Certificates
|
## 5 — Certificates
|
||||||
|
|
||||||
**Two authorities, kept separate on purpose.**
|
**Two authorities, kept separate on purpose.**
|
||||||
|
|||||||
@@ -4,7 +4,9 @@ status: designed
|
|||||||
code:
|
code:
|
||||||
- mesh-control internal/builder
|
- mesh-control internal/builder
|
||||||
- mesh-control internal/catalogue/build.go
|
- mesh-control internal/catalogue/build.go
|
||||||
updated: 2026-08-30
|
- mesh-control internal/inventory/secrets.go
|
||||||
|
- mesh-control cmd/mesh-builder
|
||||||
|
updated: 2026-08-31
|
||||||
decisions:
|
decisions:
|
||||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||||
- 02-DECISIONS/0010-delivery.md
|
- 02-DECISIONS/0010-delivery.md
|
||||||
@@ -108,6 +110,34 @@ Three properties of the builder that are decisions:
|
|||||||
is not running, and those want completely different responses — the same rule the host follows
|
is not running, and those want completely different responses — the same rule the host follows
|
||||||
about a service that does not exist
|
about a service that does not exist
|
||||||
|
|
||||||
|
### And it is a module the mesh assigns
|
||||||
|
|
||||||
|
*2026-08-31. Written after `builder issue --node`, which is the part that makes the sentence
|
||||||
|
"holding its own credential" true rather than aspirational.*
|
||||||
|
|
||||||
|
A build machine is a machine that runs the builder, and there is exactly one honest way to say
|
||||||
|
which machines those are: **assign it**. So the builder is a module like any other — an image, a
|
||||||
|
container, a working directory, and a claim so a machine does not end up running two.
|
||||||
|
|
||||||
|
The one thing that could not be a module in the ordinary way is the credential. It is not
|
||||||
|
generated, because the broker has to have been told about it, and it is not written in a manifest,
|
||||||
|
because a manifest is public and the same file goes to every machine that ever runs it. So the
|
||||||
|
mesh **creates the account, seals the URL to the machine that will use it, and discards the
|
||||||
|
plaintext** — the "given, not generated" case above, and its first user.
|
||||||
|
|
||||||
|
Nothing is printed. A credential shown on a terminal is a credential in a scrollback buffer, and
|
||||||
|
the copy that matters would then exist in two places, one of which nobody is guarding.
|
||||||
|
|
||||||
|
**What this replaces:** a builder started by hand with whatever credential was to hand, which in
|
||||||
|
practice meant the broker's administrative account. *A program documented as holding its own
|
||||||
|
credential and given somebody else's is worse than one with no story at all* — the documentation
|
||||||
|
is what stops anybody checking.
|
||||||
|
|
||||||
|
*Checked in the lab by assigning it and then asking the mesh to build a module: the credential
|
||||||
|
file arrives readable only by that machine, names the scoped account rather than the broker's own,
|
||||||
|
and the build completes — which is the only proof the credential authenticates, because a
|
||||||
|
container that is up holding a credential it cannot use looks identical from outside.*
|
||||||
|
|
||||||
## What is kept
|
## What is kept
|
||||||
|
|
||||||
**Every result, including the failures.** A failed build that leaves no trace is indistinguishable
|
**Every result, including the failures.** A failed build that leaves no trace is indistinguishable
|
||||||
@@ -167,6 +197,35 @@ Remade when the machine's sealing key changes. **Declared and not made is refuse
|
|||||||
module whose own credential is silently absent starts, fails to authenticate, and the reason is
|
module whose own credential is silently absent starts, fails to authenticate, and the reason is
|
||||||
three layers from the machine reporting it.
|
three layers from the machine reporting it.
|
||||||
|
|
||||||
|
### Some of them the mesh cannot make
|
||||||
|
|
||||||
|
*2026-08-31, from making the builder a module — the first thing to hold one.*
|
||||||
|
|
||||||
|
A generated secret is the mesh's, and remaking it costs nothing: **nothing else ever knew the old
|
||||||
|
one.** That is the assumption the paragraph above rests on, and it is not true of every secret a
|
||||||
|
module needs.
|
||||||
|
|
||||||
|
A broker account's password exists because **the broker was told about it**. A licence key exists
|
||||||
|
because somebody bought it. The mesh's job with these is to carry the value to the machine that
|
||||||
|
will use it and then be unable to read it — the same sealing, from the other direction: **given,
|
||||||
|
not generated.**
|
||||||
|
|
||||||
|
Treating the two alike is wrong in exactly one place, and it is the place nobody looks. When a
|
||||||
|
machine rejoins it has a new sealing key, and everything sealed to the old one is remade. Remaking
|
||||||
|
a *given* secret puts thirty-two random bytes where a working credential was, and every visible
|
||||||
|
signal says it worked: the mesh sealed a secret, the machine applied it, the file is there with
|
||||||
|
the right permissions. What fails is a program authenticating to something else, hours later,
|
||||||
|
with an error that names neither the mesh nor the secret.
|
||||||
|
|
||||||
|
So **where the value came from is recorded, and a given secret is never regenerated.** A rejoined
|
||||||
|
machine asking for one is refused, naming the remedy — issue it again — because the remedy is a
|
||||||
|
command somebody runs and no amount of pushing will produce a password the broker has never heard
|
||||||
|
of.
|
||||||
|
|
||||||
|
*Checked by taking a given secret, changing the machine's sealing key, and asserting the mesh
|
||||||
|
refuses rather than answers; and by asserting that two ordinary pushes hand back the same value,
|
||||||
|
without which the refusal would be a secret that never survives at all.*
|
||||||
|
|
||||||
## What one assignment gets you
|
## What one assignment gets you
|
||||||
|
|
||||||
A database module, written to see whether it could be:
|
A database module, written to see whether it could be:
|
||||||
@@ -203,3 +262,22 @@ access, or are not build output. None of that describes a digest-pinned archive,
|
|||||||
second service for one kind of immutable blob is two things to run, two to back up, and two ways
|
second service for one kind of immutable blob is two things to run, two to back up, and two ways
|
||||||
for an artifact to be missing. **Overturnable without touching anything else**: a manifest carries
|
for an artifact to be missing. **Overturnable without touching anything else**: a manifest carries
|
||||||
a URL and a digest, and neither says what served it.
|
a URL and a digest, and neither says what served it.
|
||||||
|
|
||||||
|
### And the mesh runs it
|
||||||
|
|
||||||
|
*2026-08-31.* Which registry is a **provision**, mesh-scoped: a build machine requires
|
||||||
|
`artifact-store` and is told where it is, the same way an application is told where its database
|
||||||
|
is. Nothing is configured with an address.
|
||||||
|
|
||||||
|
This closes the last thing the mesh depended on and did not run. The registry a bootstrap pulls
|
||||||
|
from belongs to whoever raised the machine; from the moment the mesh has one of its own, an
|
||||||
|
artifact's home is somewhere the mesh can move, replace and back up.
|
||||||
|
|
||||||
|
**The chicken and egg is the bootstrap's, resolved the same way.** A registry module is an
|
||||||
|
`upstream` artifact — mirrored from a registry that already exists into the one being started. The
|
||||||
|
first copy comes from outside, exactly once, and every copy after it is the mesh's.
|
||||||
|
|
||||||
|
*Checked in the lab by assigning it and then asking for `/v2/` — on the machine, and from a second
|
||||||
|
machine across the private network, because a mesh-scoped provision that only answers locally is
|
||||||
|
not one. A container that is running is not a registry that replies, and this project has paid for
|
||||||
|
that distinction once already.*
|
||||||
|
|||||||
@@ -1,9 +1,9 @@
|
|||||||
---
|
---
|
||||||
status: open
|
status: resolved
|
||||||
opened: 2026-08-22
|
opened: 2026-08-22
|
||||||
located-in: []
|
located-in: [mesh-control]
|
||||||
fixed-by:
|
fixed-by: mesh-control — a machine's filtering is computed from what it was assigned
|
||||||
amended-design:
|
amended-design: 03-DESIGN/01-to-be/08-connectivity.md
|
||||||
---
|
---
|
||||||
|
|
||||||
# 003 — A firewall rule's `scope:` is read by no code
|
# 003 — A firewall rule's `scope:` is read by no code
|
||||||
@@ -38,3 +38,29 @@ any check.
|
|||||||
- Were the five declarations intended to restrict something that is currently open? Each needs
|
- Were the five declarations intended to restrict something that is currently open? Each needs
|
||||||
checking against what the node actually exposes — the declaration cannot be trusted either
|
checking against what the node actually exposes — the declaration cannot be trusted either
|
||||||
way.
|
way.
|
||||||
|
|
||||||
|
## How it is answered
|
||||||
|
|
||||||
|
*2026-08-31.* Both halves, in Novox Mesh. HAL keeps the fault until its provisioning is switched
|
||||||
|
off, which is what this issue is now waiting on rather than a fix of its own — patching `scope:`
|
||||||
|
into something that works would mean implementing it twice, in the system being replaced.
|
||||||
|
|
||||||
|
**The unknown key.** A manifest is parsed strictly: an unknown key is refused with the key named,
|
||||||
|
the discipline the host's declaration parser has always had. `scope:` would not survive being
|
||||||
|
written today, and neither would a misspelling of anything else. This is the general fix — the
|
||||||
|
issue's own observation was that *any* invented key behaved this way, and that the one instance
|
||||||
|
was found by reading rather than by any check.
|
||||||
|
|
||||||
|
**The rule that restricts nothing.** `scope:` is not reimplemented. A module says what it listens
|
||||||
|
on and **who may reach it**, and saying from where is required rather than defaulted: a rule with
|
||||||
|
no source is open, and must say so rather than appear to restrict something. A machine's whole
|
||||||
|
rule set is then derived from every module assigned to it — so there is no second list to keep in
|
||||||
|
step, which is the condition that let the first one drift out of use unnoticed.
|
||||||
|
|
||||||
|
**And it is enforced, which is the part that makes this different from before.** The mesh renders
|
||||||
|
the rule set; a service on the node is declared to reflect that file, so replacing it restarts
|
||||||
|
what loads it. Proven in the lab against two real ports on a real machine: the declared one
|
||||||
|
answers from another machine, the undeclared one does not, and removing the module that wanted the
|
||||||
|
port closes it with nobody editing a rule.
|
||||||
|
|
||||||
|
The design is [`03-DESIGN/01-to-be/08-connectivity.md`](../../03-DESIGN/01-to-be/08-connectivity.md) §4.
|
||||||
|
|||||||
@@ -0,0 +1,56 @@
|
|||||||
|
---
|
||||||
|
status: resolved
|
||||||
|
opened: 2026-08-31
|
||||||
|
located-in: [mesh-control]
|
||||||
|
fixed-by: mesh-control — what the mesh computes is applied before what the module declared
|
||||||
|
amended-design:
|
||||||
|
---
|
||||||
|
|
||||||
|
# 013 — A file the mesh computes arrives after the service that needs it
|
||||||
|
|
||||||
|
## Symptom
|
||||||
|
|
||||||
|
Everything the control plane computes for a module — a certificate, a sealed credential, a bound
|
||||||
|
file, a rule set — was placed **after** that module's own resources in the declaration. The host
|
||||||
|
applies resources in the order it is given and
|
||||||
|
[does not sort](../../02-DECISIONS/0005-the-node-host.md), so a service or container declared in a
|
||||||
|
manifest was applied **before** the file it depends on existed.
|
||||||
|
|
||||||
|
On the first apply the service starts against a missing file and fails. The next reconcile finds
|
||||||
|
the file there and starts it.
|
||||||
|
|
||||||
|
## Why this matters
|
||||||
|
|
||||||
|
**It repairs itself, which is why nothing caught it.** A fault that is gone by the second attempt
|
||||||
|
is worse than one that persists: what gets remembered is that the thing works, and the failed
|
||||||
|
first apply is read as a machine that was briefly slow. The mesh reports a failure, then reports
|
||||||
|
success, and nobody looks again.
|
||||||
|
|
||||||
|
It was also invisible to every test that existed, because none of them combined the two halves.
|
||||||
|
Modules with computed files declared no service; modules with a service needed no computed file.
|
||||||
|
The fault lived exactly in the gap between two repositories' assumptions — the control plane
|
||||||
|
deciding an order, the host promising not to change it — which is the shape this folder exists for.
|
||||||
|
|
||||||
|
Found by reading, while writing the first module that has both: a firewall whose service must
|
||||||
|
reflect a rule set the mesh computes.
|
||||||
|
|
||||||
|
## Evidence
|
||||||
|
|
||||||
|
`internal/catalogue/declaration.go` built each module's resource list as
|
||||||
|
`append(module's own, computed...)` in six places — certificate, authority, needs, secrets, grants,
|
||||||
|
bindings. `internal/apply/apply.go` iterates `d.Resources` in order, and
|
||||||
|
`internal/declaration/declaration_test.go` states the rule directly: *order is stated, not derived.
|
||||||
|
The host must not sort.*
|
||||||
|
|
||||||
|
## What was done
|
||||||
|
|
||||||
|
The computed resources are assembled first and the module's own resources follow. Nothing the mesh
|
||||||
|
computes is derived from a module's resources, so the order is unconditionally right rather than a
|
||||||
|
heuristic — there is no case where a module's resource must precede a file the mesh made for it.
|
||||||
|
|
||||||
|
Merged after the computed-resources branch, which replaces a module's resources wholesale and
|
||||||
|
would otherwise discard everything the mesh had made for it.
|
||||||
|
|
||||||
|
*Checked by a module declaring a service that reflects a rule set, asserting the rule set is first;
|
||||||
|
and by a module whose resources are computed elsewhere, asserting its credential survives and is
|
||||||
|
still first — the case the merge point exists for.*
|
||||||
Reference in New Issue
Block a user