diff --git a/02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md b/02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md new file mode 100644 index 0000000..a520bb3 --- /dev/null +++ b/02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md @@ -0,0 +1,154 @@ +--- +topic: what runs on it +status: accepted +date: 2026-09-28 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0045-a-machine-firewall-is-the-sum-of-what-it-listens-on.md +--- + +# 138. An assignment binds an endpoint and says how far it reaches + +## Context + +[ADR 0045](0045-a-machine-firewall-is-the-sum-of-what-it-listens-on.md) settled that a machine's +packet filter is derived from what its modules declare they listen on, and that the `from` of a +listen "is the whole of public-versus-internal". That was true of the packet filter, and it turned +out to be true of nothing else. + +Reachability is now settled three times, in three places, by three mechanisms that cannot disagree +out loud ([issue 140](../04-ISSUES/140-an-endpoints-reach-is-not-declared/00-report.md)): + +- **The filter** reads a listen's source, and a per-node setting may override it. That setting has + exactly one caller in the control plane — the function that builds the node's rules. +- **The names** come from a route contribution, which names a label and a port and says nothing + about reach. The reverse proxy composes a **public** name and an **internal** name for every route + it is given, because it can. +- **The certificate authority** follows from which names exist. Measured on the control-node: an + identity provider carries a public certificate valid 90 days and an internal one valid 24 hours and + renewed daily. No assignment asked for either. + +So *this endpoint must not be public* cannot be written. It is therefore enforced by nothing, while a +public certificate for that very name is obtained automatically — the fault +[how-we-build.md](../00-META/how-we-build.md) names, an unenforced rule being indistinguishable from +a wrong one, with the additional cost that the wrong thing is done eagerly. + +And a port that is not routed cannot be spoken about at all beyond the filter. The forge serves git +over ssh; that endpoint has no name, no certificate and no way to be called public except a key only +the filter reads. + +**Two per-node settings already exist and are half of this.** One gives a module's declared port a +machine port. One overrides a declared port's source. They key on port numbers, so nothing ties a +port to the route that serves it: a route contribution names a port too, and the two are equal only +by coincidence. + +**Where this belongs is already decided.** [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md) +says a module's configuration is its assignments. Whether the forge answers git-over-ssh from the +public internet is a fact about one installation and one machine, not a property of the software — +and [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md) already refuses an +installation's decisions in a definition. + +## Considered Options + +1. **Leave reach in the manifest, as `from` today.** Rejected: it is an installation's decision + written into the definition, and it cannot differ between two machines running the same module — + which is exactly the case the forge presents. +2. **Extend the existing source override to the names and the certificate, without naming + endpoints.** Rejected: it keys on a port number. A module's route contribution names a port as + well, and nothing says the two are the same thing, so one statement cannot be made to reach all + three mechanisms. Naming the endpoint is what makes that possible. +3. **Derive reach from whether the node has a public domain recorded.** Rejected: that is a property + of the machine, and two endpoints on one machine differ — a database and a web front end on the + same host. +4. **A fourth reach for "public name, internal authority"** — a name that resolves publicly and must + not appear in a public issuance log, obtained by DNS-01. Deferred, not rejected: it is a real case + and it is a question about which challenge an authority uses, not about how far an endpoint + reaches. Left to the certificate work as an open question. +5. **Make the manifest silent on reach and require every assignment to state it.** Rejected for the + transition: every endpoint reachable today would close until an assignment named it, which is a + flag day across the whole catalogue. + +## Decision + +**A module declares named endpoints.** An endpoint is one port the module serves, with a name the +module chooses, its protocol, and what it is for. A route contribution **names the endpoint it +routes** rather than repeating a port number. The manifest says what the module serves and what it +would serve it to by default; it does not say what this installation does with it. + +**An assignment binds each endpoint and says how far it reaches.** Per node: the machine port the +endpoint is published on, and its **reach** — one of `internal`, `public` or `both`. An assignment +that states nothing keeps the manifest's default, so no machine changes until an assignment says so. + +**Reach means all three mechanisms at once, and is the only thing that decides them.** + +- `internal` — the filter opens the machine port to the private network; the proxy serves the + internal name and not the public one; the certificate comes from the mesh's own authority. +- `public` — the filter opens it to anywhere; the proxy serves the public name; the certificate + comes from the public authority. +- `both` — both names, each from its own authority, and the filter opens to anywhere. + +**An endpoint that is not routed is reached but never named.** An endpoint with no route contribution +yields filter rules and nothing else: no name is composed and no certificate is requested. Git over +ssh is that case, and it is the case the model could not express. + +**The authority stops being chosen by which names happen to exist.** The proxy composes the names the +assignments asked for, and asks each name's own authority for it. A name nobody asked for is not +composed, so it is not certified. + +**The two existing settings are this, completed.** The per-node port mapping becomes the endpoint's +binding. The per-node source override becomes its reach, widened from the filter alone to the names +and the certificate as well. + +## Consequences + +- **A manifest gains endpoint names, and a route contribution names an endpoint instead of a port.** + Every routed module's manifest changes. The word ships one release before any manifest uses it, and + reaches the build machine and the control plane first. +- **One derived value is read by three things** — the filter's rules, the proxy's contributions, the + certificate request — so they can no longer disagree, and a disagreement becomes a refusal at the + assignment rather than a surprise on a machine. +- **A name that must not be public becomes writable, and therefore checkable.** It also gives + [issue 129](../04-ISSUES/129-nothing-makes-a-machine-trust-the-meshs-authority/00-report.md) a + declared answer to read: which endpoints are internal is what says whose root must be installed + where. +- **[Issue 139](../04-ISSUES/139-an-internal-route-name-resolves-to-the-consumers-node/00-report.md) + becomes answerable**: the endpoint's assignment names the machine that serves it, which is the fact + the internal name should be composed from. +- **Reach becomes reportable.** The mesh can say, per endpoint, where it is reachable from and which + authority holds its certificate — neither of which `status` can say today. +- **This narrows [ADR 0045](0045-a-machine-firewall-is-the-sum-of-what-it-listens-on.md).** Its + decision stands: the firewall is derived and host-applied, not a provider. What no longer holds is + that a listen's `from` is the whole of public-versus-internal; it is the filter's share of a + statement that also governs names and certificates. +- **What got harder:** every endpoint needs a name, including a module that serves exactly one port + and had no reason to name it. And an installation that wants a module public must now say so on the + assignment rather than inheriting it from the definition, which is more to say and the reason it is + right. + +## How it is checked + +- **One module, two endpoints, different reach.** A module declaring an internal endpoint and a + public one renders a filter opening one to the private network and one to anywhere, asserted per + chain body so a rule in the wrong chain cannot pass. +- **The names follow the reach.** The same module's routed endpoint composes the internal name only + when internal, the public name only when public, and both when both — and a certificate is + requested from the matching authority for each name composed and for no other. This fails against + the previous behaviour, where both names and both certificates are always composed, which is how + it is written. +- **An unrouted endpoint is filtered and never named.** Asserted for an endpoint with reach and no + route contribution: rules rendered, no contribution, no certificate request. +- **An assignment naming an endpoint the module does not declare is refused where it is said**, as is + a reach that is not one of the three — before it reaches a machine, because a ruleset that does not + load is a machine filtering nothing. +- **An assignment that states nothing renders byte-identically to today**, so every machine already + converged is untouched until its assignment says otherwise. + +## References + +- [ADR 0045](0045-a-machine-firewall-is-the-sum-of-what-it-listens-on.md) — narrowed here +- [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md) — where reach belongs +- [ADR 0066](0066-public-routing-is-name-agnostic.md) — the public name this composes +- [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md) — why reach is not a definition's +- [issue 140](../04-ISSUES/140-an-endpoints-reach-is-not-declared/00-report.md) — the measurement +- [issue 139](../04-ISSUES/139-an-internal-route-name-resolves-to-the-consumers-node/00-report.md), + [issue 129](../04-ISSUES/129-nothing-makes-a-machine-trust-the-meshs-authority/00-report.md) diff --git a/02-DECISIONS/0139-a-network-is-forwarded-because-a-module-declared-it.md b/02-DECISIONS/0139-a-network-is-forwarded-because-a-module-declared-it.md new file mode 100644 index 0000000..6b39398 --- /dev/null +++ b/02-DECISIONS/0139-a-network-is-forwarded-because-a-module-declared-it.md @@ -0,0 +1,136 @@ +--- +topic: what runs on it +status: accepted +date: 2026-09-28 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0137-a-machine-says-which-networks-it-routes.md +--- + +# 139. A network is forwarded because a module declared it + +## Context + +[ADR 0137](0137-a-machine-says-which-networks-it-routes.md), decided the same week, gave a machine a +way to say which networks it routes for its guests. It was written because the derived filter's +forward chain allowed two ranges named as constants in the control plane's source — the container +runtime's bridge pool, and part of the pool its compose files are given — with a comment admitting +the gap: *a machine whose runtime is configured with something else needs this to say so, which is a +thing the mesh cannot derive.* + +**It can be derived, and from the right place.** Measured on the last machine still to be converged +([issue 141](../04-ISSUES/141-the-forward-chain-does-not-follow-the-modules/00-report.md)): twenty-one +container networks, nine inside the runtime's bridge pool, twelve in the other private range, and six +of those outside the constant's lower bound — so the flip would have cut their guests off exactly as +it did on the workstation that produced 0137. + +Naming a range to cover the six is what 0137 provides for, and it is the wrong instrument. Of those +six networks, **four are networks the mesh's own modules declare**, present as network resources in +the node's plan and created by the host because a module asked for them. **Two are the predecessor's +leftovers** — compose networks of services the mesh does not run. Any range wide enough to keep the +four forwards the two as well: a firewall widened by hand to protect networks that should not exist. + +The mesh already knows which of the twenty-one are its own, because it made them. + +**And the node's configuration is meant to follow the modules assigned to it.** That is the mesh's +founding shape — the machine runs modules, and its files, its filter and its accounts are composed +from what runs there ([ADR 0005](0005-the-node-host.md), +[ADR 0010](0010-delivery.md), +[ADR 0045](0045-a-machine-firewall-is-the-sum-of-what-it-listens-on.md)). The forward chain is the +one derived thing that consults a constant and a list a person types. + +## Considered Options + +1. **Keep 0137 as it stands** — two constants plus a named list. Rejected: the list is written in + addresses, and addresses are what the runtime allocates, so the only entry safe enough to keep a + machine working is wider than the truth. It cannot distinguish a network the mesh made from one + left behind, which is the distinction that decides whether forwarding it is correct. +2. **Derive it from what the machine reports.** Still rejected, on 0137's own grounds: a test bed + creates its bridge between one declaration and the next, and a filter that follows whatever + appeared on a machine is a firewall that widens itself. **This decision is not that** — see below. +3. **Have the control plane allocate each module network's range from a pool it owns,** so it can + render the address itself. Rejected: more machinery for no gain. The runtime already allocates and + the host already knows, and taking allocation over means the mesh owning an address space it has no + other reason to own. +4. **Have each module declare its network's range.** Rejected by + [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md): a definition names no address, + and the same definition runs on machines whose runtimes have allocated differently. + +## Decision + +**A network is forwarded because a module declared it.** Per node, the forward chain forwards the +networks of the modules assigned there, and by default nothing else. A module unassigned stops being +forwarded at the next reconcile. + +**The host resolves a declared network to its addresses.** A network resource carries a name; the +runtime allocates the subnet when the network is created. So the control plane declares *forward the +networks these modules asked for* and the host — which made them, and already resolves a container by +its name — renders the addresses. [ADR 0005](0005-the-node-host.md) holds: the host applies, it does +not decide. + +**Deriving from the declaration is not deriving from the machine.** Both of 0137's objections fall +away. The set is known before the network exists, because a module declared it, so a network created +between two declarations is already in the one that asked for it. And it cannot widen itself: a +network nobody declared is never forwarded, however it appeared on the machine. + +**The runtime's own default bridge is forwarded, from what the runtime reports.** Containers that name +no module network attach to it, and it belongs to the runtime rather than to any module — so the host +renders it from what the runtime says, not from a range named in the control plane. The constants go. + +**What a machine says is for guests no module declares.** A test bed is not a module and its range is +not a module's; that is the case 0137's mechanism is for, and it keeps it — added to the derived set, +never replacing it, as 0137 decided. Narrowed to that, it is named for it. + +**Their guests keep address and name service**, per declared network, unchanged from +[ADR 0137](0137-a-machine-says-which-networks-it-routes.md): the input chain admits that network's own +DHCP and DNS and nothing else. + +## Consequences + +- **The two constants are removed**, and with them the class of fault that a machine's guests depend + on a range that describes some other machine. +- **This is a behaviour change, not a refactor.** On the machine measured, the derived set and the + constant do not cover the same ground — that is the whole reason for the record. A machine whose + module networks happen to fall inside the old ranges renders the same rules. +- **A range that exists only to keep a leftover alive becomes visible as such**, because it will not + be in the derived set and has to be said out loud to survive. +- **`node networks` narrows** to guests no module declares, and the preview says which of a machine's + networks are the mesh's and which are not, so the difference is readable before a flip rather than + after. +- **A module's declaration gains nothing.** It already declares its network; what changes is that the + filter reads it. +- **What got harder:** the host renders part of the forward chain from what it created, so the + control plane no longer holds the whole rule set as text. The rule the mesh states is the set of + networks; the addresses are the machine's. + +## How it is checked + +- **Only declared networks are forwarded.** A node with two modules that declare networks renders + forward rules for exactly those two, and none for a third network present on the machine that no + module declared. This fails against the previous behaviour, which forwards by range and cannot tell + them apart, and that is how it is written. +- **Unassigning a module removes its network's rule** at the next reconcile, asserted on the rendered + chain rather than on the intent. +- **Guests of a declared network keep address and name service**, asserted per chain body so a line in + the wrong chain cannot pass — carried from 0137. +- **The runtime's own default bridge comes from the runtime**, asserted by rendering for a runtime + whose default bridge is somewhere other than the range the constant named. +- **A machine that names a range for guests no module declares still gets it**, added to the derived + set and not replacing it. +- **Each family is matched in its own syntax**, carried from 0137: one set holding both is a syntax + error, and a ruleset that does not load is a machine filtering nothing while its unit reports a + fault. + +## References + +- [ADR 0137](0137-a-machine-says-which-networks-it-routes.md) — narrowed here; its mechanism keeps the + case it is right for +- [ADR 0005](0005-the-node-host.md) — the host applies; the addresses are the machine's +- [ADR 0045](0045-a-machine-firewall-is-the-sum-of-what-it-listens-on.md) — the filter is derived from + what runs there +- [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md) — why a module does not name its + range +- [issue 141](../04-ISSUES/141-the-forward-chain-does-not-follow-the-modules/00-report.md) — the + measurement +- [issue 137](../04-ISSUES/137-converging-a-machine-cut-off-its-own-guests/00-report.md) — the + breakage that produced 0137 diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index cac9d25..768aa53 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -218,6 +218,8 @@ python3 00-META/checks/index.py fail if stale - **0135** — [A module version prepares its state before it runs](0135-a-module-version-prepares-its-state-before-it-runs.md) - **0136** — [A step gates its module, not the machine](0136-a-step-gates-its-module-not-the-machine.md) - **0137** — [A machine says which networks it routes](0137-a-machine-says-which-networks-it-routes.md) +- **0138** — [An assignment binds an endpoint and says how far it reaches](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md) +- **0139** — [A network is forwarded because a module declared it](0139-a-network-is-forwarded-because-a-module-declared-it.md) ### How it is built diff --git a/03-DESIGN/01-to-be/08-connectivity.md b/03-DESIGN/01-to-be/08-connectivity.md index 0743f42..75f6f94 100644 --- a/03-DESIGN/01-to-be/08-connectivity.md +++ b/03-DESIGN/01-to-be/08-connectivity.md @@ -7,8 +7,10 @@ code: - mesh-controller internal/identity/authority.go - mesh-host internal/identity/serving.go - mesh-host internal/apply (the service that reflects a rule set) -updated: 2026-09-27 +updated: 2026-09-28 decisions: + - 02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md + - 02-DECISIONS/0139-a-network-is-forwarded-because-a-module-declared-it.md - 02-DECISIONS/0104-a-provision-may-be-answered-by-an-adapter-to-the-predecessor.md - 02-DECISIONS/0106-the-bus-is-nats.md - 02-DECISIONS/0105-the-mesh-adopts-the-predecessors-tunnel-in-place.md @@ -625,6 +627,42 @@ the found firewall reloads and reachable from a container on the node, that a ma enrols through the openings before and after a reload and a reboot, and that after the flip the declared port is open and the undeclared one closed. +### The networks it forwards are the ones its modules declared + +*2026-09-28, preparing the control-node's convergence — +[issue 141](../../04-ISSUES/141-the-forward-chain-does-not-follow-the-modules/00-report.md), settled by +[ADR 0139](../../02-DECISIONS/0139-a-network-is-forwarded-because-a-module-declared-it.md).* + +The forward chain denies by default, so a machine's own guests have to be allowed back in. Until now +that was two ranges named in the control plane and, since +[ADR 0137](../../02-DECISIONS/0137-a-machine-says-which-networks-it-routes.md), a list a machine could +add to. On the machine measured here, six of its container networks fell outside those ranges — and +four of the six were networks the mesh's own modules had declared and the host had created, while two +were the predecessor's leftovers. A range wide enough to keep the four keeps the two: a filter widened +by hand to protect what should not be there. + +**A network is forwarded because a module declared it.** The forward chain forwards the networks of the +modules assigned to that node and, by default, nothing else; a module unassigned stops being forwarded +at the next reconcile. The control plane says which networks, by name, and the host — which created +them — renders their addresses, because the runtime allocates the subnet and the host is what knows it. +That keeps §4's shape and this document's: the mesh decides, the host applies. + +**This is derivation from the declaration, not from the machine.** 0137 rejected reading the machine, +for two reasons that do not apply here: a network created between two declarations is already named in +the one that asked for it, and a network nobody declared is never forwarded however it appeared. What +0137's mechanism keeps is the case it was right for — guests no module declares, a test bed's range — +added to the derived set and never replacing it. + +The runtime's own default bridge, which a container attaches to when it names no module network, is the +runtime's and not a module's, so the host renders it from what the runtime reports. With that, no range +is named in the control plane at all. + +*How it is checked:* a node with two modules declaring networks renders rules for exactly those two and +none for a third present on the machine that nothing declared — which fails against forwarding by +range, and is how it was written; unassigning one removes its rule at the next reconcile; the default +bridge is asserted for a runtime whose bridge is somewhere other than the old constant named; and the +guests of a declared network keep address and name service, per chain body. + ## 5 — Certificates **Two authorities, kept separate on purpose.** @@ -710,6 +748,60 @@ that verifies against the internal root and nothing else — which cannot succee first reached the name to certify it* ([ADR 0066](../../02-DECISIONS/0066-public-routing-is-name-agnostic.md)). +## 6 — One statement behind exposure, filtering and certificates + +*2026-09-28, preparing the control-node's convergence — +[issue 140](../../04-ISSUES/140-an-endpoints-reach-is-not-declared/00-report.md), settled by +[ADR 0138](../../02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md).* + +The three sections above each decide, independently, how far a service reaches. §3 composes a name +from a label and the node's domain. §4 opens a port to the source a listen named. §5 certifies the +names that exist, from whichever authority the proxy holds. Each is coherent on its own, and together +they mean **reachability is never stated anywhere** — it is the sum of three derivations, and a sum is +not something anyone can review or refuse. + +What that costs, measured: an identity provider holding a public certificate valid 90 days and an +internal one valid 24 hours, neither asked for by any assignment, because both names existed and a +proxy certifies what it serves. And an endpoint that is not routed — git over ssh — which can be +spoken about only in the filter's vocabulary, so *this must be reachable from outside* is a setting +exactly one mechanism reads. + +**An endpoint is the thing that was missing.** A module declares named endpoints: one port it serves, +what it is for, and what it would serve that to absent any instruction. A route contribution names an +endpoint rather than repeating a port number. An assignment — which is where a module's configuration +lives ([ADR 0046](../../02-DECISIONS/0046-a-module-configuration-is-its-assignments-not-its-manifest.md)) +— then binds each endpoint to a machine port and says how far it reaches. + +One value, three readers: + +| reach | the filter opens | the proxy serves | the certificate comes from | +|---|---|---|---| +| `internal` | the machine port, to the private network | the internal name | the mesh's own authority | +| `public` | the machine port, to anywhere | the public name | the public authority | +| `both` | the machine port, to anywhere | both names | each name's own authority | + +**An endpoint that is not routed is reached and never named.** No route contribution means no name is +composed and no certificate requested, while the filter still acts on it. That is the case the model +could not express at all, and it is the ordinary case for anything that is not HTTP. + +**Nothing moves until an assignment says so.** An endpoint whose assignment is silent keeps the +default its manifest states, so every machine already converged renders exactly as it does today — +the same property §4 needed when a machine gained a way to say which networks it routes. + +This is what the certificate questions were waiting for. Which authority signs a name, whether a name +may appear in a public issuance log, and what must be trusted where are all answerable once an +endpoint says whether it is internal — and unanswerable while the proxy decides by composing every +name it can. It is also the fact +[issue 139](../../04-ISSUES/139-an-internal-route-name-resolves-to-the-consumers-node/00-report.md) +needs: an internal name should be composed from the machine serving the endpoint, which is the +assignment that bound it. + +**How it is checked** is stated with the decision: one module with two endpoints of differing reach +asserted per chain body; the names and the certificate requests following the reach and failing +against today's behaviour, where both are always composed; an unrouted endpoint filtered and never +named; an assignment naming an endpoint the module does not declare refused where it is said; and a +silent assignment rendering byte-identically to today. + ## What this removes The list is worth having in one place, because it is most of the argument: diff --git a/04-ISSUES/140-an-endpoints-reach-is-not-declared/00-report.md b/04-ISSUES/140-an-endpoints-reach-is-not-declared/00-report.md index 7fa3da1..0706b36 100644 --- a/04-ISSUES/140-an-endpoints-reach-is-not-declared/00-report.md +++ b/04-ISSUES/140-an-endpoints-reach-is-not-declared/00-report.md @@ -1,9 +1,14 @@ --- -status: open +status: located opened: 2026-09-28 -located-in: [] +located-in: + - mesh-controller internal/catalogue/manifest.go + - mesh-controller internal/catalogue/filtering.go + - mesh-controller internal/catalogue/declaration.go + - mesh-controller examples/route-proxy + - mesh-catalog (every routed module manifest) fixed-by: -amended-design: +amended-design: 03-DESIGN/01-to-be/08-connectivity.md --- # 140 — An endpoint's reach is not declared, so three mechanisms each decide it separately diff --git a/04-ISSUES/141-the-forward-chain-does-not-follow-the-modules/00-report.md b/04-ISSUES/141-the-forward-chain-does-not-follow-the-modules/00-report.md index 17c36d9..51260c5 100644 --- a/04-ISSUES/141-the-forward-chain-does-not-follow-the-modules/00-report.md +++ b/04-ISSUES/141-the-forward-chain-does-not-follow-the-modules/00-report.md @@ -1,9 +1,11 @@ --- -status: open +status: located opened: 2026-09-28 -located-in: [] +located-in: + - mesh-controller internal/catalogue/filtering.go + - mesh-host internal/apply fixed-by: -amended-design: +amended-design: 03-DESIGN/01-to-be/08-connectivity.md --- # 141 — The forward chain does not follow the modules, though the modules declare their networks