From ddfd62edf6965225872aaf9344c3220b11837287 Mon Sep 17 00:00:00 2001 From: jochen Date: Tue, 29 Sep 2026 13:07:18 +0200 Subject: [PATCH] ADR 0143: a consumer verifies the grant it is given MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A grant is four facts and a credential — the provision, the machine, the port, and who the consumer is when it connects — and it is the whole mechanism by which anything in the mesh reaches anything else. The mesh asserted it and never found out whether it was true. Issue 145 is what that cost: eleven hours of 'every module current with its source' while a module could not reach its database. The consumer verifies it, from its own network position. Not the control plane, which reaches the address by a path no consumer uses. Not the machine, whose own packets carried a source address the filter admitted while every container's was refused — so a host-side check would have passed throughout the outage it exists to catch. That is inference from the rule that was loaded, stated as such. A connection and nothing more; speaking each provision's protocol would be a second implementation of every provision. One failure is not news, only consecutive ones, and the count is reported rather than the last attempt. A consumer that is not running reads unchecked, which is a different sentence from broken. What it costs to be wrong is the constraint on all of it: a check that calls a working provision broken trains a reader to ignore the report. --- ...consumer-verifies-the-grant-it-is-given.md | 134 ++++++++++++++++++ 02-DECISIONS/README.md | 1 + 03-DESIGN/01-to-be/10-delivery.md | 38 ++++- .../00-report.md | 14 +- 4 files changed, 185 insertions(+), 2 deletions(-) create mode 100644 02-DECISIONS/0143-a-consumer-verifies-the-grant-it-is-given.md diff --git a/02-DECISIONS/0143-a-consumer-verifies-the-grant-it-is-given.md b/02-DECISIONS/0143-a-consumer-verifies-the-grant-it-is-given.md new file mode 100644 index 0000000..bce1367 --- /dev/null +++ b/02-DECISIONS/0143-a-consumer-verifies-the-grant-it-is-given.md @@ -0,0 +1,134 @@ +--- +topic: what runs on it +status: accepted +date: 2026-09-29 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0010-delivery.md +--- + +# 143. A consumer verifies the grant it is given + +## Context + +A **grant** is what the mesh writes on a consumer's machine so it can reach a provider. The real one +the forge receives for its database, as it arrives: + +``` +provision postgres-database +at +port the machine port the provider is published on +as the role the provider created for this consumer +``` + +with the credential sealed in a separate file. Four facts and a password, and they are the whole +mechanism by which anything in the mesh reaches anything else. + +**The mesh asserts that claim and never finds out whether it is true.** +[Issue 145](../04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md): +converging a machine dropped the path from a container to a port on its own machine, and for eleven +hours the mesh answered *all doing what they were told, all heard from, every module current with its +source* while a web application logged, six thousand times: + +``` +connection to server at "" (10.10.0.1), port 6852 failed: timeout expired +``` + +Every check the mesh makes passed, because every check it makes is about the relationship between the +mesh and a machine: the declaration was applied, the digest matched, every container named was running. +None of them asks whether a consumer can reach what it requires — though the mesh composed the grant +and therefore knows the consumer, the machine, the address, the port and the credential. + +**And where the check runs decides whether it catches anything.** The rule in force admitted the +machines' own addresses on the private network. A dial from the *machine* to its own address carries +exactly such a source address, so a check run by the host on its own behalf would have matched that rule +and passed — while every container on the machine was refused. This is inference from the rule that was +loaded, not a measurement: the fault was found and fixed before anyone thought to dial from the host. +It is enough to decide the question, because a check whose position differs from the consumer's is +testing something nobody asked about. + +## Considered Options + +1. **The control plane dials each provision.** Rejected, and it is the tempting one because the control + plane holds every fact. It sits on the provider's machine for most provisions here and reaches the + address by a path no consumer uses; in the measured outage it would have passed throughout. +2. **The host dials on the consumer's behalf, from the machine.** Rejected for the reason above: the + machine's network position is not the consumer's, and the one outage this exists to catch is exactly + a difference between them. +3. **Ask the module.** Rejected: a module is arbitrary software that the mesh does not write. Some could + report on their provisions and most cannot, and a check that covers the modules that opted in tells + nobody anything about the rest. +4. **Read the module's logs.** Rejected: the failure was in a log the whole time, and reading a module's + logs makes the mesh depend on the wording of software it does not control. +5. **The consumer verifies it, from its own network position.** Adopted. + +## Decision + +**A consumer verifies each grant it is given, from its own network position.** After a reconcile has +applied a grant, the machine opens a connection to the address and port that grant names, from inside +the consumer's own network namespace — the same position the consumer's software dials from, which is +the only position that answers the question the grant asks. + +**It is a connection, not a conversation.** Whether the port accepts a connection is what a grant +claims; whether the credential is right, the role exists or the schema is current is the provider's to +answer and the consumer's to discover. A check that spoke each provision's protocol would be a second +implementation of every provision, and would fail for reasons that are not the mesh's. + +**One failure is not news.** A provider restarting is ordinary, and so is a consumer between containers. +A grant is reported unreachable only after it has failed on **consecutive** reconciles, and the count is +what the machine reports rather than the last attempt — so a reader can tell "it was briefly away" from +"it has never worked". + +**A grant that cannot be checked is said to be unchecked, never assumed good.** A consumer that is not +running has no network position to dial from; that is not a broken grant and must not read as one. It is +also not a verified grant, and the two are different sentences. + +**What it costs to be wrong is the constraint on all of it.** A check that reports a working provision +broken trains a reader to ignore the report, which is worse than having none — the fault this +repository keeps finding, one level up. So the threshold is consecutive failures, the check is the +cheapest thing that answers the question, and an unknown is reported as unknown. + +**The mesh says it where it says everything else.** A machine's report carries its unreachable grants, +and `status` names them beside what is out of date — so "every module current with its source" stops +being the whole of what the mesh will tell you about a machine whose modules cannot reach each other. + +## Consequences + +- **The mesh can be wrong out loud.** It has been able to assert a grant and not check it; now a grant + that does not work is a thing the mesh says, and the eleven hours of issue 145 become minutes. +- **The host gains the ability to act from a container's network position**, which it has not needed + before. That is a real capability and the only one this needs. +- **A machine reports something that is not about the declaration.** Everything it reports today is + what it applied and what it holds; this is the first thing it says about whether what it applied + works. +- **A provision with no port is not checked**, because there is nothing to dial. Several are files and + secrets, and saying "checked" about those would be the appearance of verification that this record + exists to remove. +- **What got harder:** a reconcile does more than apply. Every grant adds a connection attempt on a + cadence, which is cheap individually and worth naming: a machine with many consumers dials once per + grant per reconcile. + +## How it is checked + +- **The outage is caught.** A bed drops the path from a consumer's network position to a provider's + port while leaving the machine's own path to it open — the exact shape of issue 145 — and the grant + reads unreachable. This fails against the previous behaviour, where nothing reported anything, and + against a check run from the machine, which passes while the consumer cannot reach it. +- **A restarting provider is not an outage.** One failed reconcile reports nothing; the count rises and + falls, and the grant reads reachable again without anybody acting. +- **A consumer that is not running reads unchecked, not broken**, asserted separately from the + unreachable case because they are different sentences. +- **A provision with no port is not claimed to be checked.** +- **The report carries the count, not the last attempt**, so "briefly away" and "never worked" are + distinguishable by a reader who sees only the report. +- **`status` names an unreachable grant**, asserted on the output, since a check nothing surfaces is + the same as no check. + +## References + +- [ADR 0010](0010-delivery.md) — the declaration is owned resources; a grant is one of them +- [ADR 0009](0009-modules-and-the-graph.md) — what a provision and a consumer are +- [issue 145](../04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md) + — the eleven hours +- [issue 136](../04-ISSUES/136-a-module-may-name-a-program-the-machine-does-not-have/00-report.md) — the + same distance between a declaration and a machine, one level down diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 4c2b005..54a1778 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -223,6 +223,7 @@ python3 00-META/checks/index.py fail if stale - **0139** — [A network is forwarded because a module declared it](0139-a-network-is-forwarded-because-a-module-declared-it.md) *(superseded)* - **0140** — [The filter constrains what arrives from outside, and says nothing about a machine's own guests](0140-the-filter-constrains-what-arrives-from-outside.md) - **0141** — [The host delivers its own successor, and versions live side by side](0141-the-host-delivers-its-own-successor.md) +- **0143** — [A consumer verifies the grant it is given](0143-a-consumer-verifies-the-grant-it-is-given.md) ### How it is built diff --git a/03-DESIGN/01-to-be/10-delivery.md b/03-DESIGN/01-to-be/10-delivery.md index 13a4f78..ad80cdf 100644 --- a/03-DESIGN/01-to-be/10-delivery.md +++ b/03-DESIGN/01-to-be/10-delivery.md @@ -5,8 +5,9 @@ code: - mesh-controller internal/builder - mesh-controller cmd/mesh-controller (build, build --behind, push, status) - mesh-controller internal/inventory/builds.go -updated: 2026-09-21 +updated: 2026-09-29 decisions: + - 02-DECISIONS/0143-a-consumer-verifies-the-grant-it-is-given.md - 02-DECISIONS/0090-a-failure-that-repeats-is-said-to-be-stuck.md - 02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md - 02-DECISIONS/0010-delivery.md @@ -226,3 +227,38 @@ when the current failure began and how many reports in a row have said it — th id, whatever the words; three make the machine stuck, and `status` says so beside the failure. The host keeps trying — stuck is what the mesh knows, not what the machine is told. *How it is checked:* an inventory test counts three identical reports, a different one, and a clean apply; the status test asserts the word appears. + +## A consumer verifies the grant it is given + +*2026-09-29, from an outage that ran eleven hours — +[issue 145](../../04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md), +settled by [ADR 0143](../../02-DECISIONS/0143-a-consumer-verifies-the-grant-it-is-given.md).* + +A grant is four facts and a credential: the provision, the machine, the port, and who the consumer is +when it connects. It is the whole mechanism by which anything in the mesh reaches anything else, and the +mesh asserted it without ever finding out whether it was true. + +What that cost: a machine was converged, the path from a container to a port on its own machine closed, +and for eleven hours the mesh answered *all heard from, every module current with its source* while a +module logged a connection timeout to its database six thousand times. Every check the mesh makes passed, +because every one is about the relationship between the mesh and a machine — applied, current, running. +None asks whether a consumer can reach what it requires. + +**So a consumer verifies its own grants, from its own network position.** Not the control plane, which +reaches the address by a path no consumer uses; not the machine, whose own packets carried a source +address the filter admitted while every container's was refused. The position is the point: a check +somewhere else is testing something nobody asked about. + +It opens a connection and nothing more. Whether the credential is right or the schema current is the +provider's to answer and the consumer's to discover; a check that spoke each provision's protocol would +be a second implementation of every provision. + +One failure is not news — a provider restarting is ordinary — so a grant reads unreachable only after +consecutive reconciles, and the machine reports the count rather than the last attempt, which is what +separates "briefly away" from "never worked". A consumer that is not running has no position to dial +from: that grant reads unchecked, which is a different sentence from broken and must not be written as +one. + +*How it is checked* is stated with the decision, and the first of them is the outage itself: a bed drops +the path from a consumer's position while leaving the machine's own open, and the grant must read +unreachable — which fails both against reporting nothing and against a check run from the machine. diff --git a/04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md b/04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md index 65afda0..0ffddcc 100644 --- a/04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md +++ b/04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md @@ -5,7 +5,7 @@ located-in: - mesh-controller internal/catalogue/filtering.go (fixed for this instance) - mesh-controller (what status reports, and what it does not ask) fixed-by: -amended-design: +amended-design: 03-DESIGN/01-to-be/10-delivery.md --- # 145 — A machine reads healthy while its modules cannot reach each other @@ -72,6 +72,18 @@ distance between a declaration and the machine, and in both cases the report was **And the eleven hours are the measurement, not the bug.** The filter fault was one line and is fixed. What is not fixed is that nothing in the mesh would have told anybody. +## What was decided + +*2026-09-29, the same day.* The first open question below — should a grant be checked, and from where — +is answered by [ADR 0143](../../02-DECISIONS/0143-a-consumer-verifies-the-grant-it-is-given.md): the +consumer verifies it, from its own network position, because that is the only position that answers what +a grant claims. A check run by the control plane or by the machine would have passed throughout this +outage, since the rule in force admitted the machines' own addresses and it was the containers that were +refused. + +The remaining questions below stand, and the record answers two of them: one failure is not news, only +consecutive ones, and a grant that cannot be checked is reported unchecked rather than assumed good. + ## Open questions - Should a grant be checked? The mesh knows the consumer, the provider, the address and the port, so a