Merge pull request 'Multiple fixes: status vocabulary aligned, 065/026/070/007 resolved, 066/069/049/046 located, 064 diagnosed' (#64) from feat/multiple-fixes into main
This commit was merged in pull request #64.
This commit is contained in:
@@ -0,0 +1,65 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-09-21
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0010-delivery.md
|
||||
---
|
||||
|
||||
# 90. A failure that repeats is said to be stuck
|
||||
|
||||
## Context
|
||||
|
||||
Delivery is a comparison, not a one-shot ([ADR 0010](0010-delivery.md)): a node re-applies the
|
||||
declaration it holds on a steady interval and reports each time. That is right for a failure that
|
||||
goes away by itself — the overlay not up yet, a registry briefly unreachable — and it makes a
|
||||
failure that will never go away look exactly the same. A resource nothing can ever apply is
|
||||
attempted, fails, is reported, and is attempted again every few minutes, indefinitely; the mesh
|
||||
keeps one report per machine, replaced, so each attempt arrives as "failed" at a fresh time.
|
||||
Nothing distinguished "failed once, will succeed when its dependency arrives" from "failed
|
||||
identically for ever", and nothing escalated the second
|
||||
([issue 065](../04-ISSUES/065-a-permanently-failing-resource-is-retried-for-ever-with-no-escalation/00-report.md)).
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **The host gives up** after some number of attempts. Rejected: the host does not know whether
|
||||
a failure is permanent — that a registry has not answered three times is not evidence it never
|
||||
will — and a host that stops trying is a node that must be pushed to again by hand.
|
||||
2. **A duration** since the failure was first seen. Rejected as the signal: a laptop shut for a
|
||||
week has had one attempt, and a week is not evidence of anything.
|
||||
3. **The controller counts identical reports**, and says when there are enough of them. Adopted.
|
||||
|
||||
## Decision
|
||||
|
||||
The mesh keeps, beside each machine's last report, when the current failure was first reported
|
||||
and how many reports in a row have said it — the same outcome, the same refusal, the same failed
|
||||
resources by id. Not by the host's words: an error carrying a duration or a counter would read as
|
||||
new on every report, and the resource looping on it is exactly what this is for. A report that
|
||||
says something different starts the count again; a clean apply clears it. **Three identical reports in a row make a machine stuck**: `status` says
|
||||
so beside the failure, with the count and the time it began, and the machine-readable status
|
||||
carries the same three facts. The host keeps retrying; being stuck is a statement about the
|
||||
mesh's knowledge, not an instruction to the machine.
|
||||
|
||||
The controller counts rather than the host, because only it sees every node: one stuck machine
|
||||
and a mesh-wide fault are different situations, and the host cannot tell them apart.
|
||||
|
||||
## Consequences
|
||||
|
||||
A resource that will never apply is visible from `status` after three reconcile intervals, to
|
||||
anyone who looks, without being asked for. What got harder: nothing on the machine changes — a
|
||||
gating failure that stops what follows still stops it, and this only makes the wait visible.
|
||||
Whether a stuck machine should also be raised as an event, and what a gating failure should do,
|
||||
stay open in the issue's own questions.
|
||||
|
||||
## How it is checked
|
||||
|
||||
An inventory test records the same failure three times and asserts the count and the unchanged
|
||||
start; the same resource failing in other words, and asserts the count went on; a different
|
||||
failure, and asserts it restarted; a clean apply, and asserts both cleared. The status command's own test asserts a stuck machine is said to be one.
|
||||
|
||||
## References
|
||||
|
||||
- [issue 065](../04-ISSUES/065-a-permanently-failing-resource-is-retried-for-ever-with-no-escalation/00-report.md)
|
||||
- [ADR 0010](0010-delivery.md)
|
||||
- [`03-DESIGN/01-to-be/10-delivery.md`](../03-DESIGN/01-to-be/10-delivery.md)
|
||||
@@ -0,0 +1,68 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
date: 2026-09-21
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0051-shared-data-is-the-operators.md
|
||||
---
|
||||
|
||||
# 91. A mount is declared, and there are three things it can be
|
||||
|
||||
## Context
|
||||
|
||||
A container's bind mount whose source does not exist is created by the container runtime, as
|
||||
root, with whatever mode it picks. So `owner` and `mode` — which exist so a module can say who
|
||||
its data belongs to — never reach the directories that hold data, and the rule that keeps a
|
||||
directory when a module goes away ([ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md))
|
||||
does not cover them, because the mesh has never heard of them
|
||||
([issue 026](../04-ISSUES/026-the-data-directories-are-not-declared/00-report.md)). Fourteen
|
||||
such mounts were declared by hand; a check that every mount is declared was then written and
|
||||
withdrawn, because it refused the builder: the builder mounts the container runtime's socket,
|
||||
which is not its data, already exists, and belongs to the machine. Declaring it as the module's
|
||||
own directory would be a lie the host would act on.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Declare the socket as a directory anyway.** Rejected: the host would create, own and
|
||||
protect a path that is the machine's.
|
||||
2. **A new manifest field** naming machine paths a module may mount. Rejected: the manifest
|
||||
already says the module needs the container runtime, and a second field would say the same
|
||||
thing in paths.
|
||||
3. **Three declarations, one for each kind of path a mount can be.** Adopted.
|
||||
|
||||
## Decision
|
||||
|
||||
A container may not mount a path the module never declared, and a path is declared in one of
|
||||
three ways, which are the three things a path can be:
|
||||
|
||||
- **the module's own** — a directory or file resource, or where a secret, a grant or a
|
||||
contribution lands. Created and owned by the mesh for this module, kept when the module goes;
|
||||
- **the operator's** — an `accesses` entry ([ADR 0051](0051-shared-data-is-the-operators.md)):
|
||||
pre-existing, shared, granted for use, never owned;
|
||||
- **the machine's** — a facility a declared capability grants. `container-runtime` grants its
|
||||
socket. The path exists, the machine owns it, and the capability is the declaration.
|
||||
|
||||
A mount under a declared directory is declared. The check runs where the manifest is parsed,
|
||||
and names the path and the three remedies.
|
||||
|
||||
## Consequences
|
||||
|
||||
Every directory that holds a module's data is one the mesh created with the module's owner and
|
||||
mode, and one ADR 0030 protects. What got harder: a manifest borrowed from a compose file no
|
||||
longer passes on the strength of its volume lines; each must say what kind of path it mounts.
|
||||
The table of what a capability grants is small and in the catalogue's parser; a new capability
|
||||
that grants a path adds a row.
|
||||
|
||||
## How it is checked
|
||||
|
||||
Manifest tests refuse an undeclared mount, accept one under a declared directory, accept one
|
||||
the module accesses, and accept the runtime's socket with the capability and refuse it without.
|
||||
A test parses every manifest in the catalogue beside the checkout and fails on any that breaks
|
||||
the rule, so the catalogue cannot drift back.
|
||||
|
||||
## References
|
||||
|
||||
- [issue 026](../04-ISSUES/026-the-data-directories-are-not-declared/00-report.md)
|
||||
- [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md), [ADR 0051](0051-shared-data-is-the-operators.md)
|
||||
- [`03-DESIGN/01-to-be/18-building-a-module.md`](../03-DESIGN/01-to-be/18-building-a-module.md)
|
||||
@@ -0,0 +1,61 @@
|
||||
---
|
||||
topic: the tiers
|
||||
status: accepted
|
||||
date: 2026-09-21
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0085-a-secret-is-a-provision.md
|
||||
---
|
||||
|
||||
# 92. An operator delivers a pair credential, and the mesh never replaces it
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0085](0085-a-secret-is-a-provision.md) names three species of secret and gives the vault
|
||||
two of them: a module's own secret, which the mesh mints, and an operator-delivered secret — a
|
||||
credential for something outside the mesh, which only a person can supply. Under 0085 a secret
|
||||
from the vault is a pair credential between the consumer and the vault. The controller's one
|
||||
command that takes a value from a person wrote only a module's own secret, sealed to one node.
|
||||
Nothing could put a value into a pair, so the third species had no entry
|
||||
([issue 070](../04-ISSUES/070-an-operator-cannot-deliver-a-pair-credential/00-report.md)), and
|
||||
an operator's credential could be held only as an own secret — un-audited, un-rotatable, the
|
||||
gap 0085 opened to close.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **A new verb on the pair.** Rejected: `secret accept` already means "a value a person
|
||||
supplied, sealed on the way in, plaintext discarded"; a second verb would mean the same.
|
||||
2. **`secret accept` grows a provider end.** Adopted.
|
||||
|
||||
## Decision
|
||||
|
||||
`secret accept <node> <module> <name> --provider <node>` seals the supplied value to the
|
||||
consumer's node, to the provider's node, and to the operator's key when the mesh has one, and
|
||||
records the pair as `accepted`. Every pair credential now says where it came from: `made` or
|
||||
`accepted`.
|
||||
|
||||
An accepted pair is never replaced by a made one. When a sealing key at either end changes, the
|
||||
mesh cannot re-seal a value it does not hold, so the read is refused and names the remedy —
|
||||
accept it again. `rotate` refuses an accepted pair for the same reason: the mesh cannot make its
|
||||
replacement, and deleting it would have the next read mint one, delivered and reported as
|
||||
applied while failing to authenticate somewhere else entirely. Rotating an accepted credential
|
||||
is accepting a new value.
|
||||
|
||||
## Consequences
|
||||
|
||||
A credential for something outside the mesh lives in the vault's ledger with the others, sealed
|
||||
to both ends and recoverable by the operator. What got harder: a mesh whose node keys change
|
||||
cannot heal an accepted pair by itself; a person is asked. That is the honest shape — the value
|
||||
was never the mesh's to make.
|
||||
|
||||
## How it is checked
|
||||
|
||||
An inventory test accepts a value into a pair, reads it back twice unchanged with origin
|
||||
`accepted`, asserts `rotate` refuses it naming the remedy while a made pair still rotates;
|
||||
another changes a node's key and asserts the read is refused, then accepts again and reads.
|
||||
|
||||
## References
|
||||
|
||||
- [issue 070](../04-ISSUES/070-an-operator-cannot-deliver-a-pair-credential/00-report.md), [issue 069](../04-ISSUES/069-one-secret-provision-yields-one-value/00-report.md)
|
||||
- [ADR 0085](0085-a-secret-is-a-provision.md)
|
||||
- [`03-DESIGN/01-to-be/24-the-secrets-vault.md`](../03-DESIGN/01-to-be/24-the-secrets-vault.md)
|
||||
@@ -87,6 +87,7 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0077** — [The parts are named controller, foundation, node — not control plane, substrate, master](0077-the-controller-and-the-foundation.md)
|
||||
- **0083** — [One push leaves the mesh consistent](0083-one-push-leaves-the-mesh-consistent.md)
|
||||
- **0088** — [The foundation filters before anything listens](0088-the-foundation-filters-before-anything-listens.md)
|
||||
- **0090** — [A failure that repeats is said to be stuck](0090-a-failure-that-repeats-is-said-to-be-stuck.md)
|
||||
|
||||
### Its tiers, from the bottom up
|
||||
|
||||
@@ -111,6 +112,7 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0075** — [An artifact store is a provision; a package registry is a different one](0075-two-stores-and-which-provides-what.md)
|
||||
- **0078** — [The store and the broker are ordinary modules](0078-the-store-and-broker-are-modules.md)
|
||||
- **0079** — [The foundation seats are named after their servers](0079-the-foundation-seats-are-named-after-their-servers.md)
|
||||
- **0092** — [An operator delivers a pair credential, and the mesh never replaces it](0092-an-operator-delivers-a-pair-credential.md)
|
||||
|
||||
### What runs on them, and how it gets there
|
||||
|
||||
@@ -140,6 +142,7 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0084** — [Which provider serves a consumer, when the mesh runs more than one](0084-which-provider-serves-a-consumer.md)
|
||||
- **0085** — [A secret is a provision, and the vault is the module that provides it](0085-a-secret-is-a-provision.md)
|
||||
- **0087** — [A seeded file is created once, and what grows in it is not the mesh's](0087-a-seeded-file-is-created-once.md)
|
||||
- **0091** — [A mount is declared, and there are three things it can be](0091-a-mount-is-declared-three-ways.md)
|
||||
|
||||
### How it is built
|
||||
|
||||
|
||||
Reference in New Issue
Block a user