diff --git a/00-META/glossary.md b/00-META/glossary.md index 5c2493bc..e177d8f2 100644 --- a/00-META/glossary.md +++ b/00-META/glossary.md @@ -172,10 +172,13 @@ every other domain has something to run on. **Recorded by** the controller and t **Purpose.** To decide, send and apply what each node runs, and to say why. **Recorded by** the controller's `inventory` database. **Upstream of** Provisioning, Connectivity, Data -and Health and repair. **Decided by** ADR 0100, 0126, 0176, 0181, 0207, 0221, 0252. +and Health and repair. **Decided by** ADR 0100, 0126, 0176, 0181, 0207, 0221, 0252, 0261. **Uses:** node, machine, module, seat, setting (Module), manifest (Module). - **assignment** — a module put on a node, with the settings that node gives it. +- **pending assignment** — an assignment of a module whose build is still running, kept, and made by the + controller when that build registers the module, or ended with why when it does not + ([ADR 0261](../02-DECISIONS/0261-an-assignment-made-while-its-modules-build-runs-is-kept-pending-and-the-controller-makes-it.md)). - **scope** — where a seat or a claim holds: `node`, `site` or `mesh`. `node` is also the scope's value in a manifest. - **capacity** — how many holders a seat takes. A capacity-1 seat is exclusive. A higher-capacity seat diff --git a/02-DECISIONS/0261-an-assignment-made-while-its-modules-build-runs-is-kept-pending-and-the-controller-makes-it.md b/02-DECISIONS/0261-an-assignment-made-while-its-modules-build-runs-is-kept-pending-and-the-controller-makes-it.md new file mode 100644 index 00000000..85c65cac --- /dev/null +++ b/02-DECISIONS/0261-an-assignment-made-while-its-modules-build-runs-is-kept-pending-and-the-controller-makes-it.md @@ -0,0 +1,99 @@ +--- +topic: what runs on it +status: accepted +date: 2026-10-08 +deciders: jochen +reconstructed: false +--- + +# 261. An assignment made while its module's build runs is kept pending, and the controller makes it + +## Context + +Since issue 300's fix, a merge that adds a module asks the build seat for it, and the module is registered +when the build's outcome is taken in, minutes later. On 2026-10-08 `assign` was called in that window and +answered "no module of that name" +([issue 325](../04-ISSUES/325-assign-said-no-such-module-while-its-build-ran/00-report.md)). The controller +kept no record of a build it had asked for until the outcome came back, so it could not tell a build in +flight from a typo. + +Until now an assignment was always made at the person's act, and refused there if it could not be made. +Keeping it for later means the controller makes an assignment after the act, without the person present. +That is a new kind of act, and it needs a rule. + +## Considered Options + +1. **Refuse, and say the build is running.** The person waits and calls `assign` again. Rejected: it brings + back the second act by hand that issue 300 removed, and nothing reminds the person to make it. +2. **Keep the assignment only when asked** (an argument such as `wait`). Rejected: a person who knows to ask + for it already knows the build is running; the one who does not is the one this is for. +3. **Keep the assignment by default while a build is in flight, and have the controller make it when the + build registers the module.** Chosen. + +## Decision + +1. **The controller keeps every build request it makes**, once the ask is made and never before, with who + asked it (a merge, a plan, a person's `build`, `assign`), the repository, the directory, the branch and + the merge's commit. A request whose ask could not be handed over is kept as not asked, with why. One + handed over whose asker stopped waiting is kept as asked with its outcome unknown: the build may still + run, so it reads as in flight until its outcome or the two-hour bound. A request is kept for 30 days. +2. **`assign` of a module the catalogue does not hold says which case it is in:** + - *a build is in flight*: a kept request for the module's directory has no outcome yet (within two + hours of asking), or its build succeeded within the last five minutes and is being registered; + - *known, not built*: there is a kept request, and its build failed, was not registered, gave no outcome, + or could not be asked; + - *unknown*: no module registered and no kept request by that name. The answer claims only that, and + names the closest names. +3. **In the first case the assignment is kept as a pending assignment**, with no argument needed. In the + second, `assign` refuses, and its `build` argument asks for the build from where the last request came + from and keeps the assignment pending on it. If a pending assignment of that module to that machine is + already waiting, it is made to wait for the new build; no build is asked that nothing waits on. +4. **The controller makes a pending assignment by the same act as a person's `assign`**, with the same + checks, when the build registers the module. It first takes the machine's hold and claims the pending + assignment, so a withdrawal cannot land in between. A refusal then ends it as refused. Making it sends + nothing, as a person's `assign` sends nothing. +5. **A pending assignment never waits silently.** It expires with the builder's own words when its build + fails or is not registered, and with a sentence when no outcome comes within two hours. The controller + settles pending assignments on a tick of its own, every minute, and on each build's outcome. A read, + such as `status`, settles nothing. +6. **An assignment that was not made is a condition** (`assignment-not-made`, a warning the operator + resolves), one per pending assignment: its id names the row, so one row's condition is never cleared for + another's. It stays open until it is answered: the module is assigned to that machine later; a newer + pending assignment of it is open or was made (a newer one that also ended unmade answers nothing, and is + its own condition); or a person takes it back with `unassign`, which takes back every one of that module + on that machine. +7. **`unassign` withdraws a waiting pending assignment.** It is refused, and says so, while the controller + is making one. A claim left by a controller that stopped is settled by the tick after five minutes, from + what the mesh holds: made if the module is assigned there, else waiting again. Ended pending assignments + are deleted after 30 days, but one raised as a condition only once that condition is cleared. A machine's + pending assignments are deleted with it, and the tick clears any of their conditions still open. + +## Consequences + +- One kind of assignment is now made by the controller, not at the person's act. The person's intention is + still the cause: nothing is assigned that a person did not ask for. +- `status` lists open pending assignments (and is not well while one is open) and those ended in the last + day; the condition carries an assignment not made until it is answered. +- The controller cannot read the forge. A module directory that no merge announced since this record and no + person asked to build still reads as unknown; the refusal says how to build it. +- A build being registered reads as in flight for five minutes after its outcome. A build recorded and never + registered reads so only after that. + +## How it is checked + +- mesh-controller `cmd/mesh-controller/pending_test.go`: each case of point 2; the claim under the hold and + the refused withdrawal (points 4, 7); a stale claim settled both ways (point 7); a failed ask that never + reads as in flight, a build no longer waited for that does, and the asker kept (point 1); `build` + re-pointing a waiting assignment (point 3); a status read that settles nothing, the tick that does, and a + silent build expiring (point 5); the condition raised, cleared by `unassign` or a later assignment, and a + second assignment not made keeping its own condition (point 6); ended rows deleted after 30 days, a + raised one kept until its condition clears, and a removed machine's conditions cleared (point 7). +- mesh-controller `internal/inventory/pending_test.go`: a pending assignment goes with its machine. +- `TestReplay325`, R325 in mesh-lab's replays register: the live case of issue 325. + +## References + +- [Issue 325](../04-ISSUES/325-assign-said-no-such-module-while-its-build-ran/00-report.md); issue 300. +- novox/mesh-controller#150, novox/mesh-lab#68. +- [ADR 0207](0207-a-module-depends-on-the-node-seats-that-apply-its-resources.md): modules assigned together + are judged together, so an act naming a module not registered is refused whole. diff --git a/03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md b/03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md index 782ff579..ce6332d9 100644 --- a/03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md +++ b/03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md @@ -4,6 +4,7 @@ status: in-progress code: [mesh-controller, mesh-host, mesh-tools, mesh-catalog, mesh-sdk, mesh-lab] updated: 2026-10-08 decisions: + - 02-DECISIONS/0261-an-assignment-made-while-its-modules-build-runs-is-kept-pending-and-the-controller-makes-it.md - 02-DECISIONS/0254-a-wait-for-a-persons-new-login-is-not-a-gates-failure-and-each-module-of-a-send-keeps-its-own-pass.md - 02-DECISIONS/0253-a-condition-says-itself-to-the-operator-in-plain-words-beside-its-summary.md - 02-DECISIONS/0240-a-module-says-how-it-is-healthy-and-the-node-engine-judges-it.md @@ -177,6 +178,14 @@ controller's suite raises, which fails a kind without words. change `reopened`. - **`status`** lists open conditions first, urgent before warning, oldest first, silenced ones marked with their expiry. The all-well sentence requires none open, silenced included. + + **Changed 2026-10-08** ([ADR 0261](../../02-DECISIONS/0261-an-assignment-made-while-its-modules-build-runs-is-kept-pending-and-the-controller-makes-it.md)): `status` also + lists every open pending assignment (one kept while its module's build runs) and every one ended in the + last day, with what ended it. The all-well sentence requires none open. An assignment that was not made + (expired or refused) is the condition `assignment-not-made`, a warning the operator resolves, open until + the module is assigned there, assigned again, or taken back with `unassign`. The controller settles + pending assignments on its own one-minute tick and on each build's outcome; reading `status` settles + nothing. - **ADR 0224's provider standing** is the first kind: `provider-failing`, raised by the provider's event, cleared by its recovery, its silence after thirty minutes the row S8 below. - **Retirement** ([ADR 0230](../../02-DECISIONS/0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md)) diff --git a/04-ISSUES/325-assign-said-no-such-module-while-its-build-ran/00-report.md b/04-ISSUES/325-assign-said-no-such-module-while-its-build-ran/00-report.md new file mode 100644 index 00000000..e1d65cb4 --- /dev/null +++ b/04-ISSUES/325-assign-said-no-such-module-while-its-build-ran/00-report.md @@ -0,0 +1,91 @@ +--- +status: located +opened: 2026-10-08 +located-in: [mesh-controller cmd/mesh-controller (acts.go, the assign act; upgrades.go, the merge's build request), mesh-controller internal/inventory (the build request and pending assignment records)] +fixed-by: mesh-controller#150, mesh-lab#68 +replay: R325 +amended-design: 03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md +--- + +# 325. Assign said "no module of that name" while the module's build ran + +## What was observed + +On 2026-10-08 the catalogue's pull request adding a new module, `sensors`, merged. Since issue 300's fix, a +merge that adds a module asks the build seat for it, and the build registers the module when its outcome is +taken in. About a minute after the merge, `mesh-controller.assign {node: "g14", module: "sensors"}` +answered: + +> no module of that name: sensors + +A few minutes later the same call worked. The build had not finished the first time. + +The operator read the answer as "you forgot to register the module" and asked whether `assign` could offer +to register it when it is not found. + +## Why it is an issue + +The answer was true and misleading. The controller had itself asked for the build a minute earlier, and it +answered as if it had never heard of the module. It kept no record of a build it had asked for until the +outcome came back, so `assign` could not tell three different situations apart: + +1. a build is in flight, and the module will exist in minutes; +2. the module's directory is known (a merge or a person asked for its build) and it was never registered, + because the build failed, was not registered, never answered, or could not be asked; +3. the name is unknown: a typo, or a module in no repository the mesh builds from. + +Each needs a different act from the person, and the one answer given fitted only the third. Issue 300 +removed the second act by hand from adding a module; this answer put it back, as a wait and a retry that +nothing told the person to make. + +## Fix + +[ADR 0261](../../02-DECISIONS/0261-an-assignment-made-while-its-modules-build-runs-is-kept-pending-and-the-controller-makes-it.md) decides it; mesh-controller#150 builds +it. + +**The controller keeps every build request it makes, once the ask is made, and `assign` reads them.** A +request is kept with who asked it (a merge, a plan, a person's `build`, `assign`), its repository, +directory and branch, and the merge's commit. A request whose ask could not be handed over is kept as not +asked, so it never reads as a build in flight; one whose asker stopped waiting may still run, and reads as +in flight until its outcome or two hours. Requests are kept for 30 days. The +outcome is the `build` record with the same id. + +An assignment of a module the catalogue does not hold now says which case it is in: + +- **A build is in flight** (no outcome yet, within two hours of asking; or built within the last five + minutes and being registered): "sensors is not registered yet: it is being built from + novox/mesh-catalog@3da80a4b (modules/sensors) since 15:38:17, as build build-…; it can be assigned once + that build registers it". The assignment is kept as a **pending assignment**, and the controller makes + it, by the same act as a person's `assign`, when the build registers the module. It claims the pending + assignment under the machine's hold first, so an `unassign` cannot land in between; an `unassign` while + it is being made is refused and says so. If the build fails, is not registered, or says nothing within + two hours, the pending assignment expires with the build's own words and raises the condition + `assignment-not-made`, one per pending assignment. That condition stays open until the module is assigned + there, assigned again and pending or made, or taken back with `unassign`. The controller settles pending assignments on its own one-minute tick and on + each build's outcome; reading `status` settles nothing. `status` lists open pending assignments and those + ended in the last day, and is not well while one is open. +- **Known, not built**: said with what came of the last request, and refused. The answer offers `build`, + and `assign` with the new argument `build: "true"`, which asks for the build from where the last request + came from and keeps the assignment pending on it. If a pending assignment is already waiting, it is made + to wait for the new build. +- **Unknown**: refused as "no module of that name", as before. The answer says only what was looked at (the + catalogue, and the build requests kept for 30 days), and adds the closest names. + +Making a pending assignment sends nothing, as a person's `assign` sends nothing; `push` sends it. + +Several modules in one act are judged together (ADR 0207). So an act naming a module that is not registered +is refused whole, with each module's case said, and nothing is kept pending. + +**Not covered.** The controller cannot read the forge, so a module directory that no merge announced since +this fix, and no person asked to build, still reads as unknown. The refusal says how to build it. + +## How it is checked + +- `TestReplay325` replays the live case: a merge adds `modules/sensors`, `assign laptop sensors` runs + before the build's outcome, and then the outcome registers it. The assignment must say the build and be + made when the build lands. On the commit before the fix it fails with "no module of that name: sensors". + It is R325 in mesh-lab's replays register (mesh-lab#68). +- `cmd/mesh-controller/pending_test.go` and `internal/inventory/pending_test.go` hold the tests ADR 0261 + lists under *How it is checked*: one per case, and one per point of its decision. + +To-be 45 §2 is amended: what `status` lists, and the new condition.