Design 47, the glossary and playbooks 04, 07 and 08 for release 6: wants and signed compositions, no plans (ADR 0539, issue 1130)
Release 6 deletes delivery plans, batches, the merge window and tiers, so design 47's sections that described them as current are rewritten to the assigner's seven rules, the controller's signed compositions per node, the node agent, credentials on a done build, the switch as release 5's last plan, the hand path by mesh-compose and a want, and the view's delivered, moving, settled and stuck events. The glossary gains composition and restates want, declaration, send and apply; the playbooks follow a delivery by the view and the have keys and call no plans, plan, push or what-if.
This commit is contained in:
+76
-35
@@ -173,8 +173,10 @@ every other domain has something to run on. **Recorded by** the controller and t
|
|||||||
mesh. It is not a separate kind of node — it is a node that additionally runs the controller (and,
|
mesh. It is not a separate kind of node — it is a node that additionally runs the controller (and,
|
||||||
today, the foundation). Lose it and the other nodes keep running what they were last told; they simply
|
today, the foundation). Lose it and the other nodes keep running what they were last told; they simply
|
||||||
cannot be told anything new.
|
cannot be told anything new.
|
||||||
- **node-engine** — the program on every node that applies what the controller declares: it receives
|
- **node-engine** — the program on every node that applies what its node should run: from release 6 of
|
||||||
the node's declaration, writes the files, runs the services and containers, and reports what it did.
|
ADR 0429 its node agent follows the node's wants and the compositions the controller signs for it, one module
|
||||||
|
at a time (ADR 0539); until then it received the node's whole declaration. It writes the files, runs the
|
||||||
|
services and containers, and says what it runs in its haves.
|
||||||
It is the engine, not a module: it owns no file's content, and every file it writes belongs to the
|
It is the engine, not a module: it owns no file's content, and every file it writes belongs to the
|
||||||
module that declared it. *Agent* is avoided because that word means a coding agent here. A record
|
module that declared it. *Agent* is avoided because that word means a coding agent here. A record
|
||||||
written before the rename keeps the old name. "The host" is retired in this repository's prose only:
|
written before the rename keeps the old name. "The host" is retired in this repository's prose only:
|
||||||
@@ -234,7 +236,7 @@ every other domain has something to run on. **Recorded by** the controller and t
|
|||||||
|
|
||||||
**Purpose.** To decide, send and apply what each node runs, and to say why.
|
**Purpose.** To decide, send and apply what each node runs, and to say why.
|
||||||
**Recorded by** the controller's `inventory` database. **Upstream of** Provisioning, Connectivity, Data
|
**Recorded by** the controller's `inventory` database. **Upstream of** Provisioning, Connectivity, Data
|
||||||
and Health and repair. **Decided by** ADR 0100, 0126, 0176, 0181, 0207, 0221, 0252, 0261, 0411.
|
and Health and repair. **Decided by** ADR 0100, 0126, 0176, 0181, 0207, 0221, 0252, 0261, 0411, 0539.
|
||||||
**Uses:** node, machine, module, seat, manifest (Module), config value (Config).
|
**Uses:** node, machine, module, seat, manifest (Module), config value (Config).
|
||||||
|
|
||||||
- **assignment** — a module put on a node. Its config values on that node (module-assignment config,
|
- **assignment** — a module put on a node. Its config values on that node (module-assignment config,
|
||||||
@@ -259,14 +261,33 @@ and Health and repair. **Decided by** ADR 0100, 0126, 0176, 0181, 0207, 0221, 02
|
|||||||
`package` on `node-package-manager`, a `container` on `node-container-runtime`
|
`package` on `node-package-manager`, a `container` on `node-container-runtime`
|
||||||
(ADR 0207).
|
(ADR 0207).
|
||||||
Not a claim: a module claims a seat it holds and declares resources.
|
Not a claim: a module claims a seat it holds and declares resources.
|
||||||
- **declaration** — what the controller sends one node: every resource of every module assigned there,
|
- **declaration** — every resource of every module assigned to one node, composed, which the controller sent
|
||||||
composed. The noun is this domain's; what a manifest says is that it *declares* (Module), and a manifest
|
that node until release 6 of ADR 0429. From release 6 nothing is sent: what a node applies is its
|
||||||
is never called a declaration. The controller's verb `plan` previews a node's declaration.
|
compositions, and the node-engine keeps the declaration it applied last only as what it starts from, its
|
||||||
- **send** — the controller giving a node its declaration. The controller's verb `push` asks for a
|
**last declaration** (ADR 0539 §9). The switch's own last declaration carries the operator's public key
|
||||||
send (of one node, or of every node); prose says send. A git push and a phone's push notification are
|
(ADR 0539 §3, §11). The noun is this domain's; what a manifest says is that it *declares* (Module), and a
|
||||||
other things (see [Homonyms](#homonyms)).
|
manifest is never called a declaration. The controller's verb `plan` previewed a node's declaration until
|
||||||
- **apply** — the node-engine making its node match its declaration; **reconcile** is doing so again
|
release 6.
|
||||||
until nothing differs.
|
- **composition** — one module's resources composed for one node at one fingerprint, signed by the controller
|
||||||
|
(or, made by hand with `mesh-compose`, by the operator), written by the controller as
|
||||||
|
`<node>.<module>.<fingerprint>` in that node's bucket `composed-<node>`, which only that node reads; the node's
|
||||||
|
own resources (adoption, firewall openings, accounts, the resolver, identities and groups) are its node
|
||||||
|
composition `<node>.node`. Its digest, with the controller's judgement of the move from what the node runs
|
||||||
|
now, is kept under the same key in `mesh-controller_composed`; a want names that digest. Derived state, kept
|
||||||
|
while its fingerprint is among the module's five newest or a want names it, and no link of the chain
|
||||||
|
(ADR 0539 §1 to §4). From release 6. Not mesh-sdk's composition, the function that composes it (see
|
||||||
|
[Homonyms](#homonyms)), nor the **composition check** (Delivery).
|
||||||
|
- **send** — the controller giving a node its declaration, until release 6 of ADR 0429. The controller's verb
|
||||||
|
`push` asked for a send (of one node, or of every node); prose says send. From release 6 nothing is sent: the
|
||||||
|
controller writes compositions and the assigner writes wants, and every act that sent writes compositions
|
||||||
|
instead (ADR 0539 §5). A git push and a phone's push notification are other things (see
|
||||||
|
[Homonyms](#homonyms)).
|
||||||
|
- **apply** — the node-engine making its node match what it should run. From release 6 of ADR 0429 its node
|
||||||
|
agent applies one module at a time, the composition whose digest the module's want names, only when the
|
||||||
|
want's fingerprint or digest differs from what runs, and removes only that module's own resources
|
||||||
|
(ADR 0539 §9); until then it applied the node's whole declaration. **reconcile** is doing so again until
|
||||||
|
nothing differs; from release 6 a failed apply is retried by the next change of the want or the composition,
|
||||||
|
never by a clock.
|
||||||
- **operator account** — the machine account of the operator's person identity on a node, read from the
|
- **operator account** — the machine account of the operator's person identity on a node, read from the
|
||||||
identity and no longer stated on the node record; empty for a node nobody logs into. Everything the mesh
|
identity and no longer stated on the node record; empty for a node nobody logs into. Everything the mesh
|
||||||
places under a person's home is resolved against this account's home and owned by it
|
places under a person's home is resolved against this account's home and owned by it
|
||||||
@@ -403,7 +424,7 @@ and the operator act. **Recorded by** the controller's `identity` and `licences`
|
|||||||
back when it fails. The domain is continuous integration and delivery, named **delivery**, and its parts live in
|
back when it fails. The domain is continuous integration and delivery, named **delivery**, and its parts live in
|
||||||
the repository `mesh-delivery` (ADR 0429 §2). **Recorded by** the `mesh-delivery` module (deliveries) and the
|
the repository `mesh-delivery` (ADR 0429 §2). **Recorded by** the `mesh-delivery` module (deliveries) and the
|
||||||
controller (the planner, the rollout) until release 6 of ADR 0429; from each release on, by the parts it builds. **Downstream of** Module, Core, Health and repair and Data.
|
controller (the planner, the rollout) until release 6 of ADR 0429; from each release on, by the parts it builds. **Downstream of** Module, Core, Health and repair and Data.
|
||||||
**Decided by** ADR 0162, 0218, 0236, 0237, 0238, 0239, 0250, 0267, 0270, 0276, 0323, 0401–0408, 0428, 0429, 0444–0447, 0524–0530.
|
**Decided by** ADR 0162, 0218, 0236, 0237, 0238, 0239, 0250, 0267, 0270, 0276, 0323, 0401–0408, 0428, 0429, 0444–0447, 0524–0530, 0539.
|
||||||
**Uses:** module, node, declaration and send (Placement), health (Health and repair), person.
|
**Uses:** module, node, declaration and send (Placement), health (Health and repair), person.
|
||||||
|
|
||||||
- **merge check** — what a pull request carries before it may merge, judging the merge of its head into a
|
- **merge check** — what a pull request carries before it may merge, judging the merge of its head into a
|
||||||
@@ -413,7 +434,8 @@ controller (the planner, the rollout) until release 6 of ADR 0429; from each rel
|
|||||||
release 4 of ADR 0429 a pull request carried two statuses, one for the modules the change touches and one
|
release 4 of ADR 0429 a pull request carried two statuses, one for the modules the change touches and one
|
||||||
for the repository's own script (ADR 0238).
|
for the repository's own script (ADR 0238).
|
||||||
- **composition check** — whether the machines a change touches still compose with it, run by the assigner
|
- **composition check** — whether the machines a change touches still compose with it, run by the assigner
|
||||||
from the modules the indexer and the builder hold, with the composition the controller sends with
|
from the modules the indexer and the builder hold, with mesh-sdk's composition, the one the controller
|
||||||
|
composes with
|
||||||
(ADR 0446). It replaces the controller's check of that kind, which composed every machine from release 4 of
|
(ADR 0446). It replaces the controller's check of that kind, which composed every machine from release 4 of
|
||||||
ADR 0429 back (issue 1303).
|
ADR 0429 back (issue 1303).
|
||||||
*Not:* ~~merge gate~~ (hq)
|
*Not:* ~~merge gate~~ (hq)
|
||||||
@@ -468,14 +490,17 @@ controller (the planner, the rollout) until release 6 of ADR 0429; from each rel
|
|||||||
Its states are one table (`proposed`, `checked`, `ready` or `rejected`, `published`, `delivering`,
|
Its states are one table (`proposed`, `checked`, `ready` or `rejected`, `published`, `delivering`,
|
||||||
`held`, and the final six: `delivered`, `failed`, `superseded`, `stopped`, `retired` and `covered`;
|
`held`, and the final six: `delivered`, `failed`, `superseded`, `stopped`, `retired` and `covered`;
|
||||||
ADR 0250,
|
ADR 0250,
|
||||||
ADR 0270). Not *change* (a diff). *Pipeline* is the predecessor's build-and-deploy
|
ADR 0270). From release 6 of ADR 0429 the gate keeps the merge's states alone (`proposed`, `ready`,
|
||||||
|
`rejected`, `held`, `merged` and `closed`), and a merged delivery is delivered when its source revision is,
|
||||||
|
derived from the wants and the haves (ADR 0539 §17). Not *change* (a diff). *Pipeline* is the predecessor's build-and-deploy
|
||||||
mechanism, which the as-is designs describe; it is never a word for a delivery.
|
mechanism, which the as-is designs describe; it is never a word for a delivery.
|
||||||
*Not:* ~~deployment~~ (hq)
|
*Not:* ~~deployment~~ (hq)
|
||||||
- **covered** — the end of a delivery that a later merge containing it delivered: every module it moves was
|
- **covered** — the end of a delivery that a later merge containing it delivered: every module it moves was
|
||||||
sent to every machine running it by the rollout of a later merge into the same trunk, delivered on its own
|
sent to every machine running it by the rollout of a later merge into the same trunk, delivered on its own
|
||||||
passing check, whose merge commit the forge says contains this one. Not *superseded*, which is a delivery
|
passing check, whose merge commit the forge says contains this one. Not *superseded*, which is a delivery
|
||||||
whose rollout a newer delivery took over before it ended; nor *delivered*, which its own rollout or an empty plan
|
whose rollout a newer delivery took over before it ended; nor *delivered*, which its own rollout or an empty plan
|
||||||
ends in (ADR 0270).
|
ends in (ADR 0270). As a state of the gate it goes at release 6; a newer delivered source revision then
|
||||||
|
covers the older merges of its repository (ADR 0539 §17).
|
||||||
- **delivery group** — two or more deliveries sharing a pull request head branch name across the mesh's
|
- **delivery group** — two or more deliveries sharing a pull request head branch name across the mesh's
|
||||||
repositories, delivered as one unit in an order declared (`after:`) or inferred by the planner. One
|
repositories, delivered as one unit in an order declared (`after:`) or inferred by the planner. One
|
||||||
level: a group holds deliveries, never groups. Its state is derived from its members, never set.
|
level: a group holds deliveries, never groups. Its state is derived from its members, never set.
|
||||||
@@ -490,16 +515,20 @@ controller (the planner, the rollout) until release 6 of ADR 0429; from each rel
|
|||||||
- **delivery plan** — what a delivery does to the mesh: its **build plan** (the modules it moves and
|
- **delivery plan** — what a delivery does to the mesh: its **build plan** (the modules it moves and
|
||||||
their dependents, in tiers), its **deploy plan** (per node, what it receives and what waits for a
|
their dependents, in tiers), its **deploy plan** (per node, what it receives and what waits for a
|
||||||
person) and its verdict (the composed nodes, the replays). Computed by the controller's planner from a
|
person) and its verdict (the composed nodes, the replays). Computed by the controller's planner from a
|
||||||
diffset; the controller's verb `delivery-plan` shows it. *Deploy* is used in this one place. Deleted at
|
diffset; the controller's verb `delivery-plan` showed it. *Deploy* is used in this one place. Deleted at
|
||||||
release 6 of ADR 0429 (§8): the word stays in use until that release is delivered, and then retires.
|
release 6 of ADR 0429 (§8, ADR 0539): the word stays in use until that release is delivered, and goes to a
|
||||||
|
*Not:* line with the as-is update that follows the delivery (playbook 02).
|
||||||
*Not:* ~~change plan~~, ~~release plan~~
|
*Not:* ~~change plan~~, ~~release plan~~
|
||||||
- **merge window** — the rolling wait in which the controller collects merges before it plans them: each
|
- **merge window** — the rolling wait in which the controller collects merges before it plans them: each
|
||||||
merge restarts it, and it closes when none arrived for its length (90 seconds by default) or at the latest
|
merge restarts it, and it closes when none arrived for its length (90 seconds by default) or at the latest
|
||||||
10 minutes after its first merge (ADR 0276). Kept as two keys with a time to live on the bus; the first to
|
10 minutes after its first merge (ADR 0276). Kept as two keys with a time to live on the bus; the first to
|
||||||
expire closes the batch (ADR 0407). Goes at release 6 of ADR 0429, when nothing collects merges (§12).
|
expire closes the batch (ADR 0407). Goes at release 6 of ADR 0429, when nothing collects merges (§12), and
|
||||||
|
its keys' bucket is deleted (ADR 0539 §13); the word goes to a *Not:* line with the as-is update after that
|
||||||
|
delivery.
|
||||||
- **batch** — the merges one merge window collected, planned once as one rollout. For a repository merged more
|
- **batch** — the merges one merge window collected, planned once as one rollout. For a repository merged more
|
||||||
than once in it, the rollout takes the latest commit, and the earlier merges are answered by it. Not *group*,
|
than once in it, the rollout takes the latest commit, and the earlier merges are answered by it. Not *group*,
|
||||||
which is a delivery group. Goes at release 6 of ADR 0429 (§11, §12).
|
which is a delivery group. Goes at release 6 of ADR 0429 (§11, §12): a merge moves the source revision and
|
||||||
|
nothing collects it; the word goes to a *Not:* line with the as-is update after that delivery.
|
||||||
- **mesh-delivery** — the repository of the delivery domain, holding its parts as modules prefixed
|
- **mesh-delivery** — the repository of the delivery domain, holding its parts as modules prefixed
|
||||||
`mesh-delivery-` (ADR 0429 §2), made by the forge module from its config (ADR 0428). Until release 3 of
|
`mesh-delivery-` (ADR 0429 §2), made by the forge module from its config (ADR 0428). Until release 3 of
|
||||||
ADR 0429 also the module in the catalogue that owns deliveries and delivery groups, holding the mesh-scoped
|
ADR 0429 also the module in the catalogue that owns deliveries and delivery groups, holding the mesh-scoped
|
||||||
@@ -512,9 +541,10 @@ controller (the planner, the rollout) until release 6 of ADR 0429; from each rel
|
|||||||
trunk), the **builder** (claims one fingerprint once and orders its build), the **checker** (the merge check on built
|
trunk), the **builder** (claims one fingerprint once and orders its build), the **checker** (the merge check on built
|
||||||
artifacts), the **gate** (merges a group and moves the source revision; *gate part* where it could be read
|
artifacts), the **gate** (merges a group and moves the source revision; *gate part* where it could be read
|
||||||
as another gate, see [Homonyms](#homonyms)), the **teller** (the operator's phone, and the only writer of an
|
as another gate, see [Homonyms](#homonyms)), the **teller** (the operator's phone, and the only writer of an
|
||||||
ask's answer), the **assigner** (writes each node's wants by its six rules), the **node agent** (the
|
ask's answer), the **assigner** (writes each node's wants by its seven rules, ADR 0539 §6), the **node
|
||||||
node-engine's own process, applying its wants), the **watchman** (presence and pulses) and the **status
|
agent** (the node-engine's own process, applying its wants), the **watchman** (presence and pulses) and the
|
||||||
view** (`mesh-delivery-cli` and a web view, writing only its answers to asks). The reviewer is the review
|
**status view** (`mesh-delivery-cli` and a web view, writing only its answers to asks and saying the events
|
||||||
|
`delivered`, `moving`, `settled` and `stuck`, derived from the keys). The reviewer is the review
|
||||||
agent's session, not a part. Built over releases 1 to 6 of ADR 0429.
|
agent's session, not a part. Built over releases 1 to 6 of ADR 0429.
|
||||||
- **fingerprint** — what identifies a module's build by its content: a hash over its source files, its
|
- **fingerprint** — what identifies a module's build by its content: a hash over its source files, its
|
||||||
manifest and the fingerprints of its build-time inputs, so equal content gives an equal fingerprint whatever
|
manifest and the fingerprints of its build-time inputs, so equal content gives an equal fingerprint whatever
|
||||||
@@ -527,11 +557,15 @@ controller (the planner, the rollout) until release 6 of ADR 0429; from each rel
|
|||||||
want and is healthy, derived from the keys and recorded by no part (§1).
|
want and is healthy, derived from the keys and recorded by no part (§1).
|
||||||
- **canary** — the first node of a module's nodes, ordered by ring then name: a new fingerprint reaches it
|
- **canary** — the first node of a module's nodes, ordered by ring then name: a new fingerprint reaches it
|
||||||
first and the module's other nodes only once it runs healthy there (ADR 0429 §6). Replaces sending to every
|
first and the module's other nodes only once it runs healthy there (ADR 0429 §6). Replaces sending to every
|
||||||
machine at once from release 6 (§15).
|
machine at once from release 6 (§15); a fingerprint that fails there is held and the canary set back, or for
|
||||||
- **want** and **have** — a want (`want.<node>.<module>`) is the fingerprint a node should run, written by the
|
a module that keeps data the operator is asked (ADR 0539 §18, §19).
|
||||||
assigner; a have (`have.<node>.<module>`) is what the node runs, its phase and health, written by that node's
|
- **want** and **have** — a want (`want.<node>.<module>`) is the fingerprint a node should run of a module and
|
||||||
agent (ADR 0429 §4, §6). A node's facts that no module owns are its node key. They replace assignments and
|
the digest of the one **composition** it allows for it, written by the assigner by its seven rules; the node's
|
||||||
rollouts at release 6 (§17); haves are written from release 1.
|
own want `want.<node>.node` names its node composition (ADR 0429 §4, §6; ADR 0539 §1, §6). A want may also
|
||||||
|
carry the operator's answer to a failed backup (ADR 0539 §16). A have (`have.<node>.<module>`) is what the
|
||||||
|
node runs: the fingerprint, the composition's digest, its phase (`held` among them) and health, written by
|
||||||
|
that node's agent. A node's facts that no module owns are its node key. They replace rollouts at release 6,
|
||||||
|
and the assignments keep placement alone (ADR 0429 §17, ADR 0539 §12); haves are written from release 1.
|
||||||
- **pulse** — a key the watchman writes for each key whose next move a machine owes, with a time to live; its
|
- **pulse** — a key the watchman writes for each key whose next move a machine owes, with a time to live; its
|
||||||
expiry is a stall, raised as a condition, so no part needs a timer (ADR 0429 §1, research 063). A person's
|
expiry is a stall, raised as a condition, so no part needs a timer (ADR 0429 §1, research 063). A person's
|
||||||
wait carries none.
|
wait carries none.
|
||||||
@@ -541,10 +575,13 @@ controller (the planner, the rollout) until release 6 of ADR 0429; from each rel
|
|||||||
merge it answers, and its record is one key of the bus's rollouts bucket, a link of the chain (ADR 0408). A primitive the delivering stage
|
merge it answers, and its record is one key of the bus's rollouts bucket, a link of the chain (ADR 0408). A primitive the delivering stage
|
||||||
asks for, not an object anyone manages. A **tier** is one step of a rollout: a module, then the modules
|
asks for, not an object anyone manages. A **tier** is one step of a rollout: a module, then the modules
|
||||||
built against it. The controller's verb `plans` lists the rollouts. A module is rolled out by its `roll`
|
built against it. The controller's verb `plans` lists the rollouts. A module is rolled out by its `roll`
|
||||||
policy. Rollouts and tiers go at release 6 of ADR 0429 (§8, §15), replaced by wants and haves (§17).
|
policy. Rollouts and tiers go at release 6 of ADR 0429 (§8, §15), replaced by wants and haves (§17); the
|
||||||
|
rollouts bucket is kept read-only with its history (ADR 0539 §12), and both words go to a *Not:* line with
|
||||||
|
the as-is update after that delivery.
|
||||||
*Not:* ~~walk~~ (hq)
|
*Not:* ~~walk~~ (hq)
|
||||||
- **first-node gate** — the judgement on a delivery's first node: healthy three times, no witness put it
|
- **first-node gate** — the judgement on a delivery's first node: healthy three times, no witness put it
|
||||||
back, no condition raised since the send (ADR 0236 §2). Not *the gate* bare (see [Homonyms](#homonyms)).
|
back, no condition raised since the send (ADR 0236 §2). Goes at release 6 of ADR 0429, replaced by the
|
||||||
|
**canary**. Not *the gate* bare (see [Homonyms](#homonyms)).
|
||||||
- **core module** and **leaf module** — a core module is one on the controller's own path (the controller,
|
- **core module** and **leaf module** — a core module is one on the controller's own path (the controller,
|
||||||
the node-engine, the tool runner, the bus, mesh-delivery) or the mesh's resolver, whose judgement protects
|
the node-engine, the tool runner, the bus, mesh-delivery) or the mesh's resolver, whose judgement protects
|
||||||
the mesh's ability to reach a node at all; a leaf module is every other module. Decided by
|
the mesh's ability to reach a node at all; a leaf module is every other module. Decided by
|
||||||
@@ -560,8 +597,9 @@ controller (the planner, the rollout) until release 6 of ADR 0429; from each rel
|
|||||||
part. Its **fix budget** is what it is held to: five minutes for a leaf module, ten for a core module, counted
|
part. Its **fix budget** is what it is held to: five minutes for a leaf module, ten for a core module, counted
|
||||||
from the move to `fixing`; an issue whose fixes move no module has none (ADR 0313). Not the delivery time,
|
from the move to `fixing`; an issue whose fixes move no module has none (ADR 0313). Not the delivery time,
|
||||||
which starts at the merge.
|
which starts at the merge.
|
||||||
- **release** — a person's word that a held delivery goes on; the verb `mesh-delivery.release`. Used in
|
- **release** — a person's word that a held delivery goes on; the verb `mesh-delivery.release`, until release 6
|
||||||
this sense only.
|
of ADR 0429, when the verb goes (ADR 0539 §17) and a held move goes on by the operator's answer to the
|
||||||
|
assigner's ask. Used in this sense only.
|
||||||
- **rollback** — putting a node back on the build it ran before, by something other than the build being
|
- **rollback** — putting a node back on the build it ran before, by something other than the build being
|
||||||
judged (a witness, ADR 0236).
|
judged (a witness, ADR 0236).
|
||||||
- **the lab**, **scenario**, **replay** — a real mesh raised to run a change against before it reaches
|
- **the lab**, **scenario**, **replay** — a real mesh raised to run a change against before it reaches
|
||||||
@@ -923,13 +961,16 @@ renaming one sense, the old sense moves to a *Not:* line and becomes mechanical.
|
|||||||
|
|
||||||
| Word | Sense | Say | Domain |
|
| Word | Sense | Say | Domain |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| plan | what the controller would send one node | that node's **declaration** (verb `plan`) | Placement |
|
| plan | what the controller would send one node, until release 6 | that node's **declaration** (verb `plan`) | Placement |
|
||||||
| | what a delivery would do to the mesh | **delivery plan** | Delivery |
|
| | what a delivery would do to the mesh, until release 6 | **delivery plan** | Delivery |
|
||||||
| | the sending of one batch of commits across nodes | **rollout** (verb `plans`) | Delivery |
|
| | the sending of one batch of commits across nodes, until release 6 | **rollout** (verb `plans`) | Delivery |
|
||||||
| | a step a person starts, like the bus's | **planned step** | Core |
|
| | a step a person starts, like the bus's | **planned step** | Core |
|
||||||
|
| composition | one module's resources for one node at one fingerprint, signed, from release 6 | **composition** | Placement |
|
||||||
|
| | the function in mesh-sdk that composes them, which the controller and the assigner both run | **mesh-sdk's composition** | Placement |
|
||||||
|
| | whether the machines a change touches still compose | **composition check** | Delivery |
|
||||||
| membership | the subjects a bus account is issued | **membership** | Provisioning |
|
| membership | the subjects a bus account is issued | **membership** | Provisioning |
|
||||||
| | an account in a group of the machine | **account group** | Placement |
|
| | an account in a group of the machine | **account group** | Placement |
|
||||||
| push | the controller giving nodes their declarations | **send** (verb `push`) | Placement |
|
| push | the controller giving nodes their declarations, until release 6 | **send** (verb `push`) | Placement |
|
||||||
| | a git push | **git push** | The record |
|
| | a git push | **git push** | The record |
|
||||||
| | a phone notification | **push notification** | Operator and conversation |
|
| | a phone notification | **push notification** | Operator and conversation |
|
||||||
| release | a person letting a held delivery go on | **release** | Delivery |
|
| release | a person letting a held delivery go on | **release** | Delivery |
|
||||||
|
|||||||
@@ -9,7 +9,7 @@ an hour, and said to the operator if it runs past.
|
|||||||
|
|
||||||
A design becomes **milestones**, and each milestone becomes **issues**:
|
A design becomes **milestones**, and each milestone becomes **issues**:
|
||||||
|
|
||||||
- **A milestone is usable on its own.** Once its delivery plan is done, the operator or the mesh can do
|
- **A milestone is usable on its own.** Once it is delivered, the operator or the mesh can do
|
||||||
something new, said in one sentence ("a Claude Code session is mirrored into a thread on the bus").
|
something new, said in one sentence ("a Claude Code session is mirrored into a thread on the bus").
|
||||||
A milestone that changes nothing anyone can use is not a milestone; fold it into the next.
|
A milestone that changes nothing anyone can use is not a milestone; fold it into the next.
|
||||||
- **A milestone is one release** of playbook [07](07-feature-branches.md): prepared, developed whole, reviewed,
|
- **A milestone is one release** of playbook [07](07-feature-branches.md): prepared, developed whole, reviewed,
|
||||||
@@ -17,9 +17,10 @@ A design becomes **milestones**, and each milestone becomes **issues**:
|
|||||||
- **A milestone is one piece of functionality, never cut by an order between its parts** (the operator, 2026-10-21:
|
- **A milestone is one piece of functionality, never cut by an order between its parts** (the operator, 2026-10-21:
|
||||||
a milestone may span several modules, repositories and dependencies such as an SDK change; 2026-10-22: "a milestone
|
a milestone may span several modules, repositories and dependencies such as an SDK change; 2026-10-22: "a milestone
|
||||||
is a milestone, we shouldn't cut it"). Which of its modules reaches the machines first (a reader of a new manifest
|
is a milestone, we shouldn't cut it"). Which of its modules reaches the machines first (a reader of a new manifest
|
||||||
word before the modules that use it, an SDK change before its users) is the delivery plan's, drawn from the module
|
word before the modules that use it, a provider before its consumers) is the assigner's rules' (design 47); an SDK
|
||||||
graph. The parts such an order touches are found in playbook [07](07-feature-branches.md) step 0 item 6 and become
|
change is built into its users. The parts such an order touches are found in playbook [07](07-feature-branches.md)
|
||||||
issues of the same milestone; an order the plan cannot keep is an issue of the build and delivery system, not a cut.
|
step 0 item 6 and become issues of the same milestone; an order those rules cannot keep is an issue of the build
|
||||||
|
and delivery system, not a cut.
|
||||||
Until 2026-10-22 this rule cut milestones by release order (design 47's release 1-2 became six milestones).
|
Until 2026-10-22 this rule cut milestones by release order (design 47's release 1-2 became six milestones).
|
||||||
- **An issue is one repository's part of a milestone,** sized so a fixing agent finishes it inside the
|
- **An issue is one repository's part of a milestone,** sized so a fixing agent finishes it inside the
|
||||||
milestone's development time. Issues that must land in order are linked `depends-on`.
|
milestone's development time. Issues that must land in order are linked `depends-on`.
|
||||||
@@ -37,8 +38,9 @@ A design becomes **milestones**, and each milestone becomes **issues**:
|
|||||||
4. **Break it into milestones.** Write them into the design, in a section *Milestones*: for each, its number,
|
4. **Break it into milestones.** Write them into the design, in a section *Milestones*: for each, its number,
|
||||||
its one-sentence outcome, the acceptance tests of the design it proves, the repositories it touches, and
|
its one-sentence outcome, the acceptance tests of the design it proves, the repositories it touches, and
|
||||||
what it depends on. Order them so each is usable as soon as it is delivered, the smallest useful one
|
what it depends on. Order them so each is usable as soon as it is delivered, the smallest useful one
|
||||||
first. A milestone is named by what it lets someone do; how it ships is shaped by the plan its changes would
|
first. A milestone is named by what it lets someone do; how it ships is shaped by the modules its changes move
|
||||||
have, asked through `mesh-delivery.what-if`: the modules it moves and their fan-out (ADR 0401 §6).
|
and the nodes that run them (ADR 0401 §6), read from the indexer's fingerprints through the status view's
|
||||||
|
`indexer_modules` and `indexer_candidates`, and from the nodes' haves (`node_haves`).
|
||||||
5. **Ask the operator to confirm the milestones**, in a short table: number, outcome, the acceptance tests it
|
5. **Ask the operator to confirm the milestones**, in a short table: number, outcome, the acceptance tests it
|
||||||
proves, repositories, the parts each repository builds. Change what they change.
|
proves, repositories, the parts each repository builds. Change what they change.
|
||||||
6. **Open the issues.** For each confirmed milestone, `issue-tracker.open` one issue per repository part, its
|
6. **Open the issues.** For each confirmed milestone, `issue-tracker.open` one issue per repository part, its
|
||||||
@@ -48,7 +50,7 @@ A design becomes **milestones**, and each milestone becomes **issues**:
|
|||||||
7. **Flip the status.** `status: in-progress`, `updated:` today; the design change goes through playbook
|
7. **Flip the status.** `status: in-progress`, `updated:` today; the design change goes through playbook
|
||||||
[08](08-small-fixes.md) (one hq pull request).
|
[08](08-small-fixes.md) (one hq pull request).
|
||||||
8. **Hand each milestone to playbook 07, one at a time.** The milestone is one release of playbook 07; its issues
|
8. **Hand each milestone to playbook 07, one at a time.** The milestone is one release of playbook 07; its issues
|
||||||
are the release's issues; the next milestone starts when the previous one's delivery plan is done. A design waits
|
are the release's issues; the next milestone starts when the previous one is delivered. A design waits
|
||||||
while another design's milestones are being built: one design is built at a time (playbook
|
while another design's milestones are being built: one design is built at a time (playbook
|
||||||
[07](07-feature-branches.md)), so this step starts only when the design before it is delivered.
|
[07](07-feature-branches.md)), so this step starts only when the design before it is delivered.
|
||||||
9. **On completion of the last milestone**, run the "when something ships" section of playbook
|
9. **On completion of the last milestone**, run the "when something ships" section of playbook
|
||||||
@@ -72,5 +74,6 @@ Read by the lead and the reviewer today; a check in hq's merge-check is tracked
|
|||||||
issues; an in-progress design without one is the smell of building without a breakdown.
|
issues; an in-progress design without one is the smell of building without a breakdown.
|
||||||
- Every issue a milestone names exists in the tracker (`issue-tracker.get`), and an issue in `fixing` names
|
- Every issue a milestone names exists in the tracker (`issue-tracker.get`), and an issue in `fixing` names
|
||||||
its milestone in its report.
|
its milestone in its report.
|
||||||
- A milestone whose issues are all `resolved` has had its delivery plan done; its outcome sentence and
|
- A milestone whose issues are all `resolved` is delivered (the status view said `delivered` for its source
|
||||||
|
revision); its outcome sentence and
|
||||||
acceptance tests are proven on the mesh before the next milestone starts.
|
acceptance tests are proven on the mesh before the next milestone starts.
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ A small fix (one repository, one issue, about an hour of work) follows
|
|||||||
minutes took forty-one.
|
minutes took forty-one.
|
||||||
- **Finished work that never ships.** A feature that kept taking new issues held thirteen landed issues
|
- **Finished work that never ships.** A feature that kept taking new issues held thirteen landed issues
|
||||||
undelivered for a day: an issue done at 21:09 was still not on the machines at 15:00 the next day.
|
undelivered for a day: an issue done at 21:09 was still not on the machines at 15:00 the next day.
|
||||||
- **Work that waits and nobody notices.** Most lost time was not work at all, but a check, a delivery plan
|
- **Work that waits and nobody notices.** Most lost time was not work at all, but a check, a delivery
|
||||||
or an agent waiting on something that had stopped, unseen.
|
or an agent waiting on something that had stopped, unseen.
|
||||||
- **Review that keeps hardening.** One fix went through seven review rounds in five and a half hours, each
|
- **Review that keeps hardening.** One fix went through seven review rounds in five and a half hours, each
|
||||||
asking for more hardening than its issue asked (issue 704, 2026-10-14). A round is for what makes the
|
asking for more hardening than its issue asked (issue 704, 2026-10-14). A round is for what makes the
|
||||||
@@ -50,26 +50,27 @@ opened **once**, after the review. It runs in five steps, and its time is bounde
|
|||||||
| 1. Develop | one fixing agent per repository builds every issue locally, the whole unit suite before every commit; then each branch is pushed and tried | the milestone's issues, 30 minutes each, and one try |
|
| 1. Develop | one fixing agent per repository builds every issue locally, the whole unit suite before every commit; then each branch is pushed and tried | the milestone's issues, 30 minutes each, and one try |
|
||||||
| 2. Review | one reviewer reads all of it with the tries' results; reports go both ways; at most 3 rounds | 15 minutes |
|
| 2. Review | one reviewer reads all of it with the tries' results; reports go both ways; at most 3 rounds | 15 minutes |
|
||||||
| 3. Check | push the reviewed heads; a head the review changed is tried again; then the pull requests are opened and the forge's merge checks run the integration tests again; no further review | at most one more try and one check run |
|
| 3. Check | push the reviewed heads; a head the review changed is tried again; then the pull requests are opened and the forge's merge checks run the integration tests again; no further review | at most one more try and one check run |
|
||||||
| 4. Deliver | merge; the delivery plan builds, publishes and sends the whole feature | its delivery plan |
|
| 4. Deliver | merge; the source revision moves, each module is built once and wanted on its nodes, its canary first | until the view says the source revision `delivered` |
|
||||||
|
|
||||||
**One milestone is one release** (the operator, 2026-10-17). A milestone is developed whole, tried before its review, reviewed once,
|
**One milestone is one release** (the operator, 2026-10-17). A milestone is developed whole, tried before its review, reviewed once,
|
||||||
opened once and delivered once. It is never cut into steps for its size or for time: every cut adds a review, a
|
opened once and delivered once. It is never cut into steps for its size or for time: every cut adds a review, a
|
||||||
merge check, an Approve and a delivery plan of waiting, so the fewest cuts is the fastest. Nor is it cut for an
|
merge check, an Approve and a delivery of waiting, so the fewest cuts is the fastest. Nor is it cut for an
|
||||||
order between its parts (the operator, 2026-10-22: "a milestone is a milestone, we shouldn't cut it"): which module
|
order between its parts (the operator, 2026-10-22: "a milestone is a milestone, we shouldn't cut it"): which module
|
||||||
reaches the machines before which is the delivery plan's, drawn from the module graph, and that is what the plan is
|
reaches the machines before which is the assigner's rules' (a build and a composition before the want, a provider
|
||||||
for. An order the plan cannot keep is a fault of the build and delivery system, opened as an issue there, never a cut
|
before its consumers, the canary before the rest; design 47), and that is what they are for. An order they cannot
|
||||||
and never a step a session takes by hand.
|
keep is a fault of the build and delivery system, opened as an issue there, never a cut and never a step a session
|
||||||
|
takes by hand.
|
||||||
A milestone is one piece of functionality, possibly spanning several modules, repositories and dependencies such as
|
A milestone is one piece of functionality, possibly spanning several modules, repositories and dependencies such as
|
||||||
an SDK change (the operator, 2026-10-21). Its pull requests in every repository share the milestone's branch name,
|
an SDK change (the operator, 2026-10-21). Its pull requests in every repository share the milestone's branch name,
|
||||||
so they are one delivery group (ADR 0482, which holds that a change across repositories is one group, one plan and
|
so they are one delivery group (ADR 0482, which holds that a change across repositories is one group, one plan and
|
||||||
one Approve, and that its developer orders no release). Nothing of the milestone merges until every issue of it is
|
one Approve, and that its developer orders no release). Nothing of the milestone merges until every issue of it is
|
||||||
built, reviewed and checked; then the one Approve and one tiered delivery plan release it all, the plan's tiers
|
built, reviewed and checked; then the one Approve releases it all as one source revision, the assigner's rules
|
||||||
keeping a dependency such as the SDK before its dependents. A small fix found during the milestone rides in this
|
keep a provider before its consumers, and an SDK change is built into the modules that use it. A small fix found during the milestone rides in this
|
||||||
group, as playbook [08](08-small-fixes.md) says.
|
group, as playbook [08](08-small-fixes.md) says.
|
||||||
|
|
||||||
**One design is built at a time** (the operator, 2026-10-17). While the milestones of one design are being
|
**One design is built at a time** (the operator, 2026-10-17). While the milestones of one design are being
|
||||||
built, no milestone of another design starts: its issues stay `open`, and nobody moves them to `diagnosing` or
|
built, no milestone of another design starts: its issues stay `open`, and nobody moves them to `diagnosing` or
|
||||||
`fixing`. The next design starts when the last milestone of the one being built has its delivery plan done. A
|
`fixing`. The next design starts when the last milestone of the one being built is delivered. A
|
||||||
lead who finds another design's milestone already in `fixing` asks its owner before starting its own. A small
|
lead who finds another design's milestone already in `fixing` asks its owner before starting its own. A small
|
||||||
fix ([playbook 08](08-small-fixes.md)) and an issue that is in no design's milestones are not held by this rule.
|
fix ([playbook 08](08-small-fixes.md)) and an issue that is in no design's milestones are not held by this rule.
|
||||||
|
|
||||||
@@ -79,7 +80,7 @@ next milestone or the next feature.
|
|||||||
|
|
||||||
**An issue opened along the way is fixed by the session that opened it** (the operator, 2026-10-21): it is never only
|
**An issue opened along the way is fixed by the session that opened it** (the operator, 2026-10-21): it is never only
|
||||||
listed under "Opened along the way" and left. A small one (one repository, about an hour) is fixed in between, as a
|
listed under "Opened along the way" and left. A small one (one repository, about an hour) is fixed in between, as a
|
||||||
small fix ([playbook 08](08-small-fixes.md)) while the milestone waits on a check, an Approve or a delivery plan; the
|
small fix ([playbook 08](08-small-fixes.md)) while the milestone waits on a check, an Approve or a delivery; the
|
||||||
rest are fixed right after the milestone's delivery, before the next milestone starts, together on one branch where
|
rest are fixed right after the milestone's delivery, before the next milestone starts, together on one branch where
|
||||||
they share a repository. Either way it stays out of the running milestone's branch and pull request, unless it blocks
|
they share a repository. Either way it stays out of the running milestone's branch and pull request, unless it blocks
|
||||||
the running milestone, so the scope above holds. Only an issue in another design's milestones (one design is built at
|
the running milestone, so the scope above holds. Only an issue in another design's milestones (one design is built at
|
||||||
@@ -121,7 +122,8 @@ issue whether it is being fixed in between, after the delivery, or waits, and on
|
|||||||
of record), each with what it does with the new value, and a criterion for each that could treat it wrongly
|
of record), each with what it does with the new value, and a criterion for each that could treat it wrongly
|
||||||
(2026-10-19: the router's sync with the controller ended every condition the controller did not list, and the
|
(2026-10-19: the router's sync with the controller ended every condition the controller did not list, and the
|
||||||
survey, which followed only the new condition's own path, never named it); how the change is checked and delivered end to end (which test proves its done-when, which
|
survey, which followed only the new condition's own path, never named it); how the change is checked and delivered end to end (which test proves its done-when, which
|
||||||
modules its delivery plan moves, to which machines); and for a group, its merge order. The context file
|
modules its merge moves, read from the indexer's candidates of its group, and to which machines); and for a
|
||||||
|
group, its merge order. The context file
|
||||||
points to these notes; an issue reread by another session or a later agent then holds what was found.
|
points to these notes; an issue reread by another session or a later agent then holds what was found.
|
||||||
When the reading is large, it starts with a read-only survey agent on the smaller model.
|
When the reading is large, it starts with a read-only survey agent on the smaller model.
|
||||||
**Each issue carries its acceptance criteria, written by the lead** (the operator, 2026-10-18: the larger
|
**Each issue carries its acceptance criteria, written by the lead** (the operator, 2026-10-18: the larger
|
||||||
@@ -148,10 +150,10 @@ issue whether it is being fixed in between, after the delivery, or waits, and on
|
|||||||
new verbs were refused on the claim); the node-engine's decoder for a new key on the wire; the route by which a
|
new verbs were refused on the claim); the node-engine's decoder for a new key on the wire; the route by which a
|
||||||
repository's first module registers; and a part that needs something the bus or a manifest cannot yet give, which
|
repository's first module registers; and a part that needs something the bus or a manifest cannot yet give, which
|
||||||
needs its decision first. Each such part is an issue of the milestone and its pull request a member of the group;
|
needs its decision first. Each such part is an issue of the milestone and its pull request a member of the group;
|
||||||
the order in which they reach the machines is the delivery plan's (the operator, 2026-10-22), never a cut. Where
|
the order in which they reach the machines is the assigner's rules' (the operator, 2026-10-22), never a cut.
|
||||||
the plan cannot keep an order (a node told of a key before it reads it, a module using a word before its readers
|
Where those rules cannot keep an order (a node told of a key before it reads it, a module using a word before its
|
||||||
are delivered), that is an issue of the build and delivery system, opened with `issue-tracker.open` and said to
|
readers are delivered), that is an issue of the build and delivery system, opened with `issue-tracker.open` and
|
||||||
the operator.
|
said to the operator.
|
||||||
7. **Write fixtures from the shape the other side writes, never from memory** (2026-10-18: a watchman's tests
|
7. **Write fixtures from the shape the other side writes, never from memory** (2026-10-18: a watchman's tests
|
||||||
mirrored a build key the controller never writes, and every test passed): copy a value from the writer's code or
|
mirrored a build key the controller never writes, and every test passed): copy a value from the writer's code or
|
||||||
from the live bus, and name where it came from.
|
from the live bus, and name where it came from.
|
||||||
@@ -329,7 +331,7 @@ issue whether it is being fixed in between, after the delivery, or waits, and on
|
|||||||
carries an integration test that starts the module whole, its process against the real bus and store, and
|
carries an integration test that starts the module whole, its process against the real bus and store, and
|
||||||
shows the milestone's acceptance sentence; the forge's checks run it (ADR 0391 §1, §6). Unit tests alone are not
|
shows the milestone's acceptance sentence; the forge's checks run it (ADR 0391 §1, §6). Unit tests alone are not
|
||||||
that proof; the mesh has no end-to-end tests yet, and the lab is not used (the operator, 2026-10-16).
|
that proof; the mesh has no end-to-end tests yet, and the lab is not used (the operator, 2026-10-16).
|
||||||
**A new manifest word's test goes through the push that meets it first, not only the verb that fills it.**
|
**A new manifest word's test goes through the first composition that meets it, not only the verb that fills it.**
|
||||||
On 2026-10-19 release 1 of feature mail-sender tested a process's own bus account through `module issue`
|
On 2026-10-19 release 1 of feature mail-sender tested a process's own bus account through `module issue`
|
||||||
alone; the first push after mesh-mail declared the process composed a random 40-character stand-in for the
|
alone; the first push after mesh-mail declared the process composed a random 40-character stand-in for the
|
||||||
account file before anyone issued it, and the process crash-looped until the account was issued and pushed by
|
account file before anyone issued it, and the process crash-looped until the account was issued and pushed by
|
||||||
@@ -357,36 +359,41 @@ check, judged by the group's controller, is then each member's gate, and the fea
|
|||||||
|
|
||||||
## Step 4 — Deliver
|
## Step 4 — Deliver
|
||||||
|
|
||||||
The merge starts the delivery plan: build, publish, send. A merge into a branch other than `main` is not a
|
The merge moves the source revision; there is no delivery plan (design 47, ADR 0539). The indexer writes each
|
||||||
delivery: no machine receives it (issue 889 while it still starts one). The lead follows it to done
|
module's new fingerprint, the builder builds it once, the controller writes its compositions, credential and grant,
|
||||||
(`mesh-controller.plans` with `wait` and `until: done`), then resolves the feature's issues with `fixed-by`
|
and the assigner writes each node's want when its seven rules hold: the module's canary first, its other nodes once
|
||||||
naming its pull requests, and removes what is left: no `feat/<slug>` branch and no `.work/<slug>` worktree
|
it runs healthy there. A merge into a branch other than `main` is not a delivery: no machine receives it.
|
||||||
survive.
|
|
||||||
|
|
||||||
**What a merge moves is read from its plan before the operator is told.** A change to mesh-sdk rebuilds every
|
**The view follows a delivery; nobody asks the controller.** The lead's watcher waits on the status view's events:
|
||||||
module that takes the SDK, with their backups and their asks for a person (issue 721): on 2026-10-20 the lead
|
`moving` (a module's want on a node differs from its have), `settled` (the have reached the want, running and
|
||||||
said an SDK merge moved nothing, and its delivery plan moved mesh-sdk with 41 dependents to all four machines.
|
healthy), `stuck` (held, a failed health check, or past the watchman's thirty minutes) and `delivered` (every want of
|
||||||
The lead reads the delivery plan in `mesh-delivery.show` and names what it moves before asking an Approve.
|
the source revision equals its fingerprint and every have its want). After each wake it reads the state again from
|
||||||
|
the view's wants page or its tools: `assigner_wants` and `assigner_want` (a want not yet at the newest fingerprint
|
||||||
|
names, in `newest` and `rule`, the rule that holds it back, and its words say why), and `node_haves` (the node
|
||||||
|
agent's haves: fingerprint, composition digest, phase and health). The delivery is done when the view says
|
||||||
|
`delivered` for its source revision; then the lead resolves the feature's issues with `fixed-by` naming its pull
|
||||||
|
requests, and removes what is left: no `feat/<slug>` branch and no `.work/<slug>` worktree survive.
|
||||||
|
|
||||||
**A failed plan's Retry is asked of the operator alone, never beside an Approve.** An Approve's merge starts a
|
**What a merge moves is read before the operator is told.** A change to mesh-sdk rebuilds every module that takes
|
||||||
plan, and the controller refuses a Retry while another plan runs; on 2026-10-20 the operator chose both at once,
|
the SDK, with their backups and their asks for a person (issue 721): on 2026-10-20 the lead said an SDK merge moved
|
||||||
the merge's plan started first, the chosen Retry was refused and nobody asked it again (issue 1408). So the lead
|
nothing, and its delivery moved mesh-sdk with 41 dependents to all four machines. The lead reads the indexer's
|
||||||
never tells the operator to answer a Retry while an Approve is open, and reads the failed plan again after the
|
candidates of the group (the view's `indexer_candidates`: each module whose fingerprint the change moves) and the
|
||||||
answer: a Retry refused is asked again once the other plan ends.
|
nodes that run them, and names what it moves before asking an Approve.
|
||||||
|
|
||||||
**A push by hand is read from its plan first.** `mesh-controller.push` sends a machine everything it is owed, not
|
**A held move is the operator's answer, asked by the assigner.** A module of policy `record` whose move would
|
||||||
only what the lead changed: on 2026-10-21 a push of the control node to tell the bus of two new accounts also moved
|
recreate a container or move data, a failed backup before an apply, and a data module whose new fingerprint failed
|
||||||
mail's module to a build its upgrade policy holds for a person's word, and recreated one of its containers. Before
|
its health check on the canary each wait for the operator's answer to the assigner's ask; the view says them
|
||||||
every push by hand, the lead reads `mesh-controller.plan` with `diff` for that machine, and for the machine holding
|
`stuck`, and a condition names the node and the module. The lead reads the ask on the router
|
||||||
the bus when a push issues accounts, and pushes only when the plan names what the lead means to change.
|
(`operator-channel.asks`) before telling the operator it waits, and adds it to the wait list. No session writes a
|
||||||
|
want to get past a hold.
|
||||||
|
|
||||||
**A new module's first assignment is its first real composition.** The composition check composes every machine as it is
|
**A new module's first assignment is its first real composition.** The composition check composes every machine as
|
||||||
assigned, so a module that no machine runs yet is composed by no check: on 2026-10-21 the build modules' first
|
it is assigned, so a module that no machine runs yet is composed by no check: on 2026-10-21 the build modules' first
|
||||||
assignment found two faults no merge check could see (issue 1537, a template the controller filled for a value the
|
assignment found two faults no merge check could see (issue 1537, a template the controller filled for a value the
|
||||||
forge serves only later; issue 1542, the node-engine refusing the module's own `.ssh`). So a new module is assigned
|
forge serves only later; issue 1542, the node-engine refusing the module's own `.ssh`). So a new module is assigned
|
||||||
to one machine first, its plan read before the push and its apply read after it (`mesh-controller.node`: the
|
to one machine first, its have read after the node agent applies it (`node_haves`: phase and health) and its node's
|
||||||
answer, and `status` for a failed resource), and taken back at once when that machine cannot compose or apply, so
|
answer read for a failed resource (`mesh-controller.node`), and taken back at once when that machine cannot compose
|
||||||
the machine keeps applying everything else; the other machines follow once it runs there.
|
or apply, so the machine keeps applying everything else; the other machines follow once it runs there.
|
||||||
|
|
||||||
## Deliver the controller, the builder or a build module without the builder
|
## Deliver the controller, the builder or a build module without the builder
|
||||||
|
|
||||||
@@ -397,16 +404,23 @@ step recorded in the hand-act log with its reason:
|
|||||||
|
|
||||||
1. **Revert, with no build.** Revert the change that broke the builder or the build module: the earlier source's fingerprint has a done key naming
|
1. **Revert, with no build.** Revert the change that broke the builder or the build module: the earlier source's fingerprint has a done key naming
|
||||||
references, so the revert's delivery builds nothing. Read first that the artifact store still holds the earlier artifact
|
references, so the revert's delivery builds nothing. Read first that the artifact store still holds the earlier artifact
|
||||||
(`mesh-controller.artifacts`): it keeps a recorded build while it is among its module's five newest or a definition names it.
|
(`mesh-controller.artifacts`): it keeps a recorded build while it is among its module's five newest or a definition names it, and the
|
||||||
|
controller keeps its composition while it is among the five newest too, so the assigner's want delivers it.
|
||||||
2. **A build by a session's call.** Call the build module's fetch and build tools on one named build node (for Go, `build_go_fetch` and then
|
2. **A build by a session's call.** Call the build module's fetch and build tools on one named build node (for Go, `build_go_fetch` and then
|
||||||
`build_go_build` with `publish: true`); the build module pushes the artifact and answers its reference. `mesh-pack` in mesh-sdk gives a
|
`build_go_build` with `publish: true`); the build module pushes the artifact and answers its reference. `mesh-pack` in mesh-sdk gives a
|
||||||
hand build the build module's form; write the Go version of each hand build down, since another version gives another digest.
|
hand build the build module's form; write the Go version of each hand build down, since another version gives another digest.
|
||||||
3. **Register by reference, at the controller's terminal only.** The operator names the module, its repository, its path and each artifact's
|
3. **Register by reference, at the controller's terminal only.** The operator names the module, its repository, its path and each artifact's
|
||||||
reference there; no verb reaches it. Say before you ask that this is a terminal step, and which notifications it raises; check that the
|
reference there; no verb reaches it. Say before you ask that this is a terminal step, and which notifications it raises; check that the
|
||||||
command exists on the node before you hand it over.
|
command exists on the node before you hand it over.
|
||||||
4. **A controller that cannot take its own fix** is put on the control-node by hand: register its hand build as in step 3, read the control-node's
|
4. **A controller that cannot take its own fix** is put back by keys the operator writes at the terminal, with the operator's administrator
|
||||||
plan, then `mesh-controller.push`. Only in the worst case does the operator send a declaration by hand at the terminal (its source and
|
account (the operator's credential), never by a declaration sent by hand (ADR 0539 §10). To an earlier controller: the operator writes
|
||||||
digest the hand build's) and register it as in step 3 so the next push keeps it.
|
`want.<control-node>.mesh-controller` naming a fingerprint whose composition is kept and that composition's digest; a controller
|
||||||
|
composes the one it replaced at its start, so that one is always kept. To a hand build never composed: register it as in step 3, then
|
||||||
|
`mesh-compose` (in mesh-sdk, beside `mesh-pack`) composes the module for the control-node from the controller's kept inputs, signs it
|
||||||
|
with the operator's key, and with `-write` and the operator's credential writes the composition into `composed-<node>` and its digest
|
||||||
|
into `mesh-controller_composed`; the operator then writes the want naming that digest. Read the composition `mesh-compose` prints
|
||||||
|
before `-write`. Each act goes into the hand-act log with its reason; say before you ask that these are terminal steps, and which
|
||||||
|
notifications they raise.
|
||||||
5. **Before a switch that moves the controller, the builder or a build module,** the lead keeps the running versions' artifacts outside the
|
5. **Before a switch that moves the controller, the builder or a build module,** the lead keeps the running versions' artifacts outside the
|
||||||
artifact store, by digest on two nodes, with module, artifact, commit, Go version and the steps that put each back; the store's nightly
|
artifact store, by digest on two nodes, with module, artifact, commit, Go version and the steps that put each back; the store's nightly
|
||||||
collection removes a blob the controller did not record.
|
collection removes a blob the controller did not record.
|
||||||
@@ -439,20 +453,21 @@ on, with when it started and its bound.
|
|||||||
| a merge check | 15 minutes | read `mesh-delivery.show` (its state and verdict), then the check's log; the check stream's backlog and the age of its oldest ask are in the delivery view (ADR 0525 §9) |
|
| a merge check | 15 minutes | read `mesh-delivery.show` (its state and verdict), then the check's log; the check stream's backlog and the age of its oldest ask are in the delivery view (ADR 0525 §9) |
|
||||||
| the Approve's ask | 5 minutes after both checks pass | read `mesh-delivery.show`: an ask not raised is a fault, said to the operator |
|
| the Approve's ask | 5 minutes after both checks pass | read `mesh-delivery.show`: an ask not raised is a fault, said to the operator |
|
||||||
| a merge after the Approve | 5 minutes | read `mesh-delivery.show` for the delivery |
|
| a merge after the Approve | 5 minutes | read `mesh-delivery.show` for the delivery |
|
||||||
| a delivery plan | its class's budget (`mesh-delivery.times`), at most 30 minutes | read `mesh-controller.plans`, then `mesh-delivery.stalled` |
|
| a delivery | its class's budget (3 minutes for a leaf module, 10 for a core module), at most 30 minutes | read the view's wants page, or `assigner_wants` against `node_haves`: a want held back names its rule, a `stuck` module its words |
|
||||||
|
|
||||||
**Every wait on the list has a watcher** (the operator, 2026-10-17). A session runs only when a message reaches
|
**Every wait on the list has a watcher** (the operator, 2026-10-17). A session runs only when a message reaches
|
||||||
it, so a check, a merge, a delivery plan, a record's review or the operator's Approve on a channel that lands
|
it, so a check, a merge, a delivery, a record's review or the operator's Approve on a channel that lands
|
||||||
while the lead's turn has ended wakes nobody: on 2026-10-17 an approved group sat half merged for 14 minutes
|
while the lead's turn has ended wakes nobody: on 2026-10-17 an approved group sat half merged for 14 minutes
|
||||||
unseen (issue 1032). For each wait the lead starts a watcher, an agent on the smaller model that changes nothing,
|
unseen (issue 1032). For each wait the lead starts a watcher, an agent on the smaller model that changes nothing,
|
||||||
never the smallest (the operator, 2026-10-17).
|
never the smallest (the operator, 2026-10-17).
|
||||||
It loops `mesh_wait` on the event families of what it watches (`mesh-delivery-gate.transition` and
|
It loops `mesh_wait` on the event families of what it watches (`mesh-delivery-gate.transition` and
|
||||||
`mesh-delivery-gate.group` for a delivery, for example; the families a watcher may wait on are those the mesh granted
|
`mesh-delivery-gate.group` for a merge, `mesh-delivery-view.moving`, `.settled`, `.stuck` and `.delivered` for its
|
||||||
|
delivery to the nodes, for example; the families a watcher may wait on are those the mesh granted
|
||||||
the tool runner, and a refusal names them) with a timeout under 60 seconds, since the call itself ends near
|
the tool runner, and a refusal names them) with a timeout under 60 seconds, since the call itself ends near
|
||||||
a minute. After every answer or timeout it reads the state from the verb that owns it (`mesh-delivery.show`,
|
a minute. After every answer or timeout it reads the state from the verb that owns it (`mesh-delivery.show`,
|
||||||
`mesh-controller.plans`, `knowledge-base.get`), because an event only wakes it (ADR 0367). It finishes, and so wakes
|
the view's `assigner_wants` and `node_haves`, `knowledge-base.get`), because an event only wakes it (ADR 0367). It finishes, and so wakes
|
||||||
the lead, when the state changes or the wait passes its bound. The lead never sleeps or polls by hand to wait.
|
the lead, when the state changes or the wait passes its bound. The lead never sleeps or polls by hand to wait.
|
||||||
**A watcher watches one wait and ends with it.** It is never re-pointed at the next pull request or plan: on
|
**A watcher watches one wait and ends with it.** It is never re-pointed at the next pull request or delivery: on
|
||||||
2026-10-19 one watcher sent on to three pull requests in turn missed that the third's checks had passed at 04:58,
|
2026-10-19 one watcher sent on to three pull requests in turn missed that the third's checks had passed at 04:58,
|
||||||
and the lead told the operator for 15 minutes that they were still running. The next wait gets a fresh watcher,
|
and the lead told the operator for 15 minutes that they were still running. The next wait gets a fresh watcher,
|
||||||
and before a report the watcher reads the owning verb once more and says only what it says now.
|
and before a report the watcher reads the owning verb once more and says only what it says now.
|
||||||
@@ -461,7 +476,7 @@ Approve, a Retry or a Release waits for them, it reads `operator-channel.asks` a
|
|||||||
on 2026-10-20 an ask was named to the operator that the router did not hold. A merge the gate passed but whose
|
on 2026-10-20 an ask was named to the operator that the router did not hold. A merge the gate passed but whose
|
||||||
ask is not on the router is the fault of the Approve's wait in the table above.
|
ask is not on the router is the fault of the Approve's wait in the table above.
|
||||||
|
|
||||||
**Every ten minutes (the operator, 2026-10-17), every two minutes while a delivery plan runs, and at once when something lands, stops or
|
**Every ten minutes (the operator, 2026-10-17), every two minutes while a delivery moves (the view says modules `moving`), and at once when something lands, stops or
|
||||||
needs the operator,** the lead reports, in one of the two layouts below (the operator, 2026-10-16: an overview
|
needs the operator,** the lead reports, in one of the two layouts below (the operator, 2026-10-16: an overview
|
||||||
of the issues, often). Times are local and absolute, from the clock (`date`), never guessed; every number has its
|
of the issues, often). Times are local and absolute, from the clock (`date`), never guessed; every number has its
|
||||||
unit and its bound or baseline; an issue is named by its number and a few words, never by its number alone.
|
unit and its bound or baseline; an issue is named by its number and a few words, never by its number alone.
|
||||||
@@ -520,7 +535,7 @@ moments, the same hour, an hq change going through [playbook 08](08-small-fixes.
|
|||||||
|
|
||||||
- **The first milestone starts:** the design's `status: in-progress` ([playbook 04](04-build-handoff.md) step 7),
|
- **The first milestone starts:** the design's `status: in-progress` ([playbook 04](04-build-handoff.md) step 7),
|
||||||
if it is not already.
|
if it is not already.
|
||||||
- **A milestone's delivery plan is done:** its issues are resolved (step 4), with `fixed-by` naming its pull
|
- **A milestone is delivered** (the view says `delivered` for its source revision): its issues are resolved (step 4), with `fixed-by` naming its pull
|
||||||
requests. The design's milestone entry is not changed: it names the milestone's issues, and whether they are
|
requests. The design's milestone entry is not changed: it names the milestone's issues, and whether they are
|
||||||
delivered, when and by what is read from the issue tracker when it is asked, never copied into the design
|
delivered, when and by what is read from the issue tracker when it is asked, never copied into the design
|
||||||
(issue 1038; AGENTS.md: cross-cutting views are generated when they are read).
|
(issue 1038; AGENTS.md: cross-cutting views are generated when they are read).
|
||||||
|
|||||||
@@ -76,8 +76,11 @@ one delivery**. Everything before the try is local and fast.
|
|||||||
integration test starting the module whole against the real bus and store, run by `mesh-controller.try` and the checker's merge check
|
integration test starting the module whole against the real bus and store, run by `mesh-controller.try` and the checker's merge check
|
||||||
(ADR 0391 §1, §6); unit tests alone are not that proof, and the lab is not used. The operator's word is waited for only when the fix changes what the operator sees, removes data,
|
(ADR 0391 §1, §6); unit tests alone are not that proof, and the lab is not used. The operator's word is waited for only when the fix changes what the operator sees, removes data,
|
||||||
or needs the operator's terminal.
|
or needs the operator's terminal.
|
||||||
7. **Follow its delivery plan to done** (`mesh-controller.plans` with `wait` and `until: done`), prove the fix on
|
7. **Follow its delivery to done**: a watcher waits on the status view's events (`mesh-delivery-view.moving`,
|
||||||
the mesh (the symptom gone, measured), and resolve the issue with `located-in` and `fixed-by` the same hour.
|
`.settled`, `.stuck` and `.delivered`) and reads the view's `assigner_wants` and `node_haves` after each wake, as
|
||||||
|
playbook [07](07-feature-branches.md) step 4 says; the delivery is done when the view says `delivered` for its
|
||||||
|
source revision. Then prove the fix on the mesh (the symptom gone, measured), and resolve the issue with
|
||||||
|
`located-in` and `fixed-by` the same hour.
|
||||||
|
|
||||||
A fix that stops before its delivery moves its issue back to `open` with a note saying how far it got
|
A fix that stops before its delivery moves its issue back to `open` with a note saying how far it got
|
||||||
(branch, last commit, what is left), as playbook [07](07-feature-branches.md) says.
|
(branch, last commit, what is left), as playbook [07](07-feature-branches.md) says.
|
||||||
@@ -111,7 +114,7 @@ code.
|
|||||||
| the merge check | 15 minutes | read `mesh-delivery.show`, the check's log and the check stream's backlog in the delivery view |
|
| the merge check | 15 minutes | read `mesh-delivery.show`, the check's log and the check stream's backlog in the delivery view |
|
||||||
| the Approve's ask | 5 minutes after both checks pass | read `mesh-delivery.show`: an ask not raised is a fault, said to the operator |
|
| the Approve's ask | 5 minutes after both checks pass | read `mesh-delivery.show`: an ask not raised is a fault, said to the operator |
|
||||||
| the merge after the Approve | 5 minutes | read `mesh-delivery.show` for the delivery |
|
| the merge after the Approve | 5 minutes | read `mesh-delivery.show` for the delivery |
|
||||||
| the delivery plan | its class's budget (`mesh-delivery.times`), at most 30 minutes | read `mesh-controller.plans`, then `mesh-delivery.stalled` |
|
| the delivery | its class's budget (3 minutes for a leaf module, 10 for a core module), at most 30 minutes | read the view's wants page, or `assigner_wants` against `node_haves`: a want held back names its rule, a `stuck` module its words |
|
||||||
|
|
||||||
A wait past its bound is said to the operator at once, with what is being done about it.
|
A wait past its bound is said to the operator at once, with what is being done about it.
|
||||||
|
|
||||||
@@ -126,7 +129,7 @@ reading itself and uses one subagent only for the review.
|
|||||||
request with many heads is a fix reviewed after its pull request was opened.
|
request with many heads is a fix reviewed after its pull request was opened.
|
||||||
- A pull request merged by hand in a temporary mode names a try with tests `.` at its head in its description
|
- A pull request merged by hand in a temporary mode names a try with tests `.` at its head in its description
|
||||||
on the forge (`gitea.gitea_get_pull_request`); one without is the step skipped.
|
on the forge (`gitea.gitea_get_pull_request`); one without is the step skipped.
|
||||||
- Its issue moves to `resolved` within the hour its delivery plan ends.
|
- Its issue moves to `resolved` within the hour the view says its source revision `delivered`.
|
||||||
- A pull request with no review record at its head (the Approve's ask says so first), a fix to a module's
|
- A pull request with no review record at its head (the Approve's ask says so first), a fix to a module's
|
||||||
running pushed with no integration test of the module whole, or a review report
|
running pushed with no integration test of the module whole, or a review report
|
||||||
asking a change outside the issue's done-when and acceptance criteria, is the failure of steps 4 to 6.
|
asking a change outside the issue's done-when and acceptance criteria, is the failure of steps 4 to 6.
|
||||||
@@ -136,7 +139,7 @@ reading itself and uses one subagent only for the review.
|
|||||||
- An issue in `fixing` with nobody on it shows in `issue-tracker.list` with `flag: abandoned`; a stopped fix
|
- An issue in `fixing` with nobody on it shows in `issue-tracker.list` with `flag: abandoned`; a stopped fix
|
||||||
moves it back to `open` with its note.
|
moves it back to `open` with its note.
|
||||||
- A fix found during a milestone is named in the lead's report with the group it rides in, or with the reason it
|
- A fix found during a milestone is named in the lead's report with the group it rides in, or with the reason it
|
||||||
went alone. `mesh-delivery.deliveries` shows the milestone's members in one group and one delivery plan per
|
went alone. `mesh-delivery.deliveries` shows the milestone's members in one group, merged as one source revision
|
||||||
milestone; a merge of mesh-sdk or mesh-controller outside the group, with no reported reason, is this rule skipped.
|
per milestone; a merge of mesh-sdk or mesh-controller outside the group, with no reported reason, is this rule skipped.
|
||||||
- The repository's own clone says `main` (`git -C <clone> branch --show-current`); another branch there is a
|
- The repository's own clone says `main` (`git -C <clone> branch --show-current`); another branch there is a
|
||||||
fix built outside a worktree (issue 1010).
|
fix built outside a worktree (issue 1010).
|
||||||
|
|||||||
@@ -141,7 +141,7 @@ code{font:12.5px ui-monospace,monospace;background:var(--soft);padding:1px 5px;b
|
|||||||
<rect x="950" y="160" width="116" height="74" rx="10" fill="var(--card)" stroke="var(--mesh)" stroke-width="2.5"/>
|
<rect x="950" y="160" width="116" height="74" rx="10" fill="var(--card)" stroke="var(--mesh)" stroke-width="2.5"/>
|
||||||
<text x="1008" y="188" text-anchor="middle" class="t">Delivered</text>
|
<text x="1008" y="188" text-anchor="middle" class="t">Delivered</text>
|
||||||
<text x="1008" y="206" text-anchor="middle" class="mute">Approve, merge,</text>
|
<text x="1008" y="206" text-anchor="middle" class="mute">Approve, merge,</text>
|
||||||
<text x="1008" y="220" text-anchor="middle" class="mute">delivery plan</text>
|
<text x="1008" y="220" text-anchor="middle" class="mute">wants and haves</text>
|
||||||
|
|
||||||
<rect x="766" y="160" width="150" height="74" rx="10" fill="var(--card)" stroke="var(--dec)" stroke-width="2" stroke-dasharray="5 4"/>
|
<rect x="766" y="160" width="150" height="74" rx="10" fill="var(--card)" stroke="var(--dec)" stroke-width="2" stroke-dasharray="5 4"/>
|
||||||
<text x="841" y="188" text-anchor="middle" class="t">As-is updated</text>
|
<text x="841" y="188" text-anchor="middle" class="t">As-is updated</text>
|
||||||
@@ -206,7 +206,7 @@ code{font:12.5px ui-monospace,monospace;background:var(--soft);padding:1px 5px;b
|
|||||||
<text x="830" y="20" text-anchor="middle" class="t" style="fill:var(--mute)">3 · Check</text>
|
<text x="830" y="20" text-anchor="middle" class="t" style="fill:var(--mute)">3 · Check</text>
|
||||||
<text x="830" y="34" text-anchor="middle" class="mute">≤ 1 more try · 1 check</text>
|
<text x="830" y="34" text-anchor="middle" class="mute">≤ 1 more try · 1 check</text>
|
||||||
<text x="1000" y="20" text-anchor="middle" class="t" style="fill:var(--mute)">4 · Deliver</text>
|
<text x="1000" y="20" text-anchor="middle" class="t" style="fill:var(--mute)">4 · Deliver</text>
|
||||||
<text x="1000" y="34" text-anchor="middle" class="mute">its delivery plan</text>
|
<text x="1000" y="34" text-anchor="middle" class="mute">until delivered</text>
|
||||||
</g>
|
</g>
|
||||||
<!-- lanes -->
|
<!-- lanes -->
|
||||||
<g>
|
<g>
|
||||||
@@ -216,7 +216,7 @@ code{font:12.5px ui-monospace,monospace;background:var(--soft);padding:1px 5px;b
|
|||||||
<text x="10" y="86" class="t">Lead</text><text x="10" y="102" class="mute">larger model</text>
|
<text x="10" y="86" class="t">Lead</text><text x="10" y="102" class="mute">larger model</text>
|
||||||
<text x="10" y="158" class="t">Fixing agents</text><text x="10" y="174" class="mute">smaller model</text>
|
<text x="10" y="158" class="t">Fixing agents</text><text x="10" y="174" class="mute">smaller model</text>
|
||||||
<text x="10" y="230" class="t">Reviewer</text><text x="10" y="246" class="mute">larger model</text>
|
<text x="10" y="230" class="t">Reviewer</text><text x="10" y="246" class="mute">larger model</text>
|
||||||
<text x="10" y="302" class="t">The mesh</text><text x="10" y="318" class="mute">try, checks, plan</text>
|
<text x="10" y="302" class="t">The mesh</text><text x="10" y="318" class="mute">try, checks, wants</text>
|
||||||
<text x="10" y="374" class="t">Operator</text><text x="10" y="390" class="mute">Telegram</text>
|
<text x="10" y="374" class="t">Operator</text><text x="10" y="390" class="mute">Telegram</text>
|
||||||
</g>
|
</g>
|
||||||
<!-- lead -->
|
<!-- lead -->
|
||||||
@@ -251,8 +251,8 @@ code{font:12.5px ui-monospace,monospace;background:var(--soft);padding:1px 5px;b
|
|||||||
<text x="830" y="294" text-anchor="middle" class="t">Forge checks</text>
|
<text x="830" y="294" text-anchor="middle" class="t">Forge checks</text>
|
||||||
<text x="830" y="312" text-anchor="middle" class="mute">gate · repo · group</text>
|
<text x="830" y="312" text-anchor="middle" class="mute">gate · repo · group</text>
|
||||||
<rect x="930" y="272" width="140" height="52" rx="8" fill="var(--card)" stroke="var(--mesh)" stroke-width="2"/>
|
<rect x="930" y="272" width="140" height="52" rx="8" fill="var(--card)" stroke="var(--mesh)" stroke-width="2"/>
|
||||||
<text x="1000" y="294" text-anchor="middle" class="t">Merge · delivery plan</text>
|
<text x="1000" y="294" text-anchor="middle" class="t">Merge · wants</text>
|
||||||
<text x="1000" y="312" text-anchor="middle" class="mute">backups first, tier by tier</text>
|
<text x="1000" y="312" text-anchor="middle" class="mute">backups first, canary first</text>
|
||||||
<!-- operator -->
|
<!-- operator -->
|
||||||
<rect x="860" y="344" width="140" height="52" rx="8" fill="var(--card)" stroke="var(--op)" stroke-width="2.5"/>
|
<rect x="860" y="344" width="140" height="52" rx="8" fill="var(--card)" stroke="var(--op)" stroke-width="2.5"/>
|
||||||
<text x="930" y="366" text-anchor="middle" class="t">Approve</text>
|
<text x="930" y="366" text-anchor="middle" class="t">Approve</text>
|
||||||
@@ -282,7 +282,7 @@ code{font:12.5px ui-monospace,monospace;background:var(--soft);padding:1px 5px;b
|
|||||||
<li>Into each issue: every file and test it touches (file:line), how it is checked and delivered, a group's merge order.</li>
|
<li>Into each issue: every file and test it touches (file:line), how it is checked and delivered, a group's merge order.</li>
|
||||||
<li><span class="tag new">new</span>Every other reader and writer of each value the change touches, each with a criterion.</li>
|
<li><span class="tag new">new</span>Every other reader and writer of each value the change touches, each with a criterion.</li>
|
||||||
<li>Acceptance criteria: given / when / then, each naming its test level.</li>
|
<li>Acceptance criteria: given / when / then, each naming its test level.</li>
|
||||||
<li>Every part the change reaches is an issue of the milestone; their order is the delivery plan's, never a cut.</li>
|
<li>Every part the change reaches is an issue of the milestone; their order is the assigner's rules', never a cut.</li>
|
||||||
<li>Fixtures copied from what the other side really writes, never from memory.</li>
|
<li>Fixtures copied from what the other side really writes, never from memory.</li>
|
||||||
<li>An issue sized over an hour is split before work starts.</li>
|
<li>An issue sized over an hour is split before work starts.</li>
|
||||||
</ul>
|
</ul>
|
||||||
@@ -465,19 +465,19 @@ const P = [
|
|||||||
{n:"07",name:"Features",kind:"build",c:"--build",
|
{n:"07",name:"Features",kind:"build",c:"--build",
|
||||||
trig:"Work that changes code across issues or repositories, too large for a small fix.",
|
trig:"Work that changes code across issues or repositories, too large for a small fix.",
|
||||||
who:"A lead (larger model), one fixing agent per repository (smaller), one reviewer (larger).",
|
who:"A lead (larger model), one fixing agent per repository (smaller), one reviewer (larger).",
|
||||||
steps:["Prepare: name the branch, make the worktrees, settle every question.","Write one context file per repository.","Write the findings and acceptance criteria into each issue.","Survey every other reader and writer of each changed value. (new)","Answer three questions in the milestone file: who deletes each new key and stream, how the switch-over is tried on one node first, how the integration level runs before the pull requests. The review checks them. (new)","List every part the change reaches; copy fixtures from the real writer.","Develop: one agent per repository, one commit per issue.","A test fails first on behaviour, not on compiling. (new)","Bring each branch up to the trunk, push, and try it. (new)","Review all of it with the tries' results; at most 3 rounds.","Push the reviewed heads; try again only what changed.","A merge by hand in a temporary mode first runs a try with tests \".\" and names it in the pull request. (new)","Open one pull request per repository, together, with Reviewed-by and Pushed-by.","The operator's Approve on Telegram; the group merges in one pass.","Follow the delivery plan to done; resolve the issues; clean up.","When the builder cannot deliver its own fix: revert, a build by a session's call, a registration at the terminal, copies kept outside the store. (new)"],
|
steps:["Prepare: name the branch, make the worktrees, settle every question.","Write one context file per repository.","Write the findings and acceptance criteria into each issue.","Survey every other reader and writer of each changed value. (new)","Answer three questions in the milestone file: who deletes each new key and stream, how the switch-over is tried on one node first, how the integration level runs before the pull requests. The review checks them. (new)","List every part the change reaches; copy fixtures from the real writer.","Develop: one agent per repository, one commit per issue.","A test fails first on behaviour, not on compiling. (new)","Bring each branch up to the trunk, push, and try it. (new)","Review all of it with the tries' results; at most 3 rounds.","Push the reviewed heads; try again only what changed.","A merge by hand in a temporary mode first runs a try with tests \".\" and names it in the pull request. (new)","Open one pull request per repository, together, with Reviewed-by and Pushed-by.","The operator's Approve on Telegram; the group merges in one pass.","Follow the delivery in the status view (moving, settled, stuck, delivered) to done; resolve the issues; clean up. (new)","When the builder cannot deliver its own fix: revert, a build by a session's call, a registration at the terminal, copies kept outside the store; a broken controller is put back by a want the operator writes, a hand build by mesh-compose and a want. (new)"],
|
||||||
prod:"One pull request per repository, one delivery plan, resolved issues, no branch or worktree left.",
|
prod:"One pull request per repository, one source revision delivered, resolved issues, no branch or worktree left.",
|
||||||
next:"02 (when it ships)",bounds:"Prepare 15 min · 30 min per issue · review 15 min, ≤ 3 rounds · one try and one check",
|
next:"02 (when it ships)",bounds:"Prepare 15 min · 30 min per issue · review 15 min, ≤ 3 rounds · one try and one check",
|
||||||
prev:["Pull requests as scratchpads","Context bought again and again","Review rounds spent on what a run would show"],
|
prev:["Pull requests as scratchpads","Context bought again and again","Review rounds spent on what a run would show"],
|
||||||
check:"The checker's one status, mesh/check; the review record; the tracker's move."},
|
check:"The checker's one status, mesh/check; the review record; the tracker's move."},
|
||||||
{n:"08",name:"Small fixes",kind:"build",c:"--build",
|
{n:"08",name:"Small fixes",kind:"build",c:"--build",
|
||||||
trig:"One issue in one repository, about an hour of work.",
|
trig:"One issue in one repository, about an hour of work.",
|
||||||
who:"The session, with one reviewer on the larger model.",
|
who:"The session, with one reviewer on the larger model.",
|
||||||
steps:["Read the issue and the code; settle any open question.","Write the acceptance criteria.","Move the issue to fixing.","Build on fix/<issue>-<slug> in its own worktree.","A test that fails today first, on behaviour. (new)","Push and try before the review. (new)","One review of the commit and its try.","Push the reviewed head; open the pull request.","The operator's Approve; the merge; the delivery plan.","Prove the fix live and resolve the same hour.","During a milestone: ride in its delivery group, unless broken for the operator now. (new)"],
|
steps:["Read the issue and the code; settle any open question.","Write the acceptance criteria.","Move the issue to fixing.","Build on fix/<issue>-<slug> in its own worktree.","A test that fails today first, on behaviour. (new)","Push and try before the review. (new)","One review of the commit and its try.","Push the reviewed head; open the pull request.","The operator's Approve; the merge; the delivery, followed in the status view. (new)","Prove the fix live and resolve the same hour.","During a milestone: ride in its delivery group, unless broken for the operator now. (new)"],
|
||||||
prod:"One pull request, one delivery, a resolved issue.",
|
prod:"One pull request, one delivery, a resolved issue.",
|
||||||
next:"—",bounds:"About an hour · one try before the review, one review, one check",
|
next:"—",bounds:"About an hour · one try before the review, one review, one check",
|
||||||
prev:["A fix reviewed after its pull request opened","A long part run twice","A fix waiting on someone else's delivery","Ten SDK merges and eight controller merges in a day, one per fix"],
|
prev:["A fix reviewed after its pull request opened","A long part run twice","A fix waiting on someone else's delivery","Ten SDK merges and eight controller merges in a day, one per fix"],
|
||||||
check:"The checker's status; the review record; the tracker's move; one delivery group and one delivery plan per milestone."},
|
check:"The checker's status; the review record; the tracker's move; one delivery group and one source revision per milestone."},
|
||||||
{n:"09",name:"Cleanup",kind:"side",c:"--side",
|
{n:"09",name:"Cleanup",kind:"side",c:"--side",
|
||||||
trig:"The operator asks, or merged branches, stale pull requests and old worktrees pile up.",
|
trig:"The operator asks, or merged branches, stale pull requests and old worktrees pile up.",
|
||||||
who:"A session on the operator's word; the operator decides every deletion.",
|
who:"A session on the operator's word; the operator decides every deletion.",
|
||||||
|
|||||||
@@ -1,9 +1,10 @@
|
|||||||
---
|
---
|
||||||
layer: to-be
|
layer: to-be
|
||||||
status: in-progress
|
status: in-progress
|
||||||
code: [mesh-controller, mesh-catalog modules/mesh-delivery, mesh-delivery, mesh-host, mesh-sdk, hq 00-META/checks]
|
code: [mesh-controller, mesh-catalog modules/mesh-delivery-gate, mesh-delivery, mesh-host, mesh-sdk, hq 00-META/checks]
|
||||||
updated: 2026-10-23
|
updated: 2026-10-24
|
||||||
decisions:
|
decisions:
|
||||||
|
- "ADR 0539"
|
||||||
- "ADR 0536"
|
- "ADR 0536"
|
||||||
- "ADR 0534"
|
- "ADR 0534"
|
||||||
- "ADR 0529"
|
- "ADR 0529"
|
||||||
@@ -88,17 +89,19 @@ decisions:
|
|||||||
|
|
||||||
**A change reaches the machines in one of two ways, both owned by mesh-delivery: a hotfix (one pull request,
|
**A change reaches the machines in one of two ways, both owned by mesh-delivery: a hotfix (one pull request,
|
||||||
delivered at its merge) or a feature (one branch name across its repositories: in two or more, one delivery
|
delivered at its merge) or a feature (one branch name across its repositories: in two or more, one delivery
|
||||||
group merged in one pass on the operator's one Approve and delivered by one delivery plan; in one, a lone
|
group merged in one pass on the operator's one Approve as one source revision; in one, a lone delivery). From
|
||||||
delivery). A design's milestones are features delivered one after another. The trunk is what runs; there is no
|
release 6 each node takes a merged change from its wants, the module's canary first (ADR 0539). A design's
|
||||||
"release branch"** (ADR 0401, ADR 0418 §1, ADR 0421 §2).
|
milestones are features delivered one after another. The trunk is what runs; there is no "release branch"**
|
||||||
|
(ADR 0401, ADR 0418 §1, ADR 0421 §2).
|
||||||
|
|
||||||
**From the delivery of issues 1781 to 1783, a milestone is a record of the issue tracker and the unit that is released.** Its issues are linked to it, and pull requests and commits to the issues, by the issue tracker's link verb, never matched from a description; a review is recorded by a verb of the delivery seat; the mesh builds and checks each pull request as it comes in, and asks the operator once to release the milestone when every issue is ready; its pull requests are the delivery group, in whatever repositories and on whatever branches, and nothing ships outside a released milestone (ADR 0493 §1 to §11). Until that delivery the branch-name rules in this design stay what runs, kept by hand (ADR 0493 §12). A change across any repositories, the mesh-sdk included, is one group whose developer orders no release, from R4e's delivery for a group with a mesh-sdk member; until then an SDK change ships before its users (ADR 0482 §1, §7).
|
**From the delivery of issues 1781 to 1783, a milestone is a record of the issue tracker and the unit that is released.** Its issues are linked to it, and pull requests and commits to the issues, by the issue tracker's link verb, never matched from a description; a review is recorded by a verb of the delivery seat; the mesh builds and checks each pull request as it comes in, and asks the operator once to release the milestone when every issue is ready; its pull requests are the delivery group, in whatever repositories and on whatever branches, and nothing ships outside a released milestone (ADR 0493 §1 to §11). Until that delivery the branch-name rules in this design stay what runs, kept by hand (ADR 0493 §12). A change across any repositories, the mesh-sdk included, is one group whose developer orders no release, from R4e's delivery for a group with a mesh-sdk member; until then an SDK change ships before its users (ADR 0482 §1, §7).
|
||||||
|
|
||||||
**Every step starts on an event, and the state lives on the bus.** The forge's events, the controller's
|
**Every step starts on an event, and the state lives on the bus.** The forge's events, the parts' keys and the
|
||||||
events and the nodes' reports move a delivery; the bus's own rules keep the queues, the merge window and the
|
nodes' haves move a delivery; nothing collects merges, plans them or orders them in tiers. From release 6 each
|
||||||
order of a plan's tiers, which only the module graph gives (ADR 0419); and one chain, from module to commit to build to rollout to assignment, is kept in the bus's
|
node applies its wants one module at a time, a new fingerprint reaching a module's canary first and its other
|
||||||
key-value store as its only source (ADR 0406, ADR 0407, ADR 0408). A running delivery never touches merging:
|
nodes once it runs healthy there (ADR 0429 §6, ADR 0539); and one chain, from module to source revision to build
|
||||||
every merge is recorded at once for the next delivery plan of what it touches.
|
to want to have, is kept in the bus's key-value store as its only source (ADR 0406, ADR 0408, ADR 0429 §17). A
|
||||||
|
running delivery never touches merging: a merge moves the source revision, and the wants follow it.
|
||||||
|
|
||||||
This is the delivery of [to-be 10](10-delivery.md) carried out. To-be 10 says what is built: the comparison
|
This is the delivery of [to-be 10](10-delivery.md) carried out. To-be 10 says what is built: the comparison
|
||||||
of source and artifacts decides, and nothing here changes that. This design says who answers for one
|
of source and artifacts decides, and nothing here changes that. This design says who answers for one
|
||||||
@@ -133,23 +136,33 @@ source revision is derived from the keys: every **want** equals its module's **f
|
|||||||
- **The gate** merges a group's pull requests once check, review and the operator's Approve agree on the same
|
- **The gate** merges a group's pull requests once check, review and the operator's Approve agree on the same
|
||||||
heads, and then moves the mesh's source revision in one write.
|
heads, and then moves the mesh's source revision in one write.
|
||||||
- **The teller** carries conditions and asks to the operator's phone and is the only writer of an ask's answer.
|
- **The teller** carries conditions and asks to the operator's phone and is the only writer of an ask's answer.
|
||||||
- **The assigner** writes which fingerprint each node should run, by six rules: the build is done, the
|
- **The assigner** writes which fingerprint each node should run, by seven rules: the build is done, the
|
||||||
fingerprint is not held, the node is the module's **canary** or the canary runs it healthy, at most `step`
|
fingerprint is not held, the node is the module's **canary** or the canary runs it healthy, at most `step`
|
||||||
other nodes are ahead when `step` is above zero, every provider runs healthy, and the module's credential for that fingerprint exists.
|
other nodes are ahead when `step` is above zero, every provider runs healthy, the module's credential for that
|
||||||
|
fingerprint exists, and the controller's **composition** of the module for that node at that fingerprint exists
|
||||||
|
(ADR 0429 §6, ADR 0539 §6). Each want `want.<node>.<module>` names the fingerprint and the digest of the one
|
||||||
|
composition it allows; the assigner also writes the node's own want `want.<node>.node` once the node's
|
||||||
|
composition exists, and never deletes a want (ADR 0539 §1, §6; ADR 0408 §4). A move a person must allow waits
|
||||||
|
for the operator's yes before its want is written (ADR 0539 §14).
|
||||||
It also runs the **composition check**: whether the machines a change touches still compose with it, read
|
It also runs the **composition check**: whether the machines a change touches still compose with it, read
|
||||||
from the keys the indexer and the builder hold and from what the controller keeps of machines, settings and
|
from the keys the indexer and the builder hold and from what the controller keeps of machines, settings and
|
||||||
grants, with the composition it sends with (ADR 0446). Its module
|
grants, with mesh-sdk's composition, the one the controller composes with (ADR 0446). Its module was made in
|
||||||
is made in R4g, in one milestone with release 5: it writes `want.<node>.<module>` by the six rules in its own bucket, never deletes one, and
|
R4g with release 5, when its wants were only compared with what the controller sent; from release 6 it writes
|
||||||
writes every difference between a want and what the controller sends with the rule that explains it (wants are only compared; the node agent
|
no difference, and the controller writes `grants.<module>.<fingerprint>` when a module's build is done
|
||||||
applying them is release 6); the controller writes `grants.<module>.<fingerprint>` where it issues a credential (ADR 0524 §4, ADR 0528).
|
(ADR 0524 §4, ADR 0528, ADR 0539 §7, §8).
|
||||||
- **The node agent**, the node-engine's own process, applies its wants one module at a time, backs up a data
|
- **The node agent**, the node-engine's own process, follows its node's wants and its own bucket of
|
||||||
module first, and writes its haves. A node's declaration is not bounded by one bus message: a file over 64 KiB
|
compositions, applies one module at a time and only what differs from what runs, verifies each composition's
|
||||||
|
signature, backs up a data module first, and writes its haves with the fingerprint and the composition's
|
||||||
|
digest it runs (ADR 0539 §2, §3, §9, §16). A composition is not bounded by one bus message: a file over 64 KiB
|
||||||
travels by digest in the bus's object store, in a bucket of that node's own that no other node can read, and a
|
travels by digest in the bus's object store, in a bucket of that node's own that no other node can read, and a
|
||||||
condition warns before the limit (ADR 0486, ADR 0488).
|
condition warns before the limit (ADR 0486, ADR 0488, ADR 0539 §4).
|
||||||
- **The watchman** turns the bus's connections into presence and mirrors every key whose next move a machine
|
- **The watchman** turns the bus's connections into presence and mirrors every key whose next move a machine
|
||||||
owes into a **pulse**, whose expiry is a stall.
|
owes into a **pulse**, whose expiry is a stall.
|
||||||
- **The status view** is the operator's command `mesh-delivery-cli` and a web view; it reads every bucket and
|
- **The status view** is the operator's command `mesh-delivery-cli` and a web view, whose wants page sets each
|
||||||
writes only its own answers to asks.
|
want beside its node's have; it reads every bucket, writes only its own answers to asks, and says four events
|
||||||
|
derived from the keys: `delivered` once for each source revision whose wants and haves all agree, and
|
||||||
|
`moving`, `settled` and `stuck` once for a module on a node at one composition (ADR 0539 §17; *A delivery*
|
||||||
|
below).
|
||||||
|
|
||||||
Nine of these are modules of the repository `mesh-delivery`, prefixed `mesh-delivery-` (ADR 0429 §2); the
|
Nine of these are modules of the repository `mesh-delivery`, prefixed `mesh-delivery-` (ADR 0429 §2); the
|
||||||
repository is made by the forge module from its config (ADR 0428; issue 1128 makes the config value reach the
|
repository is made by the forge module from its config (ADR 0428; issue 1128 makes the config value reach the
|
||||||
@@ -158,37 +171,43 @@ node at once). The reviewer stays the review agent's session.
|
|||||||
It lands in six releases, each leaving the mesh working, each one milestone below (ADR 0429 §7); releases 4 and 5 end in one milestone, R4g,
|
It lands in six releases, each leaving the mesh working, each one milestone below (ADR 0429 §7); releases 4 and 5 end in one milestone, R4g,
|
||||||
with one switch-over and no rehearsal (ADR 0524 §1). Until the
|
with one switch-over and no rehearsal (ADR 0524 §1). Until the
|
||||||
release that replaces it is delivered, each section from here to *The bootstrap* stays this design and its rule
|
release that replaces it is delivered, each section from here to *The bootstrap* stays this design and its rule
|
||||||
(ADR 0429 §21); each is rewritten when the release that replaces it is prepared (issue 1130).
|
(ADR 0429 §21); each is rewritten when the release that replaces it is prepared (issue 1130). Release 6's were
|
||||||
|
rewritten when it was prepared, from ADR 0539: until its delivery the plans of release 5 run, as the as-is
|
||||||
|
design says, and ADR 0539's clauses hold from that delivery (ADR 0539 §20).
|
||||||
|
|
||||||
## The parts
|
## The parts
|
||||||
|
|
||||||
|
From release 6 (ADR 0539):
|
||||||
|
|
||||||
```
|
```
|
||||||
forge (system webhook) ──forge.pull-request.* / merged / status──► the bus ◄──built / checked / plan-moved── controller
|
forge ──forge messages──► forge bridge ──► the bus ◄── done keys ── builder, build modules (build seats)
|
||||||
│ ▲ (planner, order,
|
│ ▲
|
||||||
│ │ registration,
|
gate: the merge, the source revision ◄┤ ├── indexer: each module's fingerprint
|
||||||
▼ │ sending)
|
controller: signed compositions, ◄┤ ├── assigner: each node's wants, by seven rules
|
||||||
mesh-delivery │
|
credentials and grants │ ├── watchman: presence, pulses, stuck keys
|
||||||
(deliveries, groups, a key per │
|
│ └── status view: delivered, moving, settled, stuck
|
||||||
pull request, the state table) ▼
|
▼
|
||||||
node-engine on every node
|
node agent in the node-engine on every node
|
||||||
build modules (build seats) ◄── asks on streams: delivery, check, recheck ────────────────────────────── (reports; backups first)
|
(reads its wants and its compositions; backs up first; writes its haves)
|
||||||
```
|
```
|
||||||
|
|
||||||
- **mesh-delivery** (the seat `mesh-delivery`) owns hotfixes, features and milestones as deliveries and delivery
|
- **The gate** (module `mesh-delivery-gate`, holding the seat `mesh-delivery`) owns hotfixes, features and
|
||||||
groups, decides each step on events, keeps one key per pull request, and asks the controller for each act
|
milestones up to the merge: it keeps one key per pull request, asks the operator's Approve, merges a group in
|
||||||
and the nodes' backup seats for the backups a delivery needs. It never sends to a machine, writes a
|
one pass and moves the source revision in one write. It asks the controller nothing, consumes no event of a
|
||||||
declaration or registers a build (ADR 0239 §25).
|
plan, never sends to a machine and writes no composition (ADR 0539 §17).
|
||||||
- **The controller** keeps planning, ordering, registration, sending and judging. It writes the chain's
|
- **The controller** keeps registration, the inputs of every node (settings and given values, secrets,
|
||||||
links it owns — builds, rollouts with their tiers, assignments — and the merge window's
|
identities and memberships, credentials, the facts), placement in its assignments bucket, and composing: it
|
||||||
keys, all on the bus, by compare-and-set, while it holds the lease.
|
writes each module's signed composition for each node, its digest and its judgement of the move, by
|
||||||
|
compare-and-set while it holds the lease, and issues a module's credential and grant when its build is done.
|
||||||
|
It plans nothing, orders nothing and sends nothing (ADR 0539 §1, §2, §5, §7, §12).
|
||||||
- **The build modules** build and check with toolchains installed on their node, each as an account of its own, and pull
|
- **The build modules** build and check with toolchains installed on their node, each as an account of its own, and pull
|
||||||
their asks from the bus's streams, deliveries first (ADR 0405, ADR 0525).
|
their asks from the bus's streams, deliveries first (ADR 0405, ADR 0525).
|
||||||
- **The forge module** (`gitea`) publishes every forge event on the bus through the forge's system webhook
|
- **The forge module** (`gitea`) publishes every forge event on the bus through the forge's system webhook
|
||||||
(issue 729): pull requests updated, synchronized, merged and closed, and commit statuses; no label or comment
|
(issue 729): pull requests updated, synchronized, merged and closed, and commit statuses; no label or comment
|
||||||
moves a delivery, and mesh-delivery subscribes to none (ADR 0421 §11). It sets statuses, keeps the delivery's view as one comment on the pull request, and appends
|
moves a delivery, and the gate subscribes to none (ADR 0421 §11). It sets statuses, keeps the delivery's view
|
||||||
to the commit's note under `refs/notes/mesh-plan`.
|
as one comment on the pull request, and appends to the commit's note under `refs/notes/mesh-plan`.
|
||||||
- **Each node's backup seat** (`node-backup`) backs up the data a delivery plan moves there before the plan
|
- **Each node's backup seat** (`node-backup`) backs up the data a module keeps on its node before the node agent
|
||||||
sends anything (ADR 0404).
|
applies a new fingerprint of it, and says on the bus when that backup ended (ADR 0404, ADR 0539 §16).
|
||||||
|
|
||||||
## Two ways a change reaches the machines
|
## Two ways a change reaches the machines
|
||||||
|
|
||||||
@@ -196,8 +215,8 @@ release that replaces it is delivered, each section from here to *The bootstrap*
|
|||||||
|
|
||||||
| Way | What it is | How it is delivered |
|
| Way | What it is | How it is delivered |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| **hotfix** | one pull request in one repository | at its merge, as a lone delivery; a lone merge is cut at once (ADR 0282 §2, as ADR 0304 §1 changed it) |
|
| **hotfix** | one pull request in one repository | at its merge: it moves the source revision, and each module it changes is wanted on its nodes by the assigner's rules (ADR 0539) |
|
||||||
| **feature** | one branch name across its repositories; in two or more, its pull requests are one delivery group; in one, a single pull request | a group is merged in one pass once its composed check passed at every member's head and the operator gave the group's one Approve (ADR 0418 §4, ADR 0421 §2); its merges are one batch and one delivery plan, each module built once and sent once (ADR 0418 §1); in one repository, as a lone delivery like a hotfix |
|
| **feature** | one branch name across its repositories; in two or more, its pull requests are one delivery group; in one, a single pull request | a group is merged in one pass once its composed check passed at every member's head and the operator gave the group's one Approve (ADR 0418 §4, ADR 0421 §2); its merges are one source revision, each module built once and wanted on each node that runs it (ADR 0418 §1, ADR 0429 §11); in one repository, as a lone delivery like a hotfix |
|
||||||
|
|
||||||
- **The trunk is what runs.** Every delivery is cut from a repository's trunk. A merge into any other branch
|
- **The trunk is what runs.** Every delivery is cut from a repository's trunk. A merge into any other branch
|
||||||
starts no delivery, and the controller says so for that merge (issue 889). Holding merged work back, on a
|
starts no delivery, and the controller says so for that merge (issue 889). Holding merged work back, on a
|
||||||
@@ -205,10 +224,11 @@ release that replaces it is delivered, each section from here to *The bootstrap*
|
|||||||
branches held the controller for 55.7 h, and 28 of 68 other merges touched a held module (research 055,
|
branches held the controller for 55.7 h, and 28 of 68 other merges touched a held module (research 055,
|
||||||
finding 3).
|
finding 3).
|
||||||
- **A design's milestones are features, one after another.** Each milestone is delivered as a feature when its
|
- **A design's milestones are features, one after another.** Each milestone is delivered as a feature when its
|
||||||
own group is ready; the next starts when the previous one's delivery plan is done (playbook 04).
|
own group is ready; the next starts when the previous one's source revision is delivered (playbook 04).
|
||||||
- **A milestone is functional, its delivery shaped technically.** A milestone is named by what the operator or
|
- **A milestone is functional, its delivery shaped technically.** A milestone is named by what the operator or
|
||||||
the mesh can do once it is delivered; how it ships is shaped by the delivery plan its changes would have, asked of
|
the mesh can do once it is delivered; how it ships is shaped by the modules its changes move and the nodes that
|
||||||
the controller's planner through mesh-delivery's `what-if` (ADR 0239 §3): the modules it moves and their fan-out.
|
run them, read from the indexer's candidates of its group through the view's `indexer_candidates` (the gate's
|
||||||
|
`what-if` went at release 6, ADR 0539 §17).
|
||||||
- **What a merge waits for.** After its checks, a merge waits for one act of a person: the operator's Approve,
|
- **What a merge waits for.** After its checks, a merge waits for one act of a person: the operator's Approve,
|
||||||
asked as soon as both required checks passed at the head (ADR 0421 §1, §3). For a group the Approve is asked
|
asked as soon as both required checks passed at the head (ADR 0421 §1, §3). For a group the Approve is asked
|
||||||
once, when every member's checks and the group's composed check passed, and covers every member's head
|
once, when every member's checks and the group's composed check passed, and covers every member's head
|
||||||
@@ -218,22 +238,33 @@ release that replaces it is delivered, each section from here to *The bootstrap*
|
|||||||
|
|
||||||
## A delivery
|
## A delivery
|
||||||
|
|
||||||
Its id is `<owner>/<repository>@<twelve characters of its commit>`. It holds:
|
*ADR 0239, ADR 0429 §1, ADR 0539 §17.*
|
||||||
|
|
||||||
|
Its id is `<owner>/<repository>@<twelve characters of its commit>`. The gate holds, up to the merge:
|
||||||
|
|
||||||
- the pull request (number, base, head branch, description), or none for a commit that reached the trunk
|
- the pull request (number, base, head branch, description), or none for a commit that reached the trunk
|
||||||
without one;
|
without one;
|
||||||
- its **delivery plan**: build plan, deploy plan and verdict, as the controller's planner computed them for
|
- its **state** and its **transitions** (the last hundred, each with when, the event, from, to and why);
|
||||||
its diffset (ADR 0238 §2, as ADR 0323 §1 changed it);
|
|
||||||
- its **state**, its **transitions** (the last hundred, each with when, the event, from, to and why), and its
|
|
||||||
**machine steps** while delivering;
|
|
||||||
- its **Approve**: the ask, the operator's answer with its warrant, and the review record the pull request was
|
- its **Approve**: the ask, the operator's answer with its warrant, and the review record the pull request was
|
||||||
opened with (ADR 0421 §5, §8);
|
opened with (ADR 0421 §5, §8);
|
||||||
- once merged, **the commit it landed on the trunk as**, and the delivery plan that delivers it: the plan of the
|
- once merged, **the commit it landed on the trunk as** and the source revision its merge moved. Every member of
|
||||||
batch that answers its merge, which may carry a later commit of its repository that contains it. Every member
|
a delivery group moves the same source revision (ADR 0429 §11).
|
||||||
of a delivery group is delivered by the same plan, and its delivery ends with that plan's result
|
|
||||||
(ADR 0418 §1).
|
|
||||||
|
|
||||||
**One key per pull request.** Beside the delivery, mesh-delivery keeps one key per open pull request on the
|
**Delivered is derived, never recorded.** A source revision is delivered when every want of a module the
|
||||||
|
indexer holds equals that module's fingerprint and every have equals its want, running and healthy (ADR 0429
|
||||||
|
§1). No part records it: the status view derives it from `source.main`, the wants and the haves, and says the
|
||||||
|
event `delivered` once for each source revision that reaches it. The issue tracker hears that event to say a
|
||||||
|
fix delivered, and a newer delivered revision covers the older merges of its repository, as the forge says one
|
||||||
|
contains the other (issue 1946).
|
||||||
|
|
||||||
|
**A module on a node is moving, settled or stuck.** The status view says `moving` when a module's want on a
|
||||||
|
node differs from its have, `settled` when the have reaches the want running and healthy, and `stuck` when the
|
||||||
|
module is held, its health check failed, or the watchman says the want stopped moving past its bound (thirty
|
||||||
|
minutes); each once for a node, a module and a composition. The messenger holds the operator's notices that
|
||||||
|
touch that module on that node from `moving` until `settled` or `stuck`, then tells them once with how it
|
||||||
|
ended, as ADR 0505 §3 has it for a delivery in motion (issue 1946).
|
||||||
|
|
||||||
|
**One key per pull request.** Beside the delivery, the gate keeps one key per open pull request on the
|
||||||
bus: its head, the trunk commit its current check is asked against, and its verdict, written by
|
bus: its head, the trunk commit its current check is asked against, and its verdict, written by
|
||||||
compare-and-set (ADR 0406 §4). It writes a new head when the forge says one, and a new trunk commit only when it
|
compare-and-set (ADR 0406 §4). It writes a new head when the forge says one, and a new trunk commit only when it
|
||||||
asks the pull request's check again (ADR 0395 §1). A running check watches that key and stops when the head or
|
asks the pull request's check again (ADR 0395 §1). A running check watches that key and stops when the head or
|
||||||
@@ -241,69 +272,60 @@ the trunk commit in it moves.
|
|||||||
|
|
||||||
## The state table
|
## The state table
|
||||||
|
|
||||||
One table, compiled into mesh-delivery, read by its code, its test, its `table` verb, the controller's
|
*ADR 0539 §17.*
|
||||||
self-check and healer H2. Each state has a **bound**: how long it may be held before `stalled` lists it.
|
|
||||||
Each state also says **what H2 may do** after the bound, which is only ever a transition the table already
|
|
||||||
allows.
|
|
||||||
|
|
||||||
| State | Means | Bound | H2 after the bound |
|
One table, compiled into the gate, read by its code, its test and its `table` verb. It holds the merge's states
|
||||||
|---|---|---|---|
|
only: what follows the merge is the wants' and the haves', and the view derives it (*A delivery* above). Each
|
||||||
| proposed | a head was announced and is being checked | 1 hour from when its check is asked; none while its verdict is out of date and no check is asked | nothing: the check's own watchdog (S6) speaks |
|
state has a **bound**: how long it may be held before `stalled` lists it, for a merge or a check past its bound.
|
||||||
| checked | a verdict arrived | 1 minute | nothing: it is decided at once |
|
|
||||||
| ready | it passed; its Approve is asked (a group member's once the group is ready), and it waits for it and the merge | none | — |
|
| State | Means | Bound |
|
||||||
| rejected | it failed its check | none | — |
|
|---|---|---|
|
||||||
| published | merged on its trunk, recorded in a batch; it waits for its plan to start | 30 minutes | `superseded` when a plan that does not answer its merge took over its plan; `go` when its plan started |
|
| proposed | a head was announced and is being checked | 1 hour from when its check is asked; none while its verdict is out of date and no check is asked |
|
||||||
| held | it waits for a person: a failed backup (ADR 0404 §4), a policy of record; it ends by itself when it moves nothing or a later merge covers it (ADR 0270) | 1 day | nothing: it is the operator's |
|
| ready | it passed; its Approve is asked (a group member's once the group is ready), and it waits for it and the merge | none |
|
||||||
| delivering | its plan runs: backups, then builds asked and sent tier by tier | 2 hours | `delivered`, `failed` or `superseded`, as the plan's key says |
|
| rejected | it failed its check | none |
|
||||||
| delivered, failed, superseded, stopped, retired, covered | final | — | — |
|
| held | it waits for a person or a fresh check: an approved merge unchecked at the trunk's head, a conflict, a member the forge refused | 1 day |
|
||||||
|
| merged | merged on its trunk; its source revision moved | final |
|
||||||
|
| closed | closed without a merge | final |
|
||||||
|
|
||||||
**A verdict holds for one trunk commit** (ADR 0323). The check judges the head merged into the trunk's head
|
**A verdict holds for one trunk commit** (ADR 0323). The check judges the head merged into the trunk's head
|
||||||
when the builder clones it, and the verdict names both. When another merge moves the trunk, a `ready`
|
when the build module clones it, and the verdict names both. When another merge moves the trunk, a `ready`
|
||||||
delivery goes back to `proposed` on the event `trunk moved`, and mesh-delivery asks the check of every active,
|
delivery goes back to `proposed` on the event `trunk moved`, and the gate asks the check of every active,
|
||||||
cleanly merging open pull request into that trunk at once (ADR 0395, *A move of the trunk asks every check
|
cleanly merging open pull request into that trunk at once (ADR 0395, *A move of the trunk asks every check
|
||||||
again* below). An Approve already asked stays; the merge waits for the fresh check, and the Approve is not asked
|
again* below). An Approve already asked stays; the merge waits for the fresh check, and the Approve is not asked
|
||||||
again for a trunk move (ADR 0421 §1, §3). The row from `proposed` to `checked` takes only a verdict that names the trunk's head.
|
again for a trunk move (ADR 0421 §1, §3). The row from `proposed` to `ready` or `rejected` takes only a verdict
|
||||||
|
that names the trunk's head.
|
||||||
|
|
||||||
**What needs no person ends by itself** (ADR 0270): a merge that moves nothing is `delivered` at its merge; a
|
**Nothing of a merged delivery is kept moving in the gate.** The delivery states of the plans (delivering, the
|
||||||
merge a later delivered merge contains is `covered`, by the forge's word that the later merge commit contains
|
backup hold, superseded, covered, stopped, retired and failed) and the machine steps went at release 6: a
|
||||||
it; a merge whose plan said its modules unchanged is `delivered` when that plan ends (issue 611); a stop whose
|
module's progress on each node is its want against its have, shown by the view (ADR 0539 §17).
|
||||||
plan ended without building its modules is read as covered once a later merge covers it (issues 474, 480).
|
|
||||||
|
|
||||||
**History is retired, not held** (ADR 0250). A delivery made from the forge's word at its merge, heard fifteen
|
|
||||||
minutes or more after it, with no plan of it open, goes to `retired` at adoption. **Nothing of a published
|
|
||||||
delivery is registered before its plan starts.**
|
|
||||||
|
|
||||||
**The machine steps.** While a delivery is `delivering`, each machine its plan reaches has a row with the
|
|
||||||
module and the build: `backed-up`, `sent`, `applied` or `failed`, and `silent` when no report came within
|
|
||||||
ApplySilentAfter (fifteen minutes). They are read from the plan's key, written by the controller from the
|
|
||||||
nodes' reports, never inferred from time.
|
|
||||||
|
|
||||||
## Order without a person
|
## Order without a person
|
||||||
|
|
||||||
*ADR 0419, ADR 0418 §2.*
|
*ADR 0419 §1, §2, ADR 0429 §6, ADR 0539 §6, §7.*
|
||||||
|
|
||||||
A delivery group has no order among its members, and nobody writes one (ADR 0419 §1). Inside a plan only the
|
A delivery group has no order among its members, and nobody writes one (ADR 0419 §1). From release 6 there are
|
||||||
module graph orders: the planner orders the build and deploy tiers by edges between modules, never between
|
no tiers: no plan orders modules into steps. What reaches a node, and when, is the assigner's seven rules, read
|
||||||
repositories or members (ADR 0418 §2, ADR 0419 §3). Modules with no edge between them share a tier.
|
per module and node from the keys:
|
||||||
|
|
||||||
| Edge | A module goes before | Read from |
|
- **A build before its want.** No want names a fingerprint whose build is not done; a module whose build waits
|
||||||
|---|---|---|
|
is named by nothing and fails nothing (rule 1; ADR 0539 §18).
|
||||||
| *built by* and *stands on* | the modules built by it or standing on it | the module graph |
|
- **A provider before its consumers.** A module's new fingerprint is wanted on a node only while every provider
|
||||||
| *SDK before its users* | every module built on the SDK, `mesh-sdk` | the module graph |
|
it requires runs healthy (rule 5).
|
||||||
| *verb before caller* | a module whose change adds to its invoked verbs a verb this module's change adds to a seat's served verbs | the manifests |
|
- **The canary before the rest.** A new fingerprint goes to the module's canary first, and to its other nodes
|
||||||
| *version skew* | (the controller, when the plan moves it) every module whose manifest the plan changes | the change |
|
once it runs healthy there, at most `step` of them ahead when `step` is above zero (rules 3 and 4).
|
||||||
|
- **A credential and a composition before the want.** The module's credential and grant are written when its
|
||||||
- **Engine before controller is the one default.** When a plan moves both the node-engine and the controller and
|
build is done, and its composition for the node must exist, before a want can name it (rules 6 and 7;
|
||||||
no edge orders them, the node-engine's tier comes first; an edge either way wins over it (ADR 0419 §4).
|
ADR 0539 §7).
|
||||||
- **Each tier says the edge that placed it.** The plan, `delivery-order`, `show` and the plan's note name, for
|
- **An SDK change ships with its users.** mesh-sdk is a library built into the modules that use it, so no
|
||||||
each module placed after another, the edge and its rule, and where the default was overruled, the edge that
|
module waits for it to be sent anywhere (ADR 0482 §1).
|
||||||
overruled it (ADR 0419 §5).
|
- **No `after:` line is read.** The gate reads no `after:` line, no refusal asks for one, and a description
|
||||||
- **A cycle is refused before anything merges.** Edges that form a cycle fail the group's composed check, or a
|
|
||||||
lone pull request's check, naming every module and edge in it. The way out is in the change itself (a verb
|
|
||||||
promised optional before it is required, ADR 0246), never an order typed by a person (ADR 0419 §6).
|
|
||||||
- **No `after:` line is read.** mesh-delivery reads no `after:` line, no refusal asks for one, and a description
|
|
||||||
that holds one is told once on its pull request that the line is not read (ADR 0419 §2).
|
that holds one is told once on its pull request that the line is not read (ADR 0419 §2).
|
||||||
|
|
||||||
|
The planner's edges (*built by*, *SDK before its users*, *verb before caller*, *version skew*) and its one
|
||||||
|
default, the node-engine before the controller, ordered the tiers of a plan until release 6 (ADR 0419 §3 to
|
||||||
|
§5). For release 6's own switch, the last plan of release 5's controller, the version-skew edge and that
|
||||||
|
default yielded so the node-engine and the controller shared one tier (ADR 0539 §11, issue 1942).
|
||||||
|
|
||||||
**The SDK by version.** A Go module names the SDK by its released version and fetches it from the mesh's forge,
|
**The SDK by version.** A Go module names the SDK by its released version and fetches it from the mesh's forge,
|
||||||
checked against its recorded checksum; vendored copies go. Inside a group, a member may name the SDK member's
|
checked against its recorded checksum; vendored copies go. Inside a group, a member may name the SDK member's
|
||||||
head: its merge check resolves that commit from the forge and builds against it, and the head stays fetchable
|
head: its merge check resolves that commit from the forge and builds against it, and the head stays fetchable
|
||||||
@@ -335,9 +357,8 @@ commit on the SDK's trunk or the head of an SDK pull request the forge holds, op
|
|||||||
- **A failed build node loses nothing.** An ask a build module took and never acknowledged goes to another node by the
|
- **A failed build node loses nothing.** An ask a build module took and never acknowledged goes to another node by the
|
||||||
stream's own acknowledgement wait, with no timer of a part; a step's end is written on the build module's step key named
|
stream's own acknowledgement wait, with no timer of a part; a step's end is written on the build module's step key named
|
||||||
by the ask's id (ADR 0525 §6).
|
by the ask's id (ADR 0525 §6).
|
||||||
- **A change is built once** (ADR 0350). A check that passes also builds what its merge would move into the
|
- **A change is built once** (ADR 0350). A build is keyed by its module's fingerprint, so the merge commit's
|
||||||
artifact store's scratch namespace under a reuse key, and the plan of the merge commit copies the build of the
|
fingerprints find the builds the check already made, and the merge builds nothing again (ADR 0453 clause 5).
|
||||||
same key, named by the check's verdict, instead of building.
|
|
||||||
|
|
||||||
### A move of the trunk asks every check again
|
### A move of the trunk asks every check again
|
||||||
|
|
||||||
@@ -369,7 +390,8 @@ commit on the SDK's trunk or the head of an SDK pull request the forge holds, op
|
|||||||
operator's one Approve covers those heads, mesh-delivery merges every member in one pass, no member waiting for
|
operator's one Approve covers those heads, mesh-delivery merges every member in one pass, no member waiting for
|
||||||
another's merge or delivery (ADR 0418 §4). The Approve is the review ADR 0382 §6 asks: the merge tool reads its
|
another's merge or delivery (ADR 0418 §4). The Approve is the review ADR 0382 §6 asks: the merge tool reads its
|
||||||
warrant for those heads in place of an approving forge review (ADR 0421 §9). A member the forge refuses is
|
warrant for those heads in place of an approving forge review (ADR 0421 §9). A member the forge refuses is
|
||||||
merged again on its fresh check while the group's batch stays open for it.
|
merged again on its fresh check while the group waits in the gate, `held`, and its source revision does not move
|
||||||
|
(ADR 0539 §17).
|
||||||
|
|
||||||
### The check from release 4
|
### The check from release 4
|
||||||
|
|
||||||
@@ -395,7 +417,7 @@ From its delivery, a check is this:
|
|||||||
- **What the status says.** Its line says the verdict, the tests run and kept at each level, the machines
|
- **What the status says.** Its line says the verdict, the tests run and kept at each level, the machines
|
||||||
composed, and how long each part took: waiting in the queue, cloning, the unit level, the integration level's
|
composed, and how long each part took: waiting in the queue, cloning, the unit level, the integration level's
|
||||||
setup and its running, and the composition. A failure names the test or the machine first. The line links to
|
setup and its running, and the composition. A failure names the test or the machine first. The line links to
|
||||||
the check's full report, which carries the delivery plan's line too (ADR 0447 §3).
|
the check's full report, which names the modules the change moves too (ADR 0447 §3).
|
||||||
- **Tests by the language's convention.** The checker runs a change's tests through the build module of each
|
- **Tests by the language's convention.** The checker runs a change's tests through the build module of each
|
||||||
module's language (milestone 6's build modules, ADR 0405), with no manifest field and no script per
|
module's language (milestone 6's build modules, ADR 0405), with no manifest field and no script per
|
||||||
repository: in Go, `go test` is the unit level and the `integration` build tag the other, so the two levels
|
repository: in Go, `go test` is the unit level and the `integration` build tag the other, so the two levels
|
||||||
@@ -423,7 +445,7 @@ From its delivery, a check is this:
|
|||||||
composes only the machines that run or could run a module the change moves or adds, or whose grants or
|
composes only the machines that run or could run a module the change moves or adds, or whose grants or
|
||||||
providers it alters; for a group, once over every member's change, read from the indexer's candidates of the
|
providers it alters; for a group, once over every member's change, read from the indexer's candidates of the
|
||||||
group, and while the indexer holds none the group's check waits and says why. It composes with the function the
|
group, and while the indexer holds none the group's check waits and says why. It composes with the function the
|
||||||
controller sends with, moved into mesh-sdk, so the two agree for one mesh-sdk version; its verdict names the
|
controller composes with, moved into mesh-sdk, so the two agree for one mesh-sdk version; its verdict names the
|
||||||
mesh-sdk version it composed with, and while the assigner and the controller run different versions the
|
mesh-sdk version it composed with, and while the assigner and the controller run different versions the
|
||||||
controller's self-check raises a condition until they agree. No judge is built from a change: a change to the
|
controller's self-check raises a condition until they agree. No judge is built from a change: a change to the
|
||||||
composition is proved by its own tests, a field only a newer composition reads goes out as optional first, and
|
composition is proved by its own tests, a field only a newer composition reads goes out as optional first, and
|
||||||
@@ -438,7 +460,7 @@ From its delivery, a check is this:
|
|||||||
- **Asked as soon as the checks pass.** When a pull request's delivery reaches `ready` (both `mesh/merge-gate` and
|
- **Asked as soon as the checks pass.** When a pull request's delivery reaches `ready` (both `mesh/merge-gate` and
|
||||||
`mesh/repo-check` passed at its head against its trunk's head), mesh-delivery asks the operator's Approve of its
|
`mesh/repo-check` passed at its head against its trunk's head), mesh-delivery asks the operator's Approve of its
|
||||||
merge into main at once, on a channel that proves who answered (ADR 0274; Telegram today). The ask names the
|
merge into main at once, on a channel that proves who answered (ADR 0274; Telegram today). The ask names the
|
||||||
pull request, its issues, the reviewer's verdict, the delivery plan and the trunk commit the checks judged. A
|
pull request, its issues, the reviewer's verdict, the modules it moves and the trunk commit the checks judged. A
|
||||||
verdict that names a trunk commit the trunk had already left raises no ask: mesh-delivery asks that head's check
|
verdict that names a trunk commit the trunk had already left raises no ask: mesh-delivery asks that head's check
|
||||||
against the trunk's head, and its passing verdict raises the ask (ADR 0421 §1).
|
against the trunk's head, and its passing verdict raises the ask (ADR 0421 §1).
|
||||||
- **A group is asked once.** One ask names every member and its head, once every member is `ready` and the
|
- **A group is asked once.** One ask names every member and its head, once every member is `ready` and the
|
||||||
@@ -464,105 +486,130 @@ From its delivery, a check is this:
|
|||||||
- **Recorded.** The ask, the answer with its warrant and the review record are kept on the delivery, shown by
|
- **Recorded.** The ask, the answer with its warrant and the review record are kept on the delivery, shown by
|
||||||
`show` and said in mesh-delivery's log; the merge's transition names the Approve (ADR 0421 §8).
|
`show` and said in mesh-delivery's log; the merge's transition names the Approve (ADR 0421 §8).
|
||||||
|
|
||||||
## The merge window and the batch
|
## After the merge: the source revision and the wants
|
||||||
|
|
||||||
*ADR 0276, ADR 0282, ADR 0299, ADR 0304, ADR 0407.*
|
*ADR 0429 §1, §6, §11, ADR 0539 §5, §6, §8, §18.*
|
||||||
|
|
||||||
- **Every merge is recorded at once.** The forge's merged event records a merge into the open batch of what it
|
- **A merge moves the source revision, and nothing collects it.** The gate writes a group's merged heads as one
|
||||||
touches, or opens one, by whichever controller holds the plans. A merge is never refused, held or dropped,
|
revision of `source.main` in one write (ADR 0429 §11). There is no merge window, no batch and no plan: the
|
||||||
and needs no person's release (issue 921). The bus's durable consumer keeps every merge event until it is
|
indexer writes each module's fingerprint at the merge, the builder claims each new fingerprint once, and the
|
||||||
recorded; no catch-up timer looks for missed merges.
|
assigner writes each want when its seven rules hold. A merge is never refused, held or dropped once it is made.
|
||||||
- **The window is two keys.** An open batch's window is a key whose time to live is `merge-window` (90 s),
|
- **Two merges close together are two revisions.** The second's fingerprints replace the first's in the
|
||||||
refreshed by each merge into the batch, beside a key set once when the batch opens whose time to live is
|
indexer's modules; a node gets the newer fingerprint's want once its rules hold, and a fingerprint nobody wants
|
||||||
`merge-window-at-most` (10 minutes). The first to expire closes the batch, heard by a watcher; no cutter
|
any more is never applied. Nothing waits for a window to close.
|
||||||
polls. A stop that records merges again into the open batch closes its window too (ADR 0299 §3).
|
- **Changes that share nothing never wait for each other.** Each module's want moves on its own build, canary
|
||||||
- **A group's merges are one batch.** A batch holding a merge of a delivery group with a member not yet merged
|
and providers, so a hotfix is never held by an unrelated feature's delivery.
|
||||||
does not close when either key expires; once the group's last member merged, the window runs again from that
|
- **A group reaches the nodes as one source revision.** Each of its modules gets its wants by the same rules,
|
||||||
merge. A group whose last member has not merged ten minutes (`merge-window-at-most`) after its first raises a
|
and the group is delivered when its revision is (*A delivery* above). A member the forge refuses holds the
|
||||||
condition naming the member and why, and nothing of the group is delivered while it waits (ADR 0418 §3).
|
group in the gate before its source revision moves (ADR 0418 §4, ADR 0539 §17).
|
||||||
- **A lone merge is cut at once**: no batch it shares a repository, module, graph edge or delivery group member
|
- **A failure is held on the canary and fixed forward.** A fingerprint that fails its health check on the
|
||||||
with is assembling (ADR 0282 §2, ADR 0304 §1, ADR 0306).
|
canary is held there and goes no further; the canary is set back, or for a module that keeps data the
|
||||||
- **The plan.** A closed batch becomes one delivery plan: every commit in it, tiered along the module graph
|
operator is asked whether to restore it and lose newer writes or leave it stopped, and the condition names
|
||||||
across repositories, at most one commit per repository, the latest of its trunk, each module it moves built once
|
the held fingerprint (ADR 0429 §6, ADR 0539 §18, §19). The fix is a new change; what other modules reached
|
||||||
and sent once to each node that runs it. It answers every merge in the batch, its own commits and the earlier
|
stays.
|
||||||
merges a later commit contains (ADR 0276 §2, as ADR 0299 §7 changed it; ADR 0418 §1). No member of a group has
|
- **The comparison with the send ended with the send.** From release 6 the assigner writes no difference; its
|
||||||
a turn: only the module graph orders its tiers (ADR 0418 §2).
|
differences bucket is kept read-only as the evidence of acceptance test 17 (ADR 0539 §8).
|
||||||
- **A plan fixes its commits when it starts.** A plan starts at its first send, after mesh-delivery's word, given once for a
|
- **Shown.** The view's wants page sets each node's want beside its have, with the controller's judgement of
|
||||||
group's plan when every member is merged (ADR 0418 §2), and
|
the move; its tools `assigner_wants`, `assigner_want`, `assigner_differences` and `node_haves` read the same
|
||||||
from then nothing that merges changes it. A plan still waiting for its word is folded into the next batch
|
keys, and its events `delivered`, `moving`, `settled` and `stuck` say them on the bus.
|
||||||
(ADR 0276 §4).
|
|
||||||
- **Plans that share something run one after another.** A batch that shares a repository, a module or a graph
|
|
||||||
edge with an open plan waits for it, and starts with every merge recorded since when that plan ends. A batch
|
|
||||||
that shares none runs beside the open ones, so a hotfix is never held by an unrelated feature's delivery.
|
|
||||||
- **The ends.** A plan done ends every delivery it answers: `delivered` for its own commits, `covered` for an
|
|
||||||
earlier merge it carried by a later commit. A plan that failed fails its own commits' deliveries; each earlier
|
|
||||||
merge it answered is then delivered on its own commit, the newest first, until one is delivered (ADR 0276 §3). A
|
|
||||||
group's plan fails as one: no member is stopped because another failed, what earlier tiers delivered stays, and
|
|
||||||
the fix is a new change (ADR 0418 §5).
|
|
||||||
- **Shown.** `plans` lists the open batch first, with its window's time left and its merges, and a batch waiting
|
|
||||||
behind an open plan names that plan; the controller says the batch as `plan-moved` in the state `assembling`.
|
|
||||||
|
|
||||||
## A backup before the send
|
## A backup before the apply
|
||||||
|
|
||||||
*ADR 0214 §7, ADR 0233, ADR 0404.*
|
*ADR 0214 §7, ADR 0233, ADR 0404, ADR 0429 §14, ADR 0539 §16.*
|
||||||
|
|
||||||
When a delivery plan is ready to send, each node it reaches backs up, through its `node-backup` seat, every item
|
Before the node agent applies a new fingerprint of a module that keeps data on its node, it asks that node's
|
||||||
the plan's moved modules keep there whose protection is a backup (a copy or a dump). An item protected by
|
`node-backup` seat to back up every item the module keeps there whose protection is a backup (a copy or a
|
||||||
`redundancy`, one that says `backup: none`, and a cache are never asked, whatever their class. A node with no backup holder backs up
|
dump), and waits for the backup holder's event that the backup ended, polling nothing. An item protected by
|
||||||
nothing, and the plan says so for that node.
|
`redundancy`, one that says `backup: none`, and a cache are never asked, whatever their class. A node with no
|
||||||
|
backup holder backs up nothing, and the have says so.
|
||||||
|
|
||||||
- **The send waits for every backup.** The controller sends nothing of the plan to any node until every backup
|
- **Nothing of the module changes before its backup.** The node agent applies the new composition only once
|
||||||
it asked has answered done.
|
every backup it asked has answered done.
|
||||||
- **A failed backup holds the delivery.** A backup that fails, or does not answer within ten minutes, holds the
|
- **A failed backup holds that module on that node.** The module keeps running what it has; the node agent
|
||||||
delivery with the node, the module and the backup's own words as the reason, and raises a condition for the
|
writes the have phase `held` with the backup's own words and applies nothing of it. The assigner asks the
|
||||||
operator. A person's `release` asks the backups again, and the send goes on when they pass.
|
operator (back up again, or go on without a backup), raises the condition naming the node, the module and the
|
||||||
|
words, and on the answer writes the node's want for the module again naming it, so the node agent reads the
|
||||||
|
answer from its own want and from nothing else (ADR 0539 §16).
|
||||||
|
- **Nothing else waits.** Other modules, and the same module on other nodes, move on their own wants.
|
||||||
- **What it costs.** About 85% of data modules back up in under 30 s; the slowest measured, postgres on the home
|
- **What it costs.** About 85% of data modules back up in under 30 s; the slowest measured, postgres on the home
|
||||||
server, about 165 s (research 055, finding 5). A plan moving no backed-up item costs nothing more.
|
server, about 165 s (research 055, finding 5). A module that keeps no backed-up item costs nothing more.
|
||||||
|
|
||||||
## The send and the rollout
|
## Compositions, wants and the apply
|
||||||
|
|
||||||
*ADR 0360, ADR 0362, ADR 0377, ADR 0406 §5.*
|
*ADR 0360 §8, §9, ADR 0377, ADR 0429 §4, §6, ADR 0539 §1 to §7, §9, §14, §15.*
|
||||||
|
|
||||||
- **Every machine at once.** A plan's builds go to every machine that runs them in one step, tier by tier; a tier
|
- **A composition per module, node and fingerprint.** The controller composes one module's resources for one
|
||||||
is asked once the tier before it is built and sent (ADR 0360 §1, §6). No machine goes first and nothing waits
|
node at one fingerprint with mesh-sdk's composition, the one the node-engine uses too, and writes it as
|
||||||
for a health reading. A module of policy `record` that would recreate a container or move data waits for a
|
`<node>.<module>.<fingerprint>` in the bucket `composed-<node>`, which only that node's account reads, beside
|
||||||
person (ADR 0377).
|
the node's own composition `<node>.node` (adoption, firewall openings, accounts, the resolver, identities and
|
||||||
- **Every move is said.** The controller says every move of a plan as `plan-moved`, a move a command makes among
|
groups). Under the same key in `mesh-controller_composed`, which the assigner reads, it writes the
|
||||||
them, so mesh-delivery needs no periodic reconcile; it reads `delivery-walks` once when it starts.
|
composition's digest and its judgement of the move from the composition the node runs now, as its have names
|
||||||
- **A rollout advances on events.** The `built` and sent events of a tier ask the next; the nodes' reports end
|
it: recreates a container, moves data, neither, or cannot be judged (ADR 0377 §3), rewritten when that have
|
||||||
the rollout (ADR 0360 §7, with ADR 0362's provisioning report). Each send puts a key whose time to live is
|
changes. No composition's content is readable outside its node (ADR 0539 §1, §2).
|
||||||
ApplySilentAfter (fifteen minutes); its expiry names the machine silent. There is no rollout ticker.
|
- **Signed.** Each composition carries the signature of the controller's key or, made by hand, of the
|
||||||
- **A failed apply is loud and fixed forward** (ADR 0360 §8); the node-engine's witness still puts back a
|
operator's key; the node agent trusts exactly these two and applies nothing of a composition whose signature
|
||||||
controller or tool runner that does not come up (ADR 0360 §9).
|
fails or whose signer is neither. The operator's public key reaches a node in the switch's last declaration,
|
||||||
|
or at its adoption for a node adopted later (ADR 0539 §3).
|
||||||
|
- **Written on events, never on a timer.** A done build of an assigned module, an assignment written or ended,
|
||||||
|
a setting or given value, a secret rotated or moved, a credential issued, an outside acceptance, a node
|
||||||
|
adopted, taken or converged, a peer added to the hub and a change of the announced reads each make the
|
||||||
|
controller compose what they touch and write the compositions that differ. A change that keeps the
|
||||||
|
fingerprint is a new revision with a new digest, and reaches the node only when the assigner writes the want
|
||||||
|
again naming it, by the same rules as a new fingerprint (ADR 0539 §5).
|
||||||
|
- **Kept only while it can be wanted.** A composition and its digest are deleted when the fingerprint is no
|
||||||
|
longer among the module's five newest and no want names it; each bucket keeps five revisions of a key; a file
|
||||||
|
over 64 KiB travels by digest in `declaration-files-<node>`, removed a day after no composition of that node
|
||||||
|
names it. Neither bucket is a link of the chain: a composition is derived and can be composed again (ADR 0539 §4).
|
||||||
|
- **Credentials and grants on a done build.** When the builder's done key names a new fingerprint of an
|
||||||
|
assigned module, the controller issues the module's bus credential and writes `grants.<module>.<fingerprint>`
|
||||||
|
before any want can name it. The bus's user list is part of the bus module's composition: a revision that
|
||||||
|
changes only the user list is wanted at once, outside the person's hold, and the node agent applies it as a
|
||||||
|
file change and a reload (ADR 0539 §7).
|
||||||
|
- **The want names the composition.** A want `want.<node>.<module>` names a fingerprint and the digest of the
|
||||||
|
one composition the assigner allows for it; the node's own want `want.<node>.node` names the node's
|
||||||
|
composition (ADR 0539 §1, §6).
|
||||||
|
- **The apply, one module at a time.** At its first start the node agent takes the declaration it kept last as
|
||||||
|
what each module runs and writes each have. Then it applies a module only when its want's fingerprint or
|
||||||
|
digest differs from what runs, and only the composition whose digest the want names, removing only that
|
||||||
|
module's own resources: a shared path stays while any composition declares it. It writes the have with the
|
||||||
|
fingerprint, the digest, the phase and the health after the module's start window. A failed apply is loud,
|
||||||
|
retried by the next change of the want or the composition, never by a clock, and fixed forward (ADR 0360 §8,
|
||||||
|
ADR 0539 §9).
|
||||||
|
- **Every want is owed.** The watchman mirrors every want as a pulse owed by that node's agent until its have
|
||||||
|
names the want's fingerprint, so a node that does not move is a stall (ADR 0539 §19).
|
||||||
|
- **A move a person must allow is asked before the want.** For a module of policy `record`, the assigner writes
|
||||||
|
no want naming a composition judged to recreate a container or move data, at a new fingerprint or a new
|
||||||
|
revision alike, until the operator answers its ask yes, on the phone or in the view; a move judged neither is
|
||||||
|
written as any want, and so is a module the node never ran; one that cannot be judged waits for the ask, which
|
||||||
|
says why. Each judgement and the way it went are shown in the view and named in the ask (ADR 0539 §14).
|
||||||
|
- **The bus is never moved by the assigner.** The controller's `bus upgrade` checks as before and then asks the
|
||||||
|
operator to write the bus node's want with the operator's credential, recorded, until issue 1125 settles the
|
||||||
|
bus's replacement (ADR 0539 §15).
|
||||||
|
- **The witness stays.** The node-engine's witness still puts back a controller or tool runner that does not
|
||||||
|
come up (ADR 0360 §9). A new node-engine runs on trial under its launcher until it writes its first have under
|
||||||
|
its own build (ADR 0539 §9).
|
||||||
|
|
||||||
## A stop on a shared plan
|
## No stop of a shared plan
|
||||||
|
|
||||||
*ADR 0299.*
|
*ADR 0539 §18, which supersedes ADR 0299.*
|
||||||
|
|
||||||
One plan carries every merge of its batch, so a stop ends only the change it is for:
|
With no plan there is nothing to stop and deliver again without the stopped change. A bad fingerprint is held
|
||||||
|
and its canary set back (ADR 0429 §6), and a module whose build is not done waits under the assigner's first
|
||||||
- **No stop by member.** A group's plan fails as one (ADR 0418 §5), so mesh-delivery asks no stop of a member
|
rule rather than named in a plan. Taking a change back off the nodes is a revert: its earlier fingerprints
|
||||||
after a failed one. The mechanism stands for a person's stop with the way `again`: the controller ends the plan
|
have done builds and, while among their modules' five newest, kept compositions, so the assigner's wants
|
||||||
and records every other merge it answered again in the open batch of its kind, whose window the stop closes.
|
deliver it like any change, building nothing (ADR 0526 §1, ADR 0539 §4).
|
||||||
- **What is left out with it.** A later merge of the stopped merge's repository, and a merge that moves a
|
|
||||||
module a left-out merge moves, would deliver the stopped change; each is left out too, its delivery `stopped`,
|
|
||||||
naming why.
|
|
||||||
- **A person's stop.** `stop` of a delivery whose plan carries others first answers what it would take down, and
|
|
||||||
acts on `way`: `again` (the others delivered again without it) or `walk` (the whole plan stopped).
|
|
||||||
- **What it does not do.** What the stopped plan already sent stays on the machines it reached; putting it back
|
|
||||||
is issue 509's.
|
|
||||||
|
|
||||||
## The delivery budget
|
## The delivery budget
|
||||||
|
|
||||||
*ADR 0282, ADR 0304 §4, ADR 0313, ADR 0360 §12, ADR 0362 §5.*
|
*ADR 0282, ADR 0304 §4, ADR 0313.*
|
||||||
|
|
||||||
A delivery's **delivery time** runs from its merge until every machine its plan sent has reported the send
|
A delivery's **delivery time** runs from its merge until every node that runs a module it moves has a have
|
||||||
applied, and for a provisioned module also placed. A **leaf** delivery's budget is three minutes, a **core**
|
equal to its want, running and healthy, which the view says as `settled` for each node and module and
|
||||||
delivery's ten. A delivery held for a person, a failed backup among them, is shown and held to no budget. Each
|
`delivered` for the source revision. A **leaf** delivery's budget is three minutes, a **core** delivery's ten.
|
||||||
phase is kept — the window, the wait behind a plan that shares something, the backups, the build, the send, the
|
A move held for a person, a failed backup among them, is shown and held to no budget. The **fix time**
|
||||||
reports — and a delivery over its budget raises `delivery.<class>.over-budget`, naming its longest phase. The
|
(ADR 0313) runs from an issue's first move to `fixing` to its last fix delivered, and contains the delivery
|
||||||
**fix time** (ADR 0313) runs from an issue's first move to `fixing` to its last fix delivered, and contains the
|
time. No verb of the gate answers the week's times since release 6: `times` went with the plans
|
||||||
delivery time.
|
(ADR 0539 §17), and the view's events carry what a measure of it reads.
|
||||||
|
|
||||||
## Builds without a toolchain image
|
## Builds without a toolchain image
|
||||||
|
|
||||||
@@ -595,36 +642,40 @@ delivery time.
|
|||||||
|
|
||||||
## One chain, on the bus alone
|
## One chain, on the bus alone
|
||||||
|
|
||||||
*ADR 0408.*
|
*ADR 0408, ADR 0429 §17, ADR 0539 §4, §12, §13.*
|
||||||
|
|
||||||
Module, commit, build, published package or artifact, rollout and assignment are one chain, each link
|
From release 6, module, source revision, build, want and have are one chain, each link answerable from the next
|
||||||
answerable from the next. A **rollout** here is the controller's record of one batch's delivery; a delivery's own
|
(ADR 0429 §17). The assignments bucket keeps placement alone, and the rollouts bucket is kept read-only with its
|
||||||
delivery plan (*A delivery* above) stays the delivery's, and names the rollout that delivers it.
|
history: no rollout is written after release 5's last plan (ADR 0539 §12).
|
||||||
|
|
||||||
| Link | Names | Written by | Bucket |
|
| Link | Names | Written by | Bucket |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| build | its module, repository and commit, its reuse key, what it published: an artifact by digest or a package release by name and version | the controller | `mesh-controller_builds` |
|
| build | its module, repository and commit, its reuse key, what it published: an artifact by digest or a package release by name and version | the controller | `mesh-controller_builds` |
|
||||||
| rollout | the commit it fixed per repository, its tiers, the builds it carried, the machine steps, and every assignment it changed with the build before and after; its states and transitions | the controller | `mesh-controller_rollouts` |
|
| want | the fingerprint a node should run of a module, and the digest of the one composition it allows; the node's own want names its node composition | the assigner | `mesh-delivery-assigner_wants` |
|
||||||
| assignment | the module and node, the build, artifact or package and commit it runs, and the rollout that put it there | the controller | `mesh-controller_assignments` |
|
| have | what the node runs of a module: the fingerprint, the composition's digest, the phase and the health | the node agent | `node-engine_have` |
|
||||||
|
| assignment | the module and node, pinned or not, ended or not: placement alone, written by `assign` and `unassign` | the controller | `mesh-controller_assignments` |
|
||||||
|
| rollout | read-only from release 6: each plan of releases 1 to 5, its tiers, the builds it carried and the assignments it changed | nobody from release 6 | `mesh-controller_rollouts` |
|
||||||
|
|
||||||
Beside the chain, on the bus too: deliveries and groups (mesh-delivery's `deliveries` and `groups`, ADR 0239 §37),
|
Beside the chain, on the bus too: deliveries and groups (the gate's `deliveries` and `groups`, ADR 0239 §37), one
|
||||||
one key per open pull request (mesh-delivery), and each open batch with its merges and its window's two keys (the
|
key per open pull request (the gate), and the compositions with their digests (`composed-<node>` and
|
||||||
controller, ADR 0407).
|
`mesh-controller_composed`), which are derived state and no link of the chain (ADR 0539 §4). The buckets of plan
|
||||||
|
state, the merge window's keys, the silent sends' keys and a building plan's hold, are deleted by the controller
|
||||||
|
at its first start of release 6 (ADR 0539 §13).
|
||||||
|
|
||||||
- **The bus is the only source.** Each bucket is written by the controller that holds the lease, by
|
- **The bus is the only source.** Each bucket is written by its one writer, by compare-and-set, and each part
|
||||||
compare-and-set, and the controller reads its own buckets to decide; a rollout resumes from its key after a
|
reads the buckets it names to decide; a part resumes from its keys after a restart. No table of the
|
||||||
restart. No table of the controller's store holds builds, rollouts, batches or assignments, so no record mirror,
|
controller's store holds builds, rollouts, batches or assignments, so no record mirror,
|
||||||
audit or healer exists for them; the nodes, settings, declared-config and predefined-keys mirrors stay as
|
audit or healer exists for them; the nodes, settings, declared-config and predefined-keys mirrors stay as
|
||||||
ADR 0289 and ADR 0356 have them. A module reads a bucket by naming it in `reads`; mesh-config reads the
|
ADR 0289 and ADR 0356 have them. A module reads a bucket by naming it in `reads`; mesh-config reads the
|
||||||
assignments bucket.
|
assignments bucket.
|
||||||
- **No key of the chain is removed.** Each bucket keeps up to 64 earlier revisions of each key as its history. A
|
- **No key of the chain is removed.** Each bucket keeps up to 64 earlier revisions of each key as its history. A
|
||||||
value is never cut to a summary and stays under the bus's largest message. The bus's storage is declared as data
|
value is never cut to a summary and stays under the bus's largest message. The bus's storage is declared as data
|
||||||
protected by a backup, so it is backed up nightly and before a delivery moves the bus.
|
protected by a backup, so it is backed up nightly and before a delivery moves the bus.
|
||||||
- **One read answers each question.** For any module on any node, one read of its assignment names the commit,
|
- **One read answers each question.** For any module on any node, one read of its have names the fingerprint and
|
||||||
the build, the artifact or package and the rollout that put it there; one read of a rollout lists every
|
the composition it runs, one read of its want what it should run, and the fingerprint names the build, its
|
||||||
assignment it changed.
|
commit and its artifact or package.
|
||||||
- **The tables go by a numbered migration**: builds, rollouts and assignments are written once into the buckets,
|
- **The tables went by a numbered migration** in milestone 5: builds, rollouts and assignments are written once into the buckets,
|
||||||
waiting build requests onto the work queues and an open batch into its keys; then the tables are dropped from the
|
waiting build requests onto the work queues and an open batch into its keys; then the tables were dropped from the
|
||||||
controller's store and its consolidated schema.
|
controller's store and its consolidated schema.
|
||||||
- **The artifact store and the package registry keep the bytes**; the chain names them (ADR 0251).
|
- **The artifact store and the package registry keep the bytes**; the chain names them (ADR 0251).
|
||||||
|
|
||||||
@@ -645,39 +696,58 @@ forge module's files of what it announced (ADR 0406 §6).
|
|||||||
|
|
||||||
## The verbs
|
## The verbs
|
||||||
|
|
||||||
**mesh-delivery's seat**:
|
**The gate's seat `mesh-delivery`** (ADR 0539 §17):
|
||||||
|
|
||||||
| Verb | Answers or does |
|
| Verb | Answers or does |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `deliveries` | every delivery not final, the final ones of the last day, and every one `stalled` still lists |
|
| `deliveries` | every delivery not final, the final ones of the last day, and every one `stalled` still lists |
|
||||||
| `show` | one delivery or group whole: plan, the edge that placed each tier, the Approve, transitions, machine steps, the chain's links |
|
| `show` | one delivery or group whole: the Approve, transitions, the review record, the source revision its merge moved |
|
||||||
| `groups` | every group with its members and its derived state |
|
| `groups` | every group with its members and its derived state |
|
||||||
| `what-if` | the delivery plan a diffset would have, asked of the controller's planner |
|
| `table` | the state table, with its bounds |
|
||||||
| `table` | the state table and the machine-step table, with bounds and H2's repairs |
|
| `stalled` | every merge or check held past its bound |
|
||||||
| `stalled` | every delivery held past its bound, with the transition H2 may take |
|
|
||||||
| `recheck` | an exception verb, no step of the flow: a check asked again for a proposed delivery nothing else asks again, an inactive one or a failed verdict gone out of date (ADR 0395 §6, ADR 0421 §12) |
|
| `recheck` | an exception verb, no step of the flow: a check asked again for a proposed delivery nothing else asks again, an inactive one or a failed verdict gone out of date (ADR 0395 §6, ADR 0421 §12) |
|
||||||
| `release` | a held delivery goes on, with why; for a failed backup it asks the backups again |
|
| `close` | the transition the table names for a stalled delivery |
|
||||||
| `stop` | a delivery to stopped, with why; on a shared plan it first answers what it would take down (ADR 0299) |
|
|
||||||
| `close` | H2's only verb: the transition the table names for a stalled delivery |
|
|
||||||
| `retire-history` | held history to retired (ADR 0250) |
|
|
||||||
| `times` | each class's budget, the week's median and worst delivery time, and the deliveries over budget |
|
|
||||||
|
|
||||||
**The controller's verbs for it**: `delivery-plan`, `delivery-order` (the edges that order a plan's tiers,
|
`times`, `what-if`, `release`, `stop` and `retire-history` went at release 6; each is answered that it no
|
||||||
ADR 0419 §2), `delivery-check`, `deliver`,
|
longer exists.
|
||||||
`delivery-stop`, `delivery-walks`, and `plans` (the open batch, the open plans, and a what-if for a repository
|
|
||||||
and its files). A person has `plans go <plan> --why`.
|
**The status view's tools** read the keys and change nothing: `assigner_wants`, `assigner_want`,
|
||||||
|
`assigner_differences` and `node_haves`, beside the indexer's and the builder's. Its events are `delivered`,
|
||||||
|
`moving`, `settled` and `stuck` (*A delivery* above).
|
||||||
|
|
||||||
|
**The controller's verbs of plans** went at release 6 (ADR 0429 §8, ADR 0539 §11): `plans`, `plan`, `push`,
|
||||||
|
`deliver`, `delivery-stop`, `delivery-backups-again`, `delivery-walks`, `delivery-order` and the plan
|
||||||
|
subcommands of `upgrade`, each answered that it no longer exists by name. `bus upgrade` stays and ends by asking
|
||||||
|
the operator for the bus node's want (ADR 0539 §15).
|
||||||
|
|
||||||
## The bootstrap
|
## The bootstrap
|
||||||
|
|
||||||
- A plan that moves the controller, the node-engine, the tool runner, the bus or mesh-delivery is the
|
*ADR 0526, ADR 0539 §9, §10, §11, §15.*
|
||||||
controller's own, started by the merge; mesh-delivery records it (ADR 0239 §31). Such a merge shares one batch
|
|
||||||
and one plan with the other merges of its delivery group, and that plan is started by the controller when the
|
- **The controller, the node-engine, the tool runner and the gate are ordinary modules.** Each new fingerprint
|
||||||
group's batch closes, not by its own merge (ADR 0418 §6).
|
gets its composition from the running controller, its credential and grant when its build is done, and its
|
||||||
- Any other plan waits for `deliver` while the seat has a holder on record; with none, the controller starts it.
|
want from the assigner; the control-node is the controller's only node and so its canary. The node-engine's
|
||||||
- A replaced controller or mesh-delivery carries on from the bus: the keys, the queues and the durable
|
witness puts back a controller or tool runner that does not come up (ADR 0360 §9), and a new node-engine is
|
||||||
consumers hold every step, so nothing is lost or repeated (ADR 0406 §6).
|
rolled back by its launcher unless it writes its first have under its own build (ADR 0539 §9). The bus stays
|
||||||
- The controller takes every module, the builder's own and the build modules among them, from the builder's done key whose artifacts
|
outside the assigner (ADR 0539 §15).
|
||||||
all name a reference; a module whose key is not yet done waits, named in the plan, and never fails it (ADR 0524 §7).
|
- **A replaced controller or part carries on from the bus**: the keys, the streams and the durable consumers hold
|
||||||
|
every step, so nothing is lost or repeated (ADR 0406 §6). A controller composes, at its start, every assigned
|
||||||
|
module and the fingerprint of the controller it replaced, so the way back to that controller is one want
|
||||||
|
(ADR 0539 §10).
|
||||||
|
- **A build before its want.** The controller takes every module, the builder's own and the build modules among
|
||||||
|
them, from the builder's done key whose artifacts all name a reference; a module whose key is not yet done
|
||||||
|
waits under the assigner's first rule and fails nothing (ADR 0524 §7, ADR 0539 §18).
|
||||||
|
- **The switch to release 6 is one plan, release 5's last.** Release 5's controller runs the group's plan with
|
||||||
|
the new node-engine and the new controller in one tier (issue 1942), so every node's last declaration names
|
||||||
|
the new node-engine and carries the operator's public key, and the control-node's names the new controller
|
||||||
|
too. Each node applies it with the node-engine it runs; the new controller writes every composition at its
|
||||||
|
start, and the assigner's wants equal what runs. For that plan one machine at a time yields: each new
|
||||||
|
node-engine is judged by its launcher, and the new controller by the node-engine's witness. A node away during
|
||||||
|
the switch finds its last declaration on the bus's stream of declarations when it returns; the controller
|
||||||
|
deletes that stream and the nodes' consumers only once every node's have names the new node-engine. The
|
||||||
|
merge waits until every node is present (ADR 0539 §11). At its first start the new controller deletes the
|
||||||
|
buckets of plan state (the merge window's, the silent sends' and the building plan's hold) and says each
|
||||||
|
deletion as its event (ADR 0539 §13).
|
||||||
|
|
||||||
### Delivering the controller, the builder or a build module without the builder
|
### Delivering the controller, the builder or a build module without the builder
|
||||||
|
|
||||||
@@ -686,7 +756,8 @@ way is written down before the switch (ADR 0526, which ADR 0522 §6 asked of R4g
|
|||||||
|
|
||||||
1. **A revert, with no build.** A broken builder or build module is undone by reverting its change: the earlier source's fingerprint has a done
|
1. **A revert, with no build.** A broken builder or build module is undone by reverting its change: the earlier source's fingerprint has a done
|
||||||
key naming references, so the delivery of the revert builds nothing. It needs the earlier artifact in the artifact store, which keeps a
|
key naming references, so the delivery of the revert builds nothing. It needs the earlier artifact in the artifact store, which keeps a
|
||||||
recorded build's artifact while it is among its module's five newest or a definition names it (ADR 0526 §1).
|
recorded build's artifact while it is among its module's five newest or a definition names it (ADR 0526 §1); from release 6 its
|
||||||
|
composition is kept while it is among the five newest too, so the assigner's want delivers it (ADR 0539 §4).
|
||||||
2. **A build by a session's call.** A session calls the build module's fetch and build tools on one named build node (for Go, a fetch and then
|
2. **A build by a session's call.** A session calls the build module's fetch and build tools on one named build node (for Go, a fetch and then
|
||||||
a build with publishing on); the build module pushes the artifact to the artifact store and answers its reference, as it does for the
|
a build with publishing on); the build module pushes the artifact to the artifact store and answers its reference, as it does for the
|
||||||
builder (ADR 0526 §2, ADR 0525 §8). `mesh-pack`, in mesh-sdk, builds and packs a Go bundle with the Go build module's recipe, so a hand
|
builder (ADR 0526 §2, ADR 0525 §8). `mesh-pack`, in mesh-sdk, builds and packs a Go bundle with the Go build module's recipe, so a hand
|
||||||
@@ -694,10 +765,13 @@ way is written down before the switch (ADR 0526, which ADR 0522 §6 asked of R4g
|
|||||||
3. **A registration by reference, at the controller's terminal only.** The operator names the module, its repository, its path and each
|
3. **A registration by reference, at the controller's terminal only.** The operator names the module, its repository, its path and each
|
||||||
artifact's reference at the controller's terminal, and the controller registers it with no builder key, recording the act with its why;
|
artifact's reference at the controller's terminal, and the controller registers it with no builder key, recording the act with its why;
|
||||||
no verb reaches it (ADR 0526 §3).
|
no verb reaches it (ADR 0526 §3).
|
||||||
4. **A controller that cannot take its own fix** is put on the control-node by hand: its hand build is registered as in step 3 and the
|
4. **A controller that cannot take its own fix** is put back by keys written at the terminal, never by a declaration sent by hand: the
|
||||||
control-node is pushed with its plan read first. In the worst case the operator sends the control-node a declaration whose source and
|
operator's credential writes `want.<control-node>.mesh-controller` to a fingerprint whose composition is kept, the replaced
|
||||||
digest are the hand build's, recorded as a declaration sent by hand, and registers it as in step 3 so that the next push keeps it
|
controller's among them, since a controller composes that one at its start. For a hand build never composed, `mesh-compose`, a
|
||||||
(ADR 0526 §4; the bootstrap of issue 1886).
|
command of mesh-sdk beside `mesh-pack`, composes the module for the node from the controller's kept inputs with mesh-sdk's
|
||||||
|
composition, signs it with the operator's key and, with the operator's credential, writes it into `composed-<node>` and its digest
|
||||||
|
into `mesh-controller_composed`; the operator then writes the want naming that digest. Each act is recorded in the hand-act log, and
|
||||||
|
the node-engine keeps no path that applies a whole declaration (ADR 0539 §10, which changes ADR 0526 §4).
|
||||||
5. **Copies kept outside the store.** Before a switch that moves the controller, the builder or a build module, the artifacts of the
|
5. **Copies kept outside the store.** Before a switch that moves the controller, the builder or a build module, the artifacts of the
|
||||||
running versions are kept outside the artifact store on two nodes, each by digest with its module, artifact, commit, Go version and the
|
running versions are kept outside the artifact store on two nodes, each by digest with its module, artifact, commit, Go version and the
|
||||||
steps that put it back, because the store's nightly collection removes a blob the controller did not record (ADR 0526 §6).
|
steps that put it back, because the store's nightly collection removes a blob the controller did not record (ADR 0526 §6).
|
||||||
@@ -727,6 +801,9 @@ What the build proves before this design is implemented (research 055's directio
|
|||||||
with no person's release and nothing held in mesh-delivery.
|
with no person's release and nothing held in mesh-delivery.
|
||||||
12. No table in the controller's store holds builds, rollouts or assignments.
|
12. No table in the controller's store holds builds, rollouts or assignments.
|
||||||
|
|
||||||
|
Tests 1, 3, 7, 8, 9, 10 and 11 were proved by the milestones that built them, on the plans and rollouts of their
|
||||||
|
time; from release 6, tests 18 and 19 prove what of them still holds without a plan.
|
||||||
|
|
||||||
From ADR 0429 (research 063's done-when, and the live proof each of its releases names):
|
From ADR 0429 (research 063's done-when, and the live proof each of its releases names):
|
||||||
|
|
||||||
13. The status view shows every module on every node from the have keys, and agrees with the controller's own
|
13. The status view shows every module on every node from the have keys, and agrees with the controller's own
|
||||||
@@ -790,6 +867,16 @@ development, the check's share about a minute):
|
|||||||
check after 30 minutes (ADR 0483, superseded); the fallback to build-agent for a module whose builder does not say it writes
|
check after 30 minutes (ADR 0483, superseded); the fallback to build-agent for a module whose builder does not say it writes
|
||||||
references; the builder as an input of every module's fingerprint (ADR 0530); and the controller's per-node capacity choice, which the
|
references; the builder as an input of every module's fingerprint (ADR 0530); and the controller's per-node capacity choice, which the
|
||||||
build modules' own pull replaces.
|
build modules' own pull replaces.
|
||||||
|
- At release 6's delivery (ADR 0429 §7, §8; ADR 0539): delivery plans, batches, the merge window, tiers and the
|
||||||
|
writing of rollouts; the plans hold and the node declaration holds; the push of whole declarations, its
|
||||||
|
retries, takeovers and healers; the controller's verbs `plans`, `plan`, `push`, `deliver`, `delivery-stop`,
|
||||||
|
`delivery-backups-again`, `delivery-walks`, `delivery-order` and the plan subcommands of `upgrade`; the
|
||||||
|
planner's edges and its engine-first default; the backups before a send; the gate's delivery states
|
||||||
|
(delivering, the backup hold, superseded, covered, stopped, retired and failed), its machine steps and its verbs
|
||||||
|
`times`, `what-if`, `release`, `stop` and `retire-history`; the messenger's hold on a plan, which holds on the
|
||||||
|
view's events instead; the assigner's comparison with the send; the node-engine's queue of whole
|
||||||
|
declarations, its orders, put-backs and five-minute reconcile; the buckets of plan state; the stop of a shared
|
||||||
|
plan (ADR 0299, superseded by ADR 0539 §18); and a declaration sent by hand (ADR 0526 §4, changed by ADR 0539 §10).
|
||||||
|
|
||||||
## Milestones
|
## Milestones
|
||||||
|
|
||||||
@@ -830,10 +917,12 @@ release reports the controller's code lines against research 063's baseline (acc
|
|||||||
| R4e | The controller asks no build: a build is keyed by its fingerprint, the new form becomes the only one as one group, and a merge builds nothing its check built; builders fetch over SSH, so an SDK change ships with its users; the vendored copies leave mesh-controller and mesh-sdk; delivered only once the builder's artifact digests equal build-agent's (ADR 0453 clauses 2, 4, 5, 6) | 16, 2, 4 in part | mesh-controller (builds are no longer asked), mesh-delivery (builder), mesh-sdk | 979 (the installed toolchains; closed with the toolchain images in R4g), 1304, the controller asking no build (opened when it is prepared) |
|
| R4e | The controller asks no build: a build is keyed by its fingerprint, the new form becomes the only one as one group, and a merge builds nothing its check built; builders fetch over SSH, so an SDK change ships with its users; the vendored copies leave mesh-controller and mesh-sdk; delivered only once the builder's artifact digests equal build-agent's (ADR 0453 clauses 2, 4, 5, 6) | 16, 2, 4 in part | mesh-controller (builds are no longer asked), mesh-delivery (builder), mesh-sdk | 979 (the installed toolchains; closed with the toolchain images in R4g), 1304, the controller asking no build (opened when it is prepared) |
|
||||||
| R4f | A pull request carries one required status that says what it proved and how long each part took; the controller's composition check and its replays are deleted with the two old statuses; the controller's deletions R3e makes possible go; each level's times are measured, and a bound fails a repository from the first check that meets it (ADR 0447, ADR 0450 §8, ADR 0453 clauses 4, 9, 10) | 21, 22, 23, 24 (the status; the scripts' absence is proved in R4g) | mesh-delivery (checker, assigner), mesh-controller (composition check, replays, `delivery-plan`, `forget-checks`, the asker `mesh-delivery` deleted), mesh-catalog (gitea) | 1299 (the composition check deleted; closes issue 709, the replays' socket), 1300, 1301, 1302 (one status), 1305 (ADR 0482 §8), 1318 |
|
| R4f | A pull request carries one required status that says what it proved and how long each part took; the controller's composition check and its replays are deleted with the two old statuses; the controller's deletions R3e makes possible go; each level's times are measured, and a bound fails a repository from the first check that meets it (ADR 0447, ADR 0450 §8, ADR 0453 clauses 4, 9, 10) | 21, 22, 23, 24 (the status; the scripts' absence is proved in R4g) | mesh-delivery (checker, assigner), mesh-controller (composition check, replays, `delivery-plan`, `forget-checks`, the asker `mesh-delivery` deleted), mesh-catalog (gitea) | 1299 (the composition check deleted; closes issue 709, the replays' socket), 1300, 1301, 1302 (one status), 1305 (ADR 0482 §8), 1318 |
|
||||||
| R4g | The last milestone of release 4 and release 5 as one, with one switch-over and no rehearsal (ADR 0524 §1): builds and checks are asks on streams that each build node pulls when it has a free slot, deliveries first, with nothing holding checks and no controller queue (ADR 0525); the assigner is made with its composition check, writes each node's wants compared with what the controller sends and every difference, and the view answers the asks that need no proof of who answered (ADR 0528, ADR 0437 §10); no module's fingerprint takes the builder (ADR 0530); the scripts of the repositories a module is built from, build-agent with its socket and Dockerfile code, the toolchain images and the controller's build code go, and the builder delivers its own module, the build modules and the controller from its keys (ADR 0453 clauses 5, 8, 10; ADR 0524 §5, §7); the ways to deliver without the builder are recorded (ADR 0526); hq's four checks are Go tests (ADR 0527); each repository's script goes (ADR 0529); the word for the composition check's predecessor is struck. Its pull requests are opened only after five readings, the revert path read, not shown on a test merge (ADR 0524 §3) | 4 (from R4g mesh-build-image, rootless, is the only part that builds an image: ADR 0487 §7), 17, 24 (no `merge-check.sh` in a repository a module is built from) | mesh-sdk, mesh-controller (build-agent removed), mesh-delivery, mesh-catalog, mesh-tools, mesh-tools-go, the node-engine (`mesh-host`), mesh-media-catalog, okioki, hq | 1887 (the flow), 1297 (the assigner), 1295 (`merge-check.sh` goes), 979 (closed with the toolchain images), 1296 and 1303 (hq's check, the old word struck), 1299, 1305, 1294, 1318 (the controller's deletions) |
|
| R4g | The last milestone of release 4 and release 5 as one, with one switch-over and no rehearsal (ADR 0524 §1): builds and checks are asks on streams that each build node pulls when it has a free slot, deliveries first, with nothing holding checks and no controller queue (ADR 0525); the assigner is made with its composition check, writes each node's wants compared with what the controller sends and every difference, and the view answers the asks that need no proof of who answered (ADR 0528, ADR 0437 §10); no module's fingerprint takes the builder (ADR 0530); the scripts of the repositories a module is built from, build-agent with its socket and Dockerfile code, the toolchain images and the controller's build code go, and the builder delivers its own module, the build modules and the controller from its keys (ADR 0453 clauses 5, 8, 10; ADR 0524 §5, §7); the ways to deliver without the builder are recorded (ADR 0526); hq's four checks are Go tests (ADR 0527); each repository's script goes (ADR 0529); the word for the composition check's predecessor is struck. Its pull requests are opened only after five readings, the revert path read, not shown on a test merge (ADR 0524 §3) | 4 (from R4g mesh-build-image, rootless, is the only part that builds an image: ADR 0487 §7), 17, 24 (no `merge-check.sh` in a repository a module is built from) | mesh-sdk, mesh-controller (build-agent removed), mesh-delivery, mesh-catalog, mesh-tools, mesh-tools-go, the node-engine (`mesh-host`), mesh-media-catalog, okioki, hq | 1887 (the flow), 1297 (the assigner), 1295 (`merge-check.sh` goes), 979 (closed with the toolchain images), 1296 and 1303 (hq's check, the old word struck), 1299, 1305, 1294, 1318 (the controller's deletions) |
|
||||||
| R6 | Each node applies from its wants, a new version reaches the module's canary first and the rest once it is healthy there, and the plans are deleted from the code | 18, 19 | the node-engine (`mesh-host`: the node agent), mesh-controller (plans deleted), mesh-delivery (the gate's delivery states go) | opened when it is prepared |
|
| R6 | Each node applies from its wants the compositions the controller signs for it, a new version reaches the module's canary first and the rest once it is healthy there, a move a person must allow is asked before its want, the view says what is moving, settled, stuck and delivered, and the plans are deleted from the code; switched over by release 5's last plan (ADR 0539) | 18, 19, 20 | mesh-sdk, the node-engine (`mesh-host`: the node agent), mesh-controller (compositions, credentials on a done build; plans deleted), mesh-delivery (assigner, watchman, view), mesh-catalog (the gate, the messenger, the issue tracker, the backup holder), hq | 1942 (the planner puts the node-engine and the controller in one tier for the switch; delivered first), 1943 (mesh-sdk: the composition's layout and signature, the want's digest, `mesh-compose`, the verb tables without the plan verbs), 1944 (the node agent), 1945 (the controller writes compositions and grants; the plans and the push deleted), 1946 (the gate keeps the merge's states; the messenger and the issue tracker hear the view), 1906 (the assigner's seventh rule and its asks, the watchman's pulses for wants, the view's wants and events), 1130 (hq) |
|
||||||
|
|
||||||
Release 5 (the assigner's wants, compared only) is not a milestone of its own: ADR 0524 §1 folds it into R4g, whose row above holds
|
Release 5 (the assigner's wants, compared only) is not a milestone of its own: ADR 0524 §1 folds it into R4g, whose row above holds
|
||||||
it; release 6, the node agent applying the wants, stays the next milestone.
|
it. Release 6 is milestone R6, prepared on 2026-10-24 with ADR 0539. Its prepare step found six repositories where its row first
|
||||||
|
named three: the gate's code is in mesh-catalog, mesh-sdk holds the composition's layout and the seat verb tables, and hq the design
|
||||||
|
and the glossary.
|
||||||
|
|
||||||
The issues are linked `depends-on` in this order; 976 depends on 975, and 979 also on 971. R3's prepare step on 2026-10-19 found ADR 0429 §20 contradicting itself and four release orders; ADR 0437 (accepted, the operator's Approve) cuts release 3 into R3a to R3d, each one release, and moves answering asks in the view to R5. R2c's prepare step on 2026-10-19 found one more order: the alive message is the only source of when the controller last heard a machine, which `nodes`, `node` and `status` read, so the controller takes that from presence (1183, R2c) and is delivered before the node-engine stops sending (1158, R2d); 1158 depends on 1183. In release 1-2, 1132 and 1136 depend on 1131, 1134 and 1135 on 1136, 1134 also on 1132 and 1128, and 1137 on 1134. Release 1-2 was confirmed as one milestone; its prepare step on 2026-10-18 found the release orders hidden in it, and the operator chose at 18:52 to cut it by them, so it is the milestones R1a to R2d above, each one release (the operator, 2026-10-19: a milestone is one release): an SDK change still ships before its users until R4e (ADR 0429 §10, ADR 0453 clause 2); a new key on the wire goes only to a node that says it reads it; a new repository's first module registers at its merge's Approve (ADR 0432, issue 1146); the bus gives the watchman connect events only through a system account (ADR 0431); and release 2's other gaps are settled by ADR 0433 and ADR 0434. A node that is down has no effect on a delivery and plays what it missed when it returns (ADR 0435, the operator, 2026-10-18; issues 1160, 1161). Milestone 3's first live group (milestone 4's, on 2026-10-17) held one Approve but merged in two passes and went out in two delivery plans: a member's tests read the other member's repository, so the first merge made the second's verdict out of date, and a closed group never asked its composed verdict again (issues 1049, 1082). The operator put a fix of milestone 3 after milestone 4 and before milestone 5 on 2026-10-18; acceptance test 1 is proved again by it. The operator put milestone 5b after milestone 5 and before milestone 6 on 2026-10-18, as an issue rather than a research effort; where the planning goes is settled by a decision record before it is built (issue 1117), and milestone 6's build changes are read against it. Milestone 2b was
|
The issues are linked `depends-on` in this order; 976 depends on 975, and 979 also on 971. R3's prepare step on 2026-10-19 found ADR 0429 §20 contradicting itself and four release orders; ADR 0437 (accepted, the operator's Approve) cuts release 3 into R3a to R3d, each one release, and moves answering asks in the view to R5. R2c's prepare step on 2026-10-19 found one more order: the alive message is the only source of when the controller last heard a machine, which `nodes`, `node` and `status` read, so the controller takes that from presence (1183, R2c) and is delivered before the node-engine stops sending (1158, R2d); 1158 depends on 1183. In release 1-2, 1132 and 1136 depend on 1131, 1134 and 1135 on 1136, 1134 also on 1132 and 1128, and 1137 on 1134. Release 1-2 was confirmed as one milestone; its prepare step on 2026-10-18 found the release orders hidden in it, and the operator chose at 18:52 to cut it by them, so it is the milestones R1a to R2d above, each one release (the operator, 2026-10-19: a milestone is one release): an SDK change still ships before its users until R4e (ADR 0429 §10, ADR 0453 clause 2); a new key on the wire goes only to a node that says it reads it; a new repository's first module registers at its merge's Approve (ADR 0432, issue 1146); the bus gives the watchman connect events only through a system account (ADR 0431); and release 2's other gaps are settled by ADR 0433 and ADR 0434. A node that is down has no effect on a delivery and plays what it missed when it returns (ADR 0435, the operator, 2026-10-18; issues 1160, 1161). Milestone 3's first live group (milestone 4's, on 2026-10-17) held one Approve but merged in two passes and went out in two delivery plans: a member's tests read the other member's repository, so the first merge made the second's verdict out of date, and a closed group never asked its composed verdict again (issues 1049, 1082). The operator put a fix of milestone 3 after milestone 4 and before milestone 5 on 2026-10-18; acceptance test 1 is proved again by it. The operator put milestone 5b after milestone 5 and before milestone 6 on 2026-10-18, as an issue rather than a research effort; where the planning goes is settled by a decision record before it is built (issue 1117), and milestone 6's build changes are read against it. Milestone 2b was
|
||||||
put before milestone 3 by the operator on 2026-10-17. Milestone 3 builds ADR 0418, ADR 0419 and ADR 0421
|
put before milestone 3 by the operator on 2026-10-17. Milestone 3 builds ADR 0418, ADR 0419 and ADR 0421
|
||||||
|
|||||||
Reference in New Issue
Block a user