Design 47's last milestones: the keeper, self-check, healers, hand-acts, provisioner, vault, artifact sweep, disk readings, credential forms (issues 774, 1952, 1953, 1954, 1955) #560

2026-10-30T00:21:39Z

mesh-tools commented

2026-10-30 00:21:39 +00:00

Delivery 2 of design 47's single release (group feat/774-l3-reland). New modules mesh-conditions, mesh-self-check (probes setting, default D13 + D-fill), mesh-healers, mesh-hand-acts; mesh-config's declared/compare/take-over/hand-back; mesh-provisioner holds provisions and plans; the vault, gitea, keycloak, postgres, route-adapter read plans; the client keeps filling secrets via the vault and falls back to the controller's assignments until the lease write; artifact-store-tools holds mesh-artifact-sweep; disk-load keeps readings; restic writes restic.good; every offer declares its credential form (checks/credentialform). 45 modules move; record modules will each ask the operator. All 22 checks pass. Main (#561, the forge's SSH_MAX_STARTUPS) merged in at 35b4509c.

Reviewed-by: rev-47@shanks at 35b4509c0a

Delivery 2 of design 47's single release (group feat/774-l3-reland). New modules mesh-conditions, mesh-self-check (probes setting, default D13 + D-fill), mesh-healers, mesh-hand-acts; mesh-config's declared/compare/take-over/hand-back; mesh-provisioner holds provisions and plans; the vault, gitea, keycloak, postgres, route-adapter read plans; the client keeps filling secrets via the vault and falls back to the controller's assignments until the lease write; artifact-store-tools holds mesh-artifact-sweep; disk-load keeps readings; restic writes restic.good; every offer declares its credential form (checks/credentialform). 45 modules move; record modules will each ask the operator. All 22 checks pass. Main (#561, the forge's SSH_MAX_STARTUPS) merged in at 35b4509c. Reviewed-by: rev-47@shanks at 35b4509c0a15f7900209f8260e4801c698efe204

mesh-tools added 89 commits 2026-10-30 00:21:39 +00:00

Revert "Merge pull request 'Revert L3: its node-engine removed every held module on novox, the bus server among them (issue 774)' (#557 ) from revert/774-l3 into main" 3dd7b734aa

This reverts commit 030f9ee4e7, reversing
changes made to 1555241393.

Pin mesh-sdk 22c9ddd in mesh-vault, the L3 re-land group's sdk head (issue 774) 071429c3be

Take mesh-config off the assignments readers: since #558 (issue 1953) it reads placement and its modules' manifests alone (issue 774) 16378e0850

nats: tools that remove one consumer and one node's keys from a bucket, with why and an audit event (issue 1952 N1-N3) aee377765e

Move the self-check out of the controller into mesh-self-check, so its runs and findings no longer need the controller's lease (issue 1952) d7247eae19

The probes whose facts are on the bus run with their assertions unchanged (D6, D7, D15, DW, H-bus); the rest answer deferred, naming what they lack, never a pass. Findings go to its own bucket for the condition keeper; each run is kept a week and said as checked. The finding layout is a copy of mesh-sdk's S3 until that lands.

nats: the tools run on the process account nats-tools with its own secret, and take the release sdk, so the bus grants them only the four removals (issue 1952 N1-N3) f7d821b023

messenger: hear the seat the take-over key names, so the keeper's conditions are not held beside the controller's (issue 1952 R1) 19bfbd82d4

mesh-watcher: hear the self-check run the take-over key names; take the release sdk and go 1.26 (issue 1952 W1) 77f6b4292c

Pin the release sdk in the build modules so a change's own failing test ends its level (issue 1289 R4c-2) 17588fee10

Write the self-check's findings and checked in mesh-sdk's health layout, so the condition keeper reads them as it reads every observer's (issue 1952) babe2f194a

Pins the group's mesh-sdk head 69fcb6c. A seat verb's run="true" is taken as the seat schema gives it.

Move the healers out of the controller into mesh-healers, acting on the keeper's events and only once the lease names no controller (issue 1952) 4c11c2c0c0

Until the lease write each act is written to would, kept a week by its bucket's age, so a would-act counts for no budget or brake. Acts begin by a create in attempts; H3 asks the grants part's assert; the brake and S15 are its own findings.

Pin checks/health to the group's mesh-sdk 69fcb6c, so it knows the health seats the new modules claim (issue 1952) 885b7a3a8e

Re-arm every running settle from the attempts kept when the healers start, so a restart loses no wake (issue 1952) 9dd27d2edb

A settle that ended while they were down is judged at once. The README says the week-long bucket age is the deleter of would and attempts for this release.

Add mesh-hand-acts, holder of the seat mesh-hand-acts, so the hand-act log has a home the controller's lease does not hold (issue 1953) e1cdeebcdf

Acts are kept 400 days under the runtime-proven caller and said as act-recorded; a warrant is read from the router's own record and recorded once per ask; hand-acts answers over both logs. The act layout is a copy of mesh-sdk's until L5-S2 lands.

Re-pin the release sdk; the bus module's tools declare the two removals they hold and leave the credential to the runtime (issue 1952 N1-N3) 2669fc29d8

nats: the tools read their credential from a word of their own, since the runtime strips MESH_BROKER_FILE (issue 1952 N1-N3) 48502c4ade

Keep the hand-act log as mesh-hand-acts' module log, the stream ADR 0565 §10 names, so every act is held in its order (issue 1953) 5b2b70cd6c

The warrants stay a bucket and no longer age, so a warrant is recorded once; one whose act was lost is written again from the router's record. hand-acts says the log's fill against its cap.

checks/health: take the release sdk, which reads the build.artifacts bus word of the nats tools (issue 1952) 43c44bcbee

Re-pin the release sdk; the nats tools' credential gets a directory of its own the tools can reach (issue 1952) 9ccb6f4060

Re-pin the release sdk; the nats tools' secret directory is /var/lib/mesh/nats-tools (issue 1952) 8fac0f89f7

Add the condition keeper, mesh-conditions, so conditions are raised and cleared from the observers' findings without the controller's lease, held while their module is delivered, and switched only on the operator's take-over (issue 1952 C1-C8, ADR 0564) d632461eb9

Say checked, healer-acted and act-recorded on their seats' subjects, where their consumers hear them, and pin mesh-sdk fbd00c8 (issues 1952, 1953) b5fdcb88c8

stdio.Emit put them on the module's subject, which no consumer of a seat's event hears. The manifests no longer list a seat's events as the module's own, which the SDK now refuses. mesh-hand-acts takes its act layout and names from mesh-sdk's handacts, and counts a repeated cause by its Repair rule.

Give each delivery's hold bound its own key's time to live and map a seat's finding to the module the placement says holds it, so the keeper holds the right findings for the right time (issue 1952 C7) 4cbdd0fc74

Wait on keys that expire on the bus instead of in-process timers, so the healers' settles and S15's fortnight survive a restart (issue 1952) 371abdaad1

A settle and the fortnight end are keys of wakes whose time to live is the wait, heard on expiry; attempts and would keep each key a week by its own time to live. Rearm stays only for a settle that expired while no instance watched. Pins mesh-sdk bc6c8a6.

Keep the hand-act log's acts 400 days, as ADR 0565 §10 has it, now that a module log may say its age; pin mesh-sdk 2e5cc4f (issue 1953) 0842361f85

Build config's switch from the controller's records: declared, compare, take-over and hand-back, so the operator approves the last move only once nothing differs (issue 1953) d34645361d

compare also resolves every placed module's values on every node both ways, as ADR 0565's insight of 2026-10-29 asks after #558. After the take-over the predefined list and pull requests' config come from the indexer and pull-declares is refused; a hand-back returns to the mirrors until the lease write. From the lease write a trusted value is set only on an Approve. Acts are recorded through mesh-hand-acts. Pins mesh-sdk 2e5cc4f.

Record a drill in mesh-hand-acts and count S15 over both logs, so the healers no longer refuse a drill (issue 1953) 45db0e650e

Serve compare, take-over and hand-back as the mesh-config seat's verbs, beside the module's own tools; pin mesh-sdk 4204afd (issue 1953) 4117ec3a57

Keep mesh-config's held values in mesh-sdk's KeptValue layout, which the checks read, and pin mesh-sdk 66ff33d (issue 1953) 077135a7f3

The layout and its keys are unchanged, so no value moves.

Refuse a manifest that records into the controller's hand-act log, naming the recorders' fallbacks until the lease write (issue 1953) 799f03fc2a

ADR 0565 §11's catalog check: every recorder records through mesh-hand-acts. The fallbacks are listed with why, and the list only shrinks.

Compose every provision and plan in mesh-provisioner from the bus on change events, its state from the providers' outcomes, and ask before retiring (issue 1954) 0a2334e2cd

The seat holds provisions and plans alone (ADR 0570 §1-§3, §5): a node is composed at its wanted builds and its newest, so a key ahead of its want is never retiring; a key no build needs is retiring with one ask, and the warrant goes into the key for its provider. The registry's placement kinds name plans; needs is emptied at start. Pins mesh-sdk 66ff33d.

Never set a provision provided without its provider's outcome, read from the entry's state and why, and pin mesh-sdk 692dbf9 (issue 1954) 6bee0b6789

A provider that writes no outcome leaves its keys asked (ADR 0570 clause 3), so a provider that fails is never hidden.

Read the outcome of a provider that makes no credential per consumer from its own have, so its consumers' keys turn provided once it runs healthy at its wanted build (issue 1954) bc2e8b1fd2

Failed is failed with the have's words; anything else stays asked, saying what it waits for. Providers that make a credential per consumer keep their bound entries.

Merge the release branch; checks/health takes its newer sdk pin (issue 1952) bc2efe658c

Judge each provider by what its manifest declares and fill what a provision depends on, so a templated pair provider is provided only after its credential (issue 1954) 451d790edd

A provider declaring bound is judged by its entries, any other by its own have, and only once every provision it depends on, its secret-<provision> first, is provided. The credential is the vault's own provision. One integration test runs the holder on a real bus over mesh-sdk's in-memory mesh: a want composes the key, the outcomes provide it, leaving the node asks. Pins mesh-sdk 6b8c694.

Every offer of twenty providers says its credential form; a check refuses one that does not; re-pin sdk 6b8c694 (issue 1952) 1f0ff6ed98

checks/credentialform: a module that provides and keeps a bound state must be read by mesh-provisioner (issue 1952) dd9b6a0485

Merge the leaves branch into the release branch, so it carries a5's work (issue 1952) d276452d46

Merge the leaves branch at dd9b6a04 into the release branch, with its check of the providers' read outcomes (issue 1952) 4cc94f06d2

Declare the credential form of seven offers, read from each provider's code, so the plan says what the software makes (issue 1954) 313aae7440

A login per consumer is pair (postgres, keycloak's client, gitea's npm registry, the vault's secret); an offer whose consumers present none is none (gitea's git and forge account, route-adapter's route).

Give every alert kind the controller raises its home in the keeper's table, as the lead settled them under the operator's yes, so compare names no kind without one (issue 1952) 1b1a94f35b

Carry plan-failed and stalled until the lease write as kinds that end with the controller, as the lead decided, so compare names no kind without a home (issue 1952) a95ad78a9a

The vault reads each plan from mesh-provisioner, says each credential's outcome in its own bound state, mints nothing for form none, and answers rotate and issued for any proven caller (issue 1954) bb18541469

Each outcome names the value by the vault's own name and generation, never the value; held is provided, issued or given waits, an entry goes only on a retiring key's Approve. The vault no longer reads the controller's assignments, so it leaves the list of their readers. Pins mesh-sdk 6b8c694.

mesh-self-check: D8 reads placement's addresses and endpoints and asks each heard holder's ban list, so the probe moves off the controller with its kind, key and words (issue 1952, ADR 0564 §6) 1ba6c0de27

The mesh's own addresses come from placement (each node's private address and its endpoint); who holds
node-intrusion-prevention, from seat-holders; who is heard, from the watchman's presence. Each heard holder is asked
seat:node-intrusion-prevention.banned@<node>, all at once. The finding is the controller's: machine.<node>.bans-the-mesh,
urgent, the same summary and words. A heard holder that does not answer makes the probe fail to run, never pass.
Rows in TestTheBusProbes; the manifest reads and invokes what the code does.

Hold the seat artifact-store in artifact-store-tools: the keep set from the store's listing, the wants and each module's five newest builds, the sweep at a build's end once the lease names no controller with each deletion recorded, and S25's image-missing (issue 1955, ADR 0571 clause 2) 1b1019972d

Read the seat artifact-store's findings in the condition keeper, so S25's image-missing comes from its holder and not the controller (issue 1955, ADR 0571 clause 2) c2a6a6f6dd

disk-load: measure the node's data items and keep each reading and seven-day peak as its own state readings (issue 1955, ADR 0571 clause 3) 2802fdf783

mesh-watcher: read who holds the controller and the bus from the placement seat's state, not the controller's seats (issue 1955, ADR 0571 clause 4) 3d3524dd69

docker: ask the artifact-store seat for the images a machine's declarations name, not the controller (issue 1955, ADR 0571 clause 2) 7b5728b1f3

nfs-server: read the backup seat's holder from the placement seat's state, not the controller's seats (issue 1955, ADR 0571 clause 4) 34a6932288

minio: read who holds a seat from the placement seat's state, not the controller's seats (issue 1955, ADR 0571 clause 4) 45902cd728

disk-load: take the node's data items from the bus and the readings layout from the sdk, with no composed file (issue 1955, ADR 0571 clause 3) f149768f5d

restic: read each item's reading and peak from disk-load's state and the item list from the sdk; the measuring leaves the backup holder (issue 1955, ADR 0571 clause 3) 44a3f24456

nfs-server: learn the backup holder's data paths from disk-load's readings, not the controller's data (issue 1955, ADR 0571 clause 3) 4ddf702145

mounts: learn where the mesh keeps data from disk-load's readings, not the controller's data (issue 1955, ADR 0571 clause 3) 03938f8244

restic: the night process is judged by its unit; the module has no runtime account for a tool check to stand beside (issue 1955) ce8ea7bbe2

disk-load: compose the node's data items through the sdk's runtime reader, not a copy of it (issue 1955) 9d0ebe556e

restic: judge the night by its health tool again beside an awake process, and keep each item's last good night as its own state good (issue 1955, ADR 0571) b1fa4855e6

mesh-catalog: take builds in from the builder's log through builder_builds_since and its done keys, not the controller's built-before (issue 1955, ADR 0571 clause 9) de86f8123d

restic: write each item's last good night when a night's files change, with no timer, on the sdk's layout; the night is judged by its unit and the awake process by the health tool (issue 1955) 86e1082fa0

Remove the compiled programs of the build modules from the tree and ignore them (issue 1577) f73f04b1e9

nats: nats_streams answers each stream's max_bytes beside its size (issue 1952) cab031bf10

Judge the store's mirror copies by what the builder's log says each build stood on, and the images a manifest pins by itself, so mirrors are recorded and image-missing covers what the controller judged (issue 1955, ADR 0571 clause 2) 63023ca5fe

mesh-provisioner-client: read plans from the seat, record no needs, leave ${secret:} for the node-engine 542c3214c6

ADR 0570 clauses 1, 2 and 6 move the plan to mesh-provisioner_plans, fold needs into the seat and give
the node-engine the secret's fill, so the client no longer holds a value (issue 1954, V3).

mesh-self-check: DA judges each machine's agent account on its engine's own report, so the probe moves off the controller with its kind, key and words (issue 1952, ADR 0266) 89f63b3ab8

The agent account a machine names comes from placement; the engine's verdicts on it from its health report
(mesh-host.health, NodeHealth.Root and Contract); whether that verdict is on now from the watchman's presence, since
the report is written on a change only. The judgement is the controller's: no report, not heard, an engine older than
the judging, no verdict, or any verdict not healthy is not confined; a verdict waiting for the setuid search is quiet
for twelve hours from when this self-check first saw it waiting. machine.<node>.agent-root, urgent, the controller's
words. Rows in TestTheBusProbes; pins mesh-sdk f3677eb.

mesh-self-check: D13 judges the data from disk-load's readings, restic's last good nights and the indexer's manifests, with the controller's kinds, keys and words (issue 1952, ADR 0233) f7f02abbb3

Hold the seat by its name mesh-artifact-sweep, as ADR 0121 names a mesh seat for its scope (issue 1955, ADR 0571 clause 2) 50d46f6e94

mesh-self-check: D-fill judges every module's bucket and log against its cap from the bus module's nats_streams, with the controller's kind, key and words (issue 1952, ADR 0297 §6) c2209864ee

Its registry row and test rows are already in f7f02abb; this is the probe they call. Each KV_<module>_<name> and
LOG_<module>_<name> stream is said filling at three quarters of its max_bytes, and kept while a finding is written
until under seven tenths. One read of every stream, so the bus module not answering is the probe failing to run, which
clears nothing; bucket-or-log-unread is not raised.

docker: ask the seat mesh-artifact-sweep for the images a machine's declarations name, as ADR 0121 names the seat (issue 1955) 62b53c2eda

mesh-self-check: give D-fill's written-finding fixture its severity, so the row keeping a fill between the thresholds reads the finding it wrote (issue 1952) 936e7308bf

disk-load: keep a reading of an item no longer declared, retired with when and why, and clear it when declared again; sdk 31e90aa (issue 1955, ADR 0233) 60231234a5

mesh-self-check: D13 raises empty-replacement and cleanup-waiting for retired readings and counts a missing backup from the first measurement; sdk 31e90aa (issue 1952, ADR 0233) 6e5fd647c0

Providers read the seat's plans and retire only on the operator's yes, removing the credential 4a329f0443

ADR 0570 clauses 2 and 5: gitea, keycloak, postgres and route-adapter read mesh-provisioner.plans and
.provisions instead of the controller's assignments. postgres hands the database to a keeper and drops the
login; keycloak replaces the client's secret and keeps the client, so the data stays (issue 1954, V4).

Carry the kinds of the probes the operator dropped (D2, D3, D4, D12, D-heard) until the lease write, so compare names them as ending with the controller and never as missing (issue 1952) 0c0e8960d8

Carry the kinds of D-root, D11 and D13's data held twice, which the operator dropped too, until the lease write (issue 1952) a783ffa497

Merge branch 'feat/774-l3-reland-leaves' into merge-leaves 0e32f95492

mesh-provisioner composes through fromruntime's reader; pin mesh-sdk fa5a6ae across A3's modules cb7a5c9939

The SDK now reads a module's `reads` through the runtime, so busread.go's own copy goes, and the
manifest is held to frombus.StatesReads (issue 1954).

Pin the newest mesh-sdk in the modules of the leaves (issue 1955) c1eb7783e8

Merge remote-tracking branch 'origin/feat/774-l3-reland' into merge-leaves 5a1916cb46

gitea waits and keeps serving while its keys are unread; pin mesh-sdk 007b6fd 475ffdcb33

After a restart the forge-account provider returned a plain error, which the helper writes as failed
with its values cleared, so every consumer lost its forge entry until the keys loaded. provider.ErrNotReady
keeps the entry serving, in waits (issue 1954).

mesh-self-check: the setting probes lists the probes that run, D13 and D-fill by default; one unlisted runs nothing and the verb says it is off by setting (the operator, 2026-10-30) 818f8d2616

Merge remote-tracking branch 'origin/feat/774-l3-reland' into merge-leaves 8432cb844f

mesh-self-check: the setting probes is read by the module's code at runtime (ADR 0348 §5) 8ae8e88ddd

Pin mesh-sdk 007b6fd, the release's final head (issue 774) 1b16720030

Keep unchanged modules on their pins, so the delivery moves only what this release changes (issue 774) fe52629c28

Pin mesh-sdk 6e4a301 in the modules the release changes, and keep the others on their pins (issue 774) 9cb177cae6

Keep the client filling secrets and reading the controller's assignments until the lease write; plans for held modules; warrant says yes fc66555ecc

Review round 3 (issue 1954): the node-engine of today writes ${secret:} as given and the new one fills only
provision keys, so the client fills secrets this release; plans exist only for modules with a want, so a
module without a seat plan is read from the controller's assignment and never leaves for want of one, and
the holder composes held modules at the build they run; the SDK's provider lets go only on "yes".

Merge remote-tracking branch 'origin/main' into final-pin 820b4ee4c5

# Conflicts:
#	modules/mesh-build-image/go.mod
#	modules/mesh-build-image/go.sum
#	modules/mesh-build-node/go.mod
#	modules/mesh-build-node/go.sum

Pin mesh-sdk 2da0de7, the release's final head, in the modules the release changes (issue 774)

mesh/delivery ready: it is merged on the Approve

Details

mesh/check checking 8e402c82bfb2 on 12c2574c9d54 | novox/mesh-catalog#560 in group feat/774-l3-reland

mesh/delivery-group group feat/774-l3-reland checking: the checker's verdict is out of date: novox/mesh-catalog@8e402c82bfb2 was judged on trunk f61265f7a0f3, …

Details

8e402c82bf

mesh-tools commented

2026-10-30 00:22:12 +00:00

Delivery novox/mesh-catalog@35b4509c0a15 — ready since 2026-10-30T00:55:13Z

Group feat/774-l3-reland: novox/hq@0c450cad25, novox/mesh-catalog@35b4509c0a, novox/mesh-controller@79f44386d2, novox/mesh-delivery@8edc83950c, novox/mesh-host@015334020a, novox/mesh-sdk@2da0de75c3, novox/mesh-tools@250aa73a56

Transitions

2026-10-30 00:55 (new) → proposed (announced): novox/mesh-catalog#560's head announced
2026-10-30 00:55 proposed → ready (accepted): a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail; or the check gates no merge (check-gates-merge false)

The commit's note under refs/notes/mesh-plan keeps every transition of this change up to its merge: git log --notes=mesh-plan.

**Delivery** `novox/mesh-catalog@35b4509c0a15` — **ready** since 2026-10-30T00:55:13Z **Group** `feat/774-l3-reland`: novox/hq@0c450cad258e, novox/mesh-catalog@35b4509c0a15, novox/mesh-controller@79f44386d24d, novox/mesh-delivery@8edc83950cea, novox/mesh-host@015334020ad2, novox/mesh-sdk@2da0de75c3f0, novox/mesh-tools@250aa73a5631 **Transitions** - 2026-10-30 00:55 (new) → proposed (announced): novox/mesh-catalog#560's head announced - 2026-10-30 00:55 proposed → ready (accepted): a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail; or the check gates no merge (check-gates-merge false) The commit's note under `refs/notes/mesh-plan` keeps every transition of this change up to its merge: `git log --notes=mesh-plan`.

jschoubben added 1 commit 2026-10-30 00:55:07 +00:00

Merge remote-tracking branch 'origin/main' into gitea-merge2

mesh/delivery ready: it is merged on the Approve

Details

mesh/check failed: shanks: novox/mesh-catalog#560: modules/mesh-build-image/mesh-build-image: a compiled program added in 17588fee10f8 and removed lat…

mesh/delivery-group group feat/774-l3-reland rejected: composed together: the checker's verdict at the members' heads (fc2ace3a8f88): shanks

Details

35b4509c0a

Some required checks failed

mesh/delivery ready: it is merged on the Approve

Details

mesh/check failed: shanks: novox/mesh-catalog#560: modules/mesh-build-image/mesh-build-image: a compiled program added in 17588fee10f8 and removed lat…

mesh/delivery-group group feat/774-l3-reland rejected: composed together: the checker's verdict at the members' heads (fc2ace3a8f88): shanks

Details

You are not authorized to merge this pull request.

This pull request can be merged automatically.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin feat/774-l3-reland:feat/774-l3-reland

git checkout feat/774-l3-reland

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-catalog#560