Commit Graph
139 Commits
Author SHA1 Message Date
jschoubben 3ed9abf658 Format the health report's aliases as gofmt does (issue 774) 2026-10-29 22:59:38 +01:00
jschoubben 552f97c919 Take the node's health report layout from mesh-sdk 94d0de5, so the engine and the watchman read one form (issue 774)
delivery.NodeHealth and its parts replace this package's copies. The message's network and units parts are aliases of
the same types, so its JSON is unchanged. The engine keeps the derivation from its statement and the write on a change.
Test: the key's bytes, decoded and encoded again by the sdk, are the same bytes.
2026-10-29 22:59:22 +01:00
jschoubben 65fbc1e6f8 Write each node's health report into its own key of mesh-host_health on a change, so the watchman raises the machine's alerts without the controller (issue 774, the operator's alert table)
Until now the six machine alerts came from a health message passing to the controller. The report is now kept in the
engine's declared state `health`, key `<node>`, created by the grants part as `acts` is. Only this node's engine
writes it, after comparing it on every look and only when what it says changed; it is read first after a restart.

It carries:
- the network parts (machine-network; network-rewritten through the resolver file part's writer and owner);
- the tunnel's peers, each seen within five minutes or not (network-unreachable);
- the failed units no module owns (machine-units);
- the accounts marked relogin needed (relogin-needed);
- the directories used as found (directory-used-as-found).

The streaks and the time of the look are left out, so a look alone writes nothing. A peer is reported seen or not,
never its handshake time, so a renewed handshake writes nothing.

The layout is internal/link's (NodeHealth) until it moves into mesh-sdk. Pins mesh-sdk 58a8324.

Tests:
- rows in TestOnTheHubAnyFreshPeerIsAHealthyTunnel: the peers, and a change only when one is seen or lost;
- a case in TestAHealthStatementIsSaidOnTheLinkOpenNow: what the report takes, and one write per change across a
  restart;
- the manifest's states in TestTheManifestConsumesTheBackupEvent.
2026-10-29 22:56:14 +01:00
jschoubben 45d0348044 Revert "Merge pull request 'Revert L3: its node-engine removed every held module on novox, the bus server among them (issue 774)' (#174) from revert/774-l3 into main"
This reverts commit 1ec73915b8, reversing
changes made to 0a4c35f731.
2026-10-29 21:04:13 +01:00
jschoubben f81e6147ae Revert L3 (mesh-host, merge 0a4c35f): its node-engine removed every held module on novox, the bus server among them, at 08:54 CET on 2026-10-29 (issue 774)
mesh/delivery merged: source.main moved to novox/mesh-host@1ec73915b807
mesh/delivery-group group revert/774-l3 merged: every member is merged
mesh/check being carried to f81e6147aee1 on 0a4c35f731a7 | from mesh-catalog 1555241393fe | novox/mesh-host#174 in group revert/774-l3
L3 lands again with two fixes: a held module is never removed, and the grants cover L3's new subjects.
2026-10-29 09:42:29 +01:00
jschoubben 6cbdb3712e Judgements and refused setuid searches, which any caller may ask, roll in one bounded key of the node's and add no key to the bucket every node shares; a key per act is kept for hand-overs, access acts and searches started (issue 774, ADR 0563 §2; rev-l3 round 2) 2026-10-28 20:35:33 +01:00
jschoubben 2a0c428ef9 The last setuid search's start and each library's own take ask are keys of their own, which no number of other acts moves (issue 774, ADR 0563 §2, §4; rev-l3 round 1) 2026-10-28 20:35:33 +01:00
jschoubben 95dd64c774 Every signed ask is verified with the controller's key the engine trusts at that ask, so after the lease write none signed by it is taken (issue 774, ADR 0563 §7, ADR 0550 §2; rev-l3 round 1) 2026-10-28 20:35:33 +01:00
jschoubben 81fa009bf4 The node seat's acts and the engine's own asks are the node-engine's declared state acts on the bus: one key each under its own node's keys, written by compare-and-set, one ask open per path made by a create; acts reads one by its id (issue 774, ADR 0563 §2, §3) 2026-10-28 20:35:33 +01:00
jschoubben aaffc19232 The node seat node-engine served on the engine's link: its verbs with their proven caller, its acts kept in its own state, a hand-over or access act asked of the operator as node-engine-<node> and performed only on the warrant (issue 774, ADR 0563 §1-§3) 2026-10-28 20:35:33 +01:00
jschoubben 479ac71050 The node trusts the vault's key beside the controller's until the lease is handed away, says what it trusts in its record, asks the vault for its own certificate, composes the vault's authority beside the controller's, and reads the kept declaration no more once every placed module has a have (issue 774, ADR 0550 §2) 2026-10-28 20:35:33 +01:00
jschoubben dd8ba97989 Node agent: wait for the bus's word that the seat's placement and candidates states were created, then watch them (issue 1126) 2026-10-28 18:43:49 +01:00
jschoubben ab22efd8b1 Node agent: follow the placement key at start only and say why when it cannot; test the reached candidate and the switched rule; word the log by the seat's state (issue 1126) 2026-10-28 18:43:49 +01:00
jschoubben 8639b373d2 Prove a missing or refused bucket is a watch error, so a node waits on the controller's compositions and tries again (issue 1126) 2026-10-28 18:43:49 +01:00
jschoubben 2fe9d678fc Node agent: answer a proposal's candidate keys and follow the seat's placement, not the indexer's groups (issue 1126)
The answer names its proposal and the revision of the node's own candidate key; a node with no key in a proposal is not asked. The watch for its own placement key is tried again until the seat's state exists, so the node switches when its key is written.
2026-10-28 18:43:49 +01:00
jschoubben cd1d2ff447 A read the bus did not answer is tried again after a bound, never kept as an answer: the shadow's answer, the node's composition and the own-secret step (issue 1126)
Met twice on 2026-10-28: shanks kept 'want.shanks.brother could not be read ... no
responders' as the own-secret step's last word, and ace kept 'the vault's own: not yet
... no responders' as its answer to seed-7, each until another input moved. Every bus
read of the composer (manifests, builds, archives, served values, the vault's own and
credentials, the have of what runs) is noted (BusFailed); an answer or a pass that met
one is made again after a growing bound (5 s to 5 min), and a pass is not handed when
the have of its sealed values did not answer. The own-secret step tells a read or an
ask that did not answer (Retry, taken again) from a key absent or the vault's word.
2026-10-28 12:11:08 +01:00
jschoubben a7cd0a7807 Node shadow: reuse a running seal only at its own slot, lay values as the controller does, answer the newest candidate first and say how it goes (issue 1126, C1 C2, T1 T2 T3)
A box is reused by slot (resource, field) after the composition, from what runs or the
have's slots; a have written before slots is reused within its module alone. Config
values are laid by mesh-sdk's one rule (never a default). The shadow answers the
newest candidate first and hands each answer as it lands, says when an answer starts,
every minute what it waits on, and when it ends, and fits an answer to the node's key.
Needs mesh-sdk fix/1126-seed-same (16b8d8a1).
2026-10-28 09:41:29 +01:00
jschoubben 4993f1b1d7 Make the switch between composers safe: a composer tells only its own run's agent, a pass whose run ended refuses nothing, a bus that pauses is tried again, a mark keeps what runs and clears at the next start; pin mesh-sdk 7359d18 (issue 1126)
Live 2026-10-28: the own run's last pass, failing on its run's end, marked
every module of the controller-mode agent that replaced it (ace 60, shanks 63),
and the marks dropped the build and digest because the have read failed in the
bus's reload pause. Now: the refusal hook is bound to the run's agent and ctx;
a pass whose context ended returns nothing; JetStream not enabled or no
responders are tried again after 1, 2 and 4 s before any mark; NotComposed reads
the have again while the bus does not answer and marks nothing it cannot read;
baseline takes a have a composer marked as what runs, so a mark it did not make
clears; each run's shadow lives with the run.
2026-10-28 00:34:10 +01:00
jschoubben 4e8fdfdf04 The way back: a node whose placement key is deleted returns to the controller's compositions without a restart, says so in its journal and its have, and answers such a candidate as returning; pin mesh-sdk 472d5f3 (issue 1126, the seed's revert)
The seed's revert (mesh-catalog #526) was refused on 'would stop running node',
while a self-composing node would have refused its placement's absence, kept
what runs and marked every have failed until a restart. endWhenUnplaced ends the
agent's run when the key goes, so it begins again on the controller's
compositions, as at a start without a key; the composer refuses nothing while
unplaced; the node's have says whose compositions it applies (Composes); the
shadow answers a deleted placement Returns, verdict same.
2026-10-27 23:25:59 +01:00
jschoubben fb0cdabe25 Answer every placement candidate in shadow in the node's own key, applying nothing, so a placement change is checked by the nodes it places before it merges (issue 1126, ADR 0559 §4) 2026-10-27 02:26:07 +01:00
jschoubben 0474a2a3cb Keep the node's own composition's sealed values in its key and read a policy within the agent's context, so a restart applies nothing again (issue 1126) 2026-10-27 02:21:36 +01:00
mesh-tools 180a7e7b80 Merge pull request 'A directory used as found is not a module's health, and a failed have names its cause (issue 1496, ADR 0557 §4, §5)' (#150) from fix/1496-start-health into main 2026-10-26 21:38:02 +00:00
jschoubben b225c7bb4c A directory used as found is not a module's health, and a failed have says why (ADR 0557 §3, issue 1496)
mesh/delivery merged: source.main moved to novox/mesh-host@180a7e7b80e2
mesh/delivery-group group fix/1496-start-health merged: every member is merged
mesh/check checking b225c7bb4cf5 on be94d2689e2f | novox/mesh-host#150 in group fix/1496-start-health
On ace, 2026-10-26 21:19, mesh-build-node's cache directory was there before the
mesh and used as found. The health statement says such a directory unhealthy, and
the have folded it into the module's health as failed, with no words, while the
module's own check answered healthy; the assigner read it as its canary failing
and held the version on all four nodes.

The directory waits for the operator's hand-over and keeps its own condition,
raised from the same statement; it no longer counts in the module's have. A have
whose health is failed or waits and whose apply said nothing carries the words of
the resource that made it so, so the next one is readable.
2026-10-26 22:06:32 +01:00
jschoubben 4476e58080 Review of L2, round 2: a refusal at the first compose after a start keeps what the module's have said runs (1); each have of a self-composing node carries the values its composition holds sealed, and the composer reuses them after a restart, so a rotated secret moves no composition at a restart (2); pin mesh-sdk dd9d9d5 (issue 1126, ADR 0547 §3)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 4476e5808047 on ff9386dcccc2 | novox/mesh-host#149 in group feat/1126-nodes-compose
2026-10-26 20:23:16 +01:00
jschoubben f8afaefba0 The node keeps the mesh's own certificate it applies, from its kept declaration at start, for its record; one agent test folded to hold the count (issue 1126, ADR 0555 §3)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: ace: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk 91413de33516, which is neither on mesh-sdk's trunk nor the grou…
2026-10-26 19:24:36 +01:00
jschoubben 02d9d7b053 The node composes its own modules once the catalogue places it: one composer per node drains a mark set by any input's change, the agent applies what it composed when it differs, and a record move waits for a yes (issue 1126, ADR 0547 §2, §3)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 02d9d7b0538f on ff9386dcccc2 | novox/mesh-host#149 in group feat/1126-nodes-compose
2026-10-26 18:44:58 +01:00
jschoubben 8be1c2ab0d The node agent writes what the machine observes of itself as its record in its own have key, only when it changed (issue 1126, ADR 0555 §3) 2026-10-26 18:32:54 +01:00
jschoubben 43f2d6020e Write the bus's user list into a have no apply here says, merged over the key's value
mesh/delivery merged: source.main moved to novox/mesh-host@94f9aa298cb8
mesh/check pass 43f2d6020e11 on 94f9aa298cb8 | carried from 7f04d1204765, mesh-catalog a4f5df797b68, mesh-sdk b4cd95e7c4a4 | unit 972 run, 423 kept, 3…
On the bus node the bus module has no want and has left the declaration, so no
pass of this engine held its fact and the follower's user list was kept in
memory and never written: the vault and the grants part read no hashes and no
key. A module's user list now reaches its key on every pass, merged into what
the key says with every other field kept; a missing key is said once, never
made up (ADR 0547 §8).
2026-10-26 13:38:40 +01:00
jschoubben 882e75c954 The bus node follows the grants part's sealed user list from take-over, the controller's otherwise, never both (ADR 0547 clauses 7, 8; issue 1945)
On the node whose compositions carry the bus's list, the engine also watches mesh-grants' key user-list.sealed. While it
says follow grants, the object it names is read by name, opened with this machine's sealing key, checked against the
key's digest and the guard, and written to the same file at 0600 only when its digest differs; the bus reloads it
itself. The have says which list is followed and this machine's sealing key, which the grants part seals to.
The follower's test waits for its watch to catch up before a later put, which raced under load. Pins mesh-sdk 175a5567.
2026-10-26 12:06:17 +01:00
jschoubben 99285cb080 Retry an apply, a fetch by digest and a have write that met the bus reloading, and keep the cause in the have
mesh/delivery merged: source.main moved to novox/mesh-host@1195cf45971e
mesh/check pass 99285cb080db on 1195cf45971e | unit 303 run, 1080 kept, 5 skipped | integration 306 run, 1082 kept, 4 skipped | over bound: unit (slow…
The bus node's engine writes the bus user list (ADR 0547 bridge); each write makes the bus reload its accounts, and
for about three seconds every JetStream request meets no responders. An apply that failed then was kept failed until a
key changed (mesh-vault on novox, 2026-10-25 23:04:24), holding every module that needs the secret seat. A transient
bus failure is now retried with a backoff (2 s, 5 s, 15 s, 60 s, then every 5 min), the have saying retrying meanwhile;
after the engine writes the user list, the transiently failed modules are tried again once the reload is over. A have
write that met it is written again the same way. shorten keeps the head and the tail, the cause.
2026-10-25 23:37:02 +01:00
jschoubben 3f02bd7ff6 The bus node writes the bus's user list alone from the newest composition of the bus module, with no want, and says it in the bus module's have (ADR 0547 §8, issue 1945)
The list reached the bus only through a want of the bus module, which nobody writes (ADR 0539 §15): on 2026-10-25 it
stood from 11:23 to 20:50 and was put back by hand. The node agent now follows its composition bucket for the newest
signed composition carrying the user list, writes that file alone through the apply's file step when it differs
(mode 0600, read back, recorded as the module's), moves nothing that reads it (the bus module's entrypoint reloads the
server in place), never goes back to an older list, looks again after every apply of the bus module, and says source,
revision, wanted and written digests, users and any error in the bus module's have (mesh-sdk UserListWritten).

Read with mesh-delivery-view.node_haves (module nats); the bus's own nats_user_can says in_force once it reloaded.
2026-10-25 21:50:49 +01:00
jschoubben b32a27c115 Owe a look when a module's change ends unjudged, so a module with nothing long-running is ok within a look of its apply
mesh/delivery merged: source.main moved to novox/mesh-host@5f16cf507a0f
mesh/check pass b32a27c115a2 on 0dfbe1a179f2 | unit 292 run, 1060 kept, 5 skipped | integration 295 run, 1062 kept, 4 skipped | over bound: unit (slow…
The facts look inside an apply runs while the module is still applying and judges nothing; the
running write then left xorg running and unknown until the five-minute statement, holding every
x11-display consumer behind it. The running write now nudges the health loop, which looks and says
at once. A declaration's report gets the same motion and composition check, and a removed module
leaves nothing marked unjudged (review of fix 15).
2026-10-25 15:46:05 +01:00
jschoubben c16e625a08 Say a module's health is not yet judged while its change is under way and until a look after it
A have said applying and failed for the controller on novox at 15:27:36, from a look at the
container being replaced, two seconds before its launch said healthy; the assigner held the
canary on it and set novox back. A look now judges only a module that is not in motion, and a
module whose composition moved carries unknown until a look after the move.
2026-10-25 15:32:59 +01:00
jschoubben 1cdb4d1a29 Say a module found running on its own have again at the agent's start, so its health is this engine's and not an older engine's unknown
xorg's have on g14 and shanks kept the unknown an engine before 958ccbd wrote: the
baseline believed the have and never handed the module to the session, so no look
rewrote its health, and the assigner held every module requiring x11-display back.
2026-10-25 15:27:23 +01:00
jschoubben 11a635796d Bound a key's history read to 10 s and test it on a real bus; the witness test waits for its process to run from its image (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-host@57b553ac5a57
mesh/check pass 11a635796df9 on 57b553ac5a57 | unit 299 run, 1043 kept, 5 skipped | integration 302 run, 1045 kept, 4 skipped | over bound: unit (slow…
2026-10-25 12:51:00 +01:00
jschoubben 8343c4fb36 Apply the revision of a composition key whose digest the want names, from the revisions the bucket keeps (issue 1945) 2026-10-25 12:42:20 +01:00
jschoubben 958ccbde3e Say a running module ok once looked at when nothing of it is judged, so mesh-host's canary can be healthy and its other nodes get the new engine (issue 1945)
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check pass 958ccbde3ed4 on f06058390c0f | unit 297 run, 1038 kept, 5 skipped | integration 300 run, 1039 kept, 4 skipped | over bound: unit (slow…
2026-10-25 11:08:10 +01:00
jschoubben e5a06cc73d The node agent waits for a backup's end on the holder's event, not by polling; the manifest consumes it (R6-A A5) 2026-10-24 17:03:30 +02:00
jschoubben 96d0322498 The node-engine reports reads-wants; a failed apply is said; the have's fingerprint and digest follow its health (R6-A A6) 2026-10-24 16:43:26 +02:00
jschoubben b2003e7a38 A composition is applied only when the controller or the operator signed it; the node agent's integration tests on an embedded bus (R6-A A9, A1, A5) 2026-10-24 16:42:11 +02:00
jschoubben 3303797bfb The node agent runs in the node-engine: watches the wants and compositions on the link, applies one module's composition, asks the backup, writes the haves (R6-A A1, A5, A6) 2026-10-24 16:39:51 +02:00
jschoubben d5dd7761c6 The node agent: follows the wants and the node's compositions, applies one module at a time, backs a data module up first, holds it on a failed backup (R6-A A1, A2, A5, A6) 2026-10-24 16:36:21 +02:00
jschoubben 1598156752 The node-engine hears no declaration: the queue, orders, put-backs, unsaid, numbers and news go; a session says the facts in the node-engine's own order, and the first have proves a build (R6-A A7, A8) 2026-10-24 16:25:11 +02:00
jschoubben d5eeb860e3 Read a file's object from the node's own bucket, declaration-files-<node>, never a shared one (issue 1695, ADR 0486 §2)
mesh-sdk is pinned to fix/1694-bucket-per-node 36d9eba51db9 until it merges.
2026-10-21 15:41:09 +02:00
jschoubben fdb59bcf4e Correct the grant list above FileObject to the subjects it calls, flow control and consumer delete included (issue 1695)
mesh/merge-gate head fdb59bcf4eb7 merged into trunk c5c4caa2a4ab — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head fdb59bcf4eb7 merged into trunk c5c4caa2a4ab — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/delivery delivered
2026-10-21 15:01:31 +02:00
jschoubben 4cfa0469fd Run the object-bucket test in the trunk's tagless suite too: the trunk's merge check judges this change (issue 1695) 2026-10-21 14:56:41 +02:00
jschoubben 45f437eea6 Run the integration tests in the merge check, resolve file references in the hand apply, refuse an unresolved one, and read an object by name without listing the bucket (issue 1695) 2026-10-21 14:50:34 +02:00
jschoubben a457b38968 Fetch a declaration's files given by digest from the bus's object bucket, verify them and report file-references (issue 1695, ADR 0486)
mesh-sdk is pinned to fix/1694-file-references 6c857e1c025f until it merges.
2026-10-21 14:34:29 +02:00
mesh-tools 454be9b269 Merge pull request 'Write a module that leaves a node as removed in its have key, so a reader can tell it stopped (issue 1369)' (#121) from fix/r3e-before-the-gate into main 2026-10-20 06:01:59 +00:00
mesh-tools d6f8b00bf8 Do not let the asking's Close shut the stand-in's connection before its cleanup (issue 604)
mesh/merge-gate head d6f8b00bf822 merged into trunk 2474bf28330f — pass: the change touches no module of the mesh's graph
mesh/repo-check head d6f8b00bf822 merged into trunk 2474bf28330f — pass: its merge-check.sh passed
mesh/delivery delivered
The asking helper registered a.Close, which closes the shared test connection; the stand-in's cleanup ran after it on a closed connection, so its consumer was never deleted and blocked the next test on the workqueue stream.
2026-10-20 01:34:27 +00:00