Author SHA1 Message Date
jschoubben f81e6147ae Revert L3 (mesh-host, merge 0a4c35f): its node-engine removed every held module on novox, the bus server among them, at 08:54 CET on 2026-10-29 (issue 774)
mesh/delivery merged: source.main moved to novox/mesh-host@1ec73915b807
mesh/delivery-group group revert/774-l3 merged: every member is merged
mesh/check being carried to f81e6147aee1 on 0a4c35f731a7 | from mesh-catalog 1555241393fe | novox/mesh-host#174 in group revert/774-l3
L3 lands again with two fixes: a held module is never removed, and the grants cover L3's new subjects.
2026-10-29 09:42:29 +01:00
mesh-tools 0a4c35f731 Merge pull request 'L3: the node trusts the vault's key, serves the node seat node-engine and keeps its acts on the bus (issue 774, ADR 0561), rebased onto L2' (#168) from feat/774-vault-mints-l2 into main 2026-10-29 07:44:19 +00:00
jschoubben 4f9227a186 Merge main (#171) into L3: pin mesh-sdk c2999a4, mesh-sdk #128's head, which holds L3's sdk and both seed fixes (issue 774)
mesh/delivery merged: source.main moved to novox/mesh-host@0a4c35f731a7
mesh/delivery-group group feat/774-vault-mints-l2 merged: every member is merged
mesh/check pass 4f9227a1860f on fb8ea0e6838c | unit 1601 run, 5122 kept, 47 skipped | integration 1637 run, 5215 kept, 49 skipped | over bound: unit, …
2026-10-29 01:08:25 +01:00
mesh-tools fb8ea0e683 Merge pull request 'Pin mesh-sdk c2999a4, so a node's seed answer holds a held module as it runs and its neighbours compose (issue 1126)' (#171) from fix/1126-held-as-runs into main 2026-10-29 00:07:06 +00:00
jschoubben a6c70803ab Pin mesh-sdk at c2999a4: a held module no longer spreads its hold to its neighbours (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-host@fb8ea0e6838c
mesh/delivery-group group fix/1126-held-as-runs merged: every member is merged
mesh/check pass a6c70803ab72 on bb6e47293909 | unit 3054 run, 234 kept, 36 skipped | integration 3075 run, 266 kept, 29 skipped | over bound: unit (sl…
novox's seed p-20261028t231141-ec84b4 refused 11 modules behind de-spiegel's hold and gitea's handed credential. The
node composes and answers through mesh-sdk's frombus, so the fix is the pin.

- The quick fix: follow one more set of words in the answer. It moves the error again.
- The thorough fix, chosen (mesh-sdk fix/1126-held-as-runs): a held module composes only as what runs of it; a consumer
  or collector composes against that and is never held for the held one's stub, and a stub's handed credentials and own
  secrets are left out as they run rather than refused.
2026-10-29 00:50:01 +01:00
jschoubben 9bd1c84e0f Pin mesh-sdk 76f7c4d, the seed fix's own head: the checker refuses a pin to main's merge commit d4c61ee (same tree) as on no pull request (issue 774)
mesh/delivery-group group feat/774-vault-mints-l2 checking: the checker's verdict is out of date: novox/mesh-delivery@64d364a7686c was judged on trunk fa075dc8…
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 9bd1c84e0f4c on fb8ea0e6838c | novox/mesh-host#168 in group feat/774-vault-mints-l2
2026-10-29 00:20:01 +01:00
jschoubben 1f3fb52470 Merge main into L3: pin mesh-sdk main d4c61ee, which holds L3's sdk and the seed's held-as-today fixes (issue 774)
mesh/delivery-group group feat/774-vault-mints-l2 rejected: composed together: the checker's verdict at the members' heads (2199dae4e7d8): ace
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: ace: the check of novox/mesh-host#168 failed: the pin of git.novox.be/novox/mesh-sdk/go at commit d4c61ee8ffd0 is on no pull reques…
2026-10-29 00:11:11 +01:00
mesh-tools bb6e472939 Merge pull request 'A node answers its seed with an old-form build, a failed or held module, and those held behind it, as they run today (issue 1126)' (#170) from fix/1126-seed-oldbuild into main 2026-10-28 23:05:08 +00:00
jschoubben 93f33e2305 Pin mesh-sdk 76f7c4d: a held module keeps its items (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-host@bb6e47293909
mesh/delivery-group group fix/1126-seed-oldbuild merged: every member is merged
mesh/check pass 93f33e2305de on d2a868f99f05 | unit 3018 run, 270 kept, 36 skipped | integration 3039 run, 302 kept, 29 skipped | over bound: unit (sl…
2026-10-28 23:56:51 +01:00
jschoubben abdf85aa4f Node agent: say how it keeps each module today, held, failed or at a build of the old form named by its have's build; pin mesh-sdk f609be7 (issue 1126) 2026-10-28 23:50:19 +01:00
jschoubben 91686fdb81 Node agent: count a module running a build of the old form as held today, so the seed keeps it as the controller does (issue 1126) 2026-10-28 23:45:28 +01:00
mesh-tools d2a868f99f Merge pull request 'Node agent: say what it holds today in its answer, so a seed holding the same is the same (issue 1126)' (#169) from fix/1126-seed-held into main 2026-10-28 22:20:59 +00:00
jschoubben a143678fd8 Pin mesh-sdk 77b48a4, the group's head (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-host@d2a868f99f05
mesh/delivery-group group fix/1126-seed-held merged: every member is merged
mesh/check pass a143678fd898 on 4f6b389a37a1 | unit 1199 run, 1015 kept, 33 skipped | integration 1218 run, 1081 kept, 26 skipped | over bound: unit (…
2026-10-28 22:41:14 +01:00
jschoubben 9914459830 Pin mesh-sdk 06b13ec, the group's head (issue 1126)
mesh/delivery-group group fix/1126-seed-held checking: 2 of 3 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 9914459830d7 on 4f6b389a37a1 | novox/mesh-host#169 in group fix/1126-seed-held
2026-10-28 22:18:26 +01:00
jschoubben b7159d71a6 Node agent: say what it holds today in its answer, so a seed that holds the same is the same; pin mesh-sdk b03bf4c (issue 1126) 2026-10-28 22:15:24 +01:00
jschoubben e0c7f36ca9 Pin mesh-sdk 8eafc0c, the L3 group's head on mesh-sdk main with #120 and #121 (issue 774)
mesh/delivery-group group feat/774-vault-mints-l2 checking: the checker's verdict is out of date: novox/mesh-host@e0c7f36ca90c was judged on trunk d2a868f99f05…
mesh/delivery closed: a newer head of the same pull request
mesh/check checking e0c7f36ca90c on bb6e47293909 | novox/mesh-host#168 in group feat/774-vault-mints-l2
2026-10-28 21:56:06 +01:00
jschoubben 9120f94aa4 Name the seat mesh-placement where the kept declaration's retirement reads the placement, as L2 moved it (issue 774, ADR 0567) 2026-10-28 20:36:41 +01:00
jschoubben 1900b40192 Pin mesh-sdk 4e6e5cc, the L3 group's head rebased on L2 (issue 774) 2026-10-28 20:36:41 +01:00
jschoubben 6cbdb3712e Judgements and refused setuid searches, which any caller may ask, roll in one bounded key of the node's and add no key to the bucket every node shares; a key per act is kept for hand-overs, access acts and searches started (issue 774, ADR 0563 §2; rev-l3 round 2) 2026-10-28 20:35:33 +01:00
jschoubben 2a0c428ef9 The last setuid search's start and each library's own take ask are keys of their own, which no number of other acts moves (issue 774, ADR 0563 §2, §4; rev-l3 round 1) 2026-10-28 20:35:33 +01:00
jschoubben a5d1f7dde4 From the lease write a directory used as found names the node seat's hand-over as its way out, following the lease the engine hears (issue 774, ADR 0563 §5; rev-l3 round 1) 2026-10-28 20:35:33 +01:00
jschoubben 95dd64c774 Every signed ask is verified with the controller's key the engine trusts at that ask, so after the lease write none signed by it is taken (issue 774, ADR 0563 §7, ADR 0550 §2; rev-l3 round 1) 2026-10-28 20:35:33 +01:00
jschoubben 81fa009bf4 The node seat's acts and the engine's own asks are the node-engine's declared state acts on the bus: one key each under its own node's keys, written by compare-and-set, one ask open per path made by a create; acts reads one by its id (issue 774, ADR 0563 §2, §3) 2026-10-28 20:35:33 +01:00
jschoubben 05ff90e3c1 The engine judges its node seat's hand-over and take as it judges the controller's ask, answers root-free for its own machine, and asks a library's take itself once (issue 774, ADR 0563 §3, §4, §6) 2026-10-28 20:35:33 +01:00
jschoubben aaffc19232 The node seat node-engine served on the engine's link: its verbs with their proven caller, its acts kept in its own state, a hand-over or access act asked of the operator as node-engine-<node> and performed only on the warrant (issue 774, ADR 0563 §1-§3) 2026-10-28 20:35:33 +01:00
jschoubben 479ac71050 The node trusts the vault's key beside the controller's until the lease is handed away, says what it trusts in its record, asks the vault for its own certificate, composes the vault's authority beside the controller's, and reads the kept declaration no more once every placed module has a have (issue 774, ADR 0550 §2) 2026-10-28 20:35:33 +01:00
mesh-tools 4f6b389a37 Merge pull request 'Node agent: compose from the seat mesh-placement and answer proposals (issue 1126, delivery 2 of 2)' (#167) from feat/1126-placement-seat into main 2026-10-28 18:33:12 +00:00
jschoubben 45f5e27526 Pin mesh-sdk to the group's head 38dbe1d (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-host@4f6b389a37a1
mesh/delivery-group group feat/1126-placement-seat merged: every member is merged
mesh/check failed: integration: meshbuildgo/cmd/mesh-build-go.TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus: the compose ask mvsg85b34d25b4 never ende…
2026-10-28 18:50:39 +01:00
jschoubben dd8ba97989 Node agent: wait for the bus's word that the seat's placement and candidates states were created, then watch them (issue 1126) 2026-10-28 18:43:49 +01:00
jschoubben ab22efd8b1 Node agent: follow the placement key at start only and say why when it cannot; test the reached candidate and the switched rule; word the log by the seat's state (issue 1126) 2026-10-28 18:43:49 +01:00
jschoubben 54aee345fc Node agent: a node that never composed its own takes a failed placement read as not placed; one that did keeps waiting (issue 1126) 2026-10-28 18:43:49 +01:00
jschoubben 8639b373d2 Prove a missing or refused bucket is a watch error, so a node waits on the controller's compositions and tries again (issue 1126) 2026-10-28 18:43:49 +01:00
jschoubben 5fc69444f6 Node agent integration tests: candidates keyed by proposal, under the node's real grant on the seat's states (issue 1126) 2026-10-28 18:43:49 +01:00
jschoubben 2fe9d678fc Node agent: answer a proposal's candidate keys and follow the seat's placement, not the indexer's groups (issue 1126)
The answer names its proposal and the revision of the node's own candidate key; a node with no key in a proposal is not asked. The watch for its own placement key is tried again until the seat's state exists, so the node switches when its key is written.
2026-10-28 18:43:49 +01:00
mesh-tools 58b4f8b6bc Merge pull request 'Reconcile a module from the machine's facts at start and on a down edge (issue 1957, ADR 0568)' (#166) from fix/1957-boot-reconcile into main 2026-10-28 17:41:54 +00:00
jschoubben bf667a602d Reconcile a module from the machine's facts at engine start and at each edge of a declared-running resource going down (issue 1957, ADR 0568)
mesh/delivery merged: source.main moved to novox/mesh-host@58b4f8b6bc07
mesh/check being carried to bf667a602d21 on 58b4f8b6bc07 | from 9a8b3870bad0
A reboot left the have running, so a unit that failed to start at boot was never started again and nothing said so. Baseline now asks the machine; the judge's look calls Agent.Reconcile once per down edge, leaving held and unwanted modules alone.
2026-10-28 18:12:35 +01:00
mesh-tools 9a8b3870ba Merge pull request 'The node-engine applies every module but the held ones and marks only those (group fix/1126-apply-culprit, issue 1126)' (#165) from fix/1126-apply-culprit into main 2026-10-28 16:34:48 +00:00
jschoubben 24c23a1d06 A module the node holds alone marks that module, keeps what runs of it, and never refuses the node (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-host@9a8b3870bad0
mesh/delivery-group group fix/1126-apply-culprit merged: every member is merged
mesh/check pass 24c23a1d066f on a548a61e86d7 | unit 2667 run, 590 kept, 36 skipped | integration 2676 run, 633 kept, 29 skipped | over bound: unit (sl…
frombus.Compose now holds a module refused alone (mesh-sdk 0de3006, the rule
its answer follows too): composed as what runs of it, with what depends on
how it is held held too. The composer hands every other module, keeps what it
handed of a held one, and tells the agent which modules it holds and why;
agent.NotComposedEach marks those alone and clears the rest. Only a refusal
of no single module still marks every module (NotComposed).

heldOut also holds out, of the node's own composition as it runs, the
resources of each module held alone, until a composition of its own is
handed: the node's own apply never removes a module leaving the fold with its
names unset (review of the apply, round 1; issue 1945 by a new path).

Pins mesh-sdk fix/1126-apply-culprit at 0de3006 (a group).
2026-10-28 17:23:26 +01:00
mesh-tools a548a61e86 Merge pull request 'The node-engine reloads the service manager when a unit file or drop-in it writes changes (group fix/shutdown-bound, issue 1126)' (#164) from fix/shutdown-bound into main 2026-10-28 15:05:10 +00:00
jschoubben bc28606d45 apply: reload units only for a unit file actually written or removed (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-host@a548a61e86d7
mesh/delivery-group group fix/shutdown-bound merged: every member is merged
mesh/check pass bc28606d4566 on 690aa662a837 | unit 2192 run, 3794 kept, 57 skipped | integration 2208 run, 3829 kept, 57 skipped | over bound: unit, …
A skipped or held file comes back on every apply; counting it would reload the
service manager on every apply of a node with a gated or held module. The same
allowlist as MovesRoot: created, updated, corrected, removed.
2026-10-28 15:49:34 +01:00
jschoubben 7f0eb9c51d apply: reload the service manager once when a file in a unit directory moved (issue 1126)
A drop-in that no service reflects, like the power module's bound on
reboot.target, was written but never loaded: the node-engine reloaded units only
for a file named in a service's restart-on, so the next reboot still ran the
target as loaded. Any file written or removed under a system unit directory, or
a drop-in directory below one, now gets one daemon-reload per apply.
2026-10-28 15:43:35 +01:00
mesh-tools 690aa662a8 Merge pull request 'The node's shadow says which input woke each answer (issue 1126)' (#163) from fix/1126-vault-own into main 2026-10-28 14:20:39 +00:00
jschoubben 8bd0b5db15 Say which input woke each placement candidate's answer, so a probe's safety check tells a loop from moving inputs (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-host@690aa662a837
mesh/delivery-group group fix/1126-vault-own merged: every member is merged
mesh/check pass 8bd0b5db15a3 on b4fd2cc210bf | unit 1639 run, 4201 kept, 46 skipped | integration 1674 run, 4215 kept, 47 skipped | over bound: unit, …
Each answer's line now reads 'answering placement candidate G (revision N; woken by X)', X being its first answer,
a retry after a read the bus did not answer, the candidate, or the inputs that changed (placement, records, modules,
values, haves, wants). The single input counter becomes one per input, which also decides whether a group is answered
again, as before.

Quick fix weighed: a counter word in the line. Thorough fix chosen: one count per input, so the line names exactly what
moved since the group's last answer.
2026-10-28 14:44:27 +01:00
mesh-tools b4fd2cc210 Merge pull request 'A one-shot at rest satisfies a stopped declaration, so the kernel's boot cleanup applies (issue 1126)' (#162) from fix/ace-tun-cleanup into main 2026-10-28 13:34:20 +00:00
jschoubben 1a021052fe Drop the never-ran reading: a start timestamp of 0 does not mean a one-shot never ran (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-host@b4fd2cc210bf
mesh/check being carried to 1a021052fe07 on b4fd2cc210bf | from 7a2af2376405
Review round 1 measured it on systemd 262.1: a finished one-shot nothing references is
unloaded, and loaded again it reads inactive, ExecMainStatus=0 and a start timestamp of
0, as one that never ran does. The previous commit's rule read such a unit stopped, so
a one-shot declared running would be started, read stopped and fail on every apply, and
the hold, the take-over and the giving back would judge it wrongly.

ServiceState keeps main's reading. ServiceAtRest and declaredState stay: they alone
satisfy a stopped declaration for a one-shot at rest, which is the kernel cleanup's
case. A one-shot declared running that must be seen to have run lingers
(RemainAfterExit) or is enabled at boot. Unit tests: the arch table has 5 rows, the
apply test 1; 1262 in all.
2026-10-28 14:29:16 +01:00
jschoubben cf738cdb90 Read a one-shot that never ran as stopped, and one at rest as stopped to a stopped declaration (issue 1126)
The kernel module's boot-time cleanup, linux-modules-cleanup.service (Type=oneshot,
declared stopped and enabled at boot), failed its delivery on ace with 'was asked to
be stopped and is running': ServiceState read every inactive one-shot with
ExecMainStatus=0 as running, and systemd says 0 for one that never ran too. Stopping
an inactive unit changes nothing, so the read back said running again; after a boot
that ran it, the same declaration would fail every apply.

ServiceState now reads ExecMainStartTimestampMonotonic: 0 means never ran, read as
stopped (a package's one-shot declared running is then started, and the hold no
longer counts it as in use). The service apply reads a one-shot at rest as stopped
to a declaration that it be stopped (ServiceAtRest), whatever its last run did.

Quick fix weighed: declare the cleanup running in the catalogue. Thorough chosen:
the reading was wrong for every one-shot, and this is its second fault of the kind
(the firewall's was the first). Unit tests: three arch one-shot tests merged into one
table (7 rows), one apply test (2 rows); 1263 to 1262.
2026-10-28 14:29:16 +01:00
mesh-tools 7a2af23764 Merge pull request 'Pin mesh-sdk f387581: the node composes refusals per module and folded modules' owned paths as the controller does (issue 1126, group fix/1126-seed-8)' (#161) from fix/1126-seed-8 into main 2026-10-28 13:27:09 +00:00
jschoubben daaa1962bd Pin mesh-sdk f387581, the group's head: a module's own refusal refuses that module, contributions in config words, folded modules own their manifest's paths (issue 1126, seed-8)
mesh/delivery merged: source.main moved to novox/mesh-host@7a2af2376405
mesh/delivery-group group fix/1126-seed-8 merged: every member is merged
mesh/check pass daaa1962bdab on df878ba1f780 | unit 3018 run, 234 kept, 36 skipped | integration 3038 run, 266 kept, 29 skipped | over bound: unit (sl…
2026-10-28 14:08:58 +01:00
mesh-tools df878ba1f7 Merge pull request 'The node answers a placement candidate once, retries a read the bus did not answer, and keeps a file's secrets entry by entry (issue 1126; group fix/1126-seed-same-3)' (#160) from fix/1126-seed-same-3 into main 2026-10-28 11:40:38 +00:00
jschoubben 1e01868d92 An unplaced node owes no pass, so no timer is armed again; every wait a test sets is put back when it ends; pin mesh-sdk 8520a4c, whose masking splits only a file's secrets (issue 1126; review of seed-same-3, round 1)
mesh/delivery merged: source.main moved to novox/mesh-host@df878ba1f780
mesh/delivery-group group fix/1126-seed-same-3 merged: every member is merged
mesh/check pass 1e01868d92c3 on c3a2ffde6755 | carried from mesh-catalog 774fcdfa93ad | unit 2188 run, 1064 kept, 36 skipped | integration 2197 run, 1…
2026-10-28 12:25:32 +01:00
jschoubben 9d04ae7edd Hold the secrets row to the have read's error, as the loop fix's Env.Held returns it (issue 1126) 2026-10-28 12:13:09 +01:00
jschoubben 16df7d0b7a Merge branch 'fix/1126-answer-once' into fix/1126-seed-same-3 2026-10-28 12:11:50 +01:00
jschoubben cd1d2ff447 A read the bus did not answer is tried again after a bound, never kept as an answer: the shadow's answer, the node's composition and the own-secret step (issue 1126)
Met twice on 2026-10-28: shanks kept 'want.shanks.brother could not be read ... no
responders' as the own-secret step's last word, and ace kept 'the vault's own: not yet
... no responders' as its answer to seed-7, each until another input moved. Every bus
read of the composer (manifests, builds, archives, served values, the vault's own and
credentials, the have of what runs) is noted (BusFailed); an answer or a pass that met
one is made again after a growing bound (5 s to 5 min), and a pass is not handed when
the have of its sealed values did not answer. The own-secret step tells a read or an
ask that did not answer (Retry, taken again) from a key absent or the vault's word.
2026-10-28 12:11:08 +01:00
jschoubben faf6a23d91 Fill go.sum for mesh-sdk 70de942 (issue 1126) 2026-10-28 12:06:25 +01:00
jschoubben 2f4e81f3b0 A file's secrets are kept entry by entry: the box that runs at one secret is reused only there, where it opens to the same value; pin mesh-sdk 70de942 (issue 1126, C1; the seed-7 probe) 2026-10-28 12:05:48 +01:00
jschoubben 4182f8fbfb The node answers a placement candidate revision once: a module's have whose health words alone moved, or a key written again with its value, no longer composes every group's answer again (issue 1126)
On shanks probe/1126-seed-7 was answered at revision 84 every 13-17 s: each look rewrote
the haves of its unhealthy modules with new words, and the shadow took any write of its
own module haves, placement, modules or config values as a change. Each answer loaded
the bus, the health tools timed out, the words moved again. Now only what an answer is
composed from wakes it (what runs, folded modules, a changed value), and each group is
answered again only when its keys' revisions or the node's inputs moved.
2026-10-28 12:02:57 +01:00
mesh-tools c3a2ffde67 Merge pull request 'The placement test's resolved values carry their source, as mesh-config writes them (issue 1126)' (#159) from fix/1126-seed-same-2 into main 2026-10-28 10:08:46 +00:00
jschoubben 6502ba6ad6 The placement test's resolved values carry their source, as mesh-config writes them: a value with no source is laid by neither the controller nor a node composing its own (issue 1126, C2; the group check of 3cb7f74b347d)
mesh/delivery merged: source.main moved to novox/mesh-host@c3a2ffde6755
mesh/check being carried to 6502ba6ad6e6 on c3a2ffde6755 | from f6c0520f26b9
2026-10-28 10:47:48 +01:00
mesh-tools f6c0520f26 Merge pull request 'Node shadow: reuse a seal only at its slot, lay values as the controller does, newest candidate first, and hand the vault a drifted own secret (issue 1126, group fix/1126-seed-same)' (#158) from fix/1126-seed-same into main 2026-10-28 09:46:47 +00:00
jschoubben c912bcab6e The node agent finds a drifted own secret's running file by its id as the node runs it, its module's name first (issue 1126, cause C4; review round 2)
mesh/delivery merged: source.main moved to novox/mesh-host@f6c0520f26b9
mesh/delivery-group group fix/1126-seed-same merged: every member is merged
mesh/check checking c912bcab6e73 on f6c0520f26b9 | novox/mesh-host#158 in group fix/1126-seed-same
Pins mesh-sdk 3028c7e (delivery.OwnNeedName). The fixtures carry the id shape captured from novox's shadow answer.
2026-10-28 10:01:38 +01:00
jschoubben 63c0b445d7 Fill go.sum for mesh-sdk 1062b33, the group's head (issue 1126) 2026-10-28 09:53:34 +01:00
jschoubben 653ff61662 The node agent hands the vault what runs of an own secret its shadow answer says another value of, through move-in's replace_running, while it applies the controller's compositions (issue 1126, cause C4)
A new step, internal/ownresync, after each shadow answer: for each own secret masked as another value, while the node
applies the controller's compositions and runs the one its want names, it asks the vault's move-in with
replace_running: true and the running value sealed to the call's key, once per value, and says what it asked and what
the vault answered. The answer written to the node's have key carries values_differ. Pins the group's mesh-sdk head
1062b33 (go.sum is filled once that head is pushed).
2026-10-28 09:51:48 +01:00
jschoubben a7cd0a7807 Node shadow: reuse a running seal only at its own slot, lay values as the controller does, answer the newest candidate first and say how it goes (issue 1126, C1 C2, T1 T2 T3)
A box is reused by slot (resource, field) after the composition, from what runs or the
have's slots; a have written before slots is reused within its module alone. Config
values are laid by mesh-sdk's one rule (never a default). The shadow answers the
newest candidate first and hands each answer as it lands, says when an answer starts,
every minute what it waits on, and when it ends, and fits an answer to the node's key.
Needs mesh-sdk fix/1126-seed-same (16b8d8a1).
2026-10-28 09:41:29 +01:00
mesh-tools 7271cf151b Merge pull request 'Node shadow: what runs is the composition at the have's digest, not the stale kept copy (issue 1126, group fix/1126-shadow-bodies)' (#157) from fix/1126-shadow-bodies into main 2026-10-28 07:46:33 +00:00
jschoubben 82d0e125b2 What runs is the composition the have names, never the copy of the declaration kept before the switch (issue 1126, probe #532: shanks' xterm, zsh and its plugins answered differing at the very digest their haves say runs)
mesh/delivery merged: source.main moved to novox/mesh-host@7271cf151b3a
mesh/delivery-group group fix/1126-shadow-bodies merged: every member is merged
mesh/check checking 82d0e125b2c0 on 7271cf151b3a | novox/mesh-host#157 in group fix/1126-shadow-bodies
2026-10-28 01:42:17 +01:00
jschoubben a4c24a1979 Pin mesh-sdk 362a3f1: a refusal names what another node cannot run here, and is answered without a composition per module (issue 1126, probe #532) 2026-10-28 01:40:18 +01:00
mesh-tools cff69893a3 Merge pull request 'Node shadow: decides 'same value' by opening seals in memory, says when it is not ready or slow, carries the bus membership (issue 1126, group fix/1126-shadow-shows)' (#156) from fix/1126-shadow-shows into main 2026-10-28 00:21:51 +00:00
jschoubben 047237c911 The shadow shows its work: differing resources with masked bodies and whether a sealed value is the same, a minute not ready or a slow answer said in the journal and the answer; pin mesh-sdk 4e0c577 (issue 1126, seed #531)
mesh/delivery merged: source.main moved to novox/mesh-host@cff69893a361
mesh/delivery-group group fix/1126-shadow-shows merged: every member is merged
mesh/check pass 047237c911a4 on 449cda1b003e | unit 2980 run, 270 kept, 36 skipped | integration 2999 run, 302 kept, 29 skipped | over bound: unit (sl…
ace, g14 and shanks answered nothing to seed #531 and said nothing either. The
shadow now says, after a minute not ready, which inputs it has not read, and
answers each group it knows refused for that; an answer running over a minute
is said while it runs and noted in the answer. The answer's differing resources
carry both bodies, masked, the node saying whether two sealed values open to the
same value (Composer.sameValue: only the comparison leaves the node).
2026-10-28 01:11:55 +01:00
mesh-tools 449cda1b00 Merge pull request 'Node agent: a switch never marks the new run's modules, a failed read or a bus pause keeps what runs, stale composer marks clear (issue 1126, group fix/1126-switch-safe)' (#155) from fix/1126-switch-safe into main 2026-10-27 23:52:28 +00:00
jschoubben a6e53a4858 A composer's mark never covers another cause, and a run that ended marks and clears nothing (issue 1126, review of the switch, round 1)
mesh/delivery merged: source.main moved to novox/mesh-host@449cda1b003e
mesh/delivery-group group fix/1126-switch-safe merged: every member is merged
mesh/check pass a6e53a485813 on 199f6b48b1df | unit 2992 run, 1008 kept, 36 skipped | integration 3025 run, 1102 kept, 29 skipped | over bound: unit (…
NotComposed marked from a read have whatever its phase: an apply that failed,
naming the want's build and digest, became a composer's mark, which baseline
then took for what runs, and never applied again. It now marks only a have that
runs, says nothing, or is a composer's mark already. And its loop, whose reads
can take seconds, checks the run's context at each module, so a run that ended
writes no have.
2026-10-28 00:41:25 +01:00
jschoubben 4993f1b1d7 Make the switch between composers safe: a composer tells only its own run's agent, a pass whose run ended refuses nothing, a bus that pauses is tried again, a mark keeps what runs and clears at the next start; pin mesh-sdk 7359d18 (issue 1126)
Live 2026-10-28: the own run's last pass, failing on its run's end, marked
every module of the controller-mode agent that replaced it (ace 60, shanks 63),
and the marks dropped the build and digest because the have read failed in the
bus's reload pause. Now: the refusal hook is bound to the run's agent and ctx;
a pass whose context ended returns nothing; JetStream not enabled or no
responders are tried again after 1, 2 and 4 s before any mark; NotComposed reads
the have again while the bus does not answer and marks nothing it cannot read;
baseline takes a have a composer marked as what runs, so a mark it did not make
clears; each run's shadow lives with the run.
2026-10-28 00:34:10 +01:00
mesh-tools 199f6b48b1 Merge pull request 'Node agent: a failed placement read picks no owner, a deleted placement returns the node to the controller without a restart (issue 1126, group fix/1126-shadow-all)' (#154) from fix/1126-shadow-all into main 2026-10-27 22:44:12 +00:00
jschoubben 4636261435 A failing placement read chooses neither composer, a deleting head is compared with the controller's compositions at the node's wants, and a placement watch that ends is said; pin mesh-sdk e32d777 (issue 1126, review of the shadow, round 1)
mesh/delivery merged: source.main moved to novox/mesh-host@199f6b48b1df
mesh/delivery-group group fix/1126-shadow-all merged: every member is merged
mesh/check checking 463626143567 on bb15aab50eda | novox/mesh-host#154 in group fix/1126-shadow-all
selfComposer took a KVHistory error for 'not placed': at any restart a timeout
or a bus not ready flipped a self-composing node to the controller's
compositions for a whole run. selfcompose.Placed chooses only on a read that
succeeds (a missing placement state is one), and tries again otherwise, said
once. The shadow's answer to a head deleting the placement reads the
controller's composition at each want in the node's bucket and compares it with
what runs; a want with none refuses, without returning.
2026-10-27 23:37:59 +01:00
jschoubben 4e8fdfdf04 The way back: a node whose placement key is deleted returns to the controller's compositions without a restart, says so in its journal and its have, and answers such a candidate as returning; pin mesh-sdk 472d5f3 (issue 1126, the seed's revert)
The seed's revert (mesh-catalog #526) was refused on 'would stop running node',
while a self-composing node would have refused its placement's absence, kept
what runs and marked every have failed until a restart. endWhenUnplaced ends the
agent's run when the key goes, so it begins again on the controller's
compositions, as at a start without a key; the composer refuses nothing while
unplaced; the node's have says whose compositions it applies (Composes); the
shadow answers a deleted placement Returns, verdict same.
2026-10-27 23:25:59 +01:00
jschoubben d00926e089 Compose with what the node runs, so a folded module of an old-form build is carried as it runs, and bound an archive read; pin mesh-sdk 2fd5ffd (issue 1126, the seed's shadow)
novox refused its own composition and its answer on de-spiegel, a module it runs
folded into its own composition at a build only the controller holds. The
composer now gives frombus what the node runs (Scope.Running), so such a module
is carried as it runs. The shadow's answer names every refusal (mesh-sdk). A
templates archive read is bounded at 30 s: nothing else held an unanswered store.
2026-10-27 23:19:40 +01:00
mesh-tools bb15aab50e Merge pull request 'Node agent: compose its own modules from their builds and providers' served keys under its own grant (issue 1126, group fix/1126-node-creds)' (#153) from fix/1126-node-creds into main 2026-10-27 20:32:37 +00:00
jschoubben 6a4718b712 Hold in a unit table which haves count a module as run inside the node's own composition, so a failed have naming nothing is caught without H8's timing (issue 1126, review round 2)
mesh/delivery merged: source.main moved to novox/mesh-host@bb15aab50eda
mesh/delivery-group group fix/1126-node-creds merged: every member is merged
mesh/check checking 6a4718b712b2 on bb15aab50eda | novox/mesh-host#153 in group fix/1126-node-creds
2026-10-27 21:29:51 +01:00
jschoubben d5adf9b72b Drop two debugging lines from the build read and H8, and say why the refused read's wait stays above the history bound (issue 1126, review round 1) 2026-10-27 21:21:19 +01:00
jschoubben 7365890298 Prove in H8 that a node reads a placed module's build and a provider's served key through its own session, refused without the grant, and take a failed have naming nothing as no module run inside the node's own; pin mesh-sdk 7a8e9f9 (issue 1126, review round 1)
H8 found it: a module the node could not compose yet got a failed have naming
no fingerprint, which counted as run inside the node's own composition, so the
next composition took it from its manifest past its build.
2026-10-27 21:18:45 +01:00
jschoubben 0862f97972 Pin mesh-sdk 538ffba, the group's head (issue 1126) 2026-10-27 21:04:02 +01:00
jschoubben 45439fd742 Compose each module from its build and read what providers elsewhere serve, pinning mesh-sdk cebc651, so a self-composing node hands its moved modules and settles mailu's domain (issue 1126)
The node read a behind module's source manifest digest and composed every module
from the indexer's raw manifest; frombus now takes the build itself (Scope.KeptAt,
a fingerprint). The reader gives frombus the builder's key, a templates archive
fetched as every archive is (apply.Fetch) once per digest and checked against it,
and another node's served keys one by one, as the node's grant allows them.
2026-10-27 21:01:16 +01:00
jschoubben 0184e2a5b8 Watch the have bucket only where the node's grant allows, so a node composing itself is not refused by the bus (issue 1126)
The fold's watch of every have key ('>') was refused (permissions violation on
CONSUMER.CREATE KV_node-engine_have), and every self-composing agent stopped
after the seed. The composer now watches '*.node' (records) and '<node>.>' (its
own haves), as buscompose grants a node; the shadow watches '<node>.>' too. The
selfcompose tests check every watch and get against PermissionsFor(KindNode),
and the agent's integration tests run the agent as the node under that grant.
2026-10-27 09:27:54 +01:00
mesh-tools 50c229f905 Merge pull request 'The node agent folds what it runs at no new-form build into its own composition, judged from its haves, so the first composition after a restart keeps the bus (issue 1126)' (#152) from fix/1126-fold into main 2026-10-27 08:09:06 +00:00
jschoubben 69b845c0f7 Read what a node runs inside its own composition from its haves, not from what the agent applied, so a restarted agent's first composition keeps the bus (issue 1126, review of the fold round 1)
mesh/delivery merged: source.main moved to novox/mesh-host@50c229f90568
mesh/delivery-group group fix/1126-fold merged: every member is merged
mesh/check checking 69b845c0f7b2 on 0d22776b20ef | novox/mesh-host#152 in group fix/1126-fold
A fresh agent composes before it has applied anything (Running empty), so
InOwn of Running folded only mesh-wireguard and the switch would drop nats.
The composer now watches the node's haves: a placed module whose have names no
new-form fingerprint is folded; one with a new-form have whose want is removed
is not folded. The shadow uses the same set, over the candidate's placement.
2026-10-27 09:03:59 +01:00
jschoubben caafd9306e Keep what the node runs inside its own composition there when it composes itself, so the seed's answer and the node's own composition keep the tunnel and the bus (issue 1126)
Pins mesh-sdk 05b67831, whose compositions fold a module no new-form build
names into the node's own. The shadow composed a placed module it runs inside
its own composition (nats, photos) at the trunk's build, as if the candidate
added it; shadowBuilds now leaves those to the fold, and both paths pass
frombus.InOwn of what runs.
2026-10-27 08:51:41 +01:00
mesh-tools 0d22776b20 Merge pull request 'The node agent answers a placement candidate in shadow, applies nothing, and keeps its own sealed values in its have (issue 1126, ADR 0559 §4)' (#151) from feat/1126-l2b into main 2026-10-27 07:21:19 +00:00
jschoubben 1803d25c3a Watch afresh and read from nothing at each shadow run, so a candidate deleted while it was down is not answered and no earlier watch stays open (issue 1126, review of L2b round 2)
mesh/delivery merged: source.main moved to novox/mesh-host@0d22776b20ef
mesh/delivery-group group feat/1126-l2b merged: every member is merged
mesh/check checking 1803d25c3a67 on 0d22776b20ef | novox/mesh-host#151 in group feat/1126-l2b
2026-10-27 02:48:54 +01:00
jschoubben d56c576e9a Begin the shadow composer again with a backoff when an input cannot be watched yet, and pin mesh-sdk's fixed head, so a node answers candidates for the whole link (issue 1126, review of L2b round 1) 2026-10-27 02:44:39 +01:00
jschoubben fbc6d6330f Extend H8 with a placement candidate the node answers same and applies nothing for, and pin mesh-sdk's group head (issue 1126) 2026-10-27 02:29:17 +01:00
jschoubben fb0cdabe25 Answer every placement candidate in shadow in the node's own key, applying nothing, so a placement change is checked by the nodes it places before it merges (issue 1126, ADR 0559 §4) 2026-10-27 02:26:07 +01:00
jschoubben 0474a2a3cb Keep the node's own composition's sealed values in its key and read a policy within the agent's context, so a restart applies nothing again (issue 1126) 2026-10-27 02:21:36 +01:00
mesh-tools 180a7e7b80 Merge pull request 'A directory used as found is not a module's health, and a failed have names its cause (issue 1496, ADR 0557 §4, §5)' (#150) from fix/1496-start-health into main 2026-10-26 21:38:02 +00:00
mesh-tools be94d2689e Merge pull request 'The node composes its own modules once the catalogue places it, and writes its record (issue 1126)' (#149) from feat/1126-nodes-compose into main 2026-10-26 21:11:36 +00:00
jschoubben b225c7bb4c A directory used as found is not a module's health, and a failed have says why (ADR 0557 §3, issue 1496)
mesh/delivery merged: source.main moved to novox/mesh-host@180a7e7b80e2
mesh/delivery-group group fix/1496-start-health merged: every member is merged
mesh/check checking b225c7bb4cf5 on be94d2689e2f | novox/mesh-host#150 in group fix/1496-start-health
On ace, 2026-10-26 21:19, mesh-build-node's cache directory was there before the
mesh and used as found. The health statement says such a directory unhealthy, and
the have folded it into the module's health as failed, with no words, while the
module's own check answered healthy; the assigner read it as its canary failing
and held the version on all four nodes.

The directory waits for the operator's hand-over and keeps its own condition,
raised from the same statement; it no longer counts in the module's have. A have
whose health is failed or waits and whose apply said nothing carries the words of
the resource that made it so, so the next one is readable.
2026-10-26 22:06:32 +01:00
jschoubben 1d44288514 Pin mesh-sdk 9796444 (l2-next: feat/1126-nodes-compose merged with sdk main #101, the standing-test race fixes) (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-host@be94d2689e2f
mesh/delivery-group group feat/1126-nodes-compose merged: every member is merged
mesh/check checking 1d442885148a on be94d2689e2f | novox/mesh-host#149 in group feat/1126-nodes-compose
2026-10-26 22:05:41 +01:00
jschoubben af2d2a2c14 Pin mesh-sdk 5f45175 (l2-next: feat/1126-nodes-compose merged with sdk main) after merging main (issue 1126) 2026-10-26 21:59:21 +01:00
jschoubben 59d77dd457 Pin mesh-sdk feat/1126-nodes-compose 03dce12, the group's head: the checker refuses a pin that is neither trunk nor the group's head (issue 1126)
mesh/delivery-group group feat/1126-nodes-compose rejected: composed together: the checker's verdict at the members' heads (0d06ff386dce): ace
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 59d77dd457da on ff9386dcccc2 | novox/mesh-host#149 in group feat/1126-nodes-compose
2026-10-26 20:35:52 +01:00
jschoubben 23d66187a8 Review of L2, round 3: a have written with no composition keeps what the module's last have said runs (fingerprint, digest, sealed values), for every writer; a record module run nothing of since the start judges its move against its last have and waits for a yes (issue 1126, ADR 0547 §3)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 23d66187a881 on ff9386dcccc2 | novox/mesh-host#149 in group feat/1126-nodes-compose
2026-10-26 20:29:07 +01:00
jschoubben 4476e58080 Review of L2, round 2: a refusal at the first compose after a start keeps what the module's have said runs (1); each have of a self-composing node carries the values its composition holds sealed, and the composer reuses them after a restart, so a rotated secret moves no composition at a restart (2); pin mesh-sdk dd9d9d5 (issue 1126, ADR 0547 §3)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 4476e5808047 on ff9386dcccc2 | novox/mesh-host#149 in group feat/1126-nodes-compose
2026-10-26 20:23:16 +01:00
jschoubben 58de6aa10a Review of L2: a node that cannot compose says so in each wanted module's have, failed, while it keeps what it runs (H2); the composer keeps the box the node runs when it opens to the vault's value, so neither the switch nor a restart moves a composition (H1); pin mesh-sdk 3968849, whose node asks the vault only for a module declaring a secret (H8's failure) (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: unit: ..test: ✔ a transient empty list for four passes retires nothing; five stable passes retire, data kept (290.408814ms) | 58de6…
2026-10-26 20:09:40 +01:00
jschoubben cdd3f3fd4f Pin mesh-sdk feat/1126-nodes-compose 9713c30: placement carries a settings-form module's values (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: ace: novox/mesh-catalog#505 does not merge into the trunk: files in conflict: .gitignore | cdd3f3fd4f63 on ff9386dcccc2 | unit 1245…
2026-10-26 19:40:41 +01:00
jschoubben 21d6583b5e The node agent begins again composing its own when the catalogue places its node, with no restart (issue 1126, ADR 0549 §4)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: unit: git.novox.be/novox/mesh-catalog/checks/nobinaries.TestTheCatalogueHoldsNoCompiledProgram: modules/mesh-config/cmd/mesh-config…
2026-10-26 19:34:24 +01:00
jschoubben bb46a6ad0f The node composes again when the vault says a secret changed, so a rotated secret or credential reaches it at once (issue 1126, ADR 0547 §2)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: integration: meshbuildgo/cmd/mesh-build-go.TestIntegrationLevelRunsAgainstARealStoreAndBusAndLeavesNothing: result = &{Level:integr…
2026-10-26 19:26:25 +01:00
jschoubben f8afaefba0 The node keeps the mesh's own certificate it applies, from its kept declaration at start, for its record; one agent test folded to hold the count (issue 1126, ADR 0555 §3)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: ace: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk 91413de33516, which is neither on mesh-sdk's trunk nor the grou…
2026-10-26 19:24:36 +01:00
jschoubben 1e38c42646 The node's composer asks the vault for the pair credentials its composition reads, as consumer and provider, sealed to its key (issue 1126, ADR 0550 §5)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: shanks: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk 91413de33516, which is neither on mesh-sdk's trunk nor the g…
2026-10-26 19:21:22 +01:00
jschoubben 1241ee7041 Pin mesh-sdk 4d5f430: the node's wants say it composes them (Self) (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: shanks: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk f1d7c7377dc4, which is neither on mesh-sdk's trunk nor the g…
2026-10-26 19:14:38 +01:00
jschoubben 2dc384d641 The node's composer asks the vault's own for its modules' secrets sealed to its key, and keeps a value first had while its generation stands; two agent tests folded to hold the count (issue 1126, ADR 0550 §5)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 2dc384d641a2 on ff9386dcccc2 | novox/mesh-host#149 in group feat/1126-nodes-compose
2026-10-26 19:05:12 +01:00
jschoubben e4dbb49d2f Integration: a node the placement names composes and applies its own module with no controller composition, and again on a changed config value (issue 1126, ADR 0547 §2)
mesh/delivery-group group feat/1126-nodes-compose checking: 4 of 5 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking e4dbb49d2fa5 on ff9386dcccc2 | novox/mesh-host#149 in group feat/1126-nodes-compose
2026-10-26 18:51:36 +01:00
jschoubben 02d9d7b053 The node composes its own modules once the catalogue places it: one composer per node drains a mark set by any input's change, the agent applies what it composed when it differs, and a record move waits for a yes (issue 1126, ADR 0547 §2, §3)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 02d9d7b0538f on ff9386dcccc2 | novox/mesh-host#149 in group feat/1126-nodes-compose
2026-10-26 18:44:58 +01:00
jschoubben 7fc0ea5478 Pin mesh-sdk feat/1126-nodes-compose 9789c83 (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 7fc0ea5478a3 on ff9386dcccc2 | novox/mesh-host#149 in group feat/1126-nodes-compose
2026-10-26 18:34:35 +01:00
jschoubben 8be1c2ab0d The node agent writes what the machine observes of itself as its record in its own have key, only when it changed (issue 1126, ADR 0555 §3) 2026-10-26 18:32:54 +01:00
mesh-tools ff9386dccc Merge pull request 'The node-engine runs yielding processes in one capped idle slice (issue 1806, ADR 0553/0554, change A)' (#148) from fix/build-cpu-yields into main 2026-10-26 14:22:54 +00:00
mesh-tools bba9081ca1 Merge pull request 'The node-engine moves a running daemon's build aside instead of deleting it (issue 734)' (#147) from fix/daemon-dir-kept into main 2026-10-26 14:16:00 +00:00
jschoubben 65cd7b4787 Retire a daemon's running build instead of deleting it before the restart (issue 734)
mesh/delivery merged: source.main moved to novox/mesh-host@bba9081ca127
mesh/delivery-group group fix/daemon-dir-kept merged: every member is merged
mesh/check pass 65cd7b47873a on bba9081ca127 | carried from 94f9aa298cb8, mesh-sdk 21252194d530 | unit 971 run, 423 kept, 32 skipped | integration 976…
A process no witness judges had its directory removed before the new bundle
was unpacked and the unit restarted, so the old process ran on in a deleted
working directory: mesh-build-go's `go version` probe failed with "getwd: no
such file or directory" and refused three checks on ace on 2026-10-26, each in
the second a new build of it arrived.

Every process is now replaced the way a judged one is: its build is renamed
under .witness/<name>/retired/ and deleted only once nothing runs from it.
"Runs from it" now counts a process's working directory as well as its image,
so the children a daemon starts keep the directory too. Forget keeps a retired
build that still drains, and the witness's sweep covers every process it keeps
a directory for, judged or not.

Two unit-file tests are folded into one, so the unit count goes down by one.
2026-10-26 15:01:55 +01:00
jschoubben a78239b2aa Pin mesh-sdk at fix/build-cpu-yields' head (ADR 0553, ADR 0554)
mesh/delivery merged: source.main moved to novox/mesh-host@ff9386dcccc2
mesh/delivery-group group fix/build-cpu-yields merged: every member is merged
mesh/check being carried to a78239b2aa05 on ff9386dcccc2 | from bba9081ca127, mesh-catalog 9a8e54d988fe
2026-10-26 14:47:02 +01:00
jschoubben 2ac0c4686a The node-engine holds the node's one build CPU share as its own config (ADR 0553 §3, round 2)
The share belongs to the part that enforces it: build-cpu-share, default 50,
set per node through the node's settings layer and read by the composer for
the build seats' runtimes. Pin mesh-sdk at fix/build-cpu-yields' head.
2026-10-26 14:40:24 +01:00
jschoubben 5233f7584d Pin mesh-sdk at fix/build-cpu-yields' head (ADR 0553) 2026-10-26 14:23:13 +01:00
jschoubben 8858dba25e Merge main into fix/build-cpu-yields 2026-10-26 14:22:43 +01:00
jschoubben 9254180cab A yielding process runs in the node's one yielding slice, which holds the shared quota (ADR 0553, issue 1806)
Review round 1: one quota per unit let three build modules take three shares.
The engine now writes meshyield.slice, top-level so its idle weight binds
against the operator's session, with the node's share as its CPUQuota; each
yielding unit joins it at nice 19. Comments say the IO controls bind only under
bfq or io.cost, not under the none scheduler NVMe disks use.
2026-10-26 14:21:24 +01:00
mesh-tools 94f9aa298c Merge pull request 'The node-engine writes the followed user list into the bus module's have even when it does not apply that module (issue 1945)' (#146) from fix/have-keeps-user-list into main 2026-10-26 12:52:15 +00:00
jschoubben ee9fcaba20 Pin mesh-sdk at fix/build-cpu-yields' head (issue 1806) 2026-10-26 13:49:41 +01:00
jschoubben 43f2d6020e Write the bus's user list into a have no apply here says, merged over the key's value
mesh/delivery merged: source.main moved to novox/mesh-host@94f9aa298cb8
mesh/check pass 43f2d6020e11 on 94f9aa298cb8 | carried from 7f04d1204765, mesh-catalog a4f5df797b68, mesh-sdk b4cd95e7c4a4 | unit 972 run, 423 kept, 3…
On the bus node the bus module has no want and has left the declaration, so no
pass of this engine held its fact and the follower's user list was kept in
memory and never written: the vault and the grants part read no hashes and no
key. A module's user list now reaches its key on every pass, merged into what
the key says with every other field kept; a missing key is said once, never
made up (ADR 0547 §8).
2026-10-26 13:38:40 +01:00
jschoubben 6250b1274b A process that yields is written at idle priority under its share of the machine (issue 1806)
A build seat's runtime took 1650% of shanks' 24 processors and slowed the
operator's desktop. The unit of a process declared with yields now carries
CPUWeight=idle, IOWeight=1, Nice=19, the lowest best-effort IO level, and
CPUQuota for its share of the processors this machine has.
2026-10-26 13:37:48 +01:00
mesh-tools 7f04d12047 Merge pull request 'Bus node: follow the grants part's sealed user list after the operator's take-over, never both sources; hand the vault the loaded hashes (issue 1945)' (#145) from feat/1945-user-list-switch into main 2026-10-26 12:21:55 +00:00
jschoubben e88ddab241 An apply of the bus module waits when the list on the machine cannot be read while the grants part's is followed, so the file is never removed as an orphan (review of the switch, issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-host@7f04d1204765
mesh/delivery-group group feat/1945-user-list-switch merged: every member is merged
mesh/check being carried to e88ddab24170 on 7f04d1204765 | novox/mesh-host#145 in group feat/1945-user-list-switch
2026-10-26 12:54:06 +01:00
jschoubben 06f70e11ee Bring in the adoption handing; the bus node writes nothing before it heard the switch, retries the key's watch and keeps the follow last heard, and an apply never writes the controller's list while the grants part's is followed (review of the switch, issue 1945)
Merges fix/1945-adopt-hashes 1a190090. A new follower (one per link) waits for the grants part's key to catch up before
its first write, so a restart after take-over never writes the controller's list first. A refused or ended watch is
tried again on link.Backoff, the have says why it is not watched, and the key last heard, kept beside the node's state,
holds meanwhile. While the grants part's list is followed, the bus module's composition is applied with the list on the
machine in place of the controller's. The grants list is never handed to the vault. Pins mesh-sdk c2a410bb.
2026-10-26 12:30:07 +01:00
jschoubben 61cec66ba8 gofmt the follower's test (issue 1945) 2026-10-26 12:08:37 +01:00
jschoubben 5e2fadce15 Pin mesh-sdk at the group's head 83ee7769 (issue 1945) 2026-10-26 12:07:10 +01:00
jschoubben 882e75c954 The bus node follows the grants part's sealed user list from take-over, the controller's otherwise, never both (ADR 0547 clauses 7, 8; issue 1945)
On the node whose compositions carry the bus's list, the engine also watches mesh-grants' key user-list.sealed. While it
says follow grants, the object it names is read by name, opened with this machine's sealing key, checked against the
key's digest and the guard, and written to the same file at 0600 only when its digest differs; the bus reloads it
itself. The have says which list is followed and this machine's sealing key, which the grants part seals to.
The follower's test waits for its watch to catch up before a later put, which raced under load. Pins mesh-sdk 175a5567.
2026-10-26 12:06:17 +01:00
jschoubben 1a190090ab The engine states its key again until it hears the vault's key, and at a slow interval after, so a vault delivered after it or a changed vault key is heard (review of the adoption handing, ADR 0547 clause 9) 2026-10-26 12:03:12 +01:00
jschoubben fd381567f4 The bus node hands the loaded list's hashes to the vault, sealed to its key, in the bus module's have (ADR 0547 clause 9, issue 1945)
The vault holds no value for the controller's own account, the node accounts and the enrolled devices, so it cannot
hash them, and take-over waits on their hashes. The bus node's engine is the one part that has them, in the list the
bus loaded. Its follower of the user list now puts every loaded user's bcrypt hash in the bus module's have, sealed to
the vault's own key (mesh-sdk delivery.AccountsHanded), so of every reader of the have bucket the vault alone opens
it; the clear part says only the list's digest, the key's fingerprint and how many users. The vault's key is heard from
its answer to node-key, which the engine already asks at start; a first or changed key looks at the list again. It is
sealed again only when the list on the machine or the vault's key changes; a refused list hands the list kept on the
machine. The vault takes from it only the users it holds no value for.

Also: the user list follower's test sent its later puts before the watch had handed the keys there before it, so two
of its cases failed on main whenever the goroutine started late; the feed now says when the watch began.
2026-10-26 11:32:36 +01:00
mesh-tools 1195cf4597 Merge pull request 'The node-engine retries an apply that met a passing bus error, so a reload does not leave a module failed (issue 1945)' (#144) from fix/apply-retries-transient into main 2026-10-25 22:42:41 +00:00
jschoubben 99285cb080 Retry an apply, a fetch by digest and a have write that met the bus reloading, and keep the cause in the have
mesh/delivery merged: source.main moved to novox/mesh-host@1195cf45971e
mesh/check pass 99285cb080db on 1195cf45971e | unit 303 run, 1080 kept, 5 skipped | integration 306 run, 1082 kept, 4 skipped | over bound: unit (slow…
The bus node's engine writes the bus user list (ADR 0547 bridge); each write makes the bus reload its accounts, and
for about three seconds every JetStream request meets no responders. An apply that failed then was kept failed until a
key changed (mesh-vault on novox, 2026-10-25 23:04:24), holding every module that needs the secret seat. A transient
bus failure is now retried with a backoff (2 s, 5 s, 15 s, 60 s, then every 5 min), the have saying retrying meanwhile;
after the engine writes the user list, the transiently failed modules are tried again once the reload is over. A have
write that met it is written again the same way. shorten keeps the head and the tail, the cause.
2026-10-25 23:37:02 +01:00
mesh-tools d75c66dc7e Merge pull request 'The bus node writes the newest signed bus user list by itself, without a want (ADR 0547 bridge, issue 1945)' (#143) from fix/1945-user-list-follows into main 2026-10-25 21:16:06 +00:00
mesh-tools e196058f6f Merge pull request 'The node-engine wakes the provisioner when an apply lets a file it refused be written (issue 924)' (#142) from fix/provisioner-retries-on-declaration into main 2026-10-25 21:08:52 +00:00
jschoubben 5e71ab6ae6 Count the bus's users from the list a digest names, take only a composition of this node and module, and refuse a list that would lock the mesh out (review of the bridge, ADR 0547 §8)
mesh/delivery merged: source.main moved to novox/mesh-host@d75c66dc7e27
mesh/delivery-group group fix/1945-user-list-follows merged: every member is merged
mesh/check pass 5e71ab6ae612 on d75c66dc7e27 | unit 303 run, 1077 kept, 5 skipped | integration 306 run, 1079 kept, 4 skipped | over bound: unit, inte…
B1: the controller sends the user list by digest (over 64 KiB), so its users are counted from the list as resolved, not
from the composition's inline content, which is empty then.
R1: a composition signed for another node or module than its key names is no source.
R2: a list without the controller's own user, or one that drops more than half the users of the file on the machine,
is not written; the have says why and the file on the machine is kept.
2026-10-25 22:00:05 +01:00
jschoubben 3f02bd7ff6 The bus node writes the bus's user list alone from the newest composition of the bus module, with no want, and says it in the bus module's have (ADR 0547 §8, issue 1945)
The list reached the bus only through a want of the bus module, which nobody writes (ADR 0539 §15): on 2026-10-25 it
stood from 11:23 to 20:50 and was put back by hand. The node agent now follows its composition bucket for the newest
signed composition carrying the user list, writes that file alone through the apply's file step when it differs
(mode 0600, read back, recorded as the module's), moves nothing that reads it (the bus module's entrypoint reloads the
server in place), never goes back to an older list, looks again after every apply of the bus module, and says source,
revision, wanted and written digests, users and any error in the bus module's have (mesh-sdk UserListWritten).

Read with mesh-delivery-view.node_haves (module nats); the bus's own nats_user_can says in_force once it reloaded.
2026-10-25 21:50:49 +01:00
jschoubben 338797a527 Knock under the apply's lock, and read what is owed first (review of this fix)
mesh/delivery merged: source.main moved to novox/mesh-host@e196058f6fb0
mesh/delivery-group group fix/provisioner-retries-on-declaration merged: every member is merged
mesh/check pass 338797a52703 on d75c66dc7e27 | unit 2301 run, 3410 kept, 10 skipped | integration 2326 run, 3431 kept, 13 skipped | over bound: unit, …
A write judged by the declaration before the apply settled could record its
refusal just after Knock found nothing owed: the original fault in a small
window. Knock now holds the apply's lock as a write does, reads no declaration
while nothing is owed, and its comment names Settled as its caller.
2026-10-25 21:49:23 +01:00
jschoubben 2c93f79429 Tell the provisioner's client to fill again once a settled apply may let through a write the filling refused (ace, 2026-10-25 21:17)
ace's filling refused mesh-build-go's files as not named as filled 32 s before
its want arrived; after the apply that named them nothing asked again, and the
runtime crash-looped on its missing broker credential. The filling now keeps
what it refused or failed and, when the node agent says a module's apply or
removal settled (agent.Env.Settled, called once what runs holds it), touches its
write socket if the declaration now names such a path, names the provisioner's
account, or a write had failed; the client watches that socket and fills every
module again. Said in the journal each time.
2026-10-25 21:42:41 +01:00
mesh-tools 5f16cf507a Merge pull request 'Say a module's health is not yet judged while it changes, and judge it as soon as its change ends (issue 1945)' (#141) from fix/r6-health-unknown into main 2026-10-25 15:04:54 +00:00
jschoubben b32a27c115 Owe a look when a module's change ends unjudged, so a module with nothing long-running is ok within a look of its apply
mesh/delivery merged: source.main moved to novox/mesh-host@5f16cf507a0f
mesh/check pass b32a27c115a2 on 0dfbe1a179f2 | unit 292 run, 1060 kept, 5 skipped | integration 295 run, 1062 kept, 4 skipped | over bound: unit (slow…
The facts look inside an apply runs while the module is still applying and judges nothing; the
running write then left xorg running and unknown until the five-minute statement, holding every
x11-display consumer behind it. The running write now nudges the health loop, which looks and says
at once. A declaration's report gets the same motion and composition check, and a removed module
leaves nothing marked unjudged (review of fix 15).
2026-10-25 15:46:05 +01:00
jschoubben c16e625a08 Say a module's health is not yet judged while its change is under way and until a look after it
A have said applying and failed for the controller on novox at 15:27:36, from a look at the
container being replaced, two seconds before its launch said healthy; the assigner held the
canary on it and set novox back. A look now judges only a module that is not in motion, and a
module whose composition moved carries unknown until a look after the move.
2026-10-25 15:32:59 +01:00
jschoubben 1cdb4d1a29 Say a module found running on its own have again at the agent's start, so its health is this engine's and not an older engine's unknown
xorg's have on g14 and shanks kept the unknown an engine before 958ccbd wrote: the
baseline believed the have and never handed the module to the session, so no look
rewrote its health, and the assigner held every module requiring x11-display back.
2026-10-25 15:27:23 +01:00
mesh-tools 0dfbe1a179 Merge pull request 'Drop the node's own copy of a resource a module composition declares, and give the kept declaration's filled paths to their modules (issue 1945)' (#140) from fix/r6-own-composition-dupes into main 2026-10-25 13:51:00 +00:00
jschoubben 1bdbf47ffa Leave out the node's own copy only by kind and id, never by a shared target (issue 1945, review)
mesh/delivery merged: source.main moved to novox/mesh-host@0dfbe1a179f2
mesh/check pass 1bdbf47ffa83 on 0dfbe1a179f2 | unit 302 run, 1044 kept, 5 skipped | integration 305 run, 1046 kept, 4 skipped | over bound: unit (slow…
A unit or container of the node's own that shares only its name with a
module's resource is not the module's: dropped, its record became an orphan
the apply removes, stopping the module's unit (1945/AC17).
2026-10-25 14:29:06 +01:00
jschoubben 16f84b53da Leave the node's own copy of a module's resource out, and carry the kept declaration's filled paths (issue 1945)
The node's own composition and a module's both declared the same files, and
the joined declaration was refused whole ("2 resources declare X"): the
node's own copy is now left out when joining and at the node's own apply, and
said once (1945/AC17). A module still running from the kept declaration now
carries its filled paths, so the filling no longer refuses them as not named
as filled (1945/AC18).
2026-10-25 14:10:52 +01:00
mesh-tools 57b553ac5a Merge pull request 'Apply the revision of a composition its want names, read from the key's history, when the latest has another digest (issue 1945)' (#139) from fix/r6-want-names-revision into main 2026-10-25 12:30:40 +00:00
jschoubben 11a635796d Bound a key's history read to 10 s and test it on a real bus; the witness test waits for its process to run from its image (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-host@57b553ac5a57
mesh/check pass 11a635796df9 on 57b553ac5a57 | unit 299 run, 1043 kept, 5 skipped | integration 302 run, 1045 kept, 4 skipped | over bound: unit (slow…
2026-10-25 12:51:00 +01:00
jschoubben 8343c4fb36 Apply the revision of a composition key whose digest the want names, from the revisions the bucket keeps (issue 1945) 2026-10-25 12:42:20 +01:00
mesh-tools 3e3e43de52 Merge pull request 'Never lose the bus to a replace of its own container (issue 1945)' (#138) from fix/r6-bus-replace into main 2026-10-25 11:25:11 +00:00
jschoubben 175fb45c19 Take a container left set aside as the next apply's, and bound an apply to 30 minutes (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-host@3e3e43de5214
mesh/check pass 175fb45c19e3 on 3e3e43de5214 | unit 299 run, 1042 kept, 5 skipped | integration 302 run, 1043 kept, 4 skipped | over bound: unit (slow…
2026-10-25 12:18:50 +01:00
jschoubben 219169dbd9 Set a running container aside until its replacement runs, and put it back when the replacement does not come up (issue 1945) 2026-10-25 12:14:41 +01:00
jschoubben b99d0b45bf Run an apply in a context the bus's loss cannot end, and say why a killed command was killed (issue 1945) 2026-10-25 12:09:28 +01:00
jschoubben f06058390c Merge pull request 'Apply again a module whose resources are gone, move a module with nothing to back up, and report ok for a running module with nothing to watch (issue 1944)' (#137) from fix/r6-reapply-missing into main
Reviewed-on: #137
2026-10-25 10:11:19 +00:00
jschoubben 958ccbde3e Say a running module ok once looked at when nothing of it is judged, so mesh-host's canary can be healthy and its other nodes get the new engine (issue 1945)
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check pass 958ccbde3ed4 on f06058390c0f | unit 297 run, 1038 kept, 5 skipped | integration 300 run, 1039 kept, 4 skipped | over bound: unit (slow…
2026-10-25 11:08:10 +01:00
jschoubben 43d1c2456f Apply a module again only for a newly missing resource, leave filled files out of the record check, and say when the applied state cannot be read (issue 1945) 2026-10-25 11:04:08 +01:00
jschoubben c46937da04 Apply a module again when the engine holds no record of its resources or its health says one is missing, and move a module with nothing to back up without a backup (issue 1944, issue 1945) 2026-10-25 10:59:21 +01:00
jschoubben cf455bc4fe Merge pull request 'Never remove a module's resources while its own composition waits to be applied (issue 1944)' (#136) from fix/r6-node-own-keeps-moved into main
Reviewed-on: #136
2026-10-25 09:23:28 +00:00
jschoubben ecf142e451 Let only a module's wanted or newest composition hold the node's own apply back, never a stale refused one (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-host@cf455bc4fe3e
mesh/check pass ecf142e4513f on cf455bc4fe3e | unit 292 run, 1033 kept, 5 skipped | integration 295 run, 1034 kept, 4 skipped | over bound: unit (slow…
2026-10-25 02:01:24 +01:00
jschoubben fa9e4483c8 Hold back the node's own apply while a module's composition cannot be read or is refused, naming it (issue 1944, issue 1945) 2026-10-25 02:56:20 +02:00
jschoubben e62c32c8c3 Leave a record another module's composition now holds, whatever its type, instead of removing it in the node's own apply; a removed module holds nothing back (issue 1944, issue 1945) 2026-10-25 02:54:58 +02:00
jschoubben 2bfde7d659 Hold out a module whose own composition waits to be applied, so the node's own apply never removes the resources it moved there (issue 1944, issue 1945) 2026-10-25 02:52:10 +02:00
jschoubben db06feed2e Merge pull request 'Refuse an apply that would write a provisioner file empty, naming the file (issue 1944)' (#135) from fix/r6-node-composition into main
Reviewed-on: #135
2026-10-25 00:17:56 +00:00
jschoubben 4e6b756bd2 Refuse an apply that would write a provisioner's file empty, read the node's own filled paths, and name the node's own composition when its apply fails (issue 1944, issue 1945)
mesh/delivery-group group fix/r6-node-composition held: a member is held: novox/mesh-sdk@10b6e202320f (merged without a passing check: only a person decides wh…
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check checking 4e6b756bd225 on b833d09d2872 | novox/mesh-host#135 in group fix/r6-node-composition
2026-10-25 01:49:00 +02:00
jschoubben b833d09d28 Merge pull request 'R6: the node-engine applies from its wants and signed compositions, one module at a time; operator-signers (issue 1944)' (#134) from feat/r6 into main
Reviewed-on: #134
2026-10-24 23:17:39 +00:00
jschoubben 81758b24a8 Pin mesh-sdk to feat/r6 748c564, which carries main's composer (issue 1943)
mesh/delivery-group group feat/r6 merged: every member is merged
mesh/delivery delivering: 0 machine step(s) passed
mesh/check checking 81758b24a8e4 on 285395d523e1 | novox/mesh-host#134 in group feat/r6
2026-10-24 23:06:00 +02:00
jschoubben 25ad5ce42c Merge remote-tracking branch 'origin/main' into feat/r6 2026-10-24 23:00:09 +02:00
jschoubben 285395d523 Merge pull request 'R4g finish: merge-check.sh gone, reads-outside markers (issues 1295, 1935)' (#133) from feat/r4g-finish into main
Reviewed-on: #133
2026-10-24 20:57:23 +00:00
jschoubben b47df31834 Pin mesh-sdk to d7a9b96 2026-10-24 21:18:33 +02:00
jschoubben fd05c8235d Pin mesh-sdk to 7f9cc0e 2026-10-24 20:43:47 +02:00
jschoubben 98d1f165b7 Pin mesh-sdk to e45bdc0 2026-10-24 20:22:00 +02:00
jschoubben 332d8f161b The carried genesis template no longer lists the controller's plan-moved event 2026-10-24 18:32:58 +02:00
jschoubben 5088755e5e gofmt 2026-10-24 18:28:22 +02:00
jschoubben e72c682450 Pin mesh-sdk to a523a8c 2026-10-24 18:18:06 +02:00
jschoubben be5f31ce66 What runs is the agent's compositions plus the kept declaration's rest: health, filling, take act, directories and the others held out of an apply read it; a restart trusts the node-engine's own haves; the bed tests run off Arch (review round 1) 2026-10-24 18:13:40 +02:00
jschoubben 67db003813 Pin mesh-sdk to the head of feat/r6 (5da87a0) 2026-10-24 17:50:20 +02:00
jschoubben e5a06cc73d The node agent waits for a backup's end on the holder's event, not by polling; the manifest consumes it (R6-A A5) 2026-10-24 17:03:30 +02:00
jschoubben 20d0998cdd The operator's public keys reach the node as the file operator-keys, fed by the node-engine's setting operator-signers (ADR 0539 clause 3) 2026-10-24 16:57:15 +02:00
jschoubben 83f16cff8d apply FILE is no hand path: mesh-compose and a want are (ADR 0539 clause 10); genesis keeps the carried bundle 2026-10-24 16:55:23 +02:00
jschoubben 96d0322498 The node-engine reports reads-wants; a failed apply is said; the have's fingerprint and digest follow its health (R6-A A6) 2026-10-24 16:43:26 +02:00
jschoubben b2003e7a38 A composition is applied only when the controller or the operator signed it; the node agent's integration tests on an embedded bus (R6-A A9, A1, A5) 2026-10-24 16:42:11 +02:00
jschoubben 3303797bfb The node agent runs in the node-engine: watches the wants and compositions on the link, applies one module's composition, asks the backup, writes the haves (R6-A A1, A5, A6) 2026-10-24 16:39:51 +02:00
jschoubben d5dd7761c6 The node agent: follows the wants and the node's compositions, applies one module at a time, backs a data module up first, holds it on a failed backup (R6-A A1, A2, A5, A6) 2026-10-24 16:36:21 +02:00
jschoubben 5f5f37b0b2 Pin mesh-sdk to feat/r6 S1-S3 (d6cef58): the composition bucket, the want's composition and backup answer, the have's composition and held phase 2026-10-24 16:31:20 +02:00
jschoubben c30b86ae22 Proved: the evidence is a have under the running build (comment) 2026-10-24 16:26:39 +02:00
jschoubben 1598156752 The node-engine hears no declaration: the queue, orders, put-backs, unsaid, numbers and news go; a session says the facts in the node-engine's own order, and the first have proves a build (R6-A A7, A8) 2026-10-24 16:25:11 +02:00
jschoubben e2b6f37168 Apply one module at a time: a scope of the module and what the others hold, and an owner record, so a removal is the module's own (R6-A A3, A4) 2026-10-24 16:16:32 +02:00
jschoubben 13bdcb694b Mark the packages whose tests read files outside their directory, found by the review (issue 1935)
mesh/delivery delivered
mesh/delivery-group group feat/r4g-finish merged: every member is merged
mesh/check checking 13bdcb694be0 on 3c3628c0ce46 | novox/mesh-host#133 in group feat/r4g-finish
2026-10-24 04:39:01 +02:00
jschoubben 89ebd0eefe Merge remote-tracking branch 'origin/feat/r4g-finish-1935' into HEAD 2026-10-24 04:26:19 +02:00
jschoubben b4f6537f80 Mark the packages whose tests read files outside their directory, so mesh-build-go never keeps their verdict (issue 1935) 2026-10-24 04:01:02 +02:00
jschoubben 7618f661ea Fail, not skip, when the checker places no controller beside the node-engine (issue 1295) 2026-10-24 03:43:49 +02:00
jschoubben 3c3628c0ce Merge pull request 'R4g (mesh-host): merge-check.sh goes; the checker judges the repository (design 47)' (#132) from feat/r4g into main
Reviewed-on: #132
2026-10-23 17:50:55 +00:00
jschoubben 4b4bb09a05 Make the apply tests fit the checker's isolated step: accept any who in the refused line, skip aloud where no group can be set (issue 1919, R4g)
mesh/delivery delivered
mesh/delivery-group group feat/r4g merged: every member is merged
mesh/check checking 4b4bb09a05a0 on 48c3e3834c69 | novox/mesh-host#132 in group feat/r4g
2026-10-23 18:54:53 +02:00
jschoubben 11b0a1fb29 Skip aloud the controller's step-nine check where the running checker places no controller (issue 1295, R4g)
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: unit: github.com/novox/mesh-host/internal/apply.TestJudgeDirectoriesSaysWhatTheWalkWouldRefuseAndChangesNothing: judge_directories_…
2026-10-23 18:35:32 +02:00
jschoubben 6ea77844d6 Judge the other genesis modules' toolchains; set aside only build-agent (issue 1152, R4g)
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: unit: github.com/novox/mesh-host/internal/bootstrap.TestTheControllersOwnManifestPassesStepNine: genesis_form_test.go:249: the cont…
2026-10-23 18:20:50 +02:00
jschoubben 2633573012 Skip aloud the tests whose subject R4g removes: genesis's build agent (issue 1152)
The build agent left mesh-controller with R4g (ADR 0524 section 5); genesis still installs it until issue 1152 decides where a new mesh's builder comes from. The tests that read its manifest say NOT JUDGED and are judged again with 1152.
2026-10-23 18:20:11 +02:00
jschoubben 976c9998d6 Delete merge-check.sh; the checker's levels run what it ran (issue 1295, S3; ADR 0529)
mesh/delivery superseded: a newer head of the same pull request
mesh/delivery-group group feat/r4g rejected: a member's own check failed
mesh/check failed: unit: github.com/novox/mesh-host/internal/bootstrap.TestGenesisInstallsTheCataloguesBuildAgent: builder_test.go:180: mesh-controlle…
2026-10-23 15:26:03 +02:00
mesh-tools 48c3e3834c Merge pull request 'Take a root-made directory in the mesh's own tree, and hold a container until the file it mounts exists (issue 1873)' (#131) from fix/1873-mesh-tree-dirs into main 2026-10-22 20:41:08 +00:00
jschoubben 0d12fb11a4 Make no container before a file it mounts is there, and clear the empty directory a runtime left at a file's path (issue 1873)
mesh/merge-gate head 0d12fb11a48b merged into trunk 59e6a6fa2a4a — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head 0d12fb11a48b merged into trunk 59e6a6fa2a4a — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/check failed: mesh-build-go: no go build node qualifies: builders disagree: go1.27.1-X:nodwarf5 recipe 74744709d277ded9 on ace, g14, shanks; go1.…
On ace the postgres container was replaced at 22:11:51 mounting superuser.secret, a
file the provisioner's client fills only after the apply. The runtime made the
missing source a directory, and every fill after it failed with "is a directory".

A container whose mount source is a file the declaration declares, not yet on the
machine, now waits and is neither created nor replaced. The write of that file
(WriteFilled) applies what reads it, as before, so the container is made once the
file is there.

What the runtime already left is cured by the mesh: a declared file whose path is
an empty directory in the mesh's own tree (by its path and by where it leads),
owned by the engine's account, has that directory removed before the file is
written, and the outcome says so. A directory with anything in it, or outside the
tree, is never removed.
2026-10-22 22:29:39 +02:00
jschoubben 03b9c47af4 Judge the mesh's tree by where a path leads, not only by its text (issue 1873, review)
A root-only link inside /var/lib/mesh pointing out of it would have made a
directory outside the tree, owned by the engine's account, the mesh's to give
away: the fault of issue 1873 again. The path and the tree's root are now both
resolved, links followed, and the directory is the tree's only when the resolved
path lies below the resolved root as well.
2026-10-22 22:27:45 +02:00
jschoubben cdbff1777e Take a directory in the mesh's own tree that the engine's account owns as the mesh's, not as found (issue 1873)
postgres's data directory moved to /var/lib/mesh/postgres. On ace and novox that
directory was there, root:root 0700, before the directory resource was applied, so
the engine used it as found (issue 339's rule) and left it to root: the postgres
provider, running as the operator's account, cannot reach its own secrets,
consumers and grants there.

/var/lib/mesh is the engine's tree (root, 0755): only the engine's account makes a
directory in it. A directory below it owned by that account is now the mesh's and
is given its declared owner and mode. Anywhere else, or owned by another account,
issue 339's rule stands.
2026-10-22 22:23:04 +02:00
mesh-tools 59e6a6fa2a Merge pull request 'Write the store's superuser secret where the postgres module now keeps it, and read the bus's release from its upstream image (issues 1873, 1875)' (#130) from fix/1873-system-state-dirs into main 2026-10-22 20:04:32 +00:00
jschoubben 8c0bb9d453 Read the bus's release from the nats module's pinned server image, not a NATS_BASE build argument (issue 1875)
mesh/repo-check head 8c0bb9d4531d merged into trunk 4065b45ae84f beside mesh-catalog#453@e0bf87c2 — pass: its merge-check.sh passed
mesh/merge-gate head 8c0bb9d4531d merged into trunk 4065b45ae84f — pass: group fix/1873-system-state-dirs composed: every machine composes with the change …
mesh/delivery delivered
mesh/delivery-group group fix/1873-system-state-dirs merged: every member is merged
mesh/check failed: unit: github.com/novox/mesh-host/internal/bootstrap.TestGenesisInstallsTheCataloguesBuildAgent: builder_test.go:180: mesh-controlle…
The nats module takes its server as an upstream artifact, pinned by digest
(issue 1641), and builds no image on a NATS_BASE argument any more. The test of
the bundle's bus read that argument, found none, and failed every mesh-host check
against the catalogue's main. Bootstrap's own code never read it: the bundle's
bus image comes from the bundle, so genesis was not affected. The test now
compares the bundle's image with the module's server by repository and digest.
2026-10-22 21:46:34 +02:00
jschoubben de809d7e0a Keep the store's superuser file in the postgres module's own directory, not in /var/lib/postgres (issue 1873)
The postgres module's own secrets move from /var/lib/postgres, the postgresql
package's directory, to its directory under /var/lib/mesh (mesh-catalog,
fix/1873-system-state-dirs). Genesis writes the store's superuser file at the
module's own-secret path, so the store it raises is the one the module adopts in
place: StoreSuperuserFile follows, to /var/lib/mesh/postgres/superuser.secret.

A directory genesis makes on the way is 0755, the mode of the mesh's directories,
and only the credential's own is 0700: MkdirAll at 0700 would have closed
/var/lib/mesh on a machine that had none yet.
2026-10-22 21:36:38 +02:00
mesh-tools 4065b45ae8 Merge pull request 'Have a replacement image before removing a running container (issue 1742)' (#129) from fix/1732-held-moves-asked into main 2026-10-21 19:07:56 +00:00
jschoubben 09c77d57b8 Have a replacement container's image before removing the running one, so an unobtainable image leaves the service up (issue 1742)
mesh/repo-check head 09c77d57b823 merged into trunk c231d1031fc7 beside mesh-controller#428@d6db3348 — pass: its merge-check.sh passed
mesh/merge-gate head 09c77d57b823 merged into trunk c231d1031fc7 — pass: group fix/1732-held-moves-asked composed: every machine composes with the change a…
mesh/delivery delivered
mesh/delivery-group group fix/1732-held-moves-asked merged: every member is merged
2026-10-21 19:16:32 +02:00
mesh-tools c231d1031f Merge pull request 'Read a declared file from the node's own object bucket, declaration-files-<node> (issue 1695, ADR 0488)' (#128) from feat/1696-declaration-files into main 2026-10-21 16:31:25 +00:00
jschoubben e597f9791c Merge main (the container-groups change) into the bucket-per-node branch, keeping the mesh-sdk pin of #64
mesh/merge-gate head e597f9791c62 merged into trunk 7e83137aa822 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head e597f9791c62 merged into trunk 7e83137aa822 — pass: its merge-check.sh passed
mesh/delivery superseded: its walk was superseded and names no walk that took it over: a walk that took it over answers its merge, and it follows that wa…
2026-10-21 18:05:17 +02:00
jschoubben e3a595d8fe Pin mesh-sdk to the merge of #64 (a bucket per node) 2026-10-21 18:03:34 +02:00
mesh-tools 7e83137aa8 Merge pull request 'Pass a container's groups to the runtime as supplementary groups and report container-groups (issue 1637, ADR 0480 clause 7)' (#127) from feat/r4d-groups into main 2026-10-21 14:45:04 +00:00
jschoubben f9e1cd183a Pin mesh-sdk to the tip of fix/1694-bucket-per-node (issue 1695) 2026-10-21 15:49:44 +02:00
jschoubben d9f8bbec0c Pin mesh-sdk to the merge of #60 (container-groups capability)
mesh/repo-check head d9f8bbec0c0b merged into trunk fd3259a3f5d7 beside mesh-controller#422@2aa291f3 — pass: its merge-check.sh passed
mesh/merge-gate head d9f8bbec0c0b merged into trunk fd3259a3f5d7 — pass: group feat/r4d-groups composed: every machine composes with the change as it did w…
mesh/delivery delivered
mesh/delivery-group group feat/r4d-groups merged: every member is merged
2026-10-21 15:48:44 +02:00
jschoubben d5eeb860e3 Read a file's object from the node's own bucket, declaration-files-<node>, never a shared one (issue 1695, ADR 0486 §2)
mesh-sdk is pinned to fix/1694-bucket-per-node 36d9eba51db9 until it merges.
2026-10-21 15:41:09 +02:00
jschoubben ef30c12291 Pass a container's groups to the runtime as supplementary groups and report container-groups (issues 1637, ADR 0480 clause 7)
mesh-sdk is pinned to feat/r4d-groups f0f66d5b8d42 (mesh-sdk #58) until it merges.
2026-10-21 15:34:37 +02:00
mesh-tools fd3259a3f5 Merge pull request 'Fetch a file the declaration names by digest from the bus's object bucket and verify it before writing, and run the integration tests in the merge check (issue 1695, ADR 0486 step 2)' (#126) from fix/1695-fetch-by-digest into main 2026-10-21 13:33:54 +00:00
jschoubben fdb59bcf4e Correct the grant list above FileObject to the subjects it calls, flow control and consumer delete included (issue 1695)
mesh/merge-gate head fdb59bcf4eb7 merged into trunk c5c4caa2a4ab — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head fdb59bcf4eb7 merged into trunk c5c4caa2a4ab — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/delivery delivered
2026-10-21 15:01:31 +02:00
jschoubben d0298a99da Pin mesh-sdk to the merge of #59 (file references) 2026-10-21 14:59:57 +02:00
jschoubben 4cfa0469fd Run the object-bucket test in the trunk's tagless suite too: the trunk's merge check judges this change (issue 1695) 2026-10-21 14:56:41 +02:00
jschoubben 45f437eea6 Run the integration tests in the merge check, resolve file references in the hand apply, refuse an unresolved one, and read an object by name without listing the bucket (issue 1695) 2026-10-21 14:50:34 +02:00
jschoubben a457b38968 Fetch a declaration's files given by digest from the bus's object bucket, verify them and report file-references (issue 1695, ADR 0486)
mesh-sdk is pinned to fix/1694-file-references 6c857e1c025f until it merges.
2026-10-21 14:34:29 +02:00
mesh-tools c5c4caa2a4 Merge pull request 'Count the home of an account a declaration makes for its own .ssh, never taking over another account's home (issue 1542)' (#125) from fix/1542-declared-account-is-a-home into main 2026-10-20 23:37:59 +00:00
mesh-tools 5a2bc03910 Let a declared account only add a home, never take one over, and match it by name alone, so no declaration hands another account's .ssh away (issue 1542, review round 1)
mesh/merge-gate head 5a2bc0391054 merged into trunk ce1b97ad973d — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head 5a2bc0391054 merged into trunk ce1b97ad973d — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-20 23:32:27 +00:00
mesh-tools 60371aadad Count the home of an account a declaration makes as a home for the directory placement guard, so its own .ssh is placed (issue 1542) 2026-10-20 23:25:37 +00:00
mesh-tools ce1b97ad97 Merge pull request 'Take the node-engine's parser and validator from mesh-sdk and delete mesh-host's copies (design 47 R4b, issue 1392)' (#124) from feat/r4b-sdk-users into main 2026-10-20 12:51:54 +00:00
mesh-tools 52600ec7bd Take the node-engine's parser from mesh-sdk: remaining imports (issue 1392)
mesh/merge-gate head 52600ec7bd61 merged into trunk cdcba9ca7375 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head 52600ec7bd61 merged into trunk cdcba9ca7375 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-20 11:17:14 +00:00
mesh-tools 278a94d59b Take the node-engine's parser from mesh-sdk: internal/apply imports (issue 1392) 2026-10-20 11:16:26 +00:00
mesh-tools 9d76759e04 Take the node-engine's parser and validator from mesh-sdk: pin, README and cmd imports (issue 1392) 2026-10-20 11:15:23 +00:00
mesh-tools a8eb24b49c Delete mesh-host's copy of the node-engine's parser and validator, which mesh-sdk holds (issue 1392), rest 2026-10-20 11:14:38 +00:00
mesh-tools da26b7f130 Delete mesh-host's copy of the node-engine's parser and validator, which mesh-sdk holds (issue 1392) 2026-10-20 11:14:30 +00:00
mesh-tools cdcba9ca73 Merge pull request 'The node-engine reports whether an unprivileged user can make a user namespace (design 47 R4a, issue 1308)' (#123) from feat/r4a-sdk-first into main 2026-10-20 07:15:53 +00:00
mesh-tools 454be9b269 Merge pull request 'Write a module that leaves a node as removed in its have key, so a reader can tell it stopped (issue 1369)' (#121) from fix/r3e-before-the-gate into main 2026-10-20 06:01:59 +00:00
mesh-tools 619e14aa8e Merge pull request 'Close the test bus only after the enrolment stand-in has removed its consumer, so the next test can subscribe (issue 604)' (#122) from fix/604-the-enrolment-stand-in-unbinds into main 2026-10-20 05:57:07 +00:00
mesh-tools d6f8b00bf8 Do not let the asking's Close shut the stand-in's connection before its cleanup (issue 604)
mesh/merge-gate head d6f8b00bf822 merged into trunk 2474bf28330f — pass: the change touches no module of the mesh's graph
mesh/repo-check head d6f8b00bf822 merged into trunk 2474bf28330f — pass: its merge-check.sh passed
mesh/delivery delivered
The asking helper registered a.Close, which closes the shared test connection; the stand-in's cleanup ran after it on a closed connection, so its consumer was never deleted and blocked the next test on the workqueue stream.
2026-10-20 01:34:27 +00:00
mesh-tools a9789b5125 Give each enrolment stand-in its own durable and wait for it to be gone (issue 604)
The stand-in shared one durable name across tests and let go with Unsubscribe alone, so a test starting before the server dropped the binding was refused with "consumer is already bound to a subscription".
2026-10-20 01:30:20 +00:00
mesh-tools 150fadfe8b Write a removal after a restart from the declaration the apply replaced (issue 1369)
mesh/repo-check head 150fadfe8bed merged into trunk 619e14aa8e57 — pass: its merge-check.sh passed
mesh/merge-gate head 150fadfe8bed merged into trunk 619e14aa8e57 — pass: group fix/r3e-before-the-gate composed: every machine composes with the change as …
mesh/delivery delivered
mesh/delivery-group group fix/r3e-before-the-gate merged: every member is merged
The set of modules this process wrote lives in memory, so a node-engine restarted before the apply that drops a module never wrote that module's removal. The report now names the modules the kept declaration held before, and they are removal candidates.
2026-10-20 01:26:33 +00:00
mesh-tools 6a2fb63bfc Write a module's have key as removed when the applied declaration no longer holds it (issue 1369)
A module that left a node kept its have key at running for ever, so nothing could tell it stopped and the gate would never take the seat. The key is written once with the phase removed over the revision read, never deleted. Pins mesh-sdk to the commit that adds the phase.
2026-10-20 01:20:52 +00:00
mesh-tools 64dfa90c3a Report whether an unprivileged user namespace can be made (issue 1308)
mesh/repo-check head 64dfa90c3a6c merged into trunk 454be9b2693d — pass: its merge-check.sh passed
mesh/merge-gate head 64dfa90c3a6c merged into trunk 454be9b2693d — pass: group feat/r4a-sdk-first composed: every machine composes with the change as it di…
mesh/delivery delivered
mesh/delivery-group group feat/r4a-sdk-first merged: every member is merged
A build module's sandbox needs one, and a module can require only a capability the node-engine detects (ADR 0448 section 5). The probe drops to an unprivileged uid first when run as root, since root always can.
2026-10-20 01:12:24 +00:00
mesh-tools 2474bf2833 Merge pull request 'The node-engine sends no alive message: presence replaces it (design 47 R2d, issue 1158)' (#120) from feat/alive-gone into main 2026-10-19 07:07:56 +00:00
jschoubben 671f284004 Delete the node-engine's alive message, presence replaces it (issue 1158)
mesh/merge-gate head 671f2840046f merged into trunk 89025ad004d4 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head 671f2840046f merged into trunk 89025ad004d4 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 02:15:00 +02:00
mesh-tools 89025ad004 Merge pull request 'Judge a filled write with the roots the last apply read, asking no runtime per file (issue 1138)' (#119) from fix/1138-filled-write-asks-no-runtime into main 2026-10-18 22:20:54 +00:00
jochen 3798673821 Judge a filled write with the roots the last apply read, asking no runtime per file (issue 1138)
mesh/merge-gate out of date: judged against trunk 8119c470d393, the trunk is now 89025ad004d4; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 8119c470d393, the trunk is now 89025ad004d4; judged again on a recheck or when it is merged
mesh/delivery delivered
2026-10-19 00:11:58 +02:00
mesh-tools 8119c470d3 Merge pull request 'Write each module's have and the node's own facts beside the report (issue 1132); the check reads build-agent from the controller (issue 1142)' (#118) from feat/delivery-have into main 2026-10-18 18:36:14 +00:00
jschoubben aab65b81f6 Read the build agent's manifest from the controller's repository beside the check, where it moved (issue 1151)
mesh/repo-check head aab65b81f65c merged into trunk 6469e8238443 — pass: its merge-check.sh passed
mesh/merge-gate head aab65b81f65c merged into trunk 6469e8238443 — pass: group feat/delivery-have composed: every machine composes with the change as it di…
mesh/delivery delivered
mesh/delivery-group group feat/delivery-have merged: every member is merged
2026-10-18 19:28:07 +02:00
jschoubben 9d9e3a46c4 Bind the have bucket once per pass, bound by the queue's timeout, and stop the pass at the first error (issue 1132) 2026-10-18 19:18:08 +02:00
jschoubben 0b391f49a2 Write each module's have and the node's own facts beside the report, grouped per module and keyed by the module the controller names (issue 1132)
The node-engine reads a resource's module and a declaration's builds, says it does in every report (reads_modules), groups the apply's outcome per module with every failing resource named, and writes have.<node>.<module> and have.<node>.node to the bucket only when a value differs, never ahead of or in place of the report (novox/hq ADR 0429 section 4).
2026-10-18 19:10:05 +02:00
mesh-tools 6469e82384 Merge pull request 'A directory says how it reacts to files it does not know; the node-engine moves out what a move-out directory does not know (issue 1024)' (#117) from feat/owned-paths into main 2026-10-17 16:20:06 +00:00
mesh-tools dae79c7643 Prove move-out on a machine that is not Arch (issue 1024)
mesh/merge-gate head dae79c76434c merged into trunk cd13f7726ef0 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head dae79c76434c merged into trunk cd13f7726ef0 — pass: its merge-check.sh passed
mesh/delivery delivered
The merge check's build container has no pacman, so the acceptance test now stubs it and names the host it was built for, as other command tests do; nothing in the node-engine itself changes.
2026-10-17 16:16:02 +00:00
mesh-tools d237dee4c9 Let a directory say how it reacts to files it does not know, and move out what a move-out directory does not know (issue 1024, ADR 0422)
mesh/merge-gate head d237dee4c9e3 merged into trunk cd13f7726ef0 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head d237dee4c9e3 merged into trunk cd13f7726ef0 — fail: its merge-check.sh failed: --- FAIL: TestAMoveOutDirectoryKeepsOnlyWhatItKnows (0.…
mesh/delivery superseded: a newer head of the same pull request
A directory or archive resource carries `unknown` (move-out, module, program) and `knows`. For move-out under a home, every look moves each path the declaration does not know into ~/.local/state/mesh-moved/<module>/<time>/, the list written before each move, without following a link, a young path waiting, a socket or pipe left and reported, nothing made when nothing moved, and another file system refused. Reviewed in two rounds.
2026-10-17 16:09:33 +00:00
mesh-tools cd13f7726e Merge pull request 'Feature 57 (config and provisioning): release mesh-host' (#116) from release/57-config-and-provisioning into main 2026-10-16 09:32:15 +00:00
mesh-tools bff1ac14a1 Feature 57 (config and provisioning): release mesh-host, the reviewed release branch on top of main
mesh/merge-gate head bff1ac14a198 merged into trunk 49c9002c04d3 beside mesh-controller#355@8c381172 — pass: builds mesh-host → ace, g14, novox, shanks; no…
mesh/repo-check head bff1ac14a198 merged into trunk 49c9002c04d3 beside mesh-controller#355@8c381172 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group release/57-config-and-provisioning merged: every member is merged
2026-10-16 08:24:13 +00:00
mesh-tools 49c9002c04 Merge pull request 'Grant the genesis controller user operator-channel.tell (issue 835)' (#113) from feat/mail-seats into main 2026-10-15 17:09:30 +00:00
jochen c3b2fce0a2 Grant the controller's new operator-channel.tell publish to the genesis user (issue 835)
mesh/merge-gate head c3b2fce0a29f merged into trunk 198fd34ec497 — pass: the change touches no module of the mesh's graph
mesh/repo-check head c3b2fce0a29f merged into trunk 198fd34ec497 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group feat/mail-seats merged: every member is merged
ADR 0380's secret-given announcement asks operator-channel.tell over the
bus; the genesis bundle's first user list must already carry that
publish subject, in the same place the controller composes it
(internal/broker, mesh-controller ms-835 435468c1), or a first node's
controller starts without the grant until its own first composition.
2026-10-15 17:49:51 +02:00
mesh-admin 198fd34ec4 Merge pull request 'Start no account's manager to look at its units (hq issue 764)' (#112) from fix/764-a-look-starts-no-user-manager into main 2026-10-14 22:38:02 +00:00
jschoubben 62af89f85d Start no account's manager to look at its units (hq issue 764)
mesh/merge-gate head 62af89f85d0d merged into trunk 1a2677882380 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head 62af89f85d0d merged into trunk 1a2677882380 — pass: its merge-check.sh passed
mesh/delivery delivered
Every 15 s the failed-units look read the own manager of every account
the declaration creates, and reading one is a login that starts it: six
module accounts' managers on the control node started and stopped all
day, most of its journal. A look only reads (ADR 0240 rule 6) and never
asks an account's manager (ADR 0177).

An account's manager is read now only when the declaration places a
user-scoped unit in it, and only after the machine's own manager says
user@<uid>.service is active, as the apply asks before it. A manager not
running is said away in the units statement, with no failed units; in
liveness its units read unknown with the apply's words, never down. The
operator's account on a desktop is still read while logged in, since a
module places a user unit there (issue 315's Razer daemon stays said).
2026-10-15 00:33:34 +02:00
mesh-admin 1a26778823 Merge pull request 'A busy container is left running until it says idle, within 15 minutes (issue 594, hq ADR 0335)' (#98) from feat/335-a-busy-container-is-replaced-when-idle into main 2026-10-13 22:20:54 +00:00
jschoubben b9a293f8c8 Merge remote-tracking branch 'origin/main' into feat/335-a-busy-container-is-replaced-when-idle
mesh/merge-gate head b9a293f8c8bc merged into trunk 0e92ef3d1b21 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head b9a293f8c8bc merged into trunk 0e92ef3d1b21 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group feat/335-a-busy-container-is-replaced-when-idle merged: every member is merged
# Conflicts:
#	cmd/mesh-host/main.go
2026-10-14 00:00:09 +02:00
jschoubben 695e022991 Keep a replacement owed after a busy wait, and clamp a wait from a later clock
A restart-on trigger that is no file counts as changed only in its own pass,
so after a busy wait the container read unchanged and was never replaced,
while the report stopped listing it (review of #98).
2026-10-13 23:59:58 +02:00
mesh-admin 0e92ef3d1b Merge pull request 'Report the networks the outward links are on, for the local network (issue 154, ADR 0340)' (#107) from fix/154-local-network into main 2026-10-13 13:07:08 +00:00
mesh-admin 12bc6a56a3 Merge pull request 'Adopt genesis's bus through the bus's planned step, its streams snapshotted first (issue 590, hq ADR 0333 §3)' (#108) from fix/590-adopt-the-bus-as-its-planned-step into main 2026-10-13 11:11:41 +00:00
jochen cdcfcd9380 Adopt genesis's bus through the bus's planned step, its streams snapshotted first (hq issue 590, ADR 0333 §3)
mesh/merge-gate head cdcfcd938016 merged into trunk 130c5d98a601 — pass: the change touches no module of the mesh's graph
mesh/repo-check head cdcfcd938016 merged into trunk 130c5d98a601 — pass: its merge-check.sh passed
mesh/delivery delivered
The broker step sent the nats module with a plain push, and the controller refuses a push that would
replace the bus: that is a planned step, bus upgrade, which snapshots the streams first and checks
them after (ADR 0236). Genesis stopped at step 18.

The step now snapshots every stream with the module's own snapshot program (no backup holder exists
yet to be asked), asks bus upgrade with that snapshot and why, says the one restart, and waits until
the bus runs the module's image, answers, and the controller ends the step done. The bundle's bus runs
the release the module's image is built on (2.11.17, by the same digest, where it ran 2.10.29), so the
step changes no server release and is said reversible.
2026-10-13 11:47:10 +02:00
mesh-admin 130c5d98a6 Merge pull request 'Build the Go toolchain at genesis's base step, beside the base (issue 616)' (#106) from fix/616-the-base-builds-every-genesis-toolchain into main 2026-10-13 09:23:04 +00:00
jochen cb7b71dcc5 Report the networks the outward links are on, for the local network (issue 154, ADR 0340)
mesh/merge-gate head cb7b71dcc568 merged into trunk 12bc6a56a38a — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head cb7b71dcc568 merged into trunk 12bc6a56a38a — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/154-local-network merged: every member is merged
A port reached from local-network is admitted from the private networks the machine is attached to, and only the machine can say which those are. Every network on a link facing outside is said among its facts, and a change is said at the next reconcile.
2026-10-13 11:20:21 +02:00
jochen 8108dd5c23 Build the Go toolchain at genesis's base step, beside the base (hq issue 616)
mesh/merge-gate head 8108dd5c23c6 merged into trunk f0d39b4a4ab0 — pass: the change touches no module of the mesh's graph
mesh/repo-check head 8108dd5c23c6 merged into trunk f0d39b4a4ab0 — pass: its merge-check.sh passed
mesh/delivery delivered
Genesis's base step built only mesh-tools, the TypeScript toolchain. The store's and the bus's modules
carry Go bundles, compiled in mesh-tools-go, so genesis stopped at the store step: "this mesh holds no
copy of it. Build mesh-tools-go first". The base step now builds mesh-tools-go too, from
--tools-go-source or the repository beside --tools-source, and a test holds every toolchain a genesis
module's bundles compile in, read from the catalogue, to the ones the step builds.
2026-10-13 11:13:44 +02:00
mesh-admin f0d39b4a4a Merge pull request 'Wait until the node applied the genesis document before the sdk step asks for a build (issue 615)' (#105) from fix/615-wait-for-the-genesis-document into main 2026-10-13 09:08:59 +00:00
jochen 89fd0c06e1 Wait until the node applied the genesis document before the sdk step asks for a build (hq issue 615)
mesh/merge-gate head 89fd0c06e102 merged into trunk 17038b2a074b — pass: the change touches no module of the mesh's graph
mesh/repo-check head 89fd0c06e102 merged into trunk 17038b2a074b — pass: its merge-check.sh passed
mesh/delivery delivered
The packages step sealed the build agent its genesis document and pushed, and the push returned as soon
as it was sent. The sdk step then asked for the SDK's build at once, and the agent reads the document
from the file the node writes when it applies that push: one genesis found it in time, the next was
refused for having neither a binding nor the document. That one push now waits until the node reports
it applied it (push --wait); every other push still returns on sending.
2026-10-13 10:59:45 +02:00
mesh-admin 17038b2a07 Merge pull request 'Raise genesis's store as the postgres module's server, so the store step adopts it as it runs (issue 609, hq ADR 0338)' (#104) from fix/609-genesis-raises-the-store-as-postgres into main 2026-10-13 08:55:41 +00:00
jochen 4dde21f0ab Raise genesis's store as the postgres module's server, so the store step adopts it as it runs (hq issue 609, ADR 0338)
mesh/merge-gate head 4dde21f0ab20 merged into trunk 7d1d6bffe30c — pass: the change touches no module of the mesh's graph
mesh/repo-check head 4dde21f0ab20 merged into trunk 7d1d6bffe30c — pass: its merge-check.sh passed
mesh/delivery delivered
The bundle raised the store as mesh-store, on upstream postgres, in a docker volume; the postgres
module keeps it as postgres, on its pgvector image, in /var/lib/mesh-store owned 999:70, reading its
superuser password from a file. The store step refused, and genesis stopped at step 17 of 24.

The bundle now raises the store with the module's name, image, data directory and password file, and
every step of the bundle and the installer reaches it by that name. The refusal to raise a store over
data it cannot open looks for a cluster in the store's directory, no longer for the old volume.
2026-10-13 10:47:53 +02:00
mesh-admin 7d1d6bffe3 Merge pull request 'Place the bus's users after every push until the bus is adopted (issue 608)' (#103) from fix/608-place-the-bus-users-after-every-push into main 2026-10-13 05:00:42 +00:00
jochen 7283209d1a Place the bus's users after every push until the bus is adopted (hq issue 608)
mesh/merge-gate head 7283209d1abc merged into trunk 64f0ab405101 — pass: the change touches no module of the mesh's graph
mesh/repo-check head 7283209d1abc merged into trunk 64f0ab405101 — pass: its merge-check.sh passed
mesh/delivery delivered
The bus reads its user list from a file that, until the broker step adopts it as the nats module,
only the installer writes, and the installer wrote it only at enrolment. The build agent's account,
minted with step 13's push, never reached the bus; the agent was refused (Authorization Violation)
and the sdk step's build waited until it was killed. Every push genesis makes now places the composed
list and has the bus re-read it, until the nats module is registered (ADR 0331 §5, ADR 0333 §4).
2026-10-13 06:58:39 +02:00
mesh-admin 64f0ab4051 Merge pull request 'genesis: seal the build agent the forge it raised, in place of the builder's npm-password (hq issue 121, ADR 0336)' (#102) from fix/121-genesis-package-registry into main 2026-10-13 03:37:15 +00:00
jochen 5f147fe6b9 Seal the build agent the forge genesis raised, in place of the builder's npm-password (hq issue 121, ADR 0336)
mesh/merge-gate head 5f147fe6b9c7 merged into trunk 0d11b4fc930d — pass: the change touches no module of the mesh's graph
mesh/repo-check head 5f147fe6b9c7 merged into trunk 0d11b4fc930d — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/121-genesis-package-registry merged: every member is merged
The build agent declares neither the retired builder's npm-password own-secret
nor a serves setting, so the packages step handed its credential to nothing
and genesis's sdk step had no way to publish. The packages step now seals the
genesis-package-registry own-secret to build-agent: the forge on this node's
loopback, the port given for the package registry, its npm path, and the
mesh-builder account with its password. setFoundationSettings no longer gives
the builder a serves.port.
2026-10-13 03:01:35 +02:00
jschoubben bb60e7b584 Leave a busy container running until it says idle, within 15 minutes
mesh/delivery superseded: a newer head of the same pull request
mesh/delivery-group group feat/335-a-busy-container-is-replaced-when-idle rejected: its order contradicts itself: novox/mesh-catalog@190608193764, novox/mesh-c…
mesh/merge-gate out of date: judged against trunk 7a41e120635b, the trunk is now 8431ac96562e; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 7a41e120635b, the trunk is now 8431ac96562e; judged again on a recheck or when it is merged
A send that replaced the build agent killed the check it ran, and the queue
started it again elsewhere (issue 594). A container may now declare busy; the
node-engine asks it before replacing it, applies everything else, says the
wait, and replaces it on the first idle pass or at the bound, kept across
restarts (novox/hq ADR 0335).
2026-10-13 00:41:32 +02:00
291 changed files with 25293 additions and 9277 deletions
+14 -1
View File
@@ -102,6 +102,19 @@ what it did not understand would apply most of a declaration and report success.
]}
```
**A directory or archive says how it treats files it did not write** (issue 1024,
[`novox/hq` ADR 0422](https://git.novox.be/novox/hq)). `unknown` is `move-out`, `module` or `program`;
absent, nothing about unknown paths is done. `knows` lists paths inside it, relative to its `path`, that
are not declared but belong there; a trailing `/` names a folder and everything in it. `knows` without
`unknown` is refused. For `move-out` the node-engine, after every apply and so on every reconcile, moves
each path inside the directory that nothing declared, no archive unpacked and `knows` does not name,
whole, into `<home>/.local/state/mesh-moved/<module>/<UTC time>/` beside a `<UTC time>.json` list. The
list names each move before it is made. Nothing is deleted. A path younger than a minute waits for the
next look; a socket, pipe or device is left and said. It is only for a directory under an account's
home on one file system with the kept folder, and otherwise the resource fails and nothing moves.
`module` and `program` are read by the module and the program that owns the path, and the node-engine
does nothing for them.
**It converges rather than executes.** Applying twice changes nothing the second time; applying
to a drifted machine returns it. A mode is *maintained*, not merely set — a permission applied
at creation is not a permission held.
@@ -244,7 +257,7 @@ without its neighbour checked out is a repository nobody can build.
```
mesh-controller: ./build/mesh-controller plan <node> --json > /tmp/d.json
mesh-host: MESH_EMITTED=/tmp/d.json go test ./internal/declaration/ -v
mesh-sdk: MESH_EMITTED=/tmp/d.json go test ./declaration/ -v (run in mesh-sdk's go/ directory)
```
**What this node says when it joins, read by the mesh:**
+6
View File
@@ -0,0 +1,6 @@
# Release 57: config and provisioning
This branch carries the work of hq design 57 (ADR 0348, ADR 0349) in this repository. Each package
merges here through its own reviewed pull request; the branch reaches main as one pull request, in one
delivery group with the branch of the same name in mesh-controller, mesh-host, mesh-sdk, mesh-catalog
and hq, after a rehearsal in the lab. This file is deleted by the last commit before that pull request.
+6
View File
@@ -116,6 +116,8 @@ const usage = `mesh-bootstrap — make a bare machine into a mesh
--tools-source the repository the shared base is built from
--tools-ref what of it to build (default main)
--tools-go-source the repository the Go toolchain is built from (default: mesh-tools-go beside --tools-source)
--tools-go-ref what of it to build (default main)
--catalog-source the catalogue REPOSITORY, for building its modules;
--catalog is the checkout that says what they are
--catalog-ref what of it to build (default main)
@@ -308,6 +310,10 @@ func newFlagSet(opts *bootstrap.Options, jsonOut *bool) *flag.FlagSet {
"the repository the shared base is built from")
set.StringVar(&opts.ToolsSource.Ref, "tools-ref", opts.ToolsSource.Ref,
"what of it to build (default main)")
set.StringVar(&opts.ToolsGoSource.Repository, "tools-go-source", opts.ToolsGoSource.Repository,
"the repository the Go toolchain is built from (default: mesh-tools-go beside --tools-source)")
set.StringVar(&opts.ToolsGoSource.Ref, "tools-go-ref", opts.ToolsGoSource.Ref,
"what of it to build (default main)")
set.StringVar(&opts.CatalogSource.Repository, "catalog-source", opts.CatalogSource.Repository,
"the catalogue REPOSITORY, for building its modules — --catalog is the checkout that says what they are")
set.StringVar(&opts.CatalogSource.Ref, "catalog-ref", opts.CatalogSource.Ref,
+39
View File
@@ -0,0 +1,39 @@
package main
import (
"context"
"testing"
"time"
)
// An apply runs in a context the bus's loss cannot end: the link's context ends when the bus goes, and the bus may go
// because of the apply itself (it replaces the bus's own container), which then must finish (issue 1945, live on
// 2026-10-25 at 11:23: novox's node-engine removed the bus container, its link's context ended, and the docker run that
// would have made the new one was never made).
func TestAnApplyOutlivesTheLinksContext(t *testing.T) {
type key struct{}
link, cancel := context.WithCancel(context.WithValue(context.Background(), key{}, "v"))
ctx, done := applyContext(link)
defer done()
cancel()
if ctx.Err() != nil {
t.Fatalf("the apply's context ended with the link's: %v", ctx.Err())
}
if ctx.Value(key{}) != "v" {
t.Fatal("the apply's context lost the link's values")
}
}
// An apply is bounded all the same: a step that hangs (a docker call that never returns) cannot hold the node agent for
// ever (the review of issue 1945).
func TestAnApplyIsBounded(t *testing.T) {
ctx, done := applyContext(context.Background())
defer done()
at, ok := ctx.Deadline()
if !ok {
t.Fatal("an apply has no bound")
}
if left := time.Until(at); left > applyBound || left < applyBound-time.Minute {
t.Fatalf("the bound is %s away, want about %s", left, applyBound)
}
}
+41
View File
@@ -0,0 +1,41 @@
package main
import (
"context"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/store"
)
// applyAndKeep applies a whole declaration the way the carried bundle is applied, and says what came of it as the facts
// the node says: what the tests of the apply's steps read. The node-engine applies no whole declaration on its own now
// (novox/hq ADR 0539 §9); this is the test's way to put a machine in a state.
func applyAndKeep(ctx context.Context, opts options, raw []byte, signed *store.Declared, sched *apply.Scheduler,
say link.Announce) link.Report {
declared, err := declaration.Parse(raw)
if err != nil {
return link.Report{Refused: err.Error()}
}
got := applyModuleHeld(ctx, opts, declared, nil, sched, say)
if got.Refused != "" {
return link.Report{Refused: got.Refused}
}
held := map[string]bool{}
for _, h := range got.Updated.Held {
held[h.ID] = true
}
report := factsOf(ctx, opts, declared, got.Updated, got.Outcome, held)
if signed != nil {
if err := store.SaveDeclared(store.DeclaredPath(opts.state), *signed); err != nil {
report.Failed = map[string]string{"keeping the declaration": err.Error()}
}
}
if got.Err != nil {
report.Failed = map[string]string{"apply": got.Err.Error()}
}
report.Failed = withUnanswered(report.Failed, got.Probe)
report.Unapplied = unapplied(declared, got.Outcome, got.Err)
return report
}
+39
View File
@@ -0,0 +1,39 @@
package main
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// authorityPath is where this engine keeps the mesh's own certificate as it last applied it (public): beside the
// certificate issued for the machine's name, read into its record (novox/hq ADR 0555 §3).
func authorityPath(state string) string { return filepath.Join(filepath.Dir(state), "authority.crt") }
// keepAuthority keeps the mesh's own certificate a composition carries (the resource a module's certificate.authority
// asks for, catalogue.AuthorityID), so the record says the authority this machine holds. A composition carrying none
// changes nothing.
func keepAuthority(state string, c delivery.Composed) error {
for _, raw := range c.Resources {
var r struct {
ID string `json:"id"`
Content string `json:"content"`
}
if json.Unmarshal(raw, &r) != nil || r.Content == "" || !strings.HasSuffix(r.ID, "."+catalogue.AuthorityID()) {
continue
}
path := authorityPath(state)
if was, err := os.ReadFile(path); err == nil && string(was) == r.Content {
return nil
}
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return err
}
return os.WriteFile(path, []byte(r.Content), 0o644)
}
return nil
}
+32
View File
@@ -0,0 +1,32 @@
package main
import (
"encoding/json"
"os"
"path/filepath"
"testing"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// Issue 1126 (ADR 0555 §3): the mesh's own certificate a composition carries is kept for the machine's record, and the
// record's keys read it; a composition carrying none changes nothing.
func TestTheAppliedAuthorityIsKeptForTheRecord(t *testing.T) {
state := filepath.Join(t.TempDir(), "state", "state.json")
res := func(id, content string) json.RawMessage {
raw, _ := json.Marshal(map[string]any{"id": id, "type": "file", "path": "/etc/x", "content": content})
return raw
}
if err := keepAuthority(state, delivery.Composed{Resources: []json.RawMessage{res("web.conf", "x")}}); err != nil {
t.Fatal(err)
}
if _, err := os.Stat(authorityPath(state)); !os.IsNotExist(err) {
t.Fatalf("a composition with no authority kept one: %v", err)
}
if err := keepAuthority(state, delivery.Composed{Resources: []json.RawMessage{res("ca-trust.certificate-authority", "CA-1")}}); err != nil {
t.Fatal(err)
}
if got := recordKeys(state)(); got.Authority != "CA-1" {
t.Errorf("the record's keys read the authority %q", got.Authority)
}
}
+42
View File
@@ -0,0 +1,42 @@
package main
import (
"context"
"encoding/json"
"strings"
"testing"
"time"
)
func backupEvent(node, module string, ok bool, at time.Time, why string) []byte {
raw, _ := json.Marshal(map[string]any{"node": node, "module": module, "ok": ok, "at": at, "error": why})
return raw
}
// A backup is done when the holder says so for this node and module, held on its words when it says it failed, and held
// when it says nothing within the bound; another node's or module's event, or one from before the ask, is not the answer
// (R6-A A5, novox/hq ADR 0429 §14).
func TestABackupEndsWhenItsEventSaysSo(t *testing.T) {
since := time.Now()
run := func(within time.Duration, events ...[]byte) (done bool, words string) {
ch := make(chan []byte, len(events))
for _, e := range events {
ch <- e
}
got := awaitBackup(context.Background(), ch, "n1", "db", since, within)
return got.Done, got.Words
}
if done, _ := run(time.Second,
backupEvent("n2", "db", true, since, ""), backupEvent("n1", "web", true, since, ""),
backupEvent("n1", "db", false, since.Add(-time.Hour), "an old night"),
backupEvent("n1", "db", true, since.Add(time.Second), "")); !done {
t.Error("the event of this node and module was not the answer")
}
if done, words := run(time.Second, backupEvent("n1", "db", false, since, "the repository is locked")); done ||
!strings.Contains(words, "repository is locked") {
t.Errorf("a failed backup: done=%v words=%q", done, words)
}
if done, words := run(50 * time.Millisecond); done || !strings.Contains(words, "did not say it was done") {
t.Errorf("silence within the bound: done=%v words=%q", done, words)
}
}
+29
View File
@@ -0,0 +1,29 @@
package main
import (
"encoding/json"
"testing"
)
// A module that declares nothing to back up on this machine is not backed up before it moves, and so never held for a
// backup that cannot run (issue 1945, live on 2026-10-25: redis and searxng on ace were held: "redis declares nothing to
// back up on this machine"). What node-backup.backed-up answers decides it.
func TestAModuleWithNothingToBackUpIsNotBackedUp(t *testing.T) {
for _, c := range []struct {
said string
nothing bool
}{
{`[{"module":"postgres","runs":1,"paths":["/var/lib/postgres"]}]`, true},
{`[{"module":"redis","runs":0,"paths":null,"data":[{"item":"cache"}]}]`, true},
{`[]`, true},
{`[{"module":"redis","runs":1,"paths":["/var/lib/redis"]}]`, false},
} {
nothing, ok := declaresNothingToBackUp(json.RawMessage(c.said), "redis")
if !ok || nothing != c.nothing {
t.Errorf("%s: nothing %v (read %v), want %v", c.said, nothing, ok, c.nothing)
}
}
if _, ok := declaresNothingToBackUp(json.RawMessage(`{"error":"x"}`), "redis"); ok {
t.Error("an answer that is not the list was read")
}
}
+34
View File
@@ -0,0 +1,34 @@
package main
import (
"context"
"sync/atomic"
"testing"
"time"
)
// While a container waits, a pass is asked for every so often, not at some long interval; with nothing waiting, none
// (novox/hq ADR 0335 §2, ADR 0360 §11).
func TestAWaitIsAskedAgainSoon(t *testing.T) {
if BusyAskEvery > time.Minute {
t.Fatalf("a waiting container is asked every %s", BusyAskEvery)
}
var asked atomic.Int32
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
busyWaiting.Store(false)
t.Cleanup(func() { busyWaiting.Store(false) })
go busyPasses(ctx, func() { asked.Add(1) }, 10*time.Millisecond)
time.Sleep(100 * time.Millisecond)
if n := asked.Load(); n != 0 {
t.Fatalf("%d passes were asked with nothing waiting", n)
}
busyWaiting.Store(true)
deadline := time.Now().Add(2 * time.Second)
for asked.Load() == 0 && time.Now().Before(deadline) {
time.Sleep(10 * time.Millisecond)
}
if asked.Load() == 0 {
t.Fatal("no pass was asked while a container waits")
}
}
+33
View File
@@ -0,0 +1,33 @@
package main
import (
"encoding/json"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// A composition that names a file the provisioner fills without saying it is filled would have this node-engine write
// that file empty: its apply is refused, loudly, before anything is written (issue 1945, live on 2026-10-25 at 01:22,
// when ace, g14 and shanks wrote claude-code's, node-env's and power's files empty).
func TestAFileItCannotProduceIsNeverWrittenEmpty(t *testing.T) {
unit := json.RawMessage(`{"id":"power.after-boot-unit","type":"file","path":"/etc/systemd/system/mesh-power-after-boot.service","mode":"0644","owner-at-runtime":"owned"}`)
c := delivery.Composed{Node: "ace", Resources: []json.RawMessage{unit}}
d, err := declarationOf(c)
if err != nil {
t.Fatal(err)
}
if err := refuseEmptyWrites(d); err == nil || !strings.Contains(err.Error(), "power.after-boot-unit") {
t.Fatalf("a file the provisioner fills, not filled here: %v", err)
}
c.Filled = []string{"/etc/systemd/system/mesh-power-after-boot.service"}
if d, _ = declarationOf(c); refuseEmptyWrites(d) != nil {
t.Fatalf("a filled file was refused: %v", refuseEmptyWrites(d))
}
plain := delivery.Composed{Node: "ace", Resources: []json.RawMessage{json.RawMessage(
`{"id":"x.f","type":"file","path":"/etc/x","mode":"0644","content":"x"}`)}}
if d, _ = declarationOf(plain); refuseEmptyWrites(d) != nil {
t.Fatalf("a file with its content was refused: %v", refuseEmptyWrites(d))
}
}
-77
View File
@@ -1,77 +0,0 @@
package main
import (
"context"
"crypto/ed25519"
"encoding/json"
"errors"
"os"
"path/filepath"
"testing"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/store"
)
// **A declaration from an older lease epoch is refused before anything is applied** (novox/hq to-be
// 45 §6, ADR 0227 rule 2). The controller that sent it lost its lease and goes on sending; this node
// applied the holder's. Refused on the host's own apply path — the one the queue's worker runs — before
// the machine is read or touched, with a report naming what was refused and what is held, and the
// kept declaration left as it was.
func TestADeclarationFromAnOlderEpochIsRefusedBeforeAnythingIsApplied(t *testing.T) {
dir := t.TempDir()
opts := options{state: filepath.Join(dir, "state.json")}
_, private, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
target := filepath.Join(dir, "a.conf")
declared := func(epoch, sequence int64) store.Declared {
body, err := json.Marshal(map[string]any{"declaration": 1, "epoch": epoch, "sequence": sequence,
"resources": []any{map[string]any{"id": "a", "type": "file", "path": target, "content": "x\n"}}})
if err != nil {
t.Fatal(err)
}
return store.Declared{Declaration: body, Signature: ed25519.Sign(private, body)}
}
held := declared(57, 3)
if err := store.SaveDeclared(store.DeclaredPath(opts.state), held); err != nil {
t.Fatal(err)
}
stale := declared(41, 12)
report := applyAndKeep(context.Background(), opts, stale.Declaration, &stale, nil, nil)
if report.OlderThan == nil || *report.OlderThan != (link.Order{Epoch: 57, Sequence: 3}) ||
report.Order != (link.Order{Epoch: 41, Sequence: 12}) || report.Declared != digestOf(stale.Declaration) ||
report.Refused == "" {
t.Fatalf("the refusal does not name what was refused and what is held: %+v", report)
}
if _, err := os.Stat(target); !errors.Is(err, os.ErrNotExist) {
t.Fatal("the refused declaration touched the machine")
}
kept, err := store.ReadDeclared(store.DeclaredPath(opts.state))
if err != nil || string(kept.Declaration) != string(held.Declaration) {
t.Fatal("the refused declaration replaced what this node kept")
}
}
// The queue's counts are kept beside the state and read back by the next host; unreadable is an
// error, never zero.
func TestTheNumbersSurviveTheHost(t *testing.T) {
state := filepath.Join(t.TempDir(), "state.json")
if sequence, refused, err := numbersBeside(state).Read(); err != nil || sequence != 0 || refused != 0 {
t.Fatalf("a node that never reported read %d, %d, %v", sequence, refused, err)
}
if err := numbersBeside(state).Save(41, 2); err != nil {
t.Fatal(err)
}
if sequence, refused, err := numbersBeside(state).Read(); err != nil || sequence != 41 || refused != 2 {
t.Fatalf("read back %d, %d, %v", sequence, refused, err)
}
if err := os.WriteFile(store.NumbersPath(state), []byte(`{"report_sequence":`), 0o600); err != nil {
t.Fatal(err)
}
if _, _, err := numbersBeside(state).Read(); err == nil {
t.Fatal("unreadable numbers were read as zero")
}
}
+116
View File
@@ -0,0 +1,116 @@
package main
import (
"context"
"fmt"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-host/internal/link"
)
// aFilesLink is a link that serves objects by name, or fails every read with err.
type aFilesLink struct {
objects map[string][]byte
err error
}
func (a aFilesLink) FileObject(_ context.Context, name string, _ int64) ([]byte, error) {
if a.err != nil {
return nil, a.err
}
if b, ok := a.objects[name]; ok {
return b, nil
}
return nil, os.ErrNotExist
}
func referenceDeclaration(path string, body []byte) []byte {
return []byte(`{"declaration":1,"resources":[{"id":"m.big","type":"file","path":"` + path + `","digest":"` +
declaration.DigestOf(body) + `","size":` + strconv.Itoa(len(body)) + `}]}`)
}
// 1695/AC1 and AC2, through the whole apply: a reference is fetched, verified and written; a differing object
// refuses the declaration whole and writes nothing.
func TestADeclarationWithAFileByDigestAppliesFromTheObject(t *testing.T) {
unansweringPacman(t)
body := []byte(strings.Repeat("a line of a big file\n", 4000))
name, _ := declaration.ObjectName(declaration.DigestOf(body))
dir := t.TempDir()
target := filepath.Join(dir, "big")
opts := options{state: filepath.Join(dir, "state.json"), out: &strings.Builder{}}
raw := referenceDeclaration(target, body)
bad := append([]byte(nil), body...)
bad[10] ^= 1
ctx := link.WithFiles(context.Background(), aFilesLink{objects: map[string][]byte{name: bad}})
report := applyAndKeep(ctx, opts, raw, nil, nil, nil)
if report.Refused == "" || !strings.Contains(report.Refused, "m.big") {
t.Errorf("1695/AC2: a differing object did not refuse the declaration: %+v", report)
}
if _, err := os.Stat(target); err == nil {
t.Error("1695/AC2: a file was written from an object that differs")
}
if failedWith(report.Refused).Transient {
t.Errorf("an object that differs is a real failure, and was taken as the bus not answering: %s", report.Refused)
}
// The bus answers no JetStream request while it reloads its accounts (2026-10-25 23:04:24, mesh-vault on the bus
// node): the failure is transient, so the agent tries it again, and the detail kept is its cause, the tail.
noResponders := fmt.Errorf("fetching the object 2bd98ebd of this node's own object bucket: reading the chunks of "+
"object %s: %w", strings.Repeat("x", 300), nats.ErrNoResponders)
ctx = link.WithFiles(context.Background(), aFilesLink{err: noResponders})
report = applyAndKeep(ctx, opts, raw, nil, nil, nil)
failed := failedWith("a file of this composition could not be had by its digest: file mesh-vault.kept: " + report.Refused)
if !failed.Transient || !strings.HasSuffix(failed.Detail, nats.ErrNoResponders.Error()) || len(failed.Detail) > 310 ||
!strings.HasPrefix(failed.Detail, "a file of this composition") {
t.Errorf("a fetch that met no responders: transient %v, detail %q (%d bytes); want transient, its head and its cause",
failed.Transient, failed.Detail, len(failed.Detail))
}
ctx = link.WithFiles(context.Background(), aFilesLink{objects: map[string][]byte{name: body}})
report = applyAndKeep(ctx, opts, raw, nil, nil, nil)
if report.Refused != "" {
t.Fatalf("1695/AC1: refused: %s", report.Refused)
}
got, err := os.ReadFile(target)
if err != nil || string(got) != string(body) {
t.Errorf("1695/AC1: the file holds %d bytes (%v), want %d", len(got), err, len(body))
}
}
// 1695/AC8: the hand-run `apply <file>` and `reconcile` have no bus: a reference whose file here verifies is applied
// as it is, any other refuses and writes nothing — never an empty file.
func TestTheHandApplyNeverWritesAReferenceEmpty(t *testing.T) {
unansweringPacman(t)
body := []byte(strings.Repeat("a line of a big file\n", 4000))
dir := t.TempDir()
target := filepath.Join(dir, "big")
opts := options{state: filepath.Join(dir, "state.json"), out: &strings.Builder{}}
raw := referenceDeclaration(target, body)
d, err := declaration.ParseFileTrusted(raw)
if err != nil {
t.Fatal(err)
}
if err := runApply(context.Background(), opts, d, raw, fromFile); err == nil || !strings.Contains(err.Error(), "digest") {
t.Errorf("1695/AC8: a reference with no file and no bus was not refused: %v", err)
}
if _, err := os.Stat(target); err == nil {
t.Error("1695/AC8: a file was written for a reference that could not be had")
}
if err := os.WriteFile(target, body, 0o644); err != nil {
t.Fatal(err)
}
d, _ = declaration.ParseFileTrusted(raw)
if err := runApply(context.Background(), opts, d, raw, fromFile); err != nil && !strings.Contains(err.Error(), "applied only in part") {
t.Fatalf("1695/AC8: %v", err)
}
if got, _ := os.ReadFile(target); string(got) != string(body) {
t.Errorf("1695/AC8: the file holds %d bytes, want %d", len(got), len(body))
}
}
+95
View File
@@ -0,0 +1,95 @@
package main
import (
"context"
"crypto/ed25519"
"errors"
"fmt"
"path/filepath"
"sync/atomic"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/filling"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
// serveFilling writes filled files on the provisioner client's request for as long as this engine serves (novox/hq
// ADR 0349 §4). A socket that cannot be served is said once, and this engine goes on applying.
func serveFilling(ctx context.Context, opts options, signer ed25519.PublicKey, say func(string)) {
e := fillingEngine(opts, signer, say)
fillingNow.Store(e)
defer fillingNow.CompareAndSwap(e, nil)
if err := filling.Serve(ctx, filling.Socket, e, say); err != nil {
say("filled files are not written on request on this machine: " + err.Error())
}
}
// fillingNow is the filling this engine serves, for knockFilling; nil while none is served.
var fillingNow atomic.Pointer[filling.Engine]
// knockFilling tells the provisioner's client to fill again what the filling refused or failed and the declaration
// applied now may let through (filling.Engine.Knock), once a module's apply or removal settled (agent.Env.Settled).
// What it told, or why it could not, is said.
func knockFilling(module string, say func(string)) {
e := fillingNow.Load()
if e == nil {
return
}
told, err := e.Knock(filling.Socket)
switch {
case err != nil:
say(fmt.Sprintf("after %s settled: %v", module, err))
case told != "":
say(fmt.Sprintf("after %s settled, the provisioner's client is told to fill again: %s", module, told))
}
}
// fillingEngine is the write of a filled file as this engine does it: the declaration it keeps, verified, read under
// the apply's lock — the same two locks every apply takes, the process's and the one beside the state — and the file
// written through the apply (apply.WriteFilled), its outcome recorded in the node's state as an apply's is.
func fillingEngine(opts options, signer ed25519.PublicKey, say func(string)) *filling.Engine {
return &filling.Engine{
Declared: func() (*declaration.Declaration, error) {
return declaredNow(opts.state, signer)
},
UIDOf: filling.UIDOf,
Lock: func() (func(), error) {
applying.Lock()
unlock, err := store.Lock(opts.state, nil)
if err != nil {
applying.Unlock()
return nil, err
}
return func() { unlock(); applying.Unlock() }, nil
},
Place: func(ctx context.Context, d *declaration.Declaration, path string, content []byte) (string, string, error) {
built, err := system.For(builtFor)
if err != nil {
return "", "", err
}
known, err := store.Load(opts.state)
if err != nil {
return "", "", err
}
report, updated, writeErr := apply.WriteFilled(ctx, built, d, known, path, content, apply.ExecRunner,
announceOr(say), apply.KeepIn(filepath.Dir(opts.state)), apply.WindowsIn(filepath.Dir(opts.state)))
// Saved whichever way it went, as an apply's: what was written is on the machine either way.
if err := store.Save(opts.state, updated); err != nil {
return "", "", errors.Join(writeErr, errors.New("written, and the node's state could not be saved: "+
err.Error()))
}
action, digest := "", ""
for _, o := range report.Outcomes {
if o.Target == path && o.Type == string(declaration.TypeFile) {
action = o.Action
}
}
if rec, ok := updated.At(string(declaration.TypeFile), path); ok {
digest = rec.Wrote
}
return action, digest, writeErr
},
}
}
+105
View File
@@ -0,0 +1,105 @@
package main
import (
"context"
"crypto/ed25519"
"encoding/json"
"os"
"os/user"
"path/filepath"
"testing"
"time"
"github.com/novox/mesh-host/internal/filling"
"github.com/novox/mesh-host/internal/store"
)
// Defends novox/hq ADR 0349 §4 (issue 668): a write of a filled file and an apply are one at a time — the write
// takes the lock the apply holds, and writes only once the apply let it go — and the write is recorded in the
// node's state as an apply's outcome is.
func TestAWriteAndAnApplyOfTheSameFileSerialise(t *testing.T) {
was := builtFor
builtFor = "arch"
t.Cleanup(func() { builtFor = was })
me, err := user.Current()
if err != nil {
t.Fatal(err)
}
if me.Uid == "0" {
t.Skip("run as root, whose account is never the provisioner's")
}
dir := t.TempDir()
opts := options{state: filepath.Join(dir, "state.json")}
public, private, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
path := filepath.Join(dir, "m.conf")
raw, err := json.Marshal(map[string]any{"declaration": 1, "provisioner_account": me.Username,
"filled_files": []string{path}, "resources": []any{map[string]any{"id": "m.conf", "type": "file",
"path": path, "content": "from the declaration\n", "owner-at-runtime": "owned", "owner": me.Username}}})
if err != nil {
t.Fatal(err)
}
if err := store.SaveDeclared(store.DeclaredPath(opts.state),
store.Declared{Declaration: raw, Signature: ed25519.Sign(private, raw)}); err != nil {
t.Fatal(err)
}
e := fillingEngine(opts, public, func(string) {})
uid := uint32(os.Getuid())
if err := e.Accepts(uid); err != nil {
t.Fatal(err)
}
// An apply holds the lock.
applying.Lock()
done := make(chan filling.Answer)
go func() {
done <- e.Write(context.Background(), uid, filling.Request{Path: path, Content: []byte("filled\n")})
}()
select {
case a := <-done:
applying.Unlock()
t.Fatalf("written while an apply held the lock: %+v", a)
case <-time.After(200 * time.Millisecond):
}
if _, err := os.Stat(path); err == nil {
applying.Unlock()
t.Fatal("the file was written while an apply held the lock")
}
applying.Unlock()
a := <-done
if a.Refused != "" || a.Failed != "" || a.Action != "created" {
t.Fatalf("the write after the apply: %+v", a)
}
got, err := os.ReadFile(path)
if err != nil || string(got) != "filled\n" {
t.Fatalf("the file holds %q (%v)", got, err)
}
known, err := store.Load(opts.state)
if err != nil {
t.Fatal(err)
}
if rec, ok := known.Find("m.conf"); !ok || rec.Wrote != a.Digest || rec.Target != path {
t.Errorf("the write is not recorded in the node's state: %+v, answered %q", rec, a.Digest)
}
// And the cross-process lock beside the state too: a write waits for a `reconcile` run by hand.
unlock, err := store.Lock(opts.state, nil)
if err != nil {
t.Fatal(err)
}
go func() {
done <- e.Write(context.Background(), uid, filling.Request{Path: path, Content: []byte("again\n")})
}()
select {
case a := <-done:
unlock()
t.Fatalf("written while another process held the apply's lock: %+v", a)
case <-time.After(200 * time.Millisecond):
}
unlock()
if a := <-done; a.Action != "updated" {
t.Fatalf("the second write: %+v", a)
}
}
-40
View File
@@ -1,40 +0,0 @@
package main
import (
"context"
"testing"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
// **The generation applied is kept, and a put-back never lowers it**: after a put-back of generation 5
// over 7, a send from generation 6 is still older than what this machine applied.
func TestThePutBackDoesNotLowerTheGenerationApplied(t *testing.T) {
built, err := system.For("arch")
if err != nil || built.Confirm(context.Background(), apply.ExecRunner) != nil {
t.Skip("this machine is not one these tests can apply on")
}
was := builtFor
builtFor = "arch"
t.Cleanup(func() { builtFor = was })
f := machineThatApplied(t, 7)
newer := f.declared(t, map[string]any{"sequence": 9, "generation": 8}, "docker.conf", f.stays)
if r := applyAndKeep(context.Background(), f.opts, newer.Declaration, &newer, nil, nil); r.Refused != "" {
t.Fatalf("a newer generation was refused: %s", r.Refused)
}
back := f.declared(t, map[string]any{"sequence": 10, "generation": 5, "put_back": true}, "docker.conf", f.stays)
if r := applyAndKeep(context.Background(), f.opts, back.Declaration, &back, nil, nil); r.Refused != "" {
t.Fatalf("a put-back was refused: %s", r.Refused)
}
state, err := store.Load(f.opts.state)
if err != nil || state.Generation != 8 {
t.Fatalf("the generation applied is %d after a put-back of 5 over 8 (%v)", state.Generation, err)
}
stale := f.declared(t, map[string]any{"sequence": 11, "generation": 7}, "docker.conf", f.stays)
if r := applyAndKeep(context.Background(), f.opts, stale.Declaration, &stale, nil, nil); r.OlderGeneration == nil {
t.Fatalf("a send from generation 7 after 8 was applied because a put-back came between: %+v", r)
}
}
-170
View File
@@ -1,170 +0,0 @@
package main
import (
"context"
"crypto/ed25519"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/store"
)
// The assignment generation a declaration was composed from (novox/hq issue 234).
//
// On 2026-10-04 a declaration newer in sequence than every other, and composed from a view of the
// assignments older than the one this machine had applied, named four fewer modules — and the host,
// ordering by sequence alone, undeclared all four. The sequence orders arrival; it cannot tell a later
// send carrying earlier content. The generation can: the node-engine refuses a declaration composed from
// an older generation than the one it applied, before anything is read from the machine.
//
// These read what the report says through its JSON, as the controller does, so they run against a
// node-engine that has never heard of a generation and fail there for that reason.
// generationFixture is a machine that applied generation 7: a file the mesh declared, kept in its state,
// and the declaration it kept — signed — that named it.
type generationFixture struct {
opts options
private ed25519.PrivateKey
kept store.Declared
// stays is the file the older generation's declaration does not name.
stays string
}
func machineThatApplied(t *testing.T, generation int64) generationFixture {
t.Helper()
dir := t.TempDir()
opts := options{state: filepath.Join(dir, "state.json")}
_, private, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
stays := filepath.Join(dir, "docker.conf")
if err := os.WriteFile(stays, []byte("kept\n"), 0o644); err != nil {
t.Fatal(err)
}
// The state as a node-engine that applied generation 7 keeps it: the file it owns, and the highest
// generation it applied. Written as JSON, so the fixture is the same bytes for any build under test.
state := map[string]any{"resources": []any{map[string]any{"id": "docker.conf", "type": "file",
"origin": store.OriginDeclared, "target": stays}}, "generation": generation}
raw, err := json.Marshal(state)
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(opts.state, raw, 0o600); err != nil {
t.Fatal(err)
}
f := generationFixture{opts: opts, private: private, stays: stays}
f.kept = f.declared(t, map[string]any{"sequence": 3, "generation": generation}, "docker.conf", stays)
if err := store.SaveDeclared(store.DeclaredPath(opts.state), f.kept); err != nil {
t.Fatal(err)
}
return f
}
// declared is a signed declaration with the envelope's keys given and one file.
func (f generationFixture) declared(t *testing.T, envelope map[string]any, id, path string) store.Declared {
t.Helper()
body := map[string]any{"declaration": 1,
"resources": []any{map[string]any{"id": id, "type": "file", "path": path, "content": "x\n"}}}
for k, v := range envelope {
body[k] = v
}
raw, err := json.Marshal(body)
if err != nil {
t.Fatal(err)
}
return store.Declared{Declaration: raw, Signature: ed25519.Sign(f.private, raw)}
}
// reportJSON is a report as the controller reads it.
func reportJSON(t *testing.T, r link.Report) map[string]any {
t.Helper()
raw, err := json.Marshal(r)
if err != nil {
t.Fatal(err)
}
var out map[string]any
if err := json.Unmarshal(raw, &out); err != nil {
t.Fatal(err)
}
return out
}
// **A newer sequence from an older generation is refused, and nothing is undeclared** (issue 234).
func TestANewerSequenceFromAnOlderGenerationIsRefusedAndNothingIsUndeclared(t *testing.T) {
f := machineThatApplied(t, 7)
other := filepath.Join(filepath.Dir(f.opts.state), "sudo.conf")
stale := f.declared(t, map[string]any{"sequence": 9, "generation": 6}, "sudo.conf", other)
report := applyAndKeep(context.Background(), f.opts, stale.Declaration, &stale, nil, nil)
said := reportJSON(t, report)
older, _ := said["older_generation"].(map[string]any)
if older == nil || older["generation"] != float64(6) || older["applied"] != float64(7) {
t.Fatalf("the refusal does not name the generation refused and the one applied: %v", said)
}
if report.Refused == "" || !strings.Contains(report.Refused, "generation 6") ||
!strings.Contains(report.Refused, "generation 7") {
t.Errorf("the refusal is not said in words naming both generations: %q", report.Refused)
}
if report.Declared != digestOf(stale.Declaration) || report.Sequence != 9 {
t.Errorf("the refusal does not name the declaration refused, so the mesh cannot name who sent it: %+v", report)
}
if _, err := os.Stat(f.stays); err != nil {
t.Fatalf("a refused declaration undeclared what the machine applied: %v", err)
}
if _, err := os.Stat(other); err == nil {
t.Error("the refused declaration touched the machine")
}
state, err := store.Load(f.opts.state)
if err != nil {
t.Fatal(err)
}
if _, owned := state.Find("docker.conf"); !owned {
t.Error("the refused declaration took a resource out of what this machine owns")
}
kept, err := store.ReadDeclared(store.DeclaredPath(f.opts.state))
if err != nil || string(kept.Declaration) != string(f.kept.Declaration) {
t.Error("the refused declaration replaced what this node kept, so the next reconcile would apply it")
}
}
// refusedForItsGeneration says a report refused its declaration for the generation it came from.
func refusedForItsGeneration(t *testing.T, r link.Report) bool {
t.Helper()
return reportJSON(t, r)["older_generation"] != nil || strings.Contains(r.Refused, "generation")
}
// refusedAsUnread says a report refused its declaration as one this node-engine cannot read.
func refusedAsUnread(r link.Report) bool { return strings.Contains(r.Refused, "not a declaration") }
// **A gate's put-back is applied, whatever generation it carries**: it carries the generation of what it
// puts back, and says it is a put-back (issue 234).
func TestAPutBackFromAnOlderGenerationIsNotRefused(t *testing.T) {
f := machineThatApplied(t, 7)
back := f.declared(t, map[string]any{"sequence": 9, "generation": 5, "put_back": true}, "docker.conf", f.stays)
report := applyAndKeep(context.Background(), f.opts, back.Declaration, &back, nil, nil)
if refusedForItsGeneration(t, report) || refusedAsUnread(report) {
t.Fatalf("a put-back was refused: %s", report.Refused)
}
}
// **A declaration that carries no generation is applied as it always was**: an older controller sends
// none, and a node-engine that refused it would strand every machine the moment the controller is older.
func TestADeclarationWithoutAGenerationIsNotRefusedForOne(t *testing.T) {
f := machineThatApplied(t, 7)
older := f.declared(t, map[string]any{"sequence": 9}, "docker.conf", f.stays)
report := applyAndKeep(context.Background(), f.opts, older.Declaration, &older, nil, nil)
if refusedForItsGeneration(t, report) || refusedAsUnread(report) {
t.Fatalf("a declaration without a generation was refused: %s", report.Refused)
}
// And the same generation again is no older.
same := f.declared(t, map[string]any{"sequence": 10, "generation": 7}, "docker.conf", f.stays)
report = applyAndKeep(context.Background(), f.opts, same.Declaration, &same, nil, nil)
if refusedForItsGeneration(t, report) || refusedAsUnread(report) {
t.Fatalf("a declaration of the generation applied was refused: %s", report.Refused)
}
}
+90
View File
@@ -0,0 +1,90 @@
package main
import (
"context"
"errors"
"path/filepath"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/liveness"
)
// owedBus is a link that takes reports and statements and keeps a have bucket in memory.
type owedBus struct {
mu sync.Mutex
kv map[string][]byte
rv map[string]uint64
}
func (b *owedBus) Report(context.Context, string, []byte) error { return nil }
func (b *owedBus) Health(context.Context, string, []byte) error { return nil }
func (b *owedBus) HaveBind(context.Context) (link.HaveBucket, error) { return b, nil }
func (b *owedBus) Get(_ context.Context, key string) ([]byte, uint64, error) {
b.mu.Lock()
defer b.mu.Unlock()
v, ok := b.kv[key]
if !ok {
return nil, 0, link.ErrHaveAbsent
}
return v, b.rv[key], nil
}
func (b *owedBus) Set(_ context.Context, key string, value []byte, revision uint64) error {
b.mu.Lock()
defer b.mu.Unlock()
if b.rv[key] != revision {
return errors.New("wrong last sequence")
}
b.kv[key], b.rv[key] = value, revision+1
return nil
}
func (b *owedBus) have(key string) (delivery.Have, bool) {
b.mu.Lock()
defer b.mu.Unlock()
h, err := delivery.DecodeHave(b.kv[key])
return h, err == nil
}
// A module with nothing long-running (xorg) is ok within one look of its apply, not at the five-minute statement: the
// facts look inside the apply runs while it is applying and judges nothing, and the running write then owes a look,
// which the health loop makes at once (review of fix 15, B1).
func TestAModuleWithNothingLongRunningIsOkWithinALookOfItsApply(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
bus := &owedBus{kv: map[string][]byte{}, rv: map[string]uint64{}}
queue := &link.Session{Membership: link.Membership{Node: "n1"}, Timeout: time.Second}
queue.Attach(ctx, bus)
j, err := liveness.Open(filepath.Join(t.TempDir(), "liveness.json"), nil)
if err != nil {
t.Fatal(err)
}
fact := link.ModuleFact{Module: "xorg", FP: "src2:" + strings.Repeat("a", 64), Composed: "sha256:" + strings.Repeat("b", 64)}
applying := fact
applying.Phase = string(delivery.PhaseApplying)
queue.NoteModule(ctx, applying)
go judgeWhatRuns(ctx, j, queue, func(string) {}, nil, nil)
// The facts look inside the apply, while the module is still applying.
queue.SayFacts(ctx, link.Report{Health: &link.Health{Contract: link.LivenessContract, At: time.Now().UTC()}})
if h, _ := bus.have("n1.xorg"); h.Health != delivery.HealthUnknown {
t.Fatalf("judged while applying: %+v", h)
}
queue.NoteModule(ctx, fact)
deadline := time.Now().Add(2 * time.Second) // well within one look (liveness.LookEvery, 15 s)
for {
h, _ := bus.have("n1.xorg")
if h.Phase == delivery.PhaseRunning && h.Health == delivery.HealthOK {
return
}
if time.Now().After(deadline) {
t.Fatalf("not ok within a look of its apply: %+v", h)
}
time.Sleep(10 * time.Millisecond)
}
}
+88
View File
@@ -0,0 +1,88 @@
package main
import (
"context"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/liveness"
)
// heldBy is the containers an open maintenance window holds still, by runtime name (ADR 0189), at now.
func heldBy(windows *apply.Windows) func(time.Time) map[string]bool {
return func(now time.Time) map[string]bool {
held := map[string]bool{}
for _, w := range windows.Open(now) {
for _, name := range w.Holds {
held[name] = true
}
}
return held
}
}
// inactiveIn is what the agent asks at its start of a module it believes runs (ADR 0568 §1): whether the machine shows a
// long-running resource the composition declares running not active, read from the service manager and the container
// runtime and from no record. A resource held by an open maintenance window, or waiting for a library, is not counted;
// one that cannot be read is not counted either (unknown is not inactive), and neither is one the manager is restarting.
func inactiveIn(rt liveness.Runtime, held func(time.Time) map[string]bool) func(context.Context, delivery.Composed) bool {
return func(ctx context.Context, c delivery.Composed) bool {
d, err := declarationOf(c)
if err != nil {
return false
}
var holding map[string]bool
if held != nil {
holding = held(time.Now())
}
return anyInactive(ctx, rt, liveness.LongRunning(d, withWaiting(nil, apply.WaitingOnLibrary(d))), holding)
}
}
// anyInactive reads each group of resources once (the containers, each service manager's units) and says whether any is
// not active.
func anyInactive(ctx context.Context, rt liveness.Runtime, resources []liveness.Resource, held map[string]bool) bool {
type group struct {
kind, scope, user string
targets []string
}
var groups []*group
at := map[string]*group{}
for _, r := range resources {
if r.Kind == liveness.KindContainer && held[r.Target] {
continue
}
g := &group{kind: r.Kind, scope: r.Scope, user: r.User}
if r.Kind == liveness.KindContainer {
g.scope, g.user = "", ""
}
key := g.kind + "/" + g.scope + "/" + g.user
if r.Kind != liveness.KindContainer {
key = "units/" + g.scope + "/" + g.user
}
if at[key] == nil {
at[key] = g
groups = append(groups, g)
}
at[key].targets = append(at[key].targets, r.Target)
}
for _, g := range groups {
var got map[string]liveness.Observed
var err error
if g.kind == liveness.KindContainer {
got, err = rt.Containers(ctx, g.targets)
} else {
got, err = rt.Units(ctx, g.scope, g.user, g.targets)
}
if err != nil {
continue
}
for _, t := range g.targets {
if o := got[t]; !o.Found || !(o.Running || o.Restarting) {
return true
}
}
}
return false
}
+89
View File
@@ -0,0 +1,89 @@
package main
import (
"context"
"encoding/json"
"errors"
"testing"
"git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-host/internal/liveness"
)
// 1957/AC2 and AC4 (unit; ADR 0568 §2, §4): the judge's look acts on a resource that goes down, once per edge. A repeated
// statement acts nothing, the edge clears when the resource is healthy again, and a resource held, restarting, starting or
// missing (the other path's) is no edge.
func testOnlyAResourceNewlyDownIsAnEdge(t *testing.T) {
state := func(id, s, reason string, missing bool) liveness.State {
return liveness.State{Resource: liveness.Resource{Module: "dnsmasq", ID: id}, State: s, Reason: reason, Missing: missing}
}
down := state("dnsmasq.unit", liveness.Unhealthy, liveness.ReasonDown, false)
steps := []struct {
name string
st []liveness.State
want []string
}{
{"down: an edge", []liveness.State{down}, []string{"dnsmasq"}},
{"said again while down: none", []liveness.State{down}, nil},
{"healthy again: clears", []liveness.State{state("dnsmasq.unit", liveness.Healthy, "", false)}, nil},
{"down again: an edge", []liveness.State{down}, []string{"dnsmasq"}},
{"held by a window: none", []liveness.State{state("dnsmasq.unit", liveness.Held, "", false)}, nil},
{"restarting: none", []liveness.State{state("dnsmasq.unit", liveness.Unhealthy, liveness.ReasonRestarting, false)}, nil},
{"starting: none", []liveness.State{state("dnsmasq.unit", liveness.Starting, "", false)}, nil},
{"missing: the other path's", []liveness.State{state("dnsmasq.unit", liveness.Unhealthy, liveness.ReasonDown, true)}, nil},
}
var last map[string]bool
for _, s := range steps {
var got []string
got, last = newlyInactive(last, liveness.Statement{Resources: s.st})
if len(got) != len(s.want) || (len(got) == 1 && got[0] != s.want[0]) {
t.Fatalf("%s: acted on %v, want %v", s.name, got, s.want)
}
}
}
type fakeRuntime struct {
units map[string]liveness.Observed
containers map[string]liveness.Observed
err error
}
func (f fakeRuntime) Containers(_ context.Context, _ []string) (map[string]liveness.Observed, error) {
return f.containers, f.err
}
func (f fakeRuntime) Units(_ context.Context, _, _ string, _ []string) (map[string]liveness.Observed, error) {
return f.units, f.err
}
// 1957/AC1 (unit; ADR 0568 §1): the machine's facts say a declared-running resource is not active when its unit is stopped
// or unknown to the manager; everything up, a restart in progress, a read that failed and a container held by a
// maintenance window are not.
func testTheMachineShowsAResourceNotActive(t *testing.T) {
svc := []byte(`{"id":"dnsmasq.unit","type":"service","unit":"dnsmasq.service","state":"running","module":"dnsmasq"}`)
ctr := []byte(`{"id":"dnsmasq.box","type":"container","name":"box","image":"x@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","module":"dnsmasq"}`)
up := liveness.Observed{Found: true, Running: true}
for _, c := range []struct {
name string
rt fakeRuntime
held map[string]bool
want bool
}{
{"all up", fakeRuntime{units: map[string]liveness.Observed{"dnsmasq.service": up}, containers: map[string]liveness.Observed{"box": up}}, nil, false},
{"unit inactive", fakeRuntime{units: map[string]liveness.Observed{"dnsmasq.service": {Found: true}}, containers: map[string]liveness.Observed{"box": up}}, nil, true},
{"unit unknown to the manager", fakeRuntime{containers: map[string]liveness.Observed{"box": up}}, nil, true},
{"unit restarting", fakeRuntime{units: map[string]liveness.Observed{"dnsmasq.service": {Found: true, Restarting: true}}, containers: map[string]liveness.Observed{"box": up}}, nil, false},
{"container not running", fakeRuntime{units: map[string]liveness.Observed{"dnsmasq.service": up}, containers: map[string]liveness.Observed{"box": {Found: true}}}, nil, true},
{"container held by a window", fakeRuntime{units: map[string]liveness.Observed{"dnsmasq.service": up}, containers: map[string]liveness.Observed{"box": {Found: true}}}, map[string]bool{"box": true}, false},
{"unread", fakeRuntime{err: errors.New("the service manager could not be read")}, nil, false},
} {
t.Run(c.name, func(t *testing.T) {
d, err := declarationOf(delivery.Composed{Node: "n1", Module: "dnsmasq", Resources: []json.RawMessage{svc, ctr}})
if err != nil {
t.Fatal(err)
}
if got := anyInactive(context.Background(), c.rt, liveness.LongRunning(d, nil), c.held); got != c.want {
t.Fatalf("inactive = %v, want %v", got, c.want)
}
})
}
}
+2 -7
View File
@@ -5,7 +5,6 @@ import (
"fmt"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/store"
)
@@ -13,16 +12,12 @@ import (
// judgeDirectories judges every directory of the declaration this node keeps, read-only (novox/hq issue 358): which
// the walk that follows no link another account could make would refuse. Nothing is made, chmodded or chowned.
func judgeDirectories(statePath string) (string, error) {
kept, err := store.ReadDeclared(store.DeclaredPath(statePath))
d, err := declaredNow(statePath, nil)
if errors.Is(err, store.ErrNothingDeclared) {
return "", errors.New("this node keeps no declaration, so it is declared no directory")
}
if err != nil {
return "", err
}
d, err := declaration.Parse(kept.Declaration)
if err != nil {
return "", fmt.Errorf("the declaration this node keeps does not read: %w", err)
return "", fmt.Errorf("what runs on this node does not read: %w", err)
}
return apply.JudgeDirectories(d), nil
}
+3 -7
View File
@@ -11,8 +11,8 @@ import (
"sync/atomic"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/store"
)
@@ -251,16 +251,12 @@ func startKeeping(ctx context.Context, say func(string)) {
// declaration this node keeps, and run beside the applies.
func accessActFrom(statePath string, timeout time.Duration) link.AccessActFunc {
return func(act link.AccessAct) (string, error) {
kept, err := store.ReadDeclared(store.DeclaredPath(statePath))
d, err := declaredNow(statePath, nil)
if errors.Is(err, store.ErrNothingDeclared) {
return "", errors.New("this node keeps no declaration, so it is declared no library")
}
if err != nil {
return "", err
}
d, err := declaration.Parse(kept.Declaration)
if err != nil {
return "", fmt.Errorf("the declaration this node keeps does not read: %w", err)
return "", fmt.Errorf("what runs on this node does not read: %w", err)
}
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
+1 -1
View File
@@ -8,8 +8,8 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/link"
)
-107
View File
@@ -1,107 +0,0 @@
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-host/internal/link"
)
// Defends novox/hq issue 160: a machine nobody pushed to still says what it is made of once a day, so the
// newest of it the mesh holds is no older than a day. Before, the reconcile said it only with something
// else (a hold, a firewall, an outward link changed), and a converged machine's picture was as old as the
// last delivery to it.
func TestAMachineNobodyPushedToSaysWhatItIsOnceADay(t *testing.T) {
at := time.Date(2026, 10, 12, 9, 0, 0, 0, time.UTC)
w := &adoptionWatch{now: func() time.Time { return at }}
quiet := link.Report{Outward: []string{"wlan0"},
Profile: map[string]any{"machine": map[string]any{"observed_at": at}}}
if !w.news(quiet) {
t.Fatal("the first reconcile after the engine started did not say what the machine is")
}
w.said(quiet)
for _, after := range []time.Duration{5 * time.Minute, 12 * time.Hour, 22*time.Hour + 55*time.Minute} {
at = time.Date(2026, 10, 12, 9, 0, 0, 0, time.UTC).Add(after)
if w.news(quiet) {
t.Errorf("%s after the mesh heard it, an unchanged reconcile was said again", after)
}
}
// The reconcile that falls past 23 hours says it, so it reaches the mesh inside the day.
at = time.Date(2026, 10, 12, 9, 0, 0, 0, time.UTC).Add(23*time.Hour + 5*time.Minute)
if !w.news(quiet) {
t.Fatal("a day on, the reconcile did not say what the machine is")
}
w.said(quiet)
at = at.Add(5 * time.Minute)
if w.news(quiet) {
t.Error("once said, it was said again at the next reconcile")
}
}
// A delivery's account carries the profile too: heard, it starts the day again.
func TestADeliveryHeardStartsTheDayAgain(t *testing.T) {
at := time.Date(2026, 10, 12, 9, 0, 0, 0, time.UTC)
w := &adoptionWatch{now: func() time.Time { return at }}
report := link.Report{Outward: []string{"eth0"}, Profile: map[string]any{"capabilities": []any{}}}
w.said(report)
at = at.Add(20 * time.Hour)
w.said(report) // a delivery's apply
at = at.Add(20 * time.Hour)
if w.news(report) {
t.Error("said again unasked 20 hours after a delivery told the mesh")
}
}
// A refusal carries nothing about the machine, so it is never said for being a day old.
func TestARefusalIsNotSaidForTheMachinesSake(t *testing.T) {
at := time.Date(2026, 10, 12, 9, 0, 0, 0, time.UTC)
w := &adoptionWatch{now: func() time.Time { return at }}
if w.machineDue(link.Report{Refused: "no"}) {
t.Error("a refusal was due for what the machine is made of")
}
}
// A laptop that slept still says what it is within a day of wall time (review of mesh-host#92). Go's monotonic
// clock stops while a machine is suspended, and Sub between two times that both carry a monotonic reading uses
// it, so a watch that kept one would count a day asleep as almost nothing. Simulated with a clock that first
// reads now (a monotonic reading), and after the sleep reads the wall time a day on: the watch must keep no
// monotonic reading, and judge the day as the wall clock says it.
func TestALaptopThatSleptStillSaysWhatItIsWithinADay(t *testing.T) {
before := time.Now() // carries a monotonic reading
asleep := before.Add(3 * time.Second)
if asleep.Sub(before) != 3*time.Second {
t.Fatal("the monotonic reading is not what this test simulates with")
}
calls := 0
w := &adoptionWatch{now: func() time.Time {
calls++
if calls == 1 {
return before
}
// The wall clock a day on, as a machine's clock reads after waking.
return asleep.Round(0).Add(24 * time.Hour)
}}
report := link.Report{Outward: []string{"wlan0"}, Profile: map[string]any{"capabilities": []any{}}}
w.said(report)
if strings.Contains(w.machineSaid.String(), "m=") {
t.Fatalf("the watch kept a monotonic reading (%s): a sleep would not count toward the day", w.machineSaid)
}
if !w.news(report) {
t.Error("a day of wall time on, a laptop that slept did not say what it is")
}
}
// And the watch's own clock never carries a monotonic reading.
func TestTheWatchClockIsWallTime(t *testing.T) {
w := &adoptionWatch{}
if s := w.clock().String(); strings.Contains(s, "m=") {
t.Errorf("the watch's clock carries a monotonic reading: %s", s)
}
w = &adoptionWatch{now: time.Now}
if s := w.clock().String(); strings.Contains(s, "m=") {
t.Errorf("a given clock's time kept its monotonic reading: %s", s)
}
}
+356 -650
View File
File diff suppressed because it is too large Load Diff
+5 -296
View File
@@ -3,7 +3,6 @@ package main
import (
"bytes"
"context"
"crypto/ed25519"
"encoding/json"
"errors"
"os"
@@ -12,10 +11,8 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/bundle"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
@@ -98,26 +95,6 @@ func TestNoCommandIsNotAnError(t *testing.T) {
}
}
func TestApplyNeedsExactlyOneDeclaration(t *testing.T) {
// `apply` takes a file where every other command takes nothing, so the leftover-argument
// rule has an exception — and an exception is where a parser stops refusing things it
// should. Both directions are checked.
if _, _, err := parseArgs([]string{"apply"}); err == nil {
t.Error("apply with no file was accepted")
}
if _, _, err := parseArgs([]string{"apply", "a.json", "b.json"}); err == nil {
t.Error("apply with two files was accepted")
}
command, opts, err := parseArgs([]string{"apply", "decl.json", "--dry-run"})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if command != "apply" || opts.file != "decl.json" || !opts.dryRun {
t.Errorf("parsed as command=%q file=%q dry-run=%v", command, opts.file, opts.dryRun)
}
}
func TestTheStateHasADocumentedDefault(t *testing.T) {
// A host that wrote its state somewhere unexpected would forget what it owns on the next
// run, and then leave everything it had applied behind forever.
@@ -135,93 +112,22 @@ func TestAFlagAfterAPositionalIsRead(t *testing.T) {
// off the front fixed the flag after the COMMAND and not the flag after its ARGUMENT: the
// standard library stops at the first non-flag argument wherever that argument is.
for _, args := range [][]string{
{"apply", "decl.json", "--dry-run", "--json"},
{"apply", "--dry-run", "decl.json", "--json"},
{"apply", "--dry-run", "--json", "decl.json"},
{"hand-over", "decl.json", "--dry-run", "--json"},
{"hand-over", "--dry-run", "decl.json", "--json"},
{"hand-over", "--dry-run", "--json", "decl.json"},
} {
command, opts, err := parseArgs(args)
if err != nil {
t.Errorf("%v: unexpected error: %v", args, err)
continue
}
if command != "apply" || opts.file != "decl.json" || !opts.dryRun || !opts.json {
if command != "hand-over" || opts.file != "decl.json" || !opts.dryRun || !opts.json {
t.Errorf("%v parsed as file=%q dry-run=%v json=%v",
args, opts.file, opts.dryRun, opts.json)
}
}
}
// Defends novox/hq ADR 0100: a reconcile on an adopted node speaks unasked only when what it holds
// or its firewall changed — which is how a predecessor still writing is caught, without a report
// every five minutes saying nothing new.
func TestAReconcileSpeaksOnlyWhenWhatIsHeldChanged(t *testing.T) {
w := &adoptionWatch{}
held := link.Report{Firewall: "ufw", Held: []link.Held{{ID: "hello-web.page"}, {ID: "hello-web.server"}}}
if !w.changed(held) {
t.Fatal("the first report of a hold was not said")
}
again := link.Report{Firewall: "ufw", Held: []link.Held{{ID: "hello-web.server"}, {ID: "hello-web.page"}}}
if w.changed(again) {
t.Error("the same holds in another order were said again")
}
rewritten := link.Report{Firewall: "ufw", Held: []link.Held{{ID: "hello-web.page", Changed: "rewritten"}, {ID: "hello-web.server"}}}
if !w.changed(rewritten) {
t.Error("a held file rewritten by something else was not said")
}
if !w.changed(link.Report{Firewall: "none", Held: rewritten.Held}) {
t.Error("a changed firewall was not said")
}
// What filters the machine is part of it (novox/hq ADR 0168): a predecessor's chain removed by
// hand, or the found firewall enabled again, is said without being asked.
filtered := link.Report{Firewall: "none", Held: rewritten.Held,
Filters: []link.Filter{{Where: "chain HAL-MESH-ONLY (iptables-legacy)", Owner: "other", Refuses: "-j DROP"}}}
if !w.changed(filtered) {
t.Error("a filter appearing was not said")
}
if !w.changed(link.Report{Firewall: "none", Held: rewritten.Held}) {
t.Error("a filter removed was not said")
}
if !w.changed(link.Report{Firewall: "none", Held: rewritten.Held, FoundFirewall: &link.FoundFirewall{Kind: "ufw", Active: true}}) {
t.Error("the found firewall coming back was not said")
}
if !worthSaying(link.Report{Filters: filtered.Filters}) {
t.Error("a report carrying only what filters the machine is not worth saying")
}
}
// What the mesh heard — a delivery's account as much as a reconcile's — counts as said: the queue
// tells the watch every report the broker took (link.Queue.Heard).
func TestWhatTheMeshHeardCountsAsSaid(t *testing.T) {
w := &adoptionWatch{}
report := link.Report{Firewall: "ufw", Held: []link.Held{{ID: "a"}}}
heard := w.said
heard(report)
if w.changed(report) {
t.Error("a reconcile repeated what the mesh had just heard")
}
}
func TestAReconcileSpeaksWhenWhatIsReachableChanged(t *testing.T) {
// The controller previews a flip from what the node last said is reachable; a port that opened
// since must reach it without waiting for the next delivery (novox/hq ADR 0100).
w := &adoptionWatch{}
before := link.Report{Firewall: "ufw", Reachable: []link.Reach{
{Protocol: "tcp", Address: "0.0.0.0", Port: 22, By: "sshd"}}}
if !w.changed(before) {
t.Fatal("the first report was not said")
}
reordered := link.Report{Firewall: "ufw", Reachable: []link.Reach{
{Protocol: "tcp", Address: "0.0.0.0", Port: 22, By: "sshd"}}}
if w.changed(reordered) {
t.Error("the same reachable set was said again")
}
opened := link.Report{Firewall: "ufw", Reachable: append(before.Reachable,
link.Reach{Protocol: "tcp", Address: "0.0.0.0", Port: 8080, By: "hello-web", Published: true, ContainerPort: 80})}
if !w.changed(opened) {
t.Error("a newly published port was not said")
}
}
// Defends the node's own record: the link and the reconcile loop both apply, and each reads the
// state, acts, and writes it back — so they must not run at the same time, or the last save loses
// what the other recorded.
@@ -270,24 +176,6 @@ func TestOnlyOneApplyRunsAtATime(t *testing.T) {
}
}
// Defends novox/hq ADR 0100: a change is counted as said only once the mesh has been told. Queued
// and lost — the link down when the reconcile spoke — it must be said again.
func TestAChangeThatNeverReachedTheMeshIsSaidAgain(t *testing.T) {
w := &adoptionWatch{}
held := link.Report{Firewall: "ufw", Held: []link.Held{{ID: "hello-web.page", Changed: "rewritten"}}}
if !w.differs(held) {
t.Fatal("the first report of a change was not new")
}
// The link was down: nothing published it, so nothing says it was said.
if !w.differs(held) {
t.Error("a change that never reached the mesh was counted as said")
}
w.said(held)
if w.differs(held) {
t.Error("a change the mesh was told was said again")
}
}
// Defends novox/hq issue 104: a declaration for the other mode than this node is in is refused at
// the point of application, whichever command delivered it, naming both — an adopted control-node
// once applied its converged genesis bundle and closed itself for forty-five minutes.
@@ -468,125 +356,6 @@ func TestADryRunChangesNothingAndListsTheActions(t *testing.T) {
}
}
// Defends novox/hq issue 104: once the mesh has told this node anything, `reconcile` holds it to
// that — never to the bundle the host carries, which genesis consumed.
func TestReconcileAfterAControllerDeclarationDoesNotReapplyTheBundle(t *testing.T) {
was := builtFor
builtFor = "arch"
t.Cleanup(func() { builtFor = was })
opts := stateWithMode(t, store.ModeAdopted)
controllerPublic, controllerPrivate, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
said := []byte(`{"declaration":1,"adoption":{"taken":[]},"resources":[{"id":"a","type":"file","path":"/tmp/a","content":"x\n"}]}`)
if err := store.SaveDeclared(store.DeclaredPath(opts.state), store.Declared{Declaration: said,
Signature: ed25519.Sign(controllerPrivate, said)}); err != nil {
t.Fatal(err)
}
// Told, and unable to prove by whom: refused, and the bundle is not applied in its place.
_, _, _, err = reconcileSource(opts)
if err == nil || !strings.Contains(err.Error(), "not applied in its place") {
t.Errorf("a node that cannot prove what it was told fell back to something: %v", err)
}
mine, err := identity.Generate("workstation")
if err != nil {
t.Fatal(err)
}
mine.Membership = identity.Membership{Broker: "198.51.100.10:5671", Fingerprint: "sha256:0",
Signer: controllerPublic, Password: "issued"}
if err := identity.Save(identity.Path(opts.state), mine); err != nil {
t.Fatal(err)
}
d, raw, from, err := reconcileSource(opts)
if err != nil {
t.Fatal(err)
}
if from != fromDeclared || !bytes.Equal(raw, said) || d.Adoption == nil {
t.Errorf("reconcile chose %d with %d bytes, not what the mesh last said", from, len(raw))
}
// And before the mesh has said anything: the bundle, as it always was. A test binary carries
// only the placeholder, and asking for it is what proves the path.
if err := os.Remove(store.DeclaredPath(opts.state)); err != nil {
t.Fatal(err)
}
_, _, _, err = reconcileSource(opts)
if !errors.Is(err, bundle.ErrEmpty) {
t.Errorf("with nothing said, reconcile did not reach for the carried bundle: %v", err)
}
}
// **A machine says which links face outside without being asked.**
//
// The mesh composes no filter for a machine that has not said (novox/hq ADR 0140), and a machine only
// speaks unasked when this fingerprint changes. Left out of it, a machine that has just learnt to say
// could speak only when a declaration arrived — and a declaration cannot be composed until it has
// spoken. A machine waiting for a push that is waiting for the machine.
func TestANewOutwardLinkIsSaidUnasked(t *testing.T) {
w := &adoptionWatch{}
first := link.Report{Firewall: "none"}
if !w.differs(first) {
t.Fatal("the first report should differ from nothing")
}
w.said(first)
// Only the links changed, and nothing about adoption.
learnt := link.Report{Firewall: "none", Outward: []string{"eth0"}}
if !w.differs(learnt) {
t.Fatal("a machine that has just learnt which links face outside would never say so, " +
"and could then never be sent a filter")
}
w.said(learnt)
if w.differs(link.Report{Firewall: "none", Outward: []string{"eth0"}}) {
t.Fatal("the same links are reported as a change, so the machine would speak on every reconcile")
}
// And a link that changes — a laptop moving from a cable to a radio — is said too, because the
// filter is written around the old one until it is.
if !w.differs(link.Report{Firewall: "none", Outward: []string{"wlan0"}}) {
t.Fatal("a changed outward link is not said, so the filter stays written around the old one")
}
}
// **A converged machine can say which links face outside, unasked.**
//
// A reconcile is otherwise silent, and the condition deciding when it speaks asked only what an
// adopted node reports — what it holds, and the firewall it found. A converged node has neither, so
// it could speak only in reply to a declaration, and the mesh composes no declaration for a node
// that has not said which links face outside (novox/hq ADR 0140). Measured: three converged machines
// sat silent while the control plane refused to send them a filter.
func TestAConvergedMachineSaysItsOutwardLinksUnasked(t *testing.T) {
// A converged node's reconcile: nothing held, no found firewall, and the links it can see.
converged := link.Report{Outward: []string{"eth0"}}
if !worthSaying(converged) {
t.Fatal("a converged machine cannot say which links face outside, so it can never be " +
"sent a filter — a machine waiting for a push that is waiting for the machine")
}
// An adopted node's reasons still hold, because that is how a predecessor still writing is caught.
if !worthSaying(link.Report{Firewall: "ufw"}) {
t.Fatal("an adopted machine no longer says which firewall it found")
}
if !worthSaying(link.Report{Held: []link.Held{{ID: "a-file"}}}) {
t.Fatal("an adopted machine no longer says what it holds")
}
// And a reconcile with nothing to say stays silent, or every machine speaks every five minutes
// about nothing.
if worthSaying(link.Report{}) {
t.Fatal("a reconcile with nothing to say speaks anyway")
}
// A refused report says nothing about the machine; the refusal is for the console.
if worthSaying(link.Report{Outward: []string{"eth0"}, Refused: "not for this node"}) {
t.Fatal("a refused report is offered as news about the machine")
}
}
// **A host running as a service says what its apply did.**
//
// The serving path passed nil where the apply writes its detail, and nil is silence. The one-shot path
@@ -611,63 +380,3 @@ func TestTheApplysLogIsNeverNil(t *testing.T) {
t.Fatalf("the apply's detail did not reach the caller's announce: %v", said)
}
}
// Defends novox/hq issues 257 and 261: a reconcile due while the link applies a newer declaration
// applies that newer one once its turn comes — never the one kept when its timer fired. Read before
// waiting, it gave back a module assigned a second earlier, and reported a declaration the mesh no
// longer recorded as sent.
func TestAReconcileAppliesWhatWasKeptWhenItsTurnComes(t *testing.T) {
built, err := system.For("arch")
if err != nil || built.Confirm(context.Background(), apply.ExecRunner) != nil {
t.Skip("this machine is not one these tests can apply on")
}
was := builtFor
builtFor = "arch"
t.Cleanup(func() { builtFor = was })
dir := t.TempDir()
opts := options{state: filepath.Join(dir, "state.json")}
public, private, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
target := filepath.Join(dir, "a.conf")
kept := func(content string) store.Declared {
body := []byte(`{"declaration":1,"resources":[{"id":"a","type":"file","path":"` + target +
`","content":"` + content + `\n"}]}`)
return store.Declared{Declaration: body, Signature: ed25519.Sign(private, body)}
}
if err := store.SaveDeclared(store.DeclaredPath(opts.state), kept("older")); err != nil {
t.Fatal(err)
}
// The link is applying: the reconcile's timer fires and it waits its turn.
applying.Lock()
done := make(chan error, 1)
go func() {
_, err := reapplyKept(context.Background(), opts, public, nil, nil)
done <- err
}()
time.Sleep(50 * time.Millisecond)
// The link's apply ends by keeping the newer declaration, and lets go.
if err := store.SaveDeclared(store.DeclaredPath(opts.state), kept("newer")); err != nil {
applying.Unlock()
t.Fatal(err)
}
applying.Unlock()
select {
case err := <-done:
if err != nil {
t.Fatal(err)
}
case <-time.After(10 * time.Second):
t.Fatal("the reconcile never ran once the node was free")
}
got, err := os.ReadFile(target)
if err != nil {
t.Fatal(err)
}
if string(got) != "newer\n" {
t.Errorf("the reconcile applied %q, what was kept before the newer declaration", got)
}
}
+2 -2
View File
@@ -11,14 +11,14 @@ import (
// serveMeshCLI answers mesh-cli on this machine for as long as this engine serves (novox/hq ADR 0272): the account
// from the kernel, the line asked of the controller over the link open at the time. A socket that cannot be served
// is said once, and this engine goes on applying; mesh-cli then says the engine does not answer.
func serveMeshCLI(ctx context.Context, queue *link.Queue, say func(string)) {
func serveMeshCLI(ctx context.Context, queue *link.Session, say func(string)) {
if err := meshcli.Serve(ctx, cli.Socket, forwardOver(queue), say); err != nil {
say("mesh-cli is not answered on this machine: " + err.Error())
}
}
// forwardOver asks the controller over the queue's link, and follows the line to what came of it.
func forwardOver(queue *link.Queue) meshcli.Forward {
func forwardOver(queue *link.Session) meshcli.Forward {
return func(ctx context.Context, asked meshcli.Asked) cli.Answer {
return meshcli.AskTheController(ctx, queue.AskController, asked)
}
+38
View File
@@ -0,0 +1,38 @@
package main
import (
"errors"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
)
// 1132/AC1, AC3: the report carries one fact per module the declaration names, with the build the controller sent for
// it; a module not yet taken on an adopted node is not one.
func TestTheReportCarriesOneFactPerModuleWithItsBuild(t *testing.T) {
d, err := declaration.Parse([]byte(`{"declaration":1,"builds":{"a":"b-1","b":"b-2"},"resources":[
{"id":"a.dir","type":"directory","path":"/srv/a","module":"a"},
{"id":"b.dir","type":"directory","path":"/srv/b","module":"b"}]}`))
if err != nil {
t.Fatal(err)
}
failure := &apply.Error{Resource: "b.dir", Err: errors.New("no space"), Failures: []apply.Failure{{Resource: "b.dir", Err: errors.New("no space")}}}
facts := moduleFacts(d, apply.Report{}, failure)
if len(facts) != 2 || facts[0].Module != "a" || facts[0].Build != "b-1" || facts[0].Failed ||
facts[1].Module != "b" || facts[1].Build != "b-2" || !facts[1].Failed || facts[1].Detail == "" {
t.Fatalf("facts %+v", facts)
}
adopted, err := declaration.Parse([]byte(`{"declaration":1,"adoption":{"taken":["a"],"untaken":{"b":["b.dir"]}},"resources":[
{"id":"a.dir","type":"directory","path":"/srv/a","module":"a"},
{"id":"b.dir","type":"directory","path":"/srv/b","module":"b"}]}`))
if err != nil {
t.Fatal(err)
}
if facts := moduleFacts(adopted, apply.Report{}, nil); len(facts) != 1 || facts[0].Module != "a" {
t.Fatalf("an untaken module is a fact: %+v", facts)
}
if moduleFacts(nil, apply.Report{}, nil) != nil {
t.Fatal("facts without a declaration")
}
}
+113
View File
@@ -0,0 +1,113 @@
package main
import (
"context"
"encoding/json"
"fmt"
"os"
osuser "os/user"
"path/filepath"
"strconv"
"strings"
"syscall"
"testing"
"time"
"github.com/novox/mesh-host/internal/apply"
)
// TestAMoveOutDirectoryKeepsOnlyWhatItKnows drives the node-engine whole through its command (issue 1024, novox/hq
// ADR 0422): a directory under an account's home that says move-out keeps the file the mesh declared and moves a
// file nobody declared into the kept folder, named in a list, and the run says it moved it. It needs no bus: the
// apply is the one a delivery and the five-minute reconcile both run.
func TestAMoveOutDirectoryKeepsOnlyWhatItKnows(t *testing.T) {
me, err := osuser.Current()
if err != nil {
t.Fatal(err)
}
// The user database lists an account as a person's only at uid 1000 or more; root, as the forge's merge check
// runs, is listed under a number of its own (it can give files to it), by the same name.
listedUID := me.Uid
if uid, _ := strconv.Atoi(me.Uid); uid < 1000 || uid == 65534 {
listedUID = "4242"
}
root := t.TempDir()
home := filepath.Join(root, "home", "agent")
dir := filepath.Join(home, ".ssh")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
passwd := filepath.Join(root, "passwd")
line := fmt.Sprintf("%s:x:%s:%s::%s:/bin/sh\n", me.Username, listedUID, me.Gid, home)
if err := os.WriteFile(passwd, []byte(line), 0o644); err != nil {
t.Fatal(err)
}
t.Cleanup(apply.UseAccountsFileForTest(passwd))
// Independent of the machine: an arch host with a pacman on PATH that answers and holds nothing, as
// TestReplay233 puts one there (it answers with an error); nothing here declares a package.
bin := t.TempDir()
if err := os.WriteFile(filepath.Join(bin, "pacman"), []byte("#!/bin/sh\nexit 0\n"), 0o755); err != nil {
t.Fatal(err)
}
t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH"))
wasBuilt := builtFor
builtFor = "arch"
t.Cleanup(func() { builtFor = wasBuilt })
stray := filepath.Join(dir, "id_stray")
if err := os.WriteFile(stray, []byte("somebody else's key"), 0o600); err != nil {
t.Fatal(err)
}
old := time.Now().Add(-time.Hour)
if err := os.Chtimes(stray, old, old); err != nil {
t.Fatal(err)
}
pipe := filepath.Join(dir, "agent.sock")
if err := syscall.Mkfifo(pipe, 0o644); err != nil {
t.Fatal(err)
}
if err := os.Chtimes(pipe, old, old); err != nil {
t.Fatal(err)
}
opts := options{state: filepath.Join(root, "state.json")}
raw := []byte(`{"declaration":1,"resources":[` +
`{"id":"ssh.dir","type":"directory","path":"` + dir + `","owner":"` + me.Username + `","unknown":"move-out"},` +
`{"id":"ssh.config","type":"file","path":"` + dir + `/config","content":"Host *\n","owner":"` + me.Username + `"}]}`)
var said []string
report := applyAndKeep(context.Background(), opts, raw, nil, nil, func(s string) { said = append(said, s) })
if report.Refused != "" || len(report.Failed) > 0 {
t.Fatalf("refused or failed: %q %v", report.Refused, report.Failed)
}
if _, err := os.Stat(filepath.Join(dir, "config")); err != nil {
t.Errorf("the declared file is not there: %v", err)
}
if _, err := os.Lstat(stray); err == nil {
t.Error("the file nobody declared stayed in the directory")
}
lists, _ := filepath.Glob(filepath.Join(home, ".local", "state", "mesh-moved", "ssh", "*.json"))
if len(lists) != 1 {
t.Fatalf("one list expected: %v", lists)
}
rawList, err := os.ReadFile(lists[0])
if err != nil {
t.Fatal(err)
}
var entries []struct{ From, To string }
if err := json.Unmarshal(rawList, &entries); err != nil || len(entries) != 1 {
t.Fatalf("the list: %s %v", rawList, err)
}
if got, err := os.ReadFile(entries[0].To); err != nil || string(got) != "somebody else's key" {
t.Errorf("the moved file is not whole in the kept folder: %q %v", got, err)
}
if len(report.MovedOut) != 1 || report.MovedOut[0].From != stray || report.MovedOut[0].To != entries[0].To {
t.Errorf("the report does not carry the move: %+v", report.MovedOut)
}
if len(report.Left) != 1 || report.Left[0].Path != pipe {
t.Errorf("the report does not list the pipe left in place: %+v", report.Left)
}
if !strings.Contains(strings.Join(said, "\n"), "moved "+stray) {
t.Errorf("the run does not say it moved the file: %v", said)
}
}
+131
View File
@@ -0,0 +1,131 @@
package main
import (
"context"
"crypto/rand"
"crypto/subtle"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"os"
"time"
"golang.org/x/crypto/nacl/box"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/link"
)
// The move of a value sealed to this machine into the vault (design 57's V3, issue 676; internal/link
// movesecret.go). The controller cannot open what it holds; this machine can open what was sealed to it, and so
// hands it to the vault, sealed to a key the vault made for the call. The value is in this process for that one
// call and written nowhere.
// vaultAsk asks one of the vault's verbs in this engine's own name and answers its result.
type vaultAsk func(verb string, args any) (json.RawMessage, error)
// moveSecretFrom is the engine's move: this machine's sealing key, read for each ask, and the vault asked over
// the link open now.
func moveSecretFrom(statePath string, queue *link.Session) link.MoveSecretFunc {
return func(m link.MoveSecret) (link.MovedSecret, error) {
key, err := identity.LoadSealingKey(identity.SealingKeyPath(statePath))
if err != nil {
return link.MovedSecret{}, err
}
ask := func(verb string, args any) (json.RawMessage, error) {
return queue.AskVault(context.Background(), verb, args)
}
moved, err := moveIntoTheVault(ask, key, m)
if err != nil {
fmt.Fprintf(os.Stderr, "mesh-host: a value was not moved into the vault: %v\n", err)
}
return moved, err
}
}
// moveIntoTheVault opens the row's value with this machine's sealing key, hands it to the vault's move-in in its two
// steps, and answers what the vault kept once its read-back — sealed to a key made here for this one call — opens
// to the same value.
func moveIntoTheVault(ask vaultAsk, key identity.SealingKey, m link.MoveSecret) (link.MovedSecret, error) {
if m.SealedTo != "" && m.SealedTo != key.Public {
return link.MovedSecret{}, errors.New("the controller's store says this value was sealed to a sealing key this " +
"machine no longer has, so it cannot be opened here")
}
value, err := key.Unseal(m.Sealed)
if err != nil {
return link.MovedSecret{}, fmt.Errorf("the value does not open with this machine's sealing key: %w", err)
}
defer wipe(value)
// This machine's sealing key, stated in its own name: the key the vault seals the core's credentials to for the
// controller to carry here (novox/hq ADR 0349 §16, issue 763), never a key the controller names.
first := map[string]any{"origin": m.Origin, "generation": m.Generation, "sealing_key": key.Public}
if !m.MadeAt.IsZero() {
first["made_at"] = m.MadeAt.UTC().Format(time.RFC3339Nano)
}
if m.Own() {
first["node"], first["module"], first["name"] = m.Node, m.Module, m.Name
} else {
first["provider"], first["provision"] = m.Provider, m.Provision
first["consumer"], first["consumer_module"] = m.Consumer, m.ConsumerModule
if m.Local != "" {
first["local"] = m.Local
}
}
raw, err := ask("move-in", first)
if err != nil {
return link.MovedSecret{}, fmt.Errorf("the vault did not open the move: %w", err)
}
var opened struct {
Call string `json:"call"`
SealTo string `json:"seal_to"`
Secret string `json:"secret"`
}
if err := json.Unmarshal(raw, &opened); err != nil || opened.Call == "" || opened.SealTo == "" {
return link.MovedSecret{}, errors.New("the vault answered no call and no key to seal to")
}
sealed, err := identity.Seal(opened.SealTo, value)
if err != nil {
return link.MovedSecret{}, fmt.Errorf("the vault's key for the call is not a sealing key: %w", err)
}
backPub, backPriv, err := box.GenerateKey(rand.Reader)
if err != nil {
return link.MovedSecret{}, err
}
defer wipe(backPriv[:])
raw, err = ask("move-in", map[string]any{"call": opened.Call, "sealed": sealed,
"read_back_to": base64.StdEncoding.EncodeToString(backPub[:])})
if err != nil {
return link.MovedSecret{}, fmt.Errorf("the vault did not take %s: %w", opened.Secret, err)
}
var kept struct {
Name string `json:"name"`
Generation int `json:"generation"`
Unchanged bool `json:"unchanged"`
ReadBack string `json:"read_back"`
}
if err := json.Unmarshal(raw, &kept); err != nil || kept.Name == "" || kept.ReadBack == "" {
return link.MovedSecret{}, fmt.Errorf("the vault answered %s with no read-back", opened.Secret)
}
blob, err := base64.StdEncoding.DecodeString(kept.ReadBack)
if err != nil {
return link.MovedSecret{}, fmt.Errorf("the vault's read-back of %s is not sealed", kept.Name)
}
back, ok := box.OpenAnonymous(nil, blob, backPub, backPriv)
if !ok {
return link.MovedSecret{}, fmt.Errorf("the vault's read-back of %s does not open with the key made for it", kept.Name)
}
defer wipe(back)
if subtle.ConstantTimeCompare(back, value) != 1 {
return link.MovedSecret{}, fmt.Errorf("the vault keeps %s, but it read back another value than the one handed "+
"in: the move of it did not hold", kept.Name)
}
return link.MovedSecret{Secret: kept.Name, Generation: kept.Generation, Unchanged: kept.Unchanged}, nil
}
func wipe(b []byte) {
for i := range b {
b[i] = 0
}
}
+134
View File
@@ -0,0 +1,134 @@
package main
import (
"crypto/rand"
"encoding/base64"
"encoding/json"
"errors"
"strings"
"testing"
"golang.org/x/crypto/nacl/box"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/link"
)
// standInVault answers move-in's two steps as the vault does: a key made for the call, the value opened with it,
// and the value kept read back sealed to the key the second step sent. lie makes its read-back another value.
type standInVault struct {
t *testing.T
pub *[32]byte
priv *[32]byte
opened map[string]any
kept string
lie string
refuse string
carried []string
}
func (v *standInVault) ask(verb string, args any) (json.RawMessage, error) {
raw, _ := json.Marshal(args)
v.carried = append(v.carried, string(raw))
if verb != "move-in" {
v.t.Fatalf("asked %s", verb)
}
if v.refuse != "" {
return nil, errors.New(v.refuse)
}
var a map[string]any
_ = json.Unmarshal(raw, &a)
if a["call"] == nil {
v.opened = a
pub, priv, _ := box.GenerateKey(rand.Reader)
v.pub, v.priv = pub, priv
return json.Marshal(map[string]string{"call": "c1", "seal_to": base64.StdEncoding.EncodeToString(pub[:]),
"secret": "own/anchor/postgres/superuser-password"})
}
blob, _ := base64.StdEncoding.DecodeString(a["sealed"].(string))
value, ok := box.OpenAnonymous(nil, blob, v.pub, v.priv)
if !ok {
v.t.Fatal("what the engine sealed does not open with the call's key")
}
v.kept = string(value)
back := v.kept
if v.lie != "" {
back = v.lie
}
key, _ := base64.StdEncoding.DecodeString(a["read_back_to"].(string))
var to [32]byte
copy(to[:], key)
boxed, _ := box.SealAnonymous(nil, []byte(back), &to, rand.Reader)
return json.Marshal(map[string]any{"name": "own/anchor/postgres/superuser-password", "generation": 1,
"read_back": base64.StdEncoding.EncodeToString(boxed)})
}
const theValue = "the-superusers-password"
func aMoveOf(t *testing.T, key identity.SealingKey) link.MoveSecret {
t.Helper()
sealed, err := identity.Seal(key.Public, []byte(theValue))
if err != nil {
t.Fatal(err)
}
return link.MoveSecret{Node: "anchor", By: "the move", MoveSecretAsk: link.MoveSecretAsk{Module: "postgres",
Name: "superuser-password", Origin: "made", Generation: "g-1", Sealed: sealed, SealedTo: key.Public}}
}
// The engine opens what was sealed to its machine, hands it to the vault sealed to the call's key — never in the
// clear — and answers what the vault kept only once the read-back opens to the same value.
func TestTheEngineMovesAValueSealedToItAndChecksTheReadBack(t *testing.T) {
key, err := identity.GenerateSealingKey()
if err != nil {
t.Fatal(err)
}
vault := &standInVault{t: t}
moved, err := moveIntoTheVault(vault.ask, key, aMoveOf(t, key))
if err != nil {
t.Fatal(err)
}
if moved.Secret != "own/anchor/postgres/superuser-password" || moved.Generation != 1 || vault.kept != theValue {
t.Fatalf("moved %+v, the vault kept %q", moved, vault.kept)
}
if vault.opened["module"] != "postgres" || vault.opened["name"] != "superuser-password" || vault.opened["node"] != "anchor" ||
vault.opened["origin"] != "made" || vault.opened["generation"] != "g-1" ||
vault.opened["sealing_key"] != key.Public {
t.Errorf("the first step named %v", vault.opened)
}
if strings.Contains(strings.Join(vault.carried, " "), theValue) {
t.Fatal("the value crossed the bus in the clear")
}
}
// A read-back that is not the value handed in is a failed move, said; so is a value this machine cannot open, and
// the vault's refusal.
func TestTheEngineSaysAMoveThatDidNotHold(t *testing.T) {
key, _ := identity.GenerateSealingKey()
vault := &standInVault{t: t, lie: "something-else"}
if _, err := moveIntoTheVault(vault.ask, key, aMoveOf(t, key)); err == nil ||
!strings.Contains(err.Error(), "read back") {
t.Fatalf("a read-back of another value: %v", err)
}
other, _ := identity.GenerateSealingKey()
if _, err := moveIntoTheVault((&standInVault{t: t}).ask, key, aMoveOf(t, other)); err == nil ||
!strings.Contains(err.Error(), "sealing key") {
t.Fatalf("a value sealed to another key: %v", err)
}
refusing := &standInVault{t: t, refuse: "the move never replaces a value the vault holds"}
if _, err := moveIntoTheVault(refusing.ask, key, aMoveOf(t, key)); err == nil ||
!strings.Contains(err.Error(), "never replaces") {
t.Fatalf("the vault's refusal: %v", err)
}
// A pair credential names both its ends.
pair := aMoveOf(t, key)
pair.Module, pair.Name = "", ""
pair.Provider, pair.Provision, pair.Consumer, pair.ConsumerModule = "anchor", "database", "shanks", "gitea"
ends := &standInVault{t: t}
if _, err := moveIntoTheVault(ends.ask, key, pair); err != nil {
t.Fatal(err)
}
if ends.opened["provider"] != "anchor" || ends.opened["consumer"] != "shanks" || ends.opened["consumer_module"] != "gitea" ||
ends.opened["node"] != nil {
t.Errorf("the pair's first step named %v", ends.opened)
}
}
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"testing"
"time"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/liveness"
"github.com/novox/mesh-host/internal/network"
)
+15
View File
@@ -0,0 +1,15 @@
package main
import (
"os"
"path/filepath"
"testing"
)
// The checker runs gofmt, go vet and both test levels by Go's convention, and the build module starts the bus the
// integration level needs (novox/hq ADR 0529): no script of the repository's own runs beside them.
func TestTheRepositoryHoldsNoCheckScript(t *testing.T) {
if _, err := os.Stat(filepath.Join("..", "..", "merge-check.sh")); err == nil {
t.Fatal("merge-check.sh is back: the checker's levels run the tests")
}
}
File diff suppressed because it is too large Load Diff
+777
View File
@@ -0,0 +1,777 @@
//go:build integration
package main
import (
"context"
"crypto/ed25519"
"encoding/base64"
"encoding/json"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats-server/v2/server"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-host/internal/agent"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/selfcompose"
"github.com/novox/mesh-host/internal/store"
)
// The node agent whole, against a real bus with JetStream started in the test: the wants bucket, the node's composition
// bucket and the have bucket, a session on the bus, and the real apply of a module's composition into a directory of the
// test. R6-A AC1, AC5 and AC9 (novox/hq ADR 0539).
const itNode = "it1"
type itBed struct {
t *testing.T
conn *nats.Conn
js nats.JetStreamContext
// nodeConn and nodeJS are the node's own, under its real grant (buscompose.PermissionsFor, KindNode): the agent runs
// on them, so a watch or a read the grant does not allow fails here as it fails on the bus (issue 1126, 2026-10-27
// 09:21: a watch of every have key stopped every node agent). conn is the test's, unrestricted, for setting up.
nodeConn *nats.Conn
nodeJS nats.JetStreamContext
wants nats.KeyValue
composed nats.KeyValue
haves nats.KeyValue
dir string
opts options
session *link.Session
controll ed25519.PrivateKey
operator ed25519.PrivateKey
said []string
// refusals are the bus's asynchronous errors on the node's connection: each permissions violation it met.
refusals []string
mu sync.Mutex
}
// aServer is an embedded bus with JetStream and two users: the test's, unrestricted, and the node's, granted exactly what
// buscompose.PermissionsFor gives a node (KindNode); it answers a connection of each.
func aServer(t *testing.T, principal buscompose.Principal, refused func(error)) (admin, node *nats.Conn) {
t.Helper()
perms, err := buscompose.PermissionsFor(principal)
if err != nil {
t.Fatal(err)
}
granted := &server.Permissions{Publish: &server.SubjectPermission{Allow: perms.Publish, Deny: perms.PublishDeny},
Subscribe: &server.SubjectPermission{Allow: perms.Subscribe}}
if perms.AllowResponses {
granted.Response = &server.ResponsePermission{MaxMsgs: 1, Expires: buscompose.ResponseTTL}
}
nodeUser := buscompose.Principal{Kind: buscompose.KindNode, Node: itNode}.Username()
srv, err := server.NewServer(&server.Options{Port: -1, JetStream: true, StoreDir: t.TempDir(), NoLog: true, NoSigs: true,
Users: []*server.User{{Username: "test", Password: "test"}, {Username: nodeUser, Password: "node", Permissions: granted}}})
if err != nil {
t.Fatal(err)
}
go srv.Start()
if !srv.ReadyForConnections(10 * time.Second) {
t.Fatal("the embedded server did not start")
}
t.Cleanup(srv.Shutdown)
if admin, err = nats.Connect(srv.ClientURL(), nats.UserInfo("test", "test")); err != nil {
t.Fatal(err)
}
t.Cleanup(admin.Close)
if node, err = nats.Connect(srv.ClientURL(), nats.UserInfo(nodeUser, "node"), nats.CustomInboxPrefix("_INBOX."+nodeUser),
nats.ErrorHandler(func(_ *nats.Conn, _ *nats.Subscription, err error) { refused(err) })); err != nil {
t.Fatal(err)
}
t.Cleanup(node.Close)
return admin, node
}
func newItBed(t *testing.T) *itBed {
t.Helper()
return newItBedAs(t, buscompose.Principal{Kind: buscompose.KindNode, Node: itNode})
}
// newItBedAs is newItBed with the node's connection under the grant of principal, a node's (KindNode).
func newItBedAs(t *testing.T, principal buscompose.Principal) *itBed {
t.Helper()
// Not skipped off Arch: a package manager that answers is put on PATH, so the apply of files runs on any build node.
bin := t.TempDir()
if err := os.WriteFile(filepath.Join(bin, "pacman"), []byte("#!/bin/sh\nexit 0\n"), 0o755); err != nil {
t.Fatal(err)
}
t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH"))
was := builtFor
builtFor = "arch"
t.Cleanup(func() { builtFor = was })
b := &itBed{t: t, dir: t.TempDir()}
b.conn, b.nodeConn = aServer(t, principal, func(err error) {
b.mu.Lock()
b.refusals = append(b.refusals, err.Error())
b.mu.Unlock()
})
var err error
if b.js, err = b.conn.JetStream(); err != nil {
t.Fatal(err)
}
if b.nodeJS, err = b.nodeConn.JetStream(); err != nil {
t.Fatal(err)
}
composedBucket, _ := delivery.ComposedBucketFor(itNode)
for _, c := range []struct {
name string
into *nats.KeyValue
history uint8
}{{delivery.AssignerWantsBucket, &b.wants, 8}, {composedBucket, &b.composed, delivery.ComposedHistory},
{delivery.HaveBucket, &b.haves, delivery.HaveHistory}} {
kv, err := b.js.CreateKeyValue(&nats.KeyValueConfig{Bucket: c.name, History: c.history})
if err != nil {
t.Fatal(err)
}
*c.into = kv
}
b.opts = options{state: filepath.Join(b.dir, "state", "state.json"), timeout: 2 * time.Second}
if err := os.MkdirAll(filepath.Dir(b.opts.state), 0o755); err != nil {
t.Fatal(err)
}
_, b.controll, _ = ed25519.GenerateKey(nil)
_, b.operator, _ = ed25519.GenerateKey(nil)
return b
}
// start runs the node agent under a session on the bus, trusting the controller's key, and the operator's when asked.
func (b *itBed) start(trustOperator bool) (stop func()) {
b.t.Helper()
if trustOperator {
pub := b.operator.Public().(ed25519.PublicKey)
if err := os.WriteFile(filepath.Join(filepath.Dir(b.opts.state), operatorKeysFile),
[]byte("# the operator\n"+base64.StdEncoding.EncodeToString(pub)+"\n"), 0o600); err != nil {
b.t.Fatal(err)
}
}
b.session = &link.Session{Membership: link.Membership{Node: itNode}, Timeout: 2 * time.Second,
Say: b.say, Keys: func() link.RecordKeys { return link.RecordKeys{} }}
b.session.Attach(context.Background(), link.OverNATS{Conn: b.nodeConn, JS: b.nodeJS, Node: itNode})
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() {
defer close(done)
nodeAgent(ctx, b.opts, b.controll.Public().(ed25519.PublicKey), b.session, nil, b.say, func() {},
&agentNow{}, func(context.Context, *agent.Agent) {})
}()
return func() { cancel(); <-done }
}
func (b *itBed) say(line string) {
b.mu.Lock()
b.said = append(b.said, line)
b.mu.Unlock()
b.t.Log(line)
}
func fpIt(c string) string { return delivery.SourcePrefix2 + strings.Repeat(c, 64) }
func fpDigest(c string) string { return "sha256:" + strings.Repeat(c, 64) }
// compose writes a module's signed composition of one file into the node's bucket and returns its digest.
func (b *itBed) compose(module, fp, content string, data bool, signer ed25519.PrivateKey) string {
b.t.Helper()
path := filepath.Join(b.dir, module+".conf")
res, _ := json.Marshal(map[string]any{"id": module + ".conf", "type": "file", "path": path, "content": content, "module": module})
c := delivery.Composed{Node: itNode, Module: module, FP: fp, Data: data, Resources: []json.RawMessage{res}}
signed, err := delivery.SignComposed(c, signer)
if err != nil {
b.t.Fatal(err)
}
raw, err := delivery.EncodeComposed(signed)
if err != nil {
b.t.Fatal(err)
}
dig, _ := delivery.ComposedDigestOf(signed)
key, _ := delivery.ComposedKey(itNode, module, fp)
if _, err := b.composed.Put(key, raw); err != nil {
b.t.Fatal(err)
}
return dig
}
func (b *itBed) want(module, fp, digest, backup string) {
b.t.Helper()
raw, err := delivery.EncodeWant(delivery.Want{FP: fp, Composed: digest, Backup: backup})
if err != nil {
b.t.Fatal(err)
}
key, _ := delivery.WantKey(itNode, module)
if _, err := b.wants.Put(key, raw); err != nil {
b.t.Fatal(err)
}
}
func (b *itBed) file(module string) string {
got, _ := os.ReadFile(filepath.Join(b.dir, module+".conf"))
return string(got)
}
func (b *itBed) have(module string) delivery.Have {
key, _ := delivery.HaveKey(itNode, module)
e, err := b.haves.Get(key)
if err != nil {
return delivery.Have{}
}
h, _ := delivery.DecodeHave(e.Value())
return h
}
func within(t *testing.T, d time.Duration, what string, ok func() bool) {
t.Helper()
deadline := time.Now().Add(d)
for !ok() {
if time.Now().After(deadline) {
t.Fatal(what)
}
time.Sleep(10 * time.Millisecond)
}
}
// R6-A AC1: given a running agent on a bus, when a want naming a digest and that composition are written, the module is
// applied within one second and its have names the fingerprint and the digest; a composition whose digest no want names
// is not applied.
func TestNatsAWantAndItsCompositionAreAppliedWithinASecond(t *testing.T) {
t.Run("1957: an engine start starts a declared-running unit", testNatsAnEngineStartStartsADeclaredRunningUnitTheHaveSaysRuns)
b := newItBed(t)
stop := b.start(false)
defer stop()
time.Sleep(300 * time.Millisecond) // the watches are up
fp := fpIt("a")
dig := b.compose("web", fp, "one\n", false, b.controll)
time.Sleep(300 * time.Millisecond)
if b.file("web") != "" {
t.Fatal("a composition no want names was applied")
}
began := time.Now()
b.want("web", fp, dig, "")
within(t, time.Second, "the module was not applied within one second of its want", func() bool { return b.file("web") == "one\n" })
t.Logf("applied %s after the want", time.Since(began))
within(t, 2*time.Second, "the have does not name the fingerprint and the digest", func() bool {
h := b.have("web")
return h.FP == fp && h.Composed == dig && h.Phase == delivery.PhaseRunning
})
// A want naming a digest the composition at its fingerprint does not have applies nothing.
fp2 := fpIt("b")
b.compose("web", fp2, "two\n", false, b.controll)
b.want("web", fp2, "sha256:"+strings.Repeat("0", 64), "")
time.Sleep(300 * time.Millisecond)
if b.file("web") != "one\n" {
t.Fatalf("a want naming another digest changed the module: %q", b.file("web"))
}
}
// R6-A AC5: a data module that ran before asks its backup before any resource changes; a backup that fails holds the
// module (nothing applied, the have says held with the backup's words); when its want then says skip, it is applied.
func TestNatsADataModuleAsksItsBackupFirstAndIsHeldWhenItFails(t *testing.T) {
b := newItBed(t)
var mu sync.Mutex
var asked []string
var onDiskWhenAsked string
if _, err := b.conn.Subscribe("mesh.seat.node-backup.tool.now."+itNode, func(m *nats.Msg) {
mu.Lock()
asked = append(asked, "now")
onDiskWhenAsked = b.file("db")
mu.Unlock()
_ = m.Respond([]byte(`{"result":{"started":"db"}}`))
// The holder says the backup ended on its event.
ev, _ := json.Marshal(map[string]any{"node": itNode, "module": "db", "ok": false, "at": time.Now().UTC(),
"error": "restic: the repository is locked"})
_ = b.conn.Publish(backedUpEvent, ev)
}); err != nil {
t.Fatal(err)
}
stop := b.start(false)
defer stop()
time.Sleep(300 * time.Millisecond)
fp1, fp2 := fpIt("a"), fpIt("b")
b.want("db", fp1, b.compose("db", fp1, "v1\n", true, b.controll), "")
within(t, 3*time.Second, "the first composition was not applied", func() bool { return b.file("db") == "v1\n" })
mu.Lock()
if len(asked) != 0 {
t.Fatalf("a module that never ran was backed up: %v", asked)
}
mu.Unlock()
dig2 := b.compose("db", fp2, "v2\n", true, b.controll)
b.want("db", fp2, dig2, "")
within(t, 3*time.Second, "the module was not held behind the failed backup", func() bool {
h := b.have("db")
return h.Phase == delivery.PhaseHeld && strings.Contains(h.Detail, "repository is locked")
})
mu.Lock()
if len(asked) != 1 || onDiskWhenAsked != "v1\n" {
t.Fatalf("the backup was asked %v, with %q on disk: it is asked before any resource changes", asked, onDiskWhenAsked)
}
mu.Unlock()
if b.file("db") != "v1\n" {
t.Fatalf("something was applied behind a failed backup: %q", b.file("db"))
}
if h := b.have("db"); h.FP != fp1 {
t.Fatalf("a held module should still name what it runs: %+v", h)
}
b.want("db", fp2, dig2, delivery.BackupSkip)
within(t, 3*time.Second, "the module was not applied once its want said skip", func() bool { return b.file("db") == "v2\n" })
within(t, 2*time.Second, "the have does not say running at the new fingerprint", func() bool {
h := b.have("db")
return h.Phase == delivery.PhaseRunning && h.FP == fp2 && h.Composed == dig2
})
}
// R6-A AC9, the hand path end to end: with no controller running, a composition signed by the operator's key and a want
// naming it are applied and the have written; the same signed by a key nobody trusts is not.
func TestNatsTheHandPathAppliesACompositionTheOperatorSigned(t *testing.T) {
b := newItBed(t)
stop := b.start(true)
defer stop()
time.Sleep(300 * time.Millisecond)
_, stranger, _ := ed25519.GenerateKey(nil)
fpS := fpIt("c")
b.want("evil", fpS, b.compose("evil", fpS, "no\n", false, stranger), "")
time.Sleep(300 * time.Millisecond)
if b.file("evil") != "" {
t.Fatal("a composition signed by a key nobody trusts was applied")
}
fp := fpIt("d")
dig := b.compose("rescue", fp, "back\n", false, b.operator)
b.want("rescue", fp, dig, "")
within(t, 2*time.Second, "the operator's composition was not applied", func() bool { return b.file("rescue") == "back\n" })
within(t, 2*time.Second, "the have was not written", func() bool {
h := b.have("rescue")
return h.FP == fp && h.Composed == dig && h.Phase == delivery.PhaseRunning
})
}
// Issue 1126 (ADR 0547 §2, the milestone's H8): the node agent started whole against a real bus, once the catalogue
// places its node (the placement written after it started), composes a module from its want, the indexer's manifest, its record and its mesh-config value, and applies it,
// with no key in the controller's compositions; a changed config value is composed again on its event and applied as a
// revision of what runs. And a catalogue pull request's placement candidate equal to its placement is answered "same"
// in the node's own key, and nothing is applied for it (ADR 0559 §4, subtask 10 of the milestone).
//
// And (issue 1126, ADR 0560) it reads, through its own session under the grant the grants part composes for it
// (buscompose.Users, so it carries ServedReads), the build of a placed module that declares one and the one config value
// a provider on another node serves from (a mail domain); under a grant without that served key the bus refuses the
// read and the node composes nothing.
func TestNatsANodeThePlacementNamesComposesAndAppliesItsOwn(t *testing.T) {
users, err := buscompose.Users(buscompose.Records{Nodes: []string{itNode, "it2"},
Assigned: map[string][]buscompose.Declared{"it2": {{Module: "mailer", Served: []string{"domain"}}}}})
if err != nil {
t.Fatal(err)
}
var principal buscompose.Principal
for _, u := range users {
if u.Kind == buscompose.KindNode && u.Node == itNode {
principal = u
}
}
if len(principal.ServedReads) != 1 {
t.Fatalf("the grants part's principal of %s reads %v of the served keys", itNode, principal.ServedReads)
}
t.Run("without the served key the bus refuses the read", func(t *testing.T) {
b := newItBed(t)
stop, _ := h8(t, b, false)
defer stop()
// 15 s, above link.historyWithin (10 s): the bus's refusal of a publish reaches only the connection's error
// handler (nats.go fails no request on it), so the refused read ends at its own bound, then the node says it.
within(t, 15*time.Second, "the bus did not refuse the served key's read, or the node did not say it", func() bool {
b.mu.Lock()
defer b.mu.Unlock()
return strings.Contains(strings.Join(b.refusals, "\n"), "Permissions Violation for Publish to \"$JS.API.CONSUMER.CREATE.KV_"+
selfcompose.ResolvedBucket) && strings.Contains(strings.Join(b.refusals, "\n"), ".it2.mailer.domain") &&
strings.Contains(strings.Join(b.said, "\n"), "what mailer serves on it2 (domain)")
})
if b.file("web") != "" {
t.Errorf("composed and applied under a grant without the served key: %q", b.file("web"))
}
})
b := newItBedAs(t, principal)
stop, st := h8(t, b, true)
defer stop()
placement, resolved, candidates := st.placement, st.resolved, st.candidates
hkey, place, fp := st.hkey, st.place, st.fp
within(t, 5*time.Second, "the node did not compose and apply its own module once placed", func() bool { return b.file("web") == "greeting=bonjour\n" })
h8rest(t, b, resolved, candidates, hkey, place, fp)
t.Run("one answer per candidate revision", func(t *testing.T) { h8once(t, b, resolved, candidates, place) })
h8back(t, b, placement, resolved, candidates, hkey)
}
// h8once is issue 1126, 2026-10-28 (shanks answered probe/1126-seed-7 at revision 84 every 13-17 s, its bus flooded):
// a candidate revision is answered once. The node's own answer written, a module's have whose health words alone move
// (each look rewrites them) and a config value written again with the value it held answer nothing again; the
// candidate's next revision is answered once.
func h8once(t *testing.T, b *itBed, resolved, candidates nats.KeyValue, place []byte) {
answering := func() int {
b.mu.Lock()
defer b.mu.Unlock()
n := 0
for _, l := range b.said {
if strings.HasPrefix(l, "answering placement candidate feat-x ") {
n++
}
}
return n
}
time.Sleep(time.Second)
settled := answering()
if settled == 0 {
t.Fatal("feat-x was never answered")
}
hkey, _ := delivery.HaveKey(itNode, "web")
for _, detail := range []string{"its tool web_health: no link to the bus is open", "its tool web_health: no answer within 10s"} {
h := b.have("web")
h.Health, h.Detail = delivery.HealthFailed, detail
raw, err := delivery.EncodeHave(h)
if err != nil {
t.Fatal(err)
}
if _, err := b.haves.Put(hkey, raw); err != nil {
t.Fatal(err)
}
if _, err := resolved.Put(itNode+".web.greeting", []byte(`{"value":"salut","source":"module"}`)); err != nil {
t.Fatal(err)
}
}
time.Sleep(1500 * time.Millisecond)
if got := answering(); got != settled {
t.Errorf("feat-x answered %d more time(s) at the same revision, with nothing it is composed from changed", got-settled)
}
same, _ := delivery.DecodePlacement(place)
craw, err := delivery.EncodePlacementCandidate(delivery.PlacementCandidate{Proposal: "feat-x", Placement: &same})
if err != nil {
t.Fatal(err)
}
ckey, _ := delivery.PlacementCandidateKey("feat-x", itNode)
if _, err := candidates.Put(ckey, craw); err != nil {
t.Fatal(err)
}
within(t, 10*time.Second, "the candidate's next revision was not answered", func() bool { return answering() > settled })
time.Sleep(1500 * time.Millisecond)
if got := answering(); got != settled+1 {
t.Errorf("the candidate's next revision was answered %d time(s), want 1", got-settled)
}
}
// h8back is the way back (issue 1126, the seed's revert): a candidate deleting the node's placement is compared with
// the controller's compositions at its wants (here none: refused, naming web), and the placement key deleted returns the agent to them without a
// restart, said in the journal and the node's have, keeping what runs.
func h8back(t *testing.T, b *itBed, placement, resolved, candidates nats.KeyValue, hkey string) {
t.Helper()
node := func() delivery.HaveNode {
e, err := b.haves.Get(hkey)
if err != nil {
return delivery.HaveNode{}
}
n, _ := delivery.DecodeHaveNode(e.Value())
return n
}
if n := node(); n.Composes != delivery.ComposesOwn {
t.Errorf("a node composing its own says it composes %q", n.Composes)
}
craw, err := delivery.EncodePlacementCandidate(delivery.PlacementCandidate{Proposal: "revert-x"})
if err != nil {
t.Fatal(err)
}
ckey, _ := delivery.PlacementCandidateKey("revert-x", itNode)
if _, err := candidates.Put(ckey, craw); err != nil {
t.Fatal(err)
}
// The controller holds no composition at web's want (the node composed its own): returning would leave web with
// nothing to apply, and the answer says so rather than "same" (review of the shadow, round 1).
within(t, 10*time.Second, "the node did not refuse returning to compositions the controller does not hold", func() bool {
a := node().Candidates["revert-x"]
return a.Proposal == "revert-x" && a.Returns == "" && a.Verdict == delivery.CandidateRefused &&
strings.Contains(a.Refused, "web: the controller holds no composition")
})
ran := b.have("web").Composed
pkey, _ := delivery.PlacementKey(itNode)
// A pass in flight at the switch (issue 1126, 2026-10-28: the last pass of the own run, failing on its run's end,
// marked the next agent's modules failed): a config value changed starts one, and the key goes at once.
if _, err := resolved.Put(itNode+".web.greeting", []byte(`{"value":"salut","source":"module"}`)); err != nil {
t.Fatal(err)
}
if err := placement.Delete(pkey); err != nil {
t.Fatal(err)
}
stopWatch := make(chan struct{})
marked := make(chan Have1126, 1)
go func() {
for {
select {
case <-stopWatch:
return
case <-time.After(20 * time.Millisecond):
}
if h := b.have("web"); h.Phase == delivery.PhaseFailed {
select {
case marked <- Have1126{h.Phase, h.Detail}:
default:
}
return
}
}
}()
defer close(stopWatch)
within(t, 15*time.Second, "the node did not return to the controller's compositions when its placement went", func() bool {
b.mu.Lock()
said := strings.Join(b.said, "\n")
b.mu.Unlock()
return strings.Contains(said, "key in the seat's placement state is gone") && node().Composes == delivery.ComposesController
})
time.Sleep(3 * time.Second) // what a pass in flight would mark lands within it
select {
case h := <-marked:
t.Errorf("a pass in flight at the switch marked web failed: %+v", h)
default:
}
if h := b.have("web"); h.Composed != ran || h.Phase == delivery.PhaseFailed || b.file("web") != "greeting=salut\n" {
t.Errorf("returning to the controller's compositions moved what runs: %+v, %q", h, b.file("web"))
}
}
// Have1126 is what H8's way back watches of web's have.
type Have1126 struct {
Phase delivery.Phase
Detail string
}
// h8state is what h8 set up that the rest of H8 writes to.
type h8state struct {
placement, resolved, candidates nats.KeyValue
hkey string
place []byte
fp string
}
// h8 sets H8's bus up and starts the agent, then places the node. With checkBuild, the placed module's build key is
// first one that does not decode, which the node must read through its session (and say), then a done one.
func h8(t *testing.T, b *itBed, checkBuild bool) (func(), h8state) {
t.Helper()
kvOf := func(bucket string) nats.KeyValue {
kv, err := b.js.CreateKeyValue(&nats.KeyValueConfig{Bucket: bucket, History: 5})
if err != nil {
t.Fatal(err)
}
return kv
}
placement, modules, manifests := kvOf(delivery.PlacementBucket), kvOf(delivery.ModulesBucket), kvOf(delivery.ManifestsBucket)
resolved := kvOf(selfcompose.ResolvedBucket)
builds := kvOf(delivery.BuilderBuildsBucket)
candidates := kvOf(delivery.PlacementCandidatesBucket)
// The controller's stream the node's facts go into, so the node's key carries the facts it says.
if _, err := b.js.AddStream(&nats.StreamConfig{Name: "CONTROL", Subjects: []string{link.ReportSubject(itNode)}}); err != nil {
t.Fatal(err)
}
path := filepath.Join(b.dir, "web.conf")
man := `{"module":"web","version":"1","config":{"greeting":{"type":"string","description":"what the file says","default":"hello"}},` +
`"resources":[{"id":"conf","type":"file","path":"` + path + `","mode":"0644","content":"greeting=${config:greeting}\n"}],` +
`"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]}}`
digest := delivery.ManifestDigest([]byte(man))
mraw, _ := delivery.EncodeManifest(delivery.Manifest{Module: "web", JSON: man})
mkey, _ := delivery.ManifestsKey(digest)
fp := fpIt("a")
modv, _ := delivery.EncodeModule(delivery.Module{FP2: fp, Manifest: digest, Repository: "novox/web", Commit: strings.Repeat("1", 40)})
place, _ := delivery.EncodePlacement(delivery.Placement{Node: itNode, Modules: []delivery.PlacedModule{{Module: "web"}}})
rec, _ := delivery.EncodeHaveNode(delivery.HaveNode{Record: &delivery.NodeRecord{Node: itNode, Capabilities: []string{"service-manager"}}})
pkey, _ := delivery.PlacementKey(itNode)
hkey, _ := delivery.HaveNodeKey(itNode)
// A provider on it2 serving from its own config value: a mail domain, which it1's composition settles.
mailer := `{"module":"mailer","version":"1","config":{"domain":{"type":"string","description":"the mail domain","required":true}},` +
`"provides":[{"name":"smtp","scope":"mesh"}],"serves":{"smtp":{"port":587,"domain":"${config:domain}"}}}`
mdigest := delivery.ManifestDigest([]byte(mailer))
mlraw, _ := delivery.EncodeManifest(delivery.Manifest{Module: "mailer", JSON: mailer})
mlkey, _ := delivery.ManifestsKey(mdigest)
mlmod, _ := delivery.EncodeModule(delivery.Module{FP2: fpIt("b"), Manifest: mdigest, Repository: "novox/mailer", Commit: strings.Repeat("2", 40)})
place2, _ := delivery.EncodePlacement(delivery.Placement{Node: "it2", Modules: []delivery.PlacedModule{{Module: "mailer"}}})
rec2, _ := delivery.EncodeHaveNode(delivery.HaveNode{Record: &delivery.NodeRecord{Node: "it2", Capabilities: []string{"service-manager"}}})
pkey2, _ := delivery.PlacementKey("it2")
hkey2, _ := delivery.HaveNodeKey("it2")
buildKey, _ := delivery.BuilderBuildKey("web", fp)
done, _ := delivery.EncodeBuilderBuild(delivery.BuilderBuild{State: delivery.BuildDone, Manifest: digest,
Artifacts: []delivery.BuiltArtifact{{Name: "files", Kind: "archive", Reference: "artifact-store://web/files/blobs/" + fpDigest("f"), Digest: fpDigest("f")}}})
firstBuild := done
if checkBuild {
firstBuild = []byte(`{"v":1,"state":"no such state"}`)
}
for _, w := range []struct {
kv nats.KeyValue
key string
value []byte
}{{manifests, mkey, mraw}, {modules, "web", modv}, {b.haves, hkey, rec},
{resolved, itNode + ".web.greeting", []byte(`{"value":"bonjour","source":"module"}`)},
{manifests, mlkey, mlraw}, {modules, "mailer", mlmod}, {b.haves, hkey2, rec2}, {placement, pkey2, place2},
{resolved, "it2.mailer.domain", []byte(`{"value":"example.org","source":"module"}`)}, {builds, buildKey, firstBuild}} {
if _, err := w.kv.Put(w.key, w.value); err != nil {
t.Fatal(err)
}
}
want, _ := delivery.EncodeWant(delivery.Want{FP: fp, Self: true})
wkey, _ := delivery.WantKey(itNode, "web")
if _, err := b.wants.Put(wkey, want); err != nil {
t.Fatal(err)
}
stop := b.start(false)
// Not placed yet: the agent applies the controller's compositions, of which there are none. The catalogue's seed then
// places the node, and the agent begins again composing its own (ADR 0549 §4), with no restart of the engine.
time.Sleep(300 * time.Millisecond)
if b.file("web") != "" {
t.Fatal("a node the catalogue does not place composed its own")
}
if _, err := placement.Put(pkey, place); err != nil {
t.Fatal(err)
}
if checkBuild {
// The build of the placed module is read through the node's session: one that does not decode is said, and nothing
// is applied; a done one, then a config event, and the node composes.
within(t, 5*time.Second, "the node did not read web's build key through its session", func() bool {
b.mu.Lock()
defer b.mu.Unlock()
return strings.Contains(strings.Join(b.said, "\n"), "the builder's "+buildKey)
})
if b.file("web") != "" {
t.Fatal("composed with a build key that does not decode")
}
if _, err := builds.Put(buildKey, done); err != nil {
t.Fatal(err)
}
if _, err := resolved.Put(itNode+".web.greeting", []byte(`{"value":"bonjour","source":"module"}`)); err != nil {
t.Fatal(err)
}
}
return stop, h8state{placement: placement, resolved: resolved, candidates: candidates, hkey: hkey, place: place, fp: fp}
}
// h8rest is H8 once the node composes its own: the have, a config revision, and the candidate answered "same".
func h8rest(t *testing.T, b *itBed, resolved, candidates nats.KeyValue, hkey string, place []byte, fp string) {
t.Helper()
within(t, 2*time.Second, "the have does not name the build", func() bool {
h := b.have("web")
return h.FP == fp && h.Composed != "" && h.Phase == delivery.PhaseRunning
})
if keys, _ := b.composed.Keys(); len(keys) != 0 {
t.Errorf("the controller's compositions hold %v; the node composed its own", keys)
}
first := b.have("web").Composed
if _, err := resolved.Put(itNode+".web.greeting", []byte(`{"value":"salut","source":"module"}`)); err != nil {
t.Fatal(err)
}
within(t, 3*time.Second, "a changed config value was not composed and applied again", func() bool { return b.file("web") == "greeting=salut\n" })
within(t, 2*time.Second, "the have does not name the revision", func() bool {
h := b.have("web")
return h.FP == fp && h.Composed != first && h.Phase == delivery.PhaseRunning
})
// The node says its facts (its record, as the test wrote it), then a candidate equal to its placement is written: the
// node answers "same" at the candidate's head and revision in its own key, and applies nothing.
b.session.SayFacts(context.Background(), link.Report{Profile: map[string]any{"capabilities": []any{
map[string]any{"name": "service-manager", "present": true}}}})
second := b.have("web").Composed
same, _ := delivery.DecodePlacement(place)
craw, err := delivery.EncodePlacementCandidate(delivery.PlacementCandidate{Proposal: "feat-x", Placement: &same})
if err != nil {
t.Fatal(err)
}
ckey, _ := delivery.PlacementCandidateKey("feat-x", itNode)
crev, err := candidates.Put(ckey, craw)
if err != nil {
t.Fatal(err)
}
var answer delivery.CandidateAnswer
within(t, 10*time.Second, "the node did not answer its candidate", func() bool {
e, err := b.haves.Get(hkey)
if err != nil {
return false
}
n, err := delivery.DecodeHaveNode(e.Value())
answer = n.Candidates["feat-x"]
return err == nil && answer.Proposal == "feat-x" && n.Record != nil
})
if answer.Verdict != delivery.CandidateSame || answer.Revision != crev || len(answer.Same) != 1 || answer.Same[0] != "web" {
t.Errorf("the answer to a candidate equal to the placement: %+v", answer)
}
if h := b.have("web"); h.Composed != second || b.file("web") != "greeting=salut\n" {
t.Errorf("a candidate moved what runs: %+v, %q", h, b.file("web"))
}
}
// 1957/AC1 (integration; ADR 0568 §1): a node agent starting on a bus that holds a `running` have for a module at its want,
// with a service manager (a systemctl on the PATH) that shows the module's unit inactive, starts the unit once and writes
// the have running; the same start with the unit active applies nothing.
func testNatsAnEngineStartStartsADeclaredRunningUnitTheHaveSaysRuns(t *testing.T) {
for _, active := range []bool{false, true} {
b := newItBed(t)
bin := t.TempDir()
stateFile, calls := filepath.Join(bin, "state"), filepath.Join(bin, "calls")
if active {
os.WriteFile(stateFile, []byte("active"), 0o644)
} else {
os.WriteFile(stateFile, []byte("inactive"), 0o644)
}
script := "#!/bin/sh\necho \"$@\" >> " + calls + "\ncase \"$1\" in\n" +
"show) printf 'LoadState=loaded\\nActiveState=%s\\nType=simple\\n' \"$(cat " + stateFile + ")\" ;;\n" +
"start) printf active > " + stateFile + " ;;\nis-enabled) echo enabled ;;\nesac\nexit 0\n"
if err := os.WriteFile(filepath.Join(bin, "systemctl"), []byte(script), 0o755); err != nil {
t.Fatal(err)
}
t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH"))
// The engine's own record of the unit, as a previous run left it, and the have it wrote then.
res, _ := json.Marshal(map[string]any{"id": "web.unit", "type": "service", "unit": "web-it.service", "state": "running", "module": "web"})
var known store.State
known.Record(store.Applied{ID: "web.unit", Type: "service", Target: "web-it.service"})
if err := store.Save(b.opts.state, known); err != nil {
t.Fatal(err)
}
fp := fpIt("a")
c := delivery.Composed{Node: itNode, Module: "web", FP: fp, Resources: []json.RawMessage{res}}
signed, _ := delivery.SignComposed(c, b.controll)
raw, _ := delivery.EncodeComposed(signed)
dig, _ := delivery.ComposedDigestOf(signed)
key, _ := delivery.ComposedKey(itNode, "web", fp)
if _, err := b.composed.Put(key, raw); err != nil {
t.Fatal(err)
}
b.want("web", fp, dig, "")
hkey, _ := delivery.HaveKey(itNode, "web")
have, err := delivery.EncodeHave(delivery.Have{FP: fp, Composed: dig, Phase: delivery.PhaseRunning, Health: delivery.HealthUnknown})
if err != nil {
t.Fatal(err)
}
if _, err := b.haves.Put(hkey, have); err != nil {
t.Fatal(err)
}
stop := b.start(false)
within(t, 5*time.Second, "the unit was not started and the have not running", func() bool {
got, _ := os.ReadFile(stateFile)
h := b.have("web")
return string(got) == "active" && h.Phase == delivery.PhaseRunning && h.Composed == dig
})
time.Sleep(300 * time.Millisecond)
stop()
starts := 0
log, _ := os.ReadFile(calls)
for _, l := range strings.Split(string(log), "\n") {
if strings.HasPrefix(l, "start") {
starts++
}
}
if want := map[bool]int{false: 1, true: 0}[active]; starts != want {
t.Fatalf("unit active=%v: started %d times, want %d", active, starts, want)
}
}
}
+42
View File
@@ -0,0 +1,42 @@
package main
import (
"io/fs"
"os"
"path/filepath"
"strings"
"testing"
)
// R6-A AC8: nothing in the node-engine receives, orders or reconciles a whole declaration (novox/hq ADR 0539 §9, ADR
// 0429 §8): no package subscribes to a node's declare subject or names the deleted symbols.
func TestNoPackageHearsOrOrdersAWholeDeclaration(t *testing.T) {
deleted := []string{
"DeclareSubject", "mesh.node.\" + node + \".declare", "ReconcileDue", "ReconcileEvery", "olderThanKept",
"refuseOlder", "olderGeneration", "appliedGeneration", "reconcileKept", "reapplyKept", "adoptionWatch",
"unsaidFile", "numbersFile", "func gather(", "func handleBody(", "Supersedes(", "ReportSequence",
"RefusedOlder", "ReadsGeneration", "ReadsModules", "GenerationRefusal", "LeftOutModuleOf",
}
root := filepath.Join("..", "..")
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() || !strings.HasSuffix(path, ".go") || strings.HasSuffix(path, "nodeclare_test.go") {
return nil
}
body, err := os.ReadFile(path)
if err != nil {
return err
}
for _, word := range deleted {
if strings.Contains(string(body), word) {
t.Errorf("%s still names %q", path, word)
}
}
return nil
})
if err != nil {
t.Fatal(err)
}
}
+142
View File
@@ -0,0 +1,142 @@
package main
import (
"context"
"encoding/json"
"fmt"
"strings"
"sync"
"time"
"github.com/novox/mesh-host/internal/identity"
)
// This machine's sealing key, stated to the vault in the engine's own name (novox/hq ADR 0372, issue 786). The vault
// records it from the machine's own account alone and, when it is a first or a changed key, says the machine's core
// secrets changed so the controller asks for them again sealed to it. Without this a machine that never moved a
// secret in (adopted after feature 57) has no key at the vault. Nothing but the public half crosses.
const (
// nodeKeyRetryFrom and nodeKeyRetryUpTo bound the wait between tries while the vault does not answer: it starts
// after the bus and may not serve the verb yet.
nodeKeyRetryFrom = 5 * time.Second
nodeKeyRetryUpTo = 5 * time.Minute
// nodeKeyLook is how often the key file is read to see whether it changed.
nodeKeyLook = 30 * time.Second
)
// heardKey is the vault's own public key as its answer to node-key gave it (ADR 0547 clause 9, issue 1945): the key the
// bus node hands the loaded list's hashes sealed to. One for the process, as the key stated is; kick is the follower of
// the user list on the link open now, told when the key is first heard or changes.
type heardKey struct {
mu sync.Mutex
pem string
kick func()
}
// vaultKey is the vault's key this engine heard.
var vaultKey = &heardKey{}
func (h *heardKey) get() string {
h.mu.Lock()
defer h.mu.Unlock()
return h.pem
}
// heard keeps the key a node-key answer gave, and tells the follower when it is a first or changed one. An answer
// with no key (a vault before this one) changes nothing.
func (h *heardKey) heard(answer json.RawMessage) {
var a struct {
VaultKey string `json:"vault_key"`
}
if json.Unmarshal(answer, &a) != nil || !strings.Contains(a.VaultKey, "PUBLIC KEY") {
return
}
h.mu.Lock()
changed := a.VaultKey != h.pem
h.pem = a.VaultKey
kick := h.kick
h.mu.Unlock()
if changed && kick != nil {
kick()
}
}
// following names the follower told when the key changes, and tells it at once when a key is known.
func (h *heardKey) following(kick func()) {
h.mu.Lock()
h.kick = kick
known := h.pem != ""
h.mu.Unlock()
if known && kick != nil {
kick()
}
}
// nodeKeyReask is how often the key is stated again though it did not change, so a changed vault key is heard: the
// vault takes the same key again and changes nothing (ADR 0547 clause 9).
const nodeKeyReask = 10 * time.Minute
// keyHearing is what the keeper does with the vault's own key in node-key's answers (ADR 0547 clause 9): heard takes
// it from an answer; held says whether one is held, and while none is the key is stated again at every look, so a vault
// delivered after this engine is heard within a look; every is how often it is stated again all the same, so a changed
// vault key is heard.
type keyHearing struct {
heard func(json.RawMessage)
held func() bool
// look is how often it is stated again while no vault key is held; every how often once one is.
look, every time.Duration
}
// keepNodeKeyStated states the sealing key load answers to the vault at start, retrying with a growing wait from
// retryFrom to retryUpTo while it does not answer, and states it again whenever the key changes. It says one line
// when a key is stated and one when a try fails for another reason than the last, and returns when ctx ends. With
// hearing, each answer goes to it, and the key is stated again while no vault key is held and at its interval.
func keepNodeKeyStated(ctx context.Context, load func() (identity.SealingKey, error), ask vaultAsk, say func(string),
retryFrom, retryUpTo time.Duration, hearing *keyHearing) {
stated, lastFailure := "", ""
var askedAt time.Time
wait := retryFrom
fail := func(why string) {
if why != lastFailure {
say("mesh-host: the sealing key was not stated to the vault, trying again: " + why)
lastFailure = why
}
wait = min(wait*2, retryUpTo)
}
look := min(nodeKeyLook, retryFrom)
for {
delay := look
key, err := load()
again := false
if hearing != nil && stated != "" {
since := time.Since(askedAt)
held := hearing.held == nil || hearing.held()
again = (!held && since >= hearing.look) || (held && hearing.every > 0 && since >= hearing.every)
}
switch {
case err != nil:
fail(err.Error())
delay = wait
case key.Public != stated || again:
if answer, err := ask("node-key", map[string]string{"key": key.Public}); err != nil {
fail(err.Error())
delay = wait
} else {
askedAt = time.Now()
if hearing != nil && hearing.heard != nil {
hearing.heard(answer)
}
if key.Public != stated {
say(fmt.Sprintf("mesh-host: this machine's sealing key %s is stated to the vault", key.Public))
}
stated, lastFailure, wait = key.Public, "", retryFrom
}
}
select {
case <-ctx.Done():
return
case <-time.After(delay):
}
}
}
+190
View File
@@ -0,0 +1,190 @@
package main
import (
"context"
"encoding/json"
"errors"
"sync"
"testing"
"time"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/link"
)
// Issue 786 (novox/hq ADR 0372): the engine states its own sealing key to the vault's node-key at start, retries
// while the vault does not answer, and states it again when the key changes; the same key is not stated twice.
type keyVault struct {
mu sync.Mutex
stated []string
fail int
verbs []string
// keyless is how many answers carry no vault key (a vault before ADR 0547 clause 9's, or not yet started); key the
// vault's key the others carry.
keyless int
key string
}
func (v *keyVault) ask(verb string, args any) (json.RawMessage, error) {
v.mu.Lock()
defer v.mu.Unlock()
v.verbs = append(v.verbs, verb)
if v.fail > 0 {
v.fail--
return nil, errors.New("the vault does not answer")
}
raw, _ := json.Marshal(args)
var a struct {
Key string `json:"key"`
}
_ = json.Unmarshal(raw, &a)
v.stated = append(v.stated, a.Key)
if v.key == "" || v.keyless > 0 {
v.keyless--
return json.RawMessage(`{"recorded":true}`), nil
}
answer, _ := json.Marshal(map[string]any{"recorded": true, "vault_key": v.key})
return answer, nil
}
func (v *keyVault) said() []string {
v.mu.Lock()
defer v.mu.Unlock()
return append([]string(nil), v.stated...)
}
func waitFor(t *testing.T, what string, ok func() bool) {
t.Helper()
for i := 0; i < 400; i++ {
if ok() {
return
}
time.Sleep(5 * time.Millisecond)
}
t.Fatalf("timed out: %s", what)
}
func TestTheEngineStatesItsSealingKeyToTheVaultAtStartRetriesAndRestatesAChange(t *testing.T) {
first, _ := identity.GenerateSealingKey()
second, _ := identity.GenerateSealingKey()
var mu sync.Mutex
current := first
load := func() (identity.SealingKey, error) {
mu.Lock()
defer mu.Unlock()
return current, nil
}
vault := &keyVault{fail: 2}
var lines []string
var lmu sync.Mutex
say := func(l string) { lmu.Lock(); lines = append(lines, l); lmu.Unlock() }
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
var hmu sync.Mutex
var answers []string
heard := func(a json.RawMessage) { hmu.Lock(); answers = append(answers, string(a)); hmu.Unlock() }
go keepNodeKeyStated(ctx, load, vault.ask, say, time.Millisecond, 5*time.Millisecond, &keyHearing{heard: heard})
waitFor(t, "the first key stated after two refusals", func() bool { return len(vault.said()) == 1 })
// Its answer is handed on, so the vault's own key in it is heard (ADR 0547 clause 9); refusals hand nothing.
waitFor(t, "the answer heard", func() bool { hmu.Lock(); defer hmu.Unlock(); return len(answers) == 1 })
if vault.said()[0] != first.Public {
t.Errorf("stated %q, want the engine's own key %q", vault.said()[0], first.Public)
}
vault.mu.Lock()
verbs := append([]string(nil), vault.verbs...)
vault.mu.Unlock()
for _, v := range verbs {
if v != "node-key" {
t.Errorf("asked %s", v)
}
}
// The same key again is not stated again.
time.Sleep(60 * time.Millisecond)
if n := len(vault.said()); n != 1 {
t.Errorf("the same key was stated %d times", n)
}
mu.Lock()
current = second
mu.Unlock()
waitFor(t, "the changed key stated", func() bool { return len(vault.said()) == 2 })
if vault.said()[1] != second.Public {
t.Errorf("stated %q after the change, want %q", vault.said()[1], second.Public)
}
lmu.Lock()
defer lmu.Unlock()
if len(lines) == 0 {
t.Error("the engine said nothing")
}
asksAgainUntilItHearsTheVaultsKey(t)
}
// ADR 0547 clause 9: the vault's key is heard though the engine's own key never changes. While none is held the key
// is stated again at each look, so a vault delivered after the engine (or answering no key yet) is heard; once one is
// held, again only at the slow interval, so a changed vault key is heard; never more.
func asksAgainUntilItHearsTheVaultsKey(t *testing.T) {
sealing, _ := identity.GenerateSealingKey()
load := func() (identity.SealingKey, error) { return sealing, nil }
pem := func(c string) string { return "-----BEGIN PUBLIC KEY-----\n" + c + "\n-----END PUBLIC KEY-----\n" }
for _, c := range []struct {
name string
keyless int
every time.Duration
change bool
wantAtLeast int
wantAtMost int
}{
{name: "a vault with no key yet is asked again until it answers one, then not", keyless: 3, wantAtLeast: 4, wantAtMost: 4},
{name: "a held key is asked again at the slow interval, and a changed one heard", every: 20 * time.Millisecond,
change: true, wantAtLeast: 3, wantAtMost: 1 << 20},
} {
t.Run(c.name, func(t *testing.T) {
vault := &keyVault{keyless: c.keyless, key: pem("AAAA")}
h := &heardKey{}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go keepNodeKeyStated(ctx, load, vault.ask, func(string) {}, time.Millisecond, 5*time.Millisecond,
&keyHearing{heard: h.heard, held: func() bool { return h.get() != "" }, look: time.Millisecond, every: c.every})
waitFor(t, "the vault's key heard", func() bool { return h.get() == pem("AAAA") })
if c.change {
vault.mu.Lock()
vault.key = pem("BBBB")
vault.mu.Unlock()
waitFor(t, "the changed vault key heard", func() bool { return h.get() == pem("BBBB") })
}
time.Sleep(60 * time.Millisecond)
n := len(vault.said())
if n < c.wantAtLeast || n > c.wantAtMost {
t.Errorf("node-key asked %d times, want %d to %d", n, c.wantAtLeast, c.wantAtMost)
}
})
}
}
// The statement a host makes names this contract (issue 786), one above the vault's box. And the vault's key its
// answer gives is kept, told to the follower once when first heard and again only when changed; an answer without one
// (an older vault) or with no public key changes nothing (ADR 0547 clause 9).
func TestTheNodeKeyContractIsThirteen(t *testing.T) {
if link.NodeKeyContract != 13 {
t.Errorf("NodeKeyContract %d", link.NodeKeyContract)
}
h := &heardKey{}
kicks := 0
h.following(func() { kicks++ })
h.heard(json.RawMessage(`{"recorded":true}`))
h.heard(json.RawMessage(`{"recorded":true,"vault_key":"not a key"}`))
if h.get() != "" || kicks != 0 {
t.Fatalf("an answer with no key was kept: %q, %d kicks", h.get(), kicks)
}
pem := "-----BEGIN PUBLIC KEY-----\nAAAA\n-----END PUBLIC KEY-----\n"
answer, _ := json.Marshal(map[string]any{"recorded": true, "vault_key": pem})
h.heard(answer)
h.heard(answer)
if h.get() != pem || kicks != 1 {
t.Fatalf("key %q, %d kicks, want the key and one", h.get(), kicks)
}
h.following(func() { kicks += 10 })
if kicks != 11 {
t.Fatalf("a follower named after the key was heard was not told: %d", kicks)
}
}
-95
View File
@@ -1,95 +0,0 @@
package main
import (
"encoding/json"
"strings"
"testing"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/store"
)
// A declaration carries no order, so a host cannot tell an older one from a newer (novox/hq
// 04-ISSUES/107). Its only identity was the digest of its bytes: "not the last" could be said,
// "older" could not.
func keptWith(t *testing.T, sequence int64) store.Declared {
t.Helper()
return keptAt(t, link.Order{Sequence: sequence})
}
func keptAt(t *testing.T, order link.Order) store.Declared {
t.Helper()
body, err := json.Marshal(map[string]any{
"declaration": 1, "resources": []any{}, "owns_nothing": true, "sequence": order.Sequence,
"epoch": order.Epoch,
})
if err != nil {
t.Fatal(err)
}
return store.Declared{Declaration: body, Signature: []byte("x")}
}
func TestAnOlderDeclarationFromTheMeshIsRefused(t *testing.T) {
err := refuseOlder(keptWith(t, 7), nil, link.Order{Sequence: 5})
if err == nil {
t.Fatal("sequence 5 was accepted over a kept 7")
}
if !strings.Contains(err.Error(), "older") || !strings.Contains(err.Error(), "nothing was applied") {
t.Fatalf("the refusal does not say what it is: %v", err)
}
}
func TestANewerOrEqualDeclarationIsNot(t *testing.T) {
if err := refuseOlder(keptWith(t, 7), nil, link.Order{Sequence: 8}); err != nil {
t.Fatalf("sequence 8 was refused over a kept 7: %v", err)
}
// Equal is the same declaration again, which reconciling is for.
if err := refuseOlder(keptWith(t, 7), nil, link.Order{Sequence: 7}); err != nil {
t.Fatalf("the same sequence was refused: %v", err)
}
}
func TestNoOrderClaimedMeansNoOrderCompared(t *testing.T) {
// An older controller sends none; a host that received before it understood them kept none.
// Refusing on a guess would strand a node the moment the controller is older than the host.
if err := refuseOlder(keptWith(t, 7), nil, link.Order{Sequence: 0}); err != nil {
t.Fatalf("a declaration claiming no order was refused: %v", err)
}
if err := refuseOlder(keptWith(t, 0), nil, link.Order{Sequence: 3}); err != nil {
t.Fatalf("a declaration was refused against a kept one that claimed no order: %v", err)
}
if err := refuseOlder(store.Declared{}, store.ErrNothingDeclared, link.Order{Sequence: 3}); err != nil {
t.Fatalf("a first declaration was refused: %v", err)
}
}
// **The lease epoch decides first** (novox/hq to-be 45 §6): a declaration from a controller that lost
// its lease is refused whatever its sequence, and a new lease holder's is taken whatever its sequence.
func TestAnOlderEpochIsRefusedAndANewerTaken(t *testing.T) {
kept := keptAt(t, link.Order{Epoch: 57, Sequence: 3})
err := refuseOlder(kept, nil, link.Order{Epoch: 41, Sequence: 12})
if err == nil || !strings.Contains(err.Error(), "epoch 41") || !strings.Contains(err.Error(), "epoch 57") {
t.Fatalf("an older epoch was not refused by name: %v", err)
}
if err := refuseOlder(kept, nil, link.Order{Epoch: 58, Sequence: 1}); err != nil {
t.Fatalf("a new lease holder's first declaration was refused for its sequence: %v", err)
}
if err := refuseOlder(kept, nil, link.Order{Epoch: 57, Sequence: 2}); err == nil {
t.Fatal("a lower sequence under the same epoch was accepted")
}
// A controller with no lease — older, or rolled back to a build from before it — is today's
// behaviour: no epoch compared.
if err := refuseOlder(kept, nil, link.Order{Sequence: 4}); err != nil {
t.Fatalf("a declaration with no epoch was refused for one: %v", err)
}
if held, older := olderThanKept(kept, nil, link.Order{Epoch: 41, Sequence: 12}); !older ||
held != (link.Order{Epoch: 57, Sequence: 3}) {
t.Fatalf("olderThanKept said %v, holding %+v", older, held)
}
// What is kept cannot be read: no order to compare, so nothing is refused on a guess.
if _, older := olderThanKept(store.Declared{Declaration: []byte("garbled")}, nil,
link.Order{Epoch: 41, Sequence: 1}); older {
t.Fatal("refused against a kept declaration nobody can read")
}
}
+94
View File
@@ -0,0 +1,94 @@
package main
import (
"context"
"encoding/json"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/ownresync"
"github.com/novox/mesh-host/internal/selfcompose"
)
// composesNow is whose compositions this run of the agent applies, once decided ("" before).
type composesNow struct {
mu sync.Mutex
now string
}
func (c *composesNow) set(s string) { c.mu.Lock(); c.now = s; c.mu.Unlock() }
func (c *composesNow) get() string { c.mu.Lock(); defer c.mu.Unlock(); return c.now }
// resyncStep is the step that hands the vault what runs of an own secret a shadow answer says another value of (issue
// 1126, cause C4; internal/ownresync): one pass at a time, and the last answers taken again once the agent has decided
// whose compositions it applies.
type resyncStep struct {
r *ownresync.Resync
one sync.Mutex
mu sync.Mutex
last map[string]delivery.CandidateAnswer
// tries and pending are the step taken again after a bound when a read or an ask the bus did not answer stopped it
// (Outcome.Retry, issue 1126, 2026-10-28), one wait at a time.
tries int
pending bool
}
func ownResync(session *link.Session, node string, say func(string), env selfcompose.Env, composes *composesNow) *resyncStep {
return &resyncStep{r: ownresync.New(ownresync.Env{Node: node, Composes: composes.get, Running: env.Running,
Get: env.Get, Unseal: env.Unseal, Seal: identity.Seal, Say: say,
Ask: func(ctx context.Context, verb string, args map[string]any) (json.RawMessage, error) {
return session.AskVault(ctx, verb, args)
}})}
}
// take keeps answers as the last and takes them.
func (s *resyncStep) take(ctx context.Context, answers map[string]delivery.CandidateAnswer) {
s.mu.Lock()
s.last = answers
s.mu.Unlock()
s.again(ctx)
}
// again takes the last answers, and once more after a bound (selfcompose.LaterWait) while the bus did not answer for one.
func (s *resyncStep) again(ctx context.Context) {
s.one.Lock()
defer s.one.Unlock()
s.mu.Lock()
last := s.last
s.mu.Unlock()
if last == nil {
return
}
retry := false
for _, o := range s.r.Take(ctx, last) {
retry = retry || o.Retry
}
s.mu.Lock()
defer s.mu.Unlock()
if !retry {
s.tries = 0
return
}
if s.pending || ctx.Err() != nil {
return
}
wait := selfcompose.LaterWait(s.tries)
s.tries++
s.pending = true
go func() {
select {
case <-ctx.Done():
case <-time.After(wait):
}
s.mu.Lock()
s.pending = false
s.mu.Unlock()
if ctx.Err() == nil {
s.again(ctx)
}
}()
}
+1 -1
View File
@@ -6,8 +6,8 @@ import (
"os/exec"
"sort"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
+1 -82
View File
@@ -8,8 +8,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/link"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/system"
)
@@ -127,83 +126,3 @@ func TestAPackageManagerThatIsNotThereRefusesWhole(t *testing.T) {
t.Error("the refused declaration touched the machine")
}
}
// Defends the review of novox/hq issue 233: **a re-apply whose result changed is said unasked**. A machine
// reconciles every five minutes and speaks only when its account is news; how the apply went was not part of
// it, so a package manager that stopped answering between deliveries was never said, and one that answered
// again never cleared. Now a failure is said once, the same failure again is not, and a clean re-apply after it
// is — which is what clears the mesh's condition.
func TestAReapplyWhoseResultChangedIsSaidUnasked(t *testing.T) {
dir := t.TempDir()
conf := filepath.Join(dir, "elsewhere", "pacman.conf") // not declared: the apply cannot make it answer
pacmanAnsweringOnce(t, conf)
opts := options{state: filepath.Join(dir, "state.json")}
raw := []byte(`{"declaration":1,"resources":[` +
`{"id":"motd","type":"file","path":"` + filepath.Join(dir, "motd") + `","content":"hello\n"},` +
`{"id":"pacman.base","type":"package","package":"base"}]}`)
w := &adoptionWatch{}
// Clean once, as it was before: base installed while pacman answered.
if err := os.MkdirAll(filepath.Dir(conf), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(conf, []byte("[options]\n"), 0o644); err != nil {
t.Fatal(err)
}
clean := applyAndKeep(context.Background(), opts, raw, nil, nil, nil)
if faulty(clean) {
t.Fatalf("the first apply was not clean: %+v", clean)
}
w.said(clean)
if w.news(applyAndKeep(context.Background(), opts, raw, nil, nil, nil)) {
t.Error("a clean re-apply after a clean one was said, so every machine speaks every five minutes")
}
// The configuration goes: the re-apply fails, and that is news once.
if err := os.Remove(conf); err != nil {
t.Fatal(err)
}
failed := applyAndKeep(context.Background(), opts, raw, nil, nil, nil)
if !faulty(failed) {
t.Fatalf("the re-apply did not fail: %+v", failed)
}
if !w.news(failed) {
t.Fatal("a re-apply that failed after a clean one was not said, so the mesh never hears the machine fails")
}
w.said(failed)
if w.news(applyAndKeep(context.Background(), opts, raw, nil, nil, nil)) {
t.Error("the same failure was said again on the next re-apply")
}
// It comes back: the clean re-apply is news, and is what clears the mesh's condition.
if err := os.WriteFile(conf, []byte("[options]\n"), 0o644); err != nil {
t.Fatal(err)
}
again := applyAndKeep(context.Background(), opts, raw, nil, nil, nil)
if faulty(again) {
t.Fatalf("the re-apply did not come back clean: %+v", again)
}
if !w.news(again) {
t.Fatal("a clean re-apply after a failed one was not said, so the mesh's condition never clears")
}
}
// A refusal is news too, once, and a reconcile with nothing to say after a clean one stays silent.
func TestARefusalIsSaidOnceAndNothingIsNotSaid(t *testing.T) {
w := &adoptionWatch{}
w.said(link.Report{})
if w.news(link.Report{}) {
t.Error("nothing was said again")
}
refused := link.Report{Refused: "pacman does not answer here"}
if !w.news(refused) {
t.Fatal("a refusal was not said")
}
w.said(refused)
if w.news(link.Report{Refused: "pacman does not answer here, at another time"}) {
t.Error("the same refusal in other words was said again")
}
if !w.news(link.Report{}) {
t.Error("a clean apply after a refusal was not said")
}
}
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+96
View File
@@ -0,0 +1,96 @@
package main
import (
"context"
"encoding/json"
"path/filepath"
"testing"
"git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-host/internal/liveness"
"github.com/novox/mesh-host/internal/store"
)
// The record check reads the engine's applied state: a composition one of whose resources has no record there is not
// recorded (issue 1945).
func TestACompositionIsRecordedOnlyWhenEveryResourceIs(t *testing.T) {
path := filepath.Join(t.TempDir(), "state.json")
var s store.State
s.Record(store.Applied{ID: "ca-trust.trust", Type: "file", Target: "/etc/ca"})
if err := store.Save(path, s); err != nil {
t.Fatal(err)
}
c := delivery.Composed{Node: "ace", Module: "ca-trust", Resources: []json.RawMessage{
json.RawMessage(`{"id":"ca-trust.trust","type":"file","module":"ca-trust","path":"/etc/ca","mode":"0644","content":"x"}`)}}
recorded := recordedIn(path, func(string) {})
if !recorded(context.Background(), c) {
t.Fatal("a composition whose every resource has a record was not recorded")
}
c.Resources = append(c.Resources, json.RawMessage(
`{"id":"ca-trust.more","type":"file","module":"ca-trust","path":"/etc/ca2","mode":"0644","content":"y"}`))
if recorded(context.Background(), c) {
t.Fatal("a composition with a resource the state holds no record of was recorded")
}
}
// The modules applied again are those with a missing resource, each once; one down but there is not among them.
func TestTheModulesAppliedAgainAreThoseWithAMissingResource(t *testing.T) {
t.Run("1957: the machine shows a resource not active", testTheMachineShowsAResourceNotActive)
st := liveness.Statement{Resources: []liveness.State{
{Resource: liveness.Resource{Module: "matrix", ID: "matrix.homeserver"}, State: liveness.Unhealthy, Missing: true},
{Resource: liveness.Resource{Module: "matrix", ID: "matrix.db"}, State: liveness.Unhealthy, Missing: true},
{Resource: liveness.Resource{Module: "ca-trust", ID: "ca-trust.trust"}, State: liveness.Unhealthy, Missing: true},
{Resource: liveness.Resource{Module: "web", ID: "web.server"}, State: liveness.Unhealthy},
}}
if got := missingModules(st); len(got) != 2 || got[0] != "ca-trust" || got[1] != "matrix" {
t.Fatalf("applied again: %v", got)
}
}
// A module is applied again only for a resource newly missing: one still missing at the next change of the statement is
// not acted on again, so a resource that cannot be made is not applied (and backed up) at every unrelated change. One
// that comes back and goes again is newly missing again (the review of issue 1945).
func TestOnlyANewlyMissingResourceAppliesItsModuleAgain(t *testing.T) {
t.Run("1957: a resource newly down is an edge", testOnlyAResourceNewlyDownIsAnEdge)
missing := func(ids ...string) liveness.Statement {
var st liveness.Statement
for _, id := range ids {
st.Resources = append(st.Resources, liveness.State{Resource: liveness.Resource{Module: "matrix", ID: id},
State: liveness.Unhealthy, Missing: true})
}
return st
}
var last map[string]bool
var got []string
got, last = newlyMissing(last, missing("matrix.homeserver"))
if len(got) != 1 {
t.Fatalf("first missing: %v", got)
}
got, last = newlyMissing(last, missing("matrix.homeserver"))
if len(got) != 0 {
t.Fatalf("still missing was acted on again: %v", got)
}
got, last = newlyMissing(last, missing())
got, last = newlyMissing(last, missing("matrix.homeserver"))
if len(got) != 1 {
t.Fatalf("missing again after it came back: %v", got)
}
}
// A file the provisioner client fills is left to it by the apply and has no record until the client writes it: it does
// not make its composition unrecorded (the review of issue 1945).
func TestAFilledFileDoesNotMakeItsCompositionUnrecorded(t *testing.T) {
path := filepath.Join(t.TempDir(), "state.json")
var s store.State
s.Record(store.Applied{ID: "ca-trust.trust", Type: "file", Target: "/etc/ca"})
if err := store.Save(path, s); err != nil {
t.Fatal(err)
}
c := delivery.Composed{Node: "ace", Module: "ca-trust", Filled: []string{"/etc/ca-filled"}, Resources: []json.RawMessage{
json.RawMessage(`{"id":"ca-trust.trust","type":"file","module":"ca-trust","path":"/etc/ca","mode":"0644","content":"x"}`),
json.RawMessage(`{"id":"ca-trust.filled","type":"file","module":"ca-trust","path":"/etc/ca-filled","mode":"0644","owner-at-runtime":"owned"}`)}}
if !recordedIn(path, func(string) {})(context.Background(), c) {
t.Fatal("a filled file the client has not written made its composition unrecorded")
}
}
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
+8 -5
View File
@@ -5,8 +5,8 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/accounts"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/liveness"
"github.com/novox/mesh-host/internal/units"
@@ -87,10 +87,13 @@ func TestTheStatementSaysAnAccountJudgedForRoot(t *testing.T) {
Groups: []string{"openrazer"}}, State: accounts.Healthy, Since: at},
}}
h := withAccounts(healthAsReported(liveness.Statement{At: at}, nil), as)
// At RootContract or later: an engine that also reads an identity's runner says OwnGroupContract (novox/hq ADR
// 0334), one above ADR 0328's LibraryContract.
if h.Contract < link.RootContract || link.RootContract != 3 || h.Contract != link.OwnGroupContract ||
link.OwnGroupContract != link.LibraryContract+1 {
// At RootContract or later: an engine that states its sealing key to the vault says NodeKeyContract (ADR 0372, issue 786), one above
// VaultBoxContract (it opens the vault's box for the core's credentials, ADR 0349 §16, issue 755), one above HandedContract (design 57 P7), one above ProvisionerContract (novox/hq ADR
// 0349 §4), one above ADR 0349 §5's OwnershipContract.
if h.Contract < link.RootContract || link.RootContract != 3 || h.Contract != link.NodeKeyContract ||
link.NodeKeyContract != link.VaultBoxContract+1 || link.VaultBoxContract != link.HandedContract+1 ||
link.HandedContract != link.ProvisionerContract+1 || link.ProvisionerContract != link.OwnershipContract+1 || link.OwnershipContract != link.BusyContract+1 ||
link.BusyContract != link.OwnGroupContract+1 {
t.Fatalf("contract %d", h.Contract)
}
got := map[string]string{}
-64
View File
@@ -1,64 +0,0 @@
package main
import (
"os"
"path/filepath"
"reflect"
"testing"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/store"
)
// The last apply's report, kept beside the state until the mesh takes it (novox/hq issue 264).
// What a host that died between applying and reporting kept is read back by the next one exactly —
// the digest and the outcome, never a new account of the machine.
func TestTheKeptReportSurvivesTheHost(t *testing.T) {
state := filepath.Join(t.TempDir(), "state.json")
made := link.Report{Declared: "sha256:d1", Applied: []string{"a"}, Failed: map[string]string{"b": "no room"},
Host: "v2"}
if err := unsaidBeside(state).Keep(made); err != nil {
t.Fatal(err)
}
kept, ok, err := unsaidBeside(state).Pending()
if err != nil || !ok {
t.Fatalf("nothing kept: %v", err)
}
if !reflect.DeepEqual(kept, made) {
t.Fatalf("kept %+v, made %+v", kept, made)
}
// Taken about another declaration, it stays; taken about this one, it goes.
if err := unsaidBeside(state).Said("sha256:other"); err != nil {
t.Fatal(err)
}
if _, ok, _ := unsaidBeside(state).Pending(); !ok {
t.Fatal("a report about another declaration cleared this one")
}
if err := unsaidBeside(state).Said("sha256:d1"); err != nil {
t.Fatal(err)
}
if _, err := os.Stat(store.UnsaidPath(state)); !os.IsNotExist(err) {
t.Fatalf("the report was taken and is still kept: %v", err)
}
}
// A node that never applied anything has nothing to say again; and an apply refused before its
// declaration was read clears an older report, which would otherwise be re-said after it.
func TestNothingKeptWhenNothingWasApplied(t *testing.T) {
state := filepath.Join(t.TempDir(), "state.json")
if _, ok, err := unsaidBeside(state).Pending(); ok || err != nil {
t.Fatalf("a node that applied nothing has a report to say: %v %v", ok, err)
}
if err := unsaidBeside(state).Keep(link.Report{Declared: "sha256:d1"}); err != nil {
t.Fatal(err)
}
if err := unsaidBeside(state).Keep(link.Report{Refused: "forged"}); err != nil {
t.Fatal(err)
}
if _, ok, _ := unsaidBeside(state).Pending(); ok {
t.Fatal("an older report is kept after a later declaration was refused")
}
}
+204
View File
@@ -0,0 +1,204 @@
package main
import (
"context"
"crypto/ed25519"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"git.novox.be/novox/mesh-sdk/go/declaration"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/seal"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
"github.com/novox/mesh-host/internal/userlist"
)
// The bridge of novox/hq ADR 0547 §8, wired: the bus node's engine writes the bus's user list alone from the newest
// composition of the bus module, under the apply's two locks as a filled file is written (filling.go), and says what
// it wrote in the bus module's have.
// userListFollower is the follower of the bus's user list for one link: the node's composition bucket, the keys the
// agent trusts, the write alone, and the have.
func userListFollower(opts options, session *link.Session, composedBucket string, trusted func() []ed25519.PublicKey,
say link.Announce, wrote func()) *userlist.Follower {
return userlist.New(userlist.Env{
Node: session.Membership.Node,
Composed: bucketWatch{session, composedBucket},
Trusted: trusted,
Resolve: func(ctx context.Context, raw json.RawMessage) ([]byte, error) {
return resolveUserList(session.WithLink(ctx), raw)
},
OnDisk: userListOnDisk,
Write: func(ctx context.Context, module string, raw json.RawMessage, body []byte) (string, bool, error) {
digest, changed, err := writeUserListAlone(session.WithLink(ctx), opts, raw, body, say)
if changed && wrote != nil {
wrote()
}
return digest, changed, err
},
Report: func(ctx context.Context, module string, w userlist.Written) {
session.NoteUserList(ctx, module, w)
},
// From take-over, the grants part's list (ADR 0547 clauses 7, 8): its one key watched, its object read by name and
// opened with this machine's sealing key, the key's public half said in the have so the grants part seals to it.
Grants: bucketWatch{session, delivery.GrantsUserListBucket},
Fetch: func(ctx context.Context, name string) ([]byte, error) {
return session.GrantsObject(ctx, name, grantsListLimit)
},
Open: func(b seal.Box) ([]byte, error) {
key, err := identity.LoadSealingKey(identity.SealingKeyPath(opts.state))
if err != nil {
return nil, err
}
return key.OpenVaultBox(b)
},
Key: func() string {
key, err := identity.LoadSealingKey(identity.SealingKeyPath(opts.state))
if err != nil {
return ""
}
pem, err := key.PublicPEM()
if err != nil {
return ""
}
return pem
},
// The grants part's key as last heard, kept beside the node's state, followed while it cannot be watched; its
// watch tried again as a passing failure of the bus is (ADR 0547 clause 8).
Kept: func() ([]byte, uint64) { return keptGrantsKey(opts) },
Keep: func(value []byte, rev uint64) { keepGrantsKey(opts, value, rev, say) },
Backoff: link.Backoff,
// The vault's key from its answer to node-key: the loaded list's hashes are handed sealed to it (ADR 0547
// clause 9).
VaultKey: vaultKey.get,
Say: announceOr(say),
})
}
// grantsListLimit is the largest sealed list the bus node reads: the live list is about a hundred kilobytes.
const grantsListLimit = 16 << 20
// userListFile is a user-list resource as the file it declares.
func userListFile(raw []byte) (*declaration.File, error) {
r, err := declaration.ParseResource(raw)
if err != nil {
return nil, fmt.Errorf("the user list is not a resource: %w", err)
}
f, ok := r.(*declaration.File)
if !ok {
return nil, fmt.Errorf("the user list is a %s, not a file", r.Kind())
}
return f, nil
}
// resolveUserList is the list a user-list resource holds: its content, or the object its digest names in this node's
// file bucket, fetched and verified (apply.ResolveFileReferences, ADR 0486 §1).
func resolveUserList(ctx context.Context, raw []byte) ([]byte, error) {
f, err := userListFile(raw)
if err != nil {
return nil, err
}
d := &declaration.Declaration{Version: declaration.Version, Resources: []declaration.Resource{f}}
if err := apply.ResolveFileReferences(ctx, d, link.FilesFrom(ctx)); err != nil {
return nil, fmt.Errorf("the user list could not be had by its digest: %w", err)
}
return []byte(d.Resources[0].(*declaration.File).Content), nil
}
// userListOnDisk is what the user list's file holds on this machine now; nil when there is none.
func userListOnDisk(raw json.RawMessage) ([]byte, error) {
f, err := userListFile(raw)
if err != nil {
return nil, err
}
body, err := os.ReadFile(f.Path)
if errors.Is(err, os.ErrNotExist) {
return nil, nil
}
return body, err
}
// writeUserListAlone writes one user-list resource alone (apply.WriteAlone) with body as its content, the node's state
// read and saved under the apply's locks, and answers the digest of the file on the machine after.
func writeUserListAlone(ctx context.Context, opts options, raw []byte, body []byte, say link.Announce) (string, bool, error) {
declared, err := userListFile(raw)
if err != nil {
return "", false, err
}
file := *declared
file.Content, file.Digest, file.Size = string(body), "", 0
applying.Lock()
defer applying.Unlock()
unlock, err := store.Lock(opts.state, nil)
if err != nil {
return "", false, err
}
defer unlock()
known, err := store.Load(opts.state)
if err != nil {
return "", false, err
}
built, err := system.For(builtFor)
if err != nil {
return "", false, err
}
out, updated, writeErr := apply.WriteAlone(ctx, built, &file, known, apply.ExecRunner, announceOr(say))
// Saved whichever way it went, as an apply's: what was written is on the machine either way.
if err := store.Save(opts.state, updated); err != nil {
writeErr = errors.Join(writeErr, errors.New("written, and the node's state could not be saved: "+err.Error()))
}
digest := ""
if body, err := os.ReadFile(file.Path); err == nil {
digest = userlist.Digest(body)
}
return digest, writeErr == nil && out.Action != "unchanged", writeErr
}
// grantsKeyPath is where the grants part's user-list key last heard is kept, beside the node's state.
func grantsKeyPath(opts options) string {
return filepath.Join(filepath.Dir(opts.state), "grants-user-list.json")
}
type keptKey struct {
Revision uint64 `json:"revision"`
Value json.RawMessage `json:"value"`
}
func keptGrantsKey(opts options) ([]byte, uint64) {
raw, err := os.ReadFile(grantsKeyPath(opts))
if err != nil {
return nil, 0
}
var k keptKey
if json.Unmarshal(raw, &k) != nil || len(k.Value) == 0 {
return nil, 0
}
return k.Value, k.Revision
}
func keepGrantsKey(opts options, value []byte, rev uint64, say link.Announce) {
path := grantsKeyPath(opts)
if value == nil {
if err := os.Remove(path); err != nil && !errors.Is(err, os.ErrNotExist) {
announceOr(say)("the grants part's user-list key kept on this machine could not be taken away: " + err.Error())
}
return
}
raw, _ := json.Marshal(keptKey{Revision: rev, Value: value})
tmp := path + ".new"
err := os.WriteFile(tmp, raw, 0o600)
if err == nil {
err = os.Rename(tmp, path)
}
if err != nil {
announceOr(say)("the grants part's user-list key could not be kept on this machine: " + err.Error())
}
}
+153
View File
@@ -0,0 +1,153 @@
package main
// Issue 755's proof on a real bus (novox/hq ADR 0349 §16): a core credential the vault sealed to this machine crosses
// the bus inside the declaration, which the controller carries and cannot open, and this engine opens it with its
// sealing key and writes the value. A listener on every subject keeps every byte that crossed; the value is never
// among them. Run against the throwaway bus the build module starts for the integration level (MESH_TEST_NATS).
import (
"context"
"encoding/base64"
"encoding/json"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/seal"
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
const coreCredential = "the-provisioner-clients-bus-password-2025"
func TestNatsTheEngineFillsACoreCredentialFromTheVaultsBoxWithNoValueInTheClear(t *testing.T) {
url := os.Getenv("MESH_TEST_NATS")
if url == "" {
t.Skip("MESH_TEST_NATS unset")
}
// This machine's sealing key, where the engine reads it.
state := filepath.Join(t.TempDir(), "state.json")
key, err := identity.GenerateSealingKey()
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(identity.SealingKeyPath(state), []byte(key.Private), 0o600); err != nil {
t.Fatal(err)
}
// Every byte that crosses.
tap, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer tap.Close()
var mu sync.Mutex
var crossed []string
if _, err := tap.Subscribe(">", func(m *nats.Msg) {
mu.Lock()
crossed = append(crossed, m.Subject+" "+string(m.Data))
mu.Unlock()
}); err != nil {
t.Fatal(err)
}
_ = tap.Flush()
// The engine hears the composition on a subject of the test (the node agent reads it from its bucket).
const subject = "mesh.test.vaultbox.composition"
engine, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer engine.Close()
heard, err := engine.SubscribeSync(subject)
if err != nil {
t.Fatal(err)
}
_ = engine.Flush()
// The vault's answer to the controller: the value in the ADR 0206 box, sealed to this machine's key. The
// controller carries it in the declaration as it is.
pub, err := key.PublicPEM()
if err != nil {
t.Fatal(err)
}
box, err := seal.Seal(coreCredential, pub)
if err != nil {
t.Fatal(err)
}
path := filepath.Join(t.TempDir(), "broker")
body, _ := json.Marshal(map[string]any{"declaration": 1, "resources": []map[string]any{
{"id": "need-provisioner-broker", "type": "file", "path": path, "vault-sealed": box}}})
controller, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer controller.Close()
if err := controller.Publish(subject, body); err != nil {
t.Fatal(err)
}
_ = controller.Flush()
msg, err := heard.NextMsg(5 * time.Second)
if err != nil {
t.Fatalf("the declaration did not reach the engine: %v", err)
}
d, err := declaration.Parse(msg.Data)
if err != nil {
t.Fatal(err)
}
sys, err := system.For("arch")
if err != nil {
t.Fatal(err)
}
if _, _, err := apply.Apply(context.Background(), sys, d, store.State{}, store.OriginCarried,
func(context.Context, string, ...string) (string, error) { return "", nil }, nil, sealOpener(state)); err != nil {
t.Fatal(err)
}
on, err := os.ReadFile(path)
if err != nil || string(on) != coreCredential {
t.Fatalf("the engine wrote %q, %v", on, err)
}
_ = tap.Flush()
time.Sleep(100 * time.Millisecond)
mu.Lock()
all := strings.Join(crossed, "\n")
mu.Unlock()
if !strings.Contains(all, subject) || !strings.Contains(all, "vault-sealed") {
t.Fatalf("the listener heard nothing of the declaration:\n%s", all)
}
for _, form := range []string{coreCredential, base64.StdEncoding.EncodeToString([]byte(coreCredential))} {
if strings.Contains(all, form) {
t.Fatal("the core credential crossed the bus in the clear")
}
}
}
// The engine opens a value the controller sealed (the NaCl box) as before, and the vault's box, with one key.
func TestTheEnginesOpenerOpensBothBoxes(t *testing.T) {
state := filepath.Join(t.TempDir(), "state.json")
key, _ := identity.GenerateSealingKey()
if err := os.WriteFile(identity.SealingKeyPath(state), []byte(key.Private), 0o600); err != nil {
t.Fatal(err)
}
open := sealOpener(state)
nacl, _ := identity.Seal(key.Public, []byte("old"))
if got, err := open(nacl); err != nil || string(got) != "old" {
t.Errorf("the controller's box opened to %q, %v", got, err)
}
pub, _ := key.PublicPEM()
box, _ := seal.Seal("new", pub)
raw, _ := json.Marshal(box)
if got, err := open(string(raw)); err != nil || string(got) != "new" {
t.Errorf("the vault's box opened to %q, %v", got, err)
}
}
+139
View File
@@ -0,0 +1,139 @@
package main
import (
"crypto/ed25519"
"encoding/json"
"errors"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-host/internal/agent"
)
func aFileOf(module, id string) json.RawMessage {
raw, _ := json.Marshal(map[string]any{"id": module + "." + id, "type": "file", "path": "/tmp/r6/" + module + "/" + id,
"content": "x", "module": module})
return raw
}
// What runs is the agent's compositions and the kept declaration's resources for each module the agent does not hold;
// the node's own parts are the kept declaration's until the agent holds the node's composition (ADR 0539 §9). One module
// under the agent must not make the rest invisible to the liveness, units, accounts, libraries, filling and the take act.
func TestWhatRunsIsTheAgentsCompositionsAndTheKeptRest(t *testing.T) {
kept := &declaration.Declaration{Identities: []declaration.Identity{{Module: "db", Name: "mesh-db"}},
Libraries: []declaration.Library{{Library: "media"}}}
everything := []delivery.Composed{
{Node: "n1", Module: "web", Resources: []json.RawMessage{aFileOf("web", "a")}},
{Node: "n1", Module: "db", Resources: []json.RawMessage{aFileOf("db", "a")}}, // still the kept declaration's
}
d, err := whatRuns(everything, false, func() (*declaration.Declaration, error) { return kept, nil })
if err != nil {
t.Fatal(err)
}
if len(d.Resources) != 2 || len(d.Identities) != 1 || len(d.Libraries) != 1 {
t.Fatalf("%d resources, %d identities, %d libraries: the kept modules and the node's kept parts are missing",
len(d.Resources), len(d.Identities), len(d.Libraries))
}
// Once the agent holds the node's composition, its parts are the node's.
node := delivery.Composed{Node: "n1", Own: delivery.NodeOwn{Identities: []declaration.Identity{{Module: "web", Name: "mesh-web"}}}}
d, err = whatRuns(append(everything, node), true, func() (*declaration.Declaration, error) { return nil, errors.New("not read") })
if err != nil {
t.Fatal(err)
}
if len(d.Identities) != 1 || d.Identities[0].Name != "mesh-web" || len(d.Libraries) != 0 {
t.Fatalf("the node's parts: %+v %+v", d.Identities, d.Libraries)
}
}
// What the take act, the judgement of directories and the filling read is what runs, not the frozen kept declaration.
func TestDeclaredNowReadsWhatRunsWhenTheAgentIsRunning(t *testing.T) {
setRunsHere(func(string, ed25519.PublicKey) (*declaration.Declaration, error) {
return &declaration.Declaration{Version: 1, Resources: nil, Provisioner: "from-agent"}, nil
})
defer setRunsHere(nil)
d, err := declaredNow("/nowhere", nil)
if err != nil || d.Provisioner != "from-agent" {
t.Fatalf("declaredNow read %+v (%v), not what runs", d, err)
}
}
// 1945/AC17 (unit): a resource the node's own composition shares with a running module's composition is declared once,
// the module's: the declaration joined of both names each file once, so the filling writes it; the node's own resources
// no module declares stay. (Live from 2026-10-25 12:43: "2 resources declare X" on every node.)
func TestTheNodesOwnCopyOfAModulesResourceIsLeftOut(t *testing.T) {
own := delivery.Composed{Node: "n1", Resources: []json.RawMessage{aFileOf("zsh", "rc"), aFileOf("mesh-wireguard", "conf")},
Filled: []string{"/tmp/r6/zsh/rc"}}
zsh := delivery.Composed{Node: "n1", Module: "zsh", FP: "src2:aa", Resources: []json.RawMessage{aFileOf("zsh", "rc")},
Filled: []string{"/tmp/r6/zsh/rc"}}
d, err := declarationOf(own, zsh)
if err != nil {
t.Fatal(err)
}
at := map[string]int{}
for _, r := range d.Resources {
at[r.Target()]++
}
if at["/tmp/r6/zsh/rc"] != 1 || at["/tmp/r6/mesh-wireguard/conf"] != 1 || len(d.Resources) != 2 {
t.Errorf("the joined declaration declares %v, want each path once", at)
}
if !d.Fills("/tmp/r6/zsh/rc") {
t.Errorf("the filled path is no longer named")
}
// The same at the node's own apply: what the others declare is theirs.
trimmed, dropped := ownWithout(own, []delivery.Composed{zsh})
if len(trimmed.Resources) != 1 || len(dropped) != 1 || dropped[0] != "zsh.rc (zsh)" {
t.Errorf("the node's own without zsh's: %d resources, dropped %v", len(trimmed.Resources), dropped)
}
if len(own.Resources) != 2 {
t.Errorf("ownWithout changed its argument")
}
// A node's own composition alone is all of it.
if d, err := declarationOf(own); err != nil || len(d.Resources) != 2 {
t.Errorf("the node's own alone: %v (%v)", d, err)
}
}
// 1945/AC18 (unit): the kept declaration's filled paths go to the module whose file is at each path, the node's own to
// the node's; and what runs from it, joined with the node's composition, names them filled.
func TestTheKeptDeclarationsFilledPathsGoToTheirModules(t *testing.T) {
raw := []byte(`{"resources":[
{"id":"claude-code.settings","type":"file","module":"claude-code","path":"/etc/cc.json"},
{"id":"screen-lock.conf","type":"file","module":"screen-lock","path":"/etc/lock.conf"},
{"id":"host.motd","type":"file","path":"/etc/motd"},
{"id":"zsh.rc","type":"file","module":"zsh","path":"/etc/zshrc"}],
"filled_files":["/etc/cc.json","/etc/lock.conf","/etc/motd"]}`)
by, filled := splitKept(raw)
if len(by["claude-code"]) != 1 || len(by[agent.NodeModule]) != 1 {
t.Fatalf("resources by module: %v", by)
}
if strings.Join(filled["claude-code"], ",") != "/etc/cc.json" || strings.Join(filled["screen-lock"], ",") != "/etc/lock.conf" ||
strings.Join(filled[agent.NodeModule], ",") != "/etc/motd" || len(filled["zsh"]) != 0 {
t.Fatalf("filled by module: %v", filled)
}
cc := delivery.Composed{Node: "n1", Module: "claude-code", Resources: by["claude-code"], Filled: filled["claude-code"]}
node := delivery.Composed{Node: "n1"}
d, err := whatRuns([]delivery.Composed{cc, node}, true, func() (*declaration.Declaration, error) { return nil, errors.New("not read") })
if err != nil || !d.Fills("/etc/cc.json") {
t.Fatalf("what runs does not name claude-code's file filled: %v (%v)", d, err)
}
if splitBy, _ := splitKept([]byte("not json")); splitBy != nil {
t.Errorf("a kept declaration that does not read split into %v", splitBy)
}
}
// 1945/AC17 (unit, review): a resource of the node's own that shares only its target with a module's (another id: a
// unit or a container of the same name) is not the module's, and is neither dropped nor left to be removed.
func TestASharedTargetWithAnotherIDIsNotDropped(t *testing.T) {
ownFile, _ := json.Marshal(map[string]any{"id": "host.zshrc", "type": "file", "path": "/tmp/r6/zsh/rc", "content": "x"})
own := delivery.Composed{Node: "n1", Resources: []json.RawMessage{ownFile}}
zsh := delivery.Composed{Node: "n1", Module: "zsh", FP: "src2:aa", Resources: []json.RawMessage{aFileOf("zsh", "rc")}}
trimmed, dropped := ownWithout(own, []delivery.Composed{zsh})
if len(dropped) != 0 || len(trimmed.Resources) != 1 {
t.Errorf("a resource sharing only its target was dropped: %v", dropped)
}
}
+10 -11
View File
@@ -41,8 +41,9 @@ func rolledBackHosts(statePath string) []string {
return upgrade.RolledBackVersions(verdicts)
}
// proving is this host waiting for the mesh to take a report it made under its own build — the
// evidence its launcher's trial waits for — and acting on it once.
// proving is this host waiting to write a have key under its own build — the evidence its launcher's trial waits for
// (novox/hq ADR 0539 §9, ADR 0227 §8: the component replaced is never the only witness of its successor) — and acting
// on it once.
type proving struct {
statePath string
version string
@@ -56,16 +57,14 @@ func unproved(statePath, version string) bool {
return err != nil || known != version
}
// heard is told every account of the machine the mesh has taken. The first one about a declaration,
// made by this build, proves it.
func (p *proving) heard(r link.Report) {
if r.Declared == "" || r.Refused != "" || r.Host != p.version {
return
}
// haveWritten is told when this build has written a have key, or found the bucket already holding what it would write.
// The first proves it. **A declaration's report does not**: a node-engine that receives none is not on trial for it, and
// one that cannot write its haves is what the launcher keeps rolling back.
func (p *proving) haveWritten() {
p.once.Do(func() {
retired, err := upgrade.Proved(p.statePath, upgrade.VersionsDir(""), p.version)
if err != nil {
p.say(fmt.Sprintf("this host reported under its own build %s, and proving it was not complete: %v", p.version, err))
p.say(fmt.Sprintf("this host wrote its first have under its own build %s, and proving it was not complete: %v", p.version, err))
}
if len(retired) > 0 {
p.say(fmt.Sprintf("host %s is proved; retired the host version(s) %s", p.version, strings.Join(retired, ", ")))
@@ -146,7 +145,7 @@ func launcherReplaced(pid int, proc, path string) bool {
}
// watchWhatThisHostPlaced judges the core builds this host placed, until ctx ends (to-be 45 §8).
func watchWhatThisHostPlaced(ctx context.Context, node string, queue *link.Queue, say func(string)) {
func watchWhatThisHostPlaced(ctx context.Context, node string, queue *link.Session, say func(string)) {
host, _ := os.Hostname()
w := &witness.Watcher{
Root: apply.DaemonRoot(), Node: node, Host: host,
@@ -159,7 +158,7 @@ func watchWhatThisHostPlaced(ctx context.Context, node string, queue *link.Queue
},
// Said now, not at the next report: the verdict is on every report from here, and the mesh
// is asked for one at once.
Concluded: func(link.Rollback) { queue.ReportAsked() },
Concluded: func(link.Rollback) { queue.FactsAsked(ctx) },
Say: say,
}
w.Watch(ctx)
+15 -23
View File
@@ -13,9 +13,9 @@ import (
"github.com/novox/mesh-host/internal/upgrade"
)
// The first account the mesh takes from this build, about a declaration, proves it to the launcher;
// nothing else does — a refusal, a report about no declaration, another build's report (to-be 45 §8).
func TestOnlyAnAccountUnderThisBuildProvesIt(t *testing.T) {
// The first have this build writes proves it to the launcher; nothing else does — not a report about a declaration, not
// a facts event, not the haves of another build (novox/hq ADR 0539 §9, ADR 0227 §8). R6-A AC7.
func TestOnlyAHaveUnderThisBuildProvesIt(t *testing.T) {
state := filepath.Join(t.TempDir(), "state.json")
t.Setenv("MESH_HOST_LIBEXEC", t.TempDir())
// Never the machine's own engine placed by hand: proving renews it (issue 356).
@@ -27,22 +27,20 @@ func TestOnlyAnAccountUnderThisBuildProvesIt(t *testing.T) {
var said []string
p := &proving{statePath: state, version: "2.0", say: func(s string) { said = append(said, s) }}
for _, r := range []link.Report{
{Host: "2.0", Refused: "no"},
{Host: "2.0"},
{Host: "1.0", Declared: "abc"},
} {
p.heard(r)
if !unproved(state, "2.0") {
t.Fatalf("%+v proved this build", r)
}
if got, _ := os.ReadFile(fallback); string(got) != "a stale engine placed by hand" {
t.Fatalf("%+v renewed the engine placed by hand", r)
}
// Facts said, a link opened, a declaration of the old kind reported: none of it is a have.
if !unproved(state, "2.0") {
t.Fatal("a build that wrote no have is proved")
}
p.heard(link.Report{Host: "2.0", Declared: "abc"})
if got, _ := os.ReadFile(fallback); string(got) != "a stale engine placed by hand" {
t.Fatal("the engine placed by hand was renewed before any have")
}
// The session tells the proof when it has a have key saying what it would write.
session := &link.Session{HaveWritten: p.haveWritten}
_ = session
p.haveWritten()
if unproved(state, "2.0") {
t.Fatalf("an account of a declaration under this build did not prove it (%v)", said)
t.Fatalf("the first have under this build did not prove it (%v)", said)
}
if got, _ := upgrade.ReadKnownGood(upgrade.KnownGoodPath(state)); got != "2.0" {
t.Fatalf("known-good is %q", got)
@@ -78,12 +76,6 @@ func TestAStandingRollbackIsSaidOnEveryReport(t *testing.T) {
got[0].Outcome != link.RolledBack {
t.Fatalf("what the report says is %+v", got)
}
if !worthSaying(link.Report{Rollbacks: got}) {
t.Fatal("a reconcile with a rollback standing does not say it")
}
if adoptionFingerprint(link.Report{Rollbacks: got}) == adoptionFingerprint(link.Report{}) {
t.Fatal("a rollback reached or ended is not news to the reconcile")
}
}
// The launcher running this host is known to have been replaced only when it certainly was: a shell
+2 -2
View File
@@ -12,7 +12,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
func bundle(t *testing.T) *declaration.Declaration {
@@ -49,7 +49,7 @@ func TestTheBundleCarriesTheFoundationAndTheControlPlaneAndNothingElse(t *testin
}
sort.Strings(images)
want := []string{"mesh-controller", "nats", "postgres"}
want := []string{"mesh-controller", "nats", "pgvector"}
if strings.Join(images, ",") != strings.Join(want, ",") {
t.Fatalf("the bundle carries %v; expected exactly %v.\n\n"+
"Adding one is a change to what every first node becomes, and to ADR 0006's "+
+34 -19
View File
@@ -28,11 +28,12 @@
// image, and this file is applied on a machine with no mesh to ask about anything (ADR 0006). The
// registry prefix is the lab's; the digests are upstream's.
//
// The store's data is a NAMED VOLUME, not a directory on the machine. A directory the host
// creates is owned by root, and the database runs as somebody else inside the container — so it
// could not write, and the container crash-looped. A named volume lets the image set up its own
// ownership, and outlives the container, which is what you want for the thing holding the mesh's
// state.
// **The store is raised where the `postgres` module keeps it** (novox/hq ADR 0338): the container
// `postgres`, on the module's pgvector image, its data in the directory /var/lib/mesh-store owned by
// the image's postgres user, its superuser password read from the module's own-secret file — so the
// store step adopts it as it runs. Pinned upstream by the module's own digest, not through the lab's
// registry prefix, because the two must be the same reference (mesh-host internal/bootstrap holds
// them to each other against the module's manifest).
{
"declaration": 1,
@@ -83,17 +84,28 @@
"boot": "enabled",
"restart-on": ["base-filter"]
},
// The store's data is a directory owned by the image's postgres user (999), made before the
// container starts: a directory the runtime made would be root's, and the database could not
// write it. Measured on the module's image 2026-10-13: it initialises a new store on a directory
// made so (novox/hq ADR 0338 §5).
{
"id": "store-data",
"type": "directory",
"path": "/var/lib/mesh-store",
"mode": "0700",
"owner": "999:70"
},
{
"id": "store",
"type": "container",
"name": "mesh-store",
"image": "192.0.2.250:5000/postgres@sha256:7abf537131b66ed5af448d90653abf1679b0c7e9a1f07efdd4c3108a401b259a",
"name": "postgres",
"image": "pgvector/pgvector@sha256:cf134a767f474095eeba57e0117be8e568e011a63f33fbf252f14c9b760f8e6f",
"env": {
"POSTGRES_PASSWORD": "bootstrap",
"PGDATA": "/var/lib/postgresql/data/pgdata"
},
"ports": ["5432:5432"],
"volumes": ["mesh-store-data:/var/lib/postgresql/data"]
"volumes": ["/var/lib/mesh-store:/var/lib/postgresql/data"]
},
// Over TCP, not the socket. While the store initialises it runs a temporary server on the
// socket ONLY, then stops it and starts the real one — so a socket check passes, the action
@@ -102,21 +114,21 @@
{
"id": "store-ready",
"type": "action",
"in": "mesh-store",
"in": "postgres",
"command": ["sh", "-c", "for i in $(seq 1 180); do pg_isready -h 127.0.0.1 -U postgres >/dev/null 2>&1 && exit 0; sleep 1; done; echo 'the store did not answer within 180s; its own last words follow'; pg_isready -h 127.0.0.1 -U postgres; tail -n 20 /var/lib/postgresql/data/log/*.log 2>/dev/null; exit 1"],
"verify": ["pg_isready", "-h", "127.0.0.1", "-U", "postgres"]
},
{
"id": "inventory-database",
"type": "action",
"in": "mesh-store",
"in": "postgres",
"command": ["sh", "-c", "psql -U postgres -c 'CREATE DATABASE inventory'"],
"verify": ["sh", "-c", "psql -U postgres -lqt | cut -d'|' -f1 | grep -qw inventory"]
},
{
"id": "identity-database",
"type": "action",
"in": "mesh-store",
"in": "postgres",
"command": ["sh", "-c", "psql -U postgres -c 'CREATE DATABASE identity'"],
"verify": ["sh", "-c", "psql -U postgres -lqt | cut -d'|' -f1 | grep -qw identity"]
},
@@ -126,20 +138,20 @@
{
"id": "licences-database",
"type": "action",
"in": "mesh-store",
"in": "postgres",
"command": ["sh", "-c", "psql -U postgres -c 'CREATE DATABASE licences'"],
"verify": ["sh", "-c", "psql -U postgres -lqt | cut -d'|' -f1 | grep -qw licences"]
},
{
"id": "context-schemas",
"type": "action",
"command": ["docker", "run", "--rm", "--network", "container:mesh-store",
"command": ["docker", "run", "--rm", "--network", "container:postgres",
"-e", "MESH_STORE_INVENTORY=postgres://postgres:bootstrap@127.0.0.1:5432/inventory?sslmode=disable",
"-e", "MESH_STORE_IDENTITY=postgres://postgres:bootstrap@127.0.0.1:5432/identity?sslmode=disable",
"-e", "MESH_STORE_LICENCES=postgres://postgres:bootstrap@127.0.0.1:5432/licences?sslmode=disable",
"192.0.2.250:5000/mesh-controller@sha256:c67db38439ff0aee242b467486765467bb95801f52175fc5727cc4e437338ace",
"migrate"],
"verify": ["sh", "-c", "docker exec mesh-store psql -U postgres -d inventory -tAc \"select to_regclass('public.node')\" | grep -qx node && docker exec mesh-store psql -U postgres -d identity -tAc \"select to_regclass('public.signing_key')\" | grep -qx signing_key && docker exec mesh-store psql -U postgres -d licences -tAc \"select to_regclass('public.licence')\" | grep -qx licence"]
"verify": ["sh", "-c", "docker exec postgres psql -U postgres -d inventory -tAc \"select to_regclass('public.node')\" | grep -qx node && docker exec postgres psql -U postgres -d identity -tAc \"select to_regclass('public.signing_key')\" | grep -qx signing_key && docker exec postgres psql -U postgres -d licences -tAc \"select to_regclass('public.licence')\" | grep -qx licence"]
},
// **The bus is raised where the `nats` module keeps it** (novox/hq ADR 0333 §1): the container
// `mesh-broker-nats`, its configuration and user list in /var/lib/nats-module/conf, its streams in
@@ -209,17 +221,20 @@
"path": "/var/lib/nats-module/conf/accounts.conf",
"mode": "0600",
"create-once": true,
"content": "// The first user list, carried by the installer because at genesis there is no mesh to\n// compose one. A bootstrap credential, rotated with the store's and replaced by the\n// controller's own composition from its first start onward.\naccounts {\n MESH {\n jetstream: enabled\n users = [\n { user: \"controller\", password: \"$2a$10$AHqJgOifIVbU41KmATiMhuXFs8xa7Wl2HuN4UVBCXdN2jIQzjqApy\", permissions: {\n publish: { allow: [\"$JS.ACK.CONTROL.controller.>\", \"$JS.ACK.EVENTS.controller.>\", \"$JS.ACK.DEAD_LETTER_NOTICES.controller.>\", \"$JS.API.>\", \"$KV.mesh-controller_asked.>\", \"$KV.mesh-controller_calls.>\", \"$KV.mesh-controller_hand-acts.>\", \"$KV.mesh-controller_conditions.>\", \"$KV.mesh-controller_condition-history.>\", \"$KV.mesh-controller_lease.>\", \"$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>\", \"$KV.SEAT_NODE_BUILD_AGENT_cancelled.>\", \"_INBOX.enrol.>\", \"mesh.assignment.>\", \"mesh.mod.*.tool.>\", \"mesh.node.>\", \"mesh.seat.mesh-build-machine.accept.>\", \"mesh.seat.node-build-agent.accept.>\", \"mesh.seat.mesh-build-machine.tool.>\", \"mesh.seat.node-build-agent.tool.>\", \"mesh.seat.mesh-controller.event.applied\", \"mesh.seat.mesh-controller.event.built-before\", \"mesh.seat.mesh-controller.event.refused\", \"mesh.seat.mesh-controller.event.rolled-back\", \"mesh.seat.mesh-controller.event.checked\", \"mesh.seat.mesh-controller.event.plan-moved\", \"mesh.seat.mesh-controller.event.condition-raised\", \"mesh.seat.mesh-controller.event.condition-changed\", \"mesh.seat.mesh-controller.event.condition-cleared\", \"mesh.seat.mesh-controller.event.doctor-heartbeat\", \"mesh.seat.mesh-controller.event.secret-replaced\", \"mesh.seat.mesh-controller.event.healer-acted\", \"$SRV.INFO\", \"mesh.seat.node-intrusion-prevention.tool.banned.*\", \"mesh.seat.node-backup.tool.backed-up.*\", \"mesh.seat.node-backup.tool.now.*\", \"mesh.seat.mesh-delivery.tool.stalled\", \"mesh.seat.mesh-delivery.tool.close\", \"mesh.seat.node-launcher.tool.secret.*\", \"mesh.seat.mesh-delivery.tool.release\", \"mesh.seat.mesh-delivery.tool.stop\", \"mesh.seat.mesh-delivery.tool.times\", \"mesh.seat.mesh-delivery.tool.groups\", \"mesh.seat.node-service-manager.tool.restart.*\", \"mesh.seat.mesh-controller.tool.plans\", \"mesh.events.dead.>\", \"mesh.again.>\"] }\n subscribe: { allow: [\"$JS.API.>\", \"$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>\", \"$JS.EVENT.ADVISORY.CONSUMER.DELETED.>\", \"_DELIVER.controller\", \"_DELIVER.controller.>\", \"_INBOX.controller.>\", \"mesh.control.>\", \"mesh.mod.*.event.provisioner.failing\", \"mesh.mod.*.event.provisioner.recovered\", \"mesh.mod.*.event.provisioner.retirement\", \"mesh.mod.gitea.event.pull.merged\", \"mesh.mod.gitea.event.pull.updated\", \"mesh.mod.mesh-catalog.event.catching-up\", \"mesh.mod.mesh-catalog.event.upgraded\", \"mesh.seat.mesh-build-machine.event.built\", \"mesh.seat.node-build-agent.event.built\", \"mesh.seat.operator-channel.event.decided.mesh-controller\", \"mesh.seat.mesh-controller.tool.>\", \"$SRV.PING\", \"$SRV.INFO\", \"$SRV.PING.mesh-controller\", \"$SRV.PING.mesh-controller.>\", \"$SRV.INFO.mesh-controller\", \"$SRV.INFO.mesh-controller.>\", \"$SRV.STATS\", \"$SRV.STATS.mesh-controller\", \"$SRV.STATS.mesh-controller.>\"] }\n allow_responses: { max: 1, ttl: \"1m\" }\n } }\n ]\n }\n}\n"
"content": "// The first user list, carried by the installer because at genesis there is no mesh to\n// compose one. A bootstrap credential, rotated with the store's and replaced by the\n// controller's own composition from its first start onward.\naccounts {\n MESH {\n jetstream: enabled\n users = [\n { user: \"controller\", password: \"$2a$10$AHqJgOifIVbU41KmATiMhuXFs8xa7Wl2HuN4UVBCXdN2jIQzjqApy\", permissions: {\n publish: { allow: [\"$JS.ACK.CONTROL.controller.>\", \"$JS.ACK.EVENTS.controller.>\", \"$JS.ACK.DEAD_LETTER_NOTICES.controller.>\", \"$JS.API.>\", \"$KV.mesh-controller_asked.>\", \"$KV.mesh-controller_assignments.>\", \"$KV.mesh-controller_calls.>\", \"$KV.mesh-controller_hand-acts.>\", \"$KV.mesh-controller_conditions.>\", \"$KV.mesh-controller_condition-history.>\", \"$KV.mesh-controller_declared-config.>\", \"$KV.mesh-controller_lease.>\", \"$KV.mesh-controller_predefined.>\", \"$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>\", \"$KV.SEAT_NODE_BUILD_AGENT_cancelled.>\", \"_INBOX.enrol.>\", \"mesh.assignment.>\", \"mesh.mod.*.tool.>\", \"mesh.node.>\", \"mesh.seat.mesh-build-machine.accept.>\", \"mesh.seat.node-build-agent.accept.>\", \"mesh.seat.mesh-build-machine.tool.>\", \"mesh.seat.node-build-agent.tool.>\", \"mesh.seat.mesh-controller.event.applied\", \"mesh.seat.mesh-controller.event.built-before\", \"mesh.seat.mesh-controller.event.refused\", \"mesh.seat.mesh-controller.event.rolled-back\", \"mesh.seat.mesh-controller.event.checked\", \"mesh.seat.mesh-controller.event.assignment-changed\", \"mesh.seat.mesh-controller.event.declared-config-changed\", \"mesh.seat.mesh-controller.event.predefined-changed\", \"mesh.seat.mesh-controller.event.condition-raised\", \"mesh.seat.mesh-controller.event.condition-changed\", \"mesh.seat.mesh-controller.event.condition-cleared\", \"mesh.seat.mesh-controller.event.doctor-heartbeat\", \"mesh.seat.mesh-controller.event.secret-replaced\", \"mesh.seat.mesh-controller.event.healer-acted\", \"$SRV.INFO\", \"mesh.seat.node-intrusion-prevention.tool.banned.*\", \"mesh.seat.node-backup.tool.backed-up.*\", \"mesh.seat.node-backup.tool.now.*\", \"mesh.seat.mesh-delivery.tool.stalled\", \"mesh.seat.mesh-delivery.tool.close\", \"mesh.seat.node-launcher.tool.secret.*\", \"mesh.seat.mesh-delivery.tool.release\", \"mesh.seat.mesh-delivery.tool.stop\", \"mesh.seat.mesh-delivery.tool.times\", \"mesh.seat.mesh-delivery.tool.groups\", \"mesh.seat.node-service-manager.tool.restart.*\", \"mesh.seat.operator-channel.tool.tell\", \"mesh.seat.mesh-controller.tool.plans\", \"mesh.events.dead.>\", \"mesh.again.>\"] }\n subscribe: { allow: [\"$JS.API.>\", \"$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>\", \"$JS.EVENT.ADVISORY.CONSUMER.DELETED.>\", \"_DELIVER.controller\", \"_DELIVER.controller.>\", \"_INBOX.controller.>\", \"mesh.control.>\", \"mesh.mod.*.event.provisioner.failing\", \"mesh.mod.*.event.provisioner.recovered\", \"mesh.mod.*.event.provisioner.retirement\", \"mesh.mod.gitea.event.pull.merged\", \"mesh.mod.gitea.event.pull.updated\", \"mesh.mod.mesh-catalog.event.catching-up\", \"mesh.mod.mesh-catalog.event.upgraded\", \"mesh.seat.mesh-build-machine.event.built\", \"mesh.seat.node-build-agent.event.built\", \"mesh.seat.operator-channel.event.decided.mesh-controller\", \"mesh.seat.mesh-controller.tool.>\", \"$SRV.PING\", \"$SRV.INFO\", \"$SRV.PING.mesh-controller\", \"$SRV.PING.mesh-controller.>\", \"$SRV.INFO.mesh-controller\", \"$SRV.INFO.mesh-controller.>\", \"$SRV.STATS\", \"$SRV.STATS.mesh-controller\", \"$SRV.STATS.mesh-controller.>\"] }\n allow_responses: { max: 1, ttl: \"1m\" }\n } }\n ]\n }\n}\n"
},
// The upstream server, until the broker step adopts it: the `nats` module's own image, built by the
// mesh, takes its place there once, on these same directories (ADR 0333 §3). The published port is the
// mesh, takes its place there once, on these same directories (ADR 0333 §3), as the bus's planned
// step (ADR 0236). The release is the one the module's image is built on (its NATS_BASE, 2.11.17),
// pinned upstream by that same digest: the adoption changes no server release, so it can be undone
// by putting the old image back and is said reversible. The published port is the
// node's bus port, 5671 unless given, which every enrolment token names; the installer gives the
// module the setting that maps its declared 4222 to it (ADR 0333 §2).
{
"id": "broker",
"type": "container",
"name": "mesh-broker-nats",
"image": "192.0.2.250:5000/nats@sha256:b83efabe3e7def1e0a4a31ec6e078999bb17c80363f881df35edc70fcb6bb927",
"image": "nats@sha256:e4bf19f15fd3218814a4e3c9e0064e1334bd8aa20d5984b9f1a0afd084f8cc00",
"ports": ["5671:4222", "127.0.0.1:8222:8222"],
"volumes": ["/var/lib/mesh-broker-nats:/data", "/var/lib/nats-module/conf:/etc/nats:ro", "/var/lib/mesh-broker-tls:/tls:ro"],
"args": ["-c", "/etc/nats/nats.conf", "-js"]
@@ -227,8 +242,8 @@
{
"id": "broker-ready",
"type": "action",
"command": ["sh", "-c", "for i in $(seq 1 60); do docker run --rm --network host --entrypoint sh 192.0.2.250:5000/nats@sha256:b83efabe3e7def1e0a4a31ec6e078999bb17c80363f881df35edc70fcb6bb927 -c 'nc -z 127.0.0.1 5671' >/dev/null 2>&1 && exit 0; sleep 1; done; exit 1"],
"verify": ["sh", "-c", "docker run --rm --network host --entrypoint sh 192.0.2.250:5000/nats@sha256:b83efabe3e7def1e0a4a31ec6e078999bb17c80363f881df35edc70fcb6bb927 -c 'nc -z 127.0.0.1 5671'"]
"command": ["sh", "-c", "for i in $(seq 1 60); do docker run --rm --network host --entrypoint sh nats@sha256:e4bf19f15fd3218814a4e3c9e0064e1334bd8aa20d5984b9f1a0afd084f8cc00 -c 'nc -z 127.0.0.1 5671' >/dev/null 2>&1 && exit 0; sleep 1; done; exit 1"],
"verify": ["sh", "-c", "docker run --rm --network host --entrypoint sh nats@sha256:e4bf19f15fd3218814a4e3c9e0064e1334bd8aa20d5984b9f1a0afd084f8cc00 -c 'nc -z 127.0.0.1 5671'"]
},
{
"id": "control-plane",
+7 -1
View File
@@ -3,14 +3,20 @@ module github.com/novox/mesh-host
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261028234900-c2999a4e13cc
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
golang.org/x/crypto v0.57.0
golang.org/x/sys v0.48.0
)
require (
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/klauspost/compress v1.20.0 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
github.com/nats-io/jwt/v2 v2.8.1 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
golang.org/x/time v0.15.0 // indirect
)
+15 -6
View File
@@ -1,11 +1,17 @@
git.novox.be/novox/mesh-sdk/go v0.1.11-0.20261009143344-f047d0a4a970 h1:9tFDQsgmI+4X7/BpZGXIr+HemPKE7YddYGqWV0lINAI=
git.novox.be/novox/mesh-sdk/go v0.1.11-0.20261009143344-f047d0a4a970/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010190644-622b6b4f556d h1:BR8uwv4BqyLaBwITuLylrd9DRUnu0D5DjJw0RuQSXbk=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010190644-622b6b4f556d/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c h1:l5onJwoIeH8yE/PjVlEeoPyXBsHp2NQiWLllz2fwX7s=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261028234900-c2999a4e13cc h1:mOk2tAKTiuvnap/BUIYz91lLVbHi/EB6JPAD6KxTPvc=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261028234900-c2999a4e13cc/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
github.com/minio/highwayhash v1.0.4/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/nats-io/jwt/v2 v2.8.1 h1:V0xpGuD/N8Mi+fQNDynXohVvp7ZztevW5io8CUWlPmU=
github.com/nats-io/jwt/v2 v2.8.1/go.mod h1:nWnOEEiVMiKHQpnAy4eXlizVEtSfzacZ1Q43LIRavZg=
github.com/nats-io/nats-server/v2 v2.11.17 h1:GKEghcFK6A+aFx11Yf1LjgLC3txAwvyhnYzhBIQZA8I=
github.com/nats-io/nats-server/v2 v2.11.17/go.mod h1:B1sFVz4StNosQ903ak4N1G01Fl/9f8e06mXpFIE2K24=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
@@ -14,5 +20,8 @@ github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
+2 -2
View File
@@ -45,7 +45,7 @@ import (
"sync"
"time"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// The states, in the words liveness says them (the controller reads them alike).
@@ -119,7 +119,7 @@ func secretsOf(d *declaration.Declaration, account string) []string {
var out []string
for _, r := range d.Resources {
f, ok := r.(*declaration.File)
if !ok || f.Path == "" || (f.Sealed == "" && len(f.Secrets) == 0) || f.Owner == account {
if !ok || f.Path == "" || (!f.Secret() && len(f.Secrets) == 0) || f.Owner == account {
continue
}
out = append(out, f.Path)
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"testing"
"time"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// The account judge (novox/hq ADR 0252, "how it is checked"): an account in a group its running session
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// An account declared never to become root without a person (novox/hq ADR 0266, "how it is checked"): each
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -14,7 +14,7 @@ import (
"fmt"
"path/filepath"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// AgentHomeRefusedError is an archive or a written-into file the engine will not place below an agent's home.
+1 -1
View File
@@ -6,7 +6,7 @@ package apply
import (
"testing"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
func TestNothingIsUnpackedOrWrittenIntoBelowAnAgentsHome(t *testing.T) {
+67
View File
@@ -0,0 +1,67 @@
package apply
import (
"context"
"fmt"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
// WriteAlone writes one file a module's composition declares, through the apply's own file step, and records it as an
// apply would; nothing else of the module is applied, and nothing that reads the file is moved (novox/hq ADR 0547 §8).
//
// **For a file whose reader takes the change itself.** The bus's user list is the one it is for: the bus module's
// entrypoint watches the file and signals the server to reload in place, so the list is in force without a want of the
// bus, which nobody writes (ADR 0539 §15), and without the container being touched. A file that a service or a container
// must be moved to read is not written alone: it goes through the module's apply, which moves what reads it.
//
// The file is written only when its content differs from what is there (the file step's own rule), read back, and
// recorded under the module whose resource it is, so the module's next apply finds it as its own and writes nothing.
// The caller holds the apply's lock. log is said the path, the action and the digest, never the content.
func WriteAlone(ctx context.Context, sys system.System, file *declaration.File, known store.State, run Runner,
log func(string)) (Outcome, store.State, error) {
if log == nil {
log = func(string) {}
}
if file.Into != "" || file.CreateOnce || file.Sealed != "" || file.VaultSealed != nil || len(file.Secrets) > 0 {
return Outcome{}, known, &Error{Resource: file.ID, Err: fmt.Errorf("%s is not a whole file of plain content, "+
"and only such a file is written alone; nothing was written", file.Path)}
}
if file.IsReference() {
return Outcome{}, known, &Error{Resource: file.ID, Err: fmt.Errorf("%s is given by digest %s and was not "+
"resolved to its content; it is not written empty", file.Path, file.Digest)}
}
if held, isHeld := known.HeldAt(file.ID); isHeld {
return Outcome{}, known, &Error{Resource: file.ID, Err: fmt.Errorf("%s is held as found for a module not "+
"yet taken (%s), so nothing was written", file.Path, held.Why)}
}
was, _ := known.Find(file.ID)
if was.ID == "" {
// Only over a file the mesh wrote: a file this node holds no record of is the module's apply's to take, with
// the original kept, never this write's. A record of the path under another id (the declaration before the
// switch named it so) is the one written to, so no second record of the file is left for an apply to remove.
rec, ok := known.At(string(declaration.TypeFile), file.Path)
if !ok {
return Outcome{}, known, &Error{Resource: file.ID, Err: fmt.Errorf("this node holds no record of %s; it "+
"is written by the module's apply first, never alone", file.Path)}
}
renamed := *file
renamed.ID = rec.ID
file, was = &renamed, rec
}
in := inputs{declares: map[string]string{file.Identity(): declaredDigest(file)}, known: &known,
groups: groupsWanted([]declaration.Resource{file}), handed: Handed{}, dirsBefore: map[string]bool{},
files: map[string]bool{file.Path: true}}
outcome, err := applyOne(ctx, sys, file, run, map[string]bool{}, in, was, nil, nil)
if err != nil {
log(fmt.Sprintf(" failed %s (%s), alone: %v", file.ID, file.Path, err))
return outcome, known, &Error{Resource: file.ID, Err: err}
}
recordKeeping(&known, file, outcome, was)
if moved(outcome) {
log(fmt.Sprintf(" %s %s (%s), alone: %s", outcome.Action, file.ID, file.Path, outcome.wrote))
}
return outcome, known, nil
}
+90
View File
@@ -0,0 +1,90 @@
package apply
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
// Defends novox/hq ADR 0547 §8: the bus node writes the bus's user list alone, with no want, through the apply's own
// file step: only when it differs, mode 0600, read back and recorded as the module's, and nothing else moves.
func TestAFileWrittenAloneMovesNothingElse(t *testing.T) {
const was, now = "user a\n", "user a\nuser b\n"
for _, c := range []struct {
name, fields, before string
recorded bool
recordedAs string // the id the record has, when not the composition's
action, refused string
after string
}{
{name: "no record of it", fields: `"content":%q,"mode":"0600"`, before: was, refused: "holds no record", after: was},
{name: "changed", fields: `"content":%q,"mode":"0600"`, before: was, recorded: true, action: "updated", after: now},
{name: "the same", fields: `"content":%q,"mode":"0600"`, before: now, recorded: true, action: "unchanged", after: now},
{name: "recorded under the id from before the switch", fields: `"content":%q,"mode":"0600"`, before: was,
recorded: true, recordedAs: "bus-users", action: "updated", after: now},
{name: "absent with a record", fields: `"content":%q,"mode":"0600"`, recorded: true, action: "created", after: now},
{name: "written into", fields: `"content":%q,"into":"block","owner-at-runtime":"managed"`, before: was,
recorded: true, refused: "only such a file", after: was},
{name: "sealed", fields: `"sealed":"c2VhbGVk"`, before: was, recorded: true,
refused: "only such a file", after: was},
} {
t.Run(c.name, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "accounts.conf")
if c.before != "" {
if err := os.WriteFile(path, []byte(c.before), 0o600); err != nil {
t.Fatal(err)
}
}
fields := strings.Replace(c.fields, "%q", fmt.Sprintf("%q", now), 1)
d := parseTrusted(t, fmt.Sprintf(`{"declaration":1,"resources":[{"id":"nats.bus-users","module":"nats",
"type":"file","path":%q,`+fields+`}]}`, path))
file := d.Resources[0].(*declaration.File)
known := store.State{}
id := file.ID
if c.recordedAs != "" {
id = c.recordedAs
}
if c.recorded {
known.Record(store.Applied{ID: id, Type: string(declaration.TypeFile), Module: "nats", Target: path,
Origin: store.OriginDeclared, Wrote: digestOf(c.before)})
}
calls := 0
counting := func(context.Context, string, ...string) (string, error) { calls++; return "", nil }
out, after, err := WriteAlone(context.Background(), archHost(t), file, known, counting, nil)
if c.refused != "" {
if err == nil || !strings.Contains(err.Error(), c.refused) {
t.Fatalf("not refused with %q: %v", c.refused, err)
}
} else if err != nil {
t.Fatal(err)
} else if out.Action != c.action {
t.Errorf("action %q, want %q", out.Action, c.action)
}
if got := readText(t, path); got != c.after {
t.Errorf("file holds %q, want %q", got, c.after)
}
if calls != 0 {
t.Errorf("%d commands were run: a file written alone moves nothing", calls)
}
if c.refused != "" {
return
}
if m := modeAt(t, path); m != 0o600 {
t.Errorf("mode %o", m)
}
rec, ok := after.Find(id)
if !ok || rec.Wrote != digestOf(now) || rec.Target != path {
t.Errorf("record %+v (found %v), want it to say %s was written", rec, ok, digestOf(now))
}
if n := len(after.Resources); n != 1 {
t.Errorf("%d records of one file", n)
}
})
}
}
+537 -131
View File
File diff suppressed because it is too large Load Diff
+83 -1
View File
@@ -9,7 +9,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
@@ -350,6 +350,42 @@ func TestAnUnknownServiceStateIsRefusedNotGuessed(t *testing.T) {
}
}
// A boot-time one-shot a package ships, declared stopped and enabled at boot, is satisfied while
// nothing of it runs, whatever its last run did (novox/hq issue 1126). The kernel module's
// linux-modules-cleanup.service failed on ace with "was asked to be stopped and is running": an
// inactive one-shot with ExecMainStatus=0 reads "running", and stopping it changes nothing.
func TestABootOneShotDeclaredStoppedIsSatisfiedAtRest(t *testing.T) {
for _, c := range []struct{ name, status string }{
{"exited 0: never ran, ran at boot, or unloaded", "0"},
} {
t.Run(c.name, func(t *testing.T) {
var asked []string
run := func(ctx context.Context, name string, args ...string) (string, error) {
args = oldShape(name, args)
switch args[0] {
case "show":
return "LoadState=loaded\nActiveState=inactive\nType=oneshot\nRemainAfterExit=no\n" +
"ExecMainStatus=" + c.status + "\n", nil
case "is-enabled":
return "enabled\n", nil
}
asked = append(asked, strings.Join(args, " "))
return "", nil
}
d := parse(t, `{"declaration":1,"resources":[
{"id":"cleanup","type":"service","unit":"linux-modules-cleanup.service","state":"stopped","boot":"enabled"}
]}`)
_, _, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginCarried, run, nil, nil)
if err != nil {
t.Fatalf("a one-shot at rest declared stopped failed: %v", err)
}
if len(asked) > 0 {
t.Errorf("the service manager was asked %q for a unit already as declared", asked)
}
})
}
}
func TestADroppedServiceIsGivenBackTheStateItWasFoundIn(t *testing.T) {
// novox/hq ADR 0118: undeclaring removes what the mesh made, gives back what it changed, and
// leaves what was the machine's. A service resource never installs a unit — so undeclaring it
@@ -1225,6 +1261,52 @@ func TestAServiceIsRestartedWhenWhatItReflectsChanges(t *testing.T) {
if reloaded < 0 || reloaded > stop {
t.Errorf("the service was restarted without the unit files being read again first; commands were %v", commands)
}
// A drop-in in a unit directory that no service reflects — the power module's bound on reboot.target
// (issue 1126) — is read by the service manager only after a reload: one per apply, and no restart.
// Unchanged, it reloads nothing.
units := filepath.Join(dir, "system")
saved := UnitDirs
UnitDirs = []string{units}
t.Cleanup(func() { UnitDirs = saved })
dropIn := filepath.Join(units, "reboot.target.d", "mesh-power.conf")
withDropIn := parse(t, fmt.Sprintf(`{"declaration":1,"resources":[
{"id":"conf","type":"file","path":%q,"content":"second\n","mode":"0644"},
{"id":"svc","type":"service","unit":"thing.service","state":"running","restart-on":["conf"]},
{"id":"units","type":"directory","path":%q,"mode":"0755"},
{"id":"drop-ins","type":"directory","path":%q,"mode":"0755"},
{"id":"bound","type":"file","path":%q,"content":"[Unit]\nJobTimeoutSec=10min\n","mode":"0644"},
{"id":"bound-too","type":"file","path":%q,"content":"[Unit]\nJobTimeoutSec=10min\n","mode":"0644"}
]}`, path, units, filepath.Dir(dropIn), dropIn, filepath.Join(filepath.Dir(dropIn), "other.conf")))
for _, row := range []struct {
name string
reloads int
}{{"a drop-in written", 1}, {"the same drop-in again", 0}} {
commands = nil
if _, state, err = Apply(context.Background(), archHost(t), withDropIn, state,
store.OriginCarried, run, nil, nil); err != nil {
t.Fatal(err)
}
reloads := 0
for _, c := range commands {
if strings.Contains(c, "daemon-reload") {
reloads++
}
if strings.Contains(c, "stop thing.service") {
t.Errorf("%s: a drop-in no service reflects restarted one: %v", row.name, commands)
}
}
if reloads != row.reloads {
t.Errorf("%s: %d reloads, want %d; commands were %v", row.name, reloads, row.reloads, commands)
}
}
// A drop-in whose module is skipped behind its step, or held as found, comes back so on every apply: it moved
// nothing, and reloads nothing.
for _, action := range []string{"skipped", "held"} {
if at := unitFileMoved([]Outcome{{ID: "bound", Type: "file", Target: dropIn, Action: action}}); at != "" {
t.Errorf("a drop-in %s reloads the service manager (%s)", action, at)
}
}
}
}
+5 -1
View File
@@ -13,7 +13,7 @@ import (
"path/filepath"
"strings"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
@@ -290,3 +290,7 @@ func unpack(body []byte, into string) (int, error) {
}
}
}
// Fetch is an archive at source as the host fetches every archive it unpacks, at most maxArchive bytes: what the node's
// own composer reads a moved module's templates archive with (issue 1126). Its digest is the caller's to check.
func Fetch(ctx context.Context, source string) ([]byte, error) { return fetch(ctx, source) }
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"path/filepath"
"strings"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
+153
View File
@@ -0,0 +1,153 @@
package apply
import (
"context"
"fmt"
"strings"
"time"
"unicode/utf8"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
// A container that says it is busy is replaced once it is idle, within a bound (novox/hq ADR 0335).
//
// **Asked, never waited on.** The apply asks the container once, and while it says busy leaves it running
// as it is and goes on with everything else of the declaration: the machine's lock is held for one ask,
// never for the wait. The wait is carried between passes in the container's record (store.Applied's
// BusySince), so the next pass — the node-engine asks again sooner than its reconcile while anything waits
// — asks again, and so does a node-engine started again in between: the bound runs from the first apply
// that would have replaced the container, whoever applied it.
// waitsBusy is the action of a container left running because it said busy (novox/hq ADR 0335 §2). Its own
// word rather than "unchanged": its declaration moved, and the machine is not yet what was declared.
const waitsBusy = "waits-busy"
// What a report says of a busy container (novox/hq ADR 0335 §2, §3).
const (
// BusyWaiting is a container left running because it said busy, waiting to be replaced.
BusyWaiting = "waiting"
// BusyReplacedIdle is a container replaced on the first pass it said idle, after it had waited.
BusyReplacedIdle = "replaced-when-idle"
// BusyReplacedAtBound is a container still busy at the bound, replaced anyway.
BusyReplacedAtBound = "replaced-at-bound"
)
// BusyWait is one container a busy command held, as a report says it.
type BusyWait struct {
// ID is the resource's id (`<module>.<resource>`), Container its runtime name.
ID string `json:"id"`
Container string `json:"container"`
// State is BusyWaiting, BusyReplacedIdle or BusyReplacedAtBound.
State string `json:"state"`
// Since is when the first apply that would have replaced it ran; Until when the bound replaces it.
Since time.Time `json:"since"`
Until time.Time `json:"until"`
// Said is the last output its busy command gave.
Said string `json:"said,omitempty"`
}
// Words is the wait in a sentence, for an outcome's detail and the console.
func (b BusyWait) Words() string {
since := b.Since.UTC().Format(time.RFC3339)
said := ""
if b.Said != "" {
said = "; its busy command last said: " + b.Said
}
switch b.State {
case BusyReplacedAtBound:
return fmt.Sprintf("replaced while still busy: it said busy since %s and the bound of %s ran out "+
"(novox/hq ADR 0335 §3)%s", since, declaration.BusyBound, said)
case BusyReplacedIdle:
return fmt.Sprintf("replaced once it said idle, after waiting since %s (novox/hq ADR 0335 §2)%s", since, said)
}
return fmt.Sprintf("busy: left running and replaced on the first pass it says idle, or at %s at the latest; "+
"waiting since %s (novox/hq ADR 0335 §2)%s", b.Until.UTC().Format(time.RFC3339), since, said)
}
// owedSinceBusy is the reason a container still owes the replacement it was left running for, or empty: its
// record keeps when its busy wait began, under the same name.
func owedSinceBusy(r *declaration.Container, previous store.Applied) string {
if previous.BusySince == nil || previous.Target != r.Name {
return ""
}
return "the replacement owed since its busy wait began at " + previous.BusySince.UTC().Format(time.RFC3339)
}
// busyClock is the clock the bound is read on. A test moves it.
var busyClock = time.Now
// busySaidMost is how much of a busy command's output a report carries: its last lines, enough to say what
// it held.
const busySaidMost = 400
// askBeforeReplacing asks a running container that declares busy whether it may be replaced now, and says
// in out what came of it. True when it waits: out is then the whole outcome of this pass. False when the
// caller replaces it — idle, or past the bound — with out carrying the wait it ended, if there was one.
func askBeforeReplacing(ctx context.Context, r *declaration.Container, run Runner, cri string,
previous store.Applied, out *Outcome) bool {
now := busyClock().UTC()
busy, said := askBusy(ctx, r, run, cri)
var since time.Time
waited := previous.BusySince != nil && previous.Target == r.Name
if waited {
since = previous.BusySince.UTC()
// A wait kept from a clock that has since gone back reads as starting now, never as later: the bound
// holds from no later than this pass.
if since.After(now) {
since = now
}
}
wait := BusyWait{ID: r.ID, Container: r.Name, Since: since, Said: said}
switch {
case !busy && !waited:
// Idle at the first ask: replaced at once, as a container that declares nothing is.
return false
case !busy:
wait.State, wait.Until = BusyReplacedIdle, since.Add(declaration.BusyBound)
out.busy = &wait
return false
}
if !waited {
since = now
wait.Since = now
}
wait.Until = since.Add(declaration.BusyBound)
if !now.Before(wait.Until) {
wait.State = BusyReplacedAtBound
out.busy = &wait
return false
}
wait.State = BusyWaiting
out.Action, out.Detail = waitsBusy, wait.Words()
out.busySince, out.busy = &since, &wait
return true
}
// askBusy runs a container's busy command inside it, by its shell, as an exec health check is run: busy is
// an exit of 0 and nothing else. An ask that could not be made, or did not answer within its timeout, is
// not busy: the bound is what keeps a container whose command never answers from being held.
func askBusy(ctx context.Context, r *declaration.Container, run Runner, cri string) (bool, string) {
ask, cancel := context.WithTimeout(ctx, r.Busy.Within())
defer cancel()
output, err := run(ask, cri, "exec", r.Name, "/bin/sh", "-c", r.Busy.Command)
said := strings.TrimSpace(output)
if err != nil && said == "" {
said = err.Error()
}
return err == nil, lastOf(said, busySaidMost)
}
// lastOf is the end of a text, at most n bytes, on one line.
func lastOf(s string, n int) string {
s = strings.Join(strings.Fields(s), " ")
if len(s) <= n {
return s
}
from := len(s) - n
for from < len(s) && !utf8.RuneStart(s[from]) {
from++
}
return "…" + s[from:]
}
+434
View File
@@ -0,0 +1,434 @@
package apply
import (
"context"
"errors"
"path/filepath"
"slices"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
// A container that says it is busy is replaced once it is idle, within a bound (novox/hq ADR 0335). The
// machine of these tests runs two containers, a build agent that declares `busy` and a web server that
// does not; every call that replaces a container is counted.
type aBusyMachine struct {
mu sync.Mutex
// containers is each container's name and the spec it was created with; running whether it runs.
containers map[string]string
busy bool // what the agent's busy command answers
said string // what it prints
asked int
calls []string
}
func newBusyMachine() *aBusyMachine {
return &aBusyMachine{containers: map[string]string{}}
}
func (m *aBusyMachine) run(_ context.Context, _ string, args ...string) (string, error) {
args = asideShape(args)
m.mu.Lock()
defer m.mu.Unlock()
switch args[0] {
case "info", "image":
return "27.0\n", nil
case "container": // inspect
name := args[len(args)-1]
spec, there := m.containers[name]
if !there {
return "", errors.New("no such container")
}
return "true\t" + spec, nil
case "exec":
m.asked++
m.calls = append(m.calls, "ask "+args[1])
if strings.Join(args[2:], " ") != "/bin/sh -c mesh-builder busy" {
return "", errors.New("asked something else: " + strings.Join(args[2:], " "))
}
if m.busy {
return m.said + "\n", nil
}
return m.said + "\n", errors.New("exited 1")
case "rm":
name := args[len(args)-1]
m.calls = append(m.calls, "rm "+name)
delete(m.containers, name)
case "run":
name := args[slices.Index(args, "--name")+1]
m.calls = append(m.calls, "create "+name)
for i, a := range args {
if a == "--label" && strings.HasPrefix(args[i+1], specLabel+"=") {
m.containers[name] = strings.TrimPrefix(args[i+1], specLabel+"=")
}
}
}
return "", nil
}
func (m *aBusyMachine) mark() int {
m.mu.Lock()
defer m.mu.Unlock()
return len(m.calls)
}
// since is every call since mark that replaced a container, or asked one.
func (m *aBusyMachine) since(mark int) []string {
m.mu.Lock()
defer m.mu.Unlock()
return append([]string(nil), m.calls[mark:]...)
}
func (m *aBusyMachine) answer(busy bool, said string) {
m.mu.Lock()
defer m.mu.Unlock()
m.busy, m.said = busy, said
}
// twoContainers is the declaration at a version: v moves the agent's image tag through its env, w the web's.
func twoContainers(t *testing.T, agent, web string) *declaration.Declaration {
t.Helper()
return parse(t, `{"declaration":1,"resources":[
{"id":"build-agent.server","type":"container","name":"mesh-build-agent","image":"`+pinned+`",
"env":{"V":"`+agent+`"},"busy":{"command":"mesh-builder busy","timeout":"5s"}},
{"id":"web.server","type":"container","name":"web","image":"`+pinned+`","env":{"V":"`+web+`"}}
]}`)
}
// atClock moves the bound's clock for one test.
func atClock(t *testing.T, at *time.Time) {
t.Helper()
was := busyClock
busyClock = func() time.Time { return *at }
t.Cleanup(func() { busyClock = was })
}
func busyOf(r Report, id string) (BusyWait, bool) {
for _, b := range r.Busy {
if b.ID == id {
return b, true
}
}
return BusyWait{}, false
}
func applyBusy(t *testing.T, m *aBusyMachine, d *declaration.Declaration, known store.State) (Report, store.State) {
t.Helper()
report, state, err := Apply(context.Background(), archHost(t), d, known, store.OriginDeclared, m.run, nil, nil)
if err != nil {
t.Fatalf("the apply failed: %v", err)
}
return report, state
}
// The decision's first check: an apply that would replace a container whose busy command exits 0 leaves it
// running, applies the rest, and replaces it on the first pass the command exits non-zero.
func TestABusyContainerIsLeftRunningAndReplacedOnTheFirstPassItSaysIdle(t *testing.T) {
now := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
atClock(t, &now)
m := newBusyMachine()
_, state := applyBusy(t, m, twoContainers(t, "1", "1"), store.State{})
// A merge moves both: the agent holds a build, the web server holds nothing.
m.answer(true, "building novox/mesh-controller (check of #280) since 00:58")
mark := m.mark()
report, state := applyBusy(t, m, twoContainers(t, "2", "2"), state)
if got := m.since(mark); !slices.Equal(got, []string{"ask mesh-build-agent", "rm web", "create web"}) {
t.Fatalf("a busy agent was not left running while the rest applied: %v", got)
}
o := outcomeOf(report, "build-agent.server")
if o.Action != waitsBusy {
t.Fatalf("a busy container was reported %q, want %q: %+v", o.Action, waitsBusy, o)
}
for _, says := range []string{"busy", "first pass it says idle", "check of #280", "ADR 0335"} {
if !strings.Contains(o.Detail, says) {
t.Errorf("the outcome does not say %q: %s", says, o.Detail)
}
}
if (Report{Outcomes: []Outcome{o}}).Changed() {
t.Errorf("a container left running reads as a change: %+v", o)
}
b, said := busyOf(report, "build-agent.server")
if !said || b.State != BusyWaiting || !b.Since.Equal(now) || !b.Until.Equal(now.Add(15*time.Minute)) ||
b.Container != "mesh-build-agent" || !strings.Contains(b.Said, "check of #280") {
t.Fatalf("the report does not say the agent waits, since when and until when: %+v", report.Busy)
}
if !report.Waiting() {
t.Error("a report with a container waiting does not say so")
}
if rec, _ := state.Find("build-agent.server"); rec.BusySince == nil || !rec.BusySince.Equal(now) {
t.Fatalf("the wait is not kept in the container's record: %+v", rec)
}
// Asked again a minute later, still busy: still left running, the wait counted from the first apply.
now = now.Add(time.Minute)
mark = m.mark()
report, state = applyBusy(t, m, twoContainers(t, "2", "2"), state)
if got := m.since(mark); !slices.Equal(got, []string{"ask mesh-build-agent"}) {
t.Fatalf("a second pass while busy did more than ask: %v", got)
}
if b, _ := busyOf(report, "build-agent.server"); b.State != BusyWaiting || !b.Since.Equal(now.Add(-time.Minute)) {
t.Fatalf("the second pass does not keep the wait from the first apply: %+v", report.Busy)
}
// The build ends: the next pass replaces it, and says it waited.
now = now.Add(time.Minute)
m.answer(false, "idle")
mark = m.mark()
report, state = applyBusy(t, m, twoContainers(t, "2", "2"), state)
if got := m.since(mark); !slices.Equal(got, []string{"ask mesh-build-agent", "rm mesh-build-agent", "create mesh-build-agent"}) {
t.Fatalf("the agent was not replaced on the first pass it said idle: %v", got)
}
if o := outcomeOf(report, "build-agent.server"); o.Action != "updated" || !strings.Contains(o.Detail, "once it said idle") {
t.Errorf("the replacement does not say it waited: %+v", o)
}
if b, _ := busyOf(report, "build-agent.server"); b.State != BusyReplacedIdle {
t.Errorf("the report does not say the agent was replaced when idle: %+v", report.Busy)
}
if report.Waiting() {
t.Error("a report with nothing waiting says something waits")
}
if rec, _ := state.Find("build-agent.server"); rec.BusySince != nil {
t.Errorf("the wait is still kept after the replacement: %+v", rec)
}
// And the pass after that has nothing to say about it.
report, _ = applyBusy(t, m, twoContainers(t, "2", "2"), state)
if len(report.Busy) != 0 {
t.Errorf("a container replaced is still said: %+v", report.Busy)
}
}
// An idle container is replaced at once, as one that declares nothing is: asked once, nothing waits.
func TestAnIdleContainerIsReplacedAtOnce(t *testing.T) {
m := newBusyMachine()
_, state := applyBusy(t, m, twoContainers(t, "1", "1"), store.State{})
m.answer(false, "")
mark := m.mark()
report, _ := applyBusy(t, m, twoContainers(t, "2", "1"), state)
if got := m.since(mark); !slices.Equal(got, []string{"ask mesh-build-agent", "rm mesh-build-agent", "create mesh-build-agent"}) {
t.Fatalf("an idle agent was not replaced at once: %v", got)
}
if len(report.Busy) != 0 {
t.Errorf("an idle container replaced at once is said as a wait: %+v", report.Busy)
}
// And a container whose declaration did not move is never asked.
mark = m.mark()
applyBusy(t, m, twoContainers(t, "2", "1"), state)
if got := m.since(mark); slices.Contains(got, "ask mesh-build-agent") {
t.Errorf("a container nothing replaces was asked: %v", got)
}
}
// Past the bound it is replaced anyway, the report carrying the command's last output — and the bound holds
// across a restart of the node-engine: the wait is read back from the state on disk.
func TestABusyContainerIsReplacedAtTheBoundEvenAfterARestart(t *testing.T) {
now := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
atClock(t, &now)
m := newBusyMachine()
_, state := applyBusy(t, m, twoContainers(t, "1", "1"), store.State{})
m.answer(true, "building novox/mesh-host (a check that never ends)")
_, state = applyBusy(t, m, twoContainers(t, "2", "1"), state)
// The node-engine stops and starts again: what it knows is what it saved.
path := filepath.Join(t.TempDir(), "state.json")
if err := store.Save(path, state); err != nil {
t.Fatal(err)
}
reloaded, err := store.Load(path)
if err != nil {
t.Fatal(err)
}
now = now.Add(14 * time.Minute)
mark := m.mark()
report, reloaded := applyBusy(t, m, twoContainers(t, "2", "1"), reloaded)
if got := m.since(mark); slices.Contains(got, "rm mesh-build-agent") {
t.Fatalf("a container busy within the bound was replaced after a restart: %v", got)
}
if b, _ := busyOf(report, "build-agent.server"); b.State != BusyWaiting || !b.Since.Equal(now.Add(-14*time.Minute)) {
t.Fatalf("a restart started the wait again: %+v", report.Busy)
}
now = now.Add(time.Minute)
mark = m.mark()
report, _ = applyBusy(t, m, twoContainers(t, "2", "1"), reloaded)
if got := m.since(mark); !slices.Equal(got, []string{"ask mesh-build-agent", "rm mesh-build-agent", "create mesh-build-agent"}) {
t.Fatalf("a container still busy at the bound was not replaced: %v", got)
}
b, _ := busyOf(report, "build-agent.server")
if b.State != BusyReplacedAtBound || !strings.Contains(b.Said, "never ends") {
t.Fatalf("the report does not say it replaced a busy container, with its last output: %+v", report.Busy)
}
if o := outcomeOf(report, "build-agent.server"); !strings.Contains(o.Detail, "still busy") ||
!strings.Contains(o.Detail, "never ends") {
t.Errorf("the outcome does not say a busy container was replaced, with what it said: %s", o.Detail)
}
}
// While one container waits, a send that changes another module on the same machine is applied at once: the
// apply returns after one ask, and holds nothing for the wait.
func TestASendToAnotherModuleIsAppliedWhileOneWaits(t *testing.T) {
now := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
atClock(t, &now)
m := newBusyMachine()
_, state := applyBusy(t, m, twoContainers(t, "1", "1"), store.State{})
m.answer(true, "building")
_, state = applyBusy(t, m, twoContainers(t, "2", "1"), state)
// The next send moves only the web server.
mark := m.mark()
done := make(chan struct{})
var report Report
go func() {
defer close(done)
report, _ = applyBusy(t, m, twoContainers(t, "2", "2"), state)
}()
select {
case <-done:
case <-time.After(5 * time.Second):
t.Fatal("an apply while a container waits did not return")
}
got := m.since(mark)
if !slices.Contains(got, "create web") || slices.Contains(got, "rm mesh-build-agent") {
t.Fatalf("the other module's send was not applied at once beside the wait: %v", got)
}
if o := outcomeOf(report, "web.server"); o.Action != "updated" {
t.Errorf("the web server was reported %q: %+v", o.Action, o)
}
}
// A container's busy is held to its shape where it arrives.
func TestBusyIsReadAndHeldToItsShape(t *testing.T) {
d := twoContainers(t, "1", "1")
c := d.Resources[0].(*declaration.Container)
if c.Busy == nil || c.Busy.Command != "mesh-builder busy" || c.Busy.Within() != 5*time.Second {
t.Fatalf("busy was not read: %+v", c.Busy)
}
for name, raw := range map[string]string{
"no command": `"busy":{"command":" "}`,
"a long timeout": `"busy":{"command":"x","timeout":"5m"}`,
"not a duration": `"busy":{"command":"x","timeout":"soon"}`,
"an unknown key": `"busy":{"command":"x","every":"1m"}`,
"on a step": `"run-once":true,"busy":{"command":"x"}`,
} {
_, err := declaration.Parse([]byte(`{"declaration":1,"resources":[{"id":"a.b","type":"container",` +
`"name":"b","image":"` + pinned + `",` + raw + `}]}`))
if err == nil {
t.Errorf("%s: a busy of that shape was accepted", name)
}
}
}
// A replacement owed to a restart-on trigger is still owed after the wait: a trigger that is no file or
// directory (a service, an identity's or a library's change) is in no container's spec and counts as changed only
// in the pass it changed, so the container must not read unchanged at the next pass and drop from the report as if
// replaced (novox/hq ADR 0335).
func TestAReplacementOwedToARestartTriggerIsMadeOnceIdle(t *testing.T) {
now := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
atClock(t, &now)
d := parse(t, `{"declaration":1,"resources":[
{"id":"build-agent.server","type":"container","name":"mesh-build-agent","image":"`+pinned+`",
"restart-on":["identity:build-agent"],"busy":{"command":"mesh-builder busy"}}
]}`)
r := d.Resources[0].(*declaration.Container)
in := inputs{declares: map[string]string{}}
m := newBusyMachine()
m.containers["mesh-build-agent"] = containerSpec(r, in)
ctx := context.Background()
// The trigger moved this pass, and the agent holds a build: left running.
m.answer(true, "building")
out, err := applyContainer(ctx, r, m.run, map[string]bool{"identity:build-agent": true}, in, store.Applied{})
if err != nil || out.Action != waitsBusy || out.busySince == nil {
t.Fatalf("a busy container was not left running for its trigger: %+v %v", out, err)
}
record := store.Applied{ID: r.ID, Target: r.Name, BusySince: out.busySince}
// The next pass: nothing triggers now, and the replacement is still owed.
now = now.Add(time.Minute)
m.answer(false, "idle")
mark := m.mark()
out, err = applyContainer(ctx, r, m.run, map[string]bool{}, in, record)
if err != nil {
t.Fatal(err)
}
if got := m.since(mark); !slices.Equal(got, []string{"ask mesh-build-agent", "rm mesh-build-agent", "create mesh-build-agent"}) {
t.Fatalf("the replacement owed to the trigger was dropped: %v (%+v)", got, out)
}
if out.busy == nil || out.busy.State != BusyReplacedIdle || out.busySince != nil {
t.Fatalf("the replacement does not say it was made once idle: %+v", out)
}
}
// A wait kept from a clock that later went back is never longer than the bound from now.
func TestAWaitFromAClockThatWentBackIsClampedToNow(t *testing.T) {
now := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
atClock(t, &now)
m := newBusyMachine()
_, state := applyBusy(t, m, twoContainers(t, "1", "1"), store.State{})
m.answer(true, "building")
_, state = applyBusy(t, m, twoContainers(t, "2", "1"), state)
now = now.Add(-time.Hour) // the clock went back an hour
report, _ := applyBusy(t, m, twoContainers(t, "2", "1"), state)
b, _ := busyOf(report, "build-agent.server")
if b.State != BusyWaiting || b.Since.After(now) || !b.Until.Equal(now.Add(15*time.Minute)) {
t.Fatalf("a wait from a later clock is not clamped to now: %+v (now %s)", b, now)
}
}
// An ask that fails — no shell in the image, a timeout, a paused container — is not busy: replaced at once.
func TestAnAskThatFailsIsIdle(t *testing.T) {
m := newBusyMachine()
_, state := applyBusy(t, m, twoContainers(t, "1", "1"), store.State{})
failing := func(ctx context.Context, name string, args ...string) (string, error) {
if args[0] == "exec" {
m.mu.Lock()
m.calls = append(m.calls, "ask "+args[1])
m.mu.Unlock()
return "", errors.New("OCI runtime exec failed: exec: \"/bin/sh\": no such file or directory")
}
return m.run(ctx, name, args...)
}
mark := m.mark()
report, _, err := Apply(context.Background(), archHost(t), twoContainers(t, "2", "1"), state,
store.OriginDeclared, failing, nil, nil)
if err != nil {
t.Fatal(err)
}
if got := m.since(mark); !slices.Equal(got, []string{"ask mesh-build-agent", "rm mesh-build-agent", "create mesh-build-agent"}) {
t.Fatalf("an ask that failed held the container: %v", got)
}
if len(report.Busy) != 0 {
t.Errorf("an ask that failed is said as a wait: %+v", report.Busy)
}
}
// A stopped container holds no work: it is replaced without being asked.
func TestAStoppedContainerIsNotAsked(t *testing.T) {
m := newBusyMachine()
_, state := applyBusy(t, m, twoContainers(t, "1", "1"), store.State{})
m.answer(true, "would say busy")
stopped := func(ctx context.Context, name string, args ...string) (string, error) {
if args[0] == "container" && args[len(args)-1] == "mesh-build-agent" {
out, err := m.run(ctx, name, args...)
return strings.Replace(out, "true\t", "false\t", 1), err
}
return m.run(ctx, name, args...)
}
mark := m.mark()
if _, _, err := Apply(context.Background(), archHost(t), twoContainers(t, "2", "1"), state,
store.OriginDeclared, stopped, nil, nil); err != nil && !strings.Contains(err.Error(), "not running") {
t.Fatal(err)
}
got := m.since(mark)
if slices.Contains(got, "ask mesh-build-agent") || !slices.Contains(got, "rm mesh-build-agent") {
t.Fatalf("a stopped container was asked, or not replaced: %v", got)
}
}
+54
View File
@@ -0,0 +1,54 @@
package apply
import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// Issue 1637 (novox/hq ADR 0480 clause 7): a container's supplementary groups reach the runtime as --group-add.
func groupsContainer(groups ...int64) *declaration.Container {
return &declaration.Container{ID: "n8n.app", Name: "n8n", Image: pinned, Groups: groups}
}
// 1637/AC1: both run commands carry one --group-add per group, and none without groups.
func TestAContainerIsAddedItsSupplementaryGroups(t *testing.T) {
with := groupsContainer(2000, 2001)
for name, args := range map[string][]string{
"detached": detachedRunArgs(with, "x"),
"foreground": foregroundRunArgs(with, "x"),
} {
got := " " + strings.Join(args, " ") + " "
if !strings.Contains(got, " --group-add 2000 ") || !strings.Contains(got, " --group-add 2001 ") {
t.Errorf("1637/AC1: the %s run does not add both groups: %s", name, got)
}
}
without := groupsContainer()
for name, args := range map[string][]string{
"detached": detachedRunArgs(without, "x"),
"foreground": foregroundRunArgs(without, "x"),
} {
if strings.Contains(strings.Join(args, " "), "--group-add") {
t.Errorf("1637/AC1: the %s run adds a group none was declared", name)
}
}
}
// 1637/AC2: a changed set of groups recreates the container; no groups leaves the spec as it was.
func TestAChangeOfGroupsRecreatesTheContainer(t *testing.T) {
none, a, b := groupsContainer(), groupsContainer(2000), groupsContainer(2000, 2001)
if containerSpec(none, inputs{}) == containerSpec(a, inputs{}) || containerSpec(a, inputs{}) == containerSpec(b, inputs{}) {
t.Error("1637/AC2: a change of groups does not change the container's spec")
}
same := groupsContainer(2000)
if containerSpec(a, inputs{}) != containerSpec(same, inputs{}) {
t.Error("1637/AC2: the same groups changed the spec")
}
plain := *none
plain.Groups = nil
if containerSpec(none, inputs{}) != containerSpec(&plain, inputs{}) {
t.Error("1637/AC2: a container with no groups has its spec moved")
}
}
@@ -6,7 +6,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
@@ -20,6 +20,9 @@ func TestAContainerJoinsTheNetworksItKeeps(t *testing.T) {
if name != "docker" {
return "", errors.New("not installed")
}
if len(args) > 0 && args[0] == "container" && strings.HasSuffix(args[len(args)-1], asideSuffix) {
return "", errors.New("no such container") // nothing was left set aside
}
ran = append(ran, strings.Join(args, " "))
switch args[0] {
case "info":
@@ -75,69 +78,6 @@ func TestAContainerJoinsTheNetworksItKeeps(t *testing.T) {
var specOfLast string
// A module the mesh left out of a declaration is not a module the mesh removed (novox/hq ADR 0163,
// rule 6): what the host wrote for it stays, recorded and said; what it holds for it stays held.
// A module simply absent is removed as it always was.
func TestALeftOutModuleIsNeitherRemovedNorForgotten(t *testing.T) {
var removed []string
gone := map[string]bool{}
run := func(_ context.Context, name string, args ...string) (string, error) {
args = oldShape(name, args)
if name != "docker" {
return "", nil
}
switch args[0] {
case "info":
return "29.0.0\n", nil
case "rm":
removed = append(removed, args[len(args)-1])
gone[args[len(args)-1]] = true
case "container":
if gone[args[len(args)-1]] {
return "", errors.New("no such container")
}
return "true\tspec", nil
}
return "", nil
}
known := store.State{
Resources: []store.Applied{
{ID: "web.server", Type: "container", Target: "web", Origin: store.OriginDeclared},
{ID: "old.server", Type: "container", Target: "old", Origin: store.OriginDeclared},
},
Held: []store.Held{{ID: "web.page", Module: "web", Kind: "file", Target: "/srv/web/index.html"}},
}
d := parse(t, `{"declaration":1,"left_out":["web"],"resources":[
{"id":"notes.conf","type":"file","path":"`+t.TempDir()+`/notes.conf","content":"x"}
]}`)
report, state, err := Apply(context.Background(), archHost(t), d, known, store.OriginDeclared, run, nil, nil)
if err != nil {
t.Fatal(err)
}
if len(removed) != 1 || removed[0] != "old" {
t.Fatalf("removed %v; only the module that is absent goes", removed)
}
if _, kept := state.At("container", "web"); !kept {
t.Fatal("the left-out module's record was forgotten")
}
if _, held := state.HeldAt("web.page"); !held {
t.Fatal("the left-out module's hold was released")
}
said := false
for _, o := range report.Outcomes {
if o.ID == "web.server" && o.Action == "unchanged" && strings.Contains(o.Detail, "web was left out of this declaration by the mesh") {
said = true
}
if o.ID == "web.server" && o.Action != "unchanged" {
t.Errorf("the left-out module's container was %s", o.Action)
}
}
if !said {
t.Fatalf("keeping the left-out module's container was not said: %+v", report.Outcomes)
}
}
// A container's capabilities reach the runtime and are part of its spec (novox/hq ADR 0170).
func TestACapabilityReachesTheRuntimeAndTheSpec(t *testing.T) {
var ran []string
run := func(_ context.Context, name string, args ...string) (string, error) {
+158
View File
@@ -0,0 +1,158 @@
package apply
import (
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// A module that declares its own system account and a .ssh in that account's home is placed (novox/hq issue 1542):
// the account is not in the user database yet when the declaration is judged, and its home is under /var/lib.
func declaredHomeFixture(t *testing.T) (home string, user *declaration.User) {
t.Helper()
dir := t.TempDir()
passwd := filepath.Join(dir, "passwd")
if err := os.WriteFile(passwd, []byte("root:x:0:0::/root:/bin/sh\n"), 0o644); err != nil {
t.Fatal(err)
}
before := passwdFile
passwdFile = passwd
t.Cleanup(func() { passwdFile = before })
home = filepath.Join(dir, "lib", "mesh-build-go")
return home, &declaration.User{ID: "mesh-build-go.account", Type: declaration.TypeUser, Name: "mesh-build-go",
Shell: "/usr/bin/nologin", Home: home}
}
// 1542/AC1: the .ssh of a declared account, owned by it, is refused by nothing.
func TestIssue1542ADeclaredAccountsSSHIsPlaced(t *testing.T) {
home, user := declaredHomeFixture(t)
ssh := &declaration.Directory{ID: "mesh-build-go.ssh-dir", Type: declaration.TypeDirectory, Path: home + "/.ssh",
Mode: "0700", Owner: "mesh-build-go"}
refused := refusedPlaces([]declaration.Resource{user, ssh})
if len(refused) != 0 {
t.Fatalf("a declared account's own .ssh was refused: %v", refused)
}
if err := (inputs{refused: refused}).refused[filepath.Clean(ssh.Path)]; err != nil {
t.Fatalf("the apply-time check refuses it: %v", err)
}
if line := JudgeDirectories(&declaration.Declaration{Resources: []declaration.Resource{user, ssh}}); strings.Contains(line, "refused already by the placement guard: ") {
t.Fatalf("the read-only judgement refuses it: %s", line)
}
}
// 1542/AC2: another account's directory below the declared home is refused as below any home, and a .ssh below
// no home of the machine or the declaration is refused with today's words.
func TestIssue1542OnlyTheDeclaredAccountsOwnIsPlaced(t *testing.T) {
home, user := declaredHomeFixture(t)
other := &declaration.Directory{ID: "notes.ssh-dir", Type: declaration.TypeDirectory, Path: home + "/.ssh",
Mode: "0700", Owner: "intruder"}
refused := refusedPlaces([]declaration.Resource{user, other})
err := refused[filepath.Clean(other.Path)]
if err == nil || !strings.Contains(err.Error(), "below mesh-build-go's home and declared intruder's") {
t.Fatalf("another account's directory below a declared home: %v", err)
}
stray := &declaration.Directory{ID: "mesh-build-go.ssh-dir", Type: declaration.TypeDirectory,
Path: filepath.Join(filepath.Dir(home), "elsewhere", ".ssh"), Mode: "0700", Owner: "mesh-build-go"}
refused = refusedPlaces([]declaration.Resource{user, stray})
err = refused[filepath.Clean(stray.Path)]
if err == nil || !strings.Contains(err.Error(), "a .ssh directory is placed only below its account's home, as that account's") {
t.Fatalf("a .ssh below no home: %v", err)
}
// Not a .ssh: a root-owned directory below the declared home is placed exactly as on main.
data := &declaration.Directory{ID: "mesh-build-go.data", Type: declaration.TypeDirectory, Path: home + "/data",
Mode: "0755", Owner: "root"}
if refused := refusedPlaces([]declaration.Resource{user, data}); len(refused) != 0 {
t.Fatalf("a root-owned directory below a declared home, not a .ssh, was refused: %v", refused)
}
// And without the account declared, the same .ssh is refused as it was.
own := &declaration.Directory{ID: "mesh-build-go.ssh-dir", Type: declaration.TypeDirectory, Path: home + "/.ssh",
Mode: "0700", Owner: "mesh-build-go"}
if refusedPlaces([]declaration.Resource{own})[filepath.Clean(own.Path)] == nil {
t.Fatal("a .ssh below an account nobody makes was placed")
}
}
// 1542/AC3: a declared account is no home to anyone but the directory placement guard.
func TestIssue1542DeclaredAccountsAreNotHomesToTheOtherGuards(t *testing.T) {
home, user := declaredHomeFixture(t)
_ = refusedPlaces([]declaration.Resource{user})
if _, ok := accountsOfHomes()[home]; ok {
t.Fatal("a declared account became a home of the machine")
}
if homeAbove(home+"/.ssh") != "" {
t.Fatal("a declared account became a home for the links and the move-out")
}
}
// A declared account is added as a home and never takes one over (reviewer, issue 1542): the person's own .ssh
// stays refused to every other account, whatever a declaration says about a user.
func TestIssue1542ADeclaredAccountTakesNoExistingHome(t *testing.T) {
dir := t.TempDir()
passwd := filepath.Join(dir, "passwd")
body := "root:x:0:0::/root:/bin/sh\njochen:x:1000:1000::/home/jochen:/bin/sh\n" +
"svc:x:998:998::/var/lib/svc:/usr/bin/nologin\n"
if err := os.WriteFile(passwd, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
before := passwdFile
passwdFile = passwd
t.Cleanup(func() { passwdFile = before })
newHome := filepath.Join(dir, "lib", "mesh-build-go")
ssh := func(path, owner string) *declaration.Directory {
return &declaration.Directory{ID: "x.ssh", Type: declaration.TypeDirectory, Path: path, Mode: "0700", Owner: owner}
}
user := func(name, home string) *declaration.User {
return &declaration.User{ID: "x.user", Type: declaration.TypeUser, Name: name, Home: home}
}
// A declared user over a person's home, whoever it is named for.
for _, name := range []string{"evil", "postgres", "root"} {
d := ssh("/home/jochen/.ssh", name)
if refusedPlaces([]declaration.Resource{user(name, "/home/jochen"), d})[d.Path] == nil {
t.Errorf("%s, declared over jochen's home, was given jochen's .ssh", name)
}
}
// A declared home below a person's home.
d := ssh("/home/jochen/sub/.ssh", "evil")
if refusedPlaces([]declaration.Resource{user("evil", "/home/jochen/sub"), d})[d.Path] == nil {
t.Error("a home declared below jochen's home was counted")
}
// A name the machine has already, declared with another home: that is not its home.
d = ssh(newHome+"/e/.ssh", "svc")
if refusedPlaces([]declaration.Resource{user("svc", newHome+"/e"), d})[d.Path] == nil {
t.Error("an existing account's declared home was counted")
}
// A number is no name: chown -1 leaves the owner as it was.
for _, owner := range []string{"-1", "-1:0", "0", "1001"} {
d := ssh(newHome+"/.ssh", owner)
if refusedPlaces([]declaration.Resource{user("mesh-build-go", newHome), d})[d.Path] == nil {
t.Errorf("owner %q was placed below a declared home", owner)
}
}
// And the account that is new, by name, still is.
d = ssh(newHome+"/.ssh", "mesh-build-go")
if got := refusedPlaces([]declaration.Resource{user("mesh-build-go", newHome), d}); len(got) != 0 {
t.Errorf("a new account's own .ssh was refused: %v", got)
}
}
// The account the mesh made, as declared, is judged the same at the next apply (issue 1542): once it is in the
// user database with exactly the declared home, its .ssh is still placed.
func TestIssue1542TheAccountTheMeshMadeIsStillPlacedAtTheNextApply(t *testing.T) {
home, user := declaredHomeFixture(t)
ssh := &declaration.Directory{ID: "mesh-build-go.ssh-dir", Type: declaration.TypeDirectory, Path: home + "/.ssh",
Mode: "0700", Owner: "mesh-build-go"}
resources := []declaration.Resource{user, ssh}
if got := refusedPlaces(resources); len(got) != 0 {
t.Fatalf("first apply, the account not yet made: %v", got)
}
body := "root:x:0:0::/root:/bin/sh\nmesh-build-go:x:998:998::" + home + ":/usr/bin/nologin\n"
if err := os.WriteFile(passwdFile, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
if got := refusedPlaces(resources); len(got) != 0 {
t.Fatalf("second apply, the account made as declared: %v", got)
}
}
+19
View File
@@ -0,0 +1,19 @@
package apply
import (
"context"
"strings"
"testing"
"time"
)
// A command killed because its context ended says so, and why: "docker exited -1:" said nothing while novox's bus
// container was half replaced (issue 1945, live on 2026-10-25 at 11:23).
func TestACommandKilledByItsContextSaysWhy(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
go func() { time.Sleep(50 * time.Millisecond); cancel() }()
_, err := ExecRunner(ctx, "sleep", "5")
if err == nil || !strings.Contains(err.Error(), "killed") || !strings.Contains(err.Error(), "context canceled") {
t.Fatalf("a command its context killed said: %v", err)
}
}
+235
View File
@@ -0,0 +1,235 @@
package apply
import (
"context"
"fmt"
"slices"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
// WriteFilled writes what the provisioner's client filled into one file the declaration names as filled (novox/hq
// ADR 0349 §4), and makes what reads the file take the change, exactly as an apply that changed it would.
//
// **The declared file says everything but the content.** Its path, mode, owner and who owns it at runtime are the
// declaration's; the client's request carries the content alone. So the file is applied as the declared one with
// that content in place of the declaration's: an owned file whole through applyFile, a seed only when absent, a
// managed file's keys through applyInto or its block through applyBlock — the same guards on where nothing is
// placed, the same Keep of an original found there, the same record. What the declaration seals or fills with
// `${secret:}` the client has already filled, so the content is written as it came; a file that carried a secret
// keeps the mode a secret is written with when the declaration gives none.
//
// **Then what reads it, in the apply's order** (afterWhatTheyRead): a service naming it under restart-on is
// restarted and under reload-on reloaded, a process or container naming it under restart-on is restarted, and a
// running container that reads it at creation — an env-file, a file mounted into it — is recreated, because what
// it read is recorded by digest and this write changed the digest. Each of those is applied and recorded as an
// apply does, and what moves goes on moving what names it. Nothing else of the declaration is touched.
//
// The caller holds the apply's lock: a write and an apply never write one file at once. log is said the path, the
// action and the digest, never the content.
//
// **It does not refresh the runtimes' data roots.** A filled write is one file, not an apply; the placement guard
// judges it with the roots the last apply read (runtimeDataRoots), falling back to the runtimes' configuration
// files when no apply has read them yet — the runtimes themselves are asked once, at an apply's start, not once
// per file (novox/hq issue 1138).
func WriteFilled(
ctx context.Context,
sys system.System,
d *declaration.Declaration,
known store.State,
path string,
content []byte,
run Runner,
log func(string),
keep Keep,
windows *Windows,
) (report Report, _ store.State, _ error) {
if log == nil {
log = func(string) {}
}
if !d.Fills(path) {
return report, known, &Error{Resource: path, Err: fmt.Errorf("this node's declaration does not name %s "+
"as a file the provisioner fills, so nothing was written", path)}
}
var declared []*declaration.File
for _, r := range d.Resources {
if f, ok := r.(*declaration.File); ok && f.Path == path {
declared = append(declared, f)
}
}
if len(declared) != 1 {
// The parser refuses a filled path no file declares; two at one path would need the request to say whose
// keys or block it carries, and it carries the content only.
return report, known, &Error{Resource: path, Err: fmt.Errorf("%d resources declare %s, and a filled file "+
"is written for exactly one, so nothing was written", len(declared), path)}
}
file := filledAs(declared[0], content)
if held, isHeld := known.HeldAt(file.ID); isHeld {
return report, known, &Error{Resource: file.ID, Err: fmt.Errorf("%s is held as found for a module not "+
"yet taken (%s), so nothing was written", path, held.Why)}
}
in := inputs{declares: map[string]string{}, known: &known, windows: windows, away: newStoreAway(windows, log),
groups: groupsWanted(d.Resources), handed: Handed{}}
in.refused = refusedPlaces(d.Resources)
in.agentHomes = rootNeverHomes(d)
in.dirsBefore = map[string]bool{}
in.files = declaredFiles(d.Resources)
// A container that reads the file and mounts a library whose disk or mount is not there is held, as a full apply
// holds it: not created over the missing disk (novox/hq ADR 0328 §9).
in.held = heldLibraries(ctx, d.Libraries)
for _, r := range d.Resources {
in.declares[r.Identity()] = declaredDigest(r)
}
changed := map[string]bool{}
// The file, as the apply loop applies it.
was, _ := known.Find(file.ID)
var keepFound Keep
if (was.ID == "" || was.Target != file.Path) && !known.Recorded(string(declaration.TypeFile), file.Path) {
keepFound = keep
}
outcome, err := applyOne(ctx, sys, file, run, changed, in, was, nil, keepFound)
if err != nil {
log(fmt.Sprintf(" failed %s (%s): %v", file.ID, file.Path, err))
return report, known, &Error{Resource: file.ID, Err: err, Done: report}
}
recordKeeping(&known, file, outcome, was)
report.Outcomes = append(report.Outcomes, outcome)
if !moved(outcome) {
return report, known, nil
}
changed[file.ID] = true
line := fmt.Sprintf(" %s %s (%s), filled: %s", outcome.Action, file.ID, file.Path, outcome.wrote)
if outcome.Detail != "" {
line += ": " + outcome.Detail
}
log(line)
// What reads it, in the apply's order, each moving what names it in turn.
var failures []*Error
for _, r := range afterWhatTheyRead(d.Resources) {
if r.Identity() == file.ID || !readsChange(r, path, changed) {
continue
}
if _, isHeld := known.HeldAt(r.Identity()); isHeld {
log(fmt.Sprintf(" held %s (%s): held as found for a module not yet taken, so it was not restarted "+
"for %s", r.Identity(), r.Target(), file.ID))
continue
}
was, _ := known.Find(r.Identity())
o, err := applyOne(ctx, sys, r, run, changed, in, was, nil, nil)
if err != nil {
if o.found != nil {
known.KeepFound(r.Identity(), store.OriginDeclared, *o.found)
}
failures = append(failures, &Error{Resource: r.Identity(), Err: err})
log(fmt.Sprintf(" failed %s (%s): %v", r.Identity(), o.Target, err))
continue
}
if o.Action == "waiting" {
if o.found != nil {
known.KeepFound(r.Identity(), store.OriginDeclared, *o.found)
}
report.Outcomes = append(report.Outcomes, o)
log(fmt.Sprintf(" waiting %s (%s): %s", o.ID, o.Target, o.Detail))
continue
}
recordKeeping(&known, r, o, was)
report.Outcomes = append(report.Outcomes, o)
if o.busy != nil {
report.Busy = append(report.Busy, *o.busy)
}
if moved(o) {
changed[r.Identity()] = true
line := fmt.Sprintf(" %s %s (%s)", o.Action, o.ID, o.Target)
if o.Detail != "" {
line += ": " + o.Detail
}
log(line)
} else if o.Action != "unchanged" {
log(fmt.Sprintf(" %s %s (%s): %s", o.Action, o.ID, o.Target, o.Detail))
}
}
if len(failures) > 0 {
first := failures[0]
first.Done = report
first.Others = len(failures) - 1
return report, known, first
}
return report, known, nil
}
// filledAs is the declared file with the filled content in place of the declaration's.
func filledAs(declared *declaration.File, content []byte) *declaration.File {
f := *declared
f.Content, f.Sealed, f.VaultSealed, f.Bytes, f.Secrets = string(content), "", nil, "", nil
if f.Mode == "" && (declared.Secret() || len(declared.Secrets) > 0) {
// Written as the declaration would have written it: a file carrying a credential is not world-readable.
f.Mode = "0600"
}
return &f
}
// readsChange reports whether applying r is what makes it take a change of what changed: a service's restart-on or
// reload-on, a process's or container's restart-on, or a running container that reads path at creation.
func readsChange(r declaration.Resource, path string, changed map[string]bool) bool {
switch res := r.(type) {
case *declaration.Service:
return len(restartedBy(res.RestartOn, changed)) > 0 || len(restartedBy(res.ReloadOn, changed)) > 0
case *declaration.Process:
return len(restartedBy(res.RestartOn, changed)) > 0
case *declaration.Container:
if len(restartedBy(res.RestartOn, changed)) > 0 {
return true
}
if res.RunOnce || res.Schedule != "" {
return false
}
if slices.Contains(res.EnvFile, path) {
return true
}
for _, v := range res.Volumes {
if mountSource(v) == path {
return true
}
}
}
return false
}
// recordKeeping records r as applied, carrying where the original it replaced was kept from its record when this
// write kept none — for as long as the resource is recorded at the same target, as the apply loop carries it.
func recordKeeping(known *store.State, r declaration.Resource, o Outcome, was store.Applied) {
kept, keptMode, keptOwner := o.kept, o.keptMode, o.keptOwner
if kept == "" && was.Target == o.Target {
kept, keptMode, keptOwner = was.Kept, was.KeptMode, was.KeptOwner
}
recordApplied(known, store.OriginDeclared, r, o, kept, keptMode, keptOwner)
}
// leftToTheClient is a filled file as an apply leaves it (novox/hq ADR 0349 §5, design 57 P7): untouched, and said
// so. A file the mesh wrote at the same path under another id, going in this apply, is handed to it as an apply hands
// a whole file (novox/hq ADR 0223): its record — what the mesh wrote there, and the original it kept — is the filled
// file's now, so the verb judges what is there against it, and the file is neither removed nor written.
func leftToTheClient(f *declaration.File, known *store.State, handedFiles map[string]store.Applied, report *Report,
log func(string)) Outcome {
if from, handed := handedFiles[f.ID]; handed {
if _, recorded := known.Find(f.ID); !recorded {
rec := from
rec.ID = f.ID
known.Record(rec)
}
delete(handedFiles, f.ID)
known.Forget(from.ID)
detail := "no longer declared; " + f.ID + " declares the file now, filled by the provisioner's client, so " +
"it was handed to it, not removed"
report.Outcomes = append(report.Outcomes, Outcome{ID: from.ID, Type: from.Type, Target: from.Target,
Action: "forgotten", Detail: detail})
log(fmt.Sprintf(" forgotten %s (%s): %s", from.ID, from.Target, detail))
}
return Outcome{ID: f.ID, Type: string(declaration.TypeFile), Target: f.Path, Action: "unchanged",
Detail: "filled by the provisioner's client, which alone has it written"}
}
+218
View File
@@ -0,0 +1,218 @@
package apply
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
// Defends novox/hq ADR 0349 §5, one writer per file (design 57 P7, issue 673): a file the declaration names as
// filled is written only through the write verb (WriteFilled), on the provisioner client's word. An apply — the
// controller's send and the five-minute re-apply alike — never writes it, never records it, and removes it as an
// orphan only when the verb placed it.
// An apply leaves a filled file alone, whatever who owns it at runtime says: absent stays absent and is not
// recorded, and what the verb wrote stays as it wrote it through every re-apply.
func TestAnApplyNeverWritesAFilledFile(t *testing.T) {
for _, c := range []struct{ name, fields, content string }{
{"owned", `"content":"","mode":"0640","owner-at-runtime":"owned"`, "port = 8080\n"},
{"seed", `"content":"","mode":"0600","owner-at-runtime":"seed"`, "user default on\n"},
{"managed block", `"into":"block","content":"","owner-at-runtime":"managed"`, "10.42.0.1 ace\n"},
} {
t.Run(c.name, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "the-file")
d := filledDecl(t, path, c.fields, "")
report, state, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginDeclared,
noServices, nil, nil)
if err != nil {
t.Fatal(err)
}
if _, err := os.Stat(path); err == nil {
t.Fatalf("an apply wrote a filled file the client has not filled: %q", readText(t, path))
}
if _, ok := state.Find("m.conf"); ok {
t.Errorf("an apply recorded a filled file it did not write")
}
if o := outcomeOf(report, "m.conf"); o.Action != "unchanged" || !strings.Contains(o.Detail, "provisioner") {
t.Errorf("an apply did not say it left the file to the provisioner's client: %+v", o)
}
_, state, err = WriteFilled(context.Background(), archHost(t), d, state, path, []byte(c.content),
noServices, nil, nil, nil)
if err != nil {
t.Fatal(err)
}
placed, _ := state.Find("m.conf")
want, wantMode := readText(t, path), modeAt(t, path)
// The re-apply, twice over: nothing written, the verb's record kept as it wrote it.
for i := 0; i < 2; i++ {
var said []string
report, state, err = Apply(context.Background(), archHost(t), d, state, store.OriginDeclared,
noServices, func(line string) { said = append(said, line) }, nil)
if err != nil {
t.Fatal(err)
}
if got := readText(t, path); got != want || modeAt(t, path) != wantMode {
t.Fatalf("re-apply %d wrote the filled file again: %q, want %q", i+1, got, want)
}
if o := outcomeOf(report, "m.conf"); o.Action != "unchanged" {
t.Errorf("re-apply %d: %+v", i+1, o)
}
if rec, _ := state.Find("m.conf"); rec.Wrote != placed.Wrote || rec.Target != path {
t.Errorf("re-apply %d changed the verb's record: %+v, was %+v", i+1, rec, placed)
}
if len(said) != 0 {
t.Errorf("re-apply %d said something of an untouched file: %q", i+1, said)
}
}
})
}
}
// What reads a filled file is moved by the verb's write, once, never again by an apply: a service under
// restart-on is not restarted and a container reading it is not recreated by the re-apply.
func TestAnApplyMovesNothingForAFilledFile(t *testing.T) {
env := filepath.Join(t.TempDir(), "forge.env")
d := filledDecl(t, env, `"content":"","mode":"0600","owner-at-runtime":"owned"`,
`{"id":"forge.server","type":"container","name":"forge","image":"`+pinned+`","env-file":["`+env+`"]}`)
m := &machine{containers: map[string]*fakeContainer{}}
_, state, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, env, []byte("PORT=5432\n"),
m.run, nil, nil, nil)
if err != nil {
t.Fatal(err)
}
m.asked = nil
if _, _, err := Apply(context.Background(), archHost(t), d, state, store.OriginDeclared, m.run, nil,
nil); err != nil {
t.Fatal(err)
}
if m.removed("forge") || m.did("docker run") {
t.Errorf("the re-apply recreated the container reading the filled file: %v", m.asked)
}
if got := readText(t, env); got != "PORT=5432\n" {
t.Errorf("the re-apply wrote the filled file: %q", got)
}
conf := filepath.Join(t.TempDir(), "daemon.conf")
d = filledDecl(t, conf, `"content":"","owner-at-runtime":"owned"`,
`{"id":"m.daemon","type":"service","unit":"m.service","state":"running","restart-on":["m.conf"]}`)
var commands []string
_, state, err = WriteFilled(context.Background(), archHost(t), d, store.State{}, conf, []byte("level = 1\n"),
recordingServices(&commands), nil, nil, nil)
if err != nil {
t.Fatal(err)
}
commands = nil
if _, _, err := Apply(context.Background(), archHost(t), d, state, store.OriginDeclared,
recordingServices(&commands), nil, nil); err != nil {
t.Fatal(err)
}
if n := count(commands, "systemctl stop m.service"); n != 0 {
t.Errorf("the re-apply restarted what reads the filled file: %v", commands)
}
}
// A filled file the verb never placed is not the mesh's to remove: one found there, or put there by hand, stays
// when the file leaves the declaration.
func TestAFilledFileTheVerbNeverPlacedIsNotRemoved(t *testing.T) {
path := filepath.Join(t.TempDir(), "the-file")
if err := os.WriteFile(path, []byte("the machine's own\n"), 0o644); err != nil {
t.Fatal(err)
}
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`, "")
_, state, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginDeclared, noServices,
nil, nil)
if err != nil {
t.Fatal(err)
}
gone := elsewhere(t)
report, state, err := Apply(context.Background(), archHost(t), gone, state, store.OriginDeclared, noServices,
nil, nil)
if err != nil {
t.Fatal(err)
}
if got := readText(t, path); got != "the machine's own\n" {
t.Errorf("a filled file the verb never placed was touched when it left: %q", got)
}
if o := outcomeOf(report, "m.conf"); o.Action == "removed" {
t.Errorf("a filled file the verb never placed was removed: %+v", o)
}
if _, ok := state.Find("m.conf"); ok {
t.Errorf("a record of the filled file was left")
}
}
// What the verb placed is the mesh's: when the file leaves the declaration it is removed as an orphan, as a file
// an apply wrote is.
func TestAFilledFileTheVerbPlacedIsRemovedWhenItLeaves(t *testing.T) {
path := filepath.Join(t.TempDir(), "the-file")
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`, "")
_, state, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginDeclared, noServices,
nil, nil)
if err != nil {
t.Fatal(err)
}
_, state, err = WriteFilled(context.Background(), archHost(t), d, state, path, []byte("port = 8080\n"),
noServices, nil, nil, nil)
if err != nil {
t.Fatal(err)
}
gone := elsewhere(t)
report, state, err := Apply(context.Background(), archHost(t), gone, state, store.OriginDeclared, noServices,
nil, nil)
if err != nil {
t.Fatal(err)
}
if _, err := os.Stat(path); err == nil {
t.Errorf("a filled file the verb placed stayed when it left the declaration: %q", readText(t, path))
}
if o := outcomeOf(report, "m.conf"); o.Action != "removed" {
t.Errorf("not removed: %+v", o)
}
if _, ok := state.Find("m.conf"); ok {
t.Errorf("its record stayed")
}
}
// A file the mesh wrote the old way and that is now filled keeps its record for the verb to judge against, and is
// not written over by the apply that hands it to the client; a file at the same path recorded under another id
// is handed to the filled one, not removed.
func TestAFileThatMovesToTheClientIsHandedNotRewritten(t *testing.T) {
path := filepath.Join(t.TempDir(), "the-file")
old := parseTrusted(t, fmt.Sprintf(`{"declaration":1,"resources":[{"id":"m.old","type":"file","path":%q,`+
`"content":"port = 1\n"}]}`, path))
_, state, err := Apply(context.Background(), archHost(t), old, store.State{}, store.OriginDeclared, noServices,
nil, nil)
if err != nil {
t.Fatal(err)
}
was, _ := state.Find("m.old")
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`, "")
_, state, err = Apply(context.Background(), archHost(t), d, state, store.OriginDeclared, noServices, nil, nil)
if err != nil {
t.Fatal(err)
}
if got := readText(t, path); got != "port = 1\n" {
t.Errorf("the file handed to the client was rewritten or removed: %q", got)
}
if _, ok := state.Find("m.old"); ok {
t.Errorf("the record going was kept beside the filled file's")
}
if rec, ok := state.Find("m.conf"); !ok || rec.Wrote != was.Wrote || rec.Target != path {
t.Errorf("the filled file did not take the record of what the mesh wrote there: %+v", rec)
}
}
// elsewhere is a declaration with nothing at the filled path: one unrelated file.
func elsewhere(t *testing.T) *declaration.Declaration {
t.Helper()
return parseTrusted(t, fmt.Sprintf(`{"declaration":1,"resources":[{"id":"n.other","type":"file","path":%q,`+
`"content":"x\n"}]}`, filepath.Join(t.TempDir(), "other")))
}
+57
View File
@@ -0,0 +1,57 @@
//go:build linux
package apply
import (
"context"
"fmt"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
// A container that reads a filled file and mounts a library whose disk is not there is held by the write verb as a
// full apply holds it (novox/hq ADR 0328 §9, issue 668): never created or recreated over the missing disk.
func TestAWriteDoesNotRecreateAContainerHeldForItsLibrary(t *testing.T) {
// No folder is given the group: the library's root is absent, and nothing is made.
gid := 61000
libraries(t, nil, nil)
base := t.TempDir()
root := filepath.Join(base, "gone", "media")
env := filepath.Join(base, "radarr.env")
raw := fmt.Sprintf(`{"declaration":1,"provisioner_account":"mesh-node-provisioner","filled_files":[%q],
"groups":[{"name":"mesh-media","gid":%d}],"libraries":[%s],"resources":[
{"id":"radarr.env","type":"file","path":%q,"content":"","mode":"0600","owner-at-runtime":"owned"},
{"id":"radarr.app","type":"container","name":"radarr","image":%q,"env-file":[%q],"volumes":[%q]}]}`,
env, gid, libJSON("media", root, gid, true, false, "movies"), env, pinned, env,
filepath.Join(root, "movies")+":/movies")
d, err := declaration.Parse([]byte(raw))
if err != nil {
t.Fatal(err)
}
var ran []string
run := func(_ context.Context, name string, args ...string) (string, error) {
ran = append(ran, name+" "+strings.Join(args, " "))
return "", nil
}
var state store.State
for _, content := range []string{"PORT=1\n", "PORT=2\n"} {
var report Report
report, state, err = WriteFilled(context.Background(), archHost(t), d, state, env, []byte(content), run,
nil, nil, nil)
if err != nil {
t.Fatal(err)
}
if o := outcomeOf(report, "radarr.app"); o.Action != "waiting" {
t.Errorf("after writing %q the container is %+v, not held for its library", content, o)
}
}
for _, line := range ran {
if strings.HasPrefix(line, "docker run") || strings.HasPrefix(line, "docker rm") {
t.Errorf("the container was created or removed over a missing disk: %v", ran)
}
}
}
+168
View File
@@ -0,0 +1,168 @@
package apply
import (
"context"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-host/internal/store"
)
// A filled write is one file, not an apply: it asks no container runtime, and judges where nothing is placed
// (issue 339) with the roots the last apply read, falling back to the runtimes' configuration files when no apply
// has read them yet (novox/hq issue 1138). These tests set the package's runtimeRoots and AskRuntimes, so none
// runs in parallel with another that reads or sets them, and each restores what it changed.
// setRuntimeRoots makes runtimeDataRoots answer roots, as if an apply had just read them (or, with nil, as if none
// ever had), restoring what a test before it left there once this one is done.
func setRuntimeRoots(t *testing.T, roots []string) {
t.Helper()
runtimeRootsMu.Lock()
before := runtimeRoots
runtimeRoots = roots
runtimeRootsMu.Unlock()
t.Cleanup(func() {
runtimeRootsMu.Lock()
runtimeRoots = before
runtimeRootsMu.Unlock()
})
}
// askNoRuntime makes AskRuntimes record the name of every runtime it is asked for, answering as one that is not
// running does (nothing, no error): were it consulted, that is what would say so. Restored once the test is done.
func askNoRuntime(t *testing.T, asked *[]string) {
t.Helper()
before := AskRuntimes
AskRuntimes = func(_ context.Context, name string, _ ...string) (string, error) {
*asked = append(*asked, name)
return "", nil
}
t.Cleanup(func() { AskRuntimes = before })
}
// aRuntimeConfig points the runtimes' configuration at a fresh, unmounted root below a temp directory, restoring
// the engine's real configuration once the test is done.
func aRuntimeConfig(t *testing.T) (dir, root string) {
t.Helper()
dir = t.TempDir()
root = filepath.Join(dir, "docker")
daemon := filepath.Join(dir, "daemon.json")
if err := os.WriteFile(daemon, []byte(`{"data-root":"`+root+`"}`), 0o644); err != nil {
t.Fatal(err)
}
before := runtimeConfigs
runtimeConfigs = runtimeFiles{daemonJSON: daemon, storageConf: filepath.Join(dir, "none.conf")}
t.Cleanup(func() { runtimeConfigs = before })
return dir, root
}
// refusedAsPlacement fails the test unless err is a PlacementRefusedError.
func refusedAsPlacement(t *testing.T, err error) {
t.Helper()
var refusal *PlacementRefusedError
if err == nil || !errors.As(err, &refusal) {
t.Fatalf("a filled file below a refused runtime root was not refused: %v", err)
}
}
// 1138/AC1: a filled write asks no runtime, and judges with the roots the last apply read.
func TestAFilledWriteAsksNoRuntimeAndUsesTheLastApplysRoots(t *testing.T) {
root := filepath.Join(t.TempDir(), "docker-root")
setRuntimeRoots(t, []string{root}) // as if an apply had just read it
var asked []string
askNoRuntime(t, &asked)
path := filepath.Join(root, "app.conf")
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`, "")
_, _, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("x\n"), noServices,
nil, nil, nil)
refusedAsPlacement(t, err)
if len(asked) != 0 {
t.Errorf("a filled write asked the container runtimes: %v", asked)
}
}
// 1138/AC2: given a real apply that read a runtime data root, a filled write below it is refused as today, with
// no apply run again in between.
func TestAFilledWriteBelowARootARealApplyReadIsRefused(t *testing.T) {
dir, root := aRuntimeConfig(t)
t.Cleanup(func() {
runtimeRootsMu.Lock()
runtimeRoots = nil
runtimeRootsMu.Unlock()
})
// A full apply of something unrelated reads the runtimes' configuration once, at its start.
elsewhere := filepath.Join(dir, "elsewhere.conf")
if _, _, err := Apply(context.Background(), archHost(t), declare(t,
fmt.Sprintf(`{"id":"m.other","type":"file","path":%q,"content":"x\n"}`, elsewhere)),
store.State{}, store.OriginDeclared, noServices, nil, nil); err != nil {
t.Fatal(err)
}
// A filled write below the runtime's root, with no apply run since, is still refused.
path := filepath.Join(root, "sub", "app.conf")
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`, "")
_, _, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("x\n"), noServices,
nil, nil, nil)
refusedAsPlacement(t, err)
}
// 1138/AC3: with no apply having read the roots yet, a filled write judges by the runtimes' configuration files,
// asking no runtime.
func TestAFilledWriteWithNoApplyYetUsesTheRuntimesConfigFiles(t *testing.T) {
setRuntimeRoots(t, nil) // no apply has read the roots yet
_, root := aRuntimeConfig(t)
var asked []string
askNoRuntime(t, &asked)
path := filepath.Join(root, "sub", "app.conf")
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`, "")
_, _, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("x\n"), noServices,
nil, nil, nil)
refusedAsPlacement(t, err)
if len(asked) != 0 {
t.Errorf("a filled write with no apply yet asked the container runtimes: %v", asked)
}
}
// 1138/AC4: a full apply still asks each runtime once, at its start, whatever the number of files it writes.
func TestAFullApplyAsksEachRuntimeOnceRegardlessOfFileCount(t *testing.T) {
dir := t.TempDir()
var asked []string
before := AskRuntimes
AskRuntimes = func(_ context.Context, name string, _ ...string) (string, error) {
asked = append(asked, name)
return "", errors.New("not running in this test")
}
t.Cleanup(func() {
AskRuntimes = before
runtimeRootsMu.Lock()
runtimeRoots = nil
runtimeRootsMu.Unlock()
})
var resources []string
for i := 0; i < 3; i++ {
resources = append(resources, fmt.Sprintf(`{"id":"m.f%d","type":"file","path":%q,"content":"x\n"}`, i,
filepath.Join(dir, fmt.Sprintf("f%d.conf", i))))
}
if _, _, err := Apply(context.Background(), archHost(t), declare(t, strings.Join(resources, ",")),
store.State{}, store.OriginDeclared, noServices, nil, nil); err != nil {
t.Fatal(err)
}
if got := count(asked, "docker"); got != 1 {
t.Errorf("docker was asked %d times for one apply of 3 files; want 1: %v", got, asked)
}
if got := count(asked, "podman"); got != 1 {
t.Errorf("podman was asked %d times for one apply of 3 files; want 1: %v", got, asked)
}
}
+271
View File
@@ -0,0 +1,271 @@
package apply
import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
// Defends novox/hq ADR 0349 §4 (issue 668): the provisioner's client has the node-engine write a file it filled,
// and the node-engine writes it as an apply of the declared file with that content would — through applyFile,
// applyInto or applyBlock by who owns it at runtime — and makes what reads it take the change as an apply does.
// filledDecl is a declaration naming path as filled, with the file's fields after id, type and path, and more
// resources after it.
func filledDecl(t *testing.T, path, fields, more string) *declaration.Declaration {
t.Helper()
if more != "" {
more = "," + more
}
return parseTrusted(t, fmt.Sprintf(`{"declaration":1,"provisioner_account":"mesh-node-provisioner",
"filled_files":[%q],"resources":[{"id":"m.conf","type":"file","path":%q,%s}%s]}`, path, path, fields, more))
}
func TestAWriteOfAPathNotFilledIsRefused(t *testing.T) {
dir := t.TempDir()
filled := filepath.Join(dir, "filled.conf")
other := filepath.Join(dir, "other.conf")
d := parseTrusted(t, fmt.Sprintf(`{"declaration":1,"filled_files":[%q],"resources":[
{"id":"m.conf","type":"file","path":%q,"content":"x\n","owner-at-runtime":"owned"},
{"id":"m.other","type":"file","path":%q,"content":"y\n"}
]}`, filled, filled, other))
for _, path := range []string{other, filepath.Join(dir, "nothing.conf")} {
_, state, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("evil\n"),
noServices, nil, nil, nil)
if err == nil || !strings.Contains(err.Error(), "does not name") {
t.Errorf("%s: a write of a path the declaration does not fill was not refused: %v", path, err)
}
if _, err := os.Stat(path); err == nil && path != other {
t.Errorf("%s was written", path)
}
if len(state.Resources) != 0 {
t.Errorf("a refused write recorded something: %+v", state.Resources)
}
}
if _, err := os.Stat(other); err == nil {
t.Errorf("%s was written", other)
}
t.Run("a filled path two resources declare", func(t *testing.T) {
path := filepath.Join(t.TempDir(), "hosts")
if err := os.WriteFile(path, []byte(workstationHosts), 0o644); err != nil {
t.Fatal(err)
}
d := parseTrusted(t, fmt.Sprintf(`{"declaration":1,"filled_files":[%q],"resources":[
{"id":"a.hosts","type":"file","path":%q,"into":"block","content":"10.42.0.1 ace\n","owner-at-runtime":"managed"},
{"id":"b.hosts","type":"file","path":%q,"into":"block","content":"10.42.0.2 g14\n","owner-at-runtime":"managed"}
]}`, path, path, path))
_, state, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("6.6.6.6 evil\n"),
noServices, nil, nil, nil)
if err == nil || !strings.Contains(err.Error(), "2 resources declare") {
t.Fatalf("a path two resources declare was not refused: %v", err)
}
if got := readText(t, path); got != workstationHosts || len(state.Resources) != 0 {
t.Errorf("something was written or recorded: %q %+v", got, state.Resources)
}
})
}
// Each ownership is written by the write verb exactly as an apply of the declared file with the same content: the
// same action, the same bytes on disk, the same mode.
func TestAFilledFileIsWrittenAsItsOwnershipSays(t *testing.T) {
for _, c := range []struct {
name, fields, content, before, action string
}{
{"owned, absent", `"content":"","mode":"0640","owner-at-runtime":"owned"`, "port = 8080\n", "", "created"},
{"owned, over what is there", `"content":"","mode":"0640","owner-at-runtime":"owned"`, "port = 8080\n",
"port = 1\n", "updated"},
{"seed, absent", `"content":"","mode":"0600","owner-at-runtime":"seed"`, "user default on\n", "", "created"},
{"seed, present", `"content":"","mode":"0600","owner-at-runtime":"seed"`, "user default on\n",
"user default on\nuser app-one on\n", "kept"},
{"managed json", `"into":"json","content":"{}","owner-at-runtime":"managed"`,
`{"insecure-registries":["10.42.0.1:5000"]}`, machinesOwn, "updated"},
{"managed block", `"into":"block","content":"x\n","owner-at-runtime":"managed"`, "10.42.0.1 ace\n",
workstationHosts, "updated"},
} {
t.Run(c.name, func(t *testing.T) {
at := func() string {
path := filepath.Join(t.TempDir(), "the-file")
if c.before != "" {
if err := os.WriteFile(path, []byte(c.before), 0o644); err != nil {
t.Fatal(err)
}
}
return path
}
written := at()
report, state, err := WriteFilled(context.Background(), archHost(t), filledDecl(t, written, c.fields, ""),
store.State{}, written, []byte(c.content), noServices, nil, nil, nil)
if err != nil {
t.Fatal(err)
}
applied := at()
withContent := strings.Replace(c.fields, fmt.Sprintf(`"content":%q`, contentOf(t, c.fields)),
fmt.Sprintf(`"content":%q`, c.content), 1)
// The same file declared the old way: an apply writes a filled file only on the client's word (P7).
plain := parseTrusted(t, fmt.Sprintf(`{"declaration":1,"resources":[{"id":"m.conf","type":"file",`+
`"path":%q,%s}]}`, applied, withContent))
want, _, err := Apply(context.Background(), archHost(t), plain, store.State{}, store.OriginDeclared,
noServices, nil, nil)
if err != nil {
t.Fatal(err)
}
if got := report.Outcomes[0].Action; got != c.action || got != want.Outcomes[0].Action {
t.Errorf("written %s; an apply of the same content %s; want %s", got, want.Outcomes[0].Action, c.action)
}
if got, apply := readText(t, written), readText(t, applied); got != apply {
t.Errorf("written, the file holds\n%q\nan apply of the same content leaves\n%q", got, apply)
}
if a, b := modeAt(t, written), modeAt(t, applied); a != b {
t.Errorf("written mode %o, an apply's %o", a, b)
}
if rec, ok := state.Find("m.conf"); !ok || rec.Target != written || rec.Origin != store.OriginDeclared {
t.Errorf("the write is not recorded as the file's: %+v", rec)
}
})
}
t.Run("a filled file that carried a secret is not world-readable", func(t *testing.T) {
path := filepath.Join(t.TempDir(), "token.env")
d := filledDecl(t, path, `"content":"TOKEN=${secret:token}\n","secrets":{"token":"c2VhbGVk"},"owner-at-runtime":"owned"`, "")
if _, _, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("TOKEN=abc\n"),
noServices, nil, nil, nil); err != nil {
t.Fatal(err)
}
if m := modeAt(t, path); m != 0o600 {
t.Errorf("a filled file that carried a secret is mode %o", m)
}
})
}
func contentOf(t *testing.T, fields string) string {
t.Helper()
var holder struct {
Content string `json:"content"`
}
if err := json.Unmarshal([]byte("{"+fields+"}"), &holder); err != nil {
t.Fatal(err)
}
return holder.Content
}
func modeAt(t *testing.T, path string) os.FileMode {
t.Helper()
info, err := os.Stat(path)
if err != nil {
t.Fatal(err)
}
return info.Mode().Perm()
}
// A file that carried a secret keeps the mode a secret is written with: the client filled the value, and the file
// is no more readable for it.
func TestAWriteUnderReloadOnReloadsItsProgramOnce(t *testing.T) {
path := filepath.Join(t.TempDir(), "daemon.conf")
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`,
`{"id":"m.daemon","type":"service","unit":"m.service","state":"running","reload-on":["m.conf"]},
{"id":"m.other","type":"service","unit":"other.service","state":"running"}`)
var commands []string
var said []string
_, state, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("level = 1\n"),
recordingServices(&commands), func(line string) { said = append(said, line) }, nil, nil)
if err != nil {
t.Fatal(err)
}
if n := count(commands, "systemctl reload m.service"); n != 1 {
t.Errorf("the program was reloaded %d times; commands were %v", n, commands)
}
joined := strings.Join(commands, "\n")
if strings.Contains(joined, "restart m.service") || strings.Contains(joined, "stop m.service") {
t.Errorf("the program was restarted, not reloaded: %v", commands)
}
if strings.Contains(joined, "other.service") {
t.Errorf("a service that does not read the file was touched: %v", commands)
}
if _, ok := state.Find("m.daemon"); !ok {
t.Errorf("the reloaded service was not recorded")
}
// The log says the path and the digest, never the content.
for _, line := range said {
if strings.Contains(line, "level = 1") {
t.Errorf("the content was said: %q", line)
}
}
if !strings.Contains(strings.Join(said, "\n"), digestOf("level = 1\n")) {
t.Errorf("the digest was not said: %q", said)
}
// The same content again moves nothing, and reloads nothing.
commands = nil
report, _, err := WriteFilled(context.Background(), archHost(t), d, state, path, []byte("level = 1\n"),
recordingServices(&commands), nil, nil, nil)
if err != nil {
t.Fatal(err)
}
if report.Outcomes[0].Action != "unchanged" || count(commands, "reload") != 0 {
t.Errorf("an unchanged write reloaded: %v %+v", commands, report.Outcomes)
}
}
func TestAWriteUnderRestartOnRestartsItsProgram(t *testing.T) {
path := filepath.Join(t.TempDir(), "daemon.conf")
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`,
`{"id":"m.daemon","type":"service","unit":"m.service","state":"running","restart-on":["m.conf"]}`)
var commands []string
if _, _, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("level = 1\n"),
recordingServices(&commands), nil, nil, nil); err != nil {
t.Fatal(err)
}
// A restart is a stop and a start, once.
stops, starts := count(commands, "systemctl stop m.service"), count(commands, "systemctl start m.service")
if stops != 1 || starts != 1 {
t.Errorf("the program was stopped %d and started %d times; commands were %v", stops, starts, commands)
}
}
// A running container that reads the file at creation is recreated, as an apply recreates it for a changed file.
func TestAWriteOfAFileAContainerReadsRecreatesIt(t *testing.T) {
env := filepath.Join(t.TempDir(), "forge.env")
d := filledDecl(t, env, `"content":"","mode":"0600","owner-at-runtime":"owned"`,
`{"id":"forge.server","type":"container","name":"forge","image":"`+pinned+`","env-file":["`+env+`"]}`)
m := &machine{containers: map[string]*fakeContainer{}}
_, state, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, env, []byte("PORT=5432\n"),
m.run, nil, nil, nil)
if err != nil {
t.Fatal(err)
}
if !m.did("docker run") {
t.Fatalf("the container was not created: %v", m.asked)
}
m.asked = nil
report, _, err := WriteFilled(context.Background(), archHost(t), d, state, env, []byte("PORT=5433\n"), m.run,
nil, nil, nil)
if err != nil {
t.Fatal(err)
}
if !m.removed("forge") || !m.did("docker run") {
t.Fatalf("the container kept what it read before the write: %v", m.asked)
}
if o := outcomeOf(report, "forge.server"); o.Action != "updated" || !strings.Contains(o.Detail, env) {
t.Errorf("the recreate did not say which file changed: %+v", o)
}
}
func count(lines []string, prefix string) int {
n := 0
for _, l := range lines {
if strings.Contains(l, prefix) {
n++
}
}
return n
}
// Two resources declaring one filled path — two modules' blocks in one file — are refused: the request carries the
// content only, so it cannot say whose block it is, and nothing is written.
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"fmt"
"strings"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
+1 -1
View File
@@ -13,7 +13,7 @@ import (
"syscall"
"time"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
+25 -2
View File
@@ -9,7 +9,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
@@ -111,7 +111,10 @@ type fakeContainer struct {
// options. The host passes `--` before every unit (third review of mesh-catalog #147), which
// internal/system's own test holds; the fakes read the unit where they always did.
func oldShape(name string, args []string) []string {
if name != "systemctl" || len(args) < 2 || args[len(args)-2] != "--" {
if name != "systemctl" {
return asideShape(args)
}
if len(args) < 2 || args[len(args)-2] != "--" {
return args
}
v := 0
@@ -185,6 +188,13 @@ func (m *machine) run(_ context.Context, name string, args ...string) (string, e
return "", errors.New("no such image")
}
return "", nil
case "pull":
// A real runtime leaves the image present after a pull.
if m.images == nil {
m.images = map[string]string{}
}
m.images[args[len(args)-1]] = "2026-01-01T00:00:00Z"
return "", nil
case "network":
if len(args) > 1 && args[1] == "inspect" {
return strings.Join(m.members[args[len(args)-1]], ",") + ",\n", nil
@@ -1121,3 +1131,16 @@ func TestAPathTheMeshOnlySetAccessOnIsStillFound(t *testing.T) {
t.Errorf("it was re-moded to %o", info.Mode().Perm())
}
}
// asideShape reads the steps of a container set aside while its replacement starts (issue 1945) as the removal the
// tests were written against: setting it aside is its removal, and what is done to the set-aside container afterwards
// is no step of theirs.
func asideShape(args []string) []string {
if len(args) == 3 && args[0] == "rename" && args[2] == args[1]+asideSuffix {
return []string{"rm", "-f", args[1]}
}
if len(args) > 0 && strings.HasSuffix(args[len(args)-1], asideSuffix) && (args[0] == "rm" || args[0] == "stop") {
return []string{"aside"}
}
return args
}
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"os"
"path/filepath"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
func chmodUnder(home, path string, mode os.FileMode) error {
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"strings"
"time"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"slices"
"sort"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
+5 -2
View File
@@ -11,7 +11,9 @@ import (
// The read-only judgement of a node's directories (novox/hq issue 358): one line each — placed, would be refused
// naming the link and who could replace it, or not there yet (placed: an apply makes it) — and nothing made or
// changed.
// changed. Who could replace the link is "every account", "the account X" or "the group X", by the account and the
// directories the test runs as: the checker's isolated step runs as the account nobody, which owns / there (issue 1919,
// R4g), so the test demands a who in any of the forms couldReplace and linkReplacer produce, not "every account" alone.
func TestJudgeDirectoriesSaysWhatTheWalkWouldRefuseAndChangesNothing(t *testing.T) {
base := t.TempDir()
plain := filepath.Join(base, "plain")
@@ -39,7 +41,8 @@ func TestJudgeDirectoriesSaysWhatTheWalkWouldRefuseAndChangesNothing(t *testing.
t.Errorf("a directory the walk reaches is not said placed: %q", said)
}
if !strings.Contains(lines[2], "would be refused") || !strings.Contains(lines[2], filepath.Join(base, "linked")) ||
!strings.Contains(lines[2], "every account") {
!(strings.Contains(lines[2], "every account (") || strings.Contains(lines[2], "the account ") ||
strings.Contains(lines[2], "the group ")) {
t.Errorf("the linked directory is not said refused naming the link and who: %q", lines[2])
}
if _, err := os.Stat(filepath.Join(base, "missing")); !os.IsNotExist(err) {
+1 -1
View File
@@ -12,7 +12,7 @@ import (
"sync"
"time"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
+13 -7
View File
@@ -11,7 +11,7 @@ import (
"sync"
"time"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
@@ -318,6 +318,17 @@ func heldRoots(verdicts map[string]libraryVerdict, libs []declaration.Library) m
return out
}
// heldLibraries is the root of every library whose disk or mount is not there, with its name, judged now and read
// only: what a full apply holds its containers for (ADR 0328 §9), for one that judges without applying libraries.
func heldLibraries(ctx context.Context, libs []declaration.Library) map[string]string {
verdicts := map[string]libraryVerdict{}
for _, l := range libs {
rec, _ := libraryRecord(l.Library)
verdicts[l.Library] = assessLibrary(ctx, l, rec)
}
return heldRoots(verdicts, libs)
}
// mountsHeld is the library a container waits for: one whose root it mounts, or mounts below, is held.
func mountsHeld(c *declaration.Container, held map[string]string) string {
for _, v := range c.Volumes {
@@ -341,12 +352,7 @@ func WaitingOnLibrary(d *declaration.Declaration) map[string]bool {
if d == nil || len(d.Libraries) == 0 {
return out
}
verdicts := map[string]libraryVerdict{}
for _, l := range d.Libraries {
rec, _ := libraryRecord(l.Library)
verdicts[l.Library] = assessLibrary(context.Background(), l, rec)
}
held := heldRoots(verdicts, d.Libraries)
held := heldLibraries(context.Background(), d.Libraries)
for _, r := range d.Resources {
if c, ok := r.(*declaration.Container); ok && mountsHeld(c, held) != "" {
out[c.ID] = true
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"fmt"
"time"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
+27 -1
View File
@@ -11,18 +11,36 @@ import (
"path/filepath"
"strconv"
"strings"
"syscall"
"testing"
"time"
"golang.org/x/sys/unix"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
// The engine's half of ADR 0328's checks (novox/hq ADR 0328, "How it is checked", issue 567). The engine runs as
// root; a test may not, and a non-root test gives its folders a group it is in itself.
// canSetGroups skips aloud a test that gives a folder a group when this step cannot. **A step that cannot set a folder's
// group judges nothing here** (issue 1919, R4g): the checker runs the tests in the build module's isolation, a user
// namespace that maps no group but the step's own, so a chown to any other group is refused (EINVAL or EPERM). A test that
// sets a group calls it first: the probe gives a file of the test's own that group, and when it is refused the test skips
// aloud; outside such an isolation it runs and is judged.
func canSetGroups(t *testing.T, gid int) {
t.Helper()
probe := filepath.Join(t.TempDir(), "probe")
if err := os.WriteFile(probe, nil, 0o600); err != nil {
t.Fatal(err)
}
if err := os.Chown(probe, -1, gid); errors.Is(err, syscall.EINVAL) || errors.Is(err, syscall.EPERM) {
t.Skipf("NOT JUDGED: this step cannot set a folder's group (a user namespace maps only its own account, as the "+
"checker's isolated step does): %v; issue 1919", err)
}
}
// testGID is a group the test may give its folders: any number as root, else one of its own added groups that is
// no root group's.
func testGID(t *testing.T) int {
@@ -121,6 +139,7 @@ func statOf(t *testing.T, path string) unix.Stat_t {
// those folders alone, never what is in them; owner, owner's and others' bits unchanged.
func TestALibraryIsKeptInItsGroup(t *testing.T) {
gid := testGID(t)
canSetGroups(t, gid)
libraries(t, nil, nil)
root := filepath.Join(t.TempDir(), "media")
if err := os.Mkdir(root, 0o755); err != nil {
@@ -170,6 +189,7 @@ func TestALibraryIsKeptInItsGroup(t *testing.T) {
// Row 9: a missing sub-path is made with the root's owner, the library's group, group rwx and setgid.
func TestAMissingSubPathIsMadeAsTheRootIs(t *testing.T) {
gid := testGID(t)
canSetGroups(t, gid)
libraries(t, nil, nil)
root := filepath.Join(t.TempDir(), "media")
if err := os.Mkdir(root, 0o751); err != nil {
@@ -226,6 +246,7 @@ func TestARootThatHoldsDataWaitsForTheOperatorsWord(t *testing.T) {
// (the act's ask, internal/link); an undeclared library refused; a walk over its bound goes on from its cursor.
func TestTheTakeActSetsTheGroupUnderTheRoot(t *testing.T) {
gid := testGID(t)
canSetGroups(t, gid)
libraries(t, nil, nil)
base := t.TempDir()
root := filepath.Join(base, "media")
@@ -512,6 +533,7 @@ func TestNoLibraryGroupIsRoot(t *testing.T) {
// A library's filesystem that cannot be named this time is unknown, never a move; another filesystem at the root is.
func TestALibraryNotNamedIsUnknownAndAnotherFilesystemIsAMove(t *testing.T) {
gid := testGID(t)
canSetGroups(t, gid)
base := t.TempDir()
pool := filepath.Join(base, "pool")
root := filepath.Join(pool, "media")
@@ -650,6 +672,7 @@ func TestALibraryIsNeverTheMachinesOwnTree(t *testing.T) {
// and sample all in shape are kept; once taken, never asked again on that filesystem.
func TestARootInShapeWithDataBelowNotInTheGroupWaits(t *testing.T) {
gid := testGID(t)
canSetGroups(t, gid)
libraries(t, nil, nil)
root := filepath.Join(t.TempDir(), "media")
movies := filepath.Join(root, "movies")
@@ -699,6 +722,7 @@ func TestARootInShapeWithDataBelowNotInTheGroupWaits(t *testing.T) {
// file never keeps its setgid bit once regrouped.
func TestRootsLinkedFilesAreLeftAndSetgidIsNeverKept(t *testing.T) {
gid := testGID(t)
canSetGroups(t, gid)
f, err := os.CreateTemp(t.TempDir(), "file")
if err != nil {
t.Fatal(err)
@@ -729,6 +753,7 @@ func TestRootsLinkedFilesAreLeftAndSetgidIsNeverKept(t *testing.T) {
// waits for the operator's word as the root would, and nothing of it is changed; an empty one is kept.
func TestASubPathHoldingOldDataWaitsForTheOperatorsWord(t *testing.T) {
gid := testGID(t)
canSetGroups(t, gid)
libraries(t, nil, nil)
root := filepath.Join(t.TempDir(), "media")
if err := os.Mkdir(root, 0o755); err != nil {
@@ -795,6 +820,7 @@ func TestAnApplyDoesNotWaitForATakeWalk(t *testing.T) {
// the root alone, says whether the data is the library's already (§7).
func TestTheSampleBeneathARootInShapeIsJudged(t *testing.T) {
gid := testGID(t)
canSetGroups(t, gid)
libraries(t, nil, nil)
root := filepath.Join(t.TempDir(), "media")
if err := os.MkdirAll(filepath.Join(root, "old"), 0o755); err != nil {
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
+2 -1
View File
@@ -11,7 +11,7 @@ import (
"testing"
"time"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
@@ -270,6 +270,7 @@ func newRegistryMachine(spec string) *aRegistryMachine {
}
func (m *aRegistryMachine) run(_ context.Context, _ string, args ...string) (string, error) {
args = asideShape(args)
m.mu.Lock()
switch args[0] {
case "info", "image":
+8 -2
View File
@@ -30,7 +30,7 @@ import (
"strconv"
"strings"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// LinkInPathError is a mesh directory refused because a link in its path is one another account could replace.
@@ -94,13 +94,14 @@ func groupNamed(gid uint32) string {
func JudgeDirectories(d *declaration.Declaration) string {
var lines []string
var placed, refused, guarded, homed int
declared := declaredHomes(d.Resources)
for _, r := range d.Resources {
dir, ok := r.(*declaration.Directory)
if !ok {
continue
}
path := filepath.Clean(dir.Path)
if err := refusePath(path, placingDirectory, moduleOfID(dir.ID), dir.Owner); err != nil {
if err := refusePathWith(path, placingDirectory, moduleOfID(dir.ID), dir.Owner, declared); err != nil {
guarded++
lines = append(lines, path+": refused already by the placement guard: "+err.Error())
continue
@@ -123,3 +124,8 @@ func JudgeDirectories(d *declaration.Declaration) string {
placed+refused+guarded+homed, placed, refused, guarded, homed)
return strings.Join(append([]string{head}, lines...), "\n")
}
// meshDataRoot is the mesh's own tree, where a module placed "mesh" keeps its directory (<data root>/mesh/<module>):
// the engine's, 0755, so only the engine's account makes a directory in it (novox/hq issue 1873). A variable so a
// test can place it.
var meshDataRoot = "/var/lib/mesh"
+1 -1
View File
@@ -14,7 +14,7 @@ import (
"golang.org/x/sys/unix"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// meshDirsByDescriptor says this system can open a directory component by component without following a link.
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)

Some files were not shown because too many files have changed in this diff Show More