Compare commits

...
Author SHA1 Message Date
jschoubben a5d0e64200 Checker seat test: fold into the closed-set seats test, no new test function (issue 774) 2026-10-27 23:00:38 +01:00
jschoubben 9fdd33b03e Checker seat: the try's end by hand is a verb too (issue 774) 2026-10-27 22:49:44 +01:00
jschoubben 2e8c2a7f4c Seat mesh-delivery-checker: the try and its read are verbs of the checker's own seat, replacing sh merge-check.sh (ADR 0547 §10, issue 774) 2026-10-27 22:49:06 +01:00
mesh-tools d48365aed0 Merge pull request 'frombus: a node composing itself takes its modules from their builds and reads what providers elsewhere serve (issue 1126, group fix/1126-node-creds)' (#109) from fix/1126-node-creds into main 2026-10-27 20:32:30 +00:00
jschoubben 7a8e9f9706 Wait for a done key whose artifact names no reference, as the controller's take-in does, and hold the fold guard with a row (issue 1126, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-sdk@d48365aed010
mesh/delivery-group group fix/1126-node-creds merged: every member is merged
mesh/check checking 7a8e9f970648 on d48365aed010 | novox/mesh-sdk#109 in group fix/1126-node-creds
2026-10-27 21:12:13 +01:00
jschoubben d7a758894a Leave a module the node runs inside its own composition to its manifest, as the fold composes it, so taking builds in moves no folded module (issue 1126) 2026-10-27 21:11:20 +01:00
jschoubben 04de3e5da2 Compose a node's own modules from their builds and read what providers elsewhere serve, so a self-composing node hands moved modules and settles served settings as the controller does (issue 1126)
frombus composed every module from the indexer's raw manifest: no moved module
was handed, so the node stood in for own secrets the vault leaves to the
provisioner (the seed's refusals on all four nodes), and a provider elsewhere
had no config values, so mailu's served domain refused shanks.

- takein: the controller's take-in of a done key (artifacts, launchers, the
  templates archive's texts, Resolve) as one SDK rule; catalogue.Launchers holds
  the launcher rule buildstep now delegates to.
- frombus: each module placed on the composing node, and each kept at another
  build (Scope.KeptAt, a fingerprint), composes from its built manifest; another
  node's modules give the keys their served values fill (Reader.Served).
- buscompose: Declared.Served and Principal.ServedReads grant every node the read
  of exactly those keys of mesh-config_resolved.
- compose.ArtifactStoreAddress: the store as the node's composition routes to it.
2026-10-27 21:11:20 +01:00
mesh-tools 9f92233a28 Merge pull request 'process: a module's own bus connection, subjects and durable consumer in one helper (issue 1242)' (#108) from fix/1242-bus-helper into main 2026-10-27 19:49:19 +00:00
jschoubben 8ad9372b19 Name another module's own-stream subject in the SDK too (issue 1242)
mesh/delivery merged: source.main moved to novox/mesh-sdk@9f92233a2890
mesh/delivery-group group fix/1242-bus-helper merged: every member is merged
mesh/check checking 8ad9372b1952 on a262c7dbe0ff | novox/mesh-sdk#108 in group fix/1242-bus-helper
2026-10-27 20:38:20 +01:00
jschoubben 3236cbad43 Give a module's own credential its dial, subjects and durable consumer in mesh-sdk, so a part no longer hand-writes them (issue 1242) 2026-10-27 20:36:14 +01:00
mesh-tools a262c7dbe0 Merge pull request 'A node folds a placed module with no new-form build into its own composition, as the controller does, so the tunnel and the bus are not dropped when nodes compose themselves (issue 1126)' (#107) from fix/1126-fold into main 2026-10-27 08:09:03 +00:00
jschoubben 05b6783103 Fold a module no new-form build names into the node's own composition, as the controller does, so a self-composing node keeps the tunnel and the bus (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-sdk@a262c7dbe0ff
mesh/delivery-group group fix/1126-fold merged: every member is merged
mesh/check checking 05b678310366 on 56de5ba84010 | novox/mesh-sdk#107 in group fix/1126-fold
mesh-wireguard (built by nobody) and every module a node runs inside its own
composition (Node.InOwn, from frombus.InOwn of what it runs) were cut out as
skipped: every node's shadow answered differs to the placement seed, and a node
composing itself after it would drop their resources. compositions now folds
them in place, module taken off, filled paths with them, as mesh-controller's
compositionsOf does; a want naming a new-form build still moves one out.
2026-10-27 08:50:26 +01:00
mesh-tools 56de5ba840 Merge pull request 'A placement change is answered by the nodes it places: the candidate contract, composing a given placement, and replicated seats in the placement check (issue 1126, ADR 0559)' (#106) from feat/1126-l2b into main 2026-10-27 07:21:03 +00:00
mesh-tools 43eef46c41 Merge pull request 'A build step names the process that runs it, so a restarted build module clears what a gone process left running (issue 1890)' (#105) from fix/1890-step-owner into main 2026-10-27 07:19:36 +00:00
jschoubben cadd7b1b2f Ask the vault in a candidate answer only for credentials of modules the node runs, and refuse a stand-in of a running module, so a placement that adds a module is answered as added (issue 1126, ADR 0559 §4)
mesh/delivery merged: source.main moved to novox/mesh-sdk@56de5ba84010
mesh/delivery-group group feat/1126-l2b merged: every member is merged
mesh/check being carried to cadd7b1b2f8f on 43eef46c4121 | from b0cbc4e6aca8, mesh-catalog 64530a8dcb08 | novox/mesh-sdk#106 in group feat/1126-l2b
2026-10-27 02:43:10 +01:00
jschoubben 062c29c3a2 Judge a node's own composition in its candidate answer only where it runs one, as its agent applies it only on a want (issue 1126) 2026-10-27 02:27:04 +01:00
jschoubben a37d2622ed Name the words of a composition failed only for an answer not heard, and what an answer's revision is, so the assigner and the checker judge a late answer alike (issue 1126) 2026-10-27 02:22:59 +01:00
jschoubben ba398e2460 Answer a placement candidate as ADR 0559 has it: added and removed modules, per-module refusals, both digests of a difference, and a node's read of every candidate key (issue 1126) 2026-10-27 02:19:27 +01:00
jschoubben 82f10c3bc9 Let frombus and the composer compose with a given placement and compare a node's compositions with what it runs, so a placement candidate can be answered and checked (issue 1126) 2026-10-27 02:12:11 +01:00
jschoubben 7e99c354ac Give a catalogue pull request's placement a candidate state and each node an answer in its own have, so a placement change is checked by the nodes it places before it merges (issue 1126) 2026-10-27 02:07:47 +01:00
jschoubben 8e81d87610 Let a replicated mesh seat be held on several nodes in CheckPlacements, so the seed of today's placement is not refused (issue 1126) 2026-10-27 02:04:01 +01:00
jochen db1007ffdd Delete a gone process's ask key instead of ending it, so its redelivery still answers the asker (issue 1890)
mesh/delivery merged: source.main moved to novox/mesh-sdk@43eef46c4121
mesh/delivery-group group fix/1890-step-owner merged: every member is merged
mesh/check pass db1007ffdda1 on b0cbc4e6aca8 | unit 6014 run, 233 kept, 16 skipped | integration 6067 run, 265 kept, 24 skipped | over bound: unit, in…
ADR 0525 §6 has the stream redeliver an ask whose node stopped, and the
asker takes the first end it reads: ending the key at a restart failed
checks a redelivery would still run. The sweep now deletes such a key,
guarded by the revision read, so the asker keeps waiting and the stale
key leaves the counts of what runs; a hand-started step of a gone owner
is still ended at once. A missing boot id is said once at start.
2026-10-27 01:29:48 +01:00
jochen 981a35b598 End a step left by a previous process at once: every step names its owner (issue 1890)
A delivery that replaced mesh-build-go left a check ask's key running, and
the new process's start sweep spared it as an ask younger than the step
bound, so it held a build slot until a hand end (issue 1890, note 21).
Each step now records its owner (boot id, pid, process start); the sweep
ends a running step of another owner at once and says its owner, while a
key with no owner keeps the earlier rule. The conformance kit gains the
left-running case every build module runs through its own start sweep.
2026-10-27 01:19:29 +01:00
mesh-tools b0cbc4e6ac Merge pull request 'A build step's failure contract: go -json and TAP parsers and a conformance kit every build module runs, so a verdict names the failing test (issue 1901)' (#104) from fix/1901-failure-contract into main 2026-10-26 23:44:13 +00:00
jschoubben a2759ec92e Merge main into fix/1901-failure-contract: the capacity tests stay one function, so the SDK's test count does not grow (issue 1901)
mesh/delivery merged: source.main moved to novox/mesh-sdk@b0cbc4e6aca8
mesh/delivery-group group fix/1901-failure-contract merged: every member is merged
mesh/check checking a2759ec92ec9 on f178323b5790 | novox/mesh-sdk#104 in group fix/1901-failure-contract
2026-10-27 00:39:10 +01:00
mesh-tools f178323b57 Merge pull request 'The mesh-vault seat promises own, the verb L2 gave the vault (issue 1126)' (#103) from fix/1126-vault-seat-own into main 2026-10-26 22:00:44 +00:00
mesh-tools 684acb922d Merge pull request 'Delivery records are true at the source: the layouts for what a repository's merges reached (issue 1280)' (#102) from fix/delivery-records-true into main 2026-10-26 21:48:43 +00:00
jschoubben e78e881ecb The vault's seat promises own, which L2's vault serves and its claim names, so the controller's catalogue check passes (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-sdk@f178323b5790
mesh/delivery-group group fix/1126-vault-seat-own merged: every member is merged
mesh/check failed: unit: github.com/novox/mesh-controller/cmd/mesh-controller.TestModuleCheckPassesTheCatalogue: the catalogue does not pass its own c…
mesh-catalog's mesh-vault claims its seat serving own (a machine's node-engine asks its own machine's secrets and
pair credentials sealed to its stated key), and buscompose already grants that call to node principals
(AsksItsOwnSecrets, VaultOwn); only the seat's verb list lacked it, so a claim naming it was refused.
2026-10-26 22:38:58 +01:00
jschoubben 68d65e6bbf Merge remote-tracking branch 'origin/main' into fix/delivery-records-true
mesh/delivery merged: source.main moved to novox/mesh-sdk@684acb922d99
mesh/delivery-group group fix/delivery-records-true merged: every member is merged
mesh/check checking 68d65e6bbf61 on 823d52051f03 | novox/mesh-sdk#102 in group fix/delivery-records-true
2026-10-26 22:19:13 +01:00
mesh-tools 823d52051f Merge pull request 'Nodes compose: placement, node records and one composer input for three callers (issue 1126)' (#98) from feat/1126-nodes-compose into main 2026-10-26 21:11:05 +00:00
jschoubben b4310dfb8e Merge remote-tracking branch 'origin/main' into fix/delivery-records-true 2026-10-26 22:04:42 +01:00
jschoubben 9796444201 Merge remote-tracking branch 'origin/main' into feat/1126-nodes-compose
mesh/delivery merged: source.main moved to novox/mesh-sdk@823d52051f03
mesh/delivery-group group feat/1126-nodes-compose merged: every member is merged
mesh/check checking 979644420117 on 7187beaf042b | novox/mesh-sdk#98 in group feat/1126-nodes-compose
# Conflicts:
#	test/standing.test.ts
2026-10-26 22:03:26 +01:00
mesh-tools 7187beaf04 Merge pull request 'The provider loop's tests wait on what they mean, not on the clock (issue 1901 note 27)' (#101) from fix/standing-test-races into main 2026-10-26 20:58:44 +00:00
jschoubben 5f45175ed0 Merge remote-tracking branch 'origin/main' into feat/1126-nodes-compose 2026-10-26 21:56:47 +01:00
jschoubben 18e541c1ba The standing test allows 10 ms between the loop's clock and the test's stamp of an announcement
mesh/delivery merged: source.main moved to novox/mesh-sdk@7187beaf042b
mesh/check pass 18e541c1badc on 7187beaf042b | carried from 2cbe21937153 | a compiled program is not refused: mesh-build-node reported no programs | u…
2026-10-26 21:55:46 +01:00
jschoubben cf930388f9 The standing test's runs last 0.6 of failingAfterMs of clock as well as two attempts, so a success that does not reset a run is caught; the addition test holds a thousand stable passes, so 'first pass' is proved (review of the standing test races) 2026-10-26 21:49:03 +01:00
jschoubben 36ae0c691b The standing and retirement tests wait on what they mean, not on a slept fraction of a bound: announcements are timed from the clock at the moment they are said, short runs are counted in attempts, an addition is proved by a hold it must not wait for (the second timing race of one kind in a day; issue 1126) 2026-10-26 21:44:06 +01:00
mesh-tools 2cbe219371 Merge pull request 'A build seat's capacity follows its CPU share and its build hours; asks say their phases (issues 1890, 1806)' (#100) from fix/capacity-follows-cpu-share into main 2026-10-26 20:33:28 +00:00
jschoubben 3a095a66be The standing test keeps c failing alone after its ask is dropped, so its only recovery is its retirement: clearing every failure at once raced the loop's read of the new list under load (issue 1126; the hidden failure of try-1793042418286545971, issue 1901) 2026-10-26 21:33:20 +01:00
jschoubben 805b4d9a35 Merge main into fix/capacity-follows-cpu-share: the build modules' binary refusal (sdk #99) beside the capacity work
mesh/delivery merged: source.main moved to novox/mesh-sdk@2cbe21937153
mesh/delivery-group group fix/capacity-follows-cpu-share merged: every member is merged
mesh/check checking 805b4d9a3536 on dae8e288d2e1 | novox/mesh-sdk#100 in group fix/capacity-follows-cpu-share
2026-10-26 21:26:58 +01:00
jschoubben d00dac140b A build module takes asks only within its node's build hours
mesh/delivery-group group fix/capacity-follows-cpu-share rejected: composed together: the checker's verdict at the members' heads (63694fe9222d): unit: ..test
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: unit: ..test: ✔ a transient empty list for four passes retires nothing; five stable passes retire, data kept (418.253369ms) | d00da…
The operator, 2026-10-26 21:18: g14 builds only from 18:00 to 06:00. A build-hours setting names a
window per node in its local time (crossing midnight allowed, absent = any hour); outside it the
capacity judged by is 0, running steps finish, and the puller wakes on one timer at the next edge.
The change is said in the log and in capacity.<node> with the window and when it opens.
2026-10-26 21:21:37 +01:00
jschoubben e44cbaf57f A module with no floor holds no head back; a merge among its changes reads not yet on the machines (review of issue 1280, ADR 0556 clause 2) 2026-10-26 21:12:48 +01:00
jschoubben 1f70c4c0aa A settle ask's group begins with a colon, which no branch name holds, so a branch named settle/... still merges on its Approve (review of issue 1280) 2026-10-26 21:12:31 +01:00
jschoubben 5ecda46124 A settle ask's group and the act its options bind, shared by the gate and the teller (ADR 0556 clause 5) 2026-10-26 21:12:31 +01:00
jschoubben aa8e9afc7c A module's floor counts every running have, healthy or not; health stays in its words (ADR 0556 clause 2, insight) 2026-10-26 21:12:31 +01:00
jschoubben 54a8afac73 The view's reach carries the reached head and each change's source revision; nothing records a merge as delivered (ADR 0556 v8) 2026-10-26 21:12:31 +01:00
jschoubben 43de3c937d Layouts of the indexer's taken and the view's reach, and the delivery seat's settle verb, so whether a merge is on the machines can be derived (issue 1280, ADR 0556) 2026-10-26 21:12:31 +01:00
jschoubben 03dce12378 The standing test keeps c failing alone after its ask is dropped, so its only recovery is its retirement: clearing every failure at once raced the loop's read of the new list under load (issue 1126; the hidden failure of try-1793042418286545971, issue 1901)
mesh/delivery-group group feat/1126-nodes-compose rejected: composed together: the checker's verdict at the members' heads (0d06ff386dce): ace
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: ace: novox/mesh-sdk#98 does not merge into the trunk: files in conflict: test/standing.test.ts | 03dce1237803 on 7187beaf042b | uni…
2026-10-26 20:33:23 +01:00
jschoubben dd9d9d5b5f Compare haves with SameHave in the round-trip tests, now that a have carries a list (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking dd9d9d5b5f01 on dae8e288d2e1 | novox/mesh-sdk#98 in group feat/1126-nodes-compose
2026-10-26 20:20:12 +01:00
jschoubben 2c0ab3a592 A have carries the values a self-composing node's running composition holds sealed to its key, so its composer reuses them after a restart (review of L2, round 2; issue 1126, ADR 0547 §3)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed (4 tests): unit: go.go vet: # git.novox.be/novox/mesh-sdk/go/delivery [git.novox.be/novox/mesh-sdk/go/delivery.test] | 2c0ab3a59288 …
2026-10-26 20:19:54 +01:00
jschoubben 47a66e3c0d Take a crashed file's cause only from what the run wrote after the last test, give a subtest's panic to that subtest, and keep the SDK's test count (issue 1901, review round 1) 2026-10-26 20:17:42 +01:00
jschoubben 3968849b1b Review of L2: the export is a placement file at its version with its modules (S1); a node asks the vault only when a module placed on it declares an own secret, so one with none composes whether or not the vault answers (H8's failure) (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 3968849b1b36 on a7a1fbd34e1d | novox/mesh-sdk#98 in group feat/1126-nodes-compose
2026-10-26 20:08:32 +01:00
jschoubben 02f50bd9d6 A build seat's capacity follows its node's build CPU, and a running ask says its phases
Since the build CPU cap (issue 1806) shanks' mesh-build-go kept capacity 3 inside a 12-core slice:
three full checks got about 4 cores each and ran past the 30-minute step bound (issue 1890).
The puller now judges by min(configured, floor(slice quota cores / cores-per-step)), the quota
read live from its cgroup, says it on change and writes capacity.<node> to the pauses state.
A run's phases (fetch and merge, digest, tests) are logged with the ask id and written to its
running step key; Asker.Wait hands every step value to a WithStepSeen hook.
2026-10-26 19:52:54 +01:00
jschoubben 81ec7bd7de Drop the failure contract's unused version constant: a result names its build module with By (issue 1901) 2026-10-26 19:50:17 +01:00
jschoubben e40dd0f56a One failure contract for a build step's failed test, its parsers and a conformance kit every build module runs (issue 1901)
Each build module parsed its own test runner's output, so a fix in one never reached the others: a failed node --test
run was worded by a passing line of its first file. buildstep.Failure names the test, its file, its cause and its own
output; ReadGoTest (moved from mesh-build-go) and ReadTAP fill it; package conformance holds every build module to it
with runner output captured with the build nodes' own toolchains.
2026-10-26 19:45:35 +01:00
jschoubben 9713c3023a Placement carries a settings-form module's config values, which mesh-config does not hold, and the export writes them (issue 1126, ADR 0555 §1): the media modules on one node would compose without them
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: ace: novox/mesh-catalog#505 does not merge into the trunk: files in conflict: .gitignore | 9713c3023a18 on a7a1fbd34e1d | unit 1245…
2026-10-26 19:38:37 +01:00
jschoubben 53c2def1a5 VaultSecretChanged is a variable (its seat is), and gofmt (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: unit: git.novox.be/novox/mesh-catalog/checks/nobinaries.TestTheCatalogueHoldsNoCompiledProgram: modules/mesh-config/cmd/mesh-config…
2026-10-26 19:25:19 +01:00
jschoubben f55b34febc A node composing its own hears the vault's secret-changed, names and generations only (issue 1126, ADR 0547 §2)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed (6 tests): unit: go.gofmt: not as gofmt writes them (run gofmt -w): go/buscompose/buscompose_test.go | f55b34febc2d on a7a1fbd34e1d …
2026-10-26 19:25:09 +01:00
jschoubben 944a35d200 A node's record carries the mesh's own certificate as the machine holds it, and frombus composes with the composing node's (issue 1126, ADR 0555 §3)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: shanks: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk 91413de33516, which is neither on mesh-sdk's trunk nor the g…
2026-10-26 19:23:06 +01:00
jschoubben 7d81e8326f frombus gives the composing node's core credentials as the vault's boxes sealed to it, from the same own secrets (issue 1126, ADR 0349 §16, ADR 0550 §5) 2026-10-26 19:22:58 +01:00
jschoubben 91413de335 compose names the pair credentials a node's composition reads as consumer and provider, and frombus asks them sealed before it composes (issue 1126, ADR 0550 §5)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: g14: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk f1d7c7377dc4, which is neither on mesh-sdk's trunk nor the grou…
2026-10-26 19:18:46 +01:00
jschoubben 6447a1c6d4 placement-seed writes each node's first placement file from the controller's export, checked before anything is written, once (issue 1126, ADR 0549 §4, ADR 0555 §6)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: shanks: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk f1d7c7377dc4, which is neither on mesh-sdk's trunk nor the g…
2026-10-26 19:17:45 +01:00
jschoubben 4d5f430224 A want the node composes itself says so (Self) and names the build alone; every other want names its composition as before, so no caller of the old form changes (issue 1126, ADR 0547 §2)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: ace: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk f1d7c7377dc4, which is neither on mesh-sdk's trunk nor the grou…
2026-10-26 19:13:56 +01:00
jschoubben dfe3ba9475 frombus exports a node's placement from a composition input, Input's inverse for what a person decides, and the controller's seat declares placement-export (issue 1126, ADR 0555 §6)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking dfe3ba94750c on a7a1fbd34e1d | novox/mesh-sdk#98 in group feat/1126-nodes-compose
2026-10-26 19:11:34 +01:00
jschoubben f1d7c7377d frombus reads the composing node's own secrets sealed to it, and node principals may ask the vault's own in their own name (issue 1126, ADR 0550 §5)
mesh/delivery-group group feat/1126-nodes-compose checking: 4 of 5 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking f1d7c7377dc4 on a7a1fbd34e1d | novox/mesh-sdk#98 in group feat/1126-nodes-compose
2026-10-26 19:03:00 +01:00
jschoubben b72de2b75a A node applying what it composed refuses a composition that stands in for anything its input lacks, the bus's user list alone held out (issue 1126, ADR 0547 §2) 2026-10-26 18:59:05 +01:00
jschoubben 6df0dfe208 Node principals read what they compose their own modules from; the composer copy is held per user to the controller's grants plus exactly these (issue 1126, ADR 0547 §2, ADR 0549 §3)
mesh/delivery-group group feat/1126-nodes-compose rejected: composed together: the checker's verdict at the members' heads (922da6d6ae76): forge
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 6df0dfe208c6 on a7a1fbd34e1d | novox/mesh-sdk#98 in group feat/1126-nodes-compose
2026-10-26 18:44:06 +01:00
jschoubben 9532b78724 A node's own want names no composition from the second milestone: the node composes its own (issue 1126, ADR 0547 §2) 2026-10-26 18:41:33 +01:00
jschoubben e49309d61c A want names the build alone from the second milestone, and the yes a person gave; a have waits for that yes with its judgement (issue 1126, ADR 0547 §2, §3) 2026-10-26 18:39:46 +01:00
jschoubben 9789c8379a Node principals read every node's own facts, where its record rides; the composer copy is checked to grant all the controller's grants and only that and the issue 1790 rows more (issue 1126, ADR 0555 §3)
mesh/delivery-group group feat/1126-nodes-compose checking: 1 of 3 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 9789c8379a74 on a7a1fbd34e1d | novox/mesh-sdk#98 in group feat/1126-nodes-compose
2026-10-26 18:33:36 +01:00
jschoubben 2b0689f654 A node's record rides in its own have key, <node>.node: no bucket, writer or grant of its own, and its key's change is what readers hear (issue 1126, ADR 0555 §3) 2026-10-26 18:29:45 +01:00
jschoubben 5b1e990eb4 The tunnel found at adoption and the names of the hub's peers are kept in the placement; whether the hub took it over and who enrolled under a peer's key are read from placement and records, as the controller reads them (issue 1126, ADR 0555) 2026-10-26 18:26:11 +01:00
jschoubben a431746c0e What a person decides of a node lives in its placement file and what the machine observes in its record; two nodes on one address or two hubs are refused (issue 1126, K2) 2026-10-26 18:17:59 +01:00
jschoubben 5db53e738f Name the catalogue repository whose placement files are read (issue 1126, ADR 0549 §1) 2026-10-26 18:10:25 +01:00
jschoubben 9f7f50de8d A merged message of the catalogue carries its placement files, so the indexer reads placement at the merge without asking the forge again (issue 1126, ADR 0549 §2) 2026-10-26 18:08:59 +01:00
jschoubben bdd4c7b875 The judgement of a move is a pure function in mesh-sdk, so a node agent judges what it composed against what it runs (issue 1126, ADR 0547 §3, ADR 0377 §3) 2026-10-26 18:07:20 +01:00
jschoubben e982072443 frombus fills the composition's input from placement, node records, the indexer's manifests and mesh-config's slice, so one composer serves the node agent, the check and the grants part (issue 1126, ADR 0547 §2, ADR 0549 §3) 2026-10-26 18:06:04 +01:00
jschoubben 146cd858a3 Placement and the node record have one layout in mesh-sdk, so the indexer, the node agents and the composition check read them alike (issue 1126, ADR 0549 §1-§2, ADR 0547 §2) 2026-10-26 18:02:13 +01:00
107 changed files with 8508 additions and 190 deletions
+15
View File
@@ -70,6 +70,11 @@ func (s AskSteps) fetch(ctx context.Context, args map[string]any) (json.RawMessa
if s.Fetch == nil {
return nil, fetched{}, Refuse("this build module fetches nothing")
}
if _, merges := args["base"]; merges {
Phase(ctx, "fetch and merge")
} else {
Phase(ctx, "fetch")
}
out, err := s.Fetch(ctx, args)
if err != nil {
return nil, fetched{}, err
@@ -126,6 +131,7 @@ func (s AskSteps) build(ctx context.Context, job delivery.BuildJob) (any, error)
if s.Copy == nil {
return nil, Refuse("this build module copies no artifact (%s)", a.Artifact)
}
Phase(ctx, "copy "+a.Artifact)
out, err = s.Copy(ctx, map[string]any{"module": job.Module, "artifact": a.Artifact, "from": a.From})
} else {
if s.Build == nil {
@@ -135,6 +141,7 @@ func (s AskSteps) build(ctx context.Context, job delivery.BuildJob) (any, error)
if job.Publish {
args["publish"] = true
}
Phase(ctx, "build "+a.Artifact)
out, err = s.Build(ctx, args)
}
if err != nil {
@@ -196,6 +203,13 @@ func (s AskSteps) level(ctx context.Context, job delivery.CheckJob, level string
return args
}
test := func(args map[string]any) (bool, error) {
what := "the whole level"
if dir, _ := args["module"].(string); dir != "" {
what = dir
} else if pkgs, _ := args["packages"].([]string); len(pkgs) > 0 {
what = fmt.Sprintf("%d packages", len(pkgs))
}
Phase(ctx, fmt.Sprintf("%s: tests of %s", level, what))
out, err := s.Test(ctx, args)
if err != nil {
return false, err
@@ -287,6 +301,7 @@ func (s AskSteps) plan(ctx context.Context, job delivery.CheckJob, level string,
if job.Run != "" {
return nil, true, true
}
Phase(ctx, level+": digest")
out, err := s.Digest(ctx, withRunner(map[string]any{"workspace": f.Workspace, "level": level}))
if err != nil {
if s.Logf != nil {
+135
View File
@@ -0,0 +1,135 @@
package buildstep
import (
"encoding/json"
"fmt"
"os"
"strings"
"time"
)
// **A node may build only within its build hours** (the operator, 2026-10-26 21:18: "we should only use g14 as a builder
// outside office hours", "from 18 o'clock until 6 in the morning"). A build module's `build-hours` setting names, per
// node, the window in the node's local time in which it takes asks; outside it the capacity judged by is 0: steps running
// finish, no new one is taken. A node it does not name, or an empty setting, builds at any hour. The puller judges it at
// every judgement and wakes on one timer to the window's next edge (a clock edge, not a poll of another part).
// BuildHoursSetting is the build modules' setting of the nodes' build windows: `<node>=HH:MM-HH:MM` pairs separated by
// commas, in the node's local time; an end before its start crosses midnight (g14=18:00-06:00).
const BuildHoursSetting = "build-hours"
// BuildWindow is one node's build hours: minutes after local midnight, Start inclusive, End exclusive.
type BuildWindow struct {
Start, End int
}
// String is the window as the setting writes it.
func (w BuildWindow) String() string {
return fmt.Sprintf("%02d:%02d-%02d:%02d", w.Start/60, w.Start%60, w.End/60, w.End%60)
}
// Open says whether t, in its own location, falls in the window.
func (w BuildWindow) Open(t time.Time) bool {
m := t.Hour()*60 + t.Minute()
if w.Start < w.End {
return m >= w.Start && m < w.End
}
return m >= w.Start || m < w.End
}
// NextEdge is the first time after t, in t's location, at which the window opens or closes.
func (w BuildWindow) NextEdge(t time.Time) time.Time {
var next time.Time
for day := 0; day <= 1; day++ {
for _, m := range []int{w.Start, w.End} {
e := time.Date(t.Year(), t.Month(), t.Day()+day, m/60, m%60, 0, 0, t.Location())
if e.After(t) && (next.IsZero() || e.Before(next)) {
next = e
}
}
}
return next
}
// ParseBuildHours is node's window in a build-hours setting: nil when the setting is empty or does not name the node.
func ParseBuildHours(setting, node string) (*BuildWindow, error) {
var found *BuildWindow
for _, pair := range strings.Split(setting, ",") {
pair = strings.TrimSpace(pair)
if pair == "" {
continue
}
name, span, ok := strings.Cut(pair, "=")
from, to, ok2 := strings.Cut(span, "-")
if !ok || !ok2 || strings.TrimSpace(name) == "" {
return nil, fmt.Errorf("the build hours %q: %q is not <node>=HH:MM-HH:MM", setting, pair)
}
start, err := minuteOf(from)
if err != nil {
return nil, fmt.Errorf("the build hours %q: %v", setting, err)
}
end, err := minuteOf(to)
if err != nil {
return nil, fmt.Errorf("the build hours %q: %v", setting, err)
}
if start == end {
return nil, fmt.Errorf("the build hours %q: %s opens and closes at the same minute", setting, pair)
}
if strings.TrimSpace(name) == node {
found = &BuildWindow{Start: start, End: end}
}
}
return found, nil
}
// minuteOf reads HH:MM as minutes after midnight.
func minuteOf(s string) (int, error) {
var h, m int
s = strings.TrimSpace(s)
if n, err := fmt.Sscanf(s, "%d:%d", &h, &m); err != nil || n != 2 || h < 0 || h > 23 || m < 0 || m > 59 || len(s) != 5 {
return 0, fmt.Errorf("%q is not a time HH:MM", s)
}
return h*60 + m, nil
}
// NodeBuildHours is node's build window, read from the module's settings file at path at every call, and why it is
// none when the value cannot be read. No path, no file value or an empty one is no window and no fault. A value that
// cannot be read is no window too, so the node builds at any hour; the setting is a preference of hours, not a safety
// bound (a node that must never build has capacity 0). It is loud: the fault is said once per distinct fault through
// logf, and the puller writes it in the capacity record that build_usage shows.
func NodeBuildHours(path, node string, logf func(string, ...any)) func() (*BuildWindow, string) {
said := ""
say := func(why string) {
if logf != nil && why != said {
said = why
logf("%s, so %s builds at any hour", why, node)
}
}
fault := func(why string) (*BuildWindow, string) {
say(why)
return nil, why
}
return func() (*BuildWindow, string) {
if path == "" {
return nil, ""
}
raw, err := os.ReadFile(path)
if err != nil {
return fault(fmt.Sprintf("the setting %s cannot be read: %v", BuildHoursSetting, err))
}
var s map[string]json.RawMessage
if err := json.Unmarshal(raw, &s); err != nil {
return fault(fmt.Sprintf("the settings file %s is not JSON: %v", path, err))
}
setting := ""
if v, ok := s[BuildHoursSetting]; ok && json.Unmarshal(v, &setting) != nil {
return fault(fmt.Sprintf("the setting %s %s is not a string", BuildHoursSetting, v))
}
w, err := ParseBuildHours(setting, node)
if err != nil {
return fault(err.Error())
}
said = ""
return w, ""
}
}
+141
View File
@@ -0,0 +1,141 @@
package buildstep
import (
"encoding/json"
"fmt"
"math"
"os"
"runtime"
"strconv"
"strings"
"git.novox.be/novox/mesh-sdk/go/buildcpu"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// **A build seat's capacity follows the CPU its node gives builds** (issue 1806's follow-up; issue 1890, 2026-10-26
// 19:41: under the 50 % cap shanks' mesh-build-go kept capacity 3, three full mesh-catalog checks shared 12 cores, and
// each step ran past the 30-minute step bound with no test run). The configured capacity is an upper bound; the node's
// build slice bounds it again by the cores a step needs: floor(the slice's quota in cores / cores-per-step), one at
// least. The quota is read live from the cgroup at every judgement, so a changed cap is followed at the next take
// without a restart, and the result is said (the seat's current verb, the status tools, `capacity.<node>` in the pauses
// state that the delivery view's build_usage reads).
// CoresPerStepSetting is a build module's setting of the processors one of its steps needs: a number above zero, the
// same on every node (a step's need is the module's, not the node's). Each build module declares it with its own default
// (DefaultCoresPerStep).
const CoresPerStepSetting = "cores-per-step"
// DefaultCoresPerStep are the build modules' defaults by language, from measured steps (2026-10-26): a full mesh-catalog
// check on mesh-build-go takes what it is given (three at once took 965 % of shanks' 1200 % slice, about 4 cores each,
// and ran past the 30-minute step bound), so a Go step is given 6, half again what starved; a Node step (tsc, esbuild, a
// test runner: median 15 s) and an image build (buildah, its RUN lines mostly fetches) need 2.
var DefaultCoresPerStep = map[string]float64{delivery.AskGo: 6, delivery.AskNode: 2, delivery.AskImage: 2}
// Quota is the build slice's CPU quota as this process's cgroup holds it, in percent of one processor (1200 is 12
// cores), and where it was read: the cgroup's cpu.max (buildcpu.ReadInForce, the tighter of its unit's and its slice's),
// else the share the composer gave (buildcpu.ShareWord) over the machine's processors, else 0 and why none is known.
func Quota() (percent int, from string) {
f := buildcpu.ReadInForce()
if f.Quota > 0 {
path := f.Slice.Path
if path == "" {
path = f.Cgroup
}
return f.Quota, "the cpu.max of " + path
}
if n, err := strconv.Atoi(strings.TrimSpace(os.Getenv(buildcpu.ShareWord))); err == nil && n >= 1 && n <= 100 {
return buildcpu.Quota(n, runtime.NumCPU()), fmt.Sprintf("the share %d%% the composer gave (%s), of %d processors; no cpu.max is read", n, buildcpu.ShareWord, runtime.NumCPU())
}
return 0, "no CPU quota: neither a cpu.max on its cgroup nor a share from the composer"
}
// EffectiveCapacity is the capacity a seat's holder judges by, from its configured capacity, its node's quota (percent of
// one processor, 0 when none is known) and the cores one of its steps needs. Without a quota it is the configured one;
// with one, min(configured, floor(quota cores / coresPerStep)), and one at least while the configured one is above
// zero. A coresPerStep of zero or less is taken as 1.
func EffectiveCapacity(configured, quotaPercent int, quotaFrom string, coresPerStep float64) delivery.Capacity {
if coresPerStep <= 0 || math.IsNaN(coresPerStep) || math.IsInf(coresPerStep, 0) {
coresPerStep = 1
}
c := delivery.Capacity{Configured: configured, Effective: configured, QuotaFrom: quotaFrom, CoresPerStep: coresPerStep}
switch {
case configured <= 0:
c.Effective = 0
c.Why = fmt.Sprintf("its configured capacity is %d, so it pulls nothing", configured)
return c
case quotaPercent <= 0:
c.Why = fmt.Sprintf("its configured capacity %d; %s, so the CPU bounds nothing", configured, quotaFrom)
return c
}
c.QuotaCores = float64(quotaPercent) / 100
c.ByCPU = int(math.Floor(c.QuotaCores/coresPerStep + 1e-9))
if c.ByCPU < 1 {
c.ByCPU = 1
}
byCPU := fmt.Sprintf("%g cores of the build slice (%s) / %g cores a step = %d", c.QuotaCores, quotaFrom, coresPerStep, c.ByCPU)
if c.ByCPU < configured {
c.Effective = c.ByCPU
c.Why = fmt.Sprintf("%d, bounded by the CPU: %s, under its configured %d", c.Effective, byCPU, configured)
} else {
c.Why = fmt.Sprintf("%d, its configured capacity; the CPU allows %s", configured, byCPU)
}
return c
}
// NodeCoresPerStep is the module's cores-per-step, read from its settings file at path at every call so a change is
// followed at the next judgement; fallback when there is no path, no value or one that is not a number above zero, said
// once per distinct fault through logf.
func NodeCoresPerStep(path string, fallback float64, logf func(string, ...any)) func() float64 {
said := ""
say := func(why string) {
if logf != nil && why != said {
said = why
logf("%s, so a step is taken to need %g cores", why, fallback)
}
}
return func() float64 {
if path == "" {
return fallback
}
raw, err := os.ReadFile(path)
if err != nil {
say(fmt.Sprintf("the setting %s cannot be read: %v", CoresPerStepSetting, err))
return fallback
}
var s map[string]json.RawMessage
if err := json.Unmarshal(raw, &s); err != nil {
say(fmt.Sprintf("the settings file %s is not JSON: %v", path, err))
return fallback
}
v, ok := s[CoresPerStepSetting]
if !ok {
return fallback // not declared yet (a module before its manifest names it): the default, nothing to say
}
n, err := coresOf(v)
if err != nil {
say(fmt.Sprintf("the setting %s %s: %v", CoresPerStepSetting, v, err))
return fallback
}
said = ""
return n
}
}
// coresOf reads a cores-per-step value: a JSON number or a string holding one, above zero.
func coresOf(v json.RawMessage) (float64, error) {
var n float64
if err := json.Unmarshal(v, &n); err != nil {
var s string
if err := json.Unmarshal(v, &s); err != nil {
return 0, fmt.Errorf("is neither a number nor a string")
}
if n, err = strconv.ParseFloat(strings.TrimSpace(s), 64); err != nil {
return 0, fmt.Errorf("is not a number")
}
}
if n <= 0 || math.IsNaN(n) || math.IsInf(n, 0) {
return 0, fmt.Errorf("is not above zero")
}
return n, nil
}
+89 -5
View File
@@ -3,12 +3,17 @@ package buildstep
import (
"os"
"path/filepath"
"strings"
"testing"
"time"
)
// 1887/A7: the node's build capacity is the build module's setting, read at every pull: the control-node at 0 pulls
// nothing, a change takes effect at the next read, and a setting that cannot be read pulls nothing.
func TestNodeCapacity(t *testing.T) {
// ADR 0553 §4: a step's threads are the composer's share over the module's own capacity; without the word, none. And
// issue 1806's follow-up: the capacity judged by is the configured one bounded by the build slice's cores over the cores
// a step needs (the three build nodes as they stand on 2026-10-26), and cores-per-step is read from the settings file.
func TestNodeCapacityAndLimits(t *testing.T) {
if got := NodeCapacity("", DefaultCapacities, "novox", nil)(); got != 0 {
t.Errorf("novox = %d by default, want 0", got)
}
@@ -35,10 +40,6 @@ func TestNodeCapacity(t *testing.T) {
if got := NodeCapacity(filepath.Join(t.TempDir(), "none"), DefaultCapacities, "shanks", nil)(); got != 0 {
t.Errorf("a missing file = %d, want 0", got)
}
}
// ADR 0553 §4: a step's threads are the composer's share over the module's own capacity; without the word, none.
func TestNodeLimits(t *testing.T) {
t.Setenv("MESH_BUILD_CPU_SHARE", "")
if l := NodeLimits(3); l.Threads != 0 || l.Slots != 3 {
t.Errorf("without the word: %+v", l)
@@ -47,4 +48,87 @@ func TestNodeLimits(t *testing.T) {
if l := NodeLimits(3); l.Percent != 75 || l.Slots != 3 || l.Threads < 1 {
t.Errorf("from the word: %+v", l)
}
for _, c := range []struct {
name string
configured, quota int
cores float64
effective, byCPU int
}{
{"shanks: 12 cores, configured 3", 3, 1200, 6, 2, 2},
{"ace: 18 cores, configured 2", 2, 1800, 6, 2, 3},
{"g14: 8 cores, configured 2", 2, 800, 6, 1, 1},
{"node on shanks: 12 cores at 2", 3, 1200, 2, 3, 6},
{"no quota: the configured one", 3, 0, 6, 3, 0},
{"configured 0 pulls nothing", 0, 1200, 6, 0, 0},
{"a quota under one step still builds one", 2, 100, 6, 1, 1},
{"no cores a step is one", 2, 150, 0, 1, 1},
} {
got := EffectiveCapacity(c.configured, c.quota, "test", c.cores)
if got.Effective != c.effective || got.ByCPU != c.byCPU || got.Why == "" {
t.Errorf("%s: %+v, want effective %d, by CPU %d", c.name, got, c.effective, c.byCPU)
}
}
file = filepath.Join(t.TempDir(), "settings.json")
var said []string
cores := NodeCoresPerStep(file, 6, func(f string, a ...any) { said = append(said, f) })
for _, c := range []struct {
file string
want float64
}{{`{"capacity":"shanks=2"}`, 6}, {`{"cores-per-step":4}`, 4}, {`{"cores-per-step":"3.5"}`, 3.5}, {`{"cores-per-step":0}`, 6}, {`{"cores-per-step":0}`, 6}} {
os.WriteFile(file, []byte(c.file), 0o600)
if got := cores(); got != c.want {
t.Errorf("cores-per-step from %s = %g, want %g", c.file, got, c.want)
}
}
if len(said) != 1 {
t.Errorf("a value not above zero is said once, said %d times", len(said))
}
// The operator, 2026-10-26 21:18: g14 builds only from 18:00 to 06:00, its local time. Outside its build hours the
// capacity judged by is 0, said with the window and when it opens; a node not named, or a bad value, builds at any hour.
cet := time.FixedZone("CET", 3600)
day := func(hh, mm int) time.Time { return time.Date(2026, 10, 26, hh, mm, 0, 0, cet) }
was := now
defer func() { now = was }()
for _, c := range []struct {
name, setting, node string
at time.Time
effective int
edge string // the window's next edge, HH:MM; "" for no window
bad bool
}{
{"g14 in the office", "g14=18:00-06:00", "g14", day(10, 0), 0, "18:00", false},
{"g14 in the evening", "g14=18:00-06:00", "g14", day(18, 0), 2, "06:00", false},
{"g14 past midnight", "g14=18:00-06:00", "g14", day(5, 59), 2, "06:00", false},
{"g14 at the close", "g14=18:00-06:00", "g14", day(6, 0), 0, "18:00", false},
{"a day window", "shanks=09:00-17:30,g14=18:00-06:00", "shanks", day(17, 29), 2, "17:30", false},
{"a node not named", "g14=18:00-06:00", "ace", day(10, 0), 2, "", false},
{"a bad time", "g14=18:00-6", "g14", day(10, 0), 2, "", true},
{"no span", "g14=18:00-18:00", "g14", day(10, 0), 2, "", true},
} {
w, err := ParseBuildHours(c.setting, c.node)
if (err != nil) != c.bad {
t.Errorf("%s: parse %v", c.name, err)
}
if edge := ""; w != nil {
edge = w.NextEdge(c.at).Format("15:04")
if edge != c.edge {
t.Errorf("%s: next edge %s, want %s", c.name, edge, c.edge)
}
} else if c.edge != "" {
t.Errorf("%s: no window, want one", c.name)
}
file := filepath.Join(t.TempDir(), "settings.json")
os.WriteFile(file, []byte(`{"build-hours":"`+c.setting+`"}`), 0o600)
now = func() time.Time { return c.at }
p := &Puller{Language: "go", Node: c.node, Capacity: func() int { return 2 }, BuildHours: NodeBuildHours(file, c.node, nil)}
got := p.CapacityDetail()
if got.Effective != c.effective || got.OutsideHours != (c.effective == 0) || (c.effective == 0 && !strings.Contains(got.Why, "outside its build hours 18:00-06:00 CET until 18:00")) {
t.Errorf("%s: %+v, want capacity %d", c.name, got, c.effective)
}
// A value that cannot be read builds at any hour, loudly: the record names the value and says why.
if c.bad && (!strings.Contains(got.BuildHoursFault, c.setting) || !strings.Contains(got.Why, "at any hour, because its build hours cannot be read")) {
t.Errorf("%s: a bad value is not said in the record: %+v", c.name, got)
}
}
}
@@ -0,0 +1,73 @@
# The conformance kit's captured runner output
Every file here is the real output of a test runner, never written by hand (issue 1901: a fixture that agrees with the
mistake proves the mistake). The toolchains are the build nodes' own, as `mesh-build-go.build_go_toolchains` and
`mesh-build-node.build_node_toolchains` reported them on 2026-10-26: go1.27.3-X:nodwarf5 on every build node, node
v26.12.1 (npm 12.2.1) on shanks, novox and g14, and typescript 5.9.3 from mesh-build-node's toolchain/package.json.
To capture a case again, make its tiny project in a fresh directory as below and run the command shown; the kit's case
table (conformance.go) says what each failure must say.
## go/ — `go test -json`
- `passfirst.jsonl` and `subtests.jsonl` (2026-10-26, go1.27.3-X:nodwarf5, for issue 1901's thorough fix): in
`/tmp/1901k/go/<case>`, `GOTOOLCHAIN=local GOFLAGS=-mod=mod go test -json -count=1 ./... > ../<case>.jsonl 2>&1`.
- passfirst: module `example.com/passfirst`, `go 1.26`. Package `a` has `TestTheFirstPackagePasses` (empty). Package `b`
has `TestTheSecondPackagePassesFirst` (empty) and `TestTheSecondPackageFails`, which calls
`t.Logf("judged against the seat as it stands")` then `t.Fatalf("the seat has 2 holders, want 1")`: a passing package
and a passing test before the failing one.
- subtests: module `example.com/subtests`; `TestTheTable` runs `t.Run("passes", func(t *testing.T) {})` and
`t.Run("fails", func(t *testing.T) { t.Errorf("got 3 machines, want 4") })`: the parent fails only by its subtest.
- `subpanic.jsonl` (2026-10-26, go1.27.3-X:nodwarf5, review round 1 of issue 1901's fix), the same way: module
`example.com/subpanic`; `TestTable` runs `t.Run("passes", func(t *testing.T) {})` and `t.Run("panics", ...)`, which
writes to a nil map. go test -json files the subtest's panic under the parent `TestTable`.
- `logthenfatal.jsonl`, `panicking`, `timeout`, `deeptimeout`, `buildfail`, `exiting`, `killed`, `passthenfail`,
`mainexit`, `race`, `racemain` were captured for issue 1901's first and third meetings (2026-10-23 and 2026-10-25,
go1.27.3) in mesh-catalog's mesh-build-go (`testdata/captured/go-test-json-*.jsonl`, named there `go-test-json-pkg-<case>`
and `go-test-json-log-then-fatal`) and moved here unchanged. From that README: each is the standard output and error of
`go test -json -count=1 ./...` (the timeouts with `-timeout 2s`; race and racemain with `-race`, CGO_ENABLED=1) in a
throwaway one-file module that fails without a test's own assertion: panicking (a nil map write in TestPanics), timeout
(TestWaitsForever sleeps past the timeout), deeptimeout (two parallel tests and sixty blocked goroutines past the
timeout, so the goroutine dump pushes the panic line out of the last 4 KB), buildfail (a type error in a test file),
exiting (os.Exit(3) in TestExits), killed (TestKilled sends itself SIGKILL), passthenfail (TestMain panics in a
goroutine after m.Run passed), mainexit (TestMain exits 2 after m.Run passed: go test prints "PASS" and then
"FAIL <pkg>"), race (TestRacesOnTheSeat writes a package variable from itself and a goroutine), racemain (TestMain does
so after m.Run passed). logthenfatal is a test `TestThree` in package `x` that logs a line with t.Logf and then fails
with a multi-line t.Fatalf naming three subscriptions.
## node/ — `npm test` running `node --test` under the TAP reporter
Captured 2026-10-26 with node v26.12.1 and npm 12.2.1 (`/usr/bin/node`, `/usr/bin/npm`), in `/tmp/1901k/node/<case>`, the
way mesh-build-node runs a test script:
PATH=/usr/bin:$PATH NODE_OPTIONS="--test-reporter=tap --test-reporter-destination=/tmp/1901k/node/<case>.tap" \
npm test > ../<case>.stdout 2> ../<case>.stderr
`<case>.tap` is the report file, `<case>.console` the stdout followed by the stderr. Each project's package.json is
`{"name":"<case>","private":true,"type":"module","scripts":{"test":"node --test --experimental-strip-types 'test/*.test.ts'"}}`
(the script every TypeScript package of the catalogue and the SDK runs), with these test files:
- two-files: `test/a.test.ts` holds `test('passes in the first file', () => {})`; `test/b.test.ts` holds
`test('passes in the second file first', () => {})` and `test('fails in the second file', ...)`, which logs
`judged against the seat as it stands` and asserts `assert.strictEqual(1 + 1, 3, 'one plus one is not three')`.
- suite: `test/a.test.ts` has `describe('the seat', ...)` with `it('passes inside', () => {})` and `it('fails inside', ...)`
asserting `assert.deepStrictEqual({ holders: 2 }, { holders: 1 })`.
- crash: `test/a.test.ts` holds `test('passes before the crash', () => {})`; `test/b.test.ts` holds
`test('throws after it returned', ...)`, which throws `new Error('the bus closed under the test')` in a `setTimeout` of
1 ms, and `test('waits after the throw', ...)`, which waits 50 ms: node fails the file with an uncaught exception.
- logcrash (review round 1): `test/a.test.ts` holds `test('passes and logs', ...)`, which logs
`retrying after Error: the store was not ready yet`; `test/b.test.ts` is crash's. A passing test's console line
holding "Error" comes before the crash's own.
- timeout: `test/a.test.ts` holds `test('passes before the wait', () => {})` and
`test('waits for an answer that never comes', { timeout: 200 }, async () => { await new Promise(() => {}); })`.
- syntax: `test/a.test.ts` holds `test('passes in the first file', () => {})`; `test/b.test.ts` is
`import test from 'node:test';` and `test('never loads', () => {` (cut off): the file does not load.
## tsc/ — the type-check
`typeerror.console` (2026-10-26, typescript 5.9.3 under node v26.12.1): in `/tmp/1901k/tsc/typeerror`, `ok.ts` is
`export const holders: number = 1;` and `seat.ts` is `import { holders } from './ok.js'; export const name: string = holders;`;
the command is mesh-build-node's type-check line without a tsconfig.json:
node <typescript>/bin/tsc --noEmit --module NodeNext --moduleResolution NodeNext --target ES2022 --rootDir . ok.ts seat.ts
and the file is its stdout followed by its stderr.
@@ -0,0 +1,6 @@
{"ImportPath":"example.com/buildfail [example.com/buildfail.test]","Action":"build-output","Output":"# example.com/buildfail [example.com/buildfail.test]\n"}
{"ImportPath":"example.com/buildfail [example.com/buildfail.test]","Action":"build-output","Output":"./p_test.go:5:40: cannot use \"seat\" (untyped string constant) as int value in variable declaration\n"}
{"ImportPath":"example.com/buildfail [example.com/buildfail.test]","Action":"build-fail"}
{"Time":"2026-10-25T15:34:33.261137737+01:00","Action":"start","Package":"example.com/buildfail"}
{"Time":"2026-10-25T15:34:33.261255019+01:00","Action":"output","Package":"example.com/buildfail","Output":"FAIL\texample.com/buildfail [build failed]\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.261268099+01:00","Action":"fail","Package":"example.com/buildfail","Elapsed":0,"FailedBuild":"example.com/buildfail [example.com/buildfail.test]"}
@@ -0,0 +1,417 @@
{"Time":"2026-10-25T15:35:18.991630993+01:00","Action":"start","Package":"example.com/deeptimeout"}
{"Time":"2026-10-25T15:35:18.993895188+01:00","Action":"run","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat"}
{"Time":"2026-10-25T15:35:18.993963909+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":"=== RUN TestHoldsTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.99399484+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":"=== PAUSE TestHoldsTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.9939982+01:00","Action":"pause","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat"}
{"Time":"2026-10-25T15:35:18.99400173+01:00","Action":"run","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus"}
{"Time":"2026-10-25T15:35:18.99400456+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"=== RUN TestWaitsForTheBus\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.99400838+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"=== PAUSE TestWaitsForTheBus\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.99401133+01:00","Action":"pause","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus"}
{"Time":"2026-10-25T15:35:18.99401441+01:00","Action":"cont","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat"}
{"Time":"2026-10-25T15:35:18.99401704+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":"=== CONT TestHoldsTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.994074141+01:00","Action":"cont","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus"}
{"Time":"2026-10-25T15:35:18.994102612+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"=== CONT TestWaitsForTheBus\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:21.00935195+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"panic: test timed out after 2s\n"}
{"Time":"2026-10-25T15:35:21.00938383+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\trunning tests:\n"}
{"Time":"2026-10-25T15:35:21.009394391+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t\tTestHoldsTheSeat (2s)\n"}
{"Time":"2026-10-25T15:35:21.009398021+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t\tTestWaitsForTheBus (2s)\n"}
{"Time":"2026-10-25T15:35:21.009401801+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009406481+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 81 [running]:\n"}
{"Time":"2026-10-25T15:35:21.009409911+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.(*M).startAlarm.func1()\n"}
{"Time":"2026-10-25T15:35:21.009413361+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2961 +0x34a\n"}
{"Time":"2026-10-25T15:35:21.009418341+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by time.goFunc\n"}
{"Time":"2026-10-25T15:35:21.009421781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/time/sleep.go:182 +0x2d\n"}
{"Time":"2026-10-25T15:35:21.009424991+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009428341+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 1 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009431721+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner.func1()\n"}
{"Time":"2026-10-25T15:35:21.009441811+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2144 +0x425\n"}
{"Time":"2026-10-25T15:35:21.009445511+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner(0x282e9702008, 0x282e9646bc8)\n"}
{"Time":"2026-10-25T15:35:21.009449641+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2201 +0x123\n"}
{"Time":"2026-10-25T15:35:21.009453471+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.runTests({0x55a45d, 0x17}, {0x55a45d, 0x17}, 0x282e95940f0, {0x6f19f0, 0x2, 0x2}, {0xc2afa2ca3b3c0c8d, 0x7737f168, ...})\n"}
{"Time":"2026-10-25T15:35:21.009462092+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2742 +0x510\n"}
{"Time":"2026-10-25T15:35:21.009467222+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.(*M).Run(0x282e96581e0)\n"}
{"Time":"2026-10-25T15:35:21.009470662+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2602 +0x6af\n"}
{"Time":"2026-10-25T15:35:21.009473692+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"main.main()\n"}
{"Time":"2026-10-25T15:35:21.009476942+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t_testmain.go:48 +0x9b\n"}
{"Time":"2026-10-25T15:35:21.009480092+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009483242+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 7 [sleep]:\n"}
{"Time":"2026-10-25T15:35:21.009486502+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"time.Sleep(0xdf8475800)\n"}
{"Time":"2026-10-25T15:35:21.009490682+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/runtime/time.go:368 +0x165\n"}
{"Time":"2026-10-25T15:35:21.009493812+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat(0x282e9702248?)\n"}
{"Time":"2026-10-25T15:35:21.009498842+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:14 +0x9e\n"}
{"Time":"2026-10-25T15:35:21.009502242+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner(0x282e9702248, 0x6d5e78)\n"}
{"Time":"2026-10-25T15:35:21.009505742+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:35:21.009509292+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:35:21.009512752+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:35:21.009515792+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009518623+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 8 [sleep]:\n"}
{"Time":"2026-10-25T15:35:21.009521303+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"time.Sleep(0xdf8475800)\n"}
{"Time":"2026-10-25T15:35:21.009524423+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/runtime/time.go:368 +0x165\n"}
{"Time":"2026-10-25T15:35:21.009531403+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestWaitsForTheBus(0x282e9702488?)\n"}
{"Time":"2026-10-25T15:35:21.009534943+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:19 +0x25\n"}
{"Time":"2026-10-25T15:35:21.009538403+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner(0x282e9702488, 0x6d5e80)\n"}
{"Time":"2026-10-25T15:35:21.009541953+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:35:21.009545233+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:35:21.009548323+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:35:21.009551343+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009554263+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 9 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009557523+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009561063+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009564513+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009571113+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009574233+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009577433+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 10 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009580823+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009584174+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009587504+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009590974+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009594004+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009597184+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 11 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009600394+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009603874+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009610954+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009614714+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009618104+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009621364+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 12 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009624704+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009628324+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009631954+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009635404+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009638734+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009641904+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 13 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009645325+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009648645+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009652065+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009655625+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009658885+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009663385+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 14 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009666735+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009670235+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009676385+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009680205+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009683315+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009686475+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 15 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009689785+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009697235+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009700795+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009704985+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009708085+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009711346+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 16 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009714586+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009717946+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009721676+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009725626+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009728866+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009732106+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 17 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009735376+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009738956+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009742326+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009746436+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009750666+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009754126+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 18 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009758286+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009761846+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009765776+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009769816+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009773167+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009780507+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 19 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009784207+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009788007+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009791627+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009796347+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009799427+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009802937+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 20 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009806667+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009809987+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009813827+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009818087+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009821397+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009824227+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 21 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009827607+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009831307+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009835167+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009839528+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009842728+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009845888+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 22 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009849168+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009852548+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009855958+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009863648+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009866938+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009870088+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 23 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009879828+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009883728+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009887608+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009891548+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009895238+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009898748+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 24 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009902349+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009907109+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009911519+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009915609+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009918939+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009922329+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 25 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009928329+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009931909+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009935489+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009939429+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009942699+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009946019+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 26 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009949559+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009953489+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.00996244+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.00996647+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.00996987+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.00997314+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 27 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.00997664+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.00998035+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.00998418+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.00998793+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.00999131+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.00999534+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 28 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.00999894+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01000268+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.0100087+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01001258+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01001605+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01002007+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 29 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01002393+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010027531+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010031351+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010035521+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010038781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010042281+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 30 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010046091+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010054261+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010057881+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010061781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010064931+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010068261+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 31 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010072151+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010075981+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010079841+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010098802+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010103202+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010106382+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 32 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010109852+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010113242+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010116952+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010120672+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010124112+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010127162+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 33 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010130662+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010134442+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010138142+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010141972+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010145362+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010152262+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 34 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010156353+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010159913+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010163533+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010167403+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010170533+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010173953+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 35 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010177453+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010181033+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010184693+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010188583+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010191753+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010195033+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 36 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010198603+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010202393+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010206123+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010210583+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010213503+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010216914+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 37 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010220134+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010223644+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010227154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010235154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010238354+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010241754+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 38 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010270284+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010275144+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010278714+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010282825+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010285795+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010288935+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 39 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010292405+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010295915+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010299545+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010311835+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010315565+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010318975+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 40 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010322655+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010326225+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010329815+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010333745+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010337095+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010340515+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 41 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010344066+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010347706+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010355596+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010359046+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010362036+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010365076+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 42 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010367956+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010370906+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010375366+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010378386+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010381006+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010383666+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 43 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010386676+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010389786+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010392636+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010395746+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010398466+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010401166+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 44 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010404226+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010407186+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010410117+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010413237+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010416157+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010418897+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 45 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010423157+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010430107+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010433197+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010451207+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010454217+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010456967+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 46 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010459877+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010463967+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010467067+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010470597+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010473418+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010476108+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 47 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010478998+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010481958+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010484978+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010488258+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010490888+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010493628+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 48 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010496588+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010499498+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010502498+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010505768+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010508558+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010514148+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 49 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010517178+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010520238+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010523478+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010526818+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010529718+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010532508+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 50 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010535639+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010538749+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010541939+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010546689+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010549869+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010552989+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 51 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010556299+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010560079+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010563479+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010567189+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010570299+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010573079+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 52 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010576319+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010579599+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010583099+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010590519+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010594029+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010596989+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 53 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01059986+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01060433+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.01060789+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01061136+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01061393+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01061667+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 54 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01061984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.0106227+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.01062566+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01062876+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01063131+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01063428+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 55 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01063719+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01063997+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.01064278+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01064582+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01064836+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01065114+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 56 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01065437+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01065724+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010671481+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010675671+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010678431+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010691171+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 57 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010694531+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010697641+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010700781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010704001+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010706611+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010709421+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 58 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010712331+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010715351+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010718431+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010721691+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010724371+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010727362+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 59 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010730592+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010733662+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010736712+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010740162+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010742922+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010745772+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 60 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010752222+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010755402+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010758542+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010761822+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010764592+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010767992+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 61 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010771062+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010774282+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010779382+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010782942+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010785722+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010788573+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 62 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010791893+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010794943+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010798083+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010801683+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010804293+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010807283+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 63 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010810443+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010813553+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010816793+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010820233+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010823063+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010829493+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 64 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010832683+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010835953+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010839223+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010842463+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010845333+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010848423+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 65 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010851663+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010854964+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010858364+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010861714+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010864914+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010867984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 66 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010870944+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010874114+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010877464+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010880504+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010883144+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010886194+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 67 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010889154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010892234+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010895524+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010902154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010904984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010907984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 68 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010911064+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010914074+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010917365+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010920505+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.013209581+01:00","Action":"output","Package":"example.com/deeptimeout","Output":"FAIL\texample.com/deeptimeout\t2.021s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:21.013258851+01:00","Action":"fail","Package":"example.com/deeptimeout","Elapsed":2.022}
@@ -0,0 +1,10 @@
{"Time":"2026-10-25T15:34:33.768742837+01:00","Action":"start","Package":"example.com/exiting"}
{"Time":"2026-10-25T15:34:33.773822287+01:00","Action":"run","Package":"example.com/exiting","Test":"TestFine"}
{"Time":"2026-10-25T15:34:33.773896608+01:00","Action":"output","Package":"example.com/exiting","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.773940319+01:00","Action":"output","Package":"example.com/exiting","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.773945359+01:00","Action":"pass","Package":"example.com/exiting","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:34:33.773952529+01:00","Action":"run","Package":"example.com/exiting","Test":"TestExits"}
{"Time":"2026-10-25T15:34:33.773956439+01:00","Action":"output","Package":"example.com/exiting","Test":"TestExits","Output":"=== RUN TestExits\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.773959829+01:00","Action":"output","Package":"example.com/exiting","Test":"TestExits","Output":" p_test.go:11: closing the store\n"}
{"Time":"2026-10-25T15:34:33.775158538+01:00","Action":"output","Package":"example.com/exiting","Output":"FAIL\texample.com/exiting\t0.006s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.775186509+01:00","Action":"fail","Package":"example.com/exiting","Elapsed":0.006}
@@ -0,0 +1,7 @@
{"Time":"2026-10-25T15:34:34.328258934+01:00","Action":"start","Package":"example.com/killed"}
{"Time":"2026-10-25T15:34:34.331465435+01:00","Action":"run","Package":"example.com/killed","Test":"TestKilled"}
{"Time":"2026-10-25T15:34:34.331699248+01:00","Action":"output","Package":"example.com/killed","Test":"TestKilled","Output":"=== RUN TestKilled\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.331853111+01:00","Action":"output","Package":"example.com/killed","Test":"TestKilled","Output":" p_test.go:10: the kernel is about to kill this\n"}
{"Time":"2026-10-25T15:34:34.334563213+01:00","Action":"output","Package":"example.com/killed","Test":"TestKilled","Output":"signal: killed\n"}
{"Time":"2026-10-25T15:34:34.334611114+01:00","Action":"output","Package":"example.com/killed","Output":"FAIL\texample.com/killed\t0.006s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.334627134+01:00","Action":"fail","Package":"example.com/killed","Elapsed":0.006}
@@ -0,0 +1,13 @@
{"Action":"start","Package":"x"}
{"Action":"run","Package":"x","Test":"TestThree"}
{"Action":"output","Package":"x","Test":"TestThree","Output":"=== RUN TestThree\n","OutputType":"frame"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" a_test.go:4: judged against /var/lib/mesh-build-go-work/id/beside/mesh-catalog as cloned\n"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" a_test.go:5: 3 subscription(s) derive to a subject no emitter owns:\n","OutputType":"error"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" a.b.c\n","OutputType":"error-continue"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" d.e.f\n","OutputType":"error-continue"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" g.h.i\n","OutputType":"error-continue"}
{"Action":"output","Package":"x","Test":"TestThree","Output":"--- FAIL: TestThree (0.00s)\n","OutputType":"frame"}
{"Action":"fail","Package":"x","Test":"TestThree","Elapsed":0}
{"Action":"output","Package":"x","Output":"FAIL\n","OutputType":"frame"}
{"Action":"output","Package":"x","Output":"FAIL\tx\t0.002s\n","OutputType":"frame"}
{"Action":"fail","Package":"x","Elapsed":0.002}
@@ -0,0 +1,8 @@
{"Time":"2026-10-25T15:34:35.274953775+01:00","Action":"start","Package":"example.com/mainexit"}
{"Time":"2026-10-25T15:34:35.278129565+01:00","Action":"run","Package":"example.com/mainexit","Test":"TestFine"}
{"Time":"2026-10-25T15:34:35.278191516+01:00","Action":"output","Package":"example.com/mainexit","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.278341979+01:00","Action":"output","Package":"example.com/mainexit","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.278351739+01:00","Action":"pass","Package":"example.com/mainexit","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:34:35.278359039+01:00","Action":"output","Package":"example.com/mainexit","Output":"PASS\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.279516487+01:00","Action":"output","Package":"example.com/mainexit","Output":"FAIL\texample.com/mainexit\t0.004s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.279539298+01:00","Action":"fail","Package":"example.com/mainexit","Elapsed":0.005}
@@ -0,0 +1,27 @@
{"Time":"2026-10-25T15:34:33.031091436+01:00","Action":"start","Package":"example.com/panicking"}
{"Time":"2026-10-25T15:34:33.036353279+01:00","Action":"run","Package":"example.com/panicking","Test":"TestBefore"}
{"Time":"2026-10-25T15:34:33.036441781+01:00","Action":"output","Package":"example.com/panicking","Test":"TestBefore","Output":"=== RUN TestBefore\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.036568163+01:00","Action":"output","Package":"example.com/panicking","Test":"TestBefore","Output":"--- PASS: TestBefore (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.036609863+01:00","Action":"pass","Package":"example.com/panicking","Test":"TestBefore","Elapsed":0}
{"Time":"2026-10-25T15:34:33.036641304+01:00","Action":"run","Package":"example.com/panicking","Test":"TestPanics"}
{"Time":"2026-10-25T15:34:33.036645914+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"=== RUN TestPanics\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.036698995+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":" p_test.go:8: about to look up the seat\n"}
{"Time":"2026-10-25T15:34:33.036773996+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"--- FAIL: TestPanics (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.03897418+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"panic: assignment to entry in nil map [recovered, repanicked]\n"}
{"Time":"2026-10-25T15:34:33.040332172+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\n"}
{"Time":"2026-10-25T15:34:33.040414493+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"goroutine 8 [running]:\n"}
{"Time":"2026-10-25T15:34:33.040521475+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"testing.tRunner.func1.2({0x6b8120, 0x6eff80})\n"}
{"Time":"2026-10-25T15:34:33.040601826+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2125 +0x232\n"}
{"Time":"2026-10-25T15:34:33.040641947+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"testing.tRunner.func1()\n"}
{"Time":"2026-10-25T15:34:33.040720648+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2128 +0x329\n"}
{"Time":"2026-10-25T15:34:33.04083286+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"panic({0x6b8120?, 0x6eff80?})\n"}
{"Time":"2026-10-25T15:34:33.040912921+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/runtime/panic.go:859 +0x125\n"}
{"Time":"2026-10-25T15:34:33.040981652+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"example.com/panicking.TestPanics(0x228226cd0488?)\n"}
{"Time":"2026-10-25T15:34:33.041059903+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/tmp/fix-1901b/panicking/p_test.go:10 +0x53\n"}
{"Time":"2026-10-25T15:34:33.041132704+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"testing.tRunner(0x228226cd0488, 0x6d5bc8)\n"}
{"Time":"2026-10-25T15:34:33.041215586+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:34:33.041221966+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:34:33.041225556+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:34:33.041984088+01:00","Action":"fail","Package":"example.com/panicking","Test":"TestPanics","Elapsed":0}
{"Time":"2026-10-25T15:34:33.041996638+01:00","Action":"output","Package":"example.com/panicking","Output":"FAIL\texample.com/panicking\t0.010s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.042044079+01:00","Action":"fail","Package":"example.com/panicking","Elapsed":0.011}
@@ -0,0 +1,22 @@
{"Time":"2026-10-26T19:40:29.574068887+01:00","Action":"start","Package":"example.com/passfirst/a"}
{"Time":"2026-10-26T19:40:29.574347185+01:00","Action":"start","Package":"example.com/passfirst/b"}
{"Time":"2026-10-26T19:40:29.576503518+01:00","Action":"run","Package":"example.com/passfirst/a","Test":"TestTheFirstPackagePasses"}
{"Time":"2026-10-26T19:40:29.576557657+01:00","Action":"output","Package":"example.com/passfirst/a","Test":"TestTheFirstPackagePasses","Output":"=== RUN TestTheFirstPackagePasses\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.576589367+01:00","Action":"output","Package":"example.com/passfirst/a","Test":"TestTheFirstPackagePasses","Output":"--- PASS: TestTheFirstPackagePasses (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.576594637+01:00","Action":"pass","Package":"example.com/passfirst/a","Test":"TestTheFirstPackagePasses","Elapsed":0}
{"Time":"2026-10-26T19:40:29.576602487+01:00","Action":"output","Package":"example.com/passfirst/a","Output":"PASS\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.576614477+01:00","Action":"run","Package":"example.com/passfirst/b","Test":"TestTheSecondPackagePassesFirst"}
{"Time":"2026-10-26T19:40:29.576626897+01:00","Action":"output","Package":"example.com/passfirst/b","Test":"TestTheSecondPackagePassesFirst","Output":"=== RUN TestTheSecondPackagePassesFirst\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.576631187+01:00","Action":"output","Package":"example.com/passfirst/b","Test":"TestTheSecondPackagePassesFirst","Output":"--- PASS: TestTheSecondPackagePassesFirst (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.576635567+01:00","Action":"pass","Package":"example.com/passfirst/b","Test":"TestTheSecondPackagePassesFirst","Elapsed":0}
{"Time":"2026-10-26T19:40:29.576638427+01:00","Action":"run","Package":"example.com/passfirst/b","Test":"TestTheSecondPackageFails"}
{"Time":"2026-10-26T19:40:29.576640647+01:00","Action":"output","Package":"example.com/passfirst/b","Test":"TestTheSecondPackageFails","Output":"=== RUN TestTheSecondPackageFails\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.576643477+01:00","Action":"output","Package":"example.com/passfirst/b","Test":"TestTheSecondPackageFails","Output":" b_test.go:8: judged against the seat as it stands\n"}
{"Time":"2026-10-26T19:40:29.576646497+01:00","Action":"output","Package":"example.com/passfirst/b","Test":"TestTheSecondPackageFails","Output":" b_test.go:9: the seat has 2 holders, want 1\n","OutputType":"error"}
{"Time":"2026-10-26T19:40:29.576650087+01:00","Action":"output","Package":"example.com/passfirst/b","Test":"TestTheSecondPackageFails","Output":"--- FAIL: TestTheSecondPackageFails (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.576652797+01:00","Action":"fail","Package":"example.com/passfirst/b","Test":"TestTheSecondPackageFails","Elapsed":0}
{"Time":"2026-10-26T19:40:29.576655177+01:00","Action":"output","Package":"example.com/passfirst/b","Output":"FAIL\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.577365541+01:00","Action":"output","Package":"example.com/passfirst/a","Output":"ok \texample.com/passfirst/a\t0.003s\n"}
{"Time":"2026-10-26T19:40:29.577382561+01:00","Action":"pass","Package":"example.com/passfirst/a","Elapsed":0.003}
{"Time":"2026-10-26T19:40:29.577380641+01:00","Action":"output","Package":"example.com/passfirst/b","Output":"FAIL\texample.com/passfirst/b\t0.003s\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.5774131+01:00","Action":"fail","Package":"example.com/passfirst/b","Elapsed":0.003}
@@ -0,0 +1,15 @@
{"Time":"2026-10-25T15:34:34.919624192+01:00","Action":"start","Package":"example.com/passthenfail"}
{"Time":"2026-10-25T15:34:34.922861224+01:00","Action":"run","Package":"example.com/passthenfail","Test":"TestFine"}
{"Time":"2026-10-25T15:34:34.922932965+01:00","Action":"output","Package":"example.com/passthenfail","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.922969205+01:00","Action":"output","Package":"example.com/passthenfail","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.922979956+01:00","Action":"pass","Package":"example.com/passthenfail","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:34:34.922989416+01:00","Action":"output","Package":"example.com/passthenfail","Output":"PASS\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.925022528+01:00","Action":"output","Package":"example.com/passthenfail","Output":"panic: the bus closed after the tests: nats: connection closed\n"}
{"Time":"2026-10-25T15:34:34.925043388+01:00","Action":"output","Package":"example.com/passthenfail","Output":"\n"}
{"Time":"2026-10-25T15:34:34.925046848+01:00","Action":"output","Package":"example.com/passthenfail","Output":"goroutine 20 [running]:\n"}
{"Time":"2026-10-25T15:34:34.925051338+01:00","Action":"output","Package":"example.com/passthenfail","Output":"example.com/passthenfail.TestMain.func1()\n"}
{"Time":"2026-10-25T15:34:34.925055878+01:00","Action":"output","Package":"example.com/passthenfail","Output":"\t/tmp/fix-1901b/passthenfail/p_test.go:13 +0x25\n"}
{"Time":"2026-10-25T15:34:34.925061428+01:00","Action":"output","Package":"example.com/passthenfail","Output":"created by example.com/passthenfail.TestMain in goroutine 1\n"}
{"Time":"2026-10-25T15:34:34.925065028+01:00","Action":"output","Package":"example.com/passthenfail","Output":"\t/tmp/fix-1901b/passthenfail/p_test.go:13 +0x25\n"}
{"Time":"2026-10-25T15:34:34.925390203+01:00","Action":"output","Package":"example.com/passthenfail","Output":"FAIL\texample.com/passthenfail\t0.005s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.925402904+01:00","Action":"fail","Package":"example.com/passthenfail","Elapsed":0.006}
@@ -0,0 +1,53 @@
{"Time":"2026-10-25T15:50:42.908557309+01:00","Action":"start","Package":"example.com/race"}
{"Time":"2026-10-25T15:50:42.924298679+01:00","Action":"run","Package":"example.com/race","Test":"TestBefore"}
{"Time":"2026-10-25T15:50:42.92438195+01:00","Action":"output","Package":"example.com/race","Test":"TestBefore","Output":"=== RUN TestBefore\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.924648842+01:00","Action":"output","Package":"example.com/race","Test":"TestBefore","Output":"--- PASS: TestBefore (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.924685133+01:00","Action":"pass","Package":"example.com/race","Test":"TestBefore","Elapsed":0}
{"Time":"2026-10-25T15:50:42.924728573+01:00","Action":"run","Package":"example.com/race","Test":"TestRacesOnTheSeat"}
{"Time":"2026-10-25T15:50:42.924733573+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"=== RUN TestRacesOnTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.92637298+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"==================\n"}
{"Time":"2026-10-25T15:50:42.92641567+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"WARNING: DATA RACE\n"}
{"Time":"2026-10-25T15:50:42.926444061+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Write at 0x000000833468 by goroutine 10:\n"}
{"Time":"2026-10-25T15:50:42.926462621+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" example.com/race.TestRacesOnTheSeat.func1()\n"}
{"Time":"2026-10-25T15:50:42.926467121+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /tmp/fix-1901b/race/p_test.go:13 +0x24\n"}
{"Time":"2026-10-25T15:50:42.926483061+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"\n"}
{"Time":"2026-10-25T15:50:42.926509741+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Previous write at 0x000000833468 by goroutine 9:\n"}
{"Time":"2026-10-25T15:50:42.926530142+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" example.com/race.TestRacesOnTheSeat()\n"}
{"Time":"2026-10-25T15:50:42.926534952+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /tmp/fix-1901b/race/p_test.go:14 +0x30\n"}
{"Time":"2026-10-25T15:50:42.926550462+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.tRunner()\n"}
{"Time":"2026-10-25T15:50:42.926554612+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2195 +0x21c\n"}
{"Time":"2026-10-25T15:50:42.926571142+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*T).Run.gowrap1()\n"}
{"Time":"2026-10-25T15:50:42.926575812+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2260 +0x38\n"}
{"Time":"2026-10-25T15:50:42.926592202+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"\n"}
{"Time":"2026-10-25T15:50:42.926607792+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Goroutine 10 (running) created at:\n"}
{"Time":"2026-10-25T15:50:42.926624292+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" example.com/race.TestRacesOnTheSeat()\n"}
{"Time":"2026-10-25T15:50:42.926628923+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /tmp/fix-1901b/race/p_test.go:13 +0x24\n"}
{"Time":"2026-10-25T15:50:42.926643853+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.tRunner()\n"}
{"Time":"2026-10-25T15:50:42.926648103+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2195 +0x21c\n"}
{"Time":"2026-10-25T15:50:42.926674843+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*T).Run.gowrap1()\n"}
{"Time":"2026-10-25T15:50:42.926680673+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2260 +0x38\n"}
{"Time":"2026-10-25T15:50:42.926695433+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"\n"}
{"Time":"2026-10-25T15:50:42.926711063+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Goroutine 9 (running) created at:\n"}
{"Time":"2026-10-25T15:50:42.926726614+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*T).Run()\n"}
{"Time":"2026-10-25T15:50:42.926730814+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2260 +0xb12\n"}
{"Time":"2026-10-25T15:50:42.926746484+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.runTests.func1()\n"}
{"Time":"2026-10-25T15:50:42.926750764+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2744 +0x84\n"}
{"Time":"2026-10-25T15:50:42.926765354+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.tRunner()\n"}
{"Time":"2026-10-25T15:50:42.926769844+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2195 +0x21c\n"}
{"Time":"2026-10-25T15:50:42.926791794+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.runTests()\n"}
{"Time":"2026-10-25T15:50:42.926796704+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2742 +0x9e9\n"}
{"Time":"2026-10-25T15:50:42.926811544+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*M).Run()\n"}
{"Time":"2026-10-25T15:50:42.926816694+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2602 +0xf44\n"}
{"Time":"2026-10-25T15:50:42.926831695+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" main.main()\n"}
{"Time":"2026-10-25T15:50:42.926835935+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" _testmain.go:50 +0x164\n"}
{"Time":"2026-10-25T15:50:42.926850905+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"==================\n"}
{"Time":"2026-10-25T15:50:42.935320241+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.go:1867: race detected during execution of test\n","OutputType":"error"}
{"Time":"2026-10-25T15:50:42.935520113+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"--- FAIL: TestRacesOnTheSeat (0.01s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.935592973+01:00","Action":"fail","Package":"example.com/race","Test":"TestRacesOnTheSeat","Elapsed":0.01}
{"Time":"2026-10-25T15:50:42.935722225+01:00","Action":"run","Package":"example.com/race","Test":"TestAfter"}
{"Time":"2026-10-25T15:50:42.935728845+01:00","Action":"output","Package":"example.com/race","Test":"TestAfter","Output":"=== RUN TestAfter\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.935995928+01:00","Action":"output","Package":"example.com/race","Test":"TestAfter","Output":"--- PASS: TestAfter (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.936006328+01:00","Action":"pass","Package":"example.com/race","Test":"TestAfter","Elapsed":0}
{"Time":"2026-10-25T15:50:42.936009978+01:00","Action":"output","Package":"example.com/race","Output":"FAIL\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.937730745+01:00","Action":"output","Package":"example.com/race","Output":"FAIL\texample.com/race\t0.028s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.937802066+01:00","Action":"fail","Package":"example.com/race","Elapsed":0.029}
@@ -0,0 +1,27 @@
{"Time":"2026-10-25T15:50:43.551701745+01:00","Action":"start","Package":"example.com/racemain"}
{"Time":"2026-10-25T15:50:43.570709708+01:00","Action":"run","Package":"example.com/racemain","Test":"TestFine"}
{"Time":"2026-10-25T15:50:43.570806059+01:00","Action":"output","Package":"example.com/racemain","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:43.57085525+01:00","Action":"output","Package":"example.com/racemain","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:43.57086149+01:00","Action":"pass","Package":"example.com/racemain","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:50:43.57086974+01:00","Action":"output","Package":"example.com/racemain","Output":"PASS\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:43.572155623+01:00","Action":"output","Package":"example.com/racemain","Output":"==================\n"}
{"Time":"2026-10-25T15:50:43.572168083+01:00","Action":"output","Package":"example.com/racemain","Output":"WARNING: DATA RACE\n"}
{"Time":"2026-10-25T15:50:43.572171933+01:00","Action":"output","Package":"example.com/racemain","Output":"Write at 0x000000836428 by goroutine 9:\n"}
{"Time":"2026-10-25T15:50:43.572176703+01:00","Action":"output","Package":"example.com/racemain","Output":" example.com/racemain.TestMain.func1()\n"}
{"Time":"2026-10-25T15:50:43.572180053+01:00","Action":"output","Package":"example.com/racemain","Output":" /tmp/fix-1901b/racemain/p_test.go:15 +0x24\n"}
{"Time":"2026-10-25T15:50:43.572184923+01:00","Action":"output","Package":"example.com/racemain","Output":"\n"}
{"Time":"2026-10-25T15:50:43.572189613+01:00","Action":"output","Package":"example.com/racemain","Output":"Previous write at 0x000000836428 by main goroutine:\n"}
{"Time":"2026-10-25T15:50:43.572192993+01:00","Action":"output","Package":"example.com/racemain","Output":" example.com/racemain.TestMain()\n"}
{"Time":"2026-10-25T15:50:43.572198573+01:00","Action":"output","Package":"example.com/racemain","Output":" /tmp/fix-1901b/racemain/p_test.go:16 +0x44\n"}
{"Time":"2026-10-25T15:50:43.572202433+01:00","Action":"output","Package":"example.com/racemain","Output":" main.main()\n"}
{"Time":"2026-10-25T15:50:43.572205623+01:00","Action":"output","Package":"example.com/racemain","Output":" _testmain.go:48 +0x171\n"}
{"Time":"2026-10-25T15:50:43.572208443+01:00","Action":"output","Package":"example.com/racemain","Output":"\n"}
{"Time":"2026-10-25T15:50:43.572211573+01:00","Action":"output","Package":"example.com/racemain","Output":"Goroutine 9 (running) created at:\n"}
{"Time":"2026-10-25T15:50:43.572214803+01:00","Action":"output","Package":"example.com/racemain","Output":" example.com/racemain.TestMain()\n"}
{"Time":"2026-10-25T15:50:43.572218043+01:00","Action":"output","Package":"example.com/racemain","Output":" /tmp/fix-1901b/racemain/p_test.go:15 +0x37\n"}
{"Time":"2026-10-25T15:50:43.572225414+01:00","Action":"output","Package":"example.com/racemain","Output":" main.main()\n"}
{"Time":"2026-10-25T15:50:43.572228654+01:00","Action":"output","Package":"example.com/racemain","Output":" _testmain.go:48 +0x171\n"}
{"Time":"2026-10-25T15:50:43.572231664+01:00","Action":"output","Package":"example.com/racemain","Output":"==================\n"}
{"Time":"2026-10-25T15:50:44.581262249+01:00","Action":"output","Package":"example.com/racemain","Output":"Found 1 data race(s)\n"}
{"Time":"2026-10-25T15:50:44.584693204+01:00","Action":"output","Package":"example.com/racemain","Output":"FAIL\texample.com/racemain\t1.031s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:44.584737425+01:00","Action":"fail","Package":"example.com/racemain","Elapsed":1.033}
@@ -0,0 +1,30 @@
{"Time":"2026-10-26T20:15:37.791161045+01:00","Action":"start","Package":"example.com/subpanic"}
{"Time":"2026-10-26T20:15:37.793205666+01:00","Action":"run","Package":"example.com/subpanic","Test":"TestTable"}
{"Time":"2026-10-26T20:15:37.793257158+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"=== RUN TestTable\n","OutputType":"frame"}
{"Time":"2026-10-26T20:15:37.793288149+01:00","Action":"run","Package":"example.com/subpanic","Test":"TestTable/passes"}
{"Time":"2026-10-26T20:15:37.793290669+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable/passes","Output":"=== RUN TestTable/passes\n","OutputType":"frame"}
{"Time":"2026-10-26T20:15:37.793298009+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable/passes","Output":"--- PASS: TestTable/passes (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T20:15:37.793311679+01:00","Action":"pass","Package":"example.com/subpanic","Test":"TestTable/passes","Elapsed":0}
{"Time":"2026-10-26T20:15:37.79331977+01:00","Action":"run","Package":"example.com/subpanic","Test":"TestTable/panics"}
{"Time":"2026-10-26T20:15:37.79332377+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable/panics","Output":"=== RUN TestTable/panics\n","OutputType":"frame"}
{"Time":"2026-10-26T20:15:37.79332773+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable/panics","Output":"--- FAIL: TestTable/panics (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T20:15:37.79333111+01:00","Action":"fail","Package":"example.com/subpanic","Test":"TestTable/panics","Elapsed":0}
{"Time":"2026-10-26T20:15:37.79333383+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"--- FAIL: TestTable (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T20:15:37.795322529+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"panic: assignment to entry in nil map [recovered, repanicked]\n"}
{"Time":"2026-10-26T20:15:37.795331489+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"\n"}
{"Time":"2026-10-26T20:15:37.795340579+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"goroutine 9 [running]:\n"}
{"Time":"2026-10-26T20:15:37.79534387+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"testing.tRunner.func1.2({0x6b7010, 0x6eef80})\n"}
{"Time":"2026-10-26T20:15:37.79534657+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"\t/usr/lib/go/src/testing/testing.go:2125 +0x232\n"}
{"Time":"2026-10-26T20:15:37.79534907+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"testing.tRunner.func1()\n"}
{"Time":"2026-10-26T20:15:37.79535172+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"\t/usr/lib/go/src/testing/testing.go:2128 +0x329\n"}
{"Time":"2026-10-26T20:15:37.79535625+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"panic({0x6b7010?, 0x6eef80?})\n"}
{"Time":"2026-10-26T20:15:37.79535879+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"\t/usr/lib/go/src/runtime/panic.go:859 +0x125\n"}
{"Time":"2026-10-26T20:15:37.79536129+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"example.com/subpanic.TestTable.func2(0x3b39436666c8?)\n"}
{"Time":"2026-10-26T20:15:37.79536452+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"\t/tmp/1901k/go/subpanic/p_test.go:7 +0x28\n"}
{"Time":"2026-10-26T20:15:37.79536782+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"testing.tRunner(0x3b39436666c8, 0x6d4b58)\n"}
{"Time":"2026-10-26T20:15:37.7953704+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-26T20:15:37.79537289+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"created by testing.(*T).Run in goroutine 7\n"}
{"Time":"2026-10-26T20:15:37.795381681+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-26T20:15:37.795731441+01:00","Action":"fail","Package":"example.com/subpanic","Test":"TestTable","Elapsed":0}
{"Time":"2026-10-26T20:15:37.795759332+01:00","Action":"output","Package":"example.com/subpanic","Output":"FAIL\texample.com/subpanic\t0.004s\n","OutputType":"frame"}
{"Time":"2026-10-26T20:15:37.795783503+01:00","Action":"fail","Package":"example.com/subpanic","Elapsed":0.005}
@@ -0,0 +1,17 @@
{"Time":"2026-10-26T19:40:29.768101399+01:00","Action":"start","Package":"example.com/subtests"}
{"Time":"2026-10-26T19:40:29.770109733+01:00","Action":"run","Package":"example.com/subtests","Test":"TestTheTable"}
{"Time":"2026-10-26T19:40:29.770152272+01:00","Action":"output","Package":"example.com/subtests","Test":"TestTheTable","Output":"=== RUN TestTheTable\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770189372+01:00","Action":"run","Package":"example.com/subtests","Test":"TestTheTable/passes"}
{"Time":"2026-10-26T19:40:29.770191882+01:00","Action":"output","Package":"example.com/subtests","Test":"TestTheTable/passes","Output":"=== RUN TestTheTable/passes\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770197442+01:00","Action":"output","Package":"example.com/subtests","Test":"TestTheTable/passes","Output":"--- PASS: TestTheTable/passes (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770204352+01:00","Action":"pass","Package":"example.com/subtests","Test":"TestTheTable/passes","Elapsed":0}
{"Time":"2026-10-26T19:40:29.770209622+01:00","Action":"run","Package":"example.com/subtests","Test":"TestTheTable/fails"}
{"Time":"2026-10-26T19:40:29.770211832+01:00","Action":"output","Package":"example.com/subtests","Test":"TestTheTable/fails","Output":"=== RUN TestTheTable/fails\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770214722+01:00","Action":"output","Package":"example.com/subtests","Test":"TestTheTable/fails","Output":" s_test.go:7: got 3 machines, want 4\n","OutputType":"error"}
{"Time":"2026-10-26T19:40:29.770218032+01:00","Action":"output","Package":"example.com/subtests","Test":"TestTheTable/fails","Output":"--- FAIL: TestTheTable/fails (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770220732+01:00","Action":"fail","Package":"example.com/subtests","Test":"TestTheTable/fails","Elapsed":0}
{"Time":"2026-10-26T19:40:29.770223522+01:00","Action":"output","Package":"example.com/subtests","Test":"TestTheTable","Output":"--- FAIL: TestTheTable (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770226292+01:00","Action":"fail","Package":"example.com/subtests","Test":"TestTheTable","Elapsed":0}
{"Time":"2026-10-26T19:40:29.770228582+01:00","Action":"output","Package":"example.com/subtests","Output":"FAIL\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770561979+01:00","Action":"output","Package":"example.com/subtests","Output":"FAIL\texample.com/subtests\t0.002s\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770581629+01:00","Action":"fail","Package":"example.com/subtests","Elapsed":0.002}
@@ -0,0 +1,43 @@
{"Time":"2026-10-25T15:34:36.049580618+01:00","Action":"start","Package":"example.com/timeout"}
{"Time":"2026-10-25T15:34:36.053250986+01:00","Action":"run","Package":"example.com/timeout","Test":"TestQuick"}
{"Time":"2026-10-25T15:34:36.053354018+01:00","Action":"output","Package":"example.com/timeout","Test":"TestQuick","Output":"=== RUN TestQuick\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:36.053459789+01:00","Action":"output","Package":"example.com/timeout","Test":"TestQuick","Output":"--- PASS: TestQuick (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:36.05348499+01:00","Action":"pass","Package":"example.com/timeout","Test":"TestQuick","Elapsed":0}
{"Time":"2026-10-25T15:34:36.0535114+01:00","Action":"run","Package":"example.com/timeout","Test":"TestWaitsForever"}
{"Time":"2026-10-25T15:34:36.05351549+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"=== RUN TestWaitsForever\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:36.053567451+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":" p_test.go:11: waiting for the bus\n"}
{"Time":"2026-10-25T15:34:38.064693307+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"panic: test timed out after 2s\n"}
{"Time":"2026-10-25T15:34:38.06487122+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\trunning tests:\n"}
{"Time":"2026-10-25T15:34:38.06491684+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t\tTestWaitsForever (2s)\n"}
{"Time":"2026-10-25T15:34:38.065034382+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\n"}
{"Time":"2026-10-25T15:34:38.065108573+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"goroutine 33 [running]:\n"}
{"Time":"2026-10-25T15:34:38.065154754+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.(*M).startAlarm.func1()\n"}
{"Time":"2026-10-25T15:34:38.065386208+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2961 +0x34a\n"}
{"Time":"2026-10-25T15:34:38.065431878+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"created by time.goFunc\n"}
{"Time":"2026-10-25T15:34:38.0655151+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/time/sleep.go:182 +0x2d\n"}
{"Time":"2026-10-25T15:34:38.06553471+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\n"}
{"Time":"2026-10-25T15:34:38.065847805+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"goroutine 1 [chan receive]:\n"}
{"Time":"2026-10-25T15:34:38.066038298+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.(*T).Run(0x3a0209bc8008, {0x5580aa?, 0x3a0209a9caa0?}, 0x6d5b10)\n"}
{"Time":"2026-10-25T15:34:38.066047488+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2268 +0x4f2\n"}
{"Time":"2026-10-25T15:34:38.066051368+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.runTests.func1(0x3a0209bc8008)\n"}
{"Time":"2026-10-25T15:34:38.066056128+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2744 +0x37\n"}
{"Time":"2026-10-25T15:34:38.066059478+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.tRunner(0x3a0209bc8008, 0x3a0209a9cbc8)\n"}
{"Time":"2026-10-25T15:34:38.066064958+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:34:38.066069599+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.runTests({0x559017, 0x13}, {0x559017, 0x13}, 0x3a0209b8e0d8, {0x6f19f0, 0x2, 0x2}, {0xc2afa2bf832a3f56, 0x773908b5, ...})\n"}
{"Time":"2026-10-25T15:34:38.066083309+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2742 +0x510\n"}
{"Time":"2026-10-25T15:34:38.066086879+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.(*M).Run(0x3a0209b8c1e0)\n"}
{"Time":"2026-10-25T15:34:38.066090269+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2602 +0x6af\n"}
{"Time":"2026-10-25T15:34:38.066093429+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"main.main()\n"}
{"Time":"2026-10-25T15:34:38.066097569+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t_testmain.go:48 +0x9b\n"}
{"Time":"2026-10-25T15:34:38.066100639+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\n"}
{"Time":"2026-10-25T15:34:38.066103769+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"goroutine 20 [sleep]:\n"}
{"Time":"2026-10-25T15:34:38.066106969+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"time.Sleep(0xdf8475800)\n"}
{"Time":"2026-10-25T15:34:38.066110279+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/runtime/time.go:368 +0x165\n"}
{"Time":"2026-10-25T15:34:38.066113639+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"example.com/timeout.TestWaitsForever(0x3a0209bc8488?)\n"}
{"Time":"2026-10-25T15:34:38.066117109+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/tmp/fix-1901b/timeout/p_test.go:12 +0x48\n"}
{"Time":"2026-10-25T15:34:38.066120239+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.tRunner(0x3a0209bc8488, 0x6d5b10)\n"}
{"Time":"2026-10-25T15:34:38.066123509+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:34:38.066126839+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:34:38.066129989+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:34:38.066986453+01:00","Action":"output","Package":"example.com/timeout","Output":"FAIL\texample.com/timeout\t2.017s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:38.067002113+01:00","Action":"fail","Package":"example.com/timeout","Elapsed":2.017}
@@ -0,0 +1,2 @@
npm notice run test
npm notice run node --test --experimental-strip-types 'test/*.test.ts'
@@ -0,0 +1,41 @@
TAP version 13
# Subtest: passes before the crash
ok 1 - passes before the crash
---
duration_ms: 0.746694
type: 'test'
...
# Subtest: throws after it returned
ok 2 - throws after it returned
---
duration_ms: 0.763364
type: 'test'
...
# Subtest: waits after the throw
ok 3 - waits after the throw
---
duration_ms: 49.953893
type: 'test'
...
# Error: Test "throws after it returned" at test/b.test.ts:2:1 generated asynchronous activity after the test ended. This activity created the error "Error: the bus closed under the test" and would have caused the test to fail, but instead triggered an uncaughtException event.
# Subtest: test/b.test.ts
not ok 2 - test/b.test.ts
---
duration_ms: 155.864774
type: 'test'
location: '/tmp/1901k/node/crash/test/b.test.ts:1:1'
failureType: 'testCodeFailure'
exitCode: 1
signal: ~
error: 'test failed'
code: 'ERR_TEST_FAILURE'
...
1..4
# tests 4
# suites 0
# pass 3
# fail 1
# cancelled 0
# skipped 0
# todo 0
# duration_ms 168.479173
@@ -0,0 +1,2 @@
npm notice run test
npm notice run node --test --experimental-strip-types 'test/*.test.ts'
@@ -0,0 +1,42 @@
TAP version 13
# retrying after Error: the store was not ready yet
# Subtest: passes and logs
ok 1 - passes and logs
---
duration_ms: 1.184595
type: 'test'
...
# Subtest: throws after it returned
ok 2 - throws after it returned
---
duration_ms: 0.748862
type: 'test'
...
# Subtest: waits after the throw
ok 3 - waits after the throw
---
duration_ms: 51.019875
type: 'test'
...
# Error: Test "throws after it returned" at test/b.test.ts:2:1 generated asynchronous activity after the test ended. This activity created the error "Error: the bus closed under the test" and would have caused the test to fail, but instead triggered an uncaughtException event.
# Subtest: test/b.test.ts
not ok 2 - test/b.test.ts
---
duration_ms: 172.91715
type: 'test'
location: '/tmp/1901k/node/logcrash/test/b.test.ts:1:1'
failureType: 'testCodeFailure'
exitCode: 1
signal: ~
error: 'test failed'
code: 'ERR_TEST_FAILURE'
...
1..4
# tests 4
# suites 0
# pass 3
# fail 1
# cancelled 0
# skipped 0
# todo 0
# duration_ms 185.744334
@@ -0,0 +1,2 @@
npm notice run test
npm notice run node --test --experimental-strip-types 'test/*.test.ts'
@@ -0,0 +1,61 @@
TAP version 13
# Subtest: the seat
# Subtest: passes inside
ok 1 - passes inside
---
duration_ms: 0.577396
type: 'test'
...
# Subtest: fails inside
not ok 2 - fails inside
---
duration_ms: 1.621198
type: 'test'
location: '/tmp/1901k/node/suite/test/a.test.ts:5:3'
failureType: 'testCodeFailure'
error: |-
Expected values to be strictly deep-equal:
+ actual - expected
 {
+ holders: 2
- holders: 1
 }
code: 'ERR_ASSERTION'
name: 'AssertionError'
expected:
holders: 1
actual:
holders: 2
operator: 'deepStrictEqual'
stack: |-
TestContext.<anonymous> (file:///tmp/1901k/node/suite/test/a.test.ts:5:37)
Test.runInAsyncScope (node:async_hooks:226:14)
Test.run (node:internal/test_runner/test:1402:25)
Suite.processPendingSubtests (node:internal/test_runner/test:974:18)
Test.postRun (node:internal/test_runner/test:1542:19)
Test.run (node:internal/test_runner/test:1467:12)
async Promise.all (index 0)
async Suite.run (node:internal/test_runner/test:1905:7)
async startSubtestAfterBootstrap (node:internal/test_runner/harness:387:3)
...
1..2
not ok 1 - the seat
---
duration_ms: 3.295664
type: 'suite'
location: '/tmp/1901k/node/suite/test/a.test.ts:3:1'
failureType: 'subtestsFailed'
error: '1 subtest failed'
code: 'ERR_TEST_FAILURE'
...
1..1
# tests 2
# suites 1
# pass 1
# fail 1
# cancelled 0
# skipped 0
# todo 0
# duration_ms 115.948112
@@ -0,0 +1,2 @@
npm notice run test
npm notice run node --test --experimental-strip-types 'test/*.test.ts'
@@ -0,0 +1,47 @@
TAP version 13
# Subtest: passes in the first file
ok 1 - passes in the first file
---
duration_ms: 0.637465
type: 'test'
...
# node:internal/modules/run_main:111
# triggerUncaughtException(
# ^
# file:///tmp/1901k/node/syntax/test/b.test.ts:2
# test('never loads', () => {
# SyntaxError [ERR_INVALID_TYPESCRIPT_SYNTAX]: Expected ',', got '<eof>'
#  at parseTypeScript (node:internal/modules/typescript:61:36)
#  at processTypeScriptCode (node:internal/modules/typescript:129:37)
#  at Object.stripTypeScriptModuleTypes (node:internal/modules/typescript:172:22)
#  at stripTypeScriptModuleTypes (node:internal/modules/esm/translators:35:49)
#  at ModuleLoader.<anonymous> (node:internal/modules/esm/translators:688:29)
#  at \#translate (node:internal/modules/esm/loader:441:20)
#  at afterLoad (node:internal/modules/esm/loader:509:29)
#  at ModuleLoader.loadAndTranslate (node:internal/modules/esm/loader:514:12)
#  at \#getOrCreateModuleJobAfterResolve (node:internal/modules/esm/loader:567:36)
#  at afterResolve (node:internal/modules/esm/loader:614:52) {
# code: 'ERR_INVALID_TYPESCRIPT_SYNTAX'
# }
# Node.js v26.12.1
# Subtest: test/b.test.ts
not ok 2 - test/b.test.ts
---
duration_ms: 87.629815
type: 'test'
location: '/tmp/1901k/node/syntax/test/b.test.ts:1:1'
failureType: 'testCodeFailure'
exitCode: 1
signal: ~
error: 'test failed'
code: 'ERR_TEST_FAILURE'
...
1..2
# tests 2
# suites 0
# pass 1
# fail 1
# cancelled 0
# skipped 0
# todo 0
# duration_ms 106.551435
@@ -0,0 +1,2 @@
npm notice run test
npm notice run node --test --experimental-strip-types 'test/*.test.ts'
@@ -0,0 +1,26 @@
TAP version 13
# Subtest: passes before the wait
ok 1 - passes before the wait
---
duration_ms: 0.683015
type: 'test'
...
# Subtest: waits for an answer that never comes
not ok 2 - waits for an answer that never comes
---
duration_ms: 201.103775
type: 'test'
location: '/tmp/1901k/node/timeout/test/a.test.ts:3:1'
failureType: 'testTimeoutFailure'
error: 'test timed out after 200ms'
code: 'ERR_TEST_FAILURE'
...
1..2
# tests 2
# suites 0
# pass 1
# fail 0
# cancelled 1
# skipped 0
# todo 0
# duration_ms 308.920061
@@ -0,0 +1,2 @@
npm notice run test
npm notice run node --test --experimental-strip-types 'test/*.test.ts'
@@ -0,0 +1,49 @@
TAP version 13
# Subtest: passes in the first file
ok 1 - passes in the first file
---
duration_ms: 0.740124
type: 'test'
...
# judged against the seat as it stands
# Subtest: passes in the second file first
ok 2 - passes in the second file first
---
duration_ms: 0.916463
type: 'test'
...
# Subtest: fails in the second file
not ok 3 - fails in the second file
---
duration_ms: 1.719236
type: 'test'
location: '/tmp/1901k/node/two-files/test/b.test.ts:4:1'
failureType: 'testCodeFailure'
error: |-
one plus one is not three
2 !== 3
code: 'ERR_ASSERTION'
name: 'AssertionError'
expected: 3
actual: 2
operator: 'strictEqual'
stack: |-
TestContext.<anonymous> (file:///tmp/1901k/node/two-files/test/b.test.ts:6:10)
Test.runInAsyncScope (node:async_hooks:226:14)
Test.run (node:internal/test_runner/test:1402:25)
Test.processPendingSubtests (node:internal/test_runner/test:974:18)
Test.postRun (node:internal/test_runner/test:1542:19)
Test.run (node:internal/test_runner/test:1467:12)
async startSubtestAfterBootstrap (node:internal/test_runner/harness:387:3)
...
1..3
# tests 3
# suites 0
# pass 2
# fail 1
# cancelled 0
# skipped 0
# todo 0
# duration_ms 135.931121
@@ -0,0 +1 @@
seat.ts(2,14): error TS2322: Type 'number' is not assignable to type 'string'.
+201
View File
@@ -0,0 +1,201 @@
// Package conformance is the kit every build module's tests run (novox/hq issue 1901, its thorough fix): real output of
// the test runners, captured with the build nodes' own toolchains (captured/README.md), and what a build module must say
// of each failure in it under the failure contract (buildstep.Failure). A build module's test calls Run with its own parse
// path, the code that turns a runner's output into the failures of its test result; the kit fails the test when a
// failure is not named, its cause is missing or wrong, or a passing line stands in its place.
//
// A catalogue check (mesh-catalog checks/failurekit) refuses a build module with a test step whose tests do not call Run,
// and any build module whose tests do not call LeftRunning, the kit's case of steps a previous process left running
// (leftrunning.go, issue 1890).
package conformance
import (
"embed"
"fmt"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/buildstep"
)
// The runners the kit holds output of.
const (
// GoTest is `go test -json -count=1 ./...`: Output.Stdout is its output (go1.27.3).
GoTest = "go test -json"
// NodeTest is a package's `npm test` running `node --test --experimental-strip-types` with
// NODE_OPTIONS=buildstep.NodeTestOptions(<file>): Output.Report is the TAP file, Output.Stdout the script's own output
// and stderr, Output.Dir the package's directory as it was captured (node v26.12.1, npm 12.2.1).
NodeTest = "node --test"
// TypeCheck is tsc's type-check (`tsc --noEmit`, typescript 5.9.3): Output.Stdout is what it wrote; its failure
// names the test "type-check".
TypeCheck = "tsc"
)
// Output is one captured run of a runner, as a build module's parse path takes it.
type Output struct {
Stdout, Stderr string
// Report is what a runner wrote to a report file of its own (node --test's TAP), "" for the others.
Report string
// Dir is the directory the run was captured in: a node test's file is said relative to it.
Dir string
}
// Parse is a build module's own parse path: the failures its test result would carry for one run that failed.
type Parse func(Output) []buildstep.Failure
// Case is one captured run and what its failure must say.
type Case struct {
Name string
Runner string
// Test is the failing test the failure names (for a package that stopped, the tests that were running; "" for none).
Test string
// File, when set, is where the failing test is.
File string
// Cause holds what the failure's cause must contain.
Cause []string
// Passing are the names of tests that passed in the run, and what they logged: no failure's words may hold them.
Passing []string
}
//go:embed captured
var captured embed.FS
// Cases are the kit's cases of a runner.
func Cases(runner string) []Case {
var out []Case
for _, c := range cases {
if c.Runner == runner {
out = append(out, c)
}
}
return out
}
var cases = []Case{
{Name: "passfirst", Runner: GoTest, Test: "TestTheSecondPackageFails", Cause: []string{"the seat has 2 holders, want 1"},
Passing: []string{"TestTheFirstPackagePasses", "TestTheSecondPackagePassesFirst"}},
{Name: "subtests", Runner: GoTest, Test: "TestTheTable/fails", Cause: []string{"got 3 machines, want 4"}, Passing: []string{"TestTheTable/passes"}},
{Name: "logthenfatal", Runner: GoTest, Test: "TestThree", Cause: []string{"3 subscription(s) derive to a subject no emitter owns:"}},
{Name: "panicking", Runner: GoTest, Test: "TestPanics", Cause: []string{"panic: assignment to entry in nil map"}},
{Name: "timeout", Runner: GoTest, Test: "TestWaitsForever", Cause: []string{"panic: test timed out after 2s", "TestWaitsForever"}},
{Name: "deeptimeout", Runner: GoTest, Test: "TestHoldsTheSeat, TestWaitsForTheBus", Cause: []string{"panic: test timed out after 2s", "TestHoldsTheSeat", "TestWaitsForTheBus"}},
{Name: "buildfail", Runner: GoTest, Cause: []string{`cannot use "seat" (untyped string constant) as int value`}},
{Name: "exiting", Runner: GoTest, Test: "TestExits", Cause: []string{"exited", "TestExits"}},
{Name: "killed", Runner: GoTest, Test: "TestKilled", Cause: []string{"signal: killed"}},
{Name: "passthenfail", Runner: GoTest, Cause: []string{"panic: the bus closed after the tests: nats: connection closed"}},
{Name: "mainexit", Runner: GoTest, Cause: []string{"passed", "exited"}},
{Name: "race", Runner: GoTest, Test: "TestRacesOnTheSeat", Cause: []string{"DATA RACE", "Write at", "example.com/race.TestRacesOnTheSeat.func1()", "p_test.go:13", "created at example.com/race.TestRacesOnTheSeat()"}},
{Name: "subpanic", Runner: GoTest, Test: "TestTable/panics", Cause: []string{"panic: assignment to entry in nil map"}, Passing: []string{"TestTable/passes"}},
{Name: "racemain", Runner: GoTest, Cause: []string{"DATA RACE", "Write at", "example.com/racemain.TestMain.func1()", "p_test.go:15", "created at example.com/racemain.TestMain()"}},
{Name: "two-files", Runner: NodeTest, Test: "fails in the second file", File: "test/b.test.ts:4:1", Cause: []string{"one plus one is not three"},
Passing: []string{"passes in the first file", "passes in the second file first"}},
{Name: "suite", Runner: NodeTest, Test: "fails inside", File: "test/a.test.ts:5:3", Cause: []string{"Expected values to be strictly deep-equal:"},
Passing: []string{"passes inside"}},
{Name: "crash", Runner: NodeTest, Test: "test/b.test.ts", File: "test/b.test.ts:1:1", Cause: []string{"the bus closed under the test"},
Passing: []string{"passes before the crash"}},
{Name: "logcrash", Runner: NodeTest, Test: "test/b.test.ts", File: "test/b.test.ts:1:1", Cause: []string{"the bus closed under the test"},
Passing: []string{"passes and logs", "retrying after Error: the store was not ready yet"}},
{Name: "timeout", Runner: NodeTest, Test: "waits for an answer that never comes", File: "test/a.test.ts:3:1", Cause: []string{"test timed out after 200ms"},
Passing: []string{"passes before the wait"}},
{Name: "syntax", Runner: NodeTest, Test: "test/b.test.ts", File: "test/b.test.ts:1:1", Cause: []string{"SyntaxError", "Expected ',', got '<eof>'"},
Passing: []string{"passes in the first file"}},
{Name: "typeerror", Runner: TypeCheck, Test: "type-check", Cause: []string{"seat.ts(2,14): error TS2322: Type 'number' is not assignable to type 'string'."}},
}
// Load is the captured output of a case.
func Load(c Case) (Output, error) {
switch c.Runner {
case GoTest:
b, err := captured.ReadFile("captured/go/" + c.Name + ".jsonl")
return Output{Stdout: string(b)}, err
case NodeTest:
report, err := captured.ReadFile("captured/node/" + c.Name + ".tap")
if err != nil {
return Output{}, err
}
console, err := captured.ReadFile("captured/node/" + c.Name + ".console")
return Output{Report: string(report), Stdout: string(console), Dir: "/tmp/1901k/node/" + c.Name}, err
default:
b, err := captured.ReadFile("captured/tsc/" + c.Name + ".console")
return Output{Stdout: string(b), Dir: "/tmp/1901k/tsc/" + c.Name}, err
}
}
// Run runs every case of runner through parse, each as a subtest, and fails a case whose failure does not keep the
// contract: exactly one failure, naming the failing test (and its file, where the runner says it), with a cause that
// holds what the case says and is no passing line, an output that starts with the cause, stays within its bound and
// holds no passing line and no passing test's name.
func Run(t *testing.T, runner string, parse Parse) {
t.Helper()
cs := Cases(runner)
if len(cs) == 0 {
t.Fatalf("the kit holds no case of the runner %q", runner)
}
for _, c := range cs {
t.Run(c.Name, func(t *testing.T) {
out, err := Load(c)
if err != nil {
t.Fatal(err)
}
for _, problem := range Judge(c, parse(out)) {
t.Error(problem)
}
})
}
}
// Judge says what is wrong with the failures a parse path made of a case; nothing when they keep the contract.
func Judge(c Case, failed []buildstep.Failure) []string {
var problems []string
say := func(s string) { problems = append(problems, c.Runner+" "+c.Name+": "+s) }
if len(failed) != 1 {
say(fmt.Sprintf("want one failure, got %d: %s", len(failed), describe(failed)))
if len(failed) == 0 {
return problems
}
}
f := failed[0]
if f.Test != c.Test {
say("the failure names the test " + quote(f.Test) + ", want " + quote(c.Test))
}
if c.File != "" && f.File != c.File {
say("the failure's file is " + quote(f.File) + ", want " + quote(c.File))
}
if why := buildstep.Hidden(f); why != "" {
say("the failure hides its cause: " + why)
}
for _, want := range c.Cause {
if !strings.Contains(f.Cause, want) {
say("the cause " + quote(f.Cause) + " does not hold " + quote(want))
}
}
if first := strings.SplitN(f.Output, "\n", 2)[0]; strings.TrimSpace(first) != strings.TrimSpace(f.Cause) {
say("the output starts with " + quote(first) + ", not with the cause")
}
if len(f.Output) > buildstep.OutputBytes+buildstep.LineBytes+64 {
say(fmt.Sprintf("the output holds %d bytes, over its bound", len(f.Output)))
}
for _, l := range strings.Split(f.Output, "\n") {
if buildstep.PassingLine(l) {
say("the output keeps the passing line " + quote(l))
}
}
for _, p := range c.Passing {
if strings.Contains(f.Output, p+"\n") || strings.HasSuffix(f.Output, p) || strings.Contains(f.Cause, p) || strings.Contains(f.Output, "- "+p) {
say("the failure's words hold the passing test " + quote(p))
}
}
return problems
}
func describe(failed []buildstep.Failure) string {
var s []string
for _, f := range failed {
s = append(s, quote(f.Package+" "+f.Test)+" ("+quote(f.Cause)+")")
}
return strings.Join(s, ", ")
}
func quote(s string) string { return fmt.Sprintf("%q", s) }
@@ -0,0 +1,167 @@
package conformance
import (
"context"
"encoding/json"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/buildstep"
)
// The failure contract (issue 1901): the SDK's own parsers keep it on every captured case of the kit (1901/AC6); the
// kit refuses what the build modules sent before it; the contract's own rules; and the Go reader's older guarantees.
func TestTheFailureContract(t *testing.T) {
t.Run("the SDK parsers keep it", func(t *testing.T) {
Run(t, GoTest, func(o Output) []buildstep.Failure { return buildstep.ReadGoTest(o.Stdout, ".").Failed })
Run(t, NodeTest, func(o Output) []buildstep.Failure { return buildstep.ReadTAP(".", "test", o.Dir, o.Report, o.Stdout) })
Run(t, TypeCheck, func(o Output) []buildstep.Failure {
return []buildstep.Failure{buildstep.ConsoleFailure(".", "type-check", o.Stdout+o.Stderr)}
})
})
// 1890/AC7: the SDK's start sweep keeps the left-running case, and the kit refuses a sweep that ends nothing or ends
// everything.
t.Run("the left-running case", func(t *testing.T) {
LeftRunning(t, func(ctx context.Context, scan buildstep.Scan, node string, before time.Time, logf func(string, ...any)) {
buildstep.EndStaleAtStart(ctx, scan, node, before, logf)
})
if p := JudgeLeftRunning(func(context.Context, buildstep.Scan, string, time.Time, func(string, ...any)) {}); len(p) != 11 {
t.Errorf("a sweep that ends nothing: %d problems, want 11 (three steps not ended and unsaid, one owner unnamed, two asks not deleted and unsaid):\n%s", len(p), strings.Join(p, "\n"))
}
wrongNode := func(ctx context.Context, scan buildstep.Scan, _ string, before time.Time, logf func(string, ...any)) {
buildstep.EndStaleAtStart(ctx, scan, "kitother", before, logf)
}
if p := JudgeLeftRunning(wrongNode); len(p) == 0 {
t.Error("a sweep of another node's steps is not refused")
}
})
// node's output cut to its first fifteen lines with no cause, whose first line was a passing test of the first file
// (mesh-build-node's lines() before the contract), and a Go failure keeping its first line.
t.Run("the kit refuses a failure that hides its cause", func(t *testing.T) {
firstLines := func(s string) string {
var keep []string
for _, l := range strings.Split(s, "\n") {
if l = strings.TrimSpace(l); l != "" && len(keep) < 15 {
keep = append(keep, l)
}
}
return strings.Join(keep, "\n")
}
for _, c := range []struct {
runner, name string
old func(Output) []buildstep.Failure
}{
{NodeTest, "two-files", func(o Output) []buildstep.Failure {
return []buildstep.Failure{{Package: ".", Test: "test", Output: firstLines(o.Report)}}
}},
{GoTest, "passfirst", func(o Output) []buildstep.Failure {
return []buildstep.Failure{{Package: "example.com/passfirst/b", Test: "TestTheSecondPackageFails", Cause: "ok \texample.com/passfirst/a", Output: "--- PASS: TestTheSecondPackagePassesFirst"}}
}},
} {
for _, k := range Cases(c.runner) {
if k.Name != c.name {
continue
}
out, err := Load(k)
if err != nil {
t.Fatal(err)
}
if problems := Judge(k, c.old(out)); len(problems) == 0 {
t.Errorf("%s %s: the kit passed a failure that hides its cause", c.runner, c.name)
}
}
}
})
// What a passing line is (a test's own line starting "ok " is none), when a failure hides its cause, and that a long
// failure keeps its last lines within the bound and starts with a real line for an older checker, which reads only
// the first line of the output cut to 120 characters (1901/AC1, AC3).
t.Run("its rules", func(t *testing.T) {
for _, c := range []struct {
line string
passing bool
}{
{"✔ the same merge has the same id (8.0ms)", true}, {"ok 1 - passes", true}, {" ok 3 - inside", true},
{"ok \texample.com/a\t0.01s", true}, {"PASS", true}, {"--- PASS: TestA (0.00s)", true},
{" --- PASS: TestA/sub (0.00s)", true}, {"✖ fails (1ms)", false}, {"not ok 2 - fails", false},
{"okay, the seat moved", false}, {"ok then: the store refused the write", false}, {"ok the seat", false}, {"FAIL", false},
} {
if buildstep.PassingLine(c.line) != c.passing {
t.Errorf("PassingLine(%q) = %v, want %v", c.line, !c.passing, c.passing)
}
}
if buildstep.Hidden(buildstep.Failure{Cause: "the seat has 2 holders"}) != "" || buildstep.Hidden(buildstep.Failure{}) == "" ||
buildstep.Hidden(buildstep.Failure{Cause: "✔ passes (1ms)"}) == "" {
t.Error("Hidden does not tell a real cause from none and from a passing line")
}
var lines []string
for i := 0; i < 400; i++ {
lines = append(lines, " x_test.go:1: log line "+strings.Repeat("y", 40))
}
lines = append(lines, " x_test.go:9: the real cause", "--- FAIL: TestX (0.00s)")
oldFirst := func(out string) string { // the older checker's rule (judge.go firstLine before issue 1901)
for _, l := range strings.Split(out, "\n") {
if l = strings.TrimSpace(l); l != "" {
return l[:min(len(l), 120)]
}
}
return ""
}
for _, cause := range []string{"the real cause", ""} {
out := buildstep.Clip(lines, cause)
if len(out) > buildstep.OutputBytes+buildstep.LineBytes {
t.Errorf("1901/AC1: kept %d bytes, bound is %d", len(out), buildstep.OutputBytes)
}
if !strings.HasSuffix(out, "the real cause\n--- FAIL: TestX (0.00s)") || !strings.Contains(out, "\n... (") {
t.Errorf("1901/AC1: the last lines or the marker are lost: %q...%q", out[:40], out[len(out)-60:])
}
if got := oldFirst(out); strings.HasPrefix(got, "... (") || got == "" || (cause != "" && got != cause) {
t.Errorf("1901/AC3: cause %q: an older checker would read %q", cause, got)
}
}
})
// The Go reader (moved from mesh-build-go): an older checker, which knows no "cause" field, still reads a failure and
// gets at least the lines it kept (1901/AC3); a test's indented log line saying "panic:" is not the cause, go's own at
// column 0 is (1901/AC4); and each package counts its own skipped tests (issue 890).
t.Run("the Go reader", func(t *testing.T) {
read := func(name string) string {
raw, err := captured.ReadFile("captured/go/" + name + ".jsonl")
if err != nil {
t.Fatal(err)
}
return string(raw)
}
b, _ := json.Marshal(buildstep.ReadGoTest(read("logthenfatal"), ".").Failed)
var old []struct {
Output string `json:"output,omitempty"`
}
if err := json.Unmarshal(b, &old); err != nil || len(old) != 1 || !strings.Contains(old[0].Output, "judged against") || !strings.Contains(old[0].Output, "3 subscription(s)") {
t.Errorf("1901/AC3: an older checker reads %+v, %v", old, err)
}
lines := strings.SplitAfter(read("deeptimeout"), "\n")
added := `{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" p_test.go:9: the child said:\n"}` + "\n" +
`{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" panic: child retried and recovered\n"}` + "\n" +
`{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" WARNING: DATA RACE\n"}` + "\n"
if got := buildstep.ReadGoTest(strings.Join(lines[:10], "")+added+strings.Join(lines[10:], ""), ".").Failed; len(got) != 1 || !strings.HasPrefix(got[0].Cause, "panic: test timed out after 2s") {
t.Errorf("1901/AC4: failures = %+v, want the timeout's panic as the cause", got)
}
res := buildstep.ReadGoTest(`{"Action":"run","Package":"m/a","Test":"TestA"}
{"Action":"skip","Package":"m/a","Test":"TestA","Elapsed":0}
{"Action":"run","Package":"m/b","Test":"TestB"}
{"Action":"pass","Package":"m/b","Test":"TestB","Elapsed":0}
{"Action":"pass","Package":"m/a","Elapsed":0}
{"Action":"pass","Package":"m/b","Elapsed":0}
`, ".")
got := map[string]int{}
for _, p := range res.Packages {
got[p.Package] = p.Skipped
}
if res.Skipped != 1 || got["m/a"] != 1 || got["m/b"] != 0 {
t.Errorf("skipped = %d, per package %v, want 1 with m/a 1 and m/b 0", res.Skipped, got)
}
})
}
+212
View File
@@ -0,0 +1,212 @@
package conformance
import (
"context"
"encoding/json"
"fmt"
"sort"
"strings"
"sync"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// **A step left running by a previous process** (novox/hq issue 1890, note 21): when a delivery replaced mesh-build-go
// on a node while a check ask ran, the new process's start sweep let the ask's key stand as running, and it held one
// of the node's build slots until a hand end. Every build module runs this case (LeftRunning) through the start sweep
// its main runs: the hand-started steps a previous process left are ended at once and loud, its asks' keys deleted (the
// stream redelivers the ask, novox/hq ADR 0525 §6, and its asker keeps waiting for that), what this process runs kept. A
// catalogue check (mesh-catalog checks/failurekit) refuses a build module whose tests do not call LeftRunning.
// Sweep is a build module's own start sweep, as its main runs it once at its start: it ends node's steps in scan that
// no process of the module runs any more (buildstep.EndStaleAtStart, here with the runtime already serving), before
// being the start of the process, saying each end with logf.
type Sweep func(ctx context.Context, scan buildstep.Scan, node string, before time.Time, logf func(string, ...any))
// LeftNode is the node the case's steps are of.
const LeftNode = "kitnode"
// leftCase is one step in the steps state before the sweep, and whether the sweep must end it.
type leftCase struct {
id, tool, owner, state string
node string
age time.Duration // started this long before the process began (negative: after it)
ended bool
deleted bool // a gone process's ask: its key is deleted, not ended
}
// gone is a previous process's owner on the same node.
var gone = "b00t-of-the-kit.4242.mvp0000"
func leftCases() []leftCase {
return []leftCase{
{id: "mvpwlkzi7e6ca8", tool: "check ask", owner: gone, age: 5 * time.Minute, deleted: true}, // note 21's ghost
{id: "mvpwlkzi7e6ca9", tool: "delivery ask", owner: gone, age: -time.Second, deleted: true},
{id: "gonehand1", tool: "build_x_test", owner: gone, age: time.Minute, ended: true},
{id: "oldhand1", tool: "build_x_fetch", age: time.Hour, ended: true}, // no owner: an older build's
{id: "mvpwlkzi7e6caa", tool: "check ask", age: time.Hour, ended: true}, // no owner, past the bound
{id: "mvpwlkzi7e6cab", tool: "check ask", age: 5 * time.Minute}, // no owner, the stream's still
{id: "mvpwlkzi7e6cac", tool: "check ask", owner: buildstep.Owner, age: time.Minute}, // this process's
{id: "minehand1", tool: "build_x_test", owner: buildstep.Owner, age: -time.Second},
{id: "donehand1", tool: "build_x_test", owner: gone, age: time.Hour, state: delivery.StepPassed},
{id: "otherhand1", tool: "build_x_test", owner: gone, age: time.Hour, node: "kitother"},
}
}
// LeftRunning runs the case through sweep and fails the test for each step the sweep ended or kept wrongly, or ended
// without saying it.
func LeftRunning(t *testing.T, sweep Sweep) {
t.Helper()
for _, problem := range JudgeLeftRunning(sweep) {
t.Error(problem)
}
}
// JudgeLeftRunning says what is wrong with what sweep did to the case; nothing when it keeps the rule.
func JudgeLeftRunning(sweep Sweep) []string {
st := &steps{vals: map[string][]byte{}, revs: map[string]uint64{}}
before := time.Now()
for _, c := range leftCases() {
node, state := c.node, c.state
if node == "" {
node = LeftNode
}
if state == "" {
state = delivery.StepRunning
}
key, _ := delivery.StepKey(node, c.id)
raw, err := delivery.EncodeStep(delivery.Step{ID: c.id, Tool: c.tool, Node: node, State: state, Started: before.Add(-c.age).UTC(), Owner: c.owner})
if err != nil {
return []string{"the kit's step " + c.id + ": " + err.Error()}
}
st.set(key, raw)
}
served := make(chan struct{})
close(served)
was := buildstep.StaleServed
buildstep.StaleServed = func() <-chan struct{} { return served }
defer func() { buildstep.StaleServed = was }()
var mu sync.Mutex
var said []string
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
sweep(ctx, st.scan(), LeftNode, before, func(f string, a ...any) {
mu.Lock()
defer mu.Unlock()
said = append(said, fmt.Sprintf(f, a...))
})
mu.Lock()
log := strings.Join(said, "\n")
mu.Unlock()
var problems []string
for _, c := range leftCases() {
node := c.node
if node == "" {
node = LeftNode
}
key, _ := delivery.StepKey(node, c.id)
raw := st.get(key)
if c.deleted {
if raw != nil {
problems = append(problems, fmt.Sprintf("left running: the ask %s of the gone owner %q is still held after the sweep (%s); "+
"its key is deleted, never ended: the stream redelivers it (ADR 0525 §6)", key, c.owner, raw))
}
if !strings.Contains(log, key) || !strings.Contains(log, c.owner) {
problems = append(problems, "left running: the delete of "+key+" is not said with its owner "+c.owner)
}
continue
}
if raw == nil {
problems = append(problems, "left running: the step "+key+" is gone after the sweep, want it kept or ended")
continue
}
s, err := delivery.DecodeStep(raw)
if err != nil {
problems = append(problems, key+": "+err.Error())
continue
}
want := c.state
if want == "" {
want = delivery.StepRunning
}
if c.ended {
want = delivery.StepFailed
}
if s.State != want {
problems = append(problems, fmt.Sprintf("left running: the step %s (%s, owner %q, %s before the start) is %s after the sweep, want %s", key, c.tool, c.owner, c.age, s.State, want))
}
if c.ended && !strings.Contains(log, key) {
problems = append(problems, "left running: the end of "+key+" is not said")
}
if c.ended && c.owner != "" && !strings.Contains(log, c.owner) {
problems = append(problems, "left running: the end of "+key+" does not name its owner "+c.owner)
}
}
sort.Strings(problems)
return problems
}
// steps is a steps state with revisions, whose Update lands only at the revision read (stdio.ErrChanged otherwise).
type steps struct {
mu sync.Mutex
vals map[string][]byte
revs map[string]uint64
}
func (s *steps) set(key string, raw []byte) {
s.mu.Lock()
defer s.mu.Unlock()
s.vals[key], s.revs[key] = raw, s.revs[key]+1
}
func (s *steps) get(key string) []byte {
s.mu.Lock()
defer s.mu.Unlock()
return s.vals[key]
}
func (s *steps) scan() buildstep.Scan {
return buildstep.Scan{
Keys: func() ([]string, error) {
s.mu.Lock()
defer s.mu.Unlock()
var out []string
for k := range s.vals {
out = append(out, k)
}
return out, nil
},
Get: func(key string) (json.RawMessage, uint64, error) {
s.mu.Lock()
defer s.mu.Unlock()
return s.vals[key], s.revs[key], nil
},
Delete: func(key string, revision uint64) error {
s.mu.Lock()
defer s.mu.Unlock()
if s.revs[key] != revision {
return stdio.ErrChanged
}
delete(s.vals, key)
s.revs[key]++
return nil
},
Update: func(key string, value any, revision uint64) (uint64, error) {
raw, err := json.Marshal(value)
if err != nil {
return 0, err
}
s.mu.Lock()
defer s.mu.Unlock()
if s.revs[key] != revision {
return 0, stdio.ErrChanged
}
s.vals[key], s.revs[key] = raw, revision+1
return revision + 1, nil
},
}
}
+147
View File
@@ -0,0 +1,147 @@
package buildstep
// The failure contract of a build step's test level (novox/hq issue 1901, its thorough fix): what a build module says of
// each test that failed, the same for every language. Each build module used to parse its own test runner's output, and
// a fix in one never reached the others: a failed Go test kept a t.Logf line, a failed node --test run kept a passing
// '✔' line of its first file. The parsers that fill the contract live here (gotest.go, tap.go) and the conformance kit
// (package conformance) holds every build module to them with captured runner output.
import (
"fmt"
"strings"
"git.novox.be/novox/mesh-sdk/go/sdkversion"
)
// Failure is one test that failed, or a package or file that failed with no test to name (a build that failed, a test
// program that panicked, a test file that did not load). The JSON names are the ones mesh-build-go and the checker have
// always used; File is new and an older reader ignores it.
type Failure struct {
// Package is the Go package's import path, or the TypeScript package's directory in the workspace.
Package string `json:"package"`
// Test names the failing test, or the tests that were running when the program stopped; "" when none can be named.
Test string `json:"test,omitempty"`
// File is where the failing test is (a Go test's file and line is in its Cause; node names the test's file:line:col).
File string `json:"file,omitempty"`
// Cause is the one line that says why: the assertion's or the error's first line, a "panic:" line, a build's first
// error, or words saying how the program stopped. **Never empty, and never a passing line** (PassingLine).
Cause string `json:"cause,omitempty"`
// Output is the failing test's own output, its last lines up to OutputBytes, Cause first: an older checker reads
// only its first line.
Output string `json:"output,omitempty"`
}
// OutputBytes is the most a failure keeps of its output, and LineBytes the most of one line: four kilobytes holds a long
// assertion message and stays far under a bus message's or a verdict's bound.
const (
OutputBytes = 4096
LineBytes = 1024
)
// By names a build module and the SDK it was built with, for a result's "by": the checker names it when a failure hides
// its cause ("the build module hid the failure (<by>)").
func By(module string) string {
return module + " with mesh-sdk " + sdkversion.Version()
}
// PassingLine says a line is a test runner's word for a test or a package that passed: node's '✔', TAP's "ok N" (at any
// indent), go test's "ok \t<pkg>", "PASS" and "--- PASS". A line of a test's own that starts "ok " is none of these. Such a line is never a failure's cause, nor part of a failure's output.
func PassingLine(l string) bool {
if strings.HasPrefix(l, "ok \t") { // go test's package line, at column 0
return true
}
t := strings.TrimSpace(l)
switch {
case t == "PASS", strings.HasPrefix(t, "✔"), strings.HasPrefix(t, "--- PASS"):
return true
}
return tapOk(t)
}
// Hidden says how a failure hides why it failed, or "" when it does not: it has no cause, or its cause is a passing line.
// The checker says it loudly (issue 1901): a build module that sends such a failure does not keep the contract.
func Hidden(f Failure) string {
c := strings.TrimSpace(f.Cause)
switch {
case c == "":
return "it sent no cause"
case PassingLine(c):
return fmt.Sprintf("its cause is a passing line (%q)", CutLine(c))
}
return ""
}
// Clip joins the last lines of a failure's output that fit OutputBytes, each cut to LineBytes, and says how many earlier
// lines it left out. **Its first line is the cause**, put ahead when the kept lines do not start with it, or, with no
// cause and lines left out, the first non-empty line, then the marker: an older checker reads only the first line.
// Passing lines are dropped.
func Clip(lines []string, cause string) string {
own := make([]string, 0, len(lines))
for _, l := range lines {
if !PassingLine(l) {
own = append(own, l)
}
}
lines = own
size, from := 0, len(lines)
for from > 0 {
n := len(CutLine(lines[from-1])) + 1
if size+n > OutputBytes && from < len(lines) {
break
}
size += n
from--
}
kept := make([]string, 0, len(lines)-from+2)
head := strings.TrimSpace(cause)
for i := 0; head == "" && i < from; i++ {
head = strings.TrimSpace(lines[i])
}
if head != "" && (from > 0 || len(lines) == 0 || strings.TrimSpace(lines[from]) != head) {
kept = append(kept, CutLine(head))
}
if from > 0 {
kept = append(kept, fmt.Sprintf("... (%d earlier lines)", from))
}
for _, l := range lines[from:] {
kept = append(kept, CutLine(l))
}
return strings.Join(kept, "\n")
}
// CutLine cuts a line to LineBytes, on a character boundary.
func CutLine(l string) string {
if len(l) <= LineBytes {
return l
}
return strings.ToValidUTF8(l[:LineBytes], "") + "..."
}
// ConsoleFailure is the failure of a command that is no test runner (go vet, gofmt, tsc, a build script): its cause is the
// first line that names an error (tsc's "error TS", go vet's "file.go:12:3: ..."), else the first line that is not a
// heading ("# <package>", npm's "> script"), and its output the command's lines without passing ones.
func ConsoleFailure(pkg, test, out string) Failure {
var lines []string
for _, l := range strings.Split(strings.ReplaceAll(stripANSI(out), "\r\n", "\n"), "\n") {
if t := strings.TrimSpace(l); t != "" && !PassingLine(t) {
lines = append(lines, strings.TrimRight(l, " \t"))
}
}
cause := ""
for _, l := range lines {
if t := strings.TrimSpace(l); strings.Contains(t, "error") || strings.Contains(t, "Error") || goPosition.MatchString(t) {
cause = t
break
}
}
for _, l := range lines {
if t := strings.TrimSpace(l); cause == "" && !strings.HasPrefix(t, "# ") && !strings.HasPrefix(t, "> ") && !strings.HasPrefix(t, "npm ") {
cause = t
}
}
if cause == "" {
cause = test + " failed and wrote nothing that says why"
}
cause = CutLine(cause)
return Failure{Package: pkg, Test: test, Cause: cause, Output: Clip(lines, cause)}
}
+409
View File
@@ -0,0 +1,409 @@
package buildstep
// The reader of `go test -json` (moved whole from mesh-catalog's mesh-build-go, issue 1901's thorough fix): per package
// what ran and for how long, and each failure under the contract (failure.go), its real cause too when the failure is no
// test's assertion (a panic, a timeout, a build error, an exit, a kill, a race).
import (
"bufio"
"encoding/json"
"fmt"
"regexp"
"slices"
"strings"
)
// GoPackage is one package's part of a Go test level.
type GoPackage struct {
Dir string `json:"dir"`
Package string `json:"package"`
Passed bool `json:"passed"`
Elapsed float64 `json:"elapsed"`
Tests int `json:"tests"`
// Skipped is how many of the package's tests skipped.
Skipped int `json:"skipped,omitempty"`
// NoTests is a package with no test files (at this level: a file with the other level's build tag is none).
NoTests bool `json:"no_tests,omitempty"`
}
// TestTime is one test and the seconds it took.
type TestTime struct {
Package string `json:"package"`
Test string `json:"test"`
Elapsed float64 `json:"elapsed"`
}
// GoTest is what one go.mod directory's `go test -json` said.
type GoTest struct {
Packages []GoPackage
Tests int
Skipped int
Times []TestTime
Failed []Failure
// Elapsed is the slowest package's running time in seconds.
Elapsed float64
// Noise is the lines that were no event: what go said when it could not run the tests at all.
Noise []string
}
// ReadGoTest reads the output of `go test -json` run in the go.mod directory dir.
func ReadGoTest(out, dir string) GoTest {
r := &goReader{dir: dir, pkgs: map[string]*GoPackage{}, builds: map[string][]string{}, pkgOut: map[string][]string{},
testOut: map[string][]string{}, causes: map[string]string{}, failedIn: map[string]bool{},
running: map[string][]string{}, passed: map[string]bool{}, built: map[string]bool{}}
sc := bufio.NewScanner(strings.NewReader(out))
sc.Buffer(make([]byte, 1<<20), 1<<26)
for sc.Scan() {
var ev goEvent
if err := json.Unmarshal(sc.Bytes(), &ev); err != nil {
if l := strings.TrimSpace(sc.Text()); l != "" {
r.res.Noise = append(r.res.Noise, l)
}
continue
}
r.event(ev)
}
r.finish()
return r.res
}
// goEvent is one line of `go test -json` (test2json, and the build events go test adds).
type goEvent struct {
Action string
Package string
Test string
Output string
OutputType string // "error" for a t.Error or t.Fatal message, "error-continue" for its further lines
ImportPath string
FailedBuild string
Elapsed float64
}
type goReader struct {
res GoTest
dir string
pkgs map[string]*GoPackage
order []string
builds map[string][]string // build output by import path
pkgOut map[string][]string
testOut map[string][]string // by package + "\x00" + test
causes map[string]string // the first error line of a test, by the same key
failedIn map[string]bool // packages with a failed test
// running is the tests that started and have not ended, by package, in the order they started: when a package
// fails with no failed test, these were running when its test program stopped.
running map[string][]string
// passed says a package's test program printed "PASS" before it failed; built says it failed to build.
passed, built map[string]bool
}
func (r *goReader) pkg(name string) *GoPackage {
p := r.pkgs[name]
if p == nil {
p = &GoPackage{Dir: r.dir, Package: name}
r.pkgs[name] = p
r.order = append(r.order, name)
}
return p
}
func keepLine(lines []string, out string) []string {
if t := strings.TrimRight(out, "\r\n"); t != "" && !strings.HasPrefix(t, "=== ") {
lines = append(lines, t)
}
return lines
}
func (r *goReader) event(ev goEvent) {
switch {
case ev.Package == "" && ev.ImportPath != "":
if ev.Action == "build-output" {
r.builds[ev.ImportPath] = keepLine(r.builds[ev.ImportPath], ev.Output)
}
case ev.Package == "":
case ev.Test == "":
p := r.pkg(ev.Package)
switch ev.Action {
case "output":
// go test's own summary of a package is never a failure's words: a "PASS" before a "FAIL <pkg>" says only
// that the tests passed and the program failed after them.
switch l := strings.TrimRight(ev.Output, "\r\n"); {
case l == "PASS":
r.passed[ev.Package] = true
case goSummaryLine(l):
default:
r.pkgOut[ev.Package] = keepLine(r.pkgOut[ev.Package], ev.Output)
}
case "pass":
p.Passed, p.Elapsed = true, ev.Elapsed
case "fail":
p.Passed, p.Elapsed = false, ev.Elapsed
if ev.FailedBuild != "" {
r.built[ev.Package] = true
r.pkgOut[ev.Package] = append(r.builds[ev.FailedBuild], r.pkgOut[ev.Package]...)
}
case "skip":
p.Passed, p.NoTests = true, true
}
default:
key := ev.Package + "\x00" + ev.Test
switch ev.Action {
case "output":
r.testOut[key] = keepLine(r.testOut[key], ev.Output)
if ev.OutputType == "error" && r.causes[key] == "" {
r.causes[key] = goErrorWords(ev.Output)
}
case "run":
r.running[ev.Package] = append(r.running[ev.Package], ev.Test)
case "pass", "fail", "skip":
r.running[ev.Package] = slices.DeleteFunc(r.running[ev.Package], func(t string) bool { return t == ev.Test })
p := r.pkg(ev.Package)
p.Tests++
r.res.Tests++
if ev.Action == "skip" {
r.res.Skipped++
p.Skipped++
}
r.res.Times = append(r.res.Times, TestTime{Package: ev.Package, Test: ev.Test, Elapsed: ev.Elapsed})
if ev.Action == "fail" {
r.failedIn[ev.Package] = true
if r.subtestFailed(ev.Package, ev.Test) {
return // a parent failed only by its subtest is that subtest's failure, said once
}
if r.subtestTakesTheCause(ev.Package, ev.Test) {
return // a subtest's panic, which go files under its parent, is said on the subtest
}
r.res.Failed = append(r.res.Failed, r.testFailure(ev.Package, ev.Test))
}
}
}
}
// subtestFailed says a failure of test under pkg was already reported for one of its subtests (they end first), each with
// a cause of its own, and the test reported no error and no runtime cause (a subtest's panic) of its own.
func (r *goReader) subtestFailed(pkg, test string) bool {
key := pkg + "\x00" + test
if r.causes[key] != "" || goRuntimeCause(r.testOut[key]) != "" {
return false
}
for _, f := range r.res.Failed {
if f.Package == pkg && strings.HasPrefix(f.Test, test+"/") {
return true
}
}
return false
}
// noMessage is the cause of a test that failed and said nothing of why.
const noMessage = "the test failed without an error message (t.Fail or t.FailNow)"
// subtestTakesTheCause gives a parent's runtime cause (a subtest's panic: go test -json files its lines under the
// parent) to the subtests of that parent that failed with no message of their own, and says it did. A parent with an
// error line of its own, or whose failed subtests each said why, is reported as itself.
func (r *goReader) subtestTakesTheCause(pkg, test string) bool {
key := pkg + "\x00" + test
if r.causes[key] != "" {
return false
}
cause := goRuntimeCause(r.testOut[key])
if cause == "" {
return false
}
took := false
for i, f := range r.res.Failed {
if f.Package == pkg && strings.HasPrefix(f.Test, test+"/") && f.Cause == noMessage {
r.res.Failed[i].Cause, r.res.Failed[i].Output = cause, Clip(r.testOut[key], cause)
took = true
}
}
return took
}
// testFailure is a test that failed: its cause is its first error line, the race it ran into, the runtime's line, or
// words saying it failed without a message.
func (r *goReader) testFailure(pkg, test string) Failure {
key := pkg + "\x00" + test
cause := r.causes[key]
if race := goRaceCause(r.testOut[key]); race != "" && (cause == "" || strings.HasPrefix(cause, "race detected")) {
cause = race // the race itself, not testing's "race detected during execution of test"
} else if cause == "" {
cause = goRuntimeCause(r.testOut[key])
}
if cause == "" {
cause = noMessage
}
return Failure{Package: pkg, Test: test, Output: Clip(r.testOut[key], cause), Cause: cause}
}
// goSummaryLine says a line is go test's summary of a package, in go's exact forms only: "PASS" or "FAIL" alone,
// "ok \t<pkg>\t<time>" and "FAIL\t<pkg>\t<time>" (a line of the package's own that starts "ok " is kept).
func goSummaryLine(l string) bool {
return l == "PASS" || l == "FAIL" || strings.HasPrefix(l, "ok \t") || strings.HasPrefix(l, "FAIL\t")
}
// goRuntimeCause is the cause in a failure's output that is not a test's assertion: a race the race detector reported;
// else the first "panic:" line, a timeout's with the tests it names as running; else the first "fatal error:" or
// "signal:" line; "" when there is none. It reads every line, so a long goroutine dump cannot hide it. **It matches only
// lines at column 0**, where go prints the runtime's: a test's indented log line holding the same words is not the cause.
func goRuntimeCause(lines []string) string {
if race := goRaceCause(lines); race != "" {
return race
}
for _, prefix := range []string{"panic: ", "fatal error: ", "signal: "} {
for i, l := range lines {
l = strings.TrimRight(l, "\r\n")
if !strings.HasPrefix(l, prefix) {
continue
}
if strings.HasPrefix(l, "panic: test timed out after") {
var names []string
for _, n := range lines[i+1:] {
if !strings.HasPrefix(n, "\t\t") {
if strings.TrimSpace(n) == "running tests:" {
continue
}
break
}
names = append(names, strings.TrimSpace(n))
}
if len(names) > 0 {
l += "; running tests: " + strings.Join(names, ", ")
}
}
return CutLine(l)
}
}
return ""
}
// goRaceCause is the first race the race detector reported in a failure's output, in one line: its access and the frame
// that made it, and where the goroutine that made it was created. "" when the output holds no race.
func goRaceCause(lines []string) string {
at := -1
for i, l := range lines {
if strings.TrimRight(l, "\r\n") == "WARNING: DATA RACE" {
at = i
break
}
}
if at < 0 {
return ""
}
frame := func(i int) string {
var fn, file string
for _, l := range lines[i+1:] {
if l = strings.TrimSpace(l); l == "" {
break
}
if fn == "" {
fn = l
continue
}
file = l
if j := strings.LastIndexByte(file, '/'); j >= 0 {
file = file[j+1:]
}
if j := strings.Index(file, " +0x"); j >= 0 {
file = file[:j]
}
break
}
if file != "" {
return fn + " (" + file + ")"
}
return fn
}
words := "WARNING: DATA RACE"
for i := at + 1; i < len(lines); i++ {
l := strings.TrimSpace(lines[i])
if l == "" || strings.HasPrefix(l, "===") {
continue
}
words += ": " + strings.TrimSuffix(l, ":")
if f := frame(i); f != "" {
words += " in " + f
}
break
}
for i := at + 1; i < len(lines); i++ {
l := strings.TrimSpace(lines[i])
if strings.HasPrefix(l, "==================") {
break
}
if strings.HasPrefix(l, "Goroutine ") && strings.HasSuffix(l, " created at:") {
if f := frame(i); f != "" {
words += "; its goroutine created at " + f
}
break
}
}
return CutLine(words)
}
// goBuildCause is the first error line of a failed build: the first line that is not go's "# <package>" heading.
func goBuildCause(lines []string) string {
for _, l := range lines {
if l = strings.TrimSpace(l); l != "" && !strings.HasPrefix(l, "# ") {
return CutLine(l)
}
}
return ""
}
// goTestNames names the tests that were running, at most three, and how many more.
func goTestNames(tests []string) string {
if len(tests) <= 3 {
return strings.Join(tests, ", ")
}
return fmt.Sprintf("%s and %d more", strings.Join(tests[:3], ", "), len(tests)-3)
}
// packageFailure is a package that failed with no failed test to name it: its test program did not build, panicked,
// timed out, was killed or exited. It names the tests that were running, keeps their output with the package's, and
// says the cause.
func (r *goReader) packageFailure(name string) Failure {
running := r.running[name]
var lines []string
for _, t := range running {
lines = append(lines, r.testOut[name+"\x00"+t]...)
}
lines = append(lines, r.pkgOut[name]...)
cause := goRuntimeCause(lines)
switch {
case cause != "":
case r.built[name]:
cause = goBuildCause(lines)
case len(running) > 0:
cause = "the test program exited while " + goTestNames(running) + " ran, with no panic: an os.Exit or a signal (go test -json reports no exit status)"
case r.passed[name]:
cause = "the tests passed, then the test program exited with a failure: an os.Exit or a panic in TestMain or after the tests (go test -json reports no exit status)"
}
if cause == "" {
cause = "the package failed with no test to name and no panic, signal or build error in its output"
}
return Failure{Package: name, Test: goTestNames(running), Output: Clip(lines, cause), Cause: cause}
}
var goFileAndLine = regexp.MustCompile(`^\S+\.go:\d+: `)
// goPosition matches a line that starts with a Go file's position, as go vet and the compiler write an error.
var goPosition = regexp.MustCompile(`^\S+\.go:\d+(:\d+)?: `)
// goErrorWords is the words of an error line of a test: trimmed, without the file and line go test put before them.
func goErrorWords(out string) string {
return strings.TrimSpace(goFileAndLine.ReplaceAllString(strings.TrimSpace(out), ""))
}
// finish moves what was read into the result: the packages in order, a failure for each failed package that has no
// failed test to name it.
func (r *goReader) finish() {
for _, name := range r.order {
p := r.pkgs[name]
if !p.Passed && !r.failedIn[name] {
r.res.Failed = append(r.res.Failed, r.packageFailure(name))
}
if !p.NoTests && p.Elapsed > r.res.Elapsed {
r.res.Elapsed = p.Elapsed
}
r.res.Packages = append(r.res.Packages, *p)
}
}
+6 -16
View File
@@ -1,24 +1,14 @@
package buildstep
import "strings"
import "git.novox.be/novox/mesh-sdk/go/catalogue"
// LauncherSuffix is what a TypeScript entrypoint's launcher is called beside it: index.js is started as
// index.serve.mjs (novox/hq ADR 0193), an ES module by its own extension whatever the bundle's package.json says.
const LauncherSuffix = ".serve.mjs"
const LauncherSuffix = catalogue.LauncherSuffix
// Launchers are the launchers a bundle's build writes, by entrypoint, relative to the bundle's root: for TypeScript
// every entrypoint `x.js` is started by `x.serve.mjs` beside it; an entrypoint that is not JavaScript and every other
// language, whose build is already executable, get none. The one rule mesh-build-node, build-agent and the controller
// share (R4e), so the name a build writes is the name a reader expects.
// Launchers are the launchers a bundle's build writes, by entrypoint, relative to the bundle's root
// (catalogue.Launchers): the one rule mesh-build-node, build-agent, the controller and a node taking in a build share
// (R4e, issue 1126), so the name a build writes is the name a reader expects.
func Launchers(language string, entrypoints []string) map[string]string {
out := map[string]string{}
if language != "typescript" {
return out
}
for _, entry := range entrypoints {
if strings.HasSuffix(entry, ".js") {
out[entry] = strings.TrimSuffix(entry, ".js") + LauncherSuffix
}
}
return out
return catalogue.Launchers(language, entrypoints)
}
+67
View File
@@ -0,0 +1,67 @@
package buildstep
import (
"context"
"strings"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// **A running ask says its phases** (issue 1890, 2026-10-26 19:09: a mesh-catalog try's step on shanks ran 30 minutes
// and its module's journal held no line naming its ask, so nobody could tell whether it fetched, linted or tested). The
// puller hands every ask's run a phase reporter in its context; RunAsk and a module's own step bodies call Phase as they
// move (fetch and merge, a level's digest, lint, a level's tests), and each phase is logged with the ask's id and its
// time since the start, and written to the ask's running step key (delivery.Step.Phase), so its asker and the delivery
// view read where it stands from the bus.
type phaseKey struct{}
// Phase says that the ask whose run ctx belongs to entered phase; nothing outside an ask's run (a tool called by hand).
func Phase(ctx context.Context, phase string) {
if r, ok := ctx.Value(phaseKey{}).(*phases); ok && r != nil {
r.enter(strings.TrimSpace(phase))
}
}
// phases is one running ask's reporter: it writes the step key again with each new phase until the step ends.
type phases struct {
mu sync.Mutex
put Put
key string
step delivery.Step
ended bool
logf func(format string, args ...any)
prefix string
}
func (r *phases) enter(phase string) {
if phase == "" {
return
}
r.mu.Lock()
defer r.mu.Unlock()
if r.ended || phase == r.step.Phase {
return
}
at := now().UTC()
r.step.Phase, r.step.PhaseAt = phase, at
if r.logf != nil {
r.logf("%s: %s, %s after its start", r.prefix, phase, at.Sub(r.step.Started).Round(time.Second))
}
if r.put == nil {
return
}
if err := write(r.put, r.key, r.step); err != nil && r.logf != nil {
r.logf("%s: the phase %s could not be written to %s: %v", r.prefix, phase, r.key, err)
}
}
// end stops the writes (the step's end is written after it) and answers the step with its last phase, for the end.
func (r *phases) end() delivery.Step {
r.mu.Lock()
defer r.mu.Unlock()
r.ended = true
return r.step
}
+201 -13
View File
@@ -77,6 +77,15 @@ type Puller struct {
// for this node, and wakes the puller on each change (watchCapacity): WatchConfigServed in a module, a test's own
// double. Until it hands a value the puller judges by Capacity.
WatchConfig func(key string, handle func(stdio.ProvisionChange) error) error
// CoresPerStep is what one ask of this module needs, read at every judgement (NodeCoresPerStep); with it the capacity
// judged by is bounded by the node's build slice: min(the configured capacity, floor(the quota's cores / CoresPerStep))
// (EffectiveCapacity). Nil bounds nothing by the CPU. Quota reads the quota (Quota when nil; a test's own).
CoresPerStep func() float64
Quota func() (percent int, from string)
// BuildHours is the node's build window, read at every judgement (NodeBuildHours); with WatchConfig set the value
// mesh-config resolves for `build-hours` takes its place once handed. Outside the window the capacity judged by is 0:
// steps running finish, no new one is taken; the puller wakes at the window's next edge. Nil builds at any hour.
BuildHours func() (*BuildWindow, string)
// Has says whether this node has a need an ask names (delivery.AskNeeds, issue 890). An ask needing what it lacks is
// declined: its step ends refused as declined (delivery.StepDeclined), the message is dropped for its asker to put
// again, and the worker it came from is not looked at again until the next pull's wait, so a node lacking the need
@@ -92,6 +101,16 @@ type Puller struct {
// watched is the capacity the config watch handed last, used in place of Capacity once hasWatched.
watched int
hasWatched bool
// hours is the build window the config watch handed last, and hoursFault why its value cannot be read; used in place
// of BuildHours once hasHours.
hours *BuildWindow
hoursFault string
hasHours bool
// judged is the capacity said last (log and capacity.<node>), so a change is said once.
judged delivery.Capacity
hasJudged bool
capWritten bool
capFault string
// declined are the kinds whose worker handed an ask this node declined, left out of the pulls without a wait until
// the next pull's wait ends.
declined map[string]bool
@@ -216,23 +235,128 @@ func (p *Puller) room() bool {
// roomNow is room, with the capacity and the asks in hand it was judged on.
func (p *Puller) roomNow() (ok bool, capacity, inHand int) {
p.mu.Lock()
watched, has := p.watched, p.hasWatched
p.mu.Unlock()
if has {
capacity = watched
} else if p.Capacity != nil {
capacity = p.Capacity()
}
capacity = p.CapacityDetail().Effective
p.mu.Lock()
defer p.mu.Unlock()
return !p.paused && p.inHand < capacity, capacity, p.inHand
}
// CapacityNow is the capacity the puller judges by: the config watch's last value, else Capacity.
// CapacityNow is the capacity the puller judges by (CapacityDetail's Effective).
func (p *Puller) CapacityNow() int {
_, capacity, _ := p.roomNow()
return capacity
return p.CapacityDetail().Effective
}
// CapacityDetail is the capacity the puller judges by now and why: the configured one (the config watch's last value,
// else Capacity), bounded by the node's build slice when CoresPerStep is set (EffectiveCapacity), its quota read now. A
// change from what it said last is logged and written to `capacity.<node>` of the pauses state (PausePut), so the
// delivery view says it too.
func (p *Puller) CapacityDetail() delivery.Capacity {
p.mu.Lock()
watched, has := p.watched, p.hasWatched
p.mu.Unlock()
configured, from := 0, "the settings file"
switch {
case has:
configured, from = watched, "mesh-config, as the config watch handed it"
case p.Capacity != nil:
configured = p.Capacity()
}
var c delivery.Capacity
if p.CoresPerStep == nil {
c = delivery.Capacity{Configured: configured, Effective: configured, QuotaFrom: "not judged: the module names no cores a step needs",
Why: fmt.Sprintf("its configured capacity %d", configured)}
} else {
quota := p.Quota
if quota == nil {
quota = Quota
}
percent, qfrom := quota()
c = EffectiveCapacity(configured, percent, qfrom, p.CoresPerStep())
}
c.ConfiguredFrom = from
w, fault := p.window()
if fault != "" {
c.BuildHoursFault = fault
c.Why = fmt.Sprintf("%s; at any hour, because its build hours cannot be read: %s", c.Why, fault)
}
if w != nil {
at := now()
zone, _ := at.Zone()
c.BuildHours = w.String() + " " + zone
if !w.Open(at) && c.Effective > 0 {
c.OutsideHours = true
c.Why = fmt.Sprintf("0, outside its build hours %s until %s (otherwise %s)", c.BuildHours, w.NextEdge(at).Format("15:04"), c.Why)
c.Effective = 0
} else if c.Effective > 0 {
c.Why = fmt.Sprintf("%s; within its build hours %s until %s", c.Why, c.BuildHours, w.NextEdge(at).Format("15:04"))
}
}
p.sayCapacity(c)
return c
}
// window is the node's build window now (the config watch's last value, else BuildHours, else none), and why its value
// cannot be read when it cannot.
func (p *Puller) window() (*BuildWindow, string) {
p.mu.Lock()
w, fault, has := p.hours, p.hoursFault, p.hasHours
p.mu.Unlock()
if has {
return w, fault
}
if p.BuildHours != nil {
return p.BuildHours()
}
return nil, ""
}
// untilEdge is how long until the build window's next edge, and false when the node has no window.
func (p *Puller) untilEdge() (time.Duration, bool) {
w, _ := p.window()
if w == nil {
return 0, false
}
at := now()
return w.NextEdge(at).Sub(at), true
}
// sayCapacity logs c when it differs from what was said last, and writes it to the node's capacity key until a write
// lands (a failed write is tried again at the next judgement, its error said once).
func (p *Puller) sayCapacity(c delivery.Capacity) {
p.mu.Lock()
same := p.hasJudged && p.judged.Effective == c.Effective && p.judged.Configured == c.Configured && p.judged.ByCPU == c.ByCPU &&
p.judged.CoresPerStep == c.CoresPerStep && p.judged.QuotaCores == c.QuotaCores && p.judged.OutsideHours == c.OutsideHours &&
p.judged.BuildHours == c.BuildHours && p.judged.BuildHoursFault == c.BuildHoursFault
if !same {
p.judged, p.hasJudged, p.capWritten = c, true, false
}
write := !p.capWritten && p.PausePut != nil
p.mu.Unlock()
if !same {
p.logf("the %s build capacity of %s is %s (configured from %s)", p.Language, p.Node, c.Why, c.ConfiguredFrom)
}
if !write {
return
}
key, err := delivery.CapacityKey(p.Node)
if err != nil {
return
}
c.At = now()
raw, err := delivery.EncodeCapacity(c)
if err == nil {
_, err = p.PausePut(key, json.RawMessage(raw))
}
p.mu.Lock()
defer p.mu.Unlock()
if err == nil {
p.capWritten, p.capFault = true, ""
return
}
if err.Error() != p.capFault {
p.capFault = err.Error()
p.logf("the %s build capacity of %s could not be written to %s (tried again at the next judgement): %v", p.Language, p.Node, key, err)
}
}
// Pull takes and runs asks until ctx ends. An ask still running then is neither ended nor acknowledged: the stream
@@ -248,6 +372,7 @@ func (p *Puller) Pull(ctx context.Context) error {
p.logf("the pause of %s cannot be read: %v; pulling as not paused", p.Node, err)
}
go p.watchCapacity() // WatchConfigServed waits for Serve, which a module starts after its pull
go p.watchHours()
wait := p.PullWait
if wait <= 0 {
wait = DefaultPullWait
@@ -260,16 +385,29 @@ func (p *Puller) Pull(ctx context.Context) error {
// Parked until something wakes it: a capacity change (the config watch), a resume, or a step's end. Every
// park logs once, so a puller that sits idle says why (issue 1887).
p.logf("the %s puller on %s parks: %d of capacity %d in hand, paused %v", p.Language, p.Node, inHand, capacity, p.Paused())
var edge <-chan time.Time // the build window's next edge, when the node has one: one timer, not a poll
stop := func() bool { return false }
if d, ok := p.untilEdge(); ok {
t := time.NewTimer(d + time.Second)
edge, stop = t.C, t.Stop
}
select {
case <-ctx.Done():
stop()
return ctx.Err()
case <-p.wake:
case <-edge:
}
stop()
_, capacity, inHand = p.roomNow()
p.logf("the %s puller on %s wakes: %d of capacity %d in hand, paused %v", p.Language, p.Node, inHand, capacity, p.Paused())
continue
}
msg, err := p.next(ctx, wait)
pullWait := wait
if d, ok := p.untilEdge(); ok && d < pullWait {
pullWait = d // a pull does not wait past the window's close
}
msg, err := p.next(ctx, pullWait)
if err != nil {
if ctx.Err() != nil {
return ctx.Err()
@@ -395,6 +533,7 @@ func (p *Puller) take(ctx context.Context, msg *Pulled) {
}
p.running[ask.ID] = RunningAsk{ID: ask.ID, Kind: ask.Kind, Module: module, Started: step.Started}
p.mu.Unlock()
p.logf("the %s ask %s starts on %s: %s", ask.Kind, ask.ID, p.Node, step.What)
go p.run(ctx, msg, ask, key, step)
}
@@ -425,7 +564,11 @@ func (p *Puller) run(ctx context.Context, msg *Pulled, ask delivery.StepAsk, key
}
}
}()
result, err := protect(ctx, func(ctx context.Context) (any, error) { return p.Run(ctx, ask) })
ph := &phases{put: p.Put, key: key, step: step, logf: p.Logf, prefix: fmt.Sprintf("the %s ask %s on %s", ask.Kind, ask.ID, p.Node)}
result, err := protect(ctx, func(ctx context.Context) (any, error) {
return p.Run(context.WithValue(ctx, phaseKey{}, ph), ask)
})
step = ph.end() // its end keeps the phase it ended in
close(done)
if ctx.Err() != nil {
// The process stops: the ask is neither ended nor acknowledged here, so the stream gives it to a node that runs.
@@ -438,6 +581,15 @@ func (p *Puller) run(ctx context.Context, msg *Pulled, ask delivery.StepAsk, key
_ = msg.Nak()
return
}
outcome := "passed"
if err != nil {
outcome = "did not pass: " + err.Error()
if len(outcome) > 300 {
outcome = outcome[:300] + "..."
}
}
p.logf("the %s ask %s ended on %s after %s, in its phase %q; %s", ask.Kind, ask.ID, p.Node,
now().Sub(step.Started).Round(time.Second), step.Phase, outcome)
if aerr := msg.Ack(); aerr != nil {
p.logf("the ask %s ended on %s and its acknowledgement failed: %v", ask.ID, p.Node, aerr)
}
@@ -474,6 +626,42 @@ func (p *Puller) watchCapacity() {
}
}
// watchHours follows the module's config value `build-hours` when WatchConfig and BuildHours are set, as watchCapacity
// follows `capacity`: each value handed is kept, used in place of BuildHours, and wakes the puller. A value that cannot
// be read is no window, said; a watch that cannot start leaves the puller on BuildHours, said once.
func (p *Puller) watchHours() {
if p.WatchConfig == nil || p.BuildHours == nil {
return
}
err := p.WatchConfig(BuildHoursSetting, func(c stdio.ProvisionChange) error {
setting := ""
var w *BuildWindow
var why string
switch {
case c.Refused != "":
why = "its build hours are no longer answered: " + c.Refused
case len(c.Value) > 0 && string(c.Value) != "null" && json.Unmarshal(c.Value, &setting) != nil:
why = fmt.Sprintf("the setting %s %s is not a string", BuildHoursSetting, c.Value)
default:
var err error
if w, err = ParseBuildHours(setting, p.Node); err != nil {
why = err.Error()
}
}
p.mu.Lock()
p.hours, p.hoursFault, p.hasHours = w, why, true
p.mu.Unlock()
if why != "" {
p.logf("the %s puller on %s builds at any hour: %s", p.Language, p.Node, why)
}
p.Wake()
return nil
})
if err != nil {
p.logf("the %s build hours of %s cannot be followed on the bus, so they are read from the settings file: %v", p.Language, p.Node, err)
}
}
// WatchConfigServed is stdio.WatchConfig once Serve has the runtime's channel (stdio.Served): a build module starts its
// pull before it serves, and a watch asked before that would be refused.
func WatchConfigServed(key string, handle func(stdio.ProvisionChange) error) error {
+58 -3
View File
@@ -104,20 +104,38 @@ func TestPullTakesDeliveryBeforeCheck(t *testing.T) {
}
// 1887/A2a, 1887/AC-pull: given capacity 2 and three asks waiting, two run and the third waits, and it starts when
// one ends, with no timer.
// one ends, with no timer. Issue 1806's follow-up: the 2 is the CPU's, a configured 3 bounded by a 12-core slice at 6
// cores a step, said once in capacity.<node>; and issue 1890: a run's phase is written to its step key and logged with
// its ask id, and its end keeps the last phase.
func TestPullHoldsAtMostItsCapacity(t *testing.T) {
src, st := NewFakeAsks(), newSteps()
asks := []delivery.StepAsk{}
for i := 0; i < 3; i++ {
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
a := testAsk(delivery.AskDelivery)
asks = append(asks, a)
_ = src.PublishAsk(a)
}
release := make(chan struct{})
var mu sync.Mutex
started := 0
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: st.put, Capacity: func() int { return 2 },
var said []string
capacities := map[string][]byte{}
quota := 1200
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: st.put, Capacity: func() int { return 3 },
CoresPerStep: func() float64 { return 6 },
Quota: func() (int, string) { mu.Lock(); defer mu.Unlock(); return quota, "a test's slice" },
PausePut: func(key string, v any) (uint64, error) {
mu.Lock()
capacities[key] = v.(json.RawMessage)
mu.Unlock()
return 1, nil
},
Logf: func(f string, a ...any) { mu.Lock(); said = append(said, fmt.Sprintf(f, a...)); mu.Unlock() },
PullWait: 10 * time.Millisecond, Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
mu.Lock()
started++
mu.Unlock()
Phase(ctx, "unit: tests of 3 packages")
<-release
return nil, nil
}}
@@ -129,12 +147,49 @@ func TestPullHoldsAtMostItsCapacity(t *testing.T) {
if p.InHand() != 2 || src.Waiting(delivery.AskDelivery) != 1 {
t.Fatalf("in hand %d, waiting %d; want 2 and 1", p.InHand(), src.Waiting(delivery.AskDelivery))
}
c := p.CapacityDetail()
if c.Effective != 2 || c.Configured != 3 || c.ByCPU != 2 || c.QuotaCores != 12 || !strings.Contains(c.Why, "bounded by the CPU") {
t.Fatalf("the capacity judged by = %+v, want 2 of a configured 3 bounded by 12 cores / 6", c)
}
mu.Lock()
raw := capacities["capacity.shanks"]
capLines := 0
for _, l := range said {
if strings.Contains(l, "build capacity of shanks is 2, bounded by the CPU") {
capLines++
}
}
mu.Unlock()
if got, err := delivery.DecodeCapacity(raw); err != nil || got.Effective != 2 || capLines != 1 {
t.Fatalf("capacity.shanks = %s (%v), said %d times; want effective 2, said once", raw, err, capLines)
}
waitFor(t, "a phase on the step key", func() bool {
key, _ := delivery.AskStepKey("shanks", p.Running()[0].ID)
s, _ := st.get(key)
return s.State == delivery.StepRunning && s.Phase == "unit: tests of 3 packages" && !s.PhaseAt.IsZero() && s.Owner == Owner // 1890/AC5
})
release <- struct{}{}
waitFor(t, "the third taken", func() bool { return src.Waiting(delivery.AskDelivery) == 0 })
// Taken from the source comes a moment before its run starts: wait for the start, not only the take.
waitFor(t, "the third started", func() bool { mu.Lock(); defer mu.Unlock(); return started == 3 })
close(release)
waitFor(t, "all acked", func() bool { a, _, _ := src.Fates(); return len(a) == 3 })
for _, a := range asks {
key, _ := delivery.AskStepKey("shanks", a.ID)
if s, _ := st.get(key); s.State != delivery.StepPassed || s.Phase != "unit: tests of 3 packages" || s.Owner != Owner {
t.Errorf("the end of %s = %+v, want passed in its last phase", a.ID, s)
}
}
mu.Lock()
logged := strings.Join(said, "\n")
quota = 600 // the cap lowered: one step at 6 cores, followed at the next judgement
mu.Unlock()
if !strings.Contains(logged, "the delivery ask "+asks[0].ID+" on shanks: unit: tests of 3 packages") {
t.Errorf("no journal line names the ask's phase: %s", logged)
}
if c := p.CapacityDetail(); c.Effective != 1 {
t.Errorf("after the quota fell to 6 cores, capacity %+v, want 1", c)
}
}
// An ask that is not one, or not this seat's, is dropped and runs nowhere. And issue 890/AC1: a check ask needing
+5 -2
View File
@@ -14,10 +14,13 @@ func SeatTools(seat, what string, p *Puller) []stdio.Tool {
Run: func(map[string]any) (any, error) { return run() }}
}
current := func() (any, error) {
return map[string]any{"node": p.Node, "capacity": p.CapacityNow(), "running": p.Running(), "paused": p.Paused()}, nil
c := p.CapacityDetail()
return map[string]any{"node": p.Node, "capacity": c.Effective, "capacity_why": c.Why, "capacity_detail": c,
"running": p.Running(), "paused": p.Paused()}, nil
}
return []stdio.Tool{
verb("current", "The "+what+" asks this node runs now (each one's id, kind and when it started), its build capacity, "+
verb("current", "The "+what+" asks this node runs now (each one's id, kind and when it started), its build capacity (the configured one, "+
"bounded by the cores its node's build slice gives over the cores a step needs) and why, "+
"and whether it is paused.", current),
verb("pause", "Take no new "+what+" ask on this node until resumed; an ask running here ends in its place. Kept "+
"across a restart (the node's pause key).", func() (any, error) {
+121 -28
View File
@@ -5,6 +5,8 @@ import (
"encoding/json"
"errors"
"fmt"
"os"
"strconv"
"strings"
"time"
@@ -14,40 +16,82 @@ import (
// **A step left running** (novox/hq issue 1890, note 5). A build module that stops while a step runs leaves its key in
// state running for good: nothing else writes that key's end, and the stream's redelivery only rewrites it when the same
// ask comes back. At its start a module ends the steps its node left running before the process began (EndStale), and
// its step-end tool ends one by hand (EndOne).
// ask comes back. At its start a module ends the steps its node left running by another process (EndStale: every step
// names its owner, delivery.Step.Owner; note 21), and its step-end tool ends one by hand (EndOne). The conformance kit's
// left-running case holds every build module to the start sweep (conformance.LeftRunning).
// RestartReason is why a step a previous process left running is ended.
const RestartReason = "the build module restarted while it ran"
// Boot is when this process began. A step started at or after it belongs to this process, which may still run it, so
// EndOne refuses it.
// Boot is when this process began. A step with no owner (written by an older build) started at or after it is taken
// for this process's, which may still run it, so EndOne refuses it.
var Boot = time.Now()
// Owner is this process as a step's owner (delivery.Step.Owner): `<boot id>.<pid>.<start>`, the node's boot id
// (/proc/sys/kernel/random/boot_id), the pid and Boot in milliseconds, base 36. The start keeps two processes apart
// that a reboot or a pid namespace gave the same pid. A running step whose owner is another is a previous process's.
var Owner = ownerOf(bootID, os.Getpid(), Boot)
// bootID is the node's boot id as Owner holds it; noBootID when it could not be read, which EndStaleAtStart says once.
var bootID = readBootID()
const noBootID = "no-boot-id"
func ownerOf(bootID string, pid int, start time.Time) string {
return bootID + "." + strconv.Itoa(pid) + "." + strconv.FormatInt(start.UnixMilli(), 36)
}
// readBootID is the node's boot id, or noBootID where it cannot be read (the pid and the start then tell the processes
// apart).
func readBootID() string {
b, err := os.ReadFile("/proc/sys/kernel/random/boot_id")
if id := strings.TrimSpace(string(b)); err == nil && id != "" {
return id
}
return noBootID
}
// Scan reads and writes a steps state with its revisions: its keys, a key's stored value and revision (nil and 0 for
// none), and a write that lands only while the key is still at the revision read (KeptState.Update), failing with
// stdio.ErrChanged when another writer was first.
// none), a write that lands only while the key is still at the revision read (KeptState.Update), and a delete of a key
// still at the revision read; both fail with stdio.ErrChanged when another writer was first.
type Scan struct {
Keys func() ([]string, error)
Get func(key string) (json.RawMessage, uint64, error)
Update func(key string, value any, revision uint64) (uint64, error)
Delete func(key string, revision uint64) error
}
// ScanOf is the Scan of a state the process keeps.
//
// Its Delete reads the key again and deletes it only while it is still at the revision read: the runtime's
// mesh/state.delete takes no revision yet, so a write landing between that read and the delete (a few milliseconds) is
// deleted with it. For a step key that is a redelivered ask begun on this node in that moment: its running key leaves
// the steps state until its run writes its end, which its asker still reads (a watch takes puts only).
func ScanOf(s stdio.KeptState) Scan {
return Scan{Keys: s.Keys, Update: s.Update, Get: func(key string) (json.RawMessage, uint64, error) {
get := func(key string) (json.RawMessage, uint64, error) {
e, err := s.Get(key)
if err != nil || e == nil {
return nil, 0, err
}
return e.Value, e.Revision, nil
}
return Scan{Keys: s.Keys, Update: s.Update, Get: get, Delete: func(key string, revision uint64) error {
_, rev, err := get(key)
if err != nil {
return err
}
if rev != revision {
return stdio.ErrChanged
}
return s.Delete(key)
}}
}
// isAsk says a step is a pulled ask's: its key is <node>.<ask id> and its tool is "<kind> ask" (a hand-started step
// has the tool's name). An ask's key is the stream's to settle, never a restart's: the puller leaves an ask it was
// running unacknowledged so the stream redelivers it, and the asker's wait takes the first end it reads on the key.
// Only once the asker's bound has passed (delivery.StepBound) is the key nobody's, and then it is ended.
// A key a gone process owns is deleted, never ended (EndStale); one with no owner is ended only once the asker's bound
// has passed (delivery.StepBound), when it is nobody's.
func isAsk(key string, s delivery.Step) bool {
_, _, err := delivery.ParseAskStepKey(key)
return err == nil && strings.HasSuffix(s.Tool, " ask")
@@ -95,11 +139,23 @@ func endRunning(scan Scan, key string, step delivery.Step, revision uint64, why
return err
}
// EndStale ends this node's steps that are running and started before `before` (the start of this process) as failed
// with RestartReason, and answers their ids. Only keys of node are read; a value that is no step is left alone. A step
// of a pulled ask is left to the stream's redelivery (isAsk) until its start is older than delivery.StepBound: its asker
// stopped waiting then, whether the stream gave it to another node or gave it up, and it is ended as failed, "past the
// step bound: its asker stopped waiting at <time>". A key written by another writer since it was read
// EndStale ends this node's running steps that no process of this module runs any more, and answers their ids. Only keys
// of node are read; a value that is no step is left alone.
// - A hand-started step with an owner (delivery.Step.Owner) that is not this process (Owner) is ended failed with
// RestartReason at once, whatever its age: its process is gone and nothing redelivers it, so nothing else would
// write its end (issue 1890, note 21).
// - A pulled ask's step with an owner that is not this process is deleted, not ended (novox/hq ADR 0525 §6): the
// stream redelivers an ask whose node stopped (the puller neither ends nor acknowledges one at its stop), and its
// asker takes the first end it reads, so an end written here would fail a check the redelivery would still run. Its
// asker's watch takes puts only and keeps waiting for the redelivery's key, its bound the backstop; the stale running
// key leaves the steps state, and with it the delivery view's and the checker's count of what runs (note 21: it
// held one of shanks' two slots).
// - A step owned by this process is left: it may run now. A redelivered ask begun on this node is owned by it.
// - A step with no owner (written by an older build) keeps the earlier rule: a hand-started step started before
// `before` (the start of this process) is ended; a pulled ask is left to the stream's redelivery (isAsk) until its
// start is older than delivery.StepBound, and then ended "past the step bound: its asker stopped waiting at <time>".
//
// Each end and delete is logged, naming the step, its owner and why. A key written by another writer since it was read
// (a redelivered ask's start, a real end) is skipped.
func EndStale(scan Scan, node string, before time.Time, logf func(string, ...any)) ([]string, error) {
keys, err := scan.Keys()
@@ -123,32 +179,59 @@ func EndStale(scan Scan, node string, before time.Time, logf func(string, ...any
if err != nil || step.State != delivery.StepRunning {
continue
}
why := RestartReason
if isAsk(k, step) {
if !step.Started.Add(delivery.StepBound).Before(now()) {
continue
}
why = "past the step bound: its asker stopped waiting at " + step.Started.Add(delivery.StepBound).UTC().Format("2006-01-02T15:04:05Z")
} else if !step.Started.Before(before) {
why, said := staleWhy(k, step, before)
if why == "" {
continue
}
if err := endRunning(scan, k, step, rev, why); err != nil {
done := "ended"
if step.Owner != "" && isAsk(k, step) {
done = "deleted, left to the stream's redelivery"
if scan.Delete == nil {
err = fmt.Errorf("the steps state takes no delete")
} else {
err = scan.Delete(k, rev)
}
} else {
err = endRunning(scan, k, step, rev, why)
}
if err != nil {
if errors.Is(err, stdio.ErrChanged) {
continue
}
return ended, fmt.Errorf("buildstep: ending the step %s: %w", k, err)
}
if logf != nil {
logf("the step %s (%s) was left running and is ended: %s", k, step.Tool, why)
logf("the step %s (%s, started %s) was left running and is %s: %s; %s", k, step.Tool,
step.Started.UTC().Format(time.RFC3339), done, why, said)
}
ended = append(ended, step.ID)
}
return ended, nil
}
// staleWhy is why a running step is ended (or, for an ask with an owner, deleted) by EndStale and what the log says of
// its owner, or "" to leave it.
func staleWhy(key string, step delivery.Step, before time.Time) (why, said string) {
switch {
case step.Owner == Owner:
return "", ""
case step.Owner != "":
return RestartReason, "its owner " + step.Owner + " is not this process " + Owner
case isAsk(key, step):
if !step.Started.Add(delivery.StepBound).Before(now()) {
return "", ""
}
return "past the step bound: its asker stopped waiting at " + step.Started.Add(delivery.StepBound).UTC().Format("2006-01-02T15:04:05Z"),
"it names no owner (an older build wrote it)"
case step.Started.Before(before):
return RestartReason, "it names no owner (an older build wrote it) and started before this process"
}
return "", ""
}
// EndOne ends this node's running step id (or the key <node>.<id>) as failed with why. A step that is not running is
// refused, and so is one started since this process began (Boot: it may still run, whether a pulled ask or a step a
// tool started), one live reports as held, and one whose key changed since it was read.
// refused, and so is one this process owns (Owner; for a step with no owner, one started since Boot: it may still run,
// whether a pulled ask or a step a tool started), one live reports as held, and one whose key changed since it was read.
func EndOne(scan Scan, node, id, why string, live func(id string) bool) (delivery.Step, error) {
id = strings.TrimSpace(id)
if n, rest, ok := strings.Cut(id, "."); ok {
@@ -178,7 +261,7 @@ func EndOne(scan Scan, node, id, why string, live func(id string) bool) (deliver
if step.State != delivery.StepRunning {
return delivery.Step{}, fmt.Errorf("the step %s is %s, not running", key, step.State)
}
if !step.Started.Before(Boot) || (live != nil && live(id)) {
if step.Owner == Owner || (step.Owner == "" && !step.Started.Before(Boot)) || (live != nil && live(id)) {
return delivery.Step{}, fmt.Errorf("the step %s was started by this process and may run now; only a step a previous process left is ended", key)
}
if err := endRunning(scan, key, step, rev, why); err != nil {
@@ -226,14 +309,24 @@ var (
StaleRetry = 5 * time.Second
)
// StaleServed is what EndStaleAtStart waits for before it reads the steps state: the runtime serving (stdio.Served).
// The conformance kit's left-running case (conformance.LeftRunning) hands it a closed channel, to run a module's own
// start sweep in its test.
var StaleServed = stdio.Served
// EndStaleAtStart is what a build module runs once at its start, in a goroutine: it waits for the runtime to serve,
// then ends this node's steps left running before `before` (EndStale), trying again when the state cannot be read yet.
// A step this process starts is never one of them: it starts after `before`.
// then ends this node's steps a previous process left running (EndStale), trying again when the state cannot be read
// yet. A step names its owner, so `before` (the start of this process) matters only for a step with no owner, written
// by an older build: such a step started before it is a previous process's. When the node's boot id cannot be read it
// says so once: Owner then tells processes apart by their pid and start only.
func EndStaleAtStart(ctx context.Context, scan Scan, node string, before time.Time, logf func(string, ...any)) {
select {
case <-ctx.Done():
return
case <-stdio.Served():
case <-StaleServed():
}
if bootID == noBootID && logf != nil {
logf("the boot id of %s cannot be read (/proc/sys/kernel/random/boot_id): its steps' owner is %s, told apart by pid and start only", node, Owner)
}
for attempt := 1; ; attempt++ {
ended, err := EndStale(scan, node, before, logf)
+52 -22
View File
@@ -2,6 +2,7 @@ package buildstep
import (
"encoding/json"
"fmt"
stdio "git.novox.be/novox/mesh-sdk/go"
"strings"
"testing"
@@ -33,6 +34,15 @@ func scanOf(k *keys, race func(key string)) Scan {
raw, err := delivery.EncodeStep(s[len(s)-1])
return raw, uint64(len(s)), err
},
Delete: func(key string, revision uint64) error {
k.mu.Lock()
defer k.mu.Unlock()
if uint64(len(k.puts[key])) != revision {
return stdio.ErrChanged
}
delete(k.puts, key)
return nil
},
Update: func(key string, value any, revision uint64) (uint64, error) {
if race != nil {
race(key)
@@ -48,65 +58,85 @@ func scanOf(k *keys, race func(key string)) Scan {
}
}
// 1890: EndStale ends only this node's hand-started steps left running before the process began and the asks past the
// step bound (a younger ask is the stream's to redeliver), and never a key written meanwhile; EndOne ends one by hand, never one this process started.
// 1890 and 1890/AC6: EndStale ends this node's hand-started running steps of another owner at once, whatever their age,
// and deletes its asks of another owner (the stream redelivers them, ADR 0525 §6); of
// the steps with no owner (an older build's), the hand-started ones left running before the process began and the asks
// past the step bound (a younger ask is the stream's to redeliver); never a step this process owns or a key written
// meanwhile. EndOne ends one by hand, never one this process owns or started.
func TestEndStaleAndEndOne(t *testing.T) {
k := newKeys()
boot := time.Now()
oldBoot := Boot
Boot = boot
defer func() { Boot = oldBoot }()
put := func(node, id, tool, state string, started time.Time) {
gone := ownerOf("b00t", 4242, boot.Add(-time.Hour))
put := func(node, id, tool, state string, started time.Time, owner string) {
key, _ := delivery.StepKey(node, id)
_, _ = k.put(key, json.RawMessage(mustStep(t, delivery.Step{ID: id, Tool: tool, Node: node, State: state, Started: started})))
_, _ = k.put(key, json.RawMessage(mustStep(t, delivery.Step{ID: id, Tool: tool, Node: node, State: state, Started: started, Owner: owner})))
}
put("ace", "old1", "build_go_fetch", delivery.StepRunning, boot.Add(-time.Hour))
put("ace", "mjk3l9x0abcdef", "check ask", delivery.StepRunning, boot.Add(-time.Hour))
put("ace", "mjk3l9x0abcdee", "check ask", delivery.StepRunning, time.Now().Add(-5*time.Minute))
put("ace", "raced1", "build_go_test", delivery.StepRunning, boot.Add(-time.Hour))
put("ace", "new1", "build_go_fetch", delivery.StepRunning, boot.Add(time.Second))
put("ace", "done1", "build_go_fetch", delivery.StepPassed, boot.Add(-time.Hour))
put("g14", "other1", "build_go_fetch", delivery.StepRunning, boot.Add(-time.Hour))
put("ace", "old1", "build_go_fetch", delivery.StepRunning, boot.Add(-time.Hour), "")
put("ace", "mjk3l9x0abcdef", "check ask", delivery.StepRunning, boot.Add(-time.Hour), "")
put("ace", "mjk3l9x0abcdee", "check ask", delivery.StepRunning, time.Now().Add(-5*time.Minute), "")
put("ace", "mjk3l9x0abcded", "check ask", delivery.StepRunning, time.Now().Add(-5*time.Minute), gone) // note 21's ghost
put("ace", "gone2", "build_go_test", delivery.StepRunning, boot.Add(time.Second), gone)
put("ace", "mine1", "check ask", delivery.StepRunning, boot.Add(-time.Hour), Owner)
put("ace", "raced1", "build_go_test", delivery.StepRunning, boot.Add(-time.Hour), "")
put("ace", "new1", "build_go_fetch", delivery.StepRunning, boot.Add(time.Second), "")
put("ace", "done1", "build_go_fetch", delivery.StepPassed, boot.Add(-time.Hour), gone)
put("g14", "other1", "build_go_fetch", delivery.StepRunning, boot.Add(-time.Hour), gone)
// raced1 is ended by its own run between the read and the write.
raced := false
scan := scanOf(k, func(key string) {
if key == "ace.raced1" && !raced {
raced = true
put("ace", "raced1", "build_go_test", delivery.StepPassed, boot.Add(-time.Hour))
put("ace", "raced1", "build_go_test", delivery.StepPassed, boot.Add(-time.Hour), "")
}
})
ended, err := EndStale(scan, "ace", boot, nil)
if err != nil || len(ended) != 2 || ended[0] == ended[1] {
var said []string
ended, err := EndStale(scan, "ace", boot, func(f string, a ...any) { said = append(said, fmt.Sprintf(f, a...)) })
if err != nil || len(ended) != 4 {
t.Fatalf("ended %v, %v", ended, err)
}
if s := k.last("ace.old1"); s.State != delivery.StepFailed || s.Reason != RestartReason || s.Ended.IsZero() {
t.Errorf("old1 = %+v", s)
if _, held := k.puts["ace.mjk3l9x0abcded"]; held {
t.Errorf("the ghost ask's key is still held: %+v", k.last("ace.mjk3l9x0abcded"))
}
for _, key := range []string{"ace.old1", "ace.gone2"} {
if s := k.last(key); s.State != delivery.StepFailed || s.Reason != RestartReason || s.Ended.IsZero() {
t.Errorf("%s = %+v", key, s)
}
}
if s := k.last("ace.mjk3l9x0abcdef"); s.State != delivery.StepFailed || !strings.HasPrefix(s.Reason, "past the step bound: its asker stopped waiting at ") {
t.Errorf("the old ask = %+v", s)
}
for key, state := range map[string]string{"ace.mjk3l9x0abcdee": delivery.StepRunning, "ace.raced1": delivery.StepPassed, "ace.new1": delivery.StepRunning, "ace.done1": delivery.StepPassed, "g14.other1": delivery.StepRunning} {
if log := strings.Join(said, "\n"); !strings.Contains(log, "ace.mjk3l9x0abcded") || !strings.Contains(log, "deleted, left to the stream's redelivery") || !strings.Contains(log, "its owner "+gone+" is not this process "+Owner) {
t.Errorf("the log does not name the ghost and its owner:\n%s", log)
}
for key, state := range map[string]string{"ace.mjk3l9x0abcdee": delivery.StepRunning, "ace.mine1": delivery.StepRunning, "ace.raced1": delivery.StepPassed,
"ace.new1": delivery.StepRunning, "ace.done1": delivery.StepPassed, "g14.other1": delivery.StepRunning} {
if s := k.last(key); s.State != state {
t.Errorf("%s = %s, want %s", key, s.State, state)
}
}
// EndOne: refuses without why, a held step, one started by this process (a pulled ask or a step tool's), a step not
// running and another node's; ends a step a previous process left.
for _, c := range []struct{ id, why string }{{"old1", ""}, {"new1", "stuck"}, {"done1", "x"}, {"g14.other1", "x"}} {
// EndOne: refuses without why, a held step, one this process owns or (with no owner) started, a step not running and
// another node's; ends a step a previous process left.
for _, c := range []struct{ id, why string }{{"old1", ""}, {"new1", "stuck"}, {"mine1", "stuck"}, {"done1", "x"}, {"g14.other1", "x"}} {
if _, err := EndOne(scan, "ace", c.id, c.why, nil); err == nil {
t.Errorf("ended %s with why %q", c.id, c.why)
}
}
put("ace", "old2", "build_go_fetch", delivery.StepRunning, boot.Add(-time.Hour))
put("ace", "old2", "build_go_fetch", delivery.StepRunning, boot.Add(time.Second), gone)
if _, err := EndOne(scan, "ace", "old2", "x", func(string) bool { return true }); err == nil {
t.Error("ended a held step")
}
if _, err := EndOne(scan, "ace", "ace.old2", "stuck for good", nil); err != nil {
t.Fatal(err)
}
if s := k.last("ace.old2"); s.State != delivery.StepFailed || s.Reason != "stuck for good" {
if s := k.last("ace.old2"); s.State != delivery.StepFailed || s.Reason != "stuck for good" || s.Owner != gone {
t.Errorf("old2 = %+v", s)
}
if a, b := ownerOf("b", 1, boot), ownerOf("b", 1, boot.Add(time.Second)); a == b {
t.Errorf("two starts of one pid share the owner %q", a)
}
}
func mustStep(t *testing.T, s delivery.Step) []byte {
+4 -3
View File
@@ -1,6 +1,7 @@
// Package buildstep is what both build modules share: the runner of a long step that answers a tool call at once
// and ends on its key in the module's `steps` state (novox/hq ADR 0469 §2; issue 1471), the toolchain guard
// (issue 1473) and the fetch of one commit over SSH (issue 1474).
// (issue 1473), the fetch of one commit over SSH (issue 1474), and the failure contract of a test level with the parsers
// of go test -json and node --test's TAP that fill it (failure.go, issue 1901; its conformance kit is package conformance).
package buildstep
import (
@@ -51,9 +52,9 @@ func Start(ctx context.Context, put Put, node, tool string, run func(ctx context
return step.ID, nil
}
// begin writes the step running under id.
// begin writes the step running under id, owned by this process (Owner).
func begin(put Put, node, tool, id, what string) (delivery.Step, string, error) {
step := delivery.Step{ID: id, Tool: tool, Node: node, State: delivery.StepRunning, Started: now().UTC(), What: what}
step := delivery.Step{ID: id, Tool: tool, Node: node, State: delivery.StepRunning, Started: now().UTC(), What: what, Owner: Owner}
key, err := delivery.StepKey(node, step.ID)
if err != nil {
return delivery.Step{}, "", err
+4 -3
View File
@@ -70,8 +70,9 @@ func TestStartAnswersAtOnceWithTheStepRunning(t *testing.T) {
t.Errorf("Start took %v", d)
}
s := k.last("ace." + id)
if s.State != delivery.StepRunning || s.Tool != "fetch" || s.Node != "ace" || s.ID != id || s.Started.IsZero() {
t.Errorf("step = %+v", s)
// 1890/AC5: the running key names this process as its owner.
if s.State != delivery.StepRunning || s.Tool != "fetch" || s.Node != "ace" || s.ID != id || s.Started.IsZero() || s.Owner != Owner || Owner == "" {
t.Errorf("step = %+v, want owner %q", s, Owner)
}
}
@@ -100,7 +101,7 @@ func TestStepEndsPassedFailedOrRefusedOnItsKey(t *testing.T) {
t.Fatalf("ended key %q", got)
}
s := k.last("ace." + id)
if s.State != c.state || s.Reason != c.reason || string(s.Result) != c.result || s.Ended.IsZero() {
if s.State != c.state || s.Reason != c.reason || string(s.Result) != c.result || s.Ended.IsZero() || s.Owner != Owner {
t.Errorf("step = %+v", s)
}
})
+192
View File
@@ -0,0 +1,192 @@
package buildstep
// The reader of `node --test` under its TAP reporter (issue 1901's thorough fix; first written in mesh-build-node by
// fix-1901ts): each failing test with its name, its file, its cause and its own block, never a passing line of an
// earlier file, which is what node's default reporter left the first fifteen lines of.
import (
"regexp"
"strings"
)
// NodeTestOptions is the NODE_OPTIONS a build module runs a package's test script with: node's TAP reporter, written to
// the file dest, so whatever the script prints on its own output the report is whole and machine-readable. A script
// that names a reporter of its own makes node refuse the run loudly ("--test-reporter must match the number of
// --test-reporter-destination"), never quietly.
func NodeTestOptions(dest string) string {
return "--test-reporter=tap --test-reporter-destination=" + dest
}
var ansiEscape = regexp.MustCompile("\x1b\\[[0-9;]*m")
func stripANSI(s string) string { return ansiEscape.ReplaceAllString(s, "") }
// ReadTAP reads the failing tests of a `node --test` run: report is what the TAP reporter wrote (NodeTestOptions'
// file), console the script's own stdout and stderr, script the name of the script that ran and dir the package's
// directory, which a test's file is made relative to. A suite that failed only because a test inside it did is not
// reported twice. A test file that did not load (a syntax error, a crash, an error after its test ended) takes its
// cause from the lines node wrote before it. When the report holds no failing test (node itself failed, or the script
// failed before its tests ran: a build, a reporter refused) the failure is the console's (ConsoleFailure).
func ReadTAP(pkg, script, dir, report, console string) []Failure {
var failed []Failure
all := strings.Split(stripANSI(strings.ReplaceAll(report, "\r\n", "\n")), "\n")
var cancelled []Failure // failed because a suite around them did: said after the failure that carries the cause
prev := 0 // the line after the last test block, passing or not: what the run wrote since belongs to the next test
for i := 0; i < len(all); i++ {
name, ok := tapNotOk(all[i])
if !ok {
if tapOk(all[i]) {
i = tapBlockEnd(all, i) - 1
prev = i + 1
}
continue
}
indent := len(all[i]) - len(strings.TrimLeft(all[i], " "))
block := []string{strings.TrimSpace(all[i])}
fields := map[string]string{}
var key string
inYAML := false
j := i + 1
for ; j < len(all); j++ {
t := strings.TrimSpace(all[j])
if !inYAML {
if t == "---" {
inYAML = true
continue
}
break
}
if t == "..." {
j++
break
}
block = append(block, strings.TrimRight(all[j], " \t"))
if k, v, found := tapKey(all[j], indent); found {
key = k
if v != "|-" && v != "|" {
fields[k] = strings.Trim(v, "'")
}
} else if key != "" && fields[key] == "" && t != "" {
fields[key] = t // the first line of a block scalar
}
}
from, at := prev, i
i, prev = j-1, j
if fields["failureType"] == "subtestsFailed" {
continue
}
cause := fields["error"]
if cause == "test failed" || cause == "" {
// a file that did not load, or an error after a test ended: node says only "test failed" and wrote the
// error as comment lines before it
var own []string
for _, l := range all[from:at] {
if t := strings.TrimSpace(l); strings.HasPrefix(t, "# ") && !strings.HasPrefix(t, "# Subtest") {
own = append(own, strings.TrimPrefix(t, "# "))
}
}
for _, l := range own {
if strings.Contains(l, "Error") {
cause = l
break
}
}
if cause == "" && len(own) > 0 {
cause = own[0]
}
block = append(own, block...)
}
if cause == "" {
cause = fields["failureType"]
}
if cause == "" {
cause = "the test failed and node said no error"
}
cause = CutLine(cause)
file := strings.TrimPrefix(fields["location"], "file://")
if dir != "" {
file = strings.TrimPrefix(file, strings.TrimSuffix(dir, "/")+"/")
}
f := Failure{Package: pkg, Test: name, File: file, Cause: cause, Output: Clip(block, cause)}
if fields["failureType"] == "cancelledByParent" {
cancelled = append(cancelled, f)
continue
}
failed = append(failed, f)
}
if failed = append(failed, cancelled...); len(failed) > 0 {
return failed
}
var comments []string
for _, l := range all {
if t := strings.TrimSpace(l); strings.HasPrefix(t, "# ") && !strings.HasPrefix(t, "# Subtest") && !tapCount(t) {
comments = append(comments, strings.TrimPrefix(t, "# "))
}
}
return []Failure{ConsoleFailure(pkg, script, strings.Join(comments, "\n")+"\n"+console)}
}
// tapCount says a comment is TAP's closing count ("# tests 3", "# pass 2", "# duration_ms 1.2").
func tapCount(t string) bool {
for _, w := range []string{"tests", "suites", "pass", "fail", "cancelled", "skipped", "todo", "duration_ms"} {
if strings.HasPrefix(t, "# "+w+" ") {
return true
}
}
return false
}
// tapOk says a line is a passing test's "ok N - name", indented or not.
func tapOk(l string) bool {
rest, ok := strings.CutPrefix(strings.TrimSpace(l), "ok ")
return ok && rest != "" && rest[0] >= '0' && rest[0] <= '9'
}
// tapBlockEnd is the line after the test line at i and its YAML block ("---" to "...").
func tapBlockEnd(all []string, i int) int {
j := i + 1
if j >= len(all) || strings.TrimSpace(all[j]) != "---" {
return j
}
for j++; j < len(all); j++ {
if strings.TrimSpace(all[j]) == "..." {
return j + 1
}
}
return j
}
// tapNotOk is the test name of a "not ok N - name" line, indented or not; a TODO or SKIP directive fails nothing.
func tapNotOk(l string) (string, bool) {
t := strings.TrimSpace(l)
rest, ok := strings.CutPrefix(t, "not ok ")
if !ok {
return "", false
}
if u := strings.ToUpper(rest); strings.Contains(u, " # TODO") || strings.Contains(u, " # SKIP") {
return "", false
}
if _, after, found := strings.Cut(rest, " - "); found {
rest = after
}
return strings.TrimSpace(rest), true
}
// tapKey is "key: value" of a field of a TAP block, whose lines sit two spaces deeper than their "not ok" line; a deeper
// line is part of a block scalar (an error's lines, a stack).
func tapKey(l string, indent int) (key, val string, ok bool) {
t := strings.TrimLeft(l, " ")
if len(l)-len(t) != indent+2 {
return "", "", false
}
k, v, found := strings.Cut(t, ": ")
if !found {
if k, found = strings.CutSuffix(t, ":"); !found {
return "", "", false
}
}
if k == "" || strings.ContainsAny(k, " ()/") {
return "", "", false
}
return k, strings.TrimSpace(v), true
}
+146 -8
View File
@@ -6,6 +6,7 @@ import (
"path/filepath"
"reflect"
"regexp"
"slices"
"strings"
"testing"
)
@@ -14,10 +15,108 @@ import (
// byte what the controller's composes. testdata/controller-user-lists.json was written by the controller's own
// internal/broker (mesh-controller branch fix/1945-mesh-grants-publishes-memberships, and fix/1790-grants-create-buckets for the grants part creating declared streams), lists and memberships, from fixtures_test.go compiled there unchanged; composed.conf is the
// controller's golden of the whole server configuration. A row that differs is the copy drifting from the controller,
// which is a fault while both compose (until the controller loses its lease, ADR 0547 §13).
// which is a fault while both compose (until the controller loses its lease, ADR 0547 §13), but for the grants the copy
// gives beyond it on purpose (beyondTheControllers): the copy grants every user everything the controller's grants, and
// only those more.
// createdAdvisory is one subscription to a bucket's creation in a composed list, as stateCreatedHeard composes it.
var createdAdvisory = regexp.MustCompile(`"\$JS\.EVENT\.ADVISORY\.STREAM\.CREATED\.KV_[^"]*", `)
// beyondTheControllers is what the copy grants beyond the controller's on purpose (novox/hq ADR 0547 §2, ADR 0555 §3,
// issue 1126), by user: a node's account reads every node's own facts, `<node>.node` of the have bucket where the node's
// record rides, and what it composes its own modules from (NodeComposeReads). After the take-over (ADR 0547 §8) the
// grants part's list is the one the bus loads; a hand-back loads the controller's, which lacks these reads, and the nodes
// keep applying without composing their own until the next take-over. Each principal's hearing of its buckets' creation
// (issue 1790) is the other row beyond, matched by createdAdvisory.
func beyondTheControllers(user string) map[string]bool {
out := map[string]bool{}
node, ok := strings.CutPrefix(user, "node.")
if !ok {
return out
}
for _, s := range append(readGrants("node-engine_have", "*.node"), NodeComposeReads(node)...) {
out[s] = true
}
return out
}
// mayBeyondTheControllers is what the copy may grant beyond the controller's where the records allow it, and need not
// where they do not: a node's ask of the vault's `own` while the vault serves it (issue 1126, ADR 0550 §5).
func mayBeyondTheControllers(user string) map[string]bool {
if _, ok := strings.CutPrefix(user, "node."); !ok {
return nil
}
return map[string]bool{VaultOwnCall(Principal{Kind: KindNode, Node: strings.TrimPrefix(user, "node.")}): true,
VaultSecretChanged: true}
}
// quotedToken is a quoted token of a composed list.
var quotedToken = regexp.MustCompile(`"[^"]*"`)
// grantsByUser splits a composed list into the text outside its users and each user's quoted tokens (its name, its
// password's hash, every subject it may publish or subscribe to), by user.
func grantsByUser(list string) (string, map[string]map[string]bool) {
var outside strings.Builder
users := map[string]map[string]bool{}
user := ""
for _, line := range strings.SplitAfter(list, "\n") {
trimmed := strings.TrimSpace(line)
if name, ok := strings.CutPrefix(trimmed, `{ user: "`); ok {
user, _, _ = strings.Cut(name, `"`)
users[user] = map[string]bool{}
} else if user != "" && trimmed == "]" {
user = ""
}
if user == "" {
outside.WriteString(line)
continue
}
for _, q := range quotedToken.FindAllString(line, -1) {
users[user][strings.Trim(q, `"`)] = true
}
}
return outside.String(), users
}
// grantsAsTheControllers fails t unless the copy's list is the controller's, user for user, but for the grants it
// names on purpose (beyondTheControllers, the creation advisories): every grant of the controller's is the copy's, and
// the copy's beyond them are exactly those, every one of them there.
func grantsAsTheControllers(t *testing.T, got, want string) {
t.Helper()
gotOutside, gotUsers := grantsByUser(got)
wantOutside, wantUsers := grantsByUser(want)
if gotOutside != wantOutside {
t.Errorf("the list differs from the controller's outside its users:\n%s", gotOutside)
}
for user, was := range wantUsers {
now, ok := gotUsers[user]
if !ok {
t.Errorf("the copy has no user %s", user)
continue
}
intended := beyondTheControllers(user)
for g := range was {
if !now[g] {
t.Errorf("%s: the copy does not grant %s, which the controller's does", user, g)
}
}
for g := range now {
if !was[g] && !intended[g] && !mayBeyondTheControllers(user)[g] && !createdAdvisory.MatchString(`"`+g+`", `) {
t.Errorf("%s: the copy grants %s beyond the controller's, and no decision names it", user, g)
}
}
for g := range intended {
if !now[g] {
t.Errorf("%s: the copy does not grant %s, which a node composing its own modules reads", user, g)
}
}
}
for user := range gotUsers {
if _, ok := wantUsers[user]; !ok {
t.Errorf("the copy has a user %s the controller's has not", user)
}
}
}
func TestTheCopiedComposerComposesAsTheControllers(t *testing.T) {
raw, err := os.ReadFile(filepath.Join("testdata", "controller-user-lists.json"))
if err != nil {
@@ -56,10 +155,7 @@ func TestTheCopiedComposerComposesAsTheControllers(t *testing.T) {
// the take-over of ADR 0547 milestone 2): each principal hears its own and read buckets created, so a state
// missing at its start is bound when it appears. The controller's composer is frozen and its list no longer
// loaded; while it is not, this row is left out of the comparison and nothing else.
list = createdAdvisory.ReplaceAllString(list, "")
if list != want[i].List {
t.Errorf("the list differs from the controller's:\n%s", list)
}
grantsAsTheControllers(t, list, want[i].List)
// And every membership the records issue, as the controller's MembershipFor makes it (ADR 0547 §5): the
// grants part publishes them.
raw, err := json.Marshal(MembershipsFor(f))
@@ -109,6 +205,50 @@ func TestTheCopiedComposerComposesAsTheControllers(t *testing.T) {
t.Error("a user of a seat not pulled by kind reads DEAD_LETTERS")
}
})
// Issue 1126 (ADR 0550 §5): a node's engine asks the vault's `own` in its own name while the vault serves it, and no
// other node's.
t.Run("a node asks the vault its own secrets", func(t *testing.T) {
one := Principal{Kind: KindNode, Node: "one", AsksItsOwnSecrets: true}
perms, err := PermissionsFor(one)
if err != nil {
t.Fatal(err)
}
if !MayPublish(perms, VaultOwnCall(one)) || MayPublish(perms, VaultOwnCall(Principal{Kind: KindNode, Node: "two"})) {
t.Errorf("node one asks own as %v, and as two %v", MayPublish(perms, VaultOwnCall(one)),
MayPublish(perms, VaultOwnCall(Principal{Kind: KindNode, Node: "two"})))
}
if plain, _ := PermissionsFor(Principal{Kind: KindNode, Node: "one"}); MayPublish(plain, VaultOwnCall(one)) {
t.Error("a node asks own while the vault does not serve it")
}
if !slices.Contains(perms.Subscribe, VaultSecretChanged) {
t.Errorf("node one does not hear the vault's secret-changed: %v", perms.Subscribe)
}
// And what a module on another node serves from its settings, that key alone (mailu's smtp domain, issue 1126).
users, err := Users(Records{Nodes: []string{"one", "two"}, Assigned: map[string][]Declared{
"two": {{Module: "mailer", Served: []string{"domain", "not.a.token"}}}}})
if err != nil {
t.Fatal(err)
}
get := func(key string) string {
return "$JS.API.DIRECT.GET.KV_" + ResolvedBucket + ".$KV." + ResolvedBucket + "." + key
}
for _, u := range users {
if u.Kind != KindNode {
continue
}
perms, err := PermissionsFor(u)
if err != nil {
t.Fatal(err)
}
if u.Node == "one" && (!reflect.DeepEqual(u.ServedReads, []string{"two.mailer.domain"}) || !MayPublish(perms, get("two.mailer.domain")) ||
MayPublish(perms, get("two.mailer.sitename"))) {
t.Errorf("node one reads %v of two's resolved values", u.ServedReads)
}
if u.Node == "two" && len(u.ServedReads) != 0 {
t.Errorf("node two is granted its own served keys apart: %v", u.ServedReads)
}
}
})
t.Run("the controller's golden server configuration", func(t *testing.T) {
seat := Seat{Name: "telegram-sender", Accepts: []string{"send"}, Emits: []string{"delivered", "failed"}}
got, err := Compose(
@@ -135,8 +275,6 @@ func TestTheCopiedComposerComposesAsTheControllers(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if got != string(want) {
t.Errorf("the configuration differs from the controller's golden:\n%s", got)
}
grantsAsTheControllers(t, got, string(want))
})
}
+43
View File
@@ -213,6 +213,14 @@ type Principal struct {
// StatesItsKeyToTheVault is a machine principal whose node-engine may ask the vault's `node-key` in its own name
// (issue 786, novox/hq ADR 0372): set while the vault's holder serves it. That one verb, and nothing else new.
StatesItsKeyToTheVault bool
// AsksItsOwnSecrets is a machine principal whose node-engine may ask the vault's `own` in its own name (novox/hq
// ADR 0547 §2, ADR 0550 §5, issue 1126): its modules' own secrets sealed to its key, for the compositions it makes
// itself. Set while the vault's holder serves it.
AsksItsOwnSecrets bool
// ServedReads are, for a machine principal, the keys of mesh-config's resolved values that other nodes' modules serve
// from (`<node>.<module>.<key>`, catalogue.ServedSettings): what every consumer is told, and so what a node composing
// its own modules reads of a provider elsewhere (issue 1126), each key alone. Sorted, none twice.
ServedReads []string
// FollowsGrantsUserList is the machine principal of the node holding mesh-broker: its node-engine reads the grants
// part's sealed user list, one key and an object by name (novox/hq ADR 0547 clause 7, GrantsUserListFollows).
FollowsGrantsUserList bool
@@ -344,6 +352,10 @@ var VaultMoveIn = SeatVerb{Seat: "mesh-vault", Verb: "move-in"}
// novox/hq ADR 0372); granted as Principal.StatesItsKeyToTheVault says.
var VaultNodeKey = SeatVerb{Seat: "mesh-vault", Verb: "node-key"}
// VaultOwn is the vault's verb a machine's node-engine asks in its own name for its modules' own secrets sealed to its
// key (issue 1126, ADR 0550 §5); granted as Principal.AsksItsOwnSecrets says.
var VaultOwn = SeatVerb{Seat: "mesh-vault", Verb: "own"}
// VerbsTheControllerActsOnAWarrant are the other seat verbs the controller calls when the operator's warrant
// chooses them (novox/hq ADR 0259): a machine's service restarted, and a walk started or stopped through the
// controller's own `plans`, a machine's held builds sent by its own `push` and the builds held after a failed release
@@ -882,6 +894,20 @@ func PermissionsFor(p Principal) (Permissions, error) {
return Permissions{}, err
}
pub = append(pub, haveKeys...)
// And every node's own facts, `<node>.node`, its record among them (issue 1126, ADR 0555 §3): a node agent composing
// its own modules reads the other nodes' public records, which its firewall and resolver need (ADR 0547 §2). Read
// only: each node still writes its own keys alone.
pub = append(pub, readGrants(delivery.HaveBucket, "*."+delivery.HaveNodeWord)...)
// And what it composes its own modules from (ADR 0547 §2, ADR 0549 §3): the indexer's placement, modules and
// manifests, the builder's builds (the manifest of a build its want names), and mesh-config's resolved values of
// its own node alone. Read only.
pub = append(pub, NodeComposeReads(p.Node)...)
// And, of the other nodes' resolved values, the keys their modules serve from (issue 1126), each one key.
served, err := ServedReadGrants(p.ServedReads)
if err != nil {
return Permissions{}, err
}
pub = append(pub, served...)
// And an object of the declaration bucket by its name, and nothing of the bucket's list or writing
// (novox/hq ADR 0486 §2, issue 1697).
fileReads, err := DeclarationFilesReads(p.Node)
@@ -915,6 +941,14 @@ func PermissionsFor(p Principal) (Permissions, error) {
if p.StatesItsKeyToTheVault {
pub = append(pub, "mesh.seat."+VaultNodeKey.Seat+"."+CallKind+"."+VaultNodeKey.Verb+"."+CallerToken(p.Username()))
}
// And, while the vault serves it, its modules' own secrets in its own name (issue 1126): sealed to its key, for the
// compositions it makes itself.
if p.AsksItsOwnSecrets {
pub = append(pub, VaultOwnCall(p))
// And the vault's word that a secret changed (names and generations, never a value), so the node composes again
// on it rather than at the next unrelated change (ADR 0547 §2).
sub = append(sub, VaultSecretChanged)
}
// And, on the node holding mesh-broker, the grants part's sealed user list it follows from take-over (novox/hq ADR
// 0547 clause 7): one key, and an object by its name.
if p.FollowsGrantsUserList {
@@ -1784,3 +1818,12 @@ const (
nodeToolsService = "node-tools"
leaseKey = "holder"
)
// VaultOwnCall is the subject a machine principal's node-engine asks the vault's `own` on, in its own name.
func VaultOwnCall(p Principal) string {
return "mesh.seat." + VaultOwn.Seat + "." + CallKind + "." + VaultOwn.Verb + "." + CallerToken(p.Username())
}
// VaultSecretChanged is the vault's event that a secret changed (a name and its generation, never a value), which a
// node composing its own hears to compose again (issue 1126, ADR 0547 §2).
var VaultSecretChanged = "mesh.mod." + VaultOwn.Seat + ".event.secret-changed"
+45
View File
@@ -3,6 +3,7 @@ package buscompose
import (
"fmt"
"git.novox.be/novox/mesh-sdk/go/announce"
"git.novox.be/novox/mesh-sdk/go/delivery"
"sort"
"strings"
"time"
@@ -306,3 +307,47 @@ func RaiseBuckets(a BucketAsserter, buckets []Bucket) (undeclared []string, err
sort.Strings(undeclared)
return undeclared, nil
}
// ResolvedBucket is mesh-config's state of resolved values, keyed `<node>.<module>.<key>`.
const ResolvedBucket = "mesh-config_resolved"
// NodeComposeReads is what a node's agent reads to compose its own modules (novox/hq ADR 0547 §2, ADR 0549 §3, issue
// 1126): every key of the indexer's placement, placement candidates, modules and manifests and of the builder's builds,
// and of mesh-config's resolved values only its own node's, `<node>.>`. Every candidate key, not only `*.<node>`: a node
// answering its candidate composes with the whole group's placement (the seed places every node at once, and a binding
// crosses nodes), and nothing in a placement is secret. Reads only; the node is checked as every subject token is by the
// caller's principal.
func NodeComposeReads(node string) []string {
var out []string
for _, bucket := range []string{delivery.PlacementBucket, delivery.PlacementCandidatesBucket, delivery.ModulesBucket,
delivery.ManifestsBucket, delivery.BuilderBuildsBucket} {
out = append(out, readGrants(bucket, "")...)
}
return append(out, readGrants(ResolvedBucket, node+".>")...)
}
// ServedReadGrants is the read of each key of mesh-config's resolved values named, `<node>.<module>.<key>`
// (Principal.ServedReads, issue 1126): a get and a watch of that key alone. Each part is checked as every subject token
// is, so a key cannot widen the grant.
func ServedReadGrants(keys []string) ([]string, error) {
var out []string
seen := map[string]bool{}
for _, k := range keys {
parts := strings.Split(k, ".")
if len(parts) != 3 {
return nil, fmt.Errorf("%q is not <node>.<module>.<key>", k)
}
for _, part := range parts {
if !safeSubject.MatchString(part) {
return nil, fmt.Errorf("%q cannot be part of a subject: a permission is a subject pattern, and this would widen it", part)
}
}
for _, g := range readGrants(ResolvedBucket, k) {
if !seen[g] {
seen[g] = true
out = append(out, g)
}
}
}
return out, nil
}
+36 -1
View File
@@ -5,6 +5,7 @@ import (
"git.novox.be/novox/mesh-sdk/go/announce"
"slices"
"sort"
"strings"
)
// Every user the composed file should contain, derived from what the mesh knows.
@@ -25,6 +26,9 @@ type Declared struct {
Emits []string
Consumes []string
Serves []string
// Served are the config keys its served values and zone name fill from its settings (catalogue.ServedSettings): what
// every node may read of its resolved values where it runs (Principal.ServedReads, issue 1126).
Served []string
// Holds are the seats this module claims, with the protocol each seat declares. A seat the
// mesh defines for itself declares no protocol, so holding one grants nothing on the bus —
// which is right: those seats are about who does a job, not about who may say what.
@@ -144,6 +148,7 @@ func Users(r Records) ([]Principal, error) {
out := []Principal{{Kind: KindController, Uses: asksSeatOf(r), AsksIssued: issuedOf(r), AsksTheVault: vaultVerbsOf(r),
AsksTheConfig: servedOf(r, VerbsTheControllerAsksTheConfig), FilesFor: sortedCopy(r.Nodes)}}
served := servedReadsOf(r)
for _, node := range sortedCopy(r.Nodes) {
witness := false
var checks []string
@@ -159,7 +164,8 @@ func Users(r Records) ([]Principal, error) {
}
out = append(out, Principal{Kind: KindNode, Node: node, WitnessesController: witness, Checks: checks,
MovesIntoTheVault: servesMoveIn(r), StatesItsKeyToTheVault: servesVerb(r, VaultNodeKey),
FollowsGrantsUserList: bus})
AsksItsOwnSecrets: servesVerb(r, VaultOwn),
FollowsGrantsUserList: bus, ServedReads: servedOn(served, node)})
// **And the machine's provisioner client, on an account of its own** (novox/hq ADR 0349 clause 11): one
// per machine that runs it, delivered to the module running it.
for _, d := range r.Assigned[node] {
@@ -458,3 +464,32 @@ func declaredStreamsOf(r Records) []string {
sort.Strings(out)
return out
}
// servedReadsOf is every key of mesh-config's resolved values a module assigned on a node serves from,
// `<node>.<module>.<key>`, sorted: each part a subject token, so a key that cannot be one is granted nothing (and the
// node composing a consumer of it refuses, naming it), never a wider read.
func servedReadsOf(r Records) []string {
var out []string
for node, declared := range r.Assigned {
for _, d := range declared {
for _, k := range d.Served {
if safeSubject.MatchString(node) && safeSubject.MatchString(d.Module) && safeSubject.MatchString(k) {
out = append(out, node+"."+d.Module+"."+k)
}
}
}
}
sort.Strings(out)
return slices.Compact(out)
}
// servedOn is, of the served keys, those of other nodes than node: its own slice it reads whole.
func servedOn(served []string, node string) []string {
var out []string
for _, k := range served {
if !strings.HasPrefix(k, node+".") {
out = append(out, k)
}
}
return out
}
+37
View File
@@ -0,0 +1,37 @@
package catalogue
// The delivery's checker (novox/hq ADR 0547 §10, issue 774): one holder for the mesh, the module
// mesh-delivery-checker, which judges a pull request's head and a pushed head the same way. The try the
// controller forwarded to it is a verb of this seat, answered by the checker with the verdict it gives a pull
// request, so the try outlives the controller.
// CheckerSeat is the seat mesh-delivery-checker holds.
const CheckerSeat = "mesh-delivery-checker"
// checkerVerbs are the checker seat's verbs: ask a try, read one, end one by hand. Optional until the checker serves them
// everywhere.
func checkerVerbs() []Verb {
return []Verb{
{Name: "try", Optional: true, Description: "Try a pushed head before its pull request is opened: the checker " +
"judges it as it judges a pull request's head, merged onto its trunk's head, its unit and integration " +
"levels through the build modules, and owes the forge nothing. Answers the try's id at once; try-read " +
"reads where it stands. A try of the same head that waits or runs is the answer to a second ask; at most " +
"4 wait or run at once, mesh-wide.",
Input: schema(map[string]string{"repository": "<owner>/<name>",
"head": "the full commit to try, 40 or 64 hex digits",
"trunk": "the trunk commit to merge it onto, or the trunk's branch for its head now; the trunk's head when absent",
"tests": "a go test -run pattern: only those tests run, and nothing is kept",
"level": "unit or integration: only this level; both when absent"},
[]string{"repository", "head"}),
Replaces: []string{"sh merge-check.sh"}},
{Name: "try-read", Optional: true, Description: "Where a try stands, as stored: queued, running, or its end " +
"(pass, fail, or error when it could not run), with what was said of it, the head and the trunk commit " +
"it was merged onto, the time it took and what failed first. Read-only.",
Input: schema(map[string]string{"id": "the try's id, try-<digits>"}, []string{"id"})},
{Name: "try-end", Optional: true, Description: "End a try that waits or runs, as error, by hand: say why (kept in " +
"what it says). Its place among the tries is free at once and the next try of its head asks anew with a new " +
"id. A try that ended already is refused.",
Input: schema(map[string]string{"id": "the try's id, try-<digits>", "why": "why it is ended, in words"},
[]string{"id", "why"})},
}
}
+13
View File
@@ -75,6 +75,19 @@ func deliveryVerbs() []Verb {
Input: schema(map[string]string{"id": "the delivery's id", "why": "the condition it answers"},
[]string{"id", "why"})},
deliveryRemovedInR6("retire-history"),
{Name: "settle", Description: "Ask the operator to settle deliveries held because their merge reached the " +
"trunk without a passing check: one ask on the operator's channel for all of them, as an Approve is asked. " +
"The verb moves nothing; on the operator's answer each goes to merged, and whether it is on the machines " +
"is derived as for any merge (novox/hq ADR 0556). Refused for a delivery not held, or held for another " +
"reason, naming why.",
Input: map[string]any{"type": "object", "additionalProperties": false, "required": []string{"ids", "why"},
"properties": map[string]any{
"ids": map[string]any{"type": "array", "items": map[string]any{"type": "string"}, "minItems": 1,
"description": "the held deliveries' ids (owner/repository@commit)"},
"why": map[string]any{"type": "string", "description": "why they may be settled, shown to the operator"},
}},
// Added after the seat's release 6 holder shipped: optional until the gate serves it everywhere.
Optional: true},
}
}
+25
View File
@@ -0,0 +1,25 @@
package catalogue
import "strings"
// LauncherSuffix is what a TypeScript entrypoint's launcher is called beside it: index.js is started as
// index.serve.mjs (novox/hq ADR 0193), an ES module by its own extension whatever the bundle's package.json says.
const LauncherSuffix = ".serve.mjs"
// Launchers are the launchers a bundle's build writes, by entrypoint, relative to the bundle's root: for TypeScript
// every entrypoint `x.js` is started by `x.serve.mjs` beside it; an entrypoint that is not JavaScript and every other
// language, whose build is already executable, get none. The one rule mesh-build-node, build-agent, the controller and
// a node taking in a build share (R4e, issue 1126; buildstep.Launchers says it for the build modules), so the name a
// build writes is the name a reader expects.
func Launchers(language string, entrypoints []string) map[string]string {
out := map[string]string{}
if language != "typescript" {
return out
}
for _, entry := range entrypoints {
if strings.HasSuffix(entry, ".js") {
out[entry] = strings.TrimSuffix(entry, ".js") + LauncherSuffix
}
}
return out
}
+12
View File
@@ -209,6 +209,16 @@ var defaultSeats = append([]Seat{
"first or changed key is recorded and the machine's core secrets are said changed; the same key " +
"again changes nothing.",
Input: schema(map[string]string{"key": "the machine's sealing key, a raw X25519 public key in base64"}, nil)},
// A machine's node-engine composes its own modules and asks their secrets sealed to its key (novox/hq ADR 0547
// §2, ADR 0550 §5, issue 1126); granted as buscompose's Principal.AsksItsOwnSecrets says.
{Name: "own", Optional: true, Description: "Answer the node-engine of one machine, about its own machine " +
"alone (as the bus proved the caller, never an argument), its modules' own secrets and the pair " +
"credentials it consumes or provides that it names, each sealed to the key it stated with node-key. A " +
"credential the vault does not hold is named missing; never a value in the clear.",
Input: map[string]any{"type": "object", "properties": map[string]any{"credentials": map[string]any{
"type": "array", "items": map[string]any{"type": "object"},
"description": "the pair credentials the machine's composition reads, each {role: consumer|provider, " +
"provision, consumer, consumer_module, provider, local}"}}}},
{Name: "move-asks", Optional: true, Description: "Copy rows of the controller's record of asks at a desk " +
"into the vault's, as they are and once, from the controller alone. No value is in any of them.",
Input: map[string]any{"type": "object", "properties": map[string]any{"asks": map[string]any{
@@ -248,6 +258,8 @@ var defaultSeats = append([]Seat{
// machine, and a group of them, recorded and ordered by its holder, which asks the controller for each act.
// What it says it says as its module (`mesh-delivery.transition`), through the runtime that launches it.
{Name: DeliverySeat, Scope: ScopeMesh, Serves: deliveryVerbs(), Decision: "novox/hq ADR 0239"},
// The delivery's checker (novox/hq ADR 0547 §10, issue 774): the try is its verb, not the controller's.
{Name: CheckerSeat, Scope: ScopeMesh, Serves: checkerVerbs(), Decision: "novox/hq ADR 0547"},
// Deferred renames (novox/hq ADR 0121): these deliver a provision, so renaming them is a
// delivering-seat migration with a mesh-wide cascade if a holder stops resolving mid-flight.
// They keep their names until that migration is done deliberately, apart from the node-* pass.
+18 -2
View File
@@ -43,6 +43,21 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
delivered[s.Delivers] = s.Name
}
}
// The checker's seat serves the try, which replaces the shell's merge check (ADR 0547 §10, ADR 0245).
var checkerVerbs []string
for _, s := range Seats() {
if s.Name == CheckerSeat {
for _, v := range s.Serves {
checkerVerbs = append(checkerVerbs, v.Name)
if v.Name == "try" && strings.Join(v.Replaces, "|") != "sh merge-check.sh" {
t.Errorf("try replaces %v", v.Replaces)
}
}
}
}
if strings.Join(checkerVerbs, ",") != "try,try-read,try-end" {
t.Errorf("the checker seat serves %v", checkerVerbs)
}
// Forty-six with the three build seats replacing node-build-agent and the retired mesh-build-machine (novox/hq
// ADR 0525 §2, §10); forty-five with node-files (novox/hq ADR 0394); forty-four again once node-reads goes.
// Forty-four with node-provisioner (novox/hq ADR 0349 clause 3); forty-three with mesh-provisioner (novox/hq ADR 0349 clause 3); forty-two with mesh-config (novox/hq ADR 0348 §9); forty-one with node-reads (novox/hq ADR 0310); forty with node-nfs-server and node-mounts (novox/hq ADR 0263); thirty-eight with node-resolver (novox/hq ADR 0247); thirty-seven with mesh-delivery (novox/hq ADR 0239); thirty-six since node-resolver-config retired
@@ -56,8 +71,9 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
// node-container-runtime (ADR 0207); nineteen with node-environment and node-login-shell (ADR 0203,
// ADR 0204); seventeen with node-build-agent (ADR 0190). One fewer once the retired
// mesh-build-machine row goes, when no registered manifest claims it.
if len(Seats()) != 46 {
t.Errorf("the mesh defines %d seats rather than 46; the set is closed, so a change here is "+
// Forty-seven with mesh-delivery-checker (novox/hq ADR 0547 §10, issue 774).
if len(Seats()) != 47 {
t.Errorf("the mesh defines %d seats rather than 47; the set is closed, so a change here is "+
"a decision (novox/hq ADR 0110): %s", len(Seats()), seatNames())
}
}
+24
View File
@@ -132,6 +132,30 @@ func Settle(base map[string]any, layers []Layer) (map[string]any, error) {
return filledFromSettings(deepCopy(base), layers, "what is served")
}
// ServedSettings is every key a module's served values and its zone's name fill from its settings (`${setting:<key>}`
// in a top-level string, as Settle fills them), sorted: what every consumer of the module is told, wherever it runs,
// and so the one part of a module's config values on its node that a node composing a consumer elsewhere reads (issue
// 1126). None for a module that serves only literals.
func ServedSettings(m Manifest) []string {
seen := map[string]bool{}
note := func(v any) {
if s, ok := v.(string); ok {
for _, k := range settingsUsed(s) {
seen[k] = true
}
}
}
for _, values := range m.Serves {
for _, v := range values {
note(v)
}
}
if m.Zone != nil {
note(m.Zone.Name)
}
return sortedKeys(seen)
}
// overridden lays settings over a contribution: a map whose keys are its contract. Only the keys the
// map already declares are touched; the rest of a layer is somebody else's business (a file's,
// another destination's) and is left to reach it there.
+3 -1
View File
@@ -121,6 +121,8 @@ func TestTheVaultsSeatPromisesTheMove(t *testing.T) {
"move-in": {"node", "module", "name", "provider", "provision", "consumer", "consumer_module", "local", "origin",
"generation", "made_at", "call", "sealed", "read_back_to"},
"move-asks": {"asks"},
// And own (issue 1126): a machine's node-engine asks its own machine's secrets, sealed to its stated key.
"own": {"credentials"},
}
for name, args := range want {
var verb *Verb
@@ -148,7 +150,7 @@ func TestTheVaultsSeatPromisesTheMove(t *testing.T) {
}
serving := Manifest{Module: "mesh-vault", Provides: []Offer{{Name: "secret", Scope: ScopeMesh}},
Claims: []Claim{{Name: "mesh-vault", Scope: ScopeMesh,
Serves: []string{"get", "accept", "secret-ask", "issued", "rotate", "move-in", "move-asks"}}}}
Serves: []string{"get", "accept", "secret-ask", "issued", "rotate", "move-in", "node-key", "move-asks", "own"}}}}
if err := CanHold(serving, seat); err != nil {
t.Fatalf("the vault serving the move was refused: %v", err)
}
+7
View File
@@ -126,6 +126,13 @@ var ControllerVerbs = []Verb{
{Name: "node", Description: "What one machine reported it can do, what it is assigned, and why.",
Input: schema(map[string]string{"node": "the machine's name"}, []string{"node"}),
Replaces: []string{"hostnamectl", "uptime"}},
{Name: "placement-export", Description: "Each machine's placement as the catalogue's placement file says it (novox/hq ADR 0555 " +
"§6, issue 1126): what a person decided of the machine (endpoint, site, hub, address on the private network, public " +
"domain, accounts, adoption, the tunnel found at adoption) and where its modules run (in order, with host ports, " +
"bindings, config placement layers, pins and holdings), read by the controller's own code from its own store, as JSON. " +
"Read-only; the one verb the controller gains for its exit (the operator, 2026-10-26 19:06:40), read once by the seed " +
"of the placement files.",
Input: schema(map[string]string{"node": "one machine's name; every machine when absent"}, nil)},
{Name: "modules", Description: "Every module the mesh holds: version, the commit it was built from, " +
"and which machines run it.",
Input: schema(nil, nil)},
+1 -1
View File
@@ -86,7 +86,7 @@ func TestTheDeliverySeatHasNoPlanVerbs(t *testing.T) {
if _, ok := rows["mark-ready"]; ok {
t.Errorf("mark-ready is still promised")
}
for _, kept := range []string{"deliveries", "show", "groups", "checks", "table", "recheck", "close", "stalled"} {
for _, kept := range []string{"deliveries", "show", "groups", "checks", "table", "recheck", "close", "stalled", "settle"} {
if v, ok := rows[kept]; !ok || strings.HasPrefix(v.Description, "Removed") {
t.Errorf("%s is not kept (ADR 0539 clause 17)", kept)
}
+139
View File
@@ -0,0 +1,139 @@
// placement-seed writes the catalogue's first placement files (novox/hq ADR 0549 §4, ADR 0555 §6, issue 1126) from the
// answer of the controller's one read-only verb `placement-export`: one `placement/<node>.json` per node, each the
// node's placement as the controller's own code read it from its own store, checked as the catalogue's check checks it
// (delivery.CheckPlacements) before anything is written. The pull request it makes merges only when its composition check
// composes every node equal to what it runs.
//
// placement-seed -catalogue <checkout of mesh-catalog> < export.json
// placement-seed -catalogue <dir> -export export.json
//
// export.json is the verb's answer as it prints it ({"<node>": <placement>, ...}, or the verb's wrapping with it under
// "answer"). It opens no network connection, bus or store; it writes only <dir>/placement/<node>.json, and refuses to
// write over a placement file already there. Exit codes: 0 written; 1 the export cannot be read, is refused by the
// check, or a file is there; 2 the flags are wrong.
package main
import (
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"os"
"path/filepath"
"sort"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
func main() {
catalogue := flag.String("catalogue", "", "the checkout of the catalogue to write placement/<node>.json into")
export := flag.String("export", "", "the verb's answer; standard input when absent")
flag.Parse()
if *catalogue == "" || flag.NArg() > 0 {
fmt.Fprintln(os.Stderr, "usage: placement-seed -catalogue <dir> [-export <file>]")
os.Exit(2)
}
in := io.Reader(os.Stdin)
if *export != "" {
f, err := os.Open(*export)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
defer f.Close()
in = f
}
written, err := Seed(in, *catalogue)
for _, path := range written {
fmt.Println(path)
}
if err != nil {
fmt.Fprintln(os.Stderr, "placement-seed:", err)
os.Exit(1)
}
}
// Seed reads the export and writes each node's placement file under root, answering the paths written. Nothing is
// written when the export does not read, the check refuses it, or a placement file is already there.
func Seed(export io.Reader, root string) ([]string, error) {
raw, err := io.ReadAll(export)
if err != nil {
return nil, err
}
placements, err := read(raw)
if err != nil {
return nil, err
}
all := func(string) bool { return true }
if refused := delivery.CheckPlacements(placements, all, all); len(refused) > 0 {
return nil, fmt.Errorf("the export is refused by the catalogue's check: %v", refused)
}
nodes := make([]string, 0, len(placements))
for node := range placements {
nodes = append(nodes, node)
}
sort.Strings(nodes)
files := map[string][]byte{}
for _, node := range nodes {
p := placements[node]
p.Revision = "" // only the indexer writes a revision, into the key
if _, err := delivery.EncodePlacement(p); err != nil {
return nil, fmt.Errorf("%s: %w", node, err)
}
p.V = delivery.Version
body, err := json.MarshalIndent(p, "", " ")
if err != nil {
return nil, err
}
path := filepath.Join(root, filepath.FromSlash(delivery.PlacementFile(node)))
if _, err := os.Stat(path); err == nil {
return nil, fmt.Errorf("%s is already there; the seed is written once (ADR 0549 §4)", path)
} else if !errors.Is(err, os.ErrNotExist) {
return nil, err
}
files[path] = append(body, '\n')
}
var written []string
for _, node := range nodes {
path := filepath.Join(root, filepath.FromSlash(delivery.PlacementFile(node)))
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return written, err
}
if err := os.WriteFile(path, files[path], 0o644); err != nil {
return written, err
}
written = append(written, path)
}
return written, nil
}
// read is the export as the verb prints it, or wrapped as a verb's answer (the controller's verbAnswer: its JSON under
// "answer").
func read(raw []byte) (map[string]delivery.Placement, error) {
var wrapped struct {
Answer json.RawMessage `json:"answer"`
}
if json.Unmarshal(raw, &wrapped) == nil && len(wrapped.Answer) > 0 {
raw = wrapped.Answer
}
var byNode map[string]json.RawMessage
if err := json.Unmarshal(raw, &byNode); err != nil {
return nil, fmt.Errorf("the export is not {<node>: <placement>}: %w", err)
}
if len(byNode) == 0 {
return nil, errors.New("the export names no node")
}
out := map[string]delivery.Placement{}
for node, body := range byNode {
p, err := delivery.DecodePlacement(body)
if err != nil {
return nil, fmt.Errorf("%s: %w", node, err)
}
if p.Node != node {
return nil, fmt.Errorf("the export's placement of %s names the node %q", node, p.Node)
}
out[node] = p
}
return out, nil
}
+61
View File
@@ -0,0 +1,61 @@
package main
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/compose/frombus"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// Issue 1126 (ADR 0549 §4, ADR 0555 §6): the seed writes one placement file per node of the export, as the catalogue's
// check and the indexer read it, from the verb's answer bare or wrapped; it writes nothing for an export the check
// refuses, and never over a file already there.
func TestTheSeedWritesEachNodesPlacementFileOnce(t *testing.T) {
export := `{"answer":{"ace":{"v":1,"node":"ace","revision":"x","decided":{"address":"10.77.0.2"},"modules":[{"module":"plex","ports":{"32400":32400}}]},` +
`"novox":{"v":1,"node":"novox","decided":{"hub":true,"address":"10.77.0.1"},"modules":[]}}}`
root := t.TempDir()
written, err := Seed(strings.NewReader(export), root)
if err != nil || len(written) != 2 {
t.Fatalf("written %v, %v", written, err)
}
raw, err := os.ReadFile(filepath.Join(root, "placement", "ace.json"))
if err != nil {
t.Fatal(err)
}
p, err := delivery.DecodePlacement(raw)
if err != nil || p.Node != "ace" || p.Revision != "" || p.HostPorts()["plex"][32400] != 32400 {
t.Errorf("ace's file reads %+v, %v", p, err)
}
if _, err := Seed(strings.NewReader(export), root); err == nil || !strings.Contains(err.Error(), "already there") {
t.Errorf("a second seed over the files: %v", err)
}
clash := `{"ace":{"v":1,"node":"ace","decided":{"address":"10.77.0.1"},"modules":[]},"novox":{"v":1,"node":"novox","decided":{"address":"10.77.0.1"},"modules":[]}}`
other := t.TempDir()
if _, err := Seed(strings.NewReader(clash), other); err == nil || !strings.Contains(err.Error(), "both have the address") {
t.Errorf("an export the check refuses: %v", err)
}
if entries, _ := os.ReadDir(filepath.Join(other, "placement")); len(entries) != 0 {
t.Errorf("a refused export wrote %v", entries)
}
// What the controller's placement-export prints, frombus.Export per node, is what the seed takes: a node with no
// module placed included.
in := compose.Input{Nodes: []compose.Node{{Name: "solo", Place: compose.Place{Hub: true, Address: "10.77.0.9"}}}}
exported, err := frombus.Export(in, "solo")
if err != nil {
t.Fatal(err)
}
answer, _ := json.Marshal(map[string]any{"answer": map[string]delivery.Placement{"solo": exported}})
if written, err := Seed(strings.NewReader(string(answer)), t.TempDir()); err != nil || len(written) != 1 {
t.Errorf("Export's answer %s was not seeded: %v, %v", answer, written, err)
}
for _, bad := range []string{`[]`, `{}`, `{"ace":{"v":1,"node":"novox","modules":[]}}`} {
if _, err := Seed(strings.NewReader(bad), t.TempDir()); err == nil {
t.Errorf("%s was seeded", bad)
}
}
}
+107 -11
View File
@@ -2,42 +2,106 @@ package compose
import (
"encoding/json"
"errors"
"fmt"
"slices"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/declaration"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/overlay"
)
// Compositions is one node's compositions (novox/hq ADR 0539 clauses 1 and 2): its declaration composed over in, as
// the controller composes it, split into each module's composition at the fingerprint fps gives it and the node's own
// (`<node>.node`), by key (delivery.ComposedKey). Each resource says its module, whatever the node reported it reads.
// A module fps gives no fingerprint for is not composed and is named in skipped, so a caller composing one module
// (mesh-hand-compose) gets that one. Each module's Data says whether its manifest declares data. The compositions are not
// (mesh-hand-compose) gets that one; but for a module the node runs inside its own composition (Node.InOwn, and the
// private network's always), which fps names at no fingerprint of the new form: its resources go into the node's own,
// their module taken off and their ids still naming it, and its filled paths with them, as the controller's composed
// writer folds them (issue 1126). Each module's Data says whether its manifest declares data. The compositions are not
// signed and not yet by digest: the writer does both (delivery.SignComposed, Composed.ByDigest).
func Compositions(in Input, node string, fps map[string]string, mode Mode) (out map[string]delivery.Composed, skipped []string, err error) {
out, skipped, _, err = compositions(in, node, fps, mode, false)
return out, skipped, err
}
// ErrStoodIn is a composition that holds a stand-in for a value the input does not carry (an own secret, a grant's
// sealed copy, a vault box, the certificate, the bus's user list) or a port it does not give: what a send would make,
// never what may be applied.
var ErrStoodIn = errors.New("compose: the composition stands in for what its input does not carry")
// BusUsersStandIn is what a composition says it stood in for when its input carries no bus user list.
const BusUsersStandIn = "the bus's user list"
// CompositionsApplied is Compositions for a node that applies what it composed itself (novox/hq ADR 0547 §2, issue
// 1126): a declaration that stands in for anything (Declared.StoodIn, Foreseen, ForeseenPorts) is refused with
// ErrStoodIn, naming each, so a stand-in is never applied as if it were the value; but for the bus's user list
// (BusUsersStandIn), which the bus node holds out of every apply of a composition it made itself.
func CompositionsApplied(in Input, node string, fps map[string]string) (map[string]delivery.Composed, []string, error) {
out, skipped, _, err := compositions(in, node, fps, Reading, true)
return out, skipped, err
}
// CompositionsShadow is CompositionsApplied for a composition that is compared and never applied (a node's answer to a
// placement candidate, novox/hq ADR 0559 §4): composed in the Foreseeing mode, so an own secret never issued (a module
// the candidate adds) is a named stand-in rather than an error, and what it stands in for is answered (stood, sorted,
// the bus's user list left out) rather than refused, so the caller judges each stand-in: a value not yet issued for a
// module the candidate adds, or one missing for a module the node runs. A module whose every value is held composes
// as CompositionsApplied composes it: the modes differ only in an own secret never issued.
func CompositionsShadow(in Input, node string, fps map[string]string) (out map[string]delivery.Composed, skipped, stood []string, err error) {
return compositions(in, node, fps, Foreseeing, false)
}
func compositions(in Input, node string, fps map[string]string, mode Mode, applied bool) (out map[string]delivery.Composed, skipped, stood []string, err error) {
reading := in
reading.Nodes = append([]Node(nil), in.Nodes...)
found := false
inOwn := map[string]bool{}
for i := range reading.Nodes {
if reading.Nodes[i].Name == node {
reading.Nodes[i].ReadsModules = true
found = true
for m, ok := range reading.Nodes[i].InOwn {
inOwn[m] = ok
}
}
}
// The private network's module is built by nobody: no fingerprint of the new form ever names it.
inOwn[overlay.Name] = true
folds := func(module string) bool {
return module != "" && inOwn[module] && !strings.HasPrefix(fps[module], delivery.SourcePrefix2)
}
if !found {
return nil, nil, fmt.Errorf("compose: the mesh has no node %q", node)
return nil, nil, nil, fmt.Errorf("compose: the mesh has no node %q", node)
}
d, problems, err := ComposeIn(reading, node, mode)
if err != nil {
return nil, nil, err
return nil, nil, nil, err
}
if len(problems) > 0 {
return nil, nil, fmt.Errorf("compose: %s's declaration is refused by the node-engine's validator: %v", node, problems)
return nil, nil, nil, fmt.Errorf("compose: %s's declaration is refused by the node-engine's validator: %v", node, problems)
}
for _, x := range d.StoodIn {
// The bus's user list is the bus node's own to follow, written alone and never from a composition the node
// made itself (ADR 0547 §7, §8): its stand-in is held out of the apply, not applied.
if x != BusUsersStandIn {
stood = append(stood, x)
}
}
stood = append(stood, d.Foreseen...)
for _, p := range d.ForeseenPorts {
stood = append(stood, "port "+p)
}
sort.Strings(stood)
stood = slices.Compact(stood) // an own secret is both stood in for and foreseen
if applied && len(stood) > 0 {
return nil, nil, nil, fmt.Errorf("%w on %s: %s", ErrStoodIn, node, strings.Join(stood, ", "))
}
body, err := d.Body()
if err != nil {
return nil, nil, err
return nil, nil, nil, err
}
var envelope struct {
Resources []json.RawMessage `json:"resources"`
@@ -46,10 +110,11 @@ func Compositions(in Input, node string, fps map[string]string, mode Mode) (out
Provisioner string `json:"provisioner_account"`
}
if err := json.Unmarshal(body, &envelope); err != nil {
return nil, nil, err
return nil, nil, nil, err
}
var kept []json.RawMessage
keptPaths := map[string]bool{}
foldedPaths := map[string]bool{}
left := map[string]bool{}
for _, raw := range envelope.Resources {
var head struct {
@@ -58,7 +123,24 @@ func Compositions(in Input, node string, fps map[string]string, mode Mode) (out
Path string `json:"path"`
}
if err := json.Unmarshal(raw, &head); err != nil {
return nil, nil, err
return nil, nil, nil, err
}
if folds(head.Module) {
// Into the node's own, in its place among the node's resources, its module taken off (mesh-controller's
// compositionsOf): the node's own composition is then the controller's, byte for byte.
var fields map[string]json.RawMessage
if err := json.Unmarshal(raw, &fields); err != nil {
return nil, nil, nil, err
}
delete(fields, "module")
if raw, err = json.Marshal(fields); err != nil {
return nil, nil, nil, err
}
if head.Type == string(declaration.TypeFile) {
foldedPaths[head.Path] = true
}
kept = append(kept, raw)
continue
}
if _, ok := fps[head.Module]; head.Module != "" && !ok {
left[head.Module] = true
@@ -69,16 +151,30 @@ func Compositions(in Input, node string, fps map[string]string, mode Mode) (out
}
kept = append(kept, raw)
}
var filled []string
// A filled file of a module's own composition goes with that module (SplitComposed); one of a folded module goes with
// the node's own, after them, in the declaration's order; one of a module left out is dropped.
var filled, ownFilled []string
for _, p := range envelope.Filled {
if keptPaths[p] {
switch {
case foldedPaths[p]:
ownFilled = append(ownFilled, p)
case keptPaths[p]:
filled = append(filled, p)
}
}
out, err = delivery.SplitComposed(node, kept, fps, filled,
delivery.NodeOwn{Adoption: envelope.Adoption, Provisioner: envelope.Provisioner})
if err != nil {
return nil, nil, err
return nil, nil, nil, err
}
if len(ownFilled) > 0 {
own, err := delivery.ComposedNodeKey(node)
if err != nil {
return nil, nil, nil, err
}
c := out[own]
c.Filled = append(c.Filled, ownFilled...)
out[own] = c
}
for key, c := range out {
if m, ok := in.Catalogue[c.Module]; ok && c.Module != "" {
@@ -90,5 +186,5 @@ func Compositions(in Input, node string, fps map[string]string, mode Mode) (out
skipped = append(skipped, m)
}
sort.Strings(skipped)
return out, skipped, nil
return out, skipped, stood, nil
}
+29 -3
View File
@@ -58,15 +58,41 @@ func TestCompositionsAreTheDeclarationSplitByModule(t *testing.T) {
t.Errorf("the compositions hold %v, the declaration %v", got, want)
}
one, skipped, err := Compositions(in, "laptop", map[string]string{"files": fpFor('b')}, Foreseeing)
if err != nil || strings.Join(skipped, ",") != "mesh-wireguard" {
one, skipped, err := Compositions(in, "laptop", map[string]string{"mesh-wireguard": fpFor('a')}, Foreseeing)
if err != nil || strings.Join(skipped, ",") != "files" {
t.Fatalf("one module: %v, skipped %v", err, skipped)
}
for _, c := range one {
if c.Module == "mesh-wireguard" {
if c.Module == "files" {
t.Errorf("a module given no fingerprint was composed")
}
}
// Issue 1126: a module the node runs inside its own composition (Node.InOwn), and the private network's always,
// given no fingerprint of the new form, is folded into the node's own as the controller's composed writer folds it:
// every resource in its place, its module taken off, its id still naming it, nothing skipped.
folding := in
folding.Nodes = append([]Node(nil), in.Nodes...)
for i := range folding.Nodes {
if folding.Nodes[i].Name == "laptop" {
folding.Nodes[i].InOwn = map[string]bool{"files": true}
}
}
all, skipped, err := Compositions(folding, "laptop", nil, Foreseeing)
nodeKey, _ := delivery.ComposedNodeKey("laptop")
if err != nil || len(skipped) != 0 || len(all) != 1 || len(all[nodeKey].Resources) != len(want) {
t.Fatalf("folded: %v, skipped %v, %d compositions, %d of %d resources in the node's own", err, skipped, len(all), len(all[nodeKey].Resources), len(want))
}
for i, raw := range all[nodeKey].Resources {
var r map[string]any
_ = json.Unmarshal(raw, &r)
if _, says := r["module"]; says || r["id"] != d.Resources[i]["id"] {
t.Errorf("folded resource %d is %v, the declaration's %v", i, r["id"], d.Resources[i]["id"])
}
}
if _, err := delivery.EncodeComposed(all[nodeKey]); err != nil {
t.Errorf("the node's own with the folded modules: %v", err)
}
if _, _, err := Compositions(in, "nowhere", fps, Foreseeing); err == nil {
t.Errorf("a node the mesh does not have was composed")
}
+64
View File
@@ -0,0 +1,64 @@
package compose
import (
"fmt"
"sort"
)
// CredentialKey names one pair credential by the ends it joins: the provision, the consumer's node and module, the
// provider's node, and the local name where a module keeps several (Credential's and Issued's key).
type CredentialKey struct {
Provision string `json:"provision"`
Consumer string `json:"consumer"`
ConsumerModule string `json:"consumer_module"`
Provider string `json:"provider"`
Local string `json:"local,omitempty"`
}
// CredentialsNeeded is every pair credential node's composition reads (novox/hq ADR 0547 §2, ADR 0550 §5, issue 1126):
// as a consumer, each need of its resolution a provider's credential answers (a need answered by a record is a licence's,
// not a pair's); as a provider, each credential it must carry a grant for (rowsFor). A node that composes its own asks the
// vault for exactly these, sealed to its key, and composes with them; one it does not get is stood in for, which the
// node's own apply refuses (CompositionsApplied).
func CredentialsNeeded(in Input, node string) (asConsumer, asProvider []CredentialKey, err error) {
c := newComposer(in)
resolved, err := c.resolution(node)
if err != nil {
return nil, nil, err
}
for _, n := range resolved.Needs {
if n.ByRecord || n.From == "" {
continue
}
asConsumer = append(asConsumer, CredentialKey{Provision: n.Name, Consumer: node, ConsumerModule: n.For,
Provider: n.From, Local: n.Local})
}
rows, err := c.rowsFor(node)
if err != nil {
return nil, nil, fmt.Errorf("compose: the credentials %s provides: %w", node, err)
}
for _, r := range rows {
asProvider = append(asProvider, CredentialKey{Provision: r.Provision, Consumer: r.Consumer,
ConsumerModule: r.ConsumerModule, Provider: node, Local: r.Local})
}
sortKeys(asConsumer)
sortKeys(asProvider)
return asConsumer, asProvider, nil
}
func sortKeys(ks []CredentialKey) {
sort.Slice(ks, func(i, j int) bool {
a, b := ks[i], ks[j]
switch {
case a.Provision != b.Provision:
return a.Provision < b.Provision
case a.Provider != b.Provider:
return a.Provider < b.Provider
case a.Consumer != b.Consumer:
return a.Consumer < b.Consumer
case a.ConsumerModule != b.ConsumerModule:
return a.ConsumerModule < b.ConsumerModule
}
return a.Local < b.Local
})
}
+20
View File
@@ -2,6 +2,7 @@ package compose
import (
"encoding/json"
"errors"
"strings"
"testing"
@@ -78,6 +79,16 @@ func TestAnOwnSecretHeldIsCarriedAndOneNotHeldIsForeseenOrLeftOut(t *testing.T)
t.Errorf("a held secret with no value is not named: %v", got.Rendering.Foreseen)
}
unchanged(t, mk, func(in Input) { rendered(t, in, "laptop", Foreseeing) })
// 1126 (ADR 0547 §2): a node applying what it composed itself never applies a stand-in: a held secret with no value in the
// Input is refused, naming it; held with its value, it composes.
fps := map[string]string{"assistant": "src2:" + strings.Repeat("a", 64), overlay.Name: "src2:" + strings.Repeat("b", 64)}
if _, _, err := CompositionsApplied(in, "laptop", fps); !errors.Is(err, ErrStoodIn) || !strings.Contains(err.Error(), "assistant/password") {
t.Errorf("a stand-in was not refused for a node's own apply: %v", err)
}
if out, _, err := CompositionsApplied(mk(), "laptop", fps); err != nil || len(out) == 0 {
t.Errorf("a held secret with its value is refused for a node's own apply: %v", err)
}
}
// An own secret issued outside the mesh that nobody gave is never made (novox/hq issue 401, ADR 0228): nothing is composed or
@@ -161,6 +172,15 @@ func aGrantMesh(t *testing.T) Input {
// the sealed copy a stand-in is composed and named (novox/hq plan.go grantsFor).
func TestAGrantIsCreatedForEachCredentialMadeForAConsumer(t *testing.T) {
in := aGrantMesh(t)
// 1126 (ADR 0550 §5): the credentials each node's composition reads, by their ends: laptop's files consumes anchor's
// s3-bucket, and anchor grants it; the node composing its own asks the vault for exactly these.
pair := CredentialKey{Provision: "s3-bucket", Consumer: "laptop", ConsumerModule: "files", Provider: "anchor"}
if consumer, provider, err := CredentialsNeeded(in, "laptop"); err != nil || len(provider) != 0 || len(consumer) != 1 || consumer[0] != pair {
t.Errorf("laptop needs %v as consumer and %v as provider (%v), want %v as consumer", consumer, provider, err, pair)
}
if consumer, provider, err := CredentialsNeeded(in, "anchor"); err != nil || len(consumer) != 0 || len(provider) != 1 || provider[0] != pair {
t.Errorf("anchor needs %v as consumer and %v as provider (%v), want %v as provider", consumer, provider, err, pair)
}
got := rendered(t, in, "anchor", Foreseeing)
if len(got.Rendering.Grants) != 1 {
t.Fatalf("grants are %+v", got.Rendering.Grants)
+172
View File
@@ -0,0 +1,172 @@
package frombus
import (
"context"
"fmt"
"regexp"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// Answer is a node's answer to its group's placement candidates (novox/hq ADR 0559 §4): s.Node composed in shadow with
// s.Placements laid over the trunk's placement (compose.CompositionsShadow, its own sealed values and boxes through r as
// for its own composition), compared with what it runs (delivery.CompareComposed). It applies nothing and writes
// nothing; head and revision are the candidate's, said back in the answer.
//
// The node's own composition is compared only where the node runs one. fps gives the build each module is composed at: what the node runs, and for a module the candidate places that the
// node does not run, the build the trunk holds; a placed module with none is added, with no build of it known. What the
// composition stands in for is judged: a value of a module the group adds (on this node or another: its credential, own
// secrets and boxes are made only once it is placed, ADR 0549 §5) is not yet issued, which is no refusal; any other
// stand-in is a value missing for a module the node runs, a refusal naming it ("node" when it is the node's own).
func Answer(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]string, running []delivery.Composed, head string, revision uint64) delivery.CandidateAnswer {
a := delivery.CandidateAnswer{Head: head, Revision: revision}
refuse := func(err error) delivery.CandidateAnswer {
a.Refused = err.Error()
a.Verdict = delivery.CandidateRefused
return a
}
trunk, err := r.Placements(ctx)
if err != nil {
return refuse(fmt.Errorf("frombus: placement: %w", err))
}
in, err := Input(ctx, r, mesh, s)
if err != nil {
return refuse(err)
}
asConsumer, asProvider, err := compose.CredentialsNeeded(in, s.Node)
if err != nil {
return refuse(err)
}
// What the node runs, by module. A consumer's credential is asked only for a module it runs: the vault refuses the
// whole ask for a consuming module not assigned to the node (mesh-vault's own), and a module the candidate adds is
// assigned only after the merge (ADR 0549 §5), so its credential is left missing, a stand-in answered as not yet
// issued under Added (review of L2b, round 1, finding 1).
runs := map[string]bool{}
for _, c := range running {
if c.Module != "" {
runs[c.Module] = true
}
}
var consumed []compose.CredentialKey
for _, k := range asConsumer {
if runs[k.ConsumerModule] {
consumed = append(consumed, k)
}
}
asConsumer = consumed
if len(asConsumer)+len(asProvider) > 0 {
if in.Credentials, in.Issued, err = r.Credentials(ctx, s.Node, asConsumer, asProvider); err != nil {
return refuse(fmt.Errorf("frombus: the credentials of %s: %w", s.Node, err))
}
}
composed, skipped, stood, err := compose.CompositionsShadow(in, s.Node, fps)
if err != nil {
return refuse(err)
}
// The node's own composition is judged only where the node runs one: one that runs none applies it only once a want
// names it (the node agent's own composer hands it on that want alone), so a candidate does not add it.
runsOwn := false
for _, c := range running {
runsOwn = runsOwn || c.Module == ""
}
if !runsOwn {
for key, c := range composed {
if c.Module == "" {
delete(composed, key)
}
}
}
cmp := delivery.CompareComposed(composed, running)
a.Same, a.Differs, a.Removed = cmp.Same, cmp.Differs, cmp.Removed
// What the group adds, by node and module: placed by a candidate and not by the trunk on that node.
added := map[string]bool{}
for node, p := range s.Placements {
if p == nil {
continue
}
was := map[string]bool{}
for _, m := range trunk[node].Modules {
was[m.Module] = true
}
for _, m := range p.Modules {
if !was[m.Module] {
added[node+"/"+m.Module] = true
}
}
}
unissued := map[string][]string{}
refusals := map[string][]string{}
for _, x := range stood {
node, module := standsInFor(x, s.Node)
switch {
case node == s.Node && runs[module]:
// A value missing for a module the node runs is a refusal, whatever the trunk places (the seed's trunk is
// empty, and every module counts as the group's).
refusals[module] = append(refusals[module], x)
case module != "" && added[node+"/"+module]:
if node == s.Node {
unissued[module] = append(unissued[module], x)
} // another node's addition: the module here that carries it differs, which the assigner judges
case module == "" || node != s.Node:
// The node's own, or a value this node holds for a module elsewhere (a grant it provides): said as the node's.
refusals[delivery.HaveNodeWord] = append(refusals[delivery.HaveNodeWord], x)
default:
refusals[module] = append(refusals[module], x)
}
}
addedHere := map[string]delivery.ModuleAdded{}
for _, m := range cmp.Added {
addedHere[m.Module] = m
}
for _, m := range skipped { // placed, and no build given to compose it at
addedHere[m] = delivery.ModuleAdded{Module: m, Unissued: []string{"no build of " + m + " is known to compose it at"}}
}
for m, xs := range unissued {
got := addedHere[m]
got.Module = m
got.Unissued = append(got.Unissued, xs...)
addedHere[m] = got
}
for _, m := range sortedKeys(addedHere) {
a.Added = append(a.Added, addedHere[m])
}
for _, m := range sortedKeys(refusals) {
if _, isAdded := addedHere[m]; isAdded {
continue // a module the candidate adds here: what it lacks is said under Added
}
a.Refusals = append(a.Refusals, delivery.ModuleRefused{Module: m, Why: "stands in for " + strings.Join(refusals[m], ", ")})
}
sort.Slice(a.Refusals, func(i, j int) bool { return a.Refusals[i].Module < a.Refusals[j].Module })
a.Verdict = delivery.VerdictOf(a)
return a
}
var (
grantStandIn = regexp.MustCompile(`^the grant of \S+ for (\S+) on (\S+)$`)
needsStandIn = regexp.MustCompile(`^(\S+) on (\S+) needs `)
)
// standsInFor is the node and module a stand-in of compose's words is for: an own secret `module/name`, a port
// `port module:port`, a grant `the grant of <provision> for <module> on <node>`, a need `<module> on <node> needs ...`;
// the module empty when the words name none (the certificate, the authority).
func standsInFor(x, node string) (string, string) {
if m := grantStandIn.FindStringSubmatch(x); m != nil {
return m[2], m[1]
}
if m := needsStandIn.FindStringSubmatch(x); m != nil {
return m[2], m[1]
}
if rest, ok := strings.CutPrefix(x, "port "); ok {
if module, _, ok := strings.Cut(rest, ":"); ok {
return node, module
}
}
if module, _, ok := strings.Cut(x, "/"); ok && !strings.Contains(x, " ") {
return node, module
}
return node, ""
}
+111
View File
@@ -0,0 +1,111 @@
package frombus
import (
"fmt"
"sort"
"strconv"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// Export is one node's placement file as the composition input the controller fills says it (novox/hq ADR 0555 §6,
// issue 1126): the inverse of Input for what a person decides, so the controller's one read-only verb
// `placement-export` answers, from its own store by its own code, exactly what the seed writes into the catalogue.
// What the machine observes (its record) is not in it.
func Export(in compose.Input, node string) (delivery.Placement, error) {
var n *compose.Node
for i := range in.Nodes {
if in.Nodes[i].Name == node {
n = &in.Nodes[i]
}
}
if n == nil {
return delivery.Placement{}, fmt.Errorf("frombus: the input has no node %q", node)
}
// At the version the placement file is read at, with a list of modules even when none is placed: the export is the
// file the seed writes (cmd/placement-seed), and the reader refuses a value of version 0 or without its modules.
p := delivery.Placement{V: delivery.Version, Node: node, Modules: []delivery.PlacedModule{}, Decided: delivery.NodeDecided{
Endpoint: n.Place.Endpoint, Site: n.Place.Site, Hub: n.Place.Hub, Address: n.Place.Address,
PublicDomain: n.PublicDomain, Account: n.Account, AccountHome: n.AccountHome, AgentAccount: n.AgentAccount,
AgentHome: n.AgentAccountHome, Adopted: n.Adopted, Taken: n.Taken}}
// The tunnel found at the node's adoption: the one its private network took over, or, on the hub, the mesh's adopted
// one whether or not the hub took it; and the names a person gave the hub's carried peers.
if t, ok := in.Tunnels[node]; ok {
rt := recordTunnelOf(t.Tunnel)
p.Decided.Tunnel = &rt
} else if in.Adopted != nil && in.Adopted.Hub == node {
rt := recordTunnelOf(in.Adopted.Tunnel)
p.Decided.Tunnel = &rt
}
if n.Place.Hub {
for _, c := range in.CarriedPeers {
if c.Named != "" {
if p.Decided.PeerNames == nil {
p.Decided.PeerNames = map[string]string{}
}
p.Decided.PeerNames[c.PublicKey] = c.Named
}
}
}
for _, module := range n.Assigned {
m := delivery.PlacedModule{Module: module}
for port, at := range n.Ports[module] {
if m.Ports == nil {
m.Ports = map[string]int{}
}
m.Ports[strconv.Itoa(port)] = at
}
for provision, to := range n.Bound[module] {
if m.Bindings == nil {
m.Bindings = map[string]delivery.Provider{}
}
m.Bindings[provision] = delivery.Provider{Node: to.Node, Module: to.Module}
}
// A module in the settings form keeps its config values in its layers, not in mesh-config: they are exported with
// the placement. A module in the config form has mesh-config's values laid into its layer, which are not.
settingsForm := !in.Catalogue[module].DeclaresConfig()
for _, layer := range n.Settings[module] {
if layer.From == catalogue.MeshWideLayer {
m.HasMesh, m.MeshWide = true, layer.Placement
if settingsForm && len(layer.Values) > 0 {
m.MeshValues = layer.Values
}
} else {
m.HasLayer, m.Layer = true, layer.Placement
if settingsForm && len(layer.Values) > 0 {
m.Values = layer.Values
}
}
}
p.Modules = append(p.Modules, m)
}
for provision, to := range n.Pins {
if p.Pins == nil {
p.Pins = map[string]delivery.Provider{}
}
p.Pins[provision] = delivery.Provider{Node: to.Node, Module: to.Module}
}
for _, h := range in.Holdings {
if h.Node == node {
p.Holds = append(p.Holds, delivery.Holding{Claim: h.Claim, Scope: h.Scope, Module: h.Module, Site: h.Site, Kind: h.Kind})
}
}
sort.SliceStable(p.Holds, func(i, j int) bool {
if p.Holds[i].Claim != p.Holds[j].Claim {
return p.Holds[i].Claim < p.Holds[j].Claim
}
return p.Holds[i].Module < p.Holds[j].Module
})
return p, nil
}
func recordTunnelOf(t compose.Tunnel) delivery.RecordTunnel {
out := delivery.RecordTunnel{Interface: t.Interface, Unit: t.Unit, Config: t.Config, Port: t.Port, MTU: t.MTU,
Address: t.Address, Range: t.Range, PublicKey: t.PublicKey}
for _, peer := range t.Peers {
out.Peers = append(out.Peers, delivery.RecordTunnelPeer{PublicKey: peer.PublicKey, Address: peer.Address})
}
return out
}
+615
View File
@@ -0,0 +1,615 @@
// Package frombus fills mesh-sdk's composition input from what is on the bus, so a node agent, the composition check
// and the grants part compose with one composer over the same inputs (novox/hq ADR 0547 §2, ADR 0549 §3; issue 1126).
//
// **One composer, three callers.** compose.Compositions is the composer; Input is how each caller fills its
// compose.Input from the bus: the node agent for its own node (ADR 0547 §2), the assigner's composition check for every
// node (ADR 0539 §6's check, ADR 0547 §4), and the grants part (ADR 0547 §5). Each reads the bus with its own credential
// through a Reader, so this package holds no bus client, no store and no timer: what it is given is all it reads.
//
// **What it reads, and from whom.**
// - which modules run on each node, with their pins, holdings, host ports, bindings and settings placement: the
// indexer's state `placement`, one key per node (ADR 0549 §2);
// - what a person decided of each node (its endpoint, site, hub, address on the private network, public domain and
// accounts): the same placement key, Placement.Decided;
// - what each node observes of itself (its tunnel key, capabilities, keys, certificate, links and networks): the
// node's own have key, `<node>.node` (HaveNode.Record); the tunnel found at a node's adoption and the names of the hub's peers are decided,
// in its placement, and whether the hub took the tunnel over and who enrolled under a peer's key are read from both;
// - every module's manifest: the indexer's `modules.<module>` and its manifests by digest; for each module placed on
// the composing node, its built manifest instead: the builder's done key at the trunk's FP2, or at the fingerprint
// the node's want names where it runs another build (Scope.KeptAt), resolved by the controller's own take-in rule
// (takein.Manifest), a moved module's templates archive read in the artifact store (issue 1126);
// - each module's config values on a node: mesh-config's resolved state, the composing node's slice alone unless the
// caller composes every node (ADR 0547 §2, the prepare step's L4); of another node, only the keys its modules'
// served values fill (catalogue.ServedSettings), which every consumer is told (issue 1126).
//
// **Own secrets and pair credentials** come from the vault sealed to the composing node's key (OwnSecrets, Credentials),
// for that node alone: as a consumer and as a provider, exactly the keys compose.CredentialsNeeded names. **Not yet
// read** (each a later subtask of issue 1126): licences (none is placed today) and core boxes. Without one
// the composition stands in for it, and Compose refuses a composition that stands in for anything (compose.ErrStoodIn):
// a node never applies a stand-in. No node is joining before ADR 0547's third milestone (ADR 0550 §6), so Joining is
// always empty.
package frombus
import (
"context"
"encoding/json"
"fmt"
"net/netip"
"slices"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/declaration"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/overlay"
"git.novox.be/novox/mesh-sdk/go/takein"
)
// Reader is the bus as a caller reads it with its own credential. Each method reads one part's state; an error is a
// read that failed, never an absent key (an absent key is an empty answer).
type Reader interface {
// Placements is every node's placement, by node (delivery.PlacementBucket).
Placements(ctx context.Context) (map[string]delivery.Placement, error)
// Records is every node's record, by node: HaveNode.Record of each node's own have key, `<node>.node` of
// delivery.HaveBucket (ADR 0555 §3).
Records(ctx context.Context) (map[string]delivery.NodeRecord, error)
// Modules is every module the indexer holds, by module (delivery.ModulesBucket).
Modules(ctx context.Context) (map[string]delivery.Module, error)
// Manifest is the raw manifest of a digest (delivery.ManifestsBucket).
Manifest(ctx context.Context, digest string) ([]byte, error)
// Values is one module's config values on one node as mesh-config resolved them (mesh-config's state `resolved`,
// keys `<node>.<module>.<key>`), decoded; nil when it has none.
Values(ctx context.Context, node, module string) (map[string]any, error)
// OwnSecrets is the own secrets of the composing node's modules, each sealed to that node's key, as the vault answers
// the node's engine (the vault's verb `own`, ADR 0550 §5); asked only for the node composing itself.
OwnSecrets(ctx context.Context, node string) ([]compose.OwnSecretRow, error)
// Credentials is, for the composing node, each pair credential it reads as a consumer, held and sealed to its key,
// and each it grants as a provider, with the copy sealed to it (the vault's `own` with the keys, ADR 0550 §5). A key
// the vault holds no value for is left out; the composition then stands in for it, and Compose refuses it.
Credentials(ctx context.Context, node string, asConsumer, asProvider []compose.CredentialKey) ([]compose.Credential, []compose.Issued, error)
// Build is a module's build at a fingerprint, the builder's key (delivery.BuilderBuildsBucket, BuilderBuildKey); nil
// when the builder holds none. The composing node composes each of its modules from its build (issue 1126).
Build(ctx context.Context, module, fp string) (*delivery.BuilderBuild, error)
// Archive is the archive at url, a kept reference routed to the artifact store as the composing node reaches it
// (catalogue.Routed), checked against digest: a moved module's templates archive, read back for its texts as the
// controller's take-in reads it (takein.Built).
Archive(ctx context.Context, url, digest string) ([]byte, error)
// Served is, of one module's config values on another node than the composing one (mesh-config's state `resolved`,
// keys `<node>.<module>.<key>`), the keys named, decoded: the keys its served values and zone name fill
// (catalogue.ServedSettings), which every consumer is told and so every node may read (issue 1126). Nil when none
// is set.
Served(ctx context.Context, node, module string, keys []string) (map[string]any, error)
}
// Mesh is what is true of the whole mesh and is no part's state on the bus yet: the private network's range and the
// internal names' suffix as genesis was told them, and the mesh's authority certificate (public). Empty fields take
// compose's defaults.
type Mesh struct {
OverlayCIDR string
InternalSuffix string
Authority string
// Predefined is the list of predefined keys mesh-config published, which a manifest's values falling back to a
// predefined key are judged against; nil reads each manifest as the indexer does (catalogue.ParseManifestForInputs).
Predefined catalogue.PredefinedList
}
// Scope says who composes. Node is the node composing; only its config values are read unless Every is set, as the
// composition check sets it to compose every node (of another node's modules, only the values their served values fill
// are read, Reader.Served). KeptAt is, by module, the fingerprint the composing node's want names where it differs from
// the indexer's trunk: that node composes the module from its build at that fingerprint, as it runs it.
//
// **Composed from its build** (issue 1126): each module placed on the composing node, and each in KeptAt, composes from
// its built manifest (takein.Manifest over the builder's done key at the trunk's FP2 or the kept fingerprint), as the
// controller composes from the manifest its take-in resolved: so a moved module is handed to the provisioner, a resource
// naming an artifact names what was built, and a templated file has its text. A placed module the builder holds no done
// build of at the trunk's FP2 composes from the indexer's manifest, as before. Every leaves the catalogue as the indexer
// read it: the check of every node judges placement, and reads no archive.
//
// Placements, when set, stands in for the indexer's placement of the nodes it names: a node's candidate placement
// (delivery.PlacementCandidate), so a node composes in shadow with a catalogue pull request's placement and compares
// the result with what it runs (issue 1126); a nil value removes the node's placement. The rest is read as always.
//
// InOwn is the modules the composing node runs inside its own composition (InOwn of what it runs), composed there again
// (compose.Node.InOwn) rather than cut out as modules no build is given for (issue 1126).
type Scope struct {
Node string
Every bool
KeptAt map[string]string
Placements map[string]*delivery.Placement
InOwn []string
}
// InOwn is, of the modules p places on its node, those whose resources the node's own composition it runs
// (`<node>.node` in running) carries, by the module their ids name (declaration.ModuleGuess), sorted: the modules the
// controller's composed writer folded there because no fingerprint of the new form names them (mesh-wireguard, a module
// run at a build of the old form; issue 1126). A node composing itself passes it as Scope.InOwn, so its own
// composition keeps them as it runs them until a want names a build of the new form for one.
func InOwn(running []delivery.Composed, p delivery.Placement) []string {
placed := map[string]bool{}
for _, m := range p.Modules {
placed[m.Module] = true
}
in := map[string]bool{}
for _, c := range running {
if c.Module != "" {
continue
}
for _, raw := range c.Resources {
var head struct {
ID string `json:"id"`
Module string `json:"module"`
}
if json.Unmarshal(raw, &head) != nil || head.Module != "" {
continue
}
if m, ok := declaration.ModuleGuess(head.ID); ok && placed[m] {
in[m] = true
}
}
}
return sortedKeys(in)
}
// Input is the composition input for s, read through r.
func Input(ctx context.Context, r Reader, mesh Mesh, s Scope) (compose.Input, error) {
placements, err := r.Placements(ctx)
if err != nil {
return compose.Input{}, fmt.Errorf("frombus: placement: %w", err)
}
if len(s.Placements) > 0 {
given := make(map[string]delivery.Placement, len(placements)+len(s.Placements))
for node, p := range placements {
given[node] = p
}
for node, p := range s.Placements {
if p == nil {
delete(given, node)
continue
}
if p.Node != node {
return compose.Input{}, fmt.Errorf("frombus: the placement given for %s names the node %q", node, p.Node)
}
given[node] = *p
}
placements = given
}
records, err := r.Records(ctx)
if err != nil {
return compose.Input{}, fmt.Errorf("frombus: node records: %w", err)
}
modules, err := r.Modules(ctx)
if err != nil {
return compose.Input{}, fmt.Errorf("frombus: the indexer's modules: %w", err)
}
if _, ok := placements[s.Node]; !ok && !s.Every {
return compose.Input{}, fmt.Errorf("frombus: %s has no placement in the catalogue", s.Node)
}
in := compose.Input{OverlayCIDR: mesh.OverlayCIDR, InternalSuffix: mesh.InternalSuffix, Authority: mesh.Authority,
Catalogue: map[string]catalogue.Manifest{}}
if in.Authority == "" {
// The mesh's own certificate, public, as the composing node holds it; for the check of every node, as the first
// node holding one does.
if rec, ok := records[s.Node]; ok && rec.Authority != "" {
in.Authority = rec.Authority
} else {
for _, node := range sortedKeys(records) {
if a := records[node].Authority; a != "" {
in.Authority = a
break
}
}
}
}
parse := func(raw []byte) (catalogue.Manifest, error) {
if mesh.Predefined != nil {
return catalogue.ParseManifestWith(raw, mesh.Predefined)
}
return catalogue.ParseManifestForInputs(raw)
}
read := func(module, digest string) (catalogue.Manifest, error) {
raw, err := r.Manifest(ctx, digest)
if err != nil {
return catalogue.Manifest{}, fmt.Errorf("frombus: the manifest of %s (digest %s): %w", module, digest, err)
}
return parse(raw)
}
for _, name := range sortedKeys(modules) {
m := modules[name]
if m.Manifest == "" {
continue // a registered module the indexer has not read: nothing to compose it from
}
man, err := read(name, m.Manifest)
if err != nil {
return compose.Input{}, err
}
in.Catalogue[name] = man
}
// The module the mesh provides itself (the private network) has no repository the indexer reads: its manifest comes
// with mesh-sdk and always wins, as the controller's providedModules and the grants part have it.
provided, err := json.Marshal(overlay.Manifest())
if err != nil {
return compose.Input{}, err
}
var private catalogue.Manifest
if err := json.Unmarshal(provided, &private); err != nil {
return compose.Input{}, fmt.Errorf("frombus: the private network's manifest mesh-sdk provides: %w", err)
}
in.Catalogue[overlay.Name] = private
var missing []string
for _, node := range nodesOf(placements, records) {
rec, hasRecord := records[node]
p := placements[node]
if !hasRecord {
if len(p.Modules) > 0 {
missing = append(missing, node)
}
continue
}
n := nodeOf(rec, p)
withValues := s.Every || node == s.Node
for _, m := range p.Modules {
var values map[string]any
if withValues {
if values, err = r.Values(ctx, node, m.Module); err != nil {
return compose.Input{}, fmt.Errorf("frombus: %s's config values on %s: %w", m.Module, node, err)
}
} else if keys := catalogue.ServedSettings(in.Catalogue[m.Module]); len(keys) > 0 {
// What a provider elsewhere serves from its settings is told to every consumer: the composing node reads
// those keys of it, and no other (issue 1126: mailu's smtp domain, a consumer's on shanks).
if values, err = r.Served(ctx, node, m.Module, keys); err != nil {
return compose.Input{}, fmt.Errorf("frombus: what %s serves on %s (%s): %w", m.Module, node,
strings.Join(keys, ", "), err)
}
}
if layers := m.Layers(node, values); layers != nil {
if n.Settings == nil {
n.Settings = map[string][]catalogue.Layer{}
}
n.Settings[m.Module] = layers
}
}
if node == s.Node {
for _, module := range s.InOwn {
if n.InOwn == nil {
n.InOwn = map[string]bool{}
}
n.InOwn[module] = true
}
}
in.Nodes = append(in.Nodes, n)
in.Holdings = append(in.Holdings, p.Held()...)
// A tunnel found at adoption is the node's private network's when the node's tunnel key is the tunnel's (the
// controller's inv.Tunnels); the hub's found tunnel is the mesh's adopted one whether or not the hub took it over
// (inv.AdoptedTunnel), and its peers are carried once it did (inv.CarriedPeers).
if t := p.Decided.Tunnel; t != nil {
if rec.TunnelKey == t.PublicKey {
if in.Tunnels == nil {
in.Tunnels = map[string]compose.FoundTunnel{}
}
in.Tunnels[node] = compose.FoundTunnel{Tunnel: tunnelOf(*t), NodeAdopted: p.Decided.Adopted}
}
if p.Decided.Hub {
in.Adopted = &compose.AdoptedTunnel{Tunnel: tunnelOf(*t), Hub: node, Adopted: rec.TunnelKey == t.PublicKey}
}
}
}
if in.Adopted != nil && in.Adopted.Adopted {
in.CarriedPeers = carriedPeers(in.Adopted.Tunnel, placements[in.Adopted.Hub].Decided.PeerNames, records)
}
if len(missing) > 0 {
return compose.Input{}, fmt.Errorf("frombus: modules are placed on %s, whose agent has written no record yet",
strings.Join(missing, ", "))
}
if !s.Every {
if _, ok := records[s.Node]; !ok {
return compose.Input{}, fmt.Errorf("frombus: %s has written no record of itself yet", s.Node)
}
if err := fromBuilds(ctx, r, &in, s, placements[s.Node], modules, read, parse); err != nil {
return compose.Input{}, err
}
// Its own modules' secrets, sealed to its own key by the vault: a composition carries them sealed (ADR 0550 §5).
// Asked only when a module placed on it declares one: a node none of whose modules has a secret composes
// whether or not the vault answers.
if declaresOwnSecrets(in, s.Node) {
if in.OwnSecrets, err = r.OwnSecrets(ctx, s.Node); err != nil {
return compose.Input{}, fmt.Errorf("frombus: the own secrets of %s's modules: %w", s.Node, err)
}
}
// The core's credentials travel as the vault's box sealed to the machine (ADR 0349 §16): the same values, which
// the composition reads for a core module as its kept boxes.
for _, o := range in.OwnSecrets {
if json.Valid([]byte(o.Sealed)) {
in.KeptCoreBoxes = append(in.KeptCoreBoxes, compose.CoreBox{Node: o.Node, Module: o.Module, Name: o.Name,
SealedTo: o.SealedTo, Box: json.RawMessage(o.Sealed)})
}
}
}
return in, nil
}
// Compose is the node's compositions over the bus (ADR 0547 §2): Input for s, then compose.CompositionsApplied at the
// fingerprints fps gives each module: composed in the Reading mode as the controller composes them, and refused when it
// stands in for anything the input does not carry (compose.ErrStoodIn), since the node applies what it composed.
func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]string) (map[string]delivery.Composed, []string, error) {
in, err := Input(ctx, r, mesh, s)
if err != nil {
return nil, nil, err
}
// The pair credentials its composition reads, asked of the vault sealed to it (ADR 0550 §5).
asConsumer, asProvider, err := compose.CredentialsNeeded(in, s.Node)
if err != nil {
return nil, nil, err
}
if len(asConsumer)+len(asProvider) > 0 {
if in.Credentials, in.Issued, err = r.Credentials(ctx, s.Node, asConsumer, asProvider); err != nil {
return nil, nil, fmt.Errorf("frombus: the credentials of %s: %w", s.Node, err)
}
}
return compose.CompositionsApplied(in, s.Node, fps)
}
// fromBuilds puts in the built manifest of each module placed on the composing node, at the trunk's FP2, and of each
// module it keeps at another build (Scope.KeptAt, into its node's Kept), read from the builder's done key and resolved
// by the one take-in rule (takein.Manifest), a templates archive read in the artifact store as the node reaches it
// (issue 1126).
func fromBuilds(ctx context.Context, r Reader, in *compose.Input, s Scope, p delivery.Placement, modules map[string]delivery.Module,
read func(module, digest string) (catalogue.Manifest, error), parse func([]byte) (catalogue.Manifest, error)) error {
address, addressErr := "", error(nil)
addressed := false
archive := func(ctx context.Context, reference, digest string) ([]byte, error) {
if !addressed {
address, addressErr = compose.ArtifactStoreAddress(*in, s.Node)
addressed = true
}
switch {
case addressErr != nil:
return nil, addressErr
case address == "":
return nil, fmt.Errorf("%s reaches no artifact store", s.Node)
}
return r.Archive(ctx, catalogue.Routed(reference, address), digest)
}
built := func(module, fp string, b *delivery.BuilderBuild) (catalogue.Manifest, error) {
if b.Manifest == "" {
return catalogue.Manifest{}, fmt.Errorf("frombus: the build of %s at %s names no manifest", module, fp)
}
src, err := read(module, b.Manifest)
if err != nil {
return catalogue.Manifest{}, err
}
resolved, err := takein.Manifest(ctx, src, *b, archive)
if err != nil {
return catalogue.Manifest{}, fmt.Errorf("frombus: %s at %s: %w", module, fp, err)
}
// Read back as the mesh holds a built manifest (the controller keeps the resolved manifest's bytes and reads them
// as any manifest): a template's text is then in the manifest's own words, its ${config:…} in the settings form.
raw, err := json.Marshal(resolved)
if err != nil {
return catalogue.Manifest{}, err
}
if resolved, err = parse(raw); err != nil {
return catalogue.Manifest{}, fmt.Errorf("frombus: the built manifest of %s at %s: %w", module, fp, err)
}
return resolved, nil
}
for _, pm := range p.Modules {
m, ok := modules[pm.Module]
if !ok || m.FP2 == "" || s.KeptAt[pm.Module] != "" || slices.Contains(s.InOwn, pm.Module) {
// A module it runs inside its own composition is composed there as before: no build of the new form is what
// it runs (Scope.InOwn).
continue
}
if man, ok := in.Catalogue[pm.Module]; !ok || (man.Build == nil && !man.Templated()) {
continue // nothing is built of it: its manifest is what it composes from
}
b, err := r.Build(ctx, pm.Module, m.FP2)
if err != nil {
return fmt.Errorf("frombus: the build of %s at %s: %w", pm.Module, m.FP2, err)
}
if b == nil || b.State != delivery.BuildDone {
continue // no done build at the trunk's: composed from the indexer's manifest, as before
}
resolved, err := built(pm.Module, m.FP2, b)
if err != nil {
return err
}
in.Catalogue[pm.Module] = resolved
}
for i := range in.Nodes {
if in.Nodes[i].Name != s.Node {
continue
}
for _, module := range sortedKeys(s.KeptAt) {
fp := s.KeptAt[module]
b, err := r.Build(ctx, module, fp)
if err != nil {
return fmt.Errorf("frombus: the build of %s at %s: %w", module, fp, err)
}
if b == nil || b.State != delivery.BuildDone {
return fmt.Errorf("frombus: %s is wanted at %s, of which the builder holds no done build", module, fp)
}
resolved, err := built(module, fp, b)
if err != nil {
return err
}
if in.Nodes[i].Kept == nil {
in.Nodes[i].Kept = map[string]catalogue.Manifest{}
}
in.Nodes[i].Kept[module] = resolved
}
}
return nil
}
// declaresOwnSecrets says a module assigned to node declares an own secret, in the manifest the node composes it from.
func declaresOwnSecrets(in compose.Input, node string) bool {
for _, n := range in.Nodes {
if n.Name != node {
continue
}
for _, module := range n.Assigned {
m, ok := n.Kept[module]
if !ok {
m = in.Catalogue[module]
}
if len(m.OwnSecrets) > 0 {
return true
}
}
}
return false
}
// nodeOf is the composition's node from its record (what the machine observes) and its placement (what a person
// decided of it, and where its modules run).
func nodeOf(rec delivery.NodeRecord, p delivery.Placement) compose.Node {
n := compose.Node{
Name: rec.Node,
Place: compose.Place{Key: rec.TunnelKey},
SealingKey: rec.SealingKey,
OutwardLinks: rec.OutwardLinks, Networks: rec.Networks, NetworksSaid: rec.NetworksSaid,
ServingKey: rec.ServingKey, Certificate: rec.Certificate, Contract: rec.Contract, Stated: rec.Contract > 0,
}
placeNode(&n, p)
if len(rec.Capabilities) > 0 {
n.Capabilities = make(map[string]bool, len(rec.Capabilities))
for _, c := range rec.Capabilities {
n.Capabilities[c] = true
}
}
return n
}
// placeNode sets what a person decided of n and where its modules run, from its placement; what the machine observes
// (its keys, links, networks, certificate) and its settings layers are left as they are.
func placeNode(n *compose.Node, p delivery.Placement) {
d := p.Decided
n.Place = compose.Place{Key: n.Place.Key, Endpoint: d.Endpoint, Site: d.Site, Hub: d.Hub, Address: d.Address}
n.Assigned, n.PublicDomain = p.Assigned(), d.PublicDomain
n.Account, n.AccountHome, n.AgentAccount, n.AgentAccountHome = d.Account, d.AccountHome, d.AgentAccount, d.AgentHome
n.Adopted, n.Taken = d.Adopted, d.Taken
n.Pins, n.Bound, n.Ports = p.ChosenPins(), p.Bound(), p.HostPorts()
}
// Placed is in with the placements given standing in for those of the nodes they name (issue 1126): each such node's
// decided parts, modules, pins, bindings, host ports, settings placement and holdings are the given placement's, its
// config values those in already carries for it (the layer of the node's own name), and what the machine observes is
// kept. A nil placement leaves the node in the mesh with nothing placed on it. A node in does not hold is refused: it has
// no record to compose it from. The found tunnel and the hub's peers are read from in as they are, not from the given
// placement. It is how a check composes a change's own placement over an input already filled (composer.Case's
// Placements).
func Placed(in compose.Input, placements map[string]*delivery.Placement) (compose.Input, error) {
if len(placements) == 0 {
return in, nil
}
out := in
out.Nodes = append([]compose.Node(nil), in.Nodes...)
index := map[string]int{}
for i, n := range out.Nodes {
index[n.Name] = i
}
for _, node := range sortedKeys(placements) {
p := placements[node]
i, ok := index[node]
if !ok {
return compose.Input{}, fmt.Errorf("frombus: a placement is given for %s, which the input holds no node for", node)
}
if p == nil {
p = &delivery.Placement{Node: node}
}
if p.Node != node {
return compose.Input{}, fmt.Errorf("frombus: the placement given for %s names the node %q", node, p.Node)
}
n := out.Nodes[i]
placeNode(&n, *p)
settings := map[string][]catalogue.Layer{}
for _, m := range p.Modules {
var values map[string]any
for _, l := range n.Settings[m.Module] {
if l.From == node {
values = l.Values
}
}
if layers := m.Layers(node, values); layers != nil {
settings[m.Module] = layers
}
}
n.Settings = nil
if len(settings) > 0 {
n.Settings = settings
}
out.Nodes[i] = n
}
out.Holdings = nil
for _, h := range in.Holdings {
if _, given := placements[h.Node]; !given {
out.Holdings = append(out.Holdings, h)
}
}
for _, node := range sortedKeys(placements) {
if p := placements[node]; p != nil {
out.Holdings = append(out.Holdings, p.Held()...)
}
}
return out, nil
}
func tunnelOf(t delivery.RecordTunnel) compose.Tunnel {
out := compose.Tunnel{Interface: t.Interface, Unit: t.Unit, Config: t.Config, Port: t.Port, MTU: t.MTU,
Address: t.Address, Range: t.Range, PublicKey: t.PublicKey}
for _, p := range t.Peers {
out.Peers = append(out.Peers, compose.TunnelPeer{PublicKey: p.PublicKey, Address: p.Address})
}
return out
}
// carriedPeers is every peer of the hub's adopted tunnel, by address, each with the node that enrolled under its key
// where one has (a node whose tunnel key is the peer's) and the name a person gave it (inv.CarriedPeers).
func carriedPeers(t compose.Tunnel, names map[string]string, records map[string]delivery.NodeRecord) []compose.CarriedPeer {
byKey := map[string]string{}
for node, rec := range records {
if rec.TunnelKey != "" {
byKey[rec.TunnelKey] = node
}
}
var out []compose.CarriedPeer
for _, p := range t.Peers {
address, _, _ := strings.Cut(p.Address, "/")
out = append(out, compose.CarriedPeer{PublicKey: p.PublicKey, Address: address, EnrolledAs: byKey[p.PublicKey], Named: names[p.PublicKey]})
}
sort.SliceStable(out, func(i, j int) bool {
a, errA := netip.ParseAddr(out[i].Address)
b, errB := netip.ParseAddr(out[j].Address)
if errA != nil || errB != nil {
return out[i].Address < out[j].Address
}
return a.Less(b)
})
return out
}
// nodesOf is every node named by a placement or a record, by name: the order the controller reads them in.
func nodesOf(placements map[string]delivery.Placement, records map[string]delivery.NodeRecord) []string {
seen := map[string]bool{}
for n := range placements {
seen[n] = true
}
for n := range records {
seen[n] = true
}
return sortedKeys(seen)
}
func sortedKeys[V any](m map[string]V) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
+687
View File
@@ -0,0 +1,687 @@
package frombus
import (
"archive/tar"
"bytes"
"compress/gzip"
"context"
"crypto/ecdh"
"crypto/rand"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"os"
"reflect"
"slices"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/overlay"
)
// bus is a Reader over values held in memory, counting whose config values were read.
type bus struct {
placements map[string]delivery.Placement
records map[string]delivery.NodeRecord
modules map[string]delivery.Module
manifests map[string][]byte
values map[string]map[string]any // by node.module
valuesRead []string
own []compose.OwnSecretRow
ownErr error
// asked is every consumer credential asked; the fake refuses the whole ask, as the vault's own does, for a consuming
// module its placement does not put on the node.
asked []compose.CredentialKey
// builds are the builder's keys, by "<module> <fp>"; archives the artifact store's blobs, by digest, and fetched
// every URL asked; served another node's values by "<node>.<module>", and servedRead every ask with its keys.
builds map[string]*delivery.BuilderBuild
archives map[string][]byte
fetched []string
served map[string]map[string]any
servedRead []string
buildsRead []string
}
func (b *bus) Build(_ context.Context, module, fp string) (*delivery.BuilderBuild, error) {
b.buildsRead = append(b.buildsRead, module)
return b.builds[module+" "+fp], nil
}
func (b *bus) Archive(_ context.Context, url, digest string) ([]byte, error) {
b.fetched = append(b.fetched, url)
body, ok := b.archives[digest]
if !ok {
return nil, errors.New("the store holds no such blob")
}
return body, nil
}
func (b *bus) Served(_ context.Context, node, module string, keys []string) (map[string]any, error) {
b.servedRead = append(b.servedRead, node+"."+module+":"+strings.Join(keys, ","))
out := map[string]any{}
for _, k := range keys {
if v, ok := b.served[node+"."+module][k]; ok {
out[k] = v
}
}
return out, nil
}
func (b *bus) Placements(context.Context) (map[string]delivery.Placement, error) {
return b.placements, nil
}
func (b *bus) Records(context.Context) (map[string]delivery.NodeRecord, error) { return b.records, nil }
func (b *bus) Modules(context.Context) (map[string]delivery.Module, error) { return b.modules, nil }
func (b *bus) Manifest(_ context.Context, digest string) ([]byte, error) {
raw, ok := b.manifests[digest]
if !ok {
return nil, errors.New("no such manifest")
}
return raw, nil
}
func (b *bus) OwnSecrets(context.Context, string) ([]compose.OwnSecretRow, error) {
return b.own, b.ownErr
}
func (b *bus) Credentials(_ context.Context, node string, asConsumer, _ []compose.CredentialKey) ([]compose.Credential, []compose.Issued, error) {
b.asked = append(b.asked, asConsumer...)
for _, k := range asConsumer {
placed := false
for _, m := range b.placements[node].Modules {
placed = placed || m.Module == k.ConsumerModule
}
if !placed {
return nil, nil, fmt.Errorf("%s is not on %s: neither assigned there nor required by a module there", k.ConsumerModule, node)
}
}
return nil, nil, nil
}
func (b *bus) Values(_ context.Context, node, module string) (map[string]any, error) {
b.valuesRead = append(b.valuesRead, node+"."+module)
return b.values[node+"."+module], nil
}
func publicKey(t *testing.T) string {
t.Helper()
k, err := ecdh.X25519().GenerateKey(rand.Reader)
if err != nil {
t.Fatal(err)
}
return base64.StdEncoding.EncodeToString(k.PublicKey().Bytes())
}
// Issue 1126 (S3, S4, S5): the bus's placement, node records, manifests and config values fill the same compose.Input the
// controller's store fills for compose's own two-machine mesh (compose's aMesh: anchor the hub, laptop), and that Input
// composes; a node reads only its own config values; what cannot be composed is refused, naming it.
func TestInput(t *testing.T) {
ctx := context.Background()
xterm, err := os.ReadFile("../../catalogue/testdata/catalogue-835bf22b-xterm.module.json")
if err != nil {
t.Fatal(err)
}
digest := delivery.ManifestDigest(xterm)
// A module in the settings form (plex and the media modules on one node, 2026-10-26): its config values are in its
// layers, not mesh-config, so the placement carries them (MeshValues, Values) and mesh-config's are not read for it.
media := []byte(`{"module":"media","version":"1","settings":{"lines":{"kind":"preference","default":0,"why":"kept"},` +
`"name":{"kind":"preference","default":"x","why":"shown"}},"own-secrets":{"token":"/srv/media/token"},"resources":[{"id":"conf","type":"file",` +
`"path":"/srv/media/conf","mode":"0644","content":"uid=${setting:lines} name=${setting:name}\n"}]}`)
mediaDigest := delivery.ManifestDigest(media)
keys := map[string]string{"anchor": publicKey(t), "laptop": publicKey(t)}
caps := []string{"container-runtime", "systemd", "wireguard"}
tunnel := &delivery.RecordTunnel{Interface: "wg0", PublicKey: keys["anchor"], Address: "10.77.0.1/24",
Peers: []delivery.RecordTunnelPeer{{PublicKey: "peer", Address: "10.77.0.9/32"}, {PublicKey: keys["laptop"], Address: "10.77.0.2/32"}}}
mk := func() *bus {
return &bus{
placements: map[string]delivery.Placement{
"anchor": {Node: "anchor", Decided: delivery.NodeDecided{Endpoint: "anchor.example:51820", Site: "here", Hub: true, Address: "10.77.0.1",
Tunnel: tunnel, PeerNames: map[string]string{"peer": "phone"}},
Modules: []delivery.PlacedModule{{Module: overlay.Name}},
Holds: []delivery.Holding{{Claim: "mesh-private-network", Scope: "mesh", Module: overlay.Name}}},
"laptop": {Node: "laptop", Decided: delivery.NodeDecided{Endpoint: "laptop.example:51820", Site: "here", Address: "10.77.0.2"},
Modules: []delivery.PlacedModule{{Module: overlay.Name},
{Module: "xterm", HasLayer: true, Ports: map[string]int{"80": 8080}},
{Module: "media", HasMesh: true, MeshValues: map[string]any{"name": "mesh"}, HasLayer: true,
Layer: map[string]any{"places": map[string]any{"data": "/srv/data"}}, Values: map[string]any{"lines": float64(1000)}}}},
},
records: map[string]delivery.NodeRecord{
"anchor": {Node: "anchor", TunnelKey: keys["anchor"], Capabilities: caps, Contract: 2},
"laptop": {Node: "laptop", TunnelKey: keys["laptop"], Capabilities: caps, Authority: "the mesh's certificate"},
},
modules: map[string]delivery.Module{"xterm": {Manifest: digest}, "media": {Manifest: mediaDigest}, "unread": {}},
manifests: map[string][]byte{digest: xterm, mediaDigest: media},
values: map[string]map[string]any{"laptop.xterm": {"font": "mono"}, "anchor.xterm": {"font": "serif"},
"laptop.media": {"lines": "not mesh-config's"}},
own: []compose.OwnSecretRow{{Node: "laptop", Module: "media", Name: "token", Origin: "made", SealedTo: "sk", Sealed: "{}"}},
}
}
// What the controller's store gives for the same mesh, as compose's aMesh and SettingsFor build it.
var private catalogue.Manifest
raw, _ := json.Marshal(overlay.Manifest())
if err := json.Unmarshal(raw, &private); err != nil {
t.Fatal(err)
}
parsed, err := catalogue.ParseManifestForInputs(xterm)
if err != nil {
t.Fatal(err)
}
parsedMedia, err := catalogue.ParseManifestForInputs(media)
if err != nil {
t.Fatal(err)
}
capsSet := map[string]bool{"container-runtime": true, "systemd": true, "wireguard": true}
tun := compose.Tunnel{Interface: "wg0", PublicKey: keys["anchor"], Address: "10.77.0.1/24",
Peers: []compose.TunnelPeer{{PublicKey: "peer", Address: "10.77.0.9/32"}, {PublicKey: keys["laptop"], Address: "10.77.0.2/32"}}}
want := compose.Input{
Catalogue: map[string]catalogue.Manifest{overlay.Name: private, "xterm": parsed, "media": parsedMedia},
Nodes: []compose.Node{
{Name: "anchor", Place: compose.Place{Key: keys["anchor"], Endpoint: "anchor.example:51820", Site: "here", Hub: true,
Address: "10.77.0.1"}, Assigned: []string{overlay.Name}, Capabilities: capsSet, Contract: 2, Stated: true},
{Name: "laptop", Place: compose.Place{Key: keys["laptop"], Endpoint: "laptop.example:51820", Site: "here",
Address: "10.77.0.2"}, Assigned: []string{overlay.Name, "xterm", "media"}, Capabilities: capsSet,
Ports: map[string]map[int]int{"xterm": {80: 8080}},
Settings: map[string][]catalogue.Layer{"xterm": {{From: "laptop", Values: map[string]any{"font": "mono"}}},
"media": {{From: catalogue.MeshWideLayer, Values: map[string]any{"name": "mesh"}},
{From: "laptop", Values: map[string]any{"lines": float64(1000)}, Placement: map[string]any{"places": map[string]any{"data": "/srv/data"}}}}}},
},
Holdings: []catalogue.Held{{Claim: "mesh-private-network", Scope: "mesh", Node: "anchor", Module: overlay.Name}},
OwnSecrets: []compose.OwnSecretRow{{Node: "laptop", Module: "media", Name: "token", Origin: "made", SealedTo: "sk", Sealed: "{}"}},
// The same value as a core credential's box, for a core module (ADR 0349 §16).
KeptCoreBoxes: []compose.CoreBox{{Node: "laptop", Module: "media", Name: "token", SealedTo: "sk", Box: json.RawMessage("{}")}},
Tunnels: map[string]compose.FoundTunnel{"anchor": {Tunnel: tun}},
Adopted: &compose.AdoptedTunnel{Tunnel: tun, Hub: "anchor", Adopted: true},
// The controller's order is by address; laptop enrolled under its peer's key, and a person named the other.
Authority: "the mesh's certificate",
CarriedPeers: []compose.CarriedPeer{{PublicKey: keys["laptop"], Address: "10.77.0.2", EnrolledAs: "laptop"},
{PublicKey: "peer", Address: "10.77.0.9", Named: "phone"}},
}
b := mk()
got, err := Input(ctx, b, Mesh{}, Scope{Node: "laptop"})
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(got, want) {
gj, _ := json.MarshalIndent(got, "", " ")
wj, _ := json.MarshalIndent(want, "", " ")
t.Fatalf("Input from the bus differs from the store's:\n got %s\nwant %s", gj, wj)
}
if !reflect.DeepEqual(b.valuesRead, []string{"laptop." + overlay.Name, "laptop.xterm", "laptop.media"}) {
t.Errorf("laptop read config values %v; a node reads its own slice alone", b.valuesRead)
}
// The same composer over both: the compositions from the bus are the store's, byte for byte.
fps := map[string]string{overlay.Name: "src2:" + strings.Repeat("a", 64), "xterm": "src2:" + strings.Repeat("b", 64),
"media": "src2:" + strings.Repeat("c", 64)}
fromBus, _, err := Compose(ctx, mk(), Mesh{}, Scope{Node: "laptop"}, fps)
if err != nil {
t.Fatal(err)
}
fromStore, _, err := compose.Compositions(want, "laptop", fps, compose.Reading)
if err != nil {
t.Fatal(err)
}
if len(fromBus) == 0 || !reflect.DeepEqual(fromBus, fromStore) {
t.Fatalf("compositions differ or are empty: %d from the bus, %d from the store", len(fromBus), len(fromStore))
}
// Issue 1126: a candidate placement equal to the node's composes what it runs, and CompareComposed says so; one
// thing changed in the group's candidate of another node (the hub's endpoint) names exactly the module it moves
// here. Placed puts the same placement over the store's input unchanged.
running := make([]delivery.Composed, 0, len(fromBus))
for _, c := range fromBus {
running = append(running, c)
}
laptop := mk().placements["laptop"]
asIs, _, err := Compose(ctx, mk(), Mesh{}, Scope{Node: "laptop", Placements: map[string]*delivery.Placement{"laptop": &laptop}}, fps)
if err != nil {
t.Fatal(err)
}
if c := delivery.CompareComposed(asIs, running); len(c.Differs)+len(c.Added)+len(c.Removed) != 0 || len(c.Same) != len(running) {
t.Errorf("the candidate equal to the placement: %+v", c)
}
moved := mk().placements["anchor"]
moved.Decided.Endpoint = "anchor.example:51821"
changed, _, err := Compose(ctx, mk(), Mesh{}, Scope{Node: "laptop", Placements: map[string]*delivery.Placement{"laptop": &laptop, "anchor": &moved}}, fps)
if err != nil {
t.Fatal(err)
}
if c := delivery.CompareComposed(changed, running); len(c.Differs) != 1 || c.Differs[0].Module != overlay.Name || c.Differs[0].Resource != overlay.Name+".overlay-config" {
t.Errorf("the hub's endpoint changed: %+v, want %s alone, naming its config", c, overlay.Name)
}
if c := delivery.CompareComposed(changed, running[:0]); len(c.Added) != len(changed) {
t.Errorf("nothing runs: %+v", c)
}
// ADR 0559 §4, the node's answer: the same; one module differing; a module the group adds, whose own secret is not
// yet issued (no refusal); a value missing for a module the node runs (a refusal); a module the candidate removes.
keeper := []byte(`{"module":"keeper","version":"1","own-secrets":{"password":"/var/lib/mesh/keeper/password"},` +
`"resources":[{"id":"state","type":"directory","path":"/var/lib/mesh/keeper","mode":"0700"}]}`)
lemurs := []byte(`{"module":"lemurs","version":"1","capabilities":["systemd"],"resources":[{"id":"service","type":"service","unit":"lemurs.service","state":"running","boot":"enabled"}]}`)
// A provider and a consumer of a pair credential on the one node (compose's objects and files).
objects, _ := json.Marshal(catalogue.Manifest{Module: "objects", Version: "1",
Provides: []catalogue.Offer{{Name: "s3-bucket", Scope: catalogue.ScopeMesh, Identity: &catalogue.OfferIdentity{Max: 20, In: "an S3 access key"}}},
Receives: map[string]string{"s3-bucket": "/var/lib/mesh/objects/mesh.json"}})
files, _ := json.Marshal(catalogue.Manifest{Module: "files", Version: "1", Requires: []string{"s3-bucket"},
Resources: []map[string]any{{"id": "config", "type": "file", "path": "/etc/files/config.json", "mode": "0600", "content": "{}", "merge": "json"}}})
one := func(placed ...string) *bus {
p := delivery.Placement{Node: "n1", Decided: delivery.NodeDecided{Hub: true, Address: "10.77.0.1"}}
for _, m := range placed {
p.Modules = append(p.Modules, delivery.PlacedModule{Module: m})
}
return &bus{placements: map[string]delivery.Placement{"n1": p},
records: map[string]delivery.NodeRecord{"n1": {Node: "n1", TunnelKey: keys["anchor"], SealingKey: "sk", Capabilities: []string{"systemd"}}},
modules: map[string]delivery.Module{"keeper": {Manifest: delivery.ManifestDigest(keeper)}, "lemurs": {Manifest: delivery.ManifestDigest(lemurs)},
"objects": {Manifest: delivery.ManifestDigest(objects)}, "files": {Manifest: delivery.ManifestDigest(files)}},
manifests: map[string][]byte{delivery.ManifestDigest(keeper): keeper, delivery.ManifestDigest(lemurs): lemurs,
delivery.ManifestDigest(objects): objects, delivery.ManifestDigest(files): files},
own: []compose.OwnSecretRow{{Node: "n1", Module: "keeper", Name: "password", Origin: "made", SealedTo: "sk", Sealed: `"box"`}}}
}
oneFps := map[string]string{"keeper": "src2:" + strings.Repeat("d", 64), "lemurs": "src2:" + strings.Repeat("e", 64),
"objects": "src2:" + strings.Repeat("f", 64), "files": "src2:" + strings.Repeat("0", 64)}
runs, _, err := Compose(ctx, one("keeper", "lemurs"), Mesh{}, Scope{Node: "n1"}, oneFps)
if err != nil {
t.Fatal(err)
}
var runsNow []delivery.Composed
for _, c := range runs {
runsNow = append(runsNow, c)
}
objectsRun, _, err := Compose(ctx, one("objects"), Mesh{}, Scope{Node: "n1"}, oneFps)
if err != nil {
t.Fatal(err)
}
var consumerRuns []delivery.Composed
for _, c := range objectsRun {
consumerRuns = append(consumerRuns, c)
}
withoutOwn := func(cs []delivery.Composed) []delivery.Composed {
var out []delivery.Composed
for _, c := range cs {
if c.Module != "" {
out = append(out, c)
}
}
return out
}
// What the controller runs on n1 for a module no want names (lemurs, at a build of the old form) and the private
// network's (built by nobody): its composed writer (mesh-controller compositionsOf) folds their resources into the
// node's own, each in its place, its module taken off; the rest split by module.
tunnelled := func() *bus {
b := one("keeper", "lemurs", overlay.Name)
p := b.placements["n1"]
p.Decided.Endpoint = "n1.example:51820"
b.placements["n1"] = p
return b
}
tunnelledAs := func() map[string]*delivery.Placement {
p := tunnelled().placements["n1"]
return map[string]*delivery.Placement{"n1": &p}
}
folded := func() []delivery.Composed {
b := tunnelled()
in, err := Input(ctx, b, Mesh{}, Scope{Node: "n1"})
if err != nil {
t.Fatal(err)
}
reading := in
reading.Nodes = append([]compose.Node(nil), in.Nodes...)
reading.Nodes[0].ReadsModules = true
d, _, err := compose.ComposeIn(reading, "n1", compose.Reading)
if err != nil {
t.Fatal(err)
}
body, err := d.Body()
if err != nil {
t.Fatal(err)
}
var envelope struct {
Resources []map[string]json.RawMessage `json:"resources"`
}
if err := json.Unmarshal(body, &envelope); err != nil {
t.Fatal(err)
}
var resources []json.RawMessage
for _, r := range envelope.Resources {
var module string
_ = json.Unmarshal(r["module"], &module)
if module == "lemurs" || module == overlay.Name {
delete(r, "module")
}
raw, _ := json.Marshal(r)
resources = append(resources, raw)
}
split, err := delivery.SplitComposed("n1", resources, map[string]string{"keeper": oneFps["keeper"]}, nil, delivery.NodeOwn{})
if err != nil {
t.Fatal(err)
}
var out []delivery.Composed
for _, c := range split {
out = append(out, c)
}
return out
}()
if got := InOwn(folded, tunnelled().placements["n1"]); !reflect.DeepEqual(got, []string{"lemurs", overlay.Name}) {
t.Errorf("InOwn of the controller's composition = %v, want lemurs and %s", got, overlay.Name)
}
placedAs := func(modules ...string) map[string]*delivery.Placement {
p := one(modules...).placements["n1"]
return map[string]*delivery.Placement{"n1": &p}
}
for name, c := range map[string]struct {
b *bus
given map[string]*delivery.Placement
running []delivery.Composed
verdict string
check func(delivery.CandidateAnswer) bool
}{
"the same": {one("keeper", "lemurs"), placedAs("keeper", "lemurs"), runsNow, delivery.CandidateSame,
func(a delivery.CandidateAnswer) bool { return len(a.Same) == 3 && a.Head == "h1" && a.Revision == 7 }},
"the node runs no own composition": {one("keeper", "lemurs"), placedAs("keeper", "lemurs"), withoutOwn(runsNow), delivery.CandidateSame,
func(a delivery.CandidateAnswer) bool { return len(a.Same) == 2 && len(a.Added) == 0 }},
"one removed": {one("keeper", "lemurs"), placedAs("keeper"), runsNow, delivery.CandidateDiffers,
func(a delivery.CandidateAnswer) bool { return reflect.DeepEqual(a.Removed, []string{"lemurs"}) }},
"one added, its secret not yet issued": {func() *bus { b := one("lemurs"); b.own = nil; return b }(), placedAs("keeper", "lemurs"),
nil, delivery.CandidateDiffers, func(a delivery.CandidateAnswer) bool {
for _, m := range a.Added {
if m.Module == "keeper" {
return reflect.DeepEqual(m.Unissued, []string{"keeper/password"}) && len(a.Refusals) == 0
}
}
return false
}},
"a value missing for a module that runs": {func() *bus { b := one("keeper", "lemurs"); b.own = nil; return b }(), placedAs("keeper", "lemurs"),
runsNow, delivery.CandidateRefused, func(a delivery.CandidateAnswer) bool {
return len(a.Refusals) == 1 && a.Refusals[0].Module == "keeper" && strings.Contains(a.Refusals[0].Why, "keeper/password")
}},
// Review of L2b, round 1, finding 1: a consumer the candidate adds has no credential yet, and the vault refuses an
// ask for a module not on the node; it is not asked, and the module is added, its credential not yet issued.
"a consumer added, the vault not asked for it": {one("objects"), placedAs("objects", "files"), consumerRuns, delivery.CandidateDiffers,
func(a delivery.CandidateAnswer) bool {
return len(a.Added) == 1 && a.Added[0].Module == "files" && len(a.Added[0].Unissued) > 0 && len(a.Refusals) == 0
}},
// And with no trunk at all (the seed), a stand-in of a module the node runs is a refusal, not an addition.
"the seed, a value missing for a module that runs": {func() *bus { b := one(); b.own = nil; return b }(), placedAs("keeper", "lemurs"),
runsNow, delivery.CandidateRefused, func(a delivery.CandidateAnswer) bool {
return len(a.Refusals) == 1 && a.Refusals[0].Module == "keeper" && len(a.Added) == 0
}},
"cannot compose": {one("keeper"), map[string]*delivery.Placement{"n1": {Node: "n2"}}, runsNow, delivery.CandidateRefused,
func(a delivery.CandidateAnswer) bool { return a.Refused != "" }},
// Issue 1126 (the seed's review): the private network's module and one at a build of the old form, which no want
// names and the controller folds into the node's own, are folded there again: the node's own is the same, and
// neither is added.
"mesh-wireguard and an old-form module, folded": {tunnelled(), tunnelledAs(),
folded, delivery.CandidateSame, func(a delivery.CandidateAnswer) bool { return len(a.Same) == 2 && len(a.Added) == 0 }},
} {
fps := oneFps
if name == "mesh-wireguard and an old-form module, folded" {
fps = map[string]string{"keeper": oneFps["keeper"]} // the node's wants: no build of lemurs or the private network
}
a := Answer(ctx, c.b, Mesh{}, Scope{Node: "n1", Placements: c.given, InOwn: InOwn(c.running, *c.given["n1"])}, fps, c.running, "h1", 7)
if a.Verdict != c.verdict || !c.check(a) {
t.Errorf("%s: %+v", name, a)
}
if raw, err := delivery.EncodeHaveNode(delivery.HaveNode{Candidates: map[string]delivery.CandidateAnswer{"g": a}}); err != nil {
t.Errorf("%s: the answer is not one a have holds: %v (%s)", name, err, raw)
}
}
if got, err := Placed(want, map[string]*delivery.Placement{"laptop": &laptop}); err != nil || !reflect.DeepEqual(got, want) {
t.Errorf("Placed with the node's own placement changes the input: %v", err)
}
if got, err := Placed(want, map[string]*delivery.Placement{"laptop": nil}); err != nil || len(got.Nodes[1].Assigned) != 0 || got.Nodes[1].Place.Key != keys["laptop"] {
t.Errorf("Placed with no placement: %+v, %v", got.Nodes[1], err)
}
if _, err := Placed(want, map[string]*delivery.Placement{"stranger": {Node: "stranger"}}); err == nil {
t.Error("Placed a node the input holds no node for")
}
if _, err := Input(ctx, mk(), Mesh{}, Scope{Node: "laptop", Placements: map[string]*delivery.Placement{"laptop": {Node: "anchor"}}}); err == nil {
t.Error("a given placement naming another node was taken")
}
// ADR 0555 §6: Export is Input's inverse for what a person decides: the store's input exported, node by node, is the
// placement the bus holds, and read back it is the store's input again.
for node, placed := range mk().placements {
got, err := Export(want, node)
if err != nil {
t.Fatal(err)
}
if gj, _ := delivery.EncodePlacement(got); !delivery.SamePlacement(got, placed) {
pj, _ := delivery.EncodePlacement(placed)
t.Errorf("%s exported as %s, want %s", node, gj, pj)
}
}
if _, err := Export(want, "stranger"); err == nil {
t.Error("a node the input lacks was exported")
}
b = mk()
if every, err := Input(ctx, b, Mesh{}, Scope{Every: true}); err != nil || every.OwnSecrets != nil {
t.Fatalf("composing every node: %v, own secrets %v (the vault answers a node about itself alone)", err, every.OwnSecrets)
}
if !reflect.DeepEqual(b.valuesRead, []string{"anchor." + overlay.Name, "laptop." + overlay.Name, "laptop.xterm", "laptop.media"}) {
// Every node's placed modules, and nothing placed nowhere: anchor places no xterm.
t.Errorf("the check read %v", b.valuesRead)
}
// A node none of whose modules declares an own secret composes without asking the vault (the review of L2: a node
// with no secret composed nothing while the vault did not answer); one whose module does, refuses without it.
quiet := mk()
quiet.ownErr = errors.New("no responders")
if _, err := Input(ctx, quiet, Mesh{}, Scope{Node: "anchor"}); err != nil {
t.Errorf("anchor, with no own secret, needed the vault: %v", err)
}
if _, err := Input(ctx, quiet, Mesh{}, Scope{Node: "laptop"}); err == nil {
t.Error("laptop's media, which declares an own secret, composed without the vault")
}
// A module the node's want keeps at another build than the trunk's composes from that build's manifest, resolved with
// what it made (issue 1126), on the composing node alone.
kept := mk()
oldXterm := []byte(`{"module":"xterm","version":"1","resources":[{"id":"tools","type":"process","name":"xterm-tools",` +
`"artifact":"tools","run":["./xterm-tools"]}],"build":{"artifacts":[{"name":"tools","kind":"bundle","language":"go","system":"arch","from":"cmd/xterm-tools","binary":"xterm-tools"}]}}`)
kept.manifests["old"] = oldXterm
oldFP := "src2:" + strings.Repeat("9", 64)
toolsRef := "artifact-store://xterm/tools/blobs/sha256:" + strings.Repeat("1", 64)
kept.builds = map[string]*delivery.BuilderBuild{"xterm " + oldFP: {State: delivery.BuildDone, Manifest: "old",
Artifacts: []delivery.BuiltArtifact{{Name: "tools", Kind: "bundle", Reference: toolsRef, Digest: "sha256:" + strings.Repeat("1", 64)}}}}
parsedOld, err := catalogue.ParseManifestForInputs(oldXterm)
if err != nil {
t.Fatal(err)
}
keptBuilt, err := parsedOld.Resolve([]catalogue.Built{{Name: "tools", Kind: "bundle", Reference: toolsRef, Digest: "sha256:" + strings.Repeat("1", 64)}})
if err != nil {
t.Fatal(err)
}
in, err := Input(ctx, kept, Mesh{}, Scope{Node: "laptop", KeptAt: map[string]string{"xterm": oldFP}})
if err != nil || !reflect.DeepEqual(in.Nodes[1].Kept, map[string]catalogue.Manifest{"xterm": keptBuilt}) || in.Nodes[0].Kept != nil {
t.Errorf("the want's build is kept on the composing node alone: %v, %v", in.Nodes, err)
}
// Issue 1126, the seed's refusals. restic and the provisioner's client as the catalogue holds them (mesh-catalog
// c05aa5e6), restic moved with an own secret the vault leaves out (it hands it to the client, mesh-vault ownnode.go
// handed): composed from the indexer's manifest restic is not handed and stands in for that secret, as novox's,
// ace's, g14's and shanks's answers refused; composed from its build, it is handed, its templates archive read
// where the node reaches the artifact store, and the composition is the one the controller's take-in gives.
resticRaw, err := os.ReadFile("testdata/catalogue-c05aa5e6-restic.module.json")
if err != nil {
t.Fatal(err)
}
clientRaw, err := os.ReadFile("testdata/catalogue-c05aa5e6-mesh-provisioner-client.module.json")
if err != nil {
t.Fatal(err)
}
store := []byte(`{"module":"store","version":"1","provides":[{"name":"artifact-store","scope":"mesh"}],` +
`"serves":{"artifact-store":{"port":5000}}}`)
texts := map[string]string{}
var packed bytes.Buffer
zw := gzip.NewWriter(&packed)
tw := tar.NewWriter(zw)
for _, f := range []string{"backups.conf", "data.conf", "retention.conf"} {
text, err := os.ReadFile("testdata/catalogue-c05aa5e6-restic.files/" + f)
if err != nil {
t.Fatal(err)
}
texts[f] = string(text)
if err := tw.WriteHeader(&tar.Header{Name: "./" + f, Mode: 0o644, Size: int64(len(text)), Typeflag: tar.TypeReg}); err != nil {
t.Fatal(err)
}
if _, err := tw.Write(text); err != nil {
t.Fatal(err)
}
}
if err := tw.Close(); err != nil {
t.Fatal(err)
}
if err := zw.Close(); err != nil {
t.Fatal(err)
}
archiveDigest := delivery.ManifestDigest(packed.Bytes())
digestOf := func(c string) string { return "sha256:" + strings.Repeat(c, 64) }
movedFps := map[string]string{"restic": "src2:" + strings.Repeat("3", 64), "mesh-provisioner-client": "src2:" + strings.Repeat("4", 64),
"store": "src2:" + strings.Repeat("5", 64)}
resticArtifacts := []delivery.BuiltArtifact{{Name: "tools", Kind: "bundle", Reference: "artifact-store://restic/tools/blobs/" + digestOf("6"), Digest: digestOf("6")},
{Name: "files", Kind: "archive", Reference: "artifact-store://restic/files/blobs/" + archiveDigest, Digest: archiveDigest}}
clientArtifacts := []delivery.BuiltArtifact{{Name: "client", Kind: "bundle", Reference: "artifact-store://mesh-provisioner-client/client/blobs/" + digestOf("7"), Digest: digestOf("7")}}
movedMesh := func(built bool) *bus {
b := one()
p := b.placements["n1"]
p.Modules = []delivery.PlacedModule{{Module: "store"}, {Module: "mesh-provisioner-client"}, {Module: "restic"}}
p.Decided.Account, p.Decided.AccountHome = "operator", "/home/operator"
b.placements["n1"] = p
rec := b.records["n1"]
rec.Contract, rec.Capabilities = compose.HandedContract, []string{"systemd", "package-manager"}
b.records["n1"] = rec
b.modules = map[string]delivery.Module{}
b.manifests = map[string][]byte{}
for module, raw := range map[string][]byte{"restic": resticRaw, "mesh-provisioner-client": clientRaw, "store": store} {
b.modules[module] = delivery.Module{FP2: movedFps[module], Manifest: delivery.ManifestDigest(raw)}
b.manifests[delivery.ManifestDigest(raw)] = raw
}
// The vault's own answer: the client's own secret; restic's is the client's to ask, so own leaves it out.
b.own = []compose.OwnSecretRow{{Node: "n1", Module: "mesh-provisioner-client", Name: "provisioner-broker", Origin: "made", SealedTo: "sk", Sealed: `"box"`}}
if built {
b.builds = map[string]*delivery.BuilderBuild{
"restic " + movedFps["restic"]: {State: delivery.BuildDone, Manifest: delivery.ManifestDigest(resticRaw), Artifacts: resticArtifacts},
"mesh-provisioner-client " + movedFps["mesh-provisioner-client"]: {State: delivery.BuildDone, Manifest: delivery.ManifestDigest(clientRaw), Artifacts: clientArtifacts},
"store " + movedFps["store"]: {State: delivery.BuildDone, Manifest: delivery.ManifestDigest(store)},
}
b.archives = map[string][]byte{archiveDigest: packed.Bytes()}
}
return b
}
if _, _, err := Compose(ctx, movedMesh(false), Mesh{}, Scope{Node: "n1"}, movedFps); err == nil || !strings.Contains(err.Error(), `restic needs a secret called "repository"`) {
t.Errorf("restic composed from the indexer's manifest: %v, want its own secret refused", err)
}
builtBus := movedMesh(true)
fromBuild, _, err := Compose(ctx, builtBus, Mesh{}, Scope{Node: "n1"}, movedFps)
if err != nil {
t.Fatalf("restic composed from its build: %v", err)
}
if !reflect.DeepEqual(builtBus.fetched, []string{"http://127.0.0.1:5000/v2/restic/files/blobs/" + archiveDigest}) {
t.Errorf("the templates archive was read at %v", builtBus.fetched)
}
resticKey, _ := delivery.ComposedKey("n1", "restic", movedFps["restic"])
if c := fromBuild[resticKey]; len(c.Filled) != 4 { // its three templated files and its own secret
t.Errorf("restic is not handed to the provisioner: its filled files are %v", c.Filled)
}
// A done key an artifact of which names no reference is waited for, in the controller's words (ADR 0498 §8).
unreferenced := movedMesh(true)
unreferenced.builds["restic "+movedFps["restic"]].Artifacts = []delivery.BuiltArtifact{{Name: "tools", Kind: "bundle", Digest: digestOf("6")}, resticArtifacts[1]}
if _, _, err := Compose(ctx, unreferenced, Mesh{}, Scope{Node: "n1"}, movedFps); err == nil || !strings.Contains(err.Error(), "the build of restic is done and waits for the reference of tools") {
t.Errorf("a done key with an unreferenced artifact: %v", err)
}
// A module the node runs inside its own composition (the fold) composes from the indexer's manifest though a done
// build of it is at the trunk's FP2: its build is not read.
folding := movedMesh(true)
foldIn, err := Input(ctx, folding, Mesh{}, Scope{Node: "n1", InOwn: []string{"mesh-provisioner-client"}})
if err != nil {
t.Fatal(err)
}
clientParsed, err := catalogue.ParseManifestForInputs(clientRaw)
if err != nil {
t.Fatal(err)
}
if slices.Contains(folding.buildsRead, "mesh-provisioner-client") || !reflect.DeepEqual(foldIn.Catalogue["mesh-provisioner-client"], clientParsed) {
t.Errorf("a folded module was taken from its build: builds read %v", folding.buildsRead)
}
// The controller's: its take-in resolves each manifest with the key's artifacts and the archive's texts, and composes
// from those over the same input.
storeIn, err := Input(ctx, movedMesh(false), Mesh{}, Scope{Node: "n1"})
if err != nil {
t.Fatal(err)
}
for module, built := range map[string][]catalogue.Built{
"restic": {{Name: "tools", Kind: "bundle", Reference: resticArtifacts[0].Reference, Digest: digestOf("6")},
{Name: "files", Kind: "archive", Reference: resticArtifacts[1].Reference, Digest: archiveDigest, Texts: texts}},
"mesh-provisioner-client": {{Name: "client", Kind: "bundle", Reference: clientArtifacts[0].Reference, Digest: digestOf("7")}},
} {
resolved, err := storeIn.Catalogue[module].Resolve(built)
if err != nil {
t.Fatal(err)
}
// Kept as its bytes and read again, as the controller's store holds a build's manifest.
raw, _ := json.Marshal(resolved)
if storeIn.Catalogue[module], err = catalogue.ParseManifestForInputs(raw); err != nil {
t.Fatal(err)
}
}
storeIn.OwnSecrets = movedMesh(true).own
fromStoreMoved, _, err := compose.Compositions(storeIn, "n1", movedFps, compose.Reading)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(fromBuild, fromStoreMoved) {
gj, _ := json.MarshalIndent(fromBuild, "", " ")
wj, _ := json.MarshalIndent(fromStoreMoved, "", " ")
t.Errorf("composed from the build differs from the controller's take-in:\n got %s\nwant %s", gj, wj)
}
// And what a provider on another node serves from its settings (mailu's smtp domain, issue 1126: shanks refused
// '${setting:domain} ... nothing sets "domain"'): read for that provider's served keys alone, through Served.
mailer := []byte(`{"module":"mailer","version":"1","config":{"domain":{"type":"string","description":"the mail domain","required":true}},` +
`"provides":[{"name":"smtp","scope":"mesh"}],"serves":{"smtp":{"port":587,"domain":"${config:domain}"}}}`)
withMailer := func(set bool) *bus {
b := mk()
p := b.placements["anchor"]
p.Modules = append(p.Modules, delivery.PlacedModule{Module: "mailer"})
b.placements["anchor"] = p
b.modules["mailer"] = delivery.Module{Manifest: delivery.ManifestDigest(mailer)}
b.manifests[delivery.ManifestDigest(mailer)] = mailer
if set {
b.served = map[string]map[string]any{"anchor.mailer": {"domain": "example.org"}}
}
return b
}
if _, _, err := Compose(ctx, withMailer(false), Mesh{}, Scope{Node: "laptop"}, fps); err == nil || !strings.Contains(err.Error(), `nothing sets "domain"`) {
t.Errorf("a provider's served setting nobody sets: %v", err)
}
served := withMailer(true)
if _, _, err := Compose(ctx, served, Mesh{}, Scope{Node: "laptop"}, fps); err != nil {
t.Errorf("laptop with what anchor's mailer serves: %v", err)
}
if !reflect.DeepEqual(served.servedRead, []string{"anchor.mailer:domain"}) || slices.Contains(served.valuesRead, "anchor.mailer") {
t.Errorf("laptop read %v of other nodes (and %v of config values): the served keys of anchor's mailer alone", served.servedRead, served.valuesRead)
}
for name, c := range map[string]struct {
change func(*bus)
scope Scope
says string
}{
"no record": {func(b *bus) { delete(b.records, "laptop") }, Scope{Node: "anchor"}, "laptop, whose agent has written no record"},
"no placement": {func(b *bus) {}, Scope{Node: "stranger"}, "stranger has no placement"},
"a manifest unread": {func(b *bus) { delete(b.manifests, digest) }, Scope{Node: "laptop"}, "the manifest of xterm"},
"a kept one unbuilt": {func(b *bus) {}, Scope{Node: "laptop", KeptAt: map[string]string{"xterm": "src2:gone"}}, "holds no done build"},
} {
b := mk()
c.change(b)
if _, err := Input(ctx, b, Mesh{}, c.scope); err == nil || !strings.Contains(err.Error(), c.says) {
t.Errorf("%s: %v, want it to say %q", name, err, c.says)
}
}
}
@@ -0,0 +1,60 @@
{
"module": "mesh-provisioner-client",
"version": "1",
"claims": [
{
"name": "node-provisioner",
"scope": "node"
}
],
"own-secrets": {
"provisioner-broker": "${dir:state}/broker"
},
"secrets-owner": "mesh-provisioner-client",
"resources": [
{
"id": "account",
"type": "user",
"name": "mesh-provisioner-client",
"shell": "/usr/bin/nologin",
"home": "/var/lib/mesh-provisioner-client"
},
{
"id": "state",
"type": "directory",
"mode": "0700",
"owner": "mesh-provisioner-client",
"place": ".",
"unknown": "program"
},
{
"id": "client",
"type": "process",
"name": "mesh-provisioner-client",
"artifact": "client",
"run": [
"./mesh-provisioner-client"
],
"env": {
"MESH_PROVISIONER_CREDENTIAL": "${dir:state}/broker",
"MESH_PROVISIONER_STATE": "${dir:state}"
},
"user": "mesh-provisioner-client",
"health": {
"kind": "unit"
}
}
],
"build": {
"artifacts": [
{
"name": "client",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/mesh-provisioner-client",
"binary": "mesh-provisioner-client"
}
]
}
}
@@ -0,0 +1,2 @@
# What the modules on this machine back up, composed by the mesh (novox/hq to-be 43). Do not edit.
${backup:lines}
@@ -0,0 +1,2 @@
# The data the modules on this machine declare, composed by the mesh (novox/hq ADR 0233). Do not edit.
${backup:data}
@@ -0,0 +1,6 @@
# How many restore points the night keeps, written by the mesh from this machine's settings keep-daily,
# keep-weekly, keep-monthly (novox/hq issue 416) and keep-hand-taken-days (novox/hq issue 453). Replaced on every push; change the settings, never this file.
daily=${config:keep-daily}
weekly=${config:keep-weekly}
monthly=${config:keep-monthly}
hand=${config:keep-hand-taken-days}
@@ -0,0 +1,199 @@
{
"module": "restic",
"version": "1",
"moved": true,
"claims": [
{
"name": "node-backup",
"scope": "node",
"serves": [
"backed-up",
"now",
"restore"
]
}
],
"emits": [
"backed-up"
],
"tools": [
"backup_health",
"backup_delete_retired",
"backup_deleted"
],
"own-secrets": {
"repository": "${dir:state}/repository.secret"
},
"data": {
"own": [
{
"id": "repository",
"path": "${dir:repository}",
"class": "rebuildable",
"backup": "none",
"why": "the restore points themselves: a copy of data kept elsewhere on this machine, never the only copy of anything; lost, the history goes and nothing live does"
},
{
"id": "state",
"path": "${dir:state}",
"class": "cache",
"why": "what the last nights and measurements were; the next night writes it again"
}
]
},
"config": {
"keep-daily": {
"type": "number",
"description": "How many nightly restore points are kept, a count from 1 to 9999: two weeks undo a mistake noticed within a fortnight. Read by the night, which thins nothing while it does not read (novox/hq issue 416); lowering it deletes the restore points beyond the new count at the next night, and that cannot be undone.",
"minimum": 1,
"maximum": 9999,
"trusted": true,
"default": 14
},
"keep-weekly": {
"type": "number",
"description": "How many weekly restore points are kept behind the dailies, a count from 1 to 9999, refused by the night like keep-daily; lowering it deletes the restore points beyond the new count at the next night, and that cannot be undone.",
"minimum": 1,
"maximum": 9999,
"trusted": true,
"default": 8
},
"keep-monthly": {
"type": "number",
"description": "How many monthly restore points are kept behind the weeklies, a count from 1 to 9999, refused by the night like keep-daily; lowering it deletes the restore points beyond the new count at the next night, and that cannot be undone.",
"minimum": 1,
"maximum": 9999,
"trusted": true,
"default": 6
},
"keep-hand-taken-days": {
"type": "number",
"description": "How many days a restore point taken by hand (node-backup.now, or the mesh's before a module moves) is kept at least, a count from 1 to 9999; it is kept until a later good night of its module exists, and this many days in any case, so the next thinning never takes the one taken against a risky change (novox/hq issue 453). Lowering it lets go of the older ones taken by hand at the next thinning, and that cannot be undone.",
"minimum": 1,
"maximum": 9999,
"trusted": true,
"default": 7
}
},
"resources": [
{
"id": "state",
"type": "directory",
"mode": "0700",
"place": ".",
"unknown": "program"
},
{
"id": "repository",
"type": "directory",
"mode": "0700",
"unknown": "program"
},
{
"id": "declared",
"type": "file",
"path": "${dir:state}/backups.conf",
"mode": "0600",
"template": "files/backups.conf"
},
{
"id": "data-declared",
"type": "file",
"path": "${dir:state}/data.conf",
"mode": "0600",
"template": "files/data.conf"
},
{
"id": "retention",
"type": "file",
"path": "${dir:state}/retention.conf",
"mode": "0644",
"trusted": true,
"template": "files/retention.conf"
},
{
"id": "tool",
"type": "package",
"package": "restic"
},
{
"id": "sqlite",
"type": "package",
"package": "sqlite"
},
{
"id": "night",
"type": "process",
"name": "restic-night",
"artifact": "tools",
"run": [
"./restic-backups",
"night"
],
"user": "${machine:account}",
"env": {
"MESH_BACKUP_DECLARED": "${dir:state}/backups.conf",
"MESH_BACKUP_REPOSITORY": "${dir:repository}",
"MESH_BACKUP_PASSWORD_FILE": "${dir:state}/repository.secret",
"MESH_BACKUP_STATE": "${dir:state}",
"MESH_BACKUP_DATA": "${dir:state}/data.conf"
},
"health": {
"kind": "tool",
"tool": "backup_health",
"interval": "60s",
"timeout": "20s",
"looks": 2,
"grace": "60s"
}
},
{
"id": "measure",
"type": "process",
"name": "restic-measure",
"artifact": "tools",
"run": [
"./restic-backups",
"measure"
],
"user": "${machine:account}",
"env": {
"MESH_BACKUP_DECLARED": "${dir:state}/backups.conf",
"MESH_BACKUP_REPOSITORY": "${dir:repository}",
"MESH_BACKUP_PASSWORD_FILE": "${dir:state}/repository.secret",
"MESH_BACKUP_STATE": "${dir:state}",
"MESH_BACKUP_DATA": "${dir:state}/data.conf"
},
"health": {
"kind": "unit"
}
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/restic-backups",
"binary": "restic-backups",
"loads": [
"restic-backups"
],
"env": {
"MESH_BACKUP_DECLARED": "${dir:state}/backups.conf",
"MESH_BACKUP_REPOSITORY": "${dir:repository}",
"MESH_BACKUP_PASSWORD_FILE": "${dir:state}/repository.secret",
"MESH_BACKUP_STATE": "${dir:state}",
"MESH_BACKUP_DATA": "${dir:state}/data.conf"
}
},
{
"name": "files",
"kind": "archive",
"from": "files"
}
]
}
}
+5
View File
@@ -189,6 +189,11 @@ type Node struct {
// Contract is the contract of its node-engine's newest statement (inv.HealthOf); Stated is whether it ever stated one.
Contract int
Stated bool
// InOwn is the modules it runs inside its own composition (`<node>.node`), not by a composition of their own, since
// no fingerprint of the new form names them where it runs them (novox/hq issue 1126): Compositions folds their
// resources into the node's own, as the controller's composed writer does, unless fps gives one of the new form.
// The private network's module (overlay.Name), which nobody builds, is folded whether or not it is named here.
InOwn map[string]bool
}
// Place is where a node is, as its placement says (inv.Overlay).
+1 -1
View File
@@ -113,7 +113,7 @@ func (c *composer) render(nodeName string, mode Mode) (Rendered, error) {
r.BusUsers = c.in.BusUsers
if r.BusUsers == "" {
r.BusUsers = ForeseenBusUsers
foreseen("the bus's user list")
foreseen(BusUsersStandIn)
}
if len(c.in.BusMissing) > 0 {
out.Said = append(out.Said, fmt.Sprintf("the bus's user list leaves out %d user(s) the mesh has minted no credential "+
+7
View File
@@ -325,6 +325,13 @@ func (c *composer) artifactStoreOnNetwork(on map[string]bool) (node, port string
return "", "", false, nil
}
// ArtifactStoreAddress is the artifact store as node reaches it, by the rule its composition routes every kept reference
// with (artifactStoreAddress): empty when node reaches none. A node composing its own modules reads a templates archive
// there (issue 1126).
func ArtifactStoreAddress(in Input, node string) (string, error) {
return newComposer(in).artifactStoreAddress(node)
}
// artifactStoreAddress is the artifact store as `forNode` reaches it: `<node>.internal:<port>` over the private network, or
// `127.0.0.1:<port>` for the node that holds the store itself when nothing is on the network yet, and empty for any other
// (novox/hq 04-ISSUES/102).
+14 -4
View File
@@ -17,6 +17,8 @@ import (
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/compose/frombus"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/sdkversion"
)
@@ -64,10 +66,15 @@ type Request struct {
}
// Case is one catalogue composed: every module's manifest by name, and per node the manifests kept at the build it runs.
//
// Placements are, by node, the placement the case composes that node with in place of what Input says of it: a change's
// own placement files, so a check composes a catalogue pull request's placement (issue 1126, frombus.Placed); a null
// placement leaves nothing placed on the node. A case without them composes Input's.
type Case struct {
Name string `json:"name"`
Manifests map[string]Manifest `json:"manifests"`
Kept map[string]map[string]Manifest `json:"kept,omitempty"`
Name string `json:"name"`
Manifests map[string]Manifest `json:"manifests"`
Kept map[string]map[string]Manifest `json:"kept,omitempty"`
Placements map[string]*delivery.Placement `json:"placements,omitempty"`
}
// Manifest is a module's manifest as its repository holds it (Raw, the bytes), or the digest the delivery indexer keeps
@@ -251,7 +258,10 @@ func runCase(r Request, c Case, depth string) (CaseVerdict, error) {
if err != nil {
return CaseVerdict{}, err
}
in := r.Input
in, err := frombus.Placed(r.Input, c.Placements)
if err != nil {
return CaseVerdict{}, fmt.Errorf("composer: the case %q: %w", c.Name, err)
}
in.Catalogue = shelf
in.Nodes = slices.Clone(in.Nodes)
in.Holdings = slices.Clone(in.Holdings)
+14
View File
@@ -10,6 +10,7 @@ import (
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
const demo = `{"module": "demo", "version": "1", "claims": [{"name": "node-files"}]}`
@@ -100,6 +101,19 @@ func TestARequestItCannotRunIsRefused(t *testing.T) {
if _, err := Run(Request{V: Version, Depth: "deep"}); err == nil {
t.Error("an unknown depth runs")
}
// A case's own placement (issue 1126) composes over a node the input holds, and is refused for one it does not.
placed := Request{V: Version, Input: compose.Input{Nodes: []compose.Node{{Name: "a"}}}, Machines: []string{"a"}, Depth: DepthResolve,
Cases: []Case{{Name: "trunk", Manifests: map[string]Manifest{"demo": {Raw: json.RawMessage(demo)}}},
{Name: "change", Manifests: map[string]Manifest{"demo": {Raw: json.RawMessage(demo)}},
Placements: map[string]*delivery.Placement{"a": {Node: "a", Modules: []delivery.PlacedModule{{Module: "ghost"}}}}}}}
if v, err := Run(placed); err != nil || !v.Cases[0].Machines["a"].Composes || v.Cases[1].Machines["a"].Composes ||
!strings.Contains(v.Cases[1].Machines["a"].Reason, "ghost") {
t.Errorf("a case composes its own placement: %+v, %v", v, err)
}
placed.Cases[1].Placements = map[string]*delivery.Placement{"z": {Node: "z"}}
if _, err := Run(placed); err == nil || !strings.Contains(err.Error(), "holds no node for") {
t.Errorf("a case placing a node the input lacks: %v", err)
}
v, err := Run(Request{V: Version, Cases: []Case{{Name: "only-read", Manifests: map[string]Manifest{"demo": {Raw: json.RawMessage(demo)}}}}})
if err != nil || len(v.Cases) != 1 || !v.Cases[0].Manifests["demo"].Accepted || v.Cases[0].Machines != nil || v.SDK == "" {
t.Fatalf("a request with no machines: %+v, %v", v, err)
+49 -10
View File
@@ -166,8 +166,9 @@ func checkFingerprint(fp, of string) error {
return nil
}
// Want is a node's want for a module: the fingerprint FP the node should run and the digest Composed of the one
// composition of it the node may apply (ADR 0539 clause 1), or Removed for a module that left the node's modules (then
// Want is a node's want for a module: the fingerprint FP the node should run and, before ADR 0547's second milestone,
// the digest Composed of the one composition of it the node may apply (ADR 0539 clause 1; from that milestone the
// node composes its own, ADR 0547 §2, and a want names none), or Removed for a module that left the node's modules (then
// both are empty). When the module's newest fingerprint is not yet allowed on this node, Newest names it and Rule the
// rule that holds the node back; both are empty when FP is the newest. Backup is the answer to a failed backup before
// the apply, BackupAgain or BackupSkip (clause 16), empty when none was asked. Words say why in a sentence. The
@@ -185,11 +186,18 @@ type Want struct {
// is released. A put's want names no newest fingerprint or rule.
Put bool `json:"put,omitempty"`
Hold bool `json:"hold,omitempty"`
// Yes is the digest of a composition a person allowed the node to move to (novox/hq ADR 0547 §3): a move of a module
// of policy record that the node agent judged to recreate a container or move its data is applied only when its
// want names the composition so allowed. Empty when none was asked.
Yes string `json:"yes,omitempty"`
// Self says the node composes this module itself (ADR 0547 §2): the want names the build alone, and no composition.
// A want that is not Self names the one composition it allows (ADR 0539 clause 1), as before.
Self bool `json:"self,omitempty"`
}
func (w Want) check() error {
if w.Removed {
if w.FP != "" || w.Composed != "" || w.Newest != "" || w.Rule != 0 || w.Backup != "" || w.Put || w.Hold {
if w.FP != "" || w.Composed != "" || w.Newest != "" || w.Rule != 0 || w.Backup != "" || w.Put || w.Hold || w.Self || w.Yes != "" {
return fmt.Errorf("delivery: a removed want names no fingerprint, composition, rule, backup answer or put")
}
return nil
@@ -203,11 +211,22 @@ func (w Want) check() error {
if err := checkFingerprint(w.FP, "a want's fingerprint"); err != nil {
return err
}
if w.Composed == "" {
return fmt.Errorf("delivery: the want of %s names no composition: a want names the digest of the one composition it allows (ADR 0539 clause 1)", w.FP)
// From ADR 0547's second milestone a node the catalogue places composes its own modules (§2): its want is Self and
// names the build alone. Any other want names the one composition it allows (ADR 0539 clause 1).
switch {
case w.Self && w.Composed != "":
return fmt.Errorf("delivery: the want of %s says the node composes it, and names a composition", w.FP)
case !w.Self && w.Composed == "":
return fmt.Errorf("delivery: the want of %s names no composition: a want names the digest of the one composition it allows (ADR 0539 clause 1), unless the node composes it itself", w.FP)
case w.Composed != "":
if err := checkDigest(w.Composed, "a want's composition digest"); err != nil {
return err
}
}
if err := checkDigest(w.Composed, "a want's composition digest"); err != nil {
return err
if w.Yes != "" {
if err := checkDigest(w.Yes, "the digest a want's yes allows"); err != nil {
return err
}
}
switch w.Backup {
case "", BackupAgain, BackupSkip:
@@ -309,11 +328,14 @@ type NodeWant struct {
V int `json:"v"`
Composed string `json:"composed"`
Words string `json:"words,omitempty"`
// Self says the node composes its own composition (ADR 0547 §2) and the want names none.
Self bool `json:"self,omitempty"`
}
// EncodeNodeWant is w as JSON at Version; it refuses a digest not of the sha256 form.
// EncodeNodeWant is w as JSON at Version; it refuses a digest not of the sha256 form, and a want that names no
// composition unless it says the node composes its own (Self, ADR 0547 §2).
func EncodeNodeWant(w NodeWant) ([]byte, error) {
if err := checkDigest(w.Composed, "the node's own want's composition digest"); err != nil {
if err := w.check(); err != nil {
return nil, err
}
w.V = Version
@@ -332,12 +354,23 @@ func DecodeNodeWant(data []byte) (NodeWant, error) {
if err := d.Decode(&w); err != nil {
return NodeWant{}, fmt.Errorf("delivery: a node's own want does not fit its shape: %w", err)
}
if err := checkDigest(w.Composed, "the node's own want's composition digest"); err != nil {
if err := w.check(); err != nil {
return NodeWant{}, err
}
return w, nil
}
// check refuses a node's own want that names no composition and does not say the node composes its own, or does both.
func (w NodeWant) check() error {
switch {
case w.Self && w.Composed != "":
return fmt.Errorf("delivery: the node's own want says the node composes it, and names a composition")
case w.Self:
return nil
}
return checkDigest(w.Composed, "the node's own want's composition digest")
}
// EncodeHeld is h as JSON at Version; it refuses a node name a key refuses.
func EncodeHeld(h Held) ([]byte, error) {
if err := h.check(); err != nil {
@@ -419,6 +452,12 @@ func ParseCompositionKey(key string) (group, headsHash string, err error) {
return group, headsHash, nil
}
// CompositionUnanswered is the Words of a composition that failed only because a node placed at the head or on the
// trunk did not answer the group's placement candidate within its bound (novox/hq ADR 0559 §5): the checker takes a
// later passing composition of the same heads in without a new check, since the failure was a part of the mesh not
// heard, not the change (ADR 0548 §1).
const CompositionUnanswered = "did not answer the placement candidate within its bound; judged again when it answers"
// CompositionVerdict is the value of a composition key: the group and its heads hash, the State (pass, fail or waits),
// the Words of a refusal or of what it waits for, the mesh-sdk commit it composed with (the assigner's own, or the
// group's mesh-sdk member's head, ADR 0482 clause 3), the machines composed, the first machine that failed, the
+17
View File
@@ -86,6 +86,9 @@ func (k Asker) Wait(ctx context.Context, a delivery.StepAsk) (delivery.Step, err
if !ok {
return delivery.Step{}, errors.New("the watch of the ask's step ended")
}
if seen := stepSeenOf(ctx); seen != nil {
seen(step)
}
if step.State != delivery.StepRunning {
return step, nil
}
@@ -108,6 +111,20 @@ func (k Asker) Wait(ctx context.Context, a delivery.StepAsk) (delivery.Step, err
}
}
type stepSeenKey struct{}
// WithStepSeen is ctx carrying seen, which Asker.Wait hands every value the ask's step key takes while it waits: the node
// that took it, each phase it says while running (delivery.Step.Phase), and its end. A try names its asks' progress so
// (issue 1890). seen runs in the waiting goroutine and must not block.
func WithStepSeen(ctx context.Context, seen func(delivery.Step)) context.Context {
return context.WithValue(ctx, stepSeenKey{}, seen)
}
func stepSeenOf(ctx context.Context) func(delivery.Step) {
seen, _ := ctx.Value(stepSeenKey{}).(func(delivery.Step))
return seen
}
// Ask puts a and waits for its step; a step that did not pass is an error naming its node, its state and its reason.
func (k Asker) Ask(ctx context.Context, a delivery.StepAsk) (delivery.Step, error) {
if err := k.Put(ctx, a); err != nil {
+4 -3
View File
@@ -26,9 +26,10 @@ func TestAnAskerPutsOnceAndReadsTheStep(t *testing.T) {
return delivery.Step{ID: a.ID, Node: "ace", State: delivery.StepPassed, Result: []byte(`{"artifacts":[]}`)}
}}
k := Asker{Publisher: f, Waiter: f}
step, err := k.Ask(context.Background(), goAsk("mvl0frv18d1957"))
if err != nil || step.Node != "ace" || len(f.Asks()) != 1 {
t.Fatalf("step %+v, %v, asks %d", step, err, len(f.Asks()))
var seen []delivery.Step // issue 1890: a waiter that asks to see each value of the step sees it
step, err := k.Ask(WithStepSeen(context.Background(), func(s delivery.Step) { seen = append(seen, s) }), goAsk("mvl0frv18d1957"))
if err != nil || step.Node != "ace" || len(f.Asks()) != 1 || len(seen) != 1 || seen[0].Node != "ace" {
t.Fatalf("step %+v, %v, asks %d, seen %+v", step, err, len(f.Asks()), seen)
}
f.Answer = func(a delivery.StepAsk) delivery.Step {
return delivery.Step{ID: a.ID, Node: "shanks", State: delivery.StepRefused, Reason: "go 1.27 installed"}
+167
View File
@@ -0,0 +1,167 @@
package delivery
import (
"bytes"
"encoding/json"
"strconv"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Comparison is what CompareComposed finds, module by module: the same, differing, added by what was composed (not run
// now) and removed by it (run now, not composed). Each list is sorted by module.
type Comparison struct {
Same []string
Differs []ModuleDiffers
Added []ModuleAdded
Removed []string
}
// CompareComposed compares the compositions a node composed (by key, as compose.Compositions answers them) with the ones
// it runs, as its agent would (ADR 0559 §4: a node's answer to a placement candidate). A module is the same when its
// composition's digest is the one it runs (ComposedDigestOf, the signature left out), so the agent would apply nothing
// of it; it differs when the digests differ (both named); it is added when composed and not run, removed when run and
// not composed. The node's own composition is named HaveNodeWord. The bus's user list (catalogue.BusUsersID) is left out
// on both sides: the bus node follows it alone and never applies it from a composition it made (ADR 0547 §2, §8). A
// difference names the first resource, by its id, whose content differs or that one side alone holds; when every
// resource is equal it says what else differs (their order, the files filled, the data, the node's own parts, a file
// carried by digest).
func CompareComposed(composed map[string]Composed, running []Composed) Comparison {
name := func(c Composed) string {
if c.Module == "" {
return HaveNodeWord
}
return c.Module
}
ours := map[string]Composed{}
for _, c := range composed {
ours[name(c)] = c
}
theirs := map[string]Composed{}
for _, c := range running {
theirs[name(c)] = c
}
modules := map[string]bool{}
for m := range ours {
modules[m] = true
}
for m := range theirs {
modules[m] = true
}
var out Comparison
for _, m := range sortedKeys(modules) {
a, composedHere := ours[m]
b, runs := theirs[m]
switch {
case !runs:
dig, _ := ComposedDigestOf(withoutBusUsers(a))
out.Added = append(out.Added, ModuleAdded{Module: m, Composed: dig})
case !composedHere:
out.Removed = append(out.Removed, m)
default:
if d, ok := compareOne(m, a, b); ok {
out.Same = append(out.Same, m)
} else {
out.Differs = append(out.Differs, d)
}
}
}
return out
}
// compareOne is one module's comparison: true when the digests agree, else what differs first.
func compareOne(module string, composed, running Composed) (ModuleDiffers, bool) {
composed, running = withoutBusUsers(composed), withoutBusUsers(running)
dc, errC := ComposedDigestOf(composed)
dr, errR := ComposedDigestOf(running)
if errC != nil {
return ModuleDiffers{Module: module, Running: dr, Why: "the composition has no digest: " + errC.Error()}, false
}
if errR == nil && dc == dr {
return ModuleDiffers{}, true
}
differs := func(resource, why string) (ModuleDiffers, bool) {
return ModuleDiffers{Module: module, Composed: dc, Running: dr, Resource: resource, Why: why}, false
}
// Words for the difference: resource by resource, each side with its large files by digest, so a file carried whole
// and the same file carried by digest read as one.
a, _, errA := composed.ByDigest()
b, _, errB := running.ByDigest()
if errA != nil || errB != nil {
return differs("", "the compositions differ and cannot be read resource by resource")
}
ra, rb := resourcesByID(a.Resources), resourcesByID(b.Resources)
ids := map[string]bool{}
for id := range ra {
ids[id] = true
}
for id := range rb {
ids[id] = true
}
for _, id := range sortedKeys(ids) {
x, inA := ra[id]
y, inB := rb[id]
switch {
case !inB:
return differs(id, "composed, and not in what runs")
case !inA:
return differs(id, "in what runs, and not composed")
case !bytes.Equal(x, y):
return differs(id, "the resource differs")
}
}
switch {
case !jsonEqual(a.Filled, b.Filled):
return differs("", "the files the provisioner fills differ")
case a.Data != b.Data:
return differs("", "whether it declares data differs")
case !jsonEqual(a.Own, b.Own):
return differs("", "the node's own parts differ")
case a.FP != b.FP:
return differs("", "the fingerprint differs")
}
if dA, err := ComposedDigestOf(a); err == nil {
if dB, err := ComposedDigestOf(b); err == nil && dA == dB {
return differs("", "the same content, a large file carried whole on one side and by digest on the other")
}
}
return differs("", "the same resources in another order")
}
// withoutBusUsers is c without the bus's user list resource (its id `bus-users`, or `<module>.bus-users`).
func withoutBusUsers(c Composed) Composed {
out := c
out.Resources = nil
for _, raw := range c.Resources {
var head struct {
ID string `json:"id"`
}
if json.Unmarshal(raw, &head) == nil && (head.ID == catalogue.BusUsersID() || strings.HasSuffix(head.ID, "."+catalogue.BusUsersID())) {
continue
}
out.Resources = append(out.Resources, raw)
}
return out
}
// resourcesByID is each resource canonical, by its id (a resource without one by its place).
func resourcesByID(resources []json.RawMessage) map[string][]byte {
out := make(map[string][]byte, len(resources))
for i, raw := range resources {
var head struct {
ID string `json:"id"`
}
_ = json.Unmarshal(raw, &head)
id := head.ID
if id == "" {
id = "#" + strconv.Itoa(i)
}
canon, err := canonicalJSON(raw)
if err != nil {
canon = raw
}
out[id] = canon
}
return out
}
+74 -8
View File
@@ -48,7 +48,9 @@ import (
"encoding/json"
"errors"
"fmt"
"reflect"
"regexp"
"slices"
"strings"
"time"
)
@@ -97,6 +99,10 @@ const (
// composition is applied, and the node waits for the answer in its want (novox/hq ADR 0539 clause 16). Detail
// carries the backup's own words.
PhaseHeld Phase = "held"
// PhaseWaitingYes is a module of policy record whose composed move the node agent judged to recreate a container or
// move its data (novox/hq ADR 0547 §3): nothing of it is applied, Judgement says the judgement and Waits the digest
// of the composition waiting, and the assigner asks a person; the node applies it once its want's Yes names Waits.
PhaseWaitingYes Phase = "waiting-yes"
)
// Health is what the module's own check says.
@@ -213,12 +219,15 @@ func parseModuleKeyPart(part string) (string, error) {
return module, nil
}
// HaveNodeWord is the last token of a node's own facts' key in HaveBucket.
const HaveNodeWord = nodeWord
// HaveNodeKey is the key, in HaveBucket, of a node's own facts: `<node>.node`.
func HaveNodeKey(node string) (string, error) {
if err := checkName("node", node); err != nil {
return "", err
}
return node + ".node", nil
return node + "." + HaveNodeWord, nil
}
// ParseHaveKey is the inverse of HaveKey and HaveNodeKey: isNode is true for `<node>.node`, with module empty.
@@ -337,6 +346,15 @@ type Have struct {
// UserList is what the node-engine of the node holding mesh-broker wrote of the bus's user list, in the have of the
// bus module (novox/hq ADR 0547 §7, §8). Nil on every other have.
UserList *UserListWritten `json:"user-list,omitempty"`
// Judgement and Waits are, in phase waiting-yes, the node agent's judgement of the move and the digest of the
// composition that waits for a person's yes (ADR 0547 §3); empty otherwise.
Judgement Judgement `json:"judgement,omitempty"`
Waits string `json:"waits,omitempty"`
// Sealed is, for a module the node composes itself, each value its running composition carries sealed to the node's
// key (a secret, a credential, a vault box), as applied (ADR 0547 §3, issue 1126): what the node's composer reuses
// after a restart while it opens to the vault's value, so a value sealed again moves no composition. Openable by the
// node alone, as the vault hands them over the bus. Empty on every other have.
Sealed []string `json:"sealed,omitempty"`
}
// UserListWritten is the bus's user list as the node-engine of the bus node wrote it to the bus module's file, applied
@@ -381,6 +399,17 @@ type HaveNode struct {
Strays json.RawMessage `json:"strays,omitempty"`
Filters json.RawMessage `json:"filters,omitempty"`
Profile json.RawMessage `json:"profile,omitempty"`
// Record is what the machine observes of itself, typed, for every node and composing part to read (issue 1126,
// ADR 0555 §3): the node's record lives in its own have key, so it needs no bucket, writer or grant of its own.
Record *NodeRecord `json:"record,omitempty"`
// Sealed is, on a node composing its own, each value its own composition carries sealed to the node's key, as
// applied (Have.Sealed for the node's own, issue 1126): a value the vault seals again after a restart moves no
// composition, so a restart applies nothing again.
Sealed []string `json:"sealed,omitempty"`
// Candidates is the node's answer to each placement candidate naming it, by group (PlacementCandidate): written by
// its shadow composer whenever an answer changes, removed when the candidate key goes. The node applies nothing of a
// candidate.
Candidates map[string]CandidateAnswer `json:"candidates,omitempty"`
}
// Presence is whether a node or a part's instance is online; the entry's own time says since when. The watchman
@@ -429,8 +458,18 @@ func decode(data []byte, into any) error {
func (h Have) check() error {
switch h.Phase {
case PhaseBackingUp, PhaseApplying, PhaseChecking, PhaseRunning, PhaseFailed, PhaseRemoved, PhaseHeld:
if h.Judgement != "" || h.Waits != "" {
return fmt.Errorf("delivery: only a have waiting for a yes names a judgement and the composition waiting")
}
case PhaseWaitingYes:
if h.Judgement != JudgedRecreates && h.Judgement != JudgedMovesData && h.Judgement != JudgedCannot {
return fmt.Errorf("delivery: a have waiting for a yes names the judgement %q, not recreates, moves-data or cannot-judge", h.Judgement)
}
if err := checkDigest(h.Waits, "the composition a have waits a yes for"); err != nil {
return err
}
default:
return fmt.Errorf("delivery: phase %q is not backing-up, applying, checking, running, failed, removed or held", h.Phase)
return fmt.Errorf("delivery: phase %q is not backing-up, applying, checking, running, failed, removed, held or waiting-yes", h.Phase)
}
if h.Composed != "" {
if err := checkDigest(h.Composed, "the have's composition digest"); err != nil {
@@ -466,17 +505,38 @@ func DecodeHave(data []byte) (Have, error) {
return h, nil
}
// EncodeHaveNode is n as JSON at Version.
func (n HaveNode) check() error {
for group, a := range n.Candidates {
if group == "" {
return fmt.Errorf("delivery: a node's candidate answer names no group")
}
if err := a.check(); err != nil {
return fmt.Errorf("delivery: the answer to %s's candidate: %w", group, err)
}
}
return nil
}
// EncodeHaveNode is n as JSON at Version; it refuses a candidate answer CandidateAnswer's rules refuse.
func EncodeHaveNode(n HaveNode) ([]byte, error) {
if err := n.check(); err != nil {
return nil, err
}
n.V = Version
return json.Marshal(n)
}
// DecodeHaveNode reads a node's facts; it refuses an unknown version (ErrUnknownVersion).
// DecodeHaveNode reads a node's facts; it refuses an unknown version (ErrUnknownVersion) and a candidate answer that
// is not one.
func DecodeHaveNode(data []byte) (HaveNode, error) {
var n HaveNode
err := decode(data, &n)
return n, err
if err := decode(data, &n); err != nil {
return n, err
}
if err := n.check(); err != nil {
return HaveNode{}, err
}
return n, nil
}
// EncodePresence is p as JSON at Version.
@@ -509,7 +569,11 @@ func DecodePulse(data []byte) (Pulse, error) {
func SameHave(a, b Have) bool {
ua, ub := a.UserList, b.UserList
a.UserList, b.UserList = nil, nil
if a != b || (ua == nil) != (ub == nil) {
if !slices.Equal(a.Sealed, b.Sealed) {
return false
}
a.Sealed, b.Sealed = nil, nil
if !reflect.DeepEqual(a, b) || (ua == nil) != (ub == nil) {
return false
}
if ua == nil {
@@ -524,7 +588,9 @@ func SameHave(a, b Have) bool {
// SameHaveNode says whether two node facts are the same value, comparing each carried JSON without its spacing.
func SameHaveNode(a, b HaveNode) bool {
return a.V == b.V && a.Composed == b.Composed && sameJSON(a.Windows, b.Windows) && sameJSON(a.Busy, b.Busy) &&
sameJSON(a.Strays, b.Strays) && sameJSON(a.Filters, b.Filters) && sameJSON(a.Profile, b.Profile)
sameJSON(a.Strays, b.Strays) && sameJSON(a.Filters, b.Filters) && sameJSON(a.Profile, b.Profile) &&
(a.Record == nil) == (b.Record == nil) && (a.Record == nil || SameNodeRecord(*a.Record, *b.Record)) &&
jsonEqual(a.Sealed, b.Sealed) && jsonEqual(a.Candidates, b.Candidates)
}
// SamePresence says whether two presences are the same value.
+40 -7
View File
@@ -14,6 +14,9 @@ import (
// 1131/AC1: the have key forms and parses back; bad names are refused, naming them.
func TestHaveKey(t *testing.T) {
t.Run("presence", testPresenceKey)
t.Run("pulse", testPulseKey)
t.Run("pulse ttl", testPulseTTL)
if HaveBucket != "node-engine_have" {
t.Fatalf("HaveBucket = %q", HaveBucket)
}
@@ -58,7 +61,7 @@ func TestHaveKey(t *testing.T) {
}
// 1131/AC2: the presence keys form and parse back; a node named part is refused.
func TestPresenceKey(t *testing.T) {
func testPresenceKey(t *testing.T) {
if PresenceBucket != "mesh-delivery-watchman_presence" {
t.Fatalf("PresenceBucket = %q", PresenceBucket)
}
@@ -98,7 +101,7 @@ func TestPresenceKey(t *testing.T) {
}
// 1131/AC3: the pulse key splits at the first dot, the key holding dots.
func TestPulseKey(t *testing.T) {
func testPulseKey(t *testing.T) {
if PulseBucket != "mesh-delivery-watchman_pulse" {
t.Fatalf("PulseBucket = %q", PulseBucket)
}
@@ -124,6 +127,7 @@ func TestPulseKey(t *testing.T) {
// 1131/AC4: each value encodes, decodes equal, and carries "v":1.
func TestRoundTrip(t *testing.T) {
t.Run("phase and health refused", testPhaseHealthRefused)
hasV1 := func(t *testing.T, b []byte) {
t.Helper()
var m map[string]any
@@ -149,7 +153,7 @@ func TestRoundTrip(t *testing.T) {
t.Fatal(err)
}
in.V = Version
if out != in {
if !SameHave(out, in) {
t.Fatalf("got %+v want %+v", out, in)
}
})
@@ -340,7 +344,7 @@ func TestUnknownVersion(t *testing.T) {
}
// 1131/AC6: a phase or health outside the lists is refused on encode and decode.
func TestPhaseHealthRefused(t *testing.T) {
func testPhaseHealthRefused(t *testing.T) {
for _, p := range []Phase{PhaseBackingUp, PhaseApplying, PhaseChecking, PhaseRunning, PhaseFailed} {
for _, h := range []Health{HealthOK, HealthFailed, HealthWaits, HealthUnknown} {
if _, err := EncodeHave(Have{Phase: p, Health: h}); err != nil {
@@ -363,10 +367,27 @@ func TestPhaseHealthRefused(t *testing.T) {
if _, err := DecodeHave([]byte(`{"v":1,"phase":"running","health":"fine"}`)); err == nil {
t.Error("bad health decoded")
}
// ADR 0547 §3 (issue 1126): a have waits for a yes with its judgement and the composition waiting, and no other have
// names either.
waits := Have{Phase: PhaseWaitingYes, Health: HealthOK, Judgement: JudgedRecreates, Waits: "sha256:" + strings.Repeat("a", 64)}
if raw, err := EncodeHave(waits); err != nil {
t.Errorf("a have waiting for a yes: %v", err)
} else if back, err := DecodeHave(raw); err != nil || back.Judgement != JudgedRecreates || back.Waits != waits.Waits {
t.Errorf("a have waiting for a yes came back %+v, %v", back, err)
}
for name, h := range map[string]Have{
"neither judged": {Phase: PhaseWaitingYes, Health: HealthOK, Judgement: JudgedNeither, Waits: waits.Waits},
"no waiting digest": {Phase: PhaseWaitingYes, Health: HealthOK, Judgement: JudgedMovesData},
"running judged": {Phase: PhaseRunning, Health: HealthOK, Judgement: JudgedRecreates},
} {
if _, err := EncodeHave(h); err == nil {
t.Errorf("%s: encoded", name)
}
}
}
// 1131/AC7: the pulse's time to live is 120 seconds minus the move's age; at the bound the move is stalled.
func TestPulseTTL(t *testing.T) {
func testPulseTTL(t *testing.T) {
for _, c := range []struct {
age time.Duration
ttl time.Duration
@@ -390,6 +411,7 @@ func TestPulseTTL(t *testing.T) {
// 1131/AC8: equal values say same, different ones do not.
func TestSame(t *testing.T) {
t.Run("have removed", testHaveRemoved)
a := Have{V: 1, Build: "b", Phase: PhaseRunning, Health: HealthOK}
b := a
if !SameHave(a, b) {
@@ -399,6 +421,17 @@ func TestSame(t *testing.T) {
if SameHave(a, b) {
t.Error("different haves same")
}
// The sealed values a self-composing node runs are part of its have (issue 1126).
sa, sb := a, a
sa.Sealed, sb.Sealed = []string{`{"box":1}`}, []string{`{"box":1}`}
if !SameHave(sa, sb) || SameHave(sa, a) {
t.Error("equal sealed values differ, or one said and one not are the same")
}
if raw, err := EncodeHave(sa); err != nil {
t.Error(err)
} else if back, err := DecodeHave(raw); err != nil || !SameHave(back, sa) {
t.Errorf("sealed values do not travel: %+v, %v", back, err)
}
// The user list is compared by what it says, never by where it is held (ADR 0547 §8).
at := time.Date(2026, 10, 25, 21, 0, 0, 0, time.UTC)
ua, ub := a, a
@@ -443,7 +476,7 @@ func TestSame(t *testing.T) {
}
}
func TestHaveRemoved(t *testing.T) { // 1370/AC3
func testHaveRemoved(t *testing.T) { // 1370/AC3
in := Have{Build: "b1", Phase: PhaseRemoved, Health: HealthUnknown, Detail: "the declaration no longer holds it"}
b, err := EncodeHave(in)
if err != nil {
@@ -454,7 +487,7 @@ func TestHaveRemoved(t *testing.T) { // 1370/AC3
t.Fatal(err)
}
in.V = Version
if out != in || out.Phase != "removed" {
if !SameHave(out, in) || out.Phase != "removed" {
t.Errorf("round trip: %+v", out)
}
}
+16 -4
View File
@@ -9,9 +9,10 @@ const oddManifest = "{\n\t\"module\": \"a\",\n \"z\": 1, \"a\": [ \"x\",\"y\"
func mergedWithInputs() Forge {
return Forge{Kind: ForgeMerged, Repository: "novox/mesh-sdk", Number: 36, Head: "aaaa", Merge: "cccc", Root: "t0",
Trees: map[string]string{"a": "t1", "b": "t2"},
Manifests: map[string]string{"a": oddManifest},
Locks: map[string]map[string]string{"a": {"go.sum": strings.Repeat("ab", 32)}}}
Trees: map[string]string{"a": "t1", "b": "t2"},
Manifests: map[string]string{"a": oddManifest},
Locks: map[string]map[string]string{"a": {"go.sum": strings.Repeat("ab", 32)}},
Placements: map[string]string{"ace": `{"v":1,"node":"ace","modules":[]}`}, PlacementsRead: true}
}
// 1442/AC1
@@ -21,7 +22,7 @@ func TestForgeMergedCarriesManifestsAndLocks(t *testing.T) {
if err != nil {
t.Fatal(err)
}
for _, frag := range []string{`"manifests":{"a":`, `"locks":{"a":{"go.sum":`} {
for _, frag := range []string{`"manifests":{"a":`, `"locks":{"a":{"go.sum":`, `"placements":{"ace":`, `"placements-read":true`} {
if !strings.Contains(string(data), frag) {
t.Errorf("the encoding lacks %s: %s", frag, data)
}
@@ -47,6 +48,9 @@ func TestForgeMergedRefusesBadInputs(t *testing.T) {
"locks outside trees": {func(f *Forge) { f.Locks["yy"] = map[string]string{"go.sum": "h"} }, `"yy"`},
"empty manifest": {func(f *Forge) { f.Manifests["b"] = "" }, `"b"`},
"empty lock hash": {func(f *Forge) { f.Locks["a"]["go.sum"] = "" }, `"a"`},
"placements unsaid": {func(f *Forge) { f.PlacementsRead = false }, "placements-read"},
"placement of no node": {func(f *Forge) { f.Placements["a.b"] = "{}" }, "names no node"},
"placements on a pull": {func(f *Forge) { f.Kind, f.Action, f.Merge, f.Root, f.Trees = ForgePull, PullOpened, "", "", nil }, "only a merged message"},
} {
f := mergedWithInputs()
tc.mut(&f)
@@ -82,6 +86,14 @@ func TestForgePullCarriesManifestsAndLocks(t *testing.T) {
if _, err := EncodeForge(f); err == nil || !strings.Contains(err.Error(), `"c"`) {
t.Errorf("locks outside Dirs: %v", err)
}
// A pull message of the catalogue carries its head's placement files whole (placement candidates, issue 1126).
f = Forge{Kind: ForgePull, Repository: CatalogueRepository, Number: 2, Head: "h", Action: PullUpdated,
Placements: map[string]string{"ace": `{"v":1,"node":"ace","modules":[]}`}, PlacementsRead: true}
if data, err := EncodeForge(f); err != nil {
t.Errorf("a catalogue pull with placement files: %v", err)
} else if got, err := DecodeForge(data); err != nil || !SameForge(got, withV(f)) {
t.Errorf("a catalogue pull's placement files changed on the way: %+v, %v", got, err)
}
for name, f := range map[string]Forge{
"refused manifests": {Kind: ForgeRefused, Repository: "a/b", Act: "x", Words: "no", Manifests: map[string]string{"a": "{}"}},
"refused locks": {Kind: ForgeRefused, Repository: "a/b", Act: "x", Words: "no", Locks: map[string]map[string]string{"a": {"go.sum": "h"}}},
+25
View File
@@ -7,6 +7,7 @@ import (
"fmt"
"reflect"
"regexp"
"sort"
"strconv"
"strings"
@@ -87,6 +88,30 @@ func MergeAskKey(group string, heads []Head) (string, error) {
return AskKindMerge + "." + part + "." + hash, nil
}
// SettleGroupPrefix begins the group of a settle ask (novox/hq ADR 0556 clause 5): a person's ask to settle deliveries held
// because their merge reached the trunk without a passing check. It is kept in the gate's asks under MergeAskKey like a merge
// ask, so the teller carries it unchanged; its options bind the act AskVerb names for it. Git forbids a colon in a branch
// name, so no pull request's group (its branch) ever begins with it.
const SettleGroupPrefix = "settle:"
// SettleGroup is the group of the settle ask of these delivery ids: the prefix and twelve hex digits of the sha256 over the
// ids, sorted and joined by a newline, so the same deliveries asked again keep one key.
func SettleGroup(ids []string) string {
sorted := append([]string(nil), ids...)
sort.Strings(sorted)
sum := sha256.Sum256([]byte(strings.Join(sorted, "\n")))
return SettleGroupPrefix + hex.EncodeToString(sum[:])[:12]
}
// AskVerb is the act the options of an ask of the gate bind, by its group: a settle ask's `mesh-delivery.settle`, any other
// `mesh-delivery.merge`. The gate composes the act with it and the teller checks a warrant against it.
func AskVerb(group string) string {
if strings.HasPrefix(group, SettleGroupPrefix) {
return "mesh-delivery.settle"
}
return "mesh-delivery.merge"
}
// ParseAskKey is the inverse of MergeAskKey: the kind, the group and the heads hash.
func ParseAskKey(key string) (kind, group, hash string, err error) {
parts := strings.Split(key, ".")
+15
View File
@@ -77,6 +77,21 @@ func TestAskID(t *testing.T) { // 1201/AC2
if AskID(k1) != AskID(k1) {
t.Error("the id is not stable")
}
// 1280 (ADR 0556 clause 5): a settle ask's group is the same for the same ids in any order, keys like any group, and its
// options bind mesh-delivery.settle; every other ask binds mesh-delivery.merge.
g := SettleGroup([]string{"novox/b@222222222222", "novox/a@111111111111"})
if g != SettleGroup([]string{"novox/a@111111111111", "novox/b@222222222222"}) || !strings.HasPrefix(g, SettleGroupPrefix) || len(g) != len(SettleGroupPrefix)+12 {
t.Errorf("SettleGroup = %q", g)
}
if k, err := MergeAskKey(g, gateHeads()); err != nil || !asks.UsableID(AskID(k)) {
t.Errorf("the settle ask's key %q: %v", k, err)
}
for group, want := range map[string]string{g: "mesh-delivery.settle", "feat/x": "mesh-delivery.merge",
"settle/foo": "mesh-delivery.merge", "settle": "mesh-delivery.merge"} { // a branch named settle/... is a merge's group
if got := AskVerb(group); got != want {
t.Errorf("AskVerb(%q) = %q, want %q", group, got, want)
}
}
}
func TestOwedKey(t *testing.T) { // 1201/AC3
+229
View File
@@ -0,0 +1,229 @@
package delivery
import (
"bytes"
"encoding/json"
"path"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// The judgement of a move (ADR 0377 §3), as a pure function over two compositions of one module, so the node agent that
// composes its own modules judges what it would apply (novox/hq ADR 0547 §3; issue 1126's S8). Moved from
// mesh-controller cmd/mesh-controller/composed_judge.go at origin/main (the controller keeps its copy, unchanged, until
// it loses the lease, ADR 0547 §1): judged, onlyTheUserList and sameResources, with their helpers, are its code.
//
// ADR 0377 §3 over two compositions of one module: **recreates** when a container resource is changed, added or
// removed; **moves data** when, with no container changed, a directory or file resource is changed, added or removed
// that a container of the module mounts or that holds or lies within one of its directories (where a module keeps its
// data), when its `data` declaration changes, or, for a module that declares data, when one of its long-running
// processes changes; **neither** otherwise, and for a module the node has never run.
// Judged is one move's judgement: what it does, its words, and whether only the bus's user list changes (ADR 0539 §7:
// a file change and a reload of the bus, no move of it, never asked).
type Judged struct {
Judgement Judgement
Words string
UserListOnly bool
}
// JudgeMove judges moving a node's module from the composition it runs to the one composed for it. Running is nil when the
// node runs no composition of the module. A composition equal to the one running (resource for resource, each
// canonical) is unchanged: no move, and no ask (ADR 0547 §3).
func JudgeMove(running *Composed, composed Composed) Judged {
if composed.Module == "" {
return Judged{Judgement: JudgedNeither, Words: "the node's own composition"}
}
if running == nil {
return Judged{Judgement: JudgedNeither, Words: "the node runs no composition of " + composed.Module}
}
if running.Data == composed.Data && SameComposedResources(*running, composed) {
return Judged{Judgement: JudgedUnchanged, Words: "equal to the composition the node runs"}
}
if onlyTheUserList(*running, composed) {
return Judged{Judgement: JudgedNeither, Words: "only the bus's user list changes", UserListOnly: true}
}
j, words := judged(*running, composed)
return Judged{Judgement: j, Words: words}
}
// busUsersSuffix ends the id of the bus module's user list as the composer writes it: `<module>.bus-users`.
const busUsersSuffix = ".bus-users"
// onlyTheUserList says two compositions of a module differ, and only in its file of the bus's user list.
func onlyTheUserList(from, to Composed) bool {
if from.Data != to.Data {
return false
}
before, after := judgedResourcesByID(from), judgedResourcesByID(to)
if len(before) != len(after) {
return false
}
changed := false
for id, now := range after {
was, ok := before[id]
if !ok {
return false
}
if bytes.Equal(was.raw, now.raw) {
continue
}
if !strings.HasSuffix(id, busUsersSuffix) || now.kind != string(declaration.TypeFile) {
return false
}
changed = true
}
return changed
}
// judged applies ADR 0377 §3 to two compositions of one module.
func judged(from, to Composed) (Judgement, string) {
before, after := judgedResourcesByID(from), judgedResourcesByID(to)
if ids := changedOf(before, after, func(kind string) bool { return kind == string(declaration.TypeContainer) }); len(ids) > 0 {
return JudgedRecreates, "it recreates " + strings.Join(ids, ", ")
}
if from.Data != to.Data {
return JudgedMovesData, "its data declaration changes"
}
places := dataPlaces(before, after)
held := func(kind string) bool {
return kind == string(declaration.TypeDirectory) || kind == string(declaration.TypeFile)
}
var moved []string
for _, id := range changedOf(before, after, held) {
r := before[id]
if r == nil {
r = after[id]
}
if withinPlaces(r.path, places) {
moved = append(moved, id)
}
}
if to.Data || from.Data {
moved = append(moved, changedOf(before, after, func(kind string) bool { return kind == string(declaration.TypeProcess) })...)
}
if len(moved) > 0 {
sort.Strings(moved)
return JudgedMovesData, "it changes where its data is kept: " + strings.Join(moved, ", ")
}
return JudgedNeither, "it recreates no container and moves no data"
}
// judgedResource is a resource as the judgement reads it: its kind, the path it acts on, what its containers mount,
// and its canonical bytes.
type judgedResource struct {
kind string
path string
mounts []string
raw []byte
}
func judgedResourcesByID(c Composed) map[string]*judgedResource {
out := map[string]*judgedResource{}
for _, raw := range c.Resources {
var head struct {
ID string `json:"id"`
Type string `json:"type"`
Path string `json:"path"`
// A container's volumes, `<path on the machine>:<path inside>[:<options>]`.
Volumes []string `json:"volumes"`
}
if err := json.Unmarshal(raw, &head); err != nil {
continue
}
r := &judgedResource{kind: head.Type, path: head.Path, raw: compactResource(raw)}
for _, v := range head.Volumes {
source, _, _ := strings.Cut(v, ":")
r.mounts = append(r.mounts, source)
}
out[head.ID] = r
}
return out
}
// compactResource is a resource's bytes canonical: its keys sorted, its spacing dropped, its numbers as written, so two
// compositions that say the same compare equal however each was written.
func compactResource(raw []byte) []byte {
var v any
d := json.NewDecoder(bytes.NewReader(raw))
d.UseNumber()
if err := d.Decode(&v); err != nil {
return raw
}
out, err := json.Marshal(v)
if err != nil {
return raw
}
return out
}
// changedOf are the ids of the resources of the kinds counted that are changed, added or removed, sorted.
func changedOf(before, after map[string]*judgedResource, counted func(kind string) bool) []string {
var out []string
for id, now := range after {
if !counted(now.kind) {
continue
}
if was, ok := before[id]; !ok || !bytes.Equal(was.raw, now.raw) {
out = append(out, id)
}
}
for id, was := range before {
if _, ok := after[id]; !ok && counted(was.kind) {
out = append(out, id)
}
}
sort.Strings(out)
return out
}
// dataPlaces are the paths where the module keeps what a move must not lose: what its containers mount, and its
// directories, before and after.
func dataPlaces(before, after map[string]*judgedResource) []string {
var out []string
for _, rs := range []map[string]*judgedResource{before, after} {
for _, r := range rs {
if r.kind == string(declaration.TypeContainer) {
out = append(out, r.mounts...)
}
if r.kind == string(declaration.TypeDirectory) && r.path != "" {
out = append(out, r.path)
}
}
}
return out
}
// withinPlaces says p is one of places, or lies within one, or holds one.
func withinPlaces(p string, places []string) bool {
if p == "" {
return false
}
p = path.Clean(p)
for _, place := range places {
if place == "" || !strings.HasPrefix(place, "/") {
continue
}
place = path.Clean(place)
if p == place || strings.HasPrefix(p, place+"/") || strings.HasPrefix(place, p+"/") {
return true
}
}
return false
}
// SameComposedResources says two compositions of a module hold the same resources in the same order, each compared
// canonical.
func SameComposedResources(a, b Composed) bool {
if len(a.Resources) != len(b.Resources) {
return false
}
for i := range a.Resources {
if !bytes.Equal(compactResource(a.Resources[i]), compactResource(b.Resources[i])) {
return false
}
}
return true
}
+68
View File
@@ -0,0 +1,68 @@
package delivery
import (
"encoding/json"
"strings"
"testing"
)
func composedOf(module string, c byte, data bool, resources ...string) Composed {
out := Composed{Node: "ace", Module: module, FP: SourcePrefix2 + strings.Repeat(string(c), 64), Data: data}
for _, r := range resources {
out.Resources = append(out.Resources, json.RawMessage(r))
}
return out
}
// The pairs mesh-controller's composed_judge_test.go captured (issue 1945 AC12), and the bus's user list, judged as the
// controller judges them (ADR 0377 §3, ADR 0539 §7), by the node agent's function (ADR 0547 §3, issue 1126 S8).
func TestJudgeMove(t *testing.T) {
const (
container = `{"id":"mesh-mail.server","type":"container","module":"mesh-mail","name":"mail","image":"mail@sha256:1111111111111111111111111111111111111111111111111111111111111111","volumes":["/var/lib/mesh-mail/spool:/spool"]}`
container2 = `{"id":"mesh-mail.server","type":"container","module":"mesh-mail","name":"mail","image":"mail@sha256:2222222222222222222222222222222222222222222222222222222222222222","volumes":["/var/lib/mesh-mail/spool:/spool"]}`
spool = `{"id":"mesh-mail.spool","type":"directory","module":"mesh-mail","path":"/var/lib/mesh-mail/spool"}`
spoolMoved = `{"id":"mesh-mail.spool","type":"directory","module":"mesh-mail","path":"/srv/mesh-mail/spool"}`
tools = `{"id":"mesh-mail.tools","type":"file","module":"mesh-mail","path":"/usr/lib/mesh-mail/tools.json","content":"a"}`
tools2 = `{"id":"mesh-mail.tools","type":"file","module":"mesh-mail","path":"/usr/lib/mesh-mail/tools.json","content":"b"}`
config = `{"id":"mesh-mail.conf","type":"file","module":"mesh-mail","path":"/var/lib/mesh-mail/spool/conf","content":"a"}`
config2 = `{"id":"mesh-mail.conf","type":"file","module":"mesh-mail","path":"/var/lib/mesh-mail/spool/conf","content":"b"}`
process = `{"id":"mesh-mail.daemon","type":"process","module":"mesh-mail","exec":"/usr/bin/a"}`
process2 = `{"id":"mesh-mail.daemon","type":"process","module":"mesh-mail","exec":"/usr/bin/b"}`
users = `{"id":"nats.bus-users","type":"file","module":"nats","path":"/etc/nats/users","content":"a"}`
users2 = `{"id":"nats.bus-users","type":"file","module":"nats","path":"/etc/nats/users","content":"b"}`
natsConf = `{"id":"nats.conf","type":"file","module":"nats","path":"/etc/nats/conf","content":"a"}`
natsConf2 = `{"id":"nats.conf","type":"file","module":"nats","path":"/etc/nats/conf","content":"b"}`
)
mail := func(c byte, data bool, rs ...string) *Composed {
m := composedOf("mesh-mail", c, data, rs...)
return &m
}
bus := func(rs ...string) *Composed { m := composedOf("nats", 'b', false, rs...); return &m }
for _, c := range []struct {
name string
from, to *Composed
want Judgement
userList bool
}{
{"a new image", mail('a', true, container, spool, tools), mail('b', true, container2, spool, tools), JudgedRecreates, false},
{"a container gone", mail('a', true, container, spool), mail('b', true, spool), JudgedRecreates, false},
{"its spool moved", mail('a', true, container, spool), mail('b', true, container, spoolMoved), JudgedMovesData, false},
{"a file in its spool", mail('a', true, container, spool, config), mail('b', true, container, spool, config2), JudgedMovesData, false},
{"its data declaration", mail('a', false, container, spool), mail('b', true, container, spool), JudgedMovesData, false},
{"a process of a module with data", mail('a', true, spool, process), mail('b', true, spool, process2), JudgedMovesData, false},
{"a process of a module without", mail('a', false, process), mail('b', false, process2), JudgedNeither, false},
{"its tools only", mail('a', true, container, spool, tools), mail('b', true, container, spool, tools2), JudgedNeither, false},
{"never ran", nil, mail('b', true, container), JudgedNeither, false},
{"equal, written otherwise", mail('a', true, `{"type":"directory","id":"mesh-mail.spool","module":"mesh-mail","path":"/var/lib/mesh-mail/spool"}`), mail('b', true, spool), JudgedUnchanged, false},
{"only the user list", bus(natsConf, users), bus(natsConf, users2), JudgedNeither, true},
{"the user list and more", bus(natsConf, users), bus(natsConf2, users2), JudgedNeither, false},
} {
got := JudgeMove(c.from, *c.to)
if got.Judgement != c.want || got.UserListOnly != c.userList || got.Words == "" {
t.Errorf("%s: %+v, want %s (user list only %v)", c.name, got, c.want, c.userList)
}
}
if got := JudgeMove(nil, Composed{Node: "ace"}); got.Judgement != JudgedNeither {
t.Errorf("the node's own composition: %+v", got)
}
}
+98
View File
@@ -0,0 +1,98 @@
package delivery
import (
"encoding/json"
"fmt"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// A node's record is what the machine observes of itself (issue 1126, ADR 0555 §3), written by its own agent in its
// own have key, `<node>.node` of the node-engine's have bucket (HaveNode.Record), whenever an observed fact changes and
// only when the value differs. The key's change is the event its readers hear: every node and the parts that compose
// watch it. Where a module runs, and every fact a person decides of the node (its endpoint, site, hub, address on the
// private network, public domain, accounts, the tunnel found at adoption), are the catalogue's placement
// (placement.go). Nothing in it is secret: its keys are public halves.
// RecordTunnel is a tunnel found on a node when it was adopted, as compose.Tunnel holds it: public keys and addresses
// only. After the node took it over the tunnel is no longer there to observe, so it is kept as what was decided at
// adoption, in the node's placement (NodeDecided.Tunnel).
type RecordTunnel struct {
Interface string `json:"interface"`
Unit string `json:"unit,omitempty"`
Config string `json:"config,omitempty"`
Port int `json:"port,omitempty"`
MTU int `json:"mtu,omitempty"`
Address string `json:"address,omitempty"`
Range string `json:"range,omitempty"`
PublicKey string `json:"public-key,omitempty"`
Peers []RecordTunnelPeer `json:"peers,omitempty"`
}
// RecordTunnelPeer is one peer of a found tunnel.
type RecordTunnelPeer struct {
PublicKey string `json:"public-key"`
Address string `json:"address,omitempty"`
}
// NodeRecord is one node's record: what the machine observes of itself (issue 1126, ADR 0555: what a person decides
// lives in the catalogue's placement file, Placement.Decided; what the machine observes lives here). TunnelKey is the
// public key of its private-network tunnel; Certificate the one the mesh issued for its internal name, public, as the
// node holds it. The tunnel found at adoption and the names given to its peers are decisions kept in the placement;
// whether the hub took its found tunnel over, and which node enrolled under a carried peer's key, are read from both.
type NodeRecord struct {
V int `json:"v"`
Node string `json:"node"`
TunnelKey string `json:"tunnel-key,omitempty"`
Capabilities []string `json:"capabilities,omitempty"`
SealingKey string `json:"sealing-key,omitempty"`
ServingKey string `json:"serving-key,omitempty"`
Certificate string `json:"certificate,omitempty"`
OutwardLinks []string `json:"outward-links,omitempty"`
Networks []catalogue.AttachedNetwork `json:"networks,omitempty"`
NetworksSaid bool `json:"networks-said,omitempty"`
Contract int `json:"contract,omitempty"`
// Authority is the mesh's own certificate as this machine holds it (public): the one its declaration last wrote where
// a module asked for it. A node composing its own composes a certificate with it (compose.Input.Authority).
Authority string `json:"authority,omitempty"`
}
func (r NodeRecord) check() error {
if err := checkName("node", r.Node); err != nil {
return err
}
for _, c := range r.Capabilities {
if c == "" {
return fmt.Errorf("delivery: the record of %s names an empty capability", r.Node)
}
}
return nil
}
// EncodeNodeRecord is r as JSON at Version; it refuses what DecodeNodeRecord refuses.
func EncodeNodeRecord(r NodeRecord) ([]byte, error) {
if err := r.check(); err != nil {
return nil, err
}
r.V = Version
return json.Marshal(r)
}
// DecodeNodeRecord reads a node record; it refuses an unknown version (ErrUnknownVersion), a node that is not a name, and
// an empty capability.
func DecodeNodeRecord(data []byte) (NodeRecord, error) {
var r NodeRecord
if err := decode(data, &r); err != nil {
return NodeRecord{}, err
}
if err := r.check(); err != nil {
return NodeRecord{}, err
}
return r, nil
}
// SameNodeRecord says whether two records are the same value, so the agent writes only when they differ.
func SameNodeRecord(a, b NodeRecord) bool {
a.V, b.V = 0, 0
return jsonEqual(a, b)
}
+73
View File
@@ -0,0 +1,73 @@
package delivery
import (
"reflect"
"regexp"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// secretNamed is facts.secretKey's pattern: a field so named would read as a secret, and the runtime's credential-field
// check (ADR 0201 §13) refuses such a field in a state every node reads.
var secretNamed = regexp.MustCompile(`(?i)(secret|passw|token|api[-_]?key|private[-_]?key|credential|bearer|cookie|salt|signing)`)
// Issue 1126 (ADR 0555): a node's record reads back, rides in its have key, refuses a node that is not a name and an empty capability, and
// neither it nor the placement file names a field like a secret.
func TestNodeRecord(t *testing.T) {
hub := NodeRecord{Node: "novox", TunnelKey: "k", Capabilities: []string{"docker"}, SealingKey: "age1x", ServingKey: "pub",
Certificate: "cert", OutwardLinks: []string{"eth0"}, Networks: []catalogue.AttachedNetwork{{Link: "eth0", Network: "192.0.2.0/24"}},
NetworksSaid: true, Contract: 3}
raw, err := EncodeNodeRecord(hub)
if err != nil {
t.Fatal(err)
}
back, err := DecodeNodeRecord(raw)
if err != nil || !SameNodeRecord(back, hub) || back.V != Version {
t.Fatalf("round trip: %+v, %v", back, err)
}
// It rides in the node's own have key: a have that differs only in its record is another value.
with, without := HaveNode{V: Version, Record: &hub}, HaveNode{V: Version}
if raw, err := EncodeHaveNode(with); err != nil {
t.Fatal(err)
} else if got, err := DecodeHaveNode(raw); err != nil || got.Record == nil || !SameHaveNode(got, with) || SameHaveNode(got, without) {
t.Fatalf("the have's record: %+v, %v", got, err)
}
changed := hub
changed.Contract = 4
if SameHaveNode(with, HaveNode{V: Version, Record: &changed}) {
t.Error("a changed record reads as the same have")
}
for name, r := range map[string]NodeRecord{
"empty capability": {Node: "ace", Capabilities: []string{""}},
"node that is not a name": {Node: "a.b"},
} {
if _, err := EncodeNodeRecord(r); err == nil {
t.Errorf("%s: taken", name)
}
}
// Every JSON field name of the record and the types it carries, walked: none reads as a secret.
var walk func(reflect.Type, string)
walk = func(ty reflect.Type, at string) {
for ty.Kind() == reflect.Pointer || ty.Kind() == reflect.Slice {
ty = ty.Elem()
}
if ty.Kind() != reflect.Struct {
return
}
for i := 0; i < ty.NumField(); i++ {
f := ty.Field(i)
name, _, _ := strings.Cut(f.Tag.Get("json"), ",")
if secretNamed.MatchString(name) || secretNamed.MatchString(f.Name) {
t.Errorf("%s.%s (%q) is named like a secret", at, f.Name, name)
}
walk(f.Type, at+"."+f.Name)
}
}
walk(reflect.TypeOf(NodeRecord{}), "NodeRecord")
// And the placement file, which holds what a person decides of a node (K2): accounts by name, never a credential.
walk(reflect.TypeOf(Placement{}), "Placement")
}
+470
View File
@@ -0,0 +1,470 @@
package delivery
import (
"encoding/json"
"fmt"
"path"
"sort"
"strconv"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Placement of modules on nodes lives in the catalogue (novox/hq ADR 0549): one file per node, `placement/<node>.json`
// in the catalogue repository, changed only by a pull request. At each trunk merge of the catalogue the indexer reads
// the files and writes each node's key `placement.<node>` in its state `placement`, only when it differs, and says
// `placement-changed` naming the node, the revision and the modules added, removed or changed (ADR 0549 §2). The node
// agents, the assigner, the grants part and mesh-config read it (§3); nothing in it is secret.
const (
// PlacementState is the indexer's state of placements, and PlacementBucket its bucket.
PlacementState = "placement"
PlacementBucket = IndexerModule + "_" + PlacementState
// PlacementHistory is the revisions the placement bucket keeps per key.
PlacementHistory = 10
// PlacementDir is the catalogue's directory of placement files, one `<node>.json` per node.
PlacementDir = "placement"
// PlacementChangedEvent is what the indexer says when a node's placement key changed.
PlacementChangedEvent = "placement-changed"
placementWord = "placement"
// CatalogueRepository is the repository whose placement files are the mesh's placement (ADR 0549 §1): the forge bridge
// reads them at each of its merges, and no other repository's.
CatalogueRepository = "novox/mesh-catalog"
)
// PlacementKey is the key, in PlacementBucket, of one node's placement: `placement.<node>`.
func PlacementKey(node string) (string, error) {
if err := checkName("node", node); err != nil {
return "", err
}
return placementWord + "." + node, nil
}
// ParsePlacementKey is the inverse of PlacementKey; it refuses any other key, naming it.
func ParsePlacementKey(key string) (node string, err error) {
node, ok := strings.CutPrefix(key, placementWord+".")
if !ok || checkName("node", node) != nil {
return "", fmt.Errorf("delivery: %q is not a placement key, placement.<node>", key)
}
return node, nil
}
// PlacementFile is the path, in the catalogue repository, of one node's placement file.
func PlacementFile(node string) string { return path.Join(PlacementDir, node+".json") }
// PlacementFileNode is the node a catalogue path is the placement file of, and whether it is one: `placement/<node>.json`
// directly under PlacementDir.
func PlacementFileNode(p string) (string, bool) {
dir, file := path.Split(p)
node, ok := strings.CutSuffix(file, ".json")
if path.Clean(dir) != PlacementDir || !ok || checkName("node", node) != nil {
return "", false
}
return node, true
}
// Provider is the module on a node that answers a requirement or holds a seat: a pin's or a binding's provider.
type Provider struct {
Node string `json:"node"`
Module string `json:"module"`
}
// Chosen is p as the composition names a chosen provider.
func (p Provider) Chosen() catalogue.Chosen { return catalogue.Chosen{Node: p.Node, Module: p.Module} }
// PlacedModule is one module placed on the node: the host port each port it declares is published on (by the port the
// software listens on), and, by provision, the provider each of its requirements that keeps data is bound to.
//
// MeshWide and Layer are where the module's things are, as a settings layer's placement holds them (catalogue.Layer's
// Placement: `places`, `accesses`, `networks`, `endpoints`, `expose`, `reach`, `ports`; ADR 0348 §11): the whole mesh's
// layer and this node's, which the controller keeps in its table `placement` (ADR 0549 §6 names it) and composes as two
// layers, the mesh's first. Each node's file repeats the mesh-wide layer of the modules placed on it, so a node reads
// its own key alone. HasLayer says the node has a layer of its own even when it holds no placement, since a layer
// that is there and empty still carries the module's config values (Layers).
//
// MeshValues and Values are the config values of a module in the settings form (its manifest declares no config, so
// mesh-config holds none for it): what a person set in the whole mesh's layer and this node's, beside the placement
// (issue 1126; on 2026-10-26 the media modules on one node, plex's puid and pgid among them). A module in the config
// form takes its values from mesh-config and has none here.
type PlacedModule struct {
Module string `json:"module"`
Ports map[string]int `json:"ports,omitempty"`
Bindings map[string]Provider `json:"bindings,omitempty"`
MeshWide map[string]any `json:"mesh-wide,omitempty"`
HasMesh bool `json:"has-mesh-wide,omitempty"`
Layer map[string]any `json:"layer,omitempty"`
HasLayer bool `json:"has-layer,omitempty"`
MeshValues map[string]any `json:"mesh-wide-values,omitempty"`
Values map[string]any `json:"values,omitempty"`
}
// Holding is one seat a module placed on the node holds (catalogue.Held without its node, which is the file's).
type Holding struct {
Claim string `json:"claim"`
Scope string `json:"scope,omitempty"`
Module string `json:"module"`
Site string `json:"site,omitempty"`
Kind string `json:"kind,omitempty"`
}
// Placement is one node's placement: the file the catalogue keeps and, with Revision, the value of its key. Modules are
// in the order the file names them, which is the order the composition reads the node's modules in. Pins is, by
// provision, the provider chosen for the node where more than one could answer.
type Placement struct {
V int `json:"v"`
Node string `json:"node"`
Revision string `json:"revision,omitempty"`
Decided NodeDecided `json:"decided"`
Modules []PlacedModule `json:"modules"`
Pins map[string]Provider `json:"pins,omitempty"`
Holds []Holding `json:"holds,omitempty"`
}
// NodeDecided is what a person decides of the node, kept in its placement file (issue 1126's K2: what a person decides
// lives in the catalogue, what the machine observes lives in its record, NodeRecord): where it is dialled (endpoint) and
// its site, whether it is the private network's hub, its address on the private network, the domain its routed names
// are composed under, its operator and agent logins by name and their homes, whether it is adopted rather than
// converged, the modules taken on an adopted node, the tunnel found on it at adoption and, on the hub, the names given
// to that tunnel's peers. The address is a decision written once, at enrolment, by a
// catalogue pull request: nothing allocates it at run time, and CheckPlacements refuses two nodes with one address.
// No secret and no key material: accounts are names, never credentials.
type NodeDecided struct {
Endpoint string `json:"endpoint,omitempty"`
Site string `json:"site,omitempty"`
Hub bool `json:"hub,omitempty"`
Address string `json:"address,omitempty"`
PublicDomain string `json:"public-domain,omitempty"`
Account string `json:"account,omitempty"`
AccountHome string `json:"account-home,omitempty"`
AgentAccount string `json:"agent-account,omitempty"`
AgentHome string `json:"agent-account-home,omitempty"`
Adopted bool `json:"adopted,omitempty"`
Taken []string `json:"taken,omitempty"`
// Tunnel is the tunnel found on the node when it was adopted, kept as decided then: once the node takes it over it is
// no longer there to observe. PeerNames is, on the hub, the name a person gave a peer of that tunnel, by its public key.
Tunnel *RecordTunnel `json:"tunnel,omitempty"`
PeerNames map[string]string `json:"peer-names,omitempty"`
}
func (p Placement) check() error {
if err := checkName("node", p.Node); err != nil {
return err
}
seen := map[string]bool{}
for _, m := range p.Modules {
if err := checkName("module", m.Module); err != nil {
return err
}
if seen[m.Module] {
return fmt.Errorf("delivery: the placement of %s names %s twice", p.Node, m.Module)
}
seen[m.Module] = true
for port, at := range m.Ports {
if n, err := strconv.Atoi(port); err != nil || n <= 0 || n > 65535 {
return fmt.Errorf("delivery: %s's port %q on %s is not a port number", m.Module, port, p.Node)
}
if at <= 0 || at > 65535 {
return fmt.Errorf("delivery: %s's port %s on %s is published on %d, which is not a port number", m.Module, port, p.Node, at)
}
}
for provision, to := range m.Bindings {
if provision == "" || to.Node == "" || to.Module == "" {
return fmt.Errorf("delivery: %s on %s has a binding with no provision, node or module", m.Module, p.Node)
}
}
}
for provision, to := range p.Pins {
if provision == "" || to.Node == "" || to.Module == "" {
return fmt.Errorf("delivery: the placement of %s has a pin with no provision, node or module", p.Node)
}
}
for _, h := range p.Holds {
if h.Claim == "" || !seen[h.Module] {
return fmt.Errorf("delivery: the placement of %s has %s hold %q, which is not a module placed there or names no seat", p.Node, h.Module, h.Claim)
}
}
return nil
}
// EncodePlacement is p as JSON at Version; it refuses what DecodePlacement refuses.
func EncodePlacement(p Placement) ([]byte, error) {
if err := p.check(); err != nil {
return nil, err
}
p.V = Version
return json.Marshal(p)
}
// DecodePlacement reads a placement file or key value; it refuses an unknown version (ErrUnknownVersion), a node or
// module that is not a name, a module named twice, a port that is not a number, a binding or pin naming no provider,
// and a holding by a module not placed on the node.
func DecodePlacement(data []byte) (Placement, error) {
var p Placement
if err := decode(data, &p); err != nil {
return Placement{}, err
}
if err := p.check(); err != nil {
return Placement{}, err
}
return p, nil
}
// SamePlacement says whether two placements are the same value, so the indexer writes only when they differ. The
// revision is not compared: a merge that changes no placement writes nothing (ADR 0549 §2).
func SamePlacement(a, b Placement) bool {
a.Revision, b.Revision = "", ""
ja, errA := EncodePlacement(a)
jb, errB := EncodePlacement(b)
return errA == nil && errB == nil && string(ja) == string(jb)
}
// PlacementChange is what `placement-changed` says: the node, the catalogue revision, and its modules added, removed
// or changed against the placement before.
type PlacementChange struct {
Node string `json:"node"`
Revision string `json:"revision"`
Added []string `json:"added,omitempty"`
Removed []string `json:"removed,omitempty"`
Changed []string `json:"changed,omitempty"`
}
// ChangeOf is what changed from before to after on one node; before is the zero Placement for a node new to placement.
// A change of the node's pins or holdings counts as a change of every module placed on it both times, since the
// composition of each may read them.
func ChangeOf(before, after Placement) PlacementChange {
c := PlacementChange{Node: after.Node, Revision: after.Revision}
if c.Node == "" {
c.Node = before.Node
}
was := map[string]PlacedModule{}
for _, m := range before.Modules {
was[m.Module] = m
}
nodeWide := !jsonEqual(before.Pins, after.Pins) || !jsonEqual(before.Holds, after.Holds)
now := map[string]bool{}
for _, m := range after.Modules {
now[m.Module] = true
old, ok := was[m.Module]
switch {
case !ok:
c.Added = append(c.Added, m.Module)
case nodeWide || !jsonEqual(old, m):
c.Changed = append(c.Changed, m.Module)
}
}
for _, m := range before.Modules {
if !now[m.Module] {
c.Removed = append(c.Removed, m.Module)
}
}
sort.Strings(c.Added)
sort.Strings(c.Removed)
sort.Strings(c.Changed)
return c
}
func jsonEqual(a, b any) bool {
ja, _ := json.Marshal(a)
jb, _ := json.Marshal(b)
return string(ja) == string(jb)
}
// Assigned is the modules placed on the node, in the file's order (compose.Node's Assigned).
func (p Placement) Assigned() []string {
out := make([]string, 0, len(p.Modules))
for _, m := range p.Modules {
out = append(out, m.Module)
}
return out
}
// ChosenPins is the node's pins as the composition reads them (compose.Node's Pins); nil when it has none.
func (p Placement) ChosenPins() map[string]catalogue.Chosen {
if len(p.Pins) == 0 {
return nil
}
out := make(map[string]catalogue.Chosen, len(p.Pins))
for provision, to := range p.Pins {
out[provision] = to.Chosen()
}
return out
}
// Bound is, by consumer module and provision, where each binding sends it (compose.Node's Bound); nil when none.
func (p Placement) Bound() map[string]map[string]catalogue.Chosen {
var out map[string]map[string]catalogue.Chosen
for _, m := range p.Modules {
if len(m.Bindings) == 0 {
continue
}
if out == nil {
out = map[string]map[string]catalogue.Chosen{}
}
out[m.Module] = make(map[string]catalogue.Chosen, len(m.Bindings))
for provision, to := range m.Bindings {
out[m.Module][provision] = to.Chosen()
}
}
return out
}
// HostPorts is, by module and the port it declares, the host port it is published on (compose.Node's Ports); nil when
// none. The check has refused a port that is not a number, so none is dropped here.
func (p Placement) HostPorts() map[string]map[int]int {
var out map[string]map[int]int
for _, m := range p.Modules {
if len(m.Ports) == 0 {
continue
}
if out == nil {
out = map[string]map[int]int{}
}
out[m.Module] = make(map[int]int, len(m.Ports))
for port, at := range m.Ports {
n, _ := strconv.Atoi(port)
out[m.Module][n] = at
}
}
return out
}
// Layers is one module's settings layers on the node, as the composition reads them (compose.Node's Settings): the
// whole mesh's layer when it has one, then the node's, which carries the module's config values as mesh-config resolved
// them for this node (values, nil when none) beside its placement; a module with no layer and no value has none. This is
// the controller's SettingsFor for a module that takes its config values from mesh-config (issue 664). A module in the
// settings form carries its values in the file (MeshValues, Values), which then stand for mesh-config's.
func (m PlacedModule) Layers(node string, values map[string]any) []catalogue.Layer {
var out []catalogue.Layer
if m.HasMesh || len(m.MeshWide) > 0 || len(m.MeshValues) > 0 {
mesh := map[string]any{}
for k, v := range m.MeshValues {
mesh[k] = v
}
out = append(out, catalogue.Layer{From: catalogue.MeshWideLayer, Values: mesh, Placement: m.MeshWide})
}
if len(m.Values) > 0 {
values = make(map[string]any, len(m.Values))
for k, v := range m.Values {
values[k] = v
}
}
if m.HasLayer || len(m.Layer) > 0 || len(values) > 0 {
if values == nil {
values = map[string]any{}
}
out = append(out, catalogue.Layer{From: node, Values: values, Placement: m.Layer})
}
return out
}
// Held is the node's holdings as the composition reads them (compose.Input's Holdings rows for this node).
func (p Placement) Held() []catalogue.Held {
out := make([]catalogue.Held, 0, len(p.Holds))
for _, h := range p.Holds {
out = append(out, catalogue.Held{Claim: h.Claim, Scope: h.Scope, Node: p.Node, Module: h.Module, Site: h.Site, Kind: h.Kind})
}
return out
}
// CheckPlacements is the catalogue's own check of every placement file (ADR 0549 §1), over the placements by node, the
// modules the catalogue holds and the nodes enrolled. It returns every refusal, in node order, never only the first: a
// placement naming a node not enrolled or a module the catalogue does not hold; two modules of one node on the same host
// port; a pin, binding or holding naming a provider that is not placed where it says; a seat of scope mesh held twice, unless the seat is replicated (catalogue.Seat's Replicated, ADR 0223), which
// is held once per node;
// two nodes with one address on the private network; more than one hub.
// A host port a manifest declares without a number and a requirement left unbound are the composition's to find (its
// ForeseenPorts and Unbound), which the catalogue's composition check reads.
func CheckPlacements(placements map[string]Placement, held func(module string) bool, enrolled func(node string) bool) []string {
var refusals []string
nodes := make([]string, 0, len(placements))
for n := range placements {
nodes = append(nodes, n)
}
sort.Strings(nodes)
placed := func(p Provider) bool {
other, ok := placements[p.Node]
if !ok {
return false
}
for _, m := range other.Modules {
if m.Module == p.Module {
return true
}
}
return false
}
meshSeats := map[string]string{}
addresses := map[string]string{}
var hubs []string
for _, node := range nodes {
p := placements[node]
if p.Node != node {
refusals = append(refusals, fmt.Sprintf("the placement file of %s names the node %q", node, p.Node))
}
if a := strings.TrimSpace(p.Decided.Address); a != "" {
if other, twice := addresses[a]; twice {
refusals = append(refusals, fmt.Sprintf("%s and %s both have the address %s on the private network", other, node, a))
}
addresses[a] = node
}
if p.Decided.Hub {
hubs = append(hubs, node)
}
if !enrolled(node) {
refusals = append(refusals, fmt.Sprintf("%s is not an enrolled node", node))
}
hostPorts := map[int]string{}
for _, m := range p.Modules {
if !held(m.Module) {
refusals = append(refusals, fmt.Sprintf("%s: the catalogue holds no module %s", node, m.Module))
}
for _, port := range sortedKeys(m.Ports) {
at := m.Ports[port]
if other, clash := hostPorts[at]; clash && other != m.Module {
refusals = append(refusals, fmt.Sprintf("%s: %s and %s are both published on host port %d", node, other, m.Module, at))
}
hostPorts[at] = m.Module
}
for _, provision := range sortedKeys(m.Bindings) {
if to := m.Bindings[provision]; !placed(to) {
refusals = append(refusals, fmt.Sprintf("%s: %s's %s is bound to %s/%s, which is not placed there", node, m.Module, provision, to.Node, to.Module))
}
}
}
for _, provision := range sortedKeys(p.Pins) {
if to := p.Pins[provision]; !placed(to) {
refusals = append(refusals, fmt.Sprintf("%s: the pin of %s names %s/%s, which is not placed there", node, provision, to.Node, to.Module))
}
}
for _, h := range p.Holds {
if h.Scope != "mesh" {
continue
}
seat := h.Claim + "/" + h.Kind
if s, known := catalogue.SeatNamed(h.Claim); known && s.Replicated {
// A replicated mesh seat (ADR 0223) is held on several nodes at once, each answering the same thing;
// it is still held once on each node.
seat += "/" + node
}
if other, twice := meshSeats[seat]; twice {
refusals = append(refusals, fmt.Sprintf("the seat %s, held once for the mesh, is held on %s and %s", h.Claim, other, node))
}
meshSeats[seat] = node
}
}
if len(hubs) > 1 {
refusals = append(refusals, fmt.Sprintf("the private network has one hub, and %s are each placed as it", strings.Join(hubs, ", ")))
}
return refusals
}
func sortedKeys[V any](m map[string]V) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
+211
View File
@@ -0,0 +1,211 @@
package delivery
import (
"encoding/json"
"fmt"
"strings"
)
// Placement candidates (issue 1126; ADR 0549 §4: a placement change passes the composition check before any node sees
// it). A catalogue pull request that changes a placement file carries every placement file at its head on its pull
// message (Forge.Placements). The indexer writes, for each node whose file at the head differs from the trunk's, the
// key `<group>.<node>` of its state `placement-candidates`, and deletes it when the pull request merges or closes. It is
// a state of its own, since the readers of `placement` refuse any key that is not `placement.<node>`.
//
// The node it names composes in shadow with the candidate (its own sealed values, the boxes it already runs) and
// compares the result with what it runs; it applies nothing. Its answer rides in its own have key, `<node>.node`
// (HaveNode.Candidates, by group), which that node alone writes; the assigner's check of the group reads every placed
// node's answer at the head and fails on any difference, refusal or silence. Event-driven both ways: the candidate key's
// write is the node's event, the `<node>.node` change the check's.
const (
// PlacementCandidatesState is the indexer's state of placement candidates, and PlacementCandidatesBucket its bucket.
PlacementCandidatesState = "placement-candidates"
PlacementCandidatesBucket = IndexerModule + "_" + PlacementCandidatesState
// PlacementCandidatesHistory is the revisions the bucket keeps per key.
PlacementCandidatesHistory = 5
)
// PlacementCandidateKey is the key, in PlacementCandidatesBucket, of one node's candidate placement in one group:
// `<group>.<node>`, the group escaped as CandidatesKey escapes it, so a node watches `*.<node>` and a group's keys are
// `<group>.*`.
func PlacementCandidateKey(group, node string) (string, error) {
g, err := groupKey(group)
if err != nil {
return "", err
}
if err := checkName("node", node); err != nil {
return "", err
}
return g + "." + node, nil
}
// ParsePlacementCandidateKey is the inverse of PlacementCandidateKey; it refuses any other key, naming it.
func ParsePlacementCandidateKey(key string) (group, node string, err error) {
g, node, ok := strings.Cut(key, ".")
if !ok || checkName("node", node) != nil {
return "", "", fmt.Errorf("delivery: %q is not a placement candidate key, <group>.<node>", key)
}
if group, err = parseGroupKey(g); err != nil {
return "", "", fmt.Errorf("delivery: %q is not a placement candidate key: %w", key, err)
}
return group, node, nil
}
// PlacementCandidate is the value of a candidate key: the group, the catalogue pull request and the head its placement
// files were read at, and the node's placement at that head; nil Placement says the head removes the node's file.
type PlacementCandidate struct {
V int `json:"v"`
Group string `json:"group"`
Pull Head `json:"pull"`
Placement *Placement `json:"placement,omitempty"`
}
func (c PlacementCandidate) check() error {
if c.Group == "" {
return fmt.Errorf("delivery: a placement candidate names no group")
}
if err := checkRepository(c.Pull.Repository); err != nil {
return err
}
if c.Pull.Number <= 0 || c.Pull.Head == "" {
return fmt.Errorf("delivery: a placement candidate needs its pull request's number and head")
}
if c.Placement != nil {
return c.Placement.check()
}
return nil
}
// EncodePlacementCandidate is c as JSON at Version; it refuses what DecodePlacementCandidate refuses.
func EncodePlacementCandidate(c PlacementCandidate) ([]byte, error) {
if err := c.check(); err != nil {
return nil, err
}
c.V = Version
if c.Placement != nil {
p := *c.Placement
p.V = Version
c.Placement = &p
}
return json.Marshal(c)
}
// DecodePlacementCandidate reads a candidate; it refuses an unknown version, no group, a pull request without number or
// head, and a placement DecodePlacement refuses.
func DecodePlacementCandidate(data []byte) (PlacementCandidate, error) {
var c PlacementCandidate
if err := decode(data, &c); err != nil {
return PlacementCandidate{}, err
}
if err := c.check(); err != nil {
return PlacementCandidate{}, err
}
return c, nil
}
// The verdicts of a node's answer to a candidate.
const (
// CandidateSame: every composition the node composes with the candidate is the one it runs, and it runs no other.
CandidateSame = "same"
// CandidateDiffers: some composition differs (Differs), is added by the candidate (Added) or removed by it (Removed);
// whether the change intends it is the assigner's to judge (ADR 0559 §5).
CandidateDiffers = "differs"
// CandidateRefused: a value is missing for a module the node runs (Refusals), or the node cannot compose with the
// candidate at all (Refused says why: an unbound requirement, a manifest it cannot read).
CandidateRefused = "refused"
)
// CandidateAnswer is a node's answer to its group's candidate keys, in HaveNode.Candidates (ADR 0559 §4): the head
// and the revision of the node's own key in the group it answered (0 when the group has no key of its own for it: its
// file is unchanged and it answers as placed at the head or on the trunk, judged by Head alone), its verdict, the modules composed the same, those that differ with both digests, those
// the candidate adds (naming the values not yet issued for each, which is no refusal: a module's credential and boxes
// are made only once it is placed, ADR 0549 §5) or removes, and each refusal. The node's own composition is named
// "node" (HaveNodeWord).
type CandidateAnswer struct {
Head string `json:"head"`
Revision uint64 `json:"revision"`
Verdict string `json:"verdict"`
Same []string `json:"same,omitempty"`
Differs []ModuleDiffers `json:"differs,omitempty"`
Added []ModuleAdded `json:"added,omitempty"`
Removed []string `json:"removed,omitempty"`
Refusals []ModuleRefused `json:"refusals,omitempty"`
// Refused is why the node could not compose with the candidate at all; Refusals is then empty.
Refused string `json:"refused,omitempty"`
}
// ModuleDiffers is one module the node runs and the candidate still places whose composition is not the one it runs:
// both digests (Composed with the candidate, Running what runs), the first resource that differs by its id, or empty
// when every resource is equal, and why in words.
type ModuleDiffers struct {
Module string `json:"module"`
Composed string `json:"composed,omitempty"`
Running string `json:"running,omitempty"`
Resource string `json:"resource,omitempty"`
Why string `json:"why"`
}
// ModuleAdded is one module the candidate places on the node and the node does not run: its composition's digest when
// it composed, and the values not yet issued for it (its own secrets, its grants, a host port), each in the words the
// composition stood in for it with.
type ModuleAdded struct {
Module string `json:"module"`
Composed string `json:"composed,omitempty"`
Unissued []string `json:"unissued,omitempty"`
}
// ModuleRefused is a value missing for a module the node runs, in the words the composition stood in for it with.
type ModuleRefused struct {
Module string `json:"module"`
Why string `json:"why"`
}
// VerdictOf is the verdict of an answer's lists: refused on any refusal, differs on any difference, addition or
// removal, same otherwise.
func VerdictOf(a CandidateAnswer) string {
switch {
case a.Refused != "" || len(a.Refusals) > 0:
return CandidateRefused
case len(a.Differs)+len(a.Added)+len(a.Removed) > 0:
return CandidateDiffers
}
return CandidateSame
}
func (a CandidateAnswer) check() error {
if a.Head == "" {
return fmt.Errorf("delivery: a candidate answer names no head")
}
switch a.Verdict {
case CandidateSame, CandidateDiffers, CandidateRefused:
default:
return fmt.Errorf("delivery: the candidate verdict %q is not same, differs or refused", a.Verdict)
}
if got := VerdictOf(a); got != a.Verdict {
return fmt.Errorf("delivery: a candidate answered %s names what makes it %s", a.Verdict, got)
}
if a.Refused != "" && len(a.Same)+len(a.Differs)+len(a.Added)+len(a.Removed)+len(a.Refusals) > 0 {
return fmt.Errorf("delivery: a candidate the node could not compose names modules")
}
for _, d := range a.Differs {
if d.Module == "" || d.Why == "" {
return fmt.Errorf("delivery: a candidate difference names no module or no why")
}
}
for _, d := range a.Added {
if d.Module == "" {
return fmt.Errorf("delivery: a module the candidate adds is not named")
}
}
for _, d := range a.Refusals {
if d.Module == "" || d.Why == "" {
return fmt.Errorf("delivery: a candidate refusal names no module or no why")
}
}
for _, m := range a.Removed {
if m == "" {
return fmt.Errorf("delivery: a module the candidate removes is not named")
}
}
return nil
}
+295
View File
@@ -0,0 +1,295 @@
package delivery
import (
"encoding/json"
"errors"
"reflect"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// ADR 0549 §1, §2: the placement key and file, the value's refusals, its reading as the composition reads a node, what
// a change says, and the catalogue's check of every file.
func TestPlacement(t *testing.T) {
t.Run("key and file", func(t *testing.T) {
key, err := PlacementKey("g14")
if err != nil || key != "placement.g14" {
t.Fatalf("PlacementKey = %q, %v", key, err)
}
if n, err := ParsePlacementKey(key); err != nil || n != "g14" {
t.Fatalf("ParsePlacementKey = %q, %v", n, err)
}
for _, bad := range []string{"want.g14.x", "placement.", "placement.a.b"} {
if _, err := ParsePlacementKey(bad); err == nil {
t.Errorf("ParsePlacementKey(%q) took it", bad)
}
}
if _, err := PlacementKey("a.b"); err == nil {
t.Error("PlacementKey took a dotted node")
}
if PlacementBucket != "mesh-delivery-indexer_placement" || PlacementFile("ace") != "placement/ace.json" {
t.Fatalf("bucket %q, file %q", PlacementBucket, PlacementFile("ace"))
}
for p, want := range map[string]string{"placement/ace.json": "ace", "./placement/ace.json": "ace",
"placement/x/ace.json": "", "modules/ace.json": "", "placement/ace.yaml": ""} {
if n, ok := PlacementFileNode(p); n != want || ok != (want != "") {
t.Errorf("PlacementFileNode(%q) = %q, %v", p, n, ok)
}
}
})
g14 := Placement{Node: "g14", Revision: "abc",
Decided: NodeDecided{Endpoint: "g14.example:51820", Site: "home", Address: "10.77.0.2", Account: "op", Taken: []string{"zsh"}},
Modules: []PlacedModule{
{Module: "zsh"},
{Module: "app", Ports: map[string]int{"8080": 18080}, Bindings: map[string]Provider{"postgres": {Node: "novox", Module: "postgres"}}},
},
Pins: map[string]Provider{"shell": {Node: "g14", Module: "zsh"}},
Holds: []Holding{{Claim: "node-shell", Scope: "node", Module: "zsh"}},
}
novox := Placement{Node: "novox", Decided: NodeDecided{Hub: true, Address: "10.77.0.1"}, Modules: []PlacedModule{{Module: "postgres", Ports: map[string]int{"5432": 5432}}},
Holds: []Holding{{Claim: "mesh-store", Scope: "mesh", Module: "postgres"}}}
t.Run("value", func(t *testing.T) {
raw, err := EncodePlacement(g14)
if err != nil {
t.Fatal(err)
}
back, err := DecodePlacement(raw)
if err != nil || back.V != Version || !SamePlacement(back, g14) {
t.Fatalf("round trip: %+v, %v", back, err)
}
moved := g14
moved.Revision = "def"
if !SamePlacement(g14, moved) {
t.Error("a new revision alone counts as a change")
}
if _, err := DecodePlacement([]byte(`{"v":9,"node":"g14","modules":[]}`)); !errors.Is(err, ErrUnknownVersion) {
t.Errorf("version 9: %v", err)
}
for name, bad := range map[string]string{
"module twice": `{"v":1,"node":"g14","modules":[{"module":"a"},{"module":"a"}]}`,
"port not number": `{"v":1,"node":"g14","modules":[{"module":"a","ports":{"http":80}}]}`,
"host port zero": `{"v":1,"node":"g14","modules":[{"module":"a","ports":{"80":0}}]}`,
"binding no node": `{"v":1,"node":"g14","modules":[{"module":"a","bindings":{"pg":{"module":"pg"}}}]}`,
"pin no module": `{"v":1,"node":"g14","modules":[],"pins":{"pg":{"node":"x"}}}`,
"hold not placed": `{"v":1,"node":"g14","modules":[],"holds":[{"claim":"s","module":"a"}]}`,
"node not a name": `{"v":1,"node":"g.14","modules":[]}`,
"value not shaped": `{"v":1,"node":"g14","modules":{}}`,
} {
if _, err := DecodePlacement([]byte(bad)); err == nil {
t.Errorf("%s: taken", name)
}
}
})
t.Run("as the composition reads it", func(t *testing.T) {
if got := g14.Assigned(); !reflect.DeepEqual(got, []string{"zsh", "app"}) {
t.Errorf("Assigned = %v, want the file's order", got)
}
if got := g14.ChosenPins(); !reflect.DeepEqual(got, map[string]catalogue.Chosen{"shell": {Node: "g14", Module: "zsh"}}) {
t.Errorf("ChosenPins = %v", got)
}
if got := g14.Bound(); !reflect.DeepEqual(got, map[string]map[string]catalogue.Chosen{"app": {"postgres": {Node: "novox", Module: "postgres"}}}) {
t.Errorf("Bound = %v", got)
}
if got := g14.HostPorts(); !reflect.DeepEqual(got, map[string]map[int]int{"app": {8080: 18080}}) {
t.Errorf("HostPorts = %v", got)
}
if got := g14.Held(); !reflect.DeepEqual(got, []catalogue.Held{{Claim: "node-shell", Scope: "node", Node: "g14", Module: "zsh"}}) {
t.Errorf("Held = %v", got)
}
layered := PlacedModule{Module: "app", HasMesh: true, Layer: map[string]any{"expose": map[string]any{"80": "mesh"}}}
want := []catalogue.Layer{
{From: catalogue.MeshWideLayer, Values: map[string]any{}},
{From: "g14", Values: map[string]any{"mode": "x"}, Placement: map[string]any{"expose": map[string]any{"80": "mesh"}}},
}
if got := layered.Layers("g14", map[string]any{"mode": "x"}); !reflect.DeepEqual(got, want) {
t.Errorf("Layers = %#v, want %#v", got, want)
}
if got := (PlacedModule{Module: "a"}).Layers("g14", map[string]any{"k": 1}); len(got) != 1 || got[0].From != "g14" {
t.Errorf("values alone make the node's layer: %#v", got)
}
if got := (PlacedModule{Module: "a", HasLayer: true}).Layers("g14", nil); len(got) != 1 || got[0].Values == nil {
t.Errorf("an empty layer of the node's own is kept, with empty values: %#v", got)
}
if got := (PlacedModule{Module: "a"}).Layers("g14", nil); got != nil {
t.Errorf("no layer and no value is no layer: %#v", got)
}
empty := Placement{Node: "x"}
if empty.ChosenPins() != nil || empty.Bound() != nil || empty.HostPorts() != nil || len(empty.Held()) != 0 {
t.Error("an empty placement reads as something")
}
})
t.Run("what a change says", func(t *testing.T) {
after := g14
after.Revision = "def"
after.Modules = []PlacedModule{{Module: "app", Ports: map[string]int{"8080": 18081}}, {Module: "new"}}
got := ChangeOf(g14, after)
want := PlacementChange{Node: "g14", Revision: "def", Added: []string{"new"}, Removed: []string{"zsh"}, Changed: []string{"app"}}
if !reflect.DeepEqual(got, want) {
t.Errorf("ChangeOf = %+v, want %+v", got, want)
}
pinned := g14
pinned.Pins = nil
if got := ChangeOf(g14, pinned); !reflect.DeepEqual(got.Changed, []string{"app", "zsh"}) {
t.Errorf("a pin change changes every module on the node: %+v", got)
}
if got := ChangeOf(Placement{}, novox); !reflect.DeepEqual(got.Added, []string{"postgres"}) || got.Node != "novox" {
t.Errorf("a new node: %+v", got)
}
})
t.Run("the catalogue's check", func(t *testing.T) {
held := func(m string) bool { return m != "ghost" }
enrolled := func(n string) bool { return n != "stranger" }
if r := CheckPlacements(map[string]Placement{"g14": g14, "novox": novox}, held, enrolled); len(r) != 0 {
t.Fatalf("a sound placement is refused: %v", r)
}
// A replicated mesh seat (ADR 0223) is held on two nodes; a seat that is not replicated is refused there.
for seat, refused := range map[string]bool{"mesh-dns-resolver": false, "mesh-broker": true} {
a := Placement{Node: "ace", Modules: []PlacedModule{{Module: "dns"}}, Holds: []Holding{{Claim: seat, Scope: "mesh", Module: "dns"}}}
b := a
b.Node = "novox"
r := CheckPlacements(map[string]Placement{"ace": a, "novox": b}, held, enrolled)
if (len(r) != 0) != refused {
t.Errorf("%s on two nodes: refusals %v, want refused %v", seat, r, refused)
}
}
clash := g14
clash.Modules = append([]PlacedModule{{Module: "ghost", Ports: map[string]int{"80": 18080}}}, g14.Modules...)
clash.Modules[2] = PlacedModule{Module: "app", Ports: map[string]int{"8080": 18080}, Bindings: map[string]Provider{"postgres": {Node: "ace", Module: "postgres"}}}
clash.Pins = map[string]Provider{"shell": {Node: "g14", Module: "bash"}}
twice := novox
twice.Node = "stranger"
twice.Decided.Address = "10.77.0.2"
r := CheckPlacements(map[string]Placement{"g14": clash, "novox": novox, "stranger": twice}, held, enrolled)
for _, want := range []string{"holds no module ghost", "ghost and app are both published on host port 18080",
"bound to ace/postgres", "pin of shell names g14/bash", "stranger is not an enrolled node",
"mesh-store, held once for the mesh, is held on novox and stranger",
"g14 and stranger both have the address 10.77.0.2", "novox, stranger are each placed as it"} {
found := false
for _, line := range r {
found = found || strings.Contains(line, want)
}
if !found {
t.Errorf("no refusal says %q; got %v", want, r)
}
}
})
// Issue 1126: a catalogue pull request's placement, one candidate key per node, and each node's answer in its have.
t.Run("candidates", func(t *testing.T) {
key, err := PlacementCandidateKey("feat/1.2", "g14")
if err != nil || key != "feat/1=2e2.g14" {
t.Fatalf("PlacementCandidateKey = %q, %v", key, err)
}
if g, n, err := ParsePlacementCandidateKey(key); err != nil || g != "feat/1.2" || n != "g14" {
t.Fatalf("ParsePlacementCandidateKey = %q, %q, %v", g, n, err)
}
for _, bad := range []string{"g14", ".g14", "feat.g.14", "feat.", "fe=t.g14"} {
if _, _, err := ParsePlacementCandidateKey(bad); err == nil {
t.Errorf("ParsePlacementCandidateKey(%q) took it", bad)
}
}
if PlacementCandidatesBucket != "mesh-delivery-indexer_placement-candidates" {
t.Errorf("bucket %q", PlacementCandidatesBucket)
}
pull := Head{Repository: CatalogueRepository, Number: 7, Head: "h1"}
for name, c := range map[string]PlacementCandidate{
"placed": {Group: "feat/x", Pull: pull, Placement: &g14},
"removed": {Group: "feat/x", Pull: pull},
} {
raw, err := EncodePlacementCandidate(c)
if err != nil {
t.Fatalf("%s: %v", name, err)
}
back, err := DecodePlacementCandidate(raw)
if err != nil || back.Group != c.Group || back.Pull != pull || (back.Placement == nil) != (c.Placement == nil) ||
(c.Placement != nil && !SamePlacement(*back.Placement, g14)) {
t.Errorf("%s: round trip %+v, %v", name, back, err)
}
}
for name, c := range map[string]PlacementCandidate{
"no group": {Pull: pull},
"no head": {Group: "g", Pull: Head{Repository: CatalogueRepository, Number: 7}},
"bad place": {Group: "g", Pull: pull, Placement: &Placement{Node: "a.b"}},
"no repo form": {Group: "g", Pull: Head{Repository: "mesh-catalog", Number: 7, Head: "h"}},
} {
if _, err := EncodePlacementCandidate(c); err == nil {
t.Errorf("%s: taken", name)
}
}
answers := map[string]CandidateAnswer{
"same": {Head: "h1", Revision: 3, Verdict: CandidateSame, Same: []string{"app", "node"}},
"differs": {Head: "h1", Revision: 3, Verdict: CandidateDiffers, Differs: []ModuleDiffers{{Module: "app", Resource: "file:/etc/app", Why: "the resource differs"}}},
"refused": {Head: "h1", Revision: 3, Verdict: CandidateRefused, Refused: "unbound postgres"},
"added": {Head: "h1", Verdict: CandidateDiffers, Same: []string{"node"}, Added: []ModuleAdded{{Module: "app", Unissued: []string{"app/token"}}}, Removed: []string{"old"}},
"a value": {Head: "h1", Verdict: CandidateRefused, Same: []string{"node"}, Refusals: []ModuleRefused{{Module: "db", Why: "stands in for db/password"}}},
}
n := HaveNode{Composed: "sha256:" + strings.Repeat("a", 64), Sealed: []string{"box"}, Candidates: answers}
raw, err := EncodeHaveNode(n)
if err != nil {
t.Fatal(err)
}
back, err := DecodeHaveNode(raw)
if err != nil || !SameHaveNode(back, HaveNode{V: Version, Composed: n.Composed, Sealed: n.Sealed, Candidates: answers}) {
t.Fatalf("a node's answers changed on the way: %+v, %v", back, err)
}
if SameHaveNode(back, HaveNode{V: Version, Composed: n.Composed, Sealed: n.Sealed}) {
t.Error("a node's have without its answers counts as the same")
}
// CompareComposed: the bus's user list is left out on both sides; anything else that differs is named.
file := func(id, content string) json.RawMessage {
return json.RawMessage(`{"id":"` + id + `","type":"file","module":"nats","path":"/etc/` + id + `","mode":"0600","content":"` + content + `"}`)
}
fp := "src2:" + strings.Repeat("a", 64)
comp := func(rs ...json.RawMessage) Composed {
return Composed{Node: "novox", Module: "nats", FP: fp, Resources: rs}
}
for name, tc := range map[string]struct {
composed, running Composed
resource, why string
}{
"user list alone": {comp(file("conf", "a"), file("nats.bus-users", "stand-in")), comp(file("conf", "a"), file("nats.bus-users", "real")), "", ""},
"a resource": {comp(file("conf", "a")), comp(file("conf", "b")), "conf", "the resource differs"},
"one side": {comp(file("conf", "a"), file("more", "x")), comp(file("conf", "a")), "more", "composed, and not in what runs"},
"order": {comp(file("conf", "a"), file("more", "x")), comp(file("more", "x"), file("conf", "a")), "", "the same resources in another order"},
} {
c := CompareComposed(map[string]Composed{"novox.nats." + fp: tc.composed}, []Composed{tc.running})
if tc.why == "" {
if len(c.Differs) != 0 || len(c.Same) != 1 {
t.Errorf("%s: %+v", name, c)
}
} else if len(c.Differs) != 1 || c.Differs[0].Resource != tc.resource || c.Differs[0].Why != tc.why ||
c.Differs[0].Composed == "" || c.Differs[0].Running == "" || c.Differs[0].Composed == c.Differs[0].Running {
t.Errorf("%s: differs %+v, want %s: %s with both digests", name, c.Differs, tc.resource, tc.why)
}
}
if c := CompareComposed(nil, []Composed{{Node: "novox", Resources: []json.RawMessage{}}}); len(c.Removed) != 1 || c.Removed[0] != "node" {
t.Errorf("the node's own composition run and not composed is removed: %+v", c)
}
if c := CompareComposed(map[string]Composed{"k": comp(file("conf", "a"))}, nil); len(c.Added) != 1 || c.Added[0].Module != "nats" || c.Added[0].Composed == "" {
t.Errorf("a composition not run is added: %+v", c)
}
for name, a := range map[string]CandidateAnswer{
"no head": {Verdict: CandidateSame},
"same with a diff": {Head: "h", Verdict: CandidateSame, Differs: answers["differs"].Differs},
"differs with none": {Head: "h", Verdict: CandidateDiffers},
"refused no reason": {Head: "h", Verdict: CandidateRefused},
"unknown verdict": {Head: "h", Verdict: "maybe"},
"diff with no module": {Head: "h", Verdict: CandidateDiffers, Differs: []ModuleDiffers{{Why: "x"}}},
"same with an added": {Head: "h", Verdict: CandidateSame, Added: []ModuleAdded{{Module: "a"}}},
"differs refusing": {Head: "h", Verdict: CandidateDiffers, Removed: []string{"a"}, Refusals: []ModuleRefused{{Module: "b", Why: "x"}}},
"refused and modules": {Head: "h", Verdict: CandidateRefused, Refused: "x", Same: []string{"a"}},
} {
if _, err := EncodeHaveNode(HaveNode{Candidates: map[string]CandidateAnswer{"g": a}}); err == nil {
t.Errorf("%s: taken", name)
}
}
})
}
+167
View File
@@ -0,0 +1,167 @@
package delivery
import (
"encoding/json"
"fmt"
"maps"
"slices"
"strings"
"time"
)
// How far a repository reached (novox/hq ADR 0556, issue 1280). The indexer says, per repository, the last merge it took in
// (TakenBucket); the delivery view keeps, per repository, how far each of its modules reached on the nodes that want it, and
// derives from it the repository's reached head (ReachBucket): the newest head of the repository at which every wanted
// module runs, on every node, a build at or after its last change. A merge at or before the reached head is on the
// machines. Nothing records a merge as delivered (ADR 0429 §1): the gate and the tracker derive it from this state.
// TakenBucket is the indexer's state `taken`: one key per repository (RepositoryKey), the last merge it took in.
const TakenBucket = IndexerModule + "_taken"
// ReachBucket is the delivery view's state `reach`: one key per repository (RepositoryKey), how far its modules reached.
const ReachBucket = ViewModule + "_reach"
// RepositoryKey is a repository's key in TakenBucket and ReachBucket: its forge name `owner/name` (RepositoryName), in
// lower case as the forge compares it, escaped as a key part (EscapeName). An empty spelling, or one with no owner, is
// refused.
func RepositoryKey(repository string) (string, error) {
name := strings.ToLower(RepositoryName(repository))
if owner, repo, ok := strings.Cut(name, "/"); !ok || owner == "" || repo == "" {
return "", fmt.Errorf("delivery: %q names no repository as owner/name", repository)
}
return EscapeName(name), nil
}
// Taken is the value of a taken key: the repository, the merge commit the indexer last took in (applied its message
// whole), and when.
type Taken struct {
V int `json:"v"`
Repository string `json:"repository"`
Merge string `json:"merge"`
At time.Time `json:"at"`
}
// EncodeTaken is t as JSON at Version; a value with no repository or merge is refused.
func EncodeTaken(t Taken) ([]byte, error) {
if t.Repository == "" || t.Merge == "" {
return nil, fmt.Errorf("delivery: a taken value names its repository and merge")
}
t.V = Version
return json.Marshal(t)
}
// DecodeTaken reads a taken value; it refuses an unknown version (ErrUnknownVersion) and one with no merge.
func DecodeTaken(data []byte) (Taken, error) {
var t Taken
if err := decode(data, &t); err != nil {
return t, err
}
if t.Repository == "" || t.Merge == "" {
return t, fmt.Errorf("delivery: a taken value names no repository or merge")
}
return t, nil
}
// Reach is the value of a reach key: how far the modules of one repository reached on the nodes. Indexed is the last
// merge the indexer took in for it (Taken.Merge), empty while it took in none. A repository the indexer holds no module
// of (a library) has no Modules. Reached is the reached head: the newest head of the repository in the source's history,
// not past Indexed, at which every module with a node that wants it has its floor at or after its last change at or before
// that head; ReachedSrc is the source revision that named it. Holds says, in words, what keeps the next head of the
// repository from being reached (a module on a node, the indexer's lag); empty when the repository's newest head is reached.
type Reach struct {
V int `json:"v"`
Repository string `json:"repository"`
Indexed string `json:"indexed,omitempty"`
Modules map[string]ModuleReach `json:"modules,omitempty"`
Reached string `json:"reached,omitempty"`
ReachedSrc uint64 `json:"reached_src,omitempty"`
Holds []string `json:"holds,omitempty"`
}
// ModuleReach is how far one module reached. FP is its fingerprint at the indexer now; Nodes how many nodes want it
// (a want not removed); Changes its fingerprints the view knows of, each with the merge commit it first appeared at,
// newest first. Floor is the oldest first-appearance commit among the fingerprints its nodes' haves run, a running have
// counting whether its health check passes or not; it is empty when a node that wants the module has no running have, or
// runs a fingerprint not among Changes. Waiting names, in words, every node where the module is not settled (its have
// different from its want, not running, or not healthy), with or without a floor: health is said here and never moves the
// floor (ADR 0556 clause 2). A module whose floor is empty while a node wants it holds no head of the repository back: a
// reader takes a merge among its Changes as not yet on the machines, with these words. A module no node wants has neither.
type ModuleReach struct {
FP string `json:"fp,omitempty"`
Nodes int `json:"nodes"`
Changes []Change `json:"changes,omitempty"`
Floor string `json:"floor,omitempty"`
Waiting string `json:"waiting,omitempty"`
}
// Change is one fingerprint of a module, the merge commit it first appeared at (Module.Commit), and the source revision
// that named that commit as the repository's head (Module.Src, or the source's history), 0 when not known.
type Change struct {
Commit string `json:"commit"`
FP string `json:"fp"`
Src uint64 `json:"src,omitempty"`
}
// MostChanges is how many changes of a module a reach value keeps: as many revisions as the indexer's modules bucket keeps.
const MostChanges = ModulesHistory
func (r Reach) check() error {
if r.Repository == "" {
return fmt.Errorf("delivery: a reach value names no repository")
}
if (r.Reached == "") != (r.ReachedSrc == 0) {
return fmt.Errorf("delivery: the reach of %s names its reached head and its source revision together", r.Repository)
}
for name, m := range r.Modules {
if name == "" {
return fmt.Errorf("delivery: a reach value names a module with no name")
}
if m.Nodes < 0 {
return fmt.Errorf("delivery: the reach of %s counts %d nodes", name, m.Nodes)
}
if len(m.Changes) > MostChanges {
return fmt.Errorf("delivery: the reach of %s keeps %d changes, more than %d", name, len(m.Changes), MostChanges)
}
for _, c := range m.Changes {
if c.Commit == "" || c.FP == "" {
return fmt.Errorf("delivery: a change of %s names no commit or fingerprint", name)
}
}
}
return nil
}
// EncodeReach is r as JSON at Version; a value that does not check is refused.
func EncodeReach(r Reach) ([]byte, error) {
if err := r.check(); err != nil {
return nil, err
}
r.V = Version
return json.Marshal(r)
}
// DecodeReach reads a reach value; it refuses an unknown version (ErrUnknownVersion) and one that does not check.
func DecodeReach(data []byte) (Reach, error) {
var r Reach
if err := decode(data, &r); err != nil {
return r, err
}
return r, r.check()
}
// SameReach says whether two reach values are the same, so the view writes only when they differ.
func SameReach(a, b Reach) bool {
if a.Repository != b.Repository || a.Indexed != b.Indexed || len(a.Modules) != len(b.Modules) ||
a.Reached != b.Reached || a.ReachedSrc != b.ReachedSrc || !slices.Equal(a.Holds, b.Holds) {
return false
}
for _, name := range slices.Sorted(maps.Keys(a.Modules)) {
x, ok := b.Modules[name]
y := a.Modules[name]
if !ok || x.FP != y.FP || x.Nodes != y.Nodes || x.Floor != y.Floor || x.Waiting != y.Waiting ||
!slices.Equal(x.Changes, y.Changes) {
return false
}
}
return true
}
+20
View File
@@ -537,6 +537,13 @@ type Forge struct {
Act string `json:"act,omitempty"`
Words string `json:"words,omitempty"`
Open []OpenPull `json:"open,omitempty"`
// Placements are, on a merged message of the catalogue repository, every placement file of the merge commit, the
// bytes exact, by node (PlacementFile; novox/hq ADR 0549 §2), and PlacementsRead says they were read, so a catalogue
// whose last placement file was removed is told from a message that read none (older writers, another repository).
// On a pull message of the catalogue (CatalogueRepository) they are every placement file at the pull request's head,
// read whole or not at all, which the indexer writes as placement candidates (PlacementCandidate).
Placements map[string]string `json:"placements,omitempty"`
PlacementsRead bool `json:"placements-read,omitempty"`
}
func (f Forge) check() error {
@@ -549,6 +556,19 @@ func (f Forge) check() error {
if len(f.Open) > 0 {
return fmt.Errorf("delivery: only an open message names open pull requests")
}
// A pull message of the catalogue carries its head's placement files, so its change of placement is composed and
// answered by the nodes it places before it merges (placement candidates, issue 1126; ADR 0549 §4).
if (len(f.Placements) > 0 || f.PlacementsRead) && f.Kind != ForgeMerged && (f.Kind != ForgePull || f.Repository != CatalogueRepository) {
return fmt.Errorf("delivery: only a merged message, or a pull message of the catalogue, carries placement files")
}
if len(f.Placements) > 0 && !f.PlacementsRead {
return fmt.Errorf("delivery: a message with placement files must say it read them (placements-read)")
}
for node := range f.Placements {
if err := checkName("node", node); err != nil {
return fmt.Errorf("delivery: a placement file names no node: %w", err)
}
}
if err := checkRepository(f.Repository); err != nil {
return err
}
+76 -1
View File
@@ -6,6 +6,7 @@ import (
"reflect"
"strings"
"testing"
"time"
)
func TestSourceKeys(t *testing.T) { // 1200/AC1
@@ -196,7 +197,23 @@ func TestSourceValuesRoundTrip(t *testing.T) { // 1200/AC3
t.Errorf("forge %s round trip: %+v %v", f.Kind, got, err)
}
}
// 1280: the taken and reach values (ADR 0556).
tk := Taken{Repository: "novox/mesh-sdk", Merge: "cccc", At: time.Date(2026, 10, 26, 20, 0, 0, 0, time.UTC)}
data, err = EncodeTaken(tk)
check("taken", data, err)
tk.V = Version
if got, err := DecodeTaken(data); err != nil || got != tk {
t.Errorf("taken round trip: %+v %v", got, err)
}
r := sampleReach()
data, err = EncodeReach(r)
check("reach", data, err)
if got, err := DecodeReach(data); err != nil || !SameReach(got, r) {
t.Errorf("reach round trip: %+v %v", got, err)
}
for name, dec := range map[string]func([]byte) error{
"taken": func(b []byte) error { _, err := DecodeTaken(b); return err },
"reach": func(b []byte) error { _, err := DecodeReach(b); return err },
"source": func(b []byte) error { _, err := DecodeSource(b); return err },
"candidate": func(b []byte) error { _, err := DecodeCandidate(b); return err },
"module": func(b []byte) error { _, err := DecodeModule(b); return err },
@@ -282,6 +299,38 @@ func TestSourceValuesRefused(t *testing.T) { // 1200/AC4
wantErr("unknown kind", err, "closed")
_, err = EncodeForge(Forge{Kind: ForgePull, Repository: "nope", Number: 1, Head: "h", Action: "opened"})
wantErr("bad repository", err, "nope")
// 1280: taken and reach values that do not hold (ADR 0556).
_, err = EncodeTaken(Taken{Repository: "a/b"})
wantErr("taken without merge", err, "merge")
r := sampleReach()
r.Repository = ""
_, err = EncodeReach(r)
wantErr("reach without repository", err, "repository")
r = sampleReach()
m := r.Modules["gitea"]
m.Changes = append(m.Changes, Change{Commit: "dddd"})
r.Modules["gitea"] = m
_, err = EncodeReach(r)
wantErr("reach with a change naming no fingerprint", err, "gitea")
r = sampleReach()
m = r.Modules["gitea"]
m.Changes = make([]Change, MostChanges+1)
r.Modules["gitea"] = m
_, err = EncodeReach(r)
wantErr("reach keeping too many changes", err, "more than")
r = sampleReach()
r.ReachedSrc = 0
_, err = EncodeReach(r)
wantErr("reach with a head and no revision", err, "reached head")
}
// sampleReach is a reach value of one repository with a module at its floor and a library-like empty one.
func sampleReach() Reach {
return Reach{V: Version, Repository: "novox/mesh-catalog", Indexed: "cccc", Reached: "cccc", ReachedSrc: 9, Modules: map[string]ModuleReach{
"gitea": {FP: "src2:a", Nodes: 1, Floor: "cccc", Waiting: "gitea on ace runs src2:a and its health check failed", Changes: []Change{{Commit: "cccc", FP: "src2:a", Src: 9}, {Commit: "bbbb", FP: "src2:b", Src: 7}}},
"zsh": {FP: "src2:z", Changes: []Change{{Commit: "aaaa", FP: "src2:z"}}},
}}
}
func TestSourceSame(t *testing.T) { // 1200/AC5
@@ -314,14 +363,40 @@ func TestSourceSame(t *testing.T) { // 1200/AC5
if SameForge(f, g) {
t.Error("different forge messages are the same")
}
if !SameReach(sampleReach(), sampleReach()) {
t.Error("equal reach values differ")
}
for name, change := range map[string]func(*Reach){
"indexed": func(r *Reach) { r.Indexed = "dddd" },
"floor": func(r *Reach) { m := r.Modules["gitea"]; m.Floor = "bbbb"; r.Modules["gitea"] = m },
"changes": func(r *Reach) { m := r.Modules["zsh"]; m.Changes = nil; r.Modules["zsh"] = m },
"modules": func(r *Reach) { delete(r.Modules, "zsh") },
"reached": func(r *Reach) { r.Reached, r.ReachedSrc = "bbbb", 7 },
"holds": func(r *Reach) { r.Holds = []string{"gitea on ace does not yet run src2:a"} },
} {
r := sampleReach()
change(&r)
if SameReach(r, sampleReach()) {
t.Errorf("reach values differing in %s are the same", name)
}
}
}
func TestSourceBuckets(t *testing.T) {
if SourceBucket != "mesh-delivery-gate_source" || CandidatesBucket != "mesh-delivery-indexer_candidates" ||
ModulesBucket != "mesh-delivery-indexer_modules" || SourceMainKey != "main" ||
SourceHistory != 64 || CandidatesHistory != 10 || ModulesHistory != 20 {
SourceHistory != 64 || CandidatesHistory != 10 || ModulesHistory != 20 ||
TakenBucket != "mesh-delivery-indexer_taken" || ReachBucket != "mesh-delivery-view_reach" {
t.Error("a bucket or history differs from the decided layout")
}
// 1280: a repository keys taken and reach by its forge name, in lower case, from either spelling (ADR 0556).
for in, want := range map[string]string{"novox/Mesh-SDK": "novox/mesh-sdk", "http://h:3000/novox/mesh-sdk.git": "novox/mesh-sdk",
"a/b.c": "a/b=2ec", "": "", "nope": "", "/x": ""} {
got, err := RepositoryKey(in)
if (want == "") != (err != nil) || got != want {
t.Errorf("RepositoryKey(%q) = %q, %v; want %q", in, got, err, want)
}
}
}
func TestForgeDirs(t *testing.T) {
+57
View File
@@ -693,3 +693,60 @@ func DecodePause(data []byte) (Pause, error) {
}
return p, nil
}
// CapacityKey is the key, in a build module's PausesState, where the module on node says the capacity it judges by
// now and why: `capacity.<node>` (issue 1806's follow-up). The pauses state has no age, so the value stands until the
// module writes another; a reader such as the delivery view's build_usage says it in place of the configured capacity.
func CapacityKey(node string) (string, error) {
if err := checkName("node", node); err != nil {
return "", err
}
return "capacity." + node, nil
}
// Capacity is a build seat's capacity on one node as its holder judges by it: the configured capacity (the module's
// setting `capacity`), bounded by the cores its node's build slice gives (its CPU quota, read live) over the cores one
// step needs (the module's setting `cores-per-step`). Effective is min(Configured, ByCPU) when a quota is known, else
// Configured; one step at least while Configured is above zero, so a node whose quota is under one step's need still
// builds, slowly.
type Capacity struct {
V int `json:"v"`
Effective int `json:"effective"`
Configured int `json:"configured"`
// ConfiguredFrom says where Configured was read: mesh-config's value as the config watch handed it, or the settings
// file.
ConfiguredFrom string `json:"configured-from,omitempty"`
// QuotaCores is the build slice's CPU quota in processors (cpu.max: 1200% is 12), 0 when none is known; QuotaFrom
// says where it was read (the cgroup, or the share the composer gave) or why none is known.
QuotaCores float64 `json:"quota-cores,omitempty"`
QuotaFrom string `json:"quota-from"`
// CoresPerStep is what one step of this module needs; ByCPU is floor(QuotaCores / CoresPerStep), at least 1, and 0
// when no quota is known.
CoresPerStep float64 `json:"cores-per-step"`
ByCPU int `json:"by-cpu,omitempty"`
// BuildHours is the node's build window (HH:MM-HH:MM, its local time and zone), absent when it builds at any hour;
// OutsideHours says the time judged fell outside it, so Effective is 0 and the next step waits for the window to open.
BuildHours string `json:"build-hours,omitempty"`
OutsideHours bool `json:"outside-hours,omitempty"`
// BuildHoursFault is why the node's build-hours value cannot be read, naming the value: it then builds at any hour.
BuildHoursFault string `json:"build-hours-fault,omitempty"`
// Why is the capacity in words: what bounds it.
Why string `json:"why"`
At time.Time `json:"at,omitzero"`
}
// EncodeCapacity is c as JSON at Version.
func EncodeCapacity(c Capacity) ([]byte, error) {
c.V = Version
c.At = c.At.UTC()
return json.Marshal(c)
}
// DecodeCapacity reads a capacity value; it refuses an unknown version (ErrUnknownVersion).
func DecodeCapacity(data []byte) (Capacity, error) {
var c Capacity
if err := decode(data, &c); err != nil {
return Capacity{}, err
}
return c, nil
}

Some files were not shown because too many files have changed in this diff Show More