Compare commits

..
Author SHA1 Message Date
jschoubben 38dbe1d3a3 Let a node agent hear the creation of the seat's placement and candidates, so it binds them with no timer (ADR 0567 §9, issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-sdk@d7c790d2422c
mesh/delivery-group group feat/1126-grants-first merged: every member is merged
mesh/check pass 38dbe1d3a388 on 98f2f2406921 | unit 1909 run, 715 kept, 3 skipped | integration 1931 run, 813 kept, 3 skipped | over bound: unit (slow…
2026-10-28 18:47:13 +01:00
jschoubben a2dbcf670b Bound a proposal's answers and ask by pulse keys whose expiry the holder hears, and name a hand-back proposal, so no timer waits (ADR 0567 §4, §7, issue 1126) 2026-10-28 18:43:39 +01:00
jschoubben e987a8280b Let a module's config values be proposed before the lease write: only a layer's places and accesses wait (ADR 0567 §13, issue 1126) 2026-10-28 18:42:05 +01:00
jschoubben 45f5f76811 Keep the placement ask among the assigner's asks: its Approve's level, not its kind, needs the proof (ADR 0567 §5, issue 1126) 2026-10-28 18:30:00 +01:00
jschoubben 4b4a173d84 Let withdraw mark a proposal for its holder to end, since the holder's process alone says the seat's events (ADR 0567 §2, issue 1126) 2026-10-28 18:21:59 +01:00
jschoubben 0282229fd9 Name the placement ask and a proposal's again and warrant, so the holder asks through the teller as its other asks (ADR 0567 §5, §10, issue 1126) 2026-10-28 18:20:27 +01:00
jschoubben 5a2a7eebfd Hold that a check's node names reach the digest tool too, so a kept verdict misses when they change (ADR 0567 §12, issue 1126) 2026-10-28 18:17:08 +01:00
jschoubben 004bbc2ce2 Carry every node's name on a check ask to the names check, so no repository lists them (ADR 0567 §12, issue 1126) 2026-10-28 18:13:57 +01:00
jschoubben fed64292f0 Read build capacity as an integer and build hours as one window, default 0: no repository names a node (ADR 0567 §12, issue 1126) 2026-10-28 18:13:18 +01:00
jschoubben 7a82384e21 Grant a node's agent the seat's placement and candidates, so it composes from the seat's state (ADR 0567 §3, §9, issue 1126) 2026-10-28 18:12:09 +01:00
jschoubben 7c565f2aa5 Move Intended and JudgeAnswers into mesh-sdk for a proposal: a difference is shown, only a refusal or silence fails (ADR 0567 §4, issue 1126) 2026-10-28 18:11:18 +01:00
jschoubben edc6f269f3 Name the seat's states, candidates by proposal and the proposal itself, so readers and the holder share one contract (ADR 0567 §1-§6, issue 1126) 2026-10-28 18:09:38 +01:00
jschoubben 902e3c9e96 Refuse placement files on a forge message and delete placement-seed: no repository holds placement (ADR 0567 §11, issue 1126) 2026-10-28 18:09:38 +01:00
jschoubben c16e6213e1 Define the seat mesh-placement with its seven verbs, so placement has one contract whoever holds it (ADR 0567, issue 1126) 2026-10-28 18:05:16 +01:00
mesh-tools 98f2f24069 Merge pull request 'A node holds the one module it cannot compose and applies the rest, as its shadow already answers (group fix/1126-apply-culprit, issue 1126)' (#118) from fix/1126-apply-culprit into main 2026-10-28 16:34:46 +00:00
jschoubben 0de3006415 A module refused alone is held, composed as what runs of it, on the node that applies as in its answer: one rule for both paths (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-sdk@98f2f2406921
mesh/delivery-group group fix/1126-apply-culprit merged: every member is merged
mesh/check pass 0de30064156a on 7aedd28f01dc | unit 2667 run, 590 kept, 36 skipped | integration 2676 run, 633 kept, 29 skipped | over bound: unit (sl…
The applying path (frombus.Compose) composed with no culprit, so one module's
refusal came back as the whole node's: novox's apply would have refused all its
modules for de-spiegel's unset route names. composeEach is now the one rule
Compose and Answer both follow:
- a module whose stub changes the refusal is held (compose.Node.Held); in the
  applying path a stand-in for a module placed here holds that module; only a
  refusal of no single module refuses the node;
- a held module is composed from what runs of it (compose.Carried of its own
  composition in Running) and carried in the node's own wherever that runs it,
  folded or leaving the fold, so the node's own apply never removes it (issue
  1945 by a new path) and what the others read of it is what runs;
- none of a held module's own composition is in what is applied;
- a module whose composition differs when the held ones are composed as stubs
  is held too; the node's own differing refuses the node;
- since a held module is composed from its new manifest, what reads that
  manifest is held with it: the providers of what it contributes to (the
  proxy its routes) and, when it listens, the filter's holder.

Quick fix weighed: a culprit loop in Compose alone. Thorough, chosen: one
shared function, since the kind met twice (seed-8 shadow, now the apply), and
review rounds 1 and 2 found its leaks.
2026-10-28 17:22:35 +01:00
mesh-tools 7aedd28f01 Merge pull request 'A module's own refusal refuses that module, not the node; contributions say unset values in config words; a folded module owns its manifest's paths (issue 1126, group fix/1126-seed-8)' (#117) from fix/1126-seed-8 into main 2026-10-28 13:27:05 +00:00
jschoubben f387581b6b A module's own refusal refuses that module, not the node: a stub whose contributions still fail the same is tried bare (issue 1126, seed-8: novox refused the whole candidate for de-spiegel's unset route names)
mesh/delivery merged: source.main moved to novox/mesh-sdk@7aedd28f01dc
mesh/delivery-group group fix/1126-seed-8 merged: every member is merged
mesh/check pass f387581b6bfb on 3d021445f743 | unit 3018 run, 234 kept, 36 skipped | integration 3038 run, 266 kept, 29 skipped | over bound: unit (sl…
2026-10-28 14:07:59 +01:00
jschoubben eda82b1598 A folded module owns its manifest's paths, as the controller tells node-files them, not every path it runs at (issue 1126, seed-8: f1-circuits, snake and txt-game shifted node-tools' owned paths on ace) 2026-10-28 14:07:59 +01:00
jschoubben 06742d0448 A contribution's unset required value is refused in its module's own form's words, as a file's is (issue 1126, seed-8: de-spiegel in the config form was told ${setting:…} and settings set) 2026-10-28 14:07:59 +01:00
mesh-tools 3d021445f7 Merge pull request 'A build module may say its node is not eligible, so asks go to a node that can build (group fix/ace-tun)' (#116) from fix/ace-tun into main 2026-10-28 13:07:40 +00:00
jschoubben 1618837ea9 A build seat's puller takes nothing while its module finds its node cannot build, and says why
mesh/delivery merged: source.main moved to novox/mesh-sdk@3d021445f743
mesh/delivery-group group fix/ace-tun merged: every member is merged
mesh/check pass 1618837ea9fe on 6097e4f3e040 | carried from mesh-catalog 25f615456423 | unit 2293 run, 3984 kept, 16 skipped | integration 2331 run, 4…
A node whose module proves a need of the machine missing (ace's /dev/net/tun, 2026-10-28) pulled
image asks and failed each one; judged ineligible, its capacity is 0 with the reason, as build hours
are, so the asks go to a node that can build them (issue 1126 note).
2026-10-28 12:50:16 +01:00
mesh-tools 6097e4f3e0 Merge pull request 'A file's secrets are sealed values entry by entry, masked and named per secret (issue 1126; group fix/1126-seed-same-3)' (#115) from fix/1126-seed-same-3 into main 2026-10-28 11:40:35 +00:00
jschoubben 8520a4c099 Only a file's secrets are split entry by entry when masked and named: a vault box under vault-sealed is one masked word and one path, judged whole by what it opens to (issue 1126, C1; review of seed-same-3, round 1)
mesh/delivery merged: source.main moved to novox/mesh-sdk@6097e4f3e040
mesh/delivery-group group fix/1126-seed-same-3 merged: every member is merged
mesh/check pass 8520a4c09960 on 79eb58efe4b0 | carried from mesh-catalog 774fcdfa93ad | unit 2188 run, 1064 kept, 36 skipped | integration 2197 run, 1…
2026-10-28 12:22:37 +01:00
jschoubben 70de9429e5 A file's secrets are sealed values entry by entry: each a slot of its own, kept where it opens to the value that runs, masked and named per secret (issue 1126, C1; the seed-7 probe: about 45 files on ace and novox differed by their secrets' bytes alone) 2026-10-28 12:02:39 +01:00
mesh-tools 79eb58efe4 Merge pull request 'Compose as the controller does, so a node's shadow can answer same (issue 1126, group fix/1126-seed-same)' (#114) from fix/1126-seed-same into main 2026-10-28 09:46:24 +00:00
jschoubben 3028c7ec4e A drifted own secret is read from a resource id as a node runs it, its module's name first (only-office.needs-jwt), so the node agent finds what to hand the vault (issue 1126, cause C4; review round 2)
mesh/delivery merged: source.main moved to novox/mesh-sdk@79eb58efe4b0
mesh/delivery-group group fix/1126-seed-same merged: every member is merged
mesh/check failed: integration: github.com/novox/mesh-host/cmd/mesh-host.TestNatsANodeThePlacementNamesComposesAndAppliesItsOwn: the node did not comp…
delivery.OwnNeedName reads <module>.needs-<name> and the bare needs-<name>; OwnValuesDiffer and the node agent use it.
The fixtures carry the id shape captured from novox's shadow answer.
2026-10-28 10:00:29 +01:00
jschoubben 1062b33d32 A node's shadow answer counts the values that drifted from the vault's, and names the own secrets among them (issue 1126, cause C4)
delivery.OwnValuesDiffer, ValuesDiffer and WithValuesDiffer read the shown bodies masked as another value; a candidate
answer carries values_differ. The node agent hands the vault those own secrets' running values through move-in's
replace_running, so no seat verb and no bus grant changes here.
2026-10-28 09:49:03 +01:00
jschoubben 16b8d8a1ee Compose as the controller does, so a node's shadow can answer same (issue 1126, C1 C2 C3 C5 C6, T1)
Seals are reused and compared by slot (resource, field), never across resources;
one rule says which resolved config values are laid; a seat path that steps back
is owned by the seat it steps back for, whatever the seat order; a folded module
of an old-form build is composed from what runs of it; the operator export's file
is carried as it runs; a differing resource names its differing JSON paths, found
before its bodies are cut, and an answer is fitted to the node's key.
2026-10-28 09:36:36 +01:00
mesh-tools be77f66563 Merge pull request 'frombus: blame only the modules a refusal names, and say which providers another node cannot run (issue 1126, group fix/1126-shadow-bodies)' (#113) from fix/1126-shadow-bodies into main 2026-10-28 07:46:23 +00:00
jschoubben 362a3f1378 A refusal for want of a provider names the modules another node cannot run in this composition, and a refusal naming no placed module is answered at once (issue 1126, probe #532)
mesh/delivery merged: source.main moved to novox/mesh-sdk@be77f6656326
mesh/delivery-group group fix/1126-shadow-bodies merged: every member is merged
mesh/check pass 362a3f1378aa on 03da552cb331 | unit 2860 run, 1141 kept, 36 skipped | integration 2887 run, 1241 kept, 29 skipped | over bound: unit (…
ace and g14 answered 'nothing in this mesh provides artifact-store … assign
distribution to a node' while novox runs it: a module another node cannot run
in this composition (catalogue Resolution.Unhostable) offers nothing, silently.
The refusal now names each such module and what it lacks, and each node left
out. And the culprit search composed the whole mesh once per placed module for a
refusal no stub changes (ace's answer took 7m46s): only the modules the refusal
names are tried, at most 8.
2026-10-28 01:39:07 +01:00
mesh-tools 03da552cb3 Merge pull request 'Shadow answers show both bodies of each difference (secrets masked); a folded module is judged inside; the bus membership is carried (issue 1126, group fix/1126-shadow-shows)' (#112) from fix/1126-shadow-shows into main 2026-10-28 00:21:47 +00:00
jschoubben 4e0c57771e Bound the bodies an answer shows, naming the rest (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-sdk@03da552cb331
mesh/delivery-group group fix/1126-shadow-shows merged: every member is merged
mesh/check pass 4e0c57771e8c on 47eb1009921d | unit 2980 run, 270 kept, 36 skipped | integration 2999 run, 302 kept, 29 skipped | over bound: unit (sl…
2026-10-28 01:09:02 +01:00
jschoubben b31c4580aa A candidate answer shows every differing resource, masked; a module folded in the node's own is judged there; the node's own carries its bus membership as it runs it (issue 1126, seed #531)
novox's answer to seed #531 named 15 resources by id alone, so why each differed
was guessed. ModuleDiffers.Bodies keeps every differing resource with both
bodies, each value under a key a secret could be in masked (the same, another
seal of the same value, another value). The five old-form modules read
'removed' because the agent also lists them by module from the declaration kept
before the switch; they are judged inside the node's own. And the node's own
lacked bus-membership, which only the controller composes from its store: the
membership the node runs is carried, until the grants part hands it sealed.
2026-10-28 01:08:49 +01:00
mesh-tools 47eb100992 Merge pull request 'frombus: a stub keeps its module's directories; compose returns nothing once its run ended (issue 1126, group fix/1126-switch-safe)' (#111) from fix/1126-switch-safe into main 2026-10-27 23:52:19 +00:00
jschoubben 7359d187db A stub keeps its module's directories, so the places its own secrets and data name still resolve (issue 1126: novox refused nats's ${dir:mesh-state})
mesh/delivery merged: source.main moved to novox/mesh-sdk@47eb1009921d
mesh/delivery-group group fix/1126-switch-safe merged: every member is merged
mesh/check pass 7359d187db8d on 47b1dcf08f85 | unit 2992 run, 1008 kept, 36 skipped | integration 3025 run, 1102 kept, 29 skipped | over bound: unit (…
2026-10-28 00:21:45 +01:00
mesh-tools 47b1dcf08f Merge pull request 'Shadow answers name every refusal, carry old-form builds as they run, and compare a deleted placement (issue 1126, group fix/1126-shadow-all)' (#110) from fix/1126-shadow-all into main 2026-10-27 22:44:08 +00:00
jschoubben e32d777530 A deleting head's answer compares the controller's compositions with what runs, and is refused only without returning; fold the shadow's tests into TestInput (issue 1126, review of the shadow, round 1)
mesh/delivery merged: source.main moved to novox/mesh-sdk@47b1dcf08f85
mesh/delivery-group group fix/1126-shadow-all merged: every member is merged
mesh/check checking e32d77753024 on d48365aed010 | novox/mesh-sdk#110 in group fix/1126-shadow-all
2026-10-27 23:32:31 +01:00
jschoubben 472d5f3144 A node's have says whose compositions it applies, and its answer to a head deleting its placement says it returns to the controller's (issue 1126, the seed's revert)
The check of the seed's revert (mesh-catalog #526) failed on 'would stop
running node': the shadow answered a deleted placement as everything removed,
while the node keeps what runs. HaveNode.Composes and ComposesWhy say whether a
node applies its own compositions or the controller's; CandidateAnswer.Returns
answers a deleted placement, verdict same, naming nothing composed.
2026-10-27 23:22:54 +01:00
jschoubben 2fd5ffdd62 A node's answer names every module it cannot compose, a folded module of an old-form build is left to the node as it runs, and no store reached says why (issue 1126, the seed's shadow)
The seed's shadow answers named the first refusal only (ace: baserow's archive,
novox: de-spiegel), so each round showed one more. A build that cannot be read or
taken in, and a module the composition refuses, are now each composed as a stub
and named. A module folded into the node's own composition whose manifest names a
build (de-spiegel, nats, photos, systemd-resolved: haves naming a build-... id)
runs a build only the controller holds: it is composed as a stub and its
resources are carried as the node runs them.
2026-10-27 23:17:05 +01:00
mesh-tools d48365aed0 Merge pull request 'frombus: a node composing itself takes its modules from their builds and reads what providers elsewhere serve (issue 1126, group fix/1126-node-creds)' (#109) from fix/1126-node-creds into main 2026-10-27 20:32:30 +00:00
jschoubben 7a8e9f9706 Wait for a done key whose artifact names no reference, as the controller's take-in does, and hold the fold guard with a row (issue 1126, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-sdk@d48365aed010
mesh/delivery-group group fix/1126-node-creds merged: every member is merged
mesh/check checking 7a8e9f970648 on d48365aed010 | novox/mesh-sdk#109 in group fix/1126-node-creds
2026-10-27 21:12:13 +01:00
jschoubben d7a758894a Leave a module the node runs inside its own composition to its manifest, as the fold composes it, so taking builds in moves no folded module (issue 1126) 2026-10-27 21:11:20 +01:00
jschoubben 04de3e5da2 Compose a node's own modules from their builds and read what providers elsewhere serve, so a self-composing node hands moved modules and settles served settings as the controller does (issue 1126)
frombus composed every module from the indexer's raw manifest: no moved module
was handed, so the node stood in for own secrets the vault leaves to the
provisioner (the seed's refusals on all four nodes), and a provider elsewhere
had no config values, so mailu's served domain refused shanks.

- takein: the controller's take-in of a done key (artifacts, launchers, the
  templates archive's texts, Resolve) as one SDK rule; catalogue.Launchers holds
  the launcher rule buildstep now delegates to.
- frombus: each module placed on the composing node, and each kept at another
  build (Scope.KeptAt, a fingerprint), composes from its built manifest; another
  node's modules give the keys their served values fill (Reader.Served).
- buscompose: Declared.Served and Principal.ServedReads grant every node the read
  of exactly those keys of mesh-config_resolved.
- compose.ArtifactStoreAddress: the store as the node's composition routes to it.
2026-10-27 21:11:20 +01:00
mesh-tools 9f92233a28 Merge pull request 'process: a module's own bus connection, subjects and durable consumer in one helper (issue 1242)' (#108) from fix/1242-bus-helper into main 2026-10-27 19:49:19 +00:00
jschoubben 8ad9372b19 Name another module's own-stream subject in the SDK too (issue 1242)
mesh/delivery merged: source.main moved to novox/mesh-sdk@9f92233a2890
mesh/delivery-group group fix/1242-bus-helper merged: every member is merged
mesh/check checking 8ad9372b1952 on a262c7dbe0ff | novox/mesh-sdk#108 in group fix/1242-bus-helper
2026-10-27 20:38:20 +01:00
jschoubben 3236cbad43 Give a module's own credential its dial, subjects and durable consumer in mesh-sdk, so a part no longer hand-writes them (issue 1242) 2026-10-27 20:36:14 +01:00
mesh-tools a262c7dbe0 Merge pull request 'A node folds a placed module with no new-form build into its own composition, as the controller does, so the tunnel and the bus are not dropped when nodes compose themselves (issue 1126)' (#107) from fix/1126-fold into main 2026-10-27 08:09:03 +00:00
jschoubben 05b6783103 Fold a module no new-form build names into the node's own composition, as the controller does, so a self-composing node keeps the tunnel and the bus (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-sdk@a262c7dbe0ff
mesh/delivery-group group fix/1126-fold merged: every member is merged
mesh/check checking 05b678310366 on 56de5ba84010 | novox/mesh-sdk#107 in group fix/1126-fold
mesh-wireguard (built by nobody) and every module a node runs inside its own
composition (Node.InOwn, from frombus.InOwn of what it runs) were cut out as
skipped: every node's shadow answered differs to the placement seed, and a node
composing itself after it would drop their resources. compositions now folds
them in place, module taken off, filled paths with them, as mesh-controller's
compositionsOf does; a want naming a new-form build still moves one out.
2026-10-27 08:50:26 +01:00
mesh-tools 56de5ba840 Merge pull request 'A placement change is answered by the nodes it places: the candidate contract, composing a given placement, and replicated seats in the placement check (issue 1126, ADR 0559)' (#106) from feat/1126-l2b into main 2026-10-27 07:21:03 +00:00
mesh-tools 43eef46c41 Merge pull request 'A build step names the process that runs it, so a restarted build module clears what a gone process left running (issue 1890)' (#105) from fix/1890-step-owner into main 2026-10-27 07:19:36 +00:00
jschoubben cadd7b1b2f Ask the vault in a candidate answer only for credentials of modules the node runs, and refuse a stand-in of a running module, so a placement that adds a module is answered as added (issue 1126, ADR 0559 §4)
mesh/delivery merged: source.main moved to novox/mesh-sdk@56de5ba84010
mesh/delivery-group group feat/1126-l2b merged: every member is merged
mesh/check being carried to cadd7b1b2f8f on 43eef46c4121 | from b0cbc4e6aca8, mesh-catalog 64530a8dcb08 | novox/mesh-sdk#106 in group feat/1126-l2b
2026-10-27 02:43:10 +01:00
jschoubben 062c29c3a2 Judge a node's own composition in its candidate answer only where it runs one, as its agent applies it only on a want (issue 1126) 2026-10-27 02:27:04 +01:00
jschoubben a37d2622ed Name the words of a composition failed only for an answer not heard, and what an answer's revision is, so the assigner and the checker judge a late answer alike (issue 1126) 2026-10-27 02:22:59 +01:00
jschoubben ba398e2460 Answer a placement candidate as ADR 0559 has it: added and removed modules, per-module refusals, both digests of a difference, and a node's read of every candidate key (issue 1126) 2026-10-27 02:19:27 +01:00
jschoubben 82f10c3bc9 Let frombus and the composer compose with a given placement and compare a node's compositions with what it runs, so a placement candidate can be answered and checked (issue 1126) 2026-10-27 02:12:11 +01:00
jschoubben 7e99c354ac Give a catalogue pull request's placement a candidate state and each node an answer in its own have, so a placement change is checked by the nodes it places before it merges (issue 1126) 2026-10-27 02:07:47 +01:00
jschoubben 8e81d87610 Let a replicated mesh seat be held on several nodes in CheckPlacements, so the seed of today's placement is not refused (issue 1126) 2026-10-27 02:04:01 +01:00
jochen db1007ffdd Delete a gone process's ask key instead of ending it, so its redelivery still answers the asker (issue 1890)
mesh/delivery merged: source.main moved to novox/mesh-sdk@43eef46c4121
mesh/delivery-group group fix/1890-step-owner merged: every member is merged
mesh/check pass db1007ffdda1 on b0cbc4e6aca8 | unit 6014 run, 233 kept, 16 skipped | integration 6067 run, 265 kept, 24 skipped | over bound: unit, in…
ADR 0525 §6 has the stream redeliver an ask whose node stopped, and the
asker takes the first end it reads: ending the key at a restart failed
checks a redelivery would still run. The sweep now deletes such a key,
guarded by the revision read, so the asker keeps waiting and the stale
key leaves the counts of what runs; a hand-started step of a gone owner
is still ended at once. A missing boot id is said once at start.
2026-10-27 01:29:48 +01:00
jochen 981a35b598 End a step left by a previous process at once: every step names its owner (issue 1890)
A delivery that replaced mesh-build-go left a check ask's key running, and
the new process's start sweep spared it as an ask younger than the step
bound, so it held a build slot until a hand end (issue 1890, note 21).
Each step now records its owner (boot id, pid, process start); the sweep
ends a running step of another owner at once and says its owner, while a
key with no owner keeps the earlier rule. The conformance kit gains the
left-running case every build module runs through its own start sweep.
2026-10-27 01:19:29 +01:00
mesh-tools b0cbc4e6ac Merge pull request 'A build step's failure contract: go -json and TAP parsers and a conformance kit every build module runs, so a verdict names the failing test (issue 1901)' (#104) from fix/1901-failure-contract into main 2026-10-26 23:44:13 +00:00
jschoubben a2759ec92e Merge main into fix/1901-failure-contract: the capacity tests stay one function, so the SDK's test count does not grow (issue 1901)
mesh/delivery merged: source.main moved to novox/mesh-sdk@b0cbc4e6aca8
mesh/delivery-group group fix/1901-failure-contract merged: every member is merged
mesh/check checking a2759ec92ec9 on f178323b5790 | novox/mesh-sdk#104 in group fix/1901-failure-contract
2026-10-27 00:39:10 +01:00
mesh-tools f178323b57 Merge pull request 'The mesh-vault seat promises own, the verb L2 gave the vault (issue 1126)' (#103) from fix/1126-vault-seat-own into main 2026-10-26 22:00:44 +00:00
mesh-tools 684acb922d Merge pull request 'Delivery records are true at the source: the layouts for what a repository's merges reached (issue 1280)' (#102) from fix/delivery-records-true into main 2026-10-26 21:48:43 +00:00
jschoubben e78e881ecb The vault's seat promises own, which L2's vault serves and its claim names, so the controller's catalogue check passes (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-sdk@f178323b5790
mesh/delivery-group group fix/1126-vault-seat-own merged: every member is merged
mesh/check failed: unit: github.com/novox/mesh-controller/cmd/mesh-controller.TestModuleCheckPassesTheCatalogue: the catalogue does not pass its own c…
mesh-catalog's mesh-vault claims its seat serving own (a machine's node-engine asks its own machine's secrets and
pair credentials sealed to its stated key), and buscompose already grants that call to node principals
(AsksItsOwnSecrets, VaultOwn); only the seat's verb list lacked it, so a claim naming it was refused.
2026-10-26 22:38:58 +01:00
jschoubben 68d65e6bbf Merge remote-tracking branch 'origin/main' into fix/delivery-records-true
mesh/delivery merged: source.main moved to novox/mesh-sdk@684acb922d99
mesh/delivery-group group fix/delivery-records-true merged: every member is merged
mesh/check checking 68d65e6bbf61 on 823d52051f03 | novox/mesh-sdk#102 in group fix/delivery-records-true
2026-10-26 22:19:13 +01:00
mesh-tools 823d52051f Merge pull request 'Nodes compose: placement, node records and one composer input for three callers (issue 1126)' (#98) from feat/1126-nodes-compose into main 2026-10-26 21:11:05 +00:00
jschoubben b4310dfb8e Merge remote-tracking branch 'origin/main' into fix/delivery-records-true 2026-10-26 22:04:42 +01:00
jschoubben 9796444201 Merge remote-tracking branch 'origin/main' into feat/1126-nodes-compose
mesh/delivery merged: source.main moved to novox/mesh-sdk@823d52051f03
mesh/delivery-group group feat/1126-nodes-compose merged: every member is merged
mesh/check checking 979644420117 on 7187beaf042b | novox/mesh-sdk#98 in group feat/1126-nodes-compose
# Conflicts:
#	test/standing.test.ts
2026-10-26 22:03:26 +01:00
mesh-tools 7187beaf04 Merge pull request 'The provider loop's tests wait on what they mean, not on the clock (issue 1901 note 27)' (#101) from fix/standing-test-races into main 2026-10-26 20:58:44 +00:00
jschoubben 5f45175ed0 Merge remote-tracking branch 'origin/main' into feat/1126-nodes-compose 2026-10-26 21:56:47 +01:00
jschoubben 18e541c1ba The standing test allows 10 ms between the loop's clock and the test's stamp of an announcement
mesh/delivery merged: source.main moved to novox/mesh-sdk@7187beaf042b
mesh/check pass 18e541c1badc on 7187beaf042b | carried from 2cbe21937153 | a compiled program is not refused: mesh-build-node reported no programs | u…
2026-10-26 21:55:46 +01:00
jschoubben cf930388f9 The standing test's runs last 0.6 of failingAfterMs of clock as well as two attempts, so a success that does not reset a run is caught; the addition test holds a thousand stable passes, so 'first pass' is proved (review of the standing test races) 2026-10-26 21:49:03 +01:00
jschoubben 36ae0c691b The standing and retirement tests wait on what they mean, not on a slept fraction of a bound: announcements are timed from the clock at the moment they are said, short runs are counted in attempts, an addition is proved by a hold it must not wait for (the second timing race of one kind in a day; issue 1126) 2026-10-26 21:44:06 +01:00
mesh-tools 2cbe219371 Merge pull request 'A build seat's capacity follows its CPU share and its build hours; asks say their phases (issues 1890, 1806)' (#100) from fix/capacity-follows-cpu-share into main 2026-10-26 20:33:28 +00:00
jschoubben 3a095a66be The standing test keeps c failing alone after its ask is dropped, so its only recovery is its retirement: clearing every failure at once raced the loop's read of the new list under load (issue 1126; the hidden failure of try-1793042418286545971, issue 1901) 2026-10-26 21:33:20 +01:00
jschoubben 805b4d9a35 Merge main into fix/capacity-follows-cpu-share: the build modules' binary refusal (sdk #99) beside the capacity work
mesh/delivery merged: source.main moved to novox/mesh-sdk@2cbe21937153
mesh/delivery-group group fix/capacity-follows-cpu-share merged: every member is merged
mesh/check checking 805b4d9a3536 on dae8e288d2e1 | novox/mesh-sdk#100 in group fix/capacity-follows-cpu-share
2026-10-26 21:26:58 +01:00
jschoubben d00dac140b A build module takes asks only within its node's build hours
mesh/delivery-group group fix/capacity-follows-cpu-share rejected: composed together: the checker's verdict at the members' heads (63694fe9222d): unit: ..test
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: unit: ..test: ✔ a transient empty list for four passes retires nothing; five stable passes retire, data kept (418.253369ms) | d00da…
The operator, 2026-10-26 21:18: g14 builds only from 18:00 to 06:00. A build-hours setting names a
window per node in its local time (crossing midnight allowed, absent = any hour); outside it the
capacity judged by is 0, running steps finish, and the puller wakes on one timer at the next edge.
The change is said in the log and in capacity.<node> with the window and when it opens.
2026-10-26 21:21:37 +01:00
jschoubben e44cbaf57f A module with no floor holds no head back; a merge among its changes reads not yet on the machines (review of issue 1280, ADR 0556 clause 2) 2026-10-26 21:12:48 +01:00
jschoubben 1f70c4c0aa A settle ask's group begins with a colon, which no branch name holds, so a branch named settle/... still merges on its Approve (review of issue 1280) 2026-10-26 21:12:31 +01:00
jschoubben 5ecda46124 A settle ask's group and the act its options bind, shared by the gate and the teller (ADR 0556 clause 5) 2026-10-26 21:12:31 +01:00
jschoubben aa8e9afc7c A module's floor counts every running have, healthy or not; health stays in its words (ADR 0556 clause 2, insight) 2026-10-26 21:12:31 +01:00
jschoubben 54a8afac73 The view's reach carries the reached head and each change's source revision; nothing records a merge as delivered (ADR 0556 v8) 2026-10-26 21:12:31 +01:00
jschoubben 43de3c937d Layouts of the indexer's taken and the view's reach, and the delivery seat's settle verb, so whether a merge is on the machines can be derived (issue 1280, ADR 0556) 2026-10-26 21:12:31 +01:00
jschoubben 03dce12378 The standing test keeps c failing alone after its ask is dropped, so its only recovery is its retirement: clearing every failure at once raced the loop's read of the new list under load (issue 1126; the hidden failure of try-1793042418286545971, issue 1901)
mesh/delivery-group group feat/1126-nodes-compose rejected: composed together: the checker's verdict at the members' heads (0d06ff386dce): ace
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: ace: novox/mesh-sdk#98 does not merge into the trunk: files in conflict: test/standing.test.ts | 03dce1237803 on 7187beaf042b | uni…
2026-10-26 20:33:23 +01:00
mesh-tools dae8e288d2 Merge pull request 'A change that adds a compiled binary is refused, in any repository (issue 1577)' (#99) from fix/1577-checker-refuses-binaries into main 2026-10-26 19:32:25 +00:00
jschoubben dd9d9d5b5f Compare haves with SameHave in the round-trip tests, now that a have carries a list (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking dd9d9d5b5f01 on dae8e288d2e1 | novox/mesh-sdk#98 in group feat/1126-nodes-compose
2026-10-26 20:20:12 +01:00
jschoubben 2c0ab3a592 A have carries the values a self-composing node's running composition holds sealed to its key, so its composer reuses them after a restart (review of L2, round 2; issue 1126, ADR 0547 §3)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed (4 tests): unit: go.go vet: # git.novox.be/novox/mesh-sdk/go/delivery [git.novox.be/novox/mesh-sdk/go/delivery.test] | 2c0ab3a59288 …
2026-10-26 20:19:54 +01:00
jschoubben 47a66e3c0d Take a crashed file's cause only from what the run wrote after the last test, give a subtest's panic to that subtest, and keep the SDK's test count (issue 1901, review round 1) 2026-10-26 20:17:42 +01:00
jschoubben 3968849b1b Review of L2: the export is a placement file at its version with its modules (S1); a node asks the vault only when a module placed on it declares an own secret, so one with none composes whether or not the vault answers (H8's failure) (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 3968849b1b36 on a7a1fbd34e1d | novox/mesh-sdk#98 in group feat/1126-nodes-compose
2026-10-26 20:08:32 +01:00
jschoubben 02f50bd9d6 A build seat's capacity follows its node's build CPU, and a running ask says its phases
Since the build CPU cap (issue 1806) shanks' mesh-build-go kept capacity 3 inside a 12-core slice:
three full checks got about 4 cores each and ran past the 30-minute step bound (issue 1890).
The puller now judges by min(configured, floor(slice quota cores / cores-per-step)), the quota
read live from its cgroup, says it on change and writes capacity.<node> to the pauses state.
A run's phases (fetch and merge, digest, tests) are logged with the ask id and written to its
running step key; Asker.Wait hands every step value to a WithStepSeen hook.
2026-10-26 19:52:54 +01:00
jschoubben 81ec7bd7de Drop the failure contract's unused version constant: a result names its build module with By (issue 1901) 2026-10-26 19:50:17 +01:00
jschoubben fe11db9293 MergeInto also names programs a commit of the branch added and a later one removed: the merge commit would keep the blob in history (issue 1577)
mesh/delivery merged: source.main moved to novox/mesh-sdk@dae8e288d2e1
mesh/delivery-group group fix/1577-checker-refuses-binaries merged: every member is merged
mesh/check checking fe11db929397 on a7a1fbd34e1d | novox/mesh-sdk#99 in group fix/1577-checker-refuses-binaries
2026-10-26 19:47:17 +01:00
jschoubben e40dd0f56a One failure contract for a build step's failed test, its parsers and a conformance kit every build module runs (issue 1901)
Each build module parsed its own test runner's output, so a fix in one never reached the others: a failed node --test
run was worded by a passing line of its first file. buildstep.Failure names the test, its file, its cause and its own
output; ReadGoTest (moved from mesh-build-go) and ReadTAP fill it; package conformance holds every build module to it
with runner output captured with the build nodes' own toolchains.
2026-10-26 19:45:35 +01:00
jschoubben 9713c3023a Placement carries a settings-form module's config values, which mesh-config does not hold, and the export writes them (issue 1126, ADR 0555 §1): the media modules on one node would compose without them
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: ace: novox/mesh-catalog#505 does not merge into the trunk: files in conflict: .gitignore | 9713c3023a18 on a7a1fbd34e1d | unit 1245…
2026-10-26 19:38:37 +01:00
jschoubben 53c2def1a5 VaultSecretChanged is a variable (its seat is), and gofmt (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: unit: git.novox.be/novox/mesh-catalog/checks/nobinaries.TestTheCatalogueHoldsNoCompiledProgram: modules/mesh-config/cmd/mesh-config…
2026-10-26 19:25:19 +01:00
jschoubben f55b34febc A node composing its own hears the vault's secret-changed, names and generations only (issue 1126, ADR 0547 §2)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed (6 tests): unit: go.gofmt: not as gofmt writes them (run gofmt -w): go/buscompose/buscompose_test.go | f55b34febc2d on a7a1fbd34e1d …
2026-10-26 19:25:09 +01:00
jschoubben 944a35d200 A node's record carries the mesh's own certificate as the machine holds it, and frombus composes with the composing node's (issue 1126, ADR 0555 §3)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: shanks: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk 91413de33516, which is neither on mesh-sdk's trunk nor the g…
2026-10-26 19:23:06 +01:00
jschoubben 7d81e8326f frombus gives the composing node's core credentials as the vault's boxes sealed to it, from the same own secrets (issue 1126, ADR 0349 §16, ADR 0550 §5) 2026-10-26 19:22:58 +01:00
jschoubben 91413de335 compose names the pair credentials a node's composition reads as consumer and provider, and frombus asks them sealed before it composes (issue 1126, ADR 0550 §5)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: g14: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk f1d7c7377dc4, which is neither on mesh-sdk's trunk nor the grou…
2026-10-26 19:18:46 +01:00
jschoubben 6447a1c6d4 placement-seed writes each node's first placement file from the controller's export, checked before anything is written, once (issue 1126, ADR 0549 §4, ADR 0555 §6)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: shanks: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk f1d7c7377dc4, which is neither on mesh-sdk's trunk nor the g…
2026-10-26 19:17:45 +01:00
jschoubben 4d5f430224 A want the node composes itself says so (Self) and names the build alone; every other want names its composition as before, so no caller of the old form changes (issue 1126, ADR 0547 §2)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: ace: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk f1d7c7377dc4, which is neither on mesh-sdk's trunk nor the grou…
2026-10-26 19:13:56 +01:00
jschoubben dfe3ba9475 frombus exports a node's placement from a composition input, Input's inverse for what a person decides, and the controller's seat declares placement-export (issue 1126, ADR 0555 §6)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking dfe3ba94750c on a7a1fbd34e1d | novox/mesh-sdk#98 in group feat/1126-nodes-compose
2026-10-26 19:11:34 +01:00
jschoubben f1d7c7377d frombus reads the composing node's own secrets sealed to it, and node principals may ask the vault's own in their own name (issue 1126, ADR 0550 §5)
mesh/delivery-group group feat/1126-nodes-compose checking: 4 of 5 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking f1d7c7377dc4 on a7a1fbd34e1d | novox/mesh-sdk#98 in group feat/1126-nodes-compose
2026-10-26 19:03:00 +01:00
jschoubben 22bc4dc8ef MergeInto names the changed files that are compiled programs, read in the workspace, so a check can refuse build output (issue 1577) 2026-10-26 18:59:35 +01:00
jschoubben b72de2b75a A node applying what it composed refuses a composition that stands in for anything its input lacks, the bus's user list alone held out (issue 1126, ADR 0547 §2) 2026-10-26 18:59:05 +01:00
jschoubben 6df0dfe208 Node principals read what they compose their own modules from; the composer copy is held per user to the controller's grants plus exactly these (issue 1126, ADR 0547 §2, ADR 0549 §3)
mesh/delivery-group group feat/1126-nodes-compose rejected: composed together: the checker's verdict at the members' heads (922da6d6ae76): forge
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 6df0dfe208c6 on a7a1fbd34e1d | novox/mesh-sdk#98 in group feat/1126-nodes-compose
2026-10-26 18:44:06 +01:00
jschoubben 9532b78724 A node's own want names no composition from the second milestone: the node composes its own (issue 1126, ADR 0547 §2) 2026-10-26 18:41:33 +01:00
jschoubben e49309d61c A want names the build alone from the second milestone, and the yes a person gave; a have waits for that yes with its judgement (issue 1126, ADR 0547 §2, §3) 2026-10-26 18:39:46 +01:00
jschoubben 9789c8379a Node principals read every node's own facts, where its record rides; the composer copy is checked to grant all the controller's grants and only that and the issue 1790 rows more (issue 1126, ADR 0555 §3)
mesh/delivery-group group feat/1126-nodes-compose checking: 1 of 3 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 9789c8379a74 on a7a1fbd34e1d | novox/mesh-sdk#98 in group feat/1126-nodes-compose
2026-10-26 18:33:36 +01:00
jschoubben 2b0689f654 A node's record rides in its own have key, <node>.node: no bucket, writer or grant of its own, and its key's change is what readers hear (issue 1126, ADR 0555 §3) 2026-10-26 18:29:45 +01:00
jschoubben 5b1e990eb4 The tunnel found at adoption and the names of the hub's peers are kept in the placement; whether the hub took it over and who enrolled under a peer's key are read from placement and records, as the controller reads them (issue 1126, ADR 0555) 2026-10-26 18:26:11 +01:00
jschoubben a431746c0e What a person decides of a node lives in its placement file and what the machine observes in its record; two nodes on one address or two hubs are refused (issue 1126, K2) 2026-10-26 18:17:59 +01:00
jschoubben 5db53e738f Name the catalogue repository whose placement files are read (issue 1126, ADR 0549 §1) 2026-10-26 18:10:25 +01:00
jschoubben 9f7f50de8d A merged message of the catalogue carries its placement files, so the indexer reads placement at the merge without asking the forge again (issue 1126, ADR 0549 §2) 2026-10-26 18:08:59 +01:00
jschoubben bdd4c7b875 The judgement of a move is a pure function in mesh-sdk, so a node agent judges what it composed against what it runs (issue 1126, ADR 0547 §3, ADR 0377 §3) 2026-10-26 18:07:20 +01:00
jschoubben e982072443 frombus fills the composition's input from placement, node records, the indexer's manifests and mesh-config's slice, so one composer serves the node agent, the check and the grants part (issue 1126, ADR 0547 §2, ADR 0549 §3) 2026-10-26 18:06:04 +01:00
jschoubben 146cd858a3 Placement and the node record have one layout in mesh-sdk, so the indexer, the node agents and the composition check read them alike (issue 1126, ADR 0549 §1-§2, ADR 0547 §2) 2026-10-26 18:02:13 +01:00
mesh-tools a7a1fbd34e Merge pull request 'A module waits for a state created after it starts (issue 1790)' (#97) from fix/1790-state-appears into main 2026-10-26 14:47:31 +00:00
mesh-tools 1888d1d230 Merge pull request 'A git or ssh-keygen failure keeps its own words (forgekey.Said)' (#96) from fix/fetch-reads-all into main 2026-10-26 14:47:25 +00:00
mesh-tools 960d6c34d2 Merge pull request 'The composer gives build modules the node's share: idle, capped builds (issue 1806, ADR 0553/0554, change B)' (#95) from fix/build-cpu-yields-compose into main 2026-10-26 14:31:13 +00:00
jschoubben ff81bdf8cc Keep git's own words in a fetch's reason: a bare "error:" line no longer stands for the forge's refusal
mesh/delivery merged: source.main moved to novox/mesh-sdk@1888d1d230b3
mesh/delivery-group group fix/fetch-reads-all merged: every member is merged
mesh/check pass ff81bdf8cc8f on 960d6c34d25e | unit 5815 run, 345 kept, 18 skipped | integration 5603 run, 640 kept, 29 skipped | over bound: unit, in…
The mirror of jschoubben/car-hunter on ace said only "git -C: error:": git writes a bare
"error:" before the forge's message, and the reason kept the first non-empty line. forgekey.Said
keeps every line that says something, leaves bare prefixes out, joins them and bounds them at
600 bytes; every git and ssh-keygen reason of buildstep and forgekey goes through it, and the
mirror's commands are named by their verb, not by -C.
2026-10-26 15:26:29 +01:00
jschoubben 65ee43f075 Merge remote-tracking branch 'origin/main' into fix/1790-state-appears
mesh/delivery merged: source.main moved to novox/mesh-sdk@a7a1fbd34e1d
mesh/delivery-group group fix/1790-state-appears merged: every member is merged
mesh/check being carried to 65ee43f07583 on a7a1fbd34e1d | from 1888d1d230b3, mesh-delivery feef96ed160e
2026-10-26 15:24:34 +01:00
jschoubben c3e8fc4ba6 process: an awaited bucket says loudly once when the bus refuses hearing its creation, and hears it again and binds at each reconnect (issue 1790, review) 2026-10-26 15:24:33 +01:00
mesh-tools 75be8acbad Merge pull request 'Build seats yield: the decoder, buildcpu and NodeLimits (issue 1806, ADR 0553/0554, change A)' (#94) from fix/build-cpu-yields into main 2026-10-26 14:22:47 +00:00
mesh-tools d103ed88ab Merge pull request 'forgekey: a command starts in a directory that exists (issue 734)' (#93) from fix/daemon-dir-kept into main 2026-10-26 14:15:57 +00:00
jschoubben 3ac644baef process: gofmt the integration fixture (issue 1790) 2026-10-26 15:13:25 +01:00
jschoubben fd7d93333e Keep a failed toolchain probe a loud refusal; only the command directory is fixed (issue 734, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-sdk@d103ed88ab17
mesh/delivery-group group fix/daemon-dir-kept merged: every member is merged
mesh/check checking fd7d93333e6b on 21252194d530 | novox/mesh-sdk#93 in group fix/daemon-dir-kept
Declining an ask whose probe failed would turn a truly broken toolchain into a
wait of up to the decline bound that then fails with an empty "needs a node
with" reason. With the node-engine no longer deleting a running build and
commands started in a directory that exists, the getwd cause is gone, so the
guard's refusal stays as it was. The decline refactor goes with it.
2026-10-26 15:11:04 +01:00
jschoubben d515c71e1c process: a bucket missing at a part's start is bound when the bus says it was created, keeping what was written meanwhile; buscompose grants each principal the creation of the buckets it may bind (issue 1790)
Since the grants part creates a manifest's buckets after a delivery (ADR 0547), a module restarted by that
delivery can open a new state before it exists; OptionalBucket answered nil and the part never looked again
(mesh-delivery's checker and builder, state hearing, 2026-10-26). AwaitBucket binds on the server's
STREAM.CREATED advisory, or at the next use, and writes the latest value kept per key. The copied composer's
agreement test leaves that one row out: the controller's list is no longer loaded since the take-over.
2026-10-26 15:11:00 +01:00
jschoubben 36061b5c56 Start commands in a directory that exists, and decline an ask whose toolchain could not be read (issue 734)
A build module runs in the directory its build was unpacked in. When the
node-engine replaced that build, every command the module started inherited a
deleted working directory, and the toolchain probe's failure became a final
refusal of the check that asked it (three checks on ace on 2026-10-26).

forgekey: ExecRunnerClean starts its command in the HOME it is given, or the
temporary directory; the other runners keep the process's directory while it
exists and use the temporary directory once it is gone.

buildstep: a guard whose probe failed answers a Refusal with Again set; the
puller declines such an ask (delivery.StepDeclined) so its asker puts it again
for another node or a later pull, instead of ending it refused. A version
mismatch stays a refusal.
2026-10-26 15:01:57 +01:00
jschoubben 4ef49d0391 Merge fix/build-cpu-yields into the composer half
mesh/delivery merged: source.main moved to novox/mesh-sdk@960d6c34d25e
mesh/check checking 4ef49d039195 on 960d6c34d25e
2026-10-26 14:46:11 +01:00
jschoubben 17b7cc480d Merge main into fix/build-cpu-yields
mesh/delivery merged: source.main moved to novox/mesh-sdk@75be8acbad4d
mesh/delivery-group group fix/build-cpu-yields merged: every member is merged
mesh/check checking 17b7cc480d6d on d103ed88ab17 | novox/mesh-sdk#94 in group fix/build-cpu-yields
2026-10-26 14:46:01 +01:00
jschoubben 6ac8c5c229 The composer gives a build seat's holder its node's one share, the node-engine's build-cpu-share (ADR 0553 §3, §4)
The second half of ADR 0553, merged only once the node-engine that decodes
yields runs on every node (§5). The share is the node-engine's own config as
the node's layers give it; each step divides it by its own module's capacity.
A share out of range bounds builds at the default and never refuses the
node's composition. A change of the share recomposes the build runtimes, which
restart once.
2026-10-26 14:42:20 +01:00
mesh-tools 21252194d5 Merge pull request 'A check ask can name needs and packages, and a node that lacks a need declines it (issue 890, ADR 0552)' (#91) from feat/890-placement into main 2026-10-26 13:40:35 +00:00
mesh-tools 545a0c9cb4 Merge pull request 'An asker of the build seats hears when its ask is given up, and the seat's users may read their own dead letters (issue 734)' (#92) from fix/734-lost-ask into main 2026-10-26 13:40:30 +00:00
jschoubben 527d87613e Without the composer's share a step has no thread limit; with it, the share over the module's own capacity (ADR 0553 round 2)
Review round 2: change A must not slow builds before any slice exists, so a
module given no share sets no GOMAXPROCS. The slice's quota bounds the total;
threads only size one step, from the node's share over the module's own
capacity (shanks: 24 x 50% / 3 = 4). The share is the node-engine's own config.
2026-10-26 14:39:39 +01:00
jschoubben 36c92861d2 Merge main into fix/build-cpu-yields 2026-10-26 14:22:26 +01:00
jschoubben 852ba81c5a Build seats yield to one shared slice per node; the composer half waits for the engines (ADR 0553, issue 1806)
Review round 1: the decoder type ships before any composer can emit it, so a
routine SDK bump never sends yields to an engine that refuses them; the
composer half moves to a later change. Every yielding process of a node goes
into one top-level slice, meshyield.slice, so the three build modules share one
quota instead of taking one each; the share is one node value, node-tools'
build-cpu-share, and each step's threads divide it by every build slot of the
node (words from the composer, a conservative fallback without them).
2026-10-26 14:20:05 +01:00
jschoubben 3203358c4c buildrun: an asker keeps whether it hears DEAD_LETTERS, and its health tool answers from it (issue 734, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-sdk@545a0c9cb43b
mesh/delivery-group group fix/734-lost-ask merged: every member is merged
mesh/check being carried to 3203358c4cb5 on 2aae6f5580f2 | from mesh-delivery c8b26735c8cf | novox/mesh-sdk#92 in group fix/734-lost-ask
A deaf asker must be loud: SayHearing keeps each Hearing in the asker's state hearing, and HearingHealth answers the
module's health tool from it, unhealthy with why while DEAD_LETTERS is not heard.
2026-10-26 14:18:41 +01:00
jschoubben 9b01c94a9e buildrun: the asker hears DEAD_LETTERS again on its next reconnect or ask, in the background, and says whether it hears; buscompose grants only what a pulled ordered consumer needs (issue 734, review round 1)
An asker started before its grant was composed was refused once and stayed deaf until it started again, silently.
DeadLetterHearing tries in the background and again on an event (Again: a reconnect, an ask waited for while deaf), and
tells each change of Hearing so the module's health tool can say it. DeadLetterReads drops the flow control a pull
consumer does not use and deletes one consumer by name.
2026-10-26 14:16:58 +01:00
jschoubben 33395f68b3 Merge main (issue 1890's step records) into feat/890-placement: a decline begins its step with what it is
mesh/delivery merged: source.main moved to novox/mesh-sdk@21252194d530
mesh/delivery-group group feat/890-placement merged: every member is merged
mesh/check checking 33395f68b309 on 2aae6f5580f2 | novox/mesh-sdk#91 in group feat/890-placement
2026-10-26 14:05:30 +01:00
jschoubben 45564e69a6 A placed check ask names the packages it tests, so the first ask's other packages are not run or counted again (issue 890, ADR 0552 §4; review round 1) 2026-10-26 14:01:05 +01:00
jschoubben d96b9fd660 buildrun: an ask the build seat gave up on ends its asker's wait as failed with why, for every asker of the build seats (issue 734)
The checker's try and the builder's lanes waited for ever on an ask given up during a bus reload: no build node writes a
step for it. LostAskWaiter hands the asker a failed step on the node bus when DEAD_LETTERS keeps the ask's copy
(HearDeadLetters, granted by DeadLetterReads), and under NotTakenBound when no node took the ask within the bound.
2026-10-26 14:00:46 +01:00
mesh-tools 2aae6f5580 Merge pull request 'buildstep: a build module ends the steps a previous process left running, and a step says what it is (issue 1890)' (#90) from fix/1890-left-running-steps into main 2026-10-26 12:57:31 +00:00
jschoubben 6b84973142 Build steps: an ask key past the step bound is ended, at start and on the module's round (issue 1890 review 2)
mesh/delivery merged: source.main moved to novox/mesh-sdk@2aae6f5580f2
mesh/delivery-group group fix/1890-left-running-steps merged: every member is merged
mesh/check pass 6b8497314297 on 2aae6f5580f2 | carried from b4cd95e7c4a4, mesh-catalog a4f5df797b68 | unit 3132 run, 3022 kept, 7 skipped | integratio…
2026-10-26 13:51:54 +01:00
jschoubben e7f0dc612e buscompose: a user of a seat pulled by kind reads its seat's dead letters, so an asker hears an ask given up (issue 734)
A check ask given up during a bus reload left its try running for ever: the asker had no grant to learn that
DEAD_LETTERS holds it. DeadLetterReads grants an ordered consumer on DEAD_LETTERS filtered to the seat stream's copies,
and nothing else of the stream.
2026-10-26 13:51:35 +01:00
jschoubben 76deab6c67 gofmt the cap's test (issue 1806) 2026-10-26 13:49:09 +01:00
jschoubben e66c6fea62 Build steps: test the ask key with a real ask id (issue 1890) 2026-10-26 13:43:05 +01:00
jschoubben 6b1c7a5613 Build steps: a restart leaves pulled asks to the stream, ends steps by compare-and-set, and step_end refuses a step this process started (issue 1890 review 1) 2026-10-26 13:42:50 +01:00
jschoubben 04902ff24a Build seats yield: idle priority and a per-node processor cap on their runtimes, threads per step within it (issue 1806)
The operator's desktop slows while it builds: the Go build runtime took 1650% of
shanks' 24 processors with nothing holding it back. A build seat's holder now
runs its runtime at idle priority under its node's cpu-cap share (declaration
Yields, composed from Seat.Yields and the module's cpu-cap setting), and
buildcpu/buildstep give each step its part of that cap in threads.
2026-10-26 13:37:00 +01:00
jschoubben bb71310337 Let a check ask name what its node needs, decline it where that is lacking, and leave unmet packages to such a node (issue 890)
A check ask's packages may need something only some build nodes have (mesh-build-node's integration tests read the
Node build module's installed toolchain). An ask now names its Needs; a holder lacking one ends the ask's step refused
as declined and drops the message for its asker to put again, skipping that worker until its next pull wait; a level's
packages whose declared needs the node lacks are left untested and named unmet, for the asker to place.
2026-10-26 13:35:12 +01:00
jschoubben 18e0a01c54 Build steps: EndStaleAtStart waits for the runtime and ends the left-running steps (issue 1890) 2026-10-26 13:25:58 +01:00
jschoubben ddafe1b1aa Build steps: a step records what it is, a restart ends the steps it left running, a tool ends one (issue 1890) 2026-10-26 13:25:13 +01:00
mesh-tools b4cd95e7c4 Merge pull request 'Take-over of the bus's user list by the grants part: sealed list layout, followed-list have, handed hashes, named readers (issue 1945)' (#89) from feat/1945-user-list-switch into main 2026-10-26 12:21:39 +00:00
mesh-tools 89c5838176 Merge pull request 'Delivery layouts for ADR 0548: merge turns, carried verdicts and the siblings a verdict read (issue 568)' (#88) from feat/548-one-check into main 2026-10-26 12:01:17 +00:00
jschoubben c2a410bb63 The have's user list says why the bus node does not watch the grants part's key (review of the switch, issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-sdk@b4cd95e7c4a4
mesh/delivery-group group feat/1945-user-list-switch merged: every member is merged
mesh/check checking c2a410bb6376 on 89c5838176b2 | novox/mesh-sdk#89 in group feat/1945-user-list-switch
2026-10-26 12:25:27 +01:00
jschoubben af66adc180 Bring in the adoption handing (fix/1945-adopt-hashes 17478994): the have's user list carries Handed beside Follows and Key; a state that names readers is not Readable to a composer that knows no readers (review of the switch, issue 1945) 2026-10-26 12:23:07 +01:00
jschoubben 9c61918ac3 delivery: a turn never holds the check of heads that wait in it, so two approved merges in one turn never hold each other's carry (ADR 0548 §3; issue 568, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-sdk@89c5838176b2
mesh/delivery-group group feat/548-one-check merged: every member is merged
mesh/check checking 9c61918ac36f on 85fa8511c4b3 | novox/mesh-sdk#88 in group feat/548-one-check
2026-10-26 12:12:53 +01:00
jschoubben 83ee7769d8 Make the grants part's state in the switch's test as the part makes a declared state, by its stream (issue 1945) 2026-10-26 12:00:29 +01:00
jschoubben 175a5567b8 The bus node follows the grants part's sealed user list, and only the grants part reads the vault's hashes (ADR 0547 clauses 7, 9; issue 1945)
The grants part writes its list as an object of its own bucket, sealed to the bus node's key, and names it in
user-list.sealed; the node-engine of the node holding mesh-broker is granted that one key and an object by name, the
grants part its bucket (the controller's copy carries the same grants, byte for byte, golden regenerated there).
A state may name its readers (announce, NamedReaders): mesh-vault.bus-accounts is read by mesh-grants alone.
2026-10-26 11:53:11 +01:00
jschoubben 1747899419 delivery: the bus node hands the loaded list's hashes to the vault, sealed, in the bus module's have (ADR 0547 clause 9, issue 1945)
The vault holds no value for the controller's own account, the node accounts and the enrolled devices, so it cannot
hash them; the one part that has their hashes is the node-engine of the bus node, in the list the bus loaded. The have
of the bus module now carries that list's hashes sealed to the vault's key (UserListWritten.Handed), which every reader
of the have bucket sees as a count, a list digest and a key fingerprint, and only the vault opens. HandedAccountsOf
reads them from a composed list, HandAccounts seals, OpenHanded opens and refuses a box sealed to another key.
2026-10-26 11:28:38 +01:00
jschoubben f5c233a928 delivery: a verdict names the siblings it read and the commits it was carried from, and a composition its fingerprint set, so a trunk or sibling move is carried, not judged again (ADR 0548 §5, §6; issue 568 S2) 2026-10-26 11:25:07 +01:00
jschoubben 0abe480dba delivery: the gate's merge turns, so the checker holds new checks off a repository an approved merge waits on (ADR 0548 §3, §4; issue 568 S1) 2026-10-26 11:24:06 +01:00
mesh-tools 85fa8511c4 Merge pull request 'Build steps: run every test step of a level and name every failure in the verdict (issue 1921)' (#87) from fix/1921-every-failure-r into main 2026-10-26 10:18:05 +00:00
jschoubben 79649c3fa9 Run every test step of a check's level after a failure, and let a verdict name every failed test (issue 1921)
mesh/delivery merged: source.main moved to novox/mesh-sdk@85fa8511c4b3
mesh/delivery-group group fix/1921-every-failure-r merged: every member is merged
mesh/check checking 79649c3fa9f4 on c3544b6a17a2 | novox/mesh-sdk#87 in group fix/1921-every-failure-r
A check ask stopped its level at the first test step that did not pass,
so the packages after the changed ones never ran and a check found one
fault per round. Every step now runs; the level still fails, and no
later level runs (ADR 0484 clause 13). The verdict gains Failures, every
failed test with its first line, at most MaxFailures and a count.
2026-10-26 10:57:02 +01:00
mesh-tools c3544b6a17 Merge pull request 'buscompose: the grants part may create a placed module's declared buckets, logs and streams, creation only (issue 1790, ADR 0547 clause 1 exception)' (#86) from fix/1790-grants-create-buckets into main 2026-10-26 08:17:16 +00:00
jschoubben 136fa8b2a8 Say how a declared bucket and module stream are created, so the grants part makes the same stream the controller does
mesh/delivery merged: source.main moved to novox/mesh-sdk@c3544b6a17a2
mesh/delivery-group group fix/1790-grants-create-buckets merged: every member is merged
mesh/check checking 136fa8b2a896 on 095d41fa3e81 | novox/mesh-sdk#86 in group fix/1790-grants-create-buckets
BucketStreamConfig and ModuleStreamConfig are the stream configurations the controller's
CreateOrUpdateKeyValue (BucketConfig) and StreamConfigFor assert at its start, pure, for the
grants part's create of a bucket or stream a manifest declares and the bus lacks. A log already
has Log.Config. Never used for an update.

Issue: 1790
2026-10-26 09:04:43 +01:00
jschoubben 8f6ee2fd18 Let the grants part create the streams manifests declare, so a new module state no longer waits for a controller start
The copied bus composer grants mesh-grants' principal $JS.API.STREAM.CREATE.<stream> and
$JS.API.STREAM.INFO.<stream> for each bucket (KV_<module>_<state>), log (LOG_<module>_<log>) and
module stream a placed module's manifest declares, each by its own name: NATS wildcards are whole
tokens, so nothing narrower than an exact name exists and nothing wider is granted. The writers
table lets the grants part create, never update or delete. The operator allowed this one grant
on 2026-10-26 08:54, beside the memberships grant of 22:31 (ADR 0547 clause 1 and 5).

The expected lists were written again by the controller's own internal/broker with the same
change (mesh-control fix/1790-grants-create-buckets); a fixture now places mesh-grants beside the
builder's state, log and stream.

Issue: 1790
2026-10-26 09:01:32 +01:00
jschoubben 095d41fa3e Merge pull request 'buscompose: the controller's bus composer in the SDK, byte-equal to the controller's, with the mesh-grants grant (ADR 0547)' (#85) from fix/1945-memberships into main
Reviewed-on: #85
2026-10-25 22:43:57 +00:00
jschoubben b7b5c6ef27 The copied composer grants the grants part reading every membership, and its memberships are held to the controller's too (ADR 0547 §5, review of mesh-grants)
mesh/delivery merged: source.main moved to novox/mesh-sdk@095d41fa3e81
mesh/delivery-group group fix/1945-memberships merged: every member is merged
mesh/check failed: integration: github.com/novox/mesh-controller/cmd/mesh-controller.TestTheDeclarationsStreamGoesOnceEveryNodeRunsTheNewEngine: with …
The grants part reads each module's last membership before it publishes, so it needs the direct get on every
membership subject, as the controller's composer now grants (fix/1945-mesh-grants-publishes-memberships). The equality
test compares every membership the fixtures issue with the controller's MembershipFor, written by its own code.
2026-10-25 22:50:21 +01:00
mesh-tools 7ac4493ce9 Merge pull request 'process: open a part's buckets as essential (stop, naming the grant) or optional (log what is not heard, keep running) (issue 1945)' (#84) from fix/1945-ask-again-and-why into main 2026-10-25 21:43:08 +00:00
jschoubben 881da7f599 The copied bus composer grants the grants part publishing memberships, as the controller's does (ADR 0547 §5)
The same line as mesh-controller f67d1553; the fixtures name a mesh-grants module and the lists were written again by
the controller's own composer on that branch, so the copy is held byte-equal to it.
2026-10-25 22:33:20 +01:00
mesh-tools 3db051be57 Merge pull request 'A node's have says which bus user list it wrote: delivery.Have's user-list field (ADR 0547 bridge, issue 1945)' (#83) from fix/1945-user-list-follows into main 2026-10-25 21:16:03 +00:00
jschoubben b5f8604d03 Copy the controller's bus composer into mesh-sdk as buscompose, held byte for byte to the controller's lists (issue 1377, ADR 0547 §5)
The grants part composes the bus's user list with the composer the controller uses (ADR 0547 §5), copied rather than
moved: the controller is not changed and keeps its own until it loses the lease. Left out: the bus client, the broker's
address from the environment and the client's two methods; the lease's two subjects are copied in.

One table test holds the copy to four meshes' lists and the server golden, written by the controller's own
internal/broker from the same fixtures (testdata/controller-user-lists.json).
2026-10-25 21:54:58 +01:00
jschoubben 0be0ffd615 Compare the have's user list by what it says, so a have is not written again for the same list (ADR 0547 §8)
mesh/delivery merged: source.main moved to novox/mesh-sdk@3db051be57ca
mesh/delivery-group group fix/1945-user-list-follows merged: every member is merged
mesh/check pass 0be0ffd6159e on db26f7e8b6f5 | unit 1467 run, 1696 kept, 7 skipped | integration 1471 run, 1740 kept, 6 skipped | over bound: unit, in…
2026-10-25 21:47:30 +01:00
jschoubben f63fa86c74 Say in the bus module's have the user list its node wrote alone, so the bridge of ADR 0547 §8 reports what the bus loads (issue 1945) 2026-10-25 21:42:13 +01:00
195 changed files with 23235 additions and 325 deletions
+29
View File
@@ -46,6 +46,10 @@ func Key(module string) (string, error) {
type State struct {
Name string `json:"name"`
Readable bool `json:"readable,omitempty"`
// Readers, when named, are the only modules that may read the state (novox/hq ADR 0547 clause 9, issue 1945): a
// state readable by some and not all, such as the vault's bus account hashes. Only ReadableBy grants them; Readable,
// which a composer that knows no readers asks, stays false for it, so such a composer grants nobody.
Readers []string `json:"readers,omitempty"`
}
// States is an announcement: the module, the machine whose instance said it, and every state the module keeps, sorted
@@ -67,6 +71,26 @@ func (s States) Keeps(state string) bool {
return false
}
// ReadableBy says whether the module announced the state readable by reader: readable, and reader among the readers
// when it names any.
func (s States) ReadableBy(state, reader string) bool {
for _, k := range s.States {
if k.Name != state {
continue
}
if len(k.Readers) > 0 {
for _, r := range k.Readers {
if r == reader {
return true
}
}
return false
}
return k.Readable
}
return false
}
// Readable says whether the module announced the state readable by others.
func (s States) Readable(state string) bool {
for _, k := range s.States {
@@ -93,6 +117,11 @@ func (s *States) check() error {
return fmt.Errorf("announce: %s announces the state %q twice", s.Module, k.Name)
}
seen[k.Name] = true
for _, r := range k.Readers {
if !name.MatchString(r) {
return fmt.Errorf("announce: %s names %q as a reader of %s, and a reader is a module: lower-case letters, digits and hyphens", s.Module, r, k.Name)
}
}
}
return nil
}
+9 -2
View File
@@ -9,7 +9,8 @@ import (
// own event subject, in a versioned value the controller keeps by module.
func TestAnAnnouncementRoundTripsSortedAndSaysWhatIsReadable(t *testing.T) {
data, err := Encode(States{Module: "mesh-controller", Node: "novox", States: []State{
{Name: "plans-building", Readable: true}, {Name: "assignments", Readable: true}, {Name: "own"}}})
{Name: "plans-building", Readable: true}, {Name: "assignments", Readable: true}, {Name: "own"},
{Name: "hashes", Readers: []string{"mesh-grants"}}}})
if err != nil {
t.Fatal(err)
}
@@ -24,12 +25,17 @@ func TestAnAnnouncementRoundTripsSortedAndSaysWhatIsReadable(t *testing.T) {
for _, s := range back.States {
names = append(names, s.Name)
}
if strings.Join(names, ",") != "assignments,own,plans-building" {
if strings.Join(names, ",") != "assignments,hashes,own,plans-building" {
t.Fatalf("the states are not sorted: %v", names)
}
if !back.Readable("plans-building") || back.Readable("own") || back.Readable("absent") {
t.Fatal("Readable does not say what was announced readable")
}
// Named readers (ADR 0547 clause 9): readable by them alone, and not readable to a composer that knows no readers.
if back.Readable("hashes") || !back.ReadableBy("hashes", "mesh-grants") || back.ReadableBy("hashes", "spy") ||
!back.ReadableBy("plans-building", "spy") || back.ReadableBy("own", "spy") {
t.Fatal("ReadableBy does not say who was named")
}
if !back.Keeps("own") || back.Keeps("absent") {
t.Fatal("Keeps does not say what was announced")
}
@@ -57,6 +63,7 @@ func TestAnAnnouncementRefusesWhatNoBucketCouldBe(t *testing.T) {
{States{Module: "a", States: []State{{Name: "x_y"}}}, `"x_y"`},
{States{Module: "a", States: []State{{Name: ""}}}, `""`},
{States{Module: "a", States: []State{{Name: "x"}, {Name: "x", Readable: true}}}, "twice"},
{States{Module: "a", States: []State{{Name: "x", Readers: []string{"b.c"}}}}, `"b.c"`},
} {
if _, err := Encode(c.s); err == nil || !strings.Contains(err.Error(), c.says) {
t.Errorf("%+v: %v, wanted a refusal saying %s", c.s, err, c.says)
+215
View File
@@ -0,0 +1,215 @@
// Package buildcpu is how much of a build node's processors its builds may take, and how they yield to everything else
// there (novox/hq ADR 0553, issue 1806; the operator, 2026-10-26: "limit CPU usage on our build nodes so the total cpu
// usage never impacts the normal workings of our nodes ... We have 3 build agents so they should share the cpu power").
//
// Three parts, each with one owner:
//
// - **Priority.** A build seat's holder (catalogue.Seat.Yields) runs at the lowest priority there is: the composer
// gives its runtime process declaration.Yields, and the node-engine places every yielding process of the node in
// one slice (declaration.YieldSlice) whose CPU weight is idle, with Nice=19 for every process it starts.
// Interactive and service work always comes first; builds take the cycles nothing else wants.
// - **One cap per node, shared.** The node's share of its processors is one value, the node-engine's own config
// `build-cpu-share` (ShareSetting), set per node through the node's settings layer. The node-engine writes it as the
// slice's CPUQuota: one bound the kernel holds for all the node's build modules together, never one per module.
// - **Threads.** The quota bounds the total; threads only size each step. A step gets the share divided by its own
// module's capacity (Threads), from the share the composer hands as a word (ShareWord): GOMAXPROCS sets every go
// command's -p, the compiler's concurrency, a test binary's processors and -parallel. A module given no word sets
// no thread limit at all: without the slice there is nothing to size a step to, and a limit then would only slow
// builds.
package buildcpu
import (
"os"
"runtime"
"strconv"
"strings"
)
// ShareSetting is the node-engine's config value of the node's processor share for its builds, in percent of the
// machine's processors, 1 to 100: one value per node, set through the node's settings layer (novox/hq ADR 0553 §3).
const ShareSetting = "build-cpu-share"
// DefaultPercent is the share where nothing sets one: half the machine, so a desktop keeps half its processors for
// itself whatever builds.
const DefaultPercent = 50
// ShareWord is the word the composer gives every yielding module (novox/hq ADR 0553 §4): the node's share, in percent.
const ShareWord = "MESH_BUILD_CPU_SHARE"
// Quota is a CPUQuota for a share of a machine with cores processors: systemd counts a quota in percent of one
// processor, so half of 24 is 1200%. At least 100%, one processor, so a share on a small machine never stops a step.
func Quota(percent, cores int) int {
q := percent * cores
if q < 100 {
q = 100
}
return q
}
// Threads is how many threads one step may run at once: its part of the share when its own module runs slots steps at
// once, at least one. The slice's quota bounds the steps of all the node's build modules together; this only sizes one.
func Threads(percent, cores, slots int) int {
if slots < 1 {
slots = 1
}
if percent < 1 || percent > 100 {
percent = DefaultPercent
}
t := cores * percent / 100 / slots
if t < 1 {
t = 1
}
return t
}
// Limits is what one step is held to on this node, as a status tool says it.
type Limits struct {
// Percent is the node's share in percent of its processors, and From where it was read: ShareWord, or "none" when
// the composer gave no share and a step runs without a thread limit.
Percent int `json:"share-percent,omitempty"`
From string `json:"share-from"`
// Cores is the machine's processor count.
Cores int `json:"cores"`
// Slots is this module's own capacity on the node: steps at once.
Slots int `json:"slots"`
// Threads is what each step is given (Threads); 0 is no limit.
Threads int `json:"threads-per-step"`
}
// FromWords is the limits of a step from the share the composer gave this process (ShareWord) and the module's own
// capacity. Without a usable share there is no limit: Threads is 0.
func FromWords(getenv func(string) string, own int) Limits {
if own < 1 {
own = 1
}
l := Limits{From: "none", Cores: runtime.NumCPU(), Slots: own}
if n, err := strconv.Atoi(strings.TrimSpace(getenv(ShareWord))); err == nil && n >= 1 && n <= 100 {
l.Percent, l.From = n, ShareWord
l.Threads = Threads(n, l.Cores, own)
}
return l
}
// Env is the environment a step's commands are given so they start no more threads than they may: GOMAXPROCS for every
// Go program (the compiler, a test binary, esbuild, buildah), UV_THREADPOOL_SIZE for Node's worker pool. Returned as
// KEY=value words to append to a command's environment; nothing for 0 threads, no limit.
func Env(threads int) []string {
if threads < 1 {
return nil
}
t := strconv.Itoa(threads)
return []string{"GOMAXPROCS=" + t, "UV_THREADPOOL_SIZE=" + t}
}
// InForce is what the kernel holds this process to, read rather than assumed (evidence over assertion): its nice
// level, its cgroup (the unit) and the cgroup's parent (the slice the node-engine placed it in), each one's CPU weight,
// idle mark and quota. A field the machine does not answer is left empty and Unread says which.
//
// The IO weight is read but binds only where the disk's scheduler honours it (bfq, or the io.cost controller); under
// the scheduler `none`, the usual one for NVMe, it does nothing, and the CPU bound is what holds.
type InForce struct {
Nice int `json:"nice"`
Cgroup string `json:"cgroup,omitempty"`
Unit Group `json:"unit"`
Slice Group `json:"slice"`
// Quota is the tighter of the two groups' cpu.max as a percent of one processor, 0 with none.
Quota int `json:"quota-percent,omitempty"`
Unread []string `json:"unread,omitempty"`
}
// Group is one cgroup's controls.
type Group struct {
Path string `json:"path,omitempty"`
CPUIdle string `json:"cpu-idle,omitempty"`
CPUWeight string `json:"cpu-weight,omitempty"`
// CPUMax is the cgroup's cpu.max: "<quota µs> <period µs>", or "max" with no quota.
CPUMax string `json:"cpu-max,omitempty"`
IOWeight string `json:"io-weight,omitempty"`
}
// cgroupRoot is where the unified hierarchy is mounted. A variable so a test can point it elsewhere.
var cgroupRoot = "/sys/fs/cgroup"
// procSelfCgroup is this process's cgroup file. A variable so a test can point it elsewhere.
var procSelfCgroup = "/proc/self/cgroup"
// procSelfStat is this process's stat file. A variable so a test can point it elsewhere.
var procSelfStat = "/proc/self/stat"
// ReadInForce reads what the kernel holds this process to.
func ReadInForce() InForce {
var f InForce
f.Nice = nice()
raw, err := os.ReadFile(procSelfCgroup)
if err != nil {
f.Unread = append(f.Unread, "cgroup: "+err.Error())
return f
}
for _, line := range strings.Split(strings.TrimSpace(string(raw)), "\n") {
if strings.HasPrefix(line, "0::") {
f.Cgroup = strings.TrimPrefix(line, "0::")
}
}
if f.Cgroup == "" {
f.Unread = append(f.Unread, "cgroup: not on the unified hierarchy")
return f
}
read := func(path string) Group {
g := Group{Path: path}
value := func(name string) string {
b, err := os.ReadFile(cgroupRoot + path + "/" + name)
if err != nil {
f.Unread = append(f.Unread, path+"/"+name+": "+err.Error())
return ""
}
return strings.TrimSpace(string(b))
}
g.CPUIdle, g.CPUWeight, g.CPUMax, g.IOWeight = value("cpu.idle"), value("cpu.weight"), value("cpu.max"), value("io.weight")
return g
}
f.Unit = read(f.Cgroup)
if i := strings.LastIndex(f.Cgroup, "/"); i > 0 {
f.Slice = read(f.Cgroup[:i])
}
for _, q := range []int{quotaOf(f.Unit.CPUMax), quotaOf(f.Slice.CPUMax)} {
if q > 0 && (f.Quota == 0 || q < f.Quota) {
f.Quota = q
}
}
return f
}
// quotaOf is a cpu.max value as a percent of one processor: "1200000 100000" is 1200; "max ..." or nothing is 0.
func quotaOf(cpuMax string) int {
fields := strings.Fields(cpuMax)
if len(fields) != 2 || fields[0] == "max" {
return 0
}
q, err1 := strconv.Atoi(fields[0])
p, err2 := strconv.Atoi(fields[1])
if err1 != nil || err2 != nil || p <= 0 {
return 0
}
return q * 100 / p
}
// nice is this process's nice level from its stat file (the 19th field, counted after the command's closing
// parenthesis, which may itself hold spaces); 0 when it cannot be read.
func nice() int {
raw, err := os.ReadFile(procSelfStat)
if err != nil {
return 0
}
s := string(raw)
i := strings.LastIndex(s, ")")
if i < 0 {
return 0
}
fields := strings.Fields(s[i+1:])
// After the command: state is field 3, so nice (field 19) is index 16.
if len(fields) < 17 {
return 0
}
n, _ := strconv.Atoi(fields[16])
return n
}
+90
View File
@@ -0,0 +1,90 @@
package buildcpu
import (
"os"
"path/filepath"
"reflect"
"runtime"
"testing"
)
// The node's share as a quota, the threads one step is given for its own module's capacity, and the word the composer
// hands, in one table (ADR 0553, issue 1806).
func TestShareQuotaAndThreads(t *testing.T) {
for _, c := range []struct {
name string
percent, cores, slots int
quota, threads int
}{
{"shanks: half of 24, three slots", 50, 24, 3, 1200, 4},
{"ace: three quarters of 24, two slots", 75, 24, 2, 1800, 9},
{"g14: half of 16, one slot", 50, 16, 1, 800, 8},
{"one processor at least", 10, 2, 4, 100, 1},
{"no slots counts as one", 100, 4, 0, 400, 4},
{"a share out of range is the default", 0, 8, 1, 0, 4},
} {
if c.quota != 0 {
if q := Quota(c.percent, c.cores); q != c.quota {
t.Errorf("%s: Quota = %d, want %d", c.name, q, c.quota)
}
}
if th := Threads(c.percent, c.cores, c.slots); th != c.threads {
t.Errorf("%s: Threads = %d, want %d", c.name, th, c.threads)
}
}
l := FromWords(func(k string) string { return map[string]string{ShareWord: "75"}[k] }, 2)
if l.Percent != 75 || l.From != ShareWord || l.Slots != 2 || l.Threads != Threads(75, runtime.NumCPU(), 2) {
t.Errorf("from the word: %+v", l)
}
// Told nothing usable, a step has no thread limit: nothing in its environment.
for _, word := range []string{"", "250", "lots"} {
l := FromWords(func(string) string { return word }, 3)
if l.Threads != 0 || l.From != "none" || Env(l.Threads) != nil {
t.Errorf("share %q: %+v, env %v", word, l, Env(l.Threads))
}
}
if got := Env(3); !reflect.DeepEqual(got, []string{"GOMAXPROCS=3", "UV_THREADPOOL_SIZE=3"}) {
t.Errorf("Env = %q", got)
}
}
// What the kernel holds a process to is read from its unit's and its slice's cgroup and its stat file, not assumed.
func TestReadInForce(t *testing.T) {
dir := t.TempDir()
cg := filepath.Join(dir, "cg")
slice := filepath.Join(cg, "meshyield.slice")
unit := filepath.Join(slice, "mesh-build-go-runtime.service")
if err := os.MkdirAll(unit, 0o755); err != nil {
t.Fatal(err)
}
for at, files := range map[string]map[string]string{
slice: {"cpu.idle": "1\n", "cpu.weight": "0\n", "cpu.max": "1200000 100000\n", "io.weight": "default 1\n"},
unit: {"cpu.idle": "0\n", "cpu.weight": "100\n", "cpu.max": "max 100000\n", "io.weight": "default 100\n"},
} {
for name, v := range files {
if err := os.WriteFile(filepath.Join(at, name), []byte(v), 0o644); err != nil {
t.Fatal(err)
}
}
}
self := filepath.Join(dir, "cgroup")
stat := filepath.Join(dir, "stat")
_ = os.WriteFile(self, []byte("0::/meshyield.slice/mesh-build-go-runtime.service\n"), 0o644)
// A command with a space and a parenthesis in it, then nice 19 in field 19.
_ = os.WriteFile(stat, []byte("42 (mesh (build) go) S 1 42 42 0 -1 4194560 1 0 0 0 0 0 0 0 39 19 8 0 1 2 3\n"), 0o644)
cgroupRoot, procSelfCgroup, procSelfStat = cg, self, stat
t.Cleanup(func() {
cgroupRoot, procSelfCgroup, procSelfStat = "/sys/fs/cgroup", "/proc/self/cgroup", "/proc/self/stat"
})
got := ReadInForce()
want := InForce{Nice: 19, Cgroup: "/meshyield.slice/mesh-build-go-runtime.service",
Unit: Group{Path: "/meshyield.slice/mesh-build-go-runtime.service", CPUIdle: "0", CPUWeight: "100",
CPUMax: "max 100000", IOWeight: "default 100"},
Slice: Group{Path: "/meshyield.slice", CPUIdle: "1", CPUWeight: "0", CPUMax: "1200000 100000",
IOWeight: "default 1"},
Quota: 1200}
if !reflect.DeepEqual(got, want) {
t.Fatalf("ReadInForce = %+v\nwant %+v", got, want)
}
}
+49 -10
View File
@@ -34,6 +34,9 @@ type AskSteps struct {
// the module's sweep.
Release func(workspace string) error
Logf func(format string, args ...any)
// Has says whether this node has a need a package's tests declare (delivery.AskNeeds, issue 890); a package needing
// what it lacks is not tested here but named in the level's Unmet, for the asker to place. Nil has nothing.
Has func(need string) bool
}
// RunAsk runs ask's job with steps and answers a delivery.BuildAnswer or a delivery.CheckAnswer.
@@ -67,6 +70,11 @@ func (s AskSteps) fetch(ctx context.Context, args map[string]any) (json.RawMessa
if s.Fetch == nil {
return nil, fetched{}, Refuse("this build module fetches nothing")
}
if _, merges := args["base"]; merges {
Phase(ctx, "fetch and merge")
} else {
Phase(ctx, "fetch")
}
out, err := s.Fetch(ctx, args)
if err != nil {
return nil, fetched{}, err
@@ -123,6 +131,7 @@ func (s AskSteps) build(ctx context.Context, job delivery.BuildJob) (any, error)
if s.Copy == nil {
return nil, Refuse("this build module copies no artifact (%s)", a.Artifact)
}
Phase(ctx, "copy "+a.Artifact)
out, err = s.Copy(ctx, map[string]any{"module": job.Module, "artifact": a.Artifact, "from": a.From})
} else {
if s.Build == nil {
@@ -132,6 +141,7 @@ func (s AskSteps) build(ctx context.Context, job delivery.BuildJob) (any, error)
if job.Publish {
args["publish"] = true
}
Phase(ctx, "build "+a.Artifact)
out, err = s.Build(ctx, args)
}
if err != nil {
@@ -190,9 +200,19 @@ func (s AskSteps) level(ctx context.Context, job delivery.CheckJob, level string
if runner != "" {
args["runner"] = runner
}
if len(job.NodeNames) > 0 {
args[delivery.NodeNamesArg] = job.NodeNames // the names check's (ADR 0567 §12)
}
return args
}
test := func(args map[string]any) (bool, error) {
what := "the whole level"
if dir, _ := args["module"].(string); dir != "" {
what = dir
} else if pkgs, _ := args["packages"].([]string); len(pkgs) > 0 {
what = fmt.Sprintf("%d packages", len(pkgs))
}
Phase(ctx, fmt.Sprintf("%s: tests of %s", level, what))
out, err := s.Test(ctx, args)
if err != nil {
return false, err
@@ -208,6 +228,9 @@ func (s AskSteps) level(ctx context.Context, job delivery.CheckJob, level string
_ = json.Unmarshal(r, &t)
return t.Passed, nil
}
// Every test step of the level runs, also after one that did not pass, so one check names every failing test of the
// level and not only the first (issue 1921); the level as a whole passes only when each of its steps did.
all := true
if s.Digest == nil {
// A module with no digest (Node) tests each module directory of the tree, those whose tests changed first.
for _, priority := range []bool{true, false} {
@@ -219,23 +242,24 @@ func (s AskSteps) level(ctx context.Context, job delivery.CheckJob, level string
if err != nil {
return false, fmt.Errorf("the %s level of %s: %w", level, dir, err)
}
if !passed {
return false, nil
}
all = all && passed
}
}
return true, nil
return all, nil
}
todo, all, keepRunner := s.plan(ctx, job, level, f, runner, withRunner, la)
todo, whole, keepRunner := s.plan(ctx, job, level, f, runner, withRunner, la)
if !keepRunner {
runner = "" // an installed module that ignores the runner judges the level as it judges any other (ADR 0535 §2)
}
if whole && len(job.Packages) > 0 {
todo, whole = job.Packages, false // a placed ask tests its packages alone, whatever the digest said (ADR 0552 §4)
}
for _, priority := range []bool{true, false} {
args := withRunner(map[string]any{"workspace": f.Workspace, "level": level})
if job.Run != "" {
args["run"] = job.Run
}
if all {
if whole {
if priority {
continue // nothing to put first: the level runs whole after
}
@@ -255,11 +279,9 @@ func (s AskSteps) level(ctx context.Context, job delivery.CheckJob, level string
if err != nil {
return false, fmt.Errorf("the %s level: %w", level, err)
}
if !passed {
return false, nil
}
all = all && passed // the packages left still run (issue 1921)
}
return true, nil
return all, nil
}
// release removes an ask's workspace, saying a failure: the module's sweep removes it later.
@@ -282,6 +304,7 @@ func (s AskSteps) plan(ctx context.Context, job delivery.CheckJob, level string,
if job.Run != "" {
return nil, true, true
}
Phase(ctx, level+": digest")
out, err := s.Digest(ctx, withRunner(map[string]any{"workspace": f.Workspace, "level": level}))
if err != nil {
if s.Logf != nil {
@@ -308,9 +331,16 @@ func (s AskSteps) plan(ctx context.Context, job delivery.CheckJob, level string,
return nil, true, runner != "" // a digest that names nothing or not the siblings keys no verdict: the level runs whole
}
la.Digest = &d
only := map[string]bool{}
for _, p := range job.Packages {
only[p] = true
}
var todo []string
for _, pd := range d.Packages {
p := pd.Package
if len(only) > 0 && !only[p] {
continue // a placed ask's other packages were judged by the first ask (ADR 0552 §4)
}
if job.Kept && s.Kept != nil {
tests, found, err := s.Kept(ctx, level, pd.Digest)
if err != nil && s.Logf != nil {
@@ -321,6 +351,15 @@ func (s AskSteps) plan(ctx context.Context, job delivery.CheckJob, level string,
continue
}
}
if needs := d.NeedsOf(p); len(delivery.Lacks(needs, s.Has)) > 0 {
// Its tests would only skip here (issue 890): it is left for a node that has what they need.
la.Unmet = append(la.Unmet, delivery.PackageNeeds{Package: p, Needs: needs})
if s.Logf != nil {
s.Logf("the %s level of %s is left to a node with %s: this node lacks %s", level, p,
strings.Join(needs, ", "), strings.Join(delivery.Lacks(needs, s.Has), ", "))
}
continue
}
todo = append(todo, p)
}
return todo, false, false
+96 -7
View File
@@ -20,6 +20,7 @@ type recorder struct {
fetch map[string]any
tests map[string]bool // package or module directory -> passed
digests map[string]string
needs []delivery.PackageNeeds
}
func (r *recorder) steps() AskSteps {
@@ -35,8 +36,10 @@ func (r *recorder) steps() AskSteps {
Build: call("build", func(args map[string]any) any {
return map[string]any{"digest": "sha256:" + args["artifact"].(string), "reference": "artifact-store://" + args["artifact"].(string)}
}),
Copy: call("copy", func(args map[string]any) any { return map[string]any{"digest": "sha256:copied"} }),
Digest: call("digest", func(map[string]any) any { return map[string]any{"level": "unit", "packages": r.digests} }),
Copy: call("copy", func(args map[string]any) any { return map[string]any{"digest": "sha256:copied"} }),
Digest: call("digest", func(map[string]any) any {
return map[string]any{"level": "unit", "packages": r.digests, "needs": r.needs}
}),
Test: call("test", func(args map[string]any) any {
passed := true
if pkgs, ok := args["packages"].([]string); ok {
@@ -109,9 +112,9 @@ func TestRunAskRefusesAnotherToolchain(t *testing.T) {
}
}
// 1887/A3, A5a, issue 1887 design limit 1: a check ask runs the levels of one check after one fetch: it digests the unit
// level, takes the kept verdicts, tests the packages whose tests changed first, and stops at the first failure, running
// no later level.
// 1887/A3, A5a: a check ask runs the levels of one check after one fetch: it digests the unit level, takes the kept
// verdicts, tests the packages whose tests changed first, then the rest also after a failure (issue 1921), and runs no
// level after one that did not pass (ADR 0484 clause 13).
func TestRunAskChecksTheLevelsOfOneCheck(t *testing.T) {
r := &recorder{
fetch: map[string]any{"workspace": "/w", "changed": []string{"pkg/b/b_test.go"}},
@@ -126,7 +129,8 @@ func TestRunAskChecksTheLevelsOfOneCheck(t *testing.T) {
got := strings.Join(r.calls, "\n")
want := `fetch {"base":"` + runCommit + `","commit":"` + runCommit + `","repository":"novox/x"}
digest {"level":"unit","workspace":"/w"}
test {"level":"unit","packages":["x/pkg/b"],"workspace":"/w"}`
test {"level":"unit","packages":["x/pkg/b"],"workspace":"/w"}
test {"level":"unit","packages":["x/pkg/a"],"workspace":"/w"}`
if got != want {
t.Fatalf("calls:\n%s\nwant:\n%s", got, want)
}
@@ -135,9 +139,55 @@ test {"level":"unit","packages":["x/pkg/b"],"workspace":"/w"}`
t.Fatalf("levels %+v, want the unit level alone", a.Levels)
}
u := a.Levels[0]
if u.Level != delivery.LevelUnit || len(u.Kept) != 1 || u.Kept[0].Package != "x/pkg/k" || u.Kept[0].Tests != 7 || len(u.Tests) != 1 || u.Digest == nil {
if u.Level != delivery.LevelUnit || len(u.Kept) != 1 || u.Kept[0].Package != "x/pkg/k" || u.Kept[0].Tests != 7 || len(u.Tests) != 2 || u.Digest == nil {
t.Fatalf("the unit level %+v", u)
}
// ADR 0567 §12: a check's node names reach the digest and test tools, for the names check (a kept verdict misses when
// they change).
r.calls = nil
if _, err := RunAsk(context.Background(), r.steps(), ask(delivery.CheckJob{Levels: []string{delivery.LevelUnit}, NodeNames: []string{"anchor"},
Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}})); err != nil || !strings.Contains(strings.Join(r.calls, "\n"), `digest {"level":"unit","node-names":["anchor"]`) ||
!strings.Contains(strings.Join(r.calls, "\n"), `test {"level":"unit","node-names":["anchor"]`) {
t.Fatalf("the node names are not handed to the digest and test tools: %v, %v", r.calls, err)
}
// Issue 890/AC2: a package declaring node-build-node is not tested on a node that lacks it but named unmet, and is
// tested on a node that has it.
for _, has := range []bool{false, true} {
r.calls, r.tests["x/pkg/b"] = nil, true
r.needs = []delivery.PackageNeeds{{Package: "x/pkg/a", Needs: []string{delivery.NeedNodeBuildNode}}}
steps := r.steps()
if has {
steps.Has = func(n string) bool { return n == delivery.NeedNodeBuildNode }
}
out, err := RunAsk(context.Background(), steps, ask(delivery.CheckJob{Levels: []string{delivery.LevelUnit}, Kept: true,
Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}}))
if err != nil {
t.Fatal(err)
}
u := out.(delivery.CheckAnswer).Levels[0]
tested := strings.Contains(strings.Join(r.calls, "\n"), `"x/pkg/a"`)
if tested != has || (len(u.Unmet) == 0) == !has || (!has && (u.Unmet[0].Package != "x/pkg/a" || delivery.UnmetNeeds([]delivery.LevelAnswer{u})[0] != delivery.NeedNodeBuildNode)) {
t.Fatalf("has %v: tested x/pkg/a %v, unmet %+v, calls %v", has, tested, u.Unmet, r.calls)
}
}
// A placed ask naming its packages tests those alone and takes no other's kept verdict, also when the level would
// run whole (ADR 0552 §4).
for _, digests := range []map[string]string{r.digests, nil} {
r.calls, r.digests = nil, digests
steps := r.steps()
steps.Has = func(string) bool { return true }
out, err := RunAsk(context.Background(), steps, ask(delivery.CheckJob{Levels: []string{delivery.LevelUnit}, Kept: true, Packages: []string{"x/pkg/a"},
Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}}))
if err != nil {
t.Fatal(err)
}
u := out.(delivery.CheckAnswer).Levels[0]
if last := r.calls[len(r.calls)-1]; !strings.HasSuffix(last, `"packages":["x/pkg/a"],"workspace":"/w"}`) || len(u.Kept) != 0 || len(u.Tests) != 1 {
t.Fatalf("digests %v: calls %v, kept %+v: want x/pkg/a alone tested, nothing taken kept", digests != nil, r.calls, u.Kept)
}
}
}
// ADR 0535 §2, §4: a merge that changes the build module's own code is judged by the candidate's runner: its digest says
@@ -203,3 +253,42 @@ func TestRunAskChecksNodeModulesAndConflicts(t *testing.T) {
t.Fatalf("a conflict: %v %v", err, r.calls)
}
}
// Issue 1921: a failure in the packages whose tests changed does not stop the level: the other packages still run, so two
// failing packages, one changed and one not, are two test results of the one level, and the level does not pass. A Node
// check runs every module directory after one fails too.
func TestRunAskRunsEveryPackageAfterAFailure(t *testing.T) {
r := &recorder{
fetch: map[string]any{"workspace": "/w", "changed": []string{"pkg/b/b_test.go"}},
digests: map[string]string{"x/pkg/a": "d-a", "x/pkg/b": "d-b"},
tests: map[string]bool{"x/pkg/a": false, "x/pkg/b": false},
}
out, err := RunAsk(context.Background(), r.steps(), ask(delivery.CheckJob{Levels: []string{delivery.LevelIntegration, delivery.LevelUnit},
Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}}))
if err != nil {
t.Fatal(err)
}
a := out.(delivery.CheckAnswer)
if len(a.Levels) != 1 || len(a.Levels[0].Tests) != 2 {
t.Fatalf("levels %+v, want the one level with both packages' results", a.Levels)
}
for i, raw := range a.Levels[0].Tests {
var res struct{ Passed bool }
if err := json.Unmarshal(raw, &res); err != nil || res.Passed {
t.Errorf("result %d %s, want a failure", i, raw)
}
}
r = &recorder{fetch: map[string]any{"workspace": "/w", "modules": []string{"a", "b"}, "changed": []string{"b/test/x.test.ts"}},
tests: map[string]bool{"a": true, "b": false}}
s := r.steps()
s.Digest = nil
out, err = RunAsk(context.Background(), s, ask(delivery.CheckJob{Levels: []string{delivery.LevelUnit, delivery.LevelIntegration},
Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}}))
if err != nil {
t.Fatal(err)
}
if a := out.(delivery.CheckAnswer); len(r.calls) != 3 || len(a.Levels) != 1 || len(a.Levels[0].Tests) != 2 {
t.Fatalf("calls %v, levels %+v: want both directories of the unit level and no integration level", r.calls, a.Levels)
}
}
+129
View File
@@ -0,0 +1,129 @@
package buildstep
import (
"encoding/json"
"fmt"
"os"
"strings"
"time"
)
// **A node may build only within its build hours** (the operator, 2026-10-26 21:18: the laptop builds only outside
// office hours, "from 18 o'clock until 6 in the morning"). A build module's `build-hours` setting is the node's window
// in its local time, a config value set on each node's layer of mesh-config (ADR 0567 §12: no repository names a node);
// outside it the capacity judged by is 0: steps running finish, no new one is taken. An empty value builds at any hour.
// The puller judges it at every judgement and wakes on one timer to the window's next edge (a clock edge, not a poll of
// another part).
// BuildHoursSetting is the build modules' setting of the node's build window: HH:MM-HH:MM in the node's local time; an
// end before its start crosses midnight (18:00-06:00).
const BuildHoursSetting = "build-hours"
// BuildWindow is one node's build hours: minutes after local midnight, Start inclusive, End exclusive.
type BuildWindow struct {
Start, End int
}
// String is the window as the setting writes it.
func (w BuildWindow) String() string {
return fmt.Sprintf("%02d:%02d-%02d:%02d", w.Start/60, w.Start%60, w.End/60, w.End%60)
}
// Open says whether t, in its own location, falls in the window.
func (w BuildWindow) Open(t time.Time) bool {
m := t.Hour()*60 + t.Minute()
if w.Start < w.End {
return m >= w.Start && m < w.End
}
return m >= w.Start || m < w.End
}
// NextEdge is the first time after t, in t's location, at which the window opens or closes.
func (w BuildWindow) NextEdge(t time.Time) time.Time {
var next time.Time
for day := 0; day <= 1; day++ {
for _, m := range []int{w.Start, w.End} {
e := time.Date(t.Year(), t.Month(), t.Day()+day, m/60, m%60, 0, 0, t.Location())
if e.After(t) && (next.IsZero() || e.Before(next)) {
next = e
}
}
}
return next
}
// ParseBuildWindow reads a build-hours value, HH:MM-HH:MM: nil when it is empty.
func ParseBuildWindow(setting string) (*BuildWindow, error) {
setting = strings.TrimSpace(setting)
if setting == "" {
return nil, nil
}
from, to, ok := strings.Cut(setting, "-")
if !ok {
return nil, fmt.Errorf("the build hours %q are not HH:MM-HH:MM", setting)
}
start, err := minuteOf(from)
if err != nil {
return nil, fmt.Errorf("the build hours %q: %v", setting, err)
}
end, err := minuteOf(to)
if err != nil {
return nil, fmt.Errorf("the build hours %q: %v", setting, err)
}
if start == end {
return nil, fmt.Errorf("the build hours %q open and close at the same minute", setting)
}
return &BuildWindow{Start: start, End: end}, nil
}
// minuteOf reads HH:MM as minutes after midnight.
func minuteOf(s string) (int, error) {
var h, m int
s = strings.TrimSpace(s)
if n, err := fmt.Sscanf(s, "%d:%d", &h, &m); err != nil || n != 2 || h < 0 || h > 23 || m < 0 || m > 59 || len(s) != 5 {
return 0, fmt.Errorf("%q is not a time HH:MM", s)
}
return h*60 + m, nil
}
// NodeBuildHours is the node's build window, read from the module's settings file at path at every call, and why it is
// none when the value cannot be read. No path, no file value or an empty one is no window and no fault. A value that
// cannot be read is no window too, so the node builds at any hour; the setting is a preference of hours, not a safety
// bound (a node that must never build has capacity 0). It is loud: the fault is said once per distinct fault through
// logf, and the puller writes it in the capacity record that build_usage shows.
func NodeBuildHours(path, node string, logf func(string, ...any)) func() (*BuildWindow, string) {
said := ""
say := func(why string) {
if logf != nil && why != said {
said = why
logf("%s, so %s builds at any hour", why, node)
}
}
fault := func(why string) (*BuildWindow, string) {
say(why)
return nil, why
}
return func() (*BuildWindow, string) {
if path == "" {
return nil, ""
}
raw, err := os.ReadFile(path)
if err != nil {
return fault(fmt.Sprintf("the setting %s cannot be read: %v", BuildHoursSetting, err))
}
var s map[string]json.RawMessage
if err := json.Unmarshal(raw, &s); err != nil {
return fault(fmt.Sprintf("the settings file %s is not JSON: %v", path, err))
}
setting := ""
if v, ok := s[BuildHoursSetting]; ok && json.Unmarshal(v, &setting) != nil {
return fault(fmt.Sprintf("the setting %s %s is not a string", BuildHoursSetting, v))
}
w, err := ParseBuildWindow(setting)
if err != nil {
return fault(err.Error())
}
said = ""
return w, ""
}
}
+31 -34
View File
@@ -4,45 +4,38 @@ import (
"encoding/json"
"fmt"
"os"
"git.novox.be/novox/mesh-sdk/go/delivery/buildrun"
)
// CapacitySetting is the build modules' setting of each node's build capacity (novox/hq ADR 0525 §5): `<node>=<asks at
// once>` pairs separated by commas, as the builder's was (ADR 0485 §1). Each build module declares it with
// DefaultCapacities as its default, and the controller lays it into the module's settings file.
// CapacitySetting is the build modules' setting of the node's build capacity (novox/hq ADR 0525 §5, ADR 0567 §12): how
// many asks the node takes at once, an integer config value set on each node's layer of mesh-config. No repository names
// a node: each build module declares it with the default 0, which pulls nothing, and the controller (or mesh-config)
// lays the node's value into the module's settings file.
const CapacitySetting = "capacity"
// DefaultCapacities is the setting's default, the builder's before R4g: the control-node (novox) and the laptop (g14)
// build nothing.
const DefaultCapacities = "shanks=3,ace=2,g14=0,novox=0"
// NodeCapacity is node's build capacity, read from the module's settings file at path at every call, so a change takes
// effect at the next pull; with no path it reads fallback. A node the setting does not name runs one ask at a time (as
// the builder's did). A file or a setting that cannot be read gives 0 and says why: pulling on a setting nobody can read
// could build on a node that must not.
func NodeCapacity(path, fallback, node string, logf func(string, ...any)) func() int {
// NodeCapacity is the node's build capacity, read from the module's settings file at path at every call, so a change
// takes effect at the next pull. No path, no file value, or a value that is not a whole number of at least 0 gives 0 and
// a value that cannot be read says why: pulling on a setting nobody can read could build on a node that must not.
func NodeCapacity(path, node string, logf func(string, ...any)) func() int {
say := func(format string, args ...any) {
if logf != nil {
logf(format, args...)
}
}
return func() int {
setting := fallback
if path != "" {
raw, err := os.ReadFile(path)
if err != nil {
say("the build capacity cannot be read, so %s pulls nothing: %v", node, err)
return 0
}
var s map[string]any
if err := json.Unmarshal(raw, &s); err != nil {
say("the settings file %s is not JSON, so %s pulls nothing: %v", path, node, err)
return 0
}
setting, _ = s[CapacitySetting].(string)
if path == "" {
return 0
}
n, err := capacityOf(setting, node)
raw, err := os.ReadFile(path)
if err != nil {
say("the build capacity cannot be read, so %s pulls nothing: %v", node, err)
return 0
}
var s map[string]json.RawMessage
if err := json.Unmarshal(raw, &s); err != nil {
say("the settings file %s is not JSON, so %s pulls nothing: %v", path, node, err)
return 0
}
n, err := capacityOf(s[CapacitySetting])
if err != nil {
say("%v, so %s pulls nothing", err, node)
return 0
@@ -51,11 +44,15 @@ func NodeCapacity(path, fallback, node string, logf func(string, ...any)) func()
}
}
// capacityOf is node's capacity in a setting's `<node>=<asks>` pairs: DefaultCapacity for a node it does not name.
func capacityOf(setting, node string) (int, error) {
caps, err := buildrun.ParseCapacity(setting)
if err != nil {
return 0, fmt.Errorf("the build capacity: %w", err)
// capacityOf is the capacity a config value holds: absent or null is 0; anything but a whole number of at least 0 is
// refused, naming it.
func capacityOf(value json.RawMessage) (int, error) {
if len(value) == 0 || string(value) == "null" {
return 0, nil
}
return (&buildrun.Places{Capacity: caps}).CapacityOf(node), nil
var n int
if err := json.Unmarshal(value, &n); err != nil || n < 0 {
return 0, fmt.Errorf("the build capacity %s is not a whole number of asks at once, 0 or more", value)
}
return n, nil
}
+141
View File
@@ -0,0 +1,141 @@
package buildstep
import (
"encoding/json"
"fmt"
"math"
"os"
"runtime"
"strconv"
"strings"
"git.novox.be/novox/mesh-sdk/go/buildcpu"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// **A build seat's capacity follows the CPU its node gives builds** (issue 1806's follow-up; issue 1890, 2026-10-26
// 19:41: under the 50 % cap the workstation's mesh-build-go kept capacity 3, three full mesh-catalog checks shared 12 cores, and
// each step ran past the 30-minute step bound with no test run). The configured capacity is an upper bound; the node's
// build slice bounds it again by the cores a step needs: floor(the slice's quota in cores / cores-per-step), one at
// least. The quota is read live from the cgroup at every judgement, so a changed cap is followed at the next take
// without a restart, and the result is said (the seat's current verb, the status tools, `capacity.<node>` in the pauses
// state that the delivery view's build_usage reads).
// CoresPerStepSetting is a build module's setting of the processors one of its steps needs: a number above zero, the
// same on every node (a step's need is the module's, not the node's). Each build module declares it with its own default
// (DefaultCoresPerStep).
const CoresPerStepSetting = "cores-per-step"
// DefaultCoresPerStep are the build modules' defaults by language, from measured steps (2026-10-26): a full mesh-catalog
// check on mesh-build-go takes what it is given (three at once took 965 % of the workstation's 1200 % slice, about 4 cores each,
// and ran past the 30-minute step bound), so a Go step is given 6, half again what starved; a Node step (tsc, esbuild, a
// test runner: median 15 s) and an image build (buildah, its RUN lines mostly fetches) need 2.
var DefaultCoresPerStep = map[string]float64{delivery.AskGo: 6, delivery.AskNode: 2, delivery.AskImage: 2}
// Quota is the build slice's CPU quota as this process's cgroup holds it, in percent of one processor (1200 is 12
// cores), and where it was read: the cgroup's cpu.max (buildcpu.ReadInForce, the tighter of its unit's and its slice's),
// else the share the composer gave (buildcpu.ShareWord) over the machine's processors, else 0 and why none is known.
func Quota() (percent int, from string) {
f := buildcpu.ReadInForce()
if f.Quota > 0 {
path := f.Slice.Path
if path == "" {
path = f.Cgroup
}
return f.Quota, "the cpu.max of " + path
}
if n, err := strconv.Atoi(strings.TrimSpace(os.Getenv(buildcpu.ShareWord))); err == nil && n >= 1 && n <= 100 {
return buildcpu.Quota(n, runtime.NumCPU()), fmt.Sprintf("the share %d%% the composer gave (%s), of %d processors; no cpu.max is read", n, buildcpu.ShareWord, runtime.NumCPU())
}
return 0, "no CPU quota: neither a cpu.max on its cgroup nor a share from the composer"
}
// EffectiveCapacity is the capacity a seat's holder judges by, from its configured capacity, its node's quota (percent of
// one processor, 0 when none is known) and the cores one of its steps needs. Without a quota it is the configured one;
// with one, min(configured, floor(quota cores / coresPerStep)), and one at least while the configured one is above
// zero. A coresPerStep of zero or less is taken as 1.
func EffectiveCapacity(configured, quotaPercent int, quotaFrom string, coresPerStep float64) delivery.Capacity {
if coresPerStep <= 0 || math.IsNaN(coresPerStep) || math.IsInf(coresPerStep, 0) {
coresPerStep = 1
}
c := delivery.Capacity{Configured: configured, Effective: configured, QuotaFrom: quotaFrom, CoresPerStep: coresPerStep}
switch {
case configured <= 0:
c.Effective = 0
c.Why = fmt.Sprintf("its configured capacity is %d, so it pulls nothing", configured)
return c
case quotaPercent <= 0:
c.Why = fmt.Sprintf("its configured capacity %d; %s, so the CPU bounds nothing", configured, quotaFrom)
return c
}
c.QuotaCores = float64(quotaPercent) / 100
c.ByCPU = int(math.Floor(c.QuotaCores/coresPerStep + 1e-9))
if c.ByCPU < 1 {
c.ByCPU = 1
}
byCPU := fmt.Sprintf("%g cores of the build slice (%s) / %g cores a step = %d", c.QuotaCores, quotaFrom, coresPerStep, c.ByCPU)
if c.ByCPU < configured {
c.Effective = c.ByCPU
c.Why = fmt.Sprintf("%d, bounded by the CPU: %s, under its configured %d", c.Effective, byCPU, configured)
} else {
c.Why = fmt.Sprintf("%d, its configured capacity; the CPU allows %s", configured, byCPU)
}
return c
}
// NodeCoresPerStep is the module's cores-per-step, read from its settings file at path at every call so a change is
// followed at the next judgement; fallback when there is no path, no value or one that is not a number above zero, said
// once per distinct fault through logf.
func NodeCoresPerStep(path string, fallback float64, logf func(string, ...any)) func() float64 {
said := ""
say := func(why string) {
if logf != nil && why != said {
said = why
logf("%s, so a step is taken to need %g cores", why, fallback)
}
}
return func() float64 {
if path == "" {
return fallback
}
raw, err := os.ReadFile(path)
if err != nil {
say(fmt.Sprintf("the setting %s cannot be read: %v", CoresPerStepSetting, err))
return fallback
}
var s map[string]json.RawMessage
if err := json.Unmarshal(raw, &s); err != nil {
say(fmt.Sprintf("the settings file %s is not JSON: %v", path, err))
return fallback
}
v, ok := s[CoresPerStepSetting]
if !ok {
return fallback // not declared yet (a module before its manifest names it): the default, nothing to say
}
n, err := coresOf(v)
if err != nil {
say(fmt.Sprintf("the setting %s %s: %v", CoresPerStepSetting, v, err))
return fallback
}
said = ""
return n
}
}
// coresOf reads a cores-per-step value: a JSON number or a string holding one, above zero.
func coresOf(v json.RawMessage) (float64, error) {
var n float64
if err := json.Unmarshal(v, &n); err != nil {
var s string
if err := json.Unmarshal(v, &s); err != nil {
return 0, fmt.Errorf("is neither a number nor a string")
}
if n, err = strconv.ParseFloat(strings.TrimSpace(s), 64); err != nil {
return 0, fmt.Errorf("is not a number")
}
}
if n <= 0 || math.IsNaN(n) || math.IsInf(n, 0) {
return 0, fmt.Errorf("is not above zero")
}
return n, nil
}
+119 -22
View File
@@ -3,36 +3,133 @@ package buildstep
import (
"os"
"path/filepath"
"strings"
"testing"
"time"
)
// 1887/A7: the node's build capacity is the build module's setting, read at every pull: the control-node at 0 pulls
// nothing, a change takes effect at the next read, and a setting that cannot be read pulls nothing.
func TestNodeCapacity(t *testing.T) {
if got := NodeCapacity("", DefaultCapacities, "novox", nil)(); got != 0 {
t.Errorf("novox = %d by default, want 0", got)
}
if got := NodeCapacity("", DefaultCapacities, "shanks", nil)(); got != 3 {
t.Errorf("shanks = %d by default, want 3", got)
// ADR 0553 §4: a step's threads are the composer's share over the module's own capacity; without the word, none. And
// issue 1806's follow-up: the capacity judged by is the configured one bounded by the build slice's cores over the cores
// a step needs (the three build nodes as they stand on 2026-10-26), and cores-per-step is read from the settings file.
func TestNodeCapacityAndLimits(t *testing.T) {
// ADR 0567 §12: the capacity is the node's integer, its default 0, and no default names a node.
if got := NodeCapacity("", "laptop", nil)(); got != 0 {
t.Errorf("no settings file = %d, want 0", got)
}
file := filepath.Join(t.TempDir(), "settings.json")
os.WriteFile(file, []byte(`{"capacity":"shanks=1,ace=2"}`), 0o600)
capacity := NodeCapacity(file, DefaultCapacities, "shanks", nil)
if got := capacity(); got != 1 {
t.Errorf("shanks = %d from the file, want 1", got)
capacity := NodeCapacity(file, "anchor", nil)
for value, want := range map[string]int{`{"capacity":1}`: 1, `{"capacity":4}`: 4, `{}`: 0, `{"capacity":null}`: 0,
`{"capacity":"anchor=3"}`: 0, `{"capacity":-1}`: 0, `{"capacity":1.5}`: 0} {
os.WriteFile(file, []byte(value), 0o600)
if got := capacity(); got != want {
t.Errorf("%s = %d, want %d", value, got, want)
}
}
os.WriteFile(file, []byte(`{"capacity":"shanks=4"}`), 0o600)
if got := capacity(); got != 4 {
t.Errorf("shanks = %d after the change, want 4", got)
}
if got := NodeCapacity(file, "", "pi", nil)(); got != 1 {
t.Errorf("a node not named = %d, want 1", got)
}
os.WriteFile(file, []byte(`{"capacity":"shanks=lots"}`), 0o600)
if got := capacity(); got != 0 {
t.Errorf("an unreadable setting = %d, want 0", got)
}
if got := NodeCapacity(filepath.Join(t.TempDir(), "none"), DefaultCapacities, "shanks", nil)(); got != 0 {
if got := NodeCapacity(filepath.Join(t.TempDir(), "none"), "anchor", nil)(); got != 0 {
t.Errorf("a missing file = %d, want 0", got)
}
t.Setenv("MESH_BUILD_CPU_SHARE", "")
if l := NodeLimits(3); l.Threads != 0 || l.Slots != 3 {
t.Errorf("without the word: %+v", l)
}
t.Setenv("MESH_BUILD_CPU_SHARE", "75")
if l := NodeLimits(3); l.Percent != 75 || l.Slots != 3 || l.Threads < 1 {
t.Errorf("from the word: %+v", l)
}
for _, c := range []struct {
name string
configured, quota int
cores float64
effective, byCPU int
}{
{"shanks: 12 cores, configured 3", 3, 1200, 6, 2, 2},
{"ace: 18 cores, configured 2", 2, 1800, 6, 2, 3},
{"g14: 8 cores, configured 2", 2, 800, 6, 1, 1},
{"node on shanks: 12 cores at 2", 3, 1200, 2, 3, 6},
{"no quota: the configured one", 3, 0, 6, 3, 0},
{"configured 0 pulls nothing", 0, 1200, 6, 0, 0},
{"a quota under one step still builds one", 2, 100, 6, 1, 1},
{"no cores a step is one", 2, 150, 0, 1, 1},
} {
got := EffectiveCapacity(c.configured, c.quota, "test", c.cores)
if got.Effective != c.effective || got.ByCPU != c.byCPU || got.Why == "" {
t.Errorf("%s: %+v, want effective %d, by CPU %d", c.name, got, c.effective, c.byCPU)
}
}
file = filepath.Join(t.TempDir(), "settings.json")
var said []string
cores := NodeCoresPerStep(file, 6, func(f string, a ...any) { said = append(said, f) })
for _, c := range []struct {
file string
want float64
}{{`{"capacity":2}`, 6}, {`{"cores-per-step":4}`, 4}, {`{"cores-per-step":"3.5"}`, 3.5}, {`{"cores-per-step":0}`, 6}, {`{"cores-per-step":0}`, 6}} {
os.WriteFile(file, []byte(c.file), 0o600)
if got := cores(); got != c.want {
t.Errorf("cores-per-step from %s = %g, want %g", c.file, got, c.want)
}
}
if len(said) != 1 {
t.Errorf("a value not above zero is said once, said %d times", len(said))
}
// The operator, 2026-10-26 21:18: g14 builds only from 18:00 to 06:00, its local time. Outside its build hours the
// capacity judged by is 0, said with the window and when it opens; a node not named, or a bad value, builds at any hour.
// And a node its module proves cannot build (ace without /dev/net/tun, 2026-10-28) judges by 0 at every hour, saying
// why, so its asks go to a node that can.
cet := time.FixedZone("CET", 3600)
day := func(hh, mm int) time.Time { return time.Date(2026, 10, 26, hh, mm, 0, 0, cet) }
was := now
defer func() { now = was }()
for _, c := range []struct {
name, setting, node string
at time.Time
effective int
edge string // the window's next edge, HH:MM; "" for no window
bad bool
ineligible string // what the module's Eligible says; "" for eligible
}{
{"the laptop in the office", "18:00-06:00", "laptop", day(10, 0), 0, "18:00", false, ""},
{"the laptop in the evening", "18:00-06:00", "laptop", day(18, 0), 2, "06:00", false, ""},
{"the laptop past midnight", "18:00-06:00", "laptop", day(5, 59), 2, "06:00", false, ""},
{"the laptop at the close", "18:00-06:00", "laptop", day(6, 0), 0, "18:00", false, ""},
{"a day window", "09:00-17:30", "anchor", day(17, 29), 2, "17:30", false, ""},
{"no window", "", "anchor", day(10, 0), 2, "", false, ""},
{"a bad time", "18:00-6", "laptop", day(10, 0), 2, "", true, ""},
{"no span", "18:00-18:00", "laptop", day(10, 0), 2, "", true, ""},
{"the old form naming a node", "laptop=18:00-06:00", "laptop", day(10, 0), 2, "", true, ""},
{"a node without /dev/net/tun", "", "anchor", day(10, 0), 0, "", false, "/dev/net/tun cannot be opened: no such device"},
{"not eligible within its hours", "18:00-06:00", "laptop", day(19, 0), 0, "06:00", false, "/dev/net/tun cannot be opened: no such device"},
} {
w, err := ParseBuildWindow(c.setting)
if (err != nil) != c.bad {
t.Errorf("%s: parse %v", c.name, err)
}
if edge := ""; w != nil {
edge = w.NextEdge(c.at).Format("15:04")
if edge != c.edge {
t.Errorf("%s: next edge %s, want %s", c.name, edge, c.edge)
}
} else if c.edge != "" {
t.Errorf("%s: no window, want one", c.name)
}
file := filepath.Join(t.TempDir(), "settings.json")
os.WriteFile(file, []byte(`{"build-hours":"`+c.setting+`"}`), 0o600)
now = func() time.Time { return c.at }
p := &Puller{Language: "go", Node: c.node, Capacity: func() int { return 2 }, BuildHours: NodeBuildHours(file, c.node, nil),
Eligible: func() string { return c.ineligible }}
got := p.CapacityDetail()
outside := c.effective == 0 && c.ineligible == ""
if got.Effective != c.effective || got.OutsideHours != outside || (outside && !strings.Contains(got.Why, "outside its build hours 18:00-06:00 CET until 18:00")) {
t.Errorf("%s: %+v, want capacity %d", c.name, got, c.effective)
}
if got.Ineligible != c.ineligible || (c.ineligible != "" && !strings.HasPrefix(got.Why, "0, not eligible: "+c.ineligible+" (otherwise ")) {
t.Errorf("%s: not eligible is not said: %+v", c.name, got)
}
// A value that cannot be read builds at any hour, loudly: the record names the value and says why.
if c.bad && (!strings.Contains(got.BuildHoursFault, c.setting) || !strings.Contains(got.Why, "at any hour, because its build hours cannot be read")) {
t.Errorf("%s: a bad value is not said in the record: %+v", c.name, got)
}
}
}
@@ -0,0 +1,73 @@
# The conformance kit's captured runner output
Every file here is the real output of a test runner, never written by hand (issue 1901: a fixture that agrees with the
mistake proves the mistake). The toolchains are the build nodes' own, as `mesh-build-go.build_go_toolchains` and
`mesh-build-node.build_node_toolchains` reported them on 2026-10-26: go1.27.3-X:nodwarf5 on every build node, node
v26.12.1 (npm 12.2.1) on shanks, novox and g14, and typescript 5.9.3 from mesh-build-node's toolchain/package.json.
To capture a case again, make its tiny project in a fresh directory as below and run the command shown; the kit's case
table (conformance.go) says what each failure must say.
## go/ — `go test -json`
- `passfirst.jsonl` and `subtests.jsonl` (2026-10-26, go1.27.3-X:nodwarf5, for issue 1901's thorough fix): in
`/tmp/1901k/go/<case>`, `GOTOOLCHAIN=local GOFLAGS=-mod=mod go test -json -count=1 ./... > ../<case>.jsonl 2>&1`.
- passfirst: module `example.com/passfirst`, `go 1.26`. Package `a` has `TestTheFirstPackagePasses` (empty). Package `b`
has `TestTheSecondPackagePassesFirst` (empty) and `TestTheSecondPackageFails`, which calls
`t.Logf("judged against the seat as it stands")` then `t.Fatalf("the seat has 2 holders, want 1")`: a passing package
and a passing test before the failing one.
- subtests: module `example.com/subtests`; `TestTheTable` runs `t.Run("passes", func(t *testing.T) {})` and
`t.Run("fails", func(t *testing.T) { t.Errorf("got 3 machines, want 4") })`: the parent fails only by its subtest.
- `subpanic.jsonl` (2026-10-26, go1.27.3-X:nodwarf5, review round 1 of issue 1901's fix), the same way: module
`example.com/subpanic`; `TestTable` runs `t.Run("passes", func(t *testing.T) {})` and `t.Run("panics", ...)`, which
writes to a nil map. go test -json files the subtest's panic under the parent `TestTable`.
- `logthenfatal.jsonl`, `panicking`, `timeout`, `deeptimeout`, `buildfail`, `exiting`, `killed`, `passthenfail`,
`mainexit`, `race`, `racemain` were captured for issue 1901's first and third meetings (2026-10-23 and 2026-10-25,
go1.27.3) in mesh-catalog's mesh-build-go (`testdata/captured/go-test-json-*.jsonl`, named there `go-test-json-pkg-<case>`
and `go-test-json-log-then-fatal`) and moved here unchanged. From that README: each is the standard output and error of
`go test -json -count=1 ./...` (the timeouts with `-timeout 2s`; race and racemain with `-race`, CGO_ENABLED=1) in a
throwaway one-file module that fails without a test's own assertion: panicking (a nil map write in TestPanics), timeout
(TestWaitsForever sleeps past the timeout), deeptimeout (two parallel tests and sixty blocked goroutines past the
timeout, so the goroutine dump pushes the panic line out of the last 4 KB), buildfail (a type error in a test file),
exiting (os.Exit(3) in TestExits), killed (TestKilled sends itself SIGKILL), passthenfail (TestMain panics in a
goroutine after m.Run passed), mainexit (TestMain exits 2 after m.Run passed: go test prints "PASS" and then
"FAIL <pkg>"), race (TestRacesOnTheSeat writes a package variable from itself and a goroutine), racemain (TestMain does
so after m.Run passed). logthenfatal is a test `TestThree` in package `x` that logs a line with t.Logf and then fails
with a multi-line t.Fatalf naming three subscriptions.
## node/ — `npm test` running `node --test` under the TAP reporter
Captured 2026-10-26 with node v26.12.1 and npm 12.2.1 (`/usr/bin/node`, `/usr/bin/npm`), in `/tmp/1901k/node/<case>`, the
way mesh-build-node runs a test script:
PATH=/usr/bin:$PATH NODE_OPTIONS="--test-reporter=tap --test-reporter-destination=/tmp/1901k/node/<case>.tap" \
npm test > ../<case>.stdout 2> ../<case>.stderr
`<case>.tap` is the report file, `<case>.console` the stdout followed by the stderr. Each project's package.json is
`{"name":"<case>","private":true,"type":"module","scripts":{"test":"node --test --experimental-strip-types 'test/*.test.ts'"}}`
(the script every TypeScript package of the catalogue and the SDK runs), with these test files:
- two-files: `test/a.test.ts` holds `test('passes in the first file', () => {})`; `test/b.test.ts` holds
`test('passes in the second file first', () => {})` and `test('fails in the second file', ...)`, which logs
`judged against the seat as it stands` and asserts `assert.strictEqual(1 + 1, 3, 'one plus one is not three')`.
- suite: `test/a.test.ts` has `describe('the seat', ...)` with `it('passes inside', () => {})` and `it('fails inside', ...)`
asserting `assert.deepStrictEqual({ holders: 2 }, { holders: 1 })`.
- crash: `test/a.test.ts` holds `test('passes before the crash', () => {})`; `test/b.test.ts` holds
`test('throws after it returned', ...)`, which throws `new Error('the bus closed under the test')` in a `setTimeout` of
1 ms, and `test('waits after the throw', ...)`, which waits 50 ms: node fails the file with an uncaught exception.
- logcrash (review round 1): `test/a.test.ts` holds `test('passes and logs', ...)`, which logs
`retrying after Error: the store was not ready yet`; `test/b.test.ts` is crash's. A passing test's console line
holding "Error" comes before the crash's own.
- timeout: `test/a.test.ts` holds `test('passes before the wait', () => {})` and
`test('waits for an answer that never comes', { timeout: 200 }, async () => { await new Promise(() => {}); })`.
- syntax: `test/a.test.ts` holds `test('passes in the first file', () => {})`; `test/b.test.ts` is
`import test from 'node:test';` and `test('never loads', () => {` (cut off): the file does not load.
## tsc/ — the type-check
`typeerror.console` (2026-10-26, typescript 5.9.3 under node v26.12.1): in `/tmp/1901k/tsc/typeerror`, `ok.ts` is
`export const holders: number = 1;` and `seat.ts` is `import { holders } from './ok.js'; export const name: string = holders;`;
the command is mesh-build-node's type-check line without a tsconfig.json:
node <typescript>/bin/tsc --noEmit --module NodeNext --moduleResolution NodeNext --target ES2022 --rootDir . ok.ts seat.ts
and the file is its stdout followed by its stderr.
@@ -0,0 +1,6 @@
{"ImportPath":"example.com/buildfail [example.com/buildfail.test]","Action":"build-output","Output":"# example.com/buildfail [example.com/buildfail.test]\n"}
{"ImportPath":"example.com/buildfail [example.com/buildfail.test]","Action":"build-output","Output":"./p_test.go:5:40: cannot use \"seat\" (untyped string constant) as int value in variable declaration\n"}
{"ImportPath":"example.com/buildfail [example.com/buildfail.test]","Action":"build-fail"}
{"Time":"2026-10-25T15:34:33.261137737+01:00","Action":"start","Package":"example.com/buildfail"}
{"Time":"2026-10-25T15:34:33.261255019+01:00","Action":"output","Package":"example.com/buildfail","Output":"FAIL\texample.com/buildfail [build failed]\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.261268099+01:00","Action":"fail","Package":"example.com/buildfail","Elapsed":0,"FailedBuild":"example.com/buildfail [example.com/buildfail.test]"}
@@ -0,0 +1,417 @@
{"Time":"2026-10-25T15:35:18.991630993+01:00","Action":"start","Package":"example.com/deeptimeout"}
{"Time":"2026-10-25T15:35:18.993895188+01:00","Action":"run","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat"}
{"Time":"2026-10-25T15:35:18.993963909+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":"=== RUN TestHoldsTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.99399484+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":"=== PAUSE TestHoldsTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.9939982+01:00","Action":"pause","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat"}
{"Time":"2026-10-25T15:35:18.99400173+01:00","Action":"run","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus"}
{"Time":"2026-10-25T15:35:18.99400456+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"=== RUN TestWaitsForTheBus\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.99400838+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"=== PAUSE TestWaitsForTheBus\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.99401133+01:00","Action":"pause","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus"}
{"Time":"2026-10-25T15:35:18.99401441+01:00","Action":"cont","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat"}
{"Time":"2026-10-25T15:35:18.99401704+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":"=== CONT TestHoldsTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.994074141+01:00","Action":"cont","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus"}
{"Time":"2026-10-25T15:35:18.994102612+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"=== CONT TestWaitsForTheBus\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:21.00935195+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"panic: test timed out after 2s\n"}
{"Time":"2026-10-25T15:35:21.00938383+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\trunning tests:\n"}
{"Time":"2026-10-25T15:35:21.009394391+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t\tTestHoldsTheSeat (2s)\n"}
{"Time":"2026-10-25T15:35:21.009398021+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t\tTestWaitsForTheBus (2s)\n"}
{"Time":"2026-10-25T15:35:21.009401801+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009406481+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 81 [running]:\n"}
{"Time":"2026-10-25T15:35:21.009409911+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.(*M).startAlarm.func1()\n"}
{"Time":"2026-10-25T15:35:21.009413361+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2961 +0x34a\n"}
{"Time":"2026-10-25T15:35:21.009418341+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by time.goFunc\n"}
{"Time":"2026-10-25T15:35:21.009421781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/time/sleep.go:182 +0x2d\n"}
{"Time":"2026-10-25T15:35:21.009424991+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009428341+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 1 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009431721+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner.func1()\n"}
{"Time":"2026-10-25T15:35:21.009441811+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2144 +0x425\n"}
{"Time":"2026-10-25T15:35:21.009445511+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner(0x282e9702008, 0x282e9646bc8)\n"}
{"Time":"2026-10-25T15:35:21.009449641+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2201 +0x123\n"}
{"Time":"2026-10-25T15:35:21.009453471+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.runTests({0x55a45d, 0x17}, {0x55a45d, 0x17}, 0x282e95940f0, {0x6f19f0, 0x2, 0x2}, {0xc2afa2ca3b3c0c8d, 0x7737f168, ...})\n"}
{"Time":"2026-10-25T15:35:21.009462092+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2742 +0x510\n"}
{"Time":"2026-10-25T15:35:21.009467222+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.(*M).Run(0x282e96581e0)\n"}
{"Time":"2026-10-25T15:35:21.009470662+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2602 +0x6af\n"}
{"Time":"2026-10-25T15:35:21.009473692+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"main.main()\n"}
{"Time":"2026-10-25T15:35:21.009476942+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t_testmain.go:48 +0x9b\n"}
{"Time":"2026-10-25T15:35:21.009480092+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009483242+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 7 [sleep]:\n"}
{"Time":"2026-10-25T15:35:21.009486502+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"time.Sleep(0xdf8475800)\n"}
{"Time":"2026-10-25T15:35:21.009490682+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/runtime/time.go:368 +0x165\n"}
{"Time":"2026-10-25T15:35:21.009493812+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat(0x282e9702248?)\n"}
{"Time":"2026-10-25T15:35:21.009498842+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:14 +0x9e\n"}
{"Time":"2026-10-25T15:35:21.009502242+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner(0x282e9702248, 0x6d5e78)\n"}
{"Time":"2026-10-25T15:35:21.009505742+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:35:21.009509292+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:35:21.009512752+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:35:21.009515792+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009518623+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 8 [sleep]:\n"}
{"Time":"2026-10-25T15:35:21.009521303+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"time.Sleep(0xdf8475800)\n"}
{"Time":"2026-10-25T15:35:21.009524423+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/runtime/time.go:368 +0x165\n"}
{"Time":"2026-10-25T15:35:21.009531403+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestWaitsForTheBus(0x282e9702488?)\n"}
{"Time":"2026-10-25T15:35:21.009534943+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:19 +0x25\n"}
{"Time":"2026-10-25T15:35:21.009538403+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner(0x282e9702488, 0x6d5e80)\n"}
{"Time":"2026-10-25T15:35:21.009541953+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:35:21.009545233+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:35:21.009548323+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:35:21.009551343+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009554263+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 9 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009557523+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009561063+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009564513+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009571113+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009574233+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009577433+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 10 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009580823+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009584174+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009587504+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009590974+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009594004+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009597184+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 11 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009600394+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009603874+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009610954+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009614714+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009618104+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009621364+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 12 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009624704+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009628324+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009631954+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009635404+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009638734+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009641904+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 13 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009645325+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009648645+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009652065+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009655625+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009658885+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009663385+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 14 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009666735+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009670235+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009676385+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009680205+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009683315+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009686475+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 15 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009689785+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009697235+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009700795+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009704985+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009708085+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009711346+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 16 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009714586+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009717946+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009721676+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009725626+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009728866+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009732106+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 17 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009735376+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009738956+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009742326+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009746436+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009750666+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009754126+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 18 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009758286+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009761846+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009765776+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009769816+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009773167+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009780507+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 19 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009784207+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009788007+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009791627+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009796347+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009799427+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009802937+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 20 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009806667+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009809987+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009813827+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009818087+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009821397+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009824227+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 21 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009827607+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009831307+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009835167+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009839528+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009842728+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009845888+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 22 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009849168+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009852548+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009855958+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009863648+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009866938+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009870088+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 23 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009879828+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009883728+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009887608+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009891548+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009895238+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009898748+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 24 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009902349+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009907109+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009911519+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009915609+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009918939+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009922329+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 25 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009928329+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009931909+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009935489+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009939429+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009942699+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009946019+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 26 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009949559+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009953489+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.00996244+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.00996647+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.00996987+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.00997314+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 27 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.00997664+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.00998035+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.00998418+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.00998793+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.00999131+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.00999534+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 28 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.00999894+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01000268+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.0100087+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01001258+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01001605+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01002007+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 29 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01002393+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010027531+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010031351+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010035521+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010038781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010042281+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 30 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010046091+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010054261+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010057881+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010061781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010064931+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010068261+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 31 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010072151+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010075981+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010079841+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010098802+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010103202+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010106382+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 32 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010109852+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010113242+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010116952+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010120672+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010124112+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010127162+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 33 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010130662+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010134442+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010138142+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010141972+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010145362+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010152262+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 34 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010156353+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010159913+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010163533+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010167403+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010170533+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010173953+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 35 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010177453+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010181033+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010184693+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010188583+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010191753+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010195033+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 36 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010198603+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010202393+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010206123+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010210583+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010213503+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010216914+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 37 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010220134+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010223644+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010227154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010235154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010238354+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010241754+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 38 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010270284+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010275144+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010278714+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010282825+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010285795+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010288935+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 39 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010292405+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010295915+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010299545+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010311835+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010315565+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010318975+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 40 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010322655+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010326225+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010329815+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010333745+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010337095+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010340515+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 41 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010344066+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010347706+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010355596+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010359046+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010362036+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010365076+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 42 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010367956+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010370906+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010375366+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010378386+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010381006+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010383666+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 43 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010386676+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010389786+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010392636+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010395746+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010398466+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010401166+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 44 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010404226+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010407186+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010410117+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010413237+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010416157+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010418897+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 45 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010423157+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010430107+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010433197+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010451207+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010454217+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010456967+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 46 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010459877+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010463967+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010467067+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010470597+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010473418+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010476108+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 47 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010478998+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010481958+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010484978+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010488258+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010490888+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010493628+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 48 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010496588+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010499498+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010502498+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010505768+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010508558+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010514148+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 49 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010517178+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010520238+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010523478+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010526818+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010529718+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010532508+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 50 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010535639+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010538749+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010541939+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010546689+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010549869+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010552989+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 51 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010556299+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010560079+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010563479+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010567189+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010570299+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010573079+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 52 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010576319+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010579599+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010583099+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010590519+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010594029+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010596989+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 53 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01059986+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01060433+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.01060789+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01061136+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01061393+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01061667+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 54 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01061984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.0106227+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.01062566+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01062876+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01063131+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01063428+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 55 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01063719+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01063997+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.01064278+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01064582+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01064836+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01065114+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 56 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01065437+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01065724+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010671481+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010675671+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010678431+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010691171+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 57 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010694531+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010697641+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010700781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010704001+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010706611+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010709421+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 58 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010712331+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010715351+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010718431+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010721691+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010724371+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010727362+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 59 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010730592+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010733662+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010736712+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010740162+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010742922+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010745772+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 60 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010752222+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010755402+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010758542+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010761822+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010764592+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010767992+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 61 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010771062+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010774282+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010779382+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010782942+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010785722+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010788573+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 62 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010791893+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010794943+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010798083+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010801683+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010804293+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010807283+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 63 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010810443+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010813553+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010816793+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010820233+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010823063+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010829493+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 64 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010832683+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010835953+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010839223+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010842463+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010845333+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010848423+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 65 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010851663+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010854964+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010858364+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010861714+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010864914+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010867984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 66 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010870944+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010874114+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010877464+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010880504+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010883144+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010886194+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 67 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010889154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010892234+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010895524+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010902154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010904984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010907984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 68 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010911064+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010914074+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010917365+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010920505+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.013209581+01:00","Action":"output","Package":"example.com/deeptimeout","Output":"FAIL\texample.com/deeptimeout\t2.021s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:21.013258851+01:00","Action":"fail","Package":"example.com/deeptimeout","Elapsed":2.022}
@@ -0,0 +1,10 @@
{"Time":"2026-10-25T15:34:33.768742837+01:00","Action":"start","Package":"example.com/exiting"}
{"Time":"2026-10-25T15:34:33.773822287+01:00","Action":"run","Package":"example.com/exiting","Test":"TestFine"}
{"Time":"2026-10-25T15:34:33.773896608+01:00","Action":"output","Package":"example.com/exiting","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.773940319+01:00","Action":"output","Package":"example.com/exiting","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.773945359+01:00","Action":"pass","Package":"example.com/exiting","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:34:33.773952529+01:00","Action":"run","Package":"example.com/exiting","Test":"TestExits"}
{"Time":"2026-10-25T15:34:33.773956439+01:00","Action":"output","Package":"example.com/exiting","Test":"TestExits","Output":"=== RUN TestExits\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.773959829+01:00","Action":"output","Package":"example.com/exiting","Test":"TestExits","Output":" p_test.go:11: closing the store\n"}
{"Time":"2026-10-25T15:34:33.775158538+01:00","Action":"output","Package":"example.com/exiting","Output":"FAIL\texample.com/exiting\t0.006s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.775186509+01:00","Action":"fail","Package":"example.com/exiting","Elapsed":0.006}
@@ -0,0 +1,7 @@
{"Time":"2026-10-25T15:34:34.328258934+01:00","Action":"start","Package":"example.com/killed"}
{"Time":"2026-10-25T15:34:34.331465435+01:00","Action":"run","Package":"example.com/killed","Test":"TestKilled"}
{"Time":"2026-10-25T15:34:34.331699248+01:00","Action":"output","Package":"example.com/killed","Test":"TestKilled","Output":"=== RUN TestKilled\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.331853111+01:00","Action":"output","Package":"example.com/killed","Test":"TestKilled","Output":" p_test.go:10: the kernel is about to kill this\n"}
{"Time":"2026-10-25T15:34:34.334563213+01:00","Action":"output","Package":"example.com/killed","Test":"TestKilled","Output":"signal: killed\n"}
{"Time":"2026-10-25T15:34:34.334611114+01:00","Action":"output","Package":"example.com/killed","Output":"FAIL\texample.com/killed\t0.006s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.334627134+01:00","Action":"fail","Package":"example.com/killed","Elapsed":0.006}
@@ -0,0 +1,13 @@
{"Action":"start","Package":"x"}
{"Action":"run","Package":"x","Test":"TestThree"}
{"Action":"output","Package":"x","Test":"TestThree","Output":"=== RUN TestThree\n","OutputType":"frame"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" a_test.go:4: judged against /var/lib/mesh-build-go-work/id/beside/mesh-catalog as cloned\n"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" a_test.go:5: 3 subscription(s) derive to a subject no emitter owns:\n","OutputType":"error"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" a.b.c\n","OutputType":"error-continue"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" d.e.f\n","OutputType":"error-continue"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" g.h.i\n","OutputType":"error-continue"}
{"Action":"output","Package":"x","Test":"TestThree","Output":"--- FAIL: TestThree (0.00s)\n","OutputType":"frame"}
{"Action":"fail","Package":"x","Test":"TestThree","Elapsed":0}
{"Action":"output","Package":"x","Output":"FAIL\n","OutputType":"frame"}
{"Action":"output","Package":"x","Output":"FAIL\tx\t0.002s\n","OutputType":"frame"}
{"Action":"fail","Package":"x","Elapsed":0.002}
@@ -0,0 +1,8 @@
{"Time":"2026-10-25T15:34:35.274953775+01:00","Action":"start","Package":"example.com/mainexit"}
{"Time":"2026-10-25T15:34:35.278129565+01:00","Action":"run","Package":"example.com/mainexit","Test":"TestFine"}
{"Time":"2026-10-25T15:34:35.278191516+01:00","Action":"output","Package":"example.com/mainexit","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.278341979+01:00","Action":"output","Package":"example.com/mainexit","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.278351739+01:00","Action":"pass","Package":"example.com/mainexit","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:34:35.278359039+01:00","Action":"output","Package":"example.com/mainexit","Output":"PASS\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.279516487+01:00","Action":"output","Package":"example.com/mainexit","Output":"FAIL\texample.com/mainexit\t0.004s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.279539298+01:00","Action":"fail","Package":"example.com/mainexit","Elapsed":0.005}
@@ -0,0 +1,27 @@
{"Time":"2026-10-25T15:34:33.031091436+01:00","Action":"start","Package":"example.com/panicking"}
{"Time":"2026-10-25T15:34:33.036353279+01:00","Action":"run","Package":"example.com/panicking","Test":"TestBefore"}
{"Time":"2026-10-25T15:34:33.036441781+01:00","Action":"output","Package":"example.com/panicking","Test":"TestBefore","Output":"=== RUN TestBefore\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.036568163+01:00","Action":"output","Package":"example.com/panicking","Test":"TestBefore","Output":"--- PASS: TestBefore (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.036609863+01:00","Action":"pass","Package":"example.com/panicking","Test":"TestBefore","Elapsed":0}
{"Time":"2026-10-25T15:34:33.036641304+01:00","Action":"run","Package":"example.com/panicking","Test":"TestPanics"}
{"Time":"2026-10-25T15:34:33.036645914+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"=== RUN TestPanics\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.036698995+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":" p_test.go:8: about to look up the seat\n"}
{"Time":"2026-10-25T15:34:33.036773996+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"--- FAIL: TestPanics (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.03897418+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"panic: assignment to entry in nil map [recovered, repanicked]\n"}
{"Time":"2026-10-25T15:34:33.040332172+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\n"}
{"Time":"2026-10-25T15:34:33.040414493+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"goroutine 8 [running]:\n"}
{"Time":"2026-10-25T15:34:33.040521475+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"testing.tRunner.func1.2({0x6b8120, 0x6eff80})\n"}
{"Time":"2026-10-25T15:34:33.040601826+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2125 +0x232\n"}
{"Time":"2026-10-25T15:34:33.040641947+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"testing.tRunner.func1()\n"}
{"Time":"2026-10-25T15:34:33.040720648+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2128 +0x329\n"}
{"Time":"2026-10-25T15:34:33.04083286+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"panic({0x6b8120?, 0x6eff80?})\n"}
{"Time":"2026-10-25T15:34:33.040912921+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/runtime/panic.go:859 +0x125\n"}
{"Time":"2026-10-25T15:34:33.040981652+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"example.com/panicking.TestPanics(0x228226cd0488?)\n"}
{"Time":"2026-10-25T15:34:33.041059903+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/tmp/fix-1901b/panicking/p_test.go:10 +0x53\n"}
{"Time":"2026-10-25T15:34:33.041132704+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"testing.tRunner(0x228226cd0488, 0x6d5bc8)\n"}
{"Time":"2026-10-25T15:34:33.041215586+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:34:33.041221966+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:34:33.041225556+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:34:33.041984088+01:00","Action":"fail","Package":"example.com/panicking","Test":"TestPanics","Elapsed":0}
{"Time":"2026-10-25T15:34:33.041996638+01:00","Action":"output","Package":"example.com/panicking","Output":"FAIL\texample.com/panicking\t0.010s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.042044079+01:00","Action":"fail","Package":"example.com/panicking","Elapsed":0.011}
@@ -0,0 +1,22 @@
{"Time":"2026-10-26T19:40:29.574068887+01:00","Action":"start","Package":"example.com/passfirst/a"}
{"Time":"2026-10-26T19:40:29.574347185+01:00","Action":"start","Package":"example.com/passfirst/b"}
{"Time":"2026-10-26T19:40:29.576503518+01:00","Action":"run","Package":"example.com/passfirst/a","Test":"TestTheFirstPackagePasses"}
{"Time":"2026-10-26T19:40:29.576557657+01:00","Action":"output","Package":"example.com/passfirst/a","Test":"TestTheFirstPackagePasses","Output":"=== RUN TestTheFirstPackagePasses\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.576589367+01:00","Action":"output","Package":"example.com/passfirst/a","Test":"TestTheFirstPackagePasses","Output":"--- PASS: TestTheFirstPackagePasses (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.576594637+01:00","Action":"pass","Package":"example.com/passfirst/a","Test":"TestTheFirstPackagePasses","Elapsed":0}
{"Time":"2026-10-26T19:40:29.576602487+01:00","Action":"output","Package":"example.com/passfirst/a","Output":"PASS\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.576614477+01:00","Action":"run","Package":"example.com/passfirst/b","Test":"TestTheSecondPackagePassesFirst"}
{"Time":"2026-10-26T19:40:29.576626897+01:00","Action":"output","Package":"example.com/passfirst/b","Test":"TestTheSecondPackagePassesFirst","Output":"=== RUN TestTheSecondPackagePassesFirst\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.576631187+01:00","Action":"output","Package":"example.com/passfirst/b","Test":"TestTheSecondPackagePassesFirst","Output":"--- PASS: TestTheSecondPackagePassesFirst (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.576635567+01:00","Action":"pass","Package":"example.com/passfirst/b","Test":"TestTheSecondPackagePassesFirst","Elapsed":0}
{"Time":"2026-10-26T19:40:29.576638427+01:00","Action":"run","Package":"example.com/passfirst/b","Test":"TestTheSecondPackageFails"}
{"Time":"2026-10-26T19:40:29.576640647+01:00","Action":"output","Package":"example.com/passfirst/b","Test":"TestTheSecondPackageFails","Output":"=== RUN TestTheSecondPackageFails\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.576643477+01:00","Action":"output","Package":"example.com/passfirst/b","Test":"TestTheSecondPackageFails","Output":" b_test.go:8: judged against the seat as it stands\n"}
{"Time":"2026-10-26T19:40:29.576646497+01:00","Action":"output","Package":"example.com/passfirst/b","Test":"TestTheSecondPackageFails","Output":" b_test.go:9: the seat has 2 holders, want 1\n","OutputType":"error"}
{"Time":"2026-10-26T19:40:29.576650087+01:00","Action":"output","Package":"example.com/passfirst/b","Test":"TestTheSecondPackageFails","Output":"--- FAIL: TestTheSecondPackageFails (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.576652797+01:00","Action":"fail","Package":"example.com/passfirst/b","Test":"TestTheSecondPackageFails","Elapsed":0}
{"Time":"2026-10-26T19:40:29.576655177+01:00","Action":"output","Package":"example.com/passfirst/b","Output":"FAIL\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.577365541+01:00","Action":"output","Package":"example.com/passfirst/a","Output":"ok \texample.com/passfirst/a\t0.003s\n"}
{"Time":"2026-10-26T19:40:29.577382561+01:00","Action":"pass","Package":"example.com/passfirst/a","Elapsed":0.003}
{"Time":"2026-10-26T19:40:29.577380641+01:00","Action":"output","Package":"example.com/passfirst/b","Output":"FAIL\texample.com/passfirst/b\t0.003s\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.5774131+01:00","Action":"fail","Package":"example.com/passfirst/b","Elapsed":0.003}
@@ -0,0 +1,15 @@
{"Time":"2026-10-25T15:34:34.919624192+01:00","Action":"start","Package":"example.com/passthenfail"}
{"Time":"2026-10-25T15:34:34.922861224+01:00","Action":"run","Package":"example.com/passthenfail","Test":"TestFine"}
{"Time":"2026-10-25T15:34:34.922932965+01:00","Action":"output","Package":"example.com/passthenfail","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.922969205+01:00","Action":"output","Package":"example.com/passthenfail","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.922979956+01:00","Action":"pass","Package":"example.com/passthenfail","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:34:34.922989416+01:00","Action":"output","Package":"example.com/passthenfail","Output":"PASS\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.925022528+01:00","Action":"output","Package":"example.com/passthenfail","Output":"panic: the bus closed after the tests: nats: connection closed\n"}
{"Time":"2026-10-25T15:34:34.925043388+01:00","Action":"output","Package":"example.com/passthenfail","Output":"\n"}
{"Time":"2026-10-25T15:34:34.925046848+01:00","Action":"output","Package":"example.com/passthenfail","Output":"goroutine 20 [running]:\n"}
{"Time":"2026-10-25T15:34:34.925051338+01:00","Action":"output","Package":"example.com/passthenfail","Output":"example.com/passthenfail.TestMain.func1()\n"}
{"Time":"2026-10-25T15:34:34.925055878+01:00","Action":"output","Package":"example.com/passthenfail","Output":"\t/tmp/fix-1901b/passthenfail/p_test.go:13 +0x25\n"}
{"Time":"2026-10-25T15:34:34.925061428+01:00","Action":"output","Package":"example.com/passthenfail","Output":"created by example.com/passthenfail.TestMain in goroutine 1\n"}
{"Time":"2026-10-25T15:34:34.925065028+01:00","Action":"output","Package":"example.com/passthenfail","Output":"\t/tmp/fix-1901b/passthenfail/p_test.go:13 +0x25\n"}
{"Time":"2026-10-25T15:34:34.925390203+01:00","Action":"output","Package":"example.com/passthenfail","Output":"FAIL\texample.com/passthenfail\t0.005s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.925402904+01:00","Action":"fail","Package":"example.com/passthenfail","Elapsed":0.006}
@@ -0,0 +1,53 @@
{"Time":"2026-10-25T15:50:42.908557309+01:00","Action":"start","Package":"example.com/race"}
{"Time":"2026-10-25T15:50:42.924298679+01:00","Action":"run","Package":"example.com/race","Test":"TestBefore"}
{"Time":"2026-10-25T15:50:42.92438195+01:00","Action":"output","Package":"example.com/race","Test":"TestBefore","Output":"=== RUN TestBefore\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.924648842+01:00","Action":"output","Package":"example.com/race","Test":"TestBefore","Output":"--- PASS: TestBefore (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.924685133+01:00","Action":"pass","Package":"example.com/race","Test":"TestBefore","Elapsed":0}
{"Time":"2026-10-25T15:50:42.924728573+01:00","Action":"run","Package":"example.com/race","Test":"TestRacesOnTheSeat"}
{"Time":"2026-10-25T15:50:42.924733573+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"=== RUN TestRacesOnTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.92637298+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"==================\n"}
{"Time":"2026-10-25T15:50:42.92641567+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"WARNING: DATA RACE\n"}
{"Time":"2026-10-25T15:50:42.926444061+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Write at 0x000000833468 by goroutine 10:\n"}
{"Time":"2026-10-25T15:50:42.926462621+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" example.com/race.TestRacesOnTheSeat.func1()\n"}
{"Time":"2026-10-25T15:50:42.926467121+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /tmp/fix-1901b/race/p_test.go:13 +0x24\n"}
{"Time":"2026-10-25T15:50:42.926483061+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"\n"}
{"Time":"2026-10-25T15:50:42.926509741+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Previous write at 0x000000833468 by goroutine 9:\n"}
{"Time":"2026-10-25T15:50:42.926530142+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" example.com/race.TestRacesOnTheSeat()\n"}
{"Time":"2026-10-25T15:50:42.926534952+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /tmp/fix-1901b/race/p_test.go:14 +0x30\n"}
{"Time":"2026-10-25T15:50:42.926550462+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.tRunner()\n"}
{"Time":"2026-10-25T15:50:42.926554612+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2195 +0x21c\n"}
{"Time":"2026-10-25T15:50:42.926571142+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*T).Run.gowrap1()\n"}
{"Time":"2026-10-25T15:50:42.926575812+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2260 +0x38\n"}
{"Time":"2026-10-25T15:50:42.926592202+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"\n"}
{"Time":"2026-10-25T15:50:42.926607792+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Goroutine 10 (running) created at:\n"}
{"Time":"2026-10-25T15:50:42.926624292+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" example.com/race.TestRacesOnTheSeat()\n"}
{"Time":"2026-10-25T15:50:42.926628923+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /tmp/fix-1901b/race/p_test.go:13 +0x24\n"}
{"Time":"2026-10-25T15:50:42.926643853+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.tRunner()\n"}
{"Time":"2026-10-25T15:50:42.926648103+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2195 +0x21c\n"}
{"Time":"2026-10-25T15:50:42.926674843+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*T).Run.gowrap1()\n"}
{"Time":"2026-10-25T15:50:42.926680673+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2260 +0x38\n"}
{"Time":"2026-10-25T15:50:42.926695433+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"\n"}
{"Time":"2026-10-25T15:50:42.926711063+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Goroutine 9 (running) created at:\n"}
{"Time":"2026-10-25T15:50:42.926726614+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*T).Run()\n"}
{"Time":"2026-10-25T15:50:42.926730814+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2260 +0xb12\n"}
{"Time":"2026-10-25T15:50:42.926746484+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.runTests.func1()\n"}
{"Time":"2026-10-25T15:50:42.926750764+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2744 +0x84\n"}
{"Time":"2026-10-25T15:50:42.926765354+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.tRunner()\n"}
{"Time":"2026-10-25T15:50:42.926769844+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2195 +0x21c\n"}
{"Time":"2026-10-25T15:50:42.926791794+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.runTests()\n"}
{"Time":"2026-10-25T15:50:42.926796704+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2742 +0x9e9\n"}
{"Time":"2026-10-25T15:50:42.926811544+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*M).Run()\n"}
{"Time":"2026-10-25T15:50:42.926816694+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2602 +0xf44\n"}
{"Time":"2026-10-25T15:50:42.926831695+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" main.main()\n"}
{"Time":"2026-10-25T15:50:42.926835935+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" _testmain.go:50 +0x164\n"}
{"Time":"2026-10-25T15:50:42.926850905+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"==================\n"}
{"Time":"2026-10-25T15:50:42.935320241+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.go:1867: race detected during execution of test\n","OutputType":"error"}
{"Time":"2026-10-25T15:50:42.935520113+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"--- FAIL: TestRacesOnTheSeat (0.01s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.935592973+01:00","Action":"fail","Package":"example.com/race","Test":"TestRacesOnTheSeat","Elapsed":0.01}
{"Time":"2026-10-25T15:50:42.935722225+01:00","Action":"run","Package":"example.com/race","Test":"TestAfter"}
{"Time":"2026-10-25T15:50:42.935728845+01:00","Action":"output","Package":"example.com/race","Test":"TestAfter","Output":"=== RUN TestAfter\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.935995928+01:00","Action":"output","Package":"example.com/race","Test":"TestAfter","Output":"--- PASS: TestAfter (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.936006328+01:00","Action":"pass","Package":"example.com/race","Test":"TestAfter","Elapsed":0}
{"Time":"2026-10-25T15:50:42.936009978+01:00","Action":"output","Package":"example.com/race","Output":"FAIL\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.937730745+01:00","Action":"output","Package":"example.com/race","Output":"FAIL\texample.com/race\t0.028s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.937802066+01:00","Action":"fail","Package":"example.com/race","Elapsed":0.029}
@@ -0,0 +1,27 @@
{"Time":"2026-10-25T15:50:43.551701745+01:00","Action":"start","Package":"example.com/racemain"}
{"Time":"2026-10-25T15:50:43.570709708+01:00","Action":"run","Package":"example.com/racemain","Test":"TestFine"}
{"Time":"2026-10-25T15:50:43.570806059+01:00","Action":"output","Package":"example.com/racemain","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:43.57085525+01:00","Action":"output","Package":"example.com/racemain","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:43.57086149+01:00","Action":"pass","Package":"example.com/racemain","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:50:43.57086974+01:00","Action":"output","Package":"example.com/racemain","Output":"PASS\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:43.572155623+01:00","Action":"output","Package":"example.com/racemain","Output":"==================\n"}
{"Time":"2026-10-25T15:50:43.572168083+01:00","Action":"output","Package":"example.com/racemain","Output":"WARNING: DATA RACE\n"}
{"Time":"2026-10-25T15:50:43.572171933+01:00","Action":"output","Package":"example.com/racemain","Output":"Write at 0x000000836428 by goroutine 9:\n"}
{"Time":"2026-10-25T15:50:43.572176703+01:00","Action":"output","Package":"example.com/racemain","Output":" example.com/racemain.TestMain.func1()\n"}
{"Time":"2026-10-25T15:50:43.572180053+01:00","Action":"output","Package":"example.com/racemain","Output":" /tmp/fix-1901b/racemain/p_test.go:15 +0x24\n"}
{"Time":"2026-10-25T15:50:43.572184923+01:00","Action":"output","Package":"example.com/racemain","Output":"\n"}
{"Time":"2026-10-25T15:50:43.572189613+01:00","Action":"output","Package":"example.com/racemain","Output":"Previous write at 0x000000836428 by main goroutine:\n"}
{"Time":"2026-10-25T15:50:43.572192993+01:00","Action":"output","Package":"example.com/racemain","Output":" example.com/racemain.TestMain()\n"}
{"Time":"2026-10-25T15:50:43.572198573+01:00","Action":"output","Package":"example.com/racemain","Output":" /tmp/fix-1901b/racemain/p_test.go:16 +0x44\n"}
{"Time":"2026-10-25T15:50:43.572202433+01:00","Action":"output","Package":"example.com/racemain","Output":" main.main()\n"}
{"Time":"2026-10-25T15:50:43.572205623+01:00","Action":"output","Package":"example.com/racemain","Output":" _testmain.go:48 +0x171\n"}
{"Time":"2026-10-25T15:50:43.572208443+01:00","Action":"output","Package":"example.com/racemain","Output":"\n"}
{"Time":"2026-10-25T15:50:43.572211573+01:00","Action":"output","Package":"example.com/racemain","Output":"Goroutine 9 (running) created at:\n"}
{"Time":"2026-10-25T15:50:43.572214803+01:00","Action":"output","Package":"example.com/racemain","Output":" example.com/racemain.TestMain()\n"}
{"Time":"2026-10-25T15:50:43.572218043+01:00","Action":"output","Package":"example.com/racemain","Output":" /tmp/fix-1901b/racemain/p_test.go:15 +0x37\n"}
{"Time":"2026-10-25T15:50:43.572225414+01:00","Action":"output","Package":"example.com/racemain","Output":" main.main()\n"}
{"Time":"2026-10-25T15:50:43.572228654+01:00","Action":"output","Package":"example.com/racemain","Output":" _testmain.go:48 +0x171\n"}
{"Time":"2026-10-25T15:50:43.572231664+01:00","Action":"output","Package":"example.com/racemain","Output":"==================\n"}
{"Time":"2026-10-25T15:50:44.581262249+01:00","Action":"output","Package":"example.com/racemain","Output":"Found 1 data race(s)\n"}
{"Time":"2026-10-25T15:50:44.584693204+01:00","Action":"output","Package":"example.com/racemain","Output":"FAIL\texample.com/racemain\t1.031s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:44.584737425+01:00","Action":"fail","Package":"example.com/racemain","Elapsed":1.033}
@@ -0,0 +1,30 @@
{"Time":"2026-10-26T20:15:37.791161045+01:00","Action":"start","Package":"example.com/subpanic"}
{"Time":"2026-10-26T20:15:37.793205666+01:00","Action":"run","Package":"example.com/subpanic","Test":"TestTable"}
{"Time":"2026-10-26T20:15:37.793257158+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"=== RUN TestTable\n","OutputType":"frame"}
{"Time":"2026-10-26T20:15:37.793288149+01:00","Action":"run","Package":"example.com/subpanic","Test":"TestTable/passes"}
{"Time":"2026-10-26T20:15:37.793290669+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable/passes","Output":"=== RUN TestTable/passes\n","OutputType":"frame"}
{"Time":"2026-10-26T20:15:37.793298009+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable/passes","Output":"--- PASS: TestTable/passes (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T20:15:37.793311679+01:00","Action":"pass","Package":"example.com/subpanic","Test":"TestTable/passes","Elapsed":0}
{"Time":"2026-10-26T20:15:37.79331977+01:00","Action":"run","Package":"example.com/subpanic","Test":"TestTable/panics"}
{"Time":"2026-10-26T20:15:37.79332377+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable/panics","Output":"=== RUN TestTable/panics\n","OutputType":"frame"}
{"Time":"2026-10-26T20:15:37.79332773+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable/panics","Output":"--- FAIL: TestTable/panics (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T20:15:37.79333111+01:00","Action":"fail","Package":"example.com/subpanic","Test":"TestTable/panics","Elapsed":0}
{"Time":"2026-10-26T20:15:37.79333383+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"--- FAIL: TestTable (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T20:15:37.795322529+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"panic: assignment to entry in nil map [recovered, repanicked]\n"}
{"Time":"2026-10-26T20:15:37.795331489+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"\n"}
{"Time":"2026-10-26T20:15:37.795340579+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"goroutine 9 [running]:\n"}
{"Time":"2026-10-26T20:15:37.79534387+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"testing.tRunner.func1.2({0x6b7010, 0x6eef80})\n"}
{"Time":"2026-10-26T20:15:37.79534657+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"\t/usr/lib/go/src/testing/testing.go:2125 +0x232\n"}
{"Time":"2026-10-26T20:15:37.79534907+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"testing.tRunner.func1()\n"}
{"Time":"2026-10-26T20:15:37.79535172+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"\t/usr/lib/go/src/testing/testing.go:2128 +0x329\n"}
{"Time":"2026-10-26T20:15:37.79535625+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"panic({0x6b7010?, 0x6eef80?})\n"}
{"Time":"2026-10-26T20:15:37.79535879+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"\t/usr/lib/go/src/runtime/panic.go:859 +0x125\n"}
{"Time":"2026-10-26T20:15:37.79536129+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"example.com/subpanic.TestTable.func2(0x3b39436666c8?)\n"}
{"Time":"2026-10-26T20:15:37.79536452+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"\t/tmp/1901k/go/subpanic/p_test.go:7 +0x28\n"}
{"Time":"2026-10-26T20:15:37.79536782+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"testing.tRunner(0x3b39436666c8, 0x6d4b58)\n"}
{"Time":"2026-10-26T20:15:37.7953704+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-26T20:15:37.79537289+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"created by testing.(*T).Run in goroutine 7\n"}
{"Time":"2026-10-26T20:15:37.795381681+01:00","Action":"output","Package":"example.com/subpanic","Test":"TestTable","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-26T20:15:37.795731441+01:00","Action":"fail","Package":"example.com/subpanic","Test":"TestTable","Elapsed":0}
{"Time":"2026-10-26T20:15:37.795759332+01:00","Action":"output","Package":"example.com/subpanic","Output":"FAIL\texample.com/subpanic\t0.004s\n","OutputType":"frame"}
{"Time":"2026-10-26T20:15:37.795783503+01:00","Action":"fail","Package":"example.com/subpanic","Elapsed":0.005}
@@ -0,0 +1,17 @@
{"Time":"2026-10-26T19:40:29.768101399+01:00","Action":"start","Package":"example.com/subtests"}
{"Time":"2026-10-26T19:40:29.770109733+01:00","Action":"run","Package":"example.com/subtests","Test":"TestTheTable"}
{"Time":"2026-10-26T19:40:29.770152272+01:00","Action":"output","Package":"example.com/subtests","Test":"TestTheTable","Output":"=== RUN TestTheTable\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770189372+01:00","Action":"run","Package":"example.com/subtests","Test":"TestTheTable/passes"}
{"Time":"2026-10-26T19:40:29.770191882+01:00","Action":"output","Package":"example.com/subtests","Test":"TestTheTable/passes","Output":"=== RUN TestTheTable/passes\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770197442+01:00","Action":"output","Package":"example.com/subtests","Test":"TestTheTable/passes","Output":"--- PASS: TestTheTable/passes (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770204352+01:00","Action":"pass","Package":"example.com/subtests","Test":"TestTheTable/passes","Elapsed":0}
{"Time":"2026-10-26T19:40:29.770209622+01:00","Action":"run","Package":"example.com/subtests","Test":"TestTheTable/fails"}
{"Time":"2026-10-26T19:40:29.770211832+01:00","Action":"output","Package":"example.com/subtests","Test":"TestTheTable/fails","Output":"=== RUN TestTheTable/fails\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770214722+01:00","Action":"output","Package":"example.com/subtests","Test":"TestTheTable/fails","Output":" s_test.go:7: got 3 machines, want 4\n","OutputType":"error"}
{"Time":"2026-10-26T19:40:29.770218032+01:00","Action":"output","Package":"example.com/subtests","Test":"TestTheTable/fails","Output":"--- FAIL: TestTheTable/fails (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770220732+01:00","Action":"fail","Package":"example.com/subtests","Test":"TestTheTable/fails","Elapsed":0}
{"Time":"2026-10-26T19:40:29.770223522+01:00","Action":"output","Package":"example.com/subtests","Test":"TestTheTable","Output":"--- FAIL: TestTheTable (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770226292+01:00","Action":"fail","Package":"example.com/subtests","Test":"TestTheTable","Elapsed":0}
{"Time":"2026-10-26T19:40:29.770228582+01:00","Action":"output","Package":"example.com/subtests","Output":"FAIL\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770561979+01:00","Action":"output","Package":"example.com/subtests","Output":"FAIL\texample.com/subtests\t0.002s\n","OutputType":"frame"}
{"Time":"2026-10-26T19:40:29.770581629+01:00","Action":"fail","Package":"example.com/subtests","Elapsed":0.002}
@@ -0,0 +1,43 @@
{"Time":"2026-10-25T15:34:36.049580618+01:00","Action":"start","Package":"example.com/timeout"}
{"Time":"2026-10-25T15:34:36.053250986+01:00","Action":"run","Package":"example.com/timeout","Test":"TestQuick"}
{"Time":"2026-10-25T15:34:36.053354018+01:00","Action":"output","Package":"example.com/timeout","Test":"TestQuick","Output":"=== RUN TestQuick\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:36.053459789+01:00","Action":"output","Package":"example.com/timeout","Test":"TestQuick","Output":"--- PASS: TestQuick (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:36.05348499+01:00","Action":"pass","Package":"example.com/timeout","Test":"TestQuick","Elapsed":0}
{"Time":"2026-10-25T15:34:36.0535114+01:00","Action":"run","Package":"example.com/timeout","Test":"TestWaitsForever"}
{"Time":"2026-10-25T15:34:36.05351549+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"=== RUN TestWaitsForever\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:36.053567451+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":" p_test.go:11: waiting for the bus\n"}
{"Time":"2026-10-25T15:34:38.064693307+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"panic: test timed out after 2s\n"}
{"Time":"2026-10-25T15:34:38.06487122+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\trunning tests:\n"}
{"Time":"2026-10-25T15:34:38.06491684+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t\tTestWaitsForever (2s)\n"}
{"Time":"2026-10-25T15:34:38.065034382+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\n"}
{"Time":"2026-10-25T15:34:38.065108573+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"goroutine 33 [running]:\n"}
{"Time":"2026-10-25T15:34:38.065154754+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.(*M).startAlarm.func1()\n"}
{"Time":"2026-10-25T15:34:38.065386208+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2961 +0x34a\n"}
{"Time":"2026-10-25T15:34:38.065431878+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"created by time.goFunc\n"}
{"Time":"2026-10-25T15:34:38.0655151+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/time/sleep.go:182 +0x2d\n"}
{"Time":"2026-10-25T15:34:38.06553471+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\n"}
{"Time":"2026-10-25T15:34:38.065847805+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"goroutine 1 [chan receive]:\n"}
{"Time":"2026-10-25T15:34:38.066038298+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.(*T).Run(0x3a0209bc8008, {0x5580aa?, 0x3a0209a9caa0?}, 0x6d5b10)\n"}
{"Time":"2026-10-25T15:34:38.066047488+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2268 +0x4f2\n"}
{"Time":"2026-10-25T15:34:38.066051368+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.runTests.func1(0x3a0209bc8008)\n"}
{"Time":"2026-10-25T15:34:38.066056128+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2744 +0x37\n"}
{"Time":"2026-10-25T15:34:38.066059478+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.tRunner(0x3a0209bc8008, 0x3a0209a9cbc8)\n"}
{"Time":"2026-10-25T15:34:38.066064958+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:34:38.066069599+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.runTests({0x559017, 0x13}, {0x559017, 0x13}, 0x3a0209b8e0d8, {0x6f19f0, 0x2, 0x2}, {0xc2afa2bf832a3f56, 0x773908b5, ...})\n"}
{"Time":"2026-10-25T15:34:38.066083309+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2742 +0x510\n"}
{"Time":"2026-10-25T15:34:38.066086879+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.(*M).Run(0x3a0209b8c1e0)\n"}
{"Time":"2026-10-25T15:34:38.066090269+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2602 +0x6af\n"}
{"Time":"2026-10-25T15:34:38.066093429+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"main.main()\n"}
{"Time":"2026-10-25T15:34:38.066097569+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t_testmain.go:48 +0x9b\n"}
{"Time":"2026-10-25T15:34:38.066100639+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\n"}
{"Time":"2026-10-25T15:34:38.066103769+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"goroutine 20 [sleep]:\n"}
{"Time":"2026-10-25T15:34:38.066106969+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"time.Sleep(0xdf8475800)\n"}
{"Time":"2026-10-25T15:34:38.066110279+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/runtime/time.go:368 +0x165\n"}
{"Time":"2026-10-25T15:34:38.066113639+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"example.com/timeout.TestWaitsForever(0x3a0209bc8488?)\n"}
{"Time":"2026-10-25T15:34:38.066117109+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/tmp/fix-1901b/timeout/p_test.go:12 +0x48\n"}
{"Time":"2026-10-25T15:34:38.066120239+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.tRunner(0x3a0209bc8488, 0x6d5b10)\n"}
{"Time":"2026-10-25T15:34:38.066123509+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:34:38.066126839+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:34:38.066129989+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:34:38.066986453+01:00","Action":"output","Package":"example.com/timeout","Output":"FAIL\texample.com/timeout\t2.017s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:38.067002113+01:00","Action":"fail","Package":"example.com/timeout","Elapsed":2.017}
@@ -0,0 +1,2 @@
npm notice run test
npm notice run node --test --experimental-strip-types 'test/*.test.ts'
@@ -0,0 +1,41 @@
TAP version 13
# Subtest: passes before the crash
ok 1 - passes before the crash
---
duration_ms: 0.746694
type: 'test'
...
# Subtest: throws after it returned
ok 2 - throws after it returned
---
duration_ms: 0.763364
type: 'test'
...
# Subtest: waits after the throw
ok 3 - waits after the throw
---
duration_ms: 49.953893
type: 'test'
...
# Error: Test "throws after it returned" at test/b.test.ts:2:1 generated asynchronous activity after the test ended. This activity created the error "Error: the bus closed under the test" and would have caused the test to fail, but instead triggered an uncaughtException event.
# Subtest: test/b.test.ts
not ok 2 - test/b.test.ts
---
duration_ms: 155.864774
type: 'test'
location: '/tmp/1901k/node/crash/test/b.test.ts:1:1'
failureType: 'testCodeFailure'
exitCode: 1
signal: ~
error: 'test failed'
code: 'ERR_TEST_FAILURE'
...
1..4
# tests 4
# suites 0
# pass 3
# fail 1
# cancelled 0
# skipped 0
# todo 0
# duration_ms 168.479173
@@ -0,0 +1,2 @@
npm notice run test
npm notice run node --test --experimental-strip-types 'test/*.test.ts'
@@ -0,0 +1,42 @@
TAP version 13
# retrying after Error: the store was not ready yet
# Subtest: passes and logs
ok 1 - passes and logs
---
duration_ms: 1.184595
type: 'test'
...
# Subtest: throws after it returned
ok 2 - throws after it returned
---
duration_ms: 0.748862
type: 'test'
...
# Subtest: waits after the throw
ok 3 - waits after the throw
---
duration_ms: 51.019875
type: 'test'
...
# Error: Test "throws after it returned" at test/b.test.ts:2:1 generated asynchronous activity after the test ended. This activity created the error "Error: the bus closed under the test" and would have caused the test to fail, but instead triggered an uncaughtException event.
# Subtest: test/b.test.ts
not ok 2 - test/b.test.ts
---
duration_ms: 172.91715
type: 'test'
location: '/tmp/1901k/node/logcrash/test/b.test.ts:1:1'
failureType: 'testCodeFailure'
exitCode: 1
signal: ~
error: 'test failed'
code: 'ERR_TEST_FAILURE'
...
1..4
# tests 4
# suites 0
# pass 3
# fail 1
# cancelled 0
# skipped 0
# todo 0
# duration_ms 185.744334
@@ -0,0 +1,2 @@
npm notice run test
npm notice run node --test --experimental-strip-types 'test/*.test.ts'
@@ -0,0 +1,61 @@
TAP version 13
# Subtest: the seat
# Subtest: passes inside
ok 1 - passes inside
---
duration_ms: 0.577396
type: 'test'
...
# Subtest: fails inside
not ok 2 - fails inside
---
duration_ms: 1.621198
type: 'test'
location: '/tmp/1901k/node/suite/test/a.test.ts:5:3'
failureType: 'testCodeFailure'
error: |-
Expected values to be strictly deep-equal:
+ actual - expected
 {
+ holders: 2
- holders: 1
 }
code: 'ERR_ASSERTION'
name: 'AssertionError'
expected:
holders: 1
actual:
holders: 2
operator: 'deepStrictEqual'
stack: |-
TestContext.<anonymous> (file:///tmp/1901k/node/suite/test/a.test.ts:5:37)
Test.runInAsyncScope (node:async_hooks:226:14)
Test.run (node:internal/test_runner/test:1402:25)
Suite.processPendingSubtests (node:internal/test_runner/test:974:18)
Test.postRun (node:internal/test_runner/test:1542:19)
Test.run (node:internal/test_runner/test:1467:12)
async Promise.all (index 0)
async Suite.run (node:internal/test_runner/test:1905:7)
async startSubtestAfterBootstrap (node:internal/test_runner/harness:387:3)
...
1..2
not ok 1 - the seat
---
duration_ms: 3.295664
type: 'suite'
location: '/tmp/1901k/node/suite/test/a.test.ts:3:1'
failureType: 'subtestsFailed'
error: '1 subtest failed'
code: 'ERR_TEST_FAILURE'
...
1..1
# tests 2
# suites 1
# pass 1
# fail 1
# cancelled 0
# skipped 0
# todo 0
# duration_ms 115.948112
@@ -0,0 +1,2 @@
npm notice run test
npm notice run node --test --experimental-strip-types 'test/*.test.ts'
@@ -0,0 +1,47 @@
TAP version 13
# Subtest: passes in the first file
ok 1 - passes in the first file
---
duration_ms: 0.637465
type: 'test'
...
# node:internal/modules/run_main:111
# triggerUncaughtException(
# ^
# file:///tmp/1901k/node/syntax/test/b.test.ts:2
# test('never loads', () => {
# SyntaxError [ERR_INVALID_TYPESCRIPT_SYNTAX]: Expected ',', got '<eof>'
#  at parseTypeScript (node:internal/modules/typescript:61:36)
#  at processTypeScriptCode (node:internal/modules/typescript:129:37)
#  at Object.stripTypeScriptModuleTypes (node:internal/modules/typescript:172:22)
#  at stripTypeScriptModuleTypes (node:internal/modules/esm/translators:35:49)
#  at ModuleLoader.<anonymous> (node:internal/modules/esm/translators:688:29)
#  at \#translate (node:internal/modules/esm/loader:441:20)
#  at afterLoad (node:internal/modules/esm/loader:509:29)
#  at ModuleLoader.loadAndTranslate (node:internal/modules/esm/loader:514:12)
#  at \#getOrCreateModuleJobAfterResolve (node:internal/modules/esm/loader:567:36)
#  at afterResolve (node:internal/modules/esm/loader:614:52) {
# code: 'ERR_INVALID_TYPESCRIPT_SYNTAX'
# }
# Node.js v26.12.1
# Subtest: test/b.test.ts
not ok 2 - test/b.test.ts
---
duration_ms: 87.629815
type: 'test'
location: '/tmp/1901k/node/syntax/test/b.test.ts:1:1'
failureType: 'testCodeFailure'
exitCode: 1
signal: ~
error: 'test failed'
code: 'ERR_TEST_FAILURE'
...
1..2
# tests 2
# suites 0
# pass 1
# fail 1
# cancelled 0
# skipped 0
# todo 0
# duration_ms 106.551435
@@ -0,0 +1,2 @@
npm notice run test
npm notice run node --test --experimental-strip-types 'test/*.test.ts'
@@ -0,0 +1,26 @@
TAP version 13
# Subtest: passes before the wait
ok 1 - passes before the wait
---
duration_ms: 0.683015
type: 'test'
...
# Subtest: waits for an answer that never comes
not ok 2 - waits for an answer that never comes
---
duration_ms: 201.103775
type: 'test'
location: '/tmp/1901k/node/timeout/test/a.test.ts:3:1'
failureType: 'testTimeoutFailure'
error: 'test timed out after 200ms'
code: 'ERR_TEST_FAILURE'
...
1..2
# tests 2
# suites 0
# pass 1
# fail 0
# cancelled 1
# skipped 0
# todo 0
# duration_ms 308.920061
@@ -0,0 +1,2 @@
npm notice run test
npm notice run node --test --experimental-strip-types 'test/*.test.ts'
@@ -0,0 +1,49 @@
TAP version 13
# Subtest: passes in the first file
ok 1 - passes in the first file
---
duration_ms: 0.740124
type: 'test'
...
# judged against the seat as it stands
# Subtest: passes in the second file first
ok 2 - passes in the second file first
---
duration_ms: 0.916463
type: 'test'
...
# Subtest: fails in the second file
not ok 3 - fails in the second file
---
duration_ms: 1.719236
type: 'test'
location: '/tmp/1901k/node/two-files/test/b.test.ts:4:1'
failureType: 'testCodeFailure'
error: |-
one plus one is not three
2 !== 3
code: 'ERR_ASSERTION'
name: 'AssertionError'
expected: 3
actual: 2
operator: 'strictEqual'
stack: |-
TestContext.<anonymous> (file:///tmp/1901k/node/two-files/test/b.test.ts:6:10)
Test.runInAsyncScope (node:async_hooks:226:14)
Test.run (node:internal/test_runner/test:1402:25)
Test.processPendingSubtests (node:internal/test_runner/test:974:18)
Test.postRun (node:internal/test_runner/test:1542:19)
Test.run (node:internal/test_runner/test:1467:12)
async startSubtestAfterBootstrap (node:internal/test_runner/harness:387:3)
...
1..3
# tests 3
# suites 0
# pass 2
# fail 1
# cancelled 0
# skipped 0
# todo 0
# duration_ms 135.931121
@@ -0,0 +1 @@
seat.ts(2,14): error TS2322: Type 'number' is not assignable to type 'string'.
+201
View File
@@ -0,0 +1,201 @@
// Package conformance is the kit every build module's tests run (novox/hq issue 1901, its thorough fix): real output of
// the test runners, captured with the build nodes' own toolchains (captured/README.md), and what a build module must say
// of each failure in it under the failure contract (buildstep.Failure). A build module's test calls Run with its own parse
// path, the code that turns a runner's output into the failures of its test result; the kit fails the test when a
// failure is not named, its cause is missing or wrong, or a passing line stands in its place.
//
// A catalogue check (mesh-catalog checks/failurekit) refuses a build module with a test step whose tests do not call Run,
// and any build module whose tests do not call LeftRunning, the kit's case of steps a previous process left running
// (leftrunning.go, issue 1890).
package conformance
import (
"embed"
"fmt"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/buildstep"
)
// The runners the kit holds output of.
const (
// GoTest is `go test -json -count=1 ./...`: Output.Stdout is its output (go1.27.3).
GoTest = "go test -json"
// NodeTest is a package's `npm test` running `node --test --experimental-strip-types` with
// NODE_OPTIONS=buildstep.NodeTestOptions(<file>): Output.Report is the TAP file, Output.Stdout the script's own output
// and stderr, Output.Dir the package's directory as it was captured (node v26.12.1, npm 12.2.1).
NodeTest = "node --test"
// TypeCheck is tsc's type-check (`tsc --noEmit`, typescript 5.9.3): Output.Stdout is what it wrote; its failure
// names the test "type-check".
TypeCheck = "tsc"
)
// Output is one captured run of a runner, as a build module's parse path takes it.
type Output struct {
Stdout, Stderr string
// Report is what a runner wrote to a report file of its own (node --test's TAP), "" for the others.
Report string
// Dir is the directory the run was captured in: a node test's file is said relative to it.
Dir string
}
// Parse is a build module's own parse path: the failures its test result would carry for one run that failed.
type Parse func(Output) []buildstep.Failure
// Case is one captured run and what its failure must say.
type Case struct {
Name string
Runner string
// Test is the failing test the failure names (for a package that stopped, the tests that were running; "" for none).
Test string
// File, when set, is where the failing test is.
File string
// Cause holds what the failure's cause must contain.
Cause []string
// Passing are the names of tests that passed in the run, and what they logged: no failure's words may hold them.
Passing []string
}
//go:embed captured
var captured embed.FS
// Cases are the kit's cases of a runner.
func Cases(runner string) []Case {
var out []Case
for _, c := range cases {
if c.Runner == runner {
out = append(out, c)
}
}
return out
}
var cases = []Case{
{Name: "passfirst", Runner: GoTest, Test: "TestTheSecondPackageFails", Cause: []string{"the seat has 2 holders, want 1"},
Passing: []string{"TestTheFirstPackagePasses", "TestTheSecondPackagePassesFirst"}},
{Name: "subtests", Runner: GoTest, Test: "TestTheTable/fails", Cause: []string{"got 3 machines, want 4"}, Passing: []string{"TestTheTable/passes"}},
{Name: "logthenfatal", Runner: GoTest, Test: "TestThree", Cause: []string{"3 subscription(s) derive to a subject no emitter owns:"}},
{Name: "panicking", Runner: GoTest, Test: "TestPanics", Cause: []string{"panic: assignment to entry in nil map"}},
{Name: "timeout", Runner: GoTest, Test: "TestWaitsForever", Cause: []string{"panic: test timed out after 2s", "TestWaitsForever"}},
{Name: "deeptimeout", Runner: GoTest, Test: "TestHoldsTheSeat, TestWaitsForTheBus", Cause: []string{"panic: test timed out after 2s", "TestHoldsTheSeat", "TestWaitsForTheBus"}},
{Name: "buildfail", Runner: GoTest, Cause: []string{`cannot use "seat" (untyped string constant) as int value`}},
{Name: "exiting", Runner: GoTest, Test: "TestExits", Cause: []string{"exited", "TestExits"}},
{Name: "killed", Runner: GoTest, Test: "TestKilled", Cause: []string{"signal: killed"}},
{Name: "passthenfail", Runner: GoTest, Cause: []string{"panic: the bus closed after the tests: nats: connection closed"}},
{Name: "mainexit", Runner: GoTest, Cause: []string{"passed", "exited"}},
{Name: "race", Runner: GoTest, Test: "TestRacesOnTheSeat", Cause: []string{"DATA RACE", "Write at", "example.com/race.TestRacesOnTheSeat.func1()", "p_test.go:13", "created at example.com/race.TestRacesOnTheSeat()"}},
{Name: "subpanic", Runner: GoTest, Test: "TestTable/panics", Cause: []string{"panic: assignment to entry in nil map"}, Passing: []string{"TestTable/passes"}},
{Name: "racemain", Runner: GoTest, Cause: []string{"DATA RACE", "Write at", "example.com/racemain.TestMain.func1()", "p_test.go:15", "created at example.com/racemain.TestMain()"}},
{Name: "two-files", Runner: NodeTest, Test: "fails in the second file", File: "test/b.test.ts:4:1", Cause: []string{"one plus one is not three"},
Passing: []string{"passes in the first file", "passes in the second file first"}},
{Name: "suite", Runner: NodeTest, Test: "fails inside", File: "test/a.test.ts:5:3", Cause: []string{"Expected values to be strictly deep-equal:"},
Passing: []string{"passes inside"}},
{Name: "crash", Runner: NodeTest, Test: "test/b.test.ts", File: "test/b.test.ts:1:1", Cause: []string{"the bus closed under the test"},
Passing: []string{"passes before the crash"}},
{Name: "logcrash", Runner: NodeTest, Test: "test/b.test.ts", File: "test/b.test.ts:1:1", Cause: []string{"the bus closed under the test"},
Passing: []string{"passes and logs", "retrying after Error: the store was not ready yet"}},
{Name: "timeout", Runner: NodeTest, Test: "waits for an answer that never comes", File: "test/a.test.ts:3:1", Cause: []string{"test timed out after 200ms"},
Passing: []string{"passes before the wait"}},
{Name: "syntax", Runner: NodeTest, Test: "test/b.test.ts", File: "test/b.test.ts:1:1", Cause: []string{"SyntaxError", "Expected ',', got '<eof>'"},
Passing: []string{"passes in the first file"}},
{Name: "typeerror", Runner: TypeCheck, Test: "type-check", Cause: []string{"seat.ts(2,14): error TS2322: Type 'number' is not assignable to type 'string'."}},
}
// Load is the captured output of a case.
func Load(c Case) (Output, error) {
switch c.Runner {
case GoTest:
b, err := captured.ReadFile("captured/go/" + c.Name + ".jsonl")
return Output{Stdout: string(b)}, err
case NodeTest:
report, err := captured.ReadFile("captured/node/" + c.Name + ".tap")
if err != nil {
return Output{}, err
}
console, err := captured.ReadFile("captured/node/" + c.Name + ".console")
return Output{Report: string(report), Stdout: string(console), Dir: "/tmp/1901k/node/" + c.Name}, err
default:
b, err := captured.ReadFile("captured/tsc/" + c.Name + ".console")
return Output{Stdout: string(b), Dir: "/tmp/1901k/tsc/" + c.Name}, err
}
}
// Run runs every case of runner through parse, each as a subtest, and fails a case whose failure does not keep the
// contract: exactly one failure, naming the failing test (and its file, where the runner says it), with a cause that
// holds what the case says and is no passing line, an output that starts with the cause, stays within its bound and
// holds no passing line and no passing test's name.
func Run(t *testing.T, runner string, parse Parse) {
t.Helper()
cs := Cases(runner)
if len(cs) == 0 {
t.Fatalf("the kit holds no case of the runner %q", runner)
}
for _, c := range cs {
t.Run(c.Name, func(t *testing.T) {
out, err := Load(c)
if err != nil {
t.Fatal(err)
}
for _, problem := range Judge(c, parse(out)) {
t.Error(problem)
}
})
}
}
// Judge says what is wrong with the failures a parse path made of a case; nothing when they keep the contract.
func Judge(c Case, failed []buildstep.Failure) []string {
var problems []string
say := func(s string) { problems = append(problems, c.Runner+" "+c.Name+": "+s) }
if len(failed) != 1 {
say(fmt.Sprintf("want one failure, got %d: %s", len(failed), describe(failed)))
if len(failed) == 0 {
return problems
}
}
f := failed[0]
if f.Test != c.Test {
say("the failure names the test " + quote(f.Test) + ", want " + quote(c.Test))
}
if c.File != "" && f.File != c.File {
say("the failure's file is " + quote(f.File) + ", want " + quote(c.File))
}
if why := buildstep.Hidden(f); why != "" {
say("the failure hides its cause: " + why)
}
for _, want := range c.Cause {
if !strings.Contains(f.Cause, want) {
say("the cause " + quote(f.Cause) + " does not hold " + quote(want))
}
}
if first := strings.SplitN(f.Output, "\n", 2)[0]; strings.TrimSpace(first) != strings.TrimSpace(f.Cause) {
say("the output starts with " + quote(first) + ", not with the cause")
}
if len(f.Output) > buildstep.OutputBytes+buildstep.LineBytes+64 {
say(fmt.Sprintf("the output holds %d bytes, over its bound", len(f.Output)))
}
for _, l := range strings.Split(f.Output, "\n") {
if buildstep.PassingLine(l) {
say("the output keeps the passing line " + quote(l))
}
}
for _, p := range c.Passing {
if strings.Contains(f.Output, p+"\n") || strings.HasSuffix(f.Output, p) || strings.Contains(f.Cause, p) || strings.Contains(f.Output, "- "+p) {
say("the failure's words hold the passing test " + quote(p))
}
}
return problems
}
func describe(failed []buildstep.Failure) string {
var s []string
for _, f := range failed {
s = append(s, quote(f.Package+" "+f.Test)+" ("+quote(f.Cause)+")")
}
return strings.Join(s, ", ")
}
func quote(s string) string { return fmt.Sprintf("%q", s) }
@@ -0,0 +1,167 @@
package conformance
import (
"context"
"encoding/json"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/buildstep"
)
// The failure contract (issue 1901): the SDK's own parsers keep it on every captured case of the kit (1901/AC6); the
// kit refuses what the build modules sent before it; the contract's own rules; and the Go reader's older guarantees.
func TestTheFailureContract(t *testing.T) {
t.Run("the SDK parsers keep it", func(t *testing.T) {
Run(t, GoTest, func(o Output) []buildstep.Failure { return buildstep.ReadGoTest(o.Stdout, ".").Failed })
Run(t, NodeTest, func(o Output) []buildstep.Failure { return buildstep.ReadTAP(".", "test", o.Dir, o.Report, o.Stdout) })
Run(t, TypeCheck, func(o Output) []buildstep.Failure {
return []buildstep.Failure{buildstep.ConsoleFailure(".", "type-check", o.Stdout+o.Stderr)}
})
})
// 1890/AC7: the SDK's start sweep keeps the left-running case, and the kit refuses a sweep that ends nothing or ends
// everything.
t.Run("the left-running case", func(t *testing.T) {
LeftRunning(t, func(ctx context.Context, scan buildstep.Scan, node string, before time.Time, logf func(string, ...any)) {
buildstep.EndStaleAtStart(ctx, scan, node, before, logf)
})
if p := JudgeLeftRunning(func(context.Context, buildstep.Scan, string, time.Time, func(string, ...any)) {}); len(p) != 11 {
t.Errorf("a sweep that ends nothing: %d problems, want 11 (three steps not ended and unsaid, one owner unnamed, two asks not deleted and unsaid):\n%s", len(p), strings.Join(p, "\n"))
}
wrongNode := func(ctx context.Context, scan buildstep.Scan, _ string, before time.Time, logf func(string, ...any)) {
buildstep.EndStaleAtStart(ctx, scan, "kitother", before, logf)
}
if p := JudgeLeftRunning(wrongNode); len(p) == 0 {
t.Error("a sweep of another node's steps is not refused")
}
})
// node's output cut to its first fifteen lines with no cause, whose first line was a passing test of the first file
// (mesh-build-node's lines() before the contract), and a Go failure keeping its first line.
t.Run("the kit refuses a failure that hides its cause", func(t *testing.T) {
firstLines := func(s string) string {
var keep []string
for _, l := range strings.Split(s, "\n") {
if l = strings.TrimSpace(l); l != "" && len(keep) < 15 {
keep = append(keep, l)
}
}
return strings.Join(keep, "\n")
}
for _, c := range []struct {
runner, name string
old func(Output) []buildstep.Failure
}{
{NodeTest, "two-files", func(o Output) []buildstep.Failure {
return []buildstep.Failure{{Package: ".", Test: "test", Output: firstLines(o.Report)}}
}},
{GoTest, "passfirst", func(o Output) []buildstep.Failure {
return []buildstep.Failure{{Package: "example.com/passfirst/b", Test: "TestTheSecondPackageFails", Cause: "ok \texample.com/passfirst/a", Output: "--- PASS: TestTheSecondPackagePassesFirst"}}
}},
} {
for _, k := range Cases(c.runner) {
if k.Name != c.name {
continue
}
out, err := Load(k)
if err != nil {
t.Fatal(err)
}
if problems := Judge(k, c.old(out)); len(problems) == 0 {
t.Errorf("%s %s: the kit passed a failure that hides its cause", c.runner, c.name)
}
}
}
})
// What a passing line is (a test's own line starting "ok " is none), when a failure hides its cause, and that a long
// failure keeps its last lines within the bound and starts with a real line for an older checker, which reads only
// the first line of the output cut to 120 characters (1901/AC1, AC3).
t.Run("its rules", func(t *testing.T) {
for _, c := range []struct {
line string
passing bool
}{
{"✔ the same merge has the same id (8.0ms)", true}, {"ok 1 - passes", true}, {" ok 3 - inside", true},
{"ok \texample.com/a\t0.01s", true}, {"PASS", true}, {"--- PASS: TestA (0.00s)", true},
{" --- PASS: TestA/sub (0.00s)", true}, {"✖ fails (1ms)", false}, {"not ok 2 - fails", false},
{"okay, the seat moved", false}, {"ok then: the store refused the write", false}, {"ok the seat", false}, {"FAIL", false},
} {
if buildstep.PassingLine(c.line) != c.passing {
t.Errorf("PassingLine(%q) = %v, want %v", c.line, !c.passing, c.passing)
}
}
if buildstep.Hidden(buildstep.Failure{Cause: "the seat has 2 holders"}) != "" || buildstep.Hidden(buildstep.Failure{}) == "" ||
buildstep.Hidden(buildstep.Failure{Cause: "✔ passes (1ms)"}) == "" {
t.Error("Hidden does not tell a real cause from none and from a passing line")
}
var lines []string
for i := 0; i < 400; i++ {
lines = append(lines, " x_test.go:1: log line "+strings.Repeat("y", 40))
}
lines = append(lines, " x_test.go:9: the real cause", "--- FAIL: TestX (0.00s)")
oldFirst := func(out string) string { // the older checker's rule (judge.go firstLine before issue 1901)
for _, l := range strings.Split(out, "\n") {
if l = strings.TrimSpace(l); l != "" {
return l[:min(len(l), 120)]
}
}
return ""
}
for _, cause := range []string{"the real cause", ""} {
out := buildstep.Clip(lines, cause)
if len(out) > buildstep.OutputBytes+buildstep.LineBytes {
t.Errorf("1901/AC1: kept %d bytes, bound is %d", len(out), buildstep.OutputBytes)
}
if !strings.HasSuffix(out, "the real cause\n--- FAIL: TestX (0.00s)") || !strings.Contains(out, "\n... (") {
t.Errorf("1901/AC1: the last lines or the marker are lost: %q...%q", out[:40], out[len(out)-60:])
}
if got := oldFirst(out); strings.HasPrefix(got, "... (") || got == "" || (cause != "" && got != cause) {
t.Errorf("1901/AC3: cause %q: an older checker would read %q", cause, got)
}
}
})
// The Go reader (moved from mesh-build-go): an older checker, which knows no "cause" field, still reads a failure and
// gets at least the lines it kept (1901/AC3); a test's indented log line saying "panic:" is not the cause, go's own at
// column 0 is (1901/AC4); and each package counts its own skipped tests (issue 890).
t.Run("the Go reader", func(t *testing.T) {
read := func(name string) string {
raw, err := captured.ReadFile("captured/go/" + name + ".jsonl")
if err != nil {
t.Fatal(err)
}
return string(raw)
}
b, _ := json.Marshal(buildstep.ReadGoTest(read("logthenfatal"), ".").Failed)
var old []struct {
Output string `json:"output,omitempty"`
}
if err := json.Unmarshal(b, &old); err != nil || len(old) != 1 || !strings.Contains(old[0].Output, "judged against") || !strings.Contains(old[0].Output, "3 subscription(s)") {
t.Errorf("1901/AC3: an older checker reads %+v, %v", old, err)
}
lines := strings.SplitAfter(read("deeptimeout"), "\n")
added := `{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" p_test.go:9: the child said:\n"}` + "\n" +
`{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" panic: child retried and recovered\n"}` + "\n" +
`{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" WARNING: DATA RACE\n"}` + "\n"
if got := buildstep.ReadGoTest(strings.Join(lines[:10], "")+added+strings.Join(lines[10:], ""), ".").Failed; len(got) != 1 || !strings.HasPrefix(got[0].Cause, "panic: test timed out after 2s") {
t.Errorf("1901/AC4: failures = %+v, want the timeout's panic as the cause", got)
}
res := buildstep.ReadGoTest(`{"Action":"run","Package":"m/a","Test":"TestA"}
{"Action":"skip","Package":"m/a","Test":"TestA","Elapsed":0}
{"Action":"run","Package":"m/b","Test":"TestB"}
{"Action":"pass","Package":"m/b","Test":"TestB","Elapsed":0}
{"Action":"pass","Package":"m/a","Elapsed":0}
{"Action":"pass","Package":"m/b","Elapsed":0}
`, ".")
got := map[string]int{}
for _, p := range res.Packages {
got[p.Package] = p.Skipped
}
if res.Skipped != 1 || got["m/a"] != 1 || got["m/b"] != 0 {
t.Errorf("skipped = %d, per package %v, want 1 with m/a 1 and m/b 0", res.Skipped, got)
}
})
}
+212
View File
@@ -0,0 +1,212 @@
package conformance
import (
"context"
"encoding/json"
"fmt"
"sort"
"strings"
"sync"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// **A step left running by a previous process** (novox/hq issue 1890, note 21): when a delivery replaced mesh-build-go
// on a node while a check ask ran, the new process's start sweep let the ask's key stand as running, and it held one
// of the node's build slots until a hand end. Every build module runs this case (LeftRunning) through the start sweep
// its main runs: the hand-started steps a previous process left are ended at once and loud, its asks' keys deleted (the
// stream redelivers the ask, novox/hq ADR 0525 §6, and its asker keeps waiting for that), what this process runs kept. A
// catalogue check (mesh-catalog checks/failurekit) refuses a build module whose tests do not call LeftRunning.
// Sweep is a build module's own start sweep, as its main runs it once at its start: it ends node's steps in scan that
// no process of the module runs any more (buildstep.EndStaleAtStart, here with the runtime already serving), before
// being the start of the process, saying each end with logf.
type Sweep func(ctx context.Context, scan buildstep.Scan, node string, before time.Time, logf func(string, ...any))
// LeftNode is the node the case's steps are of.
const LeftNode = "kitnode"
// leftCase is one step in the steps state before the sweep, and whether the sweep must end it.
type leftCase struct {
id, tool, owner, state string
node string
age time.Duration // started this long before the process began (negative: after it)
ended bool
deleted bool // a gone process's ask: its key is deleted, not ended
}
// gone is a previous process's owner on the same node.
var gone = "b00t-of-the-kit.4242.mvp0000"
func leftCases() []leftCase {
return []leftCase{
{id: "mvpwlkzi7e6ca8", tool: "check ask", owner: gone, age: 5 * time.Minute, deleted: true}, // note 21's ghost
{id: "mvpwlkzi7e6ca9", tool: "delivery ask", owner: gone, age: -time.Second, deleted: true},
{id: "gonehand1", tool: "build_x_test", owner: gone, age: time.Minute, ended: true},
{id: "oldhand1", tool: "build_x_fetch", age: time.Hour, ended: true}, // no owner: an older build's
{id: "mvpwlkzi7e6caa", tool: "check ask", age: time.Hour, ended: true}, // no owner, past the bound
{id: "mvpwlkzi7e6cab", tool: "check ask", age: 5 * time.Minute}, // no owner, the stream's still
{id: "mvpwlkzi7e6cac", tool: "check ask", owner: buildstep.Owner, age: time.Minute}, // this process's
{id: "minehand1", tool: "build_x_test", owner: buildstep.Owner, age: -time.Second},
{id: "donehand1", tool: "build_x_test", owner: gone, age: time.Hour, state: delivery.StepPassed},
{id: "otherhand1", tool: "build_x_test", owner: gone, age: time.Hour, node: "kitother"},
}
}
// LeftRunning runs the case through sweep and fails the test for each step the sweep ended or kept wrongly, or ended
// without saying it.
func LeftRunning(t *testing.T, sweep Sweep) {
t.Helper()
for _, problem := range JudgeLeftRunning(sweep) {
t.Error(problem)
}
}
// JudgeLeftRunning says what is wrong with what sweep did to the case; nothing when it keeps the rule.
func JudgeLeftRunning(sweep Sweep) []string {
st := &steps{vals: map[string][]byte{}, revs: map[string]uint64{}}
before := time.Now()
for _, c := range leftCases() {
node, state := c.node, c.state
if node == "" {
node = LeftNode
}
if state == "" {
state = delivery.StepRunning
}
key, _ := delivery.StepKey(node, c.id)
raw, err := delivery.EncodeStep(delivery.Step{ID: c.id, Tool: c.tool, Node: node, State: state, Started: before.Add(-c.age).UTC(), Owner: c.owner})
if err != nil {
return []string{"the kit's step " + c.id + ": " + err.Error()}
}
st.set(key, raw)
}
served := make(chan struct{})
close(served)
was := buildstep.StaleServed
buildstep.StaleServed = func() <-chan struct{} { return served }
defer func() { buildstep.StaleServed = was }()
var mu sync.Mutex
var said []string
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
sweep(ctx, st.scan(), LeftNode, before, func(f string, a ...any) {
mu.Lock()
defer mu.Unlock()
said = append(said, fmt.Sprintf(f, a...))
})
mu.Lock()
log := strings.Join(said, "\n")
mu.Unlock()
var problems []string
for _, c := range leftCases() {
node := c.node
if node == "" {
node = LeftNode
}
key, _ := delivery.StepKey(node, c.id)
raw := st.get(key)
if c.deleted {
if raw != nil {
problems = append(problems, fmt.Sprintf("left running: the ask %s of the gone owner %q is still held after the sweep (%s); "+
"its key is deleted, never ended: the stream redelivers it (ADR 0525 §6)", key, c.owner, raw))
}
if !strings.Contains(log, key) || !strings.Contains(log, c.owner) {
problems = append(problems, "left running: the delete of "+key+" is not said with its owner "+c.owner)
}
continue
}
if raw == nil {
problems = append(problems, "left running: the step "+key+" is gone after the sweep, want it kept or ended")
continue
}
s, err := delivery.DecodeStep(raw)
if err != nil {
problems = append(problems, key+": "+err.Error())
continue
}
want := c.state
if want == "" {
want = delivery.StepRunning
}
if c.ended {
want = delivery.StepFailed
}
if s.State != want {
problems = append(problems, fmt.Sprintf("left running: the step %s (%s, owner %q, %s before the start) is %s after the sweep, want %s", key, c.tool, c.owner, c.age, s.State, want))
}
if c.ended && !strings.Contains(log, key) {
problems = append(problems, "left running: the end of "+key+" is not said")
}
if c.ended && c.owner != "" && !strings.Contains(log, c.owner) {
problems = append(problems, "left running: the end of "+key+" does not name its owner "+c.owner)
}
}
sort.Strings(problems)
return problems
}
// steps is a steps state with revisions, whose Update lands only at the revision read (stdio.ErrChanged otherwise).
type steps struct {
mu sync.Mutex
vals map[string][]byte
revs map[string]uint64
}
func (s *steps) set(key string, raw []byte) {
s.mu.Lock()
defer s.mu.Unlock()
s.vals[key], s.revs[key] = raw, s.revs[key]+1
}
func (s *steps) get(key string) []byte {
s.mu.Lock()
defer s.mu.Unlock()
return s.vals[key]
}
func (s *steps) scan() buildstep.Scan {
return buildstep.Scan{
Keys: func() ([]string, error) {
s.mu.Lock()
defer s.mu.Unlock()
var out []string
for k := range s.vals {
out = append(out, k)
}
return out, nil
},
Get: func(key string) (json.RawMessage, uint64, error) {
s.mu.Lock()
defer s.mu.Unlock()
return s.vals[key], s.revs[key], nil
},
Delete: func(key string, revision uint64) error {
s.mu.Lock()
defer s.mu.Unlock()
if s.revs[key] != revision {
return stdio.ErrChanged
}
delete(s.vals, key)
s.revs[key]++
return nil
},
Update: func(key string, value any, revision uint64) (uint64, error) {
raw, err := json.Marshal(value)
if err != nil {
return 0, err
}
s.mu.Lock()
defer s.mu.Unlock()
if s.revs[key] != revision {
return 0, stdio.ErrChanged
}
s.vals[key], s.revs[key] = raw, revision+1
return revision + 1, nil
},
}
}
+14
View File
@@ -0,0 +1,14 @@
package buildstep
import (
"os"
"git.novox.be/novox/mesh-sdk/go/buildcpu"
)
// NodeLimits is what one step of a build module is held to (novox/hq ADR 0553 §4): the node's processor share from the
// word the composer gives the module (buildcpu.ShareWord), divided by the module's own capacity, own, over the machine's
// processors. Without the word, no thread limit: Threads is 0.
func NodeLimits(own int) buildcpu.Limits {
return buildcpu.FromWords(os.Getenv, own)
}
+147
View File
@@ -0,0 +1,147 @@
package buildstep
// The failure contract of a build step's test level (novox/hq issue 1901, its thorough fix): what a build module says of
// each test that failed, the same for every language. Each build module used to parse its own test runner's output, and
// a fix in one never reached the others: a failed Go test kept a t.Logf line, a failed node --test run kept a passing
// '✔' line of its first file. The parsers that fill the contract live here (gotest.go, tap.go) and the conformance kit
// (package conformance) holds every build module to them with captured runner output.
import (
"fmt"
"strings"
"git.novox.be/novox/mesh-sdk/go/sdkversion"
)
// Failure is one test that failed, or a package or file that failed with no test to name (a build that failed, a test
// program that panicked, a test file that did not load). The JSON names are the ones mesh-build-go and the checker have
// always used; File is new and an older reader ignores it.
type Failure struct {
// Package is the Go package's import path, or the TypeScript package's directory in the workspace.
Package string `json:"package"`
// Test names the failing test, or the tests that were running when the program stopped; "" when none can be named.
Test string `json:"test,omitempty"`
// File is where the failing test is (a Go test's file and line is in its Cause; node names the test's file:line:col).
File string `json:"file,omitempty"`
// Cause is the one line that says why: the assertion's or the error's first line, a "panic:" line, a build's first
// error, or words saying how the program stopped. **Never empty, and never a passing line** (PassingLine).
Cause string `json:"cause,omitempty"`
// Output is the failing test's own output, its last lines up to OutputBytes, Cause first: an older checker reads
// only its first line.
Output string `json:"output,omitempty"`
}
// OutputBytes is the most a failure keeps of its output, and LineBytes the most of one line: four kilobytes holds a long
// assertion message and stays far under a bus message's or a verdict's bound.
const (
OutputBytes = 4096
LineBytes = 1024
)
// By names a build module and the SDK it was built with, for a result's "by": the checker names it when a failure hides
// its cause ("the build module hid the failure (<by>)").
func By(module string) string {
return module + " with mesh-sdk " + sdkversion.Version()
}
// PassingLine says a line is a test runner's word for a test or a package that passed: node's '✔', TAP's "ok N" (at any
// indent), go test's "ok \t<pkg>", "PASS" and "--- PASS". A line of a test's own that starts "ok " is none of these. Such a line is never a failure's cause, nor part of a failure's output.
func PassingLine(l string) bool {
if strings.HasPrefix(l, "ok \t") { // go test's package line, at column 0
return true
}
t := strings.TrimSpace(l)
switch {
case t == "PASS", strings.HasPrefix(t, "✔"), strings.HasPrefix(t, "--- PASS"):
return true
}
return tapOk(t)
}
// Hidden says how a failure hides why it failed, or "" when it does not: it has no cause, or its cause is a passing line.
// The checker says it loudly (issue 1901): a build module that sends such a failure does not keep the contract.
func Hidden(f Failure) string {
c := strings.TrimSpace(f.Cause)
switch {
case c == "":
return "it sent no cause"
case PassingLine(c):
return fmt.Sprintf("its cause is a passing line (%q)", CutLine(c))
}
return ""
}
// Clip joins the last lines of a failure's output that fit OutputBytes, each cut to LineBytes, and says how many earlier
// lines it left out. **Its first line is the cause**, put ahead when the kept lines do not start with it, or, with no
// cause and lines left out, the first non-empty line, then the marker: an older checker reads only the first line.
// Passing lines are dropped.
func Clip(lines []string, cause string) string {
own := make([]string, 0, len(lines))
for _, l := range lines {
if !PassingLine(l) {
own = append(own, l)
}
}
lines = own
size, from := 0, len(lines)
for from > 0 {
n := len(CutLine(lines[from-1])) + 1
if size+n > OutputBytes && from < len(lines) {
break
}
size += n
from--
}
kept := make([]string, 0, len(lines)-from+2)
head := strings.TrimSpace(cause)
for i := 0; head == "" && i < from; i++ {
head = strings.TrimSpace(lines[i])
}
if head != "" && (from > 0 || len(lines) == 0 || strings.TrimSpace(lines[from]) != head) {
kept = append(kept, CutLine(head))
}
if from > 0 {
kept = append(kept, fmt.Sprintf("... (%d earlier lines)", from))
}
for _, l := range lines[from:] {
kept = append(kept, CutLine(l))
}
return strings.Join(kept, "\n")
}
// CutLine cuts a line to LineBytes, on a character boundary.
func CutLine(l string) string {
if len(l) <= LineBytes {
return l
}
return strings.ToValidUTF8(l[:LineBytes], "") + "..."
}
// ConsoleFailure is the failure of a command that is no test runner (go vet, gofmt, tsc, a build script): its cause is the
// first line that names an error (tsc's "error TS", go vet's "file.go:12:3: ..."), else the first line that is not a
// heading ("# <package>", npm's "> script"), and its output the command's lines without passing ones.
func ConsoleFailure(pkg, test, out string) Failure {
var lines []string
for _, l := range strings.Split(strings.ReplaceAll(stripANSI(out), "\r\n", "\n"), "\n") {
if t := strings.TrimSpace(l); t != "" && !PassingLine(t) {
lines = append(lines, strings.TrimRight(l, " \t"))
}
}
cause := ""
for _, l := range lines {
if t := strings.TrimSpace(l); strings.Contains(t, "error") || strings.Contains(t, "Error") || goPosition.MatchString(t) {
cause = t
break
}
}
for _, l := range lines {
if t := strings.TrimSpace(l); cause == "" && !strings.HasPrefix(t, "# ") && !strings.HasPrefix(t, "> ") && !strings.HasPrefix(t, "npm ") {
cause = t
}
}
if cause == "" {
cause = test + " failed and wrote nothing that says why"
}
cause = CutLine(cause)
return Failure{Package: pkg, Test: test, Cause: cause, Output: Clip(lines, cause)}
}
+6 -11
View File
@@ -71,7 +71,7 @@ func FetchCommit(ctx context.Context, run forgekey.Runner, dir, remote, commit,
full := append([]string{"-c", "credential.helper=", "-C", dir}, args...)
r := run(ctx, env, "git", full...)
if r.Status != 0 || r.Err != "" {
return "", fmt.Errorf("git %s: %s", args[0], firstLine(r.Err+" "+r.Stderr))
return "", fmt.Errorf("git %s: %s", args[0], forgekey.Said(r.Err+"\n"+r.Stderr))
}
return r.Stdout, nil
}
@@ -128,7 +128,11 @@ func UpdateMirror(ctx context.Context, run forgekey.Runner, mirror, remote, comm
git := func(args ...string) error {
r := run(ctx, env, "git", append([]string{"-c", "credential.helper="}, args...)...)
if r.Status != 0 || r.Err != "" {
return fmt.Errorf("git %s: %s", args[0], firstLine(r.Err+" "+r.Stderr))
verb := args[0]
if verb == "-C" && len(args) > 2 { // the mirror's commands are named by what they do, not by -C
verb = args[2]
}
return fmt.Errorf("git %s: %s", verb, forgekey.Said(r.Err+"\n"+r.Stderr))
}
return nil
}
@@ -190,12 +194,3 @@ func checkFetch(o fetchOptions, dir, remote, commit, ref string) error {
}
return nil
}
func firstLine(s string) string {
for _, l := range strings.Split(s, "\n") {
if l = strings.TrimSpace(l); l != "" {
return l
}
}
return ""
}
+409
View File
@@ -0,0 +1,409 @@
package buildstep
// The reader of `go test -json` (moved whole from mesh-catalog's mesh-build-go, issue 1901's thorough fix): per package
// what ran and for how long, and each failure under the contract (failure.go), its real cause too when the failure is no
// test's assertion (a panic, a timeout, a build error, an exit, a kill, a race).
import (
"bufio"
"encoding/json"
"fmt"
"regexp"
"slices"
"strings"
)
// GoPackage is one package's part of a Go test level.
type GoPackage struct {
Dir string `json:"dir"`
Package string `json:"package"`
Passed bool `json:"passed"`
Elapsed float64 `json:"elapsed"`
Tests int `json:"tests"`
// Skipped is how many of the package's tests skipped.
Skipped int `json:"skipped,omitempty"`
// NoTests is a package with no test files (at this level: a file with the other level's build tag is none).
NoTests bool `json:"no_tests,omitempty"`
}
// TestTime is one test and the seconds it took.
type TestTime struct {
Package string `json:"package"`
Test string `json:"test"`
Elapsed float64 `json:"elapsed"`
}
// GoTest is what one go.mod directory's `go test -json` said.
type GoTest struct {
Packages []GoPackage
Tests int
Skipped int
Times []TestTime
Failed []Failure
// Elapsed is the slowest package's running time in seconds.
Elapsed float64
// Noise is the lines that were no event: what go said when it could not run the tests at all.
Noise []string
}
// ReadGoTest reads the output of `go test -json` run in the go.mod directory dir.
func ReadGoTest(out, dir string) GoTest {
r := &goReader{dir: dir, pkgs: map[string]*GoPackage{}, builds: map[string][]string{}, pkgOut: map[string][]string{},
testOut: map[string][]string{}, causes: map[string]string{}, failedIn: map[string]bool{},
running: map[string][]string{}, passed: map[string]bool{}, built: map[string]bool{}}
sc := bufio.NewScanner(strings.NewReader(out))
sc.Buffer(make([]byte, 1<<20), 1<<26)
for sc.Scan() {
var ev goEvent
if err := json.Unmarshal(sc.Bytes(), &ev); err != nil {
if l := strings.TrimSpace(sc.Text()); l != "" {
r.res.Noise = append(r.res.Noise, l)
}
continue
}
r.event(ev)
}
r.finish()
return r.res
}
// goEvent is one line of `go test -json` (test2json, and the build events go test adds).
type goEvent struct {
Action string
Package string
Test string
Output string
OutputType string // "error" for a t.Error or t.Fatal message, "error-continue" for its further lines
ImportPath string
FailedBuild string
Elapsed float64
}
type goReader struct {
res GoTest
dir string
pkgs map[string]*GoPackage
order []string
builds map[string][]string // build output by import path
pkgOut map[string][]string
testOut map[string][]string // by package + "\x00" + test
causes map[string]string // the first error line of a test, by the same key
failedIn map[string]bool // packages with a failed test
// running is the tests that started and have not ended, by package, in the order they started: when a package
// fails with no failed test, these were running when its test program stopped.
running map[string][]string
// passed says a package's test program printed "PASS" before it failed; built says it failed to build.
passed, built map[string]bool
}
func (r *goReader) pkg(name string) *GoPackage {
p := r.pkgs[name]
if p == nil {
p = &GoPackage{Dir: r.dir, Package: name}
r.pkgs[name] = p
r.order = append(r.order, name)
}
return p
}
func keepLine(lines []string, out string) []string {
if t := strings.TrimRight(out, "\r\n"); t != "" && !strings.HasPrefix(t, "=== ") {
lines = append(lines, t)
}
return lines
}
func (r *goReader) event(ev goEvent) {
switch {
case ev.Package == "" && ev.ImportPath != "":
if ev.Action == "build-output" {
r.builds[ev.ImportPath] = keepLine(r.builds[ev.ImportPath], ev.Output)
}
case ev.Package == "":
case ev.Test == "":
p := r.pkg(ev.Package)
switch ev.Action {
case "output":
// go test's own summary of a package is never a failure's words: a "PASS" before a "FAIL <pkg>" says only
// that the tests passed and the program failed after them.
switch l := strings.TrimRight(ev.Output, "\r\n"); {
case l == "PASS":
r.passed[ev.Package] = true
case goSummaryLine(l):
default:
r.pkgOut[ev.Package] = keepLine(r.pkgOut[ev.Package], ev.Output)
}
case "pass":
p.Passed, p.Elapsed = true, ev.Elapsed
case "fail":
p.Passed, p.Elapsed = false, ev.Elapsed
if ev.FailedBuild != "" {
r.built[ev.Package] = true
r.pkgOut[ev.Package] = append(r.builds[ev.FailedBuild], r.pkgOut[ev.Package]...)
}
case "skip":
p.Passed, p.NoTests = true, true
}
default:
key := ev.Package + "\x00" + ev.Test
switch ev.Action {
case "output":
r.testOut[key] = keepLine(r.testOut[key], ev.Output)
if ev.OutputType == "error" && r.causes[key] == "" {
r.causes[key] = goErrorWords(ev.Output)
}
case "run":
r.running[ev.Package] = append(r.running[ev.Package], ev.Test)
case "pass", "fail", "skip":
r.running[ev.Package] = slices.DeleteFunc(r.running[ev.Package], func(t string) bool { return t == ev.Test })
p := r.pkg(ev.Package)
p.Tests++
r.res.Tests++
if ev.Action == "skip" {
r.res.Skipped++
p.Skipped++
}
r.res.Times = append(r.res.Times, TestTime{Package: ev.Package, Test: ev.Test, Elapsed: ev.Elapsed})
if ev.Action == "fail" {
r.failedIn[ev.Package] = true
if r.subtestFailed(ev.Package, ev.Test) {
return // a parent failed only by its subtest is that subtest's failure, said once
}
if r.subtestTakesTheCause(ev.Package, ev.Test) {
return // a subtest's panic, which go files under its parent, is said on the subtest
}
r.res.Failed = append(r.res.Failed, r.testFailure(ev.Package, ev.Test))
}
}
}
}
// subtestFailed says a failure of test under pkg was already reported for one of its subtests (they end first), each with
// a cause of its own, and the test reported no error and no runtime cause (a subtest's panic) of its own.
func (r *goReader) subtestFailed(pkg, test string) bool {
key := pkg + "\x00" + test
if r.causes[key] != "" || goRuntimeCause(r.testOut[key]) != "" {
return false
}
for _, f := range r.res.Failed {
if f.Package == pkg && strings.HasPrefix(f.Test, test+"/") {
return true
}
}
return false
}
// noMessage is the cause of a test that failed and said nothing of why.
const noMessage = "the test failed without an error message (t.Fail or t.FailNow)"
// subtestTakesTheCause gives a parent's runtime cause (a subtest's panic: go test -json files its lines under the
// parent) to the subtests of that parent that failed with no message of their own, and says it did. A parent with an
// error line of its own, or whose failed subtests each said why, is reported as itself.
func (r *goReader) subtestTakesTheCause(pkg, test string) bool {
key := pkg + "\x00" + test
if r.causes[key] != "" {
return false
}
cause := goRuntimeCause(r.testOut[key])
if cause == "" {
return false
}
took := false
for i, f := range r.res.Failed {
if f.Package == pkg && strings.HasPrefix(f.Test, test+"/") && f.Cause == noMessage {
r.res.Failed[i].Cause, r.res.Failed[i].Output = cause, Clip(r.testOut[key], cause)
took = true
}
}
return took
}
// testFailure is a test that failed: its cause is its first error line, the race it ran into, the runtime's line, or
// words saying it failed without a message.
func (r *goReader) testFailure(pkg, test string) Failure {
key := pkg + "\x00" + test
cause := r.causes[key]
if race := goRaceCause(r.testOut[key]); race != "" && (cause == "" || strings.HasPrefix(cause, "race detected")) {
cause = race // the race itself, not testing's "race detected during execution of test"
} else if cause == "" {
cause = goRuntimeCause(r.testOut[key])
}
if cause == "" {
cause = noMessage
}
return Failure{Package: pkg, Test: test, Output: Clip(r.testOut[key], cause), Cause: cause}
}
// goSummaryLine says a line is go test's summary of a package, in go's exact forms only: "PASS" or "FAIL" alone,
// "ok \t<pkg>\t<time>" and "FAIL\t<pkg>\t<time>" (a line of the package's own that starts "ok " is kept).
func goSummaryLine(l string) bool {
return l == "PASS" || l == "FAIL" || strings.HasPrefix(l, "ok \t") || strings.HasPrefix(l, "FAIL\t")
}
// goRuntimeCause is the cause in a failure's output that is not a test's assertion: a race the race detector reported;
// else the first "panic:" line, a timeout's with the tests it names as running; else the first "fatal error:" or
// "signal:" line; "" when there is none. It reads every line, so a long goroutine dump cannot hide it. **It matches only
// lines at column 0**, where go prints the runtime's: a test's indented log line holding the same words is not the cause.
func goRuntimeCause(lines []string) string {
if race := goRaceCause(lines); race != "" {
return race
}
for _, prefix := range []string{"panic: ", "fatal error: ", "signal: "} {
for i, l := range lines {
l = strings.TrimRight(l, "\r\n")
if !strings.HasPrefix(l, prefix) {
continue
}
if strings.HasPrefix(l, "panic: test timed out after") {
var names []string
for _, n := range lines[i+1:] {
if !strings.HasPrefix(n, "\t\t") {
if strings.TrimSpace(n) == "running tests:" {
continue
}
break
}
names = append(names, strings.TrimSpace(n))
}
if len(names) > 0 {
l += "; running tests: " + strings.Join(names, ", ")
}
}
return CutLine(l)
}
}
return ""
}
// goRaceCause is the first race the race detector reported in a failure's output, in one line: its access and the frame
// that made it, and where the goroutine that made it was created. "" when the output holds no race.
func goRaceCause(lines []string) string {
at := -1
for i, l := range lines {
if strings.TrimRight(l, "\r\n") == "WARNING: DATA RACE" {
at = i
break
}
}
if at < 0 {
return ""
}
frame := func(i int) string {
var fn, file string
for _, l := range lines[i+1:] {
if l = strings.TrimSpace(l); l == "" {
break
}
if fn == "" {
fn = l
continue
}
file = l
if j := strings.LastIndexByte(file, '/'); j >= 0 {
file = file[j+1:]
}
if j := strings.Index(file, " +0x"); j >= 0 {
file = file[:j]
}
break
}
if file != "" {
return fn + " (" + file + ")"
}
return fn
}
words := "WARNING: DATA RACE"
for i := at + 1; i < len(lines); i++ {
l := strings.TrimSpace(lines[i])
if l == "" || strings.HasPrefix(l, "===") {
continue
}
words += ": " + strings.TrimSuffix(l, ":")
if f := frame(i); f != "" {
words += " in " + f
}
break
}
for i := at + 1; i < len(lines); i++ {
l := strings.TrimSpace(lines[i])
if strings.HasPrefix(l, "==================") {
break
}
if strings.HasPrefix(l, "Goroutine ") && strings.HasSuffix(l, " created at:") {
if f := frame(i); f != "" {
words += "; its goroutine created at " + f
}
break
}
}
return CutLine(words)
}
// goBuildCause is the first error line of a failed build: the first line that is not go's "# <package>" heading.
func goBuildCause(lines []string) string {
for _, l := range lines {
if l = strings.TrimSpace(l); l != "" && !strings.HasPrefix(l, "# ") {
return CutLine(l)
}
}
return ""
}
// goTestNames names the tests that were running, at most three, and how many more.
func goTestNames(tests []string) string {
if len(tests) <= 3 {
return strings.Join(tests, ", ")
}
return fmt.Sprintf("%s and %d more", strings.Join(tests[:3], ", "), len(tests)-3)
}
// packageFailure is a package that failed with no failed test to name it: its test program did not build, panicked,
// timed out, was killed or exited. It names the tests that were running, keeps their output with the package's, and
// says the cause.
func (r *goReader) packageFailure(name string) Failure {
running := r.running[name]
var lines []string
for _, t := range running {
lines = append(lines, r.testOut[name+"\x00"+t]...)
}
lines = append(lines, r.pkgOut[name]...)
cause := goRuntimeCause(lines)
switch {
case cause != "":
case r.built[name]:
cause = goBuildCause(lines)
case len(running) > 0:
cause = "the test program exited while " + goTestNames(running) + " ran, with no panic: an os.Exit or a signal (go test -json reports no exit status)"
case r.passed[name]:
cause = "the tests passed, then the test program exited with a failure: an os.Exit or a panic in TestMain or after the tests (go test -json reports no exit status)"
}
if cause == "" {
cause = "the package failed with no test to name and no panic, signal or build error in its output"
}
return Failure{Package: name, Test: goTestNames(running), Output: Clip(lines, cause), Cause: cause}
}
var goFileAndLine = regexp.MustCompile(`^\S+\.go:\d+: `)
// goPosition matches a line that starts with a Go file's position, as go vet and the compiler write an error.
var goPosition = regexp.MustCompile(`^\S+\.go:\d+(:\d+)?: `)
// goErrorWords is the words of an error line of a test: trimmed, without the file and line go test put before them.
func goErrorWords(out string) string {
return strings.TrimSpace(goFileAndLine.ReplaceAllString(strings.TrimSpace(out), ""))
}
// finish moves what was read into the result: the packages in order, a failure for each failed package that has no
// failed test to name it.
func (r *goReader) finish() {
for _, name := range r.order {
p := r.pkgs[name]
if !p.Passed && !r.failedIn[name] {
r.res.Failed = append(r.res.Failed, r.packageFailure(name))
}
if !p.NoTests && p.Elapsed > r.res.Elapsed {
r.res.Elapsed = p.Elapsed
}
r.res.Packages = append(r.res.Packages, *p)
}
}
+6 -16
View File
@@ -1,24 +1,14 @@
package buildstep
import "strings"
import "git.novox.be/novox/mesh-sdk/go/catalogue"
// LauncherSuffix is what a TypeScript entrypoint's launcher is called beside it: index.js is started as
// index.serve.mjs (novox/hq ADR 0193), an ES module by its own extension whatever the bundle's package.json says.
const LauncherSuffix = ".serve.mjs"
const LauncherSuffix = catalogue.LauncherSuffix
// Launchers are the launchers a bundle's build writes, by entrypoint, relative to the bundle's root: for TypeScript
// every entrypoint `x.js` is started by `x.serve.mjs` beside it; an entrypoint that is not JavaScript and every other
// language, whose build is already executable, get none. The one rule mesh-build-node, build-agent and the controller
// share (R4e), so the name a build writes is the name a reader expects.
// Launchers are the launchers a bundle's build writes, by entrypoint, relative to the bundle's root
// (catalogue.Launchers): the one rule mesh-build-node, build-agent, the controller and a node taking in a build share
// (R4e, issue 1126), so the name a build writes is the name a reader expects.
func Launchers(language string, entrypoints []string) map[string]string {
out := map[string]string{}
if language != "typescript" {
return out
}
for _, entry := range entrypoints {
if strings.HasSuffix(entry, ".js") {
out[entry] = strings.TrimSuffix(entry, ".js") + LauncherSuffix
}
}
return out
return catalogue.Launchers(language, entrypoints)
}
+14 -1
View File
@@ -15,6 +15,9 @@ type MergeResult struct {
Base, Head, Merge string
Conflicts []string
Changed []string
// Programs are the changed files that are compiled programs (ExecutableKind of their first bytes, read in the workspace
// at the merge); never nil for a clean merge, so a caller tells none from a module that reports none (issue 1577).
Programs []Program
}
// MergeInto makes, in the workspace dir that FetchCommit left at commit, the merge of commit into the trunk commit
@@ -52,7 +55,7 @@ func MergeInto(ctx context.Context, run forgekey.Runner, dir, remote, base, comm
full := append([]string{"-c", "credential.helper=", "-C", dir}, args...)
r := run(ctx, env, "git", full...)
if r.Status != 0 || r.Err != "" {
return "", fmt.Errorf("git %s: %s", args[0], firstLine(r.Err+" "+r.Stderr))
return "", fmt.Errorf("git %s: %s", args[0], forgekey.Said(r.Err+"\n"+r.Stderr))
}
return r.Stdout, nil
}
@@ -98,6 +101,16 @@ func MergeInto(ctx context.Context, run forgekey.Runner, dir, remote, base, comm
return res, fmt.Errorf("the paths the merge changes cannot be read: %w", err)
}
res.Changed = strings.Fields(changed)
raw, err := git("diff", "--raw", "-z", "--no-renames", "--diff-filter=AM", base, res.Merge)
if err != nil {
return res, fmt.Errorf("the files the merge adds or modifies cannot be read: %w", err)
}
res.Programs = programsIn(dir, raw)
history, err := historyPrograms(git, base, commit, res.Programs)
if err != nil {
return res, fmt.Errorf("the commits of the branch cannot be read for compiled programs: %w", err)
}
res.Programs = append(res.Programs, history...)
return res, nil
}
+30 -3
View File
@@ -79,6 +79,30 @@ func (r *mergeRig) head(files map[string]string) string {
return h
}
// headThatAddsAndRemoves is head with an earlier commit on the branch that adds a program and a later one that removes it: the
// branch's tree is clean, its history is not. It answers the head and the earlier commit.
func (r *mergeRig) headThatAddsAndRemoves(files map[string]string) (head, added string) {
realGit(r.t, r.work, "checkout", "-q", "-B", "pr", r.fork)
writeFile(r.t, r.work, "gone/prog", "\x7fELF\x02\x01")
realGit(r.t, r.work, "add", ".")
realGit(r.t, r.work, "commit", "-q", "-m", "oops")
added = realGit(r.t, r.work, "rev-parse", "HEAD")
realGit(r.t, r.work, "rm", "-q", "gone/prog")
return r.head2(files), added
}
// head2 commits files on the current branch and pushes it as the pull request's head.
func (r *mergeRig) head2(files map[string]string) string {
for n, c := range files {
writeFile(r.t, r.work, n, c)
}
realGit(r.t, r.work, "add", ".")
realGit(r.t, r.work, "commit", "-q", "-m", "head")
h := realGit(r.t, r.work, "rev-parse", "HEAD")
realGit(r.t, r.work, "push", "-q", r.bare, "HEAD:refs/pull/1/head")
return h
}
// advance moves the trunk past the fork point with a commit changing files, pushes it and answers its sha.
func (r *mergeRig) advance(files map[string]string) string {
realGit(r.t, r.work, "checkout", "-q", "-B", "main", r.fork)
@@ -102,7 +126,7 @@ func (r *mergeRig) fetch(head string) {
// holds both sides' changes.
func TestMergeIntoACleanMergeAnswersTheMergeCommitAndTheChangedFiles(t *testing.T) {
r := newMergeRig(t)
head := r.head(map[string]string{"b.txt": "head\n"})
head, added := r.headThatAddsAndRemoves(map[string]string{"b.txt": "head\n", "prog": "\x7fELF\x02\x01"})
base := r.advance(map[string]string{"a.txt": "one\ntwo\nthree\nfour\n"})
r.fetch(head)
res, err := MergeInto(context.Background(), forgekey.ExecRunner, r.ws, r.remote, base, head, AllowFileRemote())
@@ -112,8 +136,11 @@ func TestMergeIntoACleanMergeAnswersTheMergeCommitAndTheChangedFiles(t *testing.
if res.Base != base || res.Head != head || len(res.Merge) != 40 || res.Merge == base || res.Merge == head || len(res.Conflicts) != 0 {
t.Fatalf("result = %+v", res)
}
if !reflect.DeepEqual(res.Changed, []string{"b.txt"}) {
t.Errorf("changed = %v, want only what the head adds to the trunk: b.txt", res.Changed)
if !reflect.DeepEqual(res.Changed, []string{"b.txt", "prog"}) {
t.Errorf("changed = %v, want only what the head adds to the trunk: b.txt, prog", res.Changed)
}
if !reflect.DeepEqual(res.Programs, []Program{{"prog", "ELF", ""}, {"gone/prog", "ELF", added}}) { // issue 1577
t.Errorf("programs = %v, want the ELF the head adds and the one an earlier commit added and a later one removed", res.Programs)
}
if at := realGit(t, r.ws, "rev-parse", "HEAD"); at != res.Merge {
t.Errorf("the workspace is at %s, not at the merge %s", at, res.Merge)
+18
View File
@@ -104,3 +104,21 @@ func TestUpdateMirrorRefusesBeforeRunning(t *testing.T) {
t.Errorf("ran %v", f.calls)
}
}
// The forge's refusal reaches the reason in git's own words: the mirror of a repository its account cannot read said
// only "git -C: error:" (2026-10-26), since git writes a bare "error:" line before the forge's message.
func TestUpdateMirrorSaysTheForgesRefusal(t *testing.T) {
f := &fakeGit{}
f.rule = func(args string) (r forgekey.Ran) {
if strings.Contains(args, "fetch") {
r.Status = 128
r.Stderr = "error: \nerror: User: 22:nox-builder with Key: 30:mesh:ace.mesh-build-go is not authorized to read o/r.\nerror: \nfatal: Could not read from remote repository.\n"
}
return
}
err := UpdateMirror(context.Background(), f.run, filepath.Join(t.TempDir(), "m.git"), "ssh://mesh-forge/o/r.git", sha, "")
if err == nil || !strings.Contains(err.Error(), "nox-builder with Key: 30:mesh:ace.mesh-build-go is not authorized to read o/r.") ||
!strings.Contains(err.Error(), "Could not read from remote repository") || !strings.HasPrefix(err.Error(), "git fetch: ") {
t.Errorf("err = %v", err)
}
}
+67
View File
@@ -0,0 +1,67 @@
package buildstep
import (
"context"
"strings"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// **A running ask says its phases** (issue 1890, 2026-10-26 19:09: a mesh-catalog try's step on the workstation ran 30 minutes
// and its module's journal held no line naming its ask, so nobody could tell whether it fetched, linted or tested). The
// puller hands every ask's run a phase reporter in its context; RunAsk and a module's own step bodies call Phase as they
// move (fetch and merge, a level's digest, lint, a level's tests), and each phase is logged with the ask's id and its
// time since the start, and written to the ask's running step key (delivery.Step.Phase), so its asker and the delivery
// view read where it stands from the bus.
type phaseKey struct{}
// Phase says that the ask whose run ctx belongs to entered phase; nothing outside an ask's run (a tool called by hand).
func Phase(ctx context.Context, phase string) {
if r, ok := ctx.Value(phaseKey{}).(*phases); ok && r != nil {
r.enter(strings.TrimSpace(phase))
}
}
// phases is one running ask's reporter: it writes the step key again with each new phase until the step ends.
type phases struct {
mu sync.Mutex
put Put
key string
step delivery.Step
ended bool
logf func(format string, args ...any)
prefix string
}
func (r *phases) enter(phase string) {
if phase == "" {
return
}
r.mu.Lock()
defer r.mu.Unlock()
if r.ended || phase == r.step.Phase {
return
}
at := now().UTC()
r.step.Phase, r.step.PhaseAt = phase, at
if r.logf != nil {
r.logf("%s: %s, %s after its start", r.prefix, phase, at.Sub(r.step.Started).Round(time.Second))
}
if r.put == nil {
return
}
if err := write(r.put, r.key, r.step); err != nil && r.logf != nil {
r.logf("%s: the phase %s could not be written to %s: %v", r.prefix, phase, r.key, err)
}
}
// end stops the writes (the step's end is written after it) and answers the step with its last phase, for the end.
func (r *phases) end() delivery.Step {
r.mu.Lock()
defer r.mu.Unlock()
r.ended = true
return r.step
}
+115
View File
@@ -0,0 +1,115 @@
package buildstep
import (
"bytes"
"io"
"os"
"path/filepath"
"strings"
)
// Program is a changed file whose first bytes are an executable binary: a compiled program, which is build output and not
// source (novox/hq issue 1577: `go build` in a module's folder writes one beside go.mod, and `git add -A` takes it).
type Program struct {
Path string `json:"path"`
Kind string `json:"kind"` // ELF, Mach-O or PE
// Commit is set for a program a commit of the branch added and a later one removed (or replaced): the merge's tree is clean,
// but the merge commit carries the blob into the trunk's history for good. Empty for a program the merge's tree holds.
Commit string `json:"commit,omitempty"`
}
// ExecutableKind says what executable format the first bytes of a file are, or "" when they are none. A PE needs its header
// ("PE\0\0" at the offset in bytes 0x3c) so a text starting with "MZ" is not one; a fat Mach-O (0xCAFEBABE) is told from a
// Java class by its architecture count (bytes 4 to 7, fewer than 0x20; a class has its version there).
func ExecutableKind(head []byte) string {
switch {
case bytes.HasPrefix(head, []byte("\x7fELF")):
return "ELF"
case bytes.HasPrefix(head, []byte("MZ")):
if len(head) >= 0x40 {
off := int(head[0x3c]) | int(head[0x3d])<<8 | int(head[0x3e])<<16 | int(head[0x3f])<<24
if off >= 0 && off+4 <= len(head) && bytes.Equal(head[off:off+4], []byte("PE\x00\x00")) {
return "PE"
}
}
case bytes.HasPrefix(head, []byte{0xcf, 0xfa, 0xed, 0xfe}), bytes.HasPrefix(head, []byte{0xce, 0xfa, 0xed, 0xfe}),
bytes.HasPrefix(head, []byte{0xfe, 0xed, 0xfa, 0xcf}), bytes.HasPrefix(head, []byte{0xfe, 0xed, 0xfa, 0xce}),
bytes.HasPrefix(head, []byte{0xca, 0xfe, 0xba, 0xbe}) && len(head) >= 8 && head[4] == 0 && head[5] == 0 && head[6] == 0 && head[7] < 0x20:
return "Mach-O"
}
return ""
}
// programHeadSize is how many bytes of a file are read to tell its format: enough for a PE header at its usual offsets.
const programHeadSize = 0x100
// historyPrograms names the compiled programs that a commit of base..head added or modified and the merge's tree no longer
// holds at that path (those it holds are in final already): each with the commit that added it. git is the merge's git runner.
func historyPrograms(git func(args ...string) (string, error), base, head string, final []Program) ([]Program, error) {
revs, err := git("rev-list", "--no-merges", "--reverse", base+".."+head)
if err != nil {
return nil, err
}
held := map[string]bool{}
for _, p := range final {
held[p.Path] = true
}
out, seen := []Program{}, map[string]bool{}
for _, c := range strings.Fields(revs) {
raw, err := git("diff-tree", "-r", "-z", "--no-commit-id", "--no-renames", "--root", "--diff-filter=AM", c)
if err != nil {
return nil, err
}
f := strings.Split(raw, "\x00")
for i := 0; i+1 < len(f); i += 2 {
meta := strings.Fields(strings.TrimPrefix(f[i], ":"))
if len(meta) < 4 || (meta[1] != "100644" && meta[1] != "100755") || held[f[i+1]] || seen[meta[3]] {
continue
}
seen[meta[3]] = true
blob, err := git("cat-file", "blob", meta[3])
if err != nil {
return nil, err
}
head := []byte(blob)
if len(head) > programHeadSize {
head = head[:programHeadSize]
}
if k := ExecutableKind(head); k != "" {
out = append(out, Program{Path: f[i+1], Kind: k, Commit: c})
}
}
}
return out, nil
}
// programsIn reads the output of `git diff --raw -z --no-renames --diff-filter=AM` (":<old mode> <new mode> <old sha> <new
// sha> <status>\0<path>\0" per file) and names each added or modified regular file of the tree dir that is a compiled
// program, from its first bytes read locally. A symlink (mode 120000) and a submodule (160000) are not read. It never
// answers nil, so a caller can tell no programs from none reported.
func programsIn(dir, raw string) []Program {
out := []Program{}
f := strings.Split(raw, "\x00")
for i := 0; i+1 < len(f); i += 2 {
meta := strings.Fields(strings.TrimPrefix(f[i], ":"))
if len(meta) < 2 || (meta[1] != "100644" && meta[1] != "100755") {
continue
}
p := f[i+1]
full := filepath.Join(dir, filepath.FromSlash(p))
if st, err := os.Lstat(full); err != nil || !st.Mode().IsRegular() {
continue
}
fh, err := os.Open(full)
if err != nil {
continue
}
head := make([]byte, programHeadSize)
n, _ := io.ReadFull(fh, head)
fh.Close()
if k := ExecutableKind(head[:n]); k != "" {
out = append(out, Program{Path: p, Kind: k})
}
}
return out
}
+72
View File
@@ -0,0 +1,72 @@
package buildstep
import (
"os"
"path/filepath"
"reflect"
"testing"
)
func pe(offset int, magic string) []byte {
b := make([]byte, 0x100)
copy(b, "MZ")
b[0x3c] = byte(offset)
copy(b[offset:], magic)
return b
}
func TestExecutableKindTellsProgramsFromText(t *testing.T) { // issue 1577
for _, c := range []struct {
name string
head []byte
want string
}{
{"elf", []byte("\x7fELF\x02\x01\x01"), "ELF"},
{"mach-o 64", []byte{0xcf, 0xfa, 0xed, 0xfe, 7, 0, 0, 1}, "Mach-O"},
{"mach-o 32 big", []byte{0xfe, 0xed, 0xfa, 0xce, 0, 0, 0, 7}, "Mach-O"},
{"fat mach-o", []byte{0xca, 0xfe, 0xba, 0xbe, 0, 0, 0, 2}, "Mach-O"},
{"java class", []byte{0xca, 0xfe, 0xba, 0xbe, 0, 0, 0, 0x34}, ""},
{"pe", pe(0x80, "PE\x00\x00"), "PE"},
{"mz text", []byte("MZ is a text file starting with two letters"), ""},
{"mz without a pe header", pe(0x80, "XX\x00\x00"), ""},
{"text", []byte("package main"), ""},
{"empty", nil, ""},
} {
if got := ExecutableKind(c.head); got != c.want {
t.Errorf("%s: kind = %q, want %q", c.name, got, c.want)
}
}
}
func TestProgramsInNamesTheAddedAndModifiedRegularFilesThatArePrograms(t *testing.T) { // issue 1577
dir := t.TempDir()
write := func(name string, b []byte) {
if err := os.MkdirAll(filepath.Dir(filepath.Join(dir, name)), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, name), b, 0o755); err != nil {
t.Fatal(err)
}
}
write("m/prog", []byte("\x7fELF\x02\x01"))
write("m/notes", []byte("plain text"))
write("m/mac", []byte{0xcf, 0xfa, 0xed, 0xfe, 7, 0, 0, 1})
if err := os.Symlink("prog", filepath.Join(dir, "m/link")); err != nil {
t.Fatal(err)
}
line := func(mode, path string) string { return ":000000 " + mode + " 0000 1111 A\x00" + path + "\x00" }
for _, c := range []struct {
name, raw string
want []Program
}{
{"none", "", []Program{}},
{"a program and a text", line("100755", "m/prog") + line("100644", "m/notes"), []Program{{"m/prog", "ELF", ""}}},
{"two programs", line("100644", "m/prog") + line("100755", "m/mac"), []Program{{"m/prog", "ELF", ""}, {"m/mac", "Mach-O", ""}}},
{"a symlink and a submodule are not read", line("120000", "m/link") + line("160000", "m/sub"), []Program{}},
{"a path the tree lacks", line("100644", "m/gone"), []Program{}},
} {
if got := programsIn(dir, c.raw); !reflect.DeepEqual(got, c.want) {
t.Errorf("%s: programs = %v, want %v", c.name, got, c.want)
}
}
}
+274 -22
View File
@@ -6,6 +6,7 @@ import (
"errors"
"fmt"
"sort"
"strings"
"sync"
"time"
@@ -76,6 +77,27 @@ type Puller struct {
// for this node, and wakes the puller on each change (watchCapacity): WatchConfigServed in a module, a test's own
// double. Until it hands a value the puller judges by Capacity.
WatchConfig func(key string, handle func(stdio.ProvisionChange) error) error
// CoresPerStep is what one ask of this module needs, read at every judgement (NodeCoresPerStep); with it the capacity
// judged by is bounded by the node's build slice: min(the configured capacity, floor(the quota's cores / CoresPerStep))
// (EffectiveCapacity). Nil bounds nothing by the CPU. Quota reads the quota (Quota when nil; a test's own).
CoresPerStep func() float64
Quota func() (percent int, from string)
// BuildHours is the node's build window, read at every judgement (NodeBuildHours); with WatchConfig set the value
// mesh-config resolves for `build-hours` takes its place once handed. Outside the window the capacity judged by is 0:
// steps running finish, no new one is taken; the puller wakes at the window's next edge. Nil builds at any hour.
BuildHours func() (*BuildWindow, string)
// Eligible says why this node cannot run the seat's asks at all now, and "" when it can: what the module proves of
// its machine and finds missing (mesh-build-image: /dev/net/tun cannot be opened, so buildah's network for a RUN line
// cannot be made; 2026-10-28 on ace). Read at every judgement, so it is cheap (a module answers its last look).
// While it says a reason the capacity judged by is 0, said with the reason: steps running finish, no new one is
// taken, and the asks go to a node that can run them instead of failing here. The module calls Wake when its answer
// changes. Nil is always eligible.
Eligible func() string
// Has says whether this node has a need an ask names (delivery.AskNeeds, issue 890). An ask needing what it lacks is
// declined: its step ends refused as declined (delivery.StepDeclined), the message is dropped for its asker to put
// again, and the worker it came from is not looked at again until the next pull's wait, so a node lacking the need
// does not take the asker's next copy at once. Nil has nothing.
Has func(need string) bool
mu sync.Mutex
inHand int
@@ -86,6 +108,19 @@ type Puller struct {
// watched is the capacity the config watch handed last, used in place of Capacity once hasWatched.
watched int
hasWatched bool
// hours is the build window the config watch handed last, and hoursFault why its value cannot be read; used in place
// of BuildHours once hasHours.
hours *BuildWindow
hoursFault string
hasHours bool
// judged is the capacity said last (log and capacity.<node>), so a change is said once.
judged delivery.Capacity
hasJudged bool
capWritten bool
capFault string
// declined are the kinds whose worker handed an ask this node declined, left out of the pulls without a wait until
// the next pull's wait ends.
declined map[string]bool
}
func (p *Puller) init() {
@@ -207,23 +242,136 @@ func (p *Puller) room() bool {
// roomNow is room, with the capacity and the asks in hand it was judged on.
func (p *Puller) roomNow() (ok bool, capacity, inHand int) {
p.mu.Lock()
watched, has := p.watched, p.hasWatched
p.mu.Unlock()
if has {
capacity = watched
} else if p.Capacity != nil {
capacity = p.Capacity()
}
capacity = p.CapacityDetail().Effective
p.mu.Lock()
defer p.mu.Unlock()
return !p.paused && p.inHand < capacity, capacity, p.inHand
}
// CapacityNow is the capacity the puller judges by: the config watch's last value, else Capacity.
// CapacityNow is the capacity the puller judges by (CapacityDetail's Effective).
func (p *Puller) CapacityNow() int {
_, capacity, _ := p.roomNow()
return capacity
return p.CapacityDetail().Effective
}
// CapacityDetail is the capacity the puller judges by now and why: the configured one (the config watch's last value,
// else Capacity), bounded by the node's build slice when CoresPerStep is set (EffectiveCapacity), its quota read now. A
// change from what it said last is logged and written to `capacity.<node>` of the pauses state (PausePut), so the
// delivery view says it too.
func (p *Puller) CapacityDetail() delivery.Capacity {
p.mu.Lock()
watched, has := p.watched, p.hasWatched
p.mu.Unlock()
configured, from := 0, "the settings file"
switch {
case has:
configured, from = watched, "mesh-config, as the config watch handed it"
case p.Capacity != nil:
configured = p.Capacity()
}
var c delivery.Capacity
if p.CoresPerStep == nil {
c = delivery.Capacity{Configured: configured, Effective: configured, QuotaFrom: "not judged: the module names no cores a step needs",
Why: fmt.Sprintf("its configured capacity %d", configured)}
} else {
quota := p.Quota
if quota == nil {
quota = Quota
}
percent, qfrom := quota()
c = EffectiveCapacity(configured, percent, qfrom, p.CoresPerStep())
}
c.ConfiguredFrom = from
w, fault := p.window()
if fault != "" {
c.BuildHoursFault = fault
c.Why = fmt.Sprintf("%s; at any hour, because its build hours cannot be read: %s", c.Why, fault)
}
if w != nil {
at := now()
zone, _ := at.Zone()
c.BuildHours = w.String() + " " + zone
if !w.Open(at) && c.Effective > 0 {
c.OutsideHours = true
c.Why = fmt.Sprintf("0, outside its build hours %s until %s (otherwise %s)", c.BuildHours, w.NextEdge(at).Format("15:04"), c.Why)
c.Effective = 0
} else if c.Effective > 0 {
c.Why = fmt.Sprintf("%s; within its build hours %s until %s", c.Why, c.BuildHours, w.NextEdge(at).Format("15:04"))
}
}
if p.Eligible != nil {
if why := p.Eligible(); why != "" {
c.Ineligible = why
c.Why = fmt.Sprintf("0, not eligible: %s (otherwise %s)", why, c.Why)
c.Effective = 0
}
}
p.sayCapacity(c)
return c
}
// window is the node's build window now (the config watch's last value, else BuildHours, else none), and why its value
// cannot be read when it cannot.
func (p *Puller) window() (*BuildWindow, string) {
p.mu.Lock()
w, fault, has := p.hours, p.hoursFault, p.hasHours
p.mu.Unlock()
if has {
return w, fault
}
if p.BuildHours != nil {
return p.BuildHours()
}
return nil, ""
}
// untilEdge is how long until the build window's next edge, and false when the node has no window.
func (p *Puller) untilEdge() (time.Duration, bool) {
w, _ := p.window()
if w == nil {
return 0, false
}
at := now()
return w.NextEdge(at).Sub(at), true
}
// sayCapacity logs c when it differs from what was said last, and writes it to the node's capacity key until a write
// lands (a failed write is tried again at the next judgement, its error said once).
func (p *Puller) sayCapacity(c delivery.Capacity) {
p.mu.Lock()
same := p.hasJudged && p.judged.Effective == c.Effective && p.judged.Configured == c.Configured && p.judged.ByCPU == c.ByCPU &&
p.judged.CoresPerStep == c.CoresPerStep && p.judged.QuotaCores == c.QuotaCores && p.judged.OutsideHours == c.OutsideHours &&
p.judged.BuildHours == c.BuildHours && p.judged.BuildHoursFault == c.BuildHoursFault &&
p.judged.Ineligible == c.Ineligible
if !same {
p.judged, p.hasJudged, p.capWritten = c, true, false
}
write := !p.capWritten && p.PausePut != nil
p.mu.Unlock()
if !same {
p.logf("the %s build capacity of %s is %s (configured from %s)", p.Language, p.Node, c.Why, c.ConfiguredFrom)
}
if !write {
return
}
key, err := delivery.CapacityKey(p.Node)
if err != nil {
return
}
c.At = now()
raw, err := delivery.EncodeCapacity(c)
if err == nil {
_, err = p.PausePut(key, json.RawMessage(raw))
}
p.mu.Lock()
defer p.mu.Unlock()
if err == nil {
p.capWritten, p.capFault = true, ""
return
}
if err.Error() != p.capFault {
p.capFault = err.Error()
p.logf("the %s build capacity of %s could not be written to %s (tried again at the next judgement): %v", p.Language, p.Node, key, err)
}
}
// Pull takes and runs asks until ctx ends. An ask still running then is neither ended nor acknowledged: the stream
@@ -239,6 +387,7 @@ func (p *Puller) Pull(ctx context.Context) error {
p.logf("the pause of %s cannot be read: %v; pulling as not paused", p.Node, err)
}
go p.watchCapacity() // WatchConfigServed waits for Serve, which a module starts after its pull
go p.watchHours()
wait := p.PullWait
if wait <= 0 {
wait = DefaultPullWait
@@ -251,16 +400,29 @@ func (p *Puller) Pull(ctx context.Context) error {
// Parked until something wakes it: a capacity change (the config watch), a resume, or a step's end. Every
// park logs once, so a puller that sits idle says why (issue 1887).
p.logf("the %s puller on %s parks: %d of capacity %d in hand, paused %v", p.Language, p.Node, inHand, capacity, p.Paused())
var edge <-chan time.Time // the build window's next edge, when the node has one: one timer, not a poll
stop := func() bool { return false }
if d, ok := p.untilEdge(); ok {
t := time.NewTimer(d + time.Second)
edge, stop = t.C, t.Stop
}
select {
case <-ctx.Done():
stop()
return ctx.Err()
case <-p.wake:
case <-edge:
}
stop()
_, capacity, inHand = p.roomNow()
p.logf("the %s puller on %s wakes: %d of capacity %d in hand, paused %v", p.Language, p.Node, inHand, capacity, p.Paused())
continue
}
msg, err := p.next(ctx, wait)
pullWait := wait
if d, ok := p.untilEdge(); ok && d < pullWait {
pullWait = d // a pull does not wait past the window's close
}
msg, err := p.next(ctx, pullWait)
if err != nil {
if ctx.Err() != nil {
return ctx.Err()
@@ -283,17 +445,59 @@ func (p *Puller) Pull(ctx context.Context) error {
// next is one ask from the workers in their order: one already waiting on delivery, check or recheck, and when none
// waits, a pull of the delivery worker that waits for one.
//
// A kind whose worker handed an ask this node declined is left out until that wait ended (issue 890): the pull's own
// wait, no timer of its own.
func (p *Puller) next(ctx context.Context, wait time.Duration) (*Pulled, error) {
p.mu.Lock()
declined := p.declined
p.mu.Unlock()
for _, kind := range delivery.AskKinds {
if declined[kind] {
continue
}
msg, err := p.Source.Next(ctx, kind, 0)
if err != nil || msg != nil {
return withKind(msg, kind), err
}
}
msg, err := p.Source.Next(ctx, delivery.AskDelivery, wait)
p.mu.Lock()
p.declined = nil
p.mu.Unlock()
return withKind(msg, delivery.AskDelivery), err
}
// decline ends the step of an ask this node lacks a need of, refused as declined with what it lacks, and drops the
// message: the asker puts the ask again under a new id (issue 890). A message whose decline cannot be written is handed
// back, as one whose start cannot be.
func (p *Puller) decline(msg *Pulled, ask delivery.StepAsk, lacks []string) {
step, key, err := begin(p.Put, p.Node, ask.Kind+" ask", ask.ID, AskWhat(ask))
if err == nil {
why := Refuse("the ask %s needs %s, and %s lacks %s: it is left to a node that has it", ask.ID,
strings.Join(ask.Needs, ", "), p.Node, strings.Join(lacks, ", "))
step.Ended = now().UTC()
step.State, step.Reason = delivery.StepRefused, why.Error()
step.Result, err = json.Marshal(delivery.StepDeclined{Declined: true, Needs: ask.Needs, Lacks: lacks})
if err == nil {
err = writeEnd(p.Put, key, step)
}
}
if err != nil {
p.logf("the ask %s cannot be declined on %s, so it is given back: %v", ask.ID, p.Node, err)
_ = msg.Nak()
return
}
p.logf("the ask %s is declined on %s: it needs %s, and this node lacks %s", ask.ID, p.Node, strings.Join(ask.Needs, ", "), strings.Join(lacks, ", "))
_ = msg.Term()
p.mu.Lock()
if p.declined == nil {
p.declined = map[string]bool{}
}
p.declined[msg.Kind] = true
p.mu.Unlock()
}
func withKind(msg *Pulled, kind string) *Pulled {
if msg != nil {
msg.Kind = kind
@@ -314,7 +518,11 @@ func (p *Puller) take(ctx context.Context, msg *Pulled) {
_ = msg.Term()
return
}
step, key, err := begin(p.Put, p.Node, ask.Kind+" ask", ask.ID)
if lacks := delivery.Lacks(ask.Needs, p.Has); len(lacks) > 0 {
p.decline(msg, ask, lacks)
return
}
step, key, err := begin(p.Put, p.Node, ask.Kind+" ask", ask.ID, AskWhat(ask))
if err != nil {
p.logf("the ask %s cannot start on %s: %v", ask.ID, p.Node, err)
_ = msg.Nak()
@@ -340,6 +548,7 @@ func (p *Puller) take(ctx context.Context, msg *Pulled) {
}
p.running[ask.ID] = RunningAsk{ID: ask.ID, Kind: ask.Kind, Module: module, Started: step.Started}
p.mu.Unlock()
p.logf("the %s ask %s starts on %s: %s", ask.Kind, ask.ID, p.Node, step.What)
go p.run(ctx, msg, ask, key, step)
}
@@ -370,7 +579,11 @@ func (p *Puller) run(ctx context.Context, msg *Pulled, ask delivery.StepAsk, key
}
}
}()
result, err := protect(ctx, func(ctx context.Context) (any, error) { return p.Run(ctx, ask) })
ph := &phases{put: p.Put, key: key, step: step, logf: p.Logf, prefix: fmt.Sprintf("the %s ask %s on %s", ask.Kind, ask.ID, p.Node)}
result, err := protect(ctx, func(ctx context.Context) (any, error) {
return p.Run(context.WithValue(ctx, phaseKey{}, ph), ask)
})
step = ph.end() // its end keeps the phase it ended in
close(done)
if ctx.Err() != nil {
// The process stops: the ask is neither ended nor acknowledged here, so the stream gives it to a node that runs.
@@ -383,12 +596,21 @@ func (p *Puller) run(ctx context.Context, msg *Pulled, ask delivery.StepAsk, key
_ = msg.Nak()
return
}
outcome := "passed"
if err != nil {
outcome = "did not pass: " + err.Error()
if len(outcome) > 300 {
outcome = outcome[:300] + "..."
}
}
p.logf("the %s ask %s ended on %s after %s, in its phase %q; %s", ask.Kind, ask.ID, p.Node,
now().Sub(step.Started).Round(time.Second), step.Phase, outcome)
if aerr := msg.Ack(); aerr != nil {
p.logf("the ask %s ended on %s and its acknowledgement failed: %v", ask.ID, p.Node, aerr)
}
}
// watchCapacity follows the module's config value `capacity` when WatchConfig is set (issue 1887, 2026-10-25: g14's
// watchCapacity follows the module's config value `capacity` when WatchConfig is set (issue 1887, 2026-10-25: the laptop's
// puller started at capacity 0 sat parked with capacity 2 set, because nothing woke it to read its settings file
// again). The value comes as mesh-config resolved it for this node, the event of the change itself: the runtime hands
// the value now and then each change its machine's provisioner client says moved, so nothing polls, and the puller does
@@ -419,6 +641,42 @@ func (p *Puller) watchCapacity() {
}
}
// watchHours follows the module's config value `build-hours` when WatchConfig and BuildHours are set, as watchCapacity
// follows `capacity`: each value handed is kept, used in place of BuildHours, and wakes the puller. A value that cannot
// be read is no window, said; a watch that cannot start leaves the puller on BuildHours, said once.
func (p *Puller) watchHours() {
if p.WatchConfig == nil || p.BuildHours == nil {
return
}
err := p.WatchConfig(BuildHoursSetting, func(c stdio.ProvisionChange) error {
setting := ""
var w *BuildWindow
var why string
switch {
case c.Refused != "":
why = "its build hours are no longer answered: " + c.Refused
case len(c.Value) > 0 && string(c.Value) != "null" && json.Unmarshal(c.Value, &setting) != nil:
why = fmt.Sprintf("the setting %s %s is not a string", BuildHoursSetting, c.Value)
default:
var err error
if w, err = ParseBuildWindow(setting); err != nil {
why = err.Error()
}
}
p.mu.Lock()
p.hours, p.hoursFault, p.hasHours = w, why, true
p.mu.Unlock()
if why != "" {
p.logf("the %s puller on %s builds at any hour: %s", p.Language, p.Node, why)
}
p.Wake()
return nil
})
if err != nil {
p.logf("the %s build hours of %s cannot be followed on the bus, so they are read from the settings file: %v", p.Language, p.Node, err)
}
}
// WatchConfigServed is stdio.WatchConfig once Serve has the runtime's channel (stdio.Served): a build module starts its
// pull before it serves, and a watch asked before that would be refused.
func WatchConfigServed(key string, handle func(stdio.ProvisionChange) error) error {
@@ -431,13 +689,7 @@ func (p *Puller) capacityFrom(c stdio.ProvisionChange) (int, string) {
if c.Refused != "" {
return 0, "its capacity is no longer answered: " + c.Refused
}
setting := ""
if len(c.Value) > 0 && string(c.Value) != "null" {
if err := json.Unmarshal(c.Value, &setting); err != nil {
return 0, fmt.Sprintf("its capacity %s is not a string", c.Value)
}
}
n, err := capacityOf(setting, p.Node)
n, err := capacityOf(c.Value)
if err != nil {
return 0, err.Error()
}
+2 -2
View File
@@ -22,7 +22,7 @@ func (w *configWatch) watch(key string, handle func(stdio.ProvisionChange) error
w.asked, w.handle = key, handle
value := w.value
w.mu.Unlock()
return handle(stdio.ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(`"` + value + `"`)})
return handle(stdio.ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(value)})
}
func (w *configWatch) set(t *testing.T, value string) {
@@ -33,7 +33,7 @@ func (w *configWatch) set(t *testing.T, value string) {
if handle == nil {
t.Fatal("the puller watches no config value")
}
if err := handle(stdio.ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(`"` + value + `"`)}); err != nil {
if err := handle(stdio.ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(value)}); err != nil {
t.Fatal(err)
}
}
+83 -9
View File
@@ -104,20 +104,38 @@ func TestPullTakesDeliveryBeforeCheck(t *testing.T) {
}
// 1887/A2a, 1887/AC-pull: given capacity 2 and three asks waiting, two run and the third waits, and it starts when
// one ends, with no timer.
// one ends, with no timer. Issue 1806's follow-up: the 2 is the CPU's, a configured 3 bounded by a 12-core slice at 6
// cores a step, said once in capacity.<node>; and issue 1890: a run's phase is written to its step key and logged with
// its ask id, and its end keeps the last phase.
func TestPullHoldsAtMostItsCapacity(t *testing.T) {
src, st := NewFakeAsks(), newSteps()
asks := []delivery.StepAsk{}
for i := 0; i < 3; i++ {
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
a := testAsk(delivery.AskDelivery)
asks = append(asks, a)
_ = src.PublishAsk(a)
}
release := make(chan struct{})
var mu sync.Mutex
started := 0
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: st.put, Capacity: func() int { return 2 },
var said []string
capacities := map[string][]byte{}
quota := 1200
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: st.put, Capacity: func() int { return 3 },
CoresPerStep: func() float64 { return 6 },
Quota: func() (int, string) { mu.Lock(); defer mu.Unlock(); return quota, "a test's slice" },
PausePut: func(key string, v any) (uint64, error) {
mu.Lock()
capacities[key] = v.(json.RawMessage)
mu.Unlock()
return 1, nil
},
Logf: func(f string, a ...any) { mu.Lock(); said = append(said, fmt.Sprintf(f, a...)); mu.Unlock() },
PullWait: 10 * time.Millisecond, Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
mu.Lock()
started++
mu.Unlock()
Phase(ctx, "unit: tests of 3 packages")
<-release
return nil, nil
}}
@@ -129,31 +147,87 @@ func TestPullHoldsAtMostItsCapacity(t *testing.T) {
if p.InHand() != 2 || src.Waiting(delivery.AskDelivery) != 1 {
t.Fatalf("in hand %d, waiting %d; want 2 and 1", p.InHand(), src.Waiting(delivery.AskDelivery))
}
c := p.CapacityDetail()
if c.Effective != 2 || c.Configured != 3 || c.ByCPU != 2 || c.QuotaCores != 12 || !strings.Contains(c.Why, "bounded by the CPU") {
t.Fatalf("the capacity judged by = %+v, want 2 of a configured 3 bounded by 12 cores / 6", c)
}
mu.Lock()
raw := capacities["capacity.shanks"]
capLines := 0
for _, l := range said {
if strings.Contains(l, "build capacity of shanks is 2, bounded by the CPU") {
capLines++
}
}
mu.Unlock()
if got, err := delivery.DecodeCapacity(raw); err != nil || got.Effective != 2 || capLines != 1 {
t.Fatalf("capacity.shanks = %s (%v), said %d times; want effective 2, said once", raw, err, capLines)
}
waitFor(t, "a phase on the step key", func() bool {
key, _ := delivery.AskStepKey("shanks", p.Running()[0].ID)
s, _ := st.get(key)
return s.State == delivery.StepRunning && s.Phase == "unit: tests of 3 packages" && !s.PhaseAt.IsZero() && s.Owner == Owner // 1890/AC5
})
release <- struct{}{}
waitFor(t, "the third taken", func() bool { return src.Waiting(delivery.AskDelivery) == 0 })
// Taken from the source comes a moment before its run starts: wait for the start, not only the take.
waitFor(t, "the third started", func() bool { mu.Lock(); defer mu.Unlock(); return started == 3 })
close(release)
waitFor(t, "all acked", func() bool { a, _, _ := src.Fates(); return len(a) == 3 })
for _, a := range asks {
key, _ := delivery.AskStepKey("shanks", a.ID)
if s, _ := st.get(key); s.State != delivery.StepPassed || s.Phase != "unit: tests of 3 packages" || s.Owner != Owner {
t.Errorf("the end of %s = %+v, want passed in its last phase", a.ID, s)
}
}
mu.Lock()
logged := strings.Join(said, "\n")
quota = 600 // the cap lowered: one step at 6 cores, followed at the next judgement
mu.Unlock()
if !strings.Contains(logged, "the delivery ask "+asks[0].ID+" on shanks: unit: tests of 3 packages") {
t.Errorf("no journal line names the ask's phase: %s", logged)
}
if c := p.CapacityDetail(); c.Effective != 1 {
t.Errorf("after the quota fell to 6 cores, capacity %+v, want 1", c)
}
}
// An ask that is not one, or not this seat's, is dropped and runs nowhere.
// An ask that is not one, or not this seat's, is dropped and runs nowhere. And issue 890/AC1: a check ask needing
// node-build-node is declined on a node that lacks it (its step refused as declined, naming what it lacks, the message
// dropped for the asker to put again), and runs on a node that has it.
func TestPullDropsWhatIsNoAsk(t *testing.T) {
src, st := NewFakeAsks(), newSteps()
src.Publish(delivery.AskDelivery, []byte(`{"v":1}`))
other := testAsk(delivery.AskDelivery)
other.Language = delivery.AskNode
_ = src.PublishAsk(other)
placed := testAsk(delivery.AskCheck)
placed.Needs = []string{delivery.NeedNodeBuildNode}
if err := src.PublishAsk(placed); err != nil {
t.Fatal(err)
}
ran := false
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: st.put, Capacity: func() int { return 1 },
p := &Puller{Source: src, Language: delivery.AskGo, Node: "ace", Put: st.put, Capacity: func() int { return 1 },
PullWait: 10 * time.Millisecond, Run: func(ctx context.Context, a delivery.StepAsk) (any, error) { ran = true; return nil, nil }}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
waitFor(t, "two termed", func() bool { _, tm, _ := src.Fates(); return len(tm) == 2 })
waitFor(t, "three termed", func() bool { _, tm, _ := src.Fates(); return len(tm) == 3 })
if ran {
t.Fatal("a wrong ask ran")
t.Fatal("a wrong or a declined ask ran")
}
key, _ := delivery.AskStepKey("ace", placed.ID)
s, _ := st.get(key)
if d, ok := delivery.DeclinedOf(s); !ok || len(d.Lacks) != 1 || d.Lacks[0] != delivery.NeedNodeBuildNode || !strings.Contains(s.Reason, "lacks node-build-node") {
t.Fatalf("the declined step = %+v", s)
}
has, hasSteps := NewFakeAsks(), newSteps()
_ = has.PublishAsk(placed)
q := &Puller{Source: has, Language: delivery.AskGo, Node: "g14", Put: hasSteps.put, Capacity: func() int { return 1 },
PullWait: 10 * time.Millisecond, Has: func(n string) bool { return n == delivery.NeedNodeBuildNode },
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) { return nil, nil }}
go q.Pull(ctx)
waitFor(t, "the placed ask acked where its need is met", func() bool { a, _, _ := has.Fates(); return len(a) == 1 })
}
// load reads the recorded steps state's raw values: the pause key is not a step, so it is kept apart.
@@ -208,7 +282,7 @@ func TestPullAtCapacityZeroTakesNothing(t *testing.T) {
t.Run("config watch wakes", func(t *testing.T) {
src, st := NewFakeAsks(), newSteps()
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
w := &configWatch{value: "shanks=3,g14=0"}
w := &configWatch{value: "0"}
p := &Puller{Source: src, Language: delivery.AskGo, Node: "g14", Put: st.put, PullWait: 10 * time.Millisecond,
Capacity: func() int { return 0 }, // the settings file, not refilled yet
WatchConfig: w.watch,
@@ -224,7 +298,7 @@ func TestPullAtCapacityZeroTakesNothing(t *testing.T) {
if w.asked != CapacitySetting {
t.Fatalf("the puller watches %q, want %q", w.asked, CapacitySetting)
}
w.set(t, "shanks=3,g14=2")
w.set(t, "2")
waitFor(t, "the ask taken once the capacity moved to 2", func() bool { a, _, _ := src.Fates(); return len(a) == 1 })
})
for _, c := range []struct {
+5 -2
View File
@@ -14,10 +14,13 @@ func SeatTools(seat, what string, p *Puller) []stdio.Tool {
Run: func(map[string]any) (any, error) { return run() }}
}
current := func() (any, error) {
return map[string]any{"node": p.Node, "capacity": p.CapacityNow(), "running": p.Running(), "paused": p.Paused()}, nil
c := p.CapacityDetail()
return map[string]any{"node": p.Node, "capacity": c.Effective, "capacity_why": c.Why, "capacity_detail": c,
"running": p.Running(), "paused": p.Paused()}, nil
}
return []stdio.Tool{
verb("current", "The "+what+" asks this node runs now (each one's id, kind and when it started), its build capacity, "+
verb("current", "The "+what+" asks this node runs now (each one's id, kind and when it started), its build capacity (the configured one, "+
"bounded by the cores its node's build slice gives over the cores a step needs) and why, "+
"and whether it is paused.", current),
verb("pause", "Take no new "+what+" ask on this node until resumed; an ask running here ends in its place. Kept "+
"across a restart (the node's pause key).", func() (any, error) {
+365
View File
@@ -0,0 +1,365 @@
package buildstep
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"strconv"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// **A step left running** (novox/hq issue 1890, note 5). A build module that stops while a step runs leaves its key in
// state running for good: nothing else writes that key's end, and the stream's redelivery only rewrites it when the same
// ask comes back. At its start a module ends the steps its node left running by another process (EndStale: every step
// names its owner, delivery.Step.Owner; note 21), and its step-end tool ends one by hand (EndOne). The conformance kit's
// left-running case holds every build module to the start sweep (conformance.LeftRunning).
// RestartReason is why a step a previous process left running is ended.
const RestartReason = "the build module restarted while it ran"
// Boot is when this process began. A step with no owner (written by an older build) started at or after it is taken
// for this process's, which may still run it, so EndOne refuses it.
var Boot = time.Now()
// Owner is this process as a step's owner (delivery.Step.Owner): `<boot id>.<pid>.<start>`, the node's boot id
// (/proc/sys/kernel/random/boot_id), the pid and Boot in milliseconds, base 36. The start keeps two processes apart
// that a reboot or a pid namespace gave the same pid. A running step whose owner is another is a previous process's.
var Owner = ownerOf(bootID, os.Getpid(), Boot)
// bootID is the node's boot id as Owner holds it; noBootID when it could not be read, which EndStaleAtStart says once.
var bootID = readBootID()
const noBootID = "no-boot-id"
func ownerOf(bootID string, pid int, start time.Time) string {
return bootID + "." + strconv.Itoa(pid) + "." + strconv.FormatInt(start.UnixMilli(), 36)
}
// readBootID is the node's boot id, or noBootID where it cannot be read (the pid and the start then tell the processes
// apart).
func readBootID() string {
b, err := os.ReadFile("/proc/sys/kernel/random/boot_id")
if id := strings.TrimSpace(string(b)); err == nil && id != "" {
return id
}
return noBootID
}
// Scan reads and writes a steps state with its revisions: its keys, a key's stored value and revision (nil and 0 for
// none), a write that lands only while the key is still at the revision read (KeptState.Update), and a delete of a key
// still at the revision read; both fail with stdio.ErrChanged when another writer was first.
type Scan struct {
Keys func() ([]string, error)
Get func(key string) (json.RawMessage, uint64, error)
Update func(key string, value any, revision uint64) (uint64, error)
Delete func(key string, revision uint64) error
}
// ScanOf is the Scan of a state the process keeps.
//
// Its Delete reads the key again and deletes it only while it is still at the revision read: the runtime's
// mesh/state.delete takes no revision yet, so a write landing between that read and the delete (a few milliseconds) is
// deleted with it. For a step key that is a redelivered ask begun on this node in that moment: its running key leaves
// the steps state until its run writes its end, which its asker still reads (a watch takes puts only).
func ScanOf(s stdio.KeptState) Scan {
get := func(key string) (json.RawMessage, uint64, error) {
e, err := s.Get(key)
if err != nil || e == nil {
return nil, 0, err
}
return e.Value, e.Revision, nil
}
return Scan{Keys: s.Keys, Update: s.Update, Get: get, Delete: func(key string, revision uint64) error {
_, rev, err := get(key)
if err != nil {
return err
}
if rev != revision {
return stdio.ErrChanged
}
return s.Delete(key)
}}
}
// isAsk says a step is a pulled ask's: its key is <node>.<ask id> and its tool is "<kind> ask" (a hand-started step
// has the tool's name). An ask's key is the stream's to settle, never a restart's: the puller leaves an ask it was
// running unacknowledged so the stream redelivers it, and the asker's wait takes the first end it reads on the key.
// A key a gone process owns is deleted, never ended (EndStale); one with no owner is ended only once the asker's bound
// has passed (delivery.StepBound), when it is nobody's.
func isAsk(key string, s delivery.Step) bool {
_, _, err := delivery.ParseAskStepKey(key)
return err == nil && strings.HasSuffix(s.Tool, " ask")
}
// AskWhat says what an ask is, in one line: its kind, and its repository, head and levels, or its module and
// fingerprint.
func AskWhat(a delivery.StepAsk) string {
short := func(s string) string {
if len(s) > 12 {
return s[:12]
}
return s
}
str := func(m map[string]any, k string) string { v, _ := m[k].(string); return v }
parts := []string{a.Kind + " ask"}
switch {
case a.Check != nil:
parts = append(parts, str(a.Check.Fetch, "repository")+"@"+short(str(a.Check.Fetch, "commit")), "levels "+strings.Join(a.Check.Levels, ","))
if a.Check.Run != "" {
parts = append(parts, "run "+a.Check.Run)
}
case a.Build != nil:
parts = append(parts, "module "+a.Build.Module)
if r := str(a.Build.Fetch, "repository"); r != "" {
parts = append(parts, r+"@"+short(str(a.Build.Fetch, "commit")))
}
if a.Build.FP != "" {
parts = append(parts, "fp "+short(a.Build.FP))
}
case a.Compose != nil:
parts = append(parts, "compose", str(a.Compose.Fetch, "repository")+"@"+short(str(a.Compose.Fetch, "commit")))
}
return strings.Join(parts, ", ")
}
// endRunning writes step ended failed with why, only while its key is still at revision.
func endRunning(scan Scan, key string, step delivery.Step, revision uint64, why string) error {
step.State, step.Reason, step.Ended, step.Result = delivery.StepFailed, why, now().UTC(), nil
raw, err := delivery.EncodeStep(step)
if err != nil {
return err
}
_, err = scan.Update(key, json.RawMessage(raw), revision)
return err
}
// EndStale ends this node's running steps that no process of this module runs any more, and answers their ids. Only keys
// of node are read; a value that is no step is left alone.
// - A hand-started step with an owner (delivery.Step.Owner) that is not this process (Owner) is ended failed with
// RestartReason at once, whatever its age: its process is gone and nothing redelivers it, so nothing else would
// write its end (issue 1890, note 21).
// - A pulled ask's step with an owner that is not this process is deleted, not ended (novox/hq ADR 0525 §6): the
// stream redelivers an ask whose node stopped (the puller neither ends nor acknowledges one at its stop), and its
// asker takes the first end it reads, so an end written here would fail a check the redelivery would still run. Its
// asker's watch takes puts only and keeps waiting for the redelivery's key, its bound the backstop; the stale running
// key leaves the steps state, and with it the delivery view's and the checker's count of what runs (note 21: it
// held one of a build node's two slots).
// - A step owned by this process is left: it may run now. A redelivered ask begun on this node is owned by it.
// - A step with no owner (written by an older build) keeps the earlier rule: a hand-started step started before
// `before` (the start of this process) is ended; a pulled ask is left to the stream's redelivery (isAsk) until its
// start is older than delivery.StepBound, and then ended "past the step bound: its asker stopped waiting at <time>".
//
// Each end and delete is logged, naming the step, its owner and why. A key written by another writer since it was read
// (a redelivered ask's start, a real end) is skipped.
func EndStale(scan Scan, node string, before time.Time, logf func(string, ...any)) ([]string, error) {
keys, err := scan.Keys()
if err != nil {
return nil, fmt.Errorf("buildstep: listing the steps of %s: %w", node, err)
}
var ended []string
for _, k := range keys {
n, _, perr := delivery.ParseStepKey(k)
if perr != nil || n != node {
continue
}
raw, rev, err := scan.Get(k)
if err != nil {
return ended, fmt.Errorf("buildstep: reading the step %s: %w", k, err)
}
if len(raw) == 0 {
continue
}
step, err := delivery.DecodeStep(raw)
if err != nil || step.State != delivery.StepRunning {
continue
}
why, said := staleWhy(k, step, before)
if why == "" {
continue
}
done := "ended"
if step.Owner != "" && isAsk(k, step) {
done = "deleted, left to the stream's redelivery"
if scan.Delete == nil {
err = fmt.Errorf("the steps state takes no delete")
} else {
err = scan.Delete(k, rev)
}
} else {
err = endRunning(scan, k, step, rev, why)
}
if err != nil {
if errors.Is(err, stdio.ErrChanged) {
continue
}
return ended, fmt.Errorf("buildstep: ending the step %s: %w", k, err)
}
if logf != nil {
logf("the step %s (%s, started %s) was left running and is %s: %s; %s", k, step.Tool,
step.Started.UTC().Format(time.RFC3339), done, why, said)
}
ended = append(ended, step.ID)
}
return ended, nil
}
// staleWhy is why a running step is ended (or, for an ask with an owner, deleted) by EndStale and what the log says of
// its owner, or "" to leave it.
func staleWhy(key string, step delivery.Step, before time.Time) (why, said string) {
switch {
case step.Owner == Owner:
return "", ""
case step.Owner != "":
return RestartReason, "its owner " + step.Owner + " is not this process " + Owner
case isAsk(key, step):
if !step.Started.Add(delivery.StepBound).Before(now()) {
return "", ""
}
return "past the step bound: its asker stopped waiting at " + step.Started.Add(delivery.StepBound).UTC().Format("2006-01-02T15:04:05Z"),
"it names no owner (an older build wrote it)"
case step.Started.Before(before):
return RestartReason, "it names no owner (an older build wrote it) and started before this process"
}
return "", ""
}
// EndOne ends this node's running step id (or the key <node>.<id>) as failed with why. A step that is not running is
// refused, and so is one this process owns (Owner; for a step with no owner, one started since Boot: it may still run,
// whether a pulled ask or a step a tool started), one live reports as held, and one whose key changed since it was read.
func EndOne(scan Scan, node, id, why string, live func(id string) bool) (delivery.Step, error) {
id = strings.TrimSpace(id)
if n, rest, ok := strings.Cut(id, "."); ok {
if n != node {
return delivery.Step{}, fmt.Errorf("the step %q is of node %q, this module runs on %q", id, n, node)
}
id = rest
}
if strings.TrimSpace(why) == "" {
return delivery.Step{}, fmt.Errorf("say why the step is ended (why)")
}
key, err := delivery.StepKey(node, id)
if err != nil {
return delivery.Step{}, err
}
raw, rev, err := scan.Get(key)
if err != nil {
return delivery.Step{}, err
}
if len(raw) == 0 {
return delivery.Step{}, fmt.Errorf("no step %s in the steps state", key)
}
step, err := delivery.DecodeStep(raw)
if err != nil {
return delivery.Step{}, err
}
if step.State != delivery.StepRunning {
return delivery.Step{}, fmt.Errorf("the step %s is %s, not running", key, step.State)
}
if step.Owner == Owner || (step.Owner == "" && !step.Started.Before(Boot)) || (live != nil && live(id)) {
return delivery.Step{}, fmt.Errorf("the step %s was started by this process and may run now; only a step a previous process left is ended", key)
}
if err := endRunning(scan, key, step, rev, why); err != nil {
if errors.Is(err, stdio.ErrChanged) {
return delivery.Step{}, fmt.Errorf("the step %s changed while it was being ended; read it again", key)
}
return delivery.Step{}, err
}
step.State, step.Reason = delivery.StepFailed, why
return step, nil
}
// StepEndTool is the module's step-end tool, named name (build_go_step_end): it ends one step left running, with a
// reason, refusing one this process started. p may be nil.
func StepEndTool(name string, node string, scan Scan, p *Puller) stdio.Tool {
return stdio.Tool{Name: name,
Description: "End one step of this machine that is stuck in state running (for instance left by a restart of the module) " +
"as failed, with the reason given; its asker then sees it over, so ending an ask still in its stream fails its asker. " +
"A step not running, or one this process started, is refused. " +
"id is the step id or the key <node>.<id>.",
Input: map[string]any{"type": "object", "required": []string{"id", "why"}, "properties": map[string]any{
"id": map[string]any{"type": "string", "description": "the step id, or the key <node>.<id>"},
"why": map[string]any{"type": "string", "description": "why it is ended; it becomes the step's reason"}}},
Run: func(args map[string]any) (any, error) {
id, _ := args["id"].(string)
why, _ := args["why"].(string)
var live func(string) bool
if p != nil {
live = func(id string) bool {
for _, r := range p.Running() {
if r.ID == id {
return true
}
}
return false
}
}
return EndOne(scan, node, id, why, live)
}}
}
// StaleAttempts and StaleRetry bound EndStaleAtStart's tries: the steps state is readable once the runtime serves.
var (
StaleAttempts = 6
StaleRetry = 5 * time.Second
)
// StaleServed is what EndStaleAtStart waits for before it reads the steps state: the runtime serving (stdio.Served).
// The conformance kit's left-running case (conformance.LeftRunning) hands it a closed channel, to run a module's own
// start sweep in its test.
var StaleServed = stdio.Served
// EndStaleAtStart is what a build module runs once at its start, in a goroutine: it waits for the runtime to serve,
// then ends this node's steps a previous process left running (EndStale), trying again when the state cannot be read
// yet. A step names its owner, so `before` (the start of this process) matters only for a step with no owner, written
// by an older build: such a step started before it is a previous process's. When the node's boot id cannot be read it
// says so once: Owner then tells processes apart by their pid and start only.
func EndStaleAtStart(ctx context.Context, scan Scan, node string, before time.Time, logf func(string, ...any)) {
select {
case <-ctx.Done():
return
case <-StaleServed():
}
if bootID == noBootID && logf != nil {
logf("the boot id of %s cannot be read (/proc/sys/kernel/random/boot_id): its steps' owner is %s, told apart by pid and start only", node, Owner)
}
for attempt := 1; ; attempt++ {
ended, err := EndStale(scan, node, before, logf)
if err == nil {
if logf != nil {
logf("%d steps of %s left running by a previous process ended", len(ended), node)
}
return
}
if logf != nil {
logf("ending the steps left running (try %d of %d): %v", attempt, StaleAttempts, err)
}
if attempt >= StaleAttempts {
return
}
select {
case <-ctx.Done():
return
case <-time.After(StaleRetry):
}
}
}
// EndStaleRound is EndStale as the module's periodic round runs it: at once, and only when the runtime serves (a round
// before that is skipped, the start's EndStaleAtStart and the next round cover it). It ends the steps a previous
// process left and the asks past delivery.StepBound.
func EndStaleRound(scan Scan, node string, logf func(string, ...any)) {
select {
case <-stdio.Served():
default:
return
}
if _, err := EndStale(scan, node, Boot, logf); err != nil && logf != nil {
logf("ending the steps left running: %v", err)
}
}
+161
View File
@@ -0,0 +1,161 @@
package buildstep
import (
"encoding/json"
"fmt"
stdio "git.novox.be/novox/mesh-sdk/go"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// scanOf is the steps a keys double holds, last value per key, its revision the count of its puts; a write through
// Update lands only at the revision read, as the state's does. race, when set, runs once before the next Update's check.
func scanOf(k *keys, race func(key string)) Scan {
return Scan{
Keys: func() ([]string, error) {
k.mu.Lock()
defer k.mu.Unlock()
var out []string
for key := range k.puts {
out = append(out, key)
}
return out, nil
},
Get: func(key string) (json.RawMessage, uint64, error) {
k.mu.Lock()
defer k.mu.Unlock()
s, ok := k.puts[key]
if !ok {
return nil, 0, nil
}
raw, err := delivery.EncodeStep(s[len(s)-1])
return raw, uint64(len(s)), err
},
Delete: func(key string, revision uint64) error {
k.mu.Lock()
defer k.mu.Unlock()
if uint64(len(k.puts[key])) != revision {
return stdio.ErrChanged
}
delete(k.puts, key)
return nil
},
Update: func(key string, value any, revision uint64) (uint64, error) {
if race != nil {
race(key)
}
k.mu.Lock()
n := uint64(len(k.puts[key]))
k.mu.Unlock()
if n != revision {
return 0, stdio.ErrChanged
}
return k.put(key, value)
},
}
}
// 1890 and 1890/AC6: EndStale ends this node's hand-started running steps of another owner at once, whatever their age,
// and deletes its asks of another owner (the stream redelivers them, ADR 0525 §6); of
// the steps with no owner (an older build's), the hand-started ones left running before the process began and the asks
// past the step bound (a younger ask is the stream's to redeliver); never a step this process owns or a key written
// meanwhile. EndOne ends one by hand, never one this process owns or started.
func TestEndStaleAndEndOne(t *testing.T) {
k := newKeys()
boot := time.Now()
oldBoot := Boot
Boot = boot
defer func() { Boot = oldBoot }()
gone := ownerOf("b00t", 4242, boot.Add(-time.Hour))
put := func(node, id, tool, state string, started time.Time, owner string) {
key, _ := delivery.StepKey(node, id)
_, _ = k.put(key, json.RawMessage(mustStep(t, delivery.Step{ID: id, Tool: tool, Node: node, State: state, Started: started, Owner: owner})))
}
put("ace", "old1", "build_go_fetch", delivery.StepRunning, boot.Add(-time.Hour), "")
put("ace", "mjk3l9x0abcdef", "check ask", delivery.StepRunning, boot.Add(-time.Hour), "")
put("ace", "mjk3l9x0abcdee", "check ask", delivery.StepRunning, time.Now().Add(-5*time.Minute), "")
put("ace", "mjk3l9x0abcded", "check ask", delivery.StepRunning, time.Now().Add(-5*time.Minute), gone) // note 21's ghost
put("ace", "gone2", "build_go_test", delivery.StepRunning, boot.Add(time.Second), gone)
put("ace", "mine1", "check ask", delivery.StepRunning, boot.Add(-time.Hour), Owner)
put("ace", "raced1", "build_go_test", delivery.StepRunning, boot.Add(-time.Hour), "")
put("ace", "new1", "build_go_fetch", delivery.StepRunning, boot.Add(time.Second), "")
put("ace", "done1", "build_go_fetch", delivery.StepPassed, boot.Add(-time.Hour), gone)
put("g14", "other1", "build_go_fetch", delivery.StepRunning, boot.Add(-time.Hour), gone)
// raced1 is ended by its own run between the read and the write.
raced := false
scan := scanOf(k, func(key string) {
if key == "ace.raced1" && !raced {
raced = true
put("ace", "raced1", "build_go_test", delivery.StepPassed, boot.Add(-time.Hour), "")
}
})
var said []string
ended, err := EndStale(scan, "ace", boot, func(f string, a ...any) { said = append(said, fmt.Sprintf(f, a...)) })
if err != nil || len(ended) != 4 {
t.Fatalf("ended %v, %v", ended, err)
}
if _, held := k.puts["ace.mjk3l9x0abcded"]; held {
t.Errorf("the ghost ask's key is still held: %+v", k.last("ace.mjk3l9x0abcded"))
}
for _, key := range []string{"ace.old1", "ace.gone2"} {
if s := k.last(key); s.State != delivery.StepFailed || s.Reason != RestartReason || s.Ended.IsZero() {
t.Errorf("%s = %+v", key, s)
}
}
if s := k.last("ace.mjk3l9x0abcdef"); s.State != delivery.StepFailed || !strings.HasPrefix(s.Reason, "past the step bound: its asker stopped waiting at ") {
t.Errorf("the old ask = %+v", s)
}
if log := strings.Join(said, "\n"); !strings.Contains(log, "ace.mjk3l9x0abcded") || !strings.Contains(log, "deleted, left to the stream's redelivery") || !strings.Contains(log, "its owner "+gone+" is not this process "+Owner) {
t.Errorf("the log does not name the ghost and its owner:\n%s", log)
}
for key, state := range map[string]string{"ace.mjk3l9x0abcdee": delivery.StepRunning, "ace.mine1": delivery.StepRunning, "ace.raced1": delivery.StepPassed,
"ace.new1": delivery.StepRunning, "ace.done1": delivery.StepPassed, "g14.other1": delivery.StepRunning} {
if s := k.last(key); s.State != state {
t.Errorf("%s = %s, want %s", key, s.State, state)
}
}
// EndOne: refuses without why, a held step, one this process owns or (with no owner) started, a step not running and
// another node's; ends a step a previous process left.
for _, c := range []struct{ id, why string }{{"old1", ""}, {"new1", "stuck"}, {"mine1", "stuck"}, {"done1", "x"}, {"g14.other1", "x"}} {
if _, err := EndOne(scan, "ace", c.id, c.why, nil); err == nil {
t.Errorf("ended %s with why %q", c.id, c.why)
}
}
put("ace", "old2", "build_go_fetch", delivery.StepRunning, boot.Add(time.Second), gone)
if _, err := EndOne(scan, "ace", "old2", "x", func(string) bool { return true }); err == nil {
t.Error("ended a held step")
}
if _, err := EndOne(scan, "ace", "ace.old2", "stuck for good", nil); err != nil {
t.Fatal(err)
}
if s := k.last("ace.old2"); s.State != delivery.StepFailed || s.Reason != "stuck for good" || s.Owner != gone {
t.Errorf("old2 = %+v", s)
}
if a, b := ownerOf("b", 1, boot), ownerOf("b", 1, boot.Add(time.Second)); a == b {
t.Errorf("two starts of one pid share the owner %q", a)
}
}
func mustStep(t *testing.T, s delivery.Step) []byte {
t.Helper()
raw, err := delivery.EncodeStep(s)
if err != nil {
t.Fatal(err)
}
return raw
}
// 1890: AskWhat names the repository, head and levels of a check ask and the module of a build ask.
func TestAskWhat(t *testing.T) {
check := delivery.StepAsk{Kind: "check", Check: &delivery.CheckJob{Fetch: map[string]any{"repository": "novox/mesh-sdk", "commit": "0123456789abcdef0123"}, Levels: []string{"unit", "integration"}}}
if got, want := AskWhat(check), "check ask, novox/mesh-sdk@0123456789ab, levels unit,integration"; got != want {
t.Errorf("check = %q", got)
}
build := delivery.StepAsk{Kind: "delivery", Build: &delivery.BuildJob{Module: "mesh-x", FP: "abcdef0123456789"}}
if got, want := AskWhat(build), "delivery ask, module mesh-x, fp abcdef012345"; got != want {
t.Errorf("build = %q", got)
}
}
+6 -5
View File
@@ -1,6 +1,7 @@
// Package buildstep is what both build modules share: the runner of a long step that answers a tool call at once
// and ends on its key in the module's `steps` state (novox/hq ADR 0469 §2; issue 1471), the toolchain guard
// (issue 1473) and the fetch of one commit over SSH (issue 1474).
// (issue 1473), the fetch of one commit over SSH (issue 1474), and the failure contract of a test level with the parsers
// of go test -json and node --test's TAP that fill it (failure.go, issue 1901; its conformance kit is package conformance).
package buildstep
import (
@@ -43,7 +44,7 @@ func NewID() string {
// the error's text, or refused when the error is a *Refusal; a panic in run ends the step failed, naming it, and
// the process goes on. If the running write fails, nothing was started and Start answers the error.
func Start(ctx context.Context, put Put, node, tool string, run func(ctx context.Context) (result any, err error)) (string, error) {
step, key, err := begin(put, node, tool, NewID())
step, key, err := begin(put, node, tool, NewID(), "")
if err != nil {
return "", err
}
@@ -51,9 +52,9 @@ func Start(ctx context.Context, put Put, node, tool string, run func(ctx context
return step.ID, nil
}
// begin writes the step running under id.
func begin(put Put, node, tool, id string) (delivery.Step, string, error) {
step := delivery.Step{ID: id, Tool: tool, Node: node, State: delivery.StepRunning, Started: now().UTC()}
// begin writes the step running under id, owned by this process (Owner).
func begin(put Put, node, tool, id, what string) (delivery.Step, string, error) {
step := delivery.Step{ID: id, Tool: tool, Node: node, State: delivery.StepRunning, Started: now().UTC(), What: what, Owner: Owner}
key, err := delivery.StepKey(node, step.ID)
if err != nil {
return delivery.Step{}, "", err
+4 -3
View File
@@ -70,8 +70,9 @@ func TestStartAnswersAtOnceWithTheStepRunning(t *testing.T) {
t.Errorf("Start took %v", d)
}
s := k.last("ace." + id)
if s.State != delivery.StepRunning || s.Tool != "fetch" || s.Node != "ace" || s.ID != id || s.Started.IsZero() {
t.Errorf("step = %+v", s)
// 1890/AC5: the running key names this process as its owner.
if s.State != delivery.StepRunning || s.Tool != "fetch" || s.Node != "ace" || s.ID != id || s.Started.IsZero() || s.Owner != Owner || Owner == "" {
t.Errorf("step = %+v, want owner %q", s, Owner)
}
}
@@ -100,7 +101,7 @@ func TestStepEndsPassedFailedOrRefusedOnItsKey(t *testing.T) {
t.Fatalf("ended key %q", got)
}
s := k.last("ace." + id)
if s.State != c.state || s.Reason != c.reason || string(s.Result) != c.result || s.Ended.IsZero() {
if s.State != c.state || s.Reason != c.reason || string(s.Result) != c.result || s.Ended.IsZero() || s.Owner != Owner {
t.Errorf("step = %+v", s)
}
})
+192
View File
@@ -0,0 +1,192 @@
package buildstep
// The reader of `node --test` under its TAP reporter (issue 1901's thorough fix; first written in mesh-build-node by
// fix-1901ts): each failing test with its name, its file, its cause and its own block, never a passing line of an
// earlier file, which is what node's default reporter left the first fifteen lines of.
import (
"regexp"
"strings"
)
// NodeTestOptions is the NODE_OPTIONS a build module runs a package's test script with: node's TAP reporter, written to
// the file dest, so whatever the script prints on its own output the report is whole and machine-readable. A script
// that names a reporter of its own makes node refuse the run loudly ("--test-reporter must match the number of
// --test-reporter-destination"), never quietly.
func NodeTestOptions(dest string) string {
return "--test-reporter=tap --test-reporter-destination=" + dest
}
var ansiEscape = regexp.MustCompile("\x1b\\[[0-9;]*m")
func stripANSI(s string) string { return ansiEscape.ReplaceAllString(s, "") }
// ReadTAP reads the failing tests of a `node --test` run: report is what the TAP reporter wrote (NodeTestOptions'
// file), console the script's own stdout and stderr, script the name of the script that ran and dir the package's
// directory, which a test's file is made relative to. A suite that failed only because a test inside it did is not
// reported twice. A test file that did not load (a syntax error, a crash, an error after its test ended) takes its
// cause from the lines node wrote before it. When the report holds no failing test (node itself failed, or the script
// failed before its tests ran: a build, a reporter refused) the failure is the console's (ConsoleFailure).
func ReadTAP(pkg, script, dir, report, console string) []Failure {
var failed []Failure
all := strings.Split(stripANSI(strings.ReplaceAll(report, "\r\n", "\n")), "\n")
var cancelled []Failure // failed because a suite around them did: said after the failure that carries the cause
prev := 0 // the line after the last test block, passing or not: what the run wrote since belongs to the next test
for i := 0; i < len(all); i++ {
name, ok := tapNotOk(all[i])
if !ok {
if tapOk(all[i]) {
i = tapBlockEnd(all, i) - 1
prev = i + 1
}
continue
}
indent := len(all[i]) - len(strings.TrimLeft(all[i], " "))
block := []string{strings.TrimSpace(all[i])}
fields := map[string]string{}
var key string
inYAML := false
j := i + 1
for ; j < len(all); j++ {
t := strings.TrimSpace(all[j])
if !inYAML {
if t == "---" {
inYAML = true
continue
}
break
}
if t == "..." {
j++
break
}
block = append(block, strings.TrimRight(all[j], " \t"))
if k, v, found := tapKey(all[j], indent); found {
key = k
if v != "|-" && v != "|" {
fields[k] = strings.Trim(v, "'")
}
} else if key != "" && fields[key] == "" && t != "" {
fields[key] = t // the first line of a block scalar
}
}
from, at := prev, i
i, prev = j-1, j
if fields["failureType"] == "subtestsFailed" {
continue
}
cause := fields["error"]
if cause == "test failed" || cause == "" {
// a file that did not load, or an error after a test ended: node says only "test failed" and wrote the
// error as comment lines before it
var own []string
for _, l := range all[from:at] {
if t := strings.TrimSpace(l); strings.HasPrefix(t, "# ") && !strings.HasPrefix(t, "# Subtest") {
own = append(own, strings.TrimPrefix(t, "# "))
}
}
for _, l := range own {
if strings.Contains(l, "Error") {
cause = l
break
}
}
if cause == "" && len(own) > 0 {
cause = own[0]
}
block = append(own, block...)
}
if cause == "" {
cause = fields["failureType"]
}
if cause == "" {
cause = "the test failed and node said no error"
}
cause = CutLine(cause)
file := strings.TrimPrefix(fields["location"], "file://")
if dir != "" {
file = strings.TrimPrefix(file, strings.TrimSuffix(dir, "/")+"/")
}
f := Failure{Package: pkg, Test: name, File: file, Cause: cause, Output: Clip(block, cause)}
if fields["failureType"] == "cancelledByParent" {
cancelled = append(cancelled, f)
continue
}
failed = append(failed, f)
}
if failed = append(failed, cancelled...); len(failed) > 0 {
return failed
}
var comments []string
for _, l := range all {
if t := strings.TrimSpace(l); strings.HasPrefix(t, "# ") && !strings.HasPrefix(t, "# Subtest") && !tapCount(t) {
comments = append(comments, strings.TrimPrefix(t, "# "))
}
}
return []Failure{ConsoleFailure(pkg, script, strings.Join(comments, "\n")+"\n"+console)}
}
// tapCount says a comment is TAP's closing count ("# tests 3", "# pass 2", "# duration_ms 1.2").
func tapCount(t string) bool {
for _, w := range []string{"tests", "suites", "pass", "fail", "cancelled", "skipped", "todo", "duration_ms"} {
if strings.HasPrefix(t, "# "+w+" ") {
return true
}
}
return false
}
// tapOk says a line is a passing test's "ok N - name", indented or not.
func tapOk(l string) bool {
rest, ok := strings.CutPrefix(strings.TrimSpace(l), "ok ")
return ok && rest != "" && rest[0] >= '0' && rest[0] <= '9'
}
// tapBlockEnd is the line after the test line at i and its YAML block ("---" to "...").
func tapBlockEnd(all []string, i int) int {
j := i + 1
if j >= len(all) || strings.TrimSpace(all[j]) != "---" {
return j
}
for j++; j < len(all); j++ {
if strings.TrimSpace(all[j]) == "..." {
return j + 1
}
}
return j
}
// tapNotOk is the test name of a "not ok N - name" line, indented or not; a TODO or SKIP directive fails nothing.
func tapNotOk(l string) (string, bool) {
t := strings.TrimSpace(l)
rest, ok := strings.CutPrefix(t, "not ok ")
if !ok {
return "", false
}
if u := strings.ToUpper(rest); strings.Contains(u, " # TODO") || strings.Contains(u, " # SKIP") {
return "", false
}
if _, after, found := strings.Cut(rest, " - "); found {
rest = after
}
return strings.TrimSpace(rest), true
}
// tapKey is "key: value" of a field of a TAP block, whose lines sit two spaces deeper than their "not ok" line; a deeper
// line is part of a block scalar (an error's lines, a stack).
func tapKey(l string, indent int) (key, val string, ok bool) {
t := strings.TrimLeft(l, " ")
if len(l)-len(t) != indent+2 {
return "", "", false
}
k, v, found := strings.Cut(t, ": ")
if !found {
if k, found = strings.CutSuffix(t, ":"); !found {
return "", "", false
}
}
if k == "" || strings.ContainsAny(k, " ()/") {
return "", "", false
}
return k, strings.TrimSpace(v), true
}
+3 -3
View File
@@ -36,7 +36,7 @@ var lockFiles = []string{"go.sum", "package-lock.json"}
func ModuleTrees(ctx context.Context, run forgekey.Runner, env []string, dir, commit string) (map[string]ModuleTree, error) {
r := run(ctx, env, "git", "-C", dir, "ls-files", "--", "module.json", "*/module.json")
if r.Status != 0 || r.Err != "" {
return nil, fmt.Errorf("git ls-files: %s", firstLine(r.Err+" "+r.Stderr))
return nil, fmt.Errorf("git ls-files: %s", forgekey.Said(r.Err+"\n"+r.Stderr))
}
trees := map[string]ModuleTree{}
for _, l := range strings.Split(r.Stdout, "\n") {
@@ -50,7 +50,7 @@ func ModuleTrees(ctx context.Context, run forgekey.Runner, env []string, dir, co
}
t := run(ctx, env, "git", "-C", dir, "rev-parse", spec)
if t.Status != 0 || t.Err != "" {
return nil, fmt.Errorf("git rev-parse %s: %s", spec, firstLine(t.Err+" "+t.Stderr))
return nil, fmt.Errorf("git rev-parse %s: %s", spec, forgekey.Said(t.Err+"\n"+t.Stderr))
}
mt := ModuleTree{Tree: strings.TrimSpace(t.Stdout)}
raw, err := os.ReadFile(filepath.Join(dir, filepath.FromSlash(l)))
@@ -93,7 +93,7 @@ func narrow(ctx context.Context, run forgekey.Runner, env []string, dir, commit,
}
r := run(ctx, env, "git", append([]string{"-C", dir, "ls-tree", "-z", commit, "--"}, full...)...)
if r.Status != 0 || r.Err != "" {
return fmt.Errorf("git ls-tree %s of the module %s: %s", commit, mdir, firstLine(r.Err+" "+r.Stderr))
return fmt.Errorf("git ls-tree %s of the module %s: %s", commit, mdir, forgekey.Said(r.Err+"\n"+r.Stderr))
}
objects := map[string]string{}
for _, l := range strings.Split(r.Stdout, "\x00") {
+125
View File
@@ -0,0 +1,125 @@
package buscompose
import (
"fmt"
"sort"
"strings"
)
// Do the emitters and the consumers of a catalogue agree?
//
// **The check that was missing** (novox/hq 04-ISSUES/127). Every manifest was individually
// well-formed and every derivation individually correct, and no cross-module subscription in the
// mesh matched anything: a consumer's declaration derived into a namespace nobody publishes to.
// Nothing failed, because a subscription that matches nothing is not an error — it is silence.
//
// The comparison has to be over the whole catalogue, because the two halves live in different
// manifests, and it cannot simply demand that every consumed event have a live emitter: a module
// may be installed long before the one whose events it wants. So the rule is narrower and still
// catches this: **where the emitter is present, it must emit what the consumer asked for.**
// AConsumer is one module's interest in another's events, as this check needs it.
type AConsumer struct {
Module string
Consumes []string
}
// AnEmitter is one module's events.
type AnEmitter struct {
Module string
Emits []string
}
// Disagreements are the consumed events whose emitter is in the catalogue and does not emit them.
//
// Returned as sentences rather than as structs: every one of them is read by a person deciding
// whether a manifest or a catalogue is wrong, and a pair of names without the reason is a puzzle.
func Disagreements(emitters []AnEmitter, consumers []AConsumer, seats []DeclaredSeat) []string {
emits := map[string]map[string]bool{}
for _, e := range emitters {
if emits[e.Module] == nil {
emits[e.Module] = map[string]bool{}
}
for _, name := range e.Emits {
emits[e.Module][name] = true
}
}
// A seat's events are published by its holder under the seat's name, so a consumer naming the
// seat is naming something real even though no module declares it as its own.
for _, s := range seats {
if len(s.Emits) == 0 {
continue
}
if emits[s.Name] == nil {
emits[s.Name] = map[string]bool{}
}
for _, name := range s.Emits {
emits[s.Name][name] = true
}
}
var out []string
for _, c := range consumers {
for _, pattern := range c.Consumes {
emitter, event, named := strings.Cut(pattern, ".")
// Every event from everyone, or every event from one module: both are deliberate and
// neither names a particular event to check.
if !named || emitter == "*" || emitter == catalogueTheRest || event == catalogueTheRest {
continue
}
known, present := emits[emitter]
if !present {
// Not installed here, which is ordinary: a module lives in its own repository and
// may be registered later. Nothing to compare, so nothing to say.
continue
}
if matchesAny(event, known) {
continue
}
out = append(out, fmt.Sprintf(
"%s consumes %q and %s emits %s — so that subscription would match nothing, and "+
"nothing would report it",
c.Module, pattern, emitter, listOf(known)))
}
}
sort.Strings(out)
return out
}
// matchesAny says whether one of an emitter's event names satisfies a consumer's pattern.
func matchesAny(pattern string, emitted map[string]bool) bool {
want := strings.Split(pattern, ".")
for name := range emitted {
if matches(want, strings.Split(name, ".")) {
return true
}
}
return false
}
func matches(pattern, name []string) bool {
for i, part := range pattern {
if part == catalogueTheRest {
return i < len(name)
}
if i >= len(name) {
return false
}
if part != "*" && part != name[i] {
return false
}
}
return len(pattern) == len(name)
}
func listOf(names map[string]bool) string {
if len(names) == 0 {
return "nothing"
}
out := make([]string, 0, len(names))
for n := range names {
out = append(out, n)
}
sort.Strings(out)
return strings.Join(out, ", ")
}
+97
View File
@@ -0,0 +1,97 @@
package buscompose
import (
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/announce"
)
// Who may read a module's state is what the module announced (novox/hq ADR 0495 clauses 2 and 3, ADR 0513 §3; issue
// 1790). A module says on the bus which states it keeps and which others may read (announce.Subject); the controller
// keeps the last word of each (announce.KeptBucket) and grants a reader's `reads: <owner>.<name>` only when the owner
// announced that state readable. An owner that has announced nothing yet is read as before, from the reader's `reads`
// (the transition, until every module announces), and is named as not yet announced.
// StatesAnnouncedBucket is the controller's bucket of the last announcement of each module (announce.KeptBucket).
const StatesAnnouncedBucket = announce.KeptBucket
// controllerReadable are the controller's states others may read: the record mirrors (the packages each node declares
// among them, which the pacman module reads, ADR 0507 §5), the chain's buckets, and the grants issued and the
// compositions' digests, which the assigner reads (ADR 0528 clause 2, ADR 0539 clause 6).
var controllerReadable = []string{MirrorAssignments, MirrorDeclaredConfig, MirrorPredefined, ChainBuilds, ChainRollouts,
MirrorDeclaredPackages, ControllerGrants, ControllerComposed}
// ControllerAnnouncement is what the controller announces it keeps (ADR 0495 clause 2): every bucket of its own by its local
// name, the ones others may read marked readable. It announces itself, as any module does from its code.
func ControllerAnnouncement(node string) announce.States {
s := announce.States{Module: ControllerSeat, Node: node}
readable := map[string]bool{}
for _, n := range controllerReadable {
readable[n] = true
}
for _, bucket := range ControllerBuckets() {
local, mine := strings.CutPrefix(bucket, ControllerSeat+"_")
if !mine {
continue
}
s.States = append(s.States, announce.State{Name: local, Readable: readable[local]})
}
sort.Slice(s.States, func(i, j int) bool { return s.States[i].Name < s.States[j].Name })
return s
}
// ReadsAnnounced splits a reader's `reads` (`<owner>.<name>`) by what each owner announced: granted when the owner
// announced the state readable, or announced nothing yet (the transition); refused when the owner announced and did
// not announce that state readable.
func ReadsAnnounced(reads []string, announced map[string]announce.States) (granted, refused []string) {
return ReadsAnnouncedBy("", reads, announced)
}
// ReadsAnnouncedBy is ReadsAnnounced for one reader module, which also honours the readers an owner named for a state
// (announce.State.Readers) and the composer's named readers (NamedReaders, ADR 0547 clause 9): a read named for other
// modules is refused. An empty reader is judged by the readable mark alone.
func ReadsAnnouncedBy(reader string, reads []string, announced map[string]announce.States) (granted, refused []string) {
for _, r := range reads {
owner, name, ok := strings.Cut(r, ".")
s, said := announced[owner]
switch {
case reader != "" && !ReaderMay(reader, r):
refused = append(refused, r)
case reader != "" && ok && said && s.Keeps(name):
if s.ReadableBy(name, reader) {
granted = append(granted, r)
} else {
refused = append(refused, r)
}
case !ok || !said || s.Readable(name):
granted = append(granted, r)
default:
refused = append(refused, r)
}
}
return granted, refused
}
// NotAnnounced are the owners some module reads the state of and that have announced nothing yet, sorted: read from the
// readers' `reads` as before, until they announce.
func NotAnnounced(r Records) []string {
seen := map[string]bool{}
var out []string
for _, ds := range r.Assigned {
for _, d := range ds {
for _, read := range append(append([]string(nil), d.Reads...), d.ReadsRefused...) {
owner, _, ok := strings.Cut(read, ".")
if !ok || seen[owner] {
continue
}
if _, said := r.Announced[owner]; !said {
seen[owner] = true
out = append(out, owner)
}
}
}
}
sort.Strings(out)
return out
}
+47
View File
@@ -0,0 +1,47 @@
package buscompose
import "git.novox.be/novox/mesh-sdk/go/delivery"
// The view alone reads the build seats' queues (novox/hq ADR 0525 §2, issue 1887 A8a): the shared workers' counts and the
// oldest waiting ask, to show the backlog. It may not publish an ask: only the builder and the checker ask, by their seat use.
// The watchman is granted nothing here: a waiting ask has no pulse, a taken one is read from the build module's steps (issue
// 1908). Nothing is a wildcard over a stream or a worker.
const (
// ViewModule is the module of the delivery view (mesh-delivery-view).
ViewModule = "mesh-delivery-view"
// BuilderModule is the module of the builder (mesh-delivery-builder).
BuilderModule = "mesh-delivery-builder"
)
// BuildAskReads are the API subjects the builder requests when it takes over, at its start, a key naming an ask (issue
// 1910): each build seat's stream's information and its messages by sequence, to tell whether the ask is on its stream.
// Reads only: no consumer, no pull, no acknowledgement; the builder puts its asks by its seat use, as before.
func BuildAskReads() []string {
var out []string
for _, lang := range delivery.AskLanguages {
if stream, err := delivery.AskStream(lang); err == nil {
out = append(out, "$JS.API.STREAM.INFO."+stream, "$JS.API.STREAM.MSG.GET."+stream)
}
}
return out
}
// BuildBacklogReads are the API subjects the view requests: the information of each build seat's stream and of each of its
// shared workers (delivery.AskStream, delivery.AskWorker), and the message of the stream by sequence. Reads only: no
// message is pulled, none acknowledged, none published.
func BuildBacklogReads() []string {
var out []string
for _, lang := range delivery.AskLanguages {
stream, err := delivery.AskStream(lang)
if err != nil {
continue
}
out = append(out, "$JS.API.STREAM.INFO."+stream, "$JS.API.STREAM.MSG.GET."+stream)
for _, kind := range delivery.AskKinds {
if w, err := delivery.AskWorker(lang, kind); err == nil {
out = append(out, "$JS.API.CONSUMER.INFO."+stream+"."+w)
}
}
}
return out
}
+296
View File
@@ -0,0 +1,296 @@
package buscompose
import (
"encoding/json"
"os"
"path/filepath"
"reflect"
"regexp"
"slices"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// Defends issue 1377 and novox/hq ADR 0547 §5: this is the controller's bus composer copied, and it composes byte for
// byte what the controller's composes. testdata/controller-user-lists.json was written by the controller's own
// internal/broker (mesh-controller branch fix/1945-mesh-grants-publishes-memberships, and fix/1790-grants-create-buckets for the grants part creating declared streams), lists and memberships, from fixtures_test.go compiled there unchanged; composed.conf is the
// controller's golden of the whole server configuration. A row that differs is the copy drifting from the controller,
// which is a fault while both compose (until the controller loses its lease, ADR 0547 §13), but for the grants the copy
// gives beyond it on purpose (beyondTheControllers): the copy grants every user everything the controller's grants, and
// only those more.
// createdAdvisory is one subscription to a bucket's creation in a composed list, as stateCreatedHeard composes it.
var createdAdvisory = regexp.MustCompile(`"\$JS\.EVENT\.ADVISORY\.STREAM\.CREATED\.KV_[^"]*", `)
// beyondTheControllers is what the copy grants beyond the controller's on purpose (novox/hq ADR 0547 §2, ADR 0555 §3,
// issue 1126), by user: a node's account reads every node's own facts, `<node>.node` of the have bucket where the node's
// record rides, and what it composes its own modules from (NodeComposeReads). After the take-over (ADR 0547 §8) the
// grants part's list is the one the bus loads; a hand-back loads the controller's, which lacks these reads, and the nodes
// keep applying without composing their own until the next take-over. Each principal's hearing of its buckets' creation
// (issue 1790) is the other row beyond, matched by createdAdvisory.
func beyondTheControllers(user string) map[string]bool {
out := map[string]bool{}
node, ok := strings.CutPrefix(user, "node.")
if !ok {
return out
}
for _, s := range append(append(readGrants("node-engine_have", "*.node"), NodeComposeReads(node)...), NodeComposeAdvisories()...) {
out[s] = true
}
return out
}
// mayBeyondTheControllers is what the copy may grant beyond the controller's where the records allow it, and need not
// where they do not: a node's ask of the vault's `own` while the vault serves it (issue 1126, ADR 0550 §5).
func mayBeyondTheControllers(user string) map[string]bool {
if _, ok := strings.CutPrefix(user, "node."); !ok {
return nil
}
return map[string]bool{VaultOwnCall(Principal{Kind: KindNode, Node: strings.TrimPrefix(user, "node.")}): true,
VaultSecretChanged: true}
}
// quotedToken is a quoted token of a composed list.
var quotedToken = regexp.MustCompile(`"[^"]*"`)
// grantsByUser splits a composed list into the text outside its users and each user's quoted tokens (its name, its
// password's hash, every subject it may publish or subscribe to), by user.
func grantsByUser(list string) (string, map[string]map[string]bool) {
var outside strings.Builder
users := map[string]map[string]bool{}
user := ""
for _, line := range strings.SplitAfter(list, "\n") {
trimmed := strings.TrimSpace(line)
if name, ok := strings.CutPrefix(trimmed, `{ user: "`); ok {
user, _, _ = strings.Cut(name, `"`)
users[user] = map[string]bool{}
} else if user != "" && trimmed == "]" {
user = ""
}
if user == "" {
outside.WriteString(line)
continue
}
for _, q := range quotedToken.FindAllString(line, -1) {
users[user][strings.Trim(q, `"`)] = true
}
}
return outside.String(), users
}
// grantsAsTheControllers fails t unless the copy's list is the controller's, user for user, but for the grants it
// names on purpose (beyondTheControllers, the creation advisories): every grant of the controller's is the copy's, and
// the copy's beyond them are exactly those, every one of them there.
func grantsAsTheControllers(t *testing.T, got, want string) {
t.Helper()
gotOutside, gotUsers := grantsByUser(got)
wantOutside, wantUsers := grantsByUser(want)
if gotOutside != wantOutside {
t.Errorf("the list differs from the controller's outside its users:\n%s", gotOutside)
}
for user, was := range wantUsers {
now, ok := gotUsers[user]
if !ok {
t.Errorf("the copy has no user %s", user)
continue
}
intended := beyondTheControllers(user)
for g := range was {
if !now[g] {
t.Errorf("%s: the copy does not grant %s, which the controller's does", user, g)
}
}
for g := range now {
if !was[g] && !intended[g] && !mayBeyondTheControllers(user)[g] && !createdAdvisory.MatchString(`"`+g+`", `) {
t.Errorf("%s: the copy grants %s beyond the controller's, and no decision names it", user, g)
}
}
for g := range intended {
if !now[g] {
t.Errorf("%s: the copy does not grant %s, which a node composing its own modules reads", user, g)
}
}
}
for user := range gotUsers {
if _, ok := wantUsers[user]; !ok {
t.Errorf("the copy has a user %s the controller's has not", user)
}
}
}
func TestTheCopiedComposerComposesAsTheControllers(t *testing.T) {
// ADR 0567 §3, §9: a node reads every key of the seat's placement and its candidates (it composes with the whole
// proposal laid over), and none of its open proposals; it writes neither.
reads := strings.Join(NodeComposeReads("laptop"), " ")
for bucket, granted := range map[string]bool{delivery.PlacementBucket: true, delivery.PlacementCandidatesBucket: true,
delivery.PlacementProposalsBucket: false} {
if strings.Contains(reads, "$JS.API.DIRECT.GET.KV_"+bucket+".>") != granted || strings.Contains(reads, "$KV."+bucket+".laptop") {
t.Errorf("a node is granted %s: %v, want %v", bucket, !granted, granted)
}
}
// It hears the creation of those two buckets, and of no other.
if got := NodeComposeAdvisories(); len(got) != 2 || got[0] != "$JS.EVENT.ADVISORY.STREAM.CREATED.KV_mesh-delivery-assigner_placement" ||
got[1] != "$JS.EVENT.ADVISORY.STREAM.CREATED.KV_mesh-delivery-assigner_candidates" {
t.Errorf("a node hears the creation of %v", got)
}
raw, err := os.ReadFile(filepath.Join("testdata", "controller-user-lists.json"))
if err != nil {
t.Fatal(err)
}
var want []struct {
Name, List string
Missing []string
Err string
Memberships json.RawMessage
}
if err := json.Unmarshal(raw, &want); err != nil {
t.Fatal(err)
}
fixtures := UserListFixtures()
if len(want) != len(fixtures) {
t.Fatalf("%d fixtures, %d lists the controller composed", len(fixtures), len(want))
}
for i, f := range fixtures {
t.Run(f.Name, func(t *testing.T) {
if want[i].Name != f.Name {
t.Fatalf("fixture %q against the controller's %q", f.Name, want[i].Name)
}
list, missing, err := ComposedFor(f)
got := ""
if err != nil {
got = err.Error()
}
if got != want[i].Err {
t.Fatalf("error %q, the controller's %q", got, want[i].Err)
}
if strings.Join(missing, ",") != strings.Join(want[i].Missing, ",") {
t.Errorf("left out %v, the controller %v", missing, want[i].Missing)
}
// The one row the copy composes beyond the controller's, on purpose (novox/hq issue 1790, 2026-10-26, after
// the take-over of ADR 0547 milestone 2): each principal hears its own and read buckets created, so a state
// missing at its start is bound when it appears. The controller's composer is frozen and its list no longer
// loaded; while it is not, this row is left out of the comparison and nothing else.
grantsAsTheControllers(t, list, want[i].List)
// And every membership the records issue, as the controller's MembershipFor makes it (ADR 0547 §5): the
// grants part publishes them.
raw, err := json.Marshal(MembershipsFor(f))
if err != nil {
t.Fatal(err)
}
var gotM, wasM any
if err := json.Unmarshal(raw, &gotM); err != nil {
t.Fatal(err)
}
if err := json.Unmarshal(want[i].Memberships, &wasM); err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(gotM, wasM) {
t.Errorf("the memberships differ from the controller's:\n%s", raw)
}
})
}
// An asker on a seat pulled by kind hears what that seat's workers gave up on, and no other stream's dead letters
// (novox/hq issue 734: a check ask given up during a bus reload left its try running for ever).
t.Run("an asker reads its seat's dead letters", func(t *testing.T) {
build := Seat{Name: "node-build-go", Scope: "node", Accepts: []string{"delivery", "check", "recheck"}, PulledByKind: true}
perms, err := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "mesh-delivery-checker", Uses: []Seat{build}})
if err != nil {
t.Fatal(err)
}
for subject, want := range map[string]bool{
"$JS.API.CONSUMER.CREATE.DEAD_LETTERS.x1.mesh.events.dead.SEAT_NODE_BUILD_GO.>": true,
"$JS.API.CONSUMER.CREATE.DEAD_LETTERS.x1.mesh.events.dead.SEAT_NODE_BUILD_GO.w.next": true,
"$JS.API.CONSUMER.DELETE.DEAD_LETTERS.x1": true,
"$JS.API.CONSUMER.MSG.NEXT.DEAD_LETTERS.x1": true,
"$JS.API.CONSUMER.CREATE.DEAD_LETTERS.x1.mesh.events.dead.EVENTS.>": false,
"$JS.API.CONSUMER.CREATE.DEAD_LETTERS.x1": false,
"$JS.API.STREAM.PURGE.DEAD_LETTERS": false,
"$JS.FC.DEAD_LETTERS.x1": false,
} {
if got := MayPublish(perms, subject); got != want {
t.Errorf("may publish %s: %v, want %v", subject, got, want)
}
}
plain, err := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "shop",
Uses: []Seat{{Name: "telegram-sender", Accepts: []string{"send"}}}})
if err != nil {
t.Fatal(err)
}
if MayPublish(plain, "$JS.API.CONSUMER.DELETE.DEAD_LETTERS.x1") {
t.Error("a user of a seat not pulled by kind reads DEAD_LETTERS")
}
})
// Issue 1126 (ADR 0550 §5): a node's engine asks the vault's `own` in its own name while the vault serves it, and no
// other node's.
t.Run("a node asks the vault its own secrets", func(t *testing.T) {
one := Principal{Kind: KindNode, Node: "one", AsksItsOwnSecrets: true}
perms, err := PermissionsFor(one)
if err != nil {
t.Fatal(err)
}
if !MayPublish(perms, VaultOwnCall(one)) || MayPublish(perms, VaultOwnCall(Principal{Kind: KindNode, Node: "two"})) {
t.Errorf("node one asks own as %v, and as two %v", MayPublish(perms, VaultOwnCall(one)),
MayPublish(perms, VaultOwnCall(Principal{Kind: KindNode, Node: "two"})))
}
if plain, _ := PermissionsFor(Principal{Kind: KindNode, Node: "one"}); MayPublish(plain, VaultOwnCall(one)) {
t.Error("a node asks own while the vault does not serve it")
}
if !slices.Contains(perms.Subscribe, VaultSecretChanged) {
t.Errorf("node one does not hear the vault's secret-changed: %v", perms.Subscribe)
}
// And what a module on another node serves from its settings, that key alone (mailu's smtp domain, issue 1126).
users, err := Users(Records{Nodes: []string{"one", "two"}, Assigned: map[string][]Declared{
"two": {{Module: "mailer", Served: []string{"domain", "not.a.token"}}}}})
if err != nil {
t.Fatal(err)
}
get := func(key string) string {
return "$JS.API.DIRECT.GET.KV_" + ResolvedBucket + ".$KV." + ResolvedBucket + "." + key
}
for _, u := range users {
if u.Kind != KindNode {
continue
}
perms, err := PermissionsFor(u)
if err != nil {
t.Fatal(err)
}
if u.Node == "one" && (!reflect.DeepEqual(u.ServedReads, []string{"two.mailer.domain"}) || !MayPublish(perms, get("two.mailer.domain")) ||
MayPublish(perms, get("two.mailer.sitename"))) {
t.Errorf("node one reads %v of two's resolved values", u.ServedReads)
}
if u.Node == "two" && len(u.ServedReads) != 0 {
t.Errorf("node two is granted its own served keys apart: %v", u.ServedReads)
}
}
})
t.Run("the controller's golden server configuration", func(t *testing.T) {
seat := Seat{Name: "telegram-sender", Accepts: []string{"send"}, Emits: []string{"delivered", "failed"}}
got, err := Compose(
Server{ClientPort: 4222, MonitoringPort: 8222, StoreDir: "/data",
TLSCert: "/tls/tls.crt", TLSKey: "/tls/tls.key", TLSCA: "/tls/ca.crt"},
[]Principal{
{Kind: KindController, PasswordHash: "$2a$11$cccccccccccccccccccccc"},
{Kind: KindEnrolment, Node: "one", PasswordHash: "$2a$11$eeeeeeeeeeeeeeeeeeeeee"},
{Kind: KindNode, Node: "one", PasswordHash: "$2a$11$nnnnnnnnnnnnnnnnnnnnnn"},
{Kind: KindModule, Node: "one", Module: "telegram", Holds: []Seat{seat},
Serves: []string{"status"}, PasswordHash: "$2a$11$tttttttttttttttttttttt"},
{Kind: KindModule, Node: "two", Module: "shop", Uses: []Seat{seat},
Emits: []string{"order.placed"}, PasswordHash: "$2a$11$ssssssssssssssssssssss"},
{Kind: KindModule, Node: "two", Module: "audit",
Consumes: []string{"shop.order.placed"}, PasswordHash: "$2a$11$aaaaaaaaaaaaaaaaaaaaaa"},
{Kind: KindModule, Node: "one", Module: WatchmanModule, PasswordHash: "$2a$11$wwwwwwwwwwwwwwwwwwwwww"},
{Kind: KindModule, Node: "one", Module: "nats", SnapshotsTheBus: true,
Serves: []string{"nats_streams"}, PasswordHash: "$2a$11$bbbbbbbbbbbbbbbbbbbbbb"},
})
if err != nil {
t.Fatal(err)
}
want, err := os.ReadFile(filepath.Join("testdata", "composed.conf"))
if err != nil {
t.Fatal(err)
}
grantsAsTheControllers(t, got, string(want))
})
}
+117
View File
@@ -0,0 +1,117 @@
package buscompose
import (
"regexp"
"strings"
)
// A tool call names its caller (novox/hq issue 365, by ADR 0259 §3's precedent for asks).
//
// mesh.mod.<module>.call.<tool>[.<node>].<caller>
// mesh.seat.<seat>.call.<verb>[.<node>].<caller>
//
// The last token is the bus user that published the call, and each user is granted these subjects with its own
// name there and no other — the caller is a fact the server enforces, and the tool runtime hands it to the
// module from the subject a call arrived on (mesh-tools node-tools internal/bus caller.go holds the same shape).
//
// **A kind of its own, `call`, not the `tool` subject with a token appended.** Every grant to call a tool is a
// wildcard over the `tool` kind — `.tool.<t>.*` for the instance on any machine, `mesh.mod.*.tool.>` for every
// tool — and either would match a `tool` subject with any caller appended, so a caller could name another.
// Under `call` nothing is granted but the caller-named subjects. No stream's filter covers it: a tool call is
// never persisted (design 25 §3).
//
// **Derived from the `tool` grants, in one place** (callerNamed): wherever a principal may call or answer a
// tool, it may call it in its own name, or answer it naming any caller — so no kind of principal is left
// unable to call in its own name, and a new grant to call is one in both shapes by construction.
//
// The `tool` grants stay beside these for one release, so every caller and every runtime moves without a gap:
// their retirement is hq issue 464.
const CallKind = "call"
var userName = regexp.MustCompile(`^[A-Za-z0-9_-]+(\.[A-Za-z0-9_-]+)*$`)
// CallerToken is a bus user's name as the last token of a call it publishes: each dot written `~`, which no part
// of a user's name may hold (safeSubject), so the token names that user and no other. "" for a name that is not
// a user's.
func CallerToken(user string) string {
if !userName.MatchString(user) {
return ""
}
return strings.ReplaceAll(user, ".", "~")
}
// toolGrant splits a grant on the `tool` kind — `mesh.mod.<m>.tool.<rest>` or `mesh.seat.<s>.tool.<rest>`, any
// part possibly a wildcard — at the kind; ok is false for any other subject.
func toolGrant(subject string) (head, rest string, ok bool) {
parts := strings.SplitN(subject, ".", 5)
if len(parts) != 5 || parts[0] != "mesh" || (parts[1] != "mod" && parts[1] != "seat") || parts[3] != "tool" ||
parts[2] == "" || parts[4] == "" {
return "", "", false
}
return parts[0] + "." + parts[1] + "." + parts[2], parts[4], true
}
// CalledSubject is where a call to a tool subject goes naming its caller; "" when the subject is not a tool's
// or the user is not a bus user.
func CalledSubject(subject, user string) string {
head, rest, ok := toolGrant(subject)
token := CallerToken(user)
if !ok || token == "" || strings.ContainsAny(rest, "*>") {
return ""
}
return head + "." + CallKind + "." + rest + "." + token
}
// CalledPattern is what a holder answering a tool subject also subscribes, to hear the calls that name their
// caller: the same address under the `call` kind, any caller last. "" for a subject that is not a tool's.
func CalledPattern(subject string) string {
head, rest, ok := toolGrant(subject)
if !ok || strings.ContainsAny(rest, "*>") {
return ""
}
return head + "." + CallKind + "." + rest + ".*"
}
// calledPublish is a grant to call on the `tool` kind, as the same grant in the caller's own name: its last
// token the caller's, and nothing that reaches past it. A `>` is every tail a call carries — the tool alone or
// the tool and the machine — so it becomes both, each ending in the caller.
func calledPublish(grant, token string) []string {
head, rest, ok := toolGrant(grant)
if !ok || token == "" {
return nil
}
base := head + "." + CallKind + "."
switch {
case rest == ">":
return []string{base + "*." + token, base + "*.*." + token}
case strings.HasSuffix(rest, ".>"):
return []string{base + strings.TrimSuffix(rest, ">") + "*." + token}
}
return []string{base + rest + "." + token}
}
// calledSubscribe is a grant to answer on the `tool` kind, as the same grant for the calls naming any caller.
func calledSubscribe(grant string) []string {
head, rest, ok := toolGrant(grant)
if !ok {
return nil
}
base := head + "." + CallKind + "."
if strings.HasSuffix(rest, ">") {
return []string{base + rest}
}
return []string{base + rest + ".*"}
}
// callerNamed adds, beside a principal's grants on the `tool` kind, the same grants under `call`: to publish in
// its own name, to subscribe naming anybody.
func callerNamed(user string, pub, sub []string) ([]string, []string) {
token := CallerToken(user)
for _, g := range pub {
pub = append(pub, calledPublish(g, token)...)
}
for _, g := range sub {
sub = append(sub, calledSubscribe(g)...)
}
return unique(pub), unique(sub)
}
+11
View File
@@ -0,0 +1,11 @@
package buscompose
import "strings"
// A seat's cancelled set (novox/hq ADR 0219) is a bucket the mesh no longer raises or grants (R4g, ADR 0525 clause 6): the
// controller asks no work queue, so nothing is cancelled. A bus that still holds one is not reported as state nothing declares.
// IsCancelledSet says a bucket is a seat's cancelled set rather than a module's state.
func IsCancelledSet(bucket string) bool {
return strings.HasPrefix(bucket, "SEAT_") && strings.HasSuffix(bucket, "_cancelled")
}
+196
View File
@@ -0,0 +1,196 @@
package buscompose
import (
"context"
"fmt"
"git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats.go/jetstream"
)
// The compositions' buckets (novox/hq ADR 0539 clauses 2 and 4): one per node, `composed-<node>`, holding that node's
// compositions, which only that node's node-engine reads, since a composition can carry a sealed value (ADR 0488 §2's
// reason); and `mesh-controller_composed`, holding each composition's digest and the judgement of its move, which the
// assigner reads. The controller writes both, by compare-and-set while it holds the lease; each keeps five revisions
// of a key. The names and layouts are mesh-sdk's (delivery.ComposedBucketFor, ControllerComposedBucket, ComposedKey).
// ControllerComposed is the local name of the compositions' digests, as a reader names it in `reads` after
// `mesh-controller.`: delivery.ControllerComposedBucket.
const ControllerComposed = "composed"
// ComposedValueBytes is the most a composition may hold: a file over 64 KiB travels by digest, so a module's
// composition is far smaller; just under the server's largest message, which also carries the subject and headers.
const ComposedValueBytes = 1<<20 - 8<<10
// ComposedBucketBytes is a node's composition bucket's size: five revisions of every module's compositions of the node.
const ComposedBucketBytes = 256 << 20
// ComposedDigestsBytes is the size of the bucket of digests: a value of a few hundred bytes per node, module and
// fingerprint kept.
const ComposedDigestsBytes = 64 << 20
// ComposedConfig is a node's composition bucket as the controller asserts it.
func ComposedConfig(node string) (jetstream.KeyValueConfig, error) {
bucket, err := delivery.ComposedBucketFor(node)
if err != nil {
return jetstream.KeyValueConfig{}, err
}
return jetstream.KeyValueConfig{
Bucket: bucket,
Description: "the compositions of " + node + " (novox/hq ADR 0539): each module's resources for this node at one " +
"fingerprint, and the node's own, signed; written by the controller alone, read by this node's node-engine " +
"alone; deleted when outside the module's five newest fingerprints and named by no want",
History: delivery.ComposedHistory,
MaxValueSize: ComposedValueBytes,
MaxBytes: ComposedBucketBytes,
Storage: jetstream.FileStorage,
}, nil
}
// ComposedDigestsConfig is the bucket of the compositions' digests as the controller asserts it.
func ComposedDigestsConfig() jetstream.KeyValueConfig {
return jetstream.KeyValueConfig{
Bucket: delivery.ControllerComposedBucket,
Description: "each composition's digest and the judgement of its move from what the node runs (novox/hq ADR 0539 " +
"clauses 2 and 14): written by the controller alone, read by the assigner",
History: delivery.ControllerComposedHistory,
MaxValueSize: 4 << 10,
MaxBytes: ComposedDigestsBytes,
Storage: jetstream.FileStorage,
}
}
// EnsureComposed creates the bucket of digests and each named node's composition bucket if absent and brings their
// options to match. An update, never a delete.
func EnsureComposed(ctx context.Context, js jetstream.JetStream, nodes ...string) error {
if _, err := js.CreateOrUpdateKeyValue(ctx, ComposedDigestsConfig()); err != nil {
return fmt.Errorf("asserting bucket %s: %w", delivery.ControllerComposedBucket, err)
}
for _, node := range nodes {
cfg, err := ComposedConfig(node)
if err != nil {
return err
}
if _, err := js.CreateOrUpdateKeyValue(ctx, cfg); err != nil {
return fmt.Errorf("asserting bucket %s: %w", cfg.Bucket, err)
}
}
return nil
}
// ComposedWrites is what the controller publishes to write a node's compositions: the bucket's own subjects.
func ComposedWrites(node string) (string, error) {
bucket, err := delivery.ComposedBucketFor(node)
if err != nil {
return "", err
}
return "$KV." + bucket + ".>", nil
}
// ComposedReads is what a node's node-engine publishes to follow its wants and compositions (ADR 0539 clauses 1, 2 and
// 9): every key of its own composition bucket, and its own wants (`want.<node>.<module>` and `want.<node>.node`) in the
// assigner's bucket; nothing of another node's, nor the bucket of digests.
func ComposedReads(node string) ([]string, error) {
bucket, err := delivery.ComposedBucketFor(node)
if err != nil {
return nil, err
}
if !safeSubject.MatchString(node) {
return nil, fmt.Errorf("%q cannot be part of a subject: a permission is a subject pattern, and this would widen it", node)
}
return append(readGrants(bucket, ""), readGrants(delivery.AssignerWantsBucket, "want."+node+".>")...), nil
}
// VerbsTheNodeAgentAsks are the seat verbs a node's node-engine calls of its own machine while it applies a want (ADR
// 0539 clause 16, ADR 0429 §14): its backup holder backs a data module up before it moves, and says when it was. Asked
// of its own machine only, `mesh.seat.<seat>.tool.<verb>.<node>`.
var VerbsTheNodeAgentAsks = []SeatVerb{{Seat: "node-backup", Verb: "now"}, {Seat: "node-backup", Verb: "backed-up"}}
// NodeAgentAsks are the subjects of VerbsTheNodeAgentAsks for one node.
func NodeAgentAsks(node string) ([]string, error) {
if !safeSubject.MatchString(node) {
return nil, fmt.Errorf("%q cannot be part of a subject: a permission is a subject pattern, and this would widen it", node)
}
var out []string
for _, v := range VerbsTheNodeAgentAsks {
out = append(out, "mesh.seat."+v.Seat+".tool."+v.Verb+"."+node)
}
return out, nil
}
// NodeAgentHears are the events a node's node-engine subscribes while it applies a want: the backup holder's word that a
// backup is taken (`backed-up`), from whichever module holds node-backup on its machine (restic today).
var NodeAgentHears = []string{"mesh.mod." + BackupHolderModule + ".event.backed-up"}
// BackupHolderModule is the module that holds node-backup on every node, restic: the one whose word a node-engine hears.
// A second holder of the seat is a change of this grant.
const BackupHolderModule = "restic"
// administers says a person may invoke every tool (`*`): the operator.
func administers(invokes []string) bool {
for _, i := range invokes {
if i == "*" {
return true
}
}
return false
}
// HandPathWrites is what the operator's account publishes for release 6's hand path (ADR 0539 clauses 2 and 10): a want
// of any node, written and read back, in the assigner's bucket; each named node's compositions; the digests; and each
// node's object bucket for a file over 64 KiB, whose put reads an object's metadata and purges a replaced one's chunks.
func HandPathWrites(nodes []string) ([]string, error) {
out := append([]string{"$KV." + delivery.AssignerWantsBucket + ".want.>"}, readGrants(delivery.AssignerWantsBucket, "want.>")...)
out = append(out, "$KV."+delivery.ControllerComposedBucket+".>")
out = append(out, readGrants(delivery.ControllerComposedBucket, "")...)
for _, node := range nodes {
w, err := ComposedWrites(node)
if err != nil {
return nil, err
}
bucket, _ := delivery.ComposedBucketFor(node)
out = append(out, w)
out = append(out, readGrants(bucket, "")...)
files, err := DeclarationFilesWrites(node)
if err != nil {
return nil, err
}
stream := "OBJ_" + DeclarationFilesBucket + node
out = append(out, files, "$JS.API.STREAM.INFO."+stream, "$JS.API.DIRECT.GET."+stream+".>",
"$JS.API.STREAM.MSG.GET."+stream, "$JS.API.STREAM.PURGE."+stream)
}
return out, nil
}
// DeclarationsStream is the stream of each node's last declaration, `mesh.node.<node>.declare` (issue 107). Release 6
// sends none (ADR 0539 clauses 5 and 11): the stream is kept, with each node's consumer, only so a node away during the
// switch finds its last declaration and with it the new node-engine; the controller deletes it once every node's have
// names the new node-engine, and from then on neither asserts it nor any node's consumer on it; nor makes it again once
// it is gone from the bus for any reason, since a stream made anew would hold no node's last declaration (issue 1945).
const DeclarationsStream = "NODES"
// RaiseAfterDeclarations is Raise once the declarations' stream is retired: the mesh's streams and consumers but the
// declarations' stream, and no node's consumer of a declaration.
func RaiseAfterDeclarations(r Raiser) error {
for _, s := range MeshStreams() {
if s.Name == DeclarationsStream {
continue
}
if err := r.EnsureStream(s); err != nil {
return fmt.Errorf("asserting stream %s: %w", s.Name, err)
}
}
return AssertMeshConsumers(r)
}
// TakeInConsumer is the durable consumer the controller takes done keys in from, on the stream of each of TakeInBuckets
// (novox/hq ADR 0544 §1).
const TakeInConsumer = "mesh-controller-take-in"
// TakeInBuckets are the buckets the take-in consumes: the indexer's modules (each module's trunk value) and the builder's
// builds (each build's key).
var TakeInBuckets = []string{delivery.ModulesBucket, delivery.BuilderBuildsBucket}
// ConfigFollowConsumer is the durable consumer the controller follows mesh-config's resolved values with, on that
// bucket's stream (novox/hq ADR 0545).
const ConfigFollowConsumer = "mesh-controller-config-follow"
+247
View File
@@ -0,0 +1,247 @@
package buscompose
import (
"context"
"fmt"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats.go/jetstream"
)
// The controller's own key-value buckets (novox/hq to-be 45 §1, §6, §7).
//
// **What the controller must remember across its own restart, it keeps on the bus.** A call's
// outcome lived in the memory of the process that served it (novox/hq issue 265), so a controller
// replaced while a push ran answered "no such call" for the one thing its caller had been told to
// ask about. The bus already outlives the controller and is the shape ADR 0201 gives a module's
// current state: one value per key, written by one owner, read by anybody granted it. These are the
// controller's, written by it alone — the writers table of to-be 45 §1 — and asserted on every start
// like the streams, so a bus raised from nothing has them before the first call is served.
// CallsBucket keeps every call of the mesh's own verbs and what came of it; HandActsBucket every act
// a person did by hand, with why; ConditionsBucket every condition open now (to-be 45 §2), one key
// each, and ConditionHistoryBucket every transition of one — raised, changed, silenced, cleared —
// for ninety days.
//
// **The history is a bucket of its own** because its keys expire and an open condition's must not:
// a bucket has one age for every key, and a condition open longer than the history is kept would
// otherwise vanish from the store while still true.
var (
CallsBucket = BucketName(ControllerSeat, "calls")
HandActsBucket = BucketName(ControllerSeat, "hand-acts")
ConditionsBucket = BucketName(ControllerSeat, "conditions")
ConditionHistoryBucket = BucketName(ControllerSeat, "condition-history")
// LeaseBucket holds the controller's lease (to-be 45 §6): one key, `holder`, which the instance
// allowed to act writes by compare-and-set and renews; its revision when taken is the epoch.
LeaseBucket = BucketName(ControllerSeat, "lease")
// AskedBucket keeps what the controller asked the operator about its conditions (novox/hq ADR 0259):
// each ask by its id, its options and the actions they stand for, how it ended and whether the
// controller acted on its warrant — so a restart neither asks twice nor acts twice.
AskedBucket = BucketName(ControllerSeat, "asked")
// DeliveryMergesBucket holds the merges mesh-delivery announced it made, each with the ask whose Approve it merged
// on (novox/hq ADR 0420 §7, issue 1025): per merge a key `<owner>/<repo>/<number>/<head sha>` holding the
// announcement, kept DeliveryMergesKeptFor. What lets the build of a moved module register it from its new
// repository: the controller reads the ask's warrant from the router and accepts only the operator's Approve of
// those heads. Written by the controller alone, from the announcement, when it holds the lease.
DeliveryMergesBucket = BucketName(ControllerSeat, "delivery-merges")
)
// DeliveryMergesKeptFor is how long a merge mesh-delivery announced is kept: a week, as the events are.
const DeliveryMergesKeptFor = 7 * 24 * time.Hour
// The controller's record mirrors (novox/hq ADR 0289, design 54, design 57 package F3): a bucket per kind of
// record, one key per record, written from the store by the lease holder only when the record changed, with
// one event on the controller's seat per write. Read by whatever names them in `reads` as
// `mesh-controller.<mirror>` (ADR 0201 §1), written by nobody else. **No secret in either** (ADR 0201): an
// assignment's value names its module and node; a module's declared config is what its manifest, which is
// public, declares.
//
// AssignmentsBucket holds which module is assigned to which node, keyed `<node>.<module>`; and
// DeclaredConfigBucket each registered module's config declarations as its manifest holds them, keyed by
// the module: what mesh-config resolves a config value from (ADR 0348 clauses 2 and 4); and PredefinedBucket the
// list of predefined keys the mesh published (the newest trunk build's of the catalogue, ADR 0348 §7), one key
// `list`: what mesh-config publishes as mesh-config_predefined (design 57 K3, issue 662).
var (
AssignmentsBucket = BucketName(ControllerSeat, MirrorAssignments)
DeclaredConfigBucket = BucketName(ControllerSeat, MirrorDeclaredConfig)
PredefinedBucket = BucketName(ControllerSeat, MirrorPredefined)
// DeclaredPackagesBucket holds, keyed by the node, each package the modules assigned to it declare, present or absent,
// with the declaring modules: what the pacman module's verbs read to say a package is declared (ADR 0507 §5).
DeclaredPackagesBucket = BucketName(ControllerSeat, MirrorDeclaredPackages)
)
// The mirrors' local names, as a reader names them in `reads` after `mesh-controller.`.
const (
MirrorAssignments = "assignments"
MirrorDeclaredConfig = "declared-config"
MirrorPredefined = "predefined"
// MirrorDeclaredPackages is the local name of the packages each node's modules declare (ADR 0507 §5, issue 1840/S3).
MirrorDeclaredPackages = "declared-packages"
)
// The chain's buckets (novox/hq ADR 0408, issue 1089): builds, rollouts and assignments are one chain kept on the bus
// alone, one bucket each, one key per record, written by the lease holder by compare-and-set. The assignments bucket is
// AssignmentsBucket above, which keeps its name and its readers (ADR 0408 §3). Nothing is removed from them (§4), they keep
// ChainHistory earlier revisions of every key (§5), and a value is never cut (§6).
var (
BuildsBucket = BucketName(ControllerSeat, ChainBuilds)
RolloutsBucket = BucketName(ControllerSeat, ChainRollouts)
)
// **The assignments bucket keeps the marker age (MirrorMarkerTTL) it was made with as a mirror.** A stream that allows
// messages an age cannot be made to disallow them (the server refuses the update: "message TTL status can not be
// disabled", nats-server stream.go), so asserting the bucket without the age would fail the controller's start on a
// mesh that has the bucket. The chain never purges or removes a key (ADR 0408 §4), so no marker is ever written and the
// age ends nothing; it is kept only so the stream's configuration stays one the server accepts.
//
// The chain buckets' local names, as a reader names them in `reads` after `mesh-controller.`.
const (
ChainBuilds = "builds"
ChainRollouts = "rollouts"
)
// The chain buckets' bounds. ChainHistory is the bus's largest history (ADR 0408 §5). A value is at most ChainValueBytes,
// just under the server's largest message of a mebibyte, which also carries the subject and the headers (ADR 0408 §6):
// the controller writes none larger and the move refuses to. The buckets' size is the arithmetic of the survey of
// issue 981: 8585 builds of 1 to 40 KB, 1049 rollouts of about 2 KB a revision and 64 revisions, so 512 MiB for the two
// that grow, and the assignments, 285 of under a kilobyte, 64 MiB.
const (
ChainHistory = 64
ChainValueBytes = 1<<20 - 8<<10
ChainBuildsBytes = 512 << 20
ChainRolloutsBytes = 512 << 20
ChainAssignmentBytes = 64 << 20
)
// GrantsBucket holds, keyed `<module>.<fingerprint hex>`, each fingerprint of a module whose bus credential the controller
// has issued (novox/hq ADR 0528 clause 2, ADR 0429 §6 rule 6, issue 1394): written by the controller alone once the bus's
// user list composed with the module's manifest at that fingerprint was sent to the bus's machine; read by the assigner,
// whose sixth rule moves a node's want to a fingerprint only once its grant exists. The name and layout are mesh-sdk's
// (delivery.ControllerGrantsBucket, ControllerGrantKey).
var GrantsBucket = BucketName(ControllerSeat, ControllerGrants)
// ControllerGrants is the grants bucket's local name, as a reader names it in `reads` after `mesh-controller.`.
const ControllerGrants = "grants"
// ChainBuckets are the three buckets of the chain: builds, rollouts and assignments.
func ChainBuckets() []string { return []string{BuildsBucket, RolloutsBucket, AssignmentsBucket} }
// MirrorsWrittenByAnother are the controller's mirrors whose writer is not the controller, each with its reason: the rule
// that a mirror is written by the controller alone (TestTheMirrorsListAndTheWritersTableAgree) has these exceptions and no
// other.
var MirrorsWrittenByAnother = map[string]string{}
// The mirrors' caps (design 54 §5): one value per key, no age, 64 KiB a value, 64 MiB a bucket.
const (
MirrorValueBytes = 64 << 10
MirrorBytes = 64 << 20
// MirrorMarkerTTL is the longest a purge's marker is kept (design 54 §2): the bucket allows a
// per-message age for it, and the writer gives each marker a day.
MirrorMarkerTTL = 24 * time.Hour
)
// PlanStateBuckets are the buckets of the plans' state that release 6 deletes (novox/hq ADR 0539 clause 13): the merge
// window's clock, the silent sends' waits and the building plans' hold. The controller deletes each at its first start
// and says it once (composed_start.go); none is asserted, granted or written any more.
var PlanStateBuckets = []string{BucketName(ControllerSeat, "merge-window"), BucketName(ControllerSeat, "rollout-silent"),
BucketName(ControllerSeat, "plans-building")}
// AskedKeptFor is how long an ask is kept after it was made: a month, as the router keeps its own.
const AskedKeptFor = 30 * 24 * time.Hour
// LeaseTTL is how long the lease's key lives unrenewed (to-be 45 §6): fifteen seconds, renewed
// every five. The bucket's age, so the bus forgets a holder that stopped renewing.
const LeaseTTL = 15 * time.Second
// The bounds to-be 45 §6 sets for calls: the last thousand, or fourteen days, whichever is fewer.
// A call is two keys — its record, and its answer apart so a listing does not read every answer —
// so the stream holds twice as many messages as it keeps calls.
const (
KeptCallsDurably = 1000
CallsKeptFor = 14 * 24 * time.Hour
// CallAnswerBytes is the most of one answer kept: a whole declaration is far smaller, and an
// answer larger is cut and says so.
CallAnswerBytes = 64 << 10
// HandActsKeptFor is as long as a condition's history (to-be 45 §2): an act by hand is read
// back beside what it addressed.
HandActsKeptFor = 90 * 24 * time.Hour
// ConditionHistoryKeptFor is how long a condition's transitions are kept (to-be 45 §2).
ConditionHistoryKeptFor = 90 * 24 * time.Hour
)
// IsControllerBucket says a bucket is the controller's own, not a module's state nothing declares.
func IsControllerBucket(bucket string) bool {
return bucket == CallsBucket || bucket == HandActsBucket || bucket == ConditionsBucket ||
bucket == ConditionHistoryBucket || bucket == LeaseBucket || bucket == AskedBucket ||
bucket == AssignmentsBucket || bucket == DeclaredConfigBucket || bucket == PredefinedBucket ||
bucket == DeclaredPackagesBucket ||
bucket == DeliveryMergesBucket || bucket == BuildsBucket || bucket == RolloutsBucket ||
bucket == delivery.HaveBucket || bucket == StatesAnnouncedBucket ||
bucket == GrantsBucket || bucket == delivery.ControllerComposedBucket
}
// ControllerBuckets are the controller's own buckets, in the order they are asserted.
func ControllerBuckets() []string {
return []string{LeaseBucket, CallsBucket, HandActsBucket, ConditionsBucket, ConditionHistoryBucket, AskedBucket,
AssignmentsBucket, DeclaredConfigBucket, PredefinedBucket, DeclaredPackagesBucket,
DeliveryMergesBucket, BuildsBucket, RolloutsBucket, delivery.HaveBucket, StatesAnnouncedBucket, GrantsBucket,
delivery.ControllerComposedBucket}
}
// ControllerBucketsAsserter is what raising the controller's buckets needs of a connection.
type ControllerBucketsAsserter interface {
EnsureControllerBuckets() error
}
// chainBucket is the options of a bucket of the chain: sixty-four revisions of each key, no age, a value up to
// ChainValueBytes, on disk.
func chainBucket(bucket, why string, bytes int64, markers time.Duration) jetstream.KeyValueConfig {
return jetstream.KeyValueConfig{
Bucket: bucket,
Description: why,
History: ChainHistory,
MaxValueSize: ChainValueBytes,
MaxBytes: bytes,
Storage: jetstream.FileStorage,
LimitMarkerTTL: markers,
}
}
// EnsureLeaseBucket creates the lease's bucket if absent and brings its options to match (to-be 45
// §6). **Before the lease is taken, by any candidate**: it is the lease's own precondition, and asserting
// a bucket that exists changes nothing. File storage, so its revisions — the epochs — outlive a restart of
// the bus; one raised again from nothing is moved past the highest epoch issued when the lease is taken.
func EnsureLeaseBucket(ctx context.Context, js jetstream.JetStream) error {
if _, err := js.CreateOrUpdateKeyValue(ctx, jetstream.KeyValueConfig{
Bucket: LeaseBucket,
Description: "the controller's lease (novox/hq to-be 45 §6): the key `holder`, written by compare-and-set " +
"by the one controller instance that may act and renewed every five seconds; its revision when taken " +
"is the epoch every declaration and plan write carries",
History: 1,
TTL: LeaseTTL,
MaxValueSize: 4 << 10,
MaxBytes: 1 << 20,
Storage: jetstream.FileStorage,
}); err != nil {
return fmt.Errorf("asserting bucket %s: %w", LeaseBucket, err)
}
return nil
}
// HaveBucketConfig is the options of the have bucket (novox/hq ADR 0429 §1): delivery.HaveHistory revisions of each
// key, no age, 64 MiB on disk like the assignments. A module's have is JSON of about 500 bytes, but a node's own key
// carries its whole facts (its profile among them), several kilobytes on a desktop: a value is as large as the chain's
// buckets take, since a value is never cut (novox/hq ADR 0408 §6, issue 1162).
func HaveBucketConfig() jetstream.KeyValueConfig {
return jetstream.KeyValueConfig{
Bucket: delivery.HaveBucket,
Description: "what each machine has of each module, one key `<node>.<module>` (and `<node>.node` for the machine's " +
"own facts) (novox/hq ADR 0429, design 47): written by each machine's node-engine under its own node's keys " +
"alone, never by the controller; read by the delivery view and the watchman",
History: delivery.HaveHistory,
MaxValueSize: ChainValueBytes,
MaxBytes: 64 << 20,
Storage: jetstream.FileStorage,
}
}
+88
View File
@@ -0,0 +1,88 @@
package buscompose
import (
"time"
"github.com/nats-io/nats.go/jetstream"
)
// The streams a manifest declares, as the server is asked to create them (novox/hq ADR 0547 §5, issue 1790): the
// grants part creates each one the bus does not hold yet, with what the controller asserts at its start, so the two
// make the same stream. Pure. Only a create is ever asked with these: a stream the bus holds is never updated by the
// grants part (CreateStreamGrants grants no update).
// BucketStreamConfig is a module's bucket as the stream the controller's CreateOrUpdateKeyValue makes of its options
// (mesh-controller internal/broker BucketConfig, with no marker age held): the client library's key-value shape
// (rollup, no delete, direct reads, discard new, a duplicate window of two minutes or the age when shorter) with the
// mesh's caps, file storage and one replica.
func BucketStreamConfig(b Bucket) jetstream.StreamConfig {
history := b.History
if history < 1 {
history = 1
}
age := time.Duration(b.TTLSeconds) * time.Second
dupes := 2 * time.Minute
if age > 0 && age < dupes {
dupes = age
}
cfg := jetstream.StreamConfig{
Name: "KV_" + b.Bucket(),
Description: b.Why(),
Subjects: []string{"$KV." + b.Bucket() + ".>"},
MaxMsgsPerSubject: int64(history),
MaxBytes: StateMaxBytes,
MaxAge: age,
MaxMsgSize: StateMaxValueBytes,
Storage: jetstream.FileStorage,
Replicas: 1,
AllowRollup: true,
DenyDelete: true,
Duplicates: dupes,
MaxMsgs: -1,
MaxConsumers: -1,
AllowDirect: true,
Discard: jetstream.DiscardNew,
}
if b.KeyTTL {
cfg.AllowMsgTTL, cfg.SubjectDeleteMarkerTTL = true, KeyTTLMarkerTTL
}
return cfg
}
// ModuleStreamConfig is a module's own stream as the controller asserts it (mesh-controller internal/broker
// StreamConfigFor of ModuleStream.Stream).
func ModuleStreamConfig(m ModuleStream) jetstream.StreamConfig {
s := m.Stream()
cfg := jetstream.StreamConfig{
Name: s.Name,
Subjects: s.Subjects,
Retention: jetstream.LimitsPolicy,
MaxAge: time.Duration(s.MaxAge) * time.Second,
MaxMsgsPerSubject: int64(s.MaxMsgsPerSubject),
Description: s.Why,
AllowDirect: s.Direct,
AllowMsgTTL: s.AllowMsgTTL,
}
if s.Retention == RetentionWorkQueue {
cfg.Retention = jetstream.WorkQueuePolicy
}
if s.Restricted || s.WorkQueue {
cfg.Metadata = map[string]string{}
if s.Restricted {
cfg.Metadata[RestrictedMetadata] = "true"
}
if s.WorkQueue {
cfg.Metadata[WorkQueueMetadata] = "true"
}
}
if s.MaxBytes > 0 {
cfg.MaxBytes = s.MaxBytes
}
if s.DiscardNew {
cfg.Discard = jetstream.DiscardNew
}
if s.DuplicatesSeconds > 0 {
cfg.Duplicates = time.Duration(s.DuplicatesSeconds) * time.Second
}
return cfg
}
+46
View File
@@ -0,0 +1,46 @@
package buscompose
import (
"testing"
"time"
"github.com/nats-io/nats.go/jetstream"
)
// Defends issue 1790 (ADR 0547 §5): the grants part creates a declared bucket and stream as the controller asserts them.
func TestDeclaredStreamsAreCreatedAsTheControllerAssertsThem(t *testing.T) {
plain := BucketStreamConfig(Bucket{Module: "mesh-vault", Name: "bus-accounts"})
keyed := BucketStreamConfig(Bucket{Module: "m", Name: "presence", History: 5, TTLSeconds: 60, KeyTTL: true})
queue := ModuleStreamConfig(ModuleStream{Module: "b", Name: "ASKS", Events: []string{"ask"}, MaxAgeDays: 7, MaxMiB: 32,
WorkQueue: true, Duplicates: 1})
for _, row := range []struct {
what string
got, want any
}{
{"bucket name", plain.Name, "KV_mesh-vault_bus-accounts"},
{"bucket subjects", plain.Subjects[0], "$KV.mesh-vault_bus-accounts.>"},
{"bucket history defaults to one", plain.MaxMsgsPerSubject, int64(1)},
{"bucket cap", plain.MaxBytes, int64(StateMaxBytes)},
{"bucket value cap", plain.MaxMsgSize, int32(StateMaxValueBytes)},
{"bucket duplicate window", plain.Duplicates, 2 * time.Minute},
{"bucket discards new", plain.Discard, jetstream.DiscardNew},
{"bucket direct and rollup, no delete", plain.AllowDirect && plain.AllowRollup && plain.DenyDelete, true},
{"bucket without key ttl", plain.AllowMsgTTL, false},
{"history", keyed.MaxMsgsPerSubject, int64(5)},
{"age", keyed.MaxAge, time.Minute},
{"duplicate window under the age", keyed.Duplicates, time.Minute},
{"key ttl markers", keyed.SubjectDeleteMarkerTTL, KeyTTLMarkerTTL},
{"key ttl", keyed.AllowMsgTTL, true},
{"queue retention", queue.Retention, jetstream.WorkQueuePolicy},
{"queue subjects", queue.Subjects[0], "mesh.stream.b.event.ask"},
{"queue age", queue.MaxAge, 7 * 24 * time.Hour},
{"queue cap", queue.MaxBytes, int64(32 << 20)},
{"queue discards new", queue.Discard, jetstream.DiscardNew},
{"queue duplicates", queue.Duplicates, time.Hour},
{"queue metadata", queue.Metadata[WorkQueueMetadata] + queue.Metadata[RestrictedMetadata], "truetrue"},
} {
if row.got != row.want {
t.Errorf("%s: %v, want %v", row.what, row.got, row.want)
}
}
}
+130
View File
@@ -0,0 +1,130 @@
package buscompose
import "testing"
// Defends issue 1790 and novox/hq ADR 0547 §5 (the operator, 2026-10-26 08:54): the grants part may create the stream
// of each bucket, log and stream a placed module's manifest declares, by its own name, and read it back; nothing
// wider, nothing updated or deleted, and no other principal so. Held byte-equal in the controller's copy. And ADR 0547
// clause 7 (issue 1945, the operator's second exception): the grants part writes its own object bucket of sealed user
// lists, and the node-engine of the node holding mesh-broker alone reads its one key and an object of it by name.
func TestTheGrantsPartCreatesOnlyDeclaredStreams(t *testing.T) {
records := Records{
Nodes: []string{"novox", "ace"},
Assigned: map[string][]Declared{"ace": {{Module: "plex"}}, "novox": {
{Module: "nats", SnapshotsTheBus: true},
// mesh-sdk's copy alone (ADR 0547 clause 9, named readers): the vault's hashes are read by the grants part
// alone, whatever another module's reads say; the controller's copy grants any module that declares it.
{Module: "mesh-vault", State: []Bucket{{Module: "mesh-vault", Name: "bus-accounts"}}},
{Module: "spy", Serves: []string{"spy_x"}, Reads: []string{VaultBusAccounts}},
{Module: "mesh-delivery-builder",
State: []Bucket{{Module: "mesh-delivery-builder", Name: "steps", History: 5}},
Logs: []Log{{Module: "mesh-delivery-builder", Name: "builds"}},
Streams: []ModuleStream{{Module: "mesh-delivery-builder", Name: "BUILDASKS", Events: []string{"ask"}}}},
{Module: GrantsModule, State: []Bucket{{Module: GrantsModule, Name: "assertions"}}, Reads: []string{VaultBusAccounts}},
{Module: RuntimeModule},
}},
}
users, err := Users(records)
if err != nil {
t.Fatal(err)
}
perms := map[string]Permissions{}
for _, u := range users {
if u.Kind == KindNode || u.Kind == KindNodeTools {
p, err := PermissionsFor(u)
if err != nil {
t.Fatalf("%s: %v", u.Username(), err)
}
perms[u.Username()] = p
continue
}
if u.Kind != KindModule {
continue
}
p, err := PermissionsFor(u)
if err != nil {
t.Fatalf("%s: %v", u.Username(), err)
}
perms[u.Module] = p
}
for _, row := range []struct {
who, subject string
may bool
}{
{GrantsModule, "$JS.API.STREAM.CREATE.KV_mesh-delivery-builder_steps", true},
{GrantsModule, "$JS.API.STREAM.INFO.KV_mesh-delivery-builder_steps", true},
{GrantsModule, "$JS.API.STREAM.CREATE.LOG_mesh-delivery-builder_builds", true},
{GrantsModule, "$JS.API.STREAM.CREATE.BUILDASKS", true},
{GrantsModule, "$JS.API.STREAM.CREATE.KV_mesh-grants_assertions", true},
{GrantsModule, "$JS.API.STREAM.UPDATE.KV_mesh-delivery-builder_steps", false},
{GrantsModule, "$JS.API.STREAM.DELETE.KV_mesh-delivery-builder_steps", false},
{GrantsModule, "$JS.API.STREAM.PURGE.KV_mesh-delivery-builder_steps", false},
{GrantsModule, "$JS.API.STREAM.CREATE.KV_mesh-delivery-builder_undeclared", false},
{GrantsModule, "$JS.API.STREAM.CREATE.EVENTS", false},
{GrantsModule, "$JS.API.STREAM.CREATE.ASSIGNMENTS", false},
{GrantsModule, "$JS.API.STREAM.INFO.EVENTS", false},
{"mesh-delivery-builder", "$JS.API.STREAM.CREATE.KV_mesh-delivery-builder_steps", false},
// The sealed user list (ADR 0547 clause 7): the grants part writes its own bucket and no other.
{GrantsModule, "$JS.API.STREAM.CREATE.OBJ_mesh-grants-user-list", true},
{GrantsModule, "$O.mesh-grants-user-list.C.abc", true},
{GrantsModule, "$JS.API.STREAM.PURGE.OBJ_mesh-grants-user-list", true},
{GrantsModule, "$O.declaration-files-novox.C.abc", false},
{GrantsModule, "$JS.API.STREAM.DELETE.OBJ_mesh-grants-user-list", false},
{"mesh-delivery-builder", "$O.mesh-grants-user-list.C.abc", false},
// The bus node's engine reads the one key and an object by name; nothing of the bucket's list, no write.
{"node.novox", "$JS.API.DIRECT.GET.KV_mesh-grants_user-list.$KV.mesh-grants_user-list.sealed", true},
{"node.novox", "$JS.API.CONSUMER.CREATE.KV_mesh-grants_user-list.abc.$KV.mesh-grants_user-list.sealed", true},
{"node.novox", "$JS.API.DIRECT.GET.OBJ_mesh-grants-user-list.$O.mesh-grants-user-list.M.abc", true},
{"node.novox", "$JS.API.CONSUMER.CREATE.OBJ_mesh-grants-user-list.abc.$O.mesh-grants-user-list.C.abc", true},
{"node.novox", "$JS.API.DIRECT.GET.KV_mesh-grants_user-list.$KV.mesh-grants_user-list.list", false},
{"node.novox", "$JS.API.STREAM.INFO.OBJ_mesh-grants-user-list", false},
{"node.novox", "$JS.API.STREAM.MSG.GET.OBJ_mesh-grants-user-list", false},
{"node.novox", "$O.mesh-grants-user-list.C.abc", false},
{"node.ace", "$JS.API.DIRECT.GET.KV_mesh-grants_user-list.$KV.mesh-grants_user-list.sealed", false},
{"node.ace", "$JS.API.DIRECT.GET.OBJ_mesh-grants-user-list.$O.mesh-grants-user-list.M.abc", false},
{GrantsModule, "$JS.API.DIRECT.GET.KV_mesh-vault_bus-accounts.$KV.mesh-vault_bus-accounts.x", true},
{"spy", "$JS.API.DIRECT.GET.KV_mesh-vault_bus-accounts.$KV.mesh-vault_bus-accounts.x", false},
{"novox.node-tools", "$JS.API.DIRECT.GET.KV_mesh-vault_bus-accounts.$KV.mesh-vault_bus-accounts.x", false},
} {
if got := MayPublish(perms[row.who], row.subject); got != row.may {
t.Errorf("%s may publish %s: %v, want %v", row.who, row.subject, got, row.may)
}
}
// Issue 1790 (2026-10-26): a module hears the bus create each bucket it may bind, its own and those it reads, so a
// state missing at its start is bound when it appears; no other bucket's, and no other advisory.
for _, row := range []struct {
who, subject string
may bool
}{
{"mesh-delivery-builder", "$JS.EVENT.ADVISORY.STREAM.CREATED.KV_mesh-delivery-builder_steps", true},
{GrantsModule, "$JS.EVENT.ADVISORY.STREAM.CREATED.KV_mesh-vault_bus-accounts", true},
{"mesh-delivery-builder", "$JS.EVENT.ADVISORY.STREAM.CREATED.KV_mesh-grants_assertions", false},
{"mesh-delivery-builder", "$JS.EVENT.ADVISORY.STREAM.CREATED.LOG_mesh-delivery-builder_builds", false},
{"mesh-delivery-builder", "$JS.EVENT.ADVISORY.STREAM.DELETED.KV_mesh-delivery-builder_steps", false},
} {
if got := MaySubscribe(perms[row.who], row.subject); got != row.may {
t.Errorf("%s may hear %s: %v, want %v", row.who, row.subject, got, row.may)
}
}
// A name that is no plain token is never granted, and a module other than the grants part is never granted one.
if got := CreateStreamGrants([]string{"*", ">", "KV_a.b", "KV_a_b"}); len(got) != 2 {
t.Errorf("grants for wildcard names: %v", got)
}
other, err := PermissionsFor(Principal{Kind: KindModule, Node: "novox", Module: "shop", CreatesStreams: []string{"KV_shop_x"}})
if err != nil {
t.Fatal(err)
}
if MayPublish(other, "$JS.API.STREAM.CREATE.KV_shop_x") {
t.Error("a module other than the grants part was granted a create")
}
// The writers table refuses the grants part an update, and a create of a stream it was not given.
g := Principal{Kind: KindModule, Node: "novox", Module: GrantsModule, CreatesStreams: []string{"KV_a_b"}}
if err := CheckWriters(g, []string{"$JS.API.STREAM.CREATE.KV_a_b"}); err != nil {
t.Errorf("the declared create refused: %v", err)
}
for _, refused := range []string{"$JS.API.STREAM.UPDATE.KV_a_b", "$JS.API.STREAM.CREATE.KV_a_c", "$JS.API.STREAM.CREATE.*"} {
if CheckWriters(g, []string{refused}) == nil {
t.Errorf("%s was not refused", refused)
}
}
}
+111
View File
@@ -0,0 +1,111 @@
package buscompose
import (
"context"
"fmt"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/nats-io/nats.go/jetstream"
)
// The object buckets a declaration's large files travel in (novox/hq ADR 0486, issues 1696 and 1697): **one bucket for each
// node**, `declaration-files-<node>`.
//
// **The controller alone writes them, and a node-engine reads an object by its name from its own bucket** (ADR 0486 §2): its
// account may neither list the bucket nor write it, no account may touch another node's, and no other account has any access.
// Why a bucket each: a subject pattern cannot tell a name from a wildcard. A grant of `...M.*` matches a publish to the literal
// subject `...M.*` too (probed on nats-server 2.11.17 by the review of this issue: a node created a consumer filtered on
// `$O.<bucket>.C.*` and read every object of the bucket), so in one shared bucket a node could read every other node's files.
// With a bucket each, the same trick reads only the node's own files, which are the files of its own declaration.
//
// A bucket of the bus's object store is a stream, OBJ_<bucket>, whose messages are the objects' metadata
// (`$O.<bucket>.M.<base64 of the name>`) and chunks (`$O.<bucket>.C.<nuid>`).
// DeclarationFilesBucket is the prefix of the buckets' names, which mesh-sdk fixes for the node-engine and the controller alike.
const DeclarationFilesBucket = declaration.FileBucketPrefix
// DeclarationFilesBucketOf is the bucket a node's files are in: mesh-sdk's FileBucketFor, the one place the name is composed. A
// node name it refuses (one that could carry a subject wildcard or separator into a grant) is an error, never a bucket.
func DeclarationFilesBucketOf(node string) (string, error) { return declaration.FileBucketFor(node) }
// DeclarationFilesStreamOf is the stream under a node's bucket: mesh-sdk's FileStreamFor.
func DeclarationFilesStreamOf(node string) (string, error) { return declaration.FileStreamFor(node) }
// DeclarationFilesMaxBytes is a bucket's size: the objects are immutable and regenerable, and one node's hold well under a hundred
// megabytes.
const DeclarationFilesMaxBytes = 1 << 30
// DeclarationFilesKeptFor is how long an object stays after no node's latest declaration names it (ADR 0486 §3).
const DeclarationFilesKeptFor = 24 * time.Hour
// DeclarationFilesWrites is the one pattern the controller may publish to write a node's bucket: every object's metadata and
// chunks. The server's API for the stream is the controller's `$JS.API.>`.
func DeclarationFilesWrites(node string) (string, error) {
bucket, err := DeclarationFilesBucketOf(node)
if err != nil {
return "", err
}
return "$O." + bucket + ".>", nil
}
// DeclarationFilesReads is what a node-engine's account may publish to read an object of its own bucket by its name, and nothing
// more. The set is the one node-engine's own call (mesh-host internal/link FileObject) makes, for B the node's bucket and S its
// stream (issue 1697, notes 4 and 5):
//
// - the object's metadata, by a direct get of the last message of the one subject that names it. **Only the suffixed
// form**: the bare `$JS.API.DIRECT.GET.S` takes a sequence number and would read any message of the stream.
// - the ordered consumer on that object's chunks: `$JS.API.CONSUMER.CREATE.S.<name>.$O.B.C.<nuid>`. **Not `S.>`**:
// a consumer filtered on `$O.B.M.>` would read every object's name.
// - the flow-control replies of that consumer, without which a large object stalls at the flow-control window.
// - the consumer's deletion, best effort: the consumer's inactivity threshold removes it if this is refused.
//
// **Never granted:** STREAM.INFO (a `subjects_filter` of `>` lists every name), STREAM.MSG.GET (any message by its sequence), a
// publish to `$O.B.>` (a write), and any subject of another node's stream. A literal `*` in a subject matches the grant's `*`: it
// reaches the objects of this node's own bucket and no other (above).
func DeclarationFilesReads(node string) ([]string, error) {
bucket, err := DeclarationFilesBucketOf(node)
if err != nil {
return nil, err
}
stream, err := DeclarationFilesStreamOf(node)
if err != nil {
return nil, err
}
return []string{
"$JS.API.DIRECT.GET." + stream + ".$O." + bucket + ".M.*",
"$JS.API.CONSUMER.CREATE." + stream + ".*.$O." + bucket + ".C.*",
"$JS.API.CONSUMER.DELETE." + stream + ".*",
"$JS.FC." + stream + ".>",
}, nil
}
// EnsureDeclarationFiles is the same on a JetStream the caller holds.
func EnsureDeclarationFiles(ctx context.Context, js jetstream.JetStream, nodes ...string) error {
for _, node := range nodes {
cfg, err := DeclarationFilesConfig(node)
if err != nil {
return err
}
if _, err := js.CreateOrUpdateObjectStore(ctx, cfg); err != nil {
return fmt.Errorf("asserting the object bucket %s: %w", cfg.Bucket, err)
}
}
return nil
}
// DeclarationFilesConfig is a node's bucket as the controller asserts it.
func DeclarationFilesConfig(node string) (jetstream.ObjectStoreConfig, error) {
bucket, err := DeclarationFilesBucketOf(node)
if err != nil {
return jetstream.ObjectStoreConfig{}, err
}
return jetstream.ObjectStoreConfig{
Bucket: bucket,
Description: "the contents of the files over 64 KiB of the declaration of " + node + ", each under the hex of its sha256 " +
"digest (novox/hq ADR 0486): written by the controller alone, read by the node-engine of that node by name, removed " +
"a day after the node's latest declaration no longer names it",
MaxBytes: DeclarationFilesMaxBytes,
Storage: jetstream.FileStorage,
}, nil
}
+357
View File
@@ -0,0 +1,357 @@
package buscompose
import (
"fmt"
"sort"
"strings"
)
// Streams and consumers derived from what modules declare.
//
// The mesh's own four exist before any module does (streams.go). Everything here is the other
// half: a seat's stream comes into being when the module declaring it is **registered**, and a
// consumer when a module is **assigned** — which is why ADR 0116's task 1.4 had to be narrowed to
// the foundation set. Neither has happened at genesis.
//
// All of it is a pure function of declarations. The controller is still the only writer; this is
// only what it writes.
// A Consumer is a durable subscription the controller creates on a module's behalf. A module
// declares what it reacts to, never how delivery works, so it does not name these and cannot
// misconfigure them.
type Consumer struct {
Name string
Stream string
// Filters are the subjects this consumer receives. One consumer per module with several
// filters, rather than one per consumed event: its ack subject is derived from its name, and
// a module with five consumers would need five ack permissions to ack its own deliveries.
Filters []string
// Queue is the queue group, set for a seat's worker so that "exactly one holder" survives a
// seat later being relaxed to several. Authority and delivery are kept separate on purpose.
Queue string
// Push asks the server to deliver to a subject rather than wait to be pulled.
//
// For the mesh's own consumer, where the controller wants every message to arrive in the one
// loop it already runs: pulling would mean a second goroutine fetching batches and handing
// them over, and a loop that acts on one message at a time is the property the store window
// depends on. A queue group implies this, because a group has nothing to pull from.
Push bool
// TakesOverFrom names a consumer of the same stream whose filter this one takes over (issue 973): made while
// that consumer still filters this one's subject, it starts after what that consumer has acknowledged, so a
// message the old one held unacknowledged is not lost to the change. Otherwise it starts as FromNow says.
TakesOverFrom string
// DeliverSubject, when set, is where a push consumer delivers instead of DeliverSubjectFor's.
DeliverSubject string
// AckWaitSeconds before an unacknowledged delivery is redelivered.
AckWaitSeconds int
// MaxDeliver is how often a message is handed over before the consumer gives it up; zero for no
// bound. What a consumer gives up is kept in DEAD_LETTERS by the controller (novox/hq issue 330).
MaxDeliver int
// MaxAckPending is how many deliveries the server lets stand unacknowledged at once; zero for
// the server's default, which is many. **One, for a consumer handled one at a time**
// (novox/hq issue 175): a handler that builds for minutes keeps its own message alive with a
// heartbeat, but everything handed over behind it times out unacknowledged and comes back —
// and a merge that came back rebuilt what it had just built, five times over on 2026-09-30.
// With one outstanding, the server holds the rest, and the heartbeat is keeping the message.
MaxAckPending int
// FromNow makes a consumer that does not exist yet start at the stream's end rather than its
// beginning (novox/hq issue 248). For a consumer that reacts to announcements — a merge, a build's
// outcome — on a stream that keeps a week of them: the server's default, everything the stream
// holds, replays every merge and every build of that week as if it had just happened. A consumer
// that exists keeps where it is, whatever this says; only its making is decided here.
FromNow bool
Why string
// Resettable says why this consumer may be re-made to deliver from now by the mesh itself, with
// nobody asked (novox/hq to-be 45 §7, healer H4): what a reset drops, something else catches up.
// Empty for every consumer where nothing would — a module's, whose events would be lost to it.
// Not a property of the consumer on the bus: nothing here is sent to the server.
Resettable string
}
// seatStreamName is the stream holding a seat's inbound work. Named after the seat rather than
// the module holding it, because the holder can change and the queued work must not care — which
// is the whole reason a caller addresses a seat instead of a module.
func seatStreamName(seat string) string { return "SEAT_" + upperSnake(seat) }
// SeatStreams is one work queue per declared seat, created when the declaring module is
// registered rather than when it is assigned.
//
// **The stream exists before anyone holds the seat, and that is the point.** Work queues until a
// holder appears, so installing the telegram module a week after something started sending to it
// flushes the backlog instead of having lost it. A stream created at assignment would make "the
// holder is not here yet" mean "your messages are gone".
func SeatStreams(seats []DeclaredSeat) []Stream {
sorted := append([]DeclaredSeat(nil), seats...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Name < sorted[j].Name })
var out []Stream
for _, s := range sorted {
if len(s.Accepts) == 0 {
// A seat that only emits and serves needs no stream: its events ride EVENTS and its
// tools are core request/reply, which is never persisted.
continue
}
retain := s.RetainSeconds
if retain == 0 {
retain = 7 * 24 * 60 * 60
}
out = append(out, Stream{
Name: seatStreamName(s.Name),
Subjects: []string{"mesh.seat." + s.Name + ".accept.>"},
Retention: RetentionWorkQueue,
MaxAge: retain,
Why: fmt.Sprintf("work submitted to the %s seat; one holder consumes it, and it "+
"queues while nobody does", s.Name),
})
}
return out
}
// A DeclaredSeat is a seat as the catalogue knows it. Mirrored here rather than imported so this
// package stays free of the catalogue's own types — the same reason the host mirrors the
// contracts instead of importing the sdk.
type DeclaredSeat struct {
Name string
Accepts []string
// Emits are the verbs the seat's holder publishes under the seat's own name. An event about a
// role belongs here rather than in the holder's namespace, because the name then outlives
// whoever fills it (novox/hq ADR 0121, 04-ISSUES/127).
Emits []string
// Serves are the verbs the holder answers, request and reply.
Serves []string
RetainSeconds int
// PulledByKind is Seat's: a worker per accepted kind, derived from the holders' seat traffic, not here.
PulledByKind bool
}
// ConsumerFor is the durable consumer a module's declarations imply, or false when it subscribes
// to nothing and needs none.
//
// One per module, with every consumed subject as a filter, because its ack permission is derived
// from its name: a module with a consumer per event would need an ack permission per consumer,
// and the permission list would stop being derivable from the declaration.
func ConsumerFor(p Principal) (Consumer, bool) {
// A module that reacts to anything — a module's events or a role's (novox/hq ADR 0121). Watching
// a role was missing here, so the one module that does it got no consumer at all: it started,
// connected, and its graph stayed empty with nothing anywhere reporting why.
// And one that hears its own answers on a seat it uses (novox/hq ADR 0259 §3): an asker's warrants.
hearsItsOwn := len(SeatTrafficOf(p.Module, nil, p.Uses, nil).Subscribe) > 0
hearsAStream := false
for _, h := range p.Hears {
if len(h.Filters) > 0 {
hearsAStream = true
}
}
if p.Kind != KindModule || (len(p.Consumes) == 0 && len(p.Watches) == 0 && !hearsItsOwn && !hearsAStream) {
return Consumer{}, false
}
perms, err := PermissionsFor(p)
if err != nil {
return Consumer{}, false
}
// Events, wherever they live: a module's own namespace, and the namespace of any role it watches
// (novox/hq ADR 0121). Tool subjects and inboxes are subscribed directly and are not a consumer's
// business, which is why this is a filter and not the whole list.
var filters []string
for _, s := range perms.Subscribe {
// Not a module stream's, which its own consumer on that stream hears (novox/hq ADR 0363).
if strings.Contains(s, ".event.") && !strings.HasPrefix(s, moduleStreamPrefix) {
filters = append(filters, s)
}
}
// **A module that hears only a module's stream still has one here** (novox/hq ADR 0363 §7): what it
// gives up on there is delivered again under `mesh.again.<consumer>.` on EVENTS, and only this filters it.
if len(filters) == 0 && !hearsAStream {
return Consumer{}, false
}
sort.Strings(filters)
// And the events given up on and delivered again to this consumer alone (novox/hq issue 330).
filters = append(filters, AgainFilter(consumerDurable(p)))
return Consumer{
Name: consumerDurable(p),
Stream: consumerStream(p),
Filters: filters,
AckWaitSeconds: 30,
MaxDeliver: 5,
// From now, as the controller's own consumers have been since issue 248: a module made on a stream that
// keeps a week replays none of it. What a part needs of the present it reads from its owner
// (novox/hq ADR 0443 §1, §3). A consumer that exists keeps its place whatever this says.
FromNow: true,
Why: "what " + p.Module + " declared it consumes; after max-deliver it gives an event up, and the " +
"controller keeps it in DEAD_LETTERS until a person delivers it again or drops it",
}, true
}
// ModuleConsumer is one module's durable consumer, with the module and node it is for.
type ModuleConsumer struct {
Node, Module string
Consumer Consumer
}
// ConsumersOf is every module's durable consumer the composed users imply: each module user's, and
// each module a runtime carries — a carried module with no account of its own is no user (novox/hq
// issue 195), and its consumer is still the controller's to make, since the runtime reads it on the
// module's behalf (ADR 0198). One per module and node, whichever of the two named it first.
func ConsumersOf(users []Principal) []ModuleConsumer {
var out []ModuleConsumer
seen := map[string]bool{}
add := func(p Principal) {
key := p.Node + "/" + p.Module
if seen[key] {
return
}
c, needed := ConsumerFor(p)
// **A module with a work-queue consumer of its own needs no other reason to be visited here**
// (novox/hq ADR 0436 §1, §2, §6): mesh-mail's sender consumes and watches nothing of its own, so
// ConsumerFor alone would have found nothing needed and skipped it before its one queue consumer
// was ever composed.
if !needed && len(p.Queues) == 0 {
return
}
seen[key] = true
if needed {
out = append(out, ModuleConsumer{Node: p.Node, Module: p.Module, Consumer: c})
// And one on each module stream it hears (novox/hq ADR 0363 §5).
for _, h := range p.Hears {
if len(h.Filters) == 0 {
continue
}
out = append(out, ModuleConsumer{Node: p.Node, Module: p.Module, Consumer: heardConsumer(p, h)})
}
}
// And the one durable consumer of each of its own work-queue streams, for the one process its
// bus.queues names (novox/hq ADR 0436 §1, §2, §6).
for _, q := range p.Queues {
out = append(out, ModuleConsumer{Node: p.Node, Module: p.Module, Consumer: QueueConsumer(p, q)})
}
}
for _, p := range users {
if p.Kind == KindModule {
add(p)
}
}
for _, p := range users {
if p.Kind != KindNodeTools {
continue
}
for _, d := range p.Carries {
// **A carried module's work-queue consumer is still the controller's to make** (novox/hq ADR
// 0436 §1, §2, §6), the same reason its ordinary consumer is (ADR 0198): mesh-mail has no
// account of its own (NoAccount) and is carried here, but its process does, and the consumer
// it reads through must exist and survive PruneHeard whether or not the module itself speaks.
add(Principal{Kind: KindModule, Node: p.Node, Module: d.Module, Emits: d.Emits,
Consumes: d.Consumes, Serves: d.Serves, Holds: d.Holds, Uses: d.Uses, Watches: d.Watches,
Streams: d.Streams, Hears: ComposedHears(d), Queues: d.Queues})
}
}
return out
}
// HolderConsumerFor is the worker a seat's holders share on that seat's work queue.
//
// **One worker for every holder, and each holder pulls one ask when it is idle** (novox/hq ADR
// 0190). The seat is *authority* — who may be the telegram sender — and the worker is *delivery*,
// kept separate so that relaxing one changes nothing about the other: a node-scoped seat has a
// holder per machine, and all of them take from this one consumer, so the work is shared without
// any holder knowing about the others. Pulled rather than pushed because a push consumer hands the
// next ask to whichever subscriber the server picks, busy or not, and a pulled one is asked for by
// a holder that has just become free. Which is also what ends the race issue 186 describes — asks
// delivered behind the one being worked, expiring unacknowledged and dropped after the fifth
// redelivery: nothing is delivered that nobody asked for. A long build keeps its own ask alive
// (stillWorking); the ack wait is for a holder that died.
func HolderConsumerFor(node, module string, seat DeclaredSeat) (Consumer, bool) {
if len(seat.Accepts) == 0 || seat.PulledByKind {
// A seat pulled by kind has a worker per kind, which its holders' seat traffic derives (SeatTrafficObjects): one
// worker filtered on every accept would overlap them on the work queue.
return Consumer{}, false
}
return Consumer{
Name: "SEAT_" + upperSnake(seat.Name) + "_worker",
Stream: seatStreamName(seat.Name),
Filters: []string{"mesh.seat." + seat.Name + ".accept.>"},
AckWaitSeconds: 60,
MaxDeliver: 5,
// As many in flight as there are holders working, which pulling bounds by itself: a holder
// fetches one and fetches again only after it acknowledged. The server's default stands.
Why: fmt.Sprintf("%s on %s holds %s; every holder pulls one ask at a time from this worker "+
"and acknowledges after the work is done, so a crash mid-work redelivers rather than "+
"loses and an idle holder is the one that takes the next ask", module, node, seat.Name),
}, true
}
// NodeConsumer is the durable consumer a node reads its own declaration through.
//
// **Derived from a node existing, and created by the controller, because a host cannot create it.**
// A host's account may subscribe its own declaration subject and publish its own ack subject, and
// reaches no part of the JetStream API — which is correct (the controller is the only writer of
// consumer definitions, design 25 §3) and means the consumer must be waiting before the host binds
// to it. Named after the node, because the node's ack grant is `$JS.ACK.NODES.<node>.>` and a
// consumer named anything else is one the host cannot acknowledge a delivery from.
//
// **No max-deliver, and a long ack wait.** A declaration is settled only after the node has applied
// it and reported, which is minutes on a machine pulling images; and a declaration the mesh cannot
// get a node to accept is not one to give up on, because the stream keeps only the newest per node
// anyway — so there is exactly one message per node to redeliver, for as long as that node is away.
func NodeConsumer(node string) Consumer {
return Consumer{
Name: node,
Stream: "NODES",
Filters: []string{"mesh.node." + node + ".declare"},
Push: true,
AckWaitSeconds: 300,
Why: "how " + node + " hears what it should be; last-per-subject, so a node that was away " +
"gets exactly the current declaration and nothing older",
}
}
// AssertNodeConsumers brings every known node's declaration consumer into being.
//
// Asserted on start as well as created at enrolment, for the reason the streams are: a mesh raised
// from a restored backup, or one whose bus was recreated, has node records and no consumers, and a
// node whose consumer is missing hears nothing while everything else about it looks correct.
func AssertNodeConsumers(e Ensurer, nodes []string) error {
for _, n := range nodes {
if err := e.EnsureConsumer(NodeConsumer(n)); err != nil {
return fmt.Errorf("asserting how %s hears its declaration: %w", n, err)
}
}
return nil
}
// AllOverlaps reports subject filters claimed by more than one stream, across the mesh's own and
// every derived one.
//
// NATS refuses an overlapping stream rather than merging it (verified against nats-server 2.10:
// "subjects overlap with an existing stream"), so this is not a subtle divergence — it is a
// registration that fails. Catching it here names both streams, before a half-applied mesh does.
func AllOverlaps(seats []DeclaredSeat) []string {
all := append(MeshStreams(), SeatStreams(seats)...)
seen := map[string]string{}
var clashes []string
for _, s := range all {
for _, subject := range s.Subjects {
if first, ok := seen[subject]; ok {
clashes = append(clashes, fmt.Sprintf("%s and %s both claim %s", first, s.Name, subject))
continue
}
seen[subject] = s.Name
}
}
sort.Strings(clashes)
return clashes
}
// upperSnake makes a stream name from a seat name. NATS stream names may not contain a dot,
// a space or a wildcard, and a hyphen is legal but reads badly beside the mesh's own.
func upperSnake(s string) string {
out := []rune(s)
for i, r := range out {
switch {
case r >= 'a' && r <= 'z':
out[i] = r - 32
case r == '-' || r == '.':
out[i] = '_'
}
}
return string(out)
}
+12
View File
@@ -0,0 +1,12 @@
// Package buscompose is the bus's composer: who may say what on the bus, as the user list and the server configuration
// the bus loads, derived from the mesh's records (Users, WithPasswords, ComposeAccounts, Compose), with the grants,
// streams, state and logs it composes them from.
//
// **A copy of the controller's internal/broker, composer half** (issue 1377, novox/hq ADR 0547 §5), taken at
// mesh-controller 7cd4d18b with the grants part's membership grant of f67d1553 (ADR 0547 §5) and its grant to create the streams manifests declare (issue 1790): the bus client (jetstream.go), the broker's address from the environment (broker.go, onnats.go
// but its one constant) and the two methods on that client are left out; the lease's two subjects are copied in
// (nats.go). The controller keeps its own copy, unchanged, until it loses its lease at ADR 0547's third milestone, so
// for that time two composers state who may say what and must say the same: TestTheCopiedComposerComposesAsTheControllers
// holds this copy to lists the controller's own code wrote from the same fixtures. A change to what is granted is made
// here and, while the controller composes, in the controller's copy too, never in one alone.
package buscompose
+119
View File
@@ -0,0 +1,119 @@
package buscompose
// The fixtures the copied bus composer is held to (issue 1377, ADR 0547 §5): the same source is compiled into the
// controller's internal/broker to write the expected lists, and into mesh-sdk's buscompose to compare against them.
// UserListFixture is one mesh's records, and the hashes its accounts carry.
type UserListFixture struct {
Name string
Records Records
// Hashless are users given no hash, so they are left out and named.
Hashless []string
}
func fixtureTelegramSeat() Seat {
return Seat{Name: "telegram-sender", Accepts: []string{"send"}, Emits: []string{"delivered", "failed"}}
}
// UserListFixtures are the meshes, from the controller's own tests' records outward.
func UserListFixtures() []UserListFixture {
return []UserListFixture{
{Name: "empty"},
{Name: "the controller's someRecords", Records: Records{
Nodes: []string{"two", "one"},
Assigned: map[string][]Declared{
"one": {{Module: "telegram", Serves: []string{"status"}}},
"two": {{Module: "shop", Emits: []string{"order.placed"}}},
},
Enrolling: []string{"three"},
People: map[string][]string{"ada": {"mesh-catalog.catalog_tools"}},
}},
{Name: "seats, consumers, people and a hashless user", Hashless: []string{"two.audit"}, Records: Records{
Nodes: []string{"one", "two"},
Assigned: map[string][]Declared{
"one": {
{Module: "telegram", Holds: []Seat{fixtureTelegramSeat()}, Serves: []string{"status"}},
{Module: "nats", SnapshotsTheBus: true, Serves: []string{"nats_streams"}},
{Module: "mesh-watchman"},
{Module: "mesh-grants", Emits: []string{"membership-changed"}},
},
"two": {
{Module: "shop", Uses: []Seat{fixtureTelegramSeat()}, Emits: []string{"order.placed"}},
{Module: "audit", Consumes: []string{"shop.order.placed"}},
},
},
People: map[string][]string{"ada": {"*"}, "bob": {"shop.status"}},
Interchangeable: map[string]bool{"shop": true},
}},
{Name: "state, logs, streams, queues and processes", Records: Records{
Nodes: []string{"ace", "novox"},
Assigned: map[string][]Declared{
"novox": {
{Module: "mesh-delivery-builder", Emits: []string{"built"},
State: []Bucket{{Module: "mesh-delivery-builder", Name: "steps", History: 5},
{Module: "mesh-delivery-builder", Name: "presence", PerMachine: true, KeyTTL: true, TTLSeconds: 60}},
Logs: []Log{{Module: "mesh-delivery-builder", Name: "builds", MaxMiB: 64}},
Streams: []ModuleStream{{Module: "mesh-delivery-builder", Name: "asks", Events: []string{"ask"}, MaxAgeDays: 7, MaxMiB: 32, WorkQueue: true}},
Queues: []QueueOwner{{Process: "worker", Stream: "asks", MaxDeliver: 5}},
Processes: []ProcessPrincipal{{Process: "worker", Emits: []string{"built"},
Queues: []QueueOwner{{Process: "worker", Stream: "asks", MaxDeliver: 5}},
State: []string{"steps"}, Invokes: []string{"gitea.gitea_commit"}}},
Invokes: []string{"gitea.gitea_commit"},
},
{Module: "mesh-delivery-view", Reads: []string{"mesh-delivery-builder.steps"},
KeyedReads: []KeyedRead{{Read: "mesh-delivery-builder.presence", Key: "novox"}}},
// The grants part, which may create every stream declared here (ADR 0547 §5, issue 1790).
{Module: "mesh-grants", Emits: []string{"membership-changed"},
State: []Bucket{{Module: "mesh-grants", Name: "assertions", History: 5}}},
},
"ace": {
{Module: "plex", Serves: []string{"plex_status"}, Checks: []string{"plex.plex_status"}},
{Module: "silent", NoAccount: true},
},
},
RootFree: map[string]bool{"ace": true},
}},
}
}
// HashOf is the stand-in hash a fixture's user carries.
func HashOf(user string) string {
h := "$2a$11$"
for len(h) < 29 {
h += user
}
return h[:29] + "fixturefixturefixturefixture00"
}
// ComposedFor is a fixture's user list as the composer writes it, and the users left out.
func ComposedFor(f UserListFixture) (string, []string, error) {
users, err := Users(f.Records)
if err != nil {
return "", nil, err
}
hashes := map[string]string{}
skip := map[string]bool{}
for _, u := range f.Hashless {
skip[u] = true
}
for _, u := range users {
if !skip[u.Username()] {
hashes[u.Username()] = HashOf(u.Username())
}
}
filled, missing := WithPasswords(users, hashes)
list, err := ComposeAccounts(filled)
return list, missing, err
}
// MembershipsFor is every membership a fixture's records issue, by `<node>.<module>`, as the composer makes them.
func MembershipsFor(f UserListFixture) map[string]Membership {
where := PlacementsOf(f.Records, f.Records.Interchangeable)
out := map[string]Membership{}
for node, ds := range f.Records.Assigned {
for _, d := range ds {
out[node+"."+d.Module] = MembershipFor(node, d, where)
}
}
return out
}
+184
View File
@@ -0,0 +1,184 @@
package buscompose
import (
"fmt"
"sort"
"strings"
"github.com/nats-io/nats.go/jetstream"
)
// A module's log on the bus (novox/hq ADR 0297): its record of operations, each entry appended under
// a key and kept as long as the log is.
//
// A log follows a bucket in every respect (ADR 0201): the module names it locally, the mesh derives
// its stream and subjects, the controller creates it from the catalogue on every raise and from
// registration, never a module, and **no path of the mesh removes one** — a log whose declaration
// is gone is reported, as a bucket is. Unlike a bucket, a log is its owner's alone: no other module
// reads it, so there is no read of a log to grant or issue.
//
// Pure, but for the stream's configuration, which is the server's own type so that what is tested
// is exactly what is sent; jetstream.go is the part that asks a server.
// The mesh's caps on a log: what an entry's value may weigh, what a message on the log's stream may
// weigh — the value and its headers, which the server counts in a message's size, so a value of the
// full size still fits with the expected-last-sequence header an append carries — and what a log holds
// when its module says nothing and at most.
const (
LogMaxEntryBytes = 256 * 1024
LogMaxMessageBytes = LogMaxEntryBytes + 4*1024
LogDefaultMiB = 1024
LogMostMiB = 8192
)
// A Log is one module's declared log as the bus holds it.
type Log struct {
Module string
Name string
// MaxMiB is its cap in MiB; zero is LogDefaultMiB.
MaxMiB int
}
// LogStreamPrefix starts every log's stream name, which no other stream of the mesh's starts with.
const LogStreamPrefix = "LOG_"
// LogStreamName is the stream a module's log lives in: `LOG_<module>_<name>`. The module and the
// local name are each one token with no underscore, so two modules can never derive one stream.
func LogStreamName(module, name string) string { return LogStreamPrefix + module + "_" + name }
// LogSubject is the subject a log's entries are published under, without the key: an entry for key K
// is on `<LogSubject>.<K>`.
func LogSubject(module, name string) string { return "mesh.log." + module + "." + name }
// Stream is this log's stream name.
func (l Log) Stream() string { return LogStreamName(l.Module, l.Name) }
// Subject is this log's subject, without the key.
func (l Log) Subject() string { return LogSubject(l.Module, l.Name) }
// MaxBytes is this log's cap in bytes.
func (l Log) MaxBytes() int64 {
mib := l.MaxMiB
if mib <= 0 {
mib = LogDefaultMiB
}
return int64(mib) * 1024 * 1024
}
// Why is carried into the server's description of the stream, so somebody reading the server's own
// state finds whose it is and why it is kept.
func (l Log) Why() string {
return fmt.Sprintf("%s's log %q (novox/hq ADR 0297): its record of operations, one entry per operation "+
"under its key, appended by %s alone and kept as long as the log; never removed by the mesh, because "+
"it is data", l.Module, l.Name, l.Module)
}
// Config is the stream a log is, field by field as novox/hq ADR 0297 fixes it: a file stream kept by
// limits, with no maximum age and no cap per key, whose message is an entry's value and 4 KiB of
// headers at most, that refuses a new entry when full rather than
// drop an old one, and that refuses deleting an entry or purging it. Direct gets are allowed, which
// is how the runtime reads it.
func (l Log) Config() jetstream.StreamConfig {
return jetstream.StreamConfig{
Name: l.Stream(),
Description: l.Why(),
Subjects: []string{l.Subject() + ".>"},
Storage: jetstream.FileStorage,
Retention: jetstream.LimitsPolicy,
MaxAge: 0,
MaxMsgs: -1,
MaxMsgsPerSubject: -1,
MaxBytes: l.MaxBytes(),
MaxMsgSize: LogMaxMessageBytes,
Discard: jetstream.DiscardNew,
AllowDirect: true,
DenyDelete: true,
DenyPurge: true,
Replicas: 1,
}
}
// logsIssuedFor is every log a module's code may reach, as its membership lists them: its own.
func logsIssuedFor(d Declared) []LogIssued {
var out []LogIssued
for _, l := range d.Logs {
if !safeSubject.MatchString(d.Module) || !safeSubject.MatchString(l.Name) {
continue
}
out = append(out, LogIssued{Name: l.Name, Stream: LogStreamName(d.Module, l.Name),
Subject: LogSubject(d.Module, l.Name), Writes: true})
}
return out
}
// logGrants is what a principal publishes to reach the logs its module keeps: for each, appending
// under the log's subjects, binding to its stream, and reading it directly. Nothing more — the
// runtime reads a log by direct gets alone and makes no consumer on it — and nothing of any other
// module's log, because a log is its owner's alone. Replies come on the principal's inbox, as for a
// bucket.
func logGrants(module string, names []string) []string {
if !safeSubject.MatchString(module) {
return nil
}
var out []string
for _, name := range names {
if !safeSubject.MatchString(name) {
continue
}
stream := LogStreamName(module, name)
out = append(out,
LogSubject(module, name)+".>",
"$JS.API.STREAM.INFO."+stream,
"$JS.API.DIRECT.GET."+stream,
"$JS.API.DIRECT.GET."+stream+".>")
}
return out
}
// logNames is the local names of a module's logs.
func logNames(logs []Log) []string {
out := make([]string, 0, len(logs))
for _, l := range logs {
out = append(out, l.Name)
}
return out
}
// A LogAsserter is the part of a JetStream connection log assertion needs. It has no way to remove a
// log, by design: nothing the mesh runs asks for one.
type LogAsserter interface {
// EnsureLog creates the log's stream if absent and brings its configuration to match if present,
// never deleting or recreating it.
EnsureLog(l Log) error
// LogStreams is every log stream on the server: every stream named with LogStreamPrefix.
LogStreams() ([]string, error)
}
// RaiseLogs asserts every declared log and answers the log streams on the server that nothing
// declares any more.
//
// **Those are reported, never removed** (novox/hq ADR 0297 §2, ADR 0201 §15–16): a log is a module's
// record, and a manifest edited, a module renamed or unassigned is an ordinary day's work that must
// not take a record with it. Removing one is a person's act, outside the mesh.
func RaiseLogs(a LogAsserter, logs []Log) (undeclared []string, err error) {
sorted := append([]Log(nil), logs...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Stream() < sorted[j].Stream() })
declared := map[string]bool{}
for _, l := range sorted {
if err := a.EnsureLog(l); err != nil {
return nil, fmt.Errorf("asserting %s's log %q: %w", l.Module, l.Name, err)
}
declared[l.Stream()] = true
}
names, err := a.LogStreams()
if err != nil {
return nil, fmt.Errorf("listing the bus's logs: %w", err)
}
for _, n := range names {
if strings.HasPrefix(n, LogStreamPrefix) && !declared[n] {
undeclared = append(undeclared, n)
}
}
sort.Strings(undeclared)
return undeclared, nil
}
+178
View File
@@ -0,0 +1,178 @@
package buscompose
import (
"slices"
"sort"
"strings"
)
// What the mesh issues an assignment to serve and to reach (novox/hq ADR 0160).
//
// A module's code names its tools and its events; **where they land is the mesh's to decide**, and
// it decided it twice — once in the runtime, once here, by one rule compiled into both. Now the
// controller composes a membership for every module on every machine and publishes it to a subject
// only that assignment reads; the runtime serves exactly what the membership says, and the account's
// grant is the same composition read the other way. The shape issued today is the shape the mesh
// already had, so nothing moves when a membership first arrives; only who decides it moves.
// Placements is where every module runs, for deciding which instance answers for the module.
type Placements struct {
// Nodes is each module's machines.
Nodes map[string][]string
// Interchangeable is each module whose definition says its instances are the same anywhere,
// so the module's plain subject is issued to all of them in one queue.
Interchangeable map[string]bool
// Kinds is every kind held of a kinded bench, by whom and with what capabilities (ADR 0259 §5).
Kinds []KindHeld
}
// AnswersForTheModule says whether an instance of a module on one machine is issued the module's
// plain subject: when it is the only instance, or when the definition says instances are
// interchangeable. A stateful module on two machines gets only its machines' subjects, so a call
// that names none reaches nothing rather than the wrong store.
func (p Placements) AnswersForTheModule(module string) bool {
return len(p.Nodes[module]) <= 1 || p.Interchangeable[module]
}
// MembershipFor composes one assignment's membership from what it declared and where everything
// runs. The subjects are the ones PermissionsFor grants, derived here once more only until the
// grant itself is read from the membership — which is the next step, not this one.
func MembershipFor(node string, d Declared, where Placements) Membership {
own := "mesh.mod." + d.Module
m := Membership{
Node: node, Module: d.Module,
Emits: own + ".event.{event}",
Tools: own + ".tool.tools",
}
// This machine's address always; the module's when this instance answers for the module.
m.Serves = append(m.Serves, Served{Subject: own + ".tool.{tool}." + node})
if where.AnswersForTheModule(d.Module) {
m.Serves = append(m.Serves, Served{Subject: own + ".tool.{tool}", Queue: "serve." + d.Module})
}
for _, s := range d.Holds {
if servedOnlyByTheController(s) {
continue // answered by the serving controller alone, never through a membership
}
for _, verb := range s.Serves {
m.Seats = append(m.Seats, SeatServed{Seat: s.Name, Verb: verb, Subject: seatToolSubject(s, verb, node)})
}
}
m.State = stateIssuedFor(d, node)
m.Logs = logsIssuedFor(d)
m.Streams = streamsIssuedFor(d)
m.Hears = HeardIssuedFor(node, d)
m.Waits = waitsIssuedFor(d)
t := SeatTrafficOf(d.Module, d.Holds, d.Uses, d.Watches)
for _, s := range append(append([]Seat{}, d.Uses...), d.Watches...) {
if !s.Kinded {
continue
}
for _, k := range where.Kinds {
if k.Seat == s.Name && !kindListed(t.Kinds, k) {
t.Kinds = append(t.Kinds, k)
}
}
}
if len(t.Publish)+len(t.Subscribe)+len(t.Answers)+len(t.Workers)+len(t.Records)+len(t.Kinds) > 0 {
m.SeatTraffic = &t
}
if len(d.Invokes) > 0 {
m.Reaches = map[string][]string{}
for _, t := range d.Invokes {
if t == "*" || strings.HasPrefix(t, "seat:") {
continue // every tool, or a role's: addressed by name, not resolved per instance
}
module, tool, ok := strings.Cut(t, ".")
if !ok {
continue
}
var reach []string
if where.AnswersForTheModule(module) {
reach = append(reach, "mesh.mod."+module+".tool."+tool)
}
nodes := append([]string{}, where.Nodes[module]...)
sort.Strings(nodes)
for _, n := range nodes {
reach = append(reach, "mesh.mod."+module+".tool."+tool+"."+n)
}
m.Reaches[t] = reach
}
}
return m
}
// PlacementsOf reads where everything runs from the records the bus's accounts are composed from.
func PlacementsOf(r Records, interchangeable map[string]bool) Placements {
p := Placements{Nodes: map[string][]string{}, Interchangeable: interchangeable}
for node, declared := range r.Assigned {
for _, d := range declared {
p.Nodes[d.Module] = append(p.Nodes[d.Module], node)
}
}
for _, nodes := range p.Nodes {
sort.Strings(nodes)
}
// Where the router runs: a verified sender is believed only while its machine is root-free too. A router
// placed nowhere, or on more than one machine, frees nothing.
var routerNodes []string
for node, declared := range r.Assigned {
for _, d := range declared {
for _, s := range d.Holds {
if s.Name == routerSeat && !slices.Contains(routerNodes, node) {
routerNodes = append(routerNodes, node)
}
}
}
}
routerFree := len(routerNodes) == 1 && r.RootFree[routerNodes[0]]
for node, declared := range r.Assigned {
for _, d := range declared {
for _, s := range d.Holds {
if s.Kinded && s.Kind != "" {
p.Kinds = append(p.Kinds, KindHeld{Seat: s.Name, Kind: s.Kind, Module: d.Module, Node: node,
Capabilities: placedCapabilities(s.Capabilities, d.RunsAs, routerFree && r.RootFree[node])})
}
}
}
}
sort.Slice(p.Kinds, func(i, j int) bool {
a, b := p.Kinds[i], p.Kinds[j]
if a.Seat != b.Seat {
return a.Seat < b.Seat
}
if a.Kind != b.Kind {
return a.Kind < b.Kind
}
return a.Node < b.Node
})
return p
}
// routerSeat is the seat the router of asks holds (novox/hq ADR 0259 §3).
const routerSeat = "operator-channel"
// placedCapabilities is what a kind's claim promises, as far as its placement lets the router believe it
// (novox/hq ADR 0259 §8): `verified-sender` only from a holder that runs as an account of its own, on a bus
// account of its own — never one the machine's runtime carries as the operator's account — and only while
// its machine and the router's were root-free when composed (rootFree; the review of 2026-10-09, H3). The
// membership carrying it is composed at a push, so it can outlive a pass that later fails: the router asks
// the controller's root-free verb again before it honours an approval, and that is the check that holds.
func placedCapabilities(declared []string, runsAs string, rootFree bool) []string {
var out []string
for _, c := range declared {
if c == "verified-sender" && (runsAs == "" || !rootFree) {
continue
}
out = append(out, c)
}
return out
}
func kindListed(list []KindHeld, k KindHeld) bool {
for _, x := range list {
if x.Seat == k.Seat && x.Kind == k.Kind && x.Module == k.Module && x.Node == k.Node {
return true
}
}
return false
}
+343
View File
@@ -0,0 +1,343 @@
package buscompose
import (
"fmt"
"sort"
"strings"
)
// A module's own stream on the bus (novox/hq ADR 0363).
//
// A module declares a stream as it declares a log: named locally, created by the controller from the
// catalogue on every raise and every send, never removed by the mesh. Its events are published on
// `mesh.stream.<module>.event.<event>`, a subject space the shared EVENTS stream (`mesh.mod.*.event.>`)
// does not take — the bus refuses two streams claiming one subject, so a module's stream cannot share
// EVENTS' — and a consumer hears it through a durable consumer of its own on that stream, named as on
// EVENTS. Pure; the controller's raise asserts what is composed here.
// moduleStreamPrefix starts every subject a module's own stream takes.
const moduleStreamPrefix = "mesh.stream."
// ModuleStreamSubject is where a module's event on its own stream lands: `mesh.stream.<module>.event.<event>`.
// The token before `.event.` is the emitter, as on EVENTS, so a runtime reads the same key from it.
func ModuleStreamSubject(module, event string) string {
return moduleStreamPrefix + module + ".event." + event
}
// OnAModuleStream says a subject is one a module's own stream takes.
func OnAModuleStream(subject string) bool { return strings.HasPrefix(subject, moduleStreamPrefix) }
// WirePattern is a local event pattern as the bus spells it: `**` is `>`.
func WirePattern(local string) string {
parts := strings.Split(local, ".")
for i, p := range parts {
if p == "**" {
parts[i] = ">"
}
}
return strings.Join(parts, ".")
}
// A ModuleStream is one module's declared stream as the bus holds it.
type ModuleStream struct {
Module string
// Name is its name on the bus, `MAIL`.
Name string
// Events are the events it carries, as the bus spells them (WirePattern).
Events []string
MaxAgeDays int
MaxMiB int
// Restricted says a trusted config value of the module grants who hears it (ADR 0363 §4).
Restricted bool
// WorkQueue says this stream holds ADR 0436 §6's work-queue retention: a message is removed once a
// consumer acknowledges it, and the bus refuses a new one when full rather than dropping its oldest.
WorkQueue bool
// Duplicates is the work-queue's duplicate window in hours (novox/hq ADR 0436 §6); meaningless unless
// WorkQueue.
Duplicates int
}
// Subjects are the subjects the stream takes.
func (s ModuleStream) Subjects() []string {
var out []string
for _, e := range s.Events {
out = append(out, ModuleStreamSubject(s.Module, e))
}
sort.Strings(out)
return out
}
// Stream is the stream the controller asserts: kept by limits, its age and size the module's, dropping
// its oldest when full — a consumer that was away catches up within the age, and the module's own record
// answers for anything older.
func (s ModuleStream) Stream() Stream {
why := fmt.Sprintf("%s's own stream (novox/hq ADR 0363): its events, kept %d days and at most %d MiB so a "+
"consumer that was away catches up", s.Module, s.MaxAgeDays, s.MaxMiB)
if s.Restricted {
why += "; heard only by the consumers its trusted config value grants"
}
if s.WorkQueue {
why += "; a work-queue (novox/hq ADR 0436 §6): a message is removed once its one consumer " +
"acknowledges it, and a full stream refuses a new one rather than dropping what it holds"
}
out := Stream{
Name: s.Name,
Subjects: s.Subjects(),
Retention: RetentionLimits,
MaxAge: s.MaxAgeDays * 24 * 60 * 60,
MaxBytes: int64(s.MaxMiB) * 1024 * 1024,
Why: why,
// Said on the bus, so a dead letter kept from it ages out with it (novox/hq ADR 0376 §3).
Restricted: s.Restricted,
}
if s.WorkQueue {
out.Retention = RetentionWorkQueue
out.DiscardNew = true
out.DuplicatesSeconds = s.Duplicates * 60 * 60
// Said on the bus too, reusing the one flag a restricted stream's dead letters already age and hide
// by (novox/hq ADR 0436 §6, ADR 0376 §3-4): a work-queue stream's grant is always empty (ADR 0436
// §6 refuses one that carries a grant), so this never collides with a trusted-config restriction.
out.Restricted = true
// And its own metadata, apart from the bus's work-queue retention SEAT_* and other mesh-internal
// streams share without being one of this clause's: only this flag says a dead letter kept from the
// stream is drop-only (novox/hq ADR 0436 §6), never the stream's Retention alone.
out.WorkQueue = true
}
return out
}
// QueueOwner is the one process of a module that is the durable consumer of one of its own work-queue
// streams (novox/hq ADR 0436 §1, §2, §6): named in that process's bus.queues, and checked at registration
// to be the only one (catalogue.WorkQueueStreamProblems).
type QueueOwner struct {
// Process is the process resource's id — its bus account's name, with the node and module (novox/hq ADR
// 0436 §2).
Process string
// Stream is the work-queue stream's name on the bus.
Stream string
// MaxDeliver bounds how many times a message is handed over before it is given up as a dead letter
// (novox/hq ADR 0436 §6).
MaxDeliver int
}
// QueueConsumer is the durable consumer the one process a module's bus.queues names is granted on its own
// work-queue stream (novox/hq ADR 0436 §1, §2, §6): no filter, since it is the stream's only consumer —
// unlike heardConsumer, which is one of possibly several hearing a module's ordinary stream.
func QueueConsumer(p Principal, q QueueOwner) Consumer {
return Consumer{
Name: p.Node + "_" + p.Module + "_" + q.Process,
Stream: q.Stream,
AckWaitSeconds: 30,
MaxDeliver: q.MaxDeliver,
Why: fmt.Sprintf("%s's process %s is the one consumer %s's bus.queues names for its work-queue "+
"stream %s (novox/hq ADR 0436 §6); after max-deliver %d it gives a message up, and the "+
"controller keeps it in %s, offered only drop", p.Module, q.Process, p.Module, q.Stream,
q.MaxDeliver, DeadLettersStream),
}
}
// ModuleStreams is every declared module stream as the controller asserts it, in name order.
func ModuleStreams(declared []ModuleStream) []Stream {
sorted := append([]ModuleStream(nil), declared...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Name < sorted[j].Name })
out := make([]Stream, 0, len(sorted))
for _, s := range sorted {
out = append(out, s.Stream())
}
return out
}
// Heard is a module's stream another module hears, and the subjects it hears of it: its `consumes`
// within the stream and, for a restricted stream, within what the provider's trusted config value grants it
// (ADR 0363 §5). Composed only with at least one subject.
type Heard struct {
Stream string
Emitter string
Filters []string
// Restricted says the stream is restricted by its provider's trusted config value (ADR 0363 §4): then it
// is heard only by a module on an account of its own (ADR 0376 §1, ComposedHears).
Restricted bool
}
// ComposedHears is what of a module's hearing the controller composes (novox/hq ADR 0376 §1): every stream it
// hears, but a restricted one only for a module that runs in a runtime of its own on a bus account of its own
// (`runs-as`, ADR 0259). **The tool runner never hears a restricted stream**: a module it carries reaches the bus
// through the tool runner's account, which every agent session on that machine reaches the bus as too, so a
// consumer, its grants or its bind word composed there would hand the stream to any session. A module that
// declares no `runs-as` and keeps its own container credential gets none either: only `runs-as` is an account
// the mesh keeps from agents.
func ComposedHears(d Declared) []Heard {
if d.RunsAs != "" {
return d.Hears
}
var out []Heard
for _, h := range d.Hears {
if !h.Restricted {
out = append(out, h)
}
}
return out
}
// Withheld is a restricted stream a grant gives a module that the controller composes nothing for, because the
// module has no account of its own (novox/hq ADR 0376 §2): said in `doctor` and in the controller's log, never
// silent.
type Withheld struct {
Node, Module, Stream, Emitter string
}
// Said is a withheld hearing in the operator's words: the module, the stream and the remedy.
func (w Withheld) Said() string {
return fmt.Sprintf("%s on %s is granted part of %s's restricted stream %s, and hears none of it: it is carried "+
"by the machine's tool runner, whose bus account every agent session there shares, so nothing is composed "+
"for it. Declare runs-as in %s's manifest, so it runs as an account of its own (novox/hq ADR 0376)",
w.Module, w.Node, w.Emitter, w.Stream, w.Module)
}
// WithheldOf is every granted hearing of a restricted stream composed for nothing, by machine and module.
func WithheldOf(r Records) []Withheld {
var out []Withheld
for _, node := range sortedCopy(r.Nodes) {
for _, d := range r.Assigned[node] {
if d.RunsAs != "" {
continue
}
for _, h := range d.Hears {
if h.Restricted && len(h.Filters) > 0 {
out = append(out, Withheld{Node: node, Module: d.Module, Stream: h.Stream, Emitter: h.Emitter})
}
}
}
}
return out
}
// heardConsumer is the durable consumer a module hears a module's stream through: named as its consumer
// on EVENTS, on that stream, with the subjects it was granted as its filters.
func heardConsumer(p Principal, h Heard) Consumer {
filters := append([]string(nil), h.Filters...)
sort.Strings(filters)
return Consumer{
Name: consumerDurable(p),
Stream: h.Stream,
Filters: filters,
AckWaitSeconds: 30,
MaxDeliver: 5,
// From now, like a module's consumer on EVENTS (novox/hq ADR 0443 §1).
FromNow: true,
Why: "what " + p.Module + " consumes of " + h.Emitter + "'s stream " + h.Stream + " and is granted " +
"(novox/hq ADR 0363); after max-deliver it gives an event up, and the controller keeps it in " +
"DEAD_LETTERS until a person delivers it again or drops it",
}
}
// heardGrants are what a principal publishes to read and acknowledge the consumers it hears module streams
// through: asking about each, pulling from it, acknowledging it — its own name on each stream, no other's.
func heardGrants(p Principal, hears []Heard) []string {
var out []string
durable := consumerDurable(p)
for _, h := range hears {
if !safeSubject.MatchString(h.Stream) || len(h.Filters) == 0 {
continue
}
out = append(out,
"$JS.API.CONSUMER.INFO."+h.Stream+"."+durable,
"$JS.API.CONSUMER.MSG.NEXT."+h.Stream+"."+durable,
"$JS.ACK."+h.Stream+"."+durable+".>")
}
return out
}
// ownStreamGrants are the subjects a module publishes its own streams' events on, and no other's.
func ownStreamGrants(module string, streams []ModuleStream) []string {
var out []string
for _, s := range streams {
if s.Module != module {
continue
}
out = append(out, s.Subjects()...)
}
return out
}
// streamsIssuedFor is every stream of its own a module's membership names.
func streamsIssuedFor(d Declared) []StreamIssued {
var out []StreamIssued
for _, s := range d.Streams {
if s.Module != d.Module || !safeSubject.MatchString(d.Module) {
continue
}
out = append(out, StreamIssued{Stream: s.Name, Events: append([]string(nil), s.Events...),
Subject: ModuleStreamSubject(d.Module, "{event}")})
}
return out
}
// HeardIssuedFor is every module stream an assignment hears, as its membership names it.
func HeardIssuedFor(node string, d Declared) []HeardIssued {
var out []HeardIssued
// A restricted stream's bind word only for a module of its own account (novox/hq ADR 0376 §1).
for _, h := range ComposedHears(d) {
if len(h.Filters) == 0 {
continue
}
out = append(out, HeardIssued{Stream: h.Stream, Consumer: node + "_" + d.Module})
}
sort.Slice(out, func(i, j int) bool { return out[i].Stream < out[j].Stream })
return out
}
// Pruner is the part of a JetStream connection that removes a module's consumer on a module's stream it
// may no longer hear (ADR 0363 §6). Optional: what only records what it is asked to assert has none.
type Pruner interface {
ConsumerNames(stream string) ([]string, error)
RemoveConsumer(stream, name string) error
}
// PruneHeard removes, from each module stream, every durable consumer the composition no longer asks for,
// and answers what it removed as `<stream>/<consumer>`. A consumer that loses its grant loses what it reads
// through, whatever its runtime still holds; a stream that does not exist yet has nothing to remove.
func PruneHeard(p Pruner, streams []ModuleStream, wanted []Consumer) ([]string, error) {
keep := map[string]bool{}
for _, c := range wanted {
keep[c.Stream+"/"+c.Name] = true
}
var removed []string
for _, s := range streams {
names, err := p.ConsumerNames(s.Name)
if err != nil {
return removed, fmt.Errorf("listing the consumers on %s: %w", s.Name, err)
}
for _, n := range names {
if keep[s.Name+"/"+n] {
continue
}
if err := p.RemoveConsumer(s.Name, n); err != nil {
return removed, fmt.Errorf("removing %s's consumer on %s, which is no longer granted: %w", n, s.Name, err)
}
removed = append(removed, s.Name+"/"+n)
}
}
return removed, nil
}
// EnsureHeard asserts a consumer on a module's stream (ADR 0363 §5, §6). **On a restricted stream, a consumer
// the bus will not bring to match is removed**: a pull consumer delivers to whoever pulls it, so one left with
// its wider filters after a narrowing was refused would keep handing over what the grant no longer names,
// whatever the user list now says. The next assertion makes it again with the granted filters. The refusal is
// returned either way, saying what was done, so the send says it and raises it.
func EnsureHeard(r Raiser, c Consumer, restricted bool) error {
err := r.EnsureConsumer(c)
if err == nil || !restricted {
return err
}
p, prunes := r.(Pruner)
if !prunes {
return fmt.Errorf("%w; it could not be removed here, and may still deliver more than it is granted", err)
}
if rerr := p.RemoveConsumer(c.Stream, c.Name); rerr != nil {
return fmt.Errorf("%w; REMOVING IT FAILED TOO, so it may still deliver more than it is granted: %v", err, rerr)
}
return fmt.Errorf("%w; REMOVED, so it delivers nothing it is no longer granted: the next assertion makes it "+
"again with its granted filters (novox/hq ADR 0363 §6)", err)
}
File diff suppressed because it is too large Load Diff
+5
View File
@@ -0,0 +1,5 @@
package buscompose
// NATSVar names the environment variable that says the controller is on the bus (mesh-controller internal/broker
// onnats.go); kept with the composer for the readiness it composes.
const NATSVar = "MESH_BUS_NATS"
+56
View File
@@ -0,0 +1,56 @@
package buscompose
// The bus account of one process of a module that declares bus (novox/hq ADR 0436 §1, §2, §3, §5).
//
// **Never the module's own account's grants, never the tool runner's.** A process's authority is read
// straight from its bus declaration — its emits, its queue's consumer, its state, whether it reads the
// records of its module's own asks, and the tools and seat verbs it calls — and nothing else: it holds no
// seat, serves no tool, hears no module stream, and answers nothing a caller asked (§3). Modelled on
// provisionerGrants, the other kind of process account the mesh mints (ADR 0349 clauses 6 and 11).
// processGrants is one process's whole authority: to publish, and to subscribe.
func processGrants(p Principal) (pub, sub []string, err error) {
// 1. Its own events, under its module's own namespace — the same subjects the module's own principal
// would publish them on, had it declared them there instead (novox/hq ADR 0436 §1).
own := "mesh.mod." + p.Module
for _, e := range p.Emits {
pub = append(pub, own+".event."+e)
}
// 2. The one durable consumer of each work-queue stream its bus names (novox/hq ADR 0436 §1, §2, §6):
// asked about, pulled from and acknowledged, by the same name ConsumersOf composes the consumer under
// (QueueConsumer) — and no other stream's.
for _, q := range p.Queues {
consumer := p.Node + "_" + p.Module + "_" + q.Process
pub = append(pub,
"$JS.API.CONSUMER.INFO."+q.Stream+"."+consumer,
"$JS.API.CONSUMER.MSG.NEXT."+q.Stream+"."+consumer,
"$JS.ACK."+q.Stream+"."+consumer+".>")
}
// 3. Its module's own state, by the local names its bus names alone — never the whole of what the
// module keeps (novox/hq ADR 0436 §1), read and written as stateGrants gives any module's own bucket.
kept := stateAccess{Module: p.Module, Node: p.Node, Keeps: p.State}
pub = append(pub, stateGrants(kept)...)
// And it hears each of them created, so one not on the bus at its start is bound when it appears (novox/hq issue
// 1790; process.AwaitBucket).
sub = append(sub, stateCreatedHeard(kept)...)
// 4. The router's records of its module's own asks (novox/hq ADR 0436 §1, ADR 0259 §3): the same
// direct-get subjects a module's own principal reads for the seats its Uses names that keep them —
// read here from the one list, so the two cannot disagree.
if p.Asks {
pub = append(pub, SeatTrafficOf(p.Module, nil, p.Uses, nil).Records...)
}
// 5. The tools and seat verbs it calls (novox/hq ADR 0152, ADR 0436 §1): the same derivation a
// module's own principal's invokes takes, so `callerNamed` (applied to every non-controller principal)
// adds the caller-named `call` variants for it exactly as it would for a module.
invoked, err := invokedSubjects(p.Invokes)
if err != nil {
return nil, nil, err
}
pub = append(pub, invoked...)
return pub, sub, nil
}
+217
View File
@@ -0,0 +1,217 @@
package buscompose
import (
"errors"
"fmt"
"git.novox.be/novox/mesh-sdk/go/delivery"
"strings"
"github.com/nats-io/nats.go"
)
// The provisioner client's own bus account (novox/hq ADR 0349 clauses 6 and 11, design 57 package P3).
//
// **Not the tool runner's account.** The client fills a machine's files and asks the vault for the secrets that
// machine's declaration names; the vault hands a secret only to an account of its own, because a call under the
// tool runner's account cannot be told from an agent's on that machine (ADR 0305 §3, §12). So each machine that
// runs the client has a principal of its own, KindProvisioner, named for the machine, and granted what the client
// does and nothing else: read its own machine's keys in each domain's bucket, write its own machine's keys of the
// provisioner server's needs, applied and collections (design 57 P4), read the server's registry, say what it
// applied on its own machine, and ask the vault in its own name.
// NodeProvisionerSeat is the seat the provisioner client holds on every machine (ADR 0349 clause 3).
const NodeProvisionerSeat = "node-provisioner"
// ProvisionerSecret is the own secret of the module running the client that its bus account is delivered in, as
// `broker` is a module's own (novox/hq issue 195): a module that claims NodeProvisionerSeat and declares no own
// secret of this name has nowhere to read an account, and is given none.
const ProvisionerSecret = "provisioner-broker"
// ConfigResolvedBucket is the config domain's resolved values, keyed `<node>.<module>.<key>` (ADR 0348, research
// 051): mesh-config's state `resolved`, written by mesh-config alone.
var ConfigResolvedBucket = BucketName("mesh-config", "resolved")
// ProvisionerReads are the domains' buckets a provisioner client reads its own machine's keys of, each keyed
// `<node>.…` (research 051): placement, the controller's assignments mirror (`<node>.<module>`), whose value carries
// the provisioner's plan of a module handed to it (inventory.Provisioning, design 57 P5); and config,
// mesh-config's resolved values. **Named one by one**, as the controller's asks are: a domain that moves to the
// provisioner adds its bucket here, and a bucket not named is one the client cannot read.
var ProvisionerReads = []string{ConfigResolvedBucket}
// ProvisionerReadsWhole are the buckets a provisioner client reads every key of: the controller's assignments mirror,
// where a provider's consumers are bound to it in their own machines' plans, which the provider machine's client
// reads to fill ${consumers:<provision>} with each one's credential (novox/hq ADR 0373, issue 679). It holds placement
// and plans, never a secret's value; the client writes none of it.
var ProvisionerReadsWhole = []string{AssignmentsBucket}
// BoundState is a providing module's state of entries, one per consumer (mesh-sdk go/provider, issue 678): a
// consumer's machine's client reads its own keys of it, `<node>.<module>.<provision>[.<local>]`, to fill
// ${bound:<provision>:<key>} (issue 679).
const BoundState = "bound"
// ProvisionerServerModule is the module holding the seat mesh-provisioner once for the mesh (ADR 0349 clause 3,
// design 57 P4), whose buckets the clients write their own machine's keys of.
const ProvisionerServerModule = "mesh-provisioner"
// The provisioner server's buckets (ADR 0349 clause 3): its state, created by the controller from its manifest
// like any module's, and written by its own account whole.
var (
// ProvisionerRegistryBucket is which domain answers which kind of placeholder, keyed by the kind; the
// server's alone, read whole by every client.
ProvisionerRegistryBucket = BucketName(ProvisionerServerModule, "registry")
// ProvisionerNeedsBucket is what each module on each machine needs, `<node>.<module>.<kind>.<name>`.
ProvisionerNeedsBucket = BucketName(ProvisionerServerModule, "needs")
// ProvisionerAppliedBucket is what is applied where, `<node>.<module>`.
ProvisionerAppliedBucket = BucketName(ProvisionerServerModule, "applied")
// ProvisionerCollectBucket is the items of every collection, `<node>.<collection>.<contributor>`.
ProvisionerCollectBucket = BucketName(ProvisionerServerModule, "collect")
)
// ProvisionerWrites are the server's buckets a client writes, and reads, its own machine's keys of (`<node>.>`)
// and no other's: the needs it records, what it applied, and the items of the collections on its machine. A
// client cannot write another machine's needs, or say another machine applied a file.
var ProvisionerWrites = []string{ProvisionerNeedsBucket, ProvisionerAppliedBucket, ProvisionerCollectBucket}
// VerbsTheProvisionerAsksTheVault are the vault's verbs a provisioner client asks in its own name (ADR 0349
// clause 11): `get`, a secret its machine's declaration names, answered sealed to the asker's key. On the subject
// that names its caller, so the vault knows which machine's client asks from the bus's word.
var VerbsTheProvisionerAsksTheVault = []SeatVerb{{Seat: "mesh-vault", Verb: "get"}}
// VaultSecretChangedSubject is where the vault says a secret it holds changed (`secret-changed`: which, by whom,
// when, its generation; never a value, ADR 0349 clause 11), as link.EventSubject addresses mesh-vault's event. A
// provisioner client listens to it so a rotated secret reaches its file without a new declaration (design 57 V4):
// it asks the vault again, in its own name, for a secret its machine's declaration names.
const VaultSecretChangedSubject = "mesh.mod.mesh-vault.event.secret-changed"
// ProvisionerAppliedSubject is where a machine's client says what it applied for one module, applied or failed
// with the reason (ADR 0349 clauses 4 and 15): `prov.applied.<node>.<module>`.
func ProvisionerAppliedSubject(node, module string) string {
return "prov.applied." + node + "." + module
}
// ProvisionerAskSubject is where a machine's tool runner asks that machine's provisioner client for a config value
// or a non-secret bound value of a module it carries, naming the module from the bundle that asks (novox/hq ADR
// 0370 §1, §3, §4; issue 685): `prov.ask.<node>`, which the client subscribes and the tool runner's account of the
// same machine alone may publish. The bus proves only that the tool runner asks; the module is the runtime's word,
// so the client answers no secret here (ADR 0370 §2).
func ProvisionerAskSubject(node string) string { return "prov.ask." + node }
// ProvisionerChangedSubject is where a machine's client says that a value of one module it answers moved, so the
// tool runner asks again for the module's code's watches (ADR 0349 §14, ADR 0370 §1): `prov.changed.<node>.<module>`,
// carrying which value and never the value itself, heard by that machine's tool runner alone.
func ProvisionerChangedSubject(node, module string) string {
return "prov.changed." + node + "." + module
}
// provisionerGrants is a provisioner client's whole authority: to publish, and to subscribe.
func provisionerGrants(p Principal) (pub, sub []string, err error) {
if p.Node == "" {
// Refused rather than composed into `$KV.<bucket>..>`, a pattern no machine's keys are under, and a
// name every nameless client would share.
return nil, nil, errors.New("a provisioner client names no machine, so it would have no keys of its own " +
"to read and an account it shared with every other")
}
for _, bucket := range ProvisionerReads {
pub = append(pub, machineKeysRead(bucket, p.Node)...)
}
for _, bucket := range ProvisionerReadsWhole {
pub = append(pub, wholeBucketRead(bucket)...)
}
for _, bucket := range p.BoundBuckets {
module, name, ok := strings.Cut(bucket, "_")
if !ok || name != BoundState || !safeSubject.MatchString(module) || strings.Contains(module, "_") {
return nil, nil, fmt.Errorf("%q is not a provider's bucket of entries, <module>_%s", bucket, BoundState)
}
pub = append(pub, machineKeysRead(bucket, p.Node)...)
}
// The server's buckets (design 57 P4): its own machine's keys of the needs, the applied and the
// collections, read and written, `$KV.<bucket>.<node>.>` and nothing wider; and the registry, read whole,
// which says which domain answers which kind and is the same for every machine.
for _, bucket := range ProvisionerWrites {
pub = append(pub, machineKeysRead(bucket, p.Node)...)
pub = append(pub, "$KV."+bucket+"."+p.Node+".>")
}
pub = append(pub, wholeBucketRead(ProvisionerRegistryBucket)...)
// Its own node's own have, `<node>.node`, read and never written, and no other key of the bucket: it fills every
// module again when the node-engine moves to a new composition of the node's own (issue 1945), since an apply can
// leave a filled file as the composition had it.
pub = append(pub, readGrants(delivery.HaveBucket, p.Node+".node")...)
pub = append(pub, "prov.applied."+p.Node+".>")
token := CallerToken(p.Username())
if token == "" {
return nil, nil, fmt.Errorf("%q cannot name a caller", p.Username())
}
for _, v := range VerbsTheProvisionerAsksTheVault {
pub = append(pub, "mesh.seat."+v.Seat+"."+CallKind+"."+v.Verb+"."+token)
}
// What moved of a value it answers its own machine's tool runner (ADR 0370 §1), under its own machine's name.
pub = append(pub, "prov.changed."+p.Node+".>")
// Its own reply space, for the vault's answers and its reads' and watches' deliveries; what the vault says
// changed; and its own machine's tool runner asking it for a module's config and non-secret bound values (ADR
// 0370 §3), which it answers on the ask's reply and nowhere else.
return pub, []string{p.inbox(), VaultSecretChangedSubject, ProvisionerAskSubject(p.Node)}, nil
}
// machineKeysRead is reading every key of a bucket under one machine's name, `<node>.>`, and no other key: a
// direct get of a key's own subject, and a consumer whose filter is under it — the server's create-with-filter
// form, which it holds to the filter the subject names (as stateGrants does for one key, novox/hq ADR 0260).
// Binding, deleting a consumer and flow control cannot be narrowed by key and read no value.
func machineKeysRead(bucket, node string) []string {
stream := "KV_" + bucket
keys := "$KV." + bucket + "." + node + ".>"
return []string{
"$JS.API.STREAM.INFO." + stream,
"$JS.API.DIRECT.GET." + stream + "." + keys,
"$JS.API.CONSUMER.CREATE." + stream + ".*." + keys,
"$JS.API.CONSUMER.DELETE." + stream + ".>",
"$JS.FC." + stream + ".>",
}
}
// wholeBucketRead is reading every key of a bucket and writing none: what stateGrants grants a module for a
// bucket it `reads`.
func wholeBucketRead(bucket string) []string {
stream := "KV_" + bucket
return []string{
"$JS.API.STREAM.INFO." + stream,
"$JS.API.DIRECT.GET." + stream + ".>",
"$JS.API.CONSUMER.CREATE." + stream + ".>",
"$JS.API.CONSUMER.DELETE." + stream + ".>",
"$JS.FC." + stream + ".>",
}
}
// LastProvApplied is what a machine's provisioner client last said of one module's files, as it published it on
// ProvisionerAppliedSubject and ProvAppliedStream keeps it (design 57 P6): false when it has said nothing of it in
// the stream's week.
func LastProvApplied(js nats.JetStreamContext, node, module string) ([]byte, bool, error) {
msg, err := js.GetLastMsg(ProvAppliedStream, ProvisionerAppliedSubject(node, module))
if errors.Is(err, nats.ErrMsgNotFound) {
return nil, false, nil
}
if err != nil {
return nil, false, fmt.Errorf("what %s's provisioner client said of %s cannot be read: %w", node, module, err)
}
return msg.Data, true, nil
}
// ProvAppliedReports is the newest word of every machine's provisioner client on every module it said one of, kept by
// ProvAppliedStream, by subject (`prov.applied.<node>.<module>`).
func ProvAppliedReports(js nats.JetStreamContext) (map[string][]byte, error) {
info, err := js.StreamInfo(ProvAppliedStream, &nats.StreamInfoRequest{SubjectsFilter: ProvAppliedSubjects})
if err != nil {
return nil, fmt.Errorf("the provisioner clients' reports cannot be listed: %w", err)
}
out := map[string][]byte{}
for subject := range info.State.Subjects {
msg, err := js.GetLastMsg(ProvAppliedStream, subject)
if errors.Is(err, nats.ErrMsgNotFound) {
continue // aged out between the two reads
}
if err != nil {
return nil, fmt.Errorf("the provisioner client's report on %s cannot be read: %w", subject, err)
}
out[subject] = msg.Data
}
return out, nil
}
+73
View File
@@ -0,0 +1,73 @@
package buscompose
import "fmt"
// Bringing the bus's own objects into being, in the one order that works.
//
// **Asserted on every start rather than created once at genesis.** A stream somebody deleted, a mesh
// raised from a restored backup, or a bus whose data directory was replaced all have records and no
// objects — and a node whose consumer is missing hears nothing while everything else about it looks
// correct. Idempotence is the whole requirement, and the parts are already idempotent; this is the
// order they have to be asked in.
// Raiser is everything asserting the bus's objects needs of a connection to it.
type Raiser interface {
Asserter
Ensurer
}
// Raise asserts the mesh's streams, the controller's own consumers, and one consumer per node.
//
// **The order is not a preference.** A consumer on a stream that does not exist is refused, and the
// refusal names the stream rather than the order — so somebody reading it goes looking for a deleted
// stream instead of a reversed pair of lines. Nodes last, because the one a node reads lives on a
// stream the mesh's own set defines.
func Raise(r Raiser, nodes []string) error {
if err := AssertMeshStreams(r); err != nil {
return err
}
if err := AssertMeshConsumers(r); err != nil {
return err
}
if err := AssertNodeConsumers(r, nodes); err != nil {
return err
}
return nil
}
// RaiseSeats asserts one work queue per declared seat, and the worker of whoever holds it.
//
// Separate from Raise because it is answered by a different question: the mesh's own objects exist
// because the mesh does, and a seat's exist because a module declaring one was registered. Kept
// beside it so the order is visible — a holder's worker needs the seat's stream, and a seat's stream
// needs nothing.
func RaiseSeats(r Raiser, seats []DeclaredSeat, holders map[string]Holder) error {
for _, s := range SeatStreams(seats) {
if err := r.EnsureStream(s); err != nil {
return fmt.Errorf("asserting the work queue for %s: %w", s.Name, err)
}
}
for _, s := range seats {
h, held := holders[s.Name]
if !held {
// **The stream exists and the consumer does not, on purpose.** Work queues until a
// holder appears, so installing the module a week after something started sending to it
// flushes the backlog instead of having lost it.
continue
}
c, needed := HolderConsumerFor(h.Node, h.Module, s)
if !needed {
continue
}
if err := r.EnsureConsumer(c); err != nil {
return fmt.Errorf("asserting how %s on %s works %s: %w", h.Module, h.Node, s.Name, err)
}
}
return nil
}
// Holder is which module on which machine holds a seat.
type Holder struct {
Node string
Module string
}
+139
View File
@@ -0,0 +1,139 @@
package buscompose
import (
"fmt"
"sort"
"strings"
)
// Whether a mesh could move its bus, and what is missing if not.
//
// **Asked before anything moves, and answerable from records alone.** The rollout moves every node at
// once (novox/hq ADR 0116 step 5), so there is no partial state to inspect afterwards and no half to
// roll back: either the mesh was ready or it was not. That makes a readiness question the most
// valuable thing here — it costs nothing, it can be asked of a running mesh any number of times, and
// every answer is a thing somebody can go and fix.
//
// Deliberately pure. It is handed what the mesh knows and returns sentences; nothing here connects to
// anything, so it can be asked on a workstation about a mesh it has never reached.
// Readiness is what the mesh knows about its own ability to move.
type Readiness struct {
// TheBus is the address the mesh's own traffic would move to, empty when nothing names one.
TheBus string
// ServerStanding is whether a bus is reachable at that address, as somebody checked.
ServerStanding bool
// Holder is the node running the module that holds the bus seat, empty when nothing does.
Holder string
// AccountsComposed is whether that node has been sent the composed user list.
AccountsComposed bool
// Nodes is every machine the mesh knows.
Nodes []string
// Credentialled is which of them has a credential for the new bus.
Credentialled map[string]bool
// Modules is every assigned module, as `<node>/<module>`.
Modules []string
// ModuleCredentialled is which of those has one.
ModuleCredentialled map[string]bool
}
// NotReady is every reason this mesh cannot move its bus yet, in the order somebody would fix them.
//
// Empty means ready. **Each entry names one thing and what to do about it**, because a readiness check
// that says "not ready" is a check nobody can act on — and this is read at the point where the next
// step is irreversible.
func NotReady(r Readiness) []string {
var why []string
if strings.TrimSpace(r.TheBus) == "" {
why = append(why, "nothing names the bus to move to: set "+NATSVar+" on the control node "+
"to the address the new server answers on")
}
if !r.ServerStanding {
why = append(why, "no bus is answering at that address. Step 2 of the change raises it beside "+
"the one the mesh is on, carrying nothing — assign the module that holds "+
"mesh-broker and push the machine that runs it")
}
if r.Holder == "" {
why = append(why, "no machine holds mesh-broker, so nothing would compose the bus's user "+
"list. Assign the module that claims it")
} else if !r.AccountsComposed {
why = append(why, fmt.Sprintf(
"%s holds mesh-broker and has not been sent the composed user list, so the bus would "+
"refuse every connection. `push %s`", r.Holder, r.Holder))
}
// A node with no credential cannot come back after the move, and a node that cannot come back is
// a machine the mesh has lost until somebody goes to it.
var missing []string
for _, n := range r.Nodes {
if !r.Credentialled[n] {
missing = append(missing, n)
}
}
sort.Strings(missing)
if len(missing) > 0 {
why = append(why, fmt.Sprintf(
"%d machine(s) have no credential for the new bus and would not come back: %s. Each needs "+
"one minted before the move, not after — after, there is no bus to ask over",
len(missing), strings.Join(missing, ", ")))
}
// A module without one keeps running and stops being reachable, which is a smaller fault and still
// one somebody should choose rather than discover.
var quiet []string
for _, m := range r.Modules {
if !r.ModuleCredentialled[m] {
quiet = append(quiet, m)
}
}
sort.Strings(quiet)
if len(quiet) > 0 {
why = append(why, fmt.Sprintf(
"%d module(s) have no credential for the new bus: %s. Each keeps serving and stops "+
"answering tools and hearing events until it is issued one",
len(quiet), strings.Join(quiet, ", ")))
}
return why
}
// WhatMoves is what the rollout would do, in order, for somebody reading before they commit.
//
// **Written out rather than summarised.** This is the one step with nothing to inspect afterwards, so
// the last useful moment to disagree with it is while reading this.
func WhatMoves(r Readiness) []string {
out := []string{
fmt.Sprintf("compose the bus's user list and send it to %s", holderOr(r.Holder)),
fmt.Sprintf("move this control plane to %s, and confirm it is heard", busOr(r.TheBus)),
}
nodes := append([]string(nil), r.Nodes...)
sort.Strings(nodes)
for _, n := range nodes {
out = append(out, fmt.Sprintf("move %s, and confirm it reports", n))
}
if len(r.Modules) > 0 {
out = append(out, fmt.Sprintf("move %d module runtime(s), and confirm each answers",
len(r.Modules)))
}
// **The old broker goes, and it goes last** (novox/hq ADR 0131). AMQP is not a provision, so once
// every machine reports on the new bus nothing of the mesh is left speaking to it, and its module
// is unassigned. Said as a step so nobody reads the move as leaving a second bus behind.
out = append(out, "then unassign the old broker's module: AMQP is not a provision (ADR 0131), and "+
"once every machine reports on the new bus nothing of the mesh speaks to it")
return out
}
func holderOr(node string) string {
if node == "" {
return "whichever machine holds mesh-broker"
}
return node
}
func busOr(address string) string {
if address == "" {
return "the new bus"
}
return address
}
+38
View File
@@ -0,0 +1,38 @@
package buscompose
import (
"git.novox.be/novox/mesh-sdk/go/membership"
"git.novox.be/novox/mesh-sdk/go/process"
)
// The membership the mesh issues an assignment (novox/hq ADR 0160), its subject, its stream, the bare address and the
// pin are mesh-sdk's, public so that a module's program outside this Go module, the route proxy, reads them too
// (novox/hq ADR 0480 clause 4, issue 1633). This package keeps no second copy: what follows only names them where the
// controller already did.
type (
Membership = membership.Membership
Served = membership.Served
SeatServed = membership.SeatServed
StateIssued = membership.StateIssued
LogIssued = membership.LogIssued
Worker = membership.Worker
SeatTraffic = membership.SeatTraffic
KindHeld = membership.KindHeld
StreamIssued = membership.StreamIssued
HeardIssued = membership.HeardIssued
WaitIssued = membership.WaitIssued
)
// AssignmentsStream holds every assignment's membership, the newest per subject.
const AssignmentsStream = membership.AssignmentsStream
var (
// MembershipSubject is where an assignment's own membership is published.
MembershipSubject = membership.Subject
// CredentialIn reads the user and password out of a bus address, and the address without them.
CredentialIn = membership.CredentialIn
// BareAddress is a bus address with any credential stripped.
BareAddress = membership.BareAddress
// PinnedToFingerprint accepts exactly the certificate with this SHA-256 and no other.
PinnedToFingerprint = process.PinnedToFingerprint
)
+308
View File
@@ -0,0 +1,308 @@
package buscompose
import (
"fmt"
"sort"
)
// What a module may say and take on the seats it holds, uses and watches, beyond tools (novox/hq ADR
// 0259 §3, to-be 46 §10).
//
// Three rules are added to the ones a seat always had, each a subject whose last token names who may
// publish it, granted to that publisher alone — the way a node seat's event about a machine carries the
// machine (ADR 0219):
//
// - **A verb named by its caller** (`by-caller`). A user of the seat submits that accept, and hears that
// event, under its own module's name and no other: `accept.ask.<module>`, `event.decided.<module>`. The
// holder takes every caller's accept and says the event to any caller. So an ask's asker is a fact the
// server enforces, and a warrant reaches only the asker it is for.
// - **A kinded bench** (ADR 0234 §2). A holder claims one kind and takes its own kind's accepts, says its
// own kind's events and asks its own kind's proofs, and nothing of another kind; a user submits to any
// kind. Each kind has its own worker on the seat's queue, so a holder that is away keeps its work and
// holds up no other kind.
// - **A proof** (`proofs`): core request and reply on `mesh.seat.<seat>.proof.<verb>.<kind>`. No stream's
// subjects cover it, so what travels there — a code typed by the operator — is never persisted. A kinded
// holder asks with its own kind; the modules that watch the seat answer.
//
// And two reads: **a holder's records**, a bucket the seat names, read by each user under its own name only
// (`$KV.<bucket>.<module>.>`), so an asker reads the state of its own asks and no other asker's; and on a seat pulled by
// kind, **what its workers gave up on**, the copies DEAD_LETTERS keeps of the seat's stream (DeadLetterReads, issue 734).
// BuildAskMaxDeliver bounds the deliveries of a build seat's ask (issue 1907): mesh-sdk delivery.AskMaxDeliver, written
// here as the broker mirrors the catalogue rather than importing it.
const BuildAskMaxDeliver = 5
// KindedBenches are the seats that may be kinded (ADR 0234 §2): making another is a decision, recorded.
var KindedBenches = map[string]bool{"channel": true, "intake": true}
// WorkerName is the worker a seat's holders pull from: one for the seat, or one per kind on a kinded bench.
func WorkerName(seat, kind string) string {
if kind == "" {
return "SEAT_" + upperSnake(seat) + "_worker"
}
return "SEAT_" + upperSnake(seat) + "_" + upperSnake(kind) + "_worker"
}
func namesVerb(list []string, s string) bool {
for _, x := range list {
if x == s {
return true
}
}
return false
}
// SeatTrafficOf derives one module's seat traffic from the seats it holds, uses and watches. Only seats
// carrying one of the rules above are read: every other seat is composed as it always was.
func SeatTrafficOf(module string, holds, uses, watches []Seat) SeatTraffic {
var t SeatTraffic
for _, s := range holds {
if !s.isNewTraffic() {
continue
}
kind := ""
if s.Kinded {
kind = s.Kind
if kind == "" || !safeSubject.MatchString(kind) {
// A kinded claim without a usable kind is refused at registration; here it is granted
// nothing, which is the same answer at the last place it could be asked.
continue
}
}
if len(s.Accepts) > 0 && s.PulledByKind {
// One worker per accepted kind, which every holder pulls in the order of Accepts (ADR 0525 §4, ADR 0536 §4):
// `SEAT_<SEAT>_<KIND>_worker`, filtered to `accept.<kind>` (mesh-sdk delivery.AskWorker).
for _, a := range s.Accepts {
t.Workers = append(t.Workers, Worker{Stream: seatStreamName(s.Name), Consumer: WorkerName(s.Name, a),
Filter: "mesh.seat." + s.Name + ".accept." + a})
}
} else if len(s.Accepts) > 0 {
stream := seatStreamName(s.Name)
filter := "mesh.seat." + s.Name + ".accept.>"
if kind != "" {
filter = "mesh.seat." + s.Name + ".accept.*." + kind
}
t.Workers = append(t.Workers, Worker{Stream: stream, Consumer: WorkerName(s.Name, kind), Filter: filter})
}
for _, a := range s.Accepts {
switch {
case kind != "":
t.Subscribe = append(t.Subscribe, seatSubject(s, "accept", a+"."+kind))
case namesVerb(s.ByCaller, a):
t.Subscribe = append(t.Subscribe, seatSubject(s, "accept", a+".*"))
}
}
for _, e := range s.Emits {
switch {
case kind != "":
t.Publish = append(t.Publish, seatSubject(s, "event", e+"."+kind))
case namesVerb(s.ByCaller, e):
t.Publish = append(t.Publish, seatSubject(s, "event", e+".*"))
}
}
if kind != "" {
for _, v := range s.Proofs {
t.Publish = append(t.Publish, seatSubject(s, "proof", v+"."+kind))
}
}
}
for _, s := range uses {
if !s.isNewTraffic() {
continue
}
for _, a := range s.Accepts {
switch {
case namesVerb(s.ByCaller, a):
t.Publish = append(t.Publish, seatSubject(s, "accept", a+"."+module))
case s.Kinded && module == s.DeclaredBy:
// Work for a kind is put on its queue by the bench's own router, and by no other user.
t.Publish = append(t.Publish, seatSubject(s, "accept", a+".*"))
}
}
for _, e := range s.Emits {
if namesVerb(s.ByCaller, e) {
t.Subscribe = append(t.Subscribe, seatSubject(s, "event", e+"."+module))
}
}
for _, b := range s.Records {
if !safeSubject.MatchString(b) {
continue
}
t.Records = append(t.Records, "$JS.API.DIRECT.GET.KV_"+b+".$KV."+b+"."+module+".>")
}
if s.PulledByKind && len(s.Accepts) > 0 && safeSubject.MatchString(s.Name) {
t.Records = append(t.Records, DeadLetterReads(seatStreamName(s.Name))...)
}
}
for _, w := range watches {
if w.Kinded {
for _, e := range w.Emits {
t.Subscribe = append(t.Subscribe, seatSubject(w, "event", e+".*"))
}
if module == w.DeclaredBy {
// A code is answered by the bench's own router, and by no other watcher.
for _, v := range w.Proofs {
t.Answers = append(t.Answers, seatSubject(w, "proof", v+".*"))
}
}
}
}
t.Publish = unique(t.Publish)
t.Subscribe = unique(t.Subscribe)
t.Answers = unique(t.Answers)
t.Records = unique(t.Records)
sort.Slice(t.Workers, func(i, j int) bool { return t.Workers[i].Consumer < t.Workers[j].Consumer })
return t
}
// DeadLetterReads is what a user of a seat pulled by kind is granted to hear which of the asks it put the seat's workers
// gave up on (novox/hq issue 734): an ordered consumer on DEAD_LETTERS filtered to the copies the controller keeps of that
// seat's stream (DeadLetterSubject(stream, *)), pulled, and its deletion. A copy carries the ask's own message id
// (Mesh-Dead-Msg-Id), so the asker ends the step it waits for instead of waiting for ever; it reads the seat's dead
// letters and no other stream's, and changes none.
func DeadLetterReads(stream string) []string {
return []string{
"$JS.API.STREAM.INFO." + DeadLettersStream,
"$JS.API.CONSUMER.CREATE." + DeadLettersStream + ".*." + DeadLetterSubject(stream, ">"),
"$JS.API.CONSUMER.MSG.NEXT." + DeadLettersStream + ".*",
"$JS.API.CONSUMER.DELETE." + DeadLettersStream + ".*",
}
}
// seatTrafficGrants is the bus permissions seat traffic needs: the subjects themselves, and the JetStream API a
// worker is pulled and acknowledged through and a record is read through.
func seatTrafficGrants(t SeatTraffic) (pub, sub []string) {
pub = append(pub, t.Publish...)
sub = append(sub, t.Subscribe...)
sub = append(sub, t.Answers...)
for _, w := range t.Workers {
pub = append(pub,
"$JS.API.CONSUMER.INFO."+w.Stream+"."+w.Consumer,
"$JS.API.CONSUMER.MSG.NEXT."+w.Stream+"."+w.Consumer,
"$JS.ACK."+w.Stream+"."+w.Consumer+".>")
}
pub = append(pub, t.Records...)
return pub, sub
}
// isNewTraffic says whether a seat carries any of the rules above, so a seat that carries none is
// composed exactly as before them.
func (s Seat) isNewTraffic() bool {
return s.Kinded || len(s.ByCaller) > 0 || len(s.Proofs) > 0 || len(s.Records) > 0 || s.PulledByKind
}
// plainVerbs is a seat's accepts or emits with those the rules above compose taken out: a verb named by
// its caller and every verb of a kinded bench are composed by SeatTrafficOf and nowhere else.
func plainVerbs(s Seat, verbs []string) []string {
if s.Kinded {
return nil
}
var out []string
for _, v := range verbs {
if !namesVerb(s.ByCaller, v) {
out = append(out, v)
}
}
return out
}
// SeatTrafficObjects is the work queues and workers the seat traffic of every composed user implies
// (novox/hq ADR 0259 §3): a queue for each seat a holder takes work from, and each holder's worker on it —
// one per kind on a kinded bench, filtered to that kind, so the kinds never take each other's work. Only
// seats carrying the rules above; the mesh's own seats' queues are RaiseSeats'.
func SeatTrafficObjects(users []Principal) ([]Stream, []Consumer) {
streams := map[string]Stream{}
consumers := map[string]Consumer{}
add := func(module string, holds []Seat) {
for _, w := range SeatTrafficOf(module, holds, nil, nil).Workers {
seat, byKind := "", false
for _, s := range holds {
if seatStreamName(s.Name) == w.Stream {
seat, byKind = s.Name, s.PulledByKind
}
}
streams[w.Stream] = Stream{
Name: w.Stream,
Subjects: []string{"mesh.seat." + seat + ".accept.>"},
Retention: RetentionWorkQueue,
MaxAge: 7 * 24 * 60 * 60,
Why: "work submitted to the " + seat + " seat; its holders take it, each kind its own, and it queues while nobody does",
}
c := Consumer{
Name: w.Consumer,
Stream: w.Stream,
Filters: []string{w.Filter},
AckWaitSeconds: 60,
// No bound on redelivery: a channel away for a day keeps its work, offered again later and
// later by its holder's runtime (novox/hq ADR 0259; the correctness review of 2026-10-08).
MaxDeliver: 0,
Why: module + " holds " + seat + "; it pulls one ask at a time and acknowledges once it has " +
"recorded it, so a crash redelivers rather than loses",
}
if byKind {
// A build seat's ask is offered a bounded number of times (issue 1907): the holder that takes it past
// mesh-sdk delivery.AskRuns ends it failed saying it is poison, and the bound stops a run that kills
// every node it reaches even when that holder dies too.
c.MaxDeliver = BuildAskMaxDeliver
c.Why = module + " holds " + seat + "; each holder pulls an ask when it has a free slot, and an ask that " +
"no run ends is offered at most " + fmt.Sprint(BuildAskMaxDeliver) + " times, its step failed as poison"
}
consumers[w.Consumer] = c
}
}
for _, p := range users {
switch p.Kind {
case KindModule:
add(p.Module, p.Holds)
case KindNodeTools:
for _, d := range p.Carries {
add(d.Module, d.Holds)
}
}
}
var ss []Stream
for _, s := range streams {
ss = append(ss, s)
}
sort.Slice(ss, func(i, j int) bool { return ss[i].Name < ss[j].Name })
var cs []Consumer
for _, c := range consumers {
cs = append(cs, c)
}
sort.Slice(cs, func(i, j int) bool { return cs[i].Name < cs[j].Name })
return ss, cs
}
// trustedTraffic is why a module's seat traffic is the trusted holder's (novox/hq ADR 0259 §8), or "": it
// says a seat's event to one caller each (a warrant), or holds a kind of a kinded bench that proves its sender.
func trustedTraffic(d Declared) string {
for _, s := range d.Holds {
for _, e := range s.Emits {
if namesVerb(s.ByCaller, e) {
return "it says " + s.Name + "'s " + e + " to one caller each"
}
}
if s.Kinded && namesVerb(s.Capabilities, "verified-sender") {
return "it holds " + s.Name + " of kind " + s.Kind + ", which proves its sender"
}
}
return ""
}
// TrafficQueues is the work queue of every seat naming its caller or its kind that accepts work, held or not
// (novox/hq ADR 0259 §3): what is submitted before a holder is assigned waits for it.
func TrafficQueues(seats []Seat) []Stream {
var out []Stream
seen := map[string]bool{}
for _, s := range seats {
if len(s.Accepts) == 0 || !s.isNewTraffic() || seen[s.Name] {
continue
}
seen[s.Name] = true
out = append(out, Stream{Name: seatStreamName(s.Name), Subjects: []string{"mesh.seat." + s.Name + ".accept.>"},
Retention: RetentionWorkQueue, MaxAge: 7 * 24 * 60 * 60,
Why: "work submitted to the " + s.Name + " seat; its holders take it, each kind its own, and it queues while nobody does"})
}
sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
return out
}
+359
View File
@@ -0,0 +1,359 @@
package buscompose
import (
"fmt"
"git.novox.be/novox/mesh-sdk/go/announce"
"git.novox.be/novox/mesh-sdk/go/delivery"
"sort"
"strings"
"time"
)
// A module's state on the bus (novox/hq ADR 0201, design 32 §4, design 25 §3).
//
// A module names the state it keeps (`state`) and the state of others it reads (`reads`), and each
// is a key-value bucket: the server's own last-per-subject stream with direct reads, delete markers
// and watches, which is the state relationship the mesh already uses for declarations, opened to
// modules. The controller creates every bucket from the catalogue — from registration, like a
// seat's stream, so a reader may watch before the owner runs anywhere — and no module can.
//
// Pure, like everything else in this package that decides what the bus holds; jetstream.go is the
// part that asks a server.
// The mesh's caps on a bucket, the same for every module: a value is a piece of state, not a file,
// and a bucket that grew without bound would be one module filling the bus's disk for everyone.
const (
StateMaxValueBytes = 256 * 1024
StateMaxBytes = 64 * 1024 * 1024
)
// A Bucket is one module's declared state as the bus holds it.
type Bucket struct {
Module string
Name string
// History is how many values a key keeps; zero is one.
History int
// TTLSeconds is how long a value lives; zero is until replaced or deleted.
TTLSeconds int
// PerMachine says each machine's instance writes only the key named for its machine (novox/hq ADR
// 0260), and is granted that key and no other.
PerMachine bool
// KeyTTL says the bucket allows a time to live per key (novox/hq ADR 0433 §1): its owner writes each key with its
// own, and the server writes a delete marker, kept KeyTTLMarkerTTL, when it ends.
KeyTTL bool
}
// KeyTTLMarkerTTL is how long the marker an expired key leaves is kept: long enough for a watcher to hear it.
const KeyTTLMarkerTTL = time.Minute
// BucketName is the bucket a module's state lives in: the module and the local name joined by an
// underscore, which neither may contain, so two modules can never derive one bucket.
func BucketName(module, name string) string { return module + "_" + name }
// Bucket is this bucket's name on the bus.
func (b Bucket) Bucket() string { return BucketName(b.Module, b.Name) }
// Why is carried into the server's description of the bucket, so somebody reading the server's
// own state finds whose it is and why it is kept.
func (b Bucket) Why() string {
return fmt.Sprintf("%s's state %q (novox/hq ADR 0201): its current value per key, written by %s, "+
"read by whatever declares it reads it; kept when %s is unassigned, because it is data",
b.Module, b.Name, b.Module, b.Module)
}
// bucketOfRead is the bucket a read names, `<module>.<name>`, or false when it names none.
func bucketOfRead(read string) (string, bool) {
at := strings.LastIndex(read, ".")
if at <= 0 || at == len(read)-1 {
return "", false
}
module, name := read[:at], read[at+1:]
if !safeSubject.MatchString(module) || !safeSubject.MatchString(name) {
return "", false
}
return BucketName(module, name), true
}
// readGrants is what reaching a bucket's keys needs: binding to it, a direct get, an ordered consumer for listing and
// watching and the deletion of that consumer and its flow control. key is one key's subject tail, a pattern such as
// `one.>` for a prefix of keys, or empty for every key. The bucket is made with AllowDirect, so a get is the direct get
// and never the stream's message get.
func readGrants(bucket, key string) []string {
stream := "KV_" + bucket
out := []string{"$JS.API.STREAM.INFO." + stream}
if key == "" {
out = append(out,
"$JS.API.DIRECT.GET."+stream+".>",
"$JS.API.CONSUMER.CREATE."+stream+".>")
} else {
out = append(out,
"$JS.API.DIRECT.GET."+stream+".$KV."+bucket+"."+key,
"$JS.API.CONSUMER.CREATE."+stream+".*.$KV."+bucket+"."+key)
}
return append(out,
"$JS.API.CONSUMER.DELETE."+stream+".>",
"$JS.FC."+stream+".>")
}
// NodeKeysGrants is what a node's node-engine is granted of a bucket it writes by node, `<node>.>` and no other key
// (novox/hq ADR 0429 §1: each key one writer, enforced by credentials): the writes of those keys and what a get,
// create, update and watch of them need. The node name is checked as every subject token is.
func NodeKeysGrants(bucket, node string) ([]string, error) {
if !safeSubject.MatchString(node) {
return nil, fmt.Errorf("%q cannot be part of a subject: a permission is a subject pattern, and this would widen it", node)
}
return append([]string{"$KV." + bucket + "." + node + ".>"}, readGrants(bucket, node+".>")...), nil
}
// stateGrants is what a principal publishes to reach the state its modules keep and read: for every
// bucket, binding to it, reading a key directly, and an ordered consumer for listing and watching,
// created and deleted on the bucket's own stream, with its flow control answered; for a bucket an
// owner keeps, writing under the bucket's own subjects too.
//
// **Measured against a running server, 2026-10-04** (novox/hq research 024), and each one is there
// because leaving it out failed: without STREAM.INFO nothing binds; without DIRECT.GET nothing is
// read; without CONSUMER.CREATE no key is listed and nothing is watched; without CONSUMER.DELETE a
// watch cannot be stopped and lingers on the server. A write outside these is refused by the server
// — and reaches the writer as a timeout, not a refusal, which is why the runtime refuses first.
//
// **One key, where the state is per machine** (novox/hq ADR 0260). The owner's instance on a machine
// writes and reads the key named for that machine, and a holder granted a read for its machine reads
// that key: a direct get of the key's own subject, and a consumer whose filter is that subject — the
// server's create-with-filter form, which a client uses for a watch of one key and which the server
// holds to the filter it names. What cannot be narrowed by key: binding (STREAM.INFO), deleting a
// consumer (named at random by the client) and flow control; none of them reads or writes a value.
func stateGrants(a stateAccess) []string {
var out []string
read := func(bucket, key string) { out = append(out, readGrants(bucket, key)...) }
machineKey := ""
if safeSubject.MatchString(a.Node) {
machineKey = a.Node
}
perMachine := map[string]bool{}
for _, n := range a.PerMachine {
perMachine[n] = true
}
for _, name := range a.Keeps {
if !safeSubject.MatchString(name) {
continue
}
bucket := BucketName(a.Module, name)
if perMachine[name] {
if machineKey == "" {
continue // a machine with no usable name reaches none of it
}
read(bucket, machineKey)
out = append(out, "$KV."+bucket+"."+machineKey)
continue
}
read(bucket, "")
out = append(out, "$KV."+bucket+".>")
}
// **And it says what it keeps** (novox/hq ADR 0495 clause 2, issue 1790): a module keeping state announces it on its
// own event subject, from its code; refused, its readers are read as before.
if len(a.Keeps) > 0 && safeSubject.MatchString(a.Module) {
out = append(out, announce.Subject(a.Module))
}
for _, r := range a.Reads {
if bucket, ok := bucketOfRead(r); ok {
read(bucket, "")
}
}
for _, k := range a.KeyedReads {
if bucket, ok := bucketOfRead(k.Read); ok && safeSubject.MatchString(k.Key) {
read(bucket, k.Key)
}
}
return out
}
// StreamCreatedAdvisory is the subject the bus says a stream was created on (the server's own advisory), here for a
// bucket's stream KV_<bucket>.
func StreamCreatedAdvisory(bucket string) string {
return "$JS.EVENT.ADVISORY.STREAM.CREATED.KV_" + bucket
}
// stateCreatedHeard is what a principal subscribes to so that a bucket it may bind and that is not on the bus yet is
// bound when it appears (novox/hq issue 1790, 2026-10-26: since the grants part creates a manifest's buckets, ADR 0547,
// a module's delivery can start it before its new state exists): the bus's advisory that the bucket's stream was
// created, for each bucket stateGrants lets it bind, and no other. Read-only: the server publishes it in the mesh's own
// account, and hearing it changes nothing.
func stateCreatedHeard(a stateAccess) []string {
var out []string
seen := map[string]bool{}
add := func(bucket string) {
if !seen[bucket] {
seen[bucket] = true
out = append(out, StreamCreatedAdvisory(bucket))
}
}
perMachine := map[string]bool{}
for _, n := range a.PerMachine {
perMachine[n] = true
}
for _, name := range a.Keeps {
if !safeSubject.MatchString(name) || (perMachine[name] && !safeSubject.MatchString(a.Node)) {
continue
}
add(BucketName(a.Module, name))
}
for _, r := range a.Reads {
if bucket, ok := bucketOfRead(r); ok {
add(bucket)
}
}
for _, k := range a.KeyedReads {
if bucket, ok := bucketOfRead(k.Read); ok && safeSubject.MatchString(k.Key) {
add(bucket)
}
}
return out
}
// KeyedRead is one key of another module's state a principal reads, and no other key of it.
type KeyedRead struct {
Read string `json:"read"`
Key string `json:"key"`
}
// stateAccess is what one principal may do with state: the module's own buckets (some per machine),
// what it reads whole, and what it reads for its machine's key alone.
type stateAccess struct {
Module, Node string
Keeps, PerMachine []string
Reads []string
KeyedReads []KeyedRead
}
// perMachineNames is the local names of a module's buckets keyed by machine.
func perMachineNames(buckets []Bucket) []string {
var out []string
for _, b := range buckets {
if b.PerMachine {
out = append(out, b.Name)
}
}
return out
}
// stateIssuedFor is every bucket a module's code may reach on a machine, as its membership lists them.
func stateIssuedFor(d Declared, node string) []StateIssued {
var out []StateIssued
for _, b := range d.State {
issued := StateIssued{Name: b.Name, Bucket: BucketName(d.Module, b.Name), Writes: true}
if b.PerMachine {
issued.Key = node
}
out = append(out, issued)
}
for _, r := range d.Reads {
if bucket, ok := bucketOfRead(r); ok {
out = append(out, StateIssued{Name: r, Bucket: bucket})
}
}
for _, k := range d.KeyedReads {
if bucket, ok := bucketOfRead(k.Read); ok {
out = append(out, StateIssued{Name: k.Read, Bucket: bucket, Key: k.Key})
}
}
return out
}
// stateNames is the local names of a module's own buckets.
func stateNames(buckets []Bucket) []string {
out := make([]string, 0, len(buckets))
for _, b := range buckets {
out = append(out, b.Name)
}
return out
}
// A BucketAsserter is the part of a JetStream connection bucket assertion needs.
type BucketAsserter interface {
// EnsureBucket creates the bucket if absent and brings its options to match if present, never
// discarding what it holds.
EnsureBucket(b Bucket) error
// BucketNames is every key-value bucket on the server.
BucketNames() ([]string, error)
}
// RaiseBuckets asserts every declared bucket and answers the buckets on the server that nothing
// declares any more.
//
// **Those are reported, never removed** (novox/hq ADR 0201, ADR 0030): what a module stored is
// data, and a manifest edited, a module renamed or a catalogue entry dropped is an ordinary day's
// work that must not take data with it. Removing one is a person's act.
func RaiseBuckets(a BucketAsserter, buckets []Bucket) (undeclared []string, err error) {
sorted := append([]Bucket(nil), buckets...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Bucket() < sorted[j].Bucket() })
declared := map[string]bool{}
for _, b := range sorted {
if err := a.EnsureBucket(b); err != nil {
return nil, fmt.Errorf("asserting %s's state %q: %w", b.Module, b.Name, err)
}
declared[b.Bucket()] = true
}
names, err := a.BucketNames()
if err != nil {
return nil, fmt.Errorf("listing the bus's state: %w", err)
}
for _, n := range names {
// A seat's cancelled set is the mesh's own (novox/hq ADR 0219), not a module's state; so are
// the controller's own buckets (novox/hq to-be 45 §1).
if !declared[n] && !IsCancelledSet(n) && !IsControllerBucket(n) {
undeclared = append(undeclared, n)
}
}
sort.Strings(undeclared)
return undeclared, nil
}
// ResolvedBucket is mesh-config's state of resolved values, keyed `<node>.<module>.<key>`.
const ResolvedBucket = "mesh-config_resolved"
// NodeComposeReads is what a node's agent reads to compose its own modules (novox/hq ADR 0547 §2, ADR 0567 §3, §9,
// issue 1126): every key of the seat mesh-placement's placement and candidates (both the assigner's states), of the
// indexer's modules and manifests and of the builder's builds, and of mesh-config's resolved values only its own node's,
// `<node>.>`. Every candidate key, not only `*.<node>`: a node answering its candidate composes with the whole
// proposal laid over (the seed places every node at once, and a binding crosses nodes), and nothing in a placement is
// secret. Reads only; the node is checked as every subject token is by the caller's principal.
func NodeComposeReads(node string) []string {
var out []string
for _, bucket := range []string{delivery.PlacementBucket, delivery.PlacementCandidatesBucket, delivery.ModulesBucket,
delivery.ManifestsBucket, delivery.BuilderBuildsBucket} {
out = append(out, readGrants(bucket, "")...)
}
return append(out, readGrants(ResolvedBucket, node+".>")...)
}
// NodeComposeAdvisories are the stream-created advisories of the seat mesh-placement's placement and candidates
// (ADR 0567 §9): what a node agent hears to bind each once it is made. Those two and no wider.
func NodeComposeAdvisories() []string {
return []string{StreamCreatedAdvisory(delivery.PlacementBucket), StreamCreatedAdvisory(delivery.PlacementCandidatesBucket)}
}
// ServedReadGrants is the read of each key of mesh-config's resolved values named, `<node>.<module>.<key>`
// (Principal.ServedReads, issue 1126): a get and a watch of that key alone. Each part is checked as every subject token
// is, so a key cannot widen the grant.
func ServedReadGrants(keys []string) ([]string, error) {
var out []string
seen := map[string]bool{}
for _, k := range keys {
parts := strings.Split(k, ".")
if len(parts) != 3 {
return nil, fmt.Errorf("%q is not <node>.<module>.<key>", k)
}
for _, part := range parts {
if !safeSubject.MatchString(part) {
return nil, fmt.Errorf("%q cannot be part of a subject: a permission is a subject pattern, and this would widen it", part)
}
}
for _, g := range readGrants(ResolvedBucket, k) {
if !seen[g] {
seen[g] = true
out = append(out, g)
}
}
}
return out, nil
}
+612
View File
@@ -0,0 +1,612 @@
package buscompose
import (
"fmt"
"sort"
"strings"
)
// The mesh's own streams.
//
// **These and no more** (novox/hq ADR 0116 task 1.4, as revised by ADR 0118; the two that keep what a
// consumer gave up on added for issue 330). An earlier
// reading had the controller create *every* stream at genesis, from a fixed set. That is only the
// mesh's own half: a seat's streams are created when the module declaring it is registered, and a
// module's durable consumers when it is assigned — neither of which has happened at genesis. What
// is here is the foundation, which exists before any module does.
//
// The controller is the only writer of stream definitions (design 25 §3). A module declares
// nothing about them and cannot reach the JetStream API to make one.
// Retention is how a stream decides what to keep, which is the whole of what distinguishes the
// mesh's four relationships on the wire (design 29 §4).
type Retention string
const (
// RetentionWorkQueue: a message is removed once a consumer acknowledges it. Exactly one
// worker does the work, and a worker that dies has its message redelivered.
RetentionWorkQueue Retention = "workqueue"
// RetentionLastPerSubject: only the newest message on each subject survives. This is the
// state shape — a node that was away gets exactly the current declaration and nothing older.
RetentionLastPerSubject Retention = "last_per_subject"
// RetentionLimits: kept until it ages or the stream fills. Events, where a subscriber that
// was down catches up and nobody is obliged to act.
RetentionLimits Retention = "limits"
)
// A Stream is one of the mesh's own, as the controller asserts it.
type Stream struct {
Name string
Subjects []string
Retention Retention
// MaxAge in seconds, zero for unbounded. Per stream — JetStream has no per-subject age,
// which is why differing retention between modules would mean a stream each.
MaxAge int
// MaxMsgsPerSubject caps each subject independently, so one noisy emitter cannot push
// another's events out of a shared stream. Verified: with a cap of 3, ten messages on one
// subject and one on another leave four in the stream, not three.
MaxMsgsPerSubject int
// Why is carried into the assertion so an operator reading the server's own state finds the
// reason there, rather than only in a repository they may not have.
Why string
// Direct lets a client read a subject's last message without a consumer, which is how a
// runtime reads its own membership with no JetStream API beyond one request (ADR 0160).
Direct bool
// MaxBytes bounds the stream's size, zero for unbounded. With DiscardNew a full stream refuses
// what comes next rather than dropping what it holds: the publisher is told, and says so.
MaxBytes int64
DiscardNew bool
// DuplicatesSeconds is the window in which a message id published twice is kept once; zero for the
// server's default (two minutes).
DuplicatesSeconds int
// AllowMsgTTL lets a message carry a time to live of its own (`Nats-TTL`), for what must not outlive the
// stream it came from (novox/hq ADR 0376 §3, §5).
AllowMsgTTL bool
// Restricted marks a module's stream restricted by a grant (novox/hq ADR 0363 §4), in the stream's own
// metadata, so whoever keeps one of its messages reads it from the bus (ADR 0376 §3).
Restricted bool
// WorkQueue marks a module stream declared retention: work-queue (novox/hq ADR 0436 §6), in the
// stream's own metadata, so a dead letter kept from it is read as drop-only from the bus, by the
// declaration and not by the bus's work-queue retention alone — which SEAT_* and other mesh-internal
// streams share without being one of ADR 0436's, and whose dead letters are still delivered again
// (ADR 0264 §4).
WorkQueue bool
}
// RestrictedMetadata is the key a restricted stream's metadata says it under.
const RestrictedMetadata = "mesh-restricted"
// WorkQueueMetadata is the key a module stream declared retention: work-queue (novox/hq ADR 0436 §6) says
// it under: never set by a mesh-internal work-queue stream (a seat's, a check's), only by Stream.WorkQueue.
const WorkQueueMetadata = "mesh-work-queue"
// What a durable consumer gave up on is kept (novox/hq issue 330, design 25 §3).
//
// **The server says it and keeps it; the controller copies it.** A consumer that handed a message over
// as often as it may stops offering it and publishes `$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES` with
// the stream and the message's sequence. DeadLetterNoticesStream captures those advisories as the
// server publishes them, so one said while no controller listens is still there when one starts. The
// controller's consumer on it fetches the given-up message by its sequence, while the source stream
// still holds it, and keeps a copy in DeadLettersStream under DeadLetterSubject, with the consumer,
// the subject, how often it was handed over and when it was given up. It stays there until a person
// delivers it again or drops it, with why; a condition is open for as long as it does.
const (
DeadLetterNoticesStream = "DEAD_LETTER_NOTICES"
DeadLettersStream = "DEAD_LETTERS"
// MaxDeliveriesAdvisories is the subject the server says a given-up message on.
MaxDeliveriesAdvisories = "$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>"
deadLetterPrefix = "mesh.events.dead."
againPrefix = "mesh.again."
// DeadLettersBytes bounds the kept copies. Full, the stream refuses the next copy, which is said
// as the consumer's condition; it never drops one it holds.
DeadLettersBytes = 256 << 20
)
// DeadLetterSubject is where a message one consumer gave up on is kept: `mesh.events.dead.<stream>.<consumer>`.
func DeadLetterSubject(stream, consumer string) string {
return deadLetterPrefix + stream + "." + consumer
}
// DeadLetterOf is the stream and consumer a kept message's subject names; false for any other subject.
func DeadLetterOf(subject string) (stream, consumer string, ok bool) {
rest, found := strings.CutPrefix(subject, deadLetterPrefix)
if !found {
return "", "", false
}
stream, consumer, ok = strings.Cut(rest, ".")
return stream, consumer, ok && stream != "" && consumer != "" && !strings.Contains(consumer, ".")
}
// AgainSubject is where an event given up on is delivered again to the one consumer that gave it up,
// and to nobody else: `mesh.again.<consumer>.` and the original subject without its `mesh.`. Every
// consumer on EVENTS filters its own (AgainFilter), and a module's runtime reads the event's key from
// the tokens around `.event.`, so the handler sees the same key it saw the first time.
func AgainSubject(consumer, original string) string {
return againPrefix + consumer + "." + strings.TrimPrefix(original, "mesh.")
}
// AgainFilter is the one consumer's share of the subjects events are delivered again on.
func AgainFilter(consumer string) string { return againPrefix + consumer + ".>" }
// OriginalOfAgain is the subject an event delivered again was first published on; false for a subject
// that is not one delivered again.
func OriginalOfAgain(subject string) (string, bool) {
rest, found := strings.CutPrefix(subject, againPrefix)
if !found {
return "", false
}
_, original, ok := strings.Cut(rest, ".")
if !ok || original == "" {
return "", false
}
return "mesh." + original, true
}
// MeshStreams is the foundation set, in the order a person reads it.
//
// **CONTROL names its subjects rather than taking `mesh.control.>`**, because heartbeats live
// under that prefix and must not be persisted: a lost heartbeat is the next heartbeat, and a
// stream of them is a stream of the least valuable messages the mesh sends, competing for the
// same retention as the ones that matter.
//
// **EVENTS filters on the `event` token**, which is the reason that token exists. A module's
// namespace carries both its events and its tool calls; a filter of `mesh.mod.*.>` would persist
// every tool invocation in the mesh, and a tool call must never be persisted (design 25 §3 keeps
// tools on core NATS, where a lost call is a timeout the caller already handles).
func MeshStreams() []Stream {
return []Stream{
{
Name: "CONTROL",
// A build's outcome is no longer here: it is the build-machine seat's own event, so one
// publish reaches whoever asked, the controller and the catalogue (novox/hq ADR 0121).
Subjects: []string{"mesh.control.*.report", "mesh.control.enrol"},
Retention: RetentionWorkQueue,
Why: "the store-window guarantee (ADR 0083): the controller naks with a delay while its " +
"store is away and the message is redelivered; nothing is dropped",
},
{
Name: "NODES",
Subjects: []string{"mesh.node.*.declare"},
Retention: RetentionLastPerSubject,
Why: "one declaration per node, always the newest; a node that sees sequence n refuses " +
"n-1 by construction (issue 107)",
},
{
Name: AssignmentsStream,
Subjects: []string{"mesh.assignment.*.*"},
Retention: RetentionLastPerSubject,
Direct: true,
Why: "one membership per assignment, always the newest: what the mesh issued this module " +
"on this machine to serve and to reach (ADR 0160); read directly by the runtime it is for",
},
{
Name: EventsStream,
// A seat's own events ride here too: they are 1:many like any event, and the
// `event` token keeps them clear of both the seat's work queue (`accept`) and its
// tools (`tool`), which must not be persisted.
// And an event given up on, delivered again to the one consumer that gave it up
// (novox/hq issue 330): under `mesh.again.<consumer>.`, which only that consumer filters.
Subjects: []string{"mesh.mod.*.event.>", "mesh.seat.*.event.>", againPrefix + ">"},
Retention: RetentionLimits,
MaxAge: 7 * 24 * 60 * 60,
MaxMsgsPerSubject: 10000,
// A restricted dead letter delivered again keeps what is left of its stream's age (ADR 0376 §5).
AllowMsgTTL: true,
Why: "a subscriber that was down catches up; tool traffic under the same prefix is " +
"excluded by the event token; per-subject caps keep a noisy emitter from " +
"evicting a quiet one without splitting the stream",
},
{
Name: DeadLetterNoticesStream,
Subjects: []string{MaxDeliveriesAdvisories},
Retention: RetentionWorkQueue,
MaxAge: 7 * 24 * 60 * 60,
Why: "the server's word that a consumer gave up on a message, kept until the controller has " +
"copied the message into DEAD_LETTERS (novox/hq issue 330); a week, the longest the source " +
"streams keep what they are about",
},
{
Name: DeadLettersStream,
Subjects: []string{deadLetterPrefix + ">"},
Retention: RetentionLimits,
MaxBytes: DeadLettersBytes,
DiscardNew: true,
DuplicatesSeconds: 24 * 60 * 60,
AllowMsgTTL: true,
Why: "every message a consumer gave up on, with its consumer, subject, deliveries and when, kept " +
"until a person delivers it again or drops it with why (novox/hq issue 330); no age but a restricted " +
"stream's message, which ages out with its stream (ADR 0376), and full it refuses the next copy " +
"rather than drop one it holds",
},
{
Name: ProvAppliedStream,
Subjects: []string{ProvAppliedSubjects},
Retention: RetentionLastPerSubject,
Direct: true,
MaxAge: 7 * 24 * 60 * 60,
Why: "each machine's provisioner client's newest word on each module's files, applied, waiting or " +
"failed with the reason (novox/hq ADR 0349 §4): what a walk's end and a gate wait for (ADR 0362, " +
"ADR 0349 §15), read by subject; a week, as the events are",
},
}
}
// ProvAppliedStream keeps what each machine's provisioner client last said of each module's files, newest per
// subject (novox/hq ADR 0349 §4, ADR 0362, design 57 P6): `prov.applied.<node>.<module>` lies outside `mesh.*`, so
// no other stream captures it, and a report said while no controller listened is still there to be read.
const ProvAppliedStream = "PROV_APPLIED"
// ProvAppliedSubjects is every machine's client's reports: `prov.applied.<node>.<module>`, a module's name dots and
// all (ProvisionerAppliedSubject).
const ProvAppliedSubjects = "prov.applied.>"
// eventsTheControllerFollows are the events the controller's events consumer takes: all it follows but the forge's
// merges, which the merges consumer takes (MergesConsumerName).
func eventsTheControllerFollows() []string {
var out []string
for _, subject := range ControllerFollows {
if subject != ControllerFollows[3] {
out = append(out, subject)
}
}
return append(out, ControllerHears...)
}
// DeliverSubjectFor is where a push consumer's messages land.
//
// **Per consumer, which means per stream as well as per name** (novox/hq 04-ISSUES/146). A push
// consumer delivers onto an ordinary subject, and everything subscribed to that subject gets a
// copy. The controller holds a consumer called `controller` on CONTROL and another called
// `controller` on EVENTS; while both were given `_DELIVER.controller`, the one process holding
// both subscriptions acted on every message twice — a joining machine was enrolled twice from one
// request, and the second enrolment minted a credential that replaced the one the machine had just
// been handed. Every report and every followed event doubled the same way, silently: nothing is
// redelivered, no count is wrong, the work simply happens twice.
//
// The stream belongs in it because the pair is what identifies a consumer — the server scopes a
// durable's name to its stream, and this subject was the one place that scoping was dropped. It
// stays inside what a controller may already subscribe (`_DELIVER.controller.>`).
func DeliverSubjectFor(c Consumer) string {
if c.DeliverSubject != "" {
return c.DeliverSubject
}
return "_DELIVER." + c.Name + "." + c.Stream
}
// An Asserter is the part of a JetStream connection stream assertion needs. Narrow on purpose: it
// keeps this testable without a server, and keeps the client library out of everything that only
// wants to know what the streams are.
type Asserter interface {
// EnsureStream creates the stream if absent and updates it to match if present. It must be
// idempotent: the controller asserts on every start, not only at genesis.
EnsureStream(s Stream) error
}
// AssertMeshStreams brings the foundation set into being, in order, and says which one failed
// rather than that something did.
//
// Asserted on every start rather than created once at genesis, because a stream that was deleted,
// or a mesh raised from a restored backup, must converge rather than run without the guarantee
// its messages assume. Idempotence is the whole requirement.
func AssertMeshStreams(a Asserter) error {
for _, s := range MeshStreams() {
if err := a.EnsureStream(s); err != nil {
return fmt.Errorf("asserting stream %s: %w", s.Name, err)
}
}
return nil
}
// Overlaps reports subject filters claimed by more than one stream.
//
// **Corrected against the server**: an earlier version of this comment said NATS accepts
// overlapping streams and stores the message twice. It does not — it refuses the second stream
// with "subjects overlap with an existing stream" (verified against nats-server 2.10). The check
// still earns its place, for a different reason: the server's refusal arrives when the controller
// is applying, naming one stream, at a moment when the mesh is half-configured. This one arrives
// where the set is written, names both, and cannot reach a running mesh.
//
// It also decides a design question. Because overlap is refused rather than merged, a shared
// EVENTS stream and a per-module stream cannot coexist — the module's would be refused — so
// "one stream for most, its own for a module that wants different retention" is not an option
// the server allows. It is all of one or all of the other.
func Overlaps() []string {
seen := map[string]string{}
var clashes []string
for _, s := range MeshStreams() {
for _, subject := range s.Subjects {
if first, ok := seen[subject]; ok {
clashes = append(clashes, fmt.Sprintf("%s and %s both claim %s", first, s.Name, subject))
continue
}
seen[subject] = s.Name
}
}
sort.Strings(clashes)
return clashes
}
// The mesh's own consumers.
//
// A seat's streams and a module's consumers are derived from declarations (derived.go). These two
// are not: **the controller is not a module and files no manifest**, so its authority and its
// subscriptions cannot come from a declaration that does not exist. They are named here, where the
// mesh's own streams are named, and narrowly — a controller subscribing `mesh.mod.*.event.>` would
// hear every event in the mesh, which it has no business doing and which would make its permission
// list stop explaining anything.
// ControllerName is the controller's durable consumer on each stream it reads, and the name its
// ack subject is derived from (nats.go: `$JS.ACK.<stream>.controller.>`).
const ControllerName = "controller"
// MergesConsumerName is the controller's durable consumer of the forge's merged event on EVENTS, apart from the
// events consumer (novox/hq ADR 0407 decision 3, issue 973). A merge is never dropped, so this consumer has no
// max-deliver, which would send a merge held five times to DEAD_LETTERS to wait for a person, and no healer
// resets it, which would drop the merges it holds. A consumer of its own because both of those are properties of
// a consumer, not of a subject.
const MergesConsumerName = "controller-merges"
// ControllerSeat is the role the control plane holds, and ControllerStates are the facts it states
// under it (novox/hq ADR 0134).
//
// **Written here as well as in `link`, and a test keeps them agreeing.** `link` imports `broker`, so
// `broker` cannot import `link`; a grant naming a subject the controller never publishes is authority
// nobody uses, and a controller publishing one the grant omits is refused at the moment it has
// something to say.
const ControllerSeat = "mesh-controller"
var ControllerStates = []string{"applied", "refused", "built-before",
// What is wrong, said as it changes (novox/hq to-be 45 §2): a condition raised, changed in
// severity, resolver or silence, and cleared. The operator-channel's holder and any other surface
// consume them; the controller tells nobody itself.
"condition-raised", "condition-changed", "condition-cleared",
// And the self-check's heartbeat, at the end of every run (to-be 45 §4, S10): watched from a
// machine that is not the control node, so the controller going quiet is itself said.
"doctor-heartbeat",
// And a value given by hand, replaced after its module's first good start (novox/hq ADR 0228).
"secret-replaced",
// And every act a healer takes on a condition (novox/hq to-be 45 §7, Phase 3): a repair the mesh
// made by itself is said like one a person made, never quietly.
"healer-acted",
// And a build put back after its gate failed on its first machine (novox/hq ADR 0236, to-be 45 §8).
"rolled-back",
// And a pull request's merge check, judged (novox/hq to-be 45 §9): what the forge's holder sets as
// the pull request's status, and anybody else may read.
"checked",
// (A walk kept in a new state and a tier sent to a machine are gone with the plans, novox/hq ADR 0539.)
// And a key of a record mirror written or removed (novox/hq ADR 0289, design 54 §3): the assignments and
// each registered module's declared config (design 57 F3).
"assignment-changed", "declared-config-changed", "predefined-changed",
// And a builder's key not taken in because the artifact store does not hold what it names (novox/hq issue 1937).
"artifacts-gone",
// And a bucket of the plans' state deleted at the controller's first start, and the declarations' stream once no node
// reads one (novox/hq ADR 0539 clauses 11 and 13).
"bucket-deleted"}
// BusAdvisories are what the bus server says about the mesh's own account that the controller
// reads (novox/hq to-be 45 §3, S9): a durable consumer that handed a message over as often as it
// may and gave up on it, and one that was deleted. Read-only: an advisory is the server's to
// publish, and the controller's subscription changes nothing on the bus.
var BusAdvisories = []string{
"$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>",
"$JS.EVENT.ADVISORY.CONSUMER.DELETED.>",
}
// ControllerFollows are the events the controller reacts to: the catalogue saying a module's
// current version moved, and a catalogue that has just started saying it may have missed builds.
//
// **Derived the same way a module's subscription is**, from the emitter and the bare local event
// name, rather than written out. They were written out while the catalogue still spelled its events
// as the old bus's routing keys, and the moment those were converted (novox/hq 04-ISSUES/127) a
// hard-coded pair became a controller listening to a subject nothing publishes — the same fault, from
// the other side. Deriving them means the conversion could not leave these behind.
var ControllerFollows = []string{
moduleEventSubject("mesh-catalog", "upgraded"),
moduleEventSubject("mesh-catalog", "catching-up"),
// A build's outcome, which is the build-machine role's own event now (ADR 0121) rather than a
// message on the control branch. Same three audiences, one publish: whoever asked, this, and the
// catalogue.
seatEventSubject("node-build-agent", "built"),
// The forge's merges: what moved a source, so the mesh builds what that source produces
// without anybody telling it (novox/hq 04-ISSUES/131). Appended, because the index is a name.
moduleEventSubject("gitea", "pull.merged"),
// The retired build role's outcome too, while the handover runs (novox/hq ADR 0190): the one
// build machine keeps answering on its seat until build-agent replaces it, and the outcome that
// registers build-agent itself comes from there. Appended, for the same reason as above; goes
// with the retired seat row.
seatEventSubject("mesh-build-machine", "built"),
// **Every provider's standing** (novox/hq ADR 0224): a consumer it has failed for minutes, and
// that consumer recovered. The one pattern on this list, and a narrow one — two named events,
// from whichever module provides — because the rule is about every provider, and a list of
// providers here would be a list somebody forgets to extend. On 2026-10-05 the identity provider
// failed every consumer for a day and only its journal said so (issue 179). Appended, because
// the index is a name.
moduleEventSubject("*", ProvisionerFailing),
moduleEventSubject("*", ProvisionerRecovered),
// **What becomes of a consumer the mesh stopped asking for** (novox/hq ADR 0230): retired, waiting
// for a person, re-enabled, deleted — a provider's third word, from whichever module provides.
// Appended, because the index is a name.
moduleEventSubject("*", ProvisionerRetirement),
// **A pull request's head, announced** (novox/hq to-be 45 §9): what the controller asks the build
// seat to check before it merges — every machine of the facts snapshot composed with the change.
// Appended, because the index is a name.
moduleEventSubject("gitea", "pull.updated"),
// **The operator's answers to what the controller asked** (novox/hq ADR 0259): the router's warrant, or
// the end of an ask without one, said to the controller alone under its own name. On the stream, so a
// controller that was away hears what was decided meanwhile. Appended, because the index is a name.
DecidedSubject,
}
// ControllerHears are events the controller's events consumer takes beside the ones it follows, and that need no
// subscribe grant of their own. **A push consumer delivers onto its deliver subject** (`_DELIVER.controller.>`, already
// granted), so the subject a consumer filters on is never one the controller subscribes: listing this one in
// ControllerFollows would add a subscribe grant the installer's carried user list, which lives in mesh-host, would
// have to carry too (TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose), for nothing the consumer uses.
//
// A merge mesh-delivery made, announced (novox/hq ADR 0420 §7, issue 1025): with the ask whose Approve it merged on and
// the heads that Approve covered, kept a week to read the warrant a moved module's registration needs.
var ControllerHears = []string{DeliveryMergedSubject}
// DeliveryMergedEvent is the event a module says when it merged a pull request.
const DeliveryMergedEvent = "merged"
// DeliveryMergedSubject is where the delivery's owner says it merged a pull request: one wildcard subject, the
// module that claims the delivery seat being whichever the catalogue says (novox/hq ADR 0437 §3). Who said it is
// read from the subject, and whether it claims the seat is the controller's to check.
var DeliveryMergedSubject = moduleEventSubject("*", DeliveryMergedEvent)
// DeliveryMergedSubjectOf is where one module says it merged a pull request.
func DeliveryMergedSubjectOf(module string) string {
return moduleEventSubject(module, DeliveryMergedEvent)
}
// AsksSeat is the seat an ask is made on and its warrant heard from (novox/hq ADR 0259): the router's.
const AsksSeat = "operator-channel"
// DecidedSubject is where the router says the controller's warrants: the seat's event named by the
// controller as its caller.
var DecidedSubject = seatEventSubject(AsksSeat, "decided."+ControllerSeat)
// The provider standing events, by their local names. Written here as well as in the catalogue
// (catalogue.ProvisionerEvents), which this package cannot import; a test keeps them agreeing.
const (
ProvisionerFailing = "provisioner.failing"
ProvisionerRecovered = "provisioner.recovered"
ProvisionerRetirement = "provisioner.retirement"
)
// moduleEventSubject is where one module's event lands. The same derivation PermissionsFor uses, so
// what the controller subscribes and what the emitter is permitted to publish cannot drift apart.
func moduleEventSubject(module, event string) string {
return "mesh.mod." + module + ".event." + event
}
// seatEventSubject is where a role's own event lands, derived the same way a holder's permission is.
func seatEventSubject(seat, verb string) string {
return "mesh.seat." + seat + ".event." + verb
}
// EventsStream holds every module's and every role's events, a build's log among them.
const EventsStream = "EVENTS"
// MeshConsumers is what the controller consumes, in the order a person reads it.
//
// **Unlimited redelivery on CONTROL, deliberately.** The store window's bound is the controller's,
// not the server's (window.go): a message is held with a nak-and-delay until the controller either
// takes it or gives up and says so. A max-deliver here would give up on a push that was being
// held through a store restart — the exact message the stream exists to protect — some minutes
// before the controller had finished deciding about it.
func MeshConsumers() []Consumer {
return ControllerConsumers()
}
// ControllerConsumers are the consumers the controller's own event loop takes from: MeshConsumers without the build
// seat's check workers, which the builders take from and the loop never does (issue 1063). A consumer added to the
// controller is counted here without a change elsewhere.
func ControllerConsumers() []Consumer {
return []Consumer{
{
Name: ControllerName,
Stream: "CONTROL",
Push: true,
AckWaitSeconds: 30,
Why: "the controller is the single consumer of what nodes say; explicit ack and no " +
"max-deliver, because the store window's bound is the controller's own",
},
// The forge's merges (novox/hq ADR 0407 decision 3, issue 973): never dropped, so no max-deliver and no
// reset; a merge the controller cannot record yet is held with a growing delay until it can. **Asserted before
// the events consumer**, which stops filtering merges: the order of this list is the order of assertion.
{
Name: MergesConsumerName,
Stream: "EVENTS",
// Within what the controller may subscribe already (`_DELIVER.controller.>`).
DeliverSubject: "_DELIVER." + ControllerName + ".merges",
Filters: []string{ControllerFollows[3]},
Push: true,
AckWaitSeconds: 30,
MaxAckPending: 1,
FromNow: true,
TakesOverFrom: ControllerName,
Why: "the forge's merges, one at a time; no max-deliver and never reset, because a merge is never " +
"dropped and no person releases one",
},
{
Name: ControllerName,
Stream: "EVENTS",
Filters: append(eventsTheControllerFollows(), AgainFilter(ControllerName)),
Push: true,
AckWaitSeconds: 30,
MaxDeliver: 5,
// One at a time (novox/hq issue 175): acting on a merge builds for minutes, and an
// announcement handed over behind it must wait on the server, not time out on the
// client and come back to be acted on again.
MaxAckPending: 1,
FromNow: true,
// **The one consumer the mesh resets by itself** (healer H4, issue 248): it fell a week
// behind once and held every merge after it. What a reset drops is caught up elsewhere — a
// build's outcome from the build records a plan settles from (issue 214), a provider's failing
// word by the provider saying it again every quarter of an hour (ADR 0224). **Not the merges**:
// they are the merges consumer's below, which a reset leaves alone (ADR 0407 decision 3).
Resettable: "what it drops is caught up: build outcomes from the build records (issue 214), a " +
"provider's failing word said again (ADR 0224); the forge's merges are not its, they have a " +
"consumer of their own that no reset touches",
Why: "the events the mesh's own controller reacts to, one at a time; after " +
"max-deliver it gives the event up, and the controller keeps it in DEAD_LETTERS until " +
"a person delivers it again or drops it",
},
// What the server said a consumer gave up on (novox/hq issue 330): copied into DEAD_LETTERS and
// acknowledged. No max-deliver: a notice the controller could not copy is offered again, and said.
{
Name: ControllerName,
Stream: DeadLetterNoticesStream,
Push: true,
AckWaitSeconds: 30,
Why: "the controller copies each message a consumer gave up on into DEAD_LETTERS; no max-deliver, " +
"because a notice it gave up on would lose the message it is about",
},
}
}
// NoticesConsumer is the controller's consumer on DEAD_LETTER_NOTICES (novox/hq issue 330).
func NoticesConsumer() Consumer {
for _, c := range MeshConsumers() {
if c.Stream == DeadLetterNoticesStream {
return c
}
}
panic("the mesh's consumers carry none on " + DeadLetterNoticesStream)
}
// AssertServingConsumers are the controller's own consumers its serving cannot go without: all but the
// one on DEAD_LETTER_NOTICES, which the keeper of dead letters asserts and retries by itself, so a fault
// there never stops the controller serving (novox/hq issue 330).
func AssertServingConsumers(e Ensurer) error {
for _, c := range MeshConsumers() {
if c.Stream == DeadLetterNoticesStream {
continue
}
if err := e.EnsureConsumer(c); err != nil {
return fmt.Errorf("asserting consumer %s on %s: %w", c.Name, c.Stream, err)
}
}
return nil
}
// Ensurer is the part of a JetStream connection consumer assertion needs, narrow for the reason
// Asserter is.
type Ensurer interface {
EnsureConsumer(c Consumer) error
}
// AssertMeshConsumers brings the controller's own consumers into being, and says which one failed.
//
// After the streams, necessarily: a consumer on a stream that does not exist is refused, and the
// refusal names the stream rather than the order.
func AssertMeshConsumers(e Ensurer) error {
for _, c := range MeshConsumers() {
if err := e.EnsureConsumer(c); err != nil {
return fmt.Errorf("asserting consumer %s on %s: %w", c.Name, c.Stream, err)
}
}
return nil
}
+78
View File
@@ -0,0 +1,78 @@
# Composed by the mesh controller. Do not edit: the next composition overwrites it.
# Accounts and permissions are derived from what each module declares and nothing
# else (novox/hq ADR 0043, design 29 §2).
port: 4222
http: 127.0.0.1:8222
tls {
cert_file: "/tls/tls.crt"
key_file: "/tls/tls.key"
ca_file: "/tls/ca.crt"
}
jetstream {
store_dir: "/data"
}
# The mesh's users, composed by the controller. Do not edit: the next
# composition overwrites it. Permissions are derived from what each module
# declares and nothing else (novox/hq ADR 0043, design 29 §2).
accounts {
MESH {
jetstream: enabled
imports = [
{ stream: { account: "$SYS", subject: "$SYS.ACCOUNT.MESH.CONNECT" }, to: "mesh.bus.connect" }
{ stream: { account: "$SYS", subject: "$SYS.ACCOUNT.MESH.DISCONNECT" }, to: "mesh.bus.disconnect" }
{ service: { account: "$SYS", subject: "$SYS.REQ.ACCOUNT.MESH.CONNZ" }, to: "mesh.bus.connz" }
]
users = [
{ user: "controller", password: "$2a$11$cccccccccccccccccccccc", permissions: {
publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.DEAD_LETTER_NOTICES.controller.>", "$JS.ACK.EVENTS.controller-merges.>", "$JS.ACK.EVENTS.controller.>", "$JS.ACK.KV_mesh-delivery-builder_builds.mesh-controller-take-in.>", "$JS.ACK.KV_mesh-delivery-indexer_modules.mesh-controller-take-in.>", "$JS.API.>", "$KV.mesh-controller_asked.>", "$KV.mesh-controller_assignments.>", "$KV.mesh-controller_builds.>", "$KV.mesh-controller_calls.>", "$KV.mesh-controller_composed.>", "$KV.mesh-controller_condition-history.>", "$KV.mesh-controller_conditions.>", "$KV.mesh-controller_declared-config.>", "$KV.mesh-controller_declared-packages.>", "$KV.mesh-controller_delivery-merges.>", "$KV.mesh-controller_grants.>", "$KV.mesh-controller_hand-acts.>", "$KV.mesh-controller_lease.>", "$KV.mesh-controller_predefined.>", "$KV.mesh-controller_rollouts.>", "$KV.mesh-controller_states-announced.>", "$SRV.INFO", "_INBOX.enrol.>", "mesh.again.>", "mesh.assignment.>", "mesh.events.dead.>", "mesh.mod.*.tool.>", "mesh.mod.mesh-controller.event.states", "mesh.node.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.artifacts-gone", "mesh.seat.mesh-controller.event.assignment-changed", "mesh.seat.mesh-controller.event.bucket-deleted", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.checked", "mesh.seat.mesh-controller.event.condition-changed", "mesh.seat.mesh-controller.event.condition-cleared", "mesh.seat.mesh-controller.event.condition-raised", "mesh.seat.mesh-controller.event.declared-config-changed", "mesh.seat.mesh-controller.event.doctor-heartbeat", "mesh.seat.mesh-controller.event.healer-acted", "mesh.seat.mesh-controller.event.predefined-changed", "mesh.seat.mesh-controller.event.refused", "mesh.seat.mesh-controller.event.rolled-back", "mesh.seat.mesh-controller.event.secret-replaced", "mesh.seat.mesh-controller.tool.plans", "mesh.seat.mesh-controller.tool.push", "mesh.seat.mesh-controller.tool.upgrade", "mesh.seat.mesh-delivery.tool.close", "mesh.seat.mesh-delivery.tool.groups", "mesh.seat.mesh-delivery.tool.release", "mesh.seat.mesh-delivery.tool.stalled", "mesh.seat.mesh-delivery.tool.stop", "mesh.seat.mesh-delivery.tool.times", "mesh.seat.node-backup.tool.backed-up.*", "mesh.seat.node-backup.tool.now.*", "mesh.seat.node-intrusion-prevention.tool.banned.*", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-service-manager.tool.restart.*", "mesh.seat.operator-channel.tool.tell"] }
subscribe: { allow: ["$JS.API.>", "$JS.EVENT.ADVISORY.CONSUMER.DELETED.>", "$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>", "$SRV.INFO", "$SRV.INFO.mesh-controller", "$SRV.INFO.mesh-controller.>", "$SRV.PING", "$SRV.PING.mesh-controller", "$SRV.PING.mesh-controller.>", "$SRV.STATS", "$SRV.STATS.mesh-controller", "$SRV.STATS.mesh-controller.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.*.event.provisioner.failing", "mesh.mod.*.event.provisioner.recovered", "mesh.mod.*.event.provisioner.retirement", "mesh.mod.gitea.event.pull.merged", "mesh.mod.gitea.event.pull.updated", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built", "mesh.seat.mesh-controller.tool.>", "mesh.seat.node-build-agent.event.built", "mesh.seat.operator-channel.event.decided.mesh-controller"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "enrol.one", password: "$2a$11$eeeeeeeeeeeeeeeeeeeeee", permissions: {
publish: { allow: ["mesh.control.enrol"] }
subscribe: { allow: ["_INBOX.enrol.one.>"] }
} }
{ user: "node.one", password: "$2a$11$nnnnnnnnnnnnnnnnnnnnnn", permissions: {
publish: { allow: ["$JS.ACK.NODES.one.>", "$JS.API.CONSUMER.CREATE.KV_composed-one.>", "$JS.API.CONSUMER.CREATE.KV_mesh-delivery-assigner_wants.*.$KV.mesh-delivery-assigner_wants.want.one.>", "$JS.API.CONSUMER.CREATE.KV_node-engine_have.*.$KV.node-engine_have.one.>", "$JS.API.CONSUMER.CREATE.OBJ_declaration-files-one.*.$O.declaration-files-one.C.*", "$JS.API.CONSUMER.DELETE.KV_composed-one.>", "$JS.API.CONSUMER.DELETE.KV_mesh-delivery-assigner_wants.>", "$JS.API.CONSUMER.DELETE.KV_node-engine_have.>", "$JS.API.CONSUMER.DELETE.OBJ_declaration-files-one.*", "$JS.API.CONSUMER.INFO.NODES.one", "$JS.API.DIRECT.GET.KV_composed-one.>", "$JS.API.DIRECT.GET.KV_mesh-delivery-assigner_wants.$KV.mesh-delivery-assigner_wants.want.one.>", "$JS.API.DIRECT.GET.KV_node-engine_have.$KV.node-engine_have.one.>", "$JS.API.DIRECT.GET.OBJ_declaration-files-one.$O.declaration-files-one.M.*", "$JS.API.STREAM.INFO.KV_composed-one", "$JS.API.STREAM.INFO.KV_mesh-delivery-assigner_wants", "$JS.API.STREAM.INFO.KV_node-engine_have", "$JS.FC.KV_composed-one.>", "$JS.FC.KV_mesh-delivery-assigner_wants.>", "$JS.FC.KV_node-engine_have.>", "$JS.FC.OBJ_declaration-files-one.>", "$KV.node-engine_have.one.>", "$SRV.PING.node-tools.one", "mesh.control.one.>", "mesh.seat.node-backup.call.backed-up.one.node~one", "mesh.seat.node-backup.call.now.one.node~one", "mesh.seat.node-backup.tool.backed-up.one", "mesh.seat.node-backup.tool.now.one"] }
subscribe: { allow: ["_DELIVER.one", "_DELIVER.one.>", "_INBOX.node.one.>", "mesh.mod.restic.event.backed-up", "mesh.node.one.ask.access-act", "mesh.node.one.ask.hand-over", "mesh.node.one.ask.judge-directories", "mesh.node.one.ask.move-secret", "mesh.node.one.ask.report", "mesh.node.one.ask.setuid-search", "mesh.node.one.declare"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "one.mesh-delivery-watchman", password: "$2a$11$wwwwwwwwwwwwwwwwwwwwww", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.one_mesh-delivery-watchman.>", "$JS.API.CONSUMER.INFO.EVENTS.one_mesh-delivery-watchman", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_mesh-delivery-watchman", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.one.mesh-delivery-watchman", "mesh.bus.connz"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.mesh-delivery-watchman", "$SRV.INFO.mesh-delivery-watchman.>", "$SRV.PING", "$SRV.PING.mesh-delivery-watchman", "$SRV.PING.mesh-delivery-watchman.>", "$SRV.STATS", "$SRV.STATS.mesh-delivery-watchman", "$SRV.STATS.mesh-delivery-watchman.>", "_INBOX.one.mesh-delivery-watchman.>", "mesh.assignment.one.mesh-delivery-watchman", "mesh.bus.connect", "mesh.bus.disconnect", "mesh.mod.mesh-delivery-watchman.call.>", "mesh.mod.mesh-delivery-watchman.tool.>"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "one.nats", password: "$2a$11$bbbbbbbbbbbbbbbbbbbbbb", permissions: {
publish: { allow: ["$JS.API.STREAM.INFO.*", "$JS.API.STREAM.NAMES", "$JS.API.STREAM.SNAPSHOT.*", "$JS.SNAPSHOT.ACK.>"] }
subscribe: { allow: ["_INBOX.one.nats.>"] }
} }
{ user: "one.telegram", password: "$2a$11$tttttttttttttttttttttt", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.one_telegram.>", "$JS.ACK.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker.>", "$JS.API.CONSUMER.INFO.EVENTS.one_telegram", "$JS.API.CONSUMER.INFO.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_telegram", "$JS.API.CONSUMER.MSG.NEXT.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.one.telegram", "mesh.seat.telegram-sender.event.delivered", "mesh.seat.telegram-sender.event.failed"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.telegram", "$SRV.INFO.telegram.>", "$SRV.PING", "$SRV.PING.telegram", "$SRV.PING.telegram.>", "$SRV.STATS", "$SRV.STATS.telegram", "$SRV.STATS.telegram.>", "_INBOX.one.telegram.>", "mesh.assignment.one.telegram", "mesh.mod.telegram.call.>", "mesh.mod.telegram.tool.>", "mesh.seat.telegram-sender.accept.send"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "two.audit", password: "$2a$11$aaaaaaaaaaaaaaaaaaaaaa", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_audit.>", "$JS.API.CONSUMER.INFO.EVENTS.two_audit", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_audit", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.two.audit"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.audit", "$SRV.INFO.audit.>", "$SRV.PING", "$SRV.PING.audit", "$SRV.PING.audit.>", "$SRV.STATS", "$SRV.STATS.audit", "$SRV.STATS.audit.>", "_INBOX.two.audit.>", "mesh.assignment.two.audit", "mesh.mod.audit.call.>", "mesh.mod.audit.tool.>", "mesh.mod.shop.event.order.placed"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "two.shop", password: "$2a$11$ssssssssssssssssssssss", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_shop.>", "$JS.API.CONSUMER.INFO.EVENTS.two_shop", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_shop", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.two.shop", "mesh.mod.shop.event.order.placed", "mesh.seat.telegram-sender.accept.send"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.shop", "$SRV.INFO.shop.>", "$SRV.PING", "$SRV.PING.shop", "$SRV.PING.shop.>", "$SRV.STATS", "$SRV.STATS.shop", "$SRV.STATS.shop.>", "_INBOX.two.shop.>", "mesh.assignment.two.shop", "mesh.mod.shop.call.>", "mesh.mod.shop.tool.>"] }
allow_responses: { max: 1, ttl: "1m" }
} }
]
}
"$SYS" {
exports = [
{ stream: "$SYS.ACCOUNT.MESH.CONNECT", accounts: ["MESH"] }
{ stream: "$SYS.ACCOUNT.MESH.DISCONNECT", accounts: ["MESH"] }
{ service: "$SYS.REQ.ACCOUNT.MESH.CONNZ", accounts: ["MESH"] }
]
}
}
File diff suppressed because one or more lines are too long
+111
View File
@@ -0,0 +1,111 @@
package buscompose
import (
"slices"
"strings"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// **The grants part's user list, sealed to the bus node** (novox/hq ADR 0547 clause 7, issue 1945). The grants part writes
// the list it composed as one object of its own object bucket, sealed to the key of the node holding mesh-broker, and
// names it in one key of its own state `user-list`. The node-engine of that node follows the key when the switch says
// so (clause 8), opens the object and writes the bus's user-list file.
//
// Who may do what, and nothing more:
//
// - the grants part writes its object bucket (create, put, purge of an object's old chunks, delete) and its own state
// as before;
// - the node-engine of the node holding mesh-broker reads that one key of the state, watched, and an object of the
// bucket by its name: never the bucket's list of names, never a write, never another key of the grants part's state;
// - no other principal touches either.
// GrantsUserListBucket, GrantsUserListKey, GrantsUserListObjects and GrantsUserListObjectStream are delivery's names,
// here where the grants are composed.
const (
GrantsUserListBucket = delivery.GrantsUserListBucket
GrantsUserListKey = delivery.GrantsUserListKey
GrantsUserListObjects = delivery.GrantsUserListObjects
GrantsUserListObjectStream = delivery.GrantsUserListObjectStream
)
// GrantsUserListFollows is what the node-engine of the node holding mesh-broker may publish to follow the grants part's
// list: the one key `user-list.sealed` read and watched (its own ordered consumer, filtered on that key alone), and an
// object of the bucket by its name, as a node reads its declaration's files (ADR 0486 §2: never STREAM.INFO of the object
// stream, which lists every name, and never STREAM.MSG.GET).
func GrantsUserListFollows() []string {
kv := "KV_" + GrantsUserListBucket
key := "$KV." + GrantsUserListBucket + "." + GrantsUserListKey
obj, s := GrantsUserListObjects, GrantsUserListObjectStream
return []string{
"$JS.API.DIRECT.GET." + kv + "." + key,
"$JS.API.CONSUMER.CREATE." + kv + ".*." + key,
"$JS.API.CONSUMER.DELETE." + kv + ".*",
"$JS.API.STREAM.INFO." + kv,
"$JS.FC." + kv + ".>",
"$JS.API.DIRECT.GET." + s + ".$O." + obj + ".M.*",
"$JS.API.CONSUMER.CREATE." + s + ".*.$O." + obj + ".C.*",
"$JS.API.CONSUMER.DELETE." + s + ".*",
"$JS.FC." + s + ".>",
}
}
// GrantsUserListWrites is what the grants part may publish to keep its object bucket: create it and read its info back,
// put an object (its chunks and its metadata, the old chunks of a name purged), read an object's metadata, and delete
// one. Its own state `user-list` it writes already, as every module writes its own.
func GrantsUserListWrites() []string {
obj, s := GrantsUserListObjects, GrantsUserListObjectStream
return []string{
"$JS.API.STREAM.CREATE." + s,
"$JS.API.STREAM.INFO." + s,
"$JS.API.STREAM.PURGE." + s,
"$JS.API.DIRECT.GET." + s + ".>",
"$JS.API.STREAM.MSG.GET." + s,
"$O." + obj + ".>",
}
}
// grantsOwnObjects is the grants part writing its own object bucket, and nothing of another.
func grantsOwnObjects(p Principal, pattern string) bool {
return p.Kind == KindModule && p.Module == GrantsModule &&
(pattern == "$O."+GrantsUserListObjects+".>" || pattern == "$JS.API.STREAM.CREATE."+GrantsUserListObjectStream)
}
// **Named readers** (novox/hq ADR 0547 clause 9, progressive insight of 2026-10-26; issue 1945). A state named here is
// read by the modules named and by no other, whatever another module's `reads` says: the vault's `bus-accounts` holds
// every bus account's bcrypt hash, and from take-over the grants part alone may read it. A module's announcement may name
// readers too (announce.State.Readers); both are honoured.
const VaultBusAccounts = "mesh-vault.bus-accounts"
// NamedReaders are the states only the modules named may read, by `<owner>.<name>`.
var NamedReaders = map[string][]string{VaultBusAccounts: {GrantsModule}}
// ReaderMay says whether module may read the state read (`<owner>.<name>`) by the named readers: always, for a state
// that names none.
func ReaderMay(module, read string) bool {
readers, named := NamedReaders[read]
return !named || slices.Contains(readers, module)
}
// readsNamed splits a module's reads by the named readers: those it may read, and those named for others.
func readsNamed(module string, reads []string) (granted, refused []string) {
for _, r := range reads {
if ReaderMay(module, strings.TrimSpace(r)) {
granted = append(granted, r)
} else {
refused = append(refused, r)
}
}
return granted, refused
}
// withNamedReaders is d with the reads named for other modules moved to its refused reads.
func withNamedReaders(d Declared) Declared {
granted, refused := readsNamed(d.Module, d.Reads)
if len(refused) == 0 {
return d
}
d.Reads = granted
d.ReadsRefused = append(append([]string(nil), d.ReadsRefused...), refused...)
return d
}
@@ -0,0 +1,213 @@
//go:build integration
package buscompose
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"os"
"path/filepath"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/seal"
"github.com/nats-io/nats-server/v2/server"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
)
// Defends novox/hq ADR 0547 clause 7 on a real server (issue 1945): with the list this composer writes, the grants part
// creates its object bucket and its state, puts the sealed list and names it in `user-list.sealed`; the node-engine of
// the node holding mesh-broker watches that one key, reads the object by its name and opens it with its key, and may
// neither list the bucket, nor read another key of the state, nor write; another node reads nothing of either.
func TestTheBusNodeFollowsTheGrantsPartsSealedListUnderTheComposedGrants(t *testing.T) {
records := Records{
Nodes: []string{"novox", "ace"},
Assigned: map[string][]Declared{
"novox": {{Module: "nats", SnapshotsTheBus: true},
{Module: GrantsModule, State: []Bucket{{Module: GrantsModule, Name: "user-list"}}}},
"ace": {{Module: "plex"}},
},
}
users, err := Users(records)
if err != nil {
t.Fatal(err)
}
for i := range users {
users[i].PasswordHash = "pw-" + users[i].Username()
}
accounts, err := ComposeAccounts(users)
if err != nil {
t.Fatal(err)
}
dir := t.TempDir()
conf := filepath.Join(dir, "server.conf")
if err := os.WriteFile(conf, []byte("listen: 127.0.0.1:-1\njetstream { store_dir: \""+dir+"\" }\n"+accounts), 0o600); err != nil {
t.Fatal(err)
}
opts, err := server.ProcessConfigFile(conf)
if err != nil {
t.Fatalf("the composed list is not the server's: %v", err)
}
opts.NoLog, opts.NoSigs = true, true
s, err := server.NewServer(opts)
if err != nil {
t.Fatal(err)
}
go s.Start()
if !s.ReadyForConnections(30 * time.Second) {
t.Fatal("the bus did not come up")
}
t.Cleanup(s.Shutdown)
dial := func(user string) *nats.Conn {
nc, err := nats.Connect(s.ClientURL(), nats.UserInfo(user, "pw-"+user), nats.CustomInboxPrefix("_INBOX."+user))
if err != nil {
t.Fatalf("%s: %v", user, err)
}
t.Cleanup(nc.Close)
return nc
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
// The grants part: its state and its object bucket, the sealed list put and named.
grants, err := jetstream.New(dial("novox." + GrantsModule))
if err != nil {
t.Fatal(err)
}
// Its state is made as the grants part makes a declared state, by its stream's name (CreateKeyValue would ask the
// account's info, which no module is granted).
if _, err := grants.CreateStream(ctx, jetstream.StreamConfig{Name: "KV_" + GrantsUserListBucket,
Subjects: []string{"$KV." + GrantsUserListBucket + ".>"}, MaxMsgsPerSubject: 1, AllowDirect: true,
AllowRollup: true, DenyDelete: true, Discard: jetstream.DiscardNew}); err != nil {
t.Fatalf("the grants part's state: %v", err)
}
kv, err := grants.KeyValue(ctx, GrantsUserListBucket)
if err != nil {
t.Fatalf("the grants part binds its state: %v", err)
}
objects, err := grants.CreateObjectStore(ctx, jetstream.ObjectStoreConfig{Bucket: GrantsUserListObjects})
if err != nil {
t.Fatalf("the grants part's object bucket: %v", err)
}
node, err := seal.GenerateKeyPair()
if err != nil {
t.Fatal(err)
}
list := []byte(accounts)
digest := delivery.UserListDigest(list)
object, err := delivery.SealUserList(list, node.PublicKey)
if err != nil {
t.Fatal(err)
}
name := delivery.UserListObjectName(digest)
for range 2 { // put twice: a second put of a name purges its old chunks
if _, err := objects.PutBytes(ctx, name, object); err != nil {
t.Fatalf("the grants part put its object: %v", err)
}
}
value, err := delivery.EncodeGrantsUserList(delivery.GrantsUserList{Follow: delivery.FollowGrants, Digest: digest,
Object: name, SealedTo: delivery.SealingKeyID(node.PublicKey), Node: "novox", Revision: 1, Users: len(users)})
if err != nil {
t.Fatal(err)
}
if _, err := kv.Put(ctx, GrantsUserListKey, value); err != nil {
t.Fatal(err)
}
if _, err := kv.Put(ctx, "list", []byte(`{"v":1}`)); err != nil {
t.Fatal(err)
}
// The bus node's engine: the key watched, the object by name (as mesh-host's link reads a declaration's file).
busNode := dial("node.novox")
js, err := busNode.JetStream(nats.MaxWait(5 * time.Second))
if err != nil {
t.Fatal(err)
}
keys, err := js.KeyValue(GrantsUserListBucket)
if err != nil {
t.Fatalf("the bus node binds the grants part's state: %v", err)
}
w, err := keys.Watch(GrantsUserListKey)
if err != nil {
t.Fatalf("the bus node watches the key: %v", err)
}
defer w.Stop()
var got delivery.GrantsUserList
select {
case e := <-w.Updates():
if e == nil {
t.Fatal("the watch found no value")
}
if got, err = delivery.DecodeGrantsUserList(e.Value()); err != nil {
t.Fatal(err)
}
case <-ctx.Done():
t.Fatal("the watch said nothing")
}
read := func(js nats.JetStreamContext, name string) ([]byte, error) {
meta, err := js.GetLastMsg(GrantsUserListObjectStream, "$O."+GrantsUserListObjects+".M."+
base64.URLEncoding.EncodeToString([]byte(name)), nats.DirectGet())
if err != nil {
return nil, err
}
var info struct {
NUID string `json:"nuid"`
Size int `json:"size"`
}
if err := json.Unmarshal(meta.Data, &info); err != nil {
return nil, err
}
sub, err := js.SubscribeSync("$O."+GrantsUserListObjects+".C."+info.NUID, nats.OrderedConsumer(),
nats.BindStream(GrantsUserListObjectStream))
if err != nil {
return nil, err
}
defer sub.Unsubscribe()
var body []byte
for len(body) < info.Size {
m, err := sub.NextMsg(5 * time.Second)
if err != nil {
return nil, err
}
body = append(body, m.Data...)
}
return body, nil
}
body, err := read(js, got.Object)
if err != nil {
t.Fatalf("the bus node reads the object by its name: %v", err)
}
opened, err := delivery.OpenUserList(body, got.Digest, func(b seal.Box) ([]byte, error) {
s, err := seal.Open(b, node.PrivateKey)
return []byte(s), err
})
if err != nil || !bytes.Equal(opened, list) {
t.Fatalf("the bus node opened %d bytes, %v", len(opened), err)
}
// And nothing beyond: no other key, no list of the bucket, no write.
if _, err := keys.Get("list"); err == nil {
t.Error("the bus node read another key of the grants part's state")
}
if _, err := js.StreamInfo(GrantsUserListObjectStream); err == nil {
t.Error("the bus node read the object stream's info, which lists every name")
}
if _, err := js.Publish("$O."+GrantsUserListObjects+".M.x", []byte("x")); err == nil {
t.Error("the bus node wrote the grants part's bucket")
}
// Another node reads nothing of it.
other, err := dial("node.ace").JetStream(nats.MaxWait(2 * time.Second))
if err != nil {
t.Fatal(err)
}
if _, err := read(other, got.Object); err == nil {
t.Error("another node read the object")
}
if _, err := other.GetLastMsg("KV_"+GrantsUserListBucket, "$KV."+GrantsUserListBucket+"."+GrantsUserListKey, nats.DirectGet()); err == nil {
t.Errorf("another node read the key: %v", err)
}
}
+495
View File
@@ -0,0 +1,495 @@
package buscompose
import (
"fmt"
"git.novox.be/novox/mesh-sdk/go/announce"
"slices"
"sort"
"strings"
)
// Every user the composed file should contain, derived from what the mesh knows.
//
// **The list is derived, never kept.** A stored user list would be a second account of who may
// reach the bus, able to disagree with the records it came from — and the disagreement would be
// invisible, because both would look internally consistent. So this is a pure function of the
// mesh's records, run again every time the file is written.
//
// Records are mirrored into this package's own types rather than imported from the catalogue, for
// the reason DeclaredSeat is: composing authority is a different job from parsing a manifest, and
// this package stays free of the other's types so a change to a manifest field cannot quietly widen
// a permission.
// Declared is one module on one node, as composing its authority needs it.
type Declared struct {
Module string
Emits []string
Consumes []string
Serves []string
// Served are the config keys its served values and zone name fill from its settings (catalogue.ServedSettings): what
// every node may read of its resolved values where it runs (Principal.ServedReads, issue 1126).
Served []string
// Holds are the seats this module claims, with the protocol each seat declares. A seat the
// mesh defines for itself declares no protocol, so holding one grants nothing on the bus —
// which is right: those seats are about who does a job, not about who may say what.
Holds []Seat
// Uses are the seats this module sends to.
Uses []Seat
// Issued are the tools that answer a credential its software issued, one per offer that says so (novox/hq
// ADR 0305): the controller is granted to ask each in its own name, and no other tool so.
Issued []string
// Watches are the seats whose events it consumes.
Watches []Seat
// Invokes are the tools it calls, `<module>.<tool>` or `*` (novox/hq ADR 0152).
Invokes []string
// Waits are the event families a session may wait on through the tool runner, the tool runner's module's
// alone (novox/hq ADR 0367 §2): its account subscribes them for a call, through no consumer.
Waits []Wait
// State is the state it keeps, each a bucket its instances write (novox/hq ADR 0201).
State []Bucket
// Logs are the logs it keeps, each a stream its instances append to (novox/hq ADR 0297).
Logs []Log
// Streams are the streams it owns, each its events' one place (novox/hq ADR 0363).
Streams []ModuleStream
// Queues are its own work-queue streams and the one process each is the consumer of (novox/hq ADR 0436
// §1, §2, §6): one durable consumer per entry, named `<node>_<module>_<process id>`.
Queues []QueueOwner
// Hears are the module streams it hears on this machine, each with the subjects it was granted
// (novox/hq ADR 0363 §5); HearsAStream says its `consumes` names a module that owns a stream, granted
// or not, so its membership is issued again when a grant changes.
Hears []Heard
HearsAStream bool
// Reads are other modules' state it reads, each `<module>.<name>` (novox/hq ADR 0201).
Reads []string
// ReadsRefused are the reads its manifest names that the owner's announcement does not make readable (novox/hq ADR
// 0495 clause 3, issue 1790): granted nothing, kept to be said.
ReadsRefused []string
// KeyedReads are keys of other modules' state it reads, each one key alone: what a seat's holder is
// granted for the pieces the modules beside it offer (novox/hq ADR 0260).
KeyedReads []KeyedRead
// NoAccount says the module declares no own secret named broker, so no account could ever be
// delivered to it and nothing can connect as it (novox/hq issue 195). Said in the negative so a
// record that does not say is composed as it always was.
NoAccount bool
// SnapshotsTheBus says the module holds mesh-broker — it is the bus — and so is the one module
// granted the snapshot API, to copy the bus's streams for the night's backup (novox/hq ADR 0235).
SnapshotsTheBus bool
// Checks are the module's own tools its health asks, each `<module>.<tool>` (novox/hq ADR 0240, to-be
// 48 §3): the machine's node-engine asks them of its own node tools, and is granted that and no more.
Checks []string
// Loads says the module has a compiled bundle whose entrypoints the machine's tool runtime loads and runs
// (novox/hq ADR 0175; issue 1329): a helper that reaches the bus on the runner's account, so the runner
// must carry the module although it declares no tools.
Loads bool
// RunsAs is the account the module runs as in a runtime of its own (novox/hq ADR 0259 §8): it is never
// carried by the machine's runtime, and reaches the bus on its own account.
RunsAs string
// ProvisionsNode says the module runs the machine's provisioner client: it claims NodeProvisionerSeat and
// declares the own secret ProvisionerSecret its account is delivered in (novox/hq ADR 0349 clause 11, design 57
// P3). The client is given a bus account of its own, never the module's or the tool runner's.
ProvisionsNode bool
// Processes are this module's own processes that declare bus (novox/hq ADR 0436 §1), each getting a
// principal of its own: granted exactly what its bus says, and nothing of this module's other grants —
// composed whether or not the module itself has an account (NoAccount), because a process account is
// delivered in its own secret, broker-<process id>, never the module's `broker`.
Processes []ProcessPrincipal
}
// ProcessPrincipal is one process of a module that declares bus (novox/hq ADR 0436 §1, §2), as the broker
// composes its own account from: its own emits, its own work-queue consumers, its own state, whether it
// reads the router's records of its module's own asks, and the tools and seat verbs it calls.
type ProcessPrincipal struct {
// Process is the process resource's id, naming the account <node>.<module>.<process id>.
Process string
// Emits are the events this process may publish, named as the module's own Emits are.
Emits []string
// Queues are the module's own work-queue streams this process is the one consumer of.
Queues []QueueOwner
// State is the module's own state, by local name, this process reads and writes.
State []string
// Asks says whether this process reads the records of its module's own asks (novox/hq ADR 0259 §3), on
// whichever seats the module uses that keep them (the router's, operator-channel, among others).
Asks bool
// Invokes are the tools and seat verbs this process calls, each <module>.<tool> or a role's
// seat:<seat>.<verb>, or * for every tool on the mesh.
Invokes []string
}
// Records is what composing a user list needs to know about the mesh, and nothing more.
type Records struct {
// Nodes is every machine the mesh knows. Each gets a host user.
Nodes []string
// Assigned is the modules on each node, as they declare themselves.
Assigned map[string][]Declared
// Enrolling is every node with a live token — one enrolment user each, because the inbox an
// answer goes to is scoped to the token and a shared one is one machine reading another's
// sealed credentials (design 25 §6).
Enrolling []string
// People is each person's name against the tools they may invoke, `*` for an administrator.
People map[string][]string
// Interchangeable is each module whose definition says its instances are the same anywhere
// (ADR 0160), which decides whether the module's plain subject is issued to every instance.
Interchangeable map[string]bool
// RootFree is each machine judged root-free when this was composed (novox/hq ADR 0259 §8): it names an
// account agents run as, judged unable to become root by its node-engine, and serves no login shell
// execute. A machine absent is not free: no record of a pass is no pass.
RootFree map[string]bool
// Announced is each module's last announcement of the states it keeps (novox/hq ADR 0495, issue 1790), as the
// controller keeps it; a module absent has announced nothing yet. Nil where none was read.
Announced map[string]announce.States
}
// Users is every user the composed file should contain, in the order it will be written.
//
// The controller is always first and always present: a mesh whose own controller is not in the file
// is a mesh that cannot be told anything, and there is no state of the records in which that is
// correct.
func Users(r Records) ([]Principal, error) {
out := []Principal{{Kind: KindController, Uses: asksSeatOf(r), AsksIssued: issuedOf(r), AsksTheVault: vaultVerbsOf(r),
AsksTheConfig: servedOf(r, VerbsTheControllerAsksTheConfig), FilesFor: sortedCopy(r.Nodes)}}
served := servedReadsOf(r)
for _, node := range sortedCopy(r.Nodes) {
witness := false
var checks []string
bus := false
for _, d := range r.Assigned[node] {
if d.Module == controllerModule {
witness = true
}
if d.SnapshotsTheBus {
bus = true
}
checks = append(checks, d.Checks...)
}
out = append(out, Principal{Kind: KindNode, Node: node, WitnessesController: witness, Checks: checks,
MovesIntoTheVault: servesMoveIn(r), StatesItsKeyToTheVault: servesVerb(r, VaultNodeKey),
AsksItsOwnSecrets: servesVerb(r, VaultOwn),
FollowsGrantsUserList: bus, ServedReads: servedOn(served, node)})
// **And the machine's provisioner client, on an account of its own** (novox/hq ADR 0349 clause 11): one
// per machine that runs it, delivered to the module running it.
for _, d := range r.Assigned[node] {
if d.ProvisionsNode {
out = append(out, Principal{Kind: KindProvisioner, Node: node, Module: d.Module,
BoundBuckets: boundBucketsOf(r)})
break
}
}
// **Where the runtime is assigned, the machine gets one runtime principal in place of the
// runtime module's own** (novox/hq ADR 0175, to-be 38). It carries every module on the
// node: its serving grants are the union of theirs. Every other module keeps its own
// principal — a module still serving tools from its own container holds its own
// credential until it moves, and the two serve side by side in the meantime.
runtimeHere := false
for _, d := range r.Assigned[node] {
if d.Module == RuntimeModule {
runtimeHere = true
}
}
for _, d := range r.Assigned[node] {
// A state with named readers is read by them alone (ADR 0547 clause 9): a read named for another module is
// granted nothing, here and in the runtime carrying the module.
d = withNamedReaders(d)
// **A process account is its own, delivered in its own secret** (broker-<process id>, novox/hq
// ADR 0436 §1, §2), never the module's `broker` — so it is composed whether or not the module
// itself has an account, and is never carried by the machine's runtime, which runs as the
// operator's account (ADR 0436 §5): added here, before either of those is judged.
for _, pr := range d.Processes {
out = append(out, Principal{
Kind: KindProcess, Node: node, Module: d.Module, Process: pr.Process,
Emits: pr.Emits, Queues: pr.Queues, State: pr.State, Uses: d.Uses, Asks: pr.Asks,
Invokes: pr.Invokes,
})
}
if runtimeHere && d.Module == RuntimeModule {
continue
}
// **A module with nowhere to read an account is no user** (novox/hq issue 195). `module
// issue` refuses it one (issue 078: an account nothing reads is an orphan), so its user
// could only ever be left out of the file for want of a password — and every such module
// was named, on every status and plan, as a credential the mesh had not minted. Where the
// runtime is, it speaks for the module; where it is not, the module cannot speak at all,
// and a user would not change that.
if d.NoAccount {
continue
}
out = append(out, Principal{
Kind: KindModule, Node: node, Module: d.Module,
Emits: d.Emits, Consumes: d.Consumes, Serves: d.Serves,
Holds: d.Holds, Uses: d.Uses, Watches: d.Watches, Invokes: d.Invokes,
State: stateNames(d.State), Reads: d.Reads, SnapshotsTheBus: d.SnapshotsTheBus,
PerMachine: perMachineNames(d.State), KeyedReads: d.KeyedReads,
Logs: logNames(d.Logs), Streams: d.Streams, Hears: ComposedHears(d), Queues: d.Queues,
})
// **The grants part creates what a manifest declares** (novox/hq ADR 0547 §5; the operator, 2026-10-26
// 08:54): every placed module's buckets, logs and streams, each by its stream's name.
if d.Module == GrantsModule {
out[len(out)-1].CreatesStreams = declaredStreamsOf(r)
}
// **The vault's holder asks the desks and the issuing providers itself** (issue 723), under the
// account its own runtime runs on (ADR 0259 §8's runs-as): the one module granted it.
if holdsSeat(d, VaultSeat) {
out[len(out)-1].AsksTheDesk = true
out[len(out)-1].AsksIssued = issuedOf(r)
}
}
if runtimeHere {
var carried []Declared
for _, d := range r.Assigned[node] {
if d.RunsAs == "" && runnerServes(d) {
d = withNamedReaders(d)
// Never a restricted stream for what the tool runner carries (novox/hq ADR 0376 §1).
d.Hears = ComposedHears(d)
carried = append(carried, d)
}
}
out = append(out, Principal{Kind: KindNodeTools, Node: node, Module: RuntimeModule, Carries: carried})
}
}
for _, node := range sortedCopy(r.Enrolling) {
out = append(out, Principal{Kind: KindEnrolment, Node: node})
}
for _, person := range sortedNames(r.People) {
p := Principal{Kind: KindPerson, Module: person, Invokes: r.People[person]}
if administers(p.Invokes) {
// The operator's hand path writes each node's compositions (ADR 0539 clause 10).
p.FilesFor = sortedCopy(r.Nodes)
}
out = append(out, p)
}
// Refused here rather than discovered by the server. Two users with one name is a file the
// server reads as one of them, and which one depends on the order — so a module assigned to a
// node twice, or a person named after nothing, is a composition that must not be written.
seen := map[string]string{}
for _, p := range out {
name := p.Username()
if name == "" || name == "." {
return nil, fmt.Errorf("a %s user has no name, so nothing could authenticate as it", p.Kind)
}
if first, already := seen[name]; already {
return nil, fmt.Errorf(
"two users would be called %q (a %s and a %s): the server would read the file as "+
"one of them, and which one depends on the order", name, first, p.Kind)
}
seen[name] = string(p.Kind)
}
return out, nil
}
// WithPasswords fills each user's hash from what the mesh minted, and says which users have none.
//
// **Separated from Users because they fail differently.** A user missing from the records is a bug
// in deriving them; a user with no password is a step that has not happened yet — a module assigned
// but never given a credential, a node enrolled before this existed. The second is ordinary and its
// remedy is to mint one, so it is named rather than returned as an error, and the caller decides
// whether a partial composition is worth writing.
func WithPasswords(principals []Principal, hashes map[string]string) (filled []Principal, missing []string) {
for _, p := range principals {
hash, ok := hashes[p.Username()]
if !ok || hash == "" {
missing = append(missing, p.Username())
continue
}
p.PasswordHash = hash
filled = append(filled, p)
}
return filled, missing
}
func sortedCopy(in []string) []string {
out := append([]string(nil), in...)
sort.Strings(out)
return out
}
func sortedNames(in map[string][]string) []string {
out := make([]string, 0, len(in))
for k := range in {
out = append(out, k)
}
sort.Strings(out)
return out
}
// controllerModule is the controller's module: the machine assigned it witnesses its upgrades.
const controllerModule = "mesh-controller"
// asksSeatOf is the seat an ask is made on, as its holder declares it (novox/hq ADR 0259): the controller
// asks the operator through it like any other user, and is granted what its declaration names for a caller.
// None while nothing holds it.
// issuedOf is every tool of a module assigned anywhere that answers a credential its software issued, as
// `<module>.<tool>` (novox/hq ADR 0305): what the controller asks in its own name, and nothing beside.
func issuedOf(r Records) []string {
seen := map[string]bool{}
var out []string
for _, node := range sortedCopy(r.Nodes) {
for _, d := range r.Assigned[node] {
for _, t := range d.Issued {
if key := d.Module + "." + t; !seen[key] {
seen[key] = true
out = append(out, key)
}
}
}
}
sort.Strings(out)
return out
}
func asksSeatOf(r Records) []Seat {
for _, node := range sortedCopy(r.Nodes) {
for _, d := range r.Assigned[node] {
for _, s := range d.Holds {
if s.Name == AsksSeat && namesVerb(s.ByCaller, "ask") {
seat := s
seat.Kind, seat.Capabilities = "", nil
return []Seat{seat}
}
}
}
}
return nil
}
// VaultSeat is the vault's seat: its holder alone asks the desks and the issuing providers in its own name.
const VaultSeat = "mesh-vault"
func holdsSeat(d Declared, seat string) bool {
for _, s := range d.Holds {
if s.Name == seat {
return true
}
}
return false
}
// vaultVerbsOf are the vault's intake verbs its holder serves, as the records say (issue 675): what the controller
// is granted to ask it in its own name. None while no holder serves them.
func vaultVerbsOf(r Records) []string { return servedOf(r, VerbsTheControllerAsksTheVault) }
// servedOf are the verbs of asked that a seat's holder serves, as the records say, by name.
func servedOf(r Records, asked []SeatVerb) []string {
seen := map[string]bool{}
var out []string
for _, node := range sortedCopy(r.Nodes) {
for _, d := range r.Assigned[node] {
for _, s := range d.Holds {
for _, v := range asked {
if s.Name == v.Seat && slices.Contains(s.Serves, v.Verb) && !seen[v.Verb] {
seen[v.Verb] = true
out = append(out, v.Verb)
}
}
}
}
}
sort.Strings(out)
return out
}
// servesMoveIn says the vault's holder serves move-in, as the records say (issue 676): what lets each machine's
// node-engine ask it in its own name.
func servesMoveIn(r Records) bool { return servesVerb(r, VaultMoveIn) }
// servesVerb says a holder of verb's seat serves it, as the records say.
func servesVerb(r Records, verb SeatVerb) bool {
for _, node := range r.Nodes {
for _, d := range r.Assigned[node] {
for _, s := range d.Holds {
if s.Name == verb.Seat && slices.Contains(s.Serves, verb.Verb) {
return true
}
}
}
}
return false
}
// boundBucketsOf is every provider's bucket of entries, `<module>_bound` (mesh-sdk go/provider, issue 678): the state
// named bound of every module placed anywhere, once each, in name order. A consumer on any machine may be bound to
// it, so each machine's provisioner client reads its own keys of each (issue 679).
func boundBucketsOf(r Records) []string {
seen := map[string]bool{}
for _, mods := range r.Assigned {
for _, d := range mods {
for _, b := range d.State {
if b.Name == BoundState {
seen[b.Bucket()] = true
}
}
}
}
out := make([]string, 0, len(seen))
for b := range seen {
out = append(out, b)
}
sort.Strings(out)
return out
}
// runnerServes says the machine's tool runner has something of this module to serve (novox/hq ADR 0436 §1,
// ADR 0437 §4, issue 1329): tools or loads it answers, a seat it holds, event families a session waits on,
// a bundle the runtime loads (Loads), or a check its health asks. A module with none of these is a process of its own on its own credential, and
// carrying it would hand every session on the machine the runner's account with that module's state, seat
// traffic and streams. A module with no account of its own (NoAccount) is spoken for by the runner, so it
// stays carried, and so does the runtime module itself. Whether the module has its own broker secret is not
// the test: a module with tools and its own credential is still carried (ADR 0175).
func runnerServes(d Declared) bool {
return d.Module == RuntimeModule || d.NoAccount || d.Loads || len(d.Serves) > 0 || len(d.Holds) > 0 || len(d.Waits) > 0 || len(d.Checks) > 0
}
// declaredStreamsOf is the stream of every bucket, log and stream a module placed anywhere declares, once each, in name
// order (novox/hq ADR 0547 §5, issue 1790): `KV_<module>_<state>`, `LOG_<module>_<log>` and each module stream's own
// name. What the grants part may create, and nothing beside.
func declaredStreamsOf(r Records) []string {
seen := map[string]bool{}
for _, mods := range r.Assigned {
for _, d := range mods {
for _, b := range d.State {
seen["KV_"+b.Bucket()] = true
}
for _, l := range d.Logs {
seen[l.Stream()] = true
}
for _, s := range d.Streams {
seen[s.Name] = true
}
}
}
out := make([]string, 0, len(seen))
for s := range seen {
out = append(out, s)
}
sort.Strings(out)
return out
}
// servedReadsOf is every key of mesh-config's resolved values a module assigned on a node serves from,
// `<node>.<module>.<key>`, sorted: each part a subject token, so a key that cannot be one is granted nothing (and the
// node composing a consumer of it refuses, naming it), never a wider read.
func servedReadsOf(r Records) []string {
var out []string
for node, declared := range r.Assigned {
for _, d := range declared {
for _, k := range d.Served {
if safeSubject.MatchString(node) && safeSubject.MatchString(d.Module) && safeSubject.MatchString(k) {
out = append(out, node+"."+d.Module+"."+k)
}
}
}
}
sort.Strings(out)
return slices.Compact(out)
}
// servedOn is, of the served keys, those of other nodes than node: its own slice it reads whole.
func servedOn(served []string, node string) []string {
var out []string
for _, k := range served {
if !strings.HasPrefix(k, node+".") {
out = append(out, k)
}
}
return out
}
+94
View File
@@ -0,0 +1,94 @@
package buscompose
import (
"fmt"
"strings"
)
// What a session may wait on through the mesh MCP server (novox/hq ADR 0367).
//
// **A grant apart from `consumes`, and the tool runner's alone.** The tool runner's module names in its
// manifest the event families a session may wait on, one by one; the runtime's account is granted a plain
// subscription on those subjects and nothing wider — no durable consumer, no queue of its own — and its
// membership names the same list, from which `mesh_wait` refuses any other. Both are read from this one
// composition, so the grant and what the tool runner offers cannot disagree, and nothing but the manifest
// adds a family (§6).
// Wait is one event family the tool runner's module declares: its name as the manifest gives it, and the
// emitter — a seat or a module — and the event, as the inventory resolved them.
type Wait struct {
// Name is the family as the manifest names it, `<emitter>.<event>`: what a session asks for.
Name string
Emitter string
// Seat says the emitter is a seat, whose events live under the seat's namespace (novox/hq ADR 0121).
Seat bool
// Event is the event, with any further token the subject carries: `decided.claude-code`, the asker of a
// seat that names it last (ADR 0259 §3).
Event string
// Last names what the seat's event subject ends in — "kind" for a kinded bench, "asker" for a seat naming
// its caller (ADR 0259 §3) — and is empty for an event that ends at its name. A family needing it is waited
// on with it given, and refused without.
Last string
}
// WaitSubject is where a family's events land. Each token is checked before it becomes a subject: a
// wildcard, a dot inside a name or an empty token would widen the grant.
func WaitSubject(w Wait) (string, error) {
if !safeSubject.MatchString(w.Emitter) {
return "", fmt.Errorf("%q cannot name an emitter of a waited event: it would widen the subject", w.Emitter)
}
for _, t := range strings.Split(w.Event, ".") {
if !safeSubject.MatchString(t) {
return "", fmt.Errorf("%q cannot name a waited event: each of its names is plain, never a wildcard "+
"(novox/hq ADR 0367 §2)", w.Event)
}
}
if w.Last != "" && !strings.Contains(w.Event, ".") {
return "", fmt.Errorf("%s names no %s: %s's events are said for each %s, so a family is %s.%s.<%s>; "+
"without it no event would ever be heard (novox/hq ADR 0367 §5)",
w.Name, w.Last, w.Emitter, w.Last, w.Emitter, w.Event, w.Last)
}
if w.Seat {
return "mesh.seat." + w.Emitter + ".event." + w.Event, nil
}
return "mesh.mod." + w.Emitter + ".event." + w.Event, nil
}
// waitGrants is the subscription the tool runner's account is granted for its module's families, and an
// error for a carried module other than the tool runner's that declares any (ADR 0367 §2).
func waitGrants(carries []Declared) ([]string, error) {
var sub []string
for _, d := range carries {
if len(d.Waits) == 0 {
continue
}
if d.Module != RuntimeModule {
return nil, fmt.Errorf("%s declares what a session waits on; only %s, the tool runner's module, "+
"does (novox/hq ADR 0367 §2)", d.Module, RuntimeModule)
}
for _, w := range d.Waits {
s, err := WaitSubject(w)
if err != nil {
return nil, err
}
sub = append(sub, s)
}
}
return sub, nil
}
// waitsIssuedFor is the families a membership names: the tool runner's module's, and nobody else's.
func waitsIssuedFor(d Declared) []WaitIssued {
if d.Module != RuntimeModule {
return nil
}
var out []WaitIssued
for _, w := range d.Waits {
s, err := WaitSubject(w)
if err != nil {
continue // refused where the account is composed, which names it
}
out = append(out, WaitIssued{Event: w.Name, Subject: s})
}
return out
}
+326
View File
@@ -0,0 +1,326 @@
package buscompose
import (
"fmt"
"slices"
"strings"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// The writers table (novox/hq to-be 45 §1, ADR 0227 rule 1), compiled in.
//
// **Every kind of state the core keeps has one writer; anyone else asks it.** The table is the design's,
// row for row, with what each row writes on the bus where it writes there. Two things read it: the
// composition of every principal's grants (PermissionsFor), which **refuses a grant that lets a
// principal publish on a subject another writes** — so a second writer cannot be granted by accident,
// and the composition says which state and whose — and the test beside it, which walks the rows
// against the design's list and the composition of a whole mesh. Changing a writer is a change to
// this table, through a decision.
// WriterRow is one row of the writers table.
type WriterRow struct {
State string
Writer string
KeptIn string
Others string
// Subjects are the bus subjects a write of this state is a publish to; none for state kept off the
// bus (the controller's store, a module's own) or not built yet.
Subjects []string
// Writes says a principal granted a publish pattern overlapping Subjects is this state's writer —
// given the pattern, because a machine writes its own report and no other's.
Writes func(p Principal, pattern string) bool
// Shared says why more than one principal may publish here; empty for one writer.
Shared string
}
// isController, and the other writers' tests, are who a row's writer is as a principal.
func isController(p Principal, _ string) bool { return p.Kind == KindController }
// createsDeclared is the controller, or the grants part creating one stream a manifest declares, by its own name: the
// create alone, never an update or a delete (novox/hq ADR 0547 §5; the operator, 2026-10-26 08:54).
func createsDeclared(p Principal, pattern string) bool {
if isController(p, pattern) {
return true
}
name, ok := strings.CutPrefix(pattern, "$JS.API.STREAM.CREATE.")
if ok && p.Kind == KindModule && p.Module == GrantsModule && name == GrantsUserListObjectStream {
return true
}
return ok && p.Kind == KindModule && p.Module == GrantsModule && slices.Contains(p.CreatesStreams, name) &&
safeSubject.MatchString(name)
}
// ownMachine is a node writing a subject whose third token is its own name, and no wildcard there.
func ownMachine(p Principal, pattern string) bool {
tokens := strings.Split(pattern, ".")
return p.Kind == KindNode && len(tokens) > 2 && tokens[2] == p.Node
}
// holdsSeatOf is a principal holding the seat a `mesh.seat.<seat>.…` pattern names: its module's own
// principal, or the node tools carrying a module that holds it (ADR 0175, the runtime is its modules).
func holdsSeatOf(p Principal, pattern string) bool {
tokens := strings.Split(pattern, ".")
if len(tokens) < 3 {
return false
}
seat := tokens[2]
holds := func(seats []Seat) bool {
return slices.ContainsFunc(seats, func(s Seat) bool { return s.Name == seat })
}
switch p.Kind {
case KindModule:
return holds(p.Holds)
case KindNodeTools:
return slices.ContainsFunc(p.Carries, func(d Declared) bool { return holds(d.Holds) })
}
return false
}
// ownModule is a module publishing under its own name, or the node tools carrying it.
func ownModule(p Principal, pattern string) bool {
tokens := strings.Split(pattern, ".")
if len(tokens) < 3 {
return false
}
module := tokens[2]
switch p.Kind {
case KindModule:
return p.Module == module
case KindNodeTools:
return slices.ContainsFunc(p.Carries, func(d Declared) bool { return d.Module == module })
}
return false
}
// ownAnnouncer is a module, a process of it or the node tools carrying it saying what that module keeps, or the
// controller saying what it keeps itself (novox/hq ADR 0495 clause 2): each under its own name alone.
func ownAnnouncer(p Principal, pattern string) bool {
tokens := strings.Split(pattern, ".")
if len(tokens) < 3 {
return false
}
switch p.Kind {
case KindController:
return tokens[2] == ControllerSeat
case KindProcess:
return p.Module == tokens[2]
}
return ownModule(p, pattern)
}
// ownProvisioner is a machine's provisioner client saying a report about its own machine: the third token of
// `prov.applied.<node>.…` is its machine (novox/hq ADR 0349 §4, design 57 P3).
func ownProvisioner(p Principal, pattern string) bool {
tokens := strings.Split(pattern, ".")
return p.Kind == KindProvisioner && len(tokens) > 3 && tokens[2] == p.Node
}
// ownNodeKeys is a node writing the have keys of its own node: `$KV.<bucket>.<node>.…`, the node in the third token
// and no wildcard there (novox/hq ADR 0429 §1).
func ownNodeKeys(p Principal, pattern string) bool {
tokens := strings.Split(pattern, ".")
return p.Kind == KindNode && len(tokens) > 2 && tokens[2] == p.Node
}
// kvOf is a bucket's write subjects.
func kvOf(bucket string) []string { return []string{"$KV." + bucket + ".>"} }
// WritersTable is to-be 45 §1, in its order.
var WritersTable = []WriterRow{
{State: "a machine's declaration", Writer: "controller (lease holder)", KeptIn: "the bus, last per subject",
Others: "read", Subjects: []string{"mesh.node.*.declare"}, Writes: isController},
// The operator's hand-over of a directory used as found, asked of the machine's engine at the controller's
// terminal (novox/hq issue 356). One publisher; and because a responder can still reach the subject through a
// reply, the ask is signed with the mesh's key and the engine verifies it — the row bounds who is granted the
// publish, the signature who is believed.
{State: "a hand-over asked of a machine", Writer: "controller, at its terminal", KeptIn: "the machine, beside its state",
Others: "the engine verifies the mesh's signature and records it, or refuses",
Subjects: []string{"mesh.node.*.ask.hand-over"}, Writes: isController},
// The operator asking a machine's engine for a fresh search for setuid programs, at the controller's
// terminal (novox/hq issue 361): signed as a hand-over is, under a signing context of its own.
{State: "a fresh setuid search asked of a machine", Writer: "controller, at its terminal",
KeptIn: "the machine, which throws its last search away", Others: "the engine verifies the mesh's signature and starts it, or refuses",
Subjects: []string{"mesh.node.*.ask.setuid-search"}, Writes: isController},
// A read-only judgement of a machine's declared directories asked of its engine (novox/hq issue 358): signed as a
// hand-over is, under a signing context of its own. It changes nothing on the machine.
{State: "a judgement of a machine's directories asked of it", Writer: "controller",
KeptIn: "nowhere: the engine answers it and keeps nothing", Others: "the engine verifies the mesh's signature and answers, or refuses",
Subjects: []string{"mesh.node.*.ask.judge-directories"}, Writes: isController},
{State: "a machine's applied state and its report", Writer: "the node-engine's apply queue",
KeptIn: "the machine; the report on the bus", Others: "the reconcile and a delivery enqueue, never apply",
// And its health statement between reports (novox/hq ADR 0240): the same writer stating the same
// machine, inside the grant it already had (`mesh.control.<its own>.>`). And a line mesh-cli was given
// on the machine (novox/hq ADR 0272): the node is what the controller judges the terminal by, so that
// subject is this machine's engine's alone.
Subjects: []string{"mesh.control.*.report", "mesh.control.*.health", "mesh.control.*.cli"}, Writes: ownMachine},
{State: "the controller lease", Writer: "the controller instance holding it", KeptIn: "key-value " + LeaseBucket,
Others: "a candidate waits", Subjects: kvOf(LeaseBucket), Writes: isController},
// The merges mesh-delivery announced, for a moved module's registration (ADR 0420 §7), written by the lease holder.
{State: "the merges mesh-delivery announced", Writer: "controller (lease holder)",
KeptIn: "key-value " + DeliveryMergesBucket, Others: "read at a moved module's registration",
Subjects: kvOf(DeliveryMergesBucket), Writes: isController},
// And the compositions (novox/hq ADR 0539 clauses 2 and 4): each node's in its own bucket, which only that node's
// node-engine reads, and each digest with the judgement of its move, which the assigner reads.
{State: "the compositions and their digests", Writer: "controller (lease holder), compare-and-set",
KeptIn: "key-value composed-<node> for each node, and " + delivery.ControllerComposedBucket,
Others: "a node-engine reads its own node's; the assigner reads the digests",
Subjects: kvOf(delivery.ControllerComposedBucket), Writes: isControllerOrTheHandPath},
{State: "conditions", Writer: "controller", KeptIn: "key-value " + ConditionsBucket + " (and its history, " +
ConditionHistoryBucket + "); how often each was observed, when last and its newest evidence in the " +
"controller's store, so a look that changes nothing writes no bucket", Others: "raise or clear only through observations the controller reads",
Subjects: append(kvOf(ConditionsBucket), kvOf(ConditionHistoryBucket)...), Writes: isController},
{State: "calls and their outcomes", Writer: "controller", KeptIn: "key-value " + CallsBucket,
Others: "read by id", Subjects: kvOf(CallsBucket), Writes: isController},
{State: "the hand-act log", Writer: "controller, through the verbs that act", KeptIn: "key-value " + HandActsBucket,
Others: "—", Subjects: kvOf(HandActsBucket), Writes: isController},
// What the healers did (novox/hq to-be 45 §7, Phase 3): the controller's alone, in its store — the
// budgets and the mesh-wide brake are counted from it, so a controller restarting cannot reset them.
{State: "the healers' acts and their brake", Writer: "controller (lease holder), each act begun before it is made",
KeptIn: "the controller's store", Others: "read through healers; each act said as healer-acted and in its condition's tried"},
{State: "stream definitions and bus permissions", Writer: "controller", KeptIn: "the bus", Others: "—",
// A stream's definition, and a durable consumer's by the API that names it so. Not every
// consumer create: a module watching its own bucket makes and deletes an ordered consumer on the
// bucket's stream (ADR 0201), which defines nothing the mesh keeps.
Subjects: []string{"$JS.API.STREAM.CREATE.>", "$JS.API.STREAM.UPDATE.>", "$JS.API.STREAM.DELETE.>",
"$JS.API.CONSUMER.DURABLE.CREATE.>"},
Writes: createsDeclared,
Shared: "the grants part creates a stream a manifest declares, by its name, and never updates or deletes one (novox/hq ADR 0547 §5; the operator, 2026-10-26 08:54)"},
{State: "builds and their outcomes", Writer: "the build seat's holder", KeptIn: "its own state",
Others: "the controller asks",
Subjects: []string{"mesh.seat.node-build-agent.event.built", "mesh.seat.mesh-build-machine.event.built"},
Writes: holdsSeatOf,
Shared: "every machine holding the build seat answers the asks it took; each outcome names its ask"},
// What a module keeps and who may read it (novox/hq ADR 0495 clauses 2 and 3, issue 1790): said by the module, kept by
// the controller.
{State: "a module's states announced", Writer: "the module itself, from its code, under its own name",
KeptIn: "the bus (EVENTS)", Others: "the controller keeps the last of each and grants readers from it",
Subjects: []string{"mesh.mod.*.event.states"}, Writes: ownAnnouncer},
{State: "the states announced", Writer: "controller, from each module's announcement",
KeptIn: "key-value " + StatesAnnouncedBucket, Others: "the composition of the bus's users reads it",
Subjects: kvOf(StatesAnnouncedBucket), Writes: isController},
{State: "a merge announced", Writer: "one announcer per forge (the hook, or the poll when the hook is absent — never both)",
KeptIn: "the bus", Others: "—", Subjects: []string{"mesh.mod.*.event.pull.merged"}, Writes: ownModule},
// A pull request's head, before it merges (novox/hq to-be 45 §9): the same one announcer.
{State: "a pull request's head announced", Writer: "the forge's announcer, the one that announces its merges",
KeptIn: "the bus", Others: "the controller asks the build seat to check it", Subjects: []string{"mesh.mod.*.event.pull.updated"},
Writes: ownModule},
{State: "a pull request's merge check", Writer: "the build seat's holder that ran it, said as the controller's `checked`",
KeptIn: "the bus", Others: "the forge's holder sets it as the pull request's status"},
{State: "a provider's standing", Writer: "the provider", KeptIn: "the provider's events",
Others: "the controller keeps the newest word as a condition",
Subjects: []string{"mesh.mod.*.event.provisioner.failing", "mesh.mod.*.event.provisioner.recovered",
"mesh.mod.*.event.provisioner.retirement"},
Writes: ownModule},
{State: "the operator-channel's open messages", Writer: "the seat's holder", KeptIn: "its own key-value state",
Others: "—"},
{State: "the facts snapshot", Writer: "controller", KeptIn: "the artifact store, facts/latest",
Others: "the build seat reads"},
// What a consumer gave up on (novox/hq issue 330): copied by the controller from the server's notice,
// and delivered again or dropped only through its verb, with why.
{State: "a message a consumer gave up on", Writer: "controller", KeptIn: "the bus, the stream " + DeadLettersStream,
Others: "read, delivered again or dropped through the controller's dead-letters verb",
Subjects: []string{deadLetterPrefix + ">", againPrefix + ">"}, Writes: isController},
// The record mirrors (novox/hq ADR 0289 §9, design 54 §2): rows of this table, so a grant that lets
// anything else write them is refused at composition. Built so far: the assignments and each registered
// module's declared config (design 57 F3), and the list of predefined keys kept (design 57 K3).
{State: "the assignments mirror", Writer: "controller (lease holder): the verbs that assign, and the sends that record what runs; " +
"a mirror of its store only until the assignments are moved to the bus (novox/hq ADR 0408)",
KeptIn: "key-value " + AssignmentsBucket + ", the record once moved", Others: "read through `reads` as mesh-controller." + MirrorAssignments,
Subjects: kvOf(AssignmentsBucket), Writes: isController},
// The chain (novox/hq ADR 0408): builds and rollouts beside the assignments above, which keep their bucket. Not
// mirrors of the store but the record, written by the lease holder by compare-and-set and removed by nobody.
{State: "the builds of the chain", Writer: "controller (lease holder), compare-and-set on the key's revision",
KeptIn: "key-value " + BuildsBucket + ", sixty-four revisions of each key, none removed",
Others: "read through `reads` as mesh-controller." + ChainBuilds, Subjects: kvOf(BuildsBucket), Writes: isController},
{State: "the rollouts of the chain", Writer: "controller (lease holder), compare-and-set on the key's revision",
KeptIn: "key-value " + RolloutsBucket + ", sixty-four revisions of each key, none removed",
Others: "read through `reads` as mesh-controller." + ChainRollouts, Subjects: kvOf(RolloutsBucket), Writes: isController},
// The grants issued (novox/hq ADR 0528 clause 2, issue 1394): the controller's, read by the assigner for its sixth rule.
{State: "the grants issued", Writer: "controller (lease holder), once the bus's user list carrying a module's manifest at a fingerprint was sent",
KeptIn: "key-value " + GrantsBucket + ", one key a module's fingerprint, none removed",
Others: "read through `reads` as mesh-controller." + ControllerGrants, Subjects: kvOf(GrantsBucket), Writes: isController},
// What each machine has of each module (novox/hq ADR 0429 §1, design 47): written by that machine's node-engine under
// its own node's keys, never by the controller; the view and the watchman read it.
{State: "a machine's have", Writer: "the node-engine of the machine, under its own keys `<node>.>`",
KeptIn: "key-value " + delivery.HaveBucket + ", twenty revisions of each key",
Others: "read through `reads` as node-engine.have", Subjects: kvOf(delivery.HaveBucket), Writes: ownNodeKeys},
// The contents of a declaration's files over 64 KiB (novox/hq ADR 0486 §2): the controller alone writes the bucket, each
// object under the digest of its content; a node-engine reads one by its name, with no list and no write.
{State: "a declaration's large files", Writer: "controller, each object under the sha256 digest of its content; the " +
"operator's account on release 6's hand path alone (ADR 0539 clause 10)",
KeptIn: "the object bucket " + DeclarationFilesBucket + "-<node> of each node, an object removed a day after the " +
"node's latest declaration no longer names it",
Others: "a node-engine reads an object of its own bucket by its name and may neither list the bucket nor write it",
Subjects: []string{"$O.*.>"}, Writes: func(p Principal, pattern string) bool {
return isControllerOrTheHandPath(p, pattern) || grantsOwnObjects(p, pattern)
},
Shared: "the grants part writes its own object bucket " + GrantsUserListObjects + ", the user list sealed to the bus node, and no other (novox/hq ADR 0547 clause 7)"},
{State: "the declared-config mirror", Writer: "controller (lease holder), from its store, only on a change",
KeptIn: "key-value " + DeclaredConfigBucket, Others: "read through `reads` as mesh-controller." + MirrorDeclaredConfig,
Subjects: kvOf(DeclaredConfigBucket), Writes: isController},
{State: "the predefined-keys mirror", Writer: "controller (lease holder), from its store, only on a change",
KeptIn: "key-value " + PredefinedBucket, Others: "read through `reads` as mesh-controller." + MirrorPredefined,
Subjects: kvOf(PredefinedBucket), Writes: isController},
{State: "the declared-packages mirror", Writer: "controller (lease holder), from the composition of each node, only on a change",
KeptIn: "key-value " + DeclaredPackagesBucket, Others: "read through `reads` as mesh-controller." + MirrorDeclaredPackages,
Subjects: kvOf(DeclaredPackagesBucket), Writes: isController},
// What a machine's provisioner client placed of a module's files (novox/hq ADR 0349 §4, design 57 P6): its own
// machine's word alone, which a walk's end and a gate wait for (ADR 0362, ADR 0349 §15). Kept newest per module
// in the stream ProvAppliedStream, and by the client in the provisioner server's `applied` bucket.
{State: "a module's files placed on a machine", Writer: "that machine's provisioner client",
KeptIn: "the bus, the stream " + ProvAppliedStream + ", newest per module; and key-value " +
ProvisionerAppliedBucket,
Others: "the controller reads it at a walk's end and a gate; a failure is that machine's failed apply",
Subjects: []string{"prov.applied.*.>"}, Writes: ownProvisioner},
}
// CheckWriters refuses a grant that lets a principal publish on a subject the writers table gives
// another writer (to-be 45 §1): which state, whose, and the pattern that would make a second writer.
func CheckWriters(p Principal, publish []string) error {
var problems []string
for _, pattern := range publish {
for _, row := range WritersTable {
if row.Writes == nil {
continue
}
for _, subject := range row.Subjects {
if !SubjectsOverlap(pattern, subject) || row.Writes(p, pattern) {
continue
}
problems = append(problems, fmt.Sprintf("%s may publish %s, which writes %s (%s), whose writer is %s",
p.Username(), pattern, row.State, subject, row.Writer))
}
}
}
if len(problems) == 0 {
return nil
}
return fmt.Errorf("a second writer would be granted (novox/hq to-be 45 §1, one writer per piece of state):\n %s",
strings.Join(problems, "\n "))
}
// SubjectsOverlap says some subject matches both patterns: `*` is one token, `>` one or more to the end.
func SubjectsOverlap(a, b string) bool {
x, y := strings.Split(a, "."), strings.Split(b, ".")
for i := 0; ; i++ {
switch {
case i == len(x) && i == len(y):
return true
case i == len(x) || i == len(y):
return false
case x[i] == ">" || y[i] == ">":
return true
case x[i] == "*" || y[i] == "*" || x[i] == y[i]:
continue
default:
return false
}
}
}
// isControllerOrTheHandPath is the controller, or the operator's account (a person who may invoke every tool) writing
// release 6's hand path: the one second writer ADR 0539 clauses 2 and 10 name, for a composition made by hand.
func isControllerOrTheHandPath(p Principal, pattern string) bool {
return isController(p, pattern) || (p.Kind == KindPerson && administers(p.Invokes))
}
+13
View File
@@ -445,6 +445,19 @@ func TestAnUnsetRequiredValueIsRefusedInTheConfigFormsWords(t *testing.T) {
t.Fatalf("not typed as a value nobody gave: %v", err)
}
// A contribution's value too (issue 1126, seed-8: de-spiegel's route names were refused with ${setting:…}).
withRoute := inConfigForm()
withRoute["contributes"] = map[string]any{"route": map[string]any{"site": map[string]any{"name": "${config:sender}", "port": 80}}}
routed, err := parseRaw(t, withRoute)
if err != nil {
t.Fatal(err)
}
_, err = Resolution{}.composed(routed, "route", routed.ContributesMany["route"]["site"], nil, "notifier contributing site to route")
if !errors.As(err, &unset) || unset.Module != "notifier" || !strings.Contains(err.Error(),
`notifier has a contribution site to route, in "name", that says ${config:sender}`) || strings.Contains(err.Error(), "${setting:") {
t.Fatalf("a contribution's unset value is not said in the config form's words: %v", err)
}
settingsForm := notifier()
settingsForm.Resources[0]["content"] = "from = ${setting:domain}\n"
err = unsetInConfigWords(settingInto(settingsForm.Resources[0], nil, "notifier"), settingsForm)
+20 -1
View File
@@ -10,6 +10,7 @@ package catalogue
import (
"encoding/json"
"errors"
"fmt"
"sort"
"strconv"
@@ -138,6 +139,9 @@ type Rendering struct {
// Kept is every operator-sealed secret in the mesh, for a module that `keeps` them. Nil when
// nothing on this node keeps them, or the mesh has no operator key.
Kept *KeptExport
// KeptRuns is, by module, the file of the export as the node runs it, for a composer that holds no export (a node
// composing its own: the export is the controller's store's, issue 1126, C6): carried as it runs when Kept is nil.
KeptRuns map[string]map[string]any
// OutwardLinks is the links this machine reported as facing outside it, which the filter is
// written around (novox/hq ADR 0140). Empty means the machine has not said, and the mesh
@@ -929,6 +933,14 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
}
// Read by the module as its secrets' owner when it names one: the vault, as its own account (issue 723).
first = append(first, ownedBy(m.SecretsOwner, file))
} else if runs, ok := with.KeptRuns[m.Module]; m.Keeps != "" && ok {
file := make(map[string]any, len(runs))
for k, v := range runs {
file[k] = v
}
file["id"] = KeptID()
delete(file, "module")
first = append(first, file)
}
if m.Computed != "" {
mine, part := generated[m.Module]
@@ -1821,7 +1833,14 @@ func (r Resolution) composed(m Manifest, to string, raw map[string]any, layers [
// module's files is no part of it (novox/hq 04-ISSUES/173).
values, err := overridden(raw, WithDefaults(m, layers), what)
if err != nil {
return nil, fmt.Errorf("%s: %w", what, err)
// A required value nothing sets is said in the words of the module's form, as a file's, a unit's and an owner's
// are (issue 1126, seed-8: de-spiegel in the config form was refused with ${setting:…} and `settings set`).
var unset *UnsetSettingError
if errors.As(err, &unset) && unset.Module == "" {
unset.Module = m.Module
unset.where = fmt.Sprintf("a contribution%s, in %q,", strings.TrimPrefix(what, m.Module+" contributing"), unset.field)
}
return nil, unsetInConfigWords(fmt.Errorf("%s: %w", what, err), m)
}
reaches, err := Reaches(m, layers)
if err != nil {
+13
View File
@@ -75,6 +75,19 @@ func deliveryVerbs() []Verb {
Input: schema(map[string]string{"id": "the delivery's id", "why": "the condition it answers"},
[]string{"id", "why"})},
deliveryRemovedInR6("retire-history"),
{Name: "settle", Description: "Ask the operator to settle deliveries held because their merge reached the " +
"trunk without a passing check: one ask on the operator's channel for all of them, as an Approve is asked. " +
"The verb moves nothing; on the operator's answer each goes to merged, and whether it is on the machines " +
"is derived as for any merge (novox/hq ADR 0556). Refused for a delivery not held, or held for another " +
"reason, naming why.",
Input: map[string]any{"type": "object", "additionalProperties": false, "required": []string{"ids", "why"},
"properties": map[string]any{
"ids": map[string]any{"type": "array", "items": map[string]any{"type": "string"}, "minItems": 1,
"description": "the held deliveries' ids (owner/repository@commit)"},
"why": map[string]any{"type": "string", "description": "why they may be settled, shown to the operator"},
}},
// Added after the seat's release 6 holder shipped: optional until the gate serves it everywhere.
Optional: true},
}
}

Some files were not shown because too many files have changed in this diff Show More