Compare commits

..
Author SHA1 Message Date
jschoubben 173a7654ba The assigner, the indexer and the grants part tell their followers when the bus made their consumer (issue 1966, recovery)
mesh/delivery ready: it is merged on the Approve
mesh/check failed (5): unit: git.novox.be/novox/mesh-sdk/go/catalogue.TestTheResolverForwardsToFixedUpstrea…; and 4 more | 173a7654ba61 on 4a59c17c6886
mesh/delivery-group group fix/1966-views-durable rejected: composed together: the checker's verdict at the members' heads (603dd79ad282): unit: git.novox.be/no…
A consumer is made from now, so an event said between a first view and the consumer is never delivered; the hub says when the bus made it (EventHub.OnBound) and each follower asks again if behind (Follower.ConsumerMade).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 02:27:38 +01:00
mesh-tools 4a59c17c68 Merge pull request 'The assigner reads the seat list, so a proposal no longer refuses a replicated seat (issue 1966, recovery)' (#183) from fix/1966-assigner-verbs-seat-list into main 2026-11-08 23:39:24 +00:00
jschoubben e5f782cc07 The assigner's process and its seat verbs read the seat list from the seat mesh-catalog (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-delivery@4a59c17c6886
mesh/check checking e5f782cc07d4 on 4a59c17c6886
Neither runtime of the assigner gave catalogue a seat list, so propose judged
every mesh seat as held once (mesh-dns-resolver, replicated, refused every
proposal) and seat-holders marked every seat outside the list. Both now follow
the list as the grants part does (ADR 0636 §1, §4, §7): a proposal laid again
is not ended and seat-holders is not written until the list is read, and the
two seat list tools read and read it again.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-09 00:34:32 +01:00
jschoubben d1a6950071 The assigner's check says a seat the unread seat list does not name waits for the list, never held once (issue 1966)
A mesh seat held on two nodes was refused as held once whenever catalogue did
not name it, and a host that has not read the seat list holds only the
bootstrapped definitions: mesh-dns-resolver, a replicated seat, refused every
proposal (ADR 0636 §1).

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-09 00:31:27 +01:00
mesh-tools 4f9b15a43f Merge pull request 'Grants purges old sealed lists' keys, and once at its start the ones a delete left behind (issue 1966, recovery)' (#182) from fix/1966-user-list-purge into main 2026-11-08 23:22:50 +00:00
jschoubben 126b40e6f5 Grants purges, once at its start, the sealed keys a delete left behind (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-delivery@4f9b15a43f05
mesh/check checking 126b40e6f585 on 4f9b15a43f05
The markers and the values of the lists taken away before the purge still fill
the state user-list. Before its first pass the process purges every
sealed-grants key whose last operation is a delete or purge marker, and every
user's key of a list the index of kept lists does not name; never a kept
list's key nor another key of the state, and nothing when the index cannot be
read. It logs the counts and the state's bytes before and after.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-09 00:22:00 +01:00
jschoubben de320684f2 Grants purges the old sealed lists' keys instead of deleting them (issue 1966)
A key-value delete only adds a marker: every value of the per-user keys of the
lists older than the two kept stayed in the state user-list's stream, which
reached 94% of its 64 MiB cap on 2026-11-08 (5,968 keys, 13,228 values). At the
cap the bus refuses grants' next write of the sealed list and deliveries lock.
The part now takes those keys away through a purger on the process's own
JetStream connection, a publish to the key's own subject the account already
writes on; the tools, with no purger, still delete.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-09 00:21:49 +01:00
mesh-tools fa8258af4e Merge pull request 'Grants creates a one-filter consumer on the plain create subject it is granted (issue 1966, recovery)' (#181) from fix/grants-consumer-plain-create into main 2026-11-08 22:57:13 +00:00
jschoubben 56018993e0 Grants creates a one-filter consumer on the plain create subject it is granted
mesh/delivery merged: source.main moved to novox/mesh-delivery@fa8258af4e6d
mesh/check checking 56018993e0d9 on fa8258af4e6d
With exactly one filter in FilterSubject, nats.go creates on $JS.API.CONSUMER.CREATE.<stream>.<name>.<filter>,
a subject the grants part is never granted (CreateConsumerGrants keeps filters out of grants), so the creates
of ASKS.novox_mesh-delivery-indexer, ASKS.shanks_mesh-delivery-builder and FORGE.shanks_mesh-delivery-builder
were refused and timed out (2026-11-08, issue 1966). Every filter now goes in FilterSubjects, which the
client creates on the plain subject.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 23:56:58 +01:00
mesh-tools 7d8e2060a7 Merge pull request 'Grants composes from the haves while the wants view is unanswered, and its conditions reach the operator (issue 1966, recovery)' (#180) from fix/1966-first-view-one-read into main 2026-11-08 22:31:58 +00:00
jschoubben 160fca6a47 Say in grants' log how long each first-view page took (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-delivery@7d8e2060a740
mesh/check checking 160fca6a4767 on 7d8e2060a740
The wants page failed 226 times on a call's bound with nothing in the log to
show how far from it the answer was.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 23:30:28 +01:00
jschoubben e79a546eb7 Compose what runs from the haves while the wants view is unanswered (issue 1966, recovery)
Since 16:39 the running assigner cannot answer its wants page within a call's
bound, and the failed wants read held the whole user list, so no new build got
its credential. As the assignments view already does, an unanswered wants view
now composes what runs from each have's fingerprint, is said as a gap that is no
failed read, and keeps the pass to one that only adds (keepsLoaded), so the list
is held only where it would take from a loaded user.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 23:29:26 +01:00
jschoubben e7779d2afd Open every grants condition's explanation as the operator-channel requires (issue 1966, recovery)
The channel refuses a raised condition whose explanation does not open with
"Needs you: " and its needs, so seven of grants' conditions, among them
first-view-unanswered, never reached the operator. One helper now writes both,
and every condition the unit tests say passes the channel's own check.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 23:27:39 +01:00
mesh-tools 3f96301177 Merge pull request 'Grant the bus keeper its account's information, which every bucket create asks first (recovery)' (#179) from fix/keeper-account-info into main 2026-11-08 22:07:15 +00:00
jschoubben 66f31c8e97 Grant the bus keeper its account's information, which every bucket create asks first
mesh/delivery merged: source.main moved to novox/mesh-delivery@3f96301177b1
mesh/check checking 66f31c8e97a6 on f9cadbfdbb93
The keeper on novox (nats fea2c02b) has been refused $JS.API.INFO every 30 s since
21:58 UTC: the client library's CreateKeyValue asks the account's information before
it creates a bucket, so each create of KV_distribution_* and KV_gitea_* timed out and
the keeper never finished a fill pass (health failed). Read-only: the account's limits
and usage, never a stream's data.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 23:07:00 +01:00
mesh-tools f9cadbfdbb Merge pull request 'Grants compose each membership from the build that runs as well as the trunk's, so a running build keeps its reads (issue 1966, recovery)' (#178) from fix/1966-membership-keeps-running-reads into main 2026-11-08 16:03:19 +00:00
jschoubben 5452fa92a7 Grants keep the last published membership when the running build cannot be read; holders stay as merged (issue 1966, review round 2)
mesh/delivery merged: source.main moved to novox/mesh-delivery@f9cadbfdbb93
mesh/check failed (2): integration: mesh-delivery-assigner/cmd/mesh-delivery-assigner.TestAProposalIsAnswer…; and 1 more | 5452fa92a731 on f9cadbfdbb93
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 17:02:17 +01:00
jschoubben 563a624a6d Grants read the have's build for memberships and holdings, and a failed read of it holds the pass back (issue 1966, review round 1)
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 17:00:15 +01:00
jschoubben 77862abb12 Grants compose a membership from the build each machine runs, the want's and the trunk's, so none loses a read (issue 1966)
A running build older than the trunk lost reads it still uses (the vault's read of the assigner's placement), failing every node-engine pass. Memberships now come from the union of the have's build, the want's and the trunk's manifest.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 16:57:13 +01:00
mesh-tools 37e7ea3794 Merge pull request 'Grants compose holders through the SDK's SeatsToPassOn, so mesh-catalog is granted to its only claimant (ADR 0660, issue 1966, recovery)' (#177) from fix/1966-seat-pass-shared into main 2026-11-08 14:47:19 +00:00
jschoubben d88c0f269d Grants test: compare the record against a deep copy, so a written holding fails it (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-delivery@37e7ea379429
mesh/delivery-group group fix/1966-seat-pass-shared merged: every member is merged
mesh/check checking d88c0f269d5f on f9cadbfdbb93 | novox/mesh-delivery#177 in group fix/1966-seat-pass-shared
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 15:44:19 +01:00
jschoubben bddb6b1eb3 Grants compose the holders on record through mesh-sdk's SeatsToPassOn, so a seat whose holder's manifest left it is granted to its only claimant (issue 1966, ADR 0660)
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 15:35:26 +01:00
mesh-tools 9c6bf236b6 Merge pull request 'The indexer writes no module whose bytes it already holds (issue 1966, recovery)' (#175) from fix/1966-indexer-noop-writes into main 2026-11-08 12:02:19 +00:00
mesh-tools 8f57df2d77 Merge pull request 'The assigner's verbs and the checker's tools ask the indexer's tools instead of reading its state (issue 1966, recovery)' (#176) from fix/1966-assigner-seat-reads into main 2026-11-08 12:01:49 +00:00
jschoubben 0a82e33518 The checker's tools and the assigner's put ask the indexer's tools instead of reading its state (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-delivery@8f57df2d7757
mesh/check failed (3): mesh-build-go: the go.mod of novox/mesh-sdk at 396511b7a276, beside the change, cann…; and 2 more | 0a82e3351804 on 9c6bf236b6b1
The second instance of the kind: checker_status and checker_pull read the indexer's
candidates and modules from its bus state, which the checker's manifest does not
declare under reads, so the runtime refuses them ("keeps and reads no state
mesh-delivery-indexer.candidates"). They now ask indexer_read_candidates and
indexer_read_modules, the indexer's read tools the manifest already invokes: the
keys with no argument, one key's value with key (ADR 0596). The assigner's put
read its newest fingerprint from the same refused state and now asks the
indexer's first view of modules, as the seat verbs do. A sweep of every module
finds no other read of another module's state that its manifest does not declare;
the checker's tools carry a test that refuses one.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 11:58:42 +01:00
jschoubben 0c4660eea8 The assigner's seat verbs ask the indexer's first views instead of reading its state (issue 1966, recovery)
mesh-module-assigner.propose and mesh-wants' upgrade and bus read the indexer's
modules and manifests straight from its bus state, which the runtime refuses
since those reads moved to the indexer's first-view tools (ADR 0615 §1, ADR 0621):
"mesh-delivery-assigner keeps and reads no state called mesh-delivery-indexer.modules".
Another module's data is asked through its tools, never read from its state
(ADR 0596). The verbs now ask mesh-delivery-indexer.modules and .manifests through
the tool runner, every page, at each call; the manifest already invokes both and
the indexer answers them to the assigner.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 11:54:37 +01:00
jschoubben 00232c7197 The indexer writes no module whose bytes it already holds (issue 1966, recovery)
mesh/delivery ready: it is merged on the Approve
mesh/check failed (3): mesh-build-go: the go.mod of novox/mesh-sdk at 396511b7a276, beside the change, cann…; and 2 more | 00232c71977b on 9c6bf236b6b1
Every recompute wrote every key of its modules state again with identical bytes: recomputeLocked
compared the package versions with reflect.DeepEqual across two map types (the contract's
map[string]string and sdkstep.PackageVersions), which never match. Each write was a views
event, and a reader of the views took each as a gap and asked the whole first view again.

The comparison is by maps.Equal now, and write itself skips an answer whose bytes are the
bytes held, for every caller and every state it writes; the merge's written list counts only
real writes.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 11:50:52 +01:00
mesh-tools 7f41218552 Merge pull request 'A bootstrapped seat counts as read by every node-engine, and node users get the seat list (issue 1966, recovery)' (#174) from fix/1966-seat-reads-bootstrap into main 2026-11-08 03:45:01 +00:00
jschoubben ee6fd92b4d The assigner and the indexer's hold count a bootstrapped seat as read (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-delivery@7f412185525e
mesh/delivery-group group fix/1966-seat-reads-bootstrap merged: every member is merged
mesh/check failed (2): mesh-delivery-checker: no published of the tagged version of go-sdk of mesh-delivery…; and 1 more | ee6fd92b4d1a on 7f412185525e
Every reader of the seat list holds the bootstrapped seats from its start
(ADR 0636 §2), so a manifest naming one is read by every node-engine
whatever its statement names. graph.MissingReads, in the Delivery domain's
SDK, is the one rule both holds judge with; mesh-sdk is not changed.

ace's engine states no seat, so the want of the node-engine's own build
0bd8cbad (needing mesh-catalog) and of restic (needing node-backup) were
held on ace: the engine that would state them could not be delivered. The
indexer's hold judges only mesh-wide reads today, so its verdicts do not
change; it takes the rule so the two stay one.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 03:53:11 +01:00
jschoubben ffcce927fe Each node's account calls the engine's invokes and hears its seat events on the seat (issue 1966)
The grants part built the node-engine's declaration with its consumes alone,
so a node's account was granted none of the engine manifest's invokes: ace's
engine could not publish mesh.seat.mesh-catalog.call.seat-definitions.node~ace,
read no seat list and composed nothing new. And its consumes of
mesh-catalog's seat-definition events were granted on the emitter's subject
only, where the engine subscribes the seat's (ADR 0636 §4).

The engine's declaration now carries its invokes and the seats it watches,
split by the same consumedOf a placed module's consumes are; the node's
account is granted the invokes as a module's are (in its own caller token
by callerNamed) and the watched events on each seat's subject by the same
watchedEvents a module's use. Its consumes stay granted on the emitter's
subjects too, so nothing granted before is taken away.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 03:50:03 +01:00
mesh-tools 68581bed7e Merge pull request 'The indexer and the checker know a Go module the trunk holds a record of (issue 1966, bootstrap recovery)' (#173) from fix/1966-known-packages-from-trunk into main 2026-11-08 00:25:48 +00:00
jschoubben 2e9f23f348 Round 1: known by the module path alone, never a package inside it
mesh/delivery merged: source.main moved to novox/mesh-delivery@68581bed7e36
mesh/check failed: mesh-build-go: the change's manifests cannot be judged: the seat list is not read (the seat list is … | 2e9f23f34828 on 68581bed7e36
The review of 0f5c186b: counting each record's packages made 94 sub-package
paths, commands among them, known names, which design 32 forbids; the seven
names refused live are each their record's module (issue 1966).

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 01:24:54 +01:00
jschoubben 0f5c186b9c The judge knows the Go packages the trunk holds a record of (issue 1966, recovery)
The indexer's judge refused manifests whose `packages` name a Go package that no
manifest declares as a package artifact: such a package has no series key, and
knownPackages took series keys only, though each module the trunk published has
a per-module record in the packages state naming its Go module and packages.
knownPackages (and the checker's PackageNames) now also count each record's
module and packages[].package; an unreadable state still judges no name.

Quick vs thorough: this is the quick fix, the smallest change that lets the
trunk's own records name what the manifests name. The thorough fix is that a
build module builds Go package artifacts, so every Go package gets a series
(the registry milestone); then the records stop being a source of names. Done
now because the 15 refused manifests hold the bootstrap of the registry delivery.
2026-11-08 01:14:52 +01:00
mesh-tools 838dbd59ed Merge pull request 'mesh-grants: a pass composing part of the mesh only grows a membership (issue 1966, bootstrap recovery)' (#172) from fix/1966-grants-partial-pass-grows into main 2026-11-07 23:39:26 +00:00
jschoubben f3f0a5e695 mesh-grants: a pass composing part of the mesh publishes the union of the memberships, so it only grows (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-delivery@838dbd59ed5d
Fault: the new assigner composes a membership that drops the placement seats and adds its state view-marks. The partial pass
kept the published one whole, so the assigner's tools never got view-marks, its assignments view never answered, and the
pass never left its partial state (third bootstrap loop of the kind after #170 and #171).

Kind: the bootstrap hold freezes whole entries, so a grant needed to end the hold is never given.
Quick fix: special-case the assigner (or let additions through for it). Rejected: the next module that gains a state meets the same loop.
Thorough fix: while a pass keeps what the bus loaded, publish the union of the membership last published and the one composed
(every list, deduplicated, published entries first; the private network, contributions and bindings the pass's), published
when it differs; the pass says what it kept beyond the composed one. The user list already unions (KeepLoaded), additions pass there.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 00:33:24 +01:00
mesh-tools cd7f85c74a Merge pull request 'mesh-grants keeps what the bus loaded while it composes only part of the mesh (issue 1966, bootstrap recovery)' (#171) from fix/1966-grants-keeps-what-is-loaded into main 2026-11-07 22:35:23 +00:00
jschoubben ab2bbe9ff6 mesh-grants: a partial pass keeps every deny and every published membership
mesh/delivery merged: source.main moved to novox/mesh-delivery@cd7f85c74a75
Review round 1 of issue 1966's keep-what-is-loaded fix.

- KeepLoaded intersected the denies, and the composer gives every composed user an empty
  deny, so a user in force with a wide publish and a deny was sealed with no deny. The
  union of every deny any list says is kept now, and Losses counts a deny dropped.
- A pass composing part of the mesh (the seat list unread, or placed by the haves) no
  longer narrows a published membership: one whose published value serves, hears or gives
  more than the pass composes is kept as published and said so (not_published), so a tool
  runner or a seat holder goes on serving what it serves while keeping its login.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-07 23:33:07 +01:00
jschoubben dead64ee6e mesh-grants: a pass composing part of the mesh keeps what the bus loaded
Issue 1966: the list sealed 2026-11-07 23:09, composed while the seat list was unread
(only the bootstrapped seats compose), took the node seats from the tool runners, the
vault's reads from the node-engines and the seat queues from the build modules, and left
out ledger.novox.mesh-knowledge.

- While the seat list is unread (the gate's MayAct with no seat) or a pass is placed by
  the haves, each user of the loaded list and of the list in force is sealed with the
  union of what the pass composes and what those lists give it (publish, subscribe,
  responses; a deny only where every list saying one denies it), kept the way a refused
  principal is; the users so kept are the pass's kept_loaded.
- In such a pass a list that still takes a permission from, or leaves out, a loaded user
  is held (user-list-held), naming each user and how many permissions it loses.
- A held module keeps its ledger account, ledger.<node>.<module>, with its other users:
  heldModuleUsers matched <node>.<module>[.<process>] only, which dropped the ledger user.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-07 23:28:17 +01:00
mesh-tools dcc48b82d0 Merge pull request 'mesh-grants places modules by the node-engines' haves while the assigner's assignments view does not answer (issue 1966, bootstrap recovery)' (#170) from fix/1966-grants-places-by-haves into main 2026-11-07 22:05:11 +00:00
jschoubben 69b6b89af5 mesh-grants: the bootstrap row proves a removed have places nothing
mesh/delivery merged: source.main moved to novox/mesh-delivery@dcc48b82d0cb
Review round 1 of issue 1966's fix: the membership check alone did not pin the removed
phase; the row now reads the inputs and requires mod-y off b while its have is removed.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-07 22:51:35 +01:00
jschoubben ffaff3d3aa mesh-grants: prove the haves fallback lets the indexer serve the seat list's verb
Issue 1966: the seat list is read through mesh-catalog's seat-definitions, which got no
responders only because nothing was placed. A row shows that, the assignments view
unanswered, the indexer placed by its have (its real manifest) is composed to hear
mesh.seat.mesh-catalog.* and its membership serving seat-definitions is published, while
the seat list stays unread and other modules still wait for it.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-07 22:40:31 +01:00
jschoubben 9756dd6f1f mesh-grants: place by the node-engines' haves while the assignments view does not answer
Bootstrap only (issue 1966, notes 63 and 64; the operator chose it 2026-11-07 22:19,
bending ADR 0567 §9 for the bootstrap). An assigner that serves no first views answers
no assignments, so the grants part placed nothing and wrote no grant, membership or
bucket for anything, not even for the assigner that would serve the view.

When the assignments read fails and the haves read, each module is placed on the node
whose have says it runs (any phase but removed), the bus node found among them, and what
runs composed from each have's fingerprint instead of the assigner's wants. Said as a gap
naming the fallback, and kept as Inputs.ByHaves: such a pass retires no account and deletes
no withheld hearing (ReadFailures), and seals no list that drops a user of the list in
force or of the loaded list (holdsBack). Once the view answers, the fallback is unused.

Quick fix; the thorough one stays the module-graph edge (the assigner's views before the
grants part that reads them) in the delivery plan.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-07 22:37:56 +01:00
mesh-tools b73315ffe0 Merge pull request 'Record today's forge merges in the merge gate (issue 1966, bootstrap)' (#169) from fix/1966-gate-record-delivery into main 2026-11-07 20:21:43 +00:00
jschoubben 38d86366bf Record today's forge merges in the merge gate: an empty change merged through the gate, so the forge bridge can build the merged message for #166, #167 and #168 (issue 1966, the operator's bootstrap)
mesh/delivery merged: source.main moved to novox/mesh-delivery@b73315ffe05c
mesh/check failed: shanks: the check of novox/mesh-delivery#169 failed: downloading the asked version 08580d25d2dc49900… | 38d86366bf3a on b73315ffe05c
2026-11-07 21:12:57 +01:00
jschoubben a37ee99b41 Merge pull request 'The indexer publishes manifest-accepted and manifest-refused as events, without waiting for the stream's acknowledgement (issue 1966)' (#168) from fix/1966-indexer-event-refused into main
Reviewed-on: #168
2026-11-07 19:58:28 +00:00
jschoubben 71a56d4606 The indexer publishes manifest-accepted and manifest-refused as events, without waiting for the stream's acknowledgement (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-delivery@a37ee99b41d5
mesh/check pass 71a56d4606c9 on b73315ffe05c | carried from a37ee99b41d5 | unit 495 run, 947 kept | integration 494 run, 1014 kept | composition passe…
Both were published with the JetStream acknowledgement awaited, a request and response, for ten seconds each, and
manifest-accepted once more on failure. On a bus refusing the subject (the old grants do not grant the new indexer
mesh.seat.mesh-catalog.event.manifest-accepted) no acknowledgement ever came, so each judging pass took about twenty
seconds a manifest and the indexer never reached the forge's messages nor the seat definitions: the take-in stood
still. Now both are published plainly on their subject with the dedup id in Nats-Msg-Id, as an event is (the
operator, 2026-11-07: everything is event-driven); the stream that keeps the subject drops a duplicate by its id, and a
refusal is said on the connection's error handler, as the journal already shows. An error of the publish itself (no
connection) still goes to the caller, which retries once and writes the unsaid finding.

Quick vs thorough: thorough for these two events (every publish of a seat event by the indexer is now an event);
sayTakeIn keeps its acknowledged publish, said in the issue for 47/G.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 20:57:02 +01:00
jschoubben 5013bb06e0 Merge pull request 'Put the indexer's old seatfiles module back for now, so the running indexer can take in mesh-delivery's merges (issue 1966)' (#167) from fix/1966-bootstrap-seatfiles into main
Reviewed-on: #167
2026-11-07 19:06:01 +00:00
jschoubben e01d265dec Put the indexer's old seatfiles module back for now, so the running indexer can take in mesh-delivery's merges (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-delivery@5013bb06e0aa
mesh/check pass e01d265decb3 on b73315ffe05c | carried from a37ee99b41d5 | unit 495 run, 947 kept | integration 494 run, 1014 kept | over bound: integ…
The running indexer (built at the registry merge) publishes every Go module of a repository it published before in
one forge listing; seatfiles, published at 6bc232eb and removed since, is not in the tree, so the listing fails and
every take-in of mesh-delivery is 'computed and not written' (journal 19:56:29: 'the directory
modules/mesh-delivery-indexer/seatfiles is not in novox/mesh-delivery'). mesh-delivery #165 fixed that in the new
indexer, which cannot run yet. The module comes back as it stood at 6bc232eb, imported by nothing, until the new
indexer runs; the follow-up removes it. The operator's bootstrap of 2026-11-07.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 20:00:26 +01:00
jschoubben 267b53635a Merge pull request 'Take the Go package artifacts out of the delivery parts for now, so the running builder builds them (issue 1966)' (#166) from fix/1966-bootstrap-no-packages into main
Reviewed-on: #166
2026-11-07 18:50:46 +00:00
jschoubben e0f510facb Take the Go package artifacts out of the delivery parts for now, so the running builder builds them (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-delivery@267b53635a1e
mesh/delivery-group group fix/1966-bootstrap-no-packages merged: every member is merged
mesh/check checking e0f510facb17 on b73315ffe05c | novox/mesh-delivery#166 in group fix/1966-bootstrap-no-packages
The running builder predates Go package artifacts ("refused kind: package of language go ... no build module
builds it"), so it can build neither the new builder, the grants module, the Go build module nor the checker:
each new part is needed before the others can be built or delivered. For the bootstrap of the registry
delivery, at the operator's word (19:4x, "do the temporary change"), the package artifacts leave the manifests of
the parts that run; the contracts and SDK modules, which run on no node, keep theirs. They come back in one change
once the new builder runs. Quick vs thorough: quick; the thorough fix is 47/G's tier order (issue 1921 note 80).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 19:48:57 +01:00
mesh-tools 2e43922827 Merge pull request 'A module the tree no longer holds does not stop the others' packages being published (issue 1585)' (#165) from fix/1585-take-in-own-packages into main 2026-11-07 15:06:11 +00:00
jschoubben 3422e1a6f8 A module the tree no longer holds does not stop the others' packages being published (issue 1585)
mesh/delivery merged: source.main moved to novox/mesh-delivery@2e439228275a
mesh/check checking 3422e1a6f8fd on 2e439228275a
publish asked the forge for every published module of the repository in one call; when the merge removed one (seatfiles), the whole call failed and a revision's own new Go modules stayed unpublished, so the take-in waited for a version only its own take-in would publish. packagesAt now reads the modules one by one when the tree lacks a directory and leaves the missing one out, said.
2026-11-07 15:51:38 +01:00
mesh-tools 8ad2bf6951 Merge pull request 'Registry milestone: mesh-delivery (issue 1966)' (#151) from feat/registry-milestone into main 2026-11-07 14:19:28 +00:00
jschoubben 4735fc0b5f Merge main (#164, a check composes only after the change's own builds end) into the registry milestone
mesh/delivery merged: source.main moved to novox/mesh-delivery@8ad2bf6951d5
mesh/delivery-group group feat/registry-milestone merged: every member is merged
mesh/check checking 4735fc0b5fdb on 8ad2bf6951d5 | novox/mesh-delivery#151 in group feat/registry-milestone
Conflicts in the checker's judge.go, builds.go, composition_test.go and try_candidates.go:
the group's contract types and kept composition path stay, #164's behaviour is taken whole.
WaitsOnPreview reads the contracts' meshcatalogclient.NoneNoPreview instead of a copied
constant; awaitBuilds names its failures failedBuilds, beside the builds package.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 14:36:42 +01:00
mesh-tools 6bc232eb78 Merge pull request 'A pull request's check composes only after the change's own builds end (issue 1921)' (#164) from fix/1921-compose-waits-builds into main 2026-11-07 13:28:38 +00:00
jschoubben c0da09d81f A pull request's check without a previewed candidate says its composition was not judged, not its manifests (issue 1921, review note)
mesh/delivery merged: source.main moved to novox/mesh-delivery@6bc232eb78b9
mesh/check checking c0da09d81f6e on 6bc232eb78b9
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 14:21:15 +01:00
jschoubben 3f9acf0223 A check composes only after the candidate's builds it waits on have ended (issue 1921)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: novox: not composed on novox: mesh-delivery-checker: frombus: mesh-delivery-checker is wanted at src… | 3f9acf022325 on b719dc0ecf27
The indexer writes a pull request's candidate first from its pull message alone, every module's
fingerprint "no merge preview is computed", and the fingerprints only with the preview, on the same
heads. A check that began on that first write kept its case: it waited on no build, said nothing,
and the engines, reading the previewed candidate, refused the module as not yet built (mesh-catalog
#624, mesh-delivery #163). Such a check now takes the previewed candidate before it judges the
manifests, waits on the builds and composes, as a whole try already does.

A build not ended within the build-wait-bound is named and the change is not composed, never an
engine's refusal of a fingerprint it has no done build of read as the change's.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 12:48:11 +01:00
jschoubben 93e5160a82 Merge main (#163, the take-in waits end) into the registry milestone
#163's group-wait end, the retried start and its subtest are kept whole; its
names follow the registry's translation off mesh-sdk go/delivery (trunk,
takein, meshdeliveryclient), as the rest of pending.go already does.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 12:47:00 +01:00
mesh-tools b719dc0ecf Merge pull request 'Take-in waits end: a group whose members are all taken in ends itself, a failed start is asked again by the next event (issue 1585)' (#163) from fix/1585-waits-end into main 2026-11-07 11:42:23 +00:00
jschoubben 70cd624127 Ask the check again: its composition ran before the indexer's build b5163570 was done, which is now (issue 1585; the checker's skipped build wait is noted on issue 1921)
mesh/delivery merged: source.main moved to novox/mesh-delivery@b719dc0ecf27
mesh/check being carried to 70cd624127bf on b719dc0ecf27 | from bab05c66b3cb
2026-11-07 12:38:40 +01:00
jschoubben 0520c12619 Group-end probe is best effort and a failed start is asked again by the next event, with no timer (issue 1585, review round 1)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: novox: not composed on novox: mesh-delivery-indexer: frombus: mesh-delivery-indexer is wanted at src… | 0520c126195b on bab05c66b3cb
2026-11-07 12:29:40 +01:00
jschoubben a7aa836ff9 A group wait whose members are all taken in ends itself, and the take-in at the start is tried again (issue 1585) 2026-11-07 12:25:26 +01:00
jschoubben b965a96ab3 The checker's states test counts its eleventh state, compositions (issue 1921, milestone 47/G, the group check of c5d32074c945)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 08:09:32 +01:00
jschoubben 918d21b7da The checker keeps each composition request it sends a node, and a tool reads it (issue 1921, milestone 47/G, the lead's ruling after the group check of 8a80e8e86858)
mesh/delivery-group group feat/registry-milestone checking: 7 of 7 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 918d21b7dac5 on 6bc232eb78b9 | novox/mesh-delivery#151 in group feat/registry-milestone
The request sent to the composer is kept gzip-compressed in the state compositions (ttl 3 days) under the check's key and the node, and the tool checker_composition_requests reads one by check and node or lists a check's. A request that cannot be kept fails the node's composition, naming why.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 07:18:36 +01:00
jschoubben 5990468051 The checking program's seat answer is the judging view of a seat, with digests or without (issue 1921, milestone 47/G, the lead's ruling after the group check of 616c42434ac2)
mesh/delivery-group group feat/registry-milestone checking: 7 of 7 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check failed (4): unit: git.novox.be/novox/mesh-catalog/checks/gitignored.TestNoProgramABuildLeavesIsC…; and 3 more | 599046805156 on bab05c66b3cb
A group that changes mesh-vault's, node-launcher's or operator-channel's
definition would otherwise meet the step state's credential guard again; the
laid list goes only to the manifest validator and the composer, which read no
verb's or message's schema. The test digests the definitions whole, as the
reader holds them.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 04:39:57 +01:00
jschoubben ce70589c84 A checker that hands no seat digests gets the heads' definitions without their verbs' and messages' schemas (issue 1921, milestone 47/G, the group check of 616c42434ac2)
The build seat's step state keeps the checking program's answer and refuses a
value with a field named like a credential; schema property names such as
mesh-vault's verb input 'credentials' tripped it. Neither the manifest
validator nor the composer reads those schemas: the verdicts of the head's and
the trunk's validator are byte-identical over the group, and the answer is
99,784 bytes (was 171,088).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 04:36:37 +01:00
jschoubben 4a07e7a0ce The assigner's integration test puts its responders' interest on the bus before it asks them (issue 1921, milestone 47/G, the group check of 616c42434ac2)
A manifest-reads ask the assigner made on manifest-reads-changed before the
test's SUB reached the server was answered by nobody and, by design, never
asked again; a flush is the barrier after each of the test's subscriptions.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 04:27:37 +01:00
jschoubben 932e7ccf02 A checker that hands no seat digests gets the heads' definitions with only each seat's own description (issue 1921, milestone 47/G, review round 2)
mesh/delivery-group group feat/registry-milestone checking: 7 of 7 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check failed (3): ace: the check of novox/mesh-catalog#612 failed: the unit level: the candidate's mes…; and 2 more | 932e7ccf0255 on bab05c66b3cb
G's own check is made by the running B0 checker, which hands no digests, so the
program answers every head definition: 342 KB, over the 256 KiB a run's answer may
be. Without digests the descriptions of verbs, events and their schemas are left
out (171 KB for the group's heads); neither the manifest validator nor the composer
reads one, and the validator's verdicts over the group are the same either way.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 03:41:55 +01:00
jschoubben 112ca63f38 The checking program answers only the head seat definitions the reader does not hold as they are (issue 1921, milestone 47/G, review round 1 finding 3)
The whole heads' list is 342 KB for this group, over the 256 KiB a run's answer
may be. The checker hands the program its reader's seat-to-digest list (5.6 KB)
and lays the answer, the new or changed definitions, over its own list.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 03:17:56 +01:00
jschoubben df56af0765 Every check composes with the composer built at the node-engine member's head, else at its trunk, and a change's manifests carry the check's own judge's verdicts (issue 1921, milestone 47/G D12, D13)
The parked composition-at-running path is folded as the lead's ruling Q3: one
path, so no check composes with the running engines' own code (ADR 0656 §5).
The sandbox request no longer drops a change's manifest verdict: it carries the
check's judge's, or none, which the composer reads as accepted (ADR 0578 §3).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 03:00:35 +01:00
jschoubben 2159375873 Merge the trunk: B0's facts and checking program carry the registry's steps (issue 1921, milestone 47/G D3-D11)
B0's shape wins: the running checker asks and records facts, the checking program
decides. The registry's types win: the rules and the program read the generated
checker contract and the domain SDK, and no mesh-sdk delivery import is left.
The registry's series, renames and unparsed seat definitions become facts with
their refusals as rules; the tagged versions ride on the case. The program reads
the heads' seat definitions through sdk/seatdefs and the running checker lays them
over its reader's list by seat, reading no file of the heads: the reader's list
is larger than a run input's bound.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:57:21 +01:00
jschoubben d5d9f0c258 The domain SDK carries the run ask's rules and the checker's client asks a program's input (issue 1921, milestone 47/G D2)
B0's mesh-sdk go/delivery run lines move to their registry homes: the
run job, the one-job rule, the Go check stream alone, the input's bound,
digest and key in sdk/buildstep; AskWhat's run case in sdk/buildrun; the
ask of the verb input and the check of a verdict's judged-by in
meshdeliverycheckerclient. B0's tests come in as rows of the existing
tables.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:35:31 +01:00
jschoubben ea0b6a075b The checker's seat serves a checking program's input, a verdict names the program that judged it, and the build step's part carries the run ask (issue 1921, milestone 47/G D1)
B0's run job, its input verb and judged-by move from mesh-sdk's go/delivery
into the Delivery domain's contracts, with B0's JSON names: the part
build-step carries StepAsk.run and the defs RunJob, RunProgram, RunHead,
RunInput and RunAnswer beside compose (the lead's ruling on Q1), and a
step's result the run's exit, output and said.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:27:44 +01:00
mesh-tools bab05c66b3 Merge pull request 'The checking program judges every check: the rules leave the running checker (issue 1921, B0)' (#162) from feat/b0-checking-program into main 2026-11-07 00:32:49 +00:00
jschoubben 35b800aaef Format the checker's integration test as gofmt writes it, which the group check refused (issue 1921, B0)
mesh/delivery merged: source.main moved to novox/mesh-delivery@bab05c66b3cb
mesh/delivery-group group feat/b0-checking-program merged: every member is merged
mesh/check checking 35b800aaef68 on bab05c66b3cb
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 00:22:38 +01:00
jschoubben 9c9ed7a1d8 The checker serves a checking program's input on its seat's verb input, and the build module asks it (issue 1921, B0 review round 1)
mesh/delivery ready: it is merged on the Approve
mesh/delivery-group group feat/b0-checking-program checking: 3 of 3 member(s) ready
mesh/check checking 9c9ed7a1d8f1 on 2d201f781ced | novox/mesh-delivery#162 in group feat/b0-checking-program
The operator's answer: mesh-build-go reading the checker's state inputs broke ADR 0596 §2. The run ask's input now names
the verb seat:mesh-delivery-checker.input; the checker's tools serve it from the state inputs (Reader.Input, checked
against its digest), and the state is the checker's own again, read by no other module.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 23:39:39 +01:00
jschoubben f4ab7a2bc8 Name the file that gathers the module graph's edges (issue 1921)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 23:28:27 +01:00
jschoubben 9290cdda4d The checking program judges the module graph's cycles from the edges the running checker gathers (issue 1921, B0 review round 1)
ADR 0656 §5 has every check judged by parts built from the change's heads, and no record keeps the cycle rule in the
running checker. The facts carry each ordering edge's two modules and kind, never manifests: the whole catalogue's
graph is 50 edges, 3195 bytes.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 23:28:20 +01:00
jschoubben b1d0c6881a The facts carry a level as the rules read it, and a seat list the program cannot make ends the check without a verdict asked (issue 1921, B0 review round 1)
A level's digest, unmet packages, kept packages' names and test packages are read by no rule: a mesh-catalog check
step's two levels were 78682 bytes as facts and are 4506. A build node with no job run held a check four begin bounds;
it now holds it two.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 23:24:18 +01:00
jschoubben 06cb0cdd92 Merge the line, failure-words and package-cause tests into tables: the repository's unit count goes from 805 to 791 (issue 1921)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 23:08:49 +01:00
jschoubben fc921c13d0 checker_verdicts and checker_pull say which checking program judged a verdict, its build and the heads it read (issue 1921)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 23:07:06 +01:00
jschoubben 4dfb6af6ef The begin bound of a run ask is the checker's setting; a dropped run ask and a program that does not build fail the whole check naming the program (issue 1921)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 23:06:21 +01:00
jschoubben 625007cbc1 The checker's whole-module tests answer the checking program's run asks (issue 1921)
The fake build module of the integration tests answers a run ask with the checking program's
work over the input kept in the state inputs, as mesh-build-go does; the whole-module test
reads that the verdict names the program built at the group's mesh-delivery head, and the
keeps test reads inputs as readable.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 23:00:26 +01:00
jschoubben 63bbabacbf Every check is judged by the checking program built from its heads (issue 1921)
The running checker asks the checking program twice on the build seat's job run: first the
heads' seat list, which the manifest validator's requests (change and trunk) and the
node-engine composer's carry; then the verdict of the facts it gathered. The program is built
at the group's mesh-delivery head, else at mesh-delivery's trunk, and its input is kept in the
checker's new state inputs as the exact bytes of its digest, read with checker_inputs. A run
ask no build node begins within 5 minutes is put once more, then fails the check naming the
job; a program that does not build or answer fails the check naming it, and the verdict
records who judged it (JudgedBy). A second step-seen hook on a context is added to the
first, so a try still sees its steps.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 22:58:15 +01:00
jschoubben 043e3fb979 Read seat definitions once: the indexer's reading becomes a module the checker requires (issue 1921)
The checking program's seat list parsed the definition format a second time beside the
indexer's judge. The indexer's reading moves whole into its own sibling module seatfiles
(a domain path, as takein), with ResolveRaw keeping every field as written for a request's
seats; the judge and the take-in read through it, and the checker's rules only find the
definitions and say why one is left out (the lead's ruling on D3, ADR 0634 §1).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 22:39:31 +01:00
jschoubben 35da943fb5 The checking program answers the seat list of the heads laid for it (issue 1921)
Asked -seats, it reads every seat definition the heads carry, resolves the parts it includes and answers the list
with what it left out and why, so the running checker hands the manifest validator and the composer a list it does not
read itself (ADR 0636 §4, ADR 0644 §1).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 22:35:40 +01:00
jschoubben 1b9a2bc302 The running checker's asking records facts and returns no verdict (issue 1921)
Every asking function only records what it met; the one verdict is the rules' of the facts, so nothing in the
running checker judges beside the checking program.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 22:33:01 +01:00
jschoubben e96f8fd1b6 The manifests, compiled-programs and composition rules judge facts in the checker's rules (issue 1921)
The running checker records the manifest validator's cases, each fetch's programs and the engines' answers as facts,
and the rules package judges them, with the notes it gives a reader, so the checking program judges every rule the
running checker judged. The module graph's cycle stays with the asking: its facts are whole manifests, over a run's
input bound; G's program reads them from the heads.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 22:31:22 +01:00
jschoubben cd201a79f1 Every failure the asking meets is a fact of the check, and the verdict is the rules' of the facts (issue 1921)
The running checker records each failure where it is met and the node of each step it reads; the verdict is
rules.Judge of those facts, so the checking program, given the same facts, gives the same verdict. Its own
second outcome and its node set are gone.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 22:20:42 +01:00
jschoubben 61aa338887 Add the checking program: it answers the verdict of a check's facts (issue 1921)
mesh-checking-program -verdict reads the facts from the file the build
seat's run step names, judges them with the rules package and answers
one versioned line; facts of a version it does not read are refused,
naming it, because the running checker of B0 runs G's programs
(milestone 47/B0, ADR 0644, ADR 0656).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 22:13:37 +01:00
jschoubben 03bb56481a Move the level rules and the failure words into the checker's rules package (issue 1921)
The checking program judges a check from its facts (milestone 47/B0,
ADR 0644, ADR 0656): the rules package holds Facts v1, the level rule,
the failure words and the verdict's outcome, and the running checker
records each level answer as a fact and calls the rules. A row of the
every-failure tests judges the gathered facts and finds the running
checker's verdict.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 22:12:47 +01:00
jschoubben c3d47e4733 A composition at a node-engine head without its seat list fails (issue 1966)
When the checker's seat list is not read, or a head's definitions are not
taken, the composer at a node-engine member's head was handed no list,
judged no machine, and the composition passed with warnings. It now
fails, naming why (ADR 0635 §3), and no composer is asked. The check's
readers of the members' heads are its manifest validator and that
composer (ADR 0636 §4 insight 1). The composer's Request.Seats contract
says both the judge and the composer read the list, and a composer
handed none judges no machine (ADR 0636 §1).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 12:10:36 +01:00
jschoubben a92dc95f94 The check hands the composer at a node-engine head the seat list it judges against (issue 1966)
The group try of 2026-11-06 10:07 composed with a composer that had no
seat list, so its composition failed on all four machines for paths and
holders the seats define. The checker's one list per check (the
reader's, the members' heads' laid over it, ADR 0636 §4) now goes to the
judge and the sandbox composer alike; a list the reader has not read
goes to neither, and the composer then judges no machine (ADR 0636 §1).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 12:03:05 +01:00
150 changed files with 8358 additions and 1801 deletions
@@ -77,6 +77,32 @@ func checkComposition(c *checker.VerdictsByGroupHeadsValueComposition) error {
return nil
}
// checkJudgedBy refuses a checking program named at no repository, full commit and package, kept under no program key,
// or over no head or a head at no full commit (novox/hq ADR 0644, ADR 0656; issue 1921).
func checkJudgedBy(j *checker.VerdictsByGroupHeadsValueJudgedBy) error {
const who = "meshdeliverycheckerclient: "
if j == nil {
return nil
}
if CheckRepository(j.Program.Repository) != nil || !isCommit(j.Program.Commit) || j.Program.Package == "" {
return fmt.Errorf(who+"the verdict is judged by a checking program built at no repository, full commit and package (%+v)", j.Program)
}
if !strings.HasPrefix(j.Key, "program-") {
return fmt.Errorf(who+"the verdict's checking program is kept under %q, which is no program key", j.Key)
}
if len(j.Heads) == 0 {
return fmt.Errorf(who + "the verdict's checking program read no head")
}
for _, h := range j.Heads {
if CheckRepository(h.Repository) != nil || !isCommit(h.Commit) {
return fmt.Errorf(who+"the verdict's checking program read %s at %q, which is no full commit", h.Repository, h.Commit)
}
}
return nil
}
func isCommit(s string) bool { return len(s) == 40 && strings.Trim(s, "0123456789abcdef") == "" }
// CheckVerdict refuses a verdict a reader could not place: no group, heads that are not 12 lowercase hex digits, a repository
// both merged and read beside, a carried-from that carries nothing, a series change from a series to itself or named twice, a
// level that is neither unit nor integration or counts a negative number, a time below zero, a pass that says nothing of a
@@ -145,6 +171,9 @@ func CheckVerdict(v checker.VerdictsByGroupHeadsValue) error {
if err := checkComposition(v.Composition); err != nil {
return err
}
if err := checkJudgedBy(v.JudgedBy); err != nil {
return err
}
seen := map[string]bool{}
for _, level := range v.OverBound {
if err := checkLevel(level); err != nil {
@@ -1,6 +1,7 @@
package meshdeliverycheckerclient
import (
"encoding/json"
"strings"
"testing"
@@ -44,6 +45,11 @@ func TestAVerdictIsRefusedForWhatAReaderCouldNotPlace(t *testing.T) {
v.Composition = &checker.VerdictsByGroupHeadsValueComposition{State: CompositionPass}
}, "mesh-sdk commit"},
"the same level over bound twice": {func(v *verdict) { v.OverBound = []string{LevelUnit, LevelUnit} }, "twice"},
// 1921 (B0, G): the checking program that judged it.
"judged at no full commit": {func(v *verdict) { j := judgedBy(); j.Program.Commit = "0123456"; v.JudgedBy = &j }, "checking program built at no repository, full commit"},
"judged under no key": {func(v *verdict) { j := judgedBy(); j.Key = "composer-1"; v.JudgedBy = &j }, "composer-1"},
"judged over no head": {func(v *verdict) { j := judgedBy(); j.Heads = nil; v.JudgedBy = &j }, "no head"},
"judged over a branch": {func(v *verdict) { j := judgedBy(); j.Heads[0].Commit = "main"; v.JudgedBy = &j }, "novox/mesh-delivery"},
} {
v := passing()
c.edit(&v)
@@ -51,6 +57,12 @@ func TestAVerdictIsRefusedForWhatAReaderCouldNotPlace(t *testing.T) {
t.Errorf("%s: %v, want a refusal saying %q", name, err, c.says)
}
}
judged := passing()
j := judgedBy()
judged.JudgedBy = &j
if err := CheckVerdict(judged); err != nil {
t.Errorf("a pass judged by a checking program is refused: %v", err)
}
failed := passing()
failed.State, failed.Failed = VerdictFail, &checker.VerdictsByGroupHeadsValueFailed{Machine: "ace"}
if err := CheckVerdict(failed); err != nil {
@@ -191,6 +203,33 @@ func TestTheCodecsWriteAtTheVersionAndReadBackThroughTheRules(t *testing.T) {
if _, err := EncodePreview(p); err == nil {
t.Error("a preview with no merge commit and no conflict is written")
}
// 1921 (B0, G): a checking program's input is asked of the seat's verb input by its digest, and an answer for
// another digest, or bytes that are not the digest, is refused.
body := []byte(`{"v":1}`)
digest := DigestOf(body)
for name, c := range map[string]struct {
answer any
says string
}{
"its own bytes": {InputAnswerOf(body), ""},
"another digest": {checker.InputAnswer{Digest: DigestOf([]byte("x")), Input: InputAnswerOf(body).Input}, "answered the input"},
"bytes that are not it": {checker.InputAnswer{Digest: digest, Input: InputAnswerOf([]byte(`{"v":2}`)).Input}, "whose digest is"},
"bytes in no base64": {checker.InputAnswer{Digest: digest, Input: "%%"}, "no base64"},
"no answer of the layout": {"nothing", "answered no input"},
} {
got, err := ReadInput(func(key string, b any) (json.RawMessage, error) {
if key != "seat:mesh-delivery-checker.input" || b != (checker.InputInput{Digest: digest}) {
t.Errorf("%s: asked %s %+v", name, key, b)
}
return json.Marshal(c.answer)
}, digest)
switch {
case c.says == "" && (err != nil || string(got) != string(body)):
t.Errorf("%s: %q, %v", name, got, err)
case c.says != "" && (err == nil || !strings.Contains(err.Error(), c.says)):
t.Errorf("%s: %v, want a refusal saying %q", name, err, c.says)
}
}
}
// The kept and preview keys: a key reads back to its parts, and a part that does not key is refused.
@@ -221,3 +260,10 @@ func TestKeptAndPreviewKeysReadBack(t *testing.T) {
}
}
}
func judgedBy() checker.VerdictsByGroupHeadsValueJudgedBy {
const c = "0123456789abcdef0123456789abcdef01234567"
return checker.VerdictsByGroupHeadsValueJudgedBy{
Program: checker.VerdictsByGroupHeadsValueJudgedByProgram{Repository: "novox/mesh-delivery", Commit: c, Dir: "modules/mesh-delivery-checker", Package: "p"},
Key: "program-0123456789abcdef", Built: true, Heads: []checker.VerdictsByGroupHeadsValueJudgedByHeadsItem{{Repository: "novox/mesh-delivery", Commit: c}}}
}
@@ -0,0 +1,53 @@
package meshdeliverycheckerclient
import (
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"fmt"
checker "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
)
// InputVerb is the seat's verb input as a manifest's `invokes` names it and a run ask's input names it: asked of the
// seat, on no one machine (novox/hq ADR 0644, ADR 0656; issue 1921). A checking program's input is kept by the checker
// by the sha256 of its exact bytes, and asked of it by the build module that runs the program (ADR 0596 §2).
const InputVerb = "seat:" + checker.Seat + "." + checker.VerbInput
// InputAnswerOf is the verb input's answer for body: its digest and its exact bytes, base64.
func InputAnswerOf(body []byte) checker.InputAnswer {
return checker.InputAnswer{Digest: DigestOf(body), Input: base64.StdEncoding.EncodeToString(body)}
}
// DigestOf is the digest a checking program's input is kept and asked by: the sha256 of its bytes, 64 lowercase hex
// digits.
func DigestOf(body []byte) string {
sum := sha256.Sum256(body)
return hex.EncodeToString(sum[:])
}
// ReadInput asks the verb input, with ask (a tools bundle's stdio.Ask, a process's Conn.Call), for the input kept by
// digest and answers its bytes, refusing an answer for another digest and bytes whose sha256 is not the digest.
func ReadInput(ask func(key string, body any) (json.RawMessage, error), digest string) ([]byte, error) {
const who = "meshdeliverycheckerclient: "
raw, err := ask(InputVerb, checker.InputInput{Digest: digest})
if err != nil {
return nil, fmt.Errorf(who+"asking %s for the input %s: %w", InputVerb, digest, err)
}
var a checker.InputAnswer
if err := json.Unmarshal(raw, &a); err != nil {
return nil, fmt.Errorf(who+"%s answered no input for %s: %w", InputVerb, digest, err)
}
if a.Digest != digest {
return nil, fmt.Errorf(who+"%s answered the input %q, asked %s", InputVerb, a.Digest, digest)
}
body, err := base64.StdEncoding.DecodeString(a.Input)
if err != nil {
return nil, fmt.Errorf(who+"%s answered the input %s in no base64: %w", InputVerb, digest, err)
}
if got := DigestOf(body); got != digest {
return nil, fmt.Errorf(who+"the input %s answered %d bytes whose digest is %s", digest, len(body), got)
}
return body, nil
}
@@ -17,11 +17,12 @@ const (
VerbTry = "try"
VerbTryRead = "try-read"
VerbTryEnd = "try-end"
VerbInput = "input"
)
// Verbs is every verb the seat serves, in its definition's order: a new copy each call.
func Verbs() []string {
return []string{VerbTry, VerbTryRead, VerbTryEnd}
return []string{VerbTry, VerbTryRead, VerbTryEnd, VerbInput}
}
// The holder's own events, listed beside the seat and published under the holder's subjects (ADR 0620 §2).
@@ -273,6 +274,20 @@ type TryEndAnswer struct {
ID string `json:"id"`
}
// InputInput is what input takes.
type InputInput struct {
// the input's sha256, 64 lowercase hex digits, as the run ask names it
Digest string `json:"digest"`
}
// InputAnswer is what input answers.
type InputAnswer struct {
// the digest asked
Digest string `json:"digest"`
// the input's exact bytes, base64, so no encoding on the way changes a byte the digest covers
Input string `json:"input"`
}
// VerdictHolderEventSeriesItem is a type of the contract.
type VerdictHolderEventSeriesItem struct {
// its artifact, or the package's ecosystem name as the module's packages name it
@@ -343,6 +358,29 @@ type VerdictsByGroupHeadsValueFailuresItem struct {
Words string `json:"words,omitempty"`
}
// VerdictsByGroupHeadsValueJudgedByHeadsItem is a type of the contract.
type VerdictsByGroupHeadsValueJudgedByHeadsItem struct {
Commit string `json:"commit"`
Repository string `json:"repository"`
}
// VerdictsByGroupHeadsValueJudgedByProgram is a type of the contract.
type VerdictsByGroupHeadsValueJudgedByProgram struct {
Commit string `json:"commit"`
Dir string `json:"dir"`
Package string `json:"package"`
Repository string `json:"repository"`
}
// VerdictsByGroupHeadsValueJudgedBy is the checking program that judged the check: where it was built, the key its build is kept under, whether this check's run built it, and the heads it read (ADR 0644, ADR 0656). Absent in a value written before it.
type VerdictsByGroupHeadsValueJudgedBy struct {
Built bool `json:"built,omitempty"`
Heads []VerdictsByGroupHeadsValueJudgedByHeadsItem `json:"heads"`
// program- and 16 hex digits
Key string `json:"key"`
Program VerdictsByGroupHeadsValueJudgedByProgram `json:"program"`
}
// VerdictsByGroupHeadsValueLevelsValue is a type of the contract.
type VerdictsByGroupHeadsValueLevelsValue struct {
Kept int64 `json:"kept"`
@@ -379,6 +417,8 @@ type VerdictsByGroupHeadsValue struct {
Group string `json:"group"`
// 12 hex digits
Heads string `json:"heads"`
// the checking program that judged the check: where it was built, the key its build is kept under, whether this check's run built it, and the heads it read (ADR 0644, ADR 0656). Absent in a value written before it
JudgedBy *VerdictsByGroupHeadsValueJudgedBy `json:"judged-by,omitempty"`
// by level, unit or integration
Levels map[string]VerdictsByGroupHeadsValueLevelsValue `json:"levels,omitempty"`
Machines []string `json:"machines,omitempty"`
@@ -29,6 +29,12 @@ func TestFixtures(t *testing.T) {
err := json.Unmarshal(b, &v)
return v, err
}, subjectIs: nil},
{name: "a pass judged by the checking program built at a mesh-delivery head", subject: "", given: "{\"group\":\"feat/registry\",\"heads\":\"0123456789ab\",\"judged-by\":{\"built\":true,\"heads\":[{\"commit\":\"0123456789abcdef0123456789abcdef01234567\",\"repository\":\"novox/mesh-delivery\"}],\"key\":\"program-0123456789abcdef\",\"program\":{\"commit\":\"0123456789abcdef0123456789abcdef01234567\",\"dir\":\"modules/mesh-delivery-checker\",\"package\":\"./cmd/mesh-checking-program\",\"repository\":\"novox/mesh-delivery\"}},\"levels\":{\"integration\":{\"kept\":0,\"run\":3},\"unit\":{\"kept\":0,\"run\":12}},\"state\":\"pass\",\"v\":1}", wire: "{\"group\":\"feat/registry\",\"heads\":\"0123456789ab\",\"judged-by\":{\"built\":true,\"heads\":[{\"commit\":\"0123456789abcdef0123456789abcdef01234567\",\"repository\":\"novox/mesh-delivery\"}],\"key\":\"program-0123456789abcdef\",\"program\":{\"commit\":\"0123456789abcdef0123456789abcdef01234567\",\"dir\":\"modules/mesh-delivery-checker\",\"package\":\"./cmd/mesh-checking-program\",\"repository\":\"novox/mesh-delivery\"}},\"levels\":{\"integration\":{\"kept\":0,\"run\":3},\"unit\":{\"kept\":0,\"run\":12}},\"state\":\"pass\",\"v\":1}",
decode: func(b []byte) (any, error) {
var v VerdictsByGroupHeadsValue
err := json.Unmarshal(b, &v)
return v, err
}, subjectIs: nil},
{name: "a kept pass", subject: "", given: "{\"at\":\"2026-11-05T10:00:00Z\",\"level\":\"unit\",\"package\":\"go/keys\",\"repository\":\"novox/mesh-sdk\",\"state\":\"pass\",\"tests\":4,\"took\":120,\"toolchain\":\"go1.26\",\"v\":1}", wire: "{\"at\":\"2026-11-05T10:00:00Z\",\"level\":\"unit\",\"package\":\"go/keys\",\"repository\":\"novox/mesh-sdk\",\"state\":\"pass\",\"tests\":4,\"took\":120,\"toolchain\":\"go1.26\",\"v\":1}",
decode: func(b []byte) (any, error) { var v KeptByLevelDigestValue; err := json.Unmarshal(b, &v); return v, err }, subjectIs: nil},
{name: "the status owed for a head", subject: "", given: "{\"act\":\"status=mesh/check\",\"context\":\"mesh/check\",\"state\":\"success\",\"v\":1}", wire: "{\"act\":\"status=mesh/check\",\"context\":\"mesh/check\",\"state\":\"success\",\"v\":1}",
@@ -43,6 +49,10 @@ func TestFixtures(t *testing.T) {
err := json.Unmarshal(b, &v)
return v, err
}, subjectIs: nil},
{name: "a checking program asks its input by its digest", subject: "", given: "{\"digest\":\"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\"}", wire: "{\"digest\":\"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\"}",
decode: func(b []byte) (any, error) { var v InputInput; err := json.Unmarshal(b, &v); return v, err }, subjectIs: nil},
{name: "the input answered as its exact bytes, base64", subject: "", given: "{\"digest\":\"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\",\"input\":\"eyJ2IjoxfQ==\"}", wire: "{\"digest\":\"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\",\"input\":\"eyJ2IjoxfQ==\"}",
decode: func(b []byte) (any, error) { var v InputAnswer; err := json.Unmarshal(b, &v); return v, err }, subjectIs: nil},
}
for _, c := range cases {
v, err := c.decode([]byte(c.given))
+60 -7
View File
@@ -386,6 +386,51 @@ type Program struct {
Path string `json:"path"`
}
// RunAnswer is what a run ask's step answers: whether this run built the program or took the one kept under its key, the key, the program's exit, the last line of its standard output and the first of its standard error.
type RunAnswer struct {
Built bool `json:"built"`
Exit int64 `json:"exit"`
Key string `json:"key"`
// raw when it is JSON, else a JSON string
Output any `json:"output,omitempty"`
Said string `json:"said,omitempty"`
}
// RunHead is one commit a run's program reads.
type RunHead struct {
Commit string `json:"commit"`
Repository string `json:"repository"`
}
// RunInput is a run's input by reference: the seat verb the asker serves it on (seat:<seat>.<verb>), the sha256 of its bytes and its size, at most 256 KiB.
type RunInput struct {
// 64 lowercase hex digits
Digest string `json:"digest"`
// bytes
Size int64 `json:"size"`
Verb string `json:"verb"`
}
// RunJob is a run ask's job: where its program is built, the heads it reads (each laid read-only at its repository's path), its input by reference, its arguments and what it is for.
type RunJob struct {
// the program's arguments, each a word or a flag
Args []string `json:"args,omitempty"`
// the commits the program reads, a repository at most once
Heads []RunHead `json:"heads"`
Input RunInput `json:"input"`
Program RunProgram `json:"program"`
// what the run is for, for a reader of the ask
Reader string `json:"reader,omitempty"`
}
// RunProgram is where a run's program is built: the repository and full commit fetched, the Go module's directory in it and the program's import path.
type RunProgram struct {
Commit string `json:"commit"`
Dir string `json:"dir"`
Package string `json:"package"`
Repository string `json:"repository"`
}
// Step is one step a build module runs on a node.
type Step struct {
// when (RFC 3339 time)
@@ -460,25 +505,33 @@ type StepAsk struct {
Needs []string `json:"needs,omitempty"`
// package to the tagged version a build pins
Packages map[string]string `json:"packages,omitempty"`
// a run: a program built from a change and run in the build seat's sandbox over the heads, its input asked of a verb of the asker's seat; the Go seat's check stream alone carries it, as a compose
Run *RunJob `json:"run,omitempty"`
// the layout version
V int64 `json:"v,omitempty"`
}
// StepResult is what a step answered, by the kind of ask: a build's artifacts, toolchain, imports and out-of-step modules; a check's fetch and levels; a compose's built, key and verdict; a declined ask's declined, needs and lacks. The fields of the kinds do not meet, so one layout holds the four.
// StepResult is what a step answered, by the kind of ask: a build's artifacts, toolchain, imports and out-of-step modules; a check's fetch and levels; a compose's built, key and verdict; a run's built, key, exit, output and said; a declined ask's declined, needs and lacks. The fields of the kinds do not meet, so one layout holds the five.
type StepResult struct {
Artifacts []BuiltArtifact `json:"artifacts,omitempty"`
Built bool `json:"built,omitempty"`
// the holder declined the ask: its node lacks what the ask needs
Declined bool `json:"declined,omitempty"`
Fetch *FetchAnswer `json:"fetch,omitempty"`
Imports []string `json:"imports,omitempty"`
Key string `json:"key,omitempty"`
Declined bool `json:"declined,omitempty"`
// a run's program's exit status
Exit int64 `json:"exit,omitempty"`
Fetch *FetchAnswer `json:"fetch,omitempty"`
Imports []string `json:"imports,omitempty"`
Key string `json:"key,omitempty"`
// what the holder's node lacks of it
Lacks []string `json:"lacks,omitempty"`
Levels []LevelAnswer `json:"levels,omitempty"`
// what the declined ask needs
Needs []string `json:"needs,omitempty"`
OutOfStep []string `json:"out-of-step,omitempty"`
Needs []string `json:"needs,omitempty"`
OutOfStep []string `json:"out-of-step,omitempty"`
// a run's program's last line of standard output: raw when it is JSON, else a JSON string
Output any `json:"output,omitempty"`
// a run's program's first line of standard error, or why the step could not run it
Said string `json:"said,omitempty"`
Toolchain *ToolchainUsed `json:"toolchain,omitempty"`
// the compose-candidate verdict (the composer's own layout)
Verdict any `json:"verdict,omitempty"`
@@ -49,6 +49,10 @@ func TestFixtures(t *testing.T) {
decode: func(b []byte) (any, error) { var v StepsByNodeIDValue; err := json.Unmarshal(b, &v); return v, err }, subjectIs: nil},
{name: "a finished check step whose head does not merge", subject: "", given: "{\"ended\":\"2026-11-05T10:01:00Z\",\"id\":\"c3d5\",\"node\":\"ace\",\"result\":{\"fetch\":{\"base\":\"abcdef0123456789abcdef0123456789abcdef01\",\"commit\":\"0123456789abcdef0123456789abcdef01234567\",\"conflicts\":[\"a.go\",\"sub/b.go\"],\"modules\":[\".\"],\"workspace\":\"mvii1iwz5050eb\"}},\"started\":\"2026-11-05T10:00:00Z\",\"state\":\"passed\",\"tool\":\"check\",\"v\":1}", wire: "{\"ended\":\"2026-11-05T10:01:00Z\",\"id\":\"c3d5\",\"node\":\"ace\",\"result\":{\"fetch\":{\"base\":\"abcdef0123456789abcdef0123456789abcdef01\",\"commit\":\"0123456789abcdef0123456789abcdef01234567\",\"conflicts\":[\"a.go\",\"sub/b.go\"],\"modules\":[\".\"],\"workspace\":\"mvii1iwz5050eb\"}},\"started\":\"2026-11-05T10:00:00Z\",\"state\":\"passed\",\"tool\":\"check\",\"v\":1}",
decode: func(b []byte) (any, error) { var v StepsByNodeIDValue; err := json.Unmarshal(b, &v); return v, err }, subjectIs: nil},
{name: "a run ask of the checking program over two heads", subject: "mesh.seat.node-build-go.accept.check", given: "{\"asker\":\"mesh-delivery-checker\",\"at\":\"2026-11-05T10:00:00Z\",\"id\":\"r5e6\",\"kind\":\"check\",\"language\":\"go\",\"run\":{\"args\":[\"-verdict\"],\"heads\":[{\"commit\":\"0123456789abcdef0123456789abcdef01234567\",\"repository\":\"novox/mesh-delivery\"},{\"commit\":\"0123456789abcdef0123456789abcdef01234567\",\"repository\":\"novox/mesh-sdk\"}],\"input\":{\"digest\":\"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\",\"size\":7,\"verb\":\"seat:mesh-delivery-checker.input\"},\"program\":{\"commit\":\"0123456789abcdef0123456789abcdef01234567\",\"dir\":\"modules/mesh-delivery-checker\",\"package\":\"./cmd/mesh-checking-program\",\"repository\":\"novox/mesh-delivery\"},\"reader\":\"checking program of mesh-delivery head\"},\"v\":1}", wire: "{\"asker\":\"mesh-delivery-checker\",\"at\":\"2026-11-05T10:00:00Z\",\"id\":\"r5e6\",\"kind\":\"check\",\"language\":\"go\",\"run\":{\"args\":[\"-verdict\"],\"heads\":[{\"commit\":\"0123456789abcdef0123456789abcdef01234567\",\"repository\":\"novox/mesh-delivery\"},{\"commit\":\"0123456789abcdef0123456789abcdef01234567\",\"repository\":\"novox/mesh-sdk\"}],\"input\":{\"digest\":\"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\",\"size\":7,\"verb\":\"seat:mesh-delivery-checker.input\"},\"program\":{\"commit\":\"0123456789abcdef0123456789abcdef01234567\",\"dir\":\"modules/mesh-delivery-checker\",\"package\":\"./cmd/mesh-checking-program\",\"repository\":\"novox/mesh-delivery\"},\"reader\":\"checking program of mesh-delivery head\"},\"v\":1}",
decode: func(b []byte) (any, error) { var v CheckAccept; err := json.Unmarshal(b, &v); return v, err }, subjectIs: regexp.MustCompile("^mesh\\.seat\\.node-build-go\\.accept\\.check$")},
{name: "a finished run step whose program exited non-zero", subject: "", given: "{\"ended\":\"2026-11-05T10:01:00Z\",\"id\":\"r5e6\",\"node\":\"ace\",\"result\":{\"built\":true,\"exit\":2,\"key\":\"program-0123456789abcdef\",\"said\":\"the facts name no head\"},\"started\":\"2026-11-05T10:00:00Z\",\"state\":\"failed\",\"tool\":\"check\",\"v\":1}", wire: "{\"ended\":\"2026-11-05T10:01:00Z\",\"id\":\"r5e6\",\"node\":\"ace\",\"result\":{\"built\":true,\"exit\":2,\"key\":\"program-0123456789abcdef\",\"said\":\"the facts name no head\"},\"started\":\"2026-11-05T10:00:00Z\",\"state\":\"failed\",\"tool\":\"check\",\"v\":1}",
decode: func(b []byte) (any, error) { var v StepsByNodeIDValue; err := json.Unmarshal(b, &v); return v, err }, subjectIs: nil},
}
for _, c := range cases {
v, err := c.decode([]byte(c.given))
+60 -7
View File
@@ -386,6 +386,51 @@ type Program struct {
Path string `json:"path"`
}
// RunAnswer is what a run ask's step answers: whether this run built the program or took the one kept under its key, the key, the program's exit, the last line of its standard output and the first of its standard error.
type RunAnswer struct {
Built bool `json:"built"`
Exit int64 `json:"exit"`
Key string `json:"key"`
// raw when it is JSON, else a JSON string
Output any `json:"output,omitempty"`
Said string `json:"said,omitempty"`
}
// RunHead is one commit a run's program reads.
type RunHead struct {
Commit string `json:"commit"`
Repository string `json:"repository"`
}
// RunInput is a run's input by reference: the seat verb the asker serves it on (seat:<seat>.<verb>), the sha256 of its bytes and its size, at most 256 KiB.
type RunInput struct {
// 64 lowercase hex digits
Digest string `json:"digest"`
// bytes
Size int64 `json:"size"`
Verb string `json:"verb"`
}
// RunJob is a run ask's job: where its program is built, the heads it reads (each laid read-only at its repository's path), its input by reference, its arguments and what it is for.
type RunJob struct {
// the program's arguments, each a word or a flag
Args []string `json:"args,omitempty"`
// the commits the program reads, a repository at most once
Heads []RunHead `json:"heads"`
Input RunInput `json:"input"`
Program RunProgram `json:"program"`
// what the run is for, for a reader of the ask
Reader string `json:"reader,omitempty"`
}
// RunProgram is where a run's program is built: the repository and full commit fetched, the Go module's directory in it and the program's import path.
type RunProgram struct {
Commit string `json:"commit"`
Dir string `json:"dir"`
Package string `json:"package"`
Repository string `json:"repository"`
}
// Step is one step a build module runs on a node.
type Step struct {
// when (RFC 3339 time)
@@ -460,25 +505,33 @@ type StepAsk struct {
Needs []string `json:"needs,omitempty"`
// package to the tagged version a build pins
Packages map[string]string `json:"packages,omitempty"`
// a run: a program built from a change and run in the build seat's sandbox over the heads, its input asked of a verb of the asker's seat; the Go seat's check stream alone carries it, as a compose
Run *RunJob `json:"run,omitempty"`
// the layout version
V int64 `json:"v,omitempty"`
}
// StepResult is what a step answered, by the kind of ask: a build's artifacts, toolchain, imports and out-of-step modules; a check's fetch and levels; a compose's built, key and verdict; a declined ask's declined, needs and lacks. The fields of the kinds do not meet, so one layout holds the four.
// StepResult is what a step answered, by the kind of ask: a build's artifacts, toolchain, imports and out-of-step modules; a check's fetch and levels; a compose's built, key and verdict; a run's built, key, exit, output and said; a declined ask's declined, needs and lacks. The fields of the kinds do not meet, so one layout holds the five.
type StepResult struct {
Artifacts []BuiltArtifact `json:"artifacts,omitempty"`
Built bool `json:"built,omitempty"`
// the holder declined the ask: its node lacks what the ask needs
Declined bool `json:"declined,omitempty"`
Fetch *FetchAnswer `json:"fetch,omitempty"`
Imports []string `json:"imports,omitempty"`
Key string `json:"key,omitempty"`
Declined bool `json:"declined,omitempty"`
// a run's program's exit status
Exit int64 `json:"exit,omitempty"`
Fetch *FetchAnswer `json:"fetch,omitempty"`
Imports []string `json:"imports,omitempty"`
Key string `json:"key,omitempty"`
// what the holder's node lacks of it
Lacks []string `json:"lacks,omitempty"`
Levels []LevelAnswer `json:"levels,omitempty"`
// what the declined ask needs
Needs []string `json:"needs,omitempty"`
OutOfStep []string `json:"out-of-step,omitempty"`
Needs []string `json:"needs,omitempty"`
OutOfStep []string `json:"out-of-step,omitempty"`
// a run's program's last line of standard output: raw when it is JSON, else a JSON string
Output any `json:"output,omitempty"`
// a run's program's first line of standard error, or why the step could not run it
Said string `json:"said,omitempty"`
Toolchain *ToolchainUsed `json:"toolchain,omitempty"`
// the compose-candidate verdict (the composer's own layout)
Verdict any `json:"verdict,omitempty"`
+60 -7
View File
@@ -386,6 +386,51 @@ type Program struct {
Path string `json:"path"`
}
// RunAnswer is what a run ask's step answers: whether this run built the program or took the one kept under its key, the key, the program's exit, the last line of its standard output and the first of its standard error.
type RunAnswer struct {
Built bool `json:"built"`
Exit int64 `json:"exit"`
Key string `json:"key"`
// raw when it is JSON, else a JSON string
Output any `json:"output,omitempty"`
Said string `json:"said,omitempty"`
}
// RunHead is one commit a run's program reads.
type RunHead struct {
Commit string `json:"commit"`
Repository string `json:"repository"`
}
// RunInput is a run's input by reference: the seat verb the asker serves it on (seat:<seat>.<verb>), the sha256 of its bytes and its size, at most 256 KiB.
type RunInput struct {
// 64 lowercase hex digits
Digest string `json:"digest"`
// bytes
Size int64 `json:"size"`
Verb string `json:"verb"`
}
// RunJob is a run ask's job: where its program is built, the heads it reads (each laid read-only at its repository's path), its input by reference, its arguments and what it is for.
type RunJob struct {
// the program's arguments, each a word or a flag
Args []string `json:"args,omitempty"`
// the commits the program reads, a repository at most once
Heads []RunHead `json:"heads"`
Input RunInput `json:"input"`
Program RunProgram `json:"program"`
// what the run is for, for a reader of the ask
Reader string `json:"reader,omitempty"`
}
// RunProgram is where a run's program is built: the repository and full commit fetched, the Go module's directory in it and the program's import path.
type RunProgram struct {
Commit string `json:"commit"`
Dir string `json:"dir"`
Package string `json:"package"`
Repository string `json:"repository"`
}
// Step is one step a build module runs on a node.
type Step struct {
// when (RFC 3339 time)
@@ -460,25 +505,33 @@ type StepAsk struct {
Needs []string `json:"needs,omitempty"`
// package to the tagged version a build pins
Packages map[string]string `json:"packages,omitempty"`
// a run: a program built from a change and run in the build seat's sandbox over the heads, its input asked of a verb of the asker's seat; the Go seat's check stream alone carries it, as a compose
Run *RunJob `json:"run,omitempty"`
// the layout version
V int64 `json:"v,omitempty"`
}
// StepResult is what a step answered, by the kind of ask: a build's artifacts, toolchain, imports and out-of-step modules; a check's fetch and levels; a compose's built, key and verdict; a declined ask's declined, needs and lacks. The fields of the kinds do not meet, so one layout holds the four.
// StepResult is what a step answered, by the kind of ask: a build's artifacts, toolchain, imports and out-of-step modules; a check's fetch and levels; a compose's built, key and verdict; a run's built, key, exit, output and said; a declined ask's declined, needs and lacks. The fields of the kinds do not meet, so one layout holds the five.
type StepResult struct {
Artifacts []BuiltArtifact `json:"artifacts,omitempty"`
Built bool `json:"built,omitempty"`
// the holder declined the ask: its node lacks what the ask needs
Declined bool `json:"declined,omitempty"`
Fetch *FetchAnswer `json:"fetch,omitempty"`
Imports []string `json:"imports,omitempty"`
Key string `json:"key,omitempty"`
Declined bool `json:"declined,omitempty"`
// a run's program's exit status
Exit int64 `json:"exit,omitempty"`
Fetch *FetchAnswer `json:"fetch,omitempty"`
Imports []string `json:"imports,omitempty"`
Key string `json:"key,omitempty"`
// what the holder's node lacks of it
Lacks []string `json:"lacks,omitempty"`
Levels []LevelAnswer `json:"levels,omitempty"`
// what the declined ask needs
Needs []string `json:"needs,omitempty"`
OutOfStep []string `json:"out-of-step,omitempty"`
Needs []string `json:"needs,omitempty"`
OutOfStep []string `json:"out-of-step,omitempty"`
// a run's program's last line of standard output: raw when it is JSON, else a JSON string
Output any `json:"output,omitempty"`
// a run's program's first line of standard error, or why the step could not run it
Said string `json:"said,omitempty"`
Toolchain *ToolchainUsed `json:"toolchain,omitempty"`
// the compose-candidate verdict (the composer's own layout)
Verdict any `json:"verdict,omitempty"`
+139 -1
View File
@@ -174,6 +174,10 @@
"request"
]
},
"run": {
"$ref": "#/defs/RunJob",
"description": "a run: a program built from a change and run in the build seat's sandbox over the heads, its input asked of a verb of the asker's seat; the Go seat's check stream alone carries it, as a compose"
},
"needs": {
"type": "array",
"items": {
@@ -259,6 +263,129 @@
"built"
]
},
"RunJob": {
"type": "object",
"description": "a run ask's job: where its program is built, the heads it reads (each laid read-only at its repository's path), its input by reference, its arguments and what it is for",
"properties": {
"program": {
"$ref": "#/defs/RunProgram"
},
"heads": {
"type": "array",
"items": {
"$ref": "#/defs/RunHead"
},
"description": "the commits the program reads, a repository at most once"
},
"input": {
"$ref": "#/defs/RunInput"
},
"args": {
"type": "array",
"items": {
"type": "string"
},
"description": "the program's arguments, each a word or a flag"
},
"reader": {
"type": "string",
"description": "what the run is for, for a reader of the ask"
}
},
"required": [
"program",
"heads",
"input"
]
},
"RunProgram": {
"type": "object",
"description": "where a run's program is built: the repository and full commit fetched, the Go module's directory in it and the program's import path",
"properties": {
"repository": {
"type": "string"
},
"commit": {
"type": "string"
},
"dir": {
"type": "string"
},
"package": {
"type": "string"
}
},
"required": [
"repository",
"commit",
"dir",
"package"
]
},
"RunHead": {
"type": "object",
"description": "one commit a run's program reads",
"properties": {
"repository": {
"type": "string"
},
"commit": {
"type": "string"
}
},
"required": [
"repository",
"commit"
]
},
"RunInput": {
"type": "object",
"description": "a run's input by reference: the seat verb the asker serves it on (seat:<seat>.<verb>), the sha256 of its bytes and its size, at most 256 KiB",
"properties": {
"verb": {
"type": "string"
},
"digest": {
"type": "string",
"description": "64 lowercase hex digits"
},
"size": {
"type": "integer",
"description": "bytes"
}
},
"required": [
"verb",
"digest",
"size"
]
},
"RunAnswer": {
"type": "object",
"description": "what a run ask's step answers: whether this run built the program or took the one kept under its key, the key, the program's exit, the last line of its standard output and the first of its standard error",
"properties": {
"built": {
"type": "boolean"
},
"key": {
"type": "string"
},
"exit": {
"type": "integer"
},
"output": {
"description": "raw when it is JSON, else a JSON string"
},
"said": {
"type": "string"
}
},
"required": [
"built",
"key",
"exit"
]
},
"Step": {
"type": "object",
"properties": {
@@ -678,7 +805,7 @@
},
"StepResult": {
"type": "object",
"description": "what a step answered, by the kind of ask: a build's artifacts, toolchain, imports and out-of-step modules; a check's fetch and levels; a compose's built, key and verdict; a declined ask's declined, needs and lacks. The fields of the kinds do not meet, so one layout holds the four",
"description": "what a step answered, by the kind of ask: a build's artifacts, toolchain, imports and out-of-step modules; a check's fetch and levels; a compose's built, key and verdict; a run's built, key, exit, output and said; a declined ask's declined, needs and lacks. The fields of the kinds do not meet, so one layout holds the five",
"properties": {
"artifacts": {
"type": "array",
@@ -736,6 +863,17 @@
"type": "string"
},
"description": "what the holder's node lacks of it"
},
"exit": {
"type": "integer",
"description": "a run's program's exit status"
},
"output": {
"description": "a run's program's last line of standard output: raw when it is JSON, else a JSON string"
},
"said": {
"type": "string",
"description": "a run's program's first line of standard error, or why the step could not run it"
}
}
},
@@ -9,6 +9,42 @@
"state": "running"
}
},
{
"name": "a pass judged by the checking program built at a mesh-delivery head",
"of": "state:verdicts/<group>.<heads>",
"given": {
"v": 1,
"group": "feat/registry",
"heads": "0123456789ab",
"state": "pass",
"levels": {
"unit": {
"run": 12,
"kept": 0
},
"integration": {
"run": 3,
"kept": 0
}
},
"judged-by": {
"program": {
"repository": "novox/mesh-delivery",
"commit": "0123456789abcdef0123456789abcdef01234567",
"dir": "modules/mesh-delivery-checker",
"package": "./cmd/mesh-checking-program"
},
"key": "program-0123456789abcdef",
"built": true,
"heads": [
{
"repository": "novox/mesh-delivery",
"commit": "0123456789abcdef0123456789abcdef01234567"
}
]
}
}
},
{
"name": "a kept pass",
"of": "state:kept/<level>.<digest>",
@@ -0,0 +1,17 @@
[
{
"name": "a checking program asks its input by its digest",
"of": "verb:input.input",
"given": {
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
}
},
{
"name": "the input answered as its exact bytes, base64",
"of": "verb:input.answer",
"given": {
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"input": "eyJ2IjoxfQ=="
}
}
]
@@ -140,6 +140,40 @@
"ended"
]
}
},
{
"name": "input",
"description": "A checking program's input the checker keeps, by the sha256 of its exact bytes: the facts of a check its verdict is asked of, or the heads its seat list is asked of, as a run ask names it (its input's verb and digest). Answers the digest and the bytes, base64; a digest it keeps no input of is an error. Asked by the build module that runs the program, as its manifest's invokes grants it. Read-only.",
"optional": true,
"input": {
"type": "object",
"properties": {
"digest": {
"type": "string",
"description": "the input's sha256, 64 lowercase hex digits, as the run ask names it"
}
},
"required": [
"digest"
]
},
"answer": {
"type": "object",
"properties": {
"digest": {
"type": "string",
"description": "the digest asked"
},
"input": {
"type": "string",
"description": "the input's exact bytes, base64, so no encoding on the way changes a byte the digest covers"
}
},
"required": [
"digest",
"input"
]
}
}
],
"description": "The merge check's tries: start one, read one, end one. The checker's own events stay under its own subjects (ADR 0620 §2).",
@@ -455,6 +489,65 @@
"to"
]
}
},
"judged-by": {
"type": "object",
"description": "the checking program that judged the check: where it was built, the key its build is kept under, whether this check's run built it, and the heads it read (ADR 0644, ADR 0656). Absent in a value written before it",
"properties": {
"program": {
"type": "object",
"properties": {
"repository": {
"type": "string"
},
"commit": {
"type": "string"
},
"dir": {
"type": "string"
},
"package": {
"type": "string"
}
},
"required": [
"repository",
"commit",
"dir",
"package"
]
},
"key": {
"type": "string",
"description": "program- and 16 hex digits"
},
"built": {
"type": "boolean"
},
"heads": {
"type": "array",
"items": {
"type": "object",
"properties": {
"repository": {
"type": "string"
},
"commit": {
"type": "string"
}
},
"required": [
"repository",
"commit"
]
}
}
},
"required": [
"program",
"key",
"heads"
]
}
},
"required": [
@@ -253,5 +253,64 @@
}
}
}
},
{
"name": "a run ask of the checking program over two heads",
"of": "accept:check",
"subject": "mesh.seat.node-build-go.accept.check",
"given": {
"v": 1,
"id": "r5e6",
"kind": "check",
"language": "go",
"asker": "mesh-delivery-checker",
"at": "2026-11-05T10:00:00Z",
"run": {
"program": {
"repository": "novox/mesh-delivery",
"commit": "0123456789abcdef0123456789abcdef01234567",
"dir": "modules/mesh-delivery-checker",
"package": "./cmd/mesh-checking-program"
},
"heads": [
{
"repository": "novox/mesh-delivery",
"commit": "0123456789abcdef0123456789abcdef01234567"
},
{
"repository": "novox/mesh-sdk",
"commit": "0123456789abcdef0123456789abcdef01234567"
}
],
"input": {
"verb": "seat:mesh-delivery-checker.input",
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"size": 7
},
"args": [
"-verdict"
],
"reader": "checking program of mesh-delivery head"
}
}
},
{
"name": "a finished run step whose program exited non-zero",
"of": "state:steps/<node>.<id>",
"given": {
"v": 1,
"id": "r5e6",
"tool": "check",
"node": "ace",
"state": "failed",
"started": "2026-11-05T10:00:00Z",
"ended": "2026-11-05T10:01:00Z",
"result": {
"built": true,
"key": "program-0123456789abcdef",
"exit": 2,
"said": "the facts name no head"
}
}
}
]
@@ -8,6 +8,7 @@ export const verbs = {
try: "try",
tryRead: "try-read",
tryEnd: "try-end",
input: "input",
} as const;
export const holderEvents = {
@@ -181,6 +182,20 @@ export interface TryEndAnswer {
"id": string;
}
/** is what input takes. */
export interface InputInput {
/** the input's sha256, 64 lowercase hex digits, as the run ask names it */
"digest": string;
}
/** is what input answers. */
export interface InputAnswer {
/** the digest asked */
"digest": string;
/** the input's exact bytes, base64, so no encoding on the way changes a byte the digest covers */
"input": string;
}
export interface VerdictHolderEventSeriesItem {
/** its artifact, or the package's ecosystem name as the module's packages name it */
"artifact": string;
@@ -247,6 +262,27 @@ export interface VerdictsByGroupHeadsValueFailuresItem {
"words"?: string;
}
export interface VerdictsByGroupHeadsValueJudgedByHeadsItem {
"commit": string;
"repository": string;
}
export interface VerdictsByGroupHeadsValueJudgedByProgram {
"commit": string;
"dir": string;
"package": string;
"repository": string;
}
/** the checking program that judged the check: where it was built, the key its build is kept under, whether this check's run built it, and the heads it read (ADR 0644, ADR 0656). Absent in a value written before it */
export interface VerdictsByGroupHeadsValueJudgedBy {
"built"?: boolean;
"heads": VerdictsByGroupHeadsValueJudgedByHeadsItem[];
/** program- and 16 hex digits */
"key": string;
"program": VerdictsByGroupHeadsValueJudgedByProgram;
}
export interface VerdictsByGroupHeadsValueLevelsValue {
"kept": number;
"run": number;
@@ -281,6 +317,8 @@ export interface VerdictsByGroupHeadsValue {
"group": string;
/** 12 hex digits */
"heads": string;
/** the checking program that judged the check: where it was built, the key its build is kept under, whether this check's run built it, and the heads it read (ADR 0644, ADR 0656). Absent in a value written before it */
"judged-by"?: VerdictsByGroupHeadsValueJudgedBy;
/** by level, unit or integration */
"levels"?: Record<string, VerdictsByGroupHeadsValueLevelsValue>;
"machines"?: string[];
@@ -129,6 +129,11 @@ export type PlacedBeside = buildStep.PlacedBeside;
export type Program = buildStep.Program;
export type ResumeAnswer = buildStep.ResumeAnswer;
export type ResumeInput = buildStep.ResumeInput;
export type RunAnswer = buildStep.RunAnswer;
export type RunHead = buildStep.RunHead;
export type RunInput = buildStep.RunInput;
export type RunJob = buildStep.RunJob;
export type RunProgram = buildStep.RunProgram;
export type Step = buildStep.Step;
export type StepAsk = buildStep.StepAsk;
export type StepAskBuild = buildStep.StepAskBuild;
@@ -197,6 +197,51 @@ export interface Program {
"path": string;
}
/** what a run ask's step answers: whether this run built the program or took the one kept under its key, the key, the program's exit, the last line of its standard output and the first of its standard error */
export interface RunAnswer {
"built": boolean;
"exit": number;
"key": string;
/** raw when it is JSON, else a JSON string */
"output"?: unknown;
"said"?: string;
}
/** one commit a run's program reads */
export interface RunHead {
"commit": string;
"repository": string;
}
/** a run's input by reference: the seat verb the asker serves it on (seat:<seat>.<verb>), the sha256 of its bytes and its size, at most 256 KiB */
export interface RunInput {
/** 64 lowercase hex digits */
"digest": string;
/** bytes */
"size": number;
"verb": string;
}
/** a run ask's job: where its program is built, the heads it reads (each laid read-only at its repository's path), its input by reference, its arguments and what it is for */
export interface RunJob {
/** the program's arguments, each a word or a flag */
"args"?: string[];
/** the commits the program reads, a repository at most once */
"heads": RunHead[];
"input": RunInput;
"program": RunProgram;
/** what the run is for, for a reader of the ask */
"reader"?: string;
}
/** where a run's program is built: the repository and full commit fetched, the Go module's directory in it and the program's import path */
export interface RunProgram {
"commit": string;
"dir": string;
"package": string;
"repository": string;
}
/** one step a build module runs on a node */
export interface Step {
/** when (RFC 3339 time) */
@@ -268,16 +313,20 @@ export interface StepAsk {
"needs"?: string[];
/** package to the tagged version a build pins */
"packages"?: Record<string, string>;
/** a run: a program built from a change and run in the build seat's sandbox over the heads, its input asked of a verb of the asker's seat; the Go seat's check stream alone carries it, as a compose */
"run"?: RunJob;
/** the layout version */
"v"?: number;
}
/** what a step answered, by the kind of ask: a build's artifacts, toolchain, imports and out-of-step modules; a check's fetch and levels; a compose's built, key and verdict; a declined ask's declined, needs and lacks. The fields of the kinds do not meet, so one layout holds the four */
/** what a step answered, by the kind of ask: a build's artifacts, toolchain, imports and out-of-step modules; a check's fetch and levels; a compose's built, key and verdict; a run's built, key, exit, output and said; a declined ask's declined, needs and lacks. The fields of the kinds do not meet, so one layout holds the five */
export interface StepResult {
"artifacts"?: BuiltArtifact[];
"built"?: boolean;
/** the holder declined the ask: its node lacks what the ask needs */
"declined"?: boolean;
/** a run's program's exit status */
"exit"?: number;
"fetch"?: FetchAnswer;
"imports"?: string[];
"key"?: string;
@@ -287,6 +336,10 @@ export interface StepResult {
/** what the declined ask needs */
"needs"?: string[];
"out-of-step"?: string[];
/** a run's program's last line of standard output: raw when it is JSON, else a JSON string */
"output"?: unknown;
/** a run's program's first line of standard error, or why the step could not run it */
"said"?: string;
"toolchain"?: ToolchainUsed;
/** the compose-candidate verdict (the composer's own layout) */
"verdict"?: unknown;
@@ -129,6 +129,11 @@ export type PlacedBeside = buildStep.PlacedBeside;
export type Program = buildStep.Program;
export type ResumeAnswer = buildStep.ResumeAnswer;
export type ResumeInput = buildStep.ResumeInput;
export type RunAnswer = buildStep.RunAnswer;
export type RunHead = buildStep.RunHead;
export type RunInput = buildStep.RunInput;
export type RunJob = buildStep.RunJob;
export type RunProgram = buildStep.RunProgram;
export type Step = buildStep.Step;
export type StepAsk = buildStep.StepAsk;
export type StepAskBuild = buildStep.StepAskBuild;
@@ -197,6 +197,51 @@ export interface Program {
"path": string;
}
/** what a run ask's step answers: whether this run built the program or took the one kept under its key, the key, the program's exit, the last line of its standard output and the first of its standard error */
export interface RunAnswer {
"built": boolean;
"exit": number;
"key": string;
/** raw when it is JSON, else a JSON string */
"output"?: unknown;
"said"?: string;
}
/** one commit a run's program reads */
export interface RunHead {
"commit": string;
"repository": string;
}
/** a run's input by reference: the seat verb the asker serves it on (seat:<seat>.<verb>), the sha256 of its bytes and its size, at most 256 KiB */
export interface RunInput {
/** 64 lowercase hex digits */
"digest": string;
/** bytes */
"size": number;
"verb": string;
}
/** a run ask's job: where its program is built, the heads it reads (each laid read-only at its repository's path), its input by reference, its arguments and what it is for */
export interface RunJob {
/** the program's arguments, each a word or a flag */
"args"?: string[];
/** the commits the program reads, a repository at most once */
"heads": RunHead[];
"input": RunInput;
"program": RunProgram;
/** what the run is for, for a reader of the ask */
"reader"?: string;
}
/** where a run's program is built: the repository and full commit fetched, the Go module's directory in it and the program's import path */
export interface RunProgram {
"commit": string;
"dir": string;
"package": string;
"repository": string;
}
/** one step a build module runs on a node */
export interface Step {
/** when (RFC 3339 time) */
@@ -268,16 +313,20 @@ export interface StepAsk {
"needs"?: string[];
/** package to the tagged version a build pins */
"packages"?: Record<string, string>;
/** a run: a program built from a change and run in the build seat's sandbox over the heads, its input asked of a verb of the asker's seat; the Go seat's check stream alone carries it, as a compose */
"run"?: RunJob;
/** the layout version */
"v"?: number;
}
/** what a step answered, by the kind of ask: a build's artifacts, toolchain, imports and out-of-step modules; a check's fetch and levels; a compose's built, key and verdict; a declined ask's declined, needs and lacks. The fields of the kinds do not meet, so one layout holds the four */
/** what a step answered, by the kind of ask: a build's artifacts, toolchain, imports and out-of-step modules; a check's fetch and levels; a compose's built, key and verdict; a run's built, key, exit, output and said; a declined ask's declined, needs and lacks. The fields of the kinds do not meet, so one layout holds the five */
export interface StepResult {
"artifacts"?: BuiltArtifact[];
"built"?: boolean;
/** the holder declined the ask: its node lacks what the ask needs */
"declined"?: boolean;
/** a run's program's exit status */
"exit"?: number;
"fetch"?: FetchAnswer;
"imports"?: string[];
"key"?: string;
@@ -287,6 +336,10 @@ export interface StepResult {
/** what the declined ask needs */
"needs"?: string[];
"out-of-step"?: string[];
/** a run's program's last line of standard output: raw when it is JSON, else a JSON string */
"output"?: unknown;
/** a run's program's first line of standard error, or why the step could not run it */
"said"?: string;
"toolchain"?: ToolchainUsed;
/** the compose-candidate verdict (the composer's own layout) */
"verdict"?: unknown;
@@ -129,6 +129,11 @@ export type PlacedBeside = buildStep.PlacedBeside;
export type Program = buildStep.Program;
export type ResumeAnswer = buildStep.ResumeAnswer;
export type ResumeInput = buildStep.ResumeInput;
export type RunAnswer = buildStep.RunAnswer;
export type RunHead = buildStep.RunHead;
export type RunInput = buildStep.RunInput;
export type RunJob = buildStep.RunJob;
export type RunProgram = buildStep.RunProgram;
export type Step = buildStep.Step;
export type StepAsk = buildStep.StepAsk;
export type StepAskBuild = buildStep.StepAskBuild;
@@ -197,6 +197,51 @@ export interface Program {
"path": string;
}
/** what a run ask's step answers: whether this run built the program or took the one kept under its key, the key, the program's exit, the last line of its standard output and the first of its standard error */
export interface RunAnswer {
"built": boolean;
"exit": number;
"key": string;
/** raw when it is JSON, else a JSON string */
"output"?: unknown;
"said"?: string;
}
/** one commit a run's program reads */
export interface RunHead {
"commit": string;
"repository": string;
}
/** a run's input by reference: the seat verb the asker serves it on (seat:<seat>.<verb>), the sha256 of its bytes and its size, at most 256 KiB */
export interface RunInput {
/** 64 lowercase hex digits */
"digest": string;
/** bytes */
"size": number;
"verb": string;
}
/** a run ask's job: where its program is built, the heads it reads (each laid read-only at its repository's path), its input by reference, its arguments and what it is for */
export interface RunJob {
/** the program's arguments, each a word or a flag */
"args"?: string[];
/** the commits the program reads, a repository at most once */
"heads": RunHead[];
"input": RunInput;
"program": RunProgram;
/** what the run is for, for a reader of the ask */
"reader"?: string;
}
/** where a run's program is built: the repository and full commit fetched, the Go module's directory in it and the program's import path */
export interface RunProgram {
"commit": string;
"dir": string;
"package": string;
"repository": string;
}
/** one step a build module runs on a node */
export interface Step {
/** when (RFC 3339 time) */
@@ -268,16 +313,20 @@ export interface StepAsk {
"needs"?: string[];
/** package to the tagged version a build pins */
"packages"?: Record<string, string>;
/** a run: a program built from a change and run in the build seat's sandbox over the heads, its input asked of a verb of the asker's seat; the Go seat's check stream alone carries it, as a compose */
"run"?: RunJob;
/** the layout version */
"v"?: number;
}
/** what a step answered, by the kind of ask: a build's artifacts, toolchain, imports and out-of-step modules; a check's fetch and levels; a compose's built, key and verdict; a declined ask's declined, needs and lacks. The fields of the kinds do not meet, so one layout holds the four */
/** what a step answered, by the kind of ask: a build's artifacts, toolchain, imports and out-of-step modules; a check's fetch and levels; a compose's built, key and verdict; a run's built, key, exit, output and said; a declined ask's declined, needs and lacks. The fields of the kinds do not meet, so one layout holds the five */
export interface StepResult {
"artifacts"?: BuiltArtifact[];
"built"?: boolean;
/** the holder declined the ask: its node lacks what the ask needs */
"declined"?: boolean;
/** a run's program's exit status */
"exit"?: number;
"fetch"?: FetchAnswer;
"imports"?: string[];
"key"?: string;
@@ -287,6 +336,10 @@ export interface StepResult {
/** what the declined ask needs */
"needs"?: string[];
"out-of-step"?: string[];
/** a run's program's last line of standard output: raw when it is JSON, else a JSON string */
"output"?: unknown;
/** a run's program's first line of standard error, or why the step could not run it */
"said"?: string;
"toolchain"?: ToolchainUsed;
/** the compose-candidate verdict (the composer's own layout) */
"verdict"?: unknown;
@@ -610,6 +610,10 @@ func TestWantsAreWrittenAndNoDifferenceIs(t *testing.T) {
}); err != nil {
t.Fatal(err)
}
// The responder's interest is on the bus before the event that asks it is said: the assigner asks a node once per
// manifest-reads-changed and never on a timer, so an ask that lands before this subscription reached the server is
// answered by nobody and is never asked again (the group check of 616c42434ac2 on a loaded build node).
subscribed(t, nc)
say := func(why string) {
body, _ := json.Marshal(nodeengine.ManifestReadsChangedEvent{V: 1, Node: "ace", ManifestReads: stated, At: time.Now().Format(time.RFC3339Nano), Why: why})
if _, err := b.js.Publish(b.ctx, enginereads.SubjectPrefix+"ace", body); err != nil {
@@ -677,6 +681,16 @@ func TestWantsAreWrittenAndNoDifferenceIs(t *testing.T) {
}
}
// subscribed returns once the server holds every subscription nc made: nats.go sends a SUB from its flusher, after
// Subscribe has returned, so a responder or listener the test makes is not yet heard by a call or event another
// connection sends at once. A flush's round trip is the barrier; no sleep stands in for it.
func subscribed(t *testing.T, nc *nats.Conn) {
t.Helper()
if err := nc.Flush(); err != nil {
t.Fatalf("the test's subscriptions did not reach the bus: %v", err)
}
}
// bedState is one of the bed's buckets as the engine reads' tool reads it.
type bedState struct {
b *bed
@@ -1166,6 +1180,7 @@ func TestAProposalIsAnsweredApprovedAndWritten(t *testing.T) {
}); err != nil {
t.Fatal(err)
}
subscribed(t, nc)
for _, node := range []string{"anchor", "laptop"} {
rec, _ := nodeengineclient.EncodeHaveNode(nodeengine.HaveByNodeNodeValue{Record: &nodeengine.NodeRecord{Node: node}})
b.putJSON(t, buckets.Have, node+".node", rec)
@@ -1514,6 +1529,7 @@ func serveIndexer(t *testing.T, b *bed, nc *nats.Conn) {
if err != nil {
t.Fatal(err)
}
subscribed(t, nc)
t.Cleanup(func() { _ = sub.Unsubscribe() })
}
}
@@ -1612,6 +1628,7 @@ func firstViewsBeforeAnyNewFormState(t *testing.T, b *bed, granted map[string][]
if err != nil {
t.Fatal(err)
}
subscribed(t, nc)
t.Cleanup(func() { _ = sub.Unsubscribe() })
ask := func(user, tool string) (views.Answer, error) {
return views.AskVia(func(key string, body any) (json.RawMessage, error) {
@@ -29,6 +29,7 @@ import (
oc "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/seats/operatorchannel"
"git.novox.be/novox/mesh-sdk/go/announce"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/seatlist"
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient/enginereads"
"git.novox.be/novox/mesh-sdk/go/views"
@@ -415,6 +416,12 @@ func follow(ctx context.Context, js jetstream.JetStream, nc process.Subscriber,
// The assigner's one pull of its durable on the shared events stream: the indexer's views and the engines' reads changed
// are handlers of it, started once both are registered (process.EventHub).
hub := process.NewEventHub(logger.Printf)
// The seat list (seatlist.go), read over the connection and heard on the hub, given to catalogue at every change, and a
// pass made then: registered before the hub starts.
gate := &seatGate{}
if bus != nil {
gate.follow(seatlist.ProcessBus(bus.Call, hub), wake, logger.Printf)
}
// The indexer's manifests, modules and module graph, through its first views and views family, never its buckets
// (DLV-F2r, ADR 0631): the first pass waits for the three answers, as it waits for every watch to hand what stands.
if iv != nil {
@@ -465,12 +472,12 @@ func follow(ctx context.Context, js jetstream.JetStream, nc process.Subscriber,
}
}
p := &passer{kvs: kvs, w: w, logf: logger.Printf, say: say, findings: findings, said: map[string]bool{}, answeredAt: map[string]uint64{},
reads: reads}
reads: reads, seats: gate}
// The node-engine seat's manifest-reads-changed, for one node (ADR 0618 §4): that node is asked again, with no timer.
// Heard on the assigner's durable consumer on EVENTS, which the grants part makes from the manifest's consumes; where it
// is not there yet, a node is asked again only when it comes back online.
if js != nil {
hearReadsChanged(ctx, js, hub, reads, findings, logger.Printf)
hearReadsChanged(ctx, js, hub, reads, findings, gate.eventsLive, logger.Printf)
}
if bus != nil && kvs[buckets.Puts] != nil {
p.acts = newPutActs(bus, wake, logger.Printf)
@@ -540,7 +547,7 @@ func sayVia(ctx context.Context, js jetstream.JetStream) func(oc.Condition) erro
// hearReadsChanged hears the node-engine seat's event manifest-reads-changed on the assigner's one hub of its durable
// consumer on EVENTS, and asks the node it names again; the hub also hands the indexer's views to the followers of
// indexerViews, and starts once both are registered.
func hearReadsChanged(ctx context.Context, js jetstream.JetStream, hub *process.EventHub, reads *enginereads.Keeper, findings assign.Findings, logf func(string, ...any)) {
func hearReadsChanged(ctx context.Context, js jetstream.JetStream, hub *process.EventHub, reads *enginereads.Keeper, findings assign.Findings, live func(), logf func(string, ...any)) {
account := process.Account{Node: thisNode, Module: assignment.Module}
w := notHeardWriter(findings, logf)
handle := func(m jetstream.Msg) {
@@ -566,6 +573,9 @@ func hearReadsChanged(ctx context.Context, js jetstream.JetStream, hub *process.
return
}
w.Heard(process.EventsStream)
if live != nil {
live() // the seat list, unread while the seat was silent, is read now (seatGate.eventsLive)
}
go func() {
<-ctx.Done()
hub.Stop()
@@ -71,7 +71,10 @@ type passer struct {
callsCleared bool
// The seat mesh-module-assigner's holder (seat.go, ADR 0567): its acts (nil: none is made), how it says
// the seat's events (nil: none is said), the proposals it said, and what Intended computed by proposal and base.
seat *seatActs
seat *seatActs
// seats is the seat list the process reads (seatlist.go): until it is read, seat-holders is not written and a proposal
// laid again is not ended on a seat it does not name (ADR 0636 §1). Nil reads nothing.
seats *seatGate
publish publisher
saidProposed map[string]bool
// copied says the retired state was copied into `assignments` by this process, or found empty (copyRetired).
@@ -1119,6 +1122,14 @@ func (p *passer) seatHolders(ctx context.Context) {
if len(assignments) == 0 {
return // no assignments read yet: nothing is known of who holds what, and nothing is deleted
}
if !p.seats.read() {
// ADR 0636 §1: which claim names which seat is the list's, so nothing is written from the bootstrap alone; the
// reader's change makes a pass.
if p.seats != nil {
p.seats.waitSaid.Do(func() { p.logf("seat-holders is not written until the seat list is read: %s", p.seats.health()) })
}
return
}
want := plc.SeatHoldersOf(assignments)
have := p.w.read(plc.SeatHoldersBucket)
for key, sh := range want {
@@ -293,6 +293,9 @@ func (s memStore) put(state, key string, value []byte) error {
return nil
}
func (s memStore) keys(state string) ([]string, error) {
if s[state] == nil { // as the runtime refuses a state the module neither keeps nor reads
return nil, fmt.Errorf("mesh-delivery-assigner keeps and reads no state called %q", state)
}
var out []string
for k := range s[state].values {
out = append(out, k)
@@ -362,9 +365,14 @@ func putOfTheBus(t *testing.T) {
buildKV := mem()
buildKV.values[bkey] = built
modules := mem()
modules.values["nats"] = mod
s := newPutTools(memStore{putsState: putsKV, builds.BuilderModule + ".builds": buildKV, buckets.IndexerModule + ".modules": modules})
// Issue 1966: newest is asked of the indexer's first view of modules, never read from its bus state.
indexer := &fakeIndexer{views: map[string]map[string][]byte{"mesh-delivery-indexer.modules": {"mesh-catalog": mod, "nats": mod}}}
s := newPutTools(memStore{putsState: putsKV, builds.BuilderModule + ".builds": buildKV}, func(key string, body any) (json.RawMessage, error) {
if raw, ok, err := indexer.answer(key, body); ok {
return raw, err
}
return nil, fmt.Errorf("%s is not served here", key)
})
s.now = func() time.Time { return now }
was := answerWithin
answerWithin = 0
@@ -375,8 +383,11 @@ func putOfTheBus(t *testing.T) {
if got, err := s.version("nats", "fd497ab80000"); err != nil || got != fp {
t.Errorf("the prefix of the fingerprint built reads %q, %v", got, err)
}
if got, err := s.version("nats", "newest"); err != nil || got != fp {
t.Errorf("newest reads %q, %v", got, err)
if got, err := s.version("nats", "newest"); err != nil || got != fp || indexer.asks["mesh-delivery-indexer.modules"] != 2 {
t.Errorf("newest reads %q, %v, asking %d pages of the indexer's first view (want its 2)", got, err, indexer.asks["mesh-delivery-indexer.modules"])
}
if _, err := s.version("ghost", "newest"); err == nil || !strings.Contains(err.Error(), "holds no module ghost") {
t.Errorf("newest of a module the indexer does not hold: %v", err)
}
ans, err := s.doPut(map[string]any{"node": "novox", "module": "nats", "version": fp, "hold": "true", "why": "the bus's user list"}, "shanks.node-tools")
if err != nil {
@@ -440,7 +440,12 @@ func (p *passer) mustAnswer(ctx context.Context, prop mma.Proposal, rev uint64,
func (p *passer) layAgain(ctx context.Context, prop mma.Proposal, rev uint64, current map[string]mma.Assignments, revs map[string]uint64, why string) {
held, recorded := p.lays(ctx)
again, laid, findings := assign.Propose(assign.ProposeIn{ID: prop.ID, By: prop.By, Why: prop.Why, At: prop.At, Changes: prop.Changes,
Current: current, Revisions: revs, Held: held, Recorded: recorded})
Current: current, Revisions: revs, Held: held, Recorded: recorded, SeatListRead: p.seats.read()})
if slices.ContainsFunc(findings, plc.WaitsForSeatList) {
// ADR 0636 §1: nothing is ended on a list not read; the reader's change makes a pass, which lays it again.
p.logf("the proposal %s is not laid again yet: %s", prop.ID, p.seats.health())
return
}
if len(findings) > 0 {
p.endProposal(ctx, prop, why+", and laid over the keys now it is refused: "+strings.Join(findings, "; "))
return
@@ -1,9 +1,11 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"maps"
"mesh-delivery-assigner/internal/buckets"
"slices"
"sort"
@@ -17,6 +19,7 @@ import (
mmac "git.novox.be/novox/mesh-contracts/go/clients/meshmoduleassignerclient"
mma "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/views"
"mesh-delivery-assigner/internal/assign"
plc "mesh-delivery-assigner/internal/assignment"
@@ -42,17 +45,48 @@ func (runtimeStore) history(state, key string) ([]stdio.StateRevision, error) {
}
// The states the verbs read, as the runtime names them: the seat's own by their local names, the others'
// `<module>.<state>`.
// `<module>.<state>` where the manifest declares them under `reads`.
const haveState = "node-engine.have"
// The indexer's states the verbs read, each asked of the indexer's first-view tool of that name, never read from its bus
// state (novox/hq ADR 0596; ADR 0615 §1, ADR 0621 §1, §2: the manifest invokes them, and the indexer answers them to the
// assigner alone of the delivery's parts).
const (
modulesState = buckets.IndexerModule + ".modules"
haveState = "node-engine.have"
indexerModules = "modules"
indexerManifests = "manifests"
)
// indexerView is one of the indexer's states whole, asked of its first-view tool through ask (stdio.Ask, or a test's) page
// after page, each page continued after the one before's next. It is asked again at every call: the indexer's state is
// the truth, and the verbs keep no copy of it.
func indexerView(ask func(string, any) (json.RawMessage, error), state string) (map[string]json.RawMessage, error) {
tool := buckets.IndexerModule + "." + state
if ask == nil {
return nil, fmt.Errorf("%s cannot be asked here", tool)
}
page := views.AskVia(ask, tool)
out := map[string]json.RawMessage{}
for after := ""; ; {
a, err := page(context.Background(), after)
if err != nil {
return nil, fmt.Errorf("%s does not answer: %w", tool, err)
}
maps.Copy(out, a.Values)
if a.Next == "" || a.Next == after {
return out, nil
}
after = a.Next
}
}
// seatTools serves the seat's verbs over a store, asking the controller through ask (stdio.Ask, or a test's).
type seatTools struct {
st seatStore
ask func(key string, body any) (json.RawMessage, error)
now func() time.Time
// seats is the seat list the bundle reads (seatlist.go): until it is read, propose judges no mesh seat the bootstrapped
// definitions do not name as held once (ADR 0636 §1). Nil reads nothing.
seatList *seatGate
}
func newSeatTools(st seatStore, ask func(string, any) (json.RawMessage, error)) *seatTools {
@@ -97,14 +131,15 @@ func (s *seatTools) assignments() (map[string]mma.Assignments, map[string]uint64
return out, revs, nil
}
// lays is what a proposal is checked against: the modules the indexer holds and the nodes with a record.
// lays is what a proposal is checked against: the modules the indexer holds, as its first view of modules answers them,
// and the nodes with a record.
func (s *seatTools) lays() (func(string) bool, func(string) bool, error) {
mods, err := s.st.keys(modulesState)
mods, err := indexerView(s.ask, indexerModules)
if err != nil {
return nil, nil, fmt.Errorf("the indexer's modules cannot be listed: %w", err)
}
held := map[string]bool{}
for _, k := range mods {
for k := range mods {
if name, err := meshcatalogclient.ParseModulesKey(k); err == nil {
held[name] = true
}
@@ -230,7 +265,7 @@ func (s *seatTools) doPropose(args map[string]any, by string) (any, error) {
enrollers := plc.EnrollersOf(current)
now := s.now()
p, laid, findings := assign.Propose(assign.ProposeIn{ID: plc.NewProposalID(now), By: by, Why: arg(args, "why"), At: now,
Changes: changes, Current: current, Revisions: revs, Held: held, Recorded: recorded, Warrant: warrant})
Changes: changes, Current: current, Revisions: revs, Held: held, Recorded: recorded, Warrant: warrant, SeatListRead: s.seatList.read()})
if refusal := plc.AddNodeRefusal(p, moduleOfCaller(by, enrollers), enrollers); refusal != "" {
findings = append(findings, refusal)
}
@@ -26,6 +26,7 @@ import (
"git.novox.be/novox/mesh-contracts/provisioning/go/clients/meshprovisionerclient"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/views"
"github.com/nats-io/nats.go/jetstream"
"mesh-delivery-assigner/internal/assign"
@@ -52,11 +53,13 @@ func TestTheSeatsVerbs(t *testing.T) {
anchor := mma.Assignments{V: keys.Version, Node: "anchor", Modules: []mma.PlacedModule{{Module: "web"}, {Module: "mesh-enrolment"}},
Holds: []mma.Holding{{Claim: "mesh-node-enroller", Scope: "mesh", Module: "mesh-enrolment"}}}
st := memStore{}
for _, state := range []string{mma.StateAssignments, mma.StateProposals, mma.StateCandidates, modulesState, haveState} {
for _, state := range []string{mma.StateAssignments, mma.StateProposals, mma.StateCandidates, haveState} {
st[state] = &memKV{values: map[string][]byte{}}
}
// Issue 1966: the indexer's modules are asked of its first view, never read from its bus state (ADR 0596, ADR 0621).
indexer := &fakeIndexer{views: map[string]map[string][]byte{"mesh-delivery-indexer.modules": {}}}
for _, m := range []string{"web", "shell", "mesh-enrolment"} {
st[modulesState].values[m] = []byte("{}")
indexer.views["mesh-delivery-indexer.modules"][m] = []byte("{}")
}
for _, n := range []string{"anchor", "laptop"} {
raw, _ := nodeengineclient.EncodeHaveNode(nodeengine.HaveByNodeNodeValue{Record: &nodeengine.NodeRecord{Node: n}})
@@ -65,6 +68,9 @@ func TestTheSeatsVerbs(t *testing.T) {
var asked []string
s := newSeatTools(st, func(key string, body any) (json.RawMessage, error) {
asked = append(asked, key)
if raw, ok, err := indexer.answer(key, body); ok {
return raw, err
}
return json.RawMessage(`{"ok":true}`), nil
})
s.now = func() time.Time { return time.Date(2026, 10, 28, 20, 0, 0, 0, time.UTC) }
@@ -193,6 +199,59 @@ func TestTheSeatsVerbs(t *testing.T) {
if tools := s.tools(); len(tools) != 5 || tools[2].Name != "mesh-module-assigner.propose" || tools[2].Run == nil {
t.Errorf("the seat's verbs as served: %+v", tools)
}
// Issue 1966: propose checks its modules against the indexer's first view of modules, every page of it, asked again at
// each call; a first view the indexer refuses refuses the proposal, naming the tool, and writes nothing.
for _, c := range []struct {
name, module, refused, want string
}{
{"a module on the last page", "web", "", ""},
{"a module the indexer does not hold", "ghost", "", "ghost"},
{"the first view refused", "web", "mesh-delivery-indexer.modules", "mesh-delivery-indexer.modules does not answer"},
} {
t.Run("1966: "+c.name, func(t *testing.T) {
indexer.refused, indexer.asks = c.refused, nil
maps.DeleteFunc(st[mma.StateAssignments].values, func(k string, _ []byte) bool { return strings.HasPrefix(k, "joiner.") }) // the remove-node's joiner
st[mma.StateProposals].values, st[mma.StateCandidates].values = map[string][]byte{}, map[string][]byte{}
_, err := s.doPropose(map[string]any{"why": "x", "changes": []any{map[string]any{"op": "place", "node": "laptop", "module": c.module}}}, "op")
switch {
case c.want == "" && err != nil:
t.Errorf("propose: %v", err)
case c.want != "" && (err == nil || !strings.Contains(err.Error(), c.want) || len(st[mma.StateProposals].values) != 0):
t.Errorf("propose: %v, want a refusal saying %q and nothing written", err, c.want)
}
if pages := indexer.asks["mesh-delivery-indexer.modules"]; c.refused == "" && pages != 3 {
t.Errorf("the first view of modules was asked %d pages, want its 3", pages)
}
})
}
indexer.refused = ""
// Issue 1966: the replicated mesh-dns-resolver held on two nodes is proposed through the verb as the bundle serves it,
// over the seat list it reads (ADR 0636 §1): unread, the check says it waits for the list and writes nothing; read, the
// proposal is laid.
for _, c := range []struct {
name string
silent bool
want string
}{
{"a replicated seat on two nodes waits for an unread seat list", true, "the seat list is not read yet, so whether the seat mesh-dns-resolver may be held on anchor and laptop waits for it"},
{"a replicated seat on two nodes, the seat list read", false, ""},
} {
t.Run("1966: "+c.name, func(t *testing.T) {
st[mma.StateProposals].values, st[mma.StateCandidates].values = map[string][]byte{}, map[string][]byte{}
s.seatList = gateOver(t, c.silent)
_, err := s.doPropose(map[string]any{"why": "a second resolver", "changes": []any{
map[string]any{"op": "hold", "node": "anchor", "module": "web", "claim": "mesh-dns-resolver", "scope": "mesh"},
map[string]any{"op": "place", "node": "laptop", "module": "web"},
map[string]any{"op": "hold", "node": "laptop", "module": "web", "claim": "mesh-dns-resolver", "scope": "mesh"}}}, "op")
switch {
case c.want == "" && err != nil:
t.Errorf("propose: %v", err)
case c.want != "" && (err == nil || !strings.Contains(err.Error(), c.want) || strings.Contains(err.Error(), "held once") || len(st[mma.StateProposals].values) != 0):
t.Errorf("propose: %v, want a refusal saying %q, never held once, and nothing written", err, c.want)
}
})
}
s.seatList = nil
t.Run("1953/D4-D6: the seat mesh-wants' upgrade and bus", wantsVerbs)
t.Run("1955/L7 (ADR 0571 §6): rule 7 reads the node's own composition; an unplaced node is said", func(t *testing.T) {
w := newWorld()
@@ -226,14 +285,24 @@ func TestTheSeatsVerbs(t *testing.T) {
t.Run("1955/L7 (ADR 0571 §4): seat-holders is written from assignments, only the keys a change moves, and answered by assignments seats", func(t *testing.T) {
writes := 0
kv := &countingKV{memKV: memKV{values: map[string][]byte{}}, writes: &writes}
p := &passer{w: newWorld(), logf: t.Logf, kvs: kvSet{plc.SeatHoldersBucket: kv}}
// Issue 1966: nothing is written while the seat list is unread (ADR 0636 §1), and every seat the list names is no
// seat outside it once read.
p := &passer{w: newWorld(), logf: t.Logf, kvs: kvSet{plc.SeatHoldersBucket: kv}, seats: gateOver(t, true)}
place := func(node string, holds ...mma.Holding) {
putAssignments(p.w, mma.Assignments{V: keys.Version, Node: node, Modules: []mma.PlacedModule{{Module: "web"}, {Module: "nats"}}, Holds: holds})
}
place("ace", mma.Holding{Claim: "mesh-broker", Scope: "mesh", Module: "nats"}, mma.Holding{Claim: "node-shell", Scope: "node", Module: "web"})
place("g14", mma.Holding{Claim: "node-shell", Scope: "node", Module: "web"})
ctx := context.Background()
if p.seatHolders(ctx); writes != 0 {
t.Fatalf("seat-holders was written from an unread seat list: %v", kv.values)
}
p.seats = gateOver(t, false)
p.seatHolders(ctx)
var broker mma.SeatHolders
if err := json.Unmarshal(kv.values["mesh-broker"], &broker); err != nil || broker.Outside {
t.Errorf("mesh-broker's holders %+v, %v: a seat the list names is no seat outside it", broker, err)
}
if writes != 3 || kv.values["mesh-broker"] == nil || kv.values["node-shell.ace"] == nil || kv.values["node-shell.g14"] == nil {
t.Fatalf("%d writes: %v", writes, kv.values)
}
@@ -439,9 +508,11 @@ func TestTheSeatsVerbs(t *testing.T) {
func wantsVerbs(t *testing.T) {
fpOld, fpNew := meshcatalogclient.SourcePrefix2+strings.Repeat("1", 64), meshcatalogclient.SourcePrefix2+strings.Repeat("2", 64)
st := memStore{}
for _, state := range []string{modulesState, manifestsState, policiesState, haveState, putsState, "findings", buckets.ComposedRead} {
for _, state := range []string{policiesState, haveState, putsState, "findings", buckets.ComposedRead} {
st[state] = &memKV{values: map[string][]byte{}}
}
// Issue 1966: the indexer's modules and manifests are asked of its first views, never read from its bus state.
indexer := &fakeIndexer{views: map[string]map[string][]byte{"mesh-delivery-indexer.modules": {}, "mesh-delivery-indexer.manifests": {}}}
// ADR 0593 §6, §8: what each node published, every node's bucket as one state: anchor's own composition and its web at
// a fingerprint, and laptop's web refused there.
partNew, _ := meshcatalogclient.KeyPart(fpNew)
@@ -453,8 +524,8 @@ func wantsVerbs(t *testing.T) {
manifest := func(name, extra string) {
raw := `{"module":"` + name + `","version":"1"` + extra + `}`
digest := meshcatalogclient.ManifestDigest([]byte(raw))
st[manifestsState].values[digest], _ = meshcatalogclient.EncodeManifest(meshcatalog.ManifestsByDigestValue{V: keys.Version, Module: name, Manifest: raw})
st[modulesState].values[name], _ = meshcatalogclient.EncodeModule(meshcatalog.ModulesByModuleValue{V: keys.Version, Fp2: fpNew, Manifest: digest, Repository: "novox/x", Commit: "c"})
indexer.views["mesh-delivery-indexer.manifests"][digest], _ = meshcatalogclient.EncodeManifest(meshcatalog.ManifestsByDigestValue{V: keys.Version, Module: name, Manifest: raw})
indexer.views["mesh-delivery-indexer.modules"][name], _ = meshcatalogclient.EncodeModule(meshcatalog.ModulesByModuleValue{V: keys.Version, Fp2: fpNew, Manifest: digest, Repository: "novox/x", Commit: "c"})
}
manifest("web", "")
manifest("photos", `,"upgrade":{"policy":"record","why":"a person checks the albums"}`)
@@ -465,10 +536,14 @@ func wantsVerbs(t *testing.T) {
st[haveState].values["anchor.node"], _ = nodeengineclient.EncodeHaveNode(nodeengine.HaveByNodeNodeValue{Composed: "sha256:" + strings.Repeat("c", 64)})
var asked []string
var said []string
w := newWantsTools(st, func(key string, body any) (json.RawMessage, error) {
ask := func(key string, body any) (json.RawMessage, error) {
asked = append(asked, key)
if raw, ok, err := indexer.answer(key, body); ok {
return raw, err
}
return json.RawMessage(`{"ok":true}`), nil
}, func(event, id string, body any) error { said = append(said, event+" "+id); return nil })
}
w := newWantsTools(st, ask, func(event, id string, body any) error { said = append(said, event+" "+id); return nil })
policyOf := func(module string) policyRow {
out, err := w.doUpgrade(map[string]any{"module": module}, "op")
if err != nil {
@@ -497,7 +572,7 @@ func wantsVerbs(t *testing.T) {
// Read by watch (issue 1953 L5f): the same answer, each watched state started once, and no get per key of the
// large states (the modules, their manifests, the composed record).
watches := map[string]int{}
byWatch := newWantsTools(st, nil, nil)
byWatch := newWantsTools(st, ask, nil)
byWatch.watched = stdio.WatchedStatesOver(func(state string, handle func(stdio.StateChange) error) error {
watches[state]++
for k, v := range st[state].values {
@@ -518,10 +593,13 @@ func wantsVerbs(t *testing.T) {
t.Errorf("compare by watch = %+v", got)
}
}
if n := watches[manifestsState+" by key"]; n == 0 || n > len(st[manifestsState].values) {
t.Errorf("the manifests were read by key %d times, want each once", n)
for _, state := range []string{"mesh-delivery-indexer.modules", "mesh-delivery-indexer.manifests"} {
if watches[state] != 0 || watches[state+" by key"] != 0 || indexer.asks[state] == 0 {
t.Errorf("the indexer's %s was watched %d times and read by key %d, asked of its first view %d times: want asked alone",
state, watches[state], watches[state+" by key"], indexer.asks[state])
}
}
for _, state := range []string{modulesState, haveState, mcc.FindingsState} {
for _, state := range []string{haveState, mcc.FindingsState} {
if watches[state] != 1 {
t.Errorf("%s was watched %d times, want once", state, watches[state])
}
@@ -742,3 +820,42 @@ func (k *copiesKV) Put(_ context.Context, key string, value []byte) (uint64, err
k.values[key] = value
return uint64(len(k.values)), nil
}
// fakeIndexer answers the indexer's first-view tools as its runtime does (views.FirstViewTool): each tool its state's values
// in key order, one key a page, each page continued after the page before's next; the tool named refused answers its
// refusal. asks counts the pages asked by tool.
type fakeIndexer struct {
views map[string]map[string][]byte
refused string
asks map[string]int
}
// answer is the answer to a call of key, and whether key is one of the indexer's first views at all.
func (f *fakeIndexer) answer(key string, body any) (json.RawMessage, bool, error) {
values, ok := f.views[key]
if !ok {
return nil, false, nil
}
if f.asks == nil {
f.asks = map[string]int{}
}
f.asks[key]++
if key == f.refused {
return nil, true, fmt.Errorf("%s is refused to \"novox.mesh-delivery-assigner\"", key)
}
after, _ := body.(map[string]any)["after"].(string)
a := views.Answer{V: views.Version, Owner: buckets.IndexerModule, State: strings.TrimPrefix(key, buckets.IndexerModule+"."), Revision: 7,
Values: map[string]json.RawMessage{}}
for _, k := range slices.Sorted(maps.Keys(values)) {
if k <= after {
continue
}
if len(a.Values) == 1 {
a.Next = slices.Collect(maps.Keys(a.Values))[0]
break
}
a.Values[k] = values[k]
}
raw, err := json.Marshal(a)
return raw, true, err
}
@@ -0,0 +1,101 @@
package main
import (
"errors"
"sync"
"sync/atomic"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/seatlist"
)
// The assigner reads the seat list as every part does (novox/hq ADR 0635 §1, ADR 0636 §1, §7; issue 1966): through the seat
// mesh-catalog's verb seat-definitions and its two events, from the bootstrapped definitions until the seat has answered. What
// the reader holds is given to catalogue at every change, which the check of a proposal and the seat-holders state read: which
// mesh seat is replicated, and which name is a seat's. Both of its runtimes hold one: the process (the proposals it lays again,
// the state seat-holders) over its connection and the event hub, and the tools' bundle (the seat's verb propose) over the
// runtime. Until the list is read, a mesh seat the bootstrapped definitions do not name is never judged held once
// (plc.CheckAssignments), and seat-holders is not written.
// Seat list tools (ADR 0636 §7): what the reader holds, and reading it again now, which every operator's session and the
// healers may ask.
const (
seatListTool = "assigner_seat_list"
seatListAgainTool = "assigner_seat_list_again"
)
var seatListAgainCallers = []string{stdio.AnyPerson, "mesh-healers"}
// seatGate is the reader a runtime of the assigner judges by, set once it is started; before, the list counts as unread.
type seatGate struct {
reader atomic.Pointer[seatlist.Reader]
// waitSaid says the journal was told that seat-holders waits for the list, so it is told once per wait.
waitSaid sync.Once
}
// read says the seat mesh-catalog has answered the reader: the list is read (seatlist.Status's Heard). A nil gate, or one
// whose reader has not started, has read nothing.
func (g *seatGate) read() bool {
if g == nil {
return false
}
r := g.reader.Load()
return r != nil && r.Status().Heard
}
// health is why the list is not read, or empty.
func (g *seatGate) health() string {
if g == nil {
return "the assigner follows no seat list"
}
r := g.reader.Load()
if r == nil {
return "the seat list reader has not started yet"
}
return r.Status().Health()
}
// follow starts the reader over bus: what it holds is given to catalogue at once and at every change, and changed is called
// then (the process makes a pass, which lays again what waited).
func (g *seatGate) follow(bus seatlist.Bus, changed func(), logf func(string, ...any)) *seatlist.Reader {
reader, err := seatlist.Start(seatlist.Config{Bus: bus, Bootstrap: seatlist.Bootstrapped(),
OnChange: func(seats []catalogue.Seat) {
catalogue.UseSeats(seats)
if changed != nil {
changed()
}
}})
if err != nil {
logf("the assigner cannot follow its seat list, so a mesh seat the bootstrapped definitions do not name waits for it: %v", err)
}
if reader != nil {
catalogue.UseSeats(reader.Seats())
g.reader.Store(reader)
}
return reader
}
// eventsLive is called once the process's consumer on EVENTS is heard: the seat may have said it speaks before that consumer
// existed, whose events start at the stream's end, so while the list is unread it is read now (ADR 0636 §1).
func (g *seatGate) eventsLive() {
if r := g.reader.Load(); r != nil {
r.ReloadIfUnread()
}
}
// tools are the reader's two tools over whatever reader the gate holds at the call.
func (g *seatGate) tools() []stdio.Tool {
shape := seatlist.Tools(&seatlist.Reader{}, seatListTool, seatListAgainTool, seatListAgainCallers)
for i := range shape {
run := i
shape[i].Run = func(args map[string]any) (any, error) {
r := g.reader.Load()
if r == nil {
return nil, errors.New("the seat list reader has not started yet: the runtime has not served this module")
}
return seatlist.Tools(r, seatListTool, seatListAgainTool, seatListAgainCallers)[run].Run(args)
}
}
return shape
}
@@ -0,0 +1,63 @@
package main
import (
"encoding/json"
"errors"
"sync"
"testing"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"git.novox.be/novox/mesh-sdk/go/seatlist"
"git.novox.be/novox/mesh-sdk/go/seatlist/seatlisttest"
)
// testDefinitions are seatlisttest's, parsed once.
var testDefinitions = sync.OnceValue(func() []seatdef.Definition { return seatlisttest.Definitions() })
// standInCatalog is the seat mesh-catalog as a test's reader asks it (ADR 0636 §4): every definition of seatlisttest, or,
// silent, no answer at all.
type standInCatalog struct{ silent bool }
func (c standInCatalog) Ask(key string, body any) (json.RawMessage, error) {
if c.silent || key != seatlist.Key() {
return nil, errors.New("no responders")
}
defs := testDefinitions()
seat, _ := body.(map[string]any)["seat"].(string)
if seat == "" {
l := seatlist.Listing{Revision: 1}
for _, d := range defs {
l.Seats = append(l.Seats, seatlist.Entry{Seat: d.Seat, Source: seatlist.SourceTrunk, Digest: "d-" + d.Seat})
}
return json.Marshal(l)
}
for _, d := range defs {
if d.Seat == seat {
raw, err := json.Marshal(d)
if err != nil {
return nil, err
}
return json.Marshal(seatlist.Answer{Seat: seat, Found: true, Revision: 1,
Entry: &seatlist.Entry{Seat: seat, Source: seatlist.SourceTrunk, Digest: "d-" + seat, Definition: raw}})
}
}
return json.Marshal(seatlist.Answer{Seat: seat, Revision: 1})
}
func (standInCatalog) Subscribe(string, func(string, json.RawMessage) error) error { return nil }
// gateOver is a seat gate following the stand-in, once its first read has ended. A read one gives catalogue the test list
// (catalogue.UseSeats), as a host's does, for the tests after it too: the list is the same everywhere.
func gateOver(t *testing.T, silent bool) *seatGate {
t.Helper()
g := &seatGate{}
r := g.follow(standInCatalog{silent: silent}, nil, t.Logf)
if r == nil {
t.Fatal("the reader did not start")
}
<-r.Ready()
if g.read() == silent {
t.Fatalf("the gate read %v over a stand-in silent %v: %s", g.read(), silent, g.health())
}
return g
}
@@ -4,7 +4,6 @@ import (
"encoding/json"
"errors"
"fmt"
"mesh-delivery-assigner/internal/buckets"
"mesh-delivery-assigner/internal/puts"
"os"
"regexp"
@@ -18,6 +17,7 @@ import (
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/seatlist"
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient/enginereads"
)
@@ -45,7 +45,9 @@ type store interface {
// putTools serves the tools over a store, and keeps the newest value of every put it heard, to wait on.
type putTools struct {
st store
st store
// ask asks a module's tool (stdio.Ask, or a test's): the indexer's first view of modules for newest.
ask func(key string, body any) (json.RawMessage, error)
now func() time.Time
mu sync.Mutex
// heard is every put the watch handed, by key; changed is closed and replaced at each.
@@ -53,8 +55,8 @@ type putTools struct {
changed chan struct{}
}
func newPutTools(st store) *putTools {
return &putTools{st: st, now: time.Now, heard: map[string][]byte{}, changed: make(chan struct{})}
func newPutTools(st store, ask func(string, any) (json.RawMessage, error)) *putTools {
return &putTools{st: st, ask: ask, now: time.Now, heard: map[string][]byte{}, changed: make(chan struct{})}
}
// hear takes a put's change in from the watch.
@@ -78,10 +80,12 @@ func (s *putTools) version(module, version string) (string, error) {
v := strings.ToLower(strings.TrimSpace(version))
switch {
case v == "newest":
raw, _, found, err := s.st.get(buckets.ModulesRead, module)
// Asked of the indexer's first view of modules, never read from its bus state (ADR 0596, ADR 0621; issue 1966).
mods, err := indexerView(s.ask, indexerModules)
if err != nil {
return "", fmt.Errorf("the indexer's modules cannot be read: %w", err)
}
raw, found := mods[module]
if !found {
return "", fmt.Errorf("the indexer holds no module %s at the trunk", module)
}
@@ -345,7 +349,7 @@ func (runtimeStore) keys(state string) ([]string, error) { return stdio.State(st
// serveTools serves the tools to the runtime until it stops, hearing every put to answer assigner_put with its first steps.
func serveTools() error {
s := newPutTools(runtimeStore{})
s := newPutTools(runtimeStore{}, stdio.Ask)
go func() {
<-stdio.Served()
for wait := 2 * time.Second; ; wait = min(wait*2, time.Minute) {
@@ -360,7 +364,16 @@ func serveTools() error {
time.Sleep(wait)
}
}()
tools := append(s.tools(), newSeatTools(runtimeStore{}, stdio.Ask).tools()...)
// The seat list (seatlist.go), read through the runtime once Serve has its channel: the seat's verbs judge by it.
gate := &seatGate{}
go func() {
<-stdio.Served()
gate.follow(seatlist.StdioBus(), nil, func(f string, a ...any) { fmt.Fprintf(os.Stderr, "mesh-delivery-assigner: "+f+"\n", a...) })
}()
seatVerbs := newSeatTools(runtimeStore{}, stdio.Ask)
seatVerbs.seatList = gate
tools := append(s.tools(), seatVerbs.tools()...)
tools = append(tools, gate.tools()...)
wants := newWantsTools(runtimeStore{}, stdio.Ask, func(e, id string, b any) error {
// The seat's event, said on the seat: a claimed seat's event is never the module's own.
return stdio.SeatPublish(wantsEventSubject(e), b, id)
@@ -200,6 +200,12 @@ func (iv *indexerViews) start(ctx context.Context, w *world, wake func(), settle
}
})
}
// The hub says when the bus made its consumer: a picture asked before then is asked again (views.Follower.ConsumerMade).
iv.hub.OnBound(func(created time.Time) {
for _, f := range all {
f.ConsumerMade(ctx, created)
}
})
// Not heard, the indexer's views family is the finding family-not-heard until it binds (ADR 0615 §5): the handler is
// registered on the module's hub, which the assigner starts once every handler is.
_ = notHeardWriter(iv.findings, iv.logf).Bind(process.EventsStream, "the indexer's manifests, modules and graph are the first views' alone until the assigner starts again", func() error {
@@ -32,8 +32,7 @@ import (
// The states the verbs read, as the runtime names them.
const (
policiesState = buckets.PoliciesState
manifestsState = buckets.IndexerModule + ".manifests"
policiesState = buckets.PoliciesState
// PolicyChangedEvent is the seat's event of a person's policy set or taken back.
PolicyChangedEvent = "policy-changed"
// busStepCause is the cause the bus's step is recorded under in the hand-act log.
@@ -88,19 +87,14 @@ func (s *wantsTools) direct(state string) (map[string][]byte, error) {
return out, nil
}
// manifest reads a manifest by its digest: kept once read when there is a watched copy, a manifest never changing
// under its digest.
func (s *wantsTools) manifest(digest string) ([]byte, bool, error) {
if s.watched != nil {
return s.watched.Once(manifestsState, digest)
}
raw, _, found, err := s.st.get(manifestsState, digest)
return raw, found, err
}
// manifests is every module the indexer holds, with its trunk manifest and fingerprint.
// manifests is every module the indexer holds, with its trunk manifest and fingerprint: both asked of the indexer's first
// views of modules and manifests (indexerView), never read from its bus state.
func (s *wantsTools) manifests() (map[string]catalogue.Manifest, map[string]meshcatalog.ModulesByModuleValue, error) {
mods, err := s.values(modulesState)
mods, err := indexerView(s.ask, indexerModules)
if err != nil {
return nil, nil, err
}
kept, err := indexerView(s.ask, indexerManifests)
if err != nil {
return nil, nil, err
}
@@ -114,16 +108,16 @@ func (s *wantsTools) manifests() (map[string]catalogue.Manifest, map[string]mesh
if m.Manifest == "" {
continue
}
mraw, found, err := s.manifest(m.Manifest)
if err != nil || !found {
mraw, found := kept[m.Manifest]
if !found {
continue
}
kept, err := meshcatalogclient.DecodeManifest(mraw)
stored, err := meshcatalogclient.DecodeManifest(mraw)
if err != nil {
continue
}
// Decoded, never judged (ADR 0578 §3); one that does not decode is left out of the answer, as before.
if man, err := catalogue.Decode([]byte(kept.Manifest)); err == nil {
if man, err := catalogue.Decode([]byte(stored.Manifest)); err == nil {
out[name] = man
}
}
@@ -36,6 +36,9 @@ type ProposeIn struct {
Recorded func(node string) bool
// Warrant is the enrolment's warrant an add-node carries (ADR 0587 §7); only that change has one.
Warrant *mma.Warrant
// SeatListRead says the host has read the seat list from the seat mesh-catalog (ADR 0636 §1); until then a mesh seat
// the bootstrapped definitions do not name is not judged held once (plc.CheckAssignments).
SeatListRead bool
}
// Propose lays a proposal over the current keys (§2): the proposal (state answering, each changed node's base revision),
@@ -110,7 +113,7 @@ func Propose(in ProposeIn) (mma.Proposal, map[string]*mma.Assignments, []string)
was := recorded
recorded = func(node string) bool { return joining[node] || was(node) }
}
for _, r := range plc.CheckAssignments(all, held, recorded) {
for _, r := range plc.CheckAssignments(all, held, recorded, in.SeatListRead) {
findings = append(findings, r)
}
return p, out, findings
@@ -186,14 +186,15 @@ func holds(m Module, node string, n Nodes, ordering bool) (bool, int, string) {
}
// readerLacking says why the module's want on node is held for the node's engine (ADR 0614 §4): a reader capability its
// manifest needs that the engine does not state, read as the baseline when it states none. It is held under rule 7 (the
// manifest needs that the engine does not state, read as the baseline when it states none, a bootstrapped seat counting as
// read by every engine (graph.MissingReads, ADR 0636 §2). It is held under rule 7 (the
// composition), since no rule of its own is numbered in this release, with words naming the capability; it moves once the
// engine states it. The node-engine's own new build is held only by a word of its own manifest no engine states.
//
// A build's reference forms hold the want only when the engine reads none of them (ADR 0624 §2): every engine reads a
// first form, so a build written in one is never held by its forms; the words name the forms it reads none of.
func (m Module) readerLacking(node string, n Nodes) (string, bool) {
missing := catalogue.MissingManifestReads(m.Needs, n.Reads[node])
missing := graph.MissingReads(m.Needs, n.Reads[node])
if len(missing) == 0 && forms.Current.Held(m.FormNeeds, n.Reads[node]) {
missing = slices.Clone(m.FormNeeds)
}
@@ -138,6 +138,15 @@ func TestEachRuleHoldsTheWantBack(t *testing.T) {
map[string][]string{"ace": catalogue.ManifestReads(), "g14": catalogue.ManifestReads()}, false, nil, ""},
{"the baseline's words on engines stating nothing", "mail", []string{"word:claims.serves", "seat:mesh-catalog", "form:config"}, nil, false, nil, ""},
{"the node-engine's new build, whose contract has more words than its manifest uses", "mesh-host", hostNeeds, nil, false, nil, ""},
// Issue 1966: g14's engine states its words and forms but no seat (its statement made before its seat list was in
// use); needs of the bootstrapped seats mesh-catalog and node-backup move all the same (ADR 0636 §2), one of a seat
// not bootstrapped is held.
{"mesh-catalog on an engine stating no seat", "mesh-host", []string{"seat:mesh-catalog", "word:module"},
map[string][]string{"g14": {"word:module"}}, false, nil, ""},
{"node-backup on an engine stating no seat", "restic", []string{"seat:node-backup", "word:module"},
map[string][]string{"g14": {"word:module"}}, false, nil, ""},
{"a seat not bootstrapped on an engine stating no seat: held, naming it", "mail", []string{"seat:mesh-vault", "word:module"},
map[string][]string{"g14": {"word:module"}}, true, nil, "seat:mesh-vault"},
// ADR 0624 §2, with form:new a form no engine reads unless it states it and artifact-store the first form.
{"g14 back stating {old}, a build carrying {old,new}: moves", "mail", nil, map[string][]string{"g14": {"word:module"}}, false,
[]string{"form:artifact-store", "form:new"}, ""},
@@ -67,17 +67,31 @@ func TestAssignments(t *testing.T) {
t.Run("the check of every assignments", func(t *testing.T) {
known := func(m string) bool { return m != "ghost" }
enrolled := func(n string) bool { return n != "stranger" }
if r := CheckAssignments(current, known, enrolled); len(r) != 0 {
if r := CheckAssignments(current, known, enrolled, true); len(r) != 0 {
t.Fatalf("a sound assignments is refused: %v", r)
}
// A replicated mesh seat (ADR 0223) is held on two nodes; a seat that is not replicated is refused there.
for name, refused := range map[string]bool{"mesh-dns-resolver": false, "mesh-broker": true} {
a := seat.Assignments{Node: "ace", Modules: []seat.PlacedModule{{Module: "dns"}}, Holds: []seat.Holding{{Claim: name, Scope: "mesh", Module: "dns"}}}
// A replicated mesh seat (ADR 0223) is held on two nodes; a seat that is not replicated is refused there. Issue 1966:
// a seat the list does not name, while the list is unread, waits for it and is never said held once (ADR 0636 §1);
// once read, it is judged held once.
for _, c := range []struct {
seat string
listRead bool
want string
}{
{"mesh-dns-resolver", true, ""},
{"mesh-dns-resolver", false, ""},
{"mesh-broker", true, "held once for the mesh, is held on ace and novox"},
{"mesh-broker", false, "held once for the mesh, is held on ace and novox"},
{"a-seat-not-listed", false, "the seat list is not read yet, so whether the seat a-seat-not-listed may be held on ace and novox waits for it"},
{"a-seat-not-listed", true, "held once for the mesh, is held on ace and novox"},
} {
a := seat.Assignments{Node: "ace", Modules: []seat.PlacedModule{{Module: "dns"}}, Holds: []seat.Holding{{Claim: c.seat, Scope: "mesh", Module: "dns"}}}
b := a
b.Node = "novox"
r := CheckAssignments(map[string]seat.Assignments{"ace": a, "novox": b}, known, enrolled)
if (len(r) != 0) != refused {
t.Errorf("%s on two nodes: refusals %v, want refused %v", name, r, refused)
r := CheckAssignments(map[string]seat.Assignments{"ace": a, "novox": b}, known, enrolled, c.listRead)
got := strings.Join(r, "; ")
if (c.want == "") != (len(r) == 0) || !strings.Contains(got, c.want) || (!c.listRead && c.seat == "a-seat-not-listed" && strings.Contains(got, "held once")) {
t.Errorf("%s on two nodes, the list read %v: refusals %v, want %q", c.seat, c.listRead, r, c.want)
}
}
clash := g14
@@ -89,7 +103,7 @@ func TestAssignments(t *testing.T) {
twice := novox
twice.Node = "stranger"
twice.Decided.Address = "10.77.0.2"
r := CheckAssignments(map[string]seat.Assignments{"g14": clash, "novox": novox, "stranger": twice}, known, enrolled)
r := CheckAssignments(map[string]seat.Assignments{"g14": clash, "novox": novox, "stranger": twice}, known, enrolled, true)
for _, want := range []string{"holds no module ghost", "ghost and app are both published on host port 18080",
"bound to ace/postgres", "pin of shell names g14/bash", "stranger is a node with no record",
"mesh-store, held once for the mesh, is held on novox and stranger",
@@ -33,7 +33,13 @@ var KeptApart = []struct{ A, B, Record string }{
// (catalogue.Seat's Replicated, ADR 0223), which is held once per node; two nodes with one address on the private
// network; more than one hub; two modules KeptApart on one node. A host port a manifest declares without a number and a
// requirement left unbound are the composition's to find (its ForeseenPorts and Unbound).
func CheckAssignments(assignments map[string]seat.Assignments, known func(module string) bool, enrolled func(node string) bool) []string {
//
// **A seat the list does not name is not judged held once while the list is unread** (novox/hq ADR 0636 §1, issue 1966):
// listRead says the host has read the seat list from the seat mesh-catalog; before, catalogue holds the bootstrapped
// definitions alone, and a mesh seat it does not name, held on two nodes, may be a replicated one. Its refusal then says the
// judgement waits for the list (SeatListWaits), never that it is held twice; once the list is read, a seat it does not name
// is judged as a seat held once, as before.
func CheckAssignments(assignments map[string]seat.Assignments, known func(module string) bool, enrolled func(node string) bool, listRead bool) []string {
var refusals []string
nodes := sortedKeys(assignments)
placed := func(p seat.Provider) bool {
@@ -105,13 +111,18 @@ func CheckAssignments(assignments map[string]seat.Assignments, known func(module
continue
}
name := h.Claim + "/" + h.Kind
if s, known := catalogue.SeatNamed(h.Claim); known && s.Replicated {
s, defined := catalogue.SeatNamed(h.Claim)
if defined && s.Replicated {
// A replicated mesh seat (ADR 0223) is held on several nodes at once, each answering the same thing;
// it is still held once on each node.
name += "/" + node
}
if other, twice := meshSeats[name]; twice {
refusals = append(refusals, fmt.Sprintf("the seat %s, held once for the mesh, is held on %s and %s", h.Claim, other, node))
if !defined && !listRead {
refusals = append(refusals, fmt.Sprintf(SeatListWaits, h.Claim, other, node))
} else {
refusals = append(refusals, fmt.Sprintf("the seat %s, held once for the mesh, is held on %s and %s", h.Claim, other, node))
}
}
meshSeats[name] = node
}
@@ -122,6 +133,16 @@ func CheckAssignments(assignments map[string]seat.Assignments, known func(module
return refusals
}
// SeatListWaits is the refusal of a mesh seat held on two nodes that the unread seat list does not name (ADR 0636 §1): the
// seat, and the two nodes.
const SeatListWaits = "the seat list is not read yet, so whether the seat %s may be held on %s and %s waits for it: " +
"the assigner judges it once it has read the list from the seat mesh-catalog (novox/hq ADR 0636 §1)"
// WaitsForSeatList says a refusal is SeatListWaits': it ends nothing, and is judged again once the list is read.
func WaitsForSeatList(refusal string) bool {
return strings.HasPrefix(refusal, SeatListWaits[:strings.Index(SeatListWaits, "%")])
}
// Change is what `assignments-changed` says of one node (ADR 0567 §6): the node, the revision of its key before (0: it had
// none) and after (0: the key was deleted), and its modules added, removed or changed against the assignments before.
type Change struct {
@@ -95,8 +95,8 @@ func TestManifest(t *testing.T) {
// which lists and asks again what each node's engine reads.
if !slices.Equal(m.Tools, []string{"assigner_put", "assigner_release", "assigner_puts", "assignments", "seat-holders", "wants",
"assigner_read_assignments", "assigner_read_seat_holders", "assigner_read_wants", "assigner_read_findings", "assigner_read_asks", "assigner_read_differences", "assigner_read_copies",
"assigner_first_views", "assigner_first_view_again", "assigner_engine_reads"}) {
t.Errorf("the assigner serves tools %v, want its three put tools, three first views, six read tools and assigner_engine_reads", m.Tools)
"assigner_first_views", "assigner_first_view_again", "assigner_engine_reads", "assigner_seat_list", "assigner_seat_list_again"}) {
t.Errorf("the assigner serves tools %v, want its three put tools, three first views, six read tools, assigner_engine_reads and the seat list's two (ADR 0636 §7)", m.Tools)
}
if m.RunsAs != m.Module || m.SecretsOwner != m.RunsAs {
t.Errorf("runs-as = %q: the bundle runs on the assigner's own account (ADR 0621 §1)", m.RunsAs)
@@ -120,11 +120,12 @@ func TestManifest(t *testing.T) {
t.Errorf("emits lists %s, an event the seat says", e)
}
}
// ADR 0618 §4: and it hears each node's engine say its reads changed.
if !slices.Equal(m.Consumes, []string{"mesh-delivery-indexer.views.graph", "mesh-delivery-indexer.views.manifests", "mesh-delivery-indexer.views.modules", "node-engine.manifest-reads-changed"}) {
// ADR 0618 §4: and it hears each node's engine say its reads changed; ADR 0636 §4 (issue 1966): and the seat list's two events.
if !slices.Equal(m.Consumes, []string{"mesh-catalog.seat-definition-changed", "mesh-catalog.seat-definition-removed", "mesh-delivery-indexer.views.graph", "mesh-delivery-indexer.views.manifests", "mesh-delivery-indexer.views.modules", "node-engine.manifest-reads-changed"}) {
t.Errorf("consumes = %v, want the indexer's views of graph, manifests and modules and the engines' reads changed", m.Consumes)
}
if want := []string{"mesh-delivery-indexer.graph", "mesh-delivery-indexer.manifests", "mesh-delivery-indexer.modules",
"seat:mesh-catalog.seat-definitions", // ADR 0636 §4: the seat list
"seat:mesh-hand-acts.hand-act", "seat:mesh-hand-acts.hand-acts", "seat:mesh-hand-acts.warranted", // ADR 0565 §9, §11: the seat alone
"seat:node-backup.backed-up", "seat:node-backup.now",
"seat:node-engine.manifest-reads"}; !slices.Equal(slices.Sorted(slices.Values(m.Invokes)), want) { // ADR 0618 §3
+6 -10
View File
@@ -39,6 +39,8 @@
"operator-channel"
],
"consumes": [
"mesh-catalog.seat-definition-changed",
"mesh-catalog.seat-definition-removed",
"mesh-delivery-indexer.views.graph",
"mesh-delivery-indexer.views.manifests",
"mesh-delivery-indexer.views.modules",
@@ -48,6 +50,7 @@
"mesh-delivery-indexer.graph",
"mesh-delivery-indexer.manifests",
"mesh-delivery-indexer.modules",
"seat:mesh-catalog.seat-definitions",
"seat:mesh-hand-acts.hand-act",
"seat:mesh-hand-acts.hand-acts",
"seat:mesh-hand-acts.warranted",
@@ -82,7 +85,9 @@
"assigner_read_copies",
"assigner_first_views",
"assigner_first_view_again",
"assigner_engine_reads"
"assigner_engine_reads",
"assigner_seat_list",
"assigner_seat_list_again"
],
"state": [
{ "name": "assignments", "history": 64 },
@@ -156,15 +161,6 @@
"MESH_VIEW_CALLERS_SEAT_HOLDERS": "node-tools",
"MESH_VIEW_CALLERS_WANTS": "mesh-grants"
}
},
{
"name": "asks",
"kind": "package",
"language": "go",
"paths": [
"asks"
],
"series": "0.1"
}
]
}
@@ -149,8 +149,13 @@ type stepSeenKey struct{}
// WithStepSeen is ctx carrying seen, which Asker.Wait hands every value the ask's step key takes while it waits: the node
// that took it, each phase it says while running (buildstep.Step.Phase), and its end. A try names its asks' progress so
// (issue 1890). seen runs in the waiting goroutine and must not block.
// (issue 1890). seen runs in the waiting goroutine and must not block. A seen ctx already carries is kept: both are handed
// every value, the earlier first (a check's begin bound watches the steps of a try that names them).
func WithStepSeen(ctx context.Context, seen func(buildstep.Step)) context.Context {
if before := stepSeenOf(ctx); before != nil {
later := seen
seen = func(s buildstep.Step) { before(s); later(s) }
}
return context.WithValue(ctx, stepSeenKey{}, seen)
}
@@ -26,10 +26,13 @@ func TestAnAskerPutsOnceAndReadsTheStep(t *testing.T) {
return buildstep.Step{ID: a.ID, Node: "ace", State: buildstep.StepPassed, Result: []byte(`{"artifacts":[]}`)}
}}
k := Asker{Publisher: f, Waiter: f}
var seen []buildstep.Step // issue 1890: a waiter that asks to see each value of the step sees it
step, err := k.Ask(WithStepSeen(context.Background(), func(s buildstep.Step) { seen = append(seen, s) }), goAsk("mvl0frv18d1957"))
if err != nil || step.Node != "ace" || len(f.Asks()) != 1 || len(seen) != 1 || seen[0].Node != "ace" {
t.Fatalf("step %+v, %v, asks %d, seen %+v", step, err, len(f.Asks()), seen)
var seen, also []buildstep.Step // issue 1890: a waiter that asks to see each value of the step sees it
// Issue 1921: a second seen on the same context (a check's begin bound inside a try) is added, never in place of the first.
both := WithStepSeen(WithStepSeen(context.Background(), func(s buildstep.Step) { seen = append(seen, s) }),
func(s buildstep.Step) { also = append(also, s) })
step, err := k.Ask(both, goAsk("mvl0frv18d1957"))
if err != nil || step.Node != "ace" || len(f.Asks()) != 1 || len(seen) != 1 || seen[0].Node != "ace" || len(also) != 1 {
t.Fatalf("step %+v, %v, asks %d, seen %+v and %+v", step, err, len(f.Asks()), seen, also)
}
f.Answer = func(a buildstep.StepAsk) buildstep.Step {
return buildstep.Step{ID: a.ID, Node: "shanks", State: buildstep.StepRefused, Reason: "go 1.27 installed"}
@@ -200,12 +200,10 @@ func TestManifestProcess(t *testing.T) { // 1607/AC1
if _, bus := p["bus"]; bus {
t.Errorf("the process declares a bus: it runs on its own broker credential")
}
if len(m.Build.Artifacts) != 3 {
t.Fatalf("artifacts = %+v, want the program, the tools and the package builds", m.Build.Artifacts)
}
// ADR 0634 §1, ADR 0606 §4: the module's own package is published, so a module of another repository imports it by version.
if a := m.Build.Artifacts[2]; a.Name != "builds" || a.Kind != "package" || a.Language != "go" || !reflect.DeepEqual(a.Paths, []string{"builds"}) || a.Series != "0.1" {
t.Errorf("package artifact = %+v", a)
// Issue 1966's bootstrap: the package artifact builds is out of the manifest until the new builder runs; the follow-up
// puts it back with its assertion (ADR 0634 §1, ADR 0606 §4: three artifacts, the third the package builds).
if len(m.Build.Artifacts) != 2 {
t.Fatalf("artifacts = %+v, want the program and the tools", m.Build.Artifacts)
}
if a := m.Build.Artifacts[1]; a.Name != "tools" || a.Kind != "bundle" || a.Binary != "mesh-delivery-builder-tools" || !reflect.DeepEqual(a.Loads, []string{"mesh-delivery-builder-tools"}) {
t.Errorf("tools artifact = %+v", a) // 1945
@@ -168,15 +168,6 @@
"loads": [
"mesh-delivery-builder-tools"
]
},
{
"name": "builds",
"kind": "package",
"language": "go",
"paths": [
"builds"
],
"series": "0.1"
}
]
}
@@ -0,0 +1,52 @@
// Command mesh-checking-program is the checker's checking program (novox/hq ADR 0644, ADR 0656; design 47's bootstrap
// delivery): built from a check's heads and run in the build seat's sandbox through the seat node-build-go's ask run,
// with no network and no bus account, it reads the check's facts from the file MESH_RUN_INPUT names and answers the
// verdict the judging rules give them, as one JSON line on its standard output (rules.Answer).
//
// mesh-checking-program -verdict
// mesh-checking-program -seats
//
// Asked -seats, it answers the seat list of the heads laid in the directory MESH_RUN_HEADS names (rules.SeatsAnswer).
// It refuses facts of a version it does not read, naming it, with exit 2.
package main
import (
"fmt"
"io"
"io/fs"
"os"
"slices"
"mesh-delivery-checker/rules"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
)
func main() {
os.Exit(run(os.Args[1:], os.Getenv, os.Stdout, os.Stderr))
}
// run is the command over its arguments, environment and outputs, answering its exit.
func run(args []string, getenv func(string) string, stdout, stderr io.Writer) int {
var heads fs.FS
if dir := getenv(buildstep.RunHeadsEnv); dir != "" {
heads = os.DirFS(dir)
}
input := func() ([]byte, error) {
file := getenv(buildstep.RunInputEnv)
if file == "" {
return nil, fmt.Errorf("%s names no file", buildstep.RunInputEnv)
}
return os.ReadFile(file)
}
line, err := rules.Program(args, input, heads)
if err != nil {
if heads == nil && slices.Contains(args, "-seats") {
err = fmt.Errorf("no heads: %s names no directory", buildstep.RunHeadsEnv)
}
fmt.Fprintf(stderr, "mesh-checking-program: %v\n", err)
return 2
}
fmt.Fprintf(stdout, "%s\n", line)
return 0
}
@@ -0,0 +1,342 @@
package main
import (
"bytes"
"context"
"encoding/json"
"errors"
"io/fs"
"os"
"path"
"path/filepath"
"reflect"
"strings"
"testing"
"mesh-delivery-checker/rules"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliverycheckerclient"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
"git.novox.be/novox/mesh-delivery/sdk/seatdefs"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// 1921 (B0, D2): the checking program answers, on its last line, the verdict the rules give the facts in its input, and
// refuses facts of a version it does not read, an input it is not given, and a question it is not asked, naming each.
// 1921 (B0, D3): asked -seats, it answers the seat list of the heads laid for it: a definition a head adds, resolved with
// the parts it includes; what it left out, and why; and an empty list for heads with no definition.
func TestTheCheckingProgramAnswersTheVerdictOfItsFacts(t *testing.T) {
failing, _ := json.Marshal(rules.Tested{Level: buildstep.LevelUnit, Tests: 3, Failed: []rules.Failure{{Package: "x/a", Test: "TestA", Cause: "a_test.go:9: boom"}}})
facts := rules.Facts{V: rules.FactsVersion, Group: "feat/x", Heads: "0123456789ab", Members: []rules.Member{{Repository: "novox/x", Number: 3, Head: "h"}},
Steps: []rules.Step{
{Repository: "novox/x", Number: 3, Module: "mesh-build-go", Node: "laptop", Level: &buildstep.LevelAnswer{Level: buildstep.LevelUnit, Tests: []json.RawMessage{failing}}},
{Repository: "novox/x", Number: 3, Failed: &rules.FailedFirst{Machine: "laptop", Words: "novox/x#3 has files in conflict: go.mod"}},
}}
dir := t.TempDir()
write := func(name string, v any) string {
data, _ := json.Marshal(v)
p := filepath.Join(dir, name)
if err := os.WriteFile(p, data, 0o600); err != nil {
t.Fatal(err)
}
return p
}
v1 := write("v1.json", facts)
later := facts
later.V = rules.FactsVersion + 1
v2 := write("v2.json", later)
// The heads as the build node lays them, each at its repository's path: a contracts module of its own defining a seat
// with a part, and the mesh's own contracts module defining a seat of a domain that is not its (ADR 0636 §5).
lay := func(name string, files map[string]string) string {
root := filepath.Join(dir, name)
for p, body := range files {
if err := os.MkdirAll(filepath.Dir(filepath.Join(root, p)), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, p), []byte(body), 0o600); err != nil {
t.Fatal(err)
}
}
return root
}
contracts := func(module string) string {
return `{"module": "` + module + `", "build": {"artifacts": [{"name": "go", "kind": "package", "language": "go", "generated-from": "."}]}}`
}
heads := lay("heads", map[string]string{
"novox/x/x-contracts/module.json": contracts("x-contracts"),
"novox/x/x-contracts/seats/node-build-x/seat.json": `{"format":1,"seat":"node-build-x","scope":"node","grain":"whole","domain":"Delivery","decision":"ADR 1","description":"kept","includes":["build"],"verbs":[{"name":"own","input":{"type":"object","properties":{"credentials":{"$ref":"#/defs/Sealed"}}},"answer":{"type":"object"}}],"defs":{"Sealed":{"type":"object"}}}`,
"novox/x/x-contracts/parts/build.json": `{"format":1,"part":"build","verbs":[{"name":"current","input":{"type":"object"},"answer":{"type":"object"}}],"events":[{"name":"ended","body":{"type":"object"}}]}`,
"novox/x/contracts/module.json": contracts("mesh-contracts"),
"novox/x/contracts/seats/mesh-catalog/seat.json": `{"format":1,"seat":"mesh-catalog","scope":"mesh","grain":"whole","domain":"Delivery","decision":"ADR 1"}`,
})
broken := lay("broken", map[string]string{
"novox/x/x-contracts/module.json": contracts("x-contracts"),
"novox/x/x-contracts/seats/broken/seat.json": `{"format":1,"seat":"broken","scope":"nowhere"}`,
})
none := filepath.Join(dir, "no-heads")
if err := os.MkdirAll(filepath.Join(none, "novox/x"), 0o755); err != nil {
t.Fatal(err)
}
for name, c := range map[string]struct {
args []string
input, heads string
exit int
said string
}{
"the seats": {args: []string{"-seats"}, input: v1, heads: heads},
"no seats": {args: []string{"-seats"}, input: v1, heads: none},
"a seat unparsed": {args: []string{"-seats"}, input: v1, heads: broken, exit: 2, said: "the seat definition of broken (module x-contracts) at the head of novox/x is not taken"},
"no heads laid": {args: []string{"-seats"}, exit: 2, said: buildstep.RunHeadsEnv + " names no directory"},
"the facts": {args: []string{"-verdict"}, input: v1},
"a later version": {args: []string{"-verdict"}, input: v2, exit: 2, said: "facts of version 2, this checking program reads version 1"},
"no input": {args: []string{"-verdict"}, exit: 2, said: buildstep.RunInputEnv + " names no file"},
"nothing asked": {input: v1, exit: 2, said: "nothing asked"},
"an unknown asking": {args: []string{"-judge"}, input: v1, exit: 2, said: "-judge"},
} {
var out, errs bytes.Buffer
exit := run(c.args, func(k string) string {
switch k {
case buildstep.RunInputEnv:
return c.input
case buildstep.RunHeadsEnv:
return c.heads
}
return ""
}, &out, &errs)
if exit != c.exit || !strings.Contains(errs.String(), c.said) {
t.Errorf("%s: exit %d, said %q; want %d and %q", name, exit, errs.String(), c.exit, c.said)
continue
}
if c.exit != 0 {
continue
}
lines := strings.Split(strings.TrimSpace(out.String()), "\n")
if c.heads != "" {
var sa rules.SeatsAnswer
if err := json.Unmarshal([]byte(lines[len(lines)-1]), &sa); err != nil || sa.V != rules.SeatsVersion || sa.Seats == nil {
t.Fatalf("%s: the last line %q: %v", name, lines[len(lines)-1], err)
}
if c.heads == none {
if len(sa.Seats) != 0 || len(sa.Left) != 0 {
t.Errorf("%s: %+v, want an empty list", name, sa)
}
continue
}
var d struct {
Seat string `json:"seat"`
Verbs []struct {
Name string `json:"name"`
} `json:"verbs"`
}
if len(sa.Seats) == 1 {
_ = json.Unmarshal(sa.Seats[0], &d)
}
if d.Seat != "node-build-x" || len(d.Verbs) != 2 || len(sa.Left) != 1 || !strings.Contains(sa.Left[0].Why, "whose seats mesh-delivery-contracts defines") {
t.Errorf("%s: seats %s, left %+v; want node-build-x with its part's verb, and mesh-catalog left out", name, sa.Seats, sa.Left)
}
// The group check of 616c42434ac2: a checker from before G (no digests in these facts) writes the answer into the
// build seat's step state, whose writer refuses a value with a field named like a credential, here a schema's
// property name; the answer carries no verb's or message's schema and no type only they referred to.
if len(sa.Seats) == 1 && (bytes.Contains(sa.Seats[0], []byte("credentials")) || bytes.Contains(sa.Seats[0], []byte("Sealed"))) {
t.Errorf("%s: %s carries a verb's schema to a checker that hands no digests", name, sa.Seats[0])
}
continue
}
var a rules.Answer
if err := json.Unmarshal([]byte(lines[len(lines)-1]), &a); err != nil || a.V != rules.AnswerVersion {
t.Fatalf("%s: the last line %q: %v", name, lines[len(lines)-1], err)
}
if want := rules.Judge(facts); !reflect.DeepEqual(a.Verdict, want) || a.Verdict.State != meshdeliverycheckerclient.VerdictFail || len(a.Verdict.Failures) != 2 ||
a.Verdict.Failures[1].Words != "novox/x#3 has files in conflict: go.mod" || a.Verdict.Machines[0] != "laptop" || a.Verdict.Levels[buildstep.LevelUnit].Run != 3 {
t.Errorf("%s: the verdict %+v, want %+v", name, a.Verdict, want)
}
}
// Review finding 3 of milestone 47/G: the heads of a real group carry every seat definition the reader holds, unchanged
// (this repository's own contracts, laid as its head): the answer leaves them all to the checker's list and stays
// within the bound a run's answer is written under (buildstep.RunInputBound), where the whole list would not.
t.Run("heads carrying every definition the reader holds unchanged answer none of them, within the bound", func(t *testing.T) {
real := under{prefix: "novox/mesh-delivery", fs: os.DirFS("../../../..")}
group := rules.Facts{V: rules.FactsVersion, Members: []rules.Member{{Repository: "novox/mesh-delivery", Head: "h"}}}
// An empty list of digests answers every definition whole (a checker that hands its list's digests, holding none).
whole, err := rules.SeatsOf(real, rules.Facts{V: rules.FactsVersion, Members: group.Members, SeatDigests: map[string]string{}})
if err != nil || len(whole.Seats) < 10 {
t.Fatalf("the repository's own definitions: %d, %v", len(whole.Seats), err)
}
// The reader's digests are of each definition whole, as the seat mesh-catalog answers it; the answer is its judging view.
group.SeatDigests = wholeDigests(t, real, "novox/mesh-delivery")
same, err := rules.SeatsOf(real, group)
answer, _ := json.Marshal(same)
if err != nil || len(same.Seats) != 0 || same.Same != len(whole.Seats) || len(answer) > buildstep.RunInputBound {
t.Errorf("unchanged heads answer %d definitions (%d the same), %d bytes; want none, all %d the same, within %d: %v",
len(same.Seats), same.Same, len(answer), len(whole.Seats), buildstep.RunInputBound, err)
}
// Review round 2 of G: a checker from before G (B0's, which makes G's own check) hands no digests, and the answer
// then keeps each seat's own description only, so a group's whole list stays within the bound; each still parses.
b0, err := rules.SeatsOf(real, rules.Facts{V: rules.FactsVersion, Members: group.Members})
if err != nil || len(b0.Seats) != len(whole.Seats) {
t.Fatalf("B0-shaped facts: %d definitions, %v", len(b0.Seats), err)
}
for i, raw := range b0.Seats {
var own struct {
Description string `json:"description"`
}
_ = json.Unmarshal(whole.Seats[i], &own)
var d struct {
Description string `json:"description"`
Verbs []json.RawMessage `json:"verbs"`
Events []json.RawMessage `json:"events"`
}
_ = json.Unmarshal(raw, &d)
inner := 0
for _, m := range append(d.Verbs, d.Events...) {
inner += strings.Count(string(m), `"description":"`)
}
parsed, perr := seatdef.Parse(raw)
for _, v := range parsed.Verbs {
if len(v.Input) != 0 || len(v.Answer) != 0 {
t.Errorf("B0-shaped facts answer %s's verb %s with its schemas; want none", parsed.Seat, v.Name)
}
}
for name := range parsed.Defs {
if !bytes.Contains(raw, []byte(`"#/defs/`+name+`"`)) {
t.Errorf("B0-shaped facts answer %s with the type %s, which nothing it keeps refers to", parsed.Seat, name)
}
}
if inner > 0 || d.Description != own.Description || perr != nil {
t.Errorf("B0-shaped facts answer %s with %d inner descriptions, its own %q: %v; want only its own", raw[:min(len(raw), 80)], inner, d.Description, perr)
}
}
if answer, _ := json.Marshal(b0); len(answer) > buildstep.RunInputBound || len(answer) >= len(mustJSON(t, whole))*3/4 {
t.Errorf("B0-shaped facts answer %d bytes (whole %d), want within %d and far below the whole", len(answer), len(mustJSON(t, whole)), buildstep.RunInputBound)
}
// One definition the heads change is answered alone.
for seat := range group.SeatDigests {
group.SeatDigests[seat] = "changed"
break
}
if one, err := rules.SeatsOf(real, group); err != nil || len(one.Seats) != 1 {
t.Errorf("one changed definition: answered %d, %v", len(one.Seats), err)
}
})
// The lead's ruling after G's group check of 616c42434ac2: with digests given too, a changed definition whose verb input
// has a property named credentials is answered as the validator and the composer read it, with no such field.
changed, err := rules.SeatsOf(os.DirFS(heads), rules.Facts{V: rules.FactsVersion, Members: facts.Members,
SeatDigests: map[string]string{"node-build-x": "changed"}})
if err != nil || len(changed.Seats) != 1 || bytes.Contains(changed.Seats[0], []byte("credentials")) || bytes.Contains(changed.Seats[0], []byte("Sealed")) {
t.Errorf("a changed definition with digests given: %d answered, %s, %v; want it alone, with no verb's schema", len(changed.Seats), changed.Seats, err)
}
// Review round 1 of milestone 47/B0: the module graph's cycle rule is the program's (ADR 0603 §2, ADR 0656 §5), over the
// edges the running checker gathers. A cycle the change makes fails; one the trunk has among the same modules is noted.
web, mail := rules.Edge{From: "web", To: "mail", Kind: rules.EdgeAwaits}, rules.Edge{From: "mail", To: "web", Kind: rules.EdgeAwaits}
only := func(e rules.Edge, where string) rules.Edge { e.Only = where; return e }
for _, row := range []struct {
name string
graph rules.Graph
fails string
note string
}{
{name: "no cycle", graph: rules.Graph{Trunk: true, Edges: []rules.Edge{web}}, note: "has no cycle"},
{name: "a cycle the change makes", graph: rules.Graph{Trunk: true, Edges: []rules.Edge{mail, only(web, rules.EdgeChange)}}, fails: "has the cycle mail -> web -> mail"},
{name: "a cycle the trunk has", graph: rules.Graph{Trunk: true, Edges: []rules.Edge{mail, web}}, note: "which the trunk already has"},
{name: "the trunk unread", graph: rules.Graph{Edges: []rules.Edge{mail, web}}, fails: "has the cycle mail -> web -> mail"},
{name: "a base on itself", graph: rules.Graph{Trunk: true, Edges: []rules.Edge{only(rules.Edge{From: "web", To: "web", Kind: rules.EdgeBase}, rules.EdgeChange)}}, fails: "has the cycle web -> web"},
{name: "an edge only the trunk has", graph: rules.Graph{Trunk: true, Edges: []rules.Edge{mail, only(web, rules.EdgeTrunk)}}, note: "has no cycle"},
} {
g := row.graph
a := rules.Judged(rules.Facts{V: rules.FactsVersion, Steps: []rules.Step{{Graph: &g}}})
if row.fails != "" && (a.Verdict.State != meshdeliverycheckerclient.VerdictFail || !strings.Contains(a.Verdict.Failed.Words, row.fails)) {
t.Errorf("%s: verdict %+v, want a fail saying %q", row.name, a.Verdict, row.fails)
}
if row.note != "" && (a.Verdict.State != meshdeliverycheckerclient.VerdictPass || !strings.Contains(strings.Join(a.Notes, "\n"), row.note)) {
t.Errorf("%s: verdict %s, notes %q, want a pass noting %q", row.name, a.Verdict.State, a.Notes, row.note)
}
}
}
// under is a directory laid at a repository's path of the heads, as the build node lays a head.
type under struct {
prefix string
fs fs.FS
}
func (u under) Open(name string) (fs.File, error) {
if name == u.prefix {
return u.fs.Open(".")
}
if rest, ok := strings.CutPrefix(name, u.prefix+"/"); ok {
return u.fs.Open(rest)
}
return nil, &fs.PathError{Op: "open", Path: name, Err: fs.ErrNotExist}
}
func mustJSON(t *testing.T, v any) []byte {
t.Helper()
out, err := json.Marshal(v)
if err != nil {
t.Fatal(err)
}
return out
}
// wholeDigests are the digests of a head's seat definitions whole (rules.SeatDigest), read by the domain SDK as the
// indexer reads them: what a checker's reader holds for heads that change none.
func wholeDigests(t *testing.T, root fs.FS, repository string) map[string]string {
t.Helper()
head, err := fs.Sub(root, repository)
if err != nil {
t.Fatal(err)
}
manifests := map[string]string{}
_ = fs.WalkDir(head, ".", func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() && (d.Name() == ".git" || d.Name() == "testdata" || d.Name() == "node_modules") {
return fs.SkipDir
}
if !d.IsDir() && d.Name() == "module.json" {
raw, _ := fs.ReadFile(head, p)
manifests[path.Dir(p)] = string(raw)
}
return nil
})
forge := seatdefs.Forge{
Files: func(_ context.Context, _, _ string, dirs []string) (map[string]map[string]string, error) {
out := map[string]map[string]string{}
for _, dir := range dirs {
files := map[string]string{}
if err := fs.WalkDir(head, dir, func(p string, d fs.DirEntry, err error) error {
if err == nil && !d.IsDir() {
files[strings.TrimPrefix(p, dir+"/")] = ""
}
return err
}); err == nil {
out[dir] = files
}
}
return out, nil
},
File: func(_ context.Context, _, _, file string) ([]byte, error) {
raw, err := fs.ReadFile(head, file)
if errors.Is(err, fs.ErrNotExist) {
return nil, nil
}
return raw, err
},
}
out := map[string]string{}
if len(manifests) == 0 {
t.Fatalf("no manifest at the head of %s", repository)
}
for _, c := range seatdefs.ContractsModules(manifests) {
defs, _, err := forge.Read(context.Background(), repository, "h", c)
if err != nil {
t.Fatal(err)
}
for seat, d := range defs {
raw, _ := json.Marshal(d)
out[seat] = rules.SeatDigest(raw)
}
}
return out
}
@@ -1,6 +1,6 @@
// Command mesh-delivery-checker-tools serves the checker's read tools (novox/hq ADR 0484 clause 13, issue 1686): the runtime
// launches it for a call, it reads the checker's verdicts and kept states and the indexer's candidates through the checker's
// state and reads. It writes only the tries (the try's tools) and, asking a check again by hand, one verdict. The checker is the other program: it serves no tool (ADR 0437 §4).
// launches it for a call, it reads the checker's verdicts and kept states through the checker's state and reads, and the
// indexer's candidates and modules through the indexer's read tools. It writes only the tries (the try's tools) and, asking a check again by hand, one verdict. The checker is the other program: it serves no tool (ADR 0437 §4).
package main
import (
@@ -15,8 +15,10 @@ import (
"git.novox.be/novox/mesh-contracts/health-and-repair/go/clients/meshhandactsclient"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliveryclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"git.novox.be/novox/mesh-sdk/go/views"
)
@@ -57,6 +59,52 @@ func read(s stdio.KeptState) check.StateRead {
}}
}
// ownerRead is a state of another module read through that module's ordinary read tool (views.ReadTool), asked through ask
// (stdio.Ask, or a test's): its keys with no argument, one key's value with key. Another module's data is asked through its
// tools, never read from its bus state (novox/hq ADR 0596); the manifest invokes the tool, and the owner answers every reader
// that invokes it. Each read is asked when it is read: the owner's state is the truth, and the tools keep no copy.
func ownerRead(ask func(string, any) (json.RawMessage, error), tool string) check.StateRead {
call := func(args map[string]any, into any) error {
reads.Add(1)
raw, err := ask(tool, args)
if err != nil {
return fmt.Errorf("%s does not answer: %w", tool, err)
}
answer, said, ok := buscalls.Unwrap(raw)
if !ok {
return fmt.Errorf("%s answered: %s", tool, said)
}
if err := json.Unmarshal(answer, into); err != nil {
return fmt.Errorf("%s answered what does not decode: %w", tool, err)
}
return nil
}
return check.StateRead{
Keys: func() ([]string, error) {
var a views.ReadAnswer
if err := call(map[string]any{}, &a); err != nil {
return nil, err
}
return a.Keys, nil
},
Get: func(key string) (json.RawMessage, error) {
var a views.KeyAnswer
if err := call(map[string]any{"key": key}, &a); err != nil {
return nil, err
}
if !a.Found {
return nil, nil
}
return a.Value, nil
}}
}
// The indexer's read tools the tools ask for its candidates and modules (the manifest's invokes).
const (
indexerCandidatesTool = check.IndexerModule + ".indexer_read_candidates"
indexerModulesTool = check.IndexerModule + ".indexer_read_modules"
)
// runsOf says which commit the mesh runs of a module from the indexer's modules state, as the checker's own record of it does
// (the commit the module's record was computed at); "" when the record is not readable.
type runsOf struct{ modules check.StateRead }
@@ -232,11 +280,25 @@ func SeatTryTools(tr Trying) []stdio.Tool {
return out
}
// SeatInputTools are the seat's verb input (check.InputVerb): a checking program's input as the checker keeps it, asked by
// the Go build module that runs the program (buildstep.RunInput.Read), never read from the checker's state by it.
func SeatInputTools(r check.Reader) []stdio.Tool {
return []stdio.Tool{{Name: Seat + "." + meshdeliverychecker.VerbInput,
Description: "A checking program's input the checker keeps, by the sha256 of its exact bytes, as a run ask names it: the digest " +
"and the bytes, base64. A digest it keeps no input of, or bytes that read back as another digest, is an error. Read-only.",
Input: map[string]any{"type": "object", "required": []string{"digest"}, "properties": map[string]any{
"digest": map[string]any{"type": "string", "description": "the input's sha256, 64 lowercase hex digits"}}},
Run: func(args map[string]any) (any, error) {
digest, _ := args["digest"].(string)
return r.Input(digest)
}}}
}
// Tools are the read tools over the checker's states.
func Tools(r check.Reader) []stdio.Tool {
return []stdio.Tool{
{Name: check.VerdictsReadTool,
Description: "The verdicts the checker holds, each as stored under verdicts.<group>.<heads>: its state, every failure it names one line each (every member's failed tests, refused steps and conflicts, the manifests' refusals and the composition), the tests run, kept and skipped per level, the time of each part, the trunk heads it judged against and the composition, with the full " +
Description: "The verdicts the checker holds, each as stored under verdicts.<group>.<heads>: its state, every failure it names one line each (every member's failed tests, refused steps and conflicts, the manifests' refusals and the composition), the tests run, kept and skipped per level, the time of each part, the trunk heads it judged against, the composition and which checking program judged it (judged: its build and the heads it read), with the full " +
"report of the status each pull request carries (the line the forge shows is cut at 140 characters, this is not). With a group, that group's only. Read-only.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"group": map[string]any{"type": "string", "description": "the delivery group's branch name"}}},
@@ -246,7 +308,7 @@ func Tools(r check.Reader) []stdio.Tool {
return cutAnswer(map[string]any{"verdicts": vs}, cut), err
}},
{Name: check.PullReadTool,
Description: "The verdict that judges one pull request, every failure it names one line each (all of them, not only the first the status line shows), and the full report of its status mesh/check. Read-only.",
Description: "The verdict that judges one pull request, every failure it names one line each (all of them, not only the first the status line shows), which checking program judged it (judged: its build, its key and the heads it read), and the full report of its status mesh/check. Read-only.",
Input: map[string]any{"type": "object", "required": []string{"repository", "number"}, "properties": map[string]any{
"repository": map[string]any{"type": "string", "description": "<owner>/<name>"},
"number": map[string]any{"type": "integer", "description": "the pull request's number"}}},
@@ -267,6 +329,17 @@ func Tools(r check.Reader) []stdio.Tool {
kept, cut, err := r.Kept(level, digest)
return cutAnswer(map[string]any{"kept": kept}, cut), err
}},
{Name: check.InputsReadTool,
Description: "The checking program's inputs the checker keeps, each under the sha256 of its exact bytes (inputs.<digest>, kept a day): " +
"the facts of a check its verdict is asked of, or the heads its seat list is asked of. With a digest, that input as kept; " +
"without, every digest and its size. Read-only.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"digest": map[string]any{"type": "string", "description": "the input's sha256, as a run ask names it"}}},
Run: func(args map[string]any) (any, error) {
digest, _ := args["digest"].(string)
inputs, cut, err := r.Inputs(digest)
return cutAnswer(map[string]any{"inputs": inputs}, cut), err
}},
{Name: check.PreviewsReadTool,
Description: "The merge previews the checker holds, each as stored under previews.<repository>.<head>.<trunk>: the merge commit, per module directory " +
"its tree, manifest digest and lock hashes, or the files in conflict. With a repository, that repository's; with a head (7 or more characters) " +
@@ -299,6 +372,24 @@ func Tools(r check.Reader) []stdio.Tool {
}
}
// RequestTools is the tool that reads the composition requests the checker sent to nodes (issue 1921).
func RequestTools(r check.StateRead) []stdio.Tool {
return []stdio.Tool{{Name: check.RequestsReadTool,
Description: "The composition request a check sent to a node: the composer's whole input for that node (the node-engine's composition input, the case's " +
"manifests and verdicts, the seat list), kept exactly as sent so a session reproduces the check's composition (the composer run by hand over the same " +
"request gives the check's answer). Kept in the checker's state compositions, per check key (group.heads, the 12-digit heads hash) and node, " +
"about 430 KB each (kept compressed, so a value fits the state's 256 KiB), for 3 days (the state's ttl-seconds). With a check and a node, that request whole; with a check, " +
"the nodes it kept one for; with neither, every kept request, each with its size, digest and when it was kept. Read-only.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"check": map[string]any{"type": "string", "description": "the check's key, <group>.<heads hash>, as the verdict and composition name it"},
"node": map[string]any{"type": "string", "description": "the node the request was sent for, with a check"}}},
Run: func(args map[string]any) (any, error) {
key, _ := args["check"].(string)
node, _ := args["node"].(string)
return check.ReadRequests(r, key, node)
}}}
}
// RecoveryTools are the tools that steer a check by hand (tools-are-how-a-module-is-spoken-to): ask a check again whose
// wait judged nothing (the lost check of 2026-10-28). They write the checker's verdicts through w.
func RecoveryTools(r check.Reader, w check.VerdictEntry) []stdio.Tool {
@@ -327,13 +418,13 @@ func stepBound() time.Duration {
}
func main() {
r := check.Reader{VerdictState: read(stdio.State("verdicts")), KeptState: read(stdio.State("kept")), CandidateState: read(stdio.State("mesh-delivery-indexer.candidates")),
PreviewState: read(stdio.State("previews")),
StepStates: map[string]check.StateRead{"mesh-build-go": read(stdio.State("mesh-build-go.steps")), "mesh-build-node": read(stdio.State("mesh-build-node.steps"))},
HaveState: read(stdio.State("node-engine.have")), StepBound: stepBound(),
r := check.Reader{VerdictState: read(stdio.State("verdicts")), KeptState: read(stdio.State("kept")), CandidateState: ownerRead(stdio.Ask, indexerCandidatesTool),
PreviewState: read(stdio.State("previews")), InputState: read(stdio.State(check.InputsStateName)),
StepStates: map[string]check.StateRead{"mesh-build-go": read(stdio.State("mesh-build-go.steps")), "mesh-build-node": read(stdio.State("mesh-build-node.steps"))},
HaveState: read(stdio.State("node-engine.have")), StepBound: stepBound(),
TurnState: read(stdio.State("mesh-delivery-gate.turns")), HearingState: read(stdio.State(buildrun.HearingState)),
AskState: read(stdio.State(check.AsksState)), BuildState: read(stdio.State("mesh-delivery-builder.builds"))}
r.Runs = runsOf{read(stdio.State("mesh-delivery-indexer.modules"))} // the node-engine at the commit the mesh runs, as the checker reads it
r.Runs = runsOf{ownerRead(stdio.Ask, indexerModulesTool)} // the node-engine at the commit the mesh runs, as the checker reads it
tries := stdio.State(check.TriesState)
source := stdio.State("mesh-delivery-gate.source")
trunks := func() (map[string]string, error) {
@@ -376,6 +467,8 @@ func main() {
// checker_first_views and checker_first_view_again: where the checker's first views of the indexer's and the assigner's
// states stand, and asking one again by hand (ADR 0615, DLV-M2).
tools = append(tools, views.StatusTools(check.CheckerModule, "checker_first_views", "checker_first_view_again", stdio.State(views.StatusState), meshhandactsclient.Recorder{Ask: stdio.Ask}, time.Now)...)
tools = append(tools, SeatInputTools(r)...)
tools = append(tools, RequestTools(read(stdio.State(check.RequestsStateName)))...)
if err := stdio.Serve("", logged(tools)); err != nil {
fmt.Fprintf(os.Stderr, "[mesh-delivery-checker-tools] %v\n", err)
os.Exit(1)
@@ -1,15 +1,20 @@
package main
import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"os"
"regexp"
"slices"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-checker/checkermsg"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
"mesh-delivery-checker/internal/check"
)
@@ -22,7 +27,7 @@ func TestTheToolsServedAreTheFiveReadTools(t *testing.T) {
t.Errorf("%s has no description or no run", tl.Name)
}
}
want := []string{check.VerdictsReadTool, check.PullReadTool, check.KeptReadTool, check.PreviewsReadTool, check.StatusReadTool}
want := []string{check.VerdictsReadTool, check.PullReadTool, check.KeptReadTool, check.InputsReadTool, check.PreviewsReadTool, check.StatusReadTool}
if !slices.Equal(names, want) {
t.Errorf("tools = %v, want %v", names, want)
}
@@ -34,6 +39,102 @@ func TestTheToolsServedAreTheFiveReadTools(t *testing.T) {
if _, err := recovery[0].Run(map[string]any{"group": "feat/x", "why": "lost"}); err == nil {
t.Errorf("%s answered with no state to write", check.AskAgainTool)
}
// Issue 1921: the composition requests kept by the checker are read by check and node; the tool refuses a node with no check
// and a state it cannot read, and lists a check's nodes without their bodies.
store := &memStore{vals: map[string]json.RawMessage{}}
keeper := check.RequestKeeper{Store: memPut{store}}
for _, k := range [][2]string{{"g.0123456789ab", "anchor"}, {"g.0123456789ab", "ace"}, {"h.0123456789ab", "ace"}} {
if err := keeper.Keep(context.Background(), k[0], k[1], []byte(`{"input":{"nodes":[]}}`)); err != nil {
t.Fatal(err)
}
}
rt := RequestTools(check.StateRead{Keys: store.Keys, Get: store.Get})
if len(rt) != 1 || rt[0].Name != check.RequestsReadTool || !strings.Contains(rt[0].Description, "composition request a check sent to a node") {
t.Fatalf("request tools = %+v", rt)
}
one, err := rt[0].Run(map[string]any{"check": "g.0123456789ab", "node": "anchor"})
if b, _ := json.Marshal(one); err != nil || !strings.Contains(string(b), `"request":{"input":{"nodes":[]}}`) {
t.Errorf("one request = %s, %v", b, err)
}
list, _ := rt[0].Run(map[string]any{"check": "g.0123456789ab"})
if b, _ := json.Marshal(list); strings.Count(string(b), `"node"`) != 2 || strings.Contains(string(b), "h.0123") || strings.Contains(string(b), `"request"`) {
t.Errorf("list of a check = %s", b)
}
for _, args := range []map[string]any{{"node": "anchor"}, {"check": "g.0123456789ab", "node": "gone"}} {
if _, err := rt[0].Run(args); err == nil {
t.Errorf("%v answered", args)
}
}
if _, err := RequestTools(check.StateRead{})[0].Run(map[string]any{}); err == nil {
t.Errorf("answered with no state to read")
}
// Issue 1966: another module's state is asked of its read tool, never read from its bus state (ADR 0596). Every
// stdio.State this program names of another module is one its manifest declares under reads, which the runtime grants;
// the indexer's candidates and modules are asked of indexer_read_candidates and indexer_read_modules, which it invokes.
t.Run("1966: another module's state is read only where the manifest declares it", func(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
var man struct {
Reads []string `json:"reads"`
Invokes []string `json:"invokes"`
}
if err := json.Unmarshal(raw, &man); err != nil {
t.Fatal(err)
}
src, err := os.ReadFile("main.go")
if err != nil {
t.Fatal(err)
}
for _, m := range regexp.MustCompile(`stdio\.State\("([^"]+\.[^"]+)"\)`).FindAllStringSubmatch(string(src), -1) {
if !slices.Contains(man.Reads, m[1]) {
t.Errorf("main.go reads %s, which the manifest does not declare under reads: the runtime refuses it", m[1])
}
}
for _, tool := range []string{indexerCandidatesTool, indexerModulesTool} {
if !slices.Contains(man.Invokes, tool) {
t.Errorf("the manifest does not invoke %s", tool)
}
}
})
for _, c := range []struct {
name, key, answer, want string
failed bool
}{
{name: "the keys", answer: `{"owner":"mesh-delivery-indexer","state":"candidates","keys":["a","b"]}`, want: "[a b]"},
{name: "one key, wrapped as the tool runner hands it", key: "a",
answer: `{"content":[{"type":"text","text":"{\"key\":\"a\",\"found\":true,\"value\":{\"commit\":\"c\"}}"}]}`, want: `{"commit":"c"}`},
{name: "a key the indexer does not hold", key: "gone", answer: `{"key":"gone","found":false}`, want: ""},
{name: "the tool refused", key: "a", answer: `{"content":[{"type":"text","text":"refused"}],"isError":true}`, failed: true},
} {
t.Run("1966: the indexer's read tool, "+c.name, func(t *testing.T) {
var asked []string
r := ownerRead(func(key string, body any) (json.RawMessage, error) {
b, _ := json.Marshal(body)
asked = append(asked, key+" "+string(b))
return json.RawMessage(c.answer), nil
}, indexerCandidatesTool)
var got string
var err error
if c.key == "" {
var keys []string
keys, err = r.Keys()
got = strings.Trim(strings.Join([]string{"[", strings.Join(keys, " "), "]"}, ""), " ")
} else {
var raw json.RawMessage
raw, err = r.Get(c.key)
got = string(raw)
}
want := []string{indexerCandidatesTool + " {}"}
if c.key != "" {
want = []string{indexerCandidatesTool + ` {"key":"` + c.key + `"}`}
}
if (err != nil) != c.failed || (!c.failed && got != c.want) || !slices.Equal(asked, want) {
t.Errorf("read %q, %v, asking %v; want %q, failed %v, asking %v", got, err, asked, c.want, c.failed, want)
}
})
}
// With nothing to read from a tool says so and changes nothing.
for _, tl := range Tools(check.Reader{}) {
if tl.Name == check.StatusReadTool {
@@ -145,6 +246,21 @@ func TestTheTryToolsAskAndReadATry(t *testing.T) { // ADR 0501 §5, the agreed i
t.Errorf("the verb try requires %v and says %q: want members as an alternative to one repository and head", v.Input.Required, v.Description)
}
}
// The seat's verb input (issue 1921, ADR 0596 §2): a checking program's input as kept, which the Go build module asks.
body := []byte(`{"v":1}`)
in, _ := buildstep.RunInputOf(check.InputVerb, body)
inputs := SeatInputTools(check.Reader{InputState: check.StateRead{Get: func(key string) (json.RawMessage, error) {
if key == in.Digest {
return body, nil
}
return nil, nil
}}})
if len(inputs) != 1 || inputs[0].Name != Seat+".input" {
t.Fatalf("input verbs = %+v", inputs)
}
if a, err := inputs[0].Run(map[string]any{"digest": in.Digest}); err != nil || a.(meshdeliverychecker.InputAnswer).Input != base64.StdEncoding.EncodeToString(body) {
t.Errorf("verb input = %+v, %v", a, err)
}
// A group try (issue 1921): the members, each an exact commit; members and one repository together are refused; the
// description says what it does in the words a search for "try a group" finds.
later := TryTools(Trying{Tries: store, Ender: store, Now: func() time.Time { return time.Unix(2, 5) },
@@ -191,3 +307,11 @@ func TestTheTryToolsAskAndReadATry(t *testing.T) { // ADR 0501 §5, the agreed i
t.Errorf("hq: %v", err)
}
}
// memPut writes a memStore as the checker's kept-request state is written.
type memPut struct{ m *memStore }
func (p memPut) Put(_ context.Context, key string, value []byte) (uint64, error) {
p.m.vals[key] = value
return 1, nil
}
@@ -183,3 +183,15 @@ func (a awaitedKV) Delete(ctx context.Context, key string) error {
}
return s.Delete(ctx, key)
}
// runInputs keeps the checking program's inputs in the checker's state inputs (check.InputsBucket), each under the
// sha256 of its exact bytes: an input already kept under its digest is the same bytes, so a second keep is no error.
type runInputs struct{ a awaitedKV }
func (r runInputs) Keep(ctx context.Context, digest string, body []byte) error {
_, err := r.a.Create(ctx, digest, body)
if errors.Is(err, check.ErrConflict) {
return nil
}
return err
}
@@ -16,6 +16,7 @@ import (
"sync"
"sync/atomic"
"testing"
"testing/fstest"
"time"
"github.com/nats-io/nats-server/v2/server"
@@ -48,6 +49,8 @@ import (
"git.novox.be/novox/mesh-sdk/go/keys"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/views/viewstest"
"mesh-delivery-checker/rules"
)
// busConf is the shape of the composed accounts file (mesh-controller internal/broker): one account with JetStream, the
@@ -113,6 +116,118 @@ type fakeBuildModules struct {
names []string
// packages are, by repository, the package versions the last Go check ask of it carried (StepAsk.Packages).
packages map[string]buildstep.PackageVersions
// runs counts the run asks answered, and broken, when set, is why the checking program does not build (issue 1921).
runs atomic.Int32
broken atomic.Value
// drop, when set, drops every run ask unanswered and counts it in dropped, as a build module that does not know the job
// run (the lead's ruling on milestone 47/B0).
drop atomic.Bool
dropped atomic.Int32
}
// droppingAsks is the seat's asks as a build module that does not know the job run takes them: a run ask is acknowledged and
// never begun while the fake drops them.
type droppingAsks struct {
gobuild.AskSource
f *fakeBuildModules
}
func (d droppingAsks) Next(ctx context.Context, kind string, wait time.Duration) (*gobuild.Pulled, error) {
msg, err := d.AskSource.Next(ctx, kind, wait)
if err != nil || msg == nil || !d.f.drop.Load() {
return msg, err
}
var a buildstep.StepAsk
if json.Unmarshal(msg.Data, &a) == nil && a.Run != nil {
d.f.dropped.Add(1)
return nil, msg.Ack()
}
return msg, nil
}
// run answers a run ask as mesh-build-go does: the checking program's work (rules.Program) over the input the ask names,
// asked of the seat verb it names on the bus and checked against its digest (buildstep.RunInput.Read, as mesh-build-go
// reads it), with no heads of its own laid.
func (f *fakeBuildModules) run(ctx context.Context, a buildstep.StepAsk) (any, error) {
f.runs.Add(1)
if why, _ := f.broken.Load().(string); why != "" {
return nil, errors.New(why)
}
answer := buildstep.RunAnswer{Built: true, Key: "program-0123456789abcdef"}
// As mesh-build-go does, the input is asked before the program runs, and a step whose input cannot be had fails.
body, err := meshdeliverycheckerclient.ReadInput(func(key string, body any) (json.RawMessage, error) { return askVerb(f.js.Conn(), key, body) }, a.Run.Input.Digest)
if err == nil {
err = a.Run.Input.Verify(body)
}
if err != nil {
return nil, err
}
out, err := rules.Program(a.Run.Args, func() ([]byte, error) { return body, nil }, fstest.MapFS{})
if err != nil {
answer.Exit, answer.Said = 2, err.Error()
}
answer.Output = out
return answer, nil
}
// askVerb asks a seat verb on the subject that names no caller and reads the runtime's reply, {result, error}.
func askVerb(nc *nats.Conn, key string, body any) (json.RawMessage, error) {
seat, verb, ok := strings.Cut(strings.TrimPrefix(key, "seat:"), ".")
if !ok || !strings.HasPrefix(key, "seat:") {
return nil, fmt.Errorf("%q names no seat verb", key)
}
data, err := json.Marshal(body)
if err != nil {
return nil, err
}
msg, err := nc.Request("mesh.seat."+seat+".tool."+verb, data, 5*time.Second)
if err != nil {
return nil, err
}
var r struct {
Result json.RawMessage `json:"result"`
Error string `json:"error"`
}
if err := json.Unmarshal(msg.Data, &r); err != nil {
return nil, err
}
if r.Error != "" {
return nil, errors.New(r.Error)
}
return r.Result, nil
}
// serveInputVerb answers the checker seat's verb input on the bus as the checker's tools do (check.Reader.Input over the
// checker's state inputs), and counts what it answered (issue 1921, ADR 0596 §2).
func (b *bed) serveInputVerb(t *testing.T, nc *nats.Conn) {
t.Helper()
kv := b.buckets[check.InputsBucket]
r := check.Reader{InputState: check.StateRead{Get: func(key string) (json.RawMessage, error) {
e, err := kv.Get(b.ctx, key)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return nil, nil
}
if err != nil {
return nil, err
}
return e.Value(), nil
}}}
if _, err := nc.Subscribe("mesh.seat.mesh-delivery-checker.tool."+meshdeliverychecker.VerbInput, func(m *nats.Msg) {
var ask meshdeliverychecker.InputInput
reply := map[string]any{}
if err := json.Unmarshal(m.Data, &ask); err != nil {
reply["error"] = err.Error()
} else if a, err := r.Input(ask.Digest); err != nil {
reply["error"] = err.Error()
} else {
reply["result"] = a
b.inputsAnswered.Add(1)
}
data, _ := json.Marshal(reply)
_ = m.Respond(data)
}); err != nil {
t.Fatal(err)
}
}
func (f *fakeBuildModules) count(tool string) int {
@@ -213,7 +328,7 @@ func (f *fakeBuildModules) serve(t *testing.T, module string, kept jetstream.Key
raw, _ := value.(json.RawMessage)
return kv.Put(f.ctx, key, raw)
}
p := &gobuild.Puller{Source: &gobuild.JetStreamAsks{JS: f.js, Language: language}, Language: language, Node: "n1", Put: put,
p := &gobuild.Puller{Source: droppingAsks{&gobuild.JetStreamAsks{JS: f.js, Language: language}, f}, Language: language, Node: "n1", Put: put,
Capacity: func() int { return 2 }, PullWait: 200 * time.Millisecond,
Run: func(ctx context.Context, a buildstep.StepAsk) (any, error) {
if a.Check != nil {
@@ -227,6 +342,9 @@ func (f *fakeBuildModules) serve(t *testing.T, module string, kept jetstream.Key
}
f.mu.Unlock()
}
if a.Run != nil {
return f.run(ctx, a)
}
return gobuild.RunAsk(ctx, steps, a)
}}
go func() { _ = p.Pull(f.ctx) }()
@@ -290,16 +408,18 @@ func mustRead(t *testing.T, name string) []byte {
type bed struct {
stalePin atomic.Bool
// heads are the manifests the forge holds at a member's head, by "<repository> <path> <ref>".
heads sync.Map
ctx context.Context
js jetstream.JetStream
buckets map[string]jetstream.KeyValue
tools map[string]jetstream.KeyValue
fake *fakeBuildModules
srvURL string
announce []string
mu sync.Mutex
logs saidLog
heads sync.Map
ctx context.Context
js jetstream.JetStream
buckets map[string]jetstream.KeyValue
tools map[string]jetstream.KeyValue
// inputsAnswered counts the inputs the seat's verb input answered (issue 1921).
inputsAnswered atomic.Int32
fake *fakeBuildModules
srvURL string
announce []string
mu sync.Mutex
logs saidLog
}
// saidLog is what the program said, safe for its goroutines and the test's.
@@ -471,7 +591,7 @@ func startBed(t *testing.T) *bed {
check.CheckerOwedBucket: 5, check.PreviewsBucket: 5, check.CandidatesBucket: 10,
check.ModulesBucket: 20, check.ManifestsBucket: 1, check.SourceBucket: 64,
check.PresenceBucket: 20, check.HaveBucket: 20, check.TriesBucket: 1,
check.TurnsBucket: 5, builds.BuilderBuildsBucket: 1} {
check.TurnsBucket: 5, builds.BuilderBuildsBucket: 1, check.InputsBucket: 1} {
kv, err := js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: name, History: uint8(history)})
if err != nil {
t.Fatal(err)
@@ -499,6 +619,7 @@ func startBed(t *testing.T) *bed {
b.setHealth(t, "n1", module)
}
b.serveForge(t, ctl)
b.serveInputVerb(t, ctl)
b.fake = &fakeBuildModules{js: js, ctx: ctx, calls: map[string]int{}}
b.fake.serve(t, "mesh-build-go", b.buckets[check.CheckerKeptBucket])
b.fake.serve(t, "mesh-build-node", b.buckets[check.CheckerKeptBucket])
@@ -730,6 +851,19 @@ func TestCheckerJudgesACandidateAndOwesItsStatus(t *testing.T) { // N6/AC5, 1300
if got := asked(); len(got) == 0 || got[0].Candidate != key {
t.Errorf("n1's engine was asked %+v, want the check's candidate %s", got, key)
}
// Issue 1921 (B0): the verdict is the checking program's, built at the group's mesh-delivery head over its input kept in
// the state inputs and asked of the seat's verb input, two run asks a check (the seat list, the verdict); the verdict names the program's build.
if by := v.JudgedBy; by == nil || by.Program.Commit != head1 || by.Program.Package != check.CheckingProgramPackage ||
by.Key != "program-0123456789abcdef" || len(by.Heads) != 1 || by.Heads[0].Commit != head1 {
t.Errorf("judged by %+v, want the checking program at the head %s", v.JudgedBy, head1)
}
if n := b.fake.runs.Load(); n != 4 {
t.Errorf("%d run asks, want two for each of the two checks", n)
}
// The operator's answer on B0: each run's input reached the build module through the seat's verb input on the bus.
if n := b.inputsAnswered.Load(); n != 4 {
t.Errorf("the verb input answered %d inputs, want one for each run ask", n)
}
b.fake.mu.Lock()
names := b.fake.names
b.fake.mu.Unlock()
@@ -834,28 +968,54 @@ func keptCount(t *testing.T, b *bed) int {
return len(keys)
}
// TestAHeadThatDoesNotMergeFailsNamingTheFilesAndRunsNoTest: the fetch answers the files in conflict; the verdict fails naming
// them, no test or digest step is run, a preview of the conflicts alone is written and the status is owed as failure.
func TestAHeadThatDoesNotMergeFailsNamingTheFilesAndRunsNoTest(t *testing.T) { // 1289/AC4
b := startBed(t)
b.fake.conflict.Store(true)
key := b.candidate(t, "feat/x", pull(repo, 7, head1))
b.start(t, nil)
v := b.verdict(t, key, ended)
if v.State != meshdeliverycheckerclient.VerdictFail || !strings.Contains(v.Failed.Words, "files in conflict: a.go, sub/b.go") {
t.Fatalf("verdict = %+v", v)
}
if o := b.owed(t, 7, head1, "failure"); !strings.HasPrefix(o.Text, "failed: n1: ") || !strings.Contains(o.Text, "files in conflict: a.go, sub/b.go") {
t.Errorf("owed = %+v", o)
}
if n := b.fake.count("mesh-build-go.build_go_digest") + b.fake.count("mesh-build-go.build_go_test"); n != 0 {
t.Errorf("%d digest and test steps ran after a conflict", n)
}
pk, _ := meshdeliverycheckerclient.PreviewKey(repo, head1, trunk)
if e, err := b.buckets[check.PreviewsBucket].Get(b.ctx, pk); err != nil {
t.Errorf("previews.%s: %v", pk, err)
} else if pv, err := meshdeliverycheckerclient.DecodePreview(e.Value()); err != nil || strings.Join(pv.Conflicts, ",") != "a.go,sub/b.go" || pv.Merge != "" {
t.Errorf("preview = %+v, %v, want the files in conflict alone", pv, err)
// TestACheckThatCannotBeJudgedFailsNamingWhy: a head that does not merge fails naming the files in conflict and runs no
// test (1289/AC4); a checking program whose run asks no build node begins, and one that does not build, fail the check
// naming the checking program, never judged by the running rules (issue 1921, the lead's ruling on milestone 47/B0: the
// begin bound is the checker's setting, shortened here).
func TestACheckThatCannotBeJudgedFailsNamingWhy(t *testing.T) { // 1289/AC4, 1921
for _, row := range []struct {
name string
set func(f *fakeBuildModules)
env map[string]string
words string
tests bool // the digest and test steps ran
dropped int32 // the run asks dropped: the seat list's, put twice, and no verdict asked after it
program bool // the checking program judged it
}{
{name: "a head that does not merge", set: func(f *fakeBuildModules) { f.conflict.Store(true) }, words: "files in conflict: a.go, sub/b.go", program: true},
{name: "a run ask no build node begins", set: func(f *fakeBuildModules) { f.drop.Store(true) }, env: map[string]string{"MESH_CHECKER_BEGIN_BOUND": "500ms"},
words: "no build node began the job run of the asks", tests: true, dropped: 2},
{name: "a program that does not build", set: func(f *fakeBuildModules) { f.broken.Store("go build: undefined: rules.Nothing") },
words: "by the checking program: its run ask", tests: true},
} {
t.Run(row.name, func(t *testing.T) {
b := startBed(t)
row.set(b.fake)
key := b.candidate(t, "feat/x", pull(repo, 7, head1))
b.start(t, row.env)
v := b.verdict(t, key, ended)
if v.State != meshdeliverycheckerclient.VerdictFail || !strings.Contains(v.Failed.Words, row.words) || (v.JudgedBy != nil) != row.program {
t.Fatalf("verdict = %+v, want a fail saying %q, judged by the program %v", v, row.words, row.program)
}
if o := b.owed(t, 7, head1, "failure"); !strings.HasPrefix(o.Text, "failed") ||
!strings.Contains(o.Text, map[bool]string{true: row.words, false: "the checking program"}[row.program]) {
t.Errorf("owed = %+v", o)
}
if n := b.fake.count("mesh-build-go.build_go_digest") + b.fake.count("mesh-build-go.build_go_test"); (n != 0) != row.tests {
t.Errorf("%d digest and test steps ran, want them run %v", n, row.tests)
}
if n := b.fake.dropped.Load(); n != row.dropped {
t.Errorf("%d run asks dropped, want %d", n, row.dropped)
}
if !row.tests {
pk, _ := meshdeliverycheckerclient.PreviewKey(repo, head1, trunk)
if e, err := b.buckets[check.PreviewsBucket].Get(b.ctx, pk); err != nil {
t.Errorf("previews.%s: %v", pk, err)
} else if pv, err := meshdeliverycheckerclient.DecodePreview(e.Value()); err != nil || strings.Join(pv.Conflicts, ",") != "a.go,sub/b.go" || pv.Merge != "" {
t.Errorf("preview = %+v, %v, want the files in conflict alone", pv, err)
}
}
})
}
}
@@ -1137,7 +1297,7 @@ func TestTheCheckerSaysOwedReadableOnTheBus(t *testing.T) { // R2-1
}
if said.Module != "mesh-delivery-checker" || said.Node != "n1" || !said.Keeps("owed") || !said.Readable("owed") ||
!said.Readable("previews") || !said.Readable("verdicts") || !said.Readable("kept") || !said.Readable("tries") || said.Readable("hearing") ||
!said.Keeps("asks") || said.Readable("asks") || !said.Keeps("first-views") || said.Readable("first-views") || !said.Keeps("findings") || len(said.States) != 9 {
t.Fatalf("the checker said %+v, want its nine states, asks, hearing and first-views its own (DLV-M2), with kept, owed, previews, tries and verdicts readable, tries by the delivery view (1929/AC4; 1887 A5b; build_usage)", said)
!said.Keeps("asks") || said.Readable("asks") || !said.Keeps("first-views") || said.Readable("first-views") || !said.Keeps("findings") || !said.Keeps("inputs") || said.Readable("inputs") || !said.Keeps("compositions") || said.Readable("compositions") || len(said.States) != 11 {
t.Fatalf("the checker said %+v, want its eleven states, asks, hearing, first-views, inputs and compositions its own (DLV-M2), with kept, owed, previews, tries and verdicts readable, tries by the delivery view (1929/AC4; 1887 A5b; build_usage)", said)
}
}
@@ -30,10 +30,14 @@ import (
// judgeDeps are the parts the judge of a check is made of.
type judgeDeps struct {
asks check.Asks
renames check.Renames
seatDefs check.SeatDefinitions
// seats are the seat definitions the manifests are judged against (seatlist.go).
// inputs keeps the checking program's inputs (check.InputsStateName, served on check.InputVerb).
inputs check.Inputs
// beginBound is how long a run ask may wait for a build node to begin it; check.DefaultBeginBound when 0.
beginBound time.Duration
asks check.Asks
renames check.Renames
seatDefs check.SeatDefinitions
// seats are the seat list the checker's reader holds, the base the checking program lays the heads over (seatlist.go).
seats check.SeatList
tools *buildrun.ToolchainSet
trunks *check.Trunks
@@ -46,7 +50,6 @@ type judgeDeps struct {
// published is the indexer's packages state as it is followed (packages.go).
published *publishedPackages
forge check.Forge
trees check.Trees
announce check.Announcer
builds check.BuildStates
// ends waits on the builder's builds of a candidate before the check composes (ADR 0603 §10); nil composes without waiting.
@@ -231,7 +234,7 @@ func newJudge(d judgeDeps) check.Judge {
nodes = d.nodes
}
ex := &check.Examiner{Nodes: nodes, Compositions: d.compositions, Asks: d.asks, Toolchains: d.tools,
Kept: check.Kept{Store: d.kept}, Previews: d.previews, Candidates: d.candidates, Modules: d.repos, Packages: d.repos, Published: d.published, KnownPackages: d.published, Forge: d.forge, Runs: d.repos, Builds: d.builds, BuildEnds: d.ends, BuildWait: d.buildWait, Announce: d.announce, Trunks: d.trunks, Manifests: d.repos, Read: d.repos, Dirs: d.repos, Predefined: d.predefined, Tags: d.tags, Renames: d.renames, SeatDefinitions: d.seatDefs, SeatList: d.seats, Trees: d.trees, Logf: d.logf}
Kept: check.Kept{Store: d.kept}, Previews: d.previews, Candidates: d.candidates, Modules: d.repos, Packages: d.repos, Published: d.published, KnownPackages: d.published, Forge: d.forge, Runs: d.repos, Builds: d.builds, BuildEnds: d.ends, BuildWait: d.buildWait, Announce: d.announce, Trunks: d.trunks, Manifests: d.repos, Read: d.repos, Dirs: d.repos, Predefined: d.predefined, Tags: d.tags, Renames: d.renames, SeatDefinitions: d.seatDefs, SeatList: d.seats, Inputs: d.inputs, BeginBound: d.beginBound, Logf: d.logf}
if d.nodes != nil && d.builds != nil && d.repos != nil {
ex.Serving = runningServes{have: d.nodes, builds: d.builds, read: d.repos, logf: d.logf}
}
@@ -158,6 +158,10 @@ func TestThePublishedVersionsAreTheIndexersPackagesState(t *testing.T) {
if got := p.LatestPackages(); len(got) != 1 || got[sdk] == "" {
t.Errorf("after a delete: %v", got)
}
// Issue 1966: a module record names its module, with no series to say it, and none of the packages inside it.
if got := p.PackageNames(); !reflect.DeepEqual(got, map[string]string{sdk: ""}) {
t.Errorf("the known packages from a module record = %v", got)
}
// A series key names its package by its ecosystem name, the judge's known packages (DLV-I14, design 32), and is no
// module's published version.
now := time.Now().UTC()
@@ -38,6 +38,7 @@ func TestTheCheckerAnnouncesEveryStateItKeepsAndOwedReadable(t *testing.T) { //
t.Errorf("announced %v, the manifest keeps %v", announced, manifest)
}
// kept is read by the Go build module for a check ask (issue 1887 design limit 1); tries by the delivery view (build_usage).
// inputs are the checker's own: the Go build module asks the seat's verb input for one (ADR 0596 §2, issue 1921).
if want := []string{"kept", "owed", "previews", "tries", "verdicts"}; !equal(readable, want) {
t.Errorf("readable %v, want %v", readable, want)
}
@@ -101,6 +101,12 @@ var keptStates = []announce.State{
{Name: "hearing"},
// The asks each running check put (check.AsksState): read by the checker's own checker_ask_again alone.
{Name: "asks"},
// The checking program's inputs by their digest (check.InputsStateName): the checker's own, read by its tools alone,
// which serve one on the seat's verb input to the Go build module that runs the program (check.InputVerb; ADR 0596 §2,
// issue 1921).
{Name: check.InputsStateName},
// The composition requests it sent the composer, per check and node (check.RequestsStateName): read by its tools alone.
{Name: check.RequestsStateName},
{Name: "kept", Readable: true},
{Name: "owed", Readable: true},
{Name: "previews", Readable: true},
@@ -187,6 +193,11 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
if err != nil {
return err
}
// The begin bound of a run ask of the checking program (check.DefaultBeginBound, issue 1921).
beginBound, err := durationEnv(getenv, "MESH_CHECKER_BEGIN_BOUND", check.DefaultBeginBound)
if err != nil {
return err
}
link := strings.TrimSpace(getenv("MESH_CHECKER_LINK"))
if link == "" {
link = defaultLink
@@ -215,6 +226,8 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
conn.SetReconnectHandler(func(*nats.Conn) { hearing.Again(); heard.Nudge("") })
waiter := buildrun.LostAskWaiter{Inner: heard, Dead: dead, Hearing: hearing}
asksKV := process.AwaitBucket(ctx, js, conn, check.AsksBucket, "checker_ask_again refuses every check, as it cannot tell whether its ask waits in a stream", logger.Printf)
inputsKV := process.AwaitBucket(ctx, js, conn, check.InputsBucket, "no checking program can be asked, so every check fails naming it", logger.Printf)
requestsKV := process.AwaitBucket(ctx, js, conn, check.RequestsBucket, "no composition request is kept, so every composition made with the node-engine's head fails naming it", logger.Printf)
ch := &check.Checker{
Asks: awaitedKV{asksKV},
Verdicts: kvStore{buckets[check.CheckerVerdictsBucket]},
@@ -245,11 +258,14 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
holdCallsFor(answerBound)
verdicts := &manifestVerdicts{}
compositions := check.EngineCompositions{
Requests: check.RequestKeeper{Store: requestsKV},
Asker: engineAsks{call: caller.CallOn},
Placed: placedNodes{kv: func(context.Context) (jetstream.KeyValue, error) { return assignments, nil }}.Placed,
Bound: answerBound,
Sandbox: seatSandbox{asker: buildrun.Asker{Publisher: asks, Waiter: waiter, Bound: stepBound}},
Verdicts: verdicts.all,
// The node-engine's trunk head: a group with no node-engine member composes with the composer built there (ADR 0656 §5).
Composer: func() string { return trunks.Heads()[check.NodeEngineRepository] },
}
// The tries (ADR 0501 §5) are judged by the same judge, held by the same deliveries.
tryer := &check.Tryer{Tries: kvStore{buckets[check.TriesBucket]}, Bound: checkBound, NotTaken: stepBound, Logf: logger.Printf,
@@ -290,8 +306,8 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
ch.Judge = newJudge(judgeDeps{
asks: buildrun.Asker{Publisher: asks, Waiter: waiter, Bound: stepBound}, tools: tools, trunks: trunks,
kept: kvStore{buckets[check.CheckerKeptBucket]}, previews: kvStore{buckets[check.PreviewsBucket]}, candidates: tryCandidates{candidates},
repos: repos, published: published, builds: buildStates, ends: ends, buildWait: set.buildWait, forge: forgeReads{call: caller.Call}, trees: forgeReads{call: caller.Call}, logf: logger.Printf,
announce: announce, compositions: compositions, tags: tags,
repos: repos, published: published, builds: buildStates, ends: ends, buildWait: set.buildWait, forge: forgeReads{call: caller.Call}, logf: logger.Printf,
announce: announce, compositions: compositions, tags: tags, inputs: runInputs{awaitedKV{inputsKV}}, beginBound: beginBound,
nodes: openNodeNames(ctx, js, logger.Printf),
predefined: keptPredefined{predefined},
renames: seatRenames{call: caller.Call},
@@ -78,7 +78,9 @@ func (p *publishedPackages) LatestPackages() buildstep.PackageVersions {
return out
}
// PackageNames are the packages the state's series keys name, by the name in its own ecosystem: the `<module>/<artifact>`
// PackageNames are the packages the state's series keys name, and the Go module path of each per-module record (issue
// 1966: the indexer's own knownPackages counts them the same way, with no `<module>/<artifact>` to say; never a package
// inside a module, design 32), by the name in its own ecosystem: the `<module>/<artifact>`
// each is built from (empty when the key does not say). Nil while the state is not read yet (Answered), which judges no
// name; once read never nil, an empty state included, so the judge judges every name against it.
func (p *publishedPackages) PackageNames() map[string]string {
@@ -93,6 +95,11 @@ func (p *publishedPackages) PackageNames() map[string]string {
out[s.Name] = s.Package
}
}
for _, v := range p.byKey {
if _, ok := out[v.Module]; !ok && v.Module != "" {
out[v.Module] = ""
}
}
return out
}
@@ -9,6 +9,8 @@ import (
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"mesh-delivery-checker/internal/check"
)
// tryCandidates answers the indexer's candidate of a try's group (novox/hq ADR 0593 §7, issue 1921): the candidate the
@@ -59,7 +61,7 @@ func (t tryCandidates) TryCandidate(ctx context.Context, group, heads string, bo
func waitsOnPreview(c meshcatalog.CandidatesByGroupValue) bool {
for _, p := range c.Pulls {
for _, m := range p.Fps {
if m.None2 == meshcatalogclient.NoneNoPreview {
if check.WaitsOnPreview(m) {
return true
}
}
@@ -31,7 +31,7 @@ const NodeEngineModule = nodeEngineModule
const (
catalogueRepository = "novox/mesh-catalog"
nodeEngineRepo = "novox/mesh-host"
nodeEngineRepo = NodeEngineRepository
nodeEngineModule = "mesh-host"
// networkingRepository holds route-proxy, whose manifest the node-engine's proofs of a composition read.
networkingRepository = "novox/mesh-networking"
@@ -213,3 +213,6 @@ func besideAtTrunk(c *Case, refs []BesideRef, into map[string]string) {
}
}
}
// NodeEngineRepository is the node-engine's repository, whose trunk head a check without its member composes at.
const NodeEngineRepository = "novox/mesh-host"
@@ -2,31 +2,26 @@ package check
import (
"fmt"
"strings"
"mesh-delivery-checker/rules"
)
// A change that adds or modifies a compiled program fails (issue 1577): it is build output, `go build` in a module's folder
// writes one beside go.mod, and `git add -A` takes it. The build module's fetch step, which holds the merged clone, reads the
// first bytes of each changed file and reports the programs (the build modules' fetch tools); the checker judges the answer.
// checkBinariesOf fails a pull request whose change adds or modifies a compiled program, naming each file. It judges the
// first fetch answer of the pull request, whichever build module gave it. A fetch that made the merge and names no program has
// none: the answer's `programs` is an omitted empty list (B2g), and `merge` tells a preview was made.
func (st *state) checkBinariesOf(t *target) verdictPtr {
// checkBinariesOf records what a pull request's first fetch answer read of compiled programs, whichever build module
// gave it, as a fact the rules judge (rules.JudgePrograms). A fetch that made the merge and names no program has none: the
// answer's `programs` is an omitted empty list (B2g), and `merge` tells a preview was made.
func (st *state) checkBinariesOf(t *target) {
key := fmt.Sprintf("%s#%d", t.pull.Repository, t.pull.Number)
if t.fetched.Merge == "" {
return nil // no merge was made: nothing was changed to judge
return // no merge was made: nothing was changed to judge
}
var found []string
for _, p := range t.fetched.Programs {
if p.Commit != "" {
found = append(found, fmt.Sprintf("%s: a compiled program added in %s and removed later: it stays in history; squash the branch (issue 1577)", p.Path, short12(p.Commit)))
continue
}
found = append(found, p.Path+": a compiled program is build output: remove it and add it to .gitignore (issue 1577)")
files := t.fetched.Programs
if files == nil {
files = []ProgramFile{} // reported, and none: the rules refuse nothing (a nil list is a module that reports none)
}
if len(found) > 0 {
return failed(t.node, fmt.Sprintf("%s: %s", key, strings.Join(found, "; ")))
}
return nil
st.facts = append(st.facts, rules.Step{Repository: t.pull.Repository, Number: t.pull.Number, Module: t.module, Node: t.node,
Programs: &rules.Programs{Key: key, Files: &files}})
}
@@ -124,20 +124,22 @@ func (j *Examiner) buildWait() time.Duration {
// awaitBuilds waits for the candidate's builds before the check composes (ADR 0603 §10): the check builds what the builder
// builds of the candidate and composes with it. A failed build fails the check, naming the module and the builder's words;
// a build not ended within the bound is named, and the check composes without it. It answers the words of the composition's
// warnings: what the check composes without waiting on its build, and why.
func (st *state) awaitBuilds(ctx context.Context, v *meshdeliverychecker.VerdictsByGroupHeadsValue) []string {
// a build not ended within the bound is named in late, and the check does not compose: an engine composes a module only at
// a done build, so its answer would be the builder's lateness read as the change's refusal (issue 1921). It answers the
// words of the composition's warnings (what the check composes without waiting on its build, and why) and the failures of
// the builds that failed, for the rules to take in (rules.JudgeComposition); failed says the check has failed already.
func (st *state) awaitBuilds(ctx context.Context, failed bool) (words []string, failedBuilds []FailedFirst, late string) {
wait, words := st.buildsOfTheCheck(ctx)
if len(wait) == 0 {
return words
return words, nil, ""
}
modules := slices.Sorted(maps.Keys(wait))
if v.State == meshdeliverycheckerclient.VerdictFail {
if failed {
// The check has failed already: its red status is never held back by a build (the lead, 2026-11-03).
return append(words, fmt.Sprintf("the check failed before its builds were waited on, so it composes at once, without waiting on the builds of %s (ADR 0603 §10)", strings.Join(modules, ", ")))
return append(words, fmt.Sprintf("the check failed before its builds were waited on, so it composes at once, without waiting on the builds of %s (ADR 0603 §10)", strings.Join(modules, ", "))), nil, ""
}
if st.j.BuildEnds == nil {
return append(words, fmt.Sprintf("the builds of %s cannot be heard, so the check composes without waiting on them (ADR 0603 §10)", strings.Join(modules, ", ")))
return append(words, fmt.Sprintf("the builds of %s cannot be heard, so the check composes without waiting on them (ADR 0603 §10)", strings.Join(modules, ", "))), nil, ""
}
bound := st.j.buildWait()
waits := fmt.Sprintf("waits on the builder's builds of %s before it composes, at most %s (ADR 0603 §10)", strings.Join(modules, ", "), bound)
@@ -148,25 +150,20 @@ func (st *state) awaitBuilds(ctx context.Context, v *meshdeliverychecker.Verdict
cancel()
if err != nil {
st.j.logf("%s: the builds of %s cannot be heard: %v", st.c.Key, strings.Join(modules, ", "), err)
return append(words, fmt.Sprintf("the builds of %s cannot be heard (%v), so the check composes without waiting on them (ADR 0603 §10)", strings.Join(modules, ", "), err))
return append(words, fmt.Sprintf("the builds of %s cannot be heard (%v), so the check composes without waiting on them (ADR 0603 §10)", strings.Join(modules, ", "), err)), nil, ""
}
var late []string
var notEnded []string
for _, module := range modules {
b, done := ended[module]
switch {
case !done:
late = append(late, module)
notEnded = append(notEnded, module)
case b.State == builds.BuildFailed:
f := FailedFirst{Test: BuildFailedTest + module, Words: fmt.Sprintf("the candidate's build at %s failed (rebuild with builder_build_again; the check asks itself again when the build is done): %s", wait[module], b.Error)}
if v.State != meshdeliverycheckerclient.VerdictFail {
v.State, v.Failed = meshdeliverycheckerclient.VerdictFail, &f
continue
}
addFailure(v, f)
failedBuilds = append(failedBuilds, FailedFirst{Test: BuildFailedTest + module, Words: fmt.Sprintf("the candidate's build at %s failed (rebuild with builder_build_again; the check asks itself again when the build is done): %s", wait[module], b.Error)})
}
}
if len(late) > 0 && ctx.Err() == nil {
words = append(words, fmt.Sprintf("the builds of %s did not end within the build-wait-bound of %s, so the check composes without them (ADR 0603 §10)", strings.Join(late, ", "), bound))
if len(notEnded) > 0 && ctx.Err() == nil {
late = fmt.Sprintf("the builder's builds of %s did not end within the build-wait-bound of %s, so the change is not composed: an engine composes a module only at a done build (ADR 0603 §10)", strings.Join(notEnded, ", "), bound)
}
return words
return words, failedBuilds, late
}
@@ -1,6 +1,7 @@
package check
import (
"encoding/json"
"fmt"
"maps"
"slices"
@@ -43,8 +44,18 @@ type Case struct {
Left []string
// Queued is when the checker took the case up, for the time of its queue.
Queued time.Time
// Seats is the seat list the checking program made, the checker's reader's list with the heads' definitions laid over
// it, which the manifest validator's and the node-engine composer's requests carry (ADR 0636 §1, §4); none until it is
// made.
Seats []json.RawMessage
// Try narrows a try's check (its level, its tests); nil for a pull request's.
Try *TryOptions
// Judged are the check's own judge's verdicts on the manifests the change brings, by module, as the seat mesh-catalog's
// verdict value (judgeManifests): the composer's request carries them in place of the running indexer's (ADR 0578 §2).
Judged map[string]json.RawMessage
// Tagged are the group's tagged versions of its Go packages the check heard (design 51 *Tagged versions*), nil for none:
// a composer built at the node-engine member's head is built against them, as the check's other Go builds are.
Tagged buildstep.PackageVersions
notesMu sync.Mutex
// warnings are, by pull request, what part of its check did not run (Case.warn).
@@ -5,6 +5,7 @@ import (
"errors"
"fmt"
"maps"
"mesh-delivery-checker/rules"
"slices"
"sort"
"strings"
@@ -464,7 +465,7 @@ func (ch *Checker) untrunked(ctx context.Context, c meshcatalog.CandidatesByGrou
continue
}
o := meshdeliverychecker.OwedByRepositoryNumberHeadActValue{Act: meshdeliveryclient.OwedStatus, Context: meshdeliveryclient.StatusContextCheck, State: "error",
Target: strings.TrimSuffix(ch.Link, "/") + "/" + key, Text: clipLine(words, MaxLine)}
Target: strings.TrimSuffix(ch.Link, "/") + "/" + key, Text: rules.ClipLine(words, MaxLine)}
data, err := EncodeCheckerOwed(o)
if err != nil {
ch.logf("%s: the status of %s cannot be written: %v", key, okey, err)
@@ -1,42 +0,0 @@
package check
import (
"strings"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliverycheckerclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants"
)
// withComposition is v with the check's composition taken in: a failed one fails the check, naming its machine.
func withComposition(v *meshdeliverychecker.VerdictsByGroupHeadsValue, comp meshwants.CompositionsByGroupHeadsValue) {
v.Composition = compositionOf(comp)
if v.Took == nil {
v.Took = &meshdeliverychecker.VerdictsByGroupHeadsValueTook{}
}
v.Took.Composition = comp.Took
if comp.State == meshdeliverycheckerclient.CompositionPass && len(comp.Warnings) > 0 && v.Composition != nil {
v.Composition.Words = strings.Join(comp.Warnings, "; ") // passed with a warning (ADR 0593 §9)
}
if comp.State == meshdeliverycheckerclient.CompositionFail {
machine := comp.Machine
if machine == "" {
machine = "the composition"
}
f := FailedFirst{Machine: machine, Words: comp.Words}
if v.State == meshdeliverycheckerclient.VerdictFail {
addFailure(v, f) // the tests failed too: the composition is named after them (the operator, 2026-11-02)
return
}
v.State, v.Failed = meshdeliverycheckerclient.VerdictFail, &f
}
}
// compositionOf is what the checker writes into its verdict from the assigner's composition answer c: nil while c waits (the
// check is still running).
func compositionOf(c meshwants.CompositionsByGroupHeadsValue) *meshdeliverychecker.VerdictsByGroupHeadsValueComposition {
if c.State == compositionWaits {
return nil
}
return &meshdeliverychecker.VerdictsByGroupHeadsValueComposition{State: c.State, Words: c.Words, Sdk: c.Sdk, Machines: c.Machines}
}
@@ -2,45 +2,67 @@ package check
import (
"context"
"encoding/json"
"errors"
"maps"
"mesh-delivery-checker/rules"
"slices"
"strings"
"testing"
nec "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
ne "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliverycheckerclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants"
buildsrec "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// fakeCompositions answers a check's composition from memory, recording the keys asked.
type fakeCompositions struct {
v meshwants.CompositionsByGroupHeadsValue
v rules.Composed
err error
asked []string
// seats are the seat lists the cases asked carried (Case.Seats), one per ask.
seats [][]json.RawMessage
// waits answers only when its context ends.
waits bool
}
func (f *fakeCompositions) Compose(ctx context.Context, c *Case, _ buildstep.PackageVersions) (meshwants.CompositionsByGroupHeadsValue, error) {
func (f *fakeCompositions) Compose(ctx context.Context, c *Case) (rules.Composed, error) {
f.asked = append(f.asked, c.Key)
f.seats = append(f.seats, c.Seats)
if f.waits {
<-ctx.Done()
return meshwants.CompositionsByGroupHeadsValue{}, ctx.Err()
return rules.Composed{}, ctx.Err()
}
return f.v, f.err
}
// composedOn is the engines' answers of a composition by composer on the nodes, each composing it, taking took ms.
func composedOn(composer string, took int64, nodes ...string) rules.Composed {
c := rules.Composed{Group: "feat/x", Heads: "0123456789ab", Composer: composer, Nodes: nodes, Took: took, Answers: map[string]ne.ComposeCandidateAnswer{}}
for _, n := range nodes {
c.Answers[n] = ne.ComposeCandidateAnswer{Node: n, Verdict: nec.Composed}
}
return c
}
// refusedOn is the engines' answers of a composition the node refuses with the words.
func refusedOn(composer, node, words string) rules.Composed {
return rules.Composed{Group: "feat/x", Heads: "0123456789ab", Composer: composer, Nodes: []string{node},
Answers: map[string]ne.ComposeCandidateAnswer{node: {Node: node, Verdict: nec.Refused, Refused: words}}}
}
// 1297/AC3 (its unit half): a check whose tests pass takes the assigner's composition of it, by the check's own key,
// into its verdict: a pass says it passed with the mesh-sdk and machines it composed, and is no longer "not run".
func TestAPassingCheckTakesTheAssignersComposition(t *testing.T) {
t.Run("a change to a build module or base names the modules built on it", aChangeToABuildModuleOrBaseNamesTheModulesBuiltOnIt)
b := newBench(t)
f := &fakeCompositions{v: meshwants.CompositionsByGroupHeadsValue{Group: "feat/x", Heads: "0123456789ab", State: meshdeliverycheckerclient.CompositionPass, Sdk: "v0.1.14-x", Machines: []string{"g14"}, Took: 40}}
f := &fakeCompositions{v: composedOn("v0.1.14-x", 40, "g14")}
b.ex.Compositions = f
cs := caseOf(t, pullOf(repoA, 7, head1))
cs.FPSet = "fps-of-the-candidate"
@@ -57,6 +79,39 @@ func TestAPassingCheckTakesTheAssignersComposition(t *testing.T) {
if v.Composition.Fpset == "" || v.Composition.Fpset != cs.FPSet {
t.Errorf("the composition names the fingerprint set %q, want the candidate's %q (ADR 0548 §5)", v.Composition.Fpset, cs.FPSet)
}
// Issue 1966 (the group try of 2026-11-06 10:07): the composition is handed the check's seat list, the judge's, once the
// checker's reader read it; a list not read hands none (the composer then judges no machine, ADR 0636 §1).
if len(f.seats) != 1 || f.seats[0] != nil {
t.Errorf("with no seat list reader the composition was handed %v, want none", f.seats)
}
for _, read := range []bool{true, false} {
b.ex.SeatList = fakeSeatList{read: read}
asked := len(f.seats)
rv := b.judge(t, caseOf(t, pullOf(repoA, 7, head1))).Verdict
if read {
if got := f.seats[len(f.seats)-1]; len(got) != 2 || string(got[0]) != `{"seat":"a"}` {
t.Errorf("with the list read the composition was handed %s, want the reader's two definitions", got)
}
} else if len(f.seats) != asked || rv.Failed == nil || !strings.Contains(rv.Failed.Words, "the seat list is not read") {
// A reader that has not read the list acts on nothing new (ADR 0636 §1): the check ends before any judge.
t.Errorf("with the list not read the composition was asked %d more, verdict %+v; want none and a fail naming the list", len(f.seats)-asked, rv.Failed)
}
}
// ADR 0635 §3: at a node-engine member's head a list not read fails the check, naming why, and asks no composer
// (a composer handed none would judge no machine and the composition would pass); read, it composes.
for _, read := range []bool{false, true} {
b.ex.SeatList = fakeSeatList{read: read}
asked := len(f.asked)
hv := b.judge(t, caseOf(t, pullOf(repoA, 7, head1), pullOf(nodeEngineRepo, 3, head2))).Verdict
switch {
case !read && (hv.State != meshdeliverycheckerclient.VerdictFail || hv.Failed == nil || !strings.Contains(hv.Failed.Words, "seat list") || len(f.asked) != asked):
t.Errorf("at a node-engine head with the list not read: verdict %+v, composition %+v, asked %d more; want a failed composition naming the seat list, no ask", hv, hv.Composition, len(f.asked)-asked)
case read && (hv.Composition == nil || hv.Composition.State != meshdeliverycheckerclient.CompositionPass || len(f.asked) != asked+1):
t.Errorf("at a node-engine head with the list read: composition %+v, want the composer's pass", hv.Composition)
}
}
b.ex.SeatList = nil
f.asked, f.seats = f.asked[:1], f.seats[:1]
// ADR 0548 §5: a carry whose fingerprints did not change keeps the composition it passed and asks the assigner nothing.
carry := caseOf(t, pullOf(repoA, 7, head1))
carry.CarriedFrom, carry.Composition = map[string]string{repoA: trunkA2}, v.Composition
@@ -72,8 +127,7 @@ func compositionMentions(v meshdeliverychecker.VerdictsByGroupHeadsValue, word s
// 1297/AC2 at the checker: a failed composition fails the check, naming the machine it failed at and its words.
func TestAFailedCompositionFailsTheCheck(t *testing.T) {
b := newBench(t)
b.ex.Compositions = &fakeCompositions{v: meshwants.CompositionsByGroupHeadsValue{Group: "feat/x", Heads: "0123456789ab", State: meshdeliverycheckerclient.CompositionFail, Sdk: "x",
Machine: "g14", Words: "g14 runs web, which the change removes from its source"}}
b.ex.Compositions = &fakeCompositions{v: refusedOn("x", "g14", "g14 runs web, which the change removes from its source")}
v := b.judge(t, caseOf(t, pullOf(repoA, 7, head1))).Verdict
if v.State != meshdeliverycheckerclient.VerdictFail || v.Failed.Machine != "g14" || v.Composition == nil || v.Composition.State != meshdeliverycheckerclient.CompositionFail {
t.Fatalf("verdict = %+v", v)
@@ -94,7 +148,7 @@ func TestACompositionNotHadIsNotRun(t *testing.T) {
t.Fatalf("verdict = %+v", v)
}
b2 := newBench(t)
f2 := &fakeCompositions{v: meshwants.CompositionsByGroupHeadsValue{State: meshdeliverycheckerclient.CompositionFail, Words: "w", Sdk: "x"}}
f2 := &fakeCompositions{v: refusedOn("x", "g14", "w")}
b2.ex.Compositions = f2
cs := caseOf(t, pullOf(repoA, 7, head1))
cs.Try = &TryOptions{}
@@ -170,6 +224,8 @@ func aChangeToABuildModuleOrBaseNamesTheModulesBuiltOnIt(t *testing.T) {
failed string // the failure's test, when the check fails on a build
tests bool // the tests fail: the check fails before it would wait
served []string // what the running builds serve (ADR 0614 §7); nil reads nothing of what runs
preview bool // the check began on the candidate the indexer wrote before its merge preview (issue 1921)
late bool // a build did not end within the bound: the change is not composed (issue 1921)
}{
{name: "the Go build module invokes a verb no running build serves",
manifest: map[string]catalogue.Manifest{"mesh-build-go": {Module: "mesh-build-go", Build: goBuild,
@@ -201,8 +257,10 @@ func aChangeToABuildModuleOrBaseNamesTheModulesBuiltOnIt(t *testing.T) {
builds: map[string]buildsrec.BuilderBuild{"web": {State: buildsrec.BuildBuilding}}, tests: true,
want: []string{"composes at once, without waiting on the builds of web"}},
{name: "a build does not end within the bound", manifest: map[string]catalogue.Manifest{"web": {Module: "web", Build: goBuild}},
builds: map[string]buildsrec.BuilderBuild{"web": {State: buildsrec.BuildBuilding}},
want: []string{"the builds of web did not end within the build-wait-bound"}, waited: "web"},
builds: map[string]buildsrec.BuilderBuild{"web": {State: buildsrec.BuildBuilding}}, late: true,
want: []string{"the builder's builds of web did not end within the build-wait-bound", "the change is not composed"}, waited: "web"},
{name: "the check began before the merge preview", manifest: map[string]catalogue.Manifest{"web": {Module: "web", Build: goBuild}},
builds: map[string]buildsrec.BuilderBuild{"web": done}, preview: true, waited: "web"},
{name: "a module the trunk does not hold", manifest: map[string]catalogue.Manifest{"web": {Module: "web", Build: goBuild}, "new": {Module: "new", Build: goBuild}},
builds: map[string]buildsrec.BuilderBuild{"web": done}, runs: knownRuns{"web"},
want: []string{"new: the trunk holds no record of it yet"}, waited: "web"},
@@ -217,7 +275,7 @@ func aChangeToABuildModuleOrBaseNamesTheModulesBuiltOnIt(t *testing.T) {
return r, nil
}
}
comps := &fakeCompositions{v: meshwants.CompositionsByGroupHeadsValue{Group: "feat/x", Heads: "0123456789ab", State: meshdeliverycheckerclient.CompositionPass, Sdk: "x", Machines: []string{"g14"}}}
comps := &fakeCompositions{v: composedOn("x", 0, "g14")}
b.ex.Compositions = comps
var ends *fakeEnds
if row.builds != nil {
@@ -241,8 +299,20 @@ func aChangeToABuildModuleOrBaseNamesTheModulesBuiltOnIt(t *testing.T) {
b.ex.Read = read
p := pullOf(catalogueRepository, 9, head1)
p.Fps, p.Modules = fps, slices.Sorted(maps.Keys(fps))
if row.preview {
previewed := p
b.ex.Candidates = &fakeTryCandidates{cand: meshcatalog.CandidatesByGroupValue{Fpset: "fps", Pulls: []meshcatalog.CandidatesByGroupValuePullsItem{previewed}}}
p.Fps = map[string]meshcatalog.CandidatesByGroupValuePullsItemFpsValue{}
for module, m := range fps {
p.Fps[module] = meshcatalog.CandidatesByGroupValuePullsItemFpsValue{Manifest: m.Manifest, None2: meshcatalogclient.NoneNoPreview}
}
}
v := b.judge(t, caseOf(t, p)).Verdict
if v.Composition == nil || len(comps.asked) != 1 {
asks := 1
if row.late {
asks = 0 // never asked: an engine would read the builder's lateness as the change's refusal
}
if v.Composition == nil || len(comps.asked) != asks || row.late && v.Composition.State != meshdeliverycheckerclient.CompositionNotRun {
t.Fatalf("%s: composed %d times: %+v", row.name, len(comps.asked), v)
}
if ends != nil {
@@ -270,3 +340,5 @@ func aChangeToABuildModuleOrBaseNamesTheModulesBuiltOnIt(t *testing.T) {
}
}
}
func ptr[T any](v T) *T { return &v }
@@ -4,6 +4,7 @@ import (
"context"
"encoding/json"
"fmt"
"mesh-delivery-checker/rules"
"sort"
"strings"
"sync"
@@ -13,9 +14,7 @@ import (
nec "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
ne "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliverycheckerclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
"git.novox.be/novox/mesh-delivery/sdk/composer"
)
@@ -51,7 +50,10 @@ type EngineCompositions struct {
Placed func() ([]string, error)
// Bound is the answer bound (the setting compose-answer-bound); zero is nec.AnswerBound.
Bound time.Duration
// Composer is the commit of the node-engine the mesh runs, named as what composed (ADR 0548 §6's running commit).
// Composer is the node-engine's trunk head: a check whose group has no node-engine member is composed by the composer
// built there, so every check composes with an acting part built from the change's heads or their trunk, never with
// the running engines' own code (ADR 0656 §5; the lead's ruling Q3 on milestone 47/G). Nil, or an empty head, leaves
// such a check composed by the running engines, said.
Composer func() string
// Sandbox runs the composer of a node-engine member's head; nil leaves such a group judged by the running engines,
// said in a warning.
@@ -65,9 +67,22 @@ type EngineCompositions struct {
LatestPackages() buildstep.PackageVersions
}
Verdicts func() map[string]meshcatalog.VerdictsByDigestValue
// Requests keeps each request sent to the composer, per check and node (issue 1921); nil keeps none. A request that
// cannot be kept fails the node's composition, naming why: never dropped.
Requests interface {
Keep(ctx context.Context, key, node string, body []byte) error
}
}
// composerHead is the node-engine member's head of a case, or empty.
// atWord names where a composer was built: the member's head or the node-engine's trunk.
func atWord(member bool) string {
if member {
return "head"
}
return "trunk"
}
func composerHead(c *Case) string {
if m := c.member(nodeEngineRepo); m != nil {
return m.Head
@@ -75,24 +90,29 @@ func composerHead(c *Case) string {
return ""
}
// Compose asks every placed node and judges their answers (JudgeEngines). An error is only a case it cannot ask at all.
func (e EngineCompositions) Compose(ctx context.Context, c *Case, tagged buildstep.PackageVersions) (meshwants.CompositionsByGroupHeadsValue, error) {
// Compose asks every placed node and answers their answers, for the rules to judge (rules.JudgeEngines). An error is only a
// case it cannot ask at all.
func (e EngineCompositions) Compose(ctx context.Context, c *Case) (rules.Composed, error) {
began := time.Now()
key, err := nodeengineclient.CompositionKey(c.Group, c.Heads)
if err != nil {
return meshwants.CompositionsByGroupHeadsValue{}, err
return rules.Composed{}, err
}
var nodes []string
if e.Placed != nil {
if nodes, err = e.Placed(); err != nil {
return meshwants.CompositionsByGroupHeadsValue{}, fmt.Errorf("the assignments cannot be read, so no node can be asked: %w", err)
return rules.Composed{}, fmt.Errorf("the assignments cannot be read, so no node can be asked: %w", err)
}
}
bound := e.Bound
if bound <= 0 {
bound = nec.AnswerBound
}
head := composerHead(c)
// One composition path (ADR 0656 §5): the composer built at the node-engine member's head, else at its trunk.
head, member := composerHead(c), true
if head == "" && e.Composer != nil {
head, member = e.Composer(), false
}
sandboxed := head != "" && e.Sandbox != nil
answers := map[string]ne.ComposeCandidateAnswer{}
silent := map[string]string{}
@@ -109,9 +129,9 @@ func (e EngineCompositions) Compose(ctx context.Context, c *Case, tagged buildst
// The composer's build and run are the check's to bound, not the engine's answer bound. A composer that
// cannot run is no silent node (ADR 0593 §9 lets only that pass): the node's composition fails, naming
// the head.
if composed, serr := e.inSandbox(ctx, c, tagged, head, node, a); serr != nil {
if composed, serr := e.inSandbox(ctx, c, head, node, a); serr != nil {
a = ne.ComposeCandidateAnswer{V: a.V, Node: node, Candidate: a.Candidate, Verdict: nec.Refused,
Refused: "the composer at the node-engine's head " + head + " could not compose it: " + serr.Error()}
Refused: "the composer at the node-engine's " + atWord(member) + " " + head + " could not compose it: " + serr.Error()}
} else {
a = composed
}
@@ -133,72 +153,22 @@ func (e EngineCompositions) Compose(ctx context.Context, c *Case, tagged buildst
composerName := "node-engine"
if sandboxed {
composerName = head
} else if e.Composer != nil && e.Composer() != "" {
composerName = e.Composer()
}
v := JudgeEngines(c.Group, c.Heads, composerName, nodes, answers, silent)
if head != "" && e.Sandbox == nil {
v := rules.Composed{Group: c.Group, Heads: c.HeadsHash, Composer: composerName, Nodes: nodes, Answers: answers, Silent: silent}
switch {
case member && head != "" && e.Sandbox == nil:
v.Warnings = append(v.Warnings, "the group changes the node-engine's composition code and was composed by the running engines")
case !sandboxed && e.Sandbox != nil && len(nodes) > 0:
v.Warnings = append(v.Warnings, "composed by the running engines: no composer at the node-engine's trunk is known to build")
}
v.Took = time.Since(began).Milliseconds()
return v, nil
}
// JudgeEngines is the check's composition verdict from the engines' answers (ADR 0593 §7, §9): failed when any node's
// answer is refused, naming each node and each module refused with its words (the first refusing node is the machine);
// passed otherwise. Every node in nodes without an answer (silent, by node with why, for the journal only) is named
// with nec.NoAnswer as a warning. composer names what composed, for the line. Pure.
func JudgeEngines(group, heads, composerName string, nodes []string, answers map[string]ne.ComposeCandidateAnswer, silent map[string]string) meshwants.CompositionsByGroupHeadsValue {
v := meshwants.CompositionsByGroupHeadsValue{Group: group, Heads: heads, State: meshdeliverycheckerclient.CompositionPass, Sdk: composerName}
sorted := append([]string(nil), nodes...)
sort.Strings(sorted)
var fails []string
for _, node := range sorted {
a, answered := answers[node]
if !answered {
v.Warnings = append(v.Warnings, node+": "+nec.NoAnswer)
continue
}
if a.Verdict == nec.NotJudged {
// Composed in the sandbox and not judged (ADR 0593 §9): named as no answer, with the composer's words; decided by the
// verdict, never by the words (ADR 0601 §4's kind).
v.Warnings = append(v.Warnings, node+": "+nec.NoAnswer+" ("+a.Refused+")")
continue
}
v.Machines = append(v.Machines, node)
if a.Verdict != nec.Refused {
continue
}
if v.Machine == "" {
v.Machine = node
}
var words []string
if a.Refused != "" {
words = append(words, a.Refused)
}
for _, m := range append(append([]ne.Module(nil), a.Modules...), a.Own) {
if m.State == nec.ModuleRefused {
words = append(words, m.Module+": "+m.Words)
}
}
if len(words) == 0 {
words = append(words, "refused, naming no module")
}
fails = append(fails, "not composed on "+node+": "+strings.Join(words, "; "))
}
if len(fails) > 0 {
v.State, v.Words = meshdeliverycheckerclient.CompositionFail, strings.Join(fails, "; ")
}
if len(sorted) == 0 {
v.Warnings = append(v.Warnings, "no node is placed, so the change was composed on none")
}
return v
}
// inSandbox composes a node's answer again with the composer built from the node-engine member's head, over the input the
// node's engine composed from: the node's answer becomes the sandbox's, refused with the composer's words where the
// composer refuses the node.
func (e EngineCompositions) inSandbox(ctx context.Context, c *Case, tagged buildstep.PackageVersions, head, node string, a ne.ComposeCandidateAnswer) (ne.ComposeCandidateAnswer, error) {
func (e EngineCompositions) inSandbox(ctx context.Context, c *Case, head, node string, a ne.ComposeCandidateAnswer) (ne.ComposeCandidateAnswer, error) {
if len(a.Input) == 0 {
return a, fmt.Errorf("the engine of %s answered no composition input, so the node-engine's head cannot compose it", node)
}
@@ -225,9 +195,14 @@ func (e EngineCompositions) inSandbox(ctx context.Context, c *Case, tagged build
verdicts[name] = raw
}
}
// A manifest the change brings is the check's own judge's (ADR 0578 §2, §3): its verdict there when the judge gave one,
// else none, which the composer reads as accepted; never the running indexer's (F5 of milestone 47/G).
for _, p := range c.Pulls {
for name := range p.Fps {
delete(verdicts, name) // a manifest the change brings is the checker's own judge's (ADR 0578 §2)
delete(verdicts, name)
if raw, ok := c.Judged[name]; ok {
verdicts[name] = raw
}
}
}
// The answer carries its input as the object the seat's contract names, decoded with its numbers as written
@@ -237,12 +212,21 @@ func (e EngineCompositions) inSandbox(ctx context.Context, c *Case, tagged build
return a, fmt.Errorf("the engine of %s answered a composition input that cannot be encoded again: %w", node, err)
}
req := composer.Request{V: composer.Version, Input: input, Depth: composer.DepthCompose, Machines: []string{node},
Cases: []composer.Case{{Name: "change", Manifests: manifests, Verdicts: verdicts}}}
Cases: []composer.Case{{Name: "change", Manifests: manifests, Verdicts: verdicts}}, Seats: c.Seats}
raw, err := json.Marshal(req)
if err != nil {
return a, err
}
job := buildstep.ComposeJob{Fetch: buildstep.Fetch{Repository: nodeEngineRepo, Commit: head, Packages: packagesFor(e.Published, c, tagged, nodeEngineRepo)}, Dir: meshcatalogclient.RootPath,
if e.Requests != nil {
key, err := nodeengineclient.CompositionKey(c.Group, c.Heads)
if err == nil {
err = e.Requests.Keep(ctx, key, node, raw)
}
if err != nil {
return a, err
}
}
job := buildstep.ComposeJob{Fetch: buildstep.Fetch{Repository: nodeEngineRepo, Commit: head, Packages: packagesFor(e.Published, c, c.Tagged, nodeEngineRepo)}, Dir: meshcatalogclient.RootPath,
Package: composer.Package, Request: raw, Reader: "the node-engine head of " + c.Group}
verdict, err := e.Sandbox.Compose(ctx, job)
if err != nil {
@@ -4,6 +4,7 @@ import (
"context"
"encoding/json"
"errors"
"mesh-delivery-checker/rules"
"strings"
"sync"
"testing"
@@ -70,6 +71,36 @@ func (f *fakeSandbox) Compose(_ context.Context, job buildstep.ComposeJob) (comp
return f.v(req.Machines[0]), nil
}
// keptRequests is the state of composition requests in memory; a put fails with err when it is set.
type keptRequests struct {
mu sync.Mutex
vals map[string][]byte
err error
}
func (k *keptRequests) Put(_ context.Context, key string, value []byte) (uint64, error) {
k.mu.Lock()
defer k.mu.Unlock()
if k.err != nil {
return 0, k.err
}
if k.vals == nil {
k.vals = map[string][]byte{}
}
k.vals[key] = value
return 1, nil
}
func (k *keptRequests) read() StateRead {
return StateRead{Keys: func() ([]string, error) {
var out []string
for n := range k.vals {
out = append(out, n)
}
return out, nil
}, Get: func(key string) (json.RawMessage, error) { return k.vals[key], nil }}
}
type digests map[string]string
func (d digests) ManifestDigests() map[string]string { return d }
@@ -87,15 +118,17 @@ func TestTheCompositionIsMadeFromTheEnginesAnswers(t *testing.T) {
}
cs := caseOf(t, pullOf(repoA, 7, head1))
key, _ := nodeengineclient.CompositionKey(cs.Group, cs.HeadsHash)
var last rules.Composed // the engines' answers of the last compose, for the check's verdict to take in
compose := func(e *fakeEngines, nodes ...string) meshwants.CompositionsByGroupHeadsValue {
t.Helper()
ec := EngineCompositions{Asker: e, Placed: func() ([]string, error) { return nodes, nil }, Bound: 50 * time.Millisecond,
Composer: func() string { return strings.Repeat("7", 40) }}
v, err := ec.Compose(context.Background(), cs, nil)
raw, err := ec.Compose(context.Background(), cs)
if err != nil {
t.Fatal(err)
}
return v
last = raw
return rules.JudgeEngines(raw)
}
t.Run("one node refusing fails the composition, naming the node and the module", func(t *testing.T) {
@@ -113,7 +146,7 @@ func TestTheCompositionIsMadeFromTheEnginesAnswers(t *testing.T) {
t.Errorf("asked %+v, want the check's candidate %s and no input", e.asked, key)
}
var ver meshdeliverychecker.VerdictsByGroupHeadsValue
withComposition(&ver, v)
rules.JudgeComposition(&ver, rules.Composition{Composed: &last})
if ver.State != meshdeliverycheckerclient.VerdictFail || ver.Failed.Machine != "laptop" {
t.Errorf("the check's verdict = %+v", ver)
}
@@ -128,7 +161,8 @@ func TestTheCompositionIsMadeFromTheEnginesAnswers(t *testing.T) {
if took := time.Since(began); took > 2*time.Second {
t.Errorf("the nodes were not asked in parallel within the bound: %s", took)
}
if v.State != meshdeliverycheckerclient.CompositionPass || v.Sdk != strings.Repeat("7", 40) || len(v.Machines) != 1 {
// No sandbox: the running engines composed, named so, though the node-engine's trunk head is known.
if v.State != meshdeliverycheckerclient.CompositionPass || v.Sdk != "node-engine" || len(v.Machines) != 1 {
t.Fatalf("verdict = %+v", v)
}
for _, node := range []string{"ace", "g14", "laptop"} {
@@ -138,7 +172,7 @@ func TestTheCompositionIsMadeFromTheEnginesAnswers(t *testing.T) {
}
var ver meshdeliverychecker.VerdictsByGroupHeadsValue
ver.State = meshdeliverycheckerclient.VerdictPass
withComposition(&ver, v)
rules.JudgeComposition(&ver, rules.Composition{Composed: &last})
if ver.State != meshdeliverycheckerclient.VerdictPass || !strings.Contains(composition(ver.Composition, 0, false), "laptop: "+nec.NoAnswer) {
t.Errorf("the line = %q", composition(ver.Composition, 0, false))
}
@@ -150,8 +184,11 @@ func TestTheCompositionIsMadeFromTheEnginesAnswers(t *testing.T) {
t.Fatal(err)
}
host := CaseOf(c, map[string]string{repoA: trunkA, nodeEngineRepo: trunkA})
host.Seats = []json.RawMessage{json.RawMessage(`{"seat":"node-x","scope":"node"}`)} // the heads' list (ADR 0636 §4)
withInput := composed("anchor")
withInput.Input = map[string]any{"nodes": []any{map[string]any{"name": "anchor"}}}
// Issue 1966 (the group try of 2026-11-06 10:07): the composer at the head composes against the check's seat list.
host.Seats = []json.RawMessage{json.RawMessage(`{"format":1,"seat":"node-uplink"}`)}
e := &fakeEngines{answers: map[string]ne.ComposeCandidateAnswer{"anchor": withInput}}
sb := &fakeSandbox{v: func(node string) composer.Verdict {
return composer.Verdict{V: composer.Version, Cases: []composer.CaseVerdict{{Name: "change",
@@ -159,7 +196,8 @@ func TestTheCompositionIsMadeFromTheEnginesAnswers(t *testing.T) {
}}
ec := EngineCompositions{Asker: e, Placed: func() ([]string, error) { return []string{"anchor"}, nil }, Bound: time.Second,
Sandbox: sb, Manifests: digests{"web": "sha256:" + strings.Repeat("c", 64)}, Published: published{}}
v, err := ec.Compose(context.Background(), host, nil)
raw, err := ec.Compose(context.Background(), host)
v := rules.JudgeEngines(raw)
if err != nil {
t.Fatal(err)
}
@@ -170,9 +208,28 @@ func TestTheCompositionIsMadeFromTheEnginesAnswers(t *testing.T) {
var req composer.Request
_ = json.Unmarshal(job.Request, &req)
if job.Fetch.Repository != nodeEngineRepo || job.Fetch.Commit != head2 || job.Package != composer.Package ||
string(req.Input) != `{"nodes":[{"name":"anchor"}]}` || req.Cases[0].Manifests["web"].Digest == "" {
string(req.Input) != `{"nodes":[{"name":"anchor"}]}` || req.Cases[0].Manifests["web"].Digest == "" ||
len(req.Seats) != 1 || string(req.Seats[0]) != string(host.Seats[0]) {
t.Errorf("job = %+v, request %+v", job, req)
}
// Issue 1921 (notes 66-67): the request sent to the composer is kept per check and node, exactly as sent (read back
// whole through the tool's reader); a request that cannot be kept fails the node's composition, naming why.
store := &keptRequests{}
ec.Requests = RequestKeeper{Store: store}
if _, err := ec.Compose(context.Background(), host); err != nil {
t.Fatal(err)
}
keyOfHost, _ := nodeengineclient.CompositionKey(host.Group, host.Heads)
got, err := ReadRequests(store.read(), keyOfHost, "anchor")
if b, _ := json.Marshal(got); err != nil || !strings.Contains(string(b), `"request":`+string(sb.jobs[len(sb.jobs)-1].Request)) {
t.Errorf("kept %s, %v: want the request of the last job under %s/anchor", b, err, keyOfHost)
}
store.err = errors.New("the state is full")
raw, _ = ec.Compose(context.Background(), host)
if v := rules.JudgeEngines(raw); v.State != meshdeliverycheckerclient.CompositionFail || !strings.Contains(v.Words, "could not be kept") || !strings.Contains(v.Words, "the state is full") {
t.Errorf("a request that cannot be kept gave %+v", v)
}
ec.Requests = nil
// Issue 1921: the composer is built at the group's heads, also a member that publishes no package yet.
if pkgs := job.Fetch.Packages; len(pkgs) != 1 || pkgs[trunk.ForgeModulePrefix+"mesh-delivery"] != head1 {
t.Errorf("the composer is fetched with the packages %v, want the group's mesh-delivery member at its head %s alone", pkgs, head1)
@@ -180,7 +237,8 @@ func TestTheCompositionIsMadeFromTheEnginesAnswers(t *testing.T) {
// ADR 0607 §3, §5, design 51 *Tagged versions*: once the check heard the group's tagged version of a member's Go
// package, the composer is built against that, not the member's head, as the check's other Go builds are.
const tagged = "v0.1.15-feat-x.1"
if _, err := ec.Compose(context.Background(), host, buildstep.PackageVersions{trunk.ForgeModulePrefix + "mesh-delivery": tagged}); err != nil {
host.Tagged = buildstep.PackageVersions{trunk.ForgeModulePrefix + "mesh-delivery": tagged}
if _, err := ec.Compose(context.Background(), host); err != nil {
t.Fatal(err)
}
if pkgs := sb.jobs[len(sb.jobs)-1].Fetch.Packages; len(pkgs) != 1 || pkgs[trunk.ForgeModulePrefix+"mesh-delivery"] != tagged {
@@ -202,7 +260,8 @@ func TestTheCompositionIsMadeFromTheEnginesAnswers(t *testing.T) {
sb.v = func(node string) composer.Verdict {
return composer.Verdict{V: composer.Version, Cases: []composer.CaseVerdict{{Name: "change", Machines: map[string]composer.Machine{node: row.m}}}}
}
v, err = ec.Compose(context.Background(), host, nil)
raw, err = ec.Compose(context.Background(), host)
v = rules.JudgeEngines(raw)
named := len(v.Warnings) == 1 && strings.Contains(v.Warnings[0], "anchor: "+nec.NoAnswer+" (its node-engine sends no running state for web)")
if err != nil || (v.State == meshdeliverycheckerclient.CompositionPass) != row.pass || named != row.pass {
t.Errorf("%+v: verdict %+v, %v", row.m, v, err)
@@ -211,17 +270,76 @@ func TestTheCompositionIsMadeFromTheEnginesAnswers(t *testing.T) {
// Review finding 3 (ADR 0593 §9 lets only a node that does not answer pass): a composer at the head that cannot
// run fails the composition, naming the head, never passing as a silent node.
sb.err = errors.New("the build of mesh-compose failed")
v, err = ec.Compose(context.Background(), host, nil)
raw, err = ec.Compose(context.Background(), host)
v = rules.JudgeEngines(raw)
if err != nil || v.State != meshdeliverycheckerclient.CompositionFail || !strings.Contains(v.Words, head2) ||
!strings.Contains(v.Words, "the build of mesh-compose failed") || len(v.Warnings) != 0 {
t.Errorf("a composer that cannot run: %+v, %v", v, err)
}
})
// ADR 0578 §2, §3, ADR 0656 §5 (milestone 47/G D12, D13): with no node-engine member, every node is composed by the
// composer built at the node-engine's trunk over the input its engine answers, and a manifest the change brings carries
// the check's own judge's verdict, not the running indexer's: one the running indexer refused and the judge accepted
// composes, and the verdict names the trunk's composer.
t.Run("a change's manifest the check's judge accepted is composed by the composer at the node-engine's trunk", func(t *testing.T) {
p := pullOf(repoA, 7, head1)
p.Modules = []string{"mesh-build-go"}
p.Fps = map[string]meshcatalog.CandidatesByGroupValuePullsItemFpsValue{"mesh-build-go": {Fp2: "src2:" + strings.Repeat("f", 64), Manifest: strings.Repeat("d", 64)}}
brings := caseOf(t, p)
brings.Judged = map[string]json.RawMessage{"mesh-build-go": json.RawMessage(`{"module":"mesh-build-go","state":"accepted"}`)}
trunkHead := strings.Repeat("7", 40)
withInput := composed("anchor")
withInput.Input = map[string]any{"nodes": []any{map[string]any{"name": "anchor"}}}
e := &fakeEngines{answers: map[string]ne.ComposeCandidateAnswer{"anchor": withInput}}
sb := &fakeSandbox{v: func(node string) composer.Verdict {
return composer.Verdict{V: composer.Version, Cases: []composer.CaseVerdict{{Name: "change",
Machines: map[string]composer.Machine{node: {Composes: true}}}}}
}}
indexer := map[string]meshcatalog.VerdictsByDigestValue{"mesh-build-go": {Module: "mesh-build-go", State: "refused", Problems: []string{"serves no verb input"}}, "mail": {Module: "mail", State: "accepted"}}
ec := EngineCompositions{Asker: e, Placed: func() ([]string, error) { return []string{"anchor"}, nil }, Bound: time.Second,
Sandbox: sb, Composer: func() string { return trunkHead }, Published: published{},
Verdicts: func() map[string]meshcatalog.VerdictsByDigestValue { return indexer }}
raw, err := ec.Compose(context.Background(), brings)
if err != nil {
t.Fatal(err)
}
if v := rules.JudgeEngines(raw); v.State != meshdeliverycheckerclient.CompositionPass || v.Sdk != trunkHead {
t.Fatalf("verdict = %+v, want a pass composed at the node-engine's trunk %s", v, trunkHead)
}
if len(sb.jobs) != 1 || sb.jobs[0].Fetch.Commit != trunkHead || sb.jobs[0].Fetch.Repository != nodeEngineRepo {
t.Fatalf("jobs %+v: want one job at the node-engine's trunk", sb.jobs)
}
var req composer.Request
_ = json.Unmarshal(sb.jobs[0].Request, &req)
if got := string(req.Cases[0].Verdicts["mesh-build-go"]); got != string(brings.Judged["mesh-build-go"]) || len(req.Cases[0].Verdicts) != 2 {
t.Errorf("the request's verdicts %s: want the check's judge's for mesh-build-go and the indexer's for mail", req.Cases[0].Verdicts)
}
// A manifest the judge gave no verdict on carries none, which the composer reads as accepted (ADR 0578 §3).
brings.Judged = nil
if _, err := ec.Compose(context.Background(), brings); err != nil {
t.Fatal(err)
}
var none composer.Request
_ = json.Unmarshal(sb.jobs[len(sb.jobs)-1].Request, &none)
if _, kept := none.Cases[0].Verdicts["mesh-build-go"]; kept {
t.Errorf("the running indexer's verdict on the change's manifest was kept: %s", none.Cases[0].Verdicts)
}
// A refusal the trunk's composer makes stays the node-engine's own.
sb.v = func(node string) composer.Verdict {
return composer.Verdict{V: composer.Version, Cases: []composer.CaseVerdict{{Name: "change",
Machines: map[string]composer.Machine{node: {Composes: false, Reason: "a required value is unset"}}}}}
}
raw, _ = ec.Compose(context.Background(), brings)
if v := rules.JudgeEngines(raw); v.State != meshdeliverycheckerclient.CompositionFail || !strings.Contains(v.Words, "a required value is unset") {
t.Errorf("a refusal the trunk's composer makes: %+v", v)
}
})
// Review finding 4: assignments that cannot be read is no mesh with no node: the composition is not made, said.
t.Run("an unreadable assignments makes no composition", func(t *testing.T) {
ec := EngineCompositions{Asker: &fakeEngines{}, Placed: func() ([]string, error) { return nil, errors.New("no such bucket") }}
if _, err := ec.Compose(context.Background(), cs, nil); err == nil || !strings.Contains(err.Error(), "no such bucket") {
if _, err := ec.Compose(context.Background(), cs); err == nil || !strings.Contains(err.Error(), "no such bucket") {
t.Errorf("an unreadable assignments: %v", err)
}
b := newBench(t)
@@ -3,12 +3,12 @@ package check
import (
"encoding/json"
"fmt"
"mesh-delivery-checker/rules"
"strings"
"testing"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliverycheckerclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
)
@@ -24,9 +24,17 @@ func failingResult(t *testing.T, tests int, failed ...Failure) json.RawMessage {
func judgeLevel(t *testing.T, tests ...json.RawMessage) (*meshdeliverychecker.VerdictsByGroupHeadsValue, *state) {
t.Helper()
st := &state{j: &Examiner{}, c: &Case{}, levels: map[string]*acc{buildstep.LevelUnit: {}, buildstep.LevelIntegration: {}}, nodes: map[string]bool{}}
v, _ := st.level(t.Context(), &target{pull: Pull{Repository: "novox/x"}}, buildstep.LevelAnswer{Level: buildstep.LevelIntegration, Tests: tests})
return settled(joinFailures(v)), st
st := &state{j: &Examiner{}, c: &Case{}, levels: rules.NewTally()}
st.level(t.Context(), &target{pull: Pull{Repository: "novox/x"}}, buildstep.LevelAnswer{Level: buildstep.LevelIntegration, Tests: tests})
// 1921 (B0, D1a): the running checker only gathers the level as a fact; the rules judge it, and count what it kept.
judged := rules.Judge(rules.Facts{V: rules.FactsVersion, Steps: st.facts})
if judged.Levels[buildstep.LevelIntegration] != (meshdeliverychecker.VerdictsByGroupHeadsValueLevelsValue{Run: int64(st.levels[buildstep.LevelIntegration].Run)}) {
t.Errorf("the rules count %+v, the running checker %+v", judged.Levels, *st.levels[buildstep.LevelIntegration])
}
if judged.State == meshdeliverycheckerclient.VerdictPass {
return nil, st
}
return &judged, st
}
// Issue 1921: two failing tests in two packages are two entries of the verdict's Failures and two lines of a try's
@@ -51,7 +59,7 @@ func TestTwoFailingTestsInTwoPackagesAreTwoEntries(t *testing.T) {
if len(v.Failures) != 2 || v.Failed.Test != "integration: x/a.TestA" || !strings.Contains(v.Failed.Words, "a_test.go:3: logged") {
t.Errorf("%s: verdict %+v", name, v)
}
if run := st.levels[buildstep.LevelIntegration].run; run != 10 {
if run := st.levels[buildstep.LevelIntegration].Run; run != 10 {
t.Errorf("%s: %d tests counted, want all 10", name, run)
}
if _, err := meshdeliverycheckerclient.EncodeVerdict(meshdeliverychecker.VerdictsByGroupHeadsValue{Group: "g", Heads: "0123456789ab", State: v.State, Failed: v.Failed, Failures: v.Failures}); err != nil {
@@ -74,7 +82,7 @@ func TestTwoFailingTestsInTwoPackagesAreTwoEntries(t *testing.T) {
}
// Issue 1921: past meshdeliverycheckerclient.MaxFailures the list names the first fifty and how many more, the status line says how many
// failed and names the first, cut so "and N more" stays whole, and each entry's line is cut to MaxFailureLine.
// failed and names the first, cut so "and N more" stays whole, and each entry's line is cut to rules.MaxFailureLine.
func TestFailuresAreCappedWithHowManyMore(t *testing.T) {
var fs []Failure
for i := range 60 {
@@ -84,10 +92,10 @@ func TestFailuresAreCappedWithHowManyMore(t *testing.T) {
if len(v.Failures) != meshdeliverycheckerclient.MaxFailures+1 || v.Failures[meshdeliverycheckerclient.MaxFailures].Test != "and 10 more" {
t.Fatalf("%d entries, last %+v", len(v.Failures), v.Failures[len(v.Failures)-1])
}
if len(v.Failures[0].Words) > MaxFailureLine {
if len(v.Failures[0].Words) > rules.MaxFailureLine {
t.Errorf("an entry's line is %d bytes", len(v.Failures[0].Words))
}
if n := failedCount(*v); n != 60 {
if n := rules.FailedCount(*v); n != 60 {
t.Errorf("failedCount = %d, want 60", n)
}
cs := &Case{Pulls: []Pull{{Repository: "novox/x", Head: "0123456789abcdef", Trunk: "fedcba9876543210"}}}
@@ -96,8 +104,9 @@ func TestFailuresAreCappedWithHowManyMore(t *testing.T) {
t.Errorf("line %q", line)
}
// A composition failing after them is counted in the last entry (the operator, 2026-11-02).
withComposition(v, meshwants.CompositionsByGroupHeadsValue{State: meshdeliverycheckerclient.CompositionFail, Machine: "g14", Words: "route-proxy is defined twice"})
if n := failedCount(*v); n != 61 || v.Failed.Test != "integration: x/p0.TestT" {
refused := refusedOn("x", "g14", "route-proxy is defined twice")
rules.JudgeComposition(v, rules.Composition{Composed: &refused})
if n := rules.FailedCount(*v); n != 61 || v.Failed.Test != "integration: x/p0.TestT" {
t.Errorf("after the composition failedCount = %d, failed %+v", n, v.Failed)
}
@@ -1,12 +1,15 @@
package check
import (
"cmp"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io/fs"
"mesh-delivery-checker/rules"
"os"
"path/filepath"
"reflect"
@@ -15,6 +18,7 @@ import (
"strings"
"sync"
"testing"
"testing/fstest"
"time"
"unicode/utf8"
@@ -25,7 +29,6 @@ import (
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdelivery"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
@@ -132,6 +135,69 @@ type bench struct {
// declines is how many asks naming needs a node lacking them declines before shanks, which has every need, takes
// one (issue 890); an ask naming none runs on shanks, which then has none.
declines int
// inputs are the checking program's kept inputs; heads the heads laid for it (none when nil); run, when set, answers
// a run ask in its place (a program that does not build or answer).
inputs *memKV
heads fs.FS
run func(a buildstep.StepAsk) (buildstep.Step, bool)
mu sync.Mutex
runs []buildstep.StepAsk // the run asks put
}
// benchAsks puts the bench's asks: a run ask is answered at once by the checking program's work (answerRun), written by
// write and kept apart (bench.runs); every other ask goes to inner (the fake bus by default), answered as it answers.
type benchAsks struct {
b *bench
inner buildrun.AskPublisher
write func(buildstep.Step)
}
func (b *bench) asker(inner buildrun.AskPublisher, write func(buildstep.Step)) benchAsks {
return benchAsks{b: b, inner: inner, write: write}
}
func (p benchAsks) PublishAsk(ctx context.Context, subject, id string, data []byte) error {
inner, write := p.inner, p.write
if inner == nil {
inner, write = p.b.asks, p.b.asks.Write
}
a, err := buildstep.DecodeStepAsk(data)
if err != nil || a.Run == nil {
return inner.PublishAsk(ctx, subject, id, data)
}
p.b.mu.Lock()
p.b.runs = append(p.b.runs, a)
p.b.mu.Unlock()
if step := p.b.answerRun(a); step.ID != "" {
write(step)
}
return nil
}
// trunkD is mesh-delivery's trunk the bench's checking program is built at.
var trunkD = strings.Repeat("d", 40)
// answerRun answers a run ask as the build module would: the checking program's work (rules.Program) over the input the
// ask names, read from the kept inputs and checked against its digest.
func (b *bench) answerRun(a buildstep.StepAsk) buildstep.Step {
began := time.Date(2026, 10, 21, 19, 0, 0, 0, time.UTC)
step := buildstep.Step{ID: a.ID, Tool: a.Kind + " ask", Node: "shanks", State: buildstep.StepPassed, Started: began, Ended: began.Add(time.Second)}
if b.run != nil {
if s, ok := b.run(a); ok {
return s
}
}
answer := buildstep.RunAnswer{Built: true, Key: "program-0123456789abcdef"}
out, err := rules.Program(a.Run.Args, func() ([]byte, error) {
body, _, _, _ := b.inputs.Get(context.Background(), a.Run.Input.Digest)
return body, a.Run.Input.Verify(body)
}, b.heads)
if err != nil {
answer.Exit, answer.Said = 2, err.Error()
}
answer.Output = out
step.Result, _ = json.Marshal(answer)
return step
}
// refsOf reads the siblings of a fetch's arguments, as they arrive in an ask (JSON) or as the checker gave them.
@@ -190,7 +256,7 @@ func (b *bench) answer(a buildstep.StepAsk) buildstep.Step {
// newBench is an examiner over fake build modules that merge cleanly, digest the captured fixture's packages and pass
// the captured slow fixture's result at both levels, on shanks.
func newBench(t testing.TB) *bench {
b := &bench{steps: &fakeSteps{}, kept: newMemKV(), prev: newMemKV()}
b := &bench{steps: &fakeSteps{}, kept: newMemKV(), prev: newMemKV(), inputs: newMemKV(), heads: fstest.MapFS{}}
b.fetch = captured[Fetched](t, "go-fetch-clean")
b.fetch.Modules, b.fetch.Changed, b.fetch.Trees = []string{"."}, []string{"README.md"}, nil
one := captured[Digested](t, "go-digest")
@@ -214,8 +280,8 @@ func newBench(t testing.TB) *bench {
}
now := time.Date(2026, 10, 21, 19, 0, 0, 0, time.UTC)
b.asks = &buildrun.FakeAsks{Answer: b.answer}
b.ex = &Examiner{Asks: buildrun.Asker{Publisher: b.asks, Waiter: b.asks, Bound: 5 * time.Second}, Toolchains: goEntries("shanks"), Kept: Kept{Store: b.kept}, Previews: b.prev,
Modules: langs{}, Trunks: &Trunks{}, Now: func() time.Time { now = now.Add(50 * time.Millisecond); return now }}
b.ex = &Examiner{Asks: buildrun.Asker{Publisher: b.asker(nil, nil), Waiter: b.asks, Bound: 5 * time.Second}, Toolchains: goEntries("shanks"), Kept: Kept{Store: b.kept}, Previews: b.prev,
Modules: langs{}, Trunks: &Trunks{heads: map[string]string{DeliveryRepository: trunkD}}, Inputs: memInputs{b.inputs}, Now: func() time.Time { now = now.Add(50 * time.Millisecond); return now }}
return b
}
@@ -306,6 +372,81 @@ func TestACleanMergeIsFetchedAtItsTrunkAndBothLevelsPass(t *testing.T) { // 1679
t.Errorf("the check ask carries the names %v", asks)
}
}
// Issue 1921 (B0, D4 to D6): the check's verdict is the checking program's, built at the group's mesh-delivery head,
// else at mesh-delivery's trunk, over the facts kept as its input; a program that is never begun, does not build or
// does not answer fails the check naming it, and a rule the change's program alters judges the check.
verdictOf := func(state, words string) json.RawMessage {
out, _ := json.Marshal(rules.Answer{V: rules.AnswerVersion, Verdict: meshdeliverychecker.VerdictsByGroupHeadsValue{State: state, Failed: &FailedFirst{Words: words}}})
return out
}
for _, row := range []struct {
name string
repo string // the member's repository: mesh-delivery unless named
run func(a buildstep.StepAsk, s buildstep.Step) (buildstep.Step, bool)
at string // the commit the program is built at; empty when it fails
fails string
runs int // the run asks put when it fails: each put twice when never begun, and none after the seat list's
}{
{name: "at the group's mesh-delivery head", at: head1},
{name: "at mesh-delivery's trunk", repo: "novox/mesh-catalog", at: trunkD},
{name: "never begun", run: func(buildstep.StepAsk, buildstep.Step) (buildstep.Step, bool) { return buildstep.Step{}, true },
fails: "no build node began the job run of the asks", runs: 2},
{name: "does not build", run: func(_ buildstep.StepAsk, s buildstep.Step) (buildstep.Step, bool) {
s.State, s.Reason = buildstep.StepFailed, "the program does not build: undefined: rules.Nothing"
return s, true
}, fails: "by the checking program: its run ask", runs: 1},
{name: "a rule the change alters", at: head1, run: func(a buildstep.StepAsk, s buildstep.Step) (buildstep.Step, bool) {
if a.Run.Args[0] != "-verdict" {
return s, false
}
s.Result, _ = json.Marshal(buildstep.RunAnswer{Key: "program-0123456789abcdef", Output: verdictOf(meshdeliverycheckerclient.VerdictFail, "the change's own rule refuses it")})
return s, true
}, fails: "the change's own rule refuses it"},
} {
b := newBench(t)
b.ex.BeginBound = 20 * time.Millisecond
if row.run != nil {
b.run = func(a buildstep.StepAsk) (buildstep.Step, bool) {
return row.run(a, buildstep.Step{ID: a.ID, Tool: "check ask", Node: "shanks", State: buildstep.StepPassed})
}
}
repo := cmp.Or(row.repo, repoA)
v := b.judge(t, caseOf(t, pullOf(repo, 7, head1))).Verdict
if row.fails != "" {
if v.State != meshdeliverycheckerclient.VerdictFail || !strings.Contains(v.Failed.Words, row.fails) {
t.Errorf("%s: verdict %+v, want a fail saying %q", row.name, v, row.fails)
}
if row.at == "" && (len(b.runs) != row.runs || v.JudgedBy != nil || slices.ContainsFunc(b.runs, func(a buildstep.StepAsk) bool { return a.Run.Args[0] == "-verdict" })) {
t.Errorf("%s: %d run asks, judged by %+v: want %d and none judging", row.name, len(b.runs), v.JudgedBy, row.runs)
}
} else if v.State != meshdeliverycheckerclient.VerdictPass {
t.Errorf("%s: verdict %+v", row.name, v)
}
if row.at == "" {
continue
}
if by := v.JudgedBy; by == nil || by.Program.Commit != row.at || by.Program.Package != CheckingProgramPackage || by.Key == "" ||
len(by.Heads) != 1 || by.Heads[0].Repository != repo || by.Heads[0].Commit != head1 {
t.Errorf("%s: judged by %+v, want the program at %s over the member's head", row.name, v.JudgedBy, row.at)
}
if len(b.runs) != 2 || b.runs[0].Run.Args[0] != "-seats" || b.runs[1].Run.Args[0] != "-verdict" {
t.Fatalf("%s: run asks %+v, want the seat list and then the verdict", row.name, b.runs)
}
in := b.runs[1].Run.Input // the facts kept as their exact bytes under their digest
body, _, found, _ := b.inputs.Get(context.Background(), in.Digest)
if !found || in.Verify(body) != nil || in.Verb != InputVerb ||
!strings.Contains(string(body), `"group":"feat/x"`) {
t.Errorf("%s: the verdict's input %+v is not kept whole: %s", row.name, in, body)
}
// Review round 1 of milestone 47/B0: a level is carried as the rules read it, without its digest, its packages or
// its kept packages' names, so a group's facts stay within the input's bound.
for _, unread := range []string{`"digest":{`, `"packages":[`, `"unmet"`, `"package":"m/`} {
if strings.Contains(string(body), unread) {
t.Errorf("%s: the facts carry %s, which no rule reads: %s", row.name, unread, body)
}
}
}
}
// namesOf is the nodes' names as a test gives them.
@@ -452,8 +593,7 @@ func TestANodePackageIsNeverKeptAndRunsEveryCheck(t *testing.T) { // 1288/AC4, 1
func TestEveryFailureOfEveryMemberLevelAndTheCompositionIsNamed(t *testing.T) {
for _, conflicts := range []bool{false, true} {
b := newBench(t)
comps := &fakeCompositions{v: meshwants.CompositionsByGroupHeadsValue{Group: "feat/x", Heads: "0123456789ab", State: meshdeliverycheckerclient.CompositionFail, Sdk: "x",
Machine: "g14", Words: "route-proxy is defined twice"}, waits: conflicts}
comps := &fakeCompositions{v: refusedOn("x", "g14", "route-proxy is defined twice"), waits: conflicts}
b.ex.Compositions = comps
b.fetch.Changed = []string{"modules/m/internal/b/b_test.go"}
b.digest[buildstep.LevelUnit] = Digested{Level: "unit", Packages: map[string]string{"m/internal/a": digA, "m/internal/b": digB}}
@@ -487,7 +627,7 @@ func TestEveryFailureOfEveryMemberLevelAndTheCompositionIsNamed(t *testing.T) {
got = append(got, firstFailed(meshdeliverychecker.VerdictsByGroupHeadsValue{Failed: &FailedFirst{Test: f.Test, Machine: f.Machine, Words: f.Words}}))
}
tests := []string{"unit: fixture/failing/bad.TestThreeIsFour", "integration: fixture/failing/bad.TestThreeIsFour"}
want := append(append(slices.Clone(tests), tests...), "g14: route-proxy is defined twice") // both members fail their tests
want := append(append(slices.Clone(tests), tests...), "g14: not composed on g14: route-proxy is defined twice") // both members fail their tests
if conflicts {
want = append([]string{"shanks: novox/mesh-sdk#12 does not merge into the trunk: files in conflict: a.go"}, tests...)
}
@@ -557,6 +697,10 @@ func TestAHeadThatDoesNotMergeEndsFailedNamingTheFilesAndRunsNoTest(t *testing.T
if n := len(b.steps.tool("digest")) + len(b.steps.tool("test")); n != 0 {
t.Errorf("%d digest and test steps ran after a conflict", n)
}
// 1921/D1c: the node that ran the step is a fact, so the verdict judged from the facts names its machine.
if m := out.Verdict.Machines; len(m) != 1 || m[0] != "shanks" {
t.Errorf("machines = %v, want the node that ran the step", m)
}
}
func TestALevelWhoseDigestIsNotNamedRunsWholeAndKeepsNothing(t *testing.T) { // 1287/AC4
@@ -927,7 +1071,7 @@ func TestAnAskWhoseStepNeverEndsEndsTheCheckLoudly(t *testing.T) {
b.asks.Answer = func(a buildstep.StepAsk) buildstep.Step {
return buildstep.Step{ID: a.ID, Tool: "check ask", Node: "shanks", State: buildstep.StepRunning, Started: time.Now()}
}
b.ex.Asks = buildrun.Asker{Publisher: b.asks, Waiter: b.asks, Bound: 20 * time.Millisecond}
b.ex.Asks = buildrun.Asker{Publisher: b.asker(nil, nil), Waiter: b.asks, Bound: 20 * time.Millisecond}
out := b.judge(t, caseOf(t, pullOf(repoA, 7, head1)))
if f := out.Verdict.Failed; out.Verdict.State != meshdeliverycheckerclient.VerdictFail || !strings.Contains(f.Words, "on shanks did not end within") {
t.Fatalf("verdict = %+v", out.Verdict)
@@ -967,7 +1111,7 @@ func TestACheckAskWaitingInItsStreamIsNotBounded(t *testing.T) {
b := newBench(t)
answer := b.asks.Answer
b.asks.Answer = nil
b.ex.Asks = buildrun.Asker{Publisher: b.asks, Waiter: b.asks, Bound: 20 * time.Millisecond}
b.ex.Asks = buildrun.Asker{Publisher: b.asker(nil, nil), Waiter: b.asks, Bound: 20 * time.Millisecond}
done := make(chan Outcome, 1)
go func() { done <- b.judge(t, caseOf(t, pullOf(repoA, 7, head1))) }()
for deadline := time.Now().Add(10 * time.Second); len(b.asks.Asks()) == 0; time.Sleep(time.Millisecond) {
@@ -1076,6 +1220,9 @@ func TestTheIndexersJudgeJudgesTheChangesManifests(t *testing.T) {
// (ADR 0636 §4, ADR 0633 §6); "outside" is mesh-contracts, whose definition of another domain's seat is not taken
// (ADR 0636 §5); "misnamed" is the domain's module whose seats/mesh-catalog/seat.json names another seat.
heads string
// headSeats lays heads that add a seat: the checking program's seat list is carried by every judge ask's request (ADR
// 0636 §4, issue 1921 D4).
headSeats bool
}{
{name: "the seat list read is the request's Seats", seats: "read"},
{name: "the members' heads' definitions over the list's, for the trunk's manifest validator too", seats: "read", indexer: "ok", heads: "domain"},
@@ -1099,6 +1246,7 @@ func TestTheIndexersJudgeJudgesTheChangesManifests(t *testing.T) {
{name: "a package series kept", series: "0.1"},
{name: "a package new in the group", series: "0.1", known: []string{"git.novox.be/novox/mesh-sdk/go", "git.novox.be/novox/web/go"}},
{name: "an answered packages state holding no name", emptyState: true, known: []string{}},
{name: "the heads' seat list is carried by the head's judge and the trunk's", headSeats: true, indexer: "ok", ownJudge: true},
{name: "accepted in a group try", try: true},
{name: "refused in a group try", try: true, refused: "web has no id", fails: "the indexer's judge at mesh-delivery's trunk " + strings.Repeat("d", 40) + ` refuses the manifest of web: web has no id`},
{name: "a cycle the group try makes", try: true, cycle: "provisions", fails: "the change's module graph has the cycle mail -> web -> mail"},
@@ -1119,6 +1267,12 @@ func TestTheIndexersJudgeJudgesTheChangesManifests(t *testing.T) {
{name: "refused", refused: "web requires \"amqp\"\nweb has no id", fails: "the indexer's judge at mesh-delivery's trunk " + strings.Repeat("d", 40) + ` refuses the manifest of web: web requires "amqp"; web has no id`},
} {
b := newBench(t)
if row.headSeats {
b.heads = fstest.MapFS{
repoA + "/modules/x-contracts/module.json": {Data: []byte(`{"module": "x-contracts", "build": {"artifacts": [{"name": "go", "kind": "package", "language": "go", "generated-from": "."}]}}`)},
repoA + "/modules/x-contracts/seats/node-x/seat.json": {Data: []byte(`{"format":1,"seat":"node-x","scope":"node","grain":"whole","domain":"Delivery","decision":"ADR 1"}`)},
}
}
b.ex.Manifests = fixedDigests{"mail": strings.Repeat("a", 64), "web": strings.Repeat("b", 64)}
b.ex.Trunks = fixedTrunks{heads: map[string]string{DeliveryRepository: trunk}}
b.ex.Predefined = keptList(`{"font-size": {"type": "number", "description": "the size"}}`)
@@ -1233,13 +1387,14 @@ func TestTheIndexersJudgeJudgesTheChangesManifests(t *testing.T) {
}
dir := "modules/" + contracts
p.Modules = append(p.Modules, contracts)
b.ex.Forge = headFiles{
repoA + " " + dir + "/module.json " + head1: `{"module": "` + contracts + `", "build": {"artifacts": [{"name": "go", "kind": "package", "language": "go", "generated-from": "."}]}}`,
repoA + " " + dir + "/seats/mesh-catalog/seat.json " + head1: `{"format":1,"seat":"` + seat + `","scope":"mesh","grain":"whole","domain":"Delivery","decision":"ADR 1","includes":["lists"],` +
`"verbs":[{"name":"modules","input":{"type":"object"},"answer":{"type":"object"}}]}`,
repoA + " " + dir + "/parts/lists.json " + head1: `{"format":1,"part":"lists","verbs":[{"name":"seat-definitions","input":{"type":"object"},"answer":{"type":"object"}}]}`,
// The checking program reads the heads the build node lays for its run (D6): the running checker reads none.
b.heads = fstest.MapFS{
repoA + "/" + dir + "/module.json": {Data: []byte(`{"module": "` + contracts + `", "build": {"artifacts": [{"name": "go", "kind": "package", "language": "go", "generated-from": "."}]}}`)},
repoA + "/" + dir + "/seats/mesh-catalog/seat.json": {Data: []byte(`{"format":1,"seat":"` + seat + `","scope":"mesh","grain":"whole","domain":"Delivery","decision":"ADR 1","includes":["lists"],` +
`"verbs":[{"name":"modules","input":{"type":"object"},"answer":{"type":"object"}}]}`)},
repoA + "/" + dir + "/seats/mesh-catalog/fixtures/a.json": {Data: []byte(`{}`)},
repoA + "/" + dir + "/parts/lists.json": {Data: []byte(`{"format":1,"part":"lists","verbs":[{"name":"seat-definitions","input":{"type":"object"},"answer":{"type":"object"}}]}`)},
}
b.ex.Trees = headTrees{repoA + " " + dir + "/seats " + head1: {"mesh-catalog/seat.json": "1", "mesh-catalog/fixtures/a.json": "2"}}
}
p.Fps = map[string]meshcatalog.CandidatesByGroupValuePullsItemFpsValue{"web": {Fp2: "src2:" + strings.Repeat("f", 64), Manifest: web}}
// Issue 1921: the judge is built at the group's heads too, also a member that publishes no package yet.
@@ -1304,6 +1459,13 @@ func TestTheIndexersJudgeJudgesTheChangesManifests(t *testing.T) {
judged[0].Package != JudgePackage || judged[0].Dir != JudgeDir {
t.Fatalf("%s: judged %+v (verdict %+v)", row.name, judged, v)
}
for _, j := range judged {
var seated composer.Request
if err := json.Unmarshal(j.Request, &seated); err != nil || (len(seated.Seats) == 1) != row.headSeats ||
(row.headSeats && !strings.Contains(string(seated.Seats[0]), `"node-x"`)) {
t.Errorf("%s: the judge at %v is handed the seats %s, want the heads' list: %v", row.name, j.Fetch.Commit, seated.Seats, err)
}
}
var req composer.Request
if err := json.Unmarshal(judged[0].Request, &req); err != nil || req.Cases[0].Manifests["web"].Digest != web ||
req.Cases[1].Name != JudgeTrunk || req.Cases[1].Manifests["web"].Digest != strings.Repeat("b", 64) ||
@@ -1312,6 +1474,20 @@ func TestTheIndexersJudgeJudgesTheChangesManifests(t *testing.T) {
t.Fatalf("%s: the request %+v: %v", row.name, req, err)
}
if row.heads != "" {
// Review round 2 of G (programs.go's digest side): the checking program's -seats input carries the digest of
// every definition the reader holds, never the list, and its answer is laid over the list (below).
var digests map[string]string
b.inputs.mu.Lock()
for _, body := range b.inputs.vals {
var f rules.Facts
if json.Unmarshal(body, &f) == nil && f.SeatDigests != nil {
digests = f.SeatDigests
}
}
b.inputs.mu.Unlock()
if want := map[string]string{"a": rules.SeatDigest(json.RawMessage(`{"seat":"a"}`)), "b": rules.SeatDigest(json.RawMessage(`{"seat": "b"}`))}; !reflect.DeepEqual(digests, want) {
t.Errorf("%s: the -seats input carries the digests %v, want the reader's list's %v", row.name, digests, want)
}
var got []string
for _, raw := range req.Seats {
var d struct {
@@ -1336,7 +1512,7 @@ func TestTheIndexersJudgeJudgesTheChangesManifests(t *testing.T) {
} else if row.seats == "read" && (len(req.Seats) != 2 || string(req.Seats[0]) != `{"seat":"a"}` || string(req.Seats[1]) != `{"seat":"b"}`) {
t.Errorf("%s: the request's seats %s, want the two definitions the reader holds", row.name, req.Seats)
}
if row.seats == "" && len(req.Seats) != 0 {
if row.seats == "" && !row.headSeats && len(req.Seats) != 0 {
t.Errorf("%s: the request's seats %s, want none from a checker that follows no list", row.name, req.Seats)
}
if !reflect.DeepEqual(req.Packages, row.known) || len(req.Input) != 0 && string(req.Input) != "null" {
@@ -1360,21 +1536,6 @@ func TestTheIndexersJudgeJudgesTheChangesManifests(t *testing.T) {
// fakeSeatList is the seat list reader as a test gives it: two definitions once read, else the reader's words.
type fakeSeatList struct{ read bool }
// headTrees are the forge's listings of a directory at a commit, by "<repository> <directory> <commit>".
type headTrees map[string]map[string]string
func (h headTrees) Files(_ context.Context, repository, commit string, dirs []string) (map[string]map[string]string, error) {
out := map[string]map[string]string{}
for _, d := range dirs {
files, ok := h[repository+" "+d+" "+commit]
if !ok {
return nil, ErrNotFound
}
out[d] = files
}
return out, nil
}
func (f fakeSeatList) Definitions() ([]json.RawMessage, error) {
if !f.read {
return nil, errors.New("the seat list is held from the bootstrap (2 seats) and nothing new is acted on")
@@ -2,6 +2,7 @@ package check
import (
"encoding/json"
"mesh-delivery-checker/rules"
"strings"
"testing"
@@ -20,81 +21,88 @@ func fatalAfterLog() Failure {
}
}
func TestAFailedTestsVerdictKeepsItsWholeFailureMessage(t *testing.T) { // 1901/AC1
v := levelFailure(&target{}, "unit", Tested{Failed: []Failure{fatalAfterLog()}})
for _, want := range []string{"3 subscription(s) derive to a subject no emitter owns:", "a.b.c", "d.e.f", "g.h.i"} {
if !strings.Contains(v.Failed.Words, want) {
t.Errorf("words lack %q: %q", want, v.Failed.Words)
}
}
if len(v.Failed.Words) > MaxWords+200 {
t.Errorf("words are %d bytes, bound %d", len(v.Failed.Words), MaxWords)
}
}
func TestTheStatusLineNamesTheFailureNotTheFirstLoggedLine(t *testing.T) { // 1901/AC2
v := levelFailure(&target{}, "unit", Tested{Failed: []Failure{fatalAfterLog()}})
got := firstFailed(*v)
if !strings.Contains(got, "3 subscription(s) derive to a subject no emitter owns:") || strings.Contains(got, "judged against") || strings.Contains(got, "\n") {
t.Errorf("firstFailed = %q", got)
}
// With no cause told apart (an older build module), the words still start at the first line, as before.
f := fatalAfterLog()
f.Cause = ""
v = levelFailure(&target{}, "unit", Tested{Failed: []Failure{f}})
if !strings.HasPrefix(afterHid(v.Failed.Words), "agreement_catalogue_test.go:130") {
t.Errorf("words = %q", v.Failed.Words)
}
}
func TestWordsLongerThanTheBoundKeepTheirLastLines(t *testing.T) { // 1901/AC1
f := Failure{Output: strings.Repeat("log line of some length\n", 500) + "the cause is here"}
v := levelFailure(&target{}, "unit", Tested{Failed: []Failure{f}})
if !strings.HasSuffix(v.Failed.Words, "the cause is here") || len(v.Failed.Words) > MaxWords+200 {
t.Errorf("%d bytes, ends %q", len(v.Failed.Words), v.Failed.Words[len(v.Failed.Words)-30:])
}
_ = meshdeliverycheckerclient.VerdictFail
}
// An older build module sends no "cause": its answer is still accepted and the verdict's words are its output, as before
// (1901/AC3). A failure that hides its cause (none, or a passing line) is said loudly first in the words and in the status
// line, naming the build module and the version it says it is, and the level still fails; a failure that keeps the
// contract is worded as before (1901/AC8).
func TestAnOlderBuildModulesFailureIsAcceptedWithItsWholeOutput(t *testing.T) { // 1901/AC3, AC8
old := `{"package":"p","test":"TestX","output":"x_test.go:1: logged\nx_test.go:2: the old kept lines"}`
var f Failure
if err := json.Unmarshal([]byte(old), &f); err != nil || f.Cause != "" {
t.Fatalf("1901/AC3: err = %v, cause = %q", err, f.Cause)
}
v := levelFailure(&target{}, "unit", Tested{Failed: []Failure{f}})
if afterHid(v.Failed.Words) != "x_test.go:1: logged\nx_test.go:2: the old kept lines" {
t.Errorf("1901/AC3: words = %q", v.Failed.Words)
}
if got := firstFailed(*v); !strings.Contains(got, "the build module hid the failure") || strings.Contains(got, "\n") {
t.Errorf("1901/AC3, AC8: status = %q", got)
}
by := "mesh-build-node with mesh-sdk v0.1.14-0.20261026184535-e40dd0f56a78"
for _, c := range []struct {
name, module, by, cause, output, loud string
// A failed test's words are its whole failure message, kept within their bound by their last lines (issue 1901).
func TestAFailuresWordsAreItsWholeMessageWithinTheirBound(t *testing.T) {
for _, row := range []struct {
name string
test func(t *testing.T)
}{
{"no cause", "mesh-build-node", by, "", "test/b.test.ts: something", "the build module hid the failure (mesh-build-node at mesh-sdk v0.1.14-0.20261026184535-e40dd0f56a78): it sent no cause"},
{"a passing line as its cause", "mesh-build-node", by, "\u2714 the same merge has the same id (8.006913ms)", "", "the build module hid the failure (mesh-build-node at mesh-sdk v0.1.14-0.20261026184535-e40dd0f56a78): its cause is a passing line"},
{"from before the contract", "mesh-build-node", "", "", "\u2714 the same merge has the same id (8.006913ms)", "the build module hid the failure (mesh-build-node at a version before the failure contract): it sent no cause"},
{"a real cause", "mesh-build-node", by, "one plus one is not three", "not ok 3 - fails", ""},
{"a failed tests verdict keeps its whole failure message", func(t *testing.T) { // 1901/AC1
v := rules.LevelFailure("", "", "unit", Tested{Failed: []Failure{fatalAfterLog()}})
for _, want := range []string{"3 subscription(s) derive to a subject no emitter owns:", "a.b.c", "d.e.f", "g.h.i"} {
if !strings.Contains(v.Failed.Words, want) {
t.Errorf("words lack %q: %q", want, v.Failed.Words)
}
}
if len(v.Failed.Words) > rules.MaxWords+200 {
t.Errorf("words are %d bytes, bound %d", len(v.Failed.Words), rules.MaxWords)
}
}},
{"the status line names the failure not the first logged line", func(t *testing.T) { // 1901/AC2
v := rules.LevelFailure("", "", "unit", Tested{Failed: []Failure{fatalAfterLog()}})
got := firstFailed(*v)
if !strings.Contains(got, "3 subscription(s) derive to a subject no emitter owns:") || strings.Contains(got, "judged against") || strings.Contains(got, "\n") {
t.Errorf("firstFailed = %q", got)
}
// With no cause told apart (an older build module), the words still start at the first line, as before.
f := fatalAfterLog()
f.Cause = ""
v = rules.LevelFailure("", "", "unit", Tested{Failed: []Failure{f}})
if !strings.HasPrefix(afterHid(v.Failed.Words), "agreement_catalogue_test.go:130") {
t.Errorf("words = %q", v.Failed.Words)
}
}},
{"words longer than the bound keep their last lines", func(t *testing.T) { // 1901/AC1
f := Failure{Output: strings.Repeat("log line of some length\n", 500) + "the cause is here"}
v := rules.LevelFailure("", "", "unit", Tested{Failed: []Failure{f}})
if !strings.HasSuffix(v.Failed.Words, "the cause is here") || len(v.Failed.Words) > rules.MaxWords+200 {
t.Errorf("%d bytes, ends %q", len(v.Failed.Words), v.Failed.Words[len(v.Failed.Words)-30:])
}
_ = meshdeliverycheckerclient.VerdictFail
}},
// An older build module sends no "cause": its answer is still accepted and the verdict's words are its output, as before
// (1901/AC3). A failure that hides its cause (none, or a passing line) is said loudly first in the words and in the status
// line, naming the build module and the version it says it is, and the level still fails; a failure that keeps the
// contract is worded as before (1901/AC8).
{"an older build modules failure is accepted with its whole output", func(t *testing.T) { // 1901/AC3, AC8
old := `{"package":"p","test":"TestX","output":"x_test.go:1: logged\nx_test.go:2: the old kept lines"}`
var f Failure
if err := json.Unmarshal([]byte(old), &f); err != nil || f.Cause != "" {
t.Fatalf("1901/AC3: err = %v, cause = %q", err, f.Cause)
}
v := rules.LevelFailure("", "", "unit", Tested{Failed: []Failure{f}})
if afterHid(v.Failed.Words) != "x_test.go:1: logged\nx_test.go:2: the old kept lines" {
t.Errorf("1901/AC3: words = %q", v.Failed.Words)
}
if got := firstFailed(*v); !strings.Contains(got, "the build module hid the failure") || strings.Contains(got, "\n") {
t.Errorf("1901/AC3, AC8: status = %q", got)
}
by := "mesh-build-node with mesh-sdk v0.1.14-0.20261026184535-e40dd0f56a78"
for _, c := range []struct {
name, module, by, cause, output, loud string
}{
{"no cause", "mesh-build-node", by, "", "test/b.test.ts: something", "the build module hid the failure (mesh-build-node at mesh-sdk v0.1.14-0.20261026184535-e40dd0f56a78): it sent no cause"},
{"a passing line as its cause", "mesh-build-node", by, "\u2714 the same merge has the same id (8.006913ms)", "", "the build module hid the failure (mesh-build-node at mesh-sdk v0.1.14-0.20261026184535-e40dd0f56a78): its cause is a passing line"},
{"from before the contract", "mesh-build-node", "", "", "\u2714 the same merge has the same id (8.006913ms)", "the build module hid the failure (mesh-build-node at a version before the failure contract): it sent no cause"},
{"a real cause", "mesh-build-node", by, "one plus one is not three", "not ok 3 - fails", ""},
} {
t.Run(c.name, func(t *testing.T) {
v := rules.LevelFailure("", c.module, "unit", Tested{By: c.by, Failed: []Failure{{Package: "modules/gitea", Test: "fails", Cause: c.cause, Output: c.output}}})
if v.State != meshdeliverycheckerclient.VerdictFail {
t.Errorf("1901/AC8: the verdict is %q, want it failed", v.State)
}
line := firstFailed(*v)
switch {
case c.loud == "" && strings.Contains(v.Failed.Words, "hid the failure"):
t.Errorf("1901/AC8: a failure that keeps the contract is said hidden: %q", v.Failed.Words)
case c.loud != "" && (!strings.HasPrefix(v.Failed.Words, c.loud) || !strings.Contains(line, c.loud) || !strings.Contains(v.Failures[0].Words, "hid the failure")):
t.Errorf("1901/AC8: words %q, status %q, want both to start with %q", v.Failed.Words, line, c.loud)
}
})
}
}},
} {
t.Run(c.name, func(t *testing.T) {
v := levelFailure(&target{module: c.module}, "unit", Tested{By: c.by, Failed: []Failure{{Package: "modules/gitea", Test: "fails", Cause: c.cause, Output: c.output}}})
if v.State != meshdeliverycheckerclient.VerdictFail {
t.Errorf("1901/AC8: the verdict is %q, want it failed", v.State)
}
line := firstFailed(*v)
switch {
case c.loud == "" && strings.Contains(v.Failed.Words, "hid the failure"):
t.Errorf("1901/AC8: a failure that keeps the contract is said hidden: %q", v.Failed.Words)
case c.loud != "" && (!strings.HasPrefix(v.Failed.Words, c.loud) || !strings.Contains(line, c.loud) || !strings.Contains(v.Failures[0].Words, "hid the failure")):
t.Errorf("1901/AC8: words %q, status %q, want both to start with %q", v.Failed.Words, line, c.loud)
}
})
t.Run(row.name, row.test)
}
}
@@ -68,3 +68,14 @@ func (m *memKV) Delete(_ context.Context, key string) error {
delete(m.revs, key)
return nil
}
// memInputs keeps run inputs in a memKV, under their digest.
type memInputs struct{ kv *memKV }
func (m memInputs) Keep(ctx context.Context, digest string, body []byte) error {
if _, _, found, _ := m.kv.Get(ctx, digest); found {
return nil
}
_, err := m.kv.Create(ctx, digest, body)
return err
}
@@ -1,76 +0,0 @@
package check
import (
"context"
"fmt"
"maps"
"slices"
"strings"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-delivery/sdk/composer"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-delivery/sdk/graph"
)
// The check refuses a change whose module graph has a cycle and names it (novox/hq ADR 0603 §2): modules that would each
// wait for the other for good, through their bases, the provisions and bus state they await, or the verb edges the graph
// keeps (ADR 0614 §6). A verb edge on a cycle is left out of the graph, so it is never refused (§3), and a package or a
// build module's edge is a trigger, never a wait, so a cycle through one is not refused either (§6). The graph is worked out from the
// catalogue as the change leaves it (every trunk manifest, each one a pull request touches replaced by its candidate's),
// with the contract's ManifestGraph, the way the indexer works out its graph's edges. A cycle the trunk already has among
// the same modules is the trunk's: said in the verdict's notes, not the change's failure.
// ManifestReader reads a kept manifest by its digest (the indexer's manifests state).
type ManifestReader interface {
Manifest(ctx context.Context, digest string) (catalogue.Manifest, error)
}
// refuseCycle fails the check for a cycle of the change's module graph, naming it; nil when the case touches no manifest,
// when nothing reads manifests, or when there is none. A manifest that cannot be read fails the check, said, never passes it.
func (st *state) refuseCycle(ctx context.Context) *meshdeliverychecker.VerdictsByGroupHeadsValue {
if st.j.Manifests == nil || st.j.Read == nil || st.narrowTry() {
return nil
}
trunk := st.j.Manifests.ManifestDigests()
req, touched := judgeRequest(trunk, st.c.Pulls)
if len(touched) == 0 {
return nil
}
read := func(digests map[string]composer.Manifest) (map[string]catalogue.Manifest, error) {
out := map[string]catalogue.Manifest{}
for _, module := range slices.Sorted(maps.Keys(digests)) {
m, err := st.j.Read.Manifest(ctx, digests[module].Digest)
if err != nil {
return nil, fmt.Errorf("the manifest %s of %s: %w", digests[module].Digest, module, err)
}
out[module] = m
}
return out, nil
}
change, err := read(req.Cases[0].Manifests)
if err != nil {
return fail(goModule, "the change's module graph cannot be worked out, so no cycle can be ruled out: "+err.Error())
}
cycle := graph.OfManifests(change).Cycle()
if cycle == nil {
st.c.note("the change's module graph has no cycle")
return nil
}
words := strings.Join(cycle, " -> ")
if before, err := read(req.Cases[1].Manifests); err == nil {
among := graph.Graph{}
for name, n := range graph.OfManifests(before) {
if slices.Contains(cycle, name) {
among[name] = n
}
}
if among.Cycle() != nil {
st.c.note("the module graph has the cycle " + words + ", which the trunk already has among the same modules (ADR 0603 §2)")
return nil
}
}
return fail(goModule, fmt.Sprintf("the change's module graph has the cycle %s: these modules would each wait for the other for "+
"good, through their bases, the provisions and bus state they await, or the verbs they invoke (ADR 0603 §2, ADR 0614 §6)", words))
}
@@ -0,0 +1,116 @@
package check
import (
"cmp"
"context"
"fmt"
"maps"
"slices"
"strings"
"git.novox.be/novox/mesh-delivery/sdk/composer"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"mesh-delivery-checker/rules"
"git.novox.be/novox/mesh-delivery/sdk/graph"
)
// The module graph of a change (novox/hq ADR 0603 §2, ADR 0614 §6): the running checker gathers its ordering edges, the
// checking program judges whether it has a cycle (rules.JudgeGraph, ADR 0656 §5). The graph is worked out from the
// catalogue as the change leaves it (every trunk manifest, each one a pull request touches replaced by its candidate's)
// and as the trunk has it, with the contract's ManifestGraph, the way the indexer works out its graph's edges; the facts
// carry only each edge's two modules and its kind.
// ManifestReader reads a kept manifest by its digest (the indexer's manifests state).
type ManifestReader interface {
Manifest(ctx context.Context, digest string) (catalogue.Manifest, error)
}
// gatherGraph records the change's and the trunk's module graph edges as a fact; none when the case touches no manifest
// or nothing reads manifests. A change's manifest that cannot be read fails the check, said, never passes it; a trunk's
// that cannot be read leaves the trunk's graph unread, so no cycle is the trunk's.
func (st *state) gatherGraph(ctx context.Context) {
if st.j.Manifests == nil || st.j.Read == nil || st.narrowTry() {
return
}
req, touched := judgeRequest(st.j.Manifests.ManifestDigests(), st.c.Pulls)
if len(touched) == 0 {
return
}
read := func(digests map[string]composer.Manifest) (map[string]catalogue.Manifest, error) {
out := map[string]catalogue.Manifest{}
for _, module := range slices.Sorted(maps.Keys(digests)) {
m, err := st.j.Read.Manifest(ctx, digests[module].Digest)
if err != nil {
return nil, fmt.Errorf("the manifest %s of %s: %w", digests[module].Digest, module, err)
}
out[module] = m
}
return out, nil
}
change, err := read(req.Cases[0].Manifests)
if err != nil {
st.fail(goModule, "the change's module graph cannot be worked out, so no cycle can be ruled out: "+err.Error())
return
}
g := rules.Graph{}
changed := orderingEdges(graph.OfManifests(change))
trunk := map[[2]string]string{}
if before, err := read(req.Cases[1].Manifests); err == nil {
g.Trunk = true
trunk = orderingEdges(graph.OfManifests(before))
}
for _, pair := range slices.SortedFunc(maps.Keys(changed), cmpPair) {
e := rules.Edge{From: pair[0], To: pair[1], Kind: changed[pair]}
if _, both := trunk[pair]; !both && g.Trunk {
e.Only = rules.EdgeChange
}
g.Edges = append(g.Edges, e)
}
for _, pair := range slices.SortedFunc(maps.Keys(trunk), cmpPair) {
if _, both := changed[pair]; !both {
g.Edges = append(g.Edges, rules.Edge{From: pair[0], To: pair[1], Kind: trunk[pair], Only: rules.EdgeTrunk})
}
}
st.facts = append(st.facts, rules.Step{Graph: &g})
}
func cmpPair(a, b [2]string) int {
return cmp.Or(strings.Compare(a[0], b[0]), strings.Compare(a[1], b[1]))
}
// orderingEdges are the graph's ordering edges (graph.Graph.Before) between its modules, each pair once with the kind it
// is first met by: a base, a provision or bus state awaited of another provider, or a verb the graph keeps served by
// another module (a base on itself is a cycle). A verb edge the graph leaves out (ADR 0614 §3), a toolchain and a package edge are none (§6).
func orderingEdges(g graph.Graph) map[[2]string]string {
out := map[[2]string]string{}
add := func(from, to, kind string) {
if _, ours := g[to]; !ours || (from == to && kind != rules.EdgeBase) {
return
}
if _, met := out[[2]string{from, to}]; !met {
out[[2]string{from, to}] = kind
}
}
for _, name := range slices.Sorted(maps.Keys(g)) {
n := g[name]
for _, b := range n.Bases {
add(name, b, rules.EdgeBase)
}
for _, a := range n.Awaits {
for _, p := range g.Providers(a) {
add(name, p, rules.EdgeAwaits)
}
}
for _, v := range n.Invokes {
if slices.Contains(n.LeftOut, v) {
continue
}
for _, s := range g.Servers(v) {
add(name, s, rules.EdgeInvokes)
}
}
}
return out
}
@@ -1,151 +0,0 @@
package check
// The seat definitions of the members' heads (novox/hq ADR 0636 §4, ADR 0633 §6 as its insight of 2026-11-05 (2) says): the
// manifest validator inside a check is the one reader that reads them. A manifest of a group may serve or invoke what the
// group's contracts modules add to a seat (a verb, an event), which the seat list the checker follows (the trunks') does not
// hold yet; judged without them, by the trunk's manifest validator above all, the group's new indexer would be refused for
// serving its own seat's new verb, and so would every module that calls it. The definitions are read from each member's head
// with the indexer's own reading (the domain SDK's seatdefs: the contracts module's `seats/<seat>/seat.json` with the parts it
// includes, and the rule which module may define a seat), through the forge, and put over the list's in the request's Seats.
import (
"context"
"encoding/json"
"errors"
"fmt"
"maps"
"slices"
"git.novox.be/novox/mesh-delivery/sdk/seatdefs"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Trees lists the files of a repository under directories at a commit (the forge's commit trees): {directory: {path
// relative to it: blob id}}. A directory the commit lacks is an error wrapping ErrNotFound.
type Trees interface {
Files(ctx context.Context, repository, commit string, dirs []string) (map[string]map[string]string, error)
}
// headSeats are the definitions, resolved, that the contracts modules the case's members touch hold at their heads, by
// seat; none where the checker lists no trees. They are read and ruled on as the indexer reads and rules on a revision's,
// with the domain SDK's one reading (mesh-delivery sdk/seatdefs): a seat of the mesh's own contracts is taken only from
// its domain's contracts module (ADR 0636 §5), so an outside definer's is left out and said, and a seat two touched modules
// define is refused unless one hands it over to its domain's module. A head's definition that cannot be read, does not
// parse or names another seat than its directory's is an error: the manifests cannot be judged by the group's seats then.
func (st *state) headSeats(ctx context.Context) (map[string]json.RawMessage, error) {
if st.j.Trees == nil || st.j.Forge == nil {
return nil, nil
}
out := map[string]json.RawMessage{}
by := map[string]string{} // seat -> the module whose head's definition is taken
for _, p := range st.c.Pulls {
for _, module := range p.Modules {
c, ok, err := st.contractsModule(ctx, p, module)
if err != nil {
return nil, err
}
if !ok {
continue
}
defs, unparsed, err := st.headForge().Read(ctx, p.Repository, p.Head, c)
if err != nil {
return nil, fmt.Errorf("the seat definitions of %s at the head of %s cannot be read%s: %w", module, pullKey(p), unheardWords(err), err)
}
if len(unparsed) > 0 {
seat := slices.Sorted(maps.Keys(unparsed))[0]
return nil, fmt.Errorf("the seat definition of %s (module %s) at the head of %s is not taken: %s", seat, module, pullKey(p), unparsed[seat])
}
for _, seat := range slices.Sorted(maps.Keys(defs)) {
d := defs[seat]
if why := seatdefs.Refusal(c.Module, d); why != "" {
st.c.note(fmt.Sprintf("the head of %s's definition of %s is not taken: %s", pullKey(p), seat, why))
continue
}
if prev, twice := by[seat]; twice && prev != c.Module {
switch {
case seatdefs.HandsOver(prev, c.Module, d):
case seatdefs.HandsOver(c.Module, prev, d):
continue
default:
return nil, fmt.Errorf("the seat %s is defined at the heads by both %s and %s: a seat has one definition", seat, prev, c.Module)
}
}
raw, err := json.Marshal(d)
if err != nil {
return nil, fmt.Errorf("the definition of %s cannot be encoded: %w", seat, err)
}
out[seat], by[seat] = raw, c.Module
}
}
}
return out, nil
}
// headForge is the domain SDK's reading bound to the checker's forge listing and get-file: a directory or a file the head
// does not hold is none, every other failure the forge's.
func (st *state) headForge() seatdefs.Forge {
return seatdefs.Forge{
Files: func(ctx context.Context, repository, commit string, dirs []string) (map[string]map[string]string, error) {
got, err := st.j.Trees.Files(ctx, repository, commit, dirs)
if errors.Is(err, ErrNotFound) {
return nil, nil
}
return got, err
},
File: func(ctx context.Context, repository, commit, file string) ([]byte, error) {
text, err := st.j.Forge.File(ctx, repository, file, commit)
if errors.Is(err, ErrNotFound) {
return nil, nil
}
if err != nil {
return nil, err
}
return []byte(text), nil
},
}
}
// contractsModule is a touched module as a contracts module at its member's head (its package artifact's generated-from,
// under the module's directory), and whether the module is a contracts module at all.
func (st *state) contractsModule(ctx context.Context, p Pull, module string) (seatdefs.Module, bool, error) {
var m catalogue.Manifest
dir, known := "", false
if st.j.Dirs != nil {
dir, known = st.j.Dirs.ModuleDir(module)
}
if digest := p.Fps[module].Manifest; known && digest != "" && st.j.Read != nil {
var err error
if m, err = st.j.Read.Manifest(ctx, digest); err != nil {
return seatdefs.Module{}, false, fmt.Errorf("the manifest of %s at the head of %s cannot be read: %w", module, pullKey(p), err)
}
} else {
var found bool
var err error
if m, dir, found, err = st.manifestAtHead(ctx, p, module); err != nil || !found {
return seatdefs.Module{}, false, err
}
}
c, ok := seatdefs.Of(dir, m)
return c, ok, nil
}
// overSeats is the list's definitions with each of the heads' put in place of the one of its seat, or added, sorted by seat.
func overSeats(list []json.RawMessage, heads map[string]json.RawMessage) []json.RawMessage {
if len(heads) == 0 {
return list
}
by := map[string]json.RawMessage{}
for _, raw := range list {
var d struct {
Seat string `json:"seat"`
}
_ = json.Unmarshal(raw, &d)
by[d.Seat] = raw
}
maps.Copy(by, heads)
out := make([]json.RawMessage, 0, len(by))
for _, seat := range slices.Sorted(maps.Keys(by)) {
out = append(out, by[seat])
}
return out
}
@@ -4,16 +4,16 @@ import (
"context"
"encoding/json"
"fmt"
"path"
"mesh-delivery-checker/rules"
"slices"
"sort"
"strings"
"time"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliverycheckerclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
)
@@ -50,11 +50,6 @@ const (
judgedByCandidate = "judged by the candidate's mesh-build-go at "
)
// verdictPtr is a verdict of failure a step of the check ends with.
type verdictPtr = *meshdeliverychecker.VerdictsByGroupHeadsValue
func failed(machine, words string) verdictPtr { return fail(machine, words) }
// Asks puts a check's ask on its build seat's stream and waits for its step (buildrun.Asker; ADR 0525 §1, §6): the
// checker chooses no node, starts no step and holds none; whichever build node pulls the ask runs the check.
type Asks interface {
@@ -140,16 +135,13 @@ type Examiner struct {
Manifests TrunkManifests
// SeatDefinitions reads the indexer's seat definitions of a pull request (seatdefs.go, design 26); nil reads none.
SeatDefinitions SeatDefinitions
// SeatList gives the seat definitions the change's manifests are judged against, as the seat list reader holds them
// (manifests_judge.go, novox/hq ADR 0636 §1); nil gives the judge none.
// SeatList gives the seat list the checker's reader holds from the seat mesh-catalog (verb seat-definitions): the base the
// checking program lays the heads' definitions over (programs.go, novox/hq ADR 0636 §1, §4); nil gives it none.
SeatList SeatList
// Trees lists a member's files at its head, so the seat definitions of the contracts modules a case touches are read
// at their heads and put over SeatList's (headseats.go, novox/hq ADR 0636 §4); nil reads none.
Trees Trees
// Renames asks the seat mesh-config's verb renames for a change that brings or drops a `renamed-from` (renames.go,
// design 32); nil fails such a change, saying it cannot ask.
Renames Renames
// Read reads a kept manifest by its digest: the change's module graph is worked out from them (graph_cycle.go, novox/hq
// Read reads a kept manifest by its digest: the change's module graph is worked out from them (graph_edges.go, novox/hq
// ADR 0603 §2); nil works out no graph.
Read ManifestReader
// Serving says, from what runs, whether a running healthy build serves a seat verb or module tool (novox/hq ADR 0614
@@ -170,9 +162,14 @@ type Examiner struct {
Names(ctx context.Context) ([]string, error)
}
// Candidates answers the indexer's candidate of a try's group once it wrote it from the try's previews, the manifests
// and fingerprints a group try's manifest validator and composition judge (issue 1921); nil leaves a whole try with no
// manifest or composition judged, said.
// and fingerprints a group try's manifest validator and composition judge (issue 1921), and of a pull request's check
// that began before the indexer read its merge preview; nil leaves such a case with no manifest or composition judged,
// said.
Candidates TryCandidates
// Inputs keeps the checking program's inputs (the state InputsStateName, served on InputVerb); BeginBound is how long a run ask may wait for a
// build node to begin it (DefaultBeginBound when zero).
Inputs Inputs
BeginBound time.Duration
Now func() time.Time
Logf func(format string, args ...any)
}
@@ -183,11 +180,10 @@ type TryCandidates interface {
TryCandidate(ctx context.Context, group, heads string, bound time.Duration) (meshcatalog.CandidatesByGroupValue, error)
}
// CompositionReader makes a check's composition verdict; an error is only a case that cannot be composed at all. tagged
// are the group's tagged versions of its Go packages the check heard (design 51 *Tagged versions*), nil for none: a
// composer built at the node-engine member's head is built against them, as the check's other Go builds are.
// CompositionReader composes a check: the engines' answers, which the rules judge (rules.JudgeEngines); an error is only
// a case that cannot be composed at all.
type CompositionReader interface {
Compose(ctx context.Context, c *Case, tagged buildstep.PackageVersions) (meshwants.CompositionsByGroupHeadsValue, error)
Compose(ctx context.Context, c *Case) (rules.Composed, error)
}
func (j *Examiner) now() time.Time {
@@ -269,24 +265,18 @@ func (t *target) stepArgs(args map[string]any) map[string]any {
return args
}
// acc collects a level's counts across targets.
type acc struct {
run, kept, skipped int
elapsed, setup float64
lint float64
over bool
slowest []TestTime
}
// state is one judging.
type state struct {
j *Examiner
c *Case
targets []*target
levels map[string]*acc
nodes map[string]bool
clone time.Duration
slow []string
levels rules.Tally
// facts are the facts of the check met so far, in order: what the checking program judges (milestone 47/B0).
facts []rules.Step
// composition is how the check composed, recorded after every step (compose).
composition *rules.Composition
clone time.Duration
slow []string
// beside are the siblings read at their trunk's head, by repository (ADR 0548 §6).
beside map[string]string
// judgeHead is the group's mesh-delivery member head when its merge changes the judge (ADR 0578 §2); empty judges
@@ -301,23 +291,52 @@ type state struct {
tags groupVersions
// outOfStep are, by pull request (pullKey), the generated files a level's compare step found out of step (design 26).
outOfStep map[string][]string
// noCandidate says why a whole try has no candidate of its group from the indexer, so its manifests were not judged and
// it was not composed; empty when it has one, and for a pull request's check.
// noCandidate says why the case has no previewed candidate of its heads from the indexer within the bound: for a whole
// try its manifests were not judged and it was not composed; for a pull request's check that began on the indexer's
// first write, its manifests were judged from that write and it was not composed. Empty when it has one, and for a pull
// request's check whose case already holds its fingerprints.
noCandidate string
}
// narrowTry says the case is a try narrowed to a level or to some tests: it judges no manifest, graph or composition.
func (st *state) narrowTry() bool { return st.c.Try != nil && !st.c.Try.Whole() }
// takeTryCandidate gives a whole try's pulls the fingerprints and manifests the indexer's candidate of its group holds
// (the group's pull-request check has them from its candidate, CaseOf), once the indexer wrote it from the try's previews.
// A try with no such candidate is said so, in a note and in its composition, and is no pass (Tryer.run).
// WaitsOnPreview says a candidate's module has no fingerprint yet because the indexer wrote it from a pull message alone,
// before it read the head's merge preview (meshcatalogclient.NoneNoPreview): its fingerprint comes with the preview, on the
// same heads, so a candidate that holds one is not yet the candidate of its heads previewed.
func WaitsOnPreview(m meshcatalog.CandidatesByGroupValuePullsItemFpsValue) bool {
return m.None2 == meshcatalogclient.NoneNoPreview
}
// previewPending says a module of the case's pulls waits on its merge preview (WaitsOnPreview).
func (st *state) previewPending() bool {
for _, p := range st.c.Pulls {
for _, m := range p.Fps {
if WaitsOnPreview(m) {
return true
}
}
}
return false
}
// takeTryCandidate gives a whole try's pulls the fingerprints and manifests the indexer's candidate of its group holds,
// once the indexer wrote it from the try's previews. A pull request's check has them from its candidate (CaseOf), but the
// indexer writes a pull request's candidate first from its pull message alone and its fingerprints only with the merge
// preview, on the same heads: a check that began on that first write takes the previewed candidate too, before it judges
// the manifests, waits on the builds and composes (ADR 0603 §10; issue 1921: a check otherwise waited on no build and
// the engines, reading the previewed candidate, refused its modules as not yet built). A case with no such candidate is
// said so, in a note and in its composition, and is no pass (Tryer.run).
func (st *state) takeTryCandidate(ctx context.Context) {
if st.c.Try == nil || !st.c.Try.Whole() {
if st.c.Try != nil && !st.c.Try.Whole() || st.c.Try == nil && !st.previewPending() {
return
}
no := func(why string) {
st.noCandidate = "the group's manifests and composition were not judged: " + why
what := "the group's manifests and composition were not judged: "
if st.c.Try == nil {
what = "the composition was not judged (the manifests were judged from the indexer's first write): "
}
st.noCandidate = what + why
st.c.note(st.noCandidate)
}
if st.j.Candidates == nil {
@@ -350,36 +369,46 @@ func (st *state) takeTryCandidate(ctx context.Context) {
// Judge runs the case (it is the Judge of a Checker). A failure of the change, of a merge or of a step is a verdict; an error is only a case that cannot
// be judged at all.
func (j *Examiner) Judge(ctx context.Context, c *Case) (Outcome, error) {
st := &state{j: j, c: c, levels: map[string]*acc{buildstep.LevelUnit: {}, buildstep.LevelIntegration: {}}, nodes: map[string]bool{},
st := &state{j: j, c: c, levels: rules.NewTally(),
beside: map[string]string{}}
// Every member and level runs to its end, and every manifest the change touches is judged by the indexer's judge (ADR
// 0578 §2), also after a failure: the verdict names every failure, not only the first (the operator, 2026-11-02).
tests := st.judgeAll(ctx)
// Each failure is a fact recorded where it is met; the verdict is the judging rules' of the facts.
st.judgeAll(ctx)
st.takeTryCandidate(ctx)
st.seriesChanges(ctx) // DLV-C4: named in the verdict's line, whatever the verdict
if v := settled(joinFailures(tests, st.judgeManifests(ctx), st.refuseCycle(ctx), st.judgeRenames(ctx), st.refuseUnparsed(ctx))); v != nil {
return st.finish(ctx, *v), nil
st.seriesChanges(ctx) // DLV-C4: a fact of the check, named in the verdict whatever it is
// The heads' seat list, from the checking program, before the validator and the composer are asked (ADR 0636 §4).
// A list it cannot make ends the check there, failed naming the checking program, worded from the facts so far.
var answer rules.Answer
var by *meshdeliverychecker.VerdictsByGroupHeadsValueJudgedBy
if st.seatList(ctx) {
st.judgeManifests(ctx)
st.gatherGraph(ctx)
st.judgeRenames(ctx)
st.refuseUnparsed(ctx)
st.compose(ctx)
// The verdict is the checking program's of the facts (ADR 0644 §1, ADR 0656 §5).
answer, by = st.judged(ctx)
} else {
answer = rules.Answer{V: rules.AnswerVersion, Verdict: rules.Judge(st.factsOf())}
}
return st.finish(ctx, meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictPass}), nil
}
// finish is the outcome of the verdict with the state of the builds the candidate's modules carry and the assigner's
// composition of it, read also when the tests failed, so a failed composition is named beside them.
func (st *state) finish(ctx context.Context, v meshdeliverychecker.VerdictsByGroupHeadsValue) Outcome {
out := st.outcome(v)
st.composed(ctx, &out.Verdict)
for _, n := range answer.Notes {
st.c.note(n)
}
answer.Verdict.JudgedBy = by
out := st.outcome(answer.Verdict)
out.Builds = st.j.buildNotes(ctx, st.c)
return out
return out, nil
}
// composed makes the check's composition from the engines' answers into its verdict (ADR 0593 §7, §9): a failed
// composition fails the check, naming the machine it failed at; a node that did not answer is a warning. A try is not
// composed (it has no candidate); a composition that cannot be made leaves it not run, said.
func (st *state) composed(ctx context.Context, v *meshdeliverychecker.VerdictsByGroupHeadsValue) {
// compose records how the check composed as the facts' composition, after every step (ADR 0593 §7, §9): kept by a carry,
// not run (said why), or the engines' answers with the candidate's builds awaited first (ADR 0603 §10). A narrowed try
// and an examiner that composes nothing record none.
func (st *state) compose(ctx context.Context) {
if st.c.Composition != nil && st.c.Try == nil {
// A carry whose candidate's fingerprints did not change keeps the composition it passed (ADR 0548 §5).
kept := *st.c.Composition
v.Composition = &kept
st.composition = &rules.Composition{Kept: &kept}
st.c.note("the composition is kept: no fingerprint of a module it composed changed")
return
}
@@ -387,7 +416,7 @@ func (st *state) composed(ctx context.Context, v *meshdeliverychecker.VerdictsBy
return
}
if st.noCandidate != "" {
v.Composition = &meshdeliverychecker.VerdictsByGroupHeadsValueComposition{State: meshdeliverycheckerclient.CompositionNotRun, Words: st.noCandidate}
st.composition = &rules.Composition{NotRun: st.noCandidate}
return
}
if st.j.Compositions == nil {
@@ -395,62 +424,36 @@ func (st *state) composed(ctx context.Context, v *meshdeliverychecker.VerdictsBy
}
if len(st.unpreviewed) > 0 {
// An engine would only say it cannot read a manifest the indexer will not have: the check does not ask it.
v.Composition = &meshdeliverychecker.VerdictsByGroupHeadsValueComposition{State: meshdeliverycheckerclient.CompositionNotRun, Words: "no preview of a merge of " +
strings.Join(st.unpreviewed, ", ") + " (its head does not merge or its fetch did not answer), so the change cannot be composed"}
st.composition = &rules.Composition{NotRun: "no preview of a merge of " + strings.Join(st.unpreviewed, ", ") +
" (its head does not merge or its fetch did not answer), so the change cannot be composed"}
return
}
words := st.awaitBuilds(ctx, v) // ADR 0603 §10: the check composes with the candidate's builds
comp, err := st.j.Compositions.Compose(ctx, st.c, st.tags.goPackages)
// ADR 0603 §10: the check composes with the candidate's builds; whether it failed already is the running rules' word
// on the facts so far, an asking choice only (the verdict is the rules' of every fact).
failed := rules.Judge(st.factsOf()).State == meshdeliverycheckerclient.VerdictFail
words, builds, late := st.awaitBuilds(ctx, failed)
c := &rules.Composition{Waits: words, BuildsFailed: builds, FPSet: st.c.FPSet}
st.composition = c
if late != "" {
c.NotRun = late // the builder's lateness, said by name, never the change's refusal (issue 1921)
return
}
// A composer built at the node-engine member's head is built against the group's tagged versions the check heard, as
// its other Go builds are (design 51 *Tagged versions*); it composes against the heads' seat list (seatList).
st.c.Tagged = st.tags.goPackages
comp, err := st.j.Compositions.Compose(ctx, st.c)
if err != nil {
// Said in the verdict, never left as a composition not yet run (review finding 4).
st.j.logf("%s: the composition could not be made, so it is said as not run: %v", st.c.Key, err)
v.Composition = &meshdeliverychecker.VerdictsByGroupHeadsValueComposition{State: meshdeliverycheckerclient.CompositionNotRun, Words: "not made: " + err.Error()}
c.NotRun = "not made: " + err.Error()
return
}
comp.Warnings = append(comp.Warnings, words...) // ADR 0603 §10
withComposition(v, comp)
if v.Composition != nil && v.Composition.State != meshdeliverycheckerclient.CompositionNotRun {
v.Composition.Fpset = st.c.FPSet // a carry keeps it while the fingerprints stay the same (ADR 0548 §5)
}
c.Composed = &comp
}
// outcome fills the verdict's levels, times, machines and composition from what ran.
// outcome is the verdict judged from the facts with the slowest tests, the siblings read and what judged a build module.
func (st *state) outcome(v meshdeliverychecker.VerdictsByGroupHeadsValue) Outcome {
v.Series = st.c.allSeries() // the series changes, as structures (A2j finding 2)
if v.State == meshdeliverycheckerclient.VerdictPass || len(v.Levels) == 0 {
v.Levels = map[string]meshdeliverychecker.VerdictsByGroupHeadsValueLevelsValue{}
for level, a := range st.levels {
v.Levels[level] = meshdeliverychecker.VerdictsByGroupHeadsValueLevelsValue{Run: int64(a.run), Kept: int64(a.kept), Skipped: int64(a.skipped)}
}
}
ms := func(s float64) int64 { return int64(s * 1000) }
u, i := st.levels[buildstep.LevelUnit], st.levels[buildstep.LevelIntegration]
v.Took = &meshdeliverychecker.VerdictsByGroupHeadsValueTook{Clone: st.clone.Milliseconds(), Unit: ms(u.elapsed), Lint: ms(u.lint), IntegrationSetup: ms(i.setup), IntegrationRun: ms(i.elapsed)}
for node := range st.nodes {
v.Machines = append(v.Machines, node)
}
sort.Strings(v.Machines)
for _, level := range []string{buildstep.LevelUnit, buildstep.LevelIntegration} {
if st.levels[level].over {
v.OverBound = append(v.OverBound, level)
}
}
// The composition is made from the engines' answers (ADR 0593 §7): until it is, a verdict says it was not run (ADR 0484 clause 6).
v.Composition = &meshdeliverychecker.VerdictsByGroupHeadsValueComposition{State: meshdeliverycheckerclient.CompositionNotRun}
var slow []string
for _, level := range []string{buildstep.LevelUnit, buildstep.LevelIntegration} {
a := st.levels[level]
if !a.over {
continue
}
sort.SliceStable(a.slowest, func(x, y int) bool { return a.slowest[x].Elapsed > a.slowest[y].Elapsed })
for n, s := range a.slowest {
if n == 3 {
break
}
slow = append(slow, fmt.Sprintf("%s %s %.1fs", level, path.Base(s.Package)+"."+s.Test, s.Elapsed))
}
}
slow := rules.Slowest(st.levels)
out := Outcome{Verdict: v, Slowest: slow, OutOfStep: st.outOfStep}
if len(st.beside) > 0 {
out.Beside = st.beside
@@ -480,12 +483,39 @@ func (st *state) stepOutOfStep(p Pull, files []string) {
st.outOfStep[k] = slices.Compact(all)
}
// fail is a verdict of failure at a machine.
// fail records a failure the asking met as a fact of the check, where it is met, and answers it as a verdict of failure
// at a machine: the verdict is the judging rules' of the facts (rules.Judge), so a failure only returned is no failure
// (milestone 47/B0).
func (st *state) fail(machine, words string) {
st.facts = append(st.facts, rules.Step{Failed: &FailedFirst{Machine: machine, Words: words}})
}
// fail is a verdict of failure at a machine, for a step that answers one before any member is asked (tagged); it is
// recorded as facts (failedAll), never returned as the check's verdict.
func fail(machine, words string) *meshdeliverychecker.VerdictsByGroupHeadsValue {
v := failure(machine, words)
return &v
}
// failedAll records every failure a verdict of the asking names (its list, else its first) as facts of the check.
func (st *state) failedAll(v meshdeliverychecker.VerdictsByGroupHeadsValue) {
all := failedList(v.Failures)
if len(all) == 0 {
all = []FailedFirst{failedOf(v)}
}
for _, f := range all {
st.facts = append(st.facts, rules.Step{Failed: &f})
}
}
// factsOf are the check's facts as met: what the checking program judges.
func (st *state) factsOf() rules.Facts {
f := st.factsHead()
f.Steps, f.CloneMS, f.Composition, f.Notes = st.facts, st.clone.Milliseconds(), st.composition, st.c.Notes()
f.Series = st.c.allSeries()
return f
}
// levels are the levels a case's asks run: both, or the one a try names.
func (st *state) levelsOf() []string {
if st.c.Try != nil && st.c.Try.Level != "" {
@@ -498,7 +528,7 @@ func (st *state) levelsOf() []string {
// judges each answer. A head that does not merge, a pin that is not allowed or a level that fails is a failure of that
// member; every other member is still judged, and the verdict names every failure (the operator, 2026-11-02), unjoined
// to its first lines (settled does that). Only a check whose context ended stops.
func (st *state) judgeAll(ctx context.Context) *meshdeliverychecker.VerdictsByGroupHeadsValue {
func (st *state) judgeAll(ctx context.Context) {
// The mesh-sdk member first: whether its merge changes the TypeScript package's paths decides whether the other members'
// Node fetches take the group's package (ADR 0500 §5).
pulls := slices.Clone(st.c.Pulls)
@@ -512,16 +542,18 @@ func (st *state) judgeAll(ctx context.Context) *meshdeliverychecker.VerdictsByGr
return 0
})
// A group's members are built only after each package they declare is published as the group's tagged version
// (design 51 *Tagged versions*): a package not published within the bound fails the check before any member is built.
// (design 51 *Tagged versions*): a package not published within the bound fails the check before any member is built,
// each failure a fact of the check.
if v := st.tagged(ctx); v != nil {
return v
st.failedAll(*v)
return
}
var groupPackage *buildstep.FetchPackage
var failures []*meshdeliverychecker.VerdictsByGroupHeadsValue
met := len(st.facts)
for _, p := range pulls {
if ctx.Err() != nil {
if len(failures) == 0 { // an ask that ended with the context said it already
failures = append(failures, fail(CheckerModule, ctx.Err().Error()))
if !st.failedSince(met) { // an ask that ended with the context said it already
st.fail(CheckerModule, ctx.Err().Error())
}
break
}
@@ -544,7 +576,7 @@ func (st *state) judgeAll(ctx context.Context) *meshdeliverychecker.VerdictsByGr
if bm == buildrun.GoBuildModule {
beside, err := st.j.besideFor(ctx, st.c, p)
if err != nil {
failures = append(failures, fail(forgeMachine(err, bm), err.Error()+unheardWords(err)))
st.fail(forgeMachine(err, bm), err.Error()+unheardWords(err))
continue
}
if len(beside) > 0 {
@@ -558,9 +590,7 @@ func (st *state) judgeAll(ctx context.Context) *meshdeliverychecker.VerdictsByGr
}
t.fetchArgs = args
st.targets = append(st.targets, t)
if v := st.ask(ctx, t, &merged); v != nil {
failures = append(failures, v)
}
st.ask(ctx, t, &merged)
}
// Every build module's fetch merged (each is in merged once): the preview is written, also when the member's tests
// failed, so the indexer holds its manifests and the assigner can compose the change (review of 650a1d0).
@@ -579,7 +609,16 @@ func (st *state) judgeAll(ctx context.Context) *meshdeliverychecker.VerdictsByGr
groupPackage = &buildstep.FetchPackage{Repository: p.Repository, Commit: p.Head, Ref: refOf(p.Number)}
}
}
return joinFailures(failures...)
}
// failedSince says a failure was recorded among the facts from the nth on.
func (st *state) failedSince(n int) bool {
for _, f := range st.facts[n:] {
if f.Failed != nil {
return true
}
}
return false
}
// DefaultDeclineBound is an Examiner's DeclineBound when it names none: longer than a check runs, so a node with the
@@ -590,14 +629,14 @@ const DefaultDeclineBound = 30 * time.Minute
// each level's digest, kept verdicts and tests. When the answer leaves packages unmet,
// their tests needing what that node lacks (issue 890), the passes are kept and the levels from the first such one are
// asked again of a node that has it (placed).
func (st *state) ask(ctx context.Context, t *target, merged *[]Fetched) *meshdeliverychecker.VerdictsByGroupHeadsValue {
return st.askNeeding(ctx, t, merged, st.levelsOf(), nil, nil)
func (st *state) ask(ctx context.Context, t *target, merged *[]Fetched) {
st.askNeeding(ctx, t, merged, st.levelsOf(), nil, nil)
}
// askNeeding is ask with the levels to run, what the node must have and the packages it tests: none for the first ask
// of a target; for the placed one the unmet packages' needs and those packages alone, so no package the first ask judged
// is run or counted again (ADR 0552 §4).
func (st *state) askNeeding(ctx context.Context, t *target, merged *[]Fetched, levels, needs, packages []string) *meshdeliverychecker.VerdictsByGroupHeadsValue {
func (st *state) askNeeding(ctx context.Context, t *target, merged *[]Fetched, levels, needs, packages []string) {
p := t.pull
placed := len(needs) > 0
job := &buildstep.CheckJob{Fetch: t.fetchArgs, Levels: levels, Kept: true, Packages: packages, Toolchain: t.agreed}
@@ -607,7 +646,8 @@ func (st *state) askNeeding(ctx context.Context, t *target, merged *[]Fetched, l
err = fmt.Errorf("no node record is on the bus")
}
if err != nil {
return fail(t.module, fmt.Sprintf("the check of %s#%d is not asked: the nodes' names the names check needs cannot be read: %v", p.Repository, p.Number, err))
st.fail(t.module, fmt.Sprintf("the check of %s#%d is not asked: the nodes' names the names check needs cannot be read: %v", p.Repository, p.Number, err))
return
}
job.NodeNames = names
}
@@ -636,7 +676,8 @@ func (st *state) askNeeding(ctx context.Context, t *target, merged *[]Fetched, l
st.c.Progress.Set(what, "")
st.c.Asked(a.ID)
if err := st.j.Asks.Put(ctx, a); err != nil {
return fail(t.module, fmt.Sprintf("the check of %s#%d could not be asked: %v", p.Repository, p.Number, err))
st.fail(t.module, fmt.Sprintf("the check of %s#%d could not be asked: %v", p.Repository, p.Number, err))
return
}
waiting := ctx
if watch := st.c.Watch; watch != nil {
@@ -648,9 +689,11 @@ func (st *state) askNeeding(ctx context.Context, t *target, merged *[]Fetched, l
step, err = st.j.Asks.Wait(waiting, a)
if err != nil {
if ctx.Err() != nil {
return fail(t.module, ctx.Err().Error())
st.fail(t.module, ctx.Err().Error())
return
}
return fail(t.module, fmt.Sprintf("the check of %s#%d: %v", p.Repository, p.Number, err))
st.fail(t.module, fmt.Sprintf("the check of %s#%d: %v", p.Repository, p.Number, err))
return
}
d, declined := buildstep.DeclinedOf(step)
if !declined {
@@ -671,36 +714,40 @@ func (st *state) askNeeding(ctx context.Context, t *target, merged *[]Fetched, l
if lacks == "" {
lacks = strings.Join(d.Lacks, ", ") // a node off the agreed toolchain declined it (ADR 0584)
}
return fail(step.Node, fmt.Sprintf("the check of %s#%d needs a node with %s, and none took it in %s: every node that pulled it lacks it (last %s)",
st.fail(step.Node, fmt.Sprintf("the check of %s#%d needs a node with %s, and none took it in %s: every node that pulled it lacks it (last %s)",
p.Repository, p.Number, lacks, bound, step.Node))
return
}
st.j.logf("%s#%d: the ask %s was declined on %s, which lacks %s; asked again", p.Repository, p.Number, a.ID, step.Node, strings.Join(d.Lacks, ", "))
a.ID, a.At = buildstep.NewID(), now.UTC()
}
var failures []*meshdeliverychecker.VerdictsByGroupHeadsValue
// The fetch and the digests are what of the ask's time its levels' running did not take (Took.Clone).
took := time.Duration(0)
if !step.Started.IsZero() && step.Ended.After(step.Started) {
took = step.Ended.Sub(step.Started)
}
t.node = step.Node
st.nodes[step.Node] = true
st.facts = append(st.facts, rules.Step{Repository: p.Repository, Number: p.Number, Module: t.module, Node: step.Node})
st.c.Progress.Set(fmt.Sprintf("judge the check of %s#%d", p.Repository, p.Number), step.Node)
if step.State != buildstep.StepPassed {
return fail(step.Node, fmt.Sprintf("the check of %s#%d %s: %s", p.Repository, p.Number, step.State, step.Reason))
st.fail(step.Node, fmt.Sprintf("the check of %s#%d %s: %s", p.Repository, p.Number, step.State, step.Reason))
return
}
var answer buildstep.CheckAnswer
if err := json.Unmarshal(step.Result, &answer); err != nil {
return fail(step.Node, fmt.Sprintf("the check of %s#%d answered what is not a check: %s", p.Repository, p.Number, step.Result))
st.fail(step.Node, fmt.Sprintf("the check of %s#%d answered what is not a check: %s", p.Repository, p.Number, step.Result))
return
}
if t.fetched = answer.Fetch; t.fetched.Workspace == "" {
return fail(step.Node, fmt.Sprintf("the fetch of %s#%d answered no workspace: %s", p.Repository, p.Number, step.Result))
st.fail(step.Node, fmt.Sprintf("the fetch of %s#%d answered no workspace: %s", p.Repository, p.Number, step.Result))
return
}
if len(t.fetched.Conflicts) > 0 {
if !st.narrowTry() {
st.j.writePreview(ctx, p, append(*merged, t.fetched))
}
return fail(step.Node, fmt.Sprintf("%s#%d %s%s", p.Repository, p.Number, ConflictWords, strings.Join(t.fetched.Conflicts, ", ")))
st.fail(step.Node, fmt.Sprintf("%s#%d %s%s", p.Repository, p.Number, ConflictWords, strings.Join(t.fetched.Conflicts, ", ")))
return
}
t.changed = t.fetched.Changed
if !changesBuildGo(p.Repository, t.changed) {
@@ -708,25 +755,20 @@ func (st *state) askNeeding(ctx context.Context, t *target, merged *[]Fetched, l
}
if !placed { // the placed ask fetched the same merge: its preview was read from the first
*merged = append(*merged, t.fetched)
if v := st.checkBinariesOf(t); v != nil {
failures = append(failures, v) // the levels still run: the verdict names every failure
}
st.checkBinariesOf(t) // the levels still run: the verdict names every failure
}
ran := 0.0
for _, la := range answer.Levels {
if placed {
if len(la.Unmet) > 0 {
st.clone += max(0, took-time.Duration(ran*float64(time.Second)))
return fail(step.Node, fmt.Sprintf("the %s level of %s#%d left %s untested on %s, which was asked for a node with %s",
st.fail(step.Node, fmt.Sprintf("the %s level of %s#%d left %s untested on %s, which was asked for a node with %s",
la.Level, p.Repository, p.Number, unmetNames(la.Unmet), step.Node, strings.Join(needs, ", ")))
return
}
la.Kept = nil // it names its packages and takes no other's kept verdict; this is only a guard against an older module
}
v, elapsed := st.level(ctx, t, la)
ran += elapsed
if v != nil {
failures = append(failures, v) // the next level was run too (buildstep): its failures are named as well
}
ran += st.level(ctx, t, la) // the next level was run too (buildstep): its failures are named as well
}
st.clone += max(0, took-time.Duration(ran*float64(time.Second)))
if unmet := buildstep.UnmetNeeds(answer.Levels); len(unmet) > 0 && !placed {
@@ -746,9 +788,8 @@ func (st *state) askNeeding(ctx context.Context, t *target, merged *[]Fetched, l
}
}
st.j.logf("%s#%d: %s on %s left untested; asked of a node with %s", p.Repository, p.Number, unmetOf(answer.Levels), step.Node, strings.Join(unmet, ", "))
failures = append(failures, st.askNeeding(ctx, t, merged, again, unmet, packages))
st.askNeeding(ctx, t, merged, again, unmet, packages)
}
return joinFailures(failures...)
}
// unmetNames names the unmet packages and their needs.
@@ -775,64 +816,32 @@ func unmetOf(levels []buildstep.LevelAnswer) string {
// of the packages that passed, the kept ones taken are counted, and every test result is read and counted, also after
// one that did not pass, so the verdict names every failed test of the level (issue 1921). It answers the level's
// running time (its tests' and setup's).
func (st *state) level(ctx context.Context, t *target, la buildstep.LevelAnswer) (*meshdeliverychecker.VerdictsByGroupHeadsValue, float64) {
level := la.Level
a := st.levels[level]
if a == nil {
return fail(t.node, fmt.Sprintf("the check of %s#%d answered the level %q, which is no level", t.pull.Repository, t.pull.Number, level)), 0
}
var digests map[string]string
if t.module == buildrun.GoBuildModule && la.Digest != nil {
// The packages come as a list, never keyed by a package path (issue 1930); an answer stored before reads too.
digests = la.Digest.Map()
if t.runner != "" {
t.judgedBy(la.Digest.JudgedBy) // the installed module judged it: the runner is not used again
}
func (st *state) level(ctx context.Context, t *target, la buildstep.LevelAnswer) float64 {
if t.runner != "" && t.module == buildrun.GoBuildModule && la.Digest != nil {
t.judgedBy(la.Digest.JudgedBy) // the installed module judged it: the runner is not used again
}
if t.runner != "" && la.Judged != "" {
t.judgedBy(la.Judged) // the candidate judged the level: nothing is kept (ADR 0535 §4)
}
for _, k := range la.Kept {
a.kept += k.Tests
_, elapsed, results := st.levels.Level(t.pull.Repository, t.pull.Number, t.module, t.node, la)
st.facts = append(st.facts, rules.Step{Repository: t.pull.Repository, Number: t.pull.Number, Module: t.module, Node: t.node, Level: rules.LevelFact(la)})
var digests map[string]string
if t.module == buildrun.GoBuildModule && la.Digest != nil {
// The packages come as a list, never keyed by a package path (issue 1930); an answer stored before reads too.
digests = la.Digest.Map()
}
elapsed := 0.0
var failedResults []Tested
for _, raw := range la.Tests {
var res Tested
if err := json.Unmarshal(raw, &res); err != nil {
return fail(t.node, fmt.Sprintf("the %s level of %s#%d answered what is not a result: %v", level, t.pull.Repository, t.pull.Number, err)), elapsed
}
for _, res := range results {
if t.runner != "" {
t.judgedBy(res.JudgedBy)
}
elapsed += res.Elapsed + res.Setup + res.Lint
if t.module == buildrun.NodeBuildModule {
if !res.NoTests {
a.run++ // a Node package is counted whole: its build module counts no tests (ADR 0484 clause 10)
}
} else {
a.run += res.Tests
a.skipped += res.Skipped
a.slowest = append(a.slowest, res.Slowest...)
}
a.elapsed += res.Elapsed
a.setup += res.Setup
a.lint += res.Lint
a.over = a.over || res.OverBound
st.stepOutOfStep(t.pull, res.OutOfStep)
if len(digests) > 0 {
if _, err := st.j.Kept.KeepPasses(ctx, t.pull.Repository, level, t.toolchain, res, digests, st.j.now()); err != nil {
if _, err := st.j.Kept.KeepPasses(ctx, t.pull.Repository, la.Level, t.toolchain, res, digests, st.j.now()); err != nil {
st.j.logf("%s#%d: a passing package could not be kept: %v", t.pull.Repository, t.pull.Number, err)
}
}
if !res.Passed {
failedResults = append(failedResults, res)
}
st.stepOutOfStep(t.pull, res.OutOfStep)
}
if len(failedResults) > 0 {
return levelFailure(t, level, failedResults...), elapsed
}
return nil, elapsed
return elapsed
}
// changesPackage says whether the merge of a pull request changes a file inside the named paths of one of its repository's
@@ -854,259 +863,3 @@ func (j *Examiner) changesPackage(p Pull, merged []Fetched) bool {
}
return false
}
// failedName names a failure: its package and test.
func failedName(f Failure) string {
if f.Test == "" {
return f.Package
}
return f.Package + "." + f.Test
}
// MaxWords is the most a failed test's words in a verdict hold (issue 1901). **The words keep the failure's last lines**,
// where a test says why it failed: the first line is often something it logged before (the group r4g verdict kept a
// "judged against" line and lost the t.Fatalf after it), and the 120 characters kept before cut a docker command's echo
// before docker's own error. Four kilobytes holds a long message and is small beside a verdict's other parts.
const MaxWords = 4096
// failureWords is a failure's words: its output's last lines up to MaxWords, with the line the build module told apart as
// the test's error first when the output does not start with it, so the first line of the words is the cause and the
// status line, which is cut short, names it. **A failure's words are never "PASS" or "ok"** (issue 1901, its third
// meeting: a package that failed after its tests passed was worded "PASS"): go test's summary lines are dropped, and with
// no cause told apart (an older build module) the cause is the output's first "panic:", "fatal error:" or "signal:"
// line, read before the output is cut, or the words say the tests passed and the program failed after them.
func failureWords(f Failure) string {
return failureWordsAfter(f, "")
}
// failureWordsAfter is failureWords with the line loud put first, within the same MaxWords: what the checker itself says of
// the failure (that the build module hid it), which the status line then names.
func failureWordsAfter(f Failure, loud string) string {
out, passed := dropSummary(strings.TrimSpace(f.Output))
cause := strings.TrimSpace(f.Cause)
if cause == "" {
cause = causeIn(out)
}
switch {
case cause == "" && out == "" && passed:
cause = "the tests passed, then the test program exited with a failure: an os.Exit or a panic in TestMain or after the tests"
case cause == "" && out == "":
// The build module on main sends a package that stopped (a timeout, an exit, a kill) as its "FAIL" line alone.
cause = "the package failed with no test to name and no panic, signal or build error in its output (an older build module sent no cause)"
}
if len(cause) > MaxCause {
cause = strings.ToValidUTF8(cause[:MaxCause], "") + "..."
}
// The words keep MaxWords exactly: the loud line, the cause and their newlines, the marker and the kept lines.
budget := MaxWords
if cause != "" {
budget -= len(cause) + 1
}
if loud != "" {
budget -= len(loud) + 1
}
if len(out) > budget {
const marker = "... "
tail := out[len(out)-(budget-len(marker)):]
// Start on a whole line, and a whole character.
if i := strings.IndexByte(tail, '\n'); i >= 0 {
tail = strings.TrimLeft(tail[i:], "\n")
}
out = marker + strings.ToValidUTF8(tail, "")
}
if cause != "" && !strings.HasPrefix(firstLine(out), cause) {
out = strings.TrimRight(cause+"\n"+out, "\n")
}
if loud != "" {
out = strings.TrimRight(loud+"\n"+out, "\n")
}
return out
}
// hidWords is the checker's loud line for a failure the build module sent hiding its cause (issue 1901, its thorough
// fix): with no cause, or a passing line as its cause, under the failure contract (buildstep.Hidden). It names the build
// module and the version it says it is (Tested.By, "<module> with mesh-sdk <version>"); one from before the contract
// says nothing of itself. "" when the failure keeps the contract. The check fails either way: only its words change.
func hidWords(module, by string, f Failure) string {
why := buildstep.Hidden(f)
if why == "" {
return ""
}
version := "a version before the failure contract"
if module == "" {
module = "an unnamed build module"
}
if m, v, ok := strings.Cut(by, " with "); ok && m != "" {
module, version = m, v
}
return fmt.Sprintf("the build module hid the failure (%s at %s): %s", module, version, why)
}
// MaxCause is the most a cause in a failure's words holds: one long line (the 1901 review: a cause of a kilobyte could
// push the words past MaxWords).
const MaxCause = 1024
// dropSummary drops go test's summary lines from a failure's output ("PASS", "FAIL", "ok <pkg>", "FAIL <pkg>"), and says
// whether a "PASS" was among them: the package's tests passed before it failed.
func dropSummary(out string) (string, bool) {
passed := false
lines := strings.Split(out, "\n")
kept := lines[:0]
for _, l := range lines {
// go's exact forms only (review round 1): "PASS" or "FAIL" alone, "ok \t<pkg>\t<time>", "FAIL\t<pkg>\t<time>".
switch t := strings.TrimRight(l, "\r"); {
case t == "PASS":
passed = true
case t == "FAIL", strings.HasPrefix(t, "ok \t"), strings.HasPrefix(t, "FAIL\t"):
default:
kept = append(kept, l)
}
}
return strings.TrimSpace(strings.Join(kept, "\n")), passed
}
// causeIn is a race the race detector reported, as its heading, its access and the frame that made it ("WARNING: DATA
// RACE: Write at 0x... by goroutine 9 in pkg.F.func1()"); else the first "panic:", "fatal error:" or "signal:" line of a
// failure's output, in that order; or "". **Only lines at column 0**, where go prints the runtime's: a test's indented log
// line holding the same words is not the cause (review round 1).
func causeIn(out string) string {
lines := strings.Split(out, "\n")
for i, l := range lines {
if strings.TrimRight(l, "\r") != "WARNING: DATA RACE" {
continue
}
words := "WARNING: DATA RACE"
if i+1 < len(lines) && strings.TrimSpace(lines[i+1]) != "" {
words += ": " + strings.TrimSuffix(strings.TrimSpace(lines[i+1]), ":")
if i+2 < len(lines) && strings.HasPrefix(lines[i+2], " ") {
words += " in " + strings.TrimSpace(lines[i+2])
}
}
return words
}
for _, prefix := range []string{"panic: ", "fatal error: ", "signal: "} {
for _, l := range lines {
if l = strings.TrimRight(l, "\r"); strings.HasPrefix(l, prefix) {
return l
}
}
}
return ""
}
// firstLine is the first non-empty line of a failure's words.
func firstLine(s string) string {
for _, l := range strings.Split(s, "\n") {
if l = strings.TrimSpace(l); l != "" {
return l
}
}
return ""
}
// MaxFailureLine is the most bytes of a failure's line in a verdict's Failures: the first line of its words, which is
// its cause where the build module told one apart (issue 1901). Failed keeps the first failure's words whole.
const MaxFailureLine = 300
// levelFailure turns a level that did not pass into a verdict: Failed names what failed first with its words whole,
// and Failures every failed test of the level's results with its first line, at most meshdeliverycheckerclient.MaxFailures and an entry
// saying how many more (issue 1921).
func levelFailure(t *target, level string, results ...Tested) *meshdeliverychecker.VerdictsByGroupHeadsValue {
v := meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail}
var all []FailedFirst
for _, res := range results {
if len(res.Failed) == 0 {
why := res.Why
if why == "" {
why = "the " + level + " level did not pass"
}
all = append(all, FailedFirst{Test: level + ": " + t.pull.Repository, Words: why})
continue
}
for _, f := range res.Failed {
name := failedName(f)
if t.module == buildrun.NodeBuildModule && f.Package == "" {
name = res.Module
}
all = append(all, FailedFirst{Test: level + ": " + name, Words: failureWordsAfter(f, hidWords(t.module, res.By, f))})
}
}
if len(all) == 0 { // no result: a level said failed with nothing to name
all = append(all, FailedFirst{Test: level + ": " + t.pull.Repository, Words: "the " + level + " level did not pass"})
}
first := all[0]
v.Failed = &first
v.Failures = failureItems(all) // whole: settled cuts them to their first lines once every failure of the check is joined
return &v
}
// joinFailures is one failed verdict of every failure of vs, in the order given (a nil one passed), unsettled: Failed is
// the first whole, and Failures every failure whole, or none when the one failure is no level's (a conflict, a refused
// step: one line, as before issue 1921). Nil when none failed.
func joinFailures(vs ...*meshdeliverychecker.VerdictsByGroupHeadsValue) *meshdeliverychecker.VerdictsByGroupHeadsValue {
var out *meshdeliverychecker.VerdictsByGroupHeadsValue
var all []FailedFirst
listed := false
for _, v := range vs {
if v == nil {
continue
}
if out == nil {
out = &meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail, Failed: v.Failed}
}
if len(v.Failures) > 0 {
all, listed = append(all, failedList(v.Failures)...), true
continue
}
all = append(all, failedOf(*v))
}
if out != nil && (listed || len(all) > 1) {
out.Failures = failureItems(all)
}
return out
}
// settled is a joined verdict as it is written: each failure of its list cut to its first line, at most
// meshdeliverycheckerclient.MaxFailures and then how many more (failureLines).
func settled(v *meshdeliverychecker.VerdictsByGroupHeadsValue) *meshdeliverychecker.VerdictsByGroupHeadsValue {
if v != nil && len(v.Failures) > 0 {
v.Failures = failureItems(failureLines(failedList(v.Failures)))
}
return v
}
// addFailure is a settled list of failures with one more at its end: the first failure is put first when the list is
// empty (a verdict of one failure keeps none), and past meshdeliverycheckerclient.MaxFailures the last entry counts one more.
func addFailure(v *meshdeliverychecker.VerdictsByGroupHeadsValue, f FailedFirst) {
if len(v.Failures) == 0 {
v.Failures = failureItems(failureLines([]FailedFirst{failedOf(*v)}))
}
if n := len(v.Failures); n > meshdeliverycheckerclient.MaxFailures {
v.Failures[n-1].Test = fmt.Sprintf("and %d more", failedCount(*v)-meshdeliverycheckerclient.MaxFailures+1)
return
}
v.Failures = append(v.Failures, failureItems(failureLines([]FailedFirst{f}))...)
}
// failureLines is every failure with the first line of its words, cut to MaxFailureLine, at most meshdeliverycheckerclient.MaxFailures
// and then one entry saying how many more failed.
func failureLines(all []FailedFirst) []FailedFirst {
n := min(len(all), meshdeliverycheckerclient.MaxFailures)
out := make([]FailedFirst, 0, n+1)
for _, f := range all[:n] {
out = append(out, FailedFirst{Test: f.Test, Machine: f.Machine, Words: clipLine(firstLine(f.Words), MaxFailureLine)})
}
if more := len(all) - n; more > 0 {
out = append(out, FailedFirst{Test: fmt.Sprintf("and %d more", more)})
}
return out
}
// clipLine cuts a line to at most n bytes on a whole character, marking the cut.
func clipLine(s string, n int) string {
if len(s) <= n {
return s
}
s = strings.ToValidUTF8(s[:n-3], "")
return s + "..."
}
@@ -2,6 +2,7 @@ package check
import (
"fmt"
"mesh-delivery-checker/rules"
"slices"
"strings"
"unicode/utf8"
@@ -72,7 +73,7 @@ func failedLines(v meshdeliverychecker.VerdictsByGroupHeadsValue) []string {
// are cut to fit room characters when room is not negative; the count and "and N more" are never cut.
func failedHead(v meshdeliverychecker.VerdictsByGroupHeadsValue, room int) string {
first := firstFailed(v)
n := failedCount(v)
n := rules.FailedCount(v)
if n <= 1 {
return clipRunes("failed: "+first, room)
}
@@ -95,20 +96,8 @@ func clipRunes(s string, n int) string {
return string(r[:n-1]) + "…"
}
// failedCount is how many failures a verdict names, the ones after meshdeliverycheckerclient.MaxFailures counted too.
func failedCount(v meshdeliverychecker.VerdictsByGroupHeadsValue) int {
n := len(v.Failures)
if n > meshdeliverycheckerclient.MaxFailures {
var more int
if _, err := fmt.Sscanf(v.Failures[n-1].Test, "and %d more", &more); err == nil {
return meshdeliverycheckerclient.MaxFailures + more
}
}
return n
}
// firstWordsLine is the first line of a failure's words: the words are several lines (issue 1901) and the status line one.
func firstWordsLine(w string) string { return firstLine(w) }
func firstWordsLine(w string) string { return rules.FirstLine(w) }
// pieces are the parts of a line, from the most to the least needed.
type pieces struct {
@@ -69,100 +69,103 @@ func TestAFailureNamesTheTestTheConflictOrTheRefusedStepFirst(t *testing.T) { //
}
}
func TestALevelOverItsBoundIsNamedWithItsSlowestTestsAndTheVerdictIsNotChanged(t *testing.T) { // 1300/AC3
cs, p := lineCase(t, pullOf(repoA, 7, head1))
v := passVerdict()
v.OverBound = []string{"unit"}
line := Line(v, cs, p, nil, []string{"unit pkg.TestSlow 7.0s"}, nil)
if !strings.HasPrefix(line, "pass ") || !strings.Contains(Report(v, cs, p, nil, []string{"unit pkg.TestSlow 7.0s"}, nil), "over bound: unit (slowest unit pkg.TestSlow 7.0s)") {
t.Errorf("line = %q", line)
}
}
func TestSkippedTestsAreSaidAndNeverAFlatPass(t *testing.T) { // 1300/AC6
cs, p := lineCase(t, pullOf(repoA, 7, head1))
v := passVerdict()
v.Levels[buildstep.LevelUnit] = meshdeliverychecker.VerdictsByGroupHeadsValueLevelsValue{Run: 12, Kept: 301, Skipped: 2}
if line := Line(v, cs, p, nil, nil, nil); !strings.Contains(line, "unit 12 run, 301 kept, 2 skipped") {
t.Errorf("line = %q", line)
}
}
func TestAGroupsMembersEachCarryTheSameVerdict(t *testing.T) { // 1300/AC4
cs := caseOf(t, pullOf(repoA, 7, head1), pullOf(repoSDK, 12, head2))
a := Line(passVerdict(), cs, cs.Pulls[0], nil, nil, nil)
b := Line(passVerdict(), cs, cs.Pulls[1], nil, nil, nil)
if !strings.HasPrefix(a, "pass ") || !strings.HasPrefix(b, "pass ") || !strings.Contains(Report(passVerdict(), cs, cs.Pulls[0], nil, nil, nil), "group feat/x of 2 pull requests") {
t.Errorf("lines %q and %q", a, b)
}
if !strings.Contains(a, "111111111111") || !strings.Contains(b, "222222222222") {
t.Errorf("each member's line names its own head: %q and %q", a, b)
}
}
func TestTheBuildsOfTheTouchedModulesAreSaidAndDroppedFirstWhenTheLineIsTooLong(t *testing.T) { // ADR 0484 clause 7
cs, p := lineCase(t, pullOf(repoA, 7, head1))
builds := []BuildNote{{"avahi", "built"}, {"mesh-tools", "waiting"}}
if r := Report(passVerdict(), cs, p, builds, nil, nil); !strings.Contains(r, "builds: avahi built, mesh-tools waiting") {
t.Errorf("report = %q", r)
}
line := Line(passVerdict(), cs, p, builds, nil, nil)
if utf8.RuneCountInString(line) > MaxLine {
t.Errorf("line %q is too long", line)
}
short := passVerdict()
short.Took = &meshdeliverychecker.VerdictsByGroupHeadsValueTook{}
if l := Line(short, cs, p, builds[:1], nil, nil); !strings.Contains(l, "builds: avahi built") {
t.Errorf("a short line keeps its builds: %q", l)
}
}
func TestALineIsNeverLongerThanTheLimit(t *testing.T) { // 1300/AC1
cs, p := lineCase(t, pullOf(repoA, 7, head1))
v := meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail, Failed: &FailedFirst{Test: "integration: " + strings.Repeat("pkg/", 50) + ".TestSomething", Words: strings.Repeat("because ", 40)}}
if line := Line(v, cs, p, nil, nil, nil); utf8.RuneCountInString(line) > MaxLine || !strings.HasPrefix(line, "failed: integration: ") {
t.Errorf("line = %q (%d)", line, utf8.RuneCountInString(line))
}
}
func TestARunningCheckIsPending(t *testing.T) {
cs, p := lineCase(t, pullOf(repoA, 7, head1))
if line := Line(meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictRunning}, cs, p, nil, nil, nil); line != "checking 111111111111 on aaaaaaaaaaaa" {
t.Errorf("line = %q", line)
}
}
func TestThePassedCompositionIsSaidOnlyWhenItWasRun(t *testing.T) { // ADR 0484 clause 6
cs, p := lineCase(t, pullOf(repoA, 7, head1))
v := passVerdict()
v.Composition = &meshdeliverychecker.VerdictsByGroupHeadsValueComposition{State: meshdeliverycheckerclient.CompositionPass, Sdk: "afbe371003a570f088be4a394d83aa7a9458f7ab"}
if r := Report(v, cs, p, nil, nil, nil); !strings.Contains(r, "composition passed with afbe371003a5") {
t.Errorf("report = %q", r)
}
v.Composition = nil
if r := Report(v, cs, p, nil, nil, nil); !strings.Contains(r, "composition not yet run") || strings.Contains(r, "composition passed") {
t.Errorf("a verdict with no composition: %q", r)
}
}
func TestTheCompositionsTimeAndMachinesAreSaidOnceTheAssignerReportsThem(t *testing.T) { // 1300 (C10), the assigner's verdict (issue 1297)
cs, p := lineCase(t, pullOf(repoA, 7, head1))
v := passVerdict()
v.Took.Composition = 2400
v.Composition = &meshdeliverychecker.VerdictsByGroupHeadsValueComposition{State: meshdeliverycheckerclient.CompositionPass, Sdk: "afbe371003a570f088be4a394d83aa7a9458f7ab", Machines: []string{"g14", "ace", "novox", "shanks"}}
r := Report(v, cs, p, nil, nil, nil)
for _, want := range []string{"composition passed in 2.4s with afbe371003a5 on 4 machines: ace, g14, novox, shanks"} {
if !strings.Contains(r, want) {
t.Errorf("report %q lacks %q", r, want)
}
}
if l := Line(v, cs, p, nil, nil, nil); !strings.Contains(l, "composition passed in 2.4s") || !strings.Contains(l, "on 4 machines") || utf8.RuneCountInString(l) > MaxLine {
t.Errorf("line = %q", l)
}
v.State, v.Failed = meshdeliverycheckerclient.VerdictFail, &FailedFirst{Machine: "novox", Words: "composition refused"}
v.Composition = &meshdeliverychecker.VerdictsByGroupHeadsValueComposition{State: meshdeliverycheckerclient.CompositionFail, Words: "novox: two seats claim dns", Machines: []string{"novox"}}
if r := Report(v, cs, p, nil, nil, nil); !strings.Contains(r, "composition failed in 2.4s on novox: novox: two seats claim dns") {
t.Errorf("failed composition report = %q", r)
// The status line and its full report say each part of a verdict, within the forge's limit (issue 1300, ADR 0484).
func TestTheLineAndTheReportSayEachPartOfTheVerdict(t *testing.T) {
for _, row := range []struct {
name string
test func(t *testing.T)
}{
{"a level over its bound is named with its slowest tests and the verdict is not changed", func(t *testing.T) { // 1300/AC3
cs, p := lineCase(t, pullOf(repoA, 7, head1))
v := passVerdict()
v.OverBound = []string{"unit"}
line := Line(v, cs, p, nil, []string{"unit pkg.TestSlow 7.0s"}, nil)
if !strings.HasPrefix(line, "pass ") || !strings.Contains(Report(v, cs, p, nil, []string{"unit pkg.TestSlow 7.0s"}, nil), "over bound: unit (slowest unit pkg.TestSlow 7.0s)") {
t.Errorf("line = %q", line)
}
}},
{"skipped tests are said and never a flat pass", func(t *testing.T) { // 1300/AC6
cs, p := lineCase(t, pullOf(repoA, 7, head1))
v := passVerdict()
v.Levels[buildstep.LevelUnit] = meshdeliverychecker.VerdictsByGroupHeadsValueLevelsValue{Run: 12, Kept: 301, Skipped: 2}
if line := Line(v, cs, p, nil, nil, nil); !strings.Contains(line, "unit 12 run, 301 kept, 2 skipped") {
t.Errorf("line = %q", line)
}
}},
{"a groups members each carry the same verdict", func(t *testing.T) { // 1300/AC4
cs := caseOf(t, pullOf(repoA, 7, head1), pullOf(repoSDK, 12, head2))
a := Line(passVerdict(), cs, cs.Pulls[0], nil, nil, nil)
b := Line(passVerdict(), cs, cs.Pulls[1], nil, nil, nil)
if !strings.HasPrefix(a, "pass ") || !strings.HasPrefix(b, "pass ") || !strings.Contains(Report(passVerdict(), cs, cs.Pulls[0], nil, nil, nil), "group feat/x of 2 pull requests") {
t.Errorf("lines %q and %q", a, b)
}
if !strings.Contains(a, "111111111111") || !strings.Contains(b, "222222222222") {
t.Errorf("each member's line names its own head: %q and %q", a, b)
}
}},
{"the builds of the touched modules are said and dropped first when the line is too long", func(t *testing.T) { // ADR 0484 clause 7
cs, p := lineCase(t, pullOf(repoA, 7, head1))
builds := []BuildNote{{"avahi", "built"}, {"mesh-tools", "waiting"}}
if r := Report(passVerdict(), cs, p, builds, nil, nil); !strings.Contains(r, "builds: avahi built, mesh-tools waiting") {
t.Errorf("report = %q", r)
}
line := Line(passVerdict(), cs, p, builds, nil, nil)
if utf8.RuneCountInString(line) > MaxLine {
t.Errorf("line %q is too long", line)
}
short := passVerdict()
short.Took = &meshdeliverychecker.VerdictsByGroupHeadsValueTook{}
if l := Line(short, cs, p, builds[:1], nil, nil); !strings.Contains(l, "builds: avahi built") {
t.Errorf("a short line keeps its builds: %q", l)
}
}},
{"a line is never longer than the limit", func(t *testing.T) { // 1300/AC1
cs, p := lineCase(t, pullOf(repoA, 7, head1))
v := meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail, Failed: &FailedFirst{Test: "integration: " + strings.Repeat("pkg/", 50) + ".TestSomething", Words: strings.Repeat("because ", 40)}}
if line := Line(v, cs, p, nil, nil, nil); utf8.RuneCountInString(line) > MaxLine || !strings.HasPrefix(line, "failed: integration: ") {
t.Errorf("line = %q (%d)", line, utf8.RuneCountInString(line))
}
}},
{"a running check is pending", func(t *testing.T) {
cs, p := lineCase(t, pullOf(repoA, 7, head1))
if line := Line(meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictRunning}, cs, p, nil, nil, nil); line != "checking 111111111111 on aaaaaaaaaaaa" {
t.Errorf("line = %q", line)
}
}},
{"the passed composition is said only when it was run", func(t *testing.T) { // ADR 0484 clause 6
cs, p := lineCase(t, pullOf(repoA, 7, head1))
v := passVerdict()
v.Composition = &meshdeliverychecker.VerdictsByGroupHeadsValueComposition{State: meshdeliverycheckerclient.CompositionPass, Sdk: "afbe371003a570f088be4a394d83aa7a9458f7ab"}
if r := Report(v, cs, p, nil, nil, nil); !strings.Contains(r, "composition passed with afbe371003a5") {
t.Errorf("report = %q", r)
}
v.Composition = nil
if r := Report(v, cs, p, nil, nil, nil); !strings.Contains(r, "composition not yet run") || strings.Contains(r, "composition passed") {
t.Errorf("a verdict with no composition: %q", r)
}
}},
{"the compositions time and machines are said once the assigner reports them", func(t *testing.T) { // 1300 (C10), the assigner's verdict (issue 1297)
cs, p := lineCase(t, pullOf(repoA, 7, head1))
v := passVerdict()
v.Took.Composition = 2400
v.Composition = &meshdeliverychecker.VerdictsByGroupHeadsValueComposition{State: meshdeliverycheckerclient.CompositionPass, Sdk: "afbe371003a570f088be4a394d83aa7a9458f7ab", Machines: []string{"g14", "ace", "novox", "shanks"}}
r := Report(v, cs, p, nil, nil, nil)
for _, want := range []string{"composition passed in 2.4s with afbe371003a5 on 4 machines: ace, g14, novox, shanks"} {
if !strings.Contains(r, want) {
t.Errorf("report %q lacks %q", r, want)
}
}
if l := Line(v, cs, p, nil, nil, nil); !strings.Contains(l, "composition passed in 2.4s") || !strings.Contains(l, "on 4 machines") || utf8.RuneCountInString(l) > MaxLine {
t.Errorf("line = %q", l)
}
v.State, v.Failed = meshdeliverycheckerclient.VerdictFail, &FailedFirst{Machine: "novox", Words: "composition refused"}
v.Composition = &meshdeliverychecker.VerdictsByGroupHeadsValueComposition{State: meshdeliverycheckerclient.CompositionFail, Words: "novox: two seats claim dns", Machines: []string{"novox"}}
if r := Report(v, cs, p, nil, nil, nil); !strings.Contains(r, "composition failed in 2.4s on novox: novox: two seats claim dns") {
t.Errorf("failed composition report = %q", r)
}
}},
} {
t.Run(row.name, row.test)
}
}
@@ -60,7 +60,7 @@ func (l *losingAsks) PublishAsk(ctx context.Context, subject, id string, data []
func losing(b *bench, lose int) *losingAsks {
l := &losingAsks{FakeAsks: &buildrun.FakeAsks{}, answer: b.answer, dead: &buildrun.DeadLetters{}, lose: lose}
b.ex.Asks = buildrun.Asker{Publisher: l, Waiter: buildrun.LostAskWaiter{Inner: l.FakeAsks, Dead: l.dead}, Bound: 5 * time.Second}
b.ex.Asks = buildrun.Asker{Publisher: b.asker(l, l.FakeAsks.Write), Waiter: buildrun.LostAskWaiter{Inner: l.FakeAsks, Dead: l.dead}, Bound: 5 * time.Second}
return l
}
@@ -285,7 +285,7 @@ func TestACheckWhoseAskJudgedNothingIsAskedAgainOrRead(t *testing.T) {
tr = &troubled{FakeAsks: &buildrun.FakeAsks{}, answer: b.answer, restarts: row.restarts, deaf: row.deaf,
deafIDs: map[string]bool{}, steps: map[string]buildstep.Step{}}
heard = &buildrun.HeardEnds{Inner: tr, Read: tr}
b.ex.Asks = buildrun.Asker{Publisher: tr, Waiter: heard, Bound: 5 * time.Second}
b.ex.Asks = buildrun.Asker{Publisher: b.asker(tr, tr.Write), Waiter: heard, Bound: 5 * time.Second}
asked = func() int { return len(tr.Asks()) }
}
r := newRig(t, b.ex)
@@ -6,11 +6,12 @@ import (
"errors"
"fmt"
"maps"
"mesh-delivery-checker/rules"
"slices"
"strings"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
"git.novox.be/novox/mesh-delivery/sdk/composer"
"git.novox.be/novox/mesh-sdk/go/catalogue"
@@ -35,7 +36,7 @@ const (
JudgeCase = "change"
JudgeTrunk = "trunk"
// judgeBetween is how the judge names the problems between manifests that no one module's removal ends.
judgeBetween = "(between manifests)"
judgeBetween = rules.BetweenManifests
)
// judgePaths are the paths of mesh-delivery whose change changes the judge: a group whose mesh-delivery member changes
@@ -104,16 +105,17 @@ func judgeRequest(trunk map[string]string, pulls []Pull) (req composer.Request,
{Name: JudgeTrunk, Manifests: before}}}, slices.Sorted(maps.Keys(seen))
}
// judgeManifests runs the indexer's judge over the case's manifests and fails the check for each touched manifest it
// refuses. A case that touches no manifest, or an examiner that reads no trunk manifests, judges nothing; a judge that
// cannot be asked or does not answer fails the check, said, never passes it.
func (st *state) judgeManifests(ctx context.Context) *meshdeliverychecker.VerdictsByGroupHeadsValue {
// judgeManifests runs the indexer's judge over the case's manifests and records its answers as a fact the rules judge
// (rules.JudgeManifests), with the trunk validator's answer on a new indexer (ADR 0633 clause 6). A case that touches no
// manifest, or an examiner that reads no trunk manifests, judges nothing; a judge that cannot be asked or does not answer
// is a failure, said, never a pass.
func (st *state) judgeManifests(ctx context.Context) {
if st.j.Manifests == nil || st.narrowTry() {
return nil
return
}
req, touched := judgeRequest(st.j.Manifests.ManifestDigests(), st.c.Pulls)
if len(touched) == 0 {
return nil
return
}
commit, at := st.judgeHead, "the group's mesh-delivery head "+st.judgeHead
if commit == "" {
@@ -123,7 +125,8 @@ func (st *state) judgeManifests(ctx context.Context) *meshdeliverychecker.Verdic
at = "mesh-delivery's trunk " + commit
}
if commit == "" {
return fail(goModule, "the change's manifests cannot be judged: the trunk head of "+DeliveryRepository+" is not known")
st.fail(goModule, "the change's manifests cannot be judged: the trunk head of "+DeliveryRepository+" is not known")
return
}
// The list of predefined keys the manifests are judged against (ADR 0578 §2): the catalogue member's predefined.json
// when the change alters it, else the one the indexer keeps.
@@ -132,80 +135,27 @@ func (st *state) judgeManifests(ctx context.Context) *meshdeliverychecker.Verdic
st.c.note("the manifests are judged against the predefined keys of " + from)
}
req.Packages = st.knownPackages(ctx)
if st.j.SeatList != nil {
seats, err := st.j.SeatList.Definitions()
if err != nil {
return fail(goModule, "the change's manifests cannot be judged: the seat list is not read ("+err.Error()+")")
}
req.Seats = seats
// The heads' seat list, for the change case and the trunk case both (ADR 0636 §4).
req.Seats = st.c.Seats
cv, trunkCase, ok := st.runJudge(ctx, req, touched, commit, at, st.packagesOf(DeliveryRepository))
if !ok {
return
}
// The members' heads' definitions over the list's (ADR 0636 §4): the trunk's manifest validator of a new indexer as well.
heads, err := st.headSeats(ctx)
if err != nil {
return fail(goModule, "the change's manifests cannot be judged: "+err.Error())
st.judgedVerdicts(cv, at)
m := rules.Manifests{Touched: touched, At: at, Change: cv, Trunk: trunkCase}
if !st.indexerOf(ctx, req, touched, cv, at, &m) {
return
}
req.Seats = overSeats(req.Seats, heads)
cv, trunkCase, v := st.runJudge(ctx, req, touched, commit, at, st.packagesOf(DeliveryRepository))
if v != nil {
return v
}
if v := st.refuseUnmovableIndexer(ctx, req, touched, cv, at); v != nil {
return v
}
// A new or changed manifest is judged in full; of one the change leaves as it is, a problem the trunk already had is a
// warning (it is not refused for a rule it already broke), and a problem the change causes in it, which the trunk case
// lacks, fails the check as the change's (ADR 0579 §2).
var refused, warned []string
for _, module := range slices.Sorted(maps.Keys(cv.Manifests)) {
r := cv.Manifests[module]
if r.Accepted {
continue
}
if r.Refused == "" { // held: a verdict not given is not one given (ADR 0633 clause 6)
if slices.Contains(touched, module) {
refused = append(refused, fmt.Sprintf("the indexer's judge at %s holds the manifest of %s: it gave no verdict", at, module))
}
continue
}
had := map[string]bool{}
for _, p := range strings.Split(trunkCase.Manifests[module].Refused, "\n") {
had[p] = true
}
var caused, old []string
for _, p := range strings.Split(r.Refused, "\n") {
if slices.Contains(touched, module) || !had[p] {
caused = append(caused, p)
} else {
old = append(old, p)
}
}
if len(caused) > 0 {
what := "the manifest of " + module
if module == judgeBetween {
what = "the catalogue"
}
refused = append(refused, fmt.Sprintf("the indexer's judge at %s refuses %s: %s", at, what, strings.Join(caused, "; ")))
}
if len(old) > 0 {
warned = append(warned, module+": "+strings.Join(old, "; "))
}
}
if len(warned) > 0 {
st.c.note("warnings of the indexer's judge on manifests the change leaves as they are (ADR 0579 §2): " + strings.Join(warned, " | "))
}
if len(refused) > 0 {
return fail(goModule, strings.Join(refused, "\n"))
}
st.c.note(fmt.Sprintf("the manifests of %s are accepted by the indexer's judge at %s", strings.Join(touched, ", "), at))
return nil
st.facts = append(st.facts, rules.Step{Module: goModule, Manifests: &m})
}
// runJudge asks the Go build seat to build the judge at the commit and run it over the request, and reads its change and
// trunk cases; a failed verdict when it cannot be asked or does not answer.
func (st *state) runJudge(ctx context.Context, req composer.Request, touched []string, commit, at string, versions buildstep.PackageVersions) (change, trunk composer.CaseVerdict, v *meshdeliverychecker.VerdictsByGroupHeadsValue) {
// trunk cases; false, the failure recorded, when it cannot be asked or does not answer.
func (st *state) runJudge(ctx context.Context, req composer.Request, touched []string, commit, at string, versions buildstep.PackageVersions) (change, trunk composer.CaseVerdict, ok bool) {
raw, err := json.Marshal(req)
if err != nil {
return change, trunk, fail(goModule, err.Error())
st.fail(goModule, err.Error())
return change, trunk, false
}
job := buildstep.ComposeJob{Fetch: buildstep.Fetch{Repository: DeliveryRepository, Commit: commit, Packages: versions}, Dir: JudgeDir,
Package: JudgePackage, Request: raw, Reader: "the indexer's judge at " + at}
@@ -214,48 +164,48 @@ func (st *state) runJudge(ctx context.Context, req composer.Request, touched []s
st.c.Progress.Set(fmt.Sprintf("the judge ask %s of the manifests of %s", a.ID, strings.Join(touched, ", ")), "")
st.c.Asked(a.ID)
if err := st.j.Asks.Put(ctx, a); err != nil {
return change, trunk, fail(goModule, "the judge of the change's manifests could not be asked: "+err.Error())
st.fail(goModule, "the judge of the change's manifests could not be asked: "+err.Error())
return change, trunk, false
}
step, err := st.j.Asks.Wait(ctx, a)
if err != nil {
return change, trunk, fail(goModule, "the judge of the change's manifests: "+err.Error())
st.fail(goModule, "the judge of the change's manifests: "+err.Error())
return change, trunk, false
}
change, trunk, why := judgeAnswer(step)
if why != "" {
return change, trunk, fail(goModule, fmt.Sprintf("the indexer's judge at %s did not judge the change's manifests: %s", at, why))
st.fail(goModule, fmt.Sprintf("the indexer's judge at %s did not judge the change's manifests: %s", at, why))
return change, trunk, false
}
return change, trunk, nil
return change, trunk, true
}
// CatalogSeat is the seat the indexer holds: the module whose manifest claims it is the indexer, found by the claim and
// never by its name (novox/hq ADR 0633 clause 6, ADR 0601 §4).
const CatalogSeat = "mesh-catalog"
// CatalogSeat is the seat the indexer holds (rules.CatalogSeat).
const CatalogSeat = rules.CatalogSeat
// refuseUnmovableIndexer fails the check when the trunk's manifest validator, the one the running parts hold, would
// refuse or hold the change's new indexer manifest: such an indexer would never run, and every revision behind it would
// wait for ever (ADR 0633 clause 6). The indexer is the touched module whose manifest claims the seat mesh-catalog.
// When the change judges with its own head's manifest validator (it changes it), the manifest is judged by the trunk's
// as well. The verdicts read are the trunk's manifest validator's: its change case, where refused, held (not accepted
// with no reason) or with a field set aside (a field the running manifest validator does not know waits, ADR 0601 §3)
// each fail it, naming the module and the problems.
func (st *state) refuseUnmovableIndexer(ctx context.Context, req composer.Request, touched []string, cv composer.CaseVerdict, judgedAt string) *meshdeliverychecker.VerdictsByGroupHeadsValue {
// indexerOf finds the touched module whose manifest claims the seat mesh-catalog, the indexer, and puts in m the trunk's
// manifest validator's change case on it, the one the running parts hold (ADR 0633 clause 6): when the change judges
// with its own head's validator, or at other package versions than the latest published, the trunk's is asked as well.
// False, the failure recorded, when a manifest cannot be read or the trunk's validator cannot be asked.
func (st *state) indexerOf(ctx context.Context, req composer.Request, touched []string, cv composer.CaseVerdict, judgedAt string, m *rules.Manifests) bool {
if st.j.Read == nil {
return nil
return true
}
var indexer string
for _, module := range touched {
m, err := st.j.Read.Manifest(ctx, req.Cases[0].Manifests[module].Digest)
man, err := st.j.Read.Manifest(ctx, req.Cases[0].Manifests[module].Digest)
if err != nil {
return fail(goModule, fmt.Sprintf("the manifest of %s cannot be read, so the indexer cannot be ruled out: %v", module, err))
st.fail(goModule, fmt.Sprintf("the manifest of %s cannot be read, so the indexer cannot be ruled out: %v", module, err))
return false
}
for _, c := range m.Claims {
for _, c := range man.Claims {
if c.Name == CatalogSeat {
indexer = module
}
}
}
if indexer == "" {
return nil
return true
}
// The validator the running parts hold is the trunk's, built at the latest published versions alone, never at the group's
// heads (ADR 0633 clause 6): a group's mesh-sdk or mesh-contracts head must not change the validator that judges the new
@@ -266,32 +216,17 @@ func (st *state) refuseUnmovableIndexer(ctx context.Context, req composer.Reques
trunk = st.j.Trunks.Heads()[DeliveryRepository]
}
if trunk == "" {
return fail(goModule, "the new indexer cannot be judged by the trunk's manifest validator: the trunk head of "+DeliveryRepository+" is not known")
st.fail(goModule, "the new indexer cannot be judged by the trunk's manifest validator: the trunk head of "+DeliveryRepository+" is not known")
return false
}
var v *meshdeliverychecker.VerdictsByGroupHeadsValue
judgedAt = "mesh-delivery's trunk " + trunk
if cv, _, v = st.runJudge(ctx, req, touched, trunk, judgedAt, trunkVersions); v != nil {
return v
var ok bool
if cv, _, ok = st.runJudge(ctx, req, touched, trunk, judgedAt, trunkVersions); !ok {
return false
}
}
r, ok := cv.Manifests[indexer]
var problems []string
switch {
case !ok:
problems = append(problems, "the manifest validator gave no verdict")
case r.Refused != "":
problems = append(problems, strings.Split(r.Refused, "\n")...)
case !r.Accepted:
problems = append(problems, "the manifest validator holds it, with no verdict yet")
case r.SetAside != "":
problems = append(problems, "the manifest validator does not know its fields "+r.SetAside+", so it waits")
}
if len(problems) > 0 {
return fail(goModule, fmt.Sprintf("the indexer %s, the holder of the seat %s, would never run: the trunk's manifest validator at %s does not accept its manifest: %s",
indexer, CatalogSeat, judgedAt, strings.Join(problems, "; ")))
}
st.c.note(fmt.Sprintf("the new indexer's manifest of %s is accepted by the trunk's manifest validator at %s", indexer, judgedAt))
return nil
m.Indexer, m.IndexerAt, m.OfIndexer = indexer, judgedAt, &cv
return true
}
// judgeAnswer reads the judge's change and trunk cases from its step, or why it cannot.
@@ -435,3 +370,29 @@ func (st *state) knownPackages(ctx context.Context) []string {
func (st *state) trunkJudgeVersions() buildstep.PackageVersions {
return packagesFor(st.j.Published, &Case{}, nil, DeliveryRepository)
}
// judgedVerdicts puts the check's own judge's verdict on each manifest the change brings on the case, as the seat
// mesh-catalog's verdict value, for the composer's request (ADR 0578 §2, §3; F5 of milestone 47/G): accepted, or refused
// with its words; a manifest the judge gave no verdict on carries none, which the composer reads as accepted.
func (st *state) judgedVerdicts(cv composer.CaseVerdict, at string) {
for _, p := range st.c.Pulls {
for name, fp := range p.Fps {
read, ok := cv.Manifests[name]
if fp.Manifest == "" || !ok || (!read.Accepted && read.Refused == "") {
continue
}
v := meshcatalog.VerdictsByDigestValue{Module: name, State: meshcatalogclient.VerdictAccepted, Judge: at}
if !read.Accepted {
v.State, v.Problems = meshcatalogclient.VerdictRefused, []string{read.Refused}
}
raw, err := json.Marshal(v)
if err != nil {
continue
}
if st.c.Judged == nil {
st.c.Judged = map[string]json.RawMessage{}
}
st.c.Judged[name] = raw
}
}
}
@@ -1,6 +1,7 @@
package check
import (
"mesh-delivery-checker/rules"
"strings"
"testing"
)
@@ -19,75 +20,81 @@ const (
oldPassThenRace = "PASS\n==================\nWARNING: DATA RACE\nWrite at 0x000000836428 by goroutine 9:\n example.com/racemain.TestMain.func1()\n /tmp/fix-1901b/racemain/p_test.go:15 +0x24\nPrevious write at 0x000000836428 by main goroutine:\n example.com/racemain.TestMain()\n /tmp/fix-1901b/racemain/p_test.go:16 +0x44\n main.main()\n _testmain.go:48 +0x171\nGoroutine 9 (running) created at:\n example.com/racemain.TestMain()\n /tmp/fix-1901b/racemain/p_test.go:15 +0x37\n main.main()\n _testmain.go:48 +0x171\n==================\nFound 1 data race(s)\nFAIL\texample.com/racemain\t1.031s"
)
// A failure with no cause told apart never has "PASS" or "ok" as its words: the checker takes the panic line from the
// output, or says that the tests passed and the program failed after them (1901/AC5).
func TestAPackageFailureIsNeverWordedPass(t *testing.T) { // 1901/AC5
cases := []struct {
name, output, want string
// A package's failure is worded by its cause, cut to its bound, and never as pass (issue 1901, AC5).
func TestAPackageFailuresCauseIsWordedNeverPass(t *testing.T) {
for _, row := range []struct {
name string
test func(t *testing.T)
}{
{"panic after the tests", oldPassThenPanic, "panic: the bus closed after the tests: nats: connection closed"},
{"exit after the tests", oldPassThenExit, "the tests passed, then the test program exited with a failure"},
{"race after the tests", oldPassThenRace, "WARNING: DATA RACE: Write at 0x000000836428 by goroutine 9 in example.com/racemain.TestMain.func1()"},
{"a panic deep in a long output", "PASS\npanic: the seat row vanished\n" + strings.Repeat("goroutine 7 [chan receive]:\n\tsome/frame.go:1 +0x1\n", 300) + "FAIL\tp\t2.0s", "panic: the seat row vanished"},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
v := levelFailure(&target{}, "integration", Tested{Failed: []Failure{{Package: "github.com/novox/mesh-controller/cmd/mesh-controller", Output: c.output}}})
for _, l := range strings.Split(v.Failed.Words, "\n") {
if l = strings.TrimSpace(l); l == "PASS" || strings.HasPrefix(l, "ok ") || strings.HasPrefix(l, "ok\t") {
t.Errorf("1901/AC5: the words keep %q: %q", l, v.Failed.Words)
// A failure with no cause told apart never has "PASS" or "ok" as its words: the checker takes the panic line from the
// output, or says that the tests passed and the program failed after them (1901/AC5).
{"a package failure is never worded pass", func(t *testing.T) { // 1901/AC5
cases := []struct {
name, output, want string
}{
{"panic after the tests", oldPassThenPanic, "panic: the bus closed after the tests: nats: connection closed"},
{"exit after the tests", oldPassThenExit, "the tests passed, then the test program exited with a failure"},
{"race after the tests", oldPassThenRace, "WARNING: DATA RACE: Write at 0x000000836428 by goroutine 9 in example.com/racemain.TestMain.func1()"},
{"a panic deep in a long output", "PASS\npanic: the seat row vanished\n" + strings.Repeat("goroutine 7 [chan receive]:\n\tsome/frame.go:1 +0x1\n", 300) + "FAIL\tp\t2.0s", "panic: the seat row vanished"},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
v := rules.LevelFailure("", "", "integration", Tested{Failed: []Failure{{Package: "github.com/novox/mesh-controller/cmd/mesh-controller", Output: c.output}}})
for _, l := range strings.Split(v.Failed.Words, "\n") {
if l = strings.TrimSpace(l); l == "PASS" || strings.HasPrefix(l, "ok ") || strings.HasPrefix(l, "ok\t") {
t.Errorf("1901/AC5: the words keep %q: %q", l, v.Failed.Words)
}
}
if !strings.HasPrefix(afterHid(v.Failed.Words), c.want) {
t.Errorf("1901/AC5: words after the loud line start %q, want %q", rules.FirstLine(afterHid(v.Failed.Words)), c.want)
}
if got := firstFailed(*v); !strings.Contains(got, "the build module hid the failure") || strings.HasSuffix(got, ": PASS") {
t.Errorf("1901/AC5, AC8: status = %q", got)
}
})
}
}},
// A cause the build module told apart is cut to a line's bound, so the words keep their bound (the 1901 review's note).
{"a long cause is cut to a lines bound", func(t *testing.T) { // 1901/AC5
f := Failure{Package: "p", Test: "TestX", Cause: strings.Repeat("c", 5000), Output: "x_test.go:1: something"}
v := rules.LevelFailure("", "", "unit", Tested{Failed: []Failure{f}})
if len(v.Failed.Words) > rules.MaxWords+200 {
t.Errorf("1901/AC5: words are %d bytes, bound %d", len(v.Failed.Words), rules.MaxWords)
}
}},
// The build module on main sends a package that stopped (a timeout, an exit, a kill) as only go test's last line, with no
// cause: the words then say so, never nothing (review round 1). Each output is what that build module sends for the
// capture of the same name (go-test-json-pkg-timeout, -exiting, -killed): the package's own output, its "FAIL" line.
{"a package stopped with only its fail line is still worded", func(t *testing.T) { // 1901/AC5
for _, out := range []string{"FAIL\texample.com/timeout\t2.017s", "FAIL\texample.com/exiting\t0.006s", "FAIL\texample.com/killed\t0.006s"} {
v := rules.LevelFailure("", "", "integration", Tested{Failed: []Failure{{Package: "example.com/x", Output: out}}})
if !strings.HasPrefix(afterHid(v.Failed.Words), "the package failed with no test to name and no panic, signal or build error in its output") {
t.Errorf("1901/AC5: %q is worded %q", out, v.Failed.Words)
}
}
if !strings.HasPrefix(afterHid(v.Failed.Words), c.want) {
t.Errorf("1901/AC5: words after the loud line start %q, want %q", firstLine(afterHid(v.Failed.Words)), c.want)
}},
// A line of the test's own that starts as go's summary does, or that holds a runtime's words indented, is not taken for
// either (review round 1): go prints its summary in exact forms and the runtime's lines at column 0.
{"only gos exact forms are summary or cause", func(t *testing.T) { // 1901/AC5
out := " x_test.go:3: ok so far\nok then: the seat is free\n panic: child retried and recovered\npanic: the real one\nFAIL\tp\t1.0s"
v := rules.LevelFailure("", "", "unit", Tested{Failed: []Failure{{Package: "p", Output: out}}})
if !strings.HasPrefix(afterHid(v.Failed.Words), "panic: the real one") || !strings.Contains(v.Failed.Words, "ok then: the seat is free") {
t.Errorf("1901/AC5: words = %q", v.Failed.Words)
}
if got := firstFailed(*v); !strings.Contains(got, "the build module hid the failure") || strings.HasSuffix(got, ": PASS") {
t.Errorf("1901/AC5, AC8: status = %q", got)
}},
// The words keep rules.MaxWords exactly, counting the cause, the marker and the newlines (review round 1).
{"the words keep their bound exactly", func(t *testing.T) { // 1901/AC5
for _, cause := range []string{"", "a cause", strings.Repeat("c", 3000)} {
for _, n := range []int{100, 170, 171, 172, 500} {
f := Failure{Package: "p", Test: "TestX", Cause: cause, Output: strings.Repeat("a log line of twenty\n", n) + "the end"}
if w := rules.FailureWords(f); len(w) > rules.MaxWords {
t.Errorf("1901/AC5: cause %d bytes, %d lines: words are %d bytes, bound %d", len(cause), n, len(w), rules.MaxWords)
}
}
}
})
}
}
// A cause the build module told apart is cut to a line's bound, so the words keep their bound (the 1901 review's note).
func TestALongCauseIsCutToALinesBound(t *testing.T) { // 1901/AC5
f := Failure{Package: "p", Test: "TestX", Cause: strings.Repeat("c", 5000), Output: "x_test.go:1: something"}
v := levelFailure(&target{}, "unit", Tested{Failed: []Failure{f}})
if len(v.Failed.Words) > MaxWords+200 {
t.Errorf("1901/AC5: words are %d bytes, bound %d", len(v.Failed.Words), MaxWords)
}
}
// The build module on main sends a package that stopped (a timeout, an exit, a kill) as only go test's last line, with no
// cause: the words then say so, never nothing (review round 1). Each output is what that build module sends for the
// capture of the same name (go-test-json-pkg-timeout, -exiting, -killed): the package's own output, its "FAIL" line.
func TestAPackageStoppedWithOnlyItsFailLineIsStillWorded(t *testing.T) { // 1901/AC5
for _, out := range []string{"FAIL\texample.com/timeout\t2.017s", "FAIL\texample.com/exiting\t0.006s", "FAIL\texample.com/killed\t0.006s"} {
v := levelFailure(&target{}, "integration", Tested{Failed: []Failure{{Package: "example.com/x", Output: out}}})
if !strings.HasPrefix(afterHid(v.Failed.Words), "the package failed with no test to name and no panic, signal or build error in its output") {
t.Errorf("1901/AC5: %q is worded %q", out, v.Failed.Words)
}
}
}
// A line of the test's own that starts as go's summary does, or that holds a runtime's words indented, is not taken for
// either (review round 1): go prints its summary in exact forms and the runtime's lines at column 0.
func TestOnlyGosExactFormsAreSummaryOrCause(t *testing.T) { // 1901/AC5
out := " x_test.go:3: ok so far\nok then: the seat is free\n panic: child retried and recovered\npanic: the real one\nFAIL\tp\t1.0s"
v := levelFailure(&target{}, "unit", Tested{Failed: []Failure{{Package: "p", Output: out}}})
if !strings.HasPrefix(afterHid(v.Failed.Words), "panic: the real one") || !strings.Contains(v.Failed.Words, "ok then: the seat is free") {
t.Errorf("1901/AC5: words = %q", v.Failed.Words)
}
}
// The words keep MaxWords exactly, counting the cause, the marker and the newlines (review round 1).
func TestTheWordsKeepTheirBoundExactly(t *testing.T) { // 1901/AC5
for _, cause := range []string{"", "a cause", strings.Repeat("c", 3000)} {
for _, n := range []int{100, 170, 171, 172, 500} {
f := Failure{Package: "p", Test: "TestX", Cause: cause, Output: strings.Repeat("a log line of twenty\n", n) + "the end"}
if w := failureWords(f); len(w) > MaxWords {
t.Errorf("1901/AC5: cause %d bytes, %d lines: words are %d bytes, bound %d", len(cause), n, len(w), MaxWords)
}
}
}},
} {
t.Run(row.name, row.test)
}
}
@@ -0,0 +1,291 @@
package check
import (
"context"
"encoding/json"
"fmt"
"maps"
"slices"
"strings"
"sync"
"time"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliverycheckerclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
"mesh-delivery-checker/rules"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun"
)
// The checking program (novox/hq ADR 0644 §1, ADR 0656 §5; design 47's bootstrap delivery; issue 1921): every check has
// the program built from the group's mesh-delivery member head, else from mesh-delivery's trunk, and run in the build
// seat's sandbox through the seat node-build-go's ask `run`, with every member's head laid for it. It answers the heads'
// seat list first (`-seats`), which the manifest validator's and the composer's requests carry, and then the check's
// verdict of the facts the asking gathered (`-verdict`). The running checker asks, keeps and records; it judges nothing.
const (
// InputsStateName is the checker's own state of run inputs and InputsBucket its bucket: every input by the sha256 of
// its exact bytes, read by the checker alone. InputVerb is the seat verb a run ask's input names
// (buildstep.RunInput.Verb), which the checker's tools serve from that state (Reader.Input): the Go build module asks it
// as its manifest's invokes grants it and checks the digest (buildstep.RunInput.Read), never reading the state (ADR 0596
// §2; the operator's answer on milestone 47/B0).
InputsStateName = "inputs"
InputsBucket = CheckerModule + "_" + InputsStateName
InputVerb = meshdeliverycheckerclient.InputVerb
// CheckingProgramDir and CheckingProgramPackage are where in mesh-delivery the checking program is built.
CheckingProgramDir = "modules/mesh-delivery-checker"
CheckingProgramPackage = "mesh-delivery-checker/cmd/mesh-checking-program"
// DefaultBeginBound is how long a run ask may wait for a build node to begin its step before it is put once more, and
// after that once more before the check fails naming the job: a build module that does not know the job `run` drops
// it unanswered (the lead's ruling on milestone 47/B0, 2026-11-06).
DefaultBeginBound = 5 * time.Minute
)
// Inputs keeps a run's input under its digest, exactly its bytes.
type Inputs interface {
Keep(ctx context.Context, digest string, body []byte) error
}
func (j *Examiner) beginBound() time.Duration {
if j.BeginBound > 0 {
return j.BeginBound
}
return DefaultBeginBound
}
// checkingProgram is where this check's checking program is built: the group's mesh-delivery member head, else
// mesh-delivery's trunk; false when neither is known.
func (st *state) checkingProgram() (buildstep.RunProgram, string, bool) {
p := buildstep.RunProgram{Repository: DeliveryRepository, Dir: CheckingProgramDir, Package: CheckingProgramPackage}
for _, pull := range st.c.Pulls {
if pull.Repository == DeliveryRepository {
p.Commit = pull.Head
return p, "the group's mesh-delivery head " + pull.Head, true
}
}
if st.j.Trunks != nil {
p.Commit = st.j.Trunks.Heads()[DeliveryRepository]
}
return p, "mesh-delivery's trunk " + p.Commit, p.Commit != ""
}
// runHeads are the members' heads, as the program reads them.
func (st *state) runHeads() []buildstep.RunHead {
out := make([]buildstep.RunHead, 0, len(st.c.Pulls))
for _, p := range st.c.Pulls {
out = append(out, buildstep.RunHead{Repository: p.Repository, Commit: p.Head})
}
return out
}
// runProgram has the checking program built and run with args over body, and answers its output and what judged; an
// error says why it did not answer: its input over its bound, no build node beginning the job, a step that did not
// pass, or a program that exited otherwise than 0.
func (st *state) runProgram(ctx context.Context, args []string, body []byte) (json.RawMessage, *meshdeliverychecker.VerdictsByGroupHeadsValueJudgedBy, error) {
prog, at, ok := st.checkingProgram()
if !ok {
return nil, nil, fmt.Errorf("it cannot be built: the trunk head of %s is not known", DeliveryRepository)
}
if st.j.Inputs == nil {
return nil, nil, fmt.Errorf("its input cannot be kept: the checker keeps no state %s", InputsStateName)
}
in, err := buildstep.RunInputOf(InputVerb, body)
if err != nil {
return nil, nil, err
}
if err := st.j.Inputs.Keep(ctx, in.Digest, body); err != nil {
return nil, nil, fmt.Errorf("its input could not be kept in %s: %w", InputsStateName, err)
}
job := buildstep.RunJob{Program: prog, Heads: st.runHeads(), Input: in, Args: args, Reader: "the checking program at " + at}
var ids []string
for range 2 {
a := buildstep.StepAsk{ID: buildstep.NewID(), Kind: buildstep.AskCheck, Language: buildstep.AskGo, Asker: CheckerModule,
At: st.j.now().UTC(), Packages: st.packagesOf(DeliveryRepository), Run: &job}
ids = append(ids, a.ID)
st.c.Progress.Set(fmt.Sprintf("the run ask %s of the checking program %s", a.ID, strings.Join(args, " ")), "")
st.c.Asked(a.ID)
if err := st.j.Asks.Put(ctx, a); err != nil {
return nil, nil, fmt.Errorf("its run ask could not be put: %w", err)
}
waiting := ctx
if watch := st.c.Watch; watch != nil {
id := a.ID
watch(id, nil) // a try names it at once (issue 1890)
waiting = buildrun.WithStepSeen(ctx, func(s buildstep.Step) { watch(id, &s) })
}
step, begun, err := st.waitBegun(waiting, a)
if !begun {
continue
}
if err != nil {
return nil, nil, fmt.Errorf("its run ask %s: %w", a.ID, err)
}
if step.State != buildstep.StepPassed {
return nil, nil, fmt.Errorf("its run ask %s ended %s on %s: %s", a.ID, step.State, step.Node, step.Reason)
}
var answer buildstep.RunAnswer
if err := json.Unmarshal(step.Result, &answer); err != nil {
return nil, nil, fmt.Errorf("its run ask %s answered no run answer: %w", a.ID, err)
}
if answer.Exit != 0 {
return nil, nil, fmt.Errorf("it exited %d on %s: %s", answer.Exit, step.Node, answer.Said)
}
return answer.Output, judgedByOf(prog, answer, job.Heads), nil
}
return nil, nil, fmt.Errorf("no build node began the job run of the asks %s within %s each: a build module that does not know the job run drops it",
strings.Join(ids, " and "), st.j.beginBound())
}
// waitBegun waits for a's step, and stops waiting, begun false, when no build node began it within the begin bound: a
// bound on the wait's context, no timer of its own.
func (st *state) waitBegun(ctx context.Context, a buildstep.StepAsk) (buildstep.Step, bool, error) {
wait, cancel := context.WithCancel(ctx)
defer cancel()
begin, stop := context.WithTimeout(ctx, st.j.beginBound())
defer stop()
seen := make(chan struct{})
var once sync.Once
type ended struct {
step buildstep.Step
err error
}
done := make(chan ended, 1)
go func() {
step, err := st.j.Asks.Wait(buildrun.WithStepSeen(wait, func(buildstep.Step) { once.Do(func() { close(seen) }) }), a)
done <- ended{step, err}
}()
select {
case e := <-done:
return e.step, true, e.err
case <-seen:
case <-begin.Done():
if ctx.Err() != nil {
return buildstep.Step{}, true, ctx.Err()
}
return buildstep.Step{}, false, nil
}
e := <-done
return e.step, true, e.err
}
// seatList asks the checking program for the heads' seat list, which the validator's and the composer's requests carry
// (ADR 0636 §4); the list and what it left out are the check's notes. A list it cannot make fails the check, naming why,
// and answers false: the check ends there, with no verdict asked of a program that could not answer its list (review
// round 1 of milestone 47/B0: a build node with no job run would hold the check two begin bounds more).
func (st *state) seatList(ctx context.Context) bool {
if st.narrowTry() {
return true
}
// The list the checker's reader holds from the seat mesh-catalog: a reader that has not read it acts on nothing new
// (ADR 0636 §1), so the check ends here, said.
var base []json.RawMessage
if st.j.SeatList != nil {
defs, err := st.j.SeatList.Definitions()
if err != nil {
st.fail(goModule, "the change's manifests cannot be judged: the seat list is not read ("+err.Error()+")")
return false
}
base = defs
}
// The program answers only the heads' definitions the list does not hold as they are: the list's digests ride in its
// input, never the list, whose definitions are larger than a run's input or answer may be (review finding 3 of G).
head := st.factsHead()
if len(base) > 0 {
head.SeatDigests = map[string]string{}
for _, raw := range base {
var d struct {
Seat string `json:"seat"`
}
if json.Unmarshal(raw, &d) == nil && d.Seat != "" {
head.SeatDigests[d.Seat] = rules.SeatDigest(raw)
}
}
}
body, err := json.Marshal(head)
if err == nil {
var out json.RawMessage
if out, _, err = st.runProgram(ctx, []string{"-seats"}, body); err == nil {
var a rules.SeatsAnswer
if err = json.Unmarshal(out, &a); err == nil && a.V != rules.SeatsVersion {
err = fmt.Errorf("it answered a seat list of version %d, not %d", a.V, rules.SeatsVersion)
}
if err == nil {
st.c.Seats = overSeats(base, a.Seats)
if len(a.Seats) > 0 {
st.c.note(fmt.Sprintf("the heads carry %d seat definitions new or changed (and %d the seat list holds as they are), laid over the %d the seat list holds", len(a.Seats), a.Same, len(base)))
}
for _, l := range a.Left {
st.c.note("the heads' seat list leaves out " + l.Path + ": " + l.Why)
}
return true
}
}
}
st.fail(goModule, "the heads' seat list could not be made by the checking program: "+err.Error())
return false
}
// overSeats is the list's definitions with each of the heads' put in place of the one of its seat, or added, sorted by
// seat: the checker reads of a definition only its seat's name.
func overSeats(list, heads []json.RawMessage) []json.RawMessage {
by := map[string]json.RawMessage{}
for _, l := range [][]json.RawMessage{list, heads} {
for _, raw := range l {
var d struct {
Seat string `json:"seat"`
}
if json.Unmarshal(raw, &d) == nil && d.Seat != "" {
by[d.Seat] = raw
}
}
}
var out []json.RawMessage // none when neither holds one: the requests then carry no list
for _, seat := range slices.Sorted(maps.Keys(by)) {
out = append(out, by[seat])
}
return out
}
// factsHead are the facts that name the check, without what the asking gathered.
func (st *state) factsHead() rules.Facts {
f := rules.Facts{V: rules.FactsVersion, Group: st.c.Group, Heads: st.c.Heads, Trunk: st.c.Trunk}
for _, p := range st.c.Pulls {
f.Members = append(f.Members, rules.Member{Repository: p.Repository, Number: p.Number, Head: p.Head, Trunk: p.Trunk})
}
return f
}
// judged is the checking program's answer to the check's facts (`-verdict`), with what judged; a program that cannot be
// asked, built or run, or does not answer a verdict, fails the check naming the checking program, never the running
// checker's rules in its place: the facts gain the failure and the verdict is worded from them as a failure.
func (st *state) judged(ctx context.Context) (rules.Answer, *meshdeliverychecker.VerdictsByGroupHeadsValueJudgedBy) {
body, err := json.Marshal(st.factsOf())
var by *meshdeliverychecker.VerdictsByGroupHeadsValueJudgedBy
if err == nil {
var out json.RawMessage
if out, by, err = st.runProgram(ctx, []string{"-verdict"}, body); err == nil {
var a rules.Answer
if err = json.Unmarshal(out, &a); err == nil && a.V != rules.AnswerVersion {
err = fmt.Errorf("it answered a verdict of version %d, not %d", a.V, rules.AnswerVersion)
}
if err == nil {
return a, by
}
}
}
st.fail(goModule, "the check could not be judged by the checking program: "+err.Error())
return rules.Answer{V: rules.AnswerVersion, Verdict: rules.Judge(st.factsOf())}, by
}
// judgedByOf is the verdict's judged-by of a run of the checking program: where it was built, the key its build is kept
// under, whether this run built it, and the heads it read.
func judgedByOf(prog buildstep.RunProgram, answer buildstep.RunAnswer, heads []buildstep.RunHead) *meshdeliverychecker.VerdictsByGroupHeadsValueJudgedBy {
by := &meshdeliverychecker.VerdictsByGroupHeadsValueJudgedBy{Program: meshdeliverychecker.VerdictsByGroupHeadsValueJudgedByProgram{
Repository: prog.Repository, Commit: prog.Commit, Dir: prog.Dir, Package: prog.Package}, Key: answer.Key, Built: answer.Built}
for _, h := range heads {
by.Heads = append(by.Heads, meshdeliverychecker.VerdictsByGroupHeadsValueJudgedByHeadsItem{Repository: h.Repository, Commit: h.Commit})
}
return by
}
@@ -1,6 +1,8 @@
package check
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"slices"
@@ -12,6 +14,7 @@ import (
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliverycheckerclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
@@ -25,6 +28,7 @@ const (
VerdictsReadTool = "checker_verdicts"
PullReadTool = "checker_pull"
KeptReadTool = "checker_kept"
InputsReadTool = "checker_inputs"
// PreviewsReadTool reads the previews state (issue 1680); StatusReadTool reads what the checker runs (issue 1685: nothing holds it since R4g).
PreviewsReadTool = "checker_previews"
StatusReadTool = "checker_status"
@@ -41,6 +45,7 @@ type Reader struct {
VerdictState, KeptState, CandidateState StateRead
PreviewState StateRead // the checker's merge previews
InputState StateRead // the checking program's inputs (InputsStateName)
StepStates map[string]StateRead // the build modules' steps states, by module
HaveState StateRead // the node-engine's have: whether a build module is on a node and running
StepBound time.Duration // how long the checker waits for one step; DefaultStepBound when 0
@@ -68,7 +73,10 @@ type VerdictRead struct {
Verdict json.RawMessage `json:"verdict"`
// Failed is everything the verdict names as failed, one line each, in the order judged (failedLines): every member's
// failed tests, refused steps and conflicts, the manifests' refusals and the composition (the operator, 2026-11-02).
Failed []string `json:"failed,omitempty"`
Failed []string `json:"failed,omitempty"`
// Judged says which checking program judged the verdict, in one line (judgedLine): where it was built, the key its build
// is kept under, whether this check built it, and the heads it read (issue 1921); empty for a verdict written before.
Judged string `json:"judged,omitempty"`
Reports map[string]string `json:"reports,omitempty"`
Note string `json:"note,omitempty"`
}
@@ -93,6 +101,7 @@ func (r Reader) read(key string, raw json.RawMessage) (VerdictRead, error) {
return out, fmt.Errorf("%s holds what is not a verdict: %w", key, err)
}
out.Failed = failedLines(v)
out.Judged = judgedLine(v.JudgedBy)
c, ok := r.candidate(v.Group)
if !ok || (c.Heads != v.Heads && GateHeads(c) != v.Heads) {
out.Note = "the candidate no longer holds these heads"
@@ -262,6 +271,65 @@ func (r Reader) Kept(level, digest string) (map[string]json.RawMessage, string,
return out, cut, nil
}
// Input is the answer of the seat's verb input (InputVerb): the input kept under digest, its exact bytes, which the Go build
// module hands the checking program a run ask names (buildstep.RunInput.Read). A digest that is no sha256, one the checker
// keeps nothing under, and bytes that read back as another digest are errors, never an answer.
func (r Reader) Input(digest string) (meshdeliverychecker.InputAnswer, error) {
if r.InputState.Get == nil {
return meshdeliverychecker.InputAnswer{}, fmt.Errorf("the state %s cannot be read here", InputsStateName)
}
if len(digest) != 64 || strings.Trim(digest, "0123456789abcdef") != "" {
return meshdeliverychecker.InputAnswer{}, fmt.Errorf("%q is no sha256 digest of 64 lowercase hex digits", digest)
}
raw, err := r.InputState.Get(digest)
if err != nil {
return meshdeliverychecker.InputAnswer{}, fmt.Errorf("the input %s cannot be read: %w", digest, err)
}
if len(raw) == 0 {
return meshdeliverychecker.InputAnswer{}, fmt.Errorf("the checker keeps no input %s (an input is kept a day)", digest)
}
if sum := sha256.Sum256(raw); hex.EncodeToString(sum[:]) != digest {
return meshdeliverychecker.InputAnswer{}, fmt.Errorf("the input kept under %s reads back as %d bytes of another digest", digest, len(raw))
}
return meshdeliverycheckerclient.InputAnswerOf(raw), nil
}
// Inputs are the checking program's kept inputs: with a digest that input as kept, else every digest with its size.
func (r Reader) Inputs(digest string) (map[string]json.RawMessage, string, error) {
if r.InputState.Keys == nil || r.InputState.Get == nil {
return nil, "", fmt.Errorf("the state %s cannot be read here", InputsStateName)
}
if digest != "" {
raw, err := r.InputState.Get(digest)
if err != nil {
return nil, "", err
}
if len(raw) == 0 {
return nil, "", fmt.Errorf("no input %s is kept", digest)
}
return map[string]json.RawMessage{digest: raw}, "", nil
}
keys, err := r.InputState.Keys()
if err != nil {
return nil, "", err
}
sort.Strings(keys)
keys, cut := capKeys(keys, "give a digest to read one")
sizes := make([]int, len(keys))
if err := parallel(len(keys), func(i int) error {
raw, err := r.InputState.Get(keys[i])
sizes[i] = len(raw)
return err
}); err != nil {
return nil, "", err
}
out := map[string]json.RawMessage{}
for i, k := range keys {
out[k] = json.RawMessage(fmt.Sprintf(`{"size":%d}`, sizes[i]))
}
return out, cut, nil
}
// PreviewRead is one preview as stored, with the parts of its key.
type PreviewRead struct {
Key string `json:"key"`
@@ -719,3 +787,20 @@ func (r Reader) buildsNotEnded(c meshcatalog.CandidatesByGroupValue) string {
sort.Strings(out)
return strings.Join(slices.Compact(out), ", ")
}
// judgedLine is a verdict's JudgedBy in one line, as the verdict tools answer it.
func judgedLine(by *meshdeliverychecker.VerdictsByGroupHeadsValueJudgedBy) string {
if by == nil {
return ""
}
built := "kept"
if by.Built {
built = "built for this check"
}
heads := make([]string, len(by.Heads))
for i, h := range by.Heads {
heads[i] = h.Repository + " " + h.Commit
}
return fmt.Sprintf("by the checking program %s of %s at %s (%s), %s as %s, over %s", by.Program.Package, by.Program.Repository,
by.Program.Commit, by.Program.Dir, built, by.Key, strings.Join(heads, ", "))
}
@@ -71,15 +71,23 @@ func TestTheVerdictToolsAnswerTheStoredVerdictWithTheFullReportOfEachPullRequest
if got := vs[0].Reports["novox/mesh-delivery#7"]; !strings.HasPrefix(got, "pass 111111111111 on aaaaaaaaaaaa") || !strings.Contains(got, "composition not yet run") {
t.Errorf("reports = %v", vs[0].Reports)
}
if vs[0].Judged != "" {
t.Errorf("judged = %q of a verdict with no JudgedBy", vs[0].Judged)
}
if only, _, _ := rd.Verdicts("feat/other"); len(only) != 0 {
t.Errorf("another group's verdicts = %+v", only)
}
// A failed verdict is answered with every failure it names, one line each (the operator, 2026-11-02).
raw, _ := meshdeliverycheckerclient.EncodeVerdict(meshdeliverychecker.VerdictsByGroupHeadsValue{Group: "feat/x", Heads: "0123456789ab", State: meshdeliverycheckerclient.VerdictFail,
Failed: &FailedFirst{Test: "unit: m/a.TestA", Words: "a_test.go:3: boom\nmore"},
Failures: failureItems([]FailedFirst{{Test: "unit: m/a.TestA", Words: "a_test.go:3: boom"}, {Machine: "g14", Words: "route-proxy is defined twice"}})})
if got, _ := rd.read(key, raw); strings.Join(got.Failed, " | ") != "unit: m/a.TestA: a_test.go:3: boom | g14: route-proxy is defined twice" {
t.Errorf("failed = %q", got.Failed)
Failures: failureItems([]FailedFirst{{Test: "unit: m/a.TestA", Words: "a_test.go:3: boom"}, {Machine: "g14", Words: "route-proxy is defined twice"}}),
// Issue 1921: it names the checking program that judged it, its build and the heads it read.
JudgedBy: judgedByOf(buildstep.RunProgram{Repository: repoA, Commit: head1, Dir: CheckingProgramDir, Package: CheckingProgramPackage},
buildstep.RunAnswer{Key: "program-0123456789abcdef", Built: true}, []buildstep.RunHead{{Repository: repoA, Commit: head1}})})
want := "by the checking program " + CheckingProgramPackage + " of " + repoA + " at " + head1 + " (" + CheckingProgramDir +
"), built for this check as program-0123456789abcdef, over " + repoA + " " + head1
if got, _ := rd.read(key, raw); strings.Join(got.Failed, " | ") != "unit: m/a.TestA: a_test.go:3: boom | g14: route-proxy is defined twice" || got.Judged != want {
t.Errorf("failed = %q, judged %q, want %q", got.Failed, got.Judged, want)
}
}
@@ -121,6 +129,41 @@ func TestTheKeptToolAnswersTheKeysAsStoredByLevelAndByDigest(t *testing.T) { //
if _, _, err := rd.Kept("unit", digB); err == nil {
t.Error("a digest nothing kept was answered")
}
// Issue 1921: checker_inputs answers the checking program's inputs, every digest with its size, or one as kept.
inputs := newMemKV()
body := []byte(`{"v":1,"group":"feat/x"}`)
in, _ := buildstep.RunInputOf(InputVerb, body)
_ = memInputs{inputs}.Keep(context.Background(), in.Digest, body)
rd.InputState = kvRead(inputs)
if all, _, err := rd.Inputs(""); err != nil || string(all[in.Digest]) != fmt.Sprintf(`{"size":%d}`, len(body)) {
t.Errorf("inputs = %s, %v", all, err)
}
if one, _, err := rd.Inputs(in.Digest); err != nil || string(one[in.Digest]) != string(body) {
t.Errorf("one input = %s, %v", one, err)
}
if _, _, err := rd.Inputs(strings.Repeat("0", 64)); err == nil {
t.Error("an input nothing kept was answered")
}
// The operator's answer on B0: the seat's verb input answers the exact bytes, which the build module's read takes;
// a digest that is none, one nothing is kept under, and bytes kept under another digest are errors.
if got, err := meshdeliverycheckerclient.ReadInput(func(key string, b any) (json.RawMessage, error) {
if key != "seat:mesh-delivery-checker.input" {
return nil, fmt.Errorf("asked %s", key)
}
a, err := rd.Input(b.(meshdeliverychecker.InputInput).Digest)
if err != nil {
return nil, err
}
return json.Marshal(a)
}, in.Digest); err != nil || string(got) != string(body) {
t.Errorf("the input through the verb = %s, %v", got, err)
}
_, _ = inputs.Create(context.Background(), strings.Repeat("1", 64), []byte(`{"v":2}`))
for _, digest := range []string{"d1", strings.Repeat("0", 64), strings.Repeat("1", 64)} {
if a, err := rd.Input(digest); err == nil {
t.Errorf("the verb answered %s: %+v", digest, a)
}
}
}
// 1680 live acceptance: checker_previews shows the preview key of an open pull request, as stored, narrowed by repository and head.
@@ -3,14 +3,9 @@ package check
import (
"context"
"errors"
"fmt"
"maps"
"slices"
"strings"
"mesh-delivery-checker/rules"
"git.novox.be/novox/mesh-contracts/go/seats/meshconfig"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliverycheckerclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
@@ -20,7 +15,7 @@ import (
// naming the nodes holding it open; a check that cannot ask fails, saying so.
// RenamesSeat is the verb the checker asks, as its manifest's `invokes` names it.
var RenamesSeat = "seat:" + meshconfig.Seat + "." + meshconfig.VerbRenames
var RenamesSeat = rules.RenamesSeat
// ErrRenamesNotServed is the answer of Renames when the seat's holder serves no verb renames.
var ErrRenamesNotServed = errors.New("the holder of the seat " + meshconfig.Seat + " does not serve its verb " + meshconfig.VerbRenames)
@@ -52,25 +47,20 @@ func renamesOf(was, now catalogue.Manifest) renameChange {
return rc
}
// judgeRenames fails the check for a rename the seat refuses, a window a change would drop, or an ask that cannot be made.
func (st *state) judgeRenames(ctx context.Context) *meshdeliverychecker.VerdictsByGroupHeadsValue {
// judgeRenames records, of each module a member changes that brings or drops a `renamed-from`, what it brings and drops
// and how the seat mesh-config's verb renames answered for it, as a fact the rules judge (rules.JudgeRenames): a rename
// whose value cannot be moved, and the drop of one whose window is open, fail the check (design 32).
func (st *state) judgeRenames(ctx context.Context) {
if st.c.Try != nil {
return nil
}
var out *meshdeliverychecker.VerdictsByGroupHeadsValue
failed := func(words string) {
f := FailedFirst{Machine: meshconfig.Seat, Words: words}
if out == nil {
out = &meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail, Failed: &f}
return
}
addFailure(out, f)
return
}
for _, p := range st.c.Pulls {
for _, module := range p.Modules {
r := rules.Renames{Module: module, Key: pullKey(p)}
was, now, ok, err := st.manifestPair(ctx, p, module)
if err != nil {
failed(fmt.Sprintf("%s (%s): its config renames cannot be judged: %v", module, pullKey(p), err))
r.Unread = err.Error()
st.facts = append(st.facts, rules.Step{Repository: p.Repository, Number: p.Number, Renames: &r})
continue
}
if !ok {
@@ -83,39 +73,21 @@ func (st *state) judgeRenames(ctx context.Context) *meshdeliverychecker.Verdicts
if len(rc.brought) == 0 && len(rc.dropped) == 0 {
continue
}
if st.j.Renames == nil {
failed(fmt.Sprintf("%s (%s) changes a renamed-from, and this checker cannot ask %s", module, pullKey(p), RenamesSeat))
continue
}
answer, err := st.j.Renames.Renames(ctx, module)
switch {
case errors.Is(err, ErrRenamesNotServed):
for _, key := range slices.Sorted(maps.Keys(rc.brought)) {
failed(fmt.Sprintf("%s (%s) renames %s from %s: refused, %v, so the value cannot be moved", module, pullKey(p), key, rc.brought[key], err))
}
for _, key := range slices.Sorted(maps.Keys(rc.dropped)) {
failed(fmt.Sprintf("%s (%s) drops renamed-from %s of %s: refused, %v, so whether its window is open is not known", module, pullKey(p), rc.dropped[key], key, err))
}
continue
case err != nil:
failed(fmt.Sprintf("%s (%s) changes a renamed-from, and the check cannot ask %s: %v", module, pullKey(p), RenamesSeat, err))
continue
}
for _, key := range slices.Sorted(maps.Keys(rc.dropped)) {
for _, r := range answer.Renames {
if r.Module == module && r.From == rc.dropped[key] && r.To == key && len(r.Open) > 0 {
open := slices.Sorted(slices.Values(r.Open))
failed(fmt.Sprintf("%s (%s) drops renamed-from %s of %s while its rename window is open on %s", module, pullKey(p), r.From, key, strings.Join(open, ", ")))
}
r.Brought, r.Dropped = rc.brought, rc.dropped
if st.j.Renames != nil {
answer, err := st.j.Renames.Renames(ctx, module)
switch {
case err != nil:
r.Unasked, r.NotServed = err.Error(), errors.Is(err, ErrRenamesNotServed)
default:
r.Answer = &answer
}
}
st.c.note(fmt.Sprintf("%s.%s answered for %s (%s)", meshconfig.Seat, meshconfig.VerbRenames, module, pullKey(p)))
st.facts = append(st.facts, rules.Step{Repository: p.Repository, Number: p.Number, Renames: &r})
}
}
return out
}
// headManifest is a module's manifest at the pull request's head, new on it: by its candidate's digest, else the forge's.
func (st *state) headManifest(ctx context.Context, p Pull, module string) (catalogue.Manifest, bool) {
if digest := p.Fps[module].Manifest; digest != "" && st.j.Read != nil {
if st.j.Manifests != nil && st.j.Manifests.ManifestDigests()[module] == digest {
@@ -0,0 +1,159 @@
package check
import (
"bytes"
"compress/gzip"
"context"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"sort"
"strings"
"time"
)
// The composition requests a check sends (issue 1921, notes 66-67): the request the checker sends the composer for one
// node, the checking program's whole input of a composition, is kept in the checker's own state RequestsStateName, so a
// session reproduces a check's composition exactly. The checker is a foundation part (novox/hq ADR 0619 §1-3), so its
// kept data is a section of its manifest's old form, here `state`, and not an object store asked in `requires`.
//
// **What bounds it.** A request is the node-engine's whole composition input of one node, about 430 KB as JSON. A
// state's value holds at most 256 KiB (kvlimit.ModuleStateValueBytes), so it is kept gzip-compressed (JSON shrinks about
// tenfold) in a document of its own; a request whose compressed bytes still do not fit is refused by the put, which
// fails the node's composition naming the size (never dropped). The manifest's ttl-seconds is its lifetime (3 days):
// the bus drops a value past it, so nothing waits on a timer here. A check kept again overwrites the key.
const (
// RequestsStateName is the checker's state of kept requests and RequestsBucket its bucket.
RequestsStateName = "compositions"
RequestsBucket = CheckerModule + "_" + RequestsStateName
// RequestsReadTool is the tool that reads a kept request.
RequestsReadTool = "checker_composition_requests"
)
// KeptRequest is a request as the state holds it: whose it is, and the request's JSON, compressed.
type KeptRequest struct {
V int `json:"v"`
Check string `json:"check"`
Node string `json:"node"`
KeptAt time.Time `json:"kept_at"`
// Size and Digest are the request's, uncompressed: its bytes and their sha256.
Size int `json:"size"`
Digest string `json:"digest"`
// Gzip is the request's bytes, gzip-compressed and base64-encoded.
Gzip string `json:"gzip"`
}
// RequestKey is the key a check's request to a node is kept under: the check's key (group.heads) and the node.
func RequestKey(key, node string) string { return key + "/" + node }
// RequestPutter writes a value of the state, replacing one of that key.
type RequestPutter interface {
Put(ctx context.Context, key string, value []byte) (uint64, error)
}
// RequestKeeper keeps the requests of a check.
type RequestKeeper struct {
Store RequestPutter
Now func() time.Time
}
// Keep stores body under the check's key and the node. An error says why it could not, for the verdict's words.
func (k RequestKeeper) Keep(ctx context.Context, key, node string, body []byte) error {
var packed bytes.Buffer
zw := gzip.NewWriter(&packed)
if _, err := zw.Write(body); err != nil {
return fmt.Errorf("the composition request of check %s to %s could not be compressed: %w", key, node, err)
}
if err := zw.Close(); err != nil {
return fmt.Errorf("the composition request of check %s to %s could not be compressed: %w", key, node, err)
}
now := time.Now
if k.Now != nil {
now = k.Now
}
sum := sha256.Sum256(body)
doc, err := json.Marshal(KeptRequest{V: 1, Check: key, Node: node, KeptAt: now().UTC(), Size: len(body),
Digest: hex.EncodeToString(sum[:]), Gzip: base64.StdEncoding.EncodeToString(packed.Bytes())})
if err != nil {
return err
}
if _, err := k.Store.Put(ctx, RequestKey(key, node), doc); err != nil {
return fmt.Errorf("the composition request of check %s to %s (%d bytes, %d compressed) could not be kept in the state %s: %w",
key, node, len(body), packed.Len(), RequestsStateName, err)
}
return nil
}
// ReadRequests answers the kept requests: with a check and a node, that request whole; with a check, the nodes it kept
// one for; with neither, every kept request. A node without a check is refused. get is the state's value of a key.
func ReadRequests(s StateRead, key, node string) (any, error) {
if s.Keys == nil || s.Get == nil {
return nil, fmt.Errorf("the state %s cannot be read here", RequestsStateName)
}
if node != "" && key == "" {
return nil, fmt.Errorf("a node is asked for with the check's key (group.heads) of the request")
}
if node != "" {
raw, err := s.Get(RequestKey(key, node))
if err != nil {
return nil, err
}
if len(raw) == 0 {
return nil, fmt.Errorf("no composition request of check %s to %s is kept (none was sent, or it is past its lifetime)", key, node)
}
kept, body, err := unpackRequest(raw)
if err != nil {
return nil, fmt.Errorf("the kept request %s: %w", RequestKey(key, node), err)
}
return map[string]any{"check": kept.Check, "node": kept.Node, "size": kept.Size, "digest": kept.Digest, "kept_at": kept.KeptAt, "request": json.RawMessage(body)}, nil
}
keys, err := s.Keys()
if err != nil {
return nil, err
}
sort.Strings(keys)
listed := []map[string]any{}
for _, k := range keys {
if key != "" && !strings.HasPrefix(k, key+"/") {
continue
}
raw, err := s.Get(k)
if err != nil {
return nil, err
}
var kept KeptRequest
if err := json.Unmarshal(raw, &kept); err != nil {
listed = append(listed, map[string]any{"key": k, "unreadable": err.Error()})
continue
}
listed = append(listed, map[string]any{"check": kept.Check, "node": kept.Node, "size": kept.Size, "digest": kept.Digest, "kept_at": kept.KeptAt})
}
return map[string]any{"kept": listed}, nil
}
// unpackRequest reads a kept document and its request's bytes, which must match the digest they were kept with.
func unpackRequest(raw []byte) (KeptRequest, []byte, error) {
var kept KeptRequest
if err := json.Unmarshal(raw, &kept); err != nil {
return kept, nil, fmt.Errorf("is no kept request: %w", err)
}
packed, err := base64.StdEncoding.DecodeString(kept.Gzip)
if err != nil {
return kept, nil, fmt.Errorf("holds no base64: %w", err)
}
zr, err := gzip.NewReader(bytes.NewReader(packed))
if err != nil {
return kept, nil, fmt.Errorf("holds no gzip: %w", err)
}
body, err := io.ReadAll(zr)
if err != nil {
return kept, nil, fmt.Errorf("holds no complete gzip: %w", err)
}
if sum := sha256.Sum256(body); hex.EncodeToString(sum[:]) != kept.Digest {
return kept, nil, fmt.Errorf("does not match the digest it was kept with")
}
return kept, body, nil
}
@@ -1,6 +1,8 @@
package check
import (
"mesh-delivery-checker/rules"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
)
@@ -14,61 +16,16 @@ import (
// trees. A conflict is an answer.
type Fetched = buildstep.FetchAnswer
// GoPackage is one package of a Go test result.
type GoPackage struct {
Dir string `json:"dir"`
Package string `json:"package"`
Passed bool `json:"passed"`
Elapsed float64 `json:"elapsed"`
Tests int `json:"tests"`
NoTests bool `json:"no_tests,omitempty"`
// Skipped is how many of the package's tests skipped; an older build module names none.
Skipped int `json:"skipped,omitempty"`
}
// ProgramFile is a changed file that is a compiled program (rules.ProgramFile).
type ProgramFile = rules.ProgramFile
// Failure is a test that failed, or a package that failed with no test to name: the build modules' failure contract
// (buildstep.Failure, issue 1901), its cause never empty and never a passing line from a build module that keeps it.
type Failure = buildstep.Failure
// TestTime is one test and the seconds it took.
type TestTime struct {
Package string `json:"package"`
Test string `json:"test"`
Elapsed float64 `json:"elapsed"`
}
// Tested is the answer of a test step of either build module (the Go one names packages and counts tests; the Node one
// names the package directory as Module and counts none).
type Tested struct {
Level string `json:"level"`
Module string `json:"module,omitempty"`
Passed bool `json:"passed"`
Why string `json:"why,omitempty"`
NoTests bool `json:"no_tests,omitempty"`
Tests int `json:"tests"`
Skipped int `json:"skipped"`
Packages []GoPackage `json:"packages,omitempty"`
Elapsed float64 `json:"elapsed"`
Setup float64 `json:"setup,omitempty"`
// Lint is the unit level's gofmt and go vet time, before its tests (mesh-build-go, ADR 0529 §1).
Lint float64 `json:"lint,omitempty"`
Bound float64 `json:"bound"`
SetupBound float64 `json:"setup_bound,omitempty"`
Slowest []TestTime `json:"slowest,omitempty"`
Failed []Failure `json:"failed,omitempty"`
OverBound bool `json:"over_bound"`
OverWhich string `json:"over_which,omitempty"`
// JudgedBy is the program that ran the step (ADR 0535 §5): "installed", or "candidate <merge commit>" under runner
// workspace; absent from a build module that does not take the runner.
JudgedBy string `json:"judged_by,omitempty"`
// By is the build module that answered and the SDK it was built with (buildstep.By); absent from a build module from
// before the failure contract.
By string `json:"by,omitempty"`
// OutOfStep are the generated files the step's regenerate-and-compare found out of step with their seat definitions,
// each a repository-relative path (design 26; the build modules' compare step, mesh-catalog CAT-BG5 and CAT-BN2):
// the checker names them in its verdict event. Absent from a build module that does not compare.
OutOfStep []string `json:"out-of-step,omitempty"`
}
// The test step's answer is read by the judging rules (milestone 47/B0); these name their shapes here.
type (
Failure = rules.Failure
TestTime = rules.TestTime
Tested = rules.Tested
GoPackage = rules.GoPackage
)
// Digested is the answer of the Go build module's digest step: one digest per package that has tests at the level, by import
// path.
@@ -2,21 +2,15 @@ package check
import (
"context"
"fmt"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliverycheckerclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"mesh-delivery-checker/rules"
)
// A pull request whose head's seat definition does not parse fails its check, naming the seat and where it lives (design
// 26; the lead's ruling on A2i's finding 1): the indexer keeps such a definition as the pull request's entry of its state
// seat-definitions with `unparsed` set, and cannot refuse the manifest itself, whose digest the trunk may share.
// UnparsedSeat is one seat definition of a pull request's head that does not parse: the seat (the contracts module's name
// when its index did not parse), its module's directory and the indexer's words of why.
type UnparsedSeat struct {
Seat, Module, Directory, Why string
}
// UnparsedSeat is one seat definition of a pull request's head that does not parse (rules.UnparsedSeat).
type UnparsedSeat = rules.UnparsedSeat
// SeatDefinitions reads the indexer's seat definitions of one pull request (its state seat-definitions, through its
// first view, DLV-M2).
@@ -24,33 +18,24 @@ type SeatDefinitions interface {
Unparsed(ctx context.Context, repository string, pull int) ([]UnparsedSeat, error)
}
// refuseUnparsed fails the check of each pull request with a seat definition that does not parse; a state that cannot be
// read fails it too, saying so.
func (st *state) refuseUnparsed(ctx context.Context) *meshdeliverychecker.VerdictsByGroupHeadsValue {
// refuseUnparsed records, of each pull request, the indexer's seat definitions of its head that do not parse, or why
// they cannot be read, as a fact the rules judge (rules.JudgeUnparsed).
func (st *state) refuseUnparsed(ctx context.Context) {
if st.j.SeatDefinitions == nil || st.c.Try != nil {
return nil
}
var out *meshdeliverychecker.VerdictsByGroupHeadsValue
failed := func(words string) {
f := FailedFirst{Machine: SeatDefinitionsMachine, Words: words}
if out == nil {
out = &meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail, Failed: &f}
return
}
addFailure(out, f)
return
}
for _, p := range st.c.Pulls {
u := rules.Unparsed{Key: pullKey(p)}
bad, err := st.j.SeatDefinitions.Unparsed(ctx, p.Repository, p.Number)
if err != nil {
failed(fmt.Sprintf("%s: the indexer's seat definitions of it cannot be read, so none is known to parse: %v", pullKey(p), err))
continue
u.Unread = err.Error()
}
for _, u := range bad {
failed(fmt.Sprintf("%s: the seat definition of %s (module %s, %s) does not parse: %s", pullKey(p), u.Seat, u.Module, u.Directory, u.Why))
u.Seats = bad
if u.Unread != "" || len(u.Seats) > 0 {
st.facts = append(st.facts, rules.Step{Repository: p.Repository, Number: p.Number, Unparsed: &u})
}
}
return out
}
// SeatDefinitionsMachine names what refused a seat definition in a verdict's failure.
const SeatDefinitionsMachine = "seat-definitions"
const SeatDefinitionsMachine = rules.SeatDefinitionsMachine
@@ -5,6 +5,7 @@ import (
"encoding/json"
"errors"
"fmt"
"mesh-delivery-checker/rules"
"slices"
"strings"
"testing"
@@ -14,7 +15,6 @@ import (
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliverycheckerclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-checker/checkermsg"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
)
@@ -172,7 +172,7 @@ func (f *fakeTryCandidates) TryCandidate(_ context.Context, group, heads string,
// wholeBench makes the bench judge a whole try as the checker does: the composition on every placed node and the
// indexer's candidate of the try's group.
func wholeBench(b *bench, cand meshcatalog.CandidatesByGroupValue) (*fakeCompositions, *fakeTryCandidates) {
comps := &fakeCompositions{v: meshwants.CompositionsByGroupHeadsValue{State: meshdeliverycheckerclient.CompositionPass, Sdk: "v0.1.14-x", Machines: []string{"g14", "shanks"}, Took: 40}}
comps := &fakeCompositions{v: composedOn("v0.1.14-x", 40, "g14", "shanks")}
cands := &fakeTryCandidates{cand: cand}
b.ex.Compositions, b.ex.Candidates = comps, cands
return comps, cands
@@ -258,7 +258,8 @@ func TestATryIsJudgedAsAGroupsCheckAndOwesTheForgeNothing(t *testing.T) { // ADR
if len(written.running) < 3 || len(written.running[1].Asks) != 1 || written.running[1].Steps[0].State != TryAsked {
t.Errorf("the running try's writes = %+v, want its ask named while it ran", written.running)
}
if len(got.Steps) != 2 || got.Steps[0].Node != "shanks" || got.Steps[0].State != buildstep.StepPassed || len(got.Asks) != 2 {
// Two check asks and the checking program's two run asks (the seat list and the verdict).
if len(got.Steps) != 4 || got.Steps[0].Node != "shanks" || got.Steps[0].State != buildstep.StepPassed || len(got.Asks) != 4 {
t.Errorf("the ended try's steps = %+v, asks %v", got.Steps, got.Asks)
}
for _, a := range got.Asks { // the delivery view counts their steps as a try's (build_usage)
@@ -371,21 +372,21 @@ func TestAFailingTryNamesWhatFailedAndAStepThatCannotRunIsAnError(t *testing.T)
for _, row := range []struct {
name string
cand *fakeTryCandidates
comp meshwants.CompositionsByGroupHeadsValue
comp *rules.Composed
state, said string
}{
{name: "the indexer wrote no candidate", cand: &fakeTryCandidates{err: errors.New("the indexer wrote no candidate within 5m0s")}, state: TryError, said: "the indexer wrote no candidate within 5m0s"},
{name: "a node that gave no composition answer", cand: &fakeTryCandidates{}, comp: meshwants.CompositionsByGroupHeadsValue{State: meshdeliverycheckerclient.CompositionPass, Sdk: "x", Warnings: []string{"g14: " + nodeengineclient.NoAnswer}},
{name: "a node that gave no composition answer", cand: &fakeTryCandidates{}, comp: &rules.Composed{Composer: "x", Nodes: []string{"g14"}},
state: TryError, said: "g14: " + nodeengineclient.NoAnswer},
{name: "a composition refused on a node", cand: &fakeTryCandidates{}, comp: meshwants.CompositionsByGroupHeadsValue{State: meshdeliverycheckerclient.CompositionFail, Machine: "g14", Words: "not composed on g14: web: needs a state mail does not keep"},
{name: "a composition refused on a node", cand: &fakeTryCandidates{}, comp: ptr(refusedOn("x", "g14", "web: needs a state mail does not keep")),
state: TryFail, said: "not composed on g14: web: needs a state mail does not keep"},
} {
kv := newMemKV()
b := newBench(t)
comps, _ := wholeBench(b, meshcatalog.CandidatesByGroupValue{})
b.ex.Candidates, comps.v = row.cand, row.comp
if row.comp.State == "" {
comps.v.State = meshdeliverycheckerclient.CompositionPass
b.ex.Candidates = row.cand
if row.comp != nil {
comps.v = *row.comp
}
id, _ := AskTry(memTries{kv}, tryTrunks, TryAsk{Repository: repoA, Head: head1}, time.Now())
var ends []string
+11 -11
View File
@@ -22,6 +22,7 @@ import (
type state struct {
Name string `json:"name"`
History int `json:"history,omitempty"`
TTL int `json:"ttl-seconds,omitempty"`
}
type artifact struct {
@@ -107,7 +108,7 @@ func TestManifestDeclarations(t *testing.T) { // 1678/AC1
t.Errorf("module = %q", m.Module)
}
// The histories of the checker's four seat states (checked against internal/check names below); the names follow the seat mesh-delivery-checker.
if want := []state{{Name: "asks", History: 1}, {Name: "hearing", History: 1}, {Name: "kept", History: 1}, {Name: "owed", History: 5}, {Name: "previews", History: 5}, {Name: "tries", History: 1}, {Name: "verdicts", History: 5}, {Name: "first-views", History: 1}, {Name: "findings", History: 1}}; !reflect.DeepEqual(m.State, want) {
if want := []state{{Name: "asks", History: 1}, {Name: "hearing", History: 1}, {Name: "inputs", TTL: 86400}, {Name: "compositions", TTL: 259200}, {Name: "kept", History: 1}, {Name: "owed", History: 5}, {Name: "previews", History: 5}, {Name: "tries", History: 1}, {Name: "verdicts", History: 5}, {Name: "first-views", History: 1}, {Name: "findings", History: 1}}; !reflect.DeepEqual(m.State, want) {
t.Errorf("state = %+v, want %+v", m.State, want)
}
if want := []string{"mesh-build-go.steps", "mesh-build-go.toolchains", "mesh-build-node.steps", "mesh-build-node.toolchains",
@@ -185,13 +186,14 @@ func TestManifestDeclarations(t *testing.T) { // 1678/AC1
// The read tools (issue 1686) are served by the tools bundle, not by the checker's process.
// The try's tools (ADR 0501 §5; the end by hand, issue 734) beside them: mesh-controller's try calls them. A lost check
// is asked again by hand (checker_ask_again).
if want := []string{"checker_ask_again", "checker_first_view_again", "checker_first_views", "checker_kept", "checker_previews", "checker_pull", "checker_status", "checker_tries", "checker_try", "checker_try_end", "checker_try_read", "checker_verdicts"}; !reflect.DeepEqual(sorted(m.Tools), want) {
if want := []string{"checker_ask_again", "checker_composition_requests", "checker_first_view_again", "checker_first_views", "checker_inputs", "checker_kept", "checker_previews", "checker_pull", "checker_status", "checker_tries", "checker_try", "checker_try_end", "checker_try_read", "checker_verdicts"}; !reflect.DeepEqual(sorted(m.Tools), want) {
t.Errorf("tools = %v, want %v", m.Tools, want)
}
// The try is the checker's own seat's verbs (ADR 0547 §10, issue 774), served beside the tools by the same code.
// The try is the checker's own seat's verbs (ADR 0547 §10, issue 774), served beside the tools by the same code; so is
// input, a checking program's input the Go build module asks (issue 1921, ADR 0596 §2).
if len(m.Claims) != 1 || m.Claims[0].Name != "mesh-delivery-checker" || m.Claims[0].Scope != "mesh" ||
!reflect.DeepEqual(m.Claims[0].Serves, []string{"try", "try-read", "try-end"}) {
t.Errorf("claims = %+v, want the mesh seat mesh-delivery-checker serving try, try-read and try-end", m.Claims)
!reflect.DeepEqual(m.Claims[0].Serves, []string{"try", "try-read", "try-end", "input"}) {
t.Errorf("claims = %+v, want the mesh seat mesh-delivery-checker serving try, try-read, try-end and input", m.Claims)
}
for _, k := range []string{"logs", "emits", "provides", "requires", "defines-seats", "seat", "seats", "bus"} { // config: compose-answer-bound (ADR 0593 §7)
if _, ok := top[k]; ok {
@@ -261,12 +263,10 @@ func TestManifestProcess(t *testing.T) { // 1678/AC1
t.Errorf("the process declares a bus: it runs on its own broker credential")
}
// The settings template is packed as the files archive (novox/hq ADR 0349 §2), or the indexer's judge refuses it.
if len(m.Build.Artifacts) != 4 {
t.Fatalf("artifacts = %+v, want the program, the tools, the files archive and the package checkermsg", m.Build.Artifacts)
}
// The checker's own published layouts, as forgemsg is the forge bridge's (ADR 0634 section 2): the indexer requires them by a published version.
if a := m.Build.Artifacts[3]; a.Name != "checkermsg" || a.Kind != "package" || a.Language != "go" || len(a.Paths) != 1 || a.Paths[0] != "checkermsg" || a.Series != "0.1" {
t.Errorf("the package artifact = %+v", a)
// Issue 1966's bootstrap: the package artifact checkermsg is out of the manifest until the new builder runs; the
// follow-up puts it back with its assertion (ADR 0634 section 2: four artifacts, the fourth the package checkermsg).
if len(m.Build.Artifacts) != 3 {
t.Fatalf("artifacts = %+v, want the program, the tools and the files archive", m.Build.Artifacts)
}
if a := m.Build.Artifacts[2]; a.Name != "files" || a.Kind != "archive" || a.From != "files" {
t.Errorf("files artifact = %+v, want an archive from files/", a)
+6 -10
View File
@@ -9,7 +9,8 @@
"serves": [
"try",
"try-read",
"try-end"
"try-end",
"input"
]
}
],
@@ -35,6 +36,8 @@
"state": [
{ "name": "asks", "history": 1 },
{ "name": "hearing", "history": 1 },
{ "name": "inputs", "ttl-seconds": 86400 },
{ "name": "compositions", "ttl-seconds": 259200 },
{ "name": "kept", "history": 1 },
{ "name": "owed", "history": 5 },
{ "name": "previews", "history": 5 },
@@ -60,6 +63,8 @@
],
"tools": [
"checker_ask_again",
"checker_composition_requests",
"checker_inputs",
"checker_kept",
"checker_previews",
"checker_pull",
@@ -185,15 +190,6 @@
"name": "files",
"kind": "archive",
"from": "files"
},
{
"name": "checkermsg",
"kind": "package",
"language": "go",
"paths": [
"checkermsg"
],
"series": "0.1"
}
]
}
@@ -0,0 +1,305 @@
// Package rules is the checker's judging rules, the checking program's: given the facts of one check (what the build
// modules, the manifest validator and the composer answered, and what failed in the asking), the verdict (novox/hq ADR
// 0644, ADR 0656; design 47's bootstrap delivery). The running checker asks, waits, keeps and records; it gathers the
// facts and has them judged by the checking program built from the change's heads (cmd/mesh-checking-program), which
// imports this package, so a change to a rule is judged by its own rule.
//
// It is the checker module's own package (novox/hq ADR 0634 §1): no seat carries it and no other module imports it.
package rules
import (
"encoding/json"
"fmt"
"path"
"sort"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliverycheckerclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
)
// FactsVersion is the version of Facts this program reads. A checking program built from a later change reads the facts
// a running checker of this version writes, so a version is added to, never changed; a program refuses one it does not
// read, naming it (DecodeFacts).
const FactsVersion = 1
// Facts are everything one check judges, in the order the running checker met it.
type Facts struct {
V int `json:"v"`
// Group and Heads name the check (meshdeliverychecker.VerdictsByGroupHeadsValue's Group and Heads); Trunk is each repository's trunk head merged.
Group string `json:"group"`
Heads string `json:"heads"`
Trunk map[string]string `json:"trunk,omitempty"`
Members []Member `json:"members"`
// SeatDigests are, by seat, the digests (SeatDigest) of the definitions the running checker's reader holds: the
// `-seats` answer leaves out every head definition that is the same (milestone 47/G, the lead's ruling on review
// finding 3: a whole list is over a run answer's bound). None from a checker that hands none: then every head
// definition is answered.
SeatDigests map[string]string `json:"seat-digests,omitempty"`
// Series are the changes of the members' modules' series, as structures, put in the verdict whatever it is (DLV-C4).
Series []meshdeliverychecker.VerdictsByGroupHeadsValueSeriesItem `json:"series,omitempty"`
// Steps are, in the order met, each level a build module answered and each failure the asking met.
Steps []Step `json:"steps,omitempty"`
// CloneMS is how much of the steps' time their levels' running did not take (meshdeliverychecker.VerdictsByGroupHeadsValueTook.Clone).
CloneMS int64 `json:"clone-ms,omitempty"`
// Composition is how the check composed, after every step; nil when it was not composed (a narrowed try, a checker
// that composes nothing): its composition is then not run, unsaid.
Composition *Composition `json:"composition,omitempty"`
// Notes are the running checker's notes of the check, for a reader.
Notes []string `json:"notes,omitempty"`
}
// Member is one pull request of the check.
type Member struct {
Repository string `json:"repository"`
Number int `json:"number"`
Head string `json:"head"`
Trunk string `json:"trunk,omitempty"`
}
// Step is one fact of the check, in the order met: the node that ran it, and one of a level its build module answered
// (Level, as the build module gave it), what its fetch read of compiled programs (Programs), the manifest validator's
// answers (Manifests), the module graph's ordering edges (Graph) or a failure the asking met (Failed: a step not asked, refused or not answered, a conflict, worded
// by the running checker). A step with none only says which node ran it.
type Step struct {
Repository string `json:"repository"`
Number int `json:"number"`
Module string `json:"module,omitempty"`
Node string `json:"node,omitempty"`
Level *buildstep.LevelAnswer `json:"level,omitempty"`
Failed *FailedFirst `json:"failed,omitempty"`
Programs *Programs `json:"programs,omitempty"`
Manifests *Manifests `json:"manifests,omitempty"`
// Graph is the module graph's ordering edges, as the change leaves them and as the trunk has them (JudgeGraph).
Graph *Graph `json:"graph,omitempty"`
// Renames is what a module brings and drops of `renamed-from` and how the seat mesh-config answered (JudgeRenames).
Renames *Renames `json:"renames,omitempty"`
// Unparsed is what the indexer keeps of a pull request's seat definitions that do not parse (JudgeUnparsed).
Unparsed *Unparsed `json:"unparsed,omitempty"`
}
// LevelFact is a level answer as the facts carry it: only what the rules read (Tally.Level), so a group's facts stay
// within the run input's bound (buildstep.RunInputOf). The digest, the unmet packages and the level's judged program
// are left out, the kept packages are one count, and each test result keeps only the fields Tested names, without its
// packages, bounds and judged program (review round 1 of milestone 47/B0). An answer this cannot read is carried as given,
// for the rules to refuse it.
func LevelFact(la buildstep.LevelAnswer) *buildstep.LevelAnswer {
out := &buildstep.LevelAnswer{Level: la.Level}
kept := 0
for _, k := range la.Kept {
kept += k.Tests
}
if len(la.Kept) > 0 {
out.Kept = []buildstep.KeptTaken{{Tests: kept}}
}
for _, raw := range la.Tests {
var res Tested
if err := json.Unmarshal(raw, &res); err != nil {
out.Tests = append(out.Tests, raw)
continue
}
res.Packages, res.JudgedBy, res.Bound, res.SetupBound, res.OverWhich = nil, "", 0, 0, ""
small, err := json.Marshal(res)
if err != nil {
small = raw
}
out.Tests = append(out.Tests, small)
}
return out
}
// DecodeFacts reads facts, refusing a version this program does not read, naming it.
func DecodeFacts(data []byte) (Facts, error) {
var f Facts
if err := json.Unmarshal(data, &f); err != nil {
return Facts{}, fmt.Errorf("rules: the facts are not JSON of their shape: %w", err)
}
if f.V != FactsVersion {
return Facts{}, fmt.Errorf("rules: facts of version %d, this checking program reads version %d", f.V, FactsVersion)
}
return f, nil
}
// Counts are one level's counts across a check's members.
type Counts struct {
Run, Kept, Skipped int
Elapsed, Setup float64
Lint float64
Over bool
Slowest []TestTime
}
// Tally is a check's counts by level.
type Tally map[string]*Counts
// NewTally is a tally of both levels, at nothing.
func NewTally() Tally {
return Tally{buildstep.LevelUnit: {}, buildstep.LevelIntegration: {}}
}
// Level judges one level of a member's check step as it was answered: the kept ones taken are counted, and every test
// result is read and counted, also after one that did not pass, so the verdict names every failed test of the level
// (issue 1921). It answers the level's failure (nil when it passed), its running time (its tests' and setup's) and the
// results read, for the running checker to keep the passes from.
func (t Tally) Level(repository string, number int, module, node string, la buildstep.LevelAnswer) (*meshdeliverychecker.VerdictsByGroupHeadsValue, float64, []Tested) {
a := t[la.Level]
if a == nil {
return failAt(node, fmt.Sprintf("the check of %s#%d answered the level %q, which is no level", repository, number, la.Level)), 0, nil
}
for _, k := range la.Kept {
a.Kept += k.Tests
}
elapsed := 0.0
var results, failed []Tested
for _, raw := range la.Tests {
var res Tested
if err := json.Unmarshal(raw, &res); err != nil {
return failAt(node, fmt.Sprintf("the %s level of %s#%d answered what is not a result: %v", la.Level, repository, number, err)), elapsed, results
}
results = append(results, res)
elapsed += res.Elapsed + res.Setup + res.Lint
if module == buildrun.NodeBuildModule {
if !res.NoTests {
a.Run++ // a Node package is counted whole: its build module counts no tests (ADR 0484 clause 10)
}
} else {
a.Run += res.Tests
a.Skipped += res.Skipped
a.Slowest = append(a.Slowest, res.Slowest...)
}
a.Elapsed += res.Elapsed
a.Setup += res.Setup
a.Lint += res.Lint
a.Over = a.Over || res.OverBound
if !res.Passed {
failed = append(failed, res)
}
}
if len(failed) > 0 {
return LevelFailure(repository, module, la.Level, failed...), elapsed, results
}
return nil, elapsed, results
}
// Judge is the verdict of the facts (Judged without its notes).
func Judge(f Facts) meshdeliverychecker.VerdictsByGroupHeadsValue { return Judged(f).Verdict }
// Judged is the answer of the facts: every failure in the order met, joined and settled, or a pass; the levels'
// counts, times, machines and the levels over their bound; then the composition taken in; and the notes the rules give
// a reader (the manifest validator's warnings and acceptances).
func Judged(f Facts) Answer {
var notes []string
tally := NewTally()
var failures []*meshdeliverychecker.VerdictsByGroupHeadsValue
nodes := map[string]bool{}
for _, s := range f.Steps {
if s.Node != "" {
nodes[s.Node] = true
}
switch {
case s.Failed != nil:
failures = append(failures, &meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail, Failed: s.Failed})
case s.Level != nil:
if v, _, _ := tally.Level(s.Repository, s.Number, s.Module, s.Node, *s.Level); v != nil {
failures = append(failures, v)
}
case s.Programs != nil:
if ff := JudgePrograms(s.Node, *s.Programs); ff != nil {
failures = append(failures, &meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail, Failed: ff})
}
case s.Graph != nil:
ff, said := JudgeGraph(*s.Graph)
notes = append(notes, said...)
if ff != nil {
failures = append(failures, &meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail, Failed: ff})
}
case s.Renames != nil:
ffs, said := JudgeRenames(*s.Renames)
notes = append(notes, said...)
for _, ff := range ffs {
failures = append(failures, &meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail, Failed: &ff})
}
case s.Unparsed != nil:
for _, ff := range JudgeUnparsed(*s.Unparsed) {
failures = append(failures, &meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail, Failed: &ff})
}
case s.Manifests != nil:
ff, said := JudgeManifests(*s.Manifests)
notes = append(notes, said...)
if ff != nil {
failures = append(failures, &meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail, Failed: ff})
}
}
}
v := meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictPass}
if joined := Settled(JoinFailures(failures...)); joined != nil {
v = *joined
}
v.Group, v.Heads, v.Trunk, v.Series = f.Group, f.Heads, f.Trunk, f.Series
Outcome(&v, tally, f.CloneMS, nodes)
if f.Composition != nil {
JudgeComposition(&v, *f.Composition)
}
return Answer{V: AnswerVersion, Verdict: v, Notes: notes}
}
// Outcome fills a verdict's levels (on a pass, or when it names none), times, machines and levels over their bound from
// the tally, and says its composition is not yet run (ADR 0484 clause 6).
func Outcome(v *meshdeliverychecker.VerdictsByGroupHeadsValue, tally Tally, cloneMS int64, nodes map[string]bool) {
if v.State == meshdeliverycheckerclient.VerdictPass || len(v.Levels) == 0 {
v.Levels = map[string]meshdeliverychecker.VerdictsByGroupHeadsValueLevelsValue{}
for level, a := range tally {
v.Levels[level] = meshdeliverychecker.VerdictsByGroupHeadsValueLevelsValue{Run: int64(a.Run), Kept: int64(a.Kept), Skipped: int64(a.Skipped)}
}
}
ms := func(s float64) int64 { return int64(s * 1000) }
u, i := tally[buildstep.LevelUnit], tally[buildstep.LevelIntegration]
v.Took = &meshdeliverychecker.VerdictsByGroupHeadsValueTook{Clone: cloneMS, Unit: ms(u.Elapsed), Lint: ms(u.Lint), IntegrationSetup: ms(i.Setup), IntegrationRun: ms(i.Elapsed)}
v.Machines = nil
for node := range nodes {
v.Machines = append(v.Machines, node)
}
sort.Strings(v.Machines)
v.OverBound = nil
for _, level := range []string{buildstep.LevelUnit, buildstep.LevelIntegration} {
if tally[level].Over {
v.OverBound = append(v.OverBound, level)
}
}
v.Composition = &meshdeliverychecker.VerdictsByGroupHeadsValueComposition{State: meshdeliverycheckerclient.CompositionNotRun}
}
// Slowest are, of each level over its bound, its three slowest tests, worded for the status line.
func Slowest(tally Tally) []string {
var slow []string
for _, level := range []string{buildstep.LevelUnit, buildstep.LevelIntegration} {
a := tally[level]
if !a.Over {
continue
}
sort.SliceStable(a.Slowest, func(x, y int) bool { return a.Slowest[x].Elapsed > a.Slowest[y].Elapsed })
for n, s := range a.Slowest {
if n == 3 {
break
}
slow = append(slow, fmt.Sprintf("%s %s %.1fs", level, path.Base(s.Package)+"."+s.Test, s.Elapsed))
}
}
return slow
}
func failAt(machine, words string) *meshdeliverychecker.VerdictsByGroupHeadsValue {
return &meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail, Failed: &FailedFirst{Machine: machine, Words: words}}
}
// AnswerVersion is the version of Answer this program writes.
const AnswerVersion = 1
// Answer is what the checking program answers on the last line of its standard output: its version, the verdict and the
// notes its rules give a reader of the check.
type Answer struct {
V int `json:"v"`
Verdict meshdeliverychecker.VerdictsByGroupHeadsValue `json:"verdict"`
Notes []string `json:"notes,omitempty"`
}
@@ -0,0 +1,116 @@
package rules
import (
"fmt"
"slices"
"strings"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun"
)
// The module graph's cycle rule (novox/hq ADR 0603 §2, ADR 0614 §3, §6), judged by the checking program (ADR 0656 §5):
// the check refuses a change whose module graph has a cycle and names it; a cycle the trunk already has among the same
// modules is the trunk's, said in the notes. The running checker gathers the graph's ordering edges of the catalogue as
// the change leaves it and as the trunk has it, worked out with the contract's graph (sdk/graph: bases, the provisions
// and bus state a module awaits, the verb edges the graph keeps), as module names and the edge's kind, never the
// manifests themselves, so the facts stay within the run input's bound.
// The kinds of an ordering edge.
const (
EdgeBase = "base"
EdgeAwaits = "awaits"
EdgeInvokes = "invokes"
)
// Edge is one ordering edge: From waits for To. Only is EdgeChange when only the change's graph has it, EdgeTrunk when
// only the trunk's does, empty when both have it.
type Edge struct {
From string `json:"from"`
To string `json:"to"`
Kind string `json:"kind"`
Only string `json:"only,omitempty"`
}
// Where only one graph has an edge.
const (
EdgeChange = "change"
EdgeTrunk = "trunk"
)
// Graph is the module graph's facts: every ordering edge of either graph, between modules of the catalogue. Trunk says
// whether the trunk's graph was read; when it was not, every edge is the change's and no cycle is the trunk's.
type Graph struct {
Edges []Edge `json:"edges"`
Trunk bool `json:"trunk"`
}
// JudgeGraph is the cycle rule over the graph's facts: the failure of a cycle the change makes, or the note of no cycle
// or of a cycle the trunk already has.
func JudgeGraph(g Graph) (*FailedFirst, []string) {
cycle := firstCycle(g.Edges, func(e Edge) bool { return e.Only != EdgeTrunk || !g.Trunk })
if cycle == nil {
return nil, []string{"the change's module graph has no cycle"}
}
words := strings.Join(cycle, " -> ")
if g.Trunk {
among := firstCycle(g.Edges, func(e Edge) bool {
return e.Only != EdgeChange && slices.Contains(cycle, e.From) && slices.Contains(cycle, e.To)
})
if among != nil {
return nil, []string{"the module graph has the cycle " + words + ", which the trunk already has among the same modules (ADR 0603 §2)"}
}
}
return &FailedFirst{Machine: buildrun.GoBuildModule, Words: fmt.Sprintf("the change's module graph has the cycle %s: these modules would "+
"each wait for the other for good, through their bases, the provisions and bus state they await, or the verbs they invoke (ADR 0603 §2, ADR 0614 §6)", words)}, nil
}
// firstCycle is the first cycle of the edges kept, as the modules it passes through with the first named again at its
// end, visiting the modules and each one's edges in sorted order (as sdk/graph's Cycle does), or nil for none.
func firstCycle(edges []Edge, keep func(Edge) bool) []string {
next := map[string][]string{}
for _, e := range edges {
if keep(e) && !slices.Contains(next[e.From], e.To) {
next[e.From] = append(next[e.From], e.To)
}
}
var names []string
for name, to := range next {
slices.Sort(to)
names = append(names, name)
}
slices.Sort(names)
const (
unseen = iota
onPath
done
)
state := map[string]int{}
var path []string
var visit func(string) []string
visit = func(name string) []string {
state[name] = onPath
path = append(path, name)
for _, to := range next[name] {
switch state[to] {
case onPath:
i := slices.Index(path, to)
return append(slices.Clone(path[i:]), to)
case unseen:
if c := visit(to); c != nil {
return c
}
}
}
path = path[:len(path)-1]
state[name] = done
return nil
}
for _, name := range names {
if state[name] == unseen {
if c := visit(name); c != nil {
return c
}
}
}
return nil
}
@@ -0,0 +1,292 @@
package rules
import (
"fmt"
nec "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
ne "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"maps"
"slices"
"sort"
"strings"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliverycheckerclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
"git.novox.be/novox/mesh-delivery/sdk/composer"
)
// The facts the manifest validator, the fetch's programs and the composer answered, and the rules that judge them
// (milestone 47/B0, D1b): the running checker asks and records; these rules give the verdict.
// manifestsMachine is the machine a manifest's failure names: the build seat that ran the manifest validator.
const manifestsMachine = buildrun.GoBuildModule
// CatalogSeat is the seat the indexer holds: the module whose manifest claims it is the indexer, found by the claim and
// never by its name (novox/hq ADR 0633 clause 6, ADR 0601 §4).
const CatalogSeat = "mesh-catalog"
// BetweenManifests is the manifest validator's name for what it judges between manifests (the catalogue as a whole).
const BetweenManifests = "(between manifests)"
// Manifests are the manifest validator's answers on the manifests a check touches (ADR 0578 §2, ADR 0579 §2, ADR 0633
// clause 6): its change and trunk cases where it judged (At), and, when a touched module claims the seat mesh-catalog,
// that indexer and the trunk validator's change case on it (IndexerAt names where it judged).
type Manifests struct {
Touched []string `json:"touched"`
At string `json:"at"`
Change composer.CaseVerdict `json:"change"`
Trunk composer.CaseVerdict `json:"trunk"`
Indexer string `json:"indexer,omitempty"`
IndexerAt string `json:"indexer-at,omitempty"`
OfIndexer *composer.CaseVerdict `json:"of-indexer,omitempty"`
}
// Programs are what one member's fetch read of the change's compiled programs (issue 1577): Files nil when the build
// module reported none (an older module: not refused, the running checker warns), else each changed file that is one.
type Programs struct {
Key string `json:"key"`
Files *[]ProgramFile `json:"files"`
}
// ProgramFile is a changed file that is a compiled program, its format (ELF, Mach-O or PE) and, when it was added in an
// earlier commit of the branch and removed later, that commit: the build step's own (buildstep.Program).
type ProgramFile = buildstep.Program
// Composition is how the check's composition went (ADR 0593 §7, §9; ADR 0603 §10; ADR 0548 §5): kept from the check it
// carries, not run (NotRun says why), or composed by the engines; BuildsFailed are the candidate's builds that failed
// before it composed, worded by the running checker; Waits are what it composed without waiting on, as warnings.
type Composition struct {
Kept *meshdeliverychecker.VerdictsByGroupHeadsValueComposition `json:"kept,omitempty"`
NotRun string `json:"not-run,omitempty"`
BuildsFailed []FailedFirst `json:"builds-failed,omitempty"`
Waits []string `json:"waits,omitempty"`
Composed *Composed `json:"composed,omitempty"`
FPSet string `json:"fp-set,omitempty"`
}
// Composed are the engines' answers to one composition: the nodes placed, each answering node's answer, each silent
// node with why (for the journal only), what composed (the node-engine's commit), the composer's own warnings and how
// long it took in milliseconds.
type Composed struct {
Group string `json:"group"`
Heads string `json:"heads"`
Composer string `json:"composer"`
Nodes []string `json:"nodes"`
Answers map[string]ne.ComposeCandidateAnswer `json:"answers,omitempty"`
Silent map[string]string `json:"silent,omitempty"`
Warnings []string `json:"warnings,omitempty"`
Took int64 `json:"took,omitempty"`
}
// JudgeManifests is the failure of the manifests' facts (nil when they pass) and the notes a reader gets: the trunk
// validator refusing, holding or setting aside the new indexer's manifest fails first (ADR 0633 clause 6); else a new or
// changed manifest is judged in full, a held one is refused, and of one the change leaves as it is a problem the trunk
// already had is a warning while one the change causes fails the check (ADR 0579 §2).
func JudgeManifests(m Manifests) (*FailedFirst, []string) {
var notes []string
if m.Indexer != "" {
var problems []string
ok := false
var iv composer.ManifestRead
if m.OfIndexer != nil {
iv, ok = m.OfIndexer.Manifests[m.Indexer]
}
switch {
case !ok:
problems = append(problems, "the manifest validator gave no verdict")
case iv.Refused != "":
problems = append(problems, strings.Split(iv.Refused, "\n")...)
case !iv.Accepted:
problems = append(problems, "the manifest validator holds it, with no verdict yet")
case iv.SetAside != "":
problems = append(problems, "the manifest validator does not know its fields "+iv.SetAside+", so it waits")
}
if len(problems) > 0 {
return &FailedFirst{Machine: manifestsMachine, Words: fmt.Sprintf("the indexer %s, the holder of the seat %s, would never run: the trunk's manifest validator at %s does not accept its manifest: %s",
m.Indexer, CatalogSeat, m.IndexerAt, strings.Join(problems, "; "))}, nil
}
notes = append(notes, fmt.Sprintf("the new indexer's manifest of %s is accepted by the trunk's manifest validator at %s", m.Indexer, m.IndexerAt))
}
var refused, warned []string
for _, module := range slices.Sorted(maps.Keys(m.Change.Manifests)) {
r := m.Change.Manifests[module]
if r.Accepted {
continue
}
if r.Refused == "" { // held: a verdict not given is not one given (ADR 0633 clause 6)
if slices.Contains(m.Touched, module) {
refused = append(refused, fmt.Sprintf("the indexer's judge at %s holds the manifest of %s: it gave no verdict", m.At, module))
}
continue
}
had := map[string]bool{}
for _, p := range strings.Split(m.Trunk.Manifests[module].Refused, "\n") {
had[p] = true
}
var caused, old []string
for _, p := range strings.Split(r.Refused, "\n") {
if slices.Contains(m.Touched, module) || !had[p] {
caused = append(caused, p)
} else {
old = append(old, p)
}
}
if len(caused) > 0 {
what := "the manifest of " + module
if module == BetweenManifests {
what = "the catalogue"
}
refused = append(refused, fmt.Sprintf("the indexer's judge at %s refuses %s: %s", m.At, what, strings.Join(caused, "; ")))
}
if len(old) > 0 {
warned = append(warned, module+": "+strings.Join(old, "; "))
}
}
if len(warned) > 0 {
notes = append(notes, "warnings of the indexer's judge on manifests the change leaves as they are (ADR 0579 §2): "+strings.Join(warned, " | "))
}
if len(refused) > 0 {
return &FailedFirst{Machine: manifestsMachine, Words: strings.Join(refused, "\n")}, notes
}
return nil, append(notes, fmt.Sprintf("the manifests of %s are accepted by the indexer's judge at %s", strings.Join(m.Touched, ", "), m.At))
}
// JudgePrograms fails a member whose change adds or modifies a compiled program, naming each file (issue 1577); a fetch
// that reported none, or found none, passes.
func JudgePrograms(machine string, p Programs) *FailedFirst {
if p.Files == nil {
return nil
}
var found []string
for _, f := range *p.Files {
if f.Commit != "" {
found = append(found, fmt.Sprintf("%s: a compiled program added in %s and removed later: it stays in history; squash the branch (issue 1577)", f.Path, short12(f.Commit)))
continue
}
found = append(found, f.Path+": a compiled program is build output: remove it and add it to .gitignore (issue 1577)")
}
if len(found) == 0 {
return nil
}
return &FailedFirst{Machine: machine, Words: fmt.Sprintf("%s: %s", p.Key, strings.Join(found, "; "))}
}
func short12(s string) string {
if len(s) > 12 {
return s[:12]
}
return s
}
// JudgeEngines is the check's composition verdict from the engines' answers (ADR 0593 §7, §9): failed when any node's
// answer is refused, naming each node and each module refused with its words (the first refusing node is the machine);
// passed otherwise. Every placed node without an answer is named with nec.NoAnswer as a warning.
func JudgeEngines(c Composed) meshwants.CompositionsByGroupHeadsValue {
v := meshwants.CompositionsByGroupHeadsValue{Group: c.Group, Heads: c.Heads, State: meshdeliverycheckerclient.CompositionPass, Sdk: c.Composer, Took: c.Took}
sorted := append([]string(nil), c.Nodes...)
sort.Strings(sorted)
var fails []string
for _, node := range sorted {
a, answered := c.Answers[node]
if !answered {
v.Warnings = append(v.Warnings, node+": "+nec.NoAnswer)
continue
}
if a.Verdict == nec.NotJudged {
// Composed in the sandbox and not judged (ADR 0593 §9): named as no answer, with the composer's words; decided
// by the verdict, never by the words (ADR 0601 §4's kind).
v.Warnings = append(v.Warnings, node+": "+nec.NoAnswer+" ("+a.Refused+")")
continue
}
v.Machines = append(v.Machines, node)
if a.Verdict != nec.Refused {
continue
}
if v.Machine == "" {
v.Machine = node
}
var words []string
if a.Refused != "" {
words = append(words, a.Refused)
}
for _, m := range append(append([]ne.Module(nil), a.Modules...), a.Own) {
if m.State == nec.ModuleRefused {
words = append(words, m.Module+": "+m.Words)
}
}
if len(words) == 0 {
words = append(words, "refused, naming no module")
}
fails = append(fails, "not composed on "+node+": "+strings.Join(words, "; "))
}
if len(fails) > 0 {
v.State, v.Words = meshdeliverycheckerclient.CompositionFail, strings.Join(fails, "; ")
}
if len(sorted) == 0 {
v.Warnings = append(v.Warnings, "no node is placed, so the change was composed on none")
}
v.Warnings = append(v.Warnings, c.Warnings...)
return v
}
// JudgeComposition takes the composition's facts into the verdict: a failed build of the candidate fails it, named after
// the tests' failures; a failed composition fails it, naming its machine; a passed one with warnings says them.
func JudgeComposition(v *meshdeliverychecker.VerdictsByGroupHeadsValue, c Composition) {
if c.Kept != nil {
kept := *c.Kept
v.Composition = &kept
return
}
for _, f := range c.BuildsFailed {
if v.State != meshdeliverycheckerclient.VerdictFail {
v.State, v.Failed = meshdeliverycheckerclient.VerdictFail, &f
continue
}
AddFailure(v, f)
}
if c.NotRun != "" {
v.Composition = &meshdeliverychecker.VerdictsByGroupHeadsValueComposition{State: meshdeliverycheckerclient.CompositionNotRun, Words: c.NotRun}
return
}
if c.Composed == nil {
return
}
comp := JudgeEngines(*c.Composed)
comp.Warnings = append(comp.Warnings, c.Waits...) // ADR 0603 §10
v.Composition = compositionOf(comp)
if v.Took == nil {
v.Took = &meshdeliverychecker.VerdictsByGroupHeadsValueTook{}
}
v.Took.Composition = comp.Took
if comp.State == meshdeliverycheckerclient.CompositionPass && len(comp.Warnings) > 0 && v.Composition != nil {
v.Composition.Words = strings.Join(comp.Warnings, "; ") // passed with a warning (ADR 0593 §9)
}
if comp.State == meshdeliverycheckerclient.CompositionFail {
machine := comp.Machine
if machine == "" {
machine = "the composition"
}
f := FailedFirst{Machine: machine, Words: comp.Words}
if v.State == meshdeliverycheckerclient.VerdictFail {
AddFailure(v, f) // the tests failed too: the composition is named after them (the operator, 2026-11-02)
} else {
v.State, v.Failed = meshdeliverycheckerclient.VerdictFail, &f
}
}
if v.Composition != nil && v.Composition.State != meshdeliverycheckerclient.CompositionNotRun {
v.Composition.Fpset = c.FPSet // a carry keeps it while the fingerprints stay the same (ADR 0548 §5)
}
}
// compositionWaits is the state of the assigner's composition answer that waits for what it composes from.
const compositionWaits = "waits"
// compositionOf is what the verdict carries of a composition answer c: nil while c waits (the check is still running).
func compositionOf(c meshwants.CompositionsByGroupHeadsValue) *meshdeliverychecker.VerdictsByGroupHeadsValueComposition {
if c.State == compositionWaits {
return nil
}
return &meshdeliverychecker.VerdictsByGroupHeadsValueComposition{State: c.State, Words: c.Words, Sdk: c.Sdk, Machines: c.Machines}
}
@@ -0,0 +1,38 @@
package rules
import checker "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
// FailedFirst names what failed: a test (its package and name) or a machine, with the words of the failure. It is the shape
// of the verdict's `failed` and of each entry of its `failures` (the seat's two types of it have the same fields), so the
// rules judge with one and convert at the verdict's edge (failureItems, failedList).
type FailedFirst = checker.VerdictsByGroupHeadsValueFailed
func failureItems(list []FailedFirst) []checker.VerdictsByGroupHeadsValueFailuresItem {
if list == nil {
return nil
}
out := make([]checker.VerdictsByGroupHeadsValueFailuresItem, len(list))
for i, f := range list {
out[i] = checker.VerdictsByGroupHeadsValueFailuresItem(f)
}
return out
}
func failedList(items []checker.VerdictsByGroupHeadsValueFailuresItem) []FailedFirst {
if items == nil {
return nil
}
out := make([]FailedFirst, len(items))
for i, f := range items {
out[i] = FailedFirst(f)
}
return out
}
// failedOf is the first failure a verdict names: the zero value of a verdict that names none.
func failedOf(v checker.VerdictsByGroupHeadsValue) FailedFirst {
if v.Failed == nil {
return FailedFirst{}
}
return *v.Failed
}
@@ -0,0 +1,54 @@
package rules
import (
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"io/fs"
)
// Program is the checking program's work (cmd/mesh-checking-program; novox/hq ADR 0644, ADR 0656): over its arguments,
// its input (read only when asked for the verdict) and the heads laid for it (nil when none are), the one JSON line it
// answers, or why it refuses. `-seats` answers the seat list of the heads (SeatsOf), `-verdict` the verdict of the
// facts in the input (Judged), refusing facts of a version it does not read.
func Program(args []string, input func() ([]byte, error), heads fs.FS) ([]byte, error) {
set := flag.NewFlagSet("mesh-checking-program", flag.ContinueOnError)
set.SetOutput(io.Discard)
verdict := set.Bool("verdict", false, "answer the verdict of the facts in the input")
seats := set.Bool("seats", false, "answer the seat list of the heads laid for it")
if err := set.Parse(args); err != nil {
return nil, err
}
switch {
case *seats:
if heads == nil {
return nil, errors.New("no heads are laid for it")
}
data, err := input()
if err != nil {
return nil, fmt.Errorf("the input cannot be read: %w", err)
}
facts, err := DecodeFacts(data)
if err != nil {
return nil, err
}
answer, err := SeatsOf(heads, facts)
if err != nil {
return nil, err
}
return json.Marshal(answer)
case *verdict:
data, err := input()
if err != nil {
return nil, fmt.Errorf("the input cannot be read: %w", err)
}
facts, err := DecodeFacts(data)
if err != nil {
return nil, err
}
return json.Marshal(Judged(facts))
}
return nil, errors.New("nothing asked: give -verdict or -seats")
}
@@ -0,0 +1,99 @@
package rules
import (
"fmt"
"maps"
"slices"
"strings"
"git.novox.be/novox/mesh-contracts/go/seats/meshconfig"
)
// The refusals of what the running checker asked of other seats for a check (milestone 47/G): it asks and records the
// answers as facts, and these rules decide what fails.
// RenamesSeat is the verb a change that brings or drops a `renamed-from` is asked of (design 32).
var RenamesSeat = "seat:" + meshconfig.Seat + "." + meshconfig.VerbRenames
// Renames is what one module of a member brings and drops of `renamed-from` (key to the old key), and how the seat
// mesh-config's verb renames answered for it: Answer when it answered; else Unasked says why it was not asked or did not
// answer, NotServed that its holder does not serve the verb. Unread says the module's manifests could not be read.
type Renames struct {
Module string `json:"module"`
Key string `json:"key"`
Brought map[string]string `json:"brought,omitempty"`
Dropped map[string]string `json:"dropped,omitempty"`
Answer *meshconfig.RenamesAnswer `json:"answer,omitempty"`
Unasked string `json:"unasked,omitempty"`
NotServed bool `json:"not-served,omitempty"`
Unread string `json:"unread,omitempty"`
}
// JudgeRenames fails a rename the seat cannot move the value of, and the drop of a `renamed-from` while its window is
// open on a node; it says which module the seat answered for.
func JudgeRenames(r Renames) (failed []FailedFirst, notes []string) {
at := func(words string) { failed = append(failed, FailedFirst{Machine: meshconfig.Seat, Words: words}) }
switch {
case r.Unread != "":
at(fmt.Sprintf("%s (%s): its config renames cannot be judged: %s", r.Module, r.Key, r.Unread))
return failed, nil
case r.NotServed:
for _, key := range slices.Sorted(maps.Keys(r.Brought)) {
at(fmt.Sprintf("%s (%s) renames %s from %s: refused, %s, so the value cannot be moved", r.Module, r.Key, key, r.Brought[key], r.Unasked))
}
for _, key := range slices.Sorted(maps.Keys(r.Dropped)) {
at(fmt.Sprintf("%s (%s) drops renamed-from %s of %s: refused, %s, so whether its window is open is not known", r.Module, r.Key, r.Dropped[key], key, r.Unasked))
}
return failed, nil
case r.Answer == nil && r.Unasked == "":
at(fmt.Sprintf("%s (%s) changes a renamed-from, and this checker cannot ask %s", r.Module, r.Key, RenamesSeat))
return failed, nil
case r.Answer == nil:
at(fmt.Sprintf("%s (%s) changes a renamed-from, and the check cannot ask %s: %s", r.Module, r.Key, RenamesSeat, r.Unasked))
return failed, nil
}
for _, key := range slices.Sorted(maps.Keys(r.Dropped)) {
for _, n := range r.Answer.Renames {
if n.Module == r.Module && n.From == r.Dropped[key] && n.To == key && len(n.Open) > 0 {
open := slices.Sorted(slices.Values(n.Open))
at(fmt.Sprintf("%s (%s) drops renamed-from %s of %s while its rename window is open on %s", r.Module, r.Key, n.From, key, strings.Join(open, ", ")))
}
}
}
return failed, []string{fmt.Sprintf("%s.%s answered for %s (%s)", meshconfig.Seat, meshconfig.VerbRenames, r.Module, r.Key)}
}
// SeatDefinitionsMachine names what refused a seat definition in a verdict's failure.
const SeatDefinitionsMachine = "seat-definitions"
// UnparsedSeat is one seat definition of a pull request's head that does not parse, as the indexer keeps it in its state
// seat-definitions: the seat (the contracts module's name when its index did not parse), its module's directory and the
// indexer's words of why.
type UnparsedSeat struct {
Seat string `json:"seat"`
Module string `json:"module"`
Directory string `json:"directory"`
Why string `json:"why"`
}
// Unparsed is what the indexer keeps of one pull request's seat definitions that do not parse; Unread says why its state
// could not be read.
type Unparsed struct {
Key string `json:"key"`
Seats []UnparsedSeat `json:"seats,omitempty"`
Unread string `json:"unread,omitempty"`
}
// JudgeUnparsed fails a pull request whose head's seat definition does not parse, naming the seat and where it lives
// (design 26; the lead's ruling on A2i's finding 1), and one whose seat definitions cannot be read, since none is then
// known to parse.
func JudgeUnparsed(u Unparsed) []FailedFirst {
if u.Unread != "" {
return []FailedFirst{{Machine: SeatDefinitionsMachine, Words: fmt.Sprintf("%s: the indexer's seat definitions of it cannot be read, so none is known to parse: %s", u.Key, u.Unread)}}
}
var out []FailedFirst
for _, s := range u.Seats {
out = append(out, FailedFirst{Machine: SeatDefinitionsMachine, Words: fmt.Sprintf("%s: the seat definition of %s (module %s, %s) does not parse: %s", u.Key, s.Seat, s.Module, s.Directory, s.Why)})
}
return out
}
@@ -0,0 +1,63 @@
package rules
import "git.novox.be/novox/mesh-delivery/sdk/buildstep"
// The shapes of a test step's answer, as the build modules' code writes them (mesh-catalog modules/mesh-build-go
// cmd/mesh-build-go levels.go TestResult; modules/mesh-build-node cmd/mesh-build-node check.go TestResult). Only the
// fields the rules read are named; the rest is ignored.
// GoPackage is one package of a Go test result.
type GoPackage struct {
Dir string `json:"dir"`
Package string `json:"package"`
Passed bool `json:"passed"`
Elapsed float64 `json:"elapsed"`
Tests int `json:"tests"`
NoTests bool `json:"no_tests,omitempty"`
// Skipped is how many of the package's tests skipped; an older build module names none.
Skipped int `json:"skipped,omitempty"`
}
// Failure is a test that failed, or a package that failed with no test to name: the build modules' failure contract
// (buildstep.Failure, issue 1901), its cause never empty and never a passing line from a build module that keeps it.
type Failure = buildstep.Failure
// TestTime is one test and the seconds it took.
type TestTime struct {
Package string `json:"package"`
Test string `json:"test"`
Elapsed float64 `json:"elapsed"`
}
// Tested is the answer of a test step of either build module (the Go one names packages and counts tests; the Node one
// names the package directory as Module and counts none).
type Tested struct {
Level string `json:"level"`
Module string `json:"module,omitempty"`
Passed bool `json:"passed"`
Why string `json:"why,omitempty"`
NoTests bool `json:"no_tests,omitempty"`
Tests int `json:"tests"`
Skipped int `json:"skipped"`
Packages []GoPackage `json:"packages,omitempty"`
Elapsed float64 `json:"elapsed"`
Setup float64 `json:"setup,omitempty"`
// Lint is the unit level's gofmt and go vet time, before its tests (mesh-build-go, ADR 0529 §1).
Lint float64 `json:"lint,omitempty"`
Bound float64 `json:"bound"`
SetupBound float64 `json:"setup_bound,omitempty"`
Slowest []TestTime `json:"slowest,omitempty"`
Failed []Failure `json:"failed,omitempty"`
OverBound bool `json:"over_bound"`
OverWhich string `json:"over_which,omitempty"`
// JudgedBy is the program that ran the step (ADR 0535 §5): "installed", or "candidate <merge commit>" under runner
// workspace; absent from a build module that does not take the runner.
JudgedBy string `json:"judged_by,omitempty"`
// By is the build module that answered and the SDK it was built with (buildstep.By); absent from a build module from
// before the failure contract.
By string `json:"by,omitempty"`
// OutOfStep are the generated files the step's regenerate-and-compare found out of step with their seat definitions,
// each a repository-relative path (design 26): the running checker names them in its verdict event. Absent from a
// build module that does not compare.
OutOfStep []string `json:"out-of-step,omitempty"`
}
@@ -0,0 +1,343 @@
package rules
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io/fs"
"maps"
"path"
"slices"
"strings"
"git.novox.be/novox/mesh-delivery/sdk/seatdefs"
)
// The seat list a check judges and composes against (novox/hq ADR 0636 §1, §4, ADR 0644 §1; the lead's rulings Q2 on
// milestones 47/B0 and 47/G): the checking program built from the heads reads the definitions the heads' contracts
// modules carry, by the domain SDK's one reading (sdk/seatdefs), the indexer's too, and answers them; the running checker
// lays them by seat over the list its reader holds from the seat mesh-catalog (verb seat-definitions) and reads no file
// of the heads. The reader's list is not in the program's input: the mesh's definitions are larger than a run input's
// bound (buildstep.RunInputBound).
// SeatsVersion is the version of SeatsAnswer this program writes.
const SeatsVersion = 1
// SeatsAnswer is the checking program's `-seats` answer: every seat definition the heads carry, resolved, as the requests'
// field `seats` carries them, sorted by seat, that is new or differs from the reader's (Facts.SeatDigests); how many
// were the same and left out of it (Same); and each definition left out, with why.
type SeatsAnswer struct {
V int `json:"v"`
Seats []json.RawMessage `json:"seats"`
Same int `json:"same,omitempty"`
Left []LeftOut `json:"left,omitempty"`
}
// SeatDigest is a definition's digest, the same for the same definition whatever its bytes' order and spacing: the sha256
// of its JSON with every object's keys sorted, numbers as written. Bytes that are not JSON are digested as given.
func SeatDigest(raw json.RawMessage) string {
d := json.NewDecoder(bytes.NewReader(raw))
d.UseNumber()
var v any
canonical := []byte(raw)
if d.Decode(&v) == nil {
if out, err := json.Marshal(v); err == nil {
canonical = out
}
}
sum := sha256.Sum256(canonical)
return hex.EncodeToString(sum[:])
}
// LeftOut is a seat definition the list does not take from the heads: its file and why.
type LeftOut struct {
Path string `json:"path"`
Why string `json:"why"`
}
// SeatsOf is the seat definitions of the facts' members' heads that are new or differ from the reader's (Facts.SeatDigests). heads holds each member's head at its repository's path; a contracts module is a manifest there whose package artifact names
// `generated-from` (seatdefs.Of). A seat of the mesh's own a module that is not its domain's defines (seatdefs.Refusal),
// and a seat two modules define where neither hands it over, are left out, said; a definition that does not parse is an
// error.
// Heads with no contracts module give an empty list.
func SeatsOf(heads fs.FS, f Facts) (SeatsAnswer, error) {
out := SeatsAnswer{V: SeatsVersion}
type taken struct {
raw json.RawMessage
module, file string
}
by := map[string]taken{}
refused := map[string]bool{}
repositories := map[string]string{}
for _, m := range f.Members {
repositories[m.Repository] = m.Head
}
for _, repository := range slices.Sorted(maps.Keys(repositories)) {
if _, err := fs.Stat(heads, repository); errors.Is(err, fs.ErrNotExist) {
continue // no head of it is laid: it carries no definition here
}
sub, err := fs.Sub(heads, repository)
if err != nil {
return SeatsAnswer{}, err
}
manifests, err := manifestsIn(sub)
if err != nil {
return SeatsAnswer{}, err
}
forge := localForge(sub)
for _, c := range seatdefs.ContractsModules(manifests) {
defs, unparsed, err := forge.Read(context.Background(), repository, repositories[repository], c)
if err != nil {
return SeatsAnswer{}, fmt.Errorf("the seat definitions of %s at the head of %s cannot be read: %w", c.Module, repository, err)
}
// A head's definition that does not parse, or names another seat than its directory's, leaves the group's seats
// unknown: no list is made, so no manifest is judged by a list without it (design 26).
for _, seat := range slices.Sorted(maps.Keys(unparsed)) {
return SeatsAnswer{}, fmt.Errorf("the seat definition of %s (module %s) at the head of %s is not taken: %s", seat, c.Module, repository, unparsed[seat])
}
for _, seat := range slices.Sorted(maps.Keys(defs)) {
d, file := defs[seat], defFile(repository, c, seat)
if why := seatdefs.Refusal(c.Module, d); why != "" {
out.Left = append(out.Left, LeftOut{Path: file, Why: why})
continue
}
if prev, twice := by[seat]; twice || refused[seat] {
switch {
case refused[seat]:
out.Left = append(out.Left, LeftOut{Path: file, Why: "the seat " + seat + " is defined in more than one place"})
continue
case seatdefs.HandsOver(prev.module, c.Module, d):
case seatdefs.HandsOver(c.Module, prev.module, d):
continue
default:
why := "the seat " + seat + " is defined in more than one place: " + prev.file + ", " + file
out.Left = append(out.Left, LeftOut{Path: prev.file, Why: why}, LeftOut{Path: file, Why: why})
delete(by, seat)
refused[seat] = true
continue
}
}
raw, err := json.Marshal(d)
if err != nil {
out.Left = append(out.Left, LeftOut{Path: file, Why: err.Error()})
continue
}
by[seat] = taken{raw: raw, module: c.Module, file: file}
}
}
}
out.Seats = make([]json.RawMessage, 0, len(by))
for _, seat := range slices.Sorted(maps.Keys(by)) {
if held, ok := f.SeatDigests[seat]; ok && held == SeatDigest(by[seat].raw) {
out.Same++ // the reader holds it as the heads carry it: the checker's own list keeps it
continue
}
raw := by[seat].raw
// The answer is the judging view of a seat, what the manifest validator and the composer read (the lead's ruling after
// the group check of 616c42434ac2), with digests or without: no verb's or message's schema, which neither reads, and
// which the build seat's step state, keeping this answer, would refuse where a property is named like a credential
// (mesh-vault's verb input `credentials`).
raw = withoutMessageSchemas(raw)
if f.SeatDigests == nil {
// A checker from before G hands no digests, so every definition is answered: whole they are over a run answer's
// bound, so every description but the seat's own is left out too, which decides no verdict; review round 2 of G.
raw = withoutInnerDescriptions(raw)
}
out.Seats = append(out.Seats, raw)
}
return out, nil
}
// defFile is a seat's definition file in a head, by its repository's path.
func defFile(repository string, c seatdefs.Module, seat string) string {
return path.Join(repository, c.Defs, "seats", seat, "seat.json")
}
// manifestsIn are the raw manifests of a head by their directory (the root's is "."), the generated contracts, the tests'
// data and the packages' dependencies left out.
func manifestsIn(head fs.FS) (map[string]string, error) {
out := map[string]string{}
err := fs.WalkDir(head, ".", func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() && (d.Name() == ".git" || d.Name() == "testdata" || d.Name() == "node_modules") {
return fs.SkipDir
}
if !d.IsDir() && d.Name() == "module.json" {
raw, err := fs.ReadFile(head, p)
if err != nil {
return err
}
out[path.Dir(p)] = string(raw)
}
return nil
})
return out, err
}
// localForge is the domain SDK's reading bound to a head laid on disk: a directory or a file the head does not hold is
// none.
func localForge(head fs.FS) seatdefs.Forge {
return seatdefs.Forge{
Files: func(_ context.Context, _, _ string, dirs []string) (map[string]map[string]string, error) {
out := map[string]map[string]string{}
for _, dir := range dirs {
files := map[string]string{}
err := fs.WalkDir(head, dir, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if !d.IsDir() {
files[strings.TrimPrefix(p, dir+"/")] = ""
}
return nil
})
if errors.Is(err, fs.ErrNotExist) {
continue
}
if err != nil {
return nil, err
}
out[dir] = files
}
return out, nil
},
File: func(_ context.Context, _, _, file string) ([]byte, error) {
raw, err := fs.ReadFile(head, file)
if errors.Is(err, fs.ErrNotExist) {
return nil, nil
}
return raw, err
},
}
}
// withoutMessageSchemas is a definition whose verbs' input and answer, and whose accepted messages', events' and holder
// events' bodies, are each the empty schema, and whose types (`defs`) are only those something it keeps still refers to
// (a state's key forms, a provision's form, another kept type). Every other field is kept as written; bytes that are not
// a JSON object are given back as they are.
func withoutMessageSchemas(raw json.RawMessage) json.RawMessage {
d := json.NewDecoder(bytes.NewReader(raw))
d.UseNumber()
var def map[string]any
if d.Decode(&def) != nil {
return raw
}
empty := func(list any, fields ...string) {
items, _ := list.([]any)
for _, it := range items {
m, ok := it.(map[string]any)
if !ok {
continue
}
for _, f := range fields {
if _, has := m[f]; has {
m[f] = map[string]any{}
}
}
}
}
empty(def["verbs"], "input", "answer")
for _, field := range []string{"accepts", "events", "holder-events"} {
empty(def[field], "body")
}
if defs, ok := def["defs"].(map[string]any); ok {
rest := maps.Clone(def)
delete(rest, "defs")
kept := map[string]bool{}
todo := typesReferred(rest)
for len(todo) > 0 {
name := todo[len(todo)-1]
todo = todo[:len(todo)-1]
if kept[name] || defs[name] == nil {
continue
}
kept[name] = true
todo = append(todo, typesReferred(defs[name])...)
}
maps.DeleteFunc(defs, func(name string, _ any) bool { return !kept[name] })
if len(defs) == 0 {
delete(def, "defs")
}
}
out, err := json.Marshal(def)
if err != nil {
return raw
}
return out
}
// typesReferred are the names of the definition's types (`{"$ref": "#/defs/<name>"}`) v refers to, anywhere in it.
func typesReferred(v any) []string {
var out []string
var walk func(any)
walk = func(v any) {
switch t := v.(type) {
case map[string]any:
if ref, ok := t["$ref"].(string); ok {
if name, found := strings.CutPrefix(ref, "#/defs/"); found {
out = append(out, name)
}
}
for _, x := range t {
walk(x)
}
case []any:
for _, x := range t {
walk(x)
}
}
}
walk(v)
return out
}
// schemaData are the keywords of a schema whose value is data, never a schema: what they hold is kept as written.
var schemaData = map[string]bool{"examples": true, "enum": true, "const": true, "default": true}
// withoutInnerDescriptions is a definition with only the seat's own description kept: every other `description` string,
// of a verb, an event or anything inside their schemas, is left out, every other field kept (a property named
// description is a schema, kept; a schema's data keywords are kept whole). Bytes that are not a JSON object are given
// back as they are.
func withoutInnerDescriptions(raw json.RawMessage) json.RawMessage {
d := json.NewDecoder(bytes.NewReader(raw))
d.UseNumber()
var def map[string]any
if d.Decode(&def) != nil {
return raw
}
var walk func(v any)
walk = func(v any) {
switch t := v.(type) {
case map[string]any:
for k, val := range t {
if _, prose := val.(string); prose && k == "description" {
delete(t, k)
continue
}
if !schemaData[k] {
walk(val)
}
}
case []any:
for _, e := range t {
walk(e)
}
}
}
for k, val := range def {
if k != "description" {
walk(val)
}
}
out, err := json.Marshal(def)
if err != nil {
return raw
}
return out
}
@@ -0,0 +1,282 @@
package rules
import (
"fmt"
"strings"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliverycheckerclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdeliverychecker"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun"
"git.novox.be/novox/mesh-delivery/sdk/buildstep"
)
// The words rules: how a failed test, a failed package and a list of failures are worded in a verdict (issue 1901, issue
// 1921). Moved whole from the running checker's package (milestone 47/B0): they are the checking program's.
// FailedName names a failure: its package and test.
func FailedName(f Failure) string {
if f.Test == "" {
return f.Package
}
return f.Package + "." + f.Test
}
// MaxWords is the most a failed test's words in a verdict hold (issue 1901). **The words keep the failure's last lines**,
// where a test says why it failed: the first line is often something it logged before (the group r4g verdict kept a
// "judged against" line and lost the t.Fatalf after it), and the 120 characters kept before cut a docker command's echo
// before docker's own error. Four kilobytes holds a long message and is small beside a verdict's other parts.
const MaxWords = 4096
// FailureWords is a failure's words: its output's last lines up to MaxWords, with the line the build module told apart as
// the test's error first when the output does not start with it, so the first line of the words is the cause and the
// status line, which is cut short, names it. **A failure's words are never "PASS" or "ok"** (issue 1901, its third
// meeting: a package that failed after its tests passed was worded "PASS"): go test's summary lines are dropped, and with
// no cause told apart (an older build module) the cause is the output's first "panic:", "fatal error:" or "signal:"
// line, read before the output is cut, or the words say the tests passed and the program failed after them.
func FailureWords(f Failure) string {
return FailureWordsAfter(f, "")
}
// FailureWordsAfter is FailureWords with the line loud put first, within the same MaxWords: what the checker itself says of
// the failure (that the build module hid it), which the status line then names.
func FailureWordsAfter(f Failure, loud string) string {
out, passed := DropSummary(strings.TrimSpace(f.Output))
cause := strings.TrimSpace(f.Cause)
if cause == "" {
cause = CauseIn(out)
}
switch {
case cause == "" && out == "" && passed:
cause = "the tests passed, then the test program exited with a failure: an os.Exit or a panic in TestMain or after the tests"
case cause == "" && out == "":
// The build module on main sends a package that stopped (a timeout, an exit, a kill) as its "FAIL" line alone.
cause = "the package failed with no test to name and no panic, signal or build error in its output (an older build module sent no cause)"
}
if len(cause) > MaxCause {
cause = strings.ToValidUTF8(cause[:MaxCause], "") + "..."
}
// The words keep MaxWords exactly: the loud line, the cause and their newlines, the marker and the kept lines.
budget := MaxWords
if cause != "" {
budget -= len(cause) + 1
}
if loud != "" {
budget -= len(loud) + 1
}
if len(out) > budget {
const marker = "... "
tail := out[len(out)-(budget-len(marker)):]
// Start on a whole line, and a whole character.
if i := strings.IndexByte(tail, '\n'); i >= 0 {
tail = strings.TrimLeft(tail[i:], "\n")
}
out = marker + strings.ToValidUTF8(tail, "")
}
if cause != "" && !strings.HasPrefix(FirstLine(out), cause) {
out = strings.TrimRight(cause+"\n"+out, "\n")
}
if loud != "" {
out = strings.TrimRight(loud+"\n"+out, "\n")
}
return out
}
// HidWords is the checker's loud line for a failure the build module sent hiding its cause (issue 1901, its thorough
// fix): with no cause, or a passing line as its cause, under the failure contract (buildstep.Hidden). It names the build
// module and the version it says it is (Tested.By, "<module> with mesh-sdk <version>"); one from before the contract
// says nothing of itself. "" when the failure keeps the contract. The check fails either way: only its words change.
func HidWords(module, by string, f Failure) string {
why := buildstep.Hidden(f)
if why == "" {
return ""
}
version := "a version before the failure contract"
if module == "" {
module = "an unnamed build module"
}
if m, v, ok := strings.Cut(by, " with "); ok && m != "" {
module, version = m, v
}
return fmt.Sprintf("the build module hid the failure (%s at %s): %s", module, version, why)
}
// MaxCause is the most a cause in a failure's words holds: one long line (the 1901 review: a cause of a kilobyte could
// push the words past MaxWords).
const MaxCause = 1024
// DropSummary drops go test's summary lines from a failure's output ("PASS", "FAIL", "ok <pkg>", "FAIL <pkg>"), and says
// whether a "PASS" was among them: the package's tests passed before it failed.
func DropSummary(out string) (string, bool) {
passed := false
lines := strings.Split(out, "\n")
kept := lines[:0]
for _, l := range lines {
// go's exact forms only (review round 1): "PASS" or "FAIL" alone, "ok \t<pkg>\t<time>", "FAIL\t<pkg>\t<time>".
switch t := strings.TrimRight(l, "\r"); {
case t == "PASS":
passed = true
case t == "FAIL", strings.HasPrefix(t, "ok \t"), strings.HasPrefix(t, "FAIL\t"):
default:
kept = append(kept, l)
}
}
return strings.TrimSpace(strings.Join(kept, "\n")), passed
}
// CauseIn is a race the race detector reported, as its heading, its access and the frame that made it ("WARNING: DATA
// RACE: Write at 0x... by goroutine 9 in pkg.F.func1()"); else the first "panic:", "fatal error:" or "signal:" line of a
// failure's output, in that order; or "". **Only lines at column 0**, where go prints the runtime's: a test's indented log
// line holding the same words is not the cause (review round 1).
func CauseIn(out string) string {
lines := strings.Split(out, "\n")
for i, l := range lines {
if strings.TrimRight(l, "\r") != "WARNING: DATA RACE" {
continue
}
words := "WARNING: DATA RACE"
if i+1 < len(lines) && strings.TrimSpace(lines[i+1]) != "" {
words += ": " + strings.TrimSuffix(strings.TrimSpace(lines[i+1]), ":")
if i+2 < len(lines) && strings.HasPrefix(lines[i+2], " ") {
words += " in " + strings.TrimSpace(lines[i+2])
}
}
return words
}
for _, prefix := range []string{"panic: ", "fatal error: ", "signal: "} {
for _, l := range lines {
if l = strings.TrimRight(l, "\r"); strings.HasPrefix(l, prefix) {
return l
}
}
}
return ""
}
// FirstLine is the first non-empty line of a failure's words.
func FirstLine(s string) string {
for _, l := range strings.Split(s, "\n") {
if l = strings.TrimSpace(l); l != "" {
return l
}
}
return ""
}
// MaxFailureLine is the most bytes of a failure's line in a verdict's Failures: the first line of its words, which is
// its cause where the build module told one apart (issue 1901). Failed keeps the first failure's words whole.
const MaxFailureLine = 300
// LevelFailure turns a level that did not pass into a verdict: Failed names what failed first with its words whole,
// and Failures every failed test of the level's results with its first line, at most meshdeliverycheckerclient.MaxFailures and an entry
// saying how many more (issue 1921).
func LevelFailure(repository, module, level string, results ...Tested) *meshdeliverychecker.VerdictsByGroupHeadsValue {
v := meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail}
var all []FailedFirst
for _, res := range results {
if len(res.Failed) == 0 {
why := res.Why
if why == "" {
why = "the " + level + " level did not pass"
}
all = append(all, FailedFirst{Test: level + ": " + repository, Words: why})
continue
}
for _, f := range res.Failed {
name := FailedName(f)
if module == buildrun.NodeBuildModule && f.Package == "" {
name = res.Module
}
all = append(all, FailedFirst{Test: level + ": " + name, Words: FailureWordsAfter(f, HidWords(module, res.By, f))})
}
}
if len(all) == 0 { // no result: a level said failed with nothing to name
all = append(all, FailedFirst{Test: level + ": " + repository, Words: "the " + level + " level did not pass"})
}
first := all[0]
v.Failed = &first
v.Failures = failureItems(all) // whole: Settled cuts them to their first lines once every failure of the check is joined
return &v
}
// JoinFailures is one failed verdict of every failure of vs, in the order given (a nil one passed), unsettled: Failed is
// the first whole, and Failures every failure whole, or none when the one failure is no level's (a conflict, a refused
// step: one line, as before issue 1921). Nil when none failed.
func JoinFailures(vs ...*meshdeliverychecker.VerdictsByGroupHeadsValue) *meshdeliverychecker.VerdictsByGroupHeadsValue {
var out *meshdeliverychecker.VerdictsByGroupHeadsValue
var all []FailedFirst
listed := false
for _, v := range vs {
if v == nil {
continue
}
if out == nil {
out = &meshdeliverychecker.VerdictsByGroupHeadsValue{State: meshdeliverycheckerclient.VerdictFail, Failed: v.Failed}
}
if len(v.Failures) > 0 {
all, listed = append(all, failedList(v.Failures)...), true
continue
}
all = append(all, failedOf(*v))
}
if out != nil && (listed || len(all) > 1) {
out.Failures = failureItems(all)
}
return out
}
// Settled is a joined verdict as it is written: each failure of its list cut to its first line, at most
// meshdeliverycheckerclient.MaxFailures and then how many more (FailureLines).
func Settled(v *meshdeliverychecker.VerdictsByGroupHeadsValue) *meshdeliverychecker.VerdictsByGroupHeadsValue {
if v != nil && len(v.Failures) > 0 {
v.Failures = failureItems(FailureLines(failedList(v.Failures)))
}
return v
}
// AddFailure is a Settled list of failures with one more at its end: the first failure is put first when the list is
// empty (a verdict of one failure keeps none), and past meshdeliverycheckerclient.MaxFailures the last entry counts one more.
func AddFailure(v *meshdeliverychecker.VerdictsByGroupHeadsValue, f FailedFirst) {
if len(v.Failures) == 0 {
v.Failures = failureItems(FailureLines([]FailedFirst{failedOf(*v)}))
}
if n := len(v.Failures); n > meshdeliverycheckerclient.MaxFailures {
v.Failures[n-1].Test = fmt.Sprintf("and %d more", FailedCount(*v)-meshdeliverycheckerclient.MaxFailures+1)
return
}
v.Failures = append(v.Failures, failureItems(FailureLines([]FailedFirst{f}))...)
}
// FailureLines is every failure with the first line of its words, cut to MaxFailureLine, at most meshdeliverycheckerclient.MaxFailures
// and then one entry saying how many more failed.
func FailureLines(all []FailedFirst) []FailedFirst {
n := min(len(all), meshdeliverycheckerclient.MaxFailures)
out := make([]FailedFirst, 0, n+1)
for _, f := range all[:n] {
out = append(out, FailedFirst{Test: f.Test, Machine: f.Machine, Words: ClipLine(FirstLine(f.Words), MaxFailureLine)})
}
if more := len(all) - n; more > 0 {
out = append(out, FailedFirst{Test: fmt.Sprintf("and %d more", more)})
}
return out
}
// ClipLine cuts a line to at most n bytes on a whole character, marking the cut.
func ClipLine(s string, n int) string {
if len(s) <= n {
return s
}
s = strings.ToValidUTF8(s[:n-3], "")
return s + "..."
}
// FailedCount is how many failures a verdict names, the ones after meshdeliverycheckerclient.MaxFailures counted too.
func FailedCount(v meshdeliverychecker.VerdictsByGroupHeadsValue) int {
n := len(v.Failures)
if n > meshdeliverycheckerclient.MaxFailures {
var more int
if _, err := fmt.Sscanf(v.Failures[n-1].Test, "and %d more", &more); err == nil {
return meshdeliverycheckerclient.MaxFailures + more
}
}
return n
}
@@ -152,12 +152,10 @@ func TestManifestProcess(t *testing.T) { // 1235/AC1
if _, bus := p["bus"]; bus {
t.Errorf("the process declares a bus: it runs on its own broker credential")
}
if len(m.Build.Artifacts) != 3 || m.Build.Artifacts[1].Name != "tools" || m.Build.Artifacts[1].From != "cmd/mesh-delivery-forge-bridge-tools" {
t.Fatalf("artifacts = %+v, want the program, the tools bundle and the package forgemsg", m.Build.Artifacts)
}
// ADR 0573 §1: the gate requires forgemsg by a published version, so the module declares it as a package.
if a := m.Build.Artifacts[2]; a.Name != "forgemsg" || a.Kind != "package" || a.Language != "go" || len(a.Paths) != 1 || a.Paths[0] != "forgemsg" || a.Series != "0.1" {
t.Errorf("package artifact = %+v", a)
// Issue 1966's bootstrap: the package artifact forgemsg is out of the manifest until the new builder runs; the follow-up
// puts it back with its assertion (ADR 0573 §1: three artifacts, the third the package forgemsg).
if len(m.Build.Artifacts) != 2 || m.Build.Artifacts[1].Name != "tools" || m.Build.Artifacts[1].From != "cmd/mesh-delivery-forge-bridge-tools" {
t.Fatalf("artifacts = %+v, want the program and the tools bundle", m.Build.Artifacts)
}
a := m.Build.Artifacts[0]
if st, err := os.Stat(a.From); err != nil || !st.IsDir() || a.Binary != "mesh-delivery-forge-bridge" || !strings.HasSuffix(a.From, a.Binary) {
@@ -95,15 +95,6 @@
"loads": [
"mesh-delivery-forge-bridge-tools"
]
},
{
"name": "forgemsg",
"kind": "package",
"language": "go",
"paths": [
"forgemsg"
],
"series": "0.1"
}
]
}
@@ -412,7 +412,7 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
seatReader, saySeats := followSeatList(seatBus, saySeatDefinition(js, cred), logger.Printf)
readWhenBound(seatReader, seatDefs, marks)
ix := &index.Indexer{Findings: awaitedStore{findings}, OldSections: awaitedStore{oldSections}, Requirements: awaitedStore{requirements}, Store: kvStore{candidates}, Modules: kvStore{modules}, Manifests: kvStore{manifests}, Taken: awaitedStore{taken}, Builds: known,
Verdicts: awaitedStore{verdicts}, SayRefused: sayRefused(js, cred, logger.Printf), SayAccepted: sayAccepted(js, cred),
Verdicts: awaitedStore{verdicts}, SayRefused: sayRefused(conn, cred, logger.Printf), SayAccepted: sayAccepted(conn, cred),
Predefined: awaitedStore{predefined}, ConfigCandidates: awaitedStore{configCandidates}, Logf: logger.Printf,
Named: namedVia(calls.Call), Trunk: trunkVia(calls.Call), Contains: containsVia(calls.Call), Read: readVia(calls.Call),
ReadOpen: openVia(calls.Call), Packages: awaitedStore{packages}, Files: filesVia(calls.Call), GoSum: goSumVia(calls.Call), GoMod: goModVia(calls.Call),
@@ -668,8 +668,10 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
// stream starts (ADR 0633 §2). When the build this program runs is not known yet, nothing waits for it: the consumer
// starts, and a merged message left unacknowledged asks the build again at its redelivery (clause 1). A hand take-in
// (indexer_take_in_pending) is answered at any time: the takeins watch runs already.
// A take-in that fails (a bus error, or a merge lost because the forge bridge did not answer) does not end the process (issue
// 1585, note 70): what pending holds is evaluated again at the next merged message, toolchain report or hand take-in.
if _, words, err := ix.TakeInAtStart(ctx); err != nil {
return fmt.Errorf("the take-in at start: %w", err)
logger.Printf("the take-in at start failed and is asked again at the next merged message, toolchain report or hand take-in: %v", err)
} else if words != "" {
logger.Printf("take-in at start: %s", words)
}

Some files were not shown because too many files have changed in this diff Show More