Compare commits

..
Author SHA1 Message Date
jschoubben 0300a6fd99 pacman: drop the plain-word phrases from replaces (it holds shell commands only, ADR 0245); the descriptions' first lines carry them
mesh/delivery ready: it is merged on the Approve
mesh/check pass 0300a6fd99b5 on b1e610b2135a | unit 2030 run, 2327 kept, 6 skipped | integration 2050 run, 2346 kept, 13 skipped | over bound: unit, i…
2026-10-26 14:04:09 +01:00
jschoubben b3221b6ab8 pacman: a plain-words search finds pacman_round and pacman_upgrades first (answer on the first line, phrases they replace) 2026-10-26 13:27:09 +01:00
mesh-tools 879600202a Merge pull request 'Vault: adopt the hashes of bus accounts it holds no value for, handed sealed by the bus node (issue 1945, ADR 0550 clause 4)' (#494) from feat/1945-user-list-switch into main 2026-10-26 12:21:47 +00:00
mesh-tools 565d0a848a Merge pull request 'Gate: merge turns, a merge waits for a running check of its repository, merges on a carried verdict with no second ask (ADR 0548, issue 568)' (#493) from feat/548-one-check into main 2026-10-26 12:01:28 +00:00
mesh-tools 71e094b2de Merge pull request 'mesh-build-go: count skipped tests per package, so the checker keeps no pass that skipped (issue 890)' (#492) from fix/checker-skips-not-kept into main 2026-10-26 12:01:13 +00:00
mesh-tools f96704fa0f Merge pull request 'mesh-build-node tests: never install a toolchain from the registry in the integration level; skip as NOT RUN when none is installed (issue 890)' (#491) from fix/build-node-tests-offline into main 2026-10-26 11:34:17 +00:00
mesh-tools 86fdf74a0a Merge pull request 'Desk channel: keep what was shown in memory while the store does not answer, so a clearing replaces the urgent notification (issue 384)' (#490) from fix/desk-clear-lost into main 2026-10-26 11:33:47 +00:00
mesh-tools 843281c0e5 Merge pull request 'Knowledge base: withdraw a clause's 'changed by' mark once no accepted record changes it (issue 466)' (#489) from fix/kb-stale-marks into main 2026-10-26 11:33:34 +00:00
jschoubben e9279ce3c2 gate: find an approved merge by its ask, not by which member's run was seen first, so a group's turns are written before its first member is handed at every look (ADR 0548 §3; issue 568, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-catalog@565d0a848a85
mesh/delivery-group group feat/548-one-check merged: every member is merged
mesh/check checking e9279ce3c209 on 565d0a848a85 | novox/mesh-catalog#493 in group feat/548-one-check
2026-10-26 12:32:46 +01:00
jschoubben c9414b1c98 Vault: pin mesh-sdk at the switch group's head c2a410bb, one sdk for the adoption and the switch (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-catalog@879600202a09
mesh/delivery-group group feat/1945-user-list-switch merged: every member is merged
mesh/check checking c9414b1c9856 on 879600202a09 | novox/mesh-catalog#494 in group feat/1945-user-list-switch
2026-10-26 12:31:56 +01:00
jschoubben ea53b2f920 mesh-build-go: each package counts its skipped tests (issue 890)
mesh/delivery merged: source.main moved to novox/mesh-catalog@71e094b2de59
mesh/delivery-group group fix/checker-skips-not-kept merged: every member is merged
mesh/check failed: ace: the check of novox/mesh-catalog#492 refused: the installed go version could not be read: `go version` with GOTOOLCHAIN=local f…
2026-10-26 12:29:19 +01:00
jschoubben 99cc073e2a gate: pin mesh-sdk at the group's head 9c61918ac36f, where a turn never holds the check of heads waiting in it (ADR 0548 §3; issue 568, review round 1) 2026-10-26 12:15:20 +01:00
jschoubben b94150fcdb gitea: the merge tool reads a carried verdict's line as judged on the new trunk, so it merges a carried verdict (ADR 0548 §8; issue 568, review round 1) 2026-10-26 12:15:20 +01:00
jschoubben a01715ddf5 Skip the Node build module's toolchain tests by name where no Node toolchain is installed, instead of installing one from the registry in a level that has no network; merge three tests that share one fixture
mesh/delivery merged: source.main moved to novox/mesh-catalog@f96704fa0ff4
mesh/check checking a01715ddf508 on 565d0a848a85
On ace, which holds no Node build module, five integration tests each spent 70 s in npm's network retries and failed every mesh-catalog check run there (five verdicts since 00:07). The tests now copy the node's installed toolchain or are skipped as NOT RUN; the registry is never reached. Three tests sharing the fixture are one, so the toolchain is copied three times instead of five (10 integration tests in the module, now 8).
2026-10-26 12:05:09 +01:00
jschoubben e2136ec0c9 The vault takes a handing whole once, refusals and all, and names the refused users in its last adoption (review of the adoption, ADR 0547 clause 9) 2026-10-26 12:03:14 +01:00
jschoubben a30c168512 desk-channel: take the work when the store does not answer, and keep what it holds at the next work and the minute's standing
mesh/delivery merged: source.main moved to novox/mesh-catalog@86fdf74a0a83
mesh/check checking a30c16851246 on 565d0a848a85
Review round 1: offering the work again let a redelivered raise follow its own clearing and
replace it, and a redelivery is bounded where an outage is not. The work is now taken; what
was shown stays in memory, recalled first, and is kept at the start of the next work and at
the minute's standing, stopping at the first write the store does not answer. A desk that
restarts before then loses where those are shown, and its log said how many it held.
2026-10-26 11:46:21 +01:00
jschoubben f885250913 desk-channel: keep where a message is shown when the store does not answer, so its clearing still replaces it
On 2026-10-26 09:18 the bus reloading its accounts made the desk's state store answer
'no responders' just as the urgent 'A new build of mesh-controller cannot be delivered'
was shown. The desk logged that it could not keep the notification ids and acked the work;
the router's clearing at 09:34 then found no record, was shown as a new notification, and
the urgent one stayed on the operator's screen for hours.

Now what was shown and not kept is held in memory, recalled before the store by edits,
reminders, clicks and a notifier restart, kept at the next piece of work, and its work is
offered again until the store keeps it, without being shown twice. An edit of a message
nothing records is said in the journal.
2026-10-26 11:46:09 +01:00
jschoubben 7627503d9d gate: pin mesh-sdk at the group's head f5c233a9288c, which holds the merge turns and the carried verdict (ADR 0548 §10; issue 568) 2026-10-26 11:45:30 +01:00
jschoubben 351c7c4bf4 gate: deliveries and show say an approved merge waits for the check of the named delivery on the named repository, and the turns bucket is read by the checker alone, written by the gate alone (ADR 0548 §3, §9; issue 568 G6) 2026-10-26 11:44:29 +01:00
jschoubben 55a26b22cf gate: delivery_turns reads every merge turn with the check it waits for, and delivery_hand_now hands a held merge at once on the operator's word (ADR 0548 §4; issue 568 G5) 2026-10-26 11:43:40 +01:00
jschoubben 100e74f283 gate: an approved merge whose trunk moved waits for its carried verdict and merges on it with no second ask, never on the old one (ADR 0548 §8; issue 568 G4) 2026-10-26 11:43:18 +01:00
jschoubben c8e1e2b2fa gate: a merge held past 30 minutes by another delivery's check is a stalled line naming that check and since when, said as delivery.<id>.stalled (ADR 0548 §4; issue 568 G3) 2026-10-26 11:41:42 +01:00
jschoubben d0d00f922a gate: an approved merge writes its turn on every repository it goes into before it is handed, kept for a group until its last member merged, read by the checker (ADR 0548 §3; issue 568 G2) 2026-10-26 11:41:26 +01:00
jschoubben c2bd7760b2 gate: hand no approved merge into a repository a running check of other heads judges, and hand it on that check's end (ADR 0548 §2; issue 568 G1) 2026-10-26 11:40:35 +01:00
jschoubben f0fd53281b The vault says once why a look of the bus module's have adopted nothing, and again when it changes (ADR 0547 clause 9) 2026-10-26 11:37:14 +01:00
jschoubben fb039250f6 The vault adopts the hashes of the bus accounts it holds no value for from the bus node's handing (ADR 0547 clause 9, issue 1945)
take-over waits on a hash for every user of the loaded list, and the vault holds no value for the controller's own
account, the node accounts and the enrolled devices. The bus node's engine now hands every loaded user's hash in the
bus module's have, sealed to the vault's key (mesh-host and mesh-sdk on the same branch). The vault answers its key in
node-key, reads the haves (manifest reads node-engine.have, granted from the manifest, no controller change), watches
the bus module's have, opens the handing with its own key, and writes each user it holds no value for into
bus-accounts marked adopted, with the list and node it came from, saying account-hashed. It never touches a user whose
covered account it holds or whose key is its own hash, writes an adopted key again only for another hash, and refuses
a handing from any node the assignments do not name as holding the bus module.

Tools: account_adopt adopts now from the have as the watch does; secret_own, unnarrowed by module, lists the adopted
accounts and the last adoption.
2026-10-26 11:36:33 +01:00
jschoubben 447e6a13cd Knowledge base: withdraw a clause's changed mark when its changing record's accepted links drop it, on every judgement and by a tool (stale marks)
mesh/delivery merged: source.main moved to novox/mesh-catalog@843281c0e5f0
mesh/check checking 447e6a13cd68 on 565d0a848a85
2026-10-26 11:22:12 +01:00
mesh-tools c51a46b802 Merge pull request 'Build modules: run every package's tests of a level, with a fixture of two failing packages (issue 1921)' (#488) from fix/1921-every-failure-r into main 2026-10-26 10:18:08 +00:00
jschoubben 15a5fe09b6 mesh-build-go, mesh-build-node: run every package of a check's level after a failure (issue 1921)
mesh/delivery merged: source.main moved to novox/mesh-catalog@c51a46b802d3
mesh/delivery-group group fix/1921-every-failure-r merged: every member is merged
mesh/check checking 15a5fe09b62e on c51a46b802d3 | novox/mesh-catalog#488 in group fix/1921-every-failure-r
Pins mesh-sdk fix/1921-every-failure-r, whose check ask no longer stops a level at its first failing test step. A captured fixture with a failing test in each of two packages proves the build module's test step names both.
2026-10-26 10:58:07 +01:00
mesh-tools 3ded9e7800 Merge pull request 'Vault: name a hash record's secret in a field called of, so the runtime keeps it (issue 1945)' (#487) from fix/1945-vault-hash-field into main 2026-10-26 09:00:51 +00:00
jschoubben e7aaa8420e Vault: name a hash record's secret in a field called of, so the runtime accepts it
mesh/delivery merged: source.main moved to novox/mesh-catalog@3ded9e780043
mesh/check pass e7aaa8420eac on 3ded9e780043 | unit 2018 run, 2325 kept, 5 skipped | integration 2040 run, 2344 kept, 9 skipped | over bound: unit, in…
The runtime refuses a state field whose name ends in secret, so no bus-account hash could be written on novox. The test state now applies the same check.

Issue: 1945
2026-10-26 09:25:35 +01:00
mesh-tools 22381d7522 Merge pull request 'Vault: read own secrets' generations, and hash each bus account it is handed for the grants part (issue 1945, ADR 0547 clause 9)' (#486) from fix/1945-vault-own-read into main 2026-10-26 07:48:03 +00:00
jschoubben 1c55e5d44d Vault: take and hash a process's own bus account (broker-<process id>) as a covered account, so take-over is not held by every process user
mesh/delivery merged: source.main moved to novox/mesh-catalog@22381d75224f
mesh/check failed: integration: meshbuildnode/cmd/mesh-build-node.TestAFixtureBundleBuiltTwiceHasOneDigest: no toolchain of this lock is installed und…
The controller hands each process account of a core or moved module with origin issued (ADR 0436 §2); the vault took
only broker and provisioner-broker, refused these and never hashed them. Owed hashes are now said once per pass, and
account_hash says who asked when it refuses too (review round 1).

Issue: 1945
2026-10-26 02:35:57 +01:00
jschoubben 32b912cfe5 Vault: hash each bus account it is handed into bus-accounts.<user>, once per generation, so the grants part can compose the user list without the controller
The controller still mints each covered account and hands it to the vault (ADR 0374); ADR 0547 clause 9 has the vault
write the bcrypt hash of what it holds into its own state, readable by the grants part, and say account-hashed. It
hashes on keep, at start every held account with no key of its generation, and again by hand (account_hash); a refused
write is owed and written at the next pass. secret_own says per bus account whether this generation's hash is written.

Issue: 1945
2026-10-26 02:28:28 +01:00
jschoubben e572937d97 Vault: secret_own reads the module-own secrets it keeps, as fingerprints, so a generation can be told without the value
mesh/delivery closed: a newer head of the same pull request
mesh/check pass e572937d97fa on 567736f7795a | unit 2143 run, 2198 kept, 5 skipped | integration 2165 run, 2217 kept, 9 skipped | over bound: unit, in…
Issue: 1945
2026-10-26 02:09:03 +01:00
mesh-tools 567736f779 Merge pull request 'Build modules' step and toolchain lists read only the newest keys they answer, in parallel (issue 1827)' (#485) from fix/1827-bounded-reads into main 2026-10-25 23:56:12 +00:00
jschoubben e901769cb2 Read the newest steps first: sort the node's keys by id (start order) before the cut, and say older steps were not read
mesh/delivery merged: source.main moved to novox/mesh-catalog@567736f7795a
mesh/check failed: integration: meshbuildnode/cmd/mesh-build-node.TestAFixtureBundleBuiltTwiceHasOneDigest: no toolchain of this lock is installed und…
The review found a cut read took an arbitrary 1000 keys, so the newest steps could be missing. A step id is
base-36 milliseconds plus random digits, so its string order is its start order; the newest max(limit*4, 100)
keys are read now, 1000 at most. The toolchains read logs when it drops keys past its cap.
2026-10-26 00:43:53 +01:00
jschoubben 8f9c78bcba Read only the node's own steps, in parallel and bounded, so build_*_steps answers within the call's 30 s (issue 1827)
On a steps state of 1164 keys / 20 MB (mesh-build-node: 10034 keys) the read tools listed every key and
then read each value one round trip at a time, filtering by node only after the Get, so they timed out
at 30 s. Now the keys are filtered by the node's prefix before any Get, the values are read 16 at a
time, at most 1000 a call (the answer says it was cut), and one log line gives rows read and time.
build_*_toolchains reads its values the same parallel way. The kind: a read tool that reads a whole
state key by key; every read tool of the three modules is covered.
2026-10-26 00:39:33 +01:00
mesh-tools 82e7e3e326 Merge pull request 'mesh-provisioner-client: a module that leaves or is placed again asks the vault afresh (issue 1945)' (#484) from fix/1945-held-secret-on-leave into main 2026-10-25 23:21:44 +00:00
jschoubben 363a3d3f7e Provisioner client: drop a module's held vault secrets when it leaves or is placed again, so it asks afresh (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-catalog@82e7e3e32675
mesh/check pass 363a3d3f7e8f on 82e7e3e32675 | unit 2015 run, 2322 kept, 5 skipped | integration 2037 run, 2341 kept, 9 skipped | over bound: unit, in…
A re-assigned module was filled with the generation it held before the vault minted a new one; the bus refused the account.
2026-10-26 00:03:43 +01:00
jschoubben 609013a263 Merge pull request 'Gate: a merge learnt seconds late moves the trunk; and a setting check-gates-merge so the merge check can stop gating (operator 2026-10-25 22:56)' (#483) from fix/merge-without-check-gate into main
Reviewed-on: #483
2026-10-25 22:18:41 +00:00
jschoubben 32ad59fc64 mesh-delivery-gate and gitea: a setting check-gates-merge; false asks a reviewed pull request's Approve and merges it without waiting for mesh/check
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check pass 32ad59fc6402 on 609013a2638f | unit 2010 run, 2322 kept, 5 skipped | integration 2032 run, 2341 kept, 9 skipped | over bound: unit, in…
The operator, 2026-10-25 22:56: "run the check locally only, but not as a gate, it loses us too much time". Tonight each
merge ask waited 3-7 minutes for mesh/check, and every merge into main refused the next approved merge for a check judged
on the trunk before (mesh-delivery#52). Both settings default to true, so nothing changes until the operator sets them.
False: the gate makes a reviewed pull request ready and asks its Approve with no verdict, a trunk's move puts nothing out
of date, a verdict is kept and shown, and a group still merges in one pass; the merge tool merges a head the Approve
names whatever its mesh/check says. delivery_status and show say the mode.
2026-10-25 23:07:50 +01:00
jschoubben 93f655dac9 mesh-delivery-gate: a merge learnt from the open list seconds after it landed moves its trunk and source.main; delivery_status shows source.main
mesh-delivery#51 merged 21:43:11 and the gate read the open list at 21:43:14, before the forge's merged event: taken as
learnt late, it moved no trunk and its group's finish left it out of source.main, so the checker kept judging every
mesh-delivery head on 73be3df. A late merge within replayLate of its landing is now taken as heard as it happened, and
one a gate before this left is written at the catch-up of the start (never backwards).
2026-10-25 23:01:37 +01:00
mesh-tools 573a59e3cc Merge pull request 'mesh-provisioner-client: retry a refused write on the node-engine's wake and ask the vault again at once (issue 924)' (#482) from fix/provisioner-retries-on-declaration into main 2026-10-25 21:08:48 +00:00
jschoubben d0566f9ade Fill a refused file again once the node-engine's declaration may let it through (ace, 2026-10-25 21:17)
mesh/delivery merged: source.main moved to novox/mesh-catalog@573a59e3cc07
mesh/delivery-group group fix/provisioner-retries-on-declaration merged: every member is merged
mesh/check pass d0566f9ade9a on 573a59e3cc07 | unit 2301 run, 3410 kept, 10 skipped | integration 2326 run, 3431 kept, 13 skipped | over bound: unit, …
The node-engine refused mesh-build-go's files as not named as filled 32 s before
its want arrived, and nothing filled them after the apply that named them: the
runtime crash-looped on its missing broker credential. The client now fills
every module again when the node-engine touches its write socket after such an
apply (mesh-host fix/provisioner-retries-on-declaration), says each retry and
each refusal it waits on, and asks the vault again at once for a refused secret
when that wake or the module's own assignment may have ended the refusal.
2026-10-25 21:42:34 +01:00
jschoubben 9d9dfda19b Merge pull request 'Gate: hand every member of an approved group in pin order, settle pull requests closed on the forge, and say a stopped pass as a condition (issue 1275)' (#480) from fix/1275-group-hands-every-member into main
Reviewed-on: #480
2026-10-25 19:28:33 +00:00
jschoubben cdbe2945cf Merge pull request 'Build modules' steps live two days and their pause has a state of its own, so the steps bucket no longer fills (2026-10-25)' (#481) from fix/steps-bucket-ages into main
Reviewed-on: #481
2026-10-25 18:06:39 +00:00
jschoubben 4e82a824be Announce the pauses state in the three build modules, and pin mesh-sdk 518dae5 (an unreadable pause pulls as not paused)
mesh/delivery-group group fix/steps-bucket-ages held: a member is held: novox/mesh-sdk@518dae5ec9c8 (merged without a passing check: only a person decides what…
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check pass 4e82a824be7a on cdbe2945cf6e | unit 1994 run, 2322 kept, 5 skipped | integration 2016 run, 2341 kept, 9 skipped | over bound: unit, in…
Without it in Kept, stdio.KeptState refuses the state before the bus (mesh-sdk go/keeps.go
refusedUnannounced), so the pause could be neither read nor written.
2026-10-25 18:57:08 +01:00
jschoubben 78d26867d2 Give the build modules' steps state an age of two days, and keep a node's pause in a pauses state of its own
mesh-build-go's steps bucket filled to the bus's 64 MiB cap (27,370 keys, none ever removed) and
refused the end of a passed check, so the check read failed; mesh-build-node's (17.8 MB) and
mesh-build-image's grow the same way. A step's key is per step: its reader reads its end within
the step's bound, and a check's verdict is kept by the checker, so two days is ample. The node's
pause key lived in the same bucket and would have ended by itself; it moves to a pauses state
without an age (mesh-sdk fix/steps-bucket-ages, Puller.PausePut). No node is paused today, so
nothing is carried over.
2026-10-25 18:53:04 +01:00
jschoubben 0c216d99d0 Gate takes a merge learnt after the fact as past: no trunk move, the forge's time, never source.main (issue 1275, review round 2)
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check pass 0c216d99d0c0 on 9d9dfda19b7e | unit 2181 run, 2147 kept, 5 skipped | integration 2203 run, 2166 kept, 9 skipped | over bound: unit, in…
A pull request found merged only by its absence from the open list went through PullMerged, which moved every
other open delivery of its repository back to an old trunk (ready ones to proposed, composed checks out of
date), dated the merge now, and so let a later catch-up write the old commit into source.main. pullMerged now
takes such a merge as late: its own delivery is merged and its group closed, no other trunk moves, its time is
the forge's or none, and LearntLate keeps it out of source.main in finish, settleForgeMerges and the catch-up.

The gitea tool gitea_commit_statuses now answers the pull request's merged_at, and the gate reads a gone pull
request by its number alone, so a deleted branch does not fail the read.
2026-10-25 18:37:59 +01:00
jschoubben 145c54d19c Gate settles pull requests closed with no event, looks again at a line's bound, and keeps a waiting head's condition over a restart (issue 1275, review round 1)
The conditions the gate now says would otherwise be wrong three ways:

- A pull request closed or merged on the forge with no event heard stayed ready here for days (de-spiegel #4,
  closed 2026-10-17) and would raise a condition. After a complete read of the open list (shorter than the
  forge tool's 1000), each delivery not final whose pull request is missing is read from the forge and taken as
  its close or merge; a merge learnt so does not move source.main, as it is undated and could move it back.
- A line that comes due on the clock (a state's bound, the author's bound, an answered refusal, an
  out-of-date refusal turning urgent after 10 minutes) waited for an unrelated event. The gate now asks a look
  at the earliest such time.
- After a restart the heads waiting on the forge start their bound again, so their conditions were cleared and
  raised again later. A condition is cleared only once the open list was read since start, and not while the
  forge still shows its head open and waiting.

Also: an empty path list ranks with a truncated one in the merge order, and a seat that keeps refusing
conditions is retried on the clock 10 looks in a row, then at events only, said once in the log.
2026-10-25 18:32:19 +01:00
jschoubben 67092161bd Gate hands every member of an approved group, in pin order, and says a stopped pass (issue 1275)
Met again 2026-10-25: group fix/puller-wakes-on-capacity was merged half. Three causes in the gate:

- The next member's turn compared the group's state with "delivering", a word GroupState no longer says
  since release 6 renamed it "merging" (ready.go groupTurnToMerge). Once the first member's merge was heard
  from the forge the group read merging, and the second member waited for ever. Group states are now named
  constants, the turn goes on in ready and merging (groupMayMerge), and a test holds every word to it.
- Members merged in id order, so mesh-catalog went before the mesh-sdk commit it pins. membersOf now orders
  by mergeRank: mesh-sdk first, then members that change no go.mod or go.sum, then those that do.
- No condition was raised: the controller's self-check that read `stalled` went with release 6
  (mesh-controller dc456d56). The gate now says its stalled lines as conditions itself on the
  operator-channel seat (ADR 0437 §9) after every look, clears those listed no more, and keeps which it
  raised in its new bucket conditions. A held group's line says "half merged" when a member merged.

A pass held for a required status no longer retries on a timer: the status turning success is the event
that asks the merge again.
2026-10-25 18:20:58 +01:00
mesh-tools 2df0503034 Merge pull request 'Let the build modules follow their capacity on the bus, so a node raised from capacity 0 starts pulling (issue 1887)' (#479) from fix/puller-wakes-on-capacity into main 2026-10-25 17:02:40 +00:00
jschoubben 23c827e791 Take the mesh-sdk head that wakes a puller on its capacity, within the test budget, in the three build modules (issue 1887)
mesh/delivery merged: source.main moved to novox/mesh-catalog@2df0503034dc
mesh/delivery-group group fix/puller-wakes-on-capacity merged: every member is merged
mesh/check pass 23c827e7910c on a85d90374195 | judged by the candidate's mesh-build-go at eab02c80fe86
2026-10-25 17:44:50 +01:00
jschoubben 3138570c6b Let the build modules follow their capacity on the bus, so a node raised from capacity 0 starts pulling (issue 1887)
g14's Go builder sat idle at capacity 2 because nothing woke its parked
puller. mesh-build-go, mesh-build-node and mesh-build-image now pin mesh-sdk
b9911dc1, whose puller watches the config value capacity as mesh-config
resolved it and wakes on each change, and hand it that watch.
2026-10-25 17:28:00 +01:00
jschoubben a85d903741 Merge pull request 'nextcloud: the module makes its own mesh-admin account right on a 401 and backs off on a refusal, so its shares poll stops being refused every minute (issue 1253)' (#478) from fix/nextcloud-admin-401 into main
Reviewed-on: #478
2026-10-25 16:01:37 +00:00
jschoubben 1cc54959af nextcloud: repair the account only on a 401, never reset the throttle (issue 1253, review 2)
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check pass 1cc54959afcf on a85d90374195 | unit 1994 run, 2322 kept, 5 skipped | integration 2016 run, 2341 kept, 9 skipped | over bound: unit, in…
Resetting Nextcloud's brute-force record for the docker gateway cleared it for
every client behind that address: someone guessing passwords could drive the
throttle to 429 and have the module's poll wipe it. A throttle is no evidence
the password is wrong, so a 429 is now backed off only, and the back-off
outlasts the throttle. The account's existence is read from user:info's exit
status, since user:list pages.
2026-10-25 16:57:46 +01:00
jschoubben 962bc359a7 nextcloud: repair only the module's own account, read its existence from the user list, no admin group (issue 1253, review 1)
occ's user:info says 'user not found' on stdout with an empty stderr, so the
error read as 'occ produced no output' and mesh-admin was never made on novox:
occ now says stderr and stdout both, and existence is read from user:list.
The repair could reset whatever account the user setting names, the
operator's included, and the 'admin' fallback is a hand-made install's human
administrator: only mesh-admin is repaired now, and it is the default. The
shares poll needs no admin rights, so the account gets no group. After a
repair the throttle of the caller's address is reset, and a failing repair
reaches the back-off's one loud line with occ's message.
2026-10-25 16:54:51 +01:00
jschoubben 67b13c42d3 nextcloud: repair the account on a throttle too, since a throttle follows refusals (issue 1253)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 67b13c42d32a on f27f4c8cd58c
Right after the delivery the first answer may be Nextcloud's 429, not a 401;
without a repair then the account would never be made and the back-off would
wait out a throttle that cannot end.
2026-10-25 16:40:54 +01:00
jschoubben 4021724ca4 nextcloud: make the mesh's own account match its secret when Nextcloud refuses it, and back off a refusal that stays (issue 1253)
The image creates NEXTCLOUD_ADMIN_USER only on a fresh install. novox's
instance was installed before the mesh adopted it, so mesh-admin was never
created there, and the shares poll asked Nextcloud every minute with a login
that cannot pass: 401, then Nextcloud's brute-force throttle (429), which
also held the backup before a delivery.

On a refused login the client now adds the account through occ (admin
group), or sets its password to the module's own secret, passing it by
OC_PASS so it is on no argv, and asks once more. A refusal that stays is
said once and the poll backs off from 5 minutes to an hour instead of nine
lines of XML every minute. An empty setting no longer becomes an empty name.
2026-10-25 16:40:01 +01:00
jschoubben f27f4c8cd5 Merge pull request 'mesh-build-go: a package that fails without an assertion carries its real cause, never "PASS" (issue 1901)' (#477) from fix/1901-package-cause into main
Reviewed-on: #477
2026-10-25 15:07:40 +00:00
jschoubben 155124b59a mesh-build-go: a cause is the runtime's line at column 0, and only go's exact summary lines are dropped (issue 1901, review round 1)
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/delivery-group group fix/1901-package-cause held: a member is held: novox/mesh-catalog@155124b59af4 (merged without a passing check: only a person decides…
mesh/check pass 155124b59af4 on f27f4c8cd58c | unit 2015 run, 2883 kept, 5 skipped | integration 2037 run, 2978 kept, 9 skipped | over bound: unit, in…
An indented log line of another running test that held 'panic:' won over the
real timeout panic, since the lines were trimmed before matching. Go prints
the runtime's lines at column 0; a test's log is indented.
2026-10-25 15:58:07 +01:00
jschoubben 6ec6fdaad2 mesh-build-go: a race the race detector reports is a failure's cause, in a test or after PASS (issue 1901)
mesh-controller's try was worded 'PASS' again because the race detector
reported after the tests passed and the program exited 66. The cause is now
the race: its access, the frame that made it and where its goroutine was
created, instead of PASS or testing's generic 'race detected during execution
of test'. Tests from captured go test -race -json (1901/AC4).
2026-10-25 15:52:20 +01:00
jschoubben 663f3de975 mesh-build-go: a package that fails without an assertion carries its real cause, never "PASS" (issue 1901)
A try of mesh-controller kept 'PASS' as its only failure: the package's test
program stopped after printing PASS and the package-level failure took its
first line. Now a failure that is not one test's assertion names the tests that
were running and carries the panic line (a timeout's with its running tests),
a signal or fatal error line, a build's first error, or words saying the
program exited during which tests or after they passed; go test's summary lines
are never kept. Tests from captured go test -json of eight real failures
(1901/AC4).
2026-10-25 15:38:28 +01:00
139 changed files with 6052 additions and 1981 deletions
-303
View File
@@ -34,306 +34,3 @@ modules/mesh-build-image/cmd/mesh-build-image/mesh-build-image
/checks/paths/paths
/modules/mesh-build-go/mesh-build-go
/modules/mesh-delivery-gate/mesh-delivery-gate
# A binary built in place, named like its module directory or a cmd directory, in the module root or in that cmd directory
# (issue 1921, try speed S13), for every module; the lint step of every check refuses one a change adds, a module added
# later included.
/modules/adwaita/adwaita
/modules/adwaita/adwaita-tools
/modules/adwaita/cmd/adwaita-tools/adwaita-tools
/modules/artifact-store-tools/artifact-store-tools
/modules/artifact-store-tools/cmd/artifact-store-tools/artifact-store-tools
/modules/asus-zephyrus-g14/asus-zephyrus-g14
/modules/asus-zephyrus-g14/zephyrus
/modules/asus-zephyrus-g14/cmd/zephyrus/zephyrus
/modules/audit-logger/audit-logger
/modules/avahi/avahi
/modules/avahi/avahi-tools
/modules/avahi/cmd/avahi-tools/avahi-tools
/modules/baserow/baserow
/modules/blueman/blueman
/modules/blueman/blueman-tools
/modules/blueman/cmd/blueman-tools/blueman-tools
/modules/bluetooth/bluetooth
/modules/bluetooth/bluetooth-tools
/modules/bluetooth/cmd/bluetooth-tools/bluetooth-tools
/modules/brother/brother
/modules/brother/cmd/brother/brother
/modules/buildah/buildah
/modules/ca-trust/ca-trust
/modules/claude-code/claude-code
/modules/claude-code/cmd/claude-code/claude-code
/modules/claude-licence-manager/claude-licence-manager
/modules/claude-licence-manager/cmd/claude-licence-manager/claude-licence-manager
/modules/clipmenu/clipmenu
/modules/clipmenu/clipmenu-tools
/modules/clipmenu/cmd/clipmenu-tools/clipmenu-tools
/modules/confluence/confluence
/modules/cups/cups
/modules/cups/cups-tools
/modules/cups/cmd/cups-tools/cups-tools
/modules/dbus/dbus
/modules/dbus/dbus-tools
/modules/dbus/cmd/dbus-tools/dbus-tools
/modules/desk-channel/desk-channel
/modules/desk-channel/cmd/desk-channel/desk-channel
/modules/dev-tools/dev-tools
/modules/disk-load/disk-load
/modules/disk-load/cmd/disk-load/disk-load
/modules/distribution/distribution
/modules/dmenu/dmenu
/modules/dmenu/dmenu-tools
/modules/dmenu/cmd/dmenu-tools/dmenu-tools
/modules/dnsmasq/dnsmasq
/modules/dnsmasq/dnsmasq-tools
/modules/dnsmasq/cmd/dnsmasq-tools/dnsmasq-tools
/modules/docker/docker
/modules/docker/docker-tools
/modules/docker/cmd/docker-tools/docker-tools
/modules/docker-compose/docker-compose
/modules/docker-compose/docker-compose-tools
/modules/docker-compose/cmd/docker-compose-tools/docker-compose-tools
/modules/dunst/dunst
/modules/dunst/dunst-tools
/modules/dunst/cmd/dunst-tools/dunst-tools
/modules/fail2ban/fail2ban
/modules/fail2ban/fail2ban-tools
/modules/fail2ban/cmd/fail2ban-tools/fail2ban-tools
/modules/feh/feh
/modules/feh/feh-tools
/modules/feh/cmd/feh-tools/feh-tools
/modules/flatpak/flatpak
/modules/flatpak/flatpak-tools
/modules/flatpak/cmd/flatpak-tools/flatpak-tools
/modules/fonts/fonts
/modules/fonts/fonts-tools
/modules/fonts/cmd/fonts-tools/fonts-tools
/modules/forticlient/forticlient
/modules/forticlient/forticlient-tools
/modules/forticlient/cmd/forticlient-tools/forticlient-tools
/modules/gitea/gitea
/modules/gitea/agent-teams
/modules/gitea/cmd/agent-teams/agent-teams
/modules/gitea/agent-token
/modules/gitea/cmd/agent-token/agent-token
/modules/gitea/forge
/modules/gitea/cmd/forge/forge
/modules/gitea/forge-account-provider
/modules/gitea/cmd/forge-account-provider/forge-account-provider
/modules/gitea/npm-registry
/modules/gitea/cmd/npm-registry/npm-registry
/modules/gitea/repositories
/modules/gitea/cmd/repositories/repositories
/modules/gitlab/gitlab
/modules/gnome-keyring/gnome-keyring
/modules/gnome-keyring/gnome-keyring-tools
/modules/gnome-keyring/cmd/gnome-keyring-tools/gnome-keyring-tools
/modules/go/go
/modules/grafana/grafana
/modules/hello-web/hello-web
/modules/home-assistant/home-assistant
/modules/hostname/hostname
/modules/hostname/hostname-tools
/modules/hostname/cmd/hostname-tools/hostname-tools
/modules/i3/i3
/modules/i3/i3-tools
/modules/i3/cmd/i3-tools/i3-tools
/modules/i3status-rust/i3status-rust
/modules/i3status-rust/i3status-rust-tools
/modules/i3status-rust/cmd/i3status-rust-tools/i3status-rust-tools
/modules/icecast/icecast
/modules/influxdb/influxdb
/modules/invoicing/invoicing
/modules/jira/jira
/modules/kernel/kernel
/modules/kernel/kernel-tools
/modules/kernel/cmd/kernel-tools/kernel-tools
/modules/keycloak/keycloak
/modules/keycloak/keycloak-provider
/modules/keycloak/cmd/keycloak-provider/keycloak-provider
/modules/lab/lab
/modules/lemurs/lemurs
/modules/lemurs/lemurs-tools
/modules/lemurs/cmd/lemurs-tools/lemurs-tools
/modules/letta/letta
/modules/localization/localization
/modules/localization/localization-tools
/modules/localization/cmd/localization-tools/localization-tools
/modules/local-model-consumer/local-model-consumer
/modules/logrotate/logrotate
/modules/logrotate/logrotate-tools
/modules/logrotate/cmd/logrotate-tools/logrotate-tools
/modules/mailu/mailu
/modules/mailu/automx
/modules/mailu/cmd/automx/automx
/modules/marrytts/marrytts
/modules/matrix/matrix
/modules/memory-pressure/memory-pressure
/modules/memory-pressure/cmd/memory-pressure/memory-pressure
/modules/mesh-build-go/mesh-build-go
/modules/mesh-build-go/cmd/mesh-build-go/mesh-build-go
/modules/mesh-build-image/mesh-build-image
/modules/mesh-build-image/cmd/mesh-build-image/mesh-build-image
/modules/mesh-build-node/mesh-build-node
/modules/mesh-build-node/cmd/mesh-build-node/mesh-build-node
/modules/mesh-catalog/mesh-catalog
/modules/mesh-cli/mesh-cli
/modules/mesh-cli/cmd/mesh-cli/mesh-cli
/modules/mesh-config/mesh-config
/modules/mesh-config/cmd/mesh-config/mesh-config
/modules/mesh-delivery-gate/mesh-delivery-gate
/modules/mesh-delivery-gate/cmd/mesh-delivery-gate/mesh-delivery-gate
/modules/mesh-files/mesh-files
/modules/mesh-files/cmd/mesh-files/mesh-files
/modules/mesh-issues/mesh-issues
/modules/mesh-issues/cmd/mesh-issues/mesh-issues
/modules/mesh-knowledge/mesh-knowledge
/modules/mesh-knowledge/cmd/mesh-knowledge/mesh-knowledge
/modules/mesh-mail/mesh-mail
/modules/mesh-mail/cmd/mesh-mail/mesh-mail
/modules/mesh-mail-notifier/mesh-mail-notifier
/modules/mesh-mail-notifier/cmd/mesh-mail-notifier/mesh-mail-notifier
/modules/mesh-mail-reader/mesh-mail-reader
/modules/mesh-mail-reader/cmd/mesh-mail-reader/mesh-mail-reader
/modules/mesh-mail-web/mesh-mail-web
/modules/mesh-mail-web/cmd/mesh-mail-web/mesh-mail-web
/modules/mesh-provisioner/mesh-provisioner
/modules/mesh-provisioner/cmd/mesh-provisioner/mesh-provisioner
/modules/mesh-provisioner-client/mesh-provisioner-client
/modules/mesh-provisioner-client/cmd/mesh-provisioner-client/mesh-provisioner-client
/modules/mesh-reads/mesh-reads
/modules/mesh-reads/cmd/mesh-reads/mesh-reads
/modules/mesh-vault/mesh-vault
/modules/mesh-vault/cmd/mesh-vault/mesh-vault
/modules/mesh-watcher/mesh-watcher
/modules/mesh-watcher/cmd/mesh-watcher/mesh-watcher
/modules/messenger/messenger
/modules/messenger/cmd/messenger/messenger
/modules/minio/minio
/modules/minio/object-store
/modules/minio/cmd/object-store/object-store
/modules/model-usage/model-usage
/modules/mongodb/mongodb
/modules/mosquitto/mosquitto
/modules/mounts/mounts
/modules/mounts/cmd/mounts/mounts
/modules/mssql/mssql
/modules/n8n/n8n
/modules/nats/nats
/modules/nats/mesh-nats-snapshot
/modules/nats/cmd/mesh-nats-snapshot/mesh-nats-snapshot
/modules/nats/nats-tools
/modules/nats/cmd/nats-tools/nats-tools
/modules/netcheck/netcheck
/modules/netcheck/cmd/netcheck/netcheck
/modules/networkmanager/networkmanager
/modules/networkmanager/uplink-tools
/modules/networkmanager/cmd/uplink-tools/uplink-tools
/modules/nextcloud/nextcloud
/modules/nextcloud-client/nextcloud-client
/modules/nextcloud-client/nextcloud-client-tools
/modules/nextcloud-client/cmd/nextcloud-client-tools/nextcloud-client-tools
/modules/nfs-server/nfs-server
/modules/nfs-server/cmd/nfs-server/nfs-server
/modules/nftables/nftables
/modules/nm-applet/nm-applet
/modules/nm-applet/nm-applet-tools
/modules/nm-applet/cmd/nm-applet-tools/nm-applet-tools
/modules/node-env/node-env
/modules/nodejs/nodejs
/modules/nodered/nodered
/modules/ollama/ollama
/modules/only-office/only-office
/modules/only-office/cmd/only-office/only-office
/modules/openai-consumer/openai-consumer
/modules/openrazer/openrazer
/modules/openrazer/openrazer-tools
/modules/openrazer/cmd/openrazer-tools/openrazer-tools
/modules/pacman/pacman
/modules/pacman/pacman-tools
/modules/pacman/cmd/pacman-tools/pacman-tools
/modules/photos-eef/photos-eef
/modules/photos-filip/photos-filip
/modules/picom/picom
/modules/picom/picom-tools
/modules/picom/cmd/picom-tools/picom-tools
/modules/polychromatic/polychromatic
/modules/polychromatic/polychromatic-tools
/modules/polychromatic/cmd/polychromatic-tools/polychromatic-tools
/modules/postgres/postgres
/modules/postgres/postgres-provider
/modules/postgres/cmd/postgres-provider/postgres-provider
/modules/power/power
/modules/power/cmd/power/power
/modules/powerlevel10k/powerlevel10k
/modules/records/records
/modules/records/cmd/records/records
/modules/redis/redis
/modules/restic/restic
/modules/restic/restic-backups
/modules/restic/cmd/restic-backups/restic-backups
/modules/rofi/rofi
/modules/rofi/rofi-tools
/modules/rofi/cmd/rofi-tools/rofi-tools
/modules/route-adapter/route-adapter
/modules/route-adapter/route-provider
/modules/route-adapter/cmd/route-provider/route-provider
/modules/screen-lock/screen-lock
/modules/screen-lock/screen-lock-tools
/modules/screen-lock/cmd/screen-lock-tools/screen-lock-tools
/modules/searxng/searxng
/modules/sensors/sensors
/modules/sensors/cmd/sensors/sensors
/modules/showcase/showcase
/modules/slack/slack
/modules/slack/slack-tools
/modules/slack/cmd/slack-tools/slack-tools
/modules/snapd/snapd
/modules/snapd/snapd-tools
/modules/snapd/cmd/snapd-tools/snapd-tools
/modules/ssh-client/ssh-client
/modules/ssh-client/ssh-client-tools
/modules/ssh-client/cmd/ssh-client-tools/ssh-client-tools
/modules/sshd/sshd
/modules/step-ca/step-ca
/modules/sudo/sudo
/modules/sudo/sudo-tools
/modules/sudo/cmd/sudo-tools/sudo-tools
/modules/supabase/supabase
/modules/systemd/systemd
/modules/systemd/systemd-tools
/modules/systemd/cmd/systemd-tools/systemd-tools
/modules/systemd-networkd/systemd-networkd
/modules/systemd-networkd/uplink-tools
/modules/systemd-networkd/cmd/uplink-tools/uplink-tools
/modules/systemd-resolved/systemd-resolved
/modules/systemd-resolved/resolver-tools
/modules/systemd-resolved/cmd/resolver-tools/resolver-tools
/modules/telegram/telegram
/modules/telegram/cmd/telegram/telegram
/modules/time-sync/time-sync
/modules/time-sync/time-sync-tools
/modules/time-sync/cmd/time-sync-tools/time-sync-tools
/modules/triggerhappy/triggerhappy
/modules/triggerhappy/triggerhappy-tools
/modules/triggerhappy/cmd/triggerhappy-tools/triggerhappy-tools
/modules/umami/umami
/modules/unifi/unifi
/modules/website/website
/modules/xclip/xclip
/modules/xclip/xclip-tools
/modules/xclip/cmd/xclip-tools/xclip-tools
/modules/xdg/xdg
/modules/xdg/xdg-tools
/modules/xdg/cmd/xdg-tools/xdg-tools
/modules/xorg/xorg
/modules/xorg/cmd/xorg/xorg
/modules/xterm/xterm
/modules/xterm/xterm-tools
/modules/xterm/cmd/xterm-tools/xterm-tools
/modules/zfs/zfs
/modules/zfs/zfs-tools
/modules/zfs/cmd/zfs-tools/zfs-tools
/modules/zsh/zsh
/modules/zsh-autosuggestions/zsh-autosuggestions
/modules/zsh-syntax-highlighting/zsh-syntax-highlighting
+107 -9
View File
@@ -15,6 +15,13 @@ package main
// the desk says in its standing when that happened (`answers-lost-at`), and the router, which alone knows
// what is still open and unanswered, says that again under its handle — replacing the notification in place,
// so its buttons work again. Nothing resolved, silenced or cancelled comes back.
// - **A store that does not answer does not lose where a message is shown, while the desk runs.** A message
// is shown first and its notification ids kept after; when the state store does not answer then (the bus
// reloading its accounts, 2026-10-26 09:18: the very outage that raised the condition), the work is taken
// all the same and the ids are held in memory, recalled before the store, and kept at the start of the next
// work and at the minute's standing. Otherwise the clearing of that message finds no notification to
// replace and the urgent one stays on screen for good. A desk that restarts while it still holds some loses
// where those are shown; the log says so when they are held.
// - **Standing**: it can send when machines are named in its settings; which of them answered last is in its
// status. A message no machine took is said to the router as failed, and tried again by it.
@@ -93,6 +100,9 @@ type Desk struct {
pause map[string]pauseSeen
// lost is when each machine's notifier last started again: the answers its notifications waited for are lost.
lost map[string]time.Time
// unkept is what was shown and not yet kept in the store, because it did not answer: recalled from here
// before the store, and kept at the next piece of work.
unkept map[string]Shown
}
func (d *Desk) init() {
@@ -107,6 +117,9 @@ func (d *Desk) init() {
if d.lost == nil {
d.lost = map[string]time.Time{}
}
if d.unkept == nil {
d.unkept = map[string]Shown{}
}
if d.Now == nil {
d.Now = time.Now
}
@@ -138,6 +151,7 @@ func (d *Desk) Take(subject string, body json.RawMessage) error {
d.Logf("[desk-channel] work on %s could not be read; taken and dropped", subject)
return nil
}
d.KeepUnkept()
machines := d.Machines()
if m.Secret {
// Shown only where the notifier said lately it is not paused (the review of 2026-10-09): one held back
@@ -167,12 +181,16 @@ func (d *Desk) Take(subject string, body json.RawMessage) error {
switch {
case edit:
var err error
if before, err = d.Store.Get(m.Handle); err != nil {
if before, err = d.recall(m.Handle); err != nil {
return err
}
if before == nil {
d.Logf("[desk-channel] an edit of %s, which nothing here records as shown: shown as a new message; "+
"a notification it was to replace, if any, stays", m.Handle)
}
case !m.Secret:
var err error
if before, err = d.Store.Get(m.Handle); err != nil {
if before, err = d.recall(m.Handle); err != nil {
d.Logf("[desk-channel] what is shown under this handle could not be read (%v): shown as a new message", err)
before = nil
}
@@ -218,12 +236,91 @@ func (d *Desk) Take(subject string, body json.RawMessage) error {
// Shown once and kept nowhere of this module's (a link's code): no edit or re-showing reaches it.
return nil
}
if err := d.Store.Put(m.Handle, s); err != nil {
d.Logf("[desk-channel] shown, and what was shown could not be kept: an edit of it will not reach it: %v", err)
}
d.keep(m.Handle, s)
return nil
}
// keep keeps what a handle shows in the store. When the store does not answer it is held in memory, where an
// edit, a reminder and a click still find it, and kept later (KeepUnkept). The work is taken either way: offered
// again it would race the message's own later edits, and a redelivery is bounded where an outage is not.
func (d *Desk) keep(handle string, s Shown) {
err := d.Store.Put(handle, s)
d.mu.Lock()
defer d.mu.Unlock()
if err == nil {
delete(d.unkept, handle)
return
}
d.unkept[handle] = s
d.Logf("[desk-channel] shown, and where it is shown could not be kept yet (%v): held in memory, so its edit "+
"and clearing still reach it, and kept when the store answers; %d held, lost if the desk restarts first",
err, len(d.unkept))
}
// KeepUnkept keeps in the store what it did not answer for before: at the start of each piece of work and at
// the minute's standing. It stops at the first write the store does not answer, so one call waits on it once.
func (d *Desk) KeepUnkept() {
d.init()
d.mu.Lock()
held := make(map[string]Shown, len(d.unkept))
for h, s := range d.unkept {
held[h] = s
}
d.mu.Unlock()
for h, s := range held {
if err := d.Store.Put(h, s); err != nil {
return
}
d.mu.Lock()
if cur, ok := d.unkept[h]; ok && sameShown(cur, s) {
delete(d.unkept, h)
}
d.mu.Unlock()
d.Logf("[desk-channel] where %s is shown is kept now", h)
}
}
func (d *Desk) unkeptOf(handle string) (Shown, bool) {
d.mu.Lock()
defer d.mu.Unlock()
s, ok := d.unkept[handle]
return s, ok
}
// recall is what a handle shows: from memory when the store did not keep it yet, else from the store.
func (d *Desk) recall(handle string) (*Shown, error) {
if s, ok := d.unkeptOf(handle); ok {
return &s, nil
}
return d.Store.Get(handle)
}
// handles is every handle shown: the store's and those held in memory.
func (d *Desk) handles() ([]string, error) {
hs, err := d.Store.Handles()
d.mu.Lock()
defer d.mu.Unlock()
seen := map[string]bool{}
for _, h := range hs {
seen[h] = true
}
for h := range d.unkept {
if !seen[h] {
hs = append(hs, h)
}
}
if err != nil && len(d.unkept) == 0 {
return nil, err
}
return hs, nil
}
func sameShown(a, b Shown) bool {
x, err1 := json.Marshal(a)
y, err2 := json.Marshal(b)
return err1 == nil && err2 == nil && string(x) == string(y)
}
// show puts one message on one machine's notifier, in place of `replace` when it names one.
func (d *Desk) show(machine string, m asks.Message, replace int) (int, error) {
urgency := "normal"
@@ -308,12 +405,12 @@ func (d *Desk) Chosen(event, machine, ticket string) {
}
func (d *Desk) find(ticket string) (string, *Shown) {
handles, err := d.Store.Handles()
handles, err := d.handles()
if err != nil {
return "", nil
}
for _, h := range handles {
s, err := d.Store.Get(h)
s, err := d.recall(h)
if err != nil || s == nil {
continue
}
@@ -356,9 +453,9 @@ func (d *Desk) Started(event, machine string) int {
d.lost[machine] = d.Now()
d.mu.Unlock()
n := 0
if handles, err := d.Store.Handles(); err == nil {
if handles, err := d.handles(); err == nil {
for _, h := range handles {
if s, err := d.Store.Get(h); err == nil && s != nil && len(s.Message.Buttons) > 0 {
if s, err := d.recall(h); err == nil && s != nil && len(s.Message.Buttons) > 0 {
if _, here := s.On[machine]; here {
n++
}
@@ -472,6 +569,7 @@ func (d *Desk) Status() map[string]any {
"it proves nothing about who clicked: any process of the operator's account can choose an action, so it answers acknowledgements only",
"an answer is taken only on a machine its message was shown on; a message with buttons stays until dismissed",
"what it remembers is never shown again: when a notifier started again, the answers its notifications waited for are lost; the desk says when in its standing (answers-lost-at), and the router says again what is still open and unanswered, replacing those notifications in place — nothing resolved, silenced or cancelled comes back (issue 384)",
"a store that does not answer does not lose where a message is shown while the desk runs: it is held in memory, its edit and clearing still replace it in place, and it is kept when the store answers again; a desk restarted before then loses where those are shown, and its log said how many it held",
"it can send while a named machine's notifier said lately that it is not paused; a link's code is shown only on such a machine, kept nowhere, and sent secret so the notifier keeps it out of its history",
}}
if s.Why != "" {
@@ -3,6 +3,7 @@ package main
import (
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
@@ -363,9 +364,64 @@ func TestAMessageSaidAgainUnderItsHandleReplacesItsNotification(t *testing.T) {
}
}
// downStore is a store that does not answer while down: the bus reloading its accounts (2026-10-26 09:18).
type downStore struct {
memShown
down *bool
}
func (s downStore) Get(h string) (*Shown, error) {
if *s.down {
return nil, errors.New("nats: no responders available for request")
}
return s.memShown.Get(h)
}
func (s downStore) Put(h string, v Shown) error {
if *s.down {
return errors.New("nats: no responders available for request")
}
return s.memShown.Put(h, v)
}
// The review of PR 176: a replace that fails on a machine keeps that machine's notification on record, so a
// later edit or clearing still reaches it; failing everywhere, it is said to the router as failed.
// later edit or clearing still reaches it; failing everywhere, it is said to the router as failed. And
// (2026-10-26 09:18, an urgent "A new build of mesh-controller cannot be delivered" left on screen for hours
// after its clearing): a message shown while the store does not answer keeps where it is shown in memory, its
// work is taken, its clearing replaces it in place, and it is kept once the store answers.
func TestAFailedReplaceKeepsTheNotificationItWasToReplace(t *testing.T) {
// The raise, then its clearing, taken while the store does not answer; the store then stays down or comes
// back. Each is taken (never offered again, where a redelivered raise would follow its clearing), the
// clearing replaces the urgent notification in place, and the screen and the store end on the clearing.
for _, back := range []bool{false, true} {
t.Run(fmt.Sprintf("the store does not answer, back %v", back), func(t *testing.T) {
d, shown, _, store := newDesk(t, "laptop")
down := true
d.Store = downStore{store, &down}
raised, _ := json.Marshal(asks.Message{Handle: "m9", Title: "A new build cannot be delivered", Urgent: true})
if err := d.Take(asks.ChannelSubject(asks.Send, Kind), raised); err != nil {
t.Fatalf("the raise was not taken: %v", err)
}
cleared, _ := json.Marshal(asks.Message{Handle: "m9", Title: "Cleared: a new build can be delivered"})
if err := d.Take(asks.ChannelSubject(asks.Edit, Kind), cleared); err != nil {
t.Fatalf("the clearing was not taken: %v", err)
}
d.KeepUnkept() // the minute's standing, store still down
down = !back
d.KeepUnkept()
take(t, d, asks.Send, asks.Message{Handle: "m10", Title: "another"})
if len(*shown) != 3 || (*shown)[1].args["replace_id"] != 1 || (*shown)[1].args["summary"] != "Cleared: a new build can be delivered" {
t.Fatalf("the clearing did not replace the urgent notification: %+v", *shown)
}
s, kept := store["m9"]
if back != kept || (kept && s.Message.Title != "Cleared: a new build can be delivered") {
t.Fatalf("store back %v, kept %v: %+v", back, kept, store)
}
if got, _ := d.recall("m9"); got == nil || got.Message.Title != "Cleared: a new build can be delivered" {
t.Fatalf("recalled %+v", got)
}
})
}
d, shown, said, store := newDesk(t, "laptop", "desktop")
take(t, d, asks.Send, asks.Message{Handle: "m1", Title: "2 failed services on shanks"})
first := store["m1"].On
@@ -100,6 +100,7 @@ func run(d *Desk) {
}()
go func() {
for {
d.KeepUnkept()
d.SayStanding()
time.Sleep(time.Minute)
}
Binary file not shown.
+13
View File
@@ -15,6 +15,19 @@ export function keepMainRules(env: NodeJS.ProcessEnv = process.env): boolean {
} catch { return false; }
}
/** The setting check-gates-merge (the operator, 2026-10-25 22:56: "run the check locally only, but not as a gate, it loses
* us too much time"): whether the merge tool judges mesh/check before it merges. Absent, unreadable or anything but false
* is true; false lets a head the operator's Approve names merge with mesh/check pending, failed, missing or judged on an
* older trunk (mesh-delivery-gate's setting of the same name stops waiting for it before the Approve). */
export function checkGatesMerge(env: NodeJS.ProcessEnv = process.env): boolean {
const file = env.MESH_GITEA_CONFIG_FILE;
if (!file) return true;
try {
const v = (JSON.parse(readFileSync(file, "utf8")) as Record<string, unknown>)["check-gates-merge"];
return !(v === false || v === "false");
} catch { return true; }
}
/** The 10-minute step: puts the rules right only when kept; else says once that they are left and writes nothing. */
export function mainRulesStep(keep: boolean, put: () => Promise<string[]>, say: (line: string) => void): () => Promise<void> {
let said = false;
+8 -2
View File
@@ -19,6 +19,7 @@ import { mergeableUnder, newestByContext } from "./checks.js";
import { MERGE_METHODS, Refused, checkName } from "./guard.js";
import { CHECK_CONTEXT, CHECK_STATUS, REPO_CHECK_CONTEXT, maintained, requiredOf } from "./pulls.js";
import { approvedHeadsOf, groupRefusal, memberKey, namesSiblings, notApproved, readSiblings, siblingOutOfDate, siblingsFile } from "./siblings.js";
import { checkGatesMerge } from "./mainrules.js";
import { announcementOf, askedWords, outOfDateWords, sameCommit, tookFile, tookWords, twelve, verdictAtTrunk, waitsForComposed } from "./trunk.js";
/** A head named for a merge: at least 12 hex characters of its sha (novox/hq issue 381), up to a sha-256 commit's 64. */
@@ -31,11 +32,13 @@ export interface MergeDeps {
emit?: (type: string, body: unknown) => Promise<void>;
stateDir?: string;
now?: () => Date;
/** Whether mesh/check gates the merge: the setting check-gates-merge, read at each merge (mainrules.ts). */
checkGates?: () => boolean;
}
export function mergeTool(gitea: GiteaClient, given: MergeDeps): ToolDefinition {
const deps = { ...given, emit: given.emit ?? announce, stateDir: given.stateDir ?? process.env.MESH_GITEA_STATE_DIR,
now: given.now ?? (() => new Date()) };
now: given.now ?? (() => new Date()), checkGates: given.checkGates ?? (() => checkGatesMerge()) };
return {
name: "gitea_merge_pull_request",
description: "Merge a pull request at the head you name — the commit you reviewed — when that is still its head, " +
@@ -126,7 +129,10 @@ export function mergeTool(gitea: GiteaClient, given: MergeDeps): ToolDefinition
}
const kept = maintained(required);
const byCheck = required.includes(CHECK_STATUS) || kept.length === 0;
if (byCheck) {
// **check-gates-merge false** (the operator, 2026-10-25): mesh/check is information, not a gate. A head the operator's
// Approve names merges whatever its mesh/check says, pending, failed, missing or judged on an older trunk; without
// approved_heads the check still judges.
if (byCheck && (deps.checkGates() || !approved)) {
// **mesh/check is the checker's** (ADR 0447 §4): it is judged here against the trunk's head, and refused when it
// is not current; nothing is written or asked — the checker checks a head again when its trunk moves.
const check = checks[CHECK_STATUS];
+8 -1
View File
@@ -181,6 +181,12 @@
"trusted": true,
"default": {}
},
"check-gates-merge": {
"type": "boolean",
"description": "Whether gitea_merge_pull_request judges mesh/check before it merges. True (the default): a head merges only on a passing mesh/check judged against the trunk's head (ADR 0447, ADR 0534 §4). False (the operator, 2026-10-25 22:56: \"run the check locally only, but not as a gate, it loses us too much time\"): a head the operator's Approve names in approved_heads merges whatever its mesh/check says (pending, failed, missing or judged on an older trunk); a head that moved, a branch no rule guards and a merge git refuses are still refused. Set together with mesh-delivery-gate's check-gates-merge.",
"trusted": true,
"default": true
},
"main-mergers": {
"type": "list",
"description": "The forge logins that may merge into main beside the tools' account, on every repository of the organisation (ADR 0534 §2): the operator's own, so the operator can always merge by hand at the forge. The tools' account is always on main's merge whitelist; no agent's account belongs here.",
@@ -316,7 +322,8 @@
"takes-config": [
"url",
"keep-main-rules",
"main-mergers"
"main-mergers",
"check-gates-merge"
]
},
{
+12
View File
@@ -525,6 +525,18 @@ test("gitea_merge_pull_request judges mesh/check on a main requiring no status,
state.statuses = line(TRUNK);
assert.equal(((await merge()) as Record<string, unknown>).merged, true);
// check-gates-merge false (the operator, 2026-10-25): a head the Approve names merges whatever its mesh/check says;
// without approved_heads it is refused as before.
const ungated = mergeTool(client, { gates, stateDir, emit: async () => {}, checkGates: () => false });
const mergeUngated = (args: Record<string, unknown> = { approved_heads }) =>
ungated.run({ owner: "novox", repo: "mesh-catalog", number: 7, head: REVIEWED, ...args });
for (const statuses of [[], line(TRUNK, "pending", "checking"), line(TRUNK, "failure", "fail"), line(MOVED)]) {
state.merged = false;
state.statuses = statuses;
await assert.rejects(mergeUngated({}), /Approve/);
assert.equal(((await mergeUngated()) as Record<string, unknown>).merged, true, JSON.stringify(statuses));
}
});
test("a rule named for a branch is not made weaker than the glob rule that guards it now", async () => {
+3
View File
@@ -34,6 +34,9 @@ test("both statuses say the head and the trunk commit the verdict judged, first,
const [old] = statusesFor({ ...c, trunk: undefined, gate: { verdict: "pass", summary: "every machine composes" } });
assert.equal(old.description, "pass: every machine composes");
assert.equal(trunkNamed(old.description), undefined);
// ADR 0548 §8: a carried verdict's line names the head on the new trunk first, so the merge tool merges it on that trunk.
assert.equal(trunkNamed(`pass ${HEAD.slice(0, 12)} on ${MOVED.slice(0, 12)} | carried from ${TRUNK.slice(0, 12)}, mesh-catalog 0123456789ab | unit 0 run, 4 kept`),
MOVED.slice(0, 12));
});
test("a trunk's move puts an open head's merge check out of date, naming both commits, unless a check is on its way", () => {
+2 -2
View File
@@ -508,11 +508,11 @@ function allGiteaTools(gitea: GiteaClient, admin?: () => GiteaAdmin): ToolDefini
if (!owner || !repo) throw new Error("name the repository: owner and repo");
if (!Number.isInteger(number) || number < 0) throw new Error(`${args.number} is not a pull request's number`);
let pull: { number: number; title: string; state: string; merged: boolean; base?: string; head?: string;
head_sha?: string; merge_commit_sha?: string; html_url: string } | undefined;
head_sha?: string; merge_commit_sha?: string; merged_at?: string; html_url: string } | undefined;
if (number > 0) {
const p = await gitea.getPullRequest(owner, repo, number);
pull = { number: p.number, title: p.title, state: p.state, merged: p.merged, base: p.base, head: p.head,
head_sha: p.head_sha, merge_commit_sha: p.merge_commit_sha, html_url: p.html_url };
head_sha: p.head_sha, merge_commit_sha: p.merge_commit_sha, merged_at: p.merged_at, html_url: p.html_url };
if (!ref) ref = p.head_sha ?? "";
if (!base) base = p.base ?? "";
if (!ref) throw new Error(`${owner}/${repo}#${number} names no head commit`);
@@ -14,6 +14,7 @@ var Kept = []announce.State{
{Name: "toolchains", Readable: true},
{Name: "keys", Readable: true},
{Name: "steps", Readable: true},
{Name: "pauses", Readable: true},
}
// keepStates announces Kept as this module's (stdio.Keeps): main calls it before it serves.
@@ -80,19 +80,23 @@ func (m *Module) Puller(steps []StepTool, source buildstep.AskSource) *buildstep
source = buildstep.SeatAsks{Language: delivery.AskGo}
}
capacity := m.Capacity
// The capacity's own change wakes the puller (issue 1887): the runtime's watch of the config value as mesh-config
// resolved it; the settings file is read only when that watch cannot start. A test's Capacity is its own.
var watch func(string, func(stdio.ProvisionChange) error) error
if capacity == nil {
capacity = buildstep.NodeCapacity(m.Settings, buildstep.DefaultCapacities, m.Node, m.Say)
watch = buildstep.WatchConfigServed
}
ask := m.askSteps(steps)
var load func(string) ([]byte, error)
if m.ReadSteps.Get != nil {
if m.ReadPauses.Get != nil {
load = func(key string) ([]byte, error) {
raw, err := m.ReadSteps.Get(key)
raw, err := m.ReadPauses.Get(key)
return raw, err
}
}
return &buildstep.Puller{Source: source, Language: delivery.AskGo, Node: m.Node, Put: m.Steps, Load: load,
Capacity: capacity, Logf: m.Say,
return &buildstep.Puller{Source: source, Language: delivery.AskGo, Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load,
Capacity: capacity, WatchConfig: watch, Logf: m.Say,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
if err := m.Check(ctx); err != nil {
return nil, err
@@ -1,90 +0,0 @@
package main
// Stray build output (issue 1921, try speed S13): the lint step fails a change that adds an executable (ELF or Mach-O) or
// a file over MaxAddedFile, naming each, unless the repository's .binaries-allowed lists its path. Build binaries were
// committed three times in one night; the forge keeps them for ever.
import (
"bufio"
"bytes"
"context"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
// MaxAddedFile is the largest file a change may add without .binaries-allowed naming it.
const MaxAddedFile = 5 << 20
// BinariesAllowed is the file in a repository's root listing, one path per line ("#" starts a comment), the executables
// and large files it keeps on purpose.
const BinariesAllowed = ".binaries-allowed"
// executableMagic are the first bytes of an ELF and of each Mach-O form (32 and 64 bits, both byte orders, universal).
var executableMagic = [][]byte{[]byte("\x7fELF"), {0xfe, 0xed, 0xfa, 0xce}, {0xfe, 0xed, 0xfa, 0xcf}, {0xce, 0xfa, 0xed, 0xfe},
{0xcf, 0xfa, 0xed, 0xfe}, {0xca, 0xfe, 0xba, 0xbe}}
// addedBinaries is the lint finding of the files the merge adds to the trunk that are executables or over MaxAddedFile,
// nil when there is none. The paths come from git (the merge commit against its first parent, the trunk); a tree with no
// merge (no first parent) adds nothing to judge.
func (m *Module) addedBinaries(ctx context.Context, run forgekey.Runner, tree string) []Failure {
r := run(ctx, []string{"GIT_TERMINAL_PROMPT=0"}, "git", "-C", tree, "diff", "--name-only", "--no-renames", "--diff-filter=A", "HEAD^1", "HEAD")
if r.Status != 0 || r.Err != "" {
return nil
}
allowed := map[string]bool{}
if raw, err := os.ReadFile(filepath.Join(tree, BinariesAllowed)); err == nil {
scan := bufio.NewScanner(bytes.NewReader(raw))
for scan.Scan() {
if l := strings.TrimSpace(scan.Text()); l != "" && !strings.HasPrefix(l, "#") {
allowed[filepath.ToSlash(l)] = true
}
}
}
var found []string
for _, p := range strings.Split(r.Stdout, "\n") {
if p = strings.TrimSpace(p); p == "" || allowed[p] {
continue
}
path := filepath.Join(tree, filepath.FromSlash(p))
info, err := os.Lstat(path)
if err != nil || !info.Mode().IsRegular() {
continue // narrowed away, or not a file
}
if info.Size() > MaxAddedFile {
found = append(found, fmt.Sprintf("%s (%d bytes, over %d)", p, info.Size(), MaxAddedFile))
continue
}
if isExecutable(path) {
found = append(found, p+" (an executable)")
}
}
if len(found) == 0 {
return nil
}
sort.Strings(found)
return []Failure{{Package: ".", Test: "binaries", Step: StepLint,
Output: "the change adds build output or a large file (remove it, or list its path in " + BinariesAllowed + "): " + strings.Join(found, ", ")}}
}
func isExecutable(path string) bool {
f, err := os.Open(path)
if err != nil {
return false
}
defer f.Close()
head := make([]byte, 4)
if n, _ := f.Read(head); n < 4 {
return false
}
for _, magic := range executableMagic {
if bytes.Equal(head, magic) {
return true
}
}
return false
}
@@ -1,67 +0,0 @@
package main
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
// Issue 1921, try speed S13: the lint step fails a change that adds an executable (ELF or Mach-O) or a file over 5 MB,
// naming it, unless .binaries-allowed in the repository's root lists its path; a file the change does not add is not
// judged, and neither is an allowed one.
func TestTheLintStepFailsAChangeThatAddsABinaryOrALargeFile(t *testing.T) {
m, states, ws, _ := workspaceFor(t, "slow")
tree := filepath.Join(m.Work, ws, "tree")
write := func(name string, body []byte) {
os.MkdirAll(filepath.Dir(filepath.Join(tree, name)), 0o755)
if err := os.WriteFile(filepath.Join(tree, name), body, 0o644); err != nil {
t.Fatal(err)
}
}
write("go.mod", []byte("module fixture/slow\n"))
elf := append([]byte("\x7fELF"), make([]byte, 64)...)
macho := append([]byte{0xcf, 0xfa, 0xed, 0xfe}, make([]byte, 64)...)
write("modules/x/cmd/x/x", elf)
write("modules/y/y", macho)
write("big.tar", make([]byte, 5<<20+1))
write("tools/allowed-tool", elf)
write("old/binary", elf) // there before the change
write("modules/x/doc.md", []byte("text\n"))
write(".binaries-allowed", []byte("# kept on purpose\ntools/allowed-tool\n"))
inner := m.Run
m.Run = func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
if name == "git" && strings.Contains(strings.Join(args, " "), "--diff-filter=A") {
return forgekey.Ran{Stdout: "modules/x/cmd/x/x\nmodules/y/y\nbig.tar\ntools/allowed-tool\nmodules/x/doc.md\n.binaries-allowed\n"}
}
return inner(ctx, env, name, args...)
}
res, st := runUnit(t, m, states, ws)
if st.State != delivery.StepPassed {
t.Fatalf("the step ended %s: %s", st.State, st.Reason)
}
var found []Failure
for _, f := range res.Failed {
if f.Test == "binaries" {
found = append(found, f)
}
}
if res.Passed || len(found) != 1 || found[0].Step != StepLint {
t.Fatalf("failed = %+v, want one lint finding of the binaries", res.Failed)
}
out := found[0].Output
for _, want := range []string{"modules/x/cmd/x/x", "modules/y/y", "big.tar"} {
if !strings.Contains(out, want) {
t.Errorf("the finding does not name %s: %s", want, out)
}
}
for _, not := range []string{"allowed-tool", "old/binary", "doc.md"} {
if strings.Contains(out, not) {
t.Errorf("the finding names %s: %s", not, out)
}
}
}
@@ -1,40 +0,0 @@
package main
import (
"slices"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// 1921/S11 (the lead's decision): a check compiles and runs with CGO_ENABLED=0, as a production build does, so it tests
// what ships and the warm cache matches; the lint's go vet, a compile alone, writes the shared cache, while a go test that
// runs a test binary reads it beneath the ask's own layer.
func TestACheckRunsWithoutCgoAndOnlyItsCompileWritesTheSharedCache(t *testing.T) {
m, states, ws, calls := workspaceFor(t, "failing")
var seen []shutOff
m.Run = throughBubblewrap(t, m.Run, &seen)
if _, st := runUnit(t, m, states, ws); st.State != delivery.StepPassed {
t.Fatalf("the step ended %s: %s", st.State, st.Reason)
}
for _, c := range *calls {
if strings.HasPrefix(c.line, "go ") && !slices.Contains(c.env, "CGO_ENABLED=0") {
t.Errorf("%s ran without CGO_ENABLED=0: %v", c.line, c.env)
}
}
vets, tests := 0, 0
for _, s := range seen {
switch {
case strings.Contains(s.command, " vet "):
vets++
checkShutOff(t, m, s, true) // the shared caches bound read-write
case strings.Contains(s.command, " test "):
tests++
checkShutOff(t, m, s, false)
}
}
if vets == 0 || tests == 0 {
t.Errorf("%d vets and %d tests ran shut off, want both", vets, tests)
}
}
@@ -49,51 +49,6 @@ type FetchResult struct {
// Beside are the sibling repositories placed as plain directories under the workspace's beside directory, which
// build_go_test hands to the tests as MESH_CHECK_BESIDE (ADR 0484 clause 4).
Beside []Sibling `json:"beside,omitempty"`
// Took is how long each phase of the fetch ran, in seconds (issue 1921, the try speed-up's first measurement): what
// the checker reports as one "clone" time, split.
Took *FetchTook `json:"took,omitempty"`
}
// FetchTook is the time of each phase of a fetch, in seconds; its phases add up to Total. Mirror is bringing the
// repository's mirror to the commit, Fetch the fetch and checkout into the tree, Merge the merge preview, Dissociate
// the repack that makes the tree independent of the mirror, Beside the placing of the siblings (their own mirror,
// fetch and repack), Other the narrowing and the listing of the go.mod directories, and Modules the download and
// verify of the Go modules.
type FetchTook struct {
Mirror float64 `json:"mirror"`
Fetch float64 `json:"fetch"`
Merge float64 `json:"preview"` // named for the merge preview, apart from the result's merge commit
Dissociate float64 `json:"dissociate"`
Beside float64 `json:"siblings"`
Other float64 `json:"other"`
Modules float64 `json:"modules"`
Total float64 `json:"total"`
}
// phaseClock stamps the phases of a fetch: each lap is the time since the one before, so the laps add up to the total.
type phaseClock struct {
now func() time.Time
start, since time.Time
}
func newPhaseClock(now func() time.Time) *phaseClock {
t := now()
return &phaseClock{now: now, start: t, since: t}
}
// lap adds the time since the last lap to *into.
func (c *phaseClock) lap(into *float64) {
t := c.now()
*into += t.Sub(c.since).Seconds()
c.since = t
}
func (c *phaseClock) total() float64 { return c.since.Sub(c.start).Seconds() }
// String is the one log line a fetch says of its phases.
func (t FetchTook) String() string {
return fmt.Sprintf("mirror %.1fs, fetch %.1fs, merge %.1fs, dissociate %.1fs, beside %.1fs, other %.1fs, modules %.1fs, total %.1fs",
t.Mirror, t.Fetch, t.Merge, t.Dissociate, t.Beside, t.Other, t.Modules, t.Total)
}
// fetchTool is the fetch step tool.
@@ -154,77 +109,43 @@ func (m *Module) fetch(ctx context.Context, args map[string]any) (res any, err e
}
func (m *Module) fetchInto(ctx context.Context, req buildstep.FetchRequest, id string, siblings []siblingArg) (any, error) {
res, err := m.fetchPhases(ctx, req, id, siblings)
if err != nil {
return nil, err
}
if m.Say != nil && res.Took != nil {
m.Say("the fetch of %s at %s took %s", req.Repository, short(req.Commit), res.Took)
}
return res, nil
}
func short(commit string) string {
if len(commit) > 12 {
return commit[:12]
}
return commit
}
func (m *Module) fetchPhases(ctx context.Context, req buildstep.FetchRequest, id string, siblings []siblingArg) (FetchResult, error) {
repository, commit, ref, base := req.Repository, req.Commit, req.Ref, req.Base
var err error
took := &FetchTook{}
clock := newPhaseClock(m.now)
defer func() { took.Total = clock.total() }()
root := filepath.Join(m.Work, id)
tree := filepath.Join(root, "tree")
if err := os.MkdirAll(tree, 0o700); err != nil {
return FetchResult{}, err
return nil, err
}
remote := m.remote(repository)
opts := m.fetchOptions(ctx, repository, commit, ref)
mergeOpts := m.FetchOptions
if base != "" {
// The mirror is brought to the trunk commit too, so the merge preview deepens the tree from it and asks the forge
// for no history the mirror holds (issue 1921, try speed S6).
mergeOpts = m.fetchOptions(ctx, repository, base, "")
if err := buildstep.FetchCommit(ctx, m.Run, tree, remote, commit, ref, m.fetchOptions(ctx, repository, commit, ref)...); err != nil {
return nil, err
}
clock.lap(&took.Mirror)
if err := buildstep.FetchCommit(ctx, m.Run, tree, remote, commit, ref, opts...); err != nil {
return FetchResult{}, err
}
clock.lap(&took.Fetch)
res := FetchResult{Workspace: id, Commit: commit, Took: took}
res := FetchResult{Workspace: id, Commit: commit}
if base != "" {
if err := m.merge(ctx, &res, tree, remote, base, commit, mergeOpts); err != nil {
return FetchResult{}, err
if err := m.merge(ctx, &res, tree, remote, base, commit); err != nil {
return nil, err
}
}
clock.lap(&took.Merge)
if err := m.dissociate(ctx, tree); err != nil {
return FetchResult{}, err
return nil, err
}
clock.lap(&took.Dissociate)
if res.Beside, err = m.placeSiblings(ctx, root, siblings); err != nil {
return FetchResult{}, err
return nil, err
}
clock.lap(&took.Beside)
if (base == "" || res.Merge != "") && len(req.Paths) > 0 {
if err := buildstep.Narrow(tree, req); err != nil {
return FetchResult{}, err
return nil, err
}
// Said beside the tree, so the steps that list the tree's go.mod files leave out those narrowed away.
if err := os.WriteFile(filepath.Join(root, narrowedMark), nil, 0o600); err != nil {
return FetchResult{}, err
return nil, err
}
}
dirs, err := m.goModDirs(ctx, tree)
if err != nil {
return FetchResult{}, err
return nil, err
}
res.Modules = dirs
clock.lap(&took.Other)
if base != "" && res.Merge == "" {
return res, nil // a conflict: the tree is at the trunk, nothing of the head is built or tested
}
@@ -247,10 +168,9 @@ func (m *Module) fetchPhases(ctx context.Context, req buildstep.FetchRequest, id
}()
}
wg.Wait()
clock.lap(&took.Modules)
for _, err := range errs {
if err != nil {
return FetchResult{}, err
return nil, err
}
}
return res, nil
@@ -373,7 +293,6 @@ func (m *Module) goEnv() []string {
modcache = m.GoModCache
}
return []string{
"CGO_ENABLED=0", // as a production build compiles (build.go), so a check tests what ships (issue 1921, S11)
"GOTOOLCHAIN=local",
"GOFLAGS=-mod=mod",
"GOMODCACHE=" + modcache,
@@ -3,7 +3,6 @@ package main
import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
@@ -303,61 +302,3 @@ func TestWorkAndCacheAreTheDirectoriesTheManifestPlacesOutsideTheHome(t *testing
}
}
}
// 1921/S1: the fetch says how long each phase ran (mirror, fetch, merge, dissociate, beside, other, modules), and the
// phases add up to its total, so a try's "clone" time is split into its parts.
func TestTheFetchSaysTheTimeOfEachPhaseAndTheyAddUpToTheTotal(t *testing.T) {
var mu sync.Mutex
now := time.Date(2026, 10, 25, 9, 0, 0, 0, time.UTC)
advance := func(s int) {
mu.Lock()
now = now.Add(time.Duration(s) * time.Second)
mu.Unlock()
}
f := &fakeStep{gomods: "go.mod\n", files: map[string]string{"go.mod": "module x\n", "go.sum": ""}, rule: func(line string) (forgekey.Ran, bool) {
switch {
case strings.Contains(line, "/mirror/") && strings.Contains(line, " fetch "):
advance(1)
case strings.Contains(line, " fetch --no-tags --depth 1 "):
advance(2)
case strings.Contains(line, " merge --no-ff "):
advance(3)
case strings.Contains(line, " repack "):
advance(4)
case strings.HasSuffix(line, " mod download"):
advance(5)
}
return forgekey.Ran{}, false
}}
m, states := fetchModule(t, f)
said := []string{}
m.Say = func(format string, a ...any) { said = append(said, fmt.Sprintf(format, a...)) }
m.Now = func() time.Time { mu.Lock(); defer mu.Unlock(); return now }
args := map[string]any{"repository": "novox/mesh-sdk", "commit": commitSHA, "base": strings.Repeat("b", 40)}
if _, err := fetchTool(t, m)(args); err != nil {
t.Fatal(err)
}
st := states.waitEnd(t)
if st.State != delivery.StepPassed {
t.Fatalf("the step ended %s: %s", st.State, st.Reason)
}
var res FetchResult
if err := json.Unmarshal(st.Result, &res); err != nil {
t.Fatal(err)
}
if res.Took == nil {
t.Fatalf("the result says no phase times: %s", st.Result)
}
got := *res.Took
// the mirror is brought to the head and to the base (1 s each), and the preview deepens from it (1 s) before merging
want := FetchTook{Mirror: 2, Fetch: 2, Merge: 4, Dissociate: 4, Modules: 5, Total: 17}
if got != want {
t.Errorf("phases = %+v, want %+v", got, want)
}
if sum := got.Mirror + got.Fetch + got.Merge + got.Dissociate + got.Beside + got.Other + got.Modules; sum != got.Total {
t.Errorf("the phases add up to %v, not the total %v", sum, got.Total)
}
if !strings.Contains(strings.Join(said, "\n"), "dissociate 4.0s") {
t.Errorf("no log line names the phases: %q", said)
}
}
@@ -10,10 +10,8 @@ import (
"fmt"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/buildstep"
@@ -64,99 +62,34 @@ func (m *Module) integrationHere(ctx context.Context, workspace, tree string, pa
if err != nil {
return nil, err
}
postgres, stopStore, err := m.startStore(ctx, run)
if err != nil {
return nil, fmt.Errorf("the integration level's store: %w", err)
}
defer stopStore()
nats, stopBus, err := m.startBus(ctx, filepath.Join(run, "nats"))
if err != nil {
return nil, fmt.Errorf("the integration level's bus: %w", err)
}
defer stopBus()
env := append([]string{"MESH_TEST_POSTGRES=" + postgres, "MESH_TEST_NATS=" + nats}, m.besideEnv(workspace)...)
// Compiling counts as setup: the tests are built once, here, with the addresses a TestMain may read, and the run
// below finds them built. Whether they compile is for the run to say; its failure is not read here.
// Only the directories the selected packages lie in compile (ADR 0536 §3); no selection is every directory.
byDir, err := packagesByDir(tree, dirs, packages)
if err != nil {
return nil, err
}
// The tests the tags add, not the unit tests again (issue 1921, try speed S7): only the packages holding one compile
// and run, with a -run pattern naming them. A try that names its own pattern runs it as asked.
var runByDir map[string]string
if pattern == "" {
tagged, err := taggedTests(tree, dirs, packages, integrationTags)
if err != nil {
return nil, err
}
if len(tagged) == 0 {
res := &TestResult{Level: LevelIntegration, Packages: []PackageResult{}}
res.judge(m.integrationBounds())
return res, nil // no test the tags add: nothing to start or run
}
byDir, runByDir, packages = map[string][]string{}, map[string]string{}, nil
for d, tr := range tagged {
byDir[d], runByDir[d] = tr.packages, tr.pattern
packages = append(packages, tr.packages...)
}
sort.Strings(packages)
}
// The store, the bus and the compile start together (issue 1921, try speed S5); the setup is the longest of them.
// The compile runs no test binary (-exec true), so it needs neither server: whether the tests compile is for the run
// to say, and its failure is not read here; the run below finds them built.
var (
wg sync.WaitGroup
postgres, nats string
stopStore, stopBus func()
storeErr, busErr error
crash any
crashMu sync.Mutex
)
guard := func() {
if p := recover(); p != nil {
crashMu.Lock()
crash = p
crashMu.Unlock()
}
}
wg.Add(2)
go func() { defer wg.Done(); defer guard(); postgres, stopStore, storeErr = m.startStore(ctx, run) }()
go func() {
defer wg.Done()
defer guard()
nats, stopBus, busErr = m.startBus(ctx, filepath.Join(run, "nats"))
}()
slots := m.slots(workspace) // shared with the ask's unit level (issue 1921, try speed S10)
var slotErr error
for _, d := range dirs {
if len(byDir[d]) == 0 {
continue
}
release, err := slots.acquire(ctx)
if err != nil {
slotErr = err
break
}
wg.Add(1)
go func() {
defer wg.Done()
defer release()
defer guard()
m.Run(ctx, m.goEnvFor(filepath.Join(tree, d)), "go", append([]string{"-C", filepath.Join(tree, d), "test", "-tags", strings.Join(integrationTags, ","), "-exec", "true", "-count=1"}, byDir[d]...)...)
}()
m.Run(ctx, append(m.goEnvFor(filepath.Join(tree, d)), env...), "go", append([]string{"-C", filepath.Join(tree, d), "test", "-tags", strings.Join(integrationTags, ","), "-run", "^$", "-count=1"}, byDir[d]...)...)
}
wg.Wait()
if stopStore != nil {
defer stopStore()
}
if stopBus != nil {
defer stopBus()
}
if crash != nil {
panic(crash)
}
if slotErr != nil {
return nil, slotErr
}
if storeErr != nil {
return nil, fmt.Errorf("the integration level's store: %w", storeErr)
}
if busErr != nil {
return nil, fmt.Errorf("the integration level's bus: %w", busErr)
}
env := append([]string{"MESH_TEST_POSTGRES=" + postgres, "MESH_TEST_NATS=" + nats}, m.besideEnv(workspace)...)
setup := m.clock().Sub(began)
return m.runLevel(ctx, tree, levelOpts{
workspace: workspace, run: pattern, runByDir: runByDir,
workspace: workspace, run: pattern,
level: LevelIntegration, tags: integrationTags, packages: packages, bounds: m.integrationBounds(),
env: env,
setup: setup, wall: true,
@@ -6,7 +6,6 @@ import (
"net/url"
"os"
"path/filepath"
"slices"
"strings"
"sync"
"testing"
@@ -36,8 +35,6 @@ type integFake struct {
// command runs, with its line.
gomods string
onTest func(line string)
// onCompile is called, unlocked, as each compile of the setup runs (issue 1921, try speed S5).
onCompile func(line string)
}
func newIntegFake(t *testing.T, captured string) *integFake {
@@ -74,10 +71,7 @@ func (f *integFake) run(ctx context.Context, env []string, name string, args ...
return forgekey.Ran{Stdout: f.gomods}
}
return forgekey.Ran{Stdout: "go.mod\n"}
case name == "go" && strings.Contains(line, "-exec true"):
if f.onCompile != nil {
f.onCompile(line)
}
case name == "go" && strings.Contains(line, "-run ^$"):
f.tick("compile")
case name == "go":
if f.onTest != nil {
@@ -129,32 +123,9 @@ func integModule(t *testing.T, f *integFake) (*Module, *keptStates) {
if err := os.MkdirAll(filepath.Join(m.Work, "ws1", "tree"), 0o755); err != nil {
t.Fatal(err)
}
// A test the integration tag adds, so the level has a package to run (issue 1921, try speed S7).
writeTaggedTest(t, filepath.Join(m.Work, "ws1", "tree"), "fixture/slow")
return m, states
}
// taggedTestFile is the file writeTaggedTest writes: one test the integration tag adds.
const taggedTestFile = "level_integration_test.go"
// writeTaggedTest gives the go.mod directory dir the module path mp and one test the integration tag adds, in its root
// package and in its package x.
func writeTaggedTest(t *testing.T, dir, mp string) {
t.Helper()
if err := os.WriteFile(filepath.Join(dir, "go.mod"), []byte("module "+mp+"\n\ngo 1.25.0\n"), 0o644); err != nil {
t.Fatal(err)
}
body := "//go:build integration\n\npackage x\n\nimport \"testing\"\n\nfunc TestTakesAWhile(t *testing.T) {}\n"
for _, d := range []string{dir, filepath.Join(dir, "x")} {
if err := os.MkdirAll(d, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(d, taggedTestFile), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
}
func runLevelTool(t *testing.T, m *Module, states *keptStates, level string) (TestResult, delivery.Step) {
t.Helper()
for _, tool := range m.Tools(m.StepTools()) {
@@ -343,11 +314,11 @@ func TestAStoreThatDoesNotStartEndsTheStepFailedAndLeavesNothing(t *testing.T) {
if !strings.Contains(f.lines(), " stop") {
t.Errorf("the half-started store was not stopped:\n%s", f.lines())
}
if f.busDir != "" && !f.busDown {
t.Errorf("the bus, started beside the store, was not stopped after the store failed")
if f.busDir != "" {
t.Errorf("the bus was started after the store failed")
}
if left := workspaceLeft(filepath.Join(m.Work, "ws1")); len(left) != 1 || left[0] != "tree" {
t.Errorf("the workspace holds %v, want only its tree", left)
if entries, _ := os.ReadDir(filepath.Join(m.Work, "ws1")); len(entries) != 1 || entries[0].Name() != "tree" {
t.Errorf("the workspace holds %v, want only its tree", entries)
}
}
@@ -406,138 +377,3 @@ func TestTheSocketPathStaysUnderTheLimitForALongRunDirectory(t *testing.T) {
t.Errorf("the short socket directory %s outlived the store (%v)", sock, err)
}
}
// 1921/S5: the setup compiles the go.mod directories side by side, up to the bound, beside the store's and the bus's
// start, and compiles without running a test binary, so it needs neither; the setup stays one number.
func TestTheSetupCompilesTheDirectoriesAtOnceBesideTheStoreAndTheBus(t *testing.T) {
f := newIntegFake(t, "slow")
f.gomods = "a/go.mod\nb/go.mod\nc/go.mod\nd/go.mod\ne/go.mod\n"
var mu sync.Mutex
running, most := 0, 0
storeStarting, storeUp, overlap := false, false, false
f.onCompile = func(line string) {
mu.Lock()
running++
most = max(most, running)
mu.Unlock()
time.Sleep(40 * time.Millisecond)
mu.Lock()
running--
mu.Unlock()
}
m, states := integModule(t, f)
m.Parallel = 3
tree := filepath.Join(m.Work, "ws1", "tree")
for _, d := range []string{"a", "b", "c", "d", "e"} {
os.MkdirAll(filepath.Join(tree, d), 0o755)
writeTaggedTest(t, filepath.Join(tree, d), "x/"+d)
}
inner := m.Run
m.Run = func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
line := strings.Join(args, " ")
if name == "pg_ctl" && strings.HasSuffix(line, " start") {
mu.Lock()
storeStarting = true
mu.Unlock()
time.Sleep(100 * time.Millisecond) // the store takes a while to start
defer func() { mu.Lock(); storeUp = true; mu.Unlock() }()
}
if name == "go" && strings.Contains(line, "-exec true") {
time.Sleep(10 * time.Millisecond)
mu.Lock()
overlap = overlap || (storeStarting && !storeUp)
mu.Unlock()
for _, e := range env {
if strings.HasPrefix(e, "MESH_TEST_") {
t.Errorf("the compile was given %s: it runs no test binary", e)
}
}
}
return inner(ctx, env, name, args...)
}
res, st := runLevelTool(t, m, states, "integration")
if st.State != delivery.StepPassed {
t.Fatalf("the step ended %s: %s", st.State, st.Reason)
}
if most != 3 {
t.Errorf("%d compiles ran at once, want the bound of 3", most)
}
mu.Lock()
defer mu.Unlock()
if !overlap {
t.Errorf("no compile ran while the store was starting:\n%s", f.lines())
}
if res.Setup < 0 {
t.Errorf("setup = %v", res.Setup)
}
}
// 1921/S7: the integration level runs the tests its tags add, not the unit tests again: only the packages holding a
// tagged test compile and run, with a -run pattern naming those tests; a tree with no tagged test starts no store.
func TestTheIntegrationLevelRunsOnlyTheTestsItsTagsAdd(t *testing.T) {
f := newIntegFake(t, "slow")
m, states := integModule(t, f)
tree := filepath.Join(m.Work, "ws1", "tree")
os.RemoveAll(filepath.Join(tree, "x"))
os.Remove(filepath.Join(tree, taggedTestFile))
for name, body := range map[string]string{
"go.mod": "module fixture/slow\n",
"unit/u_test.go": "package unit\n\nimport \"testing\"\n\nfunc TestUnit(t *testing.T) {}\n",
"store/s_test.go": "package store\n\nimport \"testing\"\n\nfunc TestPlain(t *testing.T) {}\n",
"store/s_integration_test.go": "//go:build integration\n\npackage store\n\nimport \"testing\"\n\nfunc TestMain(m *testing.M) {}\nfunc TestReal(t *testing.T) {}\nfunc Testable() {}\nfunc helper() {}\n",
"store/fake_test.go": "//go:build !integration\n\npackage store\n\nimport \"testing\"\n\nfunc TestFake(t *testing.T) {}\n",
"iso/iso_test.go": "//go:build integration && isolation && linux\n\npackage iso\n\nimport \"testing\"\n\nfunc TestShutOff(t *testing.T) {}\n",
"plat/p_linux_test.go": "//go:build linux\n\npackage plat\n\nimport \"testing\"\n\nfunc TestLinux(t *testing.T) {}\n",
} {
os.MkdirAll(filepath.Dir(filepath.Join(tree, name)), 0o755)
if err := os.WriteFile(filepath.Join(tree, name), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
if _, st := runLevelTool(t, m, states, "integration"); st.State != delivery.StepPassed {
t.Fatalf("the step ended %s: %s", st.State, st.Reason)
}
var compiled, ran []string
for _, c := range f.calls {
switch {
case strings.HasPrefix(c.line, "go ") && strings.Contains(c.line, "-exec true"):
compiled = append(compiled, c.line)
case strings.HasPrefix(c.line, "go ") && strings.Contains(c.line, " -json"):
ran = append(ran, c.line)
}
}
if len(ran) != 1 || !strings.Contains(ran[0], "-run ^(TestReal|TestShutOff)$") ||
!strings.HasSuffix(ran[0], " fixture/slow/iso fixture/slow/store") {
t.Errorf("the run = %q, want the tagged tests of iso and store alone", ran)
}
if len(compiled) != 1 || !strings.HasSuffix(compiled[0], " fixture/slow/iso fixture/slow/store") {
t.Errorf("the compile = %q, want iso and store alone", compiled)
}
g := newIntegFake(t, "slow")
m, states = integModule(t, g)
tree = filepath.Join(m.Work, "ws1", "tree")
os.RemoveAll(filepath.Join(tree, "x"))
os.Remove(filepath.Join(tree, taggedTestFile))
os.WriteFile(filepath.Join(tree, "u_test.go"), []byte("package x\n\nimport \"testing\"\n\nfunc TestUnit(t *testing.T) {}\n"), 0o644)
res, st := runLevelTool(t, m, states, "integration")
if st.State != delivery.StepPassed || !res.Passed || !res.NoTests {
t.Fatalf("a tree with no tagged test: %s %+v", st.State, res)
}
if strings.Contains(g.lines(), "pg_ctl") || strings.Contains(g.lines(), "go -C") {
t.Errorf("a tree with no tagged test started the store or ran go:\n%s", g.lines())
}
}
// workspaceLeft is what a workspace holds besides the directories its ask keeps there (askDirs: its cache layer and its
// bound on processes, gone with the workspace; issue 1921, S2 and S10).
func workspaceLeft(dir string) []string {
var left []string
entries, _ := os.ReadDir(dir)
for _, e := range entries {
if !slices.Contains(askDirs, e.Name()) {
left = append(left, e.Name())
}
}
return left
}
@@ -29,30 +29,21 @@ func (m *Module) caches() []string {
}
// isolation is how a run step of the workspace is shut off: the workspace (its tree, its siblings, its run directories,
// its packed artifacts) read-write, the caches beneath the ask's own layer unless write, and an environment of a PATH and
// its packed artifacts) read-write, the caches beneath a throwaway layer unless write, and an environment of a PATH and
// a locale alone (the step's HOME is its own, inside its /tmp).
func (m *Module) isolation(workspace string, write bool) buildstep.Isolation {
path := os.Getenv("PATH")
if path == "" {
path = defaultPath
}
iso := buildstep.Isolation{
return buildstep.Isolation{
Workspace: filepath.Join(m.Work, workspace),
Caches: m.caches(),
WriteCaches: write,
Env: []string{"PATH=" + path, "LC_ALL=C"},
}
if !write {
// What one command of the ask compiles is kept for its later commands (the vet, the unit tests, the integration
// level's compile), in the workspace and gone with it (issue 1921, try speed S2).
iso.CacheLayer = filepath.Join(iso.Workspace, cacheLayerDir)
}
return iso
}
// cacheLayerDir is the directory in a workspace's root where an ask keeps what its commands wrote into the caches.
const cacheLayerDir = "cachelayer"
// stepRunner is the runner of a run step of the workspace: buildstep.Isolated over m.Run, so every command it runs is
// shut off; write is true for a trunk build alone (the shared caches are written by trunk builds only). Inside is the
// integration level's own process, already running shut off (level.go): it runs its commands as they are.
@@ -20,7 +20,6 @@ import (
type shutOff struct {
flags []string
binds, roBinds, overlays []string
uppers map[string]string // the writable layer each --overlay gives a cache: cache -> upper
env []string
command string
chdir string
@@ -28,7 +27,7 @@ type shutOff struct {
// bwrapArity is how many words each bubblewrap option the SDK writes takes.
var bwrapArity = map[string]int{"--unshare-all": 0, "--die-with-parent": 0, "--new-session": 0, "--clearenv": 0,
"--proc": 1, "--dev": 1, "--tmpfs": 1, "--dir": 1, "--chdir": 1, "--overlay-src": 1, "--tmp-overlay": 1, "--overlay": 3,
"--proc": 1, "--dev": 1, "--tmpfs": 1, "--dir": 1, "--chdir": 1, "--overlay-src": 1, "--tmp-overlay": 1,
"--ro-bind": 2, "--ro-bind-try": 2, "--bind": 2, "--symlink": 2, "--setenv": 2}
// throughBubblewrap is the fake runner of a step's tests: a bubblewrap command is read back (recorded in seen) and its
@@ -57,11 +56,6 @@ func throughBubblewrap(t *testing.T, run forgekey.Runner, seen *[]shutOff) forge
s.roBinds = append(s.roBinds, args[i+1])
case "--overlay-src":
s.overlays = append(s.overlays, args[i+1])
case "--overlay":
if s.uppers == nil {
s.uppers = map[string]string{}
}
s.uppers[args[i+3]] = args[i+1]
case "--setenv":
s.env = append(s.env, args[i+1]+"="+args[i+2])
case "--chdir":
@@ -111,9 +105,6 @@ func checkShutOff(t *testing.T, m *Module, s shutOff, write bool) {
t.Errorf("a trunk build's %s does not write the shared cache %s (bound %v, layered %v)", s.command, c, bound, layered)
case !write && (bound || !layered):
t.Errorf("%s may write the shared cache %s (bound %v, layered %v): only a trunk build does", s.command, c, bound, layered)
case !write && !strings.HasPrefix(s.uppers[c], filepath.Join(m.Work, "ws1", "cachelayer")+"/"):
// 1921/S2: what one command of the ask compiles is kept for its later commands, in the workspace
t.Errorf("%s writes the cache %s into %q, not the ask's cache layer in its workspace", s.command, c, s.uppers[c])
}
}
if want := filepath.Join(m.Work, "ws1"); !slices.Contains(s.binds, want) {
@@ -140,8 +131,7 @@ func TestTheUnitLevelRunsEveryGoTestShutOff(t *testing.T) {
t.Fatalf("%d go commands ran, %d of them shut off: %v", ran, len(seen), *calls)
}
for _, s := range seen {
// the lint's go vet, a compile alone, writes the shared cache (issue 1921, S11): cgo_test.go
checkShutOff(t, m, s, strings.Contains(s.command, " vet "))
checkShutOff(t, m, s, false)
}
}
@@ -41,8 +41,6 @@ type levelSpec struct {
Run string `json:"run,omitempty"`
Setup time.Duration `json:"setup"`
Bound time.Duration `json:"bound"`
// Parallel is the ask's bound on processes, which the process shares with the ask's other level (issue 1921, S10).
Parallel int `json:"parallel,omitempty"`
}
// levelAnswer is the one line the process prints: the level's result or the digest, or why it could not run.
@@ -85,7 +83,7 @@ func (m *Module) integrationShutOff(ctx context.Context, workspace string, packa
// workspace read-write, the program alone read-only, the shared caches beneath a throwaway layer, the Node build
// module's toolchains read-only for the integration level.
func (m *Module) shutOff(ctx context.Context, exe string, spec levelSpec) (levelAnswer, error) {
spec.Work, spec.Cache, spec.Parallel = m.Work, m.Cache, m.parallel()
spec.Work, spec.Cache = m.Work, m.Cache
raw, err := json.Marshal(spec)
if err != nil {
return levelAnswer{}, err
@@ -167,7 +165,7 @@ func runLevelProcess(args []string) int {
return say(levelAnswer{Error: fmt.Sprintf("%q is not a workspace id", spec.Workspace)})
}
m := &Module{Inside: true, Recipe: Current(), Work: spec.Work, Cache: spec.Cache, Home: os.Getenv("HOME"), UnitBound: spec.Bound,
IntegrationBounds: Bounds{Setup: spec.Setup, Run: spec.Bound}, Parallel: spec.Parallel, Say: func(string, ...any) {}}
IntegrationBounds: Bounds{Setup: spec.Setup, Run: spec.Bound}, Say: func(string, ...any) {}}
m.Run = StepRunner(m.Home)
return say(m.answer(context.Background(), spec))
}
+203 -22
View File
@@ -10,6 +10,7 @@ import (
"encoding/json"
"fmt"
"regexp"
"slices"
"sort"
"strings"
"time"
@@ -40,6 +41,8 @@ type PackageResult struct {
Tests int `json:"tests"`
// NoTests is a package with no test files (at this level: a file with the other level's build tag is none).
NoTests bool `json:"no_tests,omitempty"`
// Skipped is how many of the package's tests skipped: a pass that skipped proved less than it says, so the checker keeps no verdict for it.
Skipped int `json:"skipped,omitempty"`
}
// TestTime is one test and the time it took.
@@ -52,19 +55,17 @@ type TestTime struct {
// Failure is a test that failed, or a package that failed with no test to name (a build that failed, a TestMain that
// panicked); Output is the last lines it wrote, up to outputBytes (issue 1901), and Cause the first line of what the
// test reported as its error (t.Error, t.Fatal), without its file and line, when go test said which lines those were.
// **A failure that is not one test's assertion carries its real cause too** (issue 1901, its third meeting): the
// "panic:" line (a timeout's with the tests it found running), a "fatal error:" or "signal:" line, a build's first
// error, or in words that the test program exited, while which tests ran or after its tests passed. Test then names the
// tests that were running when the program stopped. A "PASS" or "ok" line is never a failure's words.
type Failure struct {
Package string `json:"package"`
Test string `json:"test,omitempty"`
Output string `json:"output,omitempty"`
Cause string `json:"cause,omitempty"`
// Step is StepLint for a finding of the lint step (gofmt, go vet), which runs beside the unit level and is not one of
// the levels (ADR 0542 §2): the checker reports it as the lint step's failure. Empty for a test's failure.
Step string `json:"step,omitempty"`
}
// StepLint is the Step of a lint finding.
const StepLint = "lint"
// TestResult is what a test step passes with.
type TestResult struct {
Level string `json:"level"`
@@ -79,7 +80,7 @@ type TestResult struct {
Elapsed float64 `json:"elapsed"`
// Setup is the level's setup time in seconds, when it has a setup.
Setup float64 `json:"setup,omitempty"`
// Lint is the unit level's gofmt and go vet time in seconds, run beside its tests (ADR 0529 §1; issue 1921).
// Lint is the unit level's gofmt and go vet time in seconds, before its tests (ADR 0529 §1).
Lint float64 `json:"lint,omitempty"`
// Bound and SetupBound are the bounds the level was judged by, in seconds.
Bound float64 `json:"bound"`
@@ -132,11 +133,18 @@ type reader struct {
causes map[string]string // the first error line of a test, by the same key
times []TestTime
failedIn map[string]bool // packages with a failed test
// running is the tests that started and have not ended, by package, in the order they started: when a package
// fails with no failed test, these were running when its test program stopped (issue 1901).
running map[string][]string
// passed says a package's test program printed "PASS" (its tests passed) before it failed; built says its test
// program failed to build.
passed, built map[string]bool
}
func newReader(res *TestResult, dir string) *reader {
return &reader{res: res, dir: dir, pkgs: map[string]*PackageResult{}, builds: map[string][]string{},
pkgOut: map[string][]string{}, testOut: map[string][]string{}, causes: map[string]string{}, failedIn: map[string]bool{}}
pkgOut: map[string][]string{}, testOut: map[string][]string{}, causes: map[string]string{}, failedIn: map[string]bool{},
running: map[string][]string{}, passed: map[string]bool{}, built: map[string]bool{}}
}
// read takes go test's output. Lines that are not events are kept for a failure that has nothing else to say.
@@ -184,12 +192,21 @@ func (r *reader) event(ev event) {
p := r.pkg(ev.Package)
switch ev.Action {
case "output":
r.pkgOut[ev.Package] = keep(r.pkgOut[ev.Package], ev.Output)
// go test's own summary of a package ("PASS", "FAIL", "ok <pkg>", "FAIL <pkg>") is never a failure's
// words: a "PASS" before a "FAIL <pkg>" says only that the tests passed and the program failed after them.
switch l := strings.TrimRight(ev.Output, "\r\n"); {
case l == "PASS":
r.passed[ev.Package] = true
case summaryLine(l):
default:
r.pkgOut[ev.Package] = keep(r.pkgOut[ev.Package], ev.Output)
}
case "pass":
p.Passed, p.Elapsed = true, ev.Elapsed
case "fail":
p.Passed, p.Elapsed = false, ev.Elapsed
if ev.FailedBuild != "" {
r.built[ev.Package] = true
r.pkgOut[ev.Package] = append(r.builds[ev.FailedBuild], r.pkgOut[ev.Package]...)
}
case "skip":
@@ -203,26 +220,37 @@ func (r *reader) event(ev event) {
if ev.OutputType == "error" && r.causes[key] == "" {
r.causes[key] = errorWords(ev.Output)
}
case "run":
r.running[ev.Package] = append(r.running[ev.Package], ev.Test)
case "pass", "fail", "skip":
r.running[ev.Package] = slices.DeleteFunc(r.running[ev.Package], func(t string) bool { return t == ev.Test })
p := r.pkg(ev.Package)
p.Tests++
r.res.Tests++
if ev.Action == "skip" {
r.res.Skipped++
p.Skipped++
}
r.times = append(r.times, TestTime{Package: ev.Package, Test: ev.Test, Elapsed: ev.Elapsed})
if ev.Action == "fail" {
r.failedIn[ev.Package] = true
r.res.Failed = append(r.res.Failed, Failure{Package: ev.Package, Test: ev.Test, Output: clip(r.testOut[key], r.causes[key]), Cause: r.causes[key]})
cause := r.causes[key]
if race := raceCause(r.testOut[key]); race != "" && (cause == "" || strings.HasPrefix(cause, "race detected")) {
cause = race // the race itself, not testing's "race detected during execution of test"
} else if cause == "" {
cause = causeOf(r.testOut[key])
}
r.res.Failed = append(r.res.Failed, Failure{Package: ev.Package, Test: ev.Test, Output: clip(r.testOut[key], cause), Cause: cause})
}
}
}
}
// clip joins the last lines of a failure's output that fit outputBytes, each cut to outputLineBytes, and says how many
// earlier lines it left out. **When it leaves lines out, its first line is still a real one** (issue 1901/AC3): the failure's
// cause, or failing that the first non-empty line of the output, then the marker. An older checker takes only the first
// non-empty line of the output, and a marker there would be less than the older pair kept.
// earlier lines it left out. **Its first line is the failure's cause** (issue 1901/AC3, AC4), put ahead when the kept
// lines do not start with it, or, with no cause and lines left out, the first non-empty line of the output, then the
// marker. An older checker takes only the first non-empty line of the output, and a marker there would be less than the
// older pair kept.
func clip(lines []string, cause string) string {
size, from := 0, len(lines)
for from > 0 {
@@ -233,15 +261,15 @@ func clip(lines []string, cause string) string {
size += n
from--
}
kept := make([]string, 0, len(lines)-from+1)
kept := make([]string, 0, len(lines)-from+2)
head := strings.TrimSpace(cause)
for i := 0; head == "" && i < from; i++ {
head = strings.TrimSpace(lines[i])
}
if head != "" && (from > 0 || len(lines) == 0 || strings.TrimSpace(lines[from]) != head) {
kept = append(kept, cutLine(head))
}
if from > 0 {
head := strings.TrimSpace(cause)
for i := 0; head == "" && i < from; i++ {
head = strings.TrimSpace(lines[i])
}
if head != "" {
kept = append(kept, cutLine(head))
}
kept = append(kept, fmt.Sprintf("... (%d earlier lines)", from))
}
for _, l := range lines[from:] {
@@ -250,6 +278,159 @@ func clip(lines []string, cause string) string {
return strings.Join(kept, "\n")
}
// summaryLine says a line is go test's summary of a package, in go's exact forms only: "PASS" or "FAIL" alone,
// "ok \t<pkg>\t<time>" and "FAIL\t<pkg>\t<time>" (review round 1: a line of the package's own that starts "ok " is kept).
func summaryLine(l string) bool {
return l == "PASS" || l == "FAIL" || strings.HasPrefix(l, "ok \t") || strings.HasPrefix(l, "FAIL\t")
}
// causeOf is the cause in a failure's output that is not a test's assertion (issue 1901): a race the race detector
// reported (raceCause); else the first "panic:" line, a timeout's with the tests it names as running; else the first
// "fatal error:" (the runtime's) or "signal:" line (a test program killed); "" when there is none. It reads every line,
// so a long goroutine dump after the panic cannot hide it. **It matches only lines at column 0**, where go prints the
// runtime's: a test's indented log line holding the same words is not the cause (review round 1).
func causeOf(lines []string) string {
if race := raceCause(lines); race != "" {
return race
}
for _, prefix := range []string{"panic: ", "fatal error: ", "signal: "} {
for i, l := range lines {
l = strings.TrimRight(l, "\r\n") // at column 0: an indented line is a test's own log, not the runtime's
if !strings.HasPrefix(l, prefix) {
continue
}
if strings.HasPrefix(l, "panic: test timed out after") {
var names []string
for _, n := range lines[i+1:] {
if !strings.HasPrefix(n, "\t\t") {
if strings.TrimSpace(n) == "running tests:" {
continue
}
break
}
names = append(names, strings.TrimSpace(n))
}
if len(names) > 0 {
l += "; running tests: " + strings.Join(names, ", ")
}
}
return cutLine(l)
}
}
return ""
}
// raceCause is the first race the race detector reported in a failure's output, in one line (issue 1901: a race after
// the tests' "PASS" was worded "PASS"): its access and the frame that made it, and where the goroutine that made it was
// created, as "WARNING: DATA RACE: Write at 0x... by goroutine 10 in pkg.F.func1() (f_test.go:13); goroutine created at
// pkg.F() (f_test.go:13)". "" when the output holds no race.
func raceCause(lines []string) string {
at := -1
for i, l := range lines {
if strings.TrimRight(l, "\r\n") == "WARNING: DATA RACE" { // at column 0, as causeOf
at = i
break
}
}
if at < 0 {
return ""
}
// frame is the function and the file's base name and line that follow a heading line at i.
frame := func(i int) string {
var fn, file string
for _, l := range lines[i+1:] {
if l = strings.TrimSpace(l); l == "" {
break
}
if fn == "" {
fn = l
continue
}
file = l
if j := strings.LastIndexByte(file, '/'); j >= 0 {
file = file[j+1:]
}
if j := strings.Index(file, " +0x"); j >= 0 {
file = file[:j]
}
break
}
if file != "" {
return fn + " (" + file + ")"
}
return fn
}
words := "WARNING: DATA RACE"
for i := at + 1; i < len(lines); i++ {
l := strings.TrimSpace(lines[i])
if l == "" || strings.HasPrefix(l, "===") {
continue
}
words += ": " + strings.TrimSuffix(l, ":")
if f := frame(i); f != "" {
words += " in " + f
}
break
}
for i := at + 1; i < len(lines); i++ {
l := strings.TrimSpace(lines[i])
if strings.HasPrefix(l, "==================") {
break
}
if strings.HasPrefix(l, "Goroutine ") && strings.HasSuffix(l, " created at:") {
if f := frame(i); f != "" {
words += "; its goroutine created at " + f
}
break
}
}
return cutLine(words)
}
// buildCause is the first error line of a failed build: the first line that is not go's "# <package>" heading.
func buildCause(lines []string) string {
for _, l := range lines {
if l = strings.TrimSpace(l); l != "" && !strings.HasPrefix(l, "# ") {
return cutLine(l)
}
}
return ""
}
// testNames names the tests that were running, at most three, and how many more.
func testNames(tests []string) string {
if len(tests) <= 3 {
return strings.Join(tests, ", ")
}
return fmt.Sprintf("%s and %d more", strings.Join(tests[:3], ", "), len(tests)-3)
}
// packageFailure is a package that failed with no failed test to name it (issue 1901): its test program did not build,
// panicked, timed out, was killed or exited. It names the tests that were running when the program stopped, keeps
// their output with the package's, and says the cause.
func (r *reader) packageFailure(name string) Failure {
running := r.running[name]
var lines []string
for _, t := range running {
lines = append(lines, r.testOut[name+"\x00"+t]...)
}
lines = append(lines, r.pkgOut[name]...)
cause := causeOf(lines)
switch {
case cause != "":
case r.built[name]:
cause = buildCause(lines)
case len(running) > 0:
cause = "the test program exited while " + testNames(running) + " ran, with no panic: an os.Exit or a signal (go test -json reports no exit status)"
case r.passed[name]:
cause = "the tests passed, then the test program exited with a failure: an os.Exit or a panic in TestMain or after the tests (go test -json reports no exit status)"
}
if cause == "" {
cause = "the package failed with no test to name and no panic, signal or build error in its output"
}
return Failure{Package: name, Test: testNames(running), Output: clip(lines, cause), Cause: cause}
}
// cutLine cuts a line to outputLineBytes, on a character boundary.
func cutLine(l string) string {
if len(l) <= outputLineBytes {
@@ -271,7 +452,7 @@ func (r *reader) finish() {
for _, name := range r.order {
p := r.pkgs[name]
if !p.Passed && !r.failedIn[name] {
r.res.Failed = append(r.res.Failed, Failure{Package: name, Output: clip(r.pkgOut[name], "")})
r.res.Failed = append(r.res.Failed, r.packageFailure(name))
}
if !p.NoTests && p.Elapsed > r.res.Elapsed {
r.res.Elapsed = p.Elapsed
+30 -104
View File
@@ -60,23 +60,17 @@ func lintable(dir string, each func(path string) error) error {
})
}
// unformatted are the Go files of a directory gofmt would change, relative to the tree, the build tags the directory's
// files name, and whether a file is left out by one of them (a negated tag, //go:build !integration). Only files whose
// directory (relative to the go.mod directory, "." for its own) is in only are checked for gofmt; nil checks every file.
func unformatted(tree, d string, only map[string]bool) (files, tags []string, negated bool, err error) {
// unformatted are the Go files of a directory gofmt would change, relative to the tree, and the build tags they name.
func unformatted(tree, d string) (files, tags []string, err error) {
seen := map[string]bool{}
base := filepath.Join(tree, d)
err = lintable(base, func(path string) error {
err = lintable(filepath.Join(tree, d), func(path string) error {
src, err := os.ReadFile(path)
if err != nil {
return err
}
pkgDir, _ := filepath.Rel(base, filepath.Dir(path))
if only == nil || only[filepath.ToSlash(pkgDir)] {
if out, ferr := format.Source(src); ferr != nil || !bytes.Equal(out, src) {
rel, _ := filepath.Rel(tree, path)
files = append(files, filepath.ToSlash(rel))
}
if out, ferr := format.Source(src); ferr != nil || !bytes.Equal(out, src) {
rel, _ := filepath.Rel(tree, path)
files = append(files, filepath.ToSlash(rel))
}
scan := bufio.NewScanner(bytes.NewReader(src))
for scan.Scan() {
@@ -85,15 +79,8 @@ func unformatted(tree, d string, only map[string]bool) (files, tags []string, ne
break
}
if expr, ok := strings.CutPrefix(line, "//go:build "); ok {
for _, at := range buildTag.FindAllStringIndex(expr, -1) {
tag := expr[at[0]:at[1]]
if notATag[tag] || strings.HasPrefix(tag, "go1") {
continue
}
if before := strings.TrimRight(expr[:at[0]], " ("); strings.HasSuffix(before, "!") {
negated = true
}
if !seen[tag] {
for _, tag := range buildTag.FindAllString(expr, -1) {
if !notATag[tag] && !strings.HasPrefix(tag, "go1") && !seen[tag] {
seen[tag] = true
tags = append(tags, tag)
}
@@ -104,123 +91,62 @@ func unformatted(tree, d string, only map[string]bool) (files, tags []string, ne
})
sort.Strings(files)
sort.Strings(tags)
return files, tags, negated, err
return files, tags, err
}
// boundOf is a bound of n processes of this process alone.
func boundOf(n int) func(context.Context) (func(), error) {
slots := make(chan struct{}, n)
return func(context.Context) (func(), error) {
slots <- struct{}{}
return func() { <-slots }, nil
}
}
// lintScope is what of a go.mod directory d the lint covers, from the patterns the level runs there: every package
// (./..., and the directories nil) when a pattern is relative or none is given, else the import paths given and the
// directories of their files, relative to d.
func lintScope(tree, d string, patterns []string) (targets []string, dirs map[string]bool) {
if len(patterns) == 0 {
return []string{"./..."}, nil
}
raw, _ := os.ReadFile(filepath.Join(tree, d, "go.mod"))
mp := modulePath(string(raw))
dirs = map[string]bool{}
for _, p := range patterns {
if strings.HasPrefix(p, "./") || p == "." {
return []string{"./..."}, nil
}
rel, ok := strings.CutPrefix(p, mp)
if mp == "" || !ok || (rel != "" && !strings.HasPrefix(rel, "/")) {
return []string{"./..."}, nil // a package of no module here: the whole directory, as before
}
if rel = strings.TrimPrefix(rel, "/"); rel == "" {
rel = "."
}
dirs[rel] = true
targets = append(targets, p)
}
return targets, dirs
}
// lint is gofmt and go vet of one go.mod directory d of the tree, run with run in the directory's go environment, over
// the packages the level runs there (patterns, as packagesByDir gives them; issue 1921, try speed S3); a failure of either
// is a Failure of the level naming what failed. go vet runs once with every tag the directory names, which vets the
// untagged files too; only a directory where a tag leaves a file out vets also without tags, both passes at once.
func (m *Module) lint(ctx context.Context, run forgekey.Runner, tree, d string, env, patterns []string) ([]Failure, error) {
targets, only := lintScope(tree, d, patterns)
files, tags, negated, err := unformatted(tree, d, only)
// lint is gofmt and go vet of one go.mod directory d of the tree, run with run in the directory's go environment; a
// failure of either is a Failure of the level naming what failed.
func (m *Module) lint(ctx context.Context, run forgekey.Runner, tree, d string, env []string) ([]Failure, error) {
files, tags, err := unformatted(tree, d)
if err != nil {
return nil, fmt.Errorf("reading the Go files of %s: %w", d, err)
}
var failed []Failure
if len(files) > 0 {
failed = append(failed, Failure{Package: d, Test: "gofmt", Step: StepLint,
failed = append(failed, Failure{Package: d, Test: "gofmt",
Output: "not as gofmt writes them (run gofmt -w): " + strings.Join(files, ", ")})
}
sets := [][]string{tags}
if len(tags) > 0 && negated {
sets = [][]string{nil, tags}
sets := [][]string{nil}
if len(tags) > 0 {
sets = append(sets, tags)
}
type vetRun struct {
name string
r forgekey.Ran
}
runs := make([]vetRun, len(sets))
var wg sync.WaitGroup
for i, set := range sets {
for _, set := range sets {
args := []string{"-C", filepath.Join(tree, d), "vet"}
name := "go vet"
if len(set) > 0 {
args = append(args, "-tags", strings.Join(set, ","))
name += " -tags " + strings.Join(set, ",")
}
args = append(args, targets...)
runs[i].name = name
wg.Add(1)
go func() {
defer wg.Done()
runs[i].r = run(ctx, env, "go", args...)
}()
}
wg.Wait()
for _, v := range runs {
if v.r.Err != "" {
return nil, fmt.Errorf("%s in %s could not be run: %s", v.name, d, firstLine(v.r.Err+"\n"+v.r.Stderr))
args = append(args, "./...")
r := run(ctx, env, "go", args...)
if r.Err != "" {
return nil, fmt.Errorf("%s in %s could not be run: %s", name, d, firstLine(r.Err+"\n"+r.Stderr))
}
if v.r.Status != 0 {
failed = append(failed, Failure{Package: d, Test: v.name, Step: StepLint, Output: strings.TrimSpace(v.r.Stderr + "\n" + v.r.Stdout)})
if r.Status != 0 {
failed = append(failed, Failure{Package: d, Test: name, Output: strings.TrimSpace(r.Stderr + "\n" + r.Stdout)})
}
}
return failed, nil
}
// lintAll lints each directory, up to m.parallel() at once like the level's tests; the failures join in directory order
// and the error is the first directory's, so the answer does not depend on which ended first (issue 1930). byDir names
// the packages the level runs in each directory (packagesByDir); nil lints each directory whole. acquire takes a slot of
// the bound the lint shares with the ask's tests; nil gives it one of its own.
func (m *Module) lintAll(ctx context.Context, run forgekey.Runner, tree string, dirs []string, env func(d string) []string, byDir map[string][]string, acquire func(context.Context) (func(), error)) ([]Failure, error) {
// and the error is the first directory's, so the answer does not depend on which ended first (issue 1930).
func (m *Module) lintAll(ctx context.Context, run forgekey.Runner, tree string, dirs []string, env func(d string) []string) ([]Failure, error) {
type answer struct {
failed []Failure
err error
}
answers := make([]answer, len(dirs))
if acquire == nil {
acquire = boundOf(m.parallel())
}
slots := make(chan struct{}, m.parallel())
var wg sync.WaitGroup
var crash any // a panic of a lint goes on in the caller, as a panic of a test run does
var crashMu sync.Mutex
for i, d := range dirs {
release, err := acquire(ctx)
if err != nil {
answers[i].err = err
break
}
wg.Add(1)
slots <- struct{}{}
go func() {
defer wg.Done()
defer release()
defer func() { <-slots }()
defer func() {
if p := recover(); p != nil {
crashMu.Lock()
@@ -228,7 +154,7 @@ func (m *Module) lintAll(ctx context.Context, run forgekey.Runner, tree string,
crashMu.Unlock()
}
}()
answers[i].failed, answers[i].err = m.lint(ctx, run, tree, d, env(d), byDir[d])
answers[i].failed, answers[i].err = m.lint(ctx, run, tree, d, env(d))
}()
}
wg.Wait()
@@ -4,13 +4,11 @@ import (
"context"
"os"
"path/filepath"
"slices"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
@@ -30,9 +28,8 @@ func writeTree(t *testing.T, files map[string]string) string {
}
// 1887/D7-go, ADR 0529 §1: an unformatted file fails the unit level naming the file (testdata and vendor are left
// alone), and go vet runs with every tag the files name together, integration and isolation included, so a vet error
// behind isolation fails the level naming the run. One pass with every tag vets the untagged files too (issue 1921, try
// speed S3), as no file here is left out by a tag.
// alone), and go vet runs with no tag and with every tag the files name together, integration and isolation included, so
// a vet error behind isolation fails the level naming the run.
func TestTheUnitLevelRunsGofmtAndVetWithEveryTag(t *testing.T) {
tree := writeTree(t, map[string]string{
"go.mod": "module x\n\ngo 1.26\n",
@@ -54,11 +51,11 @@ func TestTheUnitLevelRunsGofmtAndVetWithEveryTag(t *testing.T) {
}
return forgekey.Ran{}
}
failed, err := (&Module{}).lint(context.Background(), run, tree, ".", nil, nil)
failed, err := (&Module{}).lint(context.Background(), run, tree, ".", nil)
if err != nil {
t.Fatal(err)
}
if strings.Join(vetted, " | ") != "vet -tags integration,isolation ./..." {
if strings.Join(vetted, " | ") != "vet ./... | vet -tags integration,isolation ./..." {
t.Errorf("vet ran %v", vetted)
}
if len(failed) != 2 || failed[0].Test != "gofmt" || !strings.Contains(failed[0].Output, "b.go") ||
@@ -70,14 +67,14 @@ func TestTheUnitLevelRunsGofmtAndVetWithEveryTag(t *testing.T) {
}
// A formatted directory that vets passes.
clean := writeTree(t, map[string]string{"go.mod": "module y\n", "a.go": "package y\n"})
if failed, err := (&Module{}).lint(context.Background(), func(context.Context, []string, string, ...string) forgekey.Ran { return forgekey.Ran{} }, clean, ".", nil, nil); err != nil || len(failed) != 0 {
if failed, err := (&Module{}).lint(context.Background(), func(context.Context, []string, string, ...string) forgekey.Ran { return forgekey.Ran{} }, clean, ".", nil); err != nil || len(failed) != 0 {
t.Errorf("a clean directory: %+v, %v", failed, err)
}
}
// 1887/D7-go: through the step, the unit level of a tree holding an unformatted file fails, naming the file.
func TestAnUnformattedFileFailsTheUnitLevel(t *testing.T) {
m, states, ws, _ := workspaceFor(t, "empty")
m, states, ws, calls := workspaceFor(t, "empty")
tree := filepath.Join(m.Work, ws, "tree")
for name, body := range map[string]string{"go.mod": "module x\n", "x.go": "package x\nfunc X() {}\n"} {
if err := os.WriteFile(filepath.Join(tree, name), []byte(body), 0o644); err != nil {
@@ -88,117 +85,50 @@ func TestAnUnformattedFileFailsTheUnitLevel(t *testing.T) {
if res.Passed || len(res.Failed) != 1 || res.Failed[0].Test != "gofmt" || !strings.Contains(res.Failed[0].Output, "x.go") {
t.Fatalf("result = %+v, want the level failed naming x.go", res)
}
}
// 1921/S4: the lint runs beside the unit level's tests, not before them: both have started before either ends, a lint
// failure still fails the level, and the result reports the lint's time and the tests' failures both.
func TestTheLintRunsBesideTheUnitTestsAndItsFailureStillFailsTheLevel(t *testing.T) {
m, states, ws, _ := workspaceFor(t, "failing")
tree := filepath.Join(m.Work, ws, "tree")
for name, body := range map[string]string{"go.mod": "module fixture/failing\n", "x.go": "package x\nfunc X() {}\n"} {
if err := os.WriteFile(filepath.Join(tree, name), []byte(body), 0o644); err != nil {
t.Fatal(err)
// Review round 2 item 5: before the level's tests, which do not run once gofmt failed.
for _, c := range *calls {
if strings.Contains(c.line, " test -json") {
t.Errorf("the tests ran after gofmt failed: %s", c.line)
}
}
m.Parallel = 4
inner := m.Run
var vetStarted, testStarted sync.Once
vet, test := make(chan struct{}), make(chan struct{})
timeout := time.After(2 * time.Second)
m.Run = func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
line := strings.Join(args, " ")
switch {
case strings.Contains(line, " vet "):
vetStarted.Do(func() { close(vet) })
select { // the vet ends only once a test has started
case <-test:
case <-timeout:
t.Error("no test started while the vet ran")
}
case strings.Contains(line, " test -json"):
testStarted.Do(func() { close(test) })
select {
case <-vet:
case <-timeout:
t.Error("no vet started while the tests ran")
}
}
return inner(ctx, env, name, args...)
}
res, st := runUnit(t, m, states, ws)
if st.State != delivery.StepPassed {
t.Fatalf("the step ended %s: %s", st.State, st.Reason)
}
if res.Passed || len(res.Failed) < 2 || res.Failed[0].Test != "gofmt" || res.Failed[1].Test != "TestThreeIsFour" {
t.Fatalf("failed = %+v, want gofmt's failure, then the test's", res.Failed)
}
// ADR 0542 §2: the lint's failure is the lint step's, not the unit level's.
if res.Failed[0].Step != StepLint || res.Failed[1].Step != "" {
t.Errorf("steps = %q, %q; want the gofmt failure the lint step's and the test's the level's", res.Failed[0].Step, res.Failed[1].Step)
}
}
// 1921/S3: the lint of a level that runs some packages vets those packages alone, in one pass with every tag their
// directory names, and reports a gofmt fault only in their files: a fault in a package the level keeps was reported
// when that package changed.
func TestTheLintOfSomePackagesVetsThemInOnePassAndFormatsOnlyTheirFiles(t *testing.T) {
tree := writeTree(t, map[string]string{
"go.mod": "module x\n\ngo 1.26\n",
"a/a.go": "package a\n\nfunc A() {}\n",
"a/a_integration_test.go": "//go:build integration\n\npackage a\n",
"b/b.go": "package b\nfunc B() {}\n",
"a/deeper/d.go": "package deeper\nfunc D() {}\n",
})
var vetted []string
run := func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
vetted = append(vetted, strings.Join(args[2:], " "))
return forgekey.Ran{}
// Issue 1930: the lint of many directories runs side by side up to the level's bound, not one after another, and the
// failures still join in directory order.
func TestTheLintOfManyDirectoriesRunsAtOnceUpToTheBound(t *testing.T) {
files := map[string]string{}
var dirs []string
for _, d := range []string{"a", "b", "c", "d", "e", "f"} {
files[d+"/go.mod"] = "module x/" + d + "\n"
files[d+"/x.go"] = "package x\n"
dirs = append(dirs, d)
}
failed, err := (&Module{}).lint(context.Background(), run, tree, ".", nil, []string{"x/a"})
if err != nil {
t.Fatal(err)
}
if strings.Join(vetted, " | ") != "vet -tags integration x/a" {
t.Errorf("vet ran %v, want one pass over x/a with its tag", vetted)
}
if len(failed) != 0 {
t.Errorf("the faults of packages the level keeps were reported: %+v", failed)
}
vetted = nil
failed, err = (&Module{}).lint(context.Background(), run, tree, ".", nil, []string{"x/b"})
if err != nil {
t.Fatal(err)
}
if len(failed) != 1 || failed[0].Test != "gofmt" || !strings.Contains(failed[0].Output, "b/b.go") || strings.Contains(failed[0].Output, "d.go") {
t.Errorf("failed = %+v, want gofmt naming b/b.go alone", failed)
}
if strings.Join(vetted, " | ") != "vet -tags integration x/b" {
t.Errorf("vet ran %v", vetted)
}
}
// 1921/S3: a file a tag leaves out (//go:build !integration) is vetted by no pass with the tag, so a directory holding
// one keeps the pass without tags beside the pass with them.
func TestADirectoryWithAFileATagLeavesOutKeepsThePassWithoutTags(t *testing.T) {
tree := writeTree(t, map[string]string{
"go.mod": "module x\n",
"a.go": "package x\n",
"fake.go": "//go:build !integration\n\npackage x\n",
"real_test.go": "//go:build integration\n\npackage x\n",
})
tree := writeTree(t, files)
var mu sync.Mutex
var vetted []string
running, most := 0, 0
run := func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
mu.Lock()
vetted = append(vetted, strings.Join(args[2:], " "))
running++
most = max(most, running)
mu.Unlock()
time.Sleep(30 * time.Millisecond)
mu.Lock()
running--
mu.Unlock()
if strings.HasSuffix(args[1], "b") || strings.HasSuffix(args[1], "e") {
return forgekey.Ran{Status: 1, Stderr: "bad " + filepath.Base(args[1])}
}
return forgekey.Ran{}
}
if _, err := (&Module{}).lint(context.Background(), run, tree, ".", nil, nil); err != nil {
m := &Module{Parallel: 3}
failed, err := m.lintAll(context.Background(), run, tree, dirs, func(string) []string { return nil })
if err != nil {
t.Fatal(err)
}
slices.Sort(vetted)
if strings.Join(vetted, " | ") != "vet -tags integration ./... | vet ./..." {
t.Errorf("vet ran %v, want the pass without tags and the pass with them", vetted)
if most < 2 || most > 3 {
t.Errorf("%d lint commands ran at once, want 2 or 3 (the bound)", most)
}
if len(failed) != 2 || failed[0].Package != "b" || failed[1].Package != "e" {
t.Errorf("failed = %+v, want b's then e's", failed)
}
}
@@ -40,6 +40,8 @@ func main() {
Emit: func(t string, b any) error { return stdio.Emit(t, b) },
Steps: stdio.State(delivery.StepsState).Put,
ReadSteps: readOf(stdio.State(delivery.StepsState)),
Pauses: stdio.State(delivery.PausesState).Put,
ReadPauses: readOf(stdio.State(delivery.PausesState)),
ReadToolchainsState: readOf(stdio.State(delivery.ToolchainsState)),
Ctx: ctx,
Say: logf,
@@ -92,9 +92,13 @@ func TestManifestKeepsItsStatesClaimsItsBuildSeatAndDeclaresItsHealthTool(t *tes
states := []string{}
for _, s := range m.State {
states = append(states, s.Name)
// A step's key is per step and dropped after its age, so the bucket cannot fill; a pause has no age.
if (s.Name == "steps") != (s.TTLSeconds > 0) {
t.Errorf("state %s has ttl-seconds %d: only steps has an age", s.Name, s.TTLSeconds)
}
}
slices.Sort(states)
if want := []string{"keys", "steps", "toolchains"}; !slices.Equal(states, want) {
if want := []string{"keys", "pauses", "steps", "toolchains"}; !slices.Equal(states, want) {
t.Errorf("state = %v, want %v", states, want)
}
if len(m.Claims) != 1 || m.Claims[0].Name != Seat {
@@ -12,7 +12,6 @@ import (
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
func fetchOnce(t *testing.T, m *Module, states *keptStates, args map[string]any) FetchResult {
@@ -189,61 +188,3 @@ func TestTheWorkspaceIsSelfContainedAfterAFetchThroughTheMirror(t *testing.T) {
t.Errorf("the workspace is not whole: %v %s", err, out)
}
}
// 1921/S6: with a base, the mirror is brought to the trunk commit too, and the merge preview deepens the tree from the
// mirror: it asks the forge for no history the mirror holds.
func TestTheMirrorIsBroughtToTheBaseAndThePreviewFetchesFromIt(t *testing.T) {
f := &fakeStep{gomods: "go.mod\n", rule: previewRules(false)}
m, states := fetchModule(t, f)
mirror := filepath.Join(m.Cache, "mirror", "novox", "mesh-sdk.git")
f.files = map[string]string{"modules/m/module.json": "{}\n", "modules/m/go.sum": "sum\n", "go.mod": "module x\n", "go.sum": ""}
fetchOnce(t, m, states, map[string]any{"repository": "novox/mesh-sdk", "commit": commitSHA, "base": baseSHA})
lines := strings.Join(f.lines(), "\n")
if !strings.Contains(lines, "-C "+mirror+" fetch --no-tags origin "+baseSHA) {
t.Errorf("the mirror was not brought to the base:\n%s", lines)
}
if !strings.Contains(lines, "fetch --no-tags --unshallow "+mirror+" "+baseSHA) || strings.Contains(lines, "--unshallow origin") {
t.Errorf("the preview fetched the trunk's history from the forge:\n%s", lines)
}
}
// 1921/S6: the siblings are placed side by side: each sibling's fetch has started before any of them ends.
func TestTheSiblingsArePlacedAtOnce(t *testing.T) {
var mu sync.Mutex
started := map[string]bool{}
both := make(chan struct{})
f := &fakeStep{gomods: "go.mod\n"}
f.rule = func(line string) (forgekey.Ran, bool) {
if !strings.Contains(line, "/beside/") || !strings.Contains(line, " fetch --no-tags --depth 1 ") {
return forgekey.Ran{}, false
}
mu.Lock()
for _, name := range []string{"mesh-catalog", "mesh-host"} {
if strings.Contains(line, "/beside/"+name) {
started[name] = true
}
}
if len(started) == 2 {
select {
case <-both:
default:
close(both)
}
}
mu.Unlock()
select {
case <-both:
case <-time.After(2 * time.Second):
return forgekey.Ran{Status: 1, Stderr: "the other sibling's fetch never started"}, true
}
return forgekey.Ran{}, false
}
m, states := fetchModule(t, f)
res := fetchOnce(t, m, states, map[string]any{"repository": "novox/mesh-controller", "commit": commitSHA, "beside": []any{
map[string]any{"repository": "novox/mesh-host", "commit": commitSHA},
map[string]any{"repository": "novox/mesh-catalog", "commit": commitSHA},
}})
if len(res.Beside) != 2 || res.Beside[0].Dir != "mesh-catalog" || res.Beside[1].Dir != "mesh-host" {
t.Errorf("beside = %+v, want both, in name order", res.Beside)
}
}
@@ -29,6 +29,9 @@ type Module struct {
Toolchains, Steps buildstep.Put
// ReadSteps and ReadToolchainsState read those two states back for the read tools (issue 1562).
ReadSteps, ReadToolchainsState StateRead
// Pauses is the Put of the pauses state, which keeps the node's pause key without an age; ReadPauses reads it back.
Pauses buildstep.Put
ReadPauses StateRead
// Emit tells the bus an event (stdio.Emit); nil emits nothing. The guard emits toolchain.reported after each write of
// the toolchains key, so the indexer, which holds no timer, learns that the bucket exists (issue 1444).
Emit func(eventType string, body any) error
@@ -0,0 +1,125 @@
package main
import (
"os"
"strings"
"testing"
)
// A failure that is not one test's assertion carries its real cause (issue 1901, its third meeting: a try of
// mesh-controller kept only "PASS" for a package that stopped half-way). Each fixture is the real `go test -json` of a
// throwaway package (testdata/captured/README).
func TestAFailureThatIsNotAnAssertionCarriesItsRealCause(t *testing.T) { // 1901/AC4
cases := []struct {
fixture string
test string // the test the failure names; "" for none
cause []string // what the cause holds
}{
{"panicking", "TestPanics", []string{"panic: assignment to entry in nil map"}},
{"timeout", "TestWaitsForever", []string{"panic: test timed out after 2s", "TestWaitsForever"}},
{"deeptimeout", "TestHoldsTheSeat, TestWaitsForTheBus", []string{"panic: test timed out after 2s", "TestHoldsTheSeat", "TestWaitsForTheBus"}},
{"buildfail", "", []string{`cannot use "seat" (untyped string constant) as int value`}},
{"exiting", "TestExits", []string{"exited", "TestExits"}},
{"killed", "TestKilled", []string{"signal: killed"}},
{"passthenfail", "", []string{"panic: the bus closed after the tests: nats: connection closed"}},
{"mainexit", "", []string{"passed", "exited"}},
{"race", "TestRacesOnTheSeat", []string{"DATA RACE", "Write at", "example.com/race.TestRacesOnTheSeat.func1()", "p_test.go:13", "created at example.com/race.TestRacesOnTheSeat()"}},
{"racemain", "", []string{"DATA RACE", "Write at", "example.com/racemain.TestMain.func1()", "p_test.go:15", "created at example.com/racemain.TestMain()"}},
}
for _, c := range cases {
t.Run(c.fixture, func(t *testing.T) {
raw, err := os.ReadFile("testdata/captured/go-test-json-pkg-" + c.fixture + ".jsonl")
if err != nil {
t.Fatal(err)
}
res := &TestResult{}
rd := newReader(res, ".")
rd.read(string(raw))
rd.finish()
if len(res.Failed) != 1 {
t.Fatalf("failures = %+v, want one", res.Failed)
}
f := res.Failed[0]
if f.Test != c.test {
t.Errorf("1901/AC4: the failure names test %q, want %q", f.Test, c.test)
}
for _, want := range c.cause {
if !strings.Contains(f.Cause, want) {
t.Errorf("1901/AC4: cause = %q, want it to hold %q", f.Cause, want)
}
}
if len(f.Cause) > outputLineBytes+3 {
t.Errorf("1901/AC4: cause holds %d bytes, over the line bound %d", len(f.Cause), outputLineBytes)
}
// Never a "PASS" or "ok" line as a failure's words, and the first line of the output, all an older
// checker reads, is the cause.
for _, l := range strings.Split(f.Output, "\n") {
if l = strings.TrimSpace(l); l == "PASS" || strings.HasPrefix(l, "ok ") || strings.HasPrefix(l, "ok\t") {
t.Errorf("1901/AC4: the output keeps the line %q: %q", l, f.Output)
}
}
if first := strings.SplitN(f.Output, "\n", 2)[0]; strings.TrimSpace(first) != strings.TrimSpace(f.Cause) {
t.Errorf("1901/AC4: the output starts with %q, want the cause %q", first, f.Cause)
}
})
}
}
// A package that passes, and a test that fails on its own assertion, keep what they had: no cause is made up for them.
func TestAnAssertionKeepsItsOwnCause(t *testing.T) { // 1901/AC4
raw, err := os.ReadFile("testdata/captured/go-test-json-log-then-fatal.jsonl")
if err != nil {
t.Fatal(err)
}
res := &TestResult{}
rd := newReader(res, ".")
rd.read(string(raw))
rd.finish()
if len(res.Failed) != 1 || !strings.HasSuffix(res.Failed[0].Cause, "3 subscription(s) derive to a subject no emitter owns:") {
t.Errorf("1901/AC4: failures = %+v", res.Failed)
}
}
// go prints the runtime's lines ("panic:", "fatal error:", "signal:", "WARNING: DATA RACE") at column 0: a test's own
// log line that holds the same words is indented, and is not the cause (review round 1). The fixture is the deeptimeout
// capture with two log lines of the first running test added: a t.Logf and its indented continuation saying "panic:".
func TestALoggedPanicIsNotTheCause(t *testing.T) { // 1901/AC4
raw, err := os.ReadFile("testdata/captured/go-test-json-pkg-deeptimeout.jsonl")
if err != nil {
t.Fatal(err)
}
lines := strings.SplitAfter(string(raw), "\n")
added := `{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" p_test.go:9: the child said:\n"}` + "\n" +
`{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" panic: child retried and recovered\n"}` + "\n" +
`{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" WARNING: DATA RACE\n"}` + "\n"
out := strings.Join(lines[:10], "") + added + strings.Join(lines[10:], "")
res := &TestResult{}
rd := newReader(res, ".")
rd.read(out)
rd.finish()
if len(res.Failed) != 1 || !strings.HasPrefix(res.Failed[0].Cause, "panic: test timed out after 2s") {
t.Errorf("1901/AC4: failures = %+v, want the timeout's panic as the cause", res.Failed)
}
}
// A package counts its own skipped tests, so the checker withholds a kept verdict from that package alone (issue 890).
func TestEachPackageCountsItsOwnSkippedTests(t *testing.T) {
log := `{"Action":"run","Package":"m/a","Test":"TestA"}
{"Action":"skip","Package":"m/a","Test":"TestA","Elapsed":0}
{"Action":"run","Package":"m/b","Test":"TestB"}
{"Action":"pass","Package":"m/b","Test":"TestB","Elapsed":0}
{"Action":"pass","Package":"m/a","Elapsed":0}
{"Action":"pass","Package":"m/b","Elapsed":0}
`
res := &TestResult{}
rd := newReader(res, ".")
rd.read(log)
rd.finish()
got := map[string]int{}
for _, p := range res.Packages {
got[p.Package] = p.Skipped
}
if res.Skipped != 1 || got["m/a"] != 1 || got["m/b"] != 0 {
t.Errorf("skipped = %d, per package %v, want 1 with m/a 1 and m/b 0", res.Skipped, got)
}
}
@@ -13,7 +13,6 @@ import (
"path/filepath"
"sort"
"strings"
"sync"
"git.novox.be/novox/mesh-sdk/go/buildstep"
)
@@ -35,9 +34,8 @@ func (m *Module) remote(repository string) string {
}
// merge makes the merge preview of the head onto base and writes the answer into res.
// opts are the fetch's options, with the mirror when it holds base.
func (m *Module) merge(ctx context.Context, res *FetchResult, tree, remote, base, commit string, opts []buildstep.Option) error {
mr, err := buildstep.MergeInto(ctx, m.Run, tree, remote, base, commit, opts...)
func (m *Module) merge(ctx context.Context, res *FetchResult, tree, remote, base, commit string) error {
mr, err := buildstep.MergeInto(ctx, m.Run, tree, remote, base, commit, m.FetchOptions...)
if err != nil {
return err
}
@@ -91,22 +89,23 @@ func (m *Module) besideDir(workspace string) string {
// placeSiblings fetches each sibling into beside/<name> of the workspace root, as a plain directory. A symlink anywhere
// in a placed tree fails the step naming it: the mesh creates none, and a test must not be led out of its sibling.
func (m *Module) placeSiblings(ctx context.Context, root string, siblings []siblingArg) ([]Sibling, error) {
// The siblings are placed side by side (issue 1921, try speed S6); the first failure in the order given is the step's.
placed := make([]Sibling, len(siblings))
errs := make([]error, len(siblings))
var wg sync.WaitGroup
for i, s := range siblings {
wg.Add(1)
go func() {
defer wg.Done()
placed[i], errs[i] = m.placeSibling(ctx, root, s)
}()
}
wg.Wait()
for _, err := range errs {
if err != nil {
var placed []Sibling
for _, s := range siblings {
name := s.repository[strings.Index(s.repository, "/")+1:]
dir := filepath.Join(root, "beside", name)
if err := os.MkdirAll(dir, 0o700); err != nil {
return nil, err
}
if err := buildstep.FetchCommit(ctx, m.Run, dir, m.remote(s.repository), s.commit, s.ref, m.fetchOptions(ctx, s.repository, s.commit, s.ref)...); err != nil {
return nil, fmt.Errorf("beside %s: %w", s.repository, err)
}
if err := m.dissociate(ctx, dir); err != nil {
return nil, fmt.Errorf("beside %s: %w", s.repository, err)
}
if link := firstSymlink(dir); link != "" {
return nil, fmt.Errorf("beside %s holds a symlink, %s", s.repository, link)
}
placed = append(placed, Sibling{Repository: s.repository, Commit: s.commit, Dir: name})
}
sort.Slice(placed, func(i, j int) bool { return placed[i].Dir < placed[j].Dir })
if len(placed) > 0 {
@@ -122,25 +121,6 @@ func (m *Module) placeSiblings(ctx context.Context, root string, siblings []sibl
return placed, nil
}
// placeSibling fetches one sibling into beside/<name> of the workspace root.
func (m *Module) placeSibling(ctx context.Context, root string, s siblingArg) (Sibling, error) {
name := s.repository[strings.Index(s.repository, "/")+1:]
dir := filepath.Join(root, "beside", name)
if err := os.MkdirAll(dir, 0o700); err != nil {
return Sibling{}, err
}
if err := buildstep.FetchCommit(ctx, m.Run, dir, m.remote(s.repository), s.commit, s.ref, m.fetchOptions(ctx, s.repository, s.commit, s.ref)...); err != nil {
return Sibling{}, fmt.Errorf("beside %s: %w", s.repository, err)
}
if err := m.dissociate(ctx, dir); err != nil {
return Sibling{}, fmt.Errorf("beside %s: %w", s.repository, err)
}
if link := firstSymlink(dir); link != "" {
return Sibling{}, fmt.Errorf("beside %s holds a symlink, %s", s.repository, link)
}
return Sibling{Repository: s.repository, Commit: s.commit, Dir: name}, nil
}
// besideRecord is the file in a workspace's root naming the siblings the fetch placed beside it.
const besideRecord = "beside.json"
@@ -85,8 +85,7 @@ func TestFetchWithABaseMergesAndAnswersThePreview(t *testing.T) {
t.Errorf("trees = %+v", res.Trees)
}
lines := strings.Join(f.lines(), "\n")
mirror := filepath.Join(m.Cache, "mirror", "novox", "mesh-sdk.git")
for _, want := range []string{"fetch --no-tags --unshallow " + mirror + " " + baseSHA, "checkout -q --detach " + baseSHA,
for _, want := range []string{"fetch --no-tags --unshallow origin " + baseSHA, "checkout -q --detach " + baseSHA,
"core.hooksPath=/dev/null", "merge --no-ff --no-edit --no-verify"} {
if !strings.Contains(lines, want) {
t.Errorf("git was not asked %q:\n%s", want, lines)
@@ -291,8 +290,8 @@ func TestTheIntegrationLevelIsGivenTheBesideDirectoryOnlyWhenSiblingsWerePlaced(
}
tested := 0
for _, c := range f.calls {
if !strings.HasPrefix(c.line, "go ") || !strings.Contains(c.line, "-tags integration") || strings.Contains(c.line, "-exec true") {
continue // the setup's compile runs no test binary, so it is given nothing a test reads (issue 1921, S5)
if !strings.HasPrefix(c.line, "go ") || !strings.Contains(c.line, "-tags integration") {
continue
}
tested++
got := ""
+97 -22
View File
@@ -7,9 +7,12 @@ package main
import (
"encoding/json"
"fmt"
"log"
"sort"
"strconv"
"strings"
"sync"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
@@ -39,6 +42,53 @@ func readOf(s stdio.KeptState) StateRead {
}}
}
// MaxStateReads is the most values one read tool reads from a state in a call, and stateReadParallel how many of them
// are in flight at once. Every value is a round trip from the tool through the runtime to the bus, so a call that read
// every key of a large state one after the other outlasted the call's 30 s (issue 1827: 1164 keys, 20 MB).
const (
MaxStateReads = 1000
stateReadParallel = 16
)
// readValues reads the values of keys, stateReadParallel at a time, in the order of keys (nil for a key with none). It
// stops starting reads after the first error and returns it.
func readValues(keys []string, get func(string) (json.RawMessage, error)) ([]json.RawMessage, error) {
out := make([]json.RawMessage, len(keys))
var (
wg sync.WaitGroup
mu sync.Mutex
fail error
next = make(chan int)
)
for w := 0; w < stateReadParallel && w < len(keys); w++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := range next {
raw, err := get(keys[i])
mu.Lock()
if err != nil && fail == nil {
fail = err
}
mu.Unlock()
out[i] = raw
}
}()
}
for i := range keys {
mu.Lock()
stop := fail != nil
mu.Unlock()
if stop {
break
}
next <- i
}
close(next)
wg.Wait()
return out, fail
}
// stepKey is the key of a step of this node: the id alone, or the whole key <node>.<id> as a step tool answered it.
func (m *Module) stepKey(id string) (string, error) {
id = strings.TrimSpace(id)
@@ -80,27 +130,41 @@ type StepLine struct {
Reason string `json:"reason,omitempty"`
}
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero).
func (m *Module) RecentSteps(limit int) ([]StepLine, error) {
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero). Only
// the node's own keys (<node>.) are listed and the newest max(limit*4, 100) of them read (at most MaxStateReads): a step's
// id is base-36 milliseconds and random digits, so its string order is its start order. cut says older steps were not
// read. The values still order the lines, since an ask step's id carries the ask time.
func (m *Module) RecentSteps(limit int) (lines []StepLine, cut bool, err error) {
if limit <= 0 {
limit = 10
}
if m.ReadSteps.Keys == nil || m.ReadSteps.Get == nil {
return nil, fmt.Errorf("the steps state cannot be read here")
return nil, false, fmt.Errorf("the steps state cannot be read here")
}
start := time.Now()
keys, err := m.ReadSteps.Keys()
if err != nil {
return nil, err
return nil, false, err
}
own := keys[:0:0]
for _, k := range keys {
if strings.HasPrefix(k, m.Node+".") {
if node, _, err := delivery.ParseStepKey(k); err == nil && node == m.Node {
own = append(own, k)
}
}
}
sort.Sort(sort.Reverse(sort.StringSlice(own)))
if n := min(max(limit*4, 100), MaxStateReads); len(own) > n {
own, cut = own[:n], true
}
raws, err := readValues(own, m.ReadSteps.Get)
log.Printf("steps read: %d of %d keys listed read in %v (cut %v)", len(own), len(keys), time.Since(start).Round(time.Millisecond), cut)
if err != nil {
return nil, cut, err
}
var steps []delivery.Step
for _, k := range keys {
if node, _, err := delivery.ParseStepKey(k); err != nil || node != m.Node {
continue
}
raw, err := m.ReadSteps.Get(k)
if err != nil {
return nil, err
}
for _, raw := range raws {
if len(raw) == 0 {
continue
}
@@ -122,7 +186,7 @@ func (m *Module) RecentSteps(limit int) ([]StepLine, error) {
}
out = append(out, l)
}
return out, nil
return out, cut, nil
}
// ReadToolchains is every key of the toolchains state (<node>.<language>) with its stored value.
@@ -130,24 +194,30 @@ func (m *Module) ReadToolchains() (map[string]delivery.Toolchain, error) {
if m.ReadToolchainsState.Keys == nil || m.ReadToolchainsState.Get == nil {
return nil, fmt.Errorf("the toolchains state cannot be read here")
}
start := time.Now()
keys, err := m.ReadToolchainsState.Keys()
if err != nil {
return nil, err
}
cut := len(keys) > MaxStateReads
if cut {
keys = keys[:MaxStateReads] // a node has a key per language: far fewer in practice
}
raws, err := readValues(keys, m.ReadToolchainsState.Get)
log.Printf("toolchains read: %d keys read in %v (cut %v: keys past %d were not read)", len(keys), time.Since(start).Round(time.Millisecond), cut, MaxStateReads)
if err != nil {
return nil, err
}
out := map[string]delivery.Toolchain{}
for _, k := range keys {
raw, err := m.ReadToolchainsState.Get(k)
if err != nil {
return nil, err
}
for i, raw := range raws {
if len(raw) == 0 {
continue
}
t, err := delivery.DecodeToolchain(raw)
if err != nil {
return nil, fmt.Errorf("%s: %w", k, err)
return nil, fmt.Errorf("%s: %w", keys[i], err)
}
out[k] = t
out[keys[i]] = t
}
return out, nil
}
@@ -171,8 +241,13 @@ func (m *Module) ReadTools() []stdio.Tool {
Input: map[string]any{"type": "object", "properties": map[string]any{
"limit": map[string]any{"type": "integer", "description": "how many, at most; 10 when left out"}}},
Run: func(args map[string]any) (any, error) {
steps, err := m.RecentSteps(intArg(args["limit"]))
return map[string]any{"node": m.Node, "steps": steps}, err
steps, cut, err := m.RecentSteps(intArg(args["limit"]))
ans := map[string]any{"node": m.Node, "steps": steps}
if cut {
ans["cut"] = true
ans["note"] = fmt.Sprintf("this node has more steps stored than one call reads; older steps were not read (ask for a larger limit, at most %d reads)", MaxStateReads)
}
return ans, err
}},
{Name: ToolchainsReadTool,
Description: "What the module's toolchains state holds: each key <node>.<language> with the version installed, the " +
@@ -3,8 +3,10 @@ package main
import (
"context"
"encoding/json"
"fmt"
"slices"
"strings"
"sync/atomic"
"testing"
"time"
@@ -145,3 +147,62 @@ func TestReadToolsAreInTheManifestAndNeedNoToolchain(t *testing.T) {
t.Errorf("the read refused: %v", err)
}
}
// 1827: on the live shape (4 nodes, 600 steps each; a steps state of 1164 keys / 20 MB timed out at 30 s) the steps tool
// lists, then reads only this node's keys, in parallel, at most MaxStateReads of them, and says when it cut.
func TestRecentStepsReadsOnlyTheNodesKeysInParallelAndBounded(t *testing.T) {
t0 := time.Date(2026, 10, 25, 9, 0, 0, 0, time.UTC)
for _, row := range []struct {
perNode, limit int
reads int64
steps int
cut bool
}{
{20, 0, 20, 10, false},
{600, 0, 100, 10, true},
{1500, 300, MaxStateReads, 300, true},
} {
var reads, inFlight, maxFlight atomic.Int64
stored := map[string][]byte{}
for _, n := range []string{"ace", "g14", "novox", "shanks"} {
for i := 0; i < row.perNode; i++ {
b, _ := delivery.EncodeStep(delivery.Step{ID: fmt.Sprintf("s%04d", i), Tool: "t", Node: n, State: delivery.StepPassed, Started: t0.Add(time.Duration(i) * time.Second)})
stored[n+".s"+fmt.Sprintf("%04d", i)] = b
}
}
m := &Module{Node: "shanks"}
m.ReadSteps = StateRead{
Keys: func() ([]string, error) {
var k []string
for key := range stored {
k = append(k, key)
}
return k, nil
},
Get: func(key string) (json.RawMessage, error) {
reads.Add(1)
now := inFlight.Add(1)
for o := maxFlight.Load(); now > o && !maxFlight.CompareAndSwap(o, now); o = maxFlight.Load() {
}
time.Sleep(time.Millisecond)
inFlight.Add(-1)
if !strings.HasPrefix(key, "shanks.") {
t.Errorf("read %s, a key of another node", key)
}
return stored[key], nil
},
}
start := time.Now()
lines, cut, err := m.RecentSteps(row.limit)
if err != nil || len(lines) != row.steps || cut != row.cut || reads.Load() != row.reads {
t.Errorf("%d a node: %d steps, cut %v, %d reads, err %v; want %d, %v, %d", row.perNode, len(lines), cut, reads.Load(), err, row.steps, row.cut, row.reads)
}
if row.cut && len(lines) > 0 && lines[0].ID != fmt.Sprintf("s%04d", row.perNode-1) {
t.Errorf("%d a node: newest answered %s", row.perNode, lines[0].ID)
}
if maxFlight.Load() < 2 {
t.Errorf("%d a node: values read one at a time", row.perNode)
}
t.Logf("%d a node: %d reads in %v (one after the other: %v)", row.perNode, reads.Load(), time.Since(start), time.Duration(reads.Load())*time.Millisecond)
}
}
@@ -24,7 +24,9 @@ func threeModules(t *testing.T, f *integFake) (*Module, *keptStates) {
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
writeTaggedTest(t, dir, "fixture/"+d)
if err := os.WriteFile(filepath.Join(dir, "go.mod"), []byte("module fixture/"+d+"\n\ngo 1.25.0\n"), 0o644); err != nil {
t.Fatal(err)
}
}
return m, states
}
@@ -70,7 +72,7 @@ func TestOnlyTheSelectedDirectoryCompilesAndRuns(t *testing.T) {
if !strings.HasPrefix(l, "go -C "+filepath.Join(tree, "b")+" test ") {
t.Errorf("go test ran outside the selected directory: %s", l)
}
if strings.Contains(l, "-exec true") {
if strings.Contains(l, "-run ^$") {
compiled++
} else {
ran++
@@ -89,7 +91,7 @@ func TestWithoutASelectionEveryDirectoryCompilesAndRuns(t *testing.T) {
compiled, ran := 0, 0
for _, l := range strings.Split(f.lines(), "\n") {
if strings.HasPrefix(l, "go ") && strings.Contains(l, " test ") {
if strings.Contains(l, "-exec true") {
if strings.Contains(l, "-run ^$") {
compiled++
} else {
ran++
@@ -1,81 +0,0 @@
package main
// The bound on the processes one ask starts (issue 1921, try speed S10; ADR 0541): the levels of an ask run side by side,
// the unit level in this process and the integration level in a shut-off process of its own, and they share one bound,
// the processors divided by the node's build capacity. The bound lives in the ask's workspace as lock files, one per
// slot, so both processes count against it.
import (
"context"
"fmt"
"os"
"path/filepath"
"runtime"
"strconv"
"sync/atomic"
"syscall"
)
// slotsDir is the directory in a workspace's root holding its slots.
const slotsDir = "slots"
// askDirs are the directories an ask keeps in its workspace's root beside its tree, gone with the workspace: its cache
// layer and its bound on processes.
var askDirs = []string{cacheLayerDir, slotsDir}
// numCPU is the node's processor count.
func numCPU() int { return runtime.NumCPU() }
// slotPool is the bound of one workspace: n slots, each a lock file under dir.
type slotPool struct {
dir string
n int
next *atomic.Uint32
}
// slotTurns spreads the waits of the commands that find every slot taken over the slots.
var slotTurns atomic.Uint32
// slots is the bound of the workspace's processes, m.parallel() slots.
func (m *Module) slots(workspace string) slotPool {
return slotPool{dir: filepath.Join(m.Work, workspace, slotsDir), n: m.parallel(), next: &slotTurns}
}
// acquire takes a slot of the pool, waiting for one when every slot is taken, and answers what frees it. A free slot is
// taken at once; when none is, the wait is on one slot, taken in turn, until its holder frees it.
func (p slotPool) acquire(ctx context.Context) (func(), error) {
if err := os.MkdirAll(p.dir, 0o700); err != nil {
return nil, fmt.Errorf("the workspace's bound on processes: %w", err)
}
take := func(i int, how int) (func(), bool, error) {
f, err := os.OpenFile(filepath.Join(p.dir, "slot-"+strconv.Itoa(i)), os.O_CREATE|os.O_RDWR, 0o600)
if err != nil {
return nil, false, err
}
if err := syscall.Flock(int(f.Fd()), how); err != nil {
f.Close()
if err == syscall.EWOULDBLOCK {
return nil, false, nil
}
return nil, false, err
}
return func() { syscall.Flock(int(f.Fd()), syscall.LOCK_UN); f.Close() }, true, nil
}
for i := 0; i < p.n; i++ {
release, ok, err := take(i, syscall.LOCK_EX|syscall.LOCK_NB)
if err != nil {
return nil, fmt.Errorf("the workspace's bound on processes: %w", err)
}
if ok {
return release, nil
}
}
if err := ctx.Err(); err != nil {
return nil, err
}
release, _, err := take(int(p.next.Add(1))%p.n, syscall.LOCK_EX)
if err != nil {
return nil, fmt.Errorf("the workspace's bound on processes: %w", err)
}
return release, nil
}
@@ -1,61 +0,0 @@
package main
import (
"context"
"strings"
"sync/atomic"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
// 1921/S10 (ADR 0541): the levels of one ask, running side by side, share one bound on the processes they start: the
// bound is the workspace's, so a slot another level of the ask holds is one this level waits for.
func TestTheLevelsOfAnAskShareOneBoundOnProcesses(t *testing.T) {
m, states, ws, _ := workspaceFor(t, "failing")
m.Parallel = 1
held, err := m.slots(ws).acquire(context.Background()) // the other level holds the one slot
if err != nil {
t.Fatal(err)
}
var ran atomic.Int32
inner := m.Run
m.Run = func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
if strings.Contains(strings.Join(args, " "), " go ") || name == "go" {
ran.Add(1)
}
return inner(ctx, env, name, args...)
}
done := make(chan delivery.Step, 1)
go func() { _, st := runUnit(t, m, states, ws); done <- st }()
time.Sleep(150 * time.Millisecond)
if n := ran.Load(); n != 0 {
t.Errorf("%d go commands ran while the ask's one slot was held by its other level", n)
}
held()
select {
case st := <-done:
if st.State != delivery.StepPassed {
t.Errorf("the step ended %s: %s", st.State, st.Reason)
}
case <-time.After(3 * time.Second):
t.Fatal("the level never ran after the slot was freed")
}
if ran.Load() == 0 {
t.Error("no go command ran")
}
}
// 1921/S10 (ADR 0541): the bound is the processors divided by the node's build capacity, not each level half of them.
func TestTheBoundIsTheProcessorsDividedByTheBuildCapacity(t *testing.T) {
m := &Module{Capacity: func() int { return 3 }}
if got, want := m.parallel(), max(1, numCPU()/3); got != want {
t.Errorf("parallel = %d, want %d", got, want)
}
m = &Module{}
if got, want := m.parallel(), max(1, numCPU()/2); got != want {
t.Errorf("with no capacity known parallel = %d, want %d", got, want)
}
}
@@ -1,161 +0,0 @@
package main
// The integration level runs the tests its tags add, not the unit tests again (issue 1921, try speed S7): `go test
// -tags integration` without -run would run every untagged test too, and the unit level ran those already. Which tests a
// tag adds is read from the source: a test function in a _test.go file whose //go:build line holds with the level's
// tags and does not hold without them.
import (
"go/ast"
"go/build/constraint"
"go/parser"
"go/token"
"os"
"path/filepath"
"runtime"
"sort"
"strings"
"unicode"
"unicode/utf8"
)
// taggedRun is what the integration level runs in one go.mod directory: the packages holding a test the tags add, by
// import path, and the -run pattern naming those tests.
type taggedRun struct {
packages []string
pattern string
}
// taggedTests reads, for each go.mod directory, the tests that tags add. selected are the packages the level was asked
// to run (import paths, as build_go_digest keys them); none, or a relative pattern, is every package. A directory with no
// test the tags add is left out of the answer.
func taggedTests(tree string, dirs, selected, tags []string) (map[string]taggedRun, error) {
want := map[string]bool{}
for _, p := range selected {
if strings.HasPrefix(p, "./") || p == "." {
want = nil
break
}
want[p] = true
}
if len(selected) == 0 {
want = nil
}
with := map[string]bool{}
for _, t := range tags {
with[t] = true
}
out := map[string]taggedRun{}
for _, d := range dirs {
raw, _ := os.ReadFile(filepath.Join(tree, d, "go.mod"))
mp := modulePath(string(raw))
base := filepath.Join(tree, d)
pkgs := map[string]bool{}
names := map[string]bool{}
err := lintable(base, func(path string) error {
if !strings.HasSuffix(path, "_test.go") {
return nil
}
src, err := os.ReadFile(path)
if err != nil {
return err
}
expr := buildLine(src)
if expr == nil || !expr.Eval(func(tag string) bool { return with[tag] || platformTag(tag) }) || expr.Eval(platformTag) {
return nil // no constraint, or one the tags do not change: the unit level's
}
rel, _ := filepath.Rel(base, filepath.Dir(path))
pkg := mp
if rel != "." {
pkg = mp + "/" + filepath.ToSlash(rel)
}
if want != nil && !want[pkg] {
return nil
}
found := testNames(path, src)
for _, n := range found {
names[n] = true
}
if len(found) > 0 {
pkgs[pkg] = true
}
return nil
})
if err != nil {
return nil, err
}
if len(pkgs) == 0 {
continue
}
run := taggedRun{packages: sortedKeys(pkgs)}
run.pattern = "^(" + strings.Join(sortedKeys(names), "|") + ")$"
out[d] = run
}
return out, nil
}
func sortedKeys(m map[string]bool) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
// buildLine is a file's //go:build constraint, nil when it has none.
func buildLine(src []byte) constraint.Expr {
for _, line := range strings.Split(string(src), "\n") {
line = strings.TrimSpace(line)
if strings.HasPrefix(line, "package ") {
break
}
if constraint.IsGoBuild(line) {
if e, err := constraint.Parse(line); err == nil {
return e
}
}
}
return nil
}
// platformTag is how a tag other than the level's evaluates, the same with the level's tags and without them: the
// machine's platform, unix, cgo and every Go version hold; any other tag does not.
func platformTag(tag string) bool {
switch tag {
case runtime.GOOS, runtime.GOARCH, "unix", "cgo", "gc":
return true
}
return strings.HasPrefix(tag, "go1.")
}
// testNames are the top-level test functions of a test file: func TestXxx, as go test runs them, TestMain left out.
func testNames(path string, src []byte) []string {
f, err := parser.ParseFile(token.NewFileSet(), path, src, parser.SkipObjectResolution)
if err != nil {
return nil // a file that does not parse fails the compile, which the run reports
}
var out []string
for _, decl := range f.Decls {
fn, ok := decl.(*ast.FuncDecl)
if !ok || fn.Recv != nil || fn.Name.Name == "TestMain" || !isTestName(fn.Name.Name) {
continue
}
out = append(out, fn.Name.Name)
}
return out
}
// isTestName says whether go test runs a function of this name as a test: Test, then nothing or a rune that is not
// lower case.
func isTestName(name string) bool {
rest, ok := strings.CutPrefix(name, "Test")
if !ok {
return false
}
if rest == "" {
return true
}
r, _ := utf8.DecodeRuneInString(rest)
return !unicode.IsLower(r)
}
+25 -68
View File
@@ -10,6 +10,7 @@ import (
"os"
"path/filepath"
"regexp"
"runtime"
"strings"
"sync"
"time"
@@ -170,9 +171,6 @@ type levelOpts struct {
// parallelism, where the unit level's is the slowest package's.
setup time.Duration
wall bool
// runByDir is the -run pattern of each go.mod directory, over run: the tests the integration level's tags add there
// (issue 1921, try speed S7).
runByDir map[string]string
// compose is a state of the level to put in the result (the store and the bus it ran against).
store, bus string
}
@@ -207,56 +205,35 @@ func (m *Module) runLevel(ctx context.Context, tree string, o levelOpts) (*TestR
todo = append(todo, d)
}
}
// The level's commands share one bound on processes with each other and with the ask's other level: the lint's and
// the tests' (issue 1921, try speed S4, S10).
slots := m.slots(o.workspace)
var lintFailed []Failure
var lintErr error
var lintDone chan struct{}
var crash any // a panic of a run or of the lint goes on in the caller, which cleans up after it
var crashMu sync.Mutex
if o.level == LevelUnit {
// gofmt and go vet of each directory the level runs, beside its tests (ADR 0529 §1; issue 1887 D7-go; issue 1921,
// try speed S4): a directory that fails them fails the level, and their time is the result's Lint.
lintDone = make(chan struct{})
go func() {
defer close(lintDone)
defer func() {
if p := recover(); p != nil {
crashMu.Lock()
crash = p
crashMu.Unlock()
}
}()
linted := m.clock()
// go vet compiles and runs nothing of the change, so it warms the shared cache for every later check, a pull
// request's head included (issue 1921, try speed S11; the operator's decision); the tests run beneath the
// ask's own layer.
compile, err := m.stepRunner(o.workspace, true)
if err != nil {
lintErr = err
return
}
lintFailed, lintErr = m.lintAll(ctx, compile, tree, todo, func(d string) []string {
return append(m.goEnvFor(filepath.Join(tree, d)), o.env...)
}, byDir, slots.acquire)
lintFailed = append(m.addedBinaries(ctx, m.Run, tree), lintFailed...) // git alone, no code of the change
res.Lint = m.clock().Sub(linted).Seconds()
}()
// gofmt and go vet of each directory the level runs come before its tests (ADR 0529 §1; issue 1887 D7-go): a
// directory that fails them fails the level at once, and their time is the result's Lint.
linted := m.clock()
failed, err := m.lintAll(ctx, run, tree, todo, func(d string) []string {
return append(m.goEnvFor(filepath.Join(tree, d)), o.env...)
})
if err != nil {
return nil, err
}
res.Lint = m.clock().Sub(linted).Seconds()
if len(failed) > 0 {
res.Failed = failed
res.judge(o.bounds)
return res, nil
}
began = m.clock() // the level's running time is its tests'
}
runs := make([]dirRun, len(todo))
slots := make(chan struct{}, m.parallel())
var wg sync.WaitGroup
var slotErr error
var crash any // a panic of a run goes on in the caller, which cleans up after it
var crashMu sync.Mutex
for i, d := range todo {
release, err := slots.acquire(ctx)
if err != nil {
slotErr = err
break
}
wg.Add(1)
slots <- struct{}{}
go func() {
defer wg.Done()
defer release()
defer func() { <-slots }()
defer func() {
if p := recover(); p != nil {
crashMu.Lock()
@@ -268,9 +245,7 @@ func (m *Module) runLevel(ctx context.Context, tree string, o levelOpts) (*TestR
if len(o.tags) > 0 {
args = append(args, "-tags", strings.Join(o.tags, ","))
}
if r := o.runByDir[d]; r != "" {
args = append(args, "-run", r)
} else if o.run != "" {
if o.run != "" {
args = append(args, "-run", o.run)
}
args = append(args, byDir[d]...)
@@ -283,19 +258,9 @@ func (m *Module) runLevel(ctx context.Context, tree string, o levelOpts) (*TestR
}()
}
wg.Wait()
if lintDone != nil {
<-lintDone
}
if crash != nil {
panic(crash)
}
if lintErr != nil {
return nil, lintErr
}
if slotErr != nil {
return nil, slotErr
}
res.Failed = append(res.Failed, lintFailed...) // the lint's failures first, then the tests'
for i, d := range todo {
dr := runs[i]
if (dr.r.Status != 0 || dr.r.Err != "") && len(dr.rd.pkgs) == 0 {
@@ -321,20 +286,12 @@ func (m *Module) runLevel(ctx context.Context, tree string, o levelOpts) (*TestR
// runAnswer is what a step runner answers.
type runAnswer = forgekey.Ran
// parallel is how many processes the levels of one ask start at once, together (ADR 0541; issue 1921, try speed S10):
// the processors divided by the node's build capacity, half of them when the capacity is not known.
// parallel is how many directories a level runs at once.
func (m *Module) parallel() int {
if m.Parallel > 0 {
return m.Parallel
}
capacity := 0
if m.Capacity != nil {
capacity = m.Capacity()
}
if capacity <= 0 {
capacity = 2
}
return max(1, numCPU()/capacity)
return max(1, runtime.NumCPU()/2)
}
// packagesByDir says which patterns run in which go.mod directory. A relative pattern (./...) runs in every directory, as
@@ -66,9 +66,7 @@ func TestUnitLevelRunsTheRealGoOnTheFixtureRepositories(t *testing.T) {
"tagged": {passed: true, noTests: true},
"empty": {passed: true, noTests: true},
"slow": {passed: true},
// go vet, the lint step beside the tests (ADR 0542 §2), names the directory and the compiler's line; the test run
// of the package that does not compile fails beside it
"broken": {failed: "."},
"broken": {failed: "."}, // go vet, before the tests since R4g (ADR 0529 §1), names the directory and the compiler's line
}
for name, w := range want {
home := t.TempDir()
@@ -86,16 +84,14 @@ func TestUnitLevelRunsTheRealGoOnTheFixtureRepositories(t *testing.T) {
if err != nil {
t.Fatalf("%s: %v", name, err)
}
lintFirst := w.failed != "." || (len(res.Failed) >= 1 && res.Failed[0].Step == StepLint)
if res.Passed != w.passed || res.NoTests != w.noTests || !lintFirst ||
(w.failed != "" && (len(res.Failed) == 0 || res.Failed[0].Package != w.failed)) {
if res.Passed != w.passed || res.NoTests != w.noTests || (w.failed != "" && (len(res.Failed) != 1 || res.Failed[0].Package != w.failed)) {
t.Errorf("%s: result = %+v", name, res)
}
}
}
// 1887/D7-go against the real go: a tree holding an unformatted file fails the check naming the file, as the lint step's
// failure (ADR 0542 §2), and the level's result carries the lint's time.
// 1887/D7-go against the real go: a tree holding an unformatted file fails the unit level naming the file, before any
// test runs, and the level's result carries the lint's time.
func TestAnUnformattedFileFailsTheRealUnitLevel(t *testing.T) {
needTools(t, "the unit level's gofmt and vet", "go", "git", "bwrap")
home := t.TempDir()
@@ -115,9 +111,8 @@ func TestAnUnformattedFileFailsTheRealUnitLevel(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if res.Passed || len(res.Failed) == 0 || res.Failed[0].Test != "gofmt" || res.Failed[0].Step != StepLint ||
!strings.Contains(res.Failed[0].Output, "ugly.go") || res.Lint <= 0 {
t.Fatalf("result = %+v, want the lint step failed by gofmt naming ugly.go, with its time", res)
if res.Passed || len(res.Failed) == 0 || res.Failed[0].Test != "gofmt" || !strings.Contains(res.Failed[0].Output, "ugly.go") || res.Tests != 0 {
t.Fatalf("result = %+v, want the level failed by gofmt naming ugly.go and no test run", res)
}
}
@@ -161,8 +156,9 @@ func TestIntegrationLevelRunsAgainstARealStoreAndBusAndLeavesNothing(t *testing.
if !res.Passed || res.Tests != 1 || res.Setup <= 0 || res.Setup > IntegrationSetup.Seconds() {
t.Errorf("result = %+v", res)
}
if left := workspaceLeft(filepath.Join(work, "ws1")); len(left) != 1 || left[0] != "tree" {
t.Errorf("the workspace holds %v besides the ask's own directories, want only its tree: the run directory is gone", left)
entries, _ := os.ReadDir(filepath.Join(work, "ws1"))
if len(entries) != 1 || entries[0].Name() != "tree" {
t.Errorf("the workspace holds %v, want only its tree: the run directory is gone", entries)
}
for deadline := time.Now().Add(5 * time.Second); len(processesNamed(work)) > 0 && time.Now().Before(deadline); time.Sleep(100 * time.Millisecond) {
}
@@ -6,7 +6,6 @@ import (
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
@@ -26,11 +25,8 @@ func capturedRunner(t *testing.T, captured string, calls *[]cmdRecord) forgekey.
if strings.Contains(string(raw), `"Action":"fail"`) {
status = 1 // go test exits 1 when a test or a build failed
}
var mu sync.Mutex // the lint and the tests of a level run at once (issue 1921, try speed S4)
return func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
mu.Lock()
*calls = append(*calls, cmdRecord{env: env, line: name + " " + strings.Join(args, " ")})
mu.Unlock()
switch {
case name == "git" && args[2] == "ls-files":
return forgekey.Ran{Stdout: "go.mod\n"}
@@ -25,3 +25,15 @@ The tar streams (the archives gunzipped) of the three archives above, captured t
gunzip and sha256) at origin/main 0205b8f7 on 2026-10-21: directory f8af1d94...d1b6, within 294e2bec...07fb, file
055cffc2...428e. The digests above hold the gzip bytes of go1.27.1. The first try of issue 1589 on the merge check's build node answered
other digests; the tests therefore compare the stream always and the digest only under go1.27.1.
go-test-json-pkg-*.jsonl are the standard output and error of `go test -json -count=1 ./...` (go1.27.3, the timeouts with
`-timeout 2s`), captured on 2026-10-25 for issue 1901 in throwaway one-file modules that each fail without a test's own
assertion: panicking (a nil map write in TestPanics), timeout (TestWaitsForever sleeps past the timeout), deeptimeout (two
parallel tests and sixty blocked goroutines past the timeout, so the goroutine dump pushes the panic line out of the last
4 KB), buildfail (a type error in a test file), exiting (os.Exit(3) in TestExits), killed (TestKilled sends itself SIGKILL),
passthenfail (TestMain panics in a goroutine after m.Run passed) and mainexit (TestMain exits 2 after m.Run passed: go test
prints "PASS" and then "FAIL <pkg>").
go-test-json-pkg-race.jsonl and go-test-json-pkg-racemain.jsonl are the same, with `go test -race -json -count=1 ./...`
(CGO_ENABLED=1, go1.27.3, 2026-10-25, issue 1901): race (TestRacesOnTheSeat writes a package variable from itself and a
goroutine) and racemain (TestMain does so after m.Run passed: go test prints "PASS", then the race report and "Found 1
data race(s)", then "FAIL <pkg>").
@@ -0,0 +1,6 @@
{"ImportPath":"example.com/buildfail [example.com/buildfail.test]","Action":"build-output","Output":"# example.com/buildfail [example.com/buildfail.test]\n"}
{"ImportPath":"example.com/buildfail [example.com/buildfail.test]","Action":"build-output","Output":"./p_test.go:5:40: cannot use \"seat\" (untyped string constant) as int value in variable declaration\n"}
{"ImportPath":"example.com/buildfail [example.com/buildfail.test]","Action":"build-fail"}
{"Time":"2026-10-25T15:34:33.261137737+01:00","Action":"start","Package":"example.com/buildfail"}
{"Time":"2026-10-25T15:34:33.261255019+01:00","Action":"output","Package":"example.com/buildfail","Output":"FAIL\texample.com/buildfail [build failed]\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.261268099+01:00","Action":"fail","Package":"example.com/buildfail","Elapsed":0,"FailedBuild":"example.com/buildfail [example.com/buildfail.test]"}
@@ -0,0 +1,417 @@
{"Time":"2026-10-25T15:35:18.991630993+01:00","Action":"start","Package":"example.com/deeptimeout"}
{"Time":"2026-10-25T15:35:18.993895188+01:00","Action":"run","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat"}
{"Time":"2026-10-25T15:35:18.993963909+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":"=== RUN TestHoldsTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.99399484+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":"=== PAUSE TestHoldsTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.9939982+01:00","Action":"pause","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat"}
{"Time":"2026-10-25T15:35:18.99400173+01:00","Action":"run","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus"}
{"Time":"2026-10-25T15:35:18.99400456+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"=== RUN TestWaitsForTheBus\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.99400838+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"=== PAUSE TestWaitsForTheBus\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.99401133+01:00","Action":"pause","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus"}
{"Time":"2026-10-25T15:35:18.99401441+01:00","Action":"cont","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat"}
{"Time":"2026-10-25T15:35:18.99401704+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":"=== CONT TestHoldsTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.994074141+01:00","Action":"cont","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus"}
{"Time":"2026-10-25T15:35:18.994102612+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"=== CONT TestWaitsForTheBus\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:21.00935195+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"panic: test timed out after 2s\n"}
{"Time":"2026-10-25T15:35:21.00938383+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\trunning tests:\n"}
{"Time":"2026-10-25T15:35:21.009394391+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t\tTestHoldsTheSeat (2s)\n"}
{"Time":"2026-10-25T15:35:21.009398021+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t\tTestWaitsForTheBus (2s)\n"}
{"Time":"2026-10-25T15:35:21.009401801+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009406481+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 81 [running]:\n"}
{"Time":"2026-10-25T15:35:21.009409911+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.(*M).startAlarm.func1()\n"}
{"Time":"2026-10-25T15:35:21.009413361+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2961 +0x34a\n"}
{"Time":"2026-10-25T15:35:21.009418341+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by time.goFunc\n"}
{"Time":"2026-10-25T15:35:21.009421781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/time/sleep.go:182 +0x2d\n"}
{"Time":"2026-10-25T15:35:21.009424991+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009428341+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 1 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009431721+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner.func1()\n"}
{"Time":"2026-10-25T15:35:21.009441811+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2144 +0x425\n"}
{"Time":"2026-10-25T15:35:21.009445511+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner(0x282e9702008, 0x282e9646bc8)\n"}
{"Time":"2026-10-25T15:35:21.009449641+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2201 +0x123\n"}
{"Time":"2026-10-25T15:35:21.009453471+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.runTests({0x55a45d, 0x17}, {0x55a45d, 0x17}, 0x282e95940f0, {0x6f19f0, 0x2, 0x2}, {0xc2afa2ca3b3c0c8d, 0x7737f168, ...})\n"}
{"Time":"2026-10-25T15:35:21.009462092+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2742 +0x510\n"}
{"Time":"2026-10-25T15:35:21.009467222+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.(*M).Run(0x282e96581e0)\n"}
{"Time":"2026-10-25T15:35:21.009470662+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2602 +0x6af\n"}
{"Time":"2026-10-25T15:35:21.009473692+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"main.main()\n"}
{"Time":"2026-10-25T15:35:21.009476942+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t_testmain.go:48 +0x9b\n"}
{"Time":"2026-10-25T15:35:21.009480092+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009483242+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 7 [sleep]:\n"}
{"Time":"2026-10-25T15:35:21.009486502+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"time.Sleep(0xdf8475800)\n"}
{"Time":"2026-10-25T15:35:21.009490682+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/runtime/time.go:368 +0x165\n"}
{"Time":"2026-10-25T15:35:21.009493812+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat(0x282e9702248?)\n"}
{"Time":"2026-10-25T15:35:21.009498842+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:14 +0x9e\n"}
{"Time":"2026-10-25T15:35:21.009502242+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner(0x282e9702248, 0x6d5e78)\n"}
{"Time":"2026-10-25T15:35:21.009505742+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:35:21.009509292+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:35:21.009512752+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:35:21.009515792+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009518623+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 8 [sleep]:\n"}
{"Time":"2026-10-25T15:35:21.009521303+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"time.Sleep(0xdf8475800)\n"}
{"Time":"2026-10-25T15:35:21.009524423+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/runtime/time.go:368 +0x165\n"}
{"Time":"2026-10-25T15:35:21.009531403+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestWaitsForTheBus(0x282e9702488?)\n"}
{"Time":"2026-10-25T15:35:21.009534943+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:19 +0x25\n"}
{"Time":"2026-10-25T15:35:21.009538403+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner(0x282e9702488, 0x6d5e80)\n"}
{"Time":"2026-10-25T15:35:21.009541953+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:35:21.009545233+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:35:21.009548323+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:35:21.009551343+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009554263+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 9 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009557523+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009561063+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009564513+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009571113+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009574233+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009577433+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 10 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009580823+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009584174+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009587504+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009590974+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009594004+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009597184+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 11 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009600394+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009603874+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009610954+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009614714+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009618104+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009621364+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 12 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009624704+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009628324+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009631954+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009635404+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009638734+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009641904+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 13 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009645325+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009648645+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009652065+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009655625+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009658885+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009663385+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 14 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009666735+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009670235+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009676385+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009680205+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009683315+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009686475+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 15 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009689785+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009697235+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009700795+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009704985+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009708085+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009711346+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 16 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009714586+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009717946+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009721676+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009725626+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009728866+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009732106+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 17 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009735376+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009738956+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009742326+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009746436+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009750666+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009754126+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 18 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009758286+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009761846+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009765776+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009769816+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009773167+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009780507+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 19 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009784207+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009788007+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009791627+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009796347+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009799427+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009802937+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 20 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009806667+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009809987+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009813827+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009818087+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009821397+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009824227+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 21 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009827607+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009831307+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009835167+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009839528+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009842728+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009845888+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 22 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009849168+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009852548+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009855958+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009863648+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009866938+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009870088+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 23 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009879828+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009883728+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009887608+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009891548+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009895238+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009898748+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 24 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009902349+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009907109+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009911519+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009915609+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009918939+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009922329+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 25 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009928329+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009931909+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009935489+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009939429+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009942699+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009946019+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 26 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009949559+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009953489+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.00996244+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.00996647+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.00996987+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.00997314+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 27 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.00997664+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.00998035+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.00998418+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.00998793+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.00999131+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.00999534+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 28 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.00999894+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01000268+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.0100087+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01001258+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01001605+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01002007+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 29 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01002393+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010027531+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010031351+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010035521+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010038781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010042281+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 30 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010046091+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010054261+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010057881+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010061781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010064931+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010068261+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 31 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010072151+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010075981+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010079841+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010098802+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010103202+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010106382+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 32 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010109852+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010113242+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010116952+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010120672+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010124112+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010127162+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 33 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010130662+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010134442+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010138142+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010141972+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010145362+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010152262+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 34 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010156353+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010159913+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010163533+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010167403+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010170533+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010173953+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 35 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010177453+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010181033+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010184693+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010188583+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010191753+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010195033+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 36 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010198603+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010202393+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010206123+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010210583+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010213503+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010216914+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 37 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010220134+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010223644+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010227154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010235154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010238354+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010241754+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 38 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010270284+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010275144+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010278714+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010282825+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010285795+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010288935+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 39 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010292405+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010295915+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010299545+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010311835+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010315565+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010318975+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 40 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010322655+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010326225+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010329815+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010333745+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010337095+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010340515+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 41 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010344066+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010347706+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010355596+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010359046+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010362036+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010365076+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 42 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010367956+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010370906+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010375366+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010378386+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010381006+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010383666+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 43 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010386676+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010389786+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010392636+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010395746+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010398466+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010401166+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 44 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010404226+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010407186+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010410117+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010413237+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010416157+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010418897+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 45 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010423157+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010430107+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010433197+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010451207+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010454217+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010456967+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 46 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010459877+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010463967+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010467067+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010470597+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010473418+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010476108+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 47 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010478998+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010481958+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010484978+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010488258+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010490888+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010493628+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 48 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010496588+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010499498+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010502498+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010505768+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010508558+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010514148+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 49 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010517178+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010520238+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010523478+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010526818+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010529718+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010532508+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 50 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010535639+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010538749+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010541939+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010546689+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010549869+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010552989+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 51 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010556299+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010560079+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010563479+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010567189+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010570299+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010573079+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 52 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010576319+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010579599+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010583099+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010590519+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010594029+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010596989+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 53 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01059986+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01060433+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.01060789+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01061136+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01061393+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01061667+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 54 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01061984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.0106227+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.01062566+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01062876+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01063131+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01063428+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 55 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01063719+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01063997+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.01064278+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01064582+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01064836+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01065114+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 56 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01065437+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01065724+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010671481+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010675671+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010678431+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010691171+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 57 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010694531+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010697641+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010700781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010704001+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010706611+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010709421+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 58 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010712331+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010715351+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010718431+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010721691+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010724371+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010727362+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 59 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010730592+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010733662+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010736712+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010740162+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010742922+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010745772+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 60 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010752222+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010755402+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010758542+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010761822+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010764592+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010767992+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 61 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010771062+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010774282+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010779382+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010782942+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010785722+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010788573+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 62 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010791893+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010794943+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010798083+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010801683+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010804293+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010807283+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 63 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010810443+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010813553+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010816793+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010820233+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010823063+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010829493+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 64 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010832683+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010835953+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010839223+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010842463+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010845333+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010848423+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 65 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010851663+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010854964+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010858364+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010861714+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010864914+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010867984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 66 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010870944+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010874114+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010877464+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010880504+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010883144+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010886194+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 67 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010889154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010892234+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010895524+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010902154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010904984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010907984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 68 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010911064+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010914074+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010917365+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010920505+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.013209581+01:00","Action":"output","Package":"example.com/deeptimeout","Output":"FAIL\texample.com/deeptimeout\t2.021s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:21.013258851+01:00","Action":"fail","Package":"example.com/deeptimeout","Elapsed":2.022}
@@ -0,0 +1,10 @@
{"Time":"2026-10-25T15:34:33.768742837+01:00","Action":"start","Package":"example.com/exiting"}
{"Time":"2026-10-25T15:34:33.773822287+01:00","Action":"run","Package":"example.com/exiting","Test":"TestFine"}
{"Time":"2026-10-25T15:34:33.773896608+01:00","Action":"output","Package":"example.com/exiting","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.773940319+01:00","Action":"output","Package":"example.com/exiting","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.773945359+01:00","Action":"pass","Package":"example.com/exiting","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:34:33.773952529+01:00","Action":"run","Package":"example.com/exiting","Test":"TestExits"}
{"Time":"2026-10-25T15:34:33.773956439+01:00","Action":"output","Package":"example.com/exiting","Test":"TestExits","Output":"=== RUN TestExits\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.773959829+01:00","Action":"output","Package":"example.com/exiting","Test":"TestExits","Output":" p_test.go:11: closing the store\n"}
{"Time":"2026-10-25T15:34:33.775158538+01:00","Action":"output","Package":"example.com/exiting","Output":"FAIL\texample.com/exiting\t0.006s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.775186509+01:00","Action":"fail","Package":"example.com/exiting","Elapsed":0.006}
@@ -0,0 +1,7 @@
{"Time":"2026-10-25T15:34:34.328258934+01:00","Action":"start","Package":"example.com/killed"}
{"Time":"2026-10-25T15:34:34.331465435+01:00","Action":"run","Package":"example.com/killed","Test":"TestKilled"}
{"Time":"2026-10-25T15:34:34.331699248+01:00","Action":"output","Package":"example.com/killed","Test":"TestKilled","Output":"=== RUN TestKilled\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.331853111+01:00","Action":"output","Package":"example.com/killed","Test":"TestKilled","Output":" p_test.go:10: the kernel is about to kill this\n"}
{"Time":"2026-10-25T15:34:34.334563213+01:00","Action":"output","Package":"example.com/killed","Test":"TestKilled","Output":"signal: killed\n"}
{"Time":"2026-10-25T15:34:34.334611114+01:00","Action":"output","Package":"example.com/killed","Output":"FAIL\texample.com/killed\t0.006s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.334627134+01:00","Action":"fail","Package":"example.com/killed","Elapsed":0.006}
@@ -0,0 +1,8 @@
{"Time":"2026-10-25T15:34:35.274953775+01:00","Action":"start","Package":"example.com/mainexit"}
{"Time":"2026-10-25T15:34:35.278129565+01:00","Action":"run","Package":"example.com/mainexit","Test":"TestFine"}
{"Time":"2026-10-25T15:34:35.278191516+01:00","Action":"output","Package":"example.com/mainexit","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.278341979+01:00","Action":"output","Package":"example.com/mainexit","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.278351739+01:00","Action":"pass","Package":"example.com/mainexit","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:34:35.278359039+01:00","Action":"output","Package":"example.com/mainexit","Output":"PASS\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.279516487+01:00","Action":"output","Package":"example.com/mainexit","Output":"FAIL\texample.com/mainexit\t0.004s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.279539298+01:00","Action":"fail","Package":"example.com/mainexit","Elapsed":0.005}
@@ -0,0 +1,27 @@
{"Time":"2026-10-25T15:34:33.031091436+01:00","Action":"start","Package":"example.com/panicking"}
{"Time":"2026-10-25T15:34:33.036353279+01:00","Action":"run","Package":"example.com/panicking","Test":"TestBefore"}
{"Time":"2026-10-25T15:34:33.036441781+01:00","Action":"output","Package":"example.com/panicking","Test":"TestBefore","Output":"=== RUN TestBefore\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.036568163+01:00","Action":"output","Package":"example.com/panicking","Test":"TestBefore","Output":"--- PASS: TestBefore (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.036609863+01:00","Action":"pass","Package":"example.com/panicking","Test":"TestBefore","Elapsed":0}
{"Time":"2026-10-25T15:34:33.036641304+01:00","Action":"run","Package":"example.com/panicking","Test":"TestPanics"}
{"Time":"2026-10-25T15:34:33.036645914+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"=== RUN TestPanics\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.036698995+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":" p_test.go:8: about to look up the seat\n"}
{"Time":"2026-10-25T15:34:33.036773996+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"--- FAIL: TestPanics (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.03897418+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"panic: assignment to entry in nil map [recovered, repanicked]\n"}
{"Time":"2026-10-25T15:34:33.040332172+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\n"}
{"Time":"2026-10-25T15:34:33.040414493+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"goroutine 8 [running]:\n"}
{"Time":"2026-10-25T15:34:33.040521475+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"testing.tRunner.func1.2({0x6b8120, 0x6eff80})\n"}
{"Time":"2026-10-25T15:34:33.040601826+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2125 +0x232\n"}
{"Time":"2026-10-25T15:34:33.040641947+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"testing.tRunner.func1()\n"}
{"Time":"2026-10-25T15:34:33.040720648+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2128 +0x329\n"}
{"Time":"2026-10-25T15:34:33.04083286+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"panic({0x6b8120?, 0x6eff80?})\n"}
{"Time":"2026-10-25T15:34:33.040912921+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/runtime/panic.go:859 +0x125\n"}
{"Time":"2026-10-25T15:34:33.040981652+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"example.com/panicking.TestPanics(0x228226cd0488?)\n"}
{"Time":"2026-10-25T15:34:33.041059903+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/tmp/fix-1901b/panicking/p_test.go:10 +0x53\n"}
{"Time":"2026-10-25T15:34:33.041132704+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"testing.tRunner(0x228226cd0488, 0x6d5bc8)\n"}
{"Time":"2026-10-25T15:34:33.041215586+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:34:33.041221966+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:34:33.041225556+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:34:33.041984088+01:00","Action":"fail","Package":"example.com/panicking","Test":"TestPanics","Elapsed":0}
{"Time":"2026-10-25T15:34:33.041996638+01:00","Action":"output","Package":"example.com/panicking","Output":"FAIL\texample.com/panicking\t0.010s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.042044079+01:00","Action":"fail","Package":"example.com/panicking","Elapsed":0.011}
@@ -0,0 +1,15 @@
{"Time":"2026-10-25T15:34:34.919624192+01:00","Action":"start","Package":"example.com/passthenfail"}
{"Time":"2026-10-25T15:34:34.922861224+01:00","Action":"run","Package":"example.com/passthenfail","Test":"TestFine"}
{"Time":"2026-10-25T15:34:34.922932965+01:00","Action":"output","Package":"example.com/passthenfail","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.922969205+01:00","Action":"output","Package":"example.com/passthenfail","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.922979956+01:00","Action":"pass","Package":"example.com/passthenfail","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:34:34.922989416+01:00","Action":"output","Package":"example.com/passthenfail","Output":"PASS\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.925022528+01:00","Action":"output","Package":"example.com/passthenfail","Output":"panic: the bus closed after the tests: nats: connection closed\n"}
{"Time":"2026-10-25T15:34:34.925043388+01:00","Action":"output","Package":"example.com/passthenfail","Output":"\n"}
{"Time":"2026-10-25T15:34:34.925046848+01:00","Action":"output","Package":"example.com/passthenfail","Output":"goroutine 20 [running]:\n"}
{"Time":"2026-10-25T15:34:34.925051338+01:00","Action":"output","Package":"example.com/passthenfail","Output":"example.com/passthenfail.TestMain.func1()\n"}
{"Time":"2026-10-25T15:34:34.925055878+01:00","Action":"output","Package":"example.com/passthenfail","Output":"\t/tmp/fix-1901b/passthenfail/p_test.go:13 +0x25\n"}
{"Time":"2026-10-25T15:34:34.925061428+01:00","Action":"output","Package":"example.com/passthenfail","Output":"created by example.com/passthenfail.TestMain in goroutine 1\n"}
{"Time":"2026-10-25T15:34:34.925065028+01:00","Action":"output","Package":"example.com/passthenfail","Output":"\t/tmp/fix-1901b/passthenfail/p_test.go:13 +0x25\n"}
{"Time":"2026-10-25T15:34:34.925390203+01:00","Action":"output","Package":"example.com/passthenfail","Output":"FAIL\texample.com/passthenfail\t0.005s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.925402904+01:00","Action":"fail","Package":"example.com/passthenfail","Elapsed":0.006}
@@ -0,0 +1,53 @@
{"Time":"2026-10-25T15:50:42.908557309+01:00","Action":"start","Package":"example.com/race"}
{"Time":"2026-10-25T15:50:42.924298679+01:00","Action":"run","Package":"example.com/race","Test":"TestBefore"}
{"Time":"2026-10-25T15:50:42.92438195+01:00","Action":"output","Package":"example.com/race","Test":"TestBefore","Output":"=== RUN TestBefore\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.924648842+01:00","Action":"output","Package":"example.com/race","Test":"TestBefore","Output":"--- PASS: TestBefore (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.924685133+01:00","Action":"pass","Package":"example.com/race","Test":"TestBefore","Elapsed":0}
{"Time":"2026-10-25T15:50:42.924728573+01:00","Action":"run","Package":"example.com/race","Test":"TestRacesOnTheSeat"}
{"Time":"2026-10-25T15:50:42.924733573+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"=== RUN TestRacesOnTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.92637298+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"==================\n"}
{"Time":"2026-10-25T15:50:42.92641567+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"WARNING: DATA RACE\n"}
{"Time":"2026-10-25T15:50:42.926444061+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Write at 0x000000833468 by goroutine 10:\n"}
{"Time":"2026-10-25T15:50:42.926462621+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" example.com/race.TestRacesOnTheSeat.func1()\n"}
{"Time":"2026-10-25T15:50:42.926467121+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /tmp/fix-1901b/race/p_test.go:13 +0x24\n"}
{"Time":"2026-10-25T15:50:42.926483061+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"\n"}
{"Time":"2026-10-25T15:50:42.926509741+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Previous write at 0x000000833468 by goroutine 9:\n"}
{"Time":"2026-10-25T15:50:42.926530142+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" example.com/race.TestRacesOnTheSeat()\n"}
{"Time":"2026-10-25T15:50:42.926534952+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /tmp/fix-1901b/race/p_test.go:14 +0x30\n"}
{"Time":"2026-10-25T15:50:42.926550462+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.tRunner()\n"}
{"Time":"2026-10-25T15:50:42.926554612+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2195 +0x21c\n"}
{"Time":"2026-10-25T15:50:42.926571142+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*T).Run.gowrap1()\n"}
{"Time":"2026-10-25T15:50:42.926575812+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2260 +0x38\n"}
{"Time":"2026-10-25T15:50:42.926592202+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"\n"}
{"Time":"2026-10-25T15:50:42.926607792+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Goroutine 10 (running) created at:\n"}
{"Time":"2026-10-25T15:50:42.926624292+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" example.com/race.TestRacesOnTheSeat()\n"}
{"Time":"2026-10-25T15:50:42.926628923+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /tmp/fix-1901b/race/p_test.go:13 +0x24\n"}
{"Time":"2026-10-25T15:50:42.926643853+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.tRunner()\n"}
{"Time":"2026-10-25T15:50:42.926648103+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2195 +0x21c\n"}
{"Time":"2026-10-25T15:50:42.926674843+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*T).Run.gowrap1()\n"}
{"Time":"2026-10-25T15:50:42.926680673+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2260 +0x38\n"}
{"Time":"2026-10-25T15:50:42.926695433+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"\n"}
{"Time":"2026-10-25T15:50:42.926711063+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Goroutine 9 (running) created at:\n"}
{"Time":"2026-10-25T15:50:42.926726614+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*T).Run()\n"}
{"Time":"2026-10-25T15:50:42.926730814+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2260 +0xb12\n"}
{"Time":"2026-10-25T15:50:42.926746484+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.runTests.func1()\n"}
{"Time":"2026-10-25T15:50:42.926750764+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2744 +0x84\n"}
{"Time":"2026-10-25T15:50:42.926765354+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.tRunner()\n"}
{"Time":"2026-10-25T15:50:42.926769844+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2195 +0x21c\n"}
{"Time":"2026-10-25T15:50:42.926791794+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.runTests()\n"}
{"Time":"2026-10-25T15:50:42.926796704+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2742 +0x9e9\n"}
{"Time":"2026-10-25T15:50:42.926811544+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*M).Run()\n"}
{"Time":"2026-10-25T15:50:42.926816694+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2602 +0xf44\n"}
{"Time":"2026-10-25T15:50:42.926831695+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" main.main()\n"}
{"Time":"2026-10-25T15:50:42.926835935+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" _testmain.go:50 +0x164\n"}
{"Time":"2026-10-25T15:50:42.926850905+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"==================\n"}
{"Time":"2026-10-25T15:50:42.935320241+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.go:1867: race detected during execution of test\n","OutputType":"error"}
{"Time":"2026-10-25T15:50:42.935520113+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"--- FAIL: TestRacesOnTheSeat (0.01s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.935592973+01:00","Action":"fail","Package":"example.com/race","Test":"TestRacesOnTheSeat","Elapsed":0.01}
{"Time":"2026-10-25T15:50:42.935722225+01:00","Action":"run","Package":"example.com/race","Test":"TestAfter"}
{"Time":"2026-10-25T15:50:42.935728845+01:00","Action":"output","Package":"example.com/race","Test":"TestAfter","Output":"=== RUN TestAfter\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.935995928+01:00","Action":"output","Package":"example.com/race","Test":"TestAfter","Output":"--- PASS: TestAfter (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.936006328+01:00","Action":"pass","Package":"example.com/race","Test":"TestAfter","Elapsed":0}
{"Time":"2026-10-25T15:50:42.936009978+01:00","Action":"output","Package":"example.com/race","Output":"FAIL\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.937730745+01:00","Action":"output","Package":"example.com/race","Output":"FAIL\texample.com/race\t0.028s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.937802066+01:00","Action":"fail","Package":"example.com/race","Elapsed":0.029}
@@ -0,0 +1,27 @@
{"Time":"2026-10-25T15:50:43.551701745+01:00","Action":"start","Package":"example.com/racemain"}
{"Time":"2026-10-25T15:50:43.570709708+01:00","Action":"run","Package":"example.com/racemain","Test":"TestFine"}
{"Time":"2026-10-25T15:50:43.570806059+01:00","Action":"output","Package":"example.com/racemain","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:43.57085525+01:00","Action":"output","Package":"example.com/racemain","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:43.57086149+01:00","Action":"pass","Package":"example.com/racemain","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:50:43.57086974+01:00","Action":"output","Package":"example.com/racemain","Output":"PASS\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:43.572155623+01:00","Action":"output","Package":"example.com/racemain","Output":"==================\n"}
{"Time":"2026-10-25T15:50:43.572168083+01:00","Action":"output","Package":"example.com/racemain","Output":"WARNING: DATA RACE\n"}
{"Time":"2026-10-25T15:50:43.572171933+01:00","Action":"output","Package":"example.com/racemain","Output":"Write at 0x000000836428 by goroutine 9:\n"}
{"Time":"2026-10-25T15:50:43.572176703+01:00","Action":"output","Package":"example.com/racemain","Output":" example.com/racemain.TestMain.func1()\n"}
{"Time":"2026-10-25T15:50:43.572180053+01:00","Action":"output","Package":"example.com/racemain","Output":" /tmp/fix-1901b/racemain/p_test.go:15 +0x24\n"}
{"Time":"2026-10-25T15:50:43.572184923+01:00","Action":"output","Package":"example.com/racemain","Output":"\n"}
{"Time":"2026-10-25T15:50:43.572189613+01:00","Action":"output","Package":"example.com/racemain","Output":"Previous write at 0x000000836428 by main goroutine:\n"}
{"Time":"2026-10-25T15:50:43.572192993+01:00","Action":"output","Package":"example.com/racemain","Output":" example.com/racemain.TestMain()\n"}
{"Time":"2026-10-25T15:50:43.572198573+01:00","Action":"output","Package":"example.com/racemain","Output":" /tmp/fix-1901b/racemain/p_test.go:16 +0x44\n"}
{"Time":"2026-10-25T15:50:43.572202433+01:00","Action":"output","Package":"example.com/racemain","Output":" main.main()\n"}
{"Time":"2026-10-25T15:50:43.572205623+01:00","Action":"output","Package":"example.com/racemain","Output":" _testmain.go:48 +0x171\n"}
{"Time":"2026-10-25T15:50:43.572208443+01:00","Action":"output","Package":"example.com/racemain","Output":"\n"}
{"Time":"2026-10-25T15:50:43.572211573+01:00","Action":"output","Package":"example.com/racemain","Output":"Goroutine 9 (running) created at:\n"}
{"Time":"2026-10-25T15:50:43.572214803+01:00","Action":"output","Package":"example.com/racemain","Output":" example.com/racemain.TestMain()\n"}
{"Time":"2026-10-25T15:50:43.572218043+01:00","Action":"output","Package":"example.com/racemain","Output":" /tmp/fix-1901b/racemain/p_test.go:15 +0x37\n"}
{"Time":"2026-10-25T15:50:43.572225414+01:00","Action":"output","Package":"example.com/racemain","Output":" main.main()\n"}
{"Time":"2026-10-25T15:50:43.572228654+01:00","Action":"output","Package":"example.com/racemain","Output":" _testmain.go:48 +0x171\n"}
{"Time":"2026-10-25T15:50:43.572231664+01:00","Action":"output","Package":"example.com/racemain","Output":"==================\n"}
{"Time":"2026-10-25T15:50:44.581262249+01:00","Action":"output","Package":"example.com/racemain","Output":"Found 1 data race(s)\n"}
{"Time":"2026-10-25T15:50:44.584693204+01:00","Action":"output","Package":"example.com/racemain","Output":"FAIL\texample.com/racemain\t1.031s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:44.584737425+01:00","Action":"fail","Package":"example.com/racemain","Elapsed":1.033}
@@ -0,0 +1,43 @@
{"Time":"2026-10-25T15:34:36.049580618+01:00","Action":"start","Package":"example.com/timeout"}
{"Time":"2026-10-25T15:34:36.053250986+01:00","Action":"run","Package":"example.com/timeout","Test":"TestQuick"}
{"Time":"2026-10-25T15:34:36.053354018+01:00","Action":"output","Package":"example.com/timeout","Test":"TestQuick","Output":"=== RUN TestQuick\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:36.053459789+01:00","Action":"output","Package":"example.com/timeout","Test":"TestQuick","Output":"--- PASS: TestQuick (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:36.05348499+01:00","Action":"pass","Package":"example.com/timeout","Test":"TestQuick","Elapsed":0}
{"Time":"2026-10-25T15:34:36.0535114+01:00","Action":"run","Package":"example.com/timeout","Test":"TestWaitsForever"}
{"Time":"2026-10-25T15:34:36.05351549+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"=== RUN TestWaitsForever\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:36.053567451+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":" p_test.go:11: waiting for the bus\n"}
{"Time":"2026-10-25T15:34:38.064693307+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"panic: test timed out after 2s\n"}
{"Time":"2026-10-25T15:34:38.06487122+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\trunning tests:\n"}
{"Time":"2026-10-25T15:34:38.06491684+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t\tTestWaitsForever (2s)\n"}
{"Time":"2026-10-25T15:34:38.065034382+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\n"}
{"Time":"2026-10-25T15:34:38.065108573+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"goroutine 33 [running]:\n"}
{"Time":"2026-10-25T15:34:38.065154754+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.(*M).startAlarm.func1()\n"}
{"Time":"2026-10-25T15:34:38.065386208+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2961 +0x34a\n"}
{"Time":"2026-10-25T15:34:38.065431878+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"created by time.goFunc\n"}
{"Time":"2026-10-25T15:34:38.0655151+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/time/sleep.go:182 +0x2d\n"}
{"Time":"2026-10-25T15:34:38.06553471+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\n"}
{"Time":"2026-10-25T15:34:38.065847805+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"goroutine 1 [chan receive]:\n"}
{"Time":"2026-10-25T15:34:38.066038298+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.(*T).Run(0x3a0209bc8008, {0x5580aa?, 0x3a0209a9caa0?}, 0x6d5b10)\n"}
{"Time":"2026-10-25T15:34:38.066047488+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2268 +0x4f2\n"}
{"Time":"2026-10-25T15:34:38.066051368+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.runTests.func1(0x3a0209bc8008)\n"}
{"Time":"2026-10-25T15:34:38.066056128+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2744 +0x37\n"}
{"Time":"2026-10-25T15:34:38.066059478+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.tRunner(0x3a0209bc8008, 0x3a0209a9cbc8)\n"}
{"Time":"2026-10-25T15:34:38.066064958+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:34:38.066069599+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.runTests({0x559017, 0x13}, {0x559017, 0x13}, 0x3a0209b8e0d8, {0x6f19f0, 0x2, 0x2}, {0xc2afa2bf832a3f56, 0x773908b5, ...})\n"}
{"Time":"2026-10-25T15:34:38.066083309+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2742 +0x510\n"}
{"Time":"2026-10-25T15:34:38.066086879+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.(*M).Run(0x3a0209b8c1e0)\n"}
{"Time":"2026-10-25T15:34:38.066090269+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2602 +0x6af\n"}
{"Time":"2026-10-25T15:34:38.066093429+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"main.main()\n"}
{"Time":"2026-10-25T15:34:38.066097569+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t_testmain.go:48 +0x9b\n"}
{"Time":"2026-10-25T15:34:38.066100639+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\n"}
{"Time":"2026-10-25T15:34:38.066103769+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"goroutine 20 [sleep]:\n"}
{"Time":"2026-10-25T15:34:38.066106969+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"time.Sleep(0xdf8475800)\n"}
{"Time":"2026-10-25T15:34:38.066110279+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/runtime/time.go:368 +0x165\n"}
{"Time":"2026-10-25T15:34:38.066113639+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"example.com/timeout.TestWaitsForever(0x3a0209bc8488?)\n"}
{"Time":"2026-10-25T15:34:38.066117109+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/tmp/fix-1901b/timeout/p_test.go:12 +0x48\n"}
{"Time":"2026-10-25T15:34:38.066120239+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.tRunner(0x3a0209bc8488, 0x6d5b10)\n"}
{"Time":"2026-10-25T15:34:38.066123509+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:34:38.066126839+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:34:38.066129989+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:34:38.066986453+01:00","Action":"output","Package":"example.com/timeout","Output":"FAIL\texample.com/timeout\t2.017s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:38.067002113+01:00","Action":"fail","Package":"example.com/timeout","Elapsed":2.017}
+1 -1
View File
@@ -3,7 +3,7 @@ module meshbuildgo
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024232922-99407c37df95
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026095702-79649c3fa9f4
github.com/distribution/distribution/v3 v3.1.2
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
+2 -10
View File
@@ -1,13 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024214154-bc16740feb0d h1:oJREw0ggKqJWJ5DEISSL0i2ZC5XqdXAidxo9NJ74TCc=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024214154-bc16740feb0d/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024225846-cb548a6a588c h1:GWHP9ozfkFXHVEL2gE4Xa5kc5e8uPO+UL9+mau6iaIs=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024225846-cb548a6a588c/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024230839-4a12ccb78d19 h1:NqU4hkk/jHdTEcGFnnXvdOJwD8dH5u+qWaCdQQ+JYMc=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024230839-4a12ccb78d19/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024232250-9b408db24a58 h1:4Damtdqlf6u6nJhvMA99peDy1jmjixtcwFXJ4bWEkLk=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024232250-9b408db24a58/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024232922-99407c37df95 h1:I8zbgGzRjmgYydPTWo8hmaZW/P/g+qGDUfu3E32/pkU=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024232922-99407c37df95/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026095702-79649c3fa9f4 h1:05sVel/UOvz4/g0dxrDOeVUL7BGakVrxPBlqRq02ZhM=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026095702-79649c3fa9f4/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/alicebob/miniredis/v2 v2.35.0 h1:QwLphYqCEAo1eu1TqPRN2jgVMPBweeQcR21jeqDCONI=
github.com/alicebob/miniredis/v2 v2.35.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
+1 -1
View File
@@ -10,7 +10,7 @@
"state": [
"toolchains",
"keys",
"steps"
{"name": "steps", "ttl-seconds": 172800}, "pauses"
],
"claims": [
{
@@ -13,6 +13,7 @@ import (
var Kept = []announce.State{
{Name: "keys", Readable: true},
{Name: "steps", Readable: true},
{Name: "pauses", Readable: true},
}
// keepStates announces Kept as this module's (stdio.Keeps): main calls it before it serves.
@@ -37,16 +37,20 @@ func (m *Module) Puller(steps []StepTool, source buildstep.AskSource) *buildstep
source = buildstep.SeatAsks{Language: delivery.AskImage}
}
capacity := m.Capacity
// The capacity's own change wakes the puller (issue 1887): the runtime's watch of the config value as mesh-config
// resolved it; the settings file is read only when that watch cannot start. A test's Capacity is its own.
var watch func(string, func(stdio.ProvisionChange) error) error
if capacity == nil {
capacity = buildstep.NodeCapacity(m.Settings, buildstep.DefaultCapacities, m.Node, m.Say)
watch = buildstep.WatchConfigServed
}
ask := m.askSteps(steps)
var load func(string) ([]byte, error)
if m.ReadSteps.Get != nil {
load = func(key string) ([]byte, error) { return m.ReadSteps.Get(key) }
if m.ReadPauses.Get != nil {
load = func(key string) ([]byte, error) { return m.ReadPauses.Get(key) }
}
return &buildstep.Puller{Source: source, Language: delivery.AskImage, Node: m.Node, Put: m.Steps, Load: load,
Capacity: capacity, Logf: m.Say,
return &buildstep.Puller{Source: source, Language: delivery.AskImage, Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load,
Capacity: capacity, WatchConfig: watch, Logf: m.Say,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
if err := m.Check(ctx); err != nil {
return nil, err
@@ -29,13 +29,15 @@ func logf(format string, a ...any) { fmt.Fprintf(os.Stderr, "[mesh-build-image]
func main() {
ctx := context.Background()
m := &Module{
Node: strings.TrimSpace(os.Getenv("MESH_NODE")),
Steps: stdio.State(delivery.StepsState).Put,
ReadSteps: readOf(stdio.State(delivery.StepsState)),
Ctx: ctx,
Say: logf,
Work: WorkDir,
Store: StoreDir,
Node: strings.TrimSpace(os.Getenv("MESH_NODE")),
Steps: stdio.State(delivery.StepsState).Put,
ReadSteps: readOf(stdio.State(delivery.StepsState)),
Pauses: stdio.State(delivery.PausesState).Put,
ReadPauses: readOf(stdio.State(delivery.PausesState)),
Ctx: ctx,
Say: logf,
Work: WorkDir,
Store: StoreDir,
}
m.Floor = floorOf(m.Node)
if u, err := user.Current(); err == nil {
@@ -97,9 +97,13 @@ func TestManifestKeepsItsStatesClaimsItsBuildSeatAndDeclaresItsHealthTool(t *tes
states := []string{}
for _, s := range m.State {
states = append(states, s.Name)
// A step's key is per step and dropped after its age, so the bucket cannot fill; a pause has no age.
if (s.Name == "steps") != (s.TTLSeconds > 0) {
t.Errorf("state %s has ttl-seconds %d: only steps has an age", s.Name, s.TTLSeconds)
}
}
slices.Sort(states)
if want := []string{"keys", "steps"}; !slices.Equal(states, want) {
if want := []string{"keys", "pauses", "steps"}; !slices.Equal(states, want) {
t.Errorf("state = %v, want %v", states, want)
}
if len(m.Claims) != 1 || m.Claims[0].Name != Seat || !slices.Equal(m.Claims[0].Serves, []string{"current", "pause", "resume"}) {
@@ -24,6 +24,9 @@ type Module struct {
// Steps is the Put of the steps state; ReadSteps reads it back for the read tools.
Steps buildstep.Put
ReadSteps StateRead
// Pauses is the Put of the pauses state, which keeps the node's pause key without an age; ReadPauses reads it back.
Pauses buildstep.Put
ReadPauses StateRead
// Ctx is the process's context, which a step runs under (the tool call's is cut at 30 s, the step's is not).
Ctx context.Context
Say func(format string, a ...any)
@@ -7,9 +7,12 @@ package main
import (
"encoding/json"
"fmt"
"log"
"sort"
"strconv"
"strings"
"sync"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
@@ -38,6 +41,53 @@ func readOf(s stdio.KeptState) StateRead {
}}
}
// MaxStateReads is the most values one read tool reads from a state in a call, and stateReadParallel how many of them
// are in flight at once. Every value is a round trip from the tool through the runtime to the bus, so a call that read
// every key of a large state one after the other outlasted the call's 30 s (issue 1827: 1164 keys, 20 MB).
const (
MaxStateReads = 1000
stateReadParallel = 16
)
// readValues reads the values of keys, stateReadParallel at a time, in the order of keys (nil for a key with none). It
// stops starting reads after the first error and returns it.
func readValues(keys []string, get func(string) (json.RawMessage, error)) ([]json.RawMessage, error) {
out := make([]json.RawMessage, len(keys))
var (
wg sync.WaitGroup
mu sync.Mutex
fail error
next = make(chan int)
)
for w := 0; w < stateReadParallel && w < len(keys); w++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := range next {
raw, err := get(keys[i])
mu.Lock()
if err != nil && fail == nil {
fail = err
}
mu.Unlock()
out[i] = raw
}
}()
}
for i := range keys {
mu.Lock()
stop := fail != nil
mu.Unlock()
if stop {
break
}
next <- i
}
close(next)
wg.Wait()
return out, fail
}
// stepKey is the key of a step of this node: the id alone, or the whole key <node>.<id> as a step tool answered it.
func (m *Module) stepKey(id string) (string, error) {
id = strings.TrimSpace(id)
@@ -79,27 +129,41 @@ type StepLine struct {
Reason string `json:"reason,omitempty"`
}
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero).
func (m *Module) RecentSteps(limit int) ([]StepLine, error) {
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero). Only
// the node's own keys (<node>.) are listed and the newest max(limit*4, 100) of them read (at most MaxStateReads): a step's
// id is base-36 milliseconds and random digits, so its string order is its start order. cut says older steps were not
// read. The values still order the lines, since an ask step's id carries the ask time.
func (m *Module) RecentSteps(limit int) (lines []StepLine, cut bool, err error) {
if limit <= 0 {
limit = 10
}
if m.ReadSteps.Keys == nil || m.ReadSteps.Get == nil {
return nil, fmt.Errorf("the steps state cannot be read here")
return nil, false, fmt.Errorf("the steps state cannot be read here")
}
start := time.Now()
keys, err := m.ReadSteps.Keys()
if err != nil {
return nil, err
return nil, false, err
}
own := keys[:0:0]
for _, k := range keys {
if strings.HasPrefix(k, m.Node+".") {
if node, _, err := delivery.ParseStepKey(k); err == nil && node == m.Node {
own = append(own, k)
}
}
}
sort.Sort(sort.Reverse(sort.StringSlice(own)))
if n := min(max(limit*4, 100), MaxStateReads); len(own) > n {
own, cut = own[:n], true
}
raws, err := readValues(own, m.ReadSteps.Get)
log.Printf("steps read: %d of %d keys listed read in %v (cut %v)", len(own), len(keys), time.Since(start).Round(time.Millisecond), cut)
if err != nil {
return nil, cut, err
}
var steps []delivery.Step
for _, k := range keys {
if node, _, err := delivery.ParseStepKey(k); err != nil || node != m.Node {
continue
}
raw, err := m.ReadSteps.Get(k)
if err != nil {
return nil, err
}
for _, raw := range raws {
if len(raw) == 0 {
continue
}
@@ -121,7 +185,7 @@ func (m *Module) RecentSteps(limit int) ([]StepLine, error) {
}
out = append(out, l)
}
return out, nil
return out, cut, nil
}
// ReadTools are the module's read tools. All are read-only.
@@ -143,8 +207,13 @@ func (m *Module) ReadTools() []stdio.Tool {
Input: map[string]any{"type": "object", "properties": map[string]any{
"limit": map[string]any{"type": "integer", "description": "how many, at most; 10 when left out"}}},
Run: func(args map[string]any) (any, error) {
steps, err := m.RecentSteps(intArg(args["limit"]))
return map[string]any{"node": m.Node, "steps": steps}, err
steps, cut, err := m.RecentSteps(intArg(args["limit"]))
ans := map[string]any{"node": m.Node, "steps": steps}
if cut {
ans["cut"] = true
ans["note"] = fmt.Sprintf("this node has more steps stored than one call reads; older steps were not read (ask for a larger limit, at most %d reads)", MaxStateReads)
}
return ans, err
}},
}
}
@@ -2,7 +2,11 @@ package main
import (
"encoding/json"
"fmt"
"strings"
"sync/atomic"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
@@ -39,8 +43,67 @@ func TestTheReadToolsAnswerAStepsResult(t *testing.T) {
if err := json.Unmarshal(got.Result, &back); err != nil || back.Digest != res.Digest {
t.Errorf("result = %s, %v", got.Result, err)
}
lines, err := m.RecentSteps(0)
lines, _, err := m.RecentSteps(0)
if err != nil || len(lines) != 1 || lines[0].Tool != BuildTool {
t.Errorf("steps = %+v, %v", lines, err)
}
}
// 1827: on the live shape (4 nodes, 600 steps each; a steps state of 1164 keys / 20 MB timed out at 30 s) the steps tool
// lists, then reads only this node's keys, in parallel, at most MaxStateReads of them, and says when it cut.
func TestRecentStepsReadsOnlyTheNodesKeysInParallelAndBounded(t *testing.T) {
t0 := time.Date(2026, 10, 25, 9, 0, 0, 0, time.UTC)
for _, row := range []struct {
perNode, limit int
reads int64
steps int
cut bool
}{
{20, 0, 20, 10, false},
{600, 0, 100, 10, true},
{1500, 300, MaxStateReads, 300, true},
} {
var reads, inFlight, maxFlight atomic.Int64
stored := map[string][]byte{}
for _, n := range []string{"ace", "g14", "novox", "shanks"} {
for i := 0; i < row.perNode; i++ {
b, _ := delivery.EncodeStep(delivery.Step{ID: fmt.Sprintf("s%04d", i), Tool: "t", Node: n, State: delivery.StepPassed, Started: t0.Add(time.Duration(i) * time.Second)})
stored[n+".s"+fmt.Sprintf("%04d", i)] = b
}
}
m := &Module{Node: "shanks"}
m.ReadSteps = StateRead{
Keys: func() ([]string, error) {
var k []string
for key := range stored {
k = append(k, key)
}
return k, nil
},
Get: func(key string) (json.RawMessage, error) {
reads.Add(1)
now := inFlight.Add(1)
for o := maxFlight.Load(); now > o && !maxFlight.CompareAndSwap(o, now); o = maxFlight.Load() {
}
time.Sleep(time.Millisecond)
inFlight.Add(-1)
if !strings.HasPrefix(key, "shanks.") {
t.Errorf("read %s, a key of another node", key)
}
return stored[key], nil
},
}
start := time.Now()
lines, cut, err := m.RecentSteps(row.limit)
if err != nil || len(lines) != row.steps || cut != row.cut || reads.Load() != row.reads {
t.Errorf("%d a node: %d steps, cut %v, %d reads, err %v; want %d, %v, %d", row.perNode, len(lines), cut, reads.Load(), err, row.steps, row.cut, row.reads)
}
if row.cut && len(lines) > 0 && lines[0].ID != fmt.Sprintf("s%04d", row.perNode-1) {
t.Errorf("%d a node: newest answered %s", row.perNode, lines[0].ID)
}
if maxFlight.Load() < 2 {
t.Errorf("%d a node: values read one at a time", row.perNode)
}
t.Logf("%d a node: %d reads in %v (one after the other: %v)", row.perNode, reads.Load(), time.Since(start), time.Duration(reads.Load())*time.Millisecond)
}
}
+1 -1
View File
@@ -3,7 +3,7 @@ module meshbuildimage
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023213200-403a5d72ab93
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261025175617-518dae5ec9c8
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023213200-403a5d72ab93 h1:FAiFY2KBKEoZSH49OZTt7/zRBIQxyl5g/KhYrpggbN4=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023213200-403a5d72ab93/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261025175617-518dae5ec9c8 h1:x4mvY7yiGTP5lnzxbiwyj1o24p05hAmzZP7xHg3XX2M=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261025175617-518dae5ec9c8/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
+1 -1
View File
@@ -9,7 +9,7 @@
],
"state": [
"keys",
"steps"
{"name": "steps", "ttl-seconds": 172800}, "pauses"
],
"claims": [
{
@@ -14,6 +14,7 @@ var Kept = []announce.State{
{Name: "toolchains", Readable: true},
{Name: "keys", Readable: true},
{Name: "steps", Readable: true},
{Name: "pauses", Readable: true},
}
// keepStates announces Kept as this module's (stdio.Keeps): main calls it before it serves.
@@ -45,16 +45,20 @@ func (m *Module) Puller(steps []StepTool, source buildstep.AskSource) *buildstep
source = buildstep.SeatAsks{Language: delivery.AskNode}
}
capacity := m.Capacity
// The capacity's own change wakes the puller (issue 1887): the runtime's watch of the config value as mesh-config
// resolved it; the settings file is read only when that watch cannot start. A test's Capacity is its own.
var watch func(string, func(stdio.ProvisionChange) error) error
if capacity == nil {
capacity = buildstep.NodeCapacity(m.Settings, buildstep.DefaultCapacities, m.Node, m.Say)
watch = buildstep.WatchConfigServed
}
ask := m.askSteps(steps)
var load func(string) ([]byte, error)
if m.ReadSteps.Get != nil {
load = func(key string) ([]byte, error) { return m.ReadSteps.Get(key) }
if m.ReadPauses.Get != nil {
load = func(key string) ([]byte, error) { return m.ReadPauses.Get(key) }
}
return &buildstep.Puller{Source: source, Language: delivery.AskNode, Node: m.Node, Put: m.Steps, Load: load,
Capacity: capacity, Logf: m.Say,
return &buildstep.Puller{Source: source, Language: delivery.AskNode, Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load,
Capacity: capacity, WatchConfig: watch, Logf: m.Say,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
if err := m.Prepare(ctx); err != nil {
return nil, err
@@ -17,11 +17,11 @@ import (
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
// realMachine gives the tests the real toolchain of the module's own lock and makes two workspaces holding one fixture
// bundle. The toolchain is the one this node's installed Node build module keeps (ADR 0533 §1): the integration level
// is given its toolchains read-only and no network, so it is copied into the test's own cache from there. Only where
// no toolchain is installed (a desk) is it installed with `npm ci --ignore-scripts` from the mesh's registry, as the
// module does, which needs the network.
// realMachine gives the tests the real toolchain of the module's own lock and makes three workspaces holding one fixture
// bundle. The toolchain is the one this node's installed Node build module keeps (ADR 0533 §1), copied into the test's
// own cache: the integration level is given its toolchains read-only and no network, so it is never installed from the
// registry here. Where no Node build module has installed one (a node without it, a desk), the test is skipped by
// name: an install from the public registry inside a shut-off level only fails, after 70 s of npm's retries.
func realMachine(t *testing.T) (*Module, string) {
t.Helper()
for _, p := range []string{"node", "npm", "bwrap"} { // bwrap: tsc, esbuild and the scripts run shut off (1309)
@@ -44,16 +44,19 @@ func realMachine(t *testing.T) (*Module, string) {
run := forgekey.ExecRunnerFor(0)
m := &Module{Node: "ace", Recipe: lock, Work: work, Run: run, Ctx: context.Background(),
Toolchain: &Toolchain{Paths: Paths{Home: home, Cache: cache}, Run: run, Registry: func() (Registry, error) {
return Registry{URL: "https://git.novox.be/api/packages/novox/npm/"}, nil
return Registry{}, fmt.Errorf("the integration test installs no toolchain from a registry: it has no network")
}}}
withStore(t, m)
if from := installedToolchain(t, filepath.Join(CacheDir, "toolchain"), filepath.Join(home, "toolchain", "package-lock.json")); from != "" {
copyToolchain(t, from, m.Toolchain.Paths.Dir(m.Recipe.Version()))
from := installedToolchain(t, filepath.Join(CacheDir, "toolchain"), filepath.Join(home, "toolchain", "package-lock.json"))
if from == "" {
t.Skipf("NOT RUN: no Node toolchain is installed under %s for this account to read (no Node build module on this node); "+
"the integration level has no network to install one, so this runs on a node that holds the Node build module", filepath.Join(CacheDir, "toolchain"))
}
if err := m.Toolchain.Ensure(context.Background(), m.Recipe.Version()); err != nil {
t.Fatalf("no toolchain of this lock is installed under %s, and installing it from the registry failed: %v", filepath.Join(CacheDir, "toolchain"), err)
copyToolchain(t, from, m.Toolchain.Paths.Dir(m.Recipe.Version()))
if err := m.Toolchain.Ensure(context.Background(), m.Recipe.Version()); err != nil { // present: opens it, installs nothing
t.Fatalf("the toolchain copied from %s cannot be used: %v", from, err)
}
for _, id := range []string{"w1", "w2"} {
for _, id := range []string{"w1", "w2", "w3"} {
for p, body := range map[string]string{
"modules/m/module.json": `{"module":"m","build":{"artifacts":[{"name":"tools","kind":"bundle","language":"typescript","entrypoints":["tools/index.js"]}]}}`,
"modules/m/package.json": `{"name":"m","type":"module","scripts":{"test":"node -e \"process.exit(0)\""}}`,
@@ -65,10 +68,36 @@ func realMachine(t *testing.T) (*Module, string) {
return m, work
}
// 1490/AC2: a fixture TypeScript bundle importing @novox/mesh-sdk, built twice by the real tsc and esbuild, has equal
// digests, and its archive holds the bundled .js, the .serve.mjs launcher and package.json.
func TestAFixtureBundleBuiltTwiceHasOneDigest(t *testing.T) {
// 1490/AC2, 1491 and issue 1869, in one test so the toolchain is copied once: the real type-check and test script of the
// fixture pass the unit level; a package whose test imports its own build output (as mesh-sdk's do) passes the unit level,
// the test step emitting it with the real tsc, shut off, before the test script runs; and the fixture TypeScript bundle
// importing @novox/mesh-sdk, built twice by the real tsc and esbuild, has equal digests, its archive holding the bundled
// .js, the .serve.mjs launcher and package.json. The unit levels run in w3, so the two builds' workspaces stay alike.
func TestTheFixturePassesTheUnitLevelAndBuildsTwiceToOneDigest(t *testing.T) {
m, work := realMachine(t)
out, err := m.test(context.Background(), map[string]any{"workspace": "w3", "module": "modules/m", "level": "unit"})
if err != nil {
t.Fatal(err)
}
if r := out.(*TestResult); !r.Passed || r.NoTests || len(r.Ran) != 2 {
t.Errorf("the fixture's unit level: result = %+v", r)
}
for p, body := range map[string]string{
"modules/p/package.json": `{"name":"p","type":"module","scripts":{"build":"tsc","test":"node --test test/a.test.mjs","prepare":"tsc"}}`,
"modules/p/tsconfig.json": `{"compilerOptions":{"module":"NodeNext","moduleResolution":"NodeNext","target":"ES2022","outDir":"dist","rootDir":"src"},"include":["src"]}`,
"modules/p/src/index.ts": "export const answer: number = 42;\n",
"modules/p/test/a.test.mjs": "import { test } from 'node:test';\nimport assert from 'node:assert';\nimport { answer } from '../dist/index.js';\n" +
"test('the build is there', () => assert.equal(answer, 42));\n",
} {
write(t, filepath.Join(work, "w3", "tree", filepath.FromSlash(p)), body, 0o644)
}
out, err = m.test(context.Background(), map[string]any{"workspace": "w3", "module": "modules/p", "level": "unit"})
if err != nil {
t.Fatal(err)
}
if r := out.(*TestResult); !r.Passed || !slices.Equal(r.Ran, []string{"build", "test"}) {
t.Errorf("the package importing its build: result = %+v", r)
}
var digests []string
for _, id := range []string{"w1", "w2"} {
out, err := m.build(context.Background(), map[string]any{"workspace": id, "module": "modules/m", "artifact": "tools"})
@@ -89,43 +118,6 @@ func TestAFixtureBundleBuiltTwiceHasOneDigest(t *testing.T) {
}
}
// 1491: the real type-check and test script of the same fixture pass the unit level.
func TestTheFixturePassesTheUnitLevel(t *testing.T) {
m, _ := realMachine(t)
out, err := m.test(context.Background(), map[string]any{"workspace": "w1", "module": "modules/m", "level": "unit"})
if err != nil {
t.Fatal(err)
}
if r := out.(*TestResult); !r.Passed || r.NoTests || len(r.Ran) != 2 {
t.Errorf("result = %+v", r)
}
}
// Issue 1869: a package whose test imports its own build output (as mesh-sdk's do) passes the unit level: the test step
// emits it with the real tsc, shut off, before the test script runs.
func TestAPackageWhoseTestsImportItsBuildPassesTheUnitLevel(t *testing.T) {
if _, err := exec.LookPath("bwrap"); err != nil {
t.Skip("NOT RUN: no bubblewrap installed where this check runs; every build node has it (the module declares it)")
}
m, work := realMachine(t)
for p, body := range map[string]string{
"modules/p/package.json": `{"name":"p","type":"module","scripts":{"build":"tsc","test":"node --test test/a.test.mjs","prepare":"tsc"}}`,
"modules/p/tsconfig.json": `{"compilerOptions":{"module":"NodeNext","moduleResolution":"NodeNext","target":"ES2022","outDir":"dist","rootDir":"src"},"include":["src"]}`,
"modules/p/src/index.ts": "export const answer: number = 42;\n",
"modules/p/test/a.test.mjs": "import { test } from 'node:test';\nimport assert from 'node:assert';\nimport { answer } from '../dist/index.js';\n" +
"test('the build is there', () => assert.equal(answer, 42));\n",
} {
write(t, filepath.Join(work, "w1", "tree", filepath.FromSlash(p)), body, 0o644)
}
out, err := m.test(context.Background(), map[string]any{"workspace": "w1", "module": "modules/p", "level": "unit"})
if err != nil {
t.Fatal(err)
}
if r := out.(*TestResult); !r.Passed || !slices.Equal(r.Ran, []string{"build", "test"}) {
t.Errorf("result = %+v", r)
}
}
// installedToolchain is the toolchain directory under toolchains that this node's Node build module installed: the
// one installed from the lock at lockPath, byte for byte, or, when the change carries a lock no delivered release has
// installed yet, the one installed last, said in the test's log (ADR 0533 §3: a new lock is proven by the module that
@@ -46,6 +46,8 @@ func main() {
Emit: func(t string, b any) error { return stdio.Emit(t, b) },
Steps: stdio.State(delivery.StepsState).Put,
ReadSteps: readOf(stdio.State(delivery.StepsState)),
Pauses: stdio.State(delivery.PausesState).Put,
ReadPauses: readOf(stdio.State(delivery.PausesState)),
ReadToolchainsState: readOf(stdio.State(delivery.ToolchainsState)),
Work: WorkDir,
Run: forgekey.ExecRunnerClean(0),
@@ -94,9 +94,13 @@ func TestManifestKeepsItsStatesClaimsItsBuildSeatAndDeclaresItsHealthTool(t *tes
states := []string{}
for _, s := range m.State {
states = append(states, s.Name)
// A step's key is per step and dropped after its age, so the bucket cannot fill; a pause has no age.
if (s.Name == "steps") != (s.TTLSeconds > 0) {
t.Errorf("state %s has ttl-seconds %d: only steps has an age", s.Name, s.TTLSeconds)
}
}
slices.Sort(states)
if want := []string{"keys", "steps", "toolchains"}; !slices.Equal(states, want) {
if want := []string{"keys", "pauses", "steps", "toolchains"}; !slices.Equal(states, want) {
t.Errorf("state = %v, want %v", states, want)
}
if len(m.Claims) != 1 || m.Claims[0].Name != Seat || !slices.Equal(m.Claims[0].Serves, []string{"current", "pause", "resume"}) {
@@ -43,6 +43,9 @@ type Module struct {
Toolchains, Steps buildstep.Put
// ReadSteps and ReadToolchainsState read those two states back for the read tools (issue 1562).
ReadSteps, ReadToolchainsState StateRead
// Pauses is the Put of the pauses state, which keeps the node's pause key without an age; ReadPauses reads it back.
Pauses buildstep.Put
ReadPauses StateRead
// Emit tells the bus an event (stdio.Emit); nil emits nothing. The guard emits toolchain.reported after each write of
// the toolchains key, so the indexer, which holds no timer, learns that the bucket exists (issue 1444).
Emit func(eventType string, body any) error
@@ -7,9 +7,12 @@ package main
import (
"encoding/json"
"fmt"
"log"
"sort"
"strconv"
"strings"
"sync"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
@@ -39,6 +42,53 @@ func readOf(s stdio.KeptState) StateRead {
}}
}
// MaxStateReads is the most values one read tool reads from a state in a call, and stateReadParallel how many of them
// are in flight at once. Every value is a round trip from the tool through the runtime to the bus, so a call that read
// every key of a large state one after the other outlasted the call's 30 s (issue 1827: 1164 keys, 20 MB).
const (
MaxStateReads = 1000
stateReadParallel = 16
)
// readValues reads the values of keys, stateReadParallel at a time, in the order of keys (nil for a key with none). It
// stops starting reads after the first error and returns it.
func readValues(keys []string, get func(string) (json.RawMessage, error)) ([]json.RawMessage, error) {
out := make([]json.RawMessage, len(keys))
var (
wg sync.WaitGroup
mu sync.Mutex
fail error
next = make(chan int)
)
for w := 0; w < stateReadParallel && w < len(keys); w++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := range next {
raw, err := get(keys[i])
mu.Lock()
if err != nil && fail == nil {
fail = err
}
mu.Unlock()
out[i] = raw
}
}()
}
for i := range keys {
mu.Lock()
stop := fail != nil
mu.Unlock()
if stop {
break
}
next <- i
}
close(next)
wg.Wait()
return out, fail
}
// stepKey is the key of a step of this node: the id alone, or the whole key <node>.<id> as a step tool answered it.
func (m *Module) stepKey(id string) (string, error) {
id = strings.TrimSpace(id)
@@ -80,27 +130,41 @@ type StepLine struct {
Reason string `json:"reason,omitempty"`
}
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero).
func (m *Module) RecentSteps(limit int) ([]StepLine, error) {
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero). Only
// the node's own keys (<node>.) are listed and the newest max(limit*4, 100) of them read (at most MaxStateReads): a step's
// id is base-36 milliseconds and random digits, so its string order is its start order. cut says older steps were not
// read. The values still order the lines, since an ask step's id carries the ask time.
func (m *Module) RecentSteps(limit int) (lines []StepLine, cut bool, err error) {
if limit <= 0 {
limit = 10
}
if m.ReadSteps.Keys == nil || m.ReadSteps.Get == nil {
return nil, fmt.Errorf("the steps state cannot be read here")
return nil, false, fmt.Errorf("the steps state cannot be read here")
}
start := time.Now()
keys, err := m.ReadSteps.Keys()
if err != nil {
return nil, err
return nil, false, err
}
own := keys[:0:0]
for _, k := range keys {
if strings.HasPrefix(k, m.Node+".") {
if node, _, err := delivery.ParseStepKey(k); err == nil && node == m.Node {
own = append(own, k)
}
}
}
sort.Sort(sort.Reverse(sort.StringSlice(own)))
if n := min(max(limit*4, 100), MaxStateReads); len(own) > n {
own, cut = own[:n], true
}
raws, err := readValues(own, m.ReadSteps.Get)
log.Printf("steps read: %d of %d keys listed read in %v (cut %v)", len(own), len(keys), time.Since(start).Round(time.Millisecond), cut)
if err != nil {
return nil, cut, err
}
var steps []delivery.Step
for _, k := range keys {
if node, _, err := delivery.ParseStepKey(k); err != nil || node != m.Node {
continue
}
raw, err := m.ReadSteps.Get(k)
if err != nil {
return nil, err
}
for _, raw := range raws {
if len(raw) == 0 {
continue
}
@@ -122,7 +186,7 @@ func (m *Module) RecentSteps(limit int) ([]StepLine, error) {
}
out = append(out, l)
}
return out, nil
return out, cut, nil
}
// ReadToolchains is every key of the toolchains state (<node>.<language>) with its stored value.
@@ -130,24 +194,30 @@ func (m *Module) ReadToolchains() (map[string]delivery.Toolchain, error) {
if m.ReadToolchainsState.Keys == nil || m.ReadToolchainsState.Get == nil {
return nil, fmt.Errorf("the toolchains state cannot be read here")
}
start := time.Now()
keys, err := m.ReadToolchainsState.Keys()
if err != nil {
return nil, err
}
cut := len(keys) > MaxStateReads
if cut {
keys = keys[:MaxStateReads] // a node has a key per language: far fewer in practice
}
raws, err := readValues(keys, m.ReadToolchainsState.Get)
log.Printf("toolchains read: %d keys read in %v (cut %v: keys past %d were not read)", len(keys), time.Since(start).Round(time.Millisecond), cut, MaxStateReads)
if err != nil {
return nil, err
}
out := map[string]delivery.Toolchain{}
for _, k := range keys {
raw, err := m.ReadToolchainsState.Get(k)
if err != nil {
return nil, err
}
for i, raw := range raws {
if len(raw) == 0 {
continue
}
t, err := delivery.DecodeToolchain(raw)
if err != nil {
return nil, fmt.Errorf("%s: %w", k, err)
return nil, fmt.Errorf("%s: %w", keys[i], err)
}
out[k] = t
out[keys[i]] = t
}
return out, nil
}
@@ -171,8 +241,13 @@ func (m *Module) ReadTools() []stdio.Tool {
Input: map[string]any{"type": "object", "properties": map[string]any{
"limit": map[string]any{"type": "integer", "description": "how many, at most; 10 when left out"}}},
Run: func(args map[string]any) (any, error) {
steps, err := m.RecentSteps(intArg(args["limit"]))
return map[string]any{"node": m.Node, "steps": steps}, err
steps, cut, err := m.RecentSteps(intArg(args["limit"]))
ans := map[string]any{"node": m.Node, "steps": steps}
if cut {
ans["cut"] = true
ans["note"] = fmt.Sprintf("this node has more steps stored than one call reads; older steps were not read (ask for a larger limit, at most %d reads)", MaxStateReads)
}
return ans, err
}},
{Name: ToolchainsReadTool,
Description: "What the module's toolchains state holds: each key <node>.<language> with the version installed, the " +
@@ -3,8 +3,10 @@ package main
import (
"context"
"encoding/json"
"fmt"
"slices"
"strings"
"sync/atomic"
"testing"
"time"
@@ -145,3 +147,62 @@ func TestReadToolsAreInTheManifestAndNeedNoToolchain(t *testing.T) {
t.Errorf("the read refused: %v", err)
}
}
// 1827: on the live shape (4 nodes, 600 steps each; a steps state of 1164 keys / 20 MB timed out at 30 s) the steps tool
// lists, then reads only this node's keys, in parallel, at most MaxStateReads of them, and says when it cut.
func TestRecentStepsReadsOnlyTheNodesKeysInParallelAndBounded(t *testing.T) {
t0 := time.Date(2026, 10, 25, 9, 0, 0, 0, time.UTC)
for _, row := range []struct {
perNode, limit int
reads int64
steps int
cut bool
}{
{20, 0, 20, 10, false},
{600, 0, 100, 10, true},
{1500, 300, MaxStateReads, 300, true},
} {
var reads, inFlight, maxFlight atomic.Int64
stored := map[string][]byte{}
for _, n := range []string{"ace", "g14", "novox", "shanks"} {
for i := 0; i < row.perNode; i++ {
b, _ := delivery.EncodeStep(delivery.Step{ID: fmt.Sprintf("s%04d", i), Tool: "t", Node: n, State: delivery.StepPassed, Started: t0.Add(time.Duration(i) * time.Second)})
stored[n+".s"+fmt.Sprintf("%04d", i)] = b
}
}
m := &Module{Node: "shanks"}
m.ReadSteps = StateRead{
Keys: func() ([]string, error) {
var k []string
for key := range stored {
k = append(k, key)
}
return k, nil
},
Get: func(key string) (json.RawMessage, error) {
reads.Add(1)
now := inFlight.Add(1)
for o := maxFlight.Load(); now > o && !maxFlight.CompareAndSwap(o, now); o = maxFlight.Load() {
}
time.Sleep(time.Millisecond)
inFlight.Add(-1)
if !strings.HasPrefix(key, "shanks.") {
t.Errorf("read %s, a key of another node", key)
}
return stored[key], nil
},
}
start := time.Now()
lines, cut, err := m.RecentSteps(row.limit)
if err != nil || len(lines) != row.steps || cut != row.cut || reads.Load() != row.reads {
t.Errorf("%d a node: %d steps, cut %v, %d reads, err %v; want %d, %v, %d", row.perNode, len(lines), cut, reads.Load(), err, row.steps, row.cut, row.reads)
}
if row.cut && len(lines) > 0 && lines[0].ID != fmt.Sprintf("s%04d", row.perNode-1) {
t.Errorf("%d a node: newest answered %s", row.perNode, lines[0].ID)
}
if maxFlight.Load() < 2 {
t.Errorf("%d a node: values read one at a time", row.perNode)
}
t.Logf("%d a node: %d reads in %v (one after the other: %v)", row.perNode, reads.Load(), time.Since(start), time.Duration(reads.Load())*time.Millisecond)
}
}
+1 -1
View File
@@ -3,7 +3,7 @@ module meshbuildnode
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024232922-99407c37df95
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026095702-79649c3fa9f4
github.com/distribution/distribution/v3 v3.1.2
github.com/nats-io/nats.go v1.54.0
)
+2 -4
View File
@@ -1,7 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024214154-bc16740feb0d h1:oJREw0ggKqJWJ5DEISSL0i2ZC5XqdXAidxo9NJ74TCc=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024214154-bc16740feb0d/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024232922-99407c37df95 h1:I8zbgGzRjmgYydPTWo8hmaZW/P/g+qGDUfu3E32/pkU=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024232922-99407c37df95/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026095702-79649c3fa9f4 h1:05sVel/UOvz4/g0dxrDOeVUL7BGakVrxPBlqRq02ZhM=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026095702-79649c3fa9f4/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/alicebob/miniredis/v2 v2.35.0 h1:QwLphYqCEAo1eu1TqPRN2jgVMPBweeQcR21jeqDCONI=
github.com/alicebob/miniredis/v2 v2.35.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
+1 -1
View File
@@ -10,7 +10,7 @@
"state": [
"toolchains",
"keys",
"steps"
{"name": "steps", "ttl-seconds": 172800}, "pauses"
],
"claims": [
{
+4 -1
View File
@@ -22,7 +22,10 @@ is gone is read as the state it came to (merged, closed or proposed) when the ga
## When a refused or rejected merge is said (ADR 0505 §4, §5)
`stalled` is what the controller raises as a condition for the operator. The gate's own list (`show`, `deliveries`)
`stalled` is what the gate says to the operator as conditions, itself, on the operator-channel seat (ADR 0437 §9, issue
1275): after every look it raises each line, raises it again when its severity or need changes, and clears each that is
listed no more; its bucket `conditions` keeps which it raised. (The controller's self-check that raised them went in
release 6.) The gate's own list (`show`, `deliveries`)
names every refusal and rejection at once, and the pull request is told at once; only `stalled` waits:
- a merge refused because its trunk moved, whose check the gate asked again, is stalled only once that check answered and
@@ -36,6 +36,15 @@ func TestTheBoundsAreSettingsWithTheirDefaults(t *testing.T) {
if h.refusalFloor() != 10*time.Minute || h.authorBound() != 2*time.Hour {
t.Fatalf("empty file: floor %s, author %s", h.refusalFloor(), h.authorBound())
}
if !h.checkGates() {
t.Fatalf("check-gates-merge unset reads false; the default is true")
}
if s, err = read(`{"check-gates-merge": false}`); err != nil || s.CheckGatesMerge == nil || *s.CheckGatesMerge || h.checkGates() {
t.Fatalf("check-gates-merge false: %+v %v, gates %v", s, err, h.checkGates())
}
if _, err := read(`{"check-gates-merge": "no"}`); err == nil {
t.Fatalf("a check-gates-merge that is not true or false was taken")
}
if _, err := read(`{"author-new-head-minutes": 0}`); err == nil {
t.Fatalf("a bound of zero minutes was taken")
}
@@ -0,0 +1,246 @@
package main
import (
"fmt"
"strings"
"sync"
"testing"
"time"
sdk "git.novox.be/novox/mesh-sdk/go/delivery"
)
// One check per delivery (novox/hq ADR 0548), the gate's part: nothing merges into a repository while a check of other heads
// judges it (§2); an approved merge holds a turn on its repositories (§3); a held merge past its bound is a stalled line, read
// and handed by two tools (§4); an approved delivery whose trunk moved merges on its carried verdict with no second ask (§8).
const otherHead = "c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3"
// approvedLone is a lone delivery into oneRepo whose check passed, approved by the operator (the world approves at once).
func approvedLone(w *world) string {
w.t.Helper()
p := pr(oneRepo, 1, oneHead, "feat/one")
requiresCheck(w, p)
w.h.PullUpdated(p)
putCheckerVerdict(w, "pass", headOf(oneRepo, 1, oneHead))
w.h.CommitStatus(checkStatus(oneRepo, oneHead, "success", oneTrunk))
return IDOf(oneRepo, oneHead)
}
// memTurns is the bucket turns in memory; it logs each write with the merges the fake bridge had made by then.
type memTurns struct {
mu sync.Mutex
m map[string]sdk.Turn
log []string
merges func() int
}
func newMemTurns(w *world) *memTurns {
return &memTurns{m: map[string]sdk.Turn{}, merges: func() int { return len(w.forge.merges) }}
}
func (f *memTurns) All() (map[string]sdk.Turn, error) {
f.mu.Lock()
defer f.mu.Unlock()
out := map[string]sdk.Turn{}
for k, v := range f.m {
out[k] = v
}
return out, nil
}
func (f *memTurns) Put(t sdk.Turn) error {
if _, err := sdk.EncodeTurn(t); err != nil {
return err
}
f.mu.Lock()
defer f.mu.Unlock()
f.m[t.Repository] = t
f.log = append(f.log, fmt.Sprintf("put %s after %d merges", t.Repository, f.merges()))
return nil
}
func (f *memTurns) Delete(repository string) error {
f.mu.Lock()
defer f.mu.Unlock()
delete(f.m, repository)
f.log = append(f.log, fmt.Sprintf("delete %s after %d merges", repository, f.merges()))
return nil
}
func (f *memTurns) said() string { f.mu.Lock(); defer f.mu.Unlock(); return strings.Join(f.log, "; ") }
func TestOneCheckPerDeliveryAtTheGate(t *testing.T) {
for name, run := range map[string]func(t *testing.T){
// §2: a merge into a repository a running verdict names is not written to merges, and is written on that verdict's pass.
"an approved merge waits for a running check on its repository and merges on its end": func(t *testing.T) {
w := newWorld(t)
putCheckerVerdict(w, "running", headOf(oneRepo, 9, otherHead))
id := approvedLone(w)
w.settleAll()
if w.op.count() != 1 || len(w.forge.merges) != 0 {
t.Fatalf("asked %d times, merges %v: want the Approve asked and the merge held", w.op.count(), w.forge.merges)
}
d := w.delivery(id)
if d.Ready == nil || !strings.Contains(d.Ready.Waits, "waits for the check of g") || !strings.Contains(d.Ready.Waits, oneRepo) ||
d.Ready.HeldSince == nil {
t.Fatalf("the merge does not say what it waits for: %+v", d.Ready)
}
// §9: deliveries says what it waits for.
for _, l := range w.h.Deliveries("", "", "", false) {
if l.ID == id && !strings.Contains(l.Waits, "approved; waits for the check of g") {
t.Errorf("deliveries says it waits for %q", l.Waits)
}
}
// §4: past 30 minutes the held merge is a stalled line naming the check it waits for and since when.
for _, l := range w.h.Stalled() {
if l.ID == id {
t.Fatalf("a stalled line within the bound: %+v", l)
}
}
w.later(turnBound + time.Minute)
w.settleAll()
var line *StalledLine
for _, l := range w.h.Stalled() {
if l.ID == id {
line = &l
}
}
if line == nil || line.Bound != "30m0s" || !strings.Contains(line.Says, "waits for the check of g") || !strings.Contains(line.Says, "held since") ||
!strings.Contains(line.Says, "checker_status") || !strings.Contains(line.H2, "delivery_hand_now") {
t.Fatalf("the stalled line of the held merge = %+v", line)
}
putCheckerVerdict(w, "pass", headOf(oneRepo, 9, otherHead))
w.settleAll()
if len(w.forge.merges) != 1 || w.op.count() != 1 {
t.Fatalf("after the check's end: merges %v, asked %d times; want merged with no second ask", w.forge.merges, w.op.count())
}
},
"a running check on another repository holds nothing": func(t *testing.T) {
w := newWorld(t)
putCheckerVerdict(w, "running", headOf(oneRepo2, 9, otherHead))
approvedLone(w)
w.settleAll()
if len(w.forge.merges) != 1 {
t.Fatalf("merges %v: a check of another repository held the merge (%s)", w.forge.merges, w.logs())
}
},
"the delivery's own running verdict does not hold it": func(t *testing.T) {
w := newWorld(t)
w.op.auto = false
id := approvedLone(w)
w.settleAll()
putCheckerVerdict(w, "running", headOf(oneRepo, 1, oneHead)) // its own heads judged again (a carry)
for _, k := range w.op.open() {
w.op.answer(k, OptionApprove)
}
w.settleAll()
if d := w.delivery(id); len(w.forge.merges) != 1 || d.Ready == nil || d.Ready.HeldSince != nil {
t.Fatalf("merges %v, run %+v: its own check held it", w.forge.merges, d.Ready)
}
},
// §3: the turn is written before the verdicts are read for the merge, and deleted after the last merge.
"a lone merge's turn is written before it is handed and deleted once it merged": func(t *testing.T) {
w := newWorld(t)
turns := newMemTurns(w)
w.h.Turns = turns
approvedLone(w)
w.settleAll()
if len(w.forge.merges) != 1 || turns.said() != "put "+oneRepo+" after 0 merges; delete "+oneRepo+" after 1 merges" {
t.Fatalf("merges %v; the turns were %q", w.forge.merges, turns.said())
}
},
"a group's turns stand on every member's repository until its last member merged": func(t *testing.T) {
w := newWorld(t)
turns := newMemTurns(w)
w.h.Turns = turns
a, b := pr(oneRepo, 1, oneHead, "feat/two"), pr(oneRepo2, 2, oneHead2, "feat/two")
requiresCheck(w, a, b)
w.h.PullUpdated(a)
w.h.PullUpdated(b)
w.h.wantForge()
putCheckerVerdict(w, "pass", headOf(oneRepo, 1, oneHead), headOf(oneRepo2, 2, oneHead2))
w.h.CommitStatus(checkStatus(oneRepo, oneHead, "success", oneTrunk))
w.h.CommitStatus(checkStatus(oneRepo2, oneHead2, "success", oneTrunk))
for range 3 {
w.settleAll()
}
want := "put " + oneRepo2 + " after 0 merges; put " + oneRepo + " after 0 merges; delete " + oneRepo2 + " after 2 merges; delete " + oneRepo + " after 2 merges"
if len(w.forge.merges) != 2 || turns.said() != want {
t.Fatalf("merges %v; the turns were %q, want %q", w.forge.merges, turns.said(), want)
}
},
// §8: an approved delivery whose trunk moved goes back to proposed, waits for its carried verdict rather than being
// handed on the old one, and merges on it with no second ask.
"an approved merge whose trunk moved merges on its carried verdict with no second ask": func(t *testing.T) {
w := newWorld(t)
putCheckerVerdict(w, "running", headOf(oneRepo, 9, otherHead)) // holds the approved merge (§2)
id := approvedLone(w)
w.settleAll()
if w.op.count() != 1 || len(w.forge.merges) != 0 {
t.Fatalf("asked %d times, merges %v; want approved and held", w.op.count(), w.forge.merges)
}
w.h.PullMerged(merged(oneRepo, 5, otherHead, oneTrunk2)) // a merge at the forge moves the trunk
putCheckerVerdict(w, "pass", headOf(oneRepo, 9, otherHead))
w.settleAll()
if len(w.forge.merges) != 0 {
t.Fatalf("merges %v: handed on a verdict judged on the old trunk", w.forge.merges)
}
d := w.delivery(id)
if d.Ready == nil || !strings.Contains(d.Ready.Waits, "being carried") {
t.Fatalf("after the trunk moved it waits for %q, want its verdict carried", d.Ready.Waits)
}
// The checker's carried line: the head on the new trunk, carried from the old one.
carried := checkStatus(oneRepo, oneHead, "success", oneTrunk2)
carried.Description = "pass " + oneHead[:12] + " on " + oneTrunk2[:12] + " | carried from " + oneTrunk[:12] + " | unit 0 run, 4 kept"
if head, trunk := checkLine(carried.Description); head != oneHead[:12] || trunk != oneTrunk2[:12] {
t.Fatalf("the carried line reads as %s on %s", head, trunk)
}
if head, trunk := checkLine("failed: a test"); head != "" || trunk != "" {
t.Fatalf("a line that names no commits gave %q %q", head, trunk)
}
putCheckerVerdict(w, "pass", headOf(oneRepo, 1, oneHead))
w.h.CommitStatus(carried)
w.settleAll()
if w.op.count() != 1 {
t.Errorf("the Approve was asked %d times, want once: it stands through the carry", w.op.count())
}
if d := w.delivery(id); d.Check == nil || d.Check.Trunk != oneTrunk2[:12] || len(w.forge.merges) != 1 {
t.Errorf("after the carry: verdict %+v, merges %v; want merged on the carried verdict", d.Check, w.forge.merges)
}
},
// §4: delivery_turns reads every turn with the check it waits for; delivery_hand_now hands a held merge at once on the
// operator's word, and refuses one no check holds.
"the turns are read, and a held merge is handed now on the operator's word": func(t *testing.T) {
w := newWorld(t)
w.h.Turns = newMemTurns(w)
putCheckerVerdict(w, "running", headOf(oneRepo, 9, otherHead))
id := approvedLone(w)
w.settleAll()
read := w.h.TurnsRead()
lines := read["turns"].([]TurnLine)
if len(lines) != 1 || lines[0].Repository != oneRepo || len(lines[0].Turn.Waiting) != 1 || lines[0].Turn.Waiting[0].WaitsFor == "" ||
!strings.Contains(lines[0].Waits["feat/one"].(string), "the check g.") || read["written"] != true {
t.Fatalf("delivery_turns = %+v", read)
}
if _, err := w.h.HandNow(id, ""); err == nil || !strings.Contains(err.Error(), "why") {
t.Errorf("a hand without the operator's why: %v", err)
}
if _, err := w.h.HandNow(IDOf(oneRepo2, oneHead2), "x"); err == nil {
t.Error("an unknown delivery was handed")
}
if _, err := w.h.HandNow(id, "the operator, on telegram: merge it now"); err != nil {
t.Fatal(err)
}
w.settleAll()
if len(w.forge.merges) != 1 || w.delivery(id).Ready.HandNow == "" {
t.Fatalf("merges %v after the hand", w.forge.merges)
}
if _, err := w.h.HandNow(id, "again"); err == nil {
t.Error("a merged delivery was handed again")
}
},
} {
t.Run(name, run)
}
}
@@ -0,0 +1,232 @@
package main
import (
"encoding/json"
"fmt"
"sort"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/asks"
)
// The gate says its own stalled lines to the operator as conditions (novox/hq issue 1275, met again 2026-10-25; ADR 0437
// §9: any module says a condition on the operator-channel seat's one subject, and the router alone chooses the channels).
// Until release 6 the controller's self-check read the `stalled` verb and raised each line as `delivery.<id>.stalled`;
// release 6 deleted that self-check (mesh-controller dc456d56) and nothing read the verb since, so a half-merged group was
// said to no one. Now, after every look, the gate raises each line `stalled` lists, raises it again when its severity or
// its need changes, and clears each it raised that is listed no more. What it raised is kept in its bucket conditions, so a
// gate that restarts clears what ended while it was away.
// ConditionStore keeps the conditions the gate has raised: the condition's key to what it said (conditionSaid).
type ConditionStore interface {
Said() (map[string]string, error)
PutSaid(said map[string]string) error
}
// saidKey is the one key of the bucket conditions.
const saidKey = "said"
// kvConditions is the bucket conditions over the runtime.
type kvConditions struct{}
func (kvConditions) Said() (map[string]string, error) {
e, err := stdio.State("conditions").Get(saidKey)
if err != nil || e == nil {
return map[string]string{}, err
}
said := map[string]string{}
if err := json.Unmarshal(e.Value, &said); err != nil {
return nil, fmt.Errorf("the conditions the gate raised are not readable: %w", err)
}
return said, nil
}
func (kvConditions) PutSaid(said map[string]string) error {
_, err := stdio.State("conditions").Put(saidKey, said)
return err
}
// sayOnSeat says a condition on the operator-channel seat under the gate's own name.
func sayOnSeat(c asks.Condition) error {
subject, err := asks.ConditionSubject("mesh-delivery-gate")
if err != nil {
return err
}
raw, err := asks.EncodeCondition(c)
if err != nil {
return err
}
return stdio.SeatPublish(subject, json.RawMessage(raw), "")
}
// conditionKeyOf is the key of a stalled line's condition: its last part is the condition's kind.
func conditionKeyOf(id string) string { return "mesh-delivery-gate.delivery." + id + ".stalled" }
// conditionSaid is what a raised condition said that, when it changes, raises it again: its severity and what it needs.
// The words of a line hold durations, which change at every look and are no reason to tell the operator again.
func conditionSaid(l StalledLine) string { return severityOfLine(l) + "|" + l.H2 + "|" + l.Needs }
func severityOfLine(l StalledLine) string {
if l.Severity == asks.SeverityUrgent {
return asks.SeverityUrgent
}
return asks.SeverityWarning
}
// conditionOf is a stalled line as the condition the operator is told.
func conditionOf(l StalledLine) asks.Condition {
headline := "stalled: " + l.ID
if strings.HasPrefix(l.Says, "group ") {
group, _, _ := strings.Cut(strings.TrimPrefix(l.Says, "group "), " ")
headline = "group " + group + " is not merged whole"
}
if r := []rune(headline); len(r) > asks.HeadlineLength {
headline = string(r[:asks.HeadlineLength-1]) + "…"
}
needs := "read mesh-delivery.show " + l.ID + " and decide"
return asks.Condition{Key: conditionKeyOf(l.ID), Severity: severityOfLine(l), State: asks.StateRaised,
Subject: asks.Scope{Scope: "delivery", ID: l.ID}, Summary: l.Says, Headline: headline, Needs: needs,
Explanation: asks.NeedsYou + needs + ". " + l.Says + " What ends it: " + l.H2 + ".",
Resolved: "The gate lists " + l.ID + " as stalled no more."}
}
// sayConditions raises and clears the conditions of the stalled lines (see above). Outside the lock; React runs it after
// the look, so it follows every event. A condition that could not be said is said again at the next look, after
// askAgainAfter at the latest.
func (h *Holder) sayConditions() {
if h.Say == nil || h.Conditions == nil {
return
}
if h.said == nil {
said, err := h.Conditions.Said()
if err != nil {
h.Logf("[mesh-delivery] the conditions raised before cannot be read, so none is said yet: %v; asked again in %s", err, askAgainAfter)
h.again("the conditions the gate raised, read")
return
}
h.said = said
}
h.mu.Lock()
if at := h.nextStall(h.Now()); !at.IsZero() {
h.lookAt(at, "a delivery whose stalled line, or its severity, comes due with no event")
}
forgeRead := h.forgeRead
waitingOnForge := map[string]bool{}
for id := range h.unchecked {
waitingOnForge[id] = true
}
h.mu.Unlock()
lines := map[string]StalledLine{}
for _, l := range h.Stalled() {
// One condition for an id: the most severe of its lines.
if o, ok := lines[l.ID]; !ok || (severityOfLine(o) != asks.SeverityUrgent && severityOfLine(l) == asks.SeverityUrgent) {
lines[l.ID] = l
}
}
ids := make([]string, 0, len(lines))
for id := range lines {
ids = append(ids, id)
}
sort.Strings(ids)
changed, failed := false, false
for _, id := range ids {
l, key := lines[id], conditionKeyOf(id)
said := conditionSaid(l)
if h.said[key] == said {
continue
}
if err := h.Say(conditionOf(l)); err != nil {
h.Logf("[mesh-delivery] the condition of %s could not be said: %v", id, err)
failed = true
continue
}
h.Logf("[mesh-delivery] %s is said to the operator as %s: %s", id, severityOfLine(l), l.Says)
h.said[key], changed = said, true
}
// Cleared only on what was read: not before the forge's open pull requests were read once since the gate started (the
// heads waiting on the forge are kept in memory alone), and not while the forge still shows the head open and waiting:
// its line is gone only because its bound runs again from a later first sight, and it comes back with the same words.
var gone []string
for key := range h.said {
id := strings.TrimSuffix(strings.TrimPrefix(key, "mesh-delivery-gate.delivery."), ".stalled")
if _, ok := lines[id]; !ok && forgeRead && !waitingOnForge[id] {
gone = append(gone, key)
}
}
sort.Strings(gone)
for _, key := range gone {
id := strings.TrimSuffix(strings.TrimPrefix(key, "mesh-delivery-gate.delivery."), ".stalled")
c := asks.Condition{Key: key, Subject: asks.Scope{Scope: "delivery", ID: id}, State: asks.StateCleared,
Resolved: "The gate lists " + id + " as stalled no more."}
if err := h.Say(c); err != nil {
h.Logf("[mesh-delivery] the condition of %s could not be cleared: %v", id, err)
failed = true
continue
}
h.Logf("[mesh-delivery] the condition of %s is cleared: it is stalled no more", id)
delete(h.said, key)
changed = true
}
if changed {
if err := h.Conditions.PutSaid(h.said); err != nil {
h.Logf("[mesh-delivery] the conditions raised cannot be kept: %v", err)
}
}
if !failed {
h.sayFailures = 0
return
}
// A seat that keeps refusing is tried again after askAgainAfter, up to sayTries looks in a row; then only at the
// next event, and said once, loudly, in the log (the operator hears nothing while the seat refuses).
h.sayFailures++
switch {
case h.sayFailures < sayTries:
h.again("a condition that could not be said to the operator")
case h.sayFailures == sayTries:
h.Logf("[mesh-delivery] NO CONDITION REACHES THE OPERATOR: the operator-channel seat refused %d looks in a row; "+
"tried again at each event from now on", sayTries)
}
}
// sayTries is how many looks in a row a condition that could not be said is tried again on the clock.
const sayTries = 10
// nextStall is the earliest time after now when a delivery's stalled line comes due, or its severity changes, with nothing
// else happening: a state's bound, the author's bound of a rejected head, a head never announced, a refusal the gate
// answers, and an out-of-date refusal turning urgent (1275/AC6). Zero when none. Under the lock.
func (h *Holder) nextStall(now time.Time) time.Time {
var next time.Time
consider := func(at time.Time) {
if at.After(now) && (next.IsZero() || at.Before(next)) {
next = at
}
}
for _, d := range h.deliveries {
if d.State.Final() {
continue
}
if b, bounded := boundOf(d); bounded && b.For > 0 {
consider(boundSince(d).Add(b.For).Add(time.Second)) // pastBound is strictly after the bound
}
r := d.Ready
if r == nil || r.Merged != nil || d.MergedAs != "" {
continue
}
if d.Base == "" && d.Number != 0 {
consider(r.Seen.Add(headHeardBound))
}
if r.Failed != "" && d.State == Rejected {
consider(d.Since.Add(h.authorBound()))
}
if r.RefusedSince != nil {
consider(r.RefusedSince.Add(h.answeredRefusalBound(d.Repository)))
consider(r.RefusedSince.Add(outOfDateFor))
}
if r.HeldSince != nil {
consider(r.HeldSince.Add(turnBound))
}
}
return next
}
@@ -67,7 +67,9 @@ func (h *Holder) CommitStatus(e ForgeStatusEvent) {
h.takeCheckStatus(e.Owner+"/"+e.Repo, e.SHA, e.State, e.Description)
}
if e.State != "success" {
h.withdrawForStatus(e)
if h.checkGates() {
h.withdrawForStatus(e) // while the check gates no merge, its status is shown and takes back no ask
}
return
}
repository := strings.ToLower(e.Owner + "/" + e.Repo)
@@ -63,7 +63,8 @@ func (h *Holder) unmergedLive(g *Group) []*Delivery {
// groupMergeable reads every member's required statuses at the head the Approve covers, and says whether all are a success.
// When one is not, or cannot be read, nothing is handed to the bridge: the group is held, said urgent (ADR 0441 §2, §4), and
// the target's turn is given back so the next event that decides whether it may merge asks it again. Outside the lock.
// the target's turn is given back so the next event that decides whether it may merge asks it again. A read that could not
// be made is a failed ask, tried again after askAgainAfter. Outside the lock.
func (h *Holder) groupMergeable(gid, target string, reads []memberRead) bool {
release := func() {
if d := h.deliveries[target]; d != nil && d.Ready != nil {
@@ -98,8 +99,9 @@ func (h *Holder) groupMergeable(gid, target string, reads []memberRead) bool {
g.Hold = &GroupHold{Kind: "blocked", Member: m.id, Says: says, Since: since}
h.keepGroup(g)
}
// No timer: the status turning success is an event on the forge (CommitStatus, mergeAgainNow), and it asks the
// merge again at once (issue 1275: a pass that stops part way resumes on an event).
release()
h.again("a group's merge held for a required status")
h.mu.Unlock()
return false
}
@@ -112,7 +114,7 @@ func (h *Holder) groupMergeable(gid, target string, reads []memberRead) bool {
return true
}
// groupLines are the groups that are held or half merged, as urgent stalled lines the controller raises as conditions:
// groupLines are the groups that are held or half merged, as urgent stalled lines the gate says as conditions (conditions.go):
// one for each, on its first member not merged, until every member is merged or a person decides (issue 1275). A member
// the merge tool refused is said by readyLines, with its group; this adds a group held for a required status, and a
// group half merged with no refusal. Under the lock.
@@ -141,12 +143,16 @@ func (h *Holder) groupLines(now time.Time) []StalledLine {
switch {
// A held member the merge tool refused is said by readyLines already: one line for an id, not two.
case g.Hold != nil && g.Hold.Kind == "blocked" && !running && !(heldOn != nil && heldOn.Ready != nil && heldOn.Ready.RefusedSince != nil):
stands := "is approved and nothing is merged"
if merged > 0 {
stands = "is half merged" // a member merged before the pass stopped (issue 1275)
}
out = append(out, StalledLine{ID: g.Hold.Member, Number: numberOf(h.deliveries[g.Hold.Member]), State: first.State,
For: now.Sub(g.Hold.Since).Round(time.Second).String(), Bound: "0s", Severity: "urgent",
H2: "none: the required status turns success, which asks the merge again — or the operator's decision",
Says: fmt.Sprintf("group %s is approved and nothing is merged (merged %d of %d): %s waits on a required status "+
Says: fmt.Sprintf("group %s %s (merged %d of %d): %s waits on a required status "+
"at its head: %s. Needed: that status turns success (its check asked again), or the operator decides.",
g.ID, merged, len(h.membersOf(g)), g.Hold.Member, g.Hold.Says)})
g.ID, stands, merged, len(h.membersOf(g)), g.Hold.Member, g.Hold.Says)})
case merged > 0 && !refused && !running:
out = append(out, StalledLine{ID: first.ID, Number: first.Number, State: first.State,
For: now.Sub(g.Updated).Round(time.Second).String(), Bound: "0s", Severity: "urgent",
@@ -21,25 +21,44 @@ const (
// approvedGroup is a group of two pull requests on one branch, checks green, composed green and the operator's Approve
// asked and not yet answered (w.op.auto is off).
func approvedGroup(t *testing.T) *world {
return approvedGroupOf(t, member{gCtl, 1, gHc, "internal/a.go"}, member{gCat, 2, gHk, "modules/app/module.json"})
}
// member is a pull request of a group in feat/g: its repository, number, head and the paths it changes.
type member struct {
repo string
number int
head string
paths string
}
// approvedGroupOf is a group feat/g of these members, each checked, composed together and asked once.
func approvedGroupOf(t *testing.T, ms ...member) *world {
t.Helper()
w := newWorld(t)
w.op.auto = false
w.h.PullUpdated(pr(gCtl, 1, gHc, "feat/g", "internal/a.go"))
w.h.PullUpdated(pr(gCat, 2, gHk, "feat/g", "modules/app/module.json"))
w.h.Checked(judged(gCtl, 1, gHc, "pass", gTrunk0))
w.h.Checked(judged(gCat, 2, gHk, "pass", gTrunk0))
for _, m := range ms {
w.h.PullUpdated(pr(m.repo, m.number, m.head, "feat/g", strings.Split(m.paths, ",")...))
}
for _, m := range ms {
w.h.Checked(judged(m.repo, m.number, m.head, "pass", gTrunk0))
}
w.settleAll()
g := w.h.groups["feat/g"]
if g == nil || g.Check == nil {
t.Fatalf("no composed check was asked: %+v", g)
}
w.h.Checked(CheckedEvent{Group: "feat/g", ID: g.Check.Asked, Verdict: "pass", Summary: "composed",
Members: []struct {
c := CheckedEvent{Group: "feat/g", ID: g.Check.Asked, Verdict: "pass", Summary: "composed"}
for _, m := range ms {
owner, repo, _ := strings.Cut(m.repo, "/")
c.Members = append(c.Members, struct {
Owner string `json:"owner"`
Repo string `json:"repo"`
Commit string `json:"commit"`
Trunk string `json:"trunk,omitempty"`
}{{Owner: "novox", Repo: "mesh-controller", Commit: gHc, Trunk: gTrunk0[:12]},
{Owner: "novox", Repo: "mesh-catalog", Commit: gHk, Trunk: gTrunk0[:12]}}})
}{Owner: owner, Repo: repo, Commit: m.head, Trunk: gTrunk0[:12]})
}
w.h.Checked(c)
w.settleAll()
if w.op.count() != 1 || len(w.op.open()) != 1 {
t.Fatalf("the green group is asked once: %d asks, open %v", w.op.count(), w.op.open())
@@ -16,6 +16,12 @@ func verdictMoving(repo string, number int, sha string, _ ...string) CheckedEven
return verdict(repo, number, sha, "pass")
}
// ungated is the setting check-gates-merge false: mesh/check gates no merge (the operator, 2026-10-25 22:56).
func ungated(w *world) {
off := false
w.h.Settings = func() Settings { return Settings{CheckGatesMerge: &off} }
}
func statusOn(w *world, repo, commit string) string {
w.forge.mu.Lock()
defer w.forge.mu.Unlock()
@@ -10,6 +10,9 @@ import (
"strings"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
sdk "git.novox.be/novox/mesh-sdk/go/delivery"
)
// The holder of the mesh-delivery seat (novox/hq ADR 0239, to-be 47): the one writer of deliveries and
@@ -37,6 +40,15 @@ type Holder struct {
Owed OwedStore
// Pulls keeps the pull requests' keys on the bus (pullkey.go); nil keeps none.
Pulls PullKeys
// Say says a condition to the operator, and Conditions keeps which the gate raised (conditions.go); nil says none.
Say func(c asks.Condition) error
Conditions ConditionStore
// said is what Conditions keeps, read at the first look; sayFailures the looks in a row a condition could not be
// said. Touched by React alone.
said map[string]string
sayFailures int
// forgeRead is whether the forge's open pull requests were read since this owner started (SeeOpenPulls).
forgeRead bool
// cancels are asks to take back, gathered under the lock and run with the other asks of the look.
cancels []func()
@@ -64,8 +76,14 @@ type Holder struct {
// unchecked are the open pull requests' heads seen without a merge check (unchecked.go).
unchecked map[string]unchecked
// checker are the checker's verdicts by the hash of the heads they judged, as its verdicts bucket holds them
// (onestatus.go, issue 1302).
checker map[string]checkerVerdict
// (onestatus.go, issue 1302); runningSince when this owner first saw each running (ADR 0548 §2, turns.go).
checker map[string]checkerVerdict
runningSince map[string]time.Time
// Turns is the gate's bucket turns (turns.go, ADR 0548 §3); nil keeps none. turnsHeld are the turns as written there,
// by repository, read once at the first look (turnsLoaded).
Turns TurnStore
turnsHeld map[string]sdk.Turn
turnsLoaded bool
// tried counts the owed effects Flush tried since this owner started, by how each ended (novox/hq issue 407).
tried owedTries
// forgeWanted is whether the forge's open pull requests are to be read at the next React: a check may have
@@ -115,6 +133,7 @@ func (h *Holder) init() {
h.lostRead = map[string]time.Time{}
h.unchecked = map[string]unchecked{}
h.checker = map[string]checkerVerdict{}
h.runningSince, h.turnsHeld = map[string]time.Time{}, map[string]sdk.Turn{}
h.started = h.Now()
}
@@ -250,7 +269,7 @@ func (h *Holder) keepGroup(g *Group) {
// settleAndKeep moves a delivery as far as its facts allow, and keeps it.
func (h *Holder) settleAndKeep(d *Delivery) {
ts := settle(d, Facts{Now: h.Now()})
ts := settle(d, h.factsAt(h.Now()))
h.moved(d, ts)
h.closeGroupOf(d)
}
@@ -367,7 +386,7 @@ func (h *Holder) pullUpdated(p PullEvent) (isNew bool) {
// the gate's status on that very commit. A decided one is taken; a pending or errored one is asked again.
if v := decidedOnHead(p.HeadChecks, now); v != nil {
d.Check = v
ts = append(ts, settle(d, Facts{Now: now})...)
ts = append(ts, settle(d, h.factsAt(now))...)
}
h.moved(d, ts)
if !h.takePlaceOf(replaced, d) {
@@ -635,7 +654,12 @@ func (h *Holder) leaveGroup(d *Delivery) {
// PullMerged takes a merge: the commit it landed as, and the delivery it was — made from the forge's word
// when this owner never heard its head (merged before it held the seat).
func (h *Holder) PullMerged(p PullEvent) {
func (h *Holder) PullMerged(p PullEvent) { h.pullMerged(p, false) }
// pullMerged takes a merge. late is a merge learnt after the fact (settleGone): its own delivery is merged and its group
// closed, but no other delivery's trunk moves (the trunk has long moved past it, by merges heard as they happened),
// its time is the forge's or none (never now), and it never moves source.main (LearntLate).
func (h *Holder) pullMerged(p PullEvent, late bool) {
if p.MergeCommit == "" {
h.Refuse(errors.New("a merge announced with no merge commit; nothing published"))
return
@@ -684,7 +708,7 @@ func (h *Holder) PullMerged(p PullEvent) {
}
d, ts = fresh, []Transition{t}
if d.Check != nil {
ts = append(ts, settle(d, Facts{Now: now})...)
ts = append(ts, settle(d, h.factsAt(now))...)
}
} else if d == nil {
head := p.HeadSHA
@@ -710,16 +734,31 @@ func (h *Holder) PullMerged(p PullEvent) {
d.MergedAs = p.MergeCommit
if t, err := time.Parse(time.RFC3339, p.MergedAt); err == nil {
d.MergedAt = t.UTC()
} else {
if late && now.Sub(d.MergedAt) < replayLate {
// Read from the forge's open pull requests seconds after the merge, before its event arrived (mesh-delivery#51,
// 2026-10-25 21:43: merged 21:43:11, learnt late 21:43:14): the trunk has not moved past it, so it is taken as a
// merge heard as it happened. Taken late, it moved no trunk and its group's finish left it out of source.main,
// which kept naming the trunk before it, and the checker judged every head on that.
late = false
}
} else if !late {
d.MergedAt = now
}
ts = append(ts, settle(d, Facts{Now: now})...)
if late {
d.LearntLate, d.InSource = true, true
if d.Ready != nil {
d.Ready.SourceMoved = true
}
}
ts = append(ts, settle(d, h.factsAt(now))...)
if d.State == Held && d.HeldWhy == "" {
d.HeldWhy = "merged without a passing check: only a person decides what follows"
}
h.moved(d, ts)
h.closeGroupOf(d)
h.trunkMoved(d, p.Base, p.MergeCommit, now)
if !late {
h.trunkMoved(d, p.Base, p.MergeCommit, now)
}
}
// trunkMoved takes a merge into a trunk as that trunk's move (novox/hq ADR 0323 §7): every other open delivery into
@@ -921,7 +960,7 @@ func (h *Holder) askChecks(now time.Time) func() {
}
}
// membersOf are a group's current members, by id order.
// membersOf are a group's current members, in the order they merge (mergeRank, then id).
func (h *Holder) membersOf(g *Group) []*Delivery {
var out []*Delivery
for _, id := range g.Members {
@@ -929,9 +968,39 @@ func (h *Holder) membersOf(g *Group) []*Delivery {
out = append(out, d)
}
}
sort.SliceStable(out, func(i, j int) bool {
if ri, rj := mergeRank(out[i]), mergeRank(out[j]); ri != rj {
return ri < rj
}
return out[i].ID < out[j].ID
})
return out
}
// SDKRepository is the repository of mesh-sdk, the library the other repositories' modules pin by commit (novox/hq ADR 0482 §3).
const SDKRepository = "novox/mesh-sdk"
// mergeRank is a member's place in its group's merge (novox/hq issue 1275): a member another member pins merges before it, so
// that the trunk never holds a pin of a commit not on its own trunk. The gate knows a pin by the paths a pull request changes:
// mesh-sdk's member goes first (the members of the other repositories pin it, ADR 0482 §3); then the members that change no
// go.mod or go.sum (they pin nothing new); then those that do (they pin another member's commit). The id orders a tie. The
// module graph is the fuller source (which module requires which), and is not the gate's to read: this is what its own
// deliveries say.
func mergeRank(d *Delivery) int {
if strings.EqualFold(d.Repository, SDKRepository) {
return 0
}
if d.PathsTruncated || len(d.Paths) == 0 {
return 2 // its paths are not all known: it may pin, so it goes after those that do not
}
for _, p := range d.Paths {
if base := p[strings.LastIndex(p, "/")+1:]; base == "go.mod" || base == "go.sum" {
return 2
}
}
return 1
}
// planGroup is what a group needs asked, and its turns.
func (h *Holder) planGroup(g *Group, members []*Delivery, now time.Time) []func() {
var asks []func()
@@ -956,7 +1025,7 @@ func (h *Holder) planGroup(g *Group, members []*Delivery, now time.Time) []func(
if held {
status = "pending" // no Approve exists while a member waits for its review (novox/hq ADR 0496 §2)
}
if state == "merging" && !(g.Check != nil && (g.Check.Verdict == "pass" || g.Check.Verdict == "warning") &&
if state == GroupMerging && !(g.Check != nil && (g.Check.Verdict == "pass" || g.Check.Verdict == "warning") &&
sameMembers(g.Check.Commits, commitsOf(members)) && g.Check.OutOfDate == "") {
// Success there lets the merge tool merge its members: only for a composed check that passed for these heads.
status = "pending"
@@ -980,6 +1049,23 @@ func (h *Holder) planGroup(g *Group, members []*Delivery, now time.Time) []func(
return asks
}
// The words GroupState says a group's state with (ADR 0539 §17). Code that decides on a group's state compares with these,
// never with a literal: a word GroupState no longer says ("delivering", retired in release 6) left the second member of
// every approved group waiting for ever (novox/hq issue 1275, met again 2026-10-25). groupWords lists them all, and a test
// holds every decision on a group's state to that list.
const (
GroupChecking = "checking"
GroupRejected = "rejected"
GroupReady = "ready"
GroupMerging = "merging"
GroupMerged = "merged"
GroupHeld = "held"
GroupClosed = "closed"
)
// groupWords are every word GroupState says.
var groupWords = []string{GroupChecking, GroupRejected, GroupReady, GroupMerging, GroupMerged, GroupHeld, GroupClosed}
// GroupState is a group's state, derived from its members and its composed check — never set apart (novox/hq
// ADR 0239 decision 4, cut to the merge's states by ADR 0539 §17): checking, rejected, ready, merging (some members
// merged), merged, held and closed. Nothing after the merge is derived here: whether a merged change reached the machines is
@@ -992,36 +1078,36 @@ func GroupState(g *Group, members []*Delivery) (string, string) {
n := len(members)
switch {
case count[Held] > 0 && g.Closed:
return "held", "a member is held: " + heldWords(members)
return GroupHeld, "a member is held: " + heldWords(members)
case count[Closed] > 0 && g.Closed:
return "closed", "a member was closed before the group merged"
return GroupClosed, "a member was closed before the group merged"
case n > 0 && count[Merged] == n:
return "merged", "every member is merged"
return GroupMerged, "every member is merged"
case g.Closed && g.Check != nil && (g.Check.OutOfDate != "" || g.Check.Verdict == "" ||
!sameMembers(g.Check.Commits, commitsOf(members))):
// A member rebased or pushed once the group was merging, or its composed check out of date: it is composed
// again for these heads, and the group is checking until then, never held (novox/hq ADR 0323 §6, issue 406).
if g.Check.OutOfDate != "" {
return "checking", g.Check.OutOfDate
return GroupChecking, g.Check.OutOfDate
}
return "checking", "composed again for its members' new heads"
return GroupChecking, "composed again for its members' new heads"
case g.Closed && g.Check != nil && g.Check.Verdict != "pass" && g.Check.Verdict != "warning":
// Composed again for new heads once it was merging, and those do not compose: its members wait (issue 902).
return "rejected", "composed together: " + g.Check.Summary
return GroupRejected, "composed together: " + g.Check.Summary
case g.Closed:
return "merging", fmt.Sprintf("%d of %d merged", count[Merged], n)
return GroupMerging, fmt.Sprintf("%d of %d merged", count[Merged], n)
case count[Rejected] > 0:
return "rejected", "a member's own check failed"
return GroupRejected, "a member's own check failed"
case g.Check != nil && g.Check.OutOfDate != "":
// Out of date counts as outstanding (ADR 0239 §18, novox/hq ADR 0323 §7): checking, saying why.
return "checking", g.Check.OutOfDate
return GroupChecking, g.Check.OutOfDate
case g.Check != nil && g.Check.Verdict != "" && g.Check.Verdict != "pass" && g.Check.Verdict != "warning":
return "rejected", "composed together: " + g.Check.Summary
return GroupRejected, "composed together: " + g.Check.Summary
case count[Ready] == n && g.Check != nil && sameMembers(g.Check.Commits, commitsOf(members)) &&
(g.Check.Verdict == "pass" || g.Check.Verdict == "warning"):
return "ready", "every member ready, and composed together they pass"
return GroupReady, "every member ready, and composed together they pass"
}
return "checking", fmt.Sprintf("%d of %d member(s) ready", count[Ready], n)
return GroupChecking, fmt.Sprintf("%d of %d member(s) ready", count[Ready], n)
}
// heldWords says the held members and why.
@@ -1040,7 +1126,7 @@ func heldWords(members []*Delivery) string {
// says its mesh/delivery-group is pending, not a success. Under the lock.
func (h *Holder) groupState(g *Group, members []*Delivery) (state, why string, held bool) {
state, why = GroupState(g, members)
if state != "ready" || g.Closed {
if state != GroupReady || g.Closed {
return state, why, false
}
var waiting []string
@@ -1063,11 +1149,11 @@ func (h *Holder) groupState(g *Group, members []*Delivery) (state, why string, h
// heads (novox/hq ADR 0382), and a pending one there would hold them for ever (novox/hq issue 902).
func groupStatusState(s string) string {
switch s {
case "ready", "merging", "merged":
case GroupReady, GroupMerging, GroupMerged:
return "success"
case "rejected":
case GroupRejected:
return "failure"
case "held", "closed":
case GroupHeld, GroupClosed:
return "error"
}
return "pending"
@@ -2,9 +2,14 @@ package main
import (
"errors"
"fmt"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
sdk "git.novox.be/novox/mesh-sdk/go/delivery"
)
const time30s = 31 * time.Second
@@ -75,24 +80,6 @@ func TestAnApprovedGroupWithAMemberWhoseRequiredStatusIsNotASuccessMergesNothing
}
}
// 1275/AC3: the merge tool refuses the first member in order; the second is not merged, and the group is said urgent,
// naming the group, the refused member and the forge's words.
func TestARefusedFirstMemberStopsTheGroupAndIsSaidUrgent(t *testing.T) {
w := approvedGroup(t)
allGreen(w)
w.forge.refuseRepo = map[string]error{gCat: refusedWords("novox/mesh-catalog#2")}
w.op.answer(w.op.open()[0], OptionApprove)
w.settleAll()
if len(w.forge.merges) != 1 || !strings.Contains(w.forge.merges[0], "mesh-catalog#2") {
t.Fatalf("wanted only the first member asked: %v", w.forge.merges)
}
lines := urgentLines(w)
if len(lines) != 1 || lines[0].ID != IDOf(gCat, gHk) || !strings.Contains(lines[0].Says, "feat/g") ||
!strings.Contains(lines[0].Says, "mesh/repo-check (missing)") || !strings.Contains(lines[0].Says, "Needed") {
t.Fatalf("not said as one urgent line: %+v", lines)
}
}
// 1275/AC4: a group half merged (the second member refused after the first merged) stays raised, pass after pass, until
// the second is merged; then it goes.
func TestAHalfMergedGroupStaysRaisedUntilEveryMemberIsMerged(t *testing.T) {
@@ -344,3 +331,440 @@ func TestASeverityOnceUrgentNeverStepsDownToAWarning(t *testing.T) {
t.Fatalf("severity %v after an out of date refusal that followed an urgent one; wanted urgent", got)
}
}
// fakeConditions is the gate's bucket conditions and the operator-channel seat's subject for conditions: what it keeps,
// and every condition said, in order.
type fakeConditions struct {
mu sync.Mutex
kept map[string]string
told []asks.Condition
}
func (f *fakeConditions) Said() (map[string]string, error) {
f.mu.Lock()
defer f.mu.Unlock()
out := map[string]string{}
for k, v := range f.kept {
out[k] = v
}
return out, nil
}
func (f *fakeConditions) PutSaid(said map[string]string) error {
f.mu.Lock()
defer f.mu.Unlock()
f.kept = map[string]string{}
for k, v := range said {
f.kept[k] = v
}
return nil
}
func (f *fakeConditions) say(c asks.Condition) error {
if _, err := asks.EncodeCondition(c); err != nil {
return err // what the router would refuse
}
f.mu.Lock()
defer f.mu.Unlock()
f.told = append(f.told, c)
return nil
}
// last is the newest condition said for a delivery id, or nil.
func (f *fakeConditions) last(id string) *asks.Condition {
f.mu.Lock()
defer f.mu.Unlock()
for i := len(f.told) - 1; i >= 0; i-- {
if f.told[i].Subject.ID == id {
c := f.told[i]
return &c
}
}
return nil
}
// hearConditions has the gate say its conditions into a fake seat and bucket.
func hearConditions(w *world) *fakeConditions {
f := &fakeConditions{kept: map[string]string{}}
w.h.Say, w.h.Conditions = f.say, f
return f
}
// handedInOrder answers each member the gate hands the forge bridge as the forge does on 2026-10-25: the forge's word that
// it merged (gitea.pull.merged) first, the bridge's word after; or the bridge's first, as forgeFirst says. It returns the
// repositories in the order they were handed.
func handedInOrder(w *world, forgeFirst bool) []string {
var handed []string
numbers := map[string]int{}
for i := 0; i < 4; i++ {
var next *sdk.MergePull
for _, p := range w.merges.last("feat/g").Pulls {
if p.State == sdk.MergeMerging {
p := p
next = &p
}
}
if next == nil || numbers[next.Repository] > 0 {
break
}
handed = append(handed, next.Repository)
numbers[next.Repository] = next.Number
mergeCommit := strings.Repeat(string(rune('a'+i)), 40)
forge := func() { w.h.PullMerged(merged(next.Repository, next.Number, next.Head.Head, mergeCommit)) }
bridge := func() { bridgeMerged(w, next.Repository, next.Number, next.Head.Head, mergeCommit) }
if forgeFirst {
forge()
w.settleAll() // ten seconds pass between the two words on 2026-10-25: the gate looks in between
bridge()
} else {
bridge()
w.settleAll()
forge()
}
w.settleAll()
}
return handed
}
// Issue 1275, met again 2026-10-25 (group fix/puller-wakes-on-capacity): every member of an approved group is handed to
// the forge bridge, in the group's order, the next on the event that the one before merged; a group whose pass stops is
// said to the operator as a condition, which is cleared once it is whole; and every word a group's state is said in is
// one the merge decides on.
func TestAnApprovedGroupIsHandedWholeInItsOrderAndAStopIsSaid(t *testing.T) {
const (
sdkRepo, hS = "novox/mesh-sdk", "a8fbabbfc37369f9d1f6977c72e5523aff3d494c"
catRepo, hK = "novox/mesh-catalog", "23c827e7910c233608af4e4a8056cbc362dcb755"
ctlRepo, hC = "novox/mesh-controller", "c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0"
)
sdkMember := member{sdkRepo, 80, hS, "go/stdio.go,go/buildstep/pull.go"}
catMember := member{catRepo, 479, hK, "modules/mesh-build-go/asks.go,modules/mesh-build-go/go.mod,modules/mesh-build-go/go.sum"}
ctlMember := member{ctlRepo, 7, hC, "cmd/mesh-controller/main.go"}
inOrder := func(members []member, forgeFirst bool, want ...string) func(t *testing.T) {
return func(t *testing.T) {
w := approvedGroupOf(t, members...)
w.merges.manual = true
allGreenFor(w, members...)
w.op.answer(w.op.open()[0], OptionApprove)
w.settleAll()
if got := handedInOrder(w, forgeFirst); strings.Join(got, " ") != strings.Join(want, " ") {
t.Fatalf("handed %v, wanted %v, every member in the group's order", got, want)
}
if g := w.h.groups["feat/g"]; g == nil {
t.Fatal("the group is gone")
} else if state, why := GroupState(g, w.h.membersOf(g)); state != GroupMerged {
t.Fatalf("the group is %s (%s), not merged whole", state, why)
}
if w.source.main == nil || len(w.source.main.Heads) != len(want) {
t.Fatalf("source.main does not hold the group as one revision: %+v", w.source.main)
}
}
}
cases := []struct {
name string
run func(t *testing.T)
}{
// 1275/AC3 (2026-10-25): the case of the day. The catalogue member pins the sdk member's commit; the forge's word
// of a merge arrives before the bridge's. Before the fix the catalogue member went first, and the sdk member was
// never handed: its group read merging, a word the turn to merge did not know.
{"sdk first, then the member that pins it, the forge's word first", inOrder([]member{catMember, sdkMember}, true, sdkRepo, catRepo)},
{"sdk first, then the member that pins it, the bridge's word first", inOrder([]member{sdkMember, catMember}, false, sdkRepo, catRepo)},
{"a member that pins nothing before one that changes a go.mod", inOrder([]member{catMember, ctlMember}, true, ctlRepo, catRepo)},
{"three members", inOrder([]member{ctlMember, catMember, sdkMember}, true, sdkRepo, ctlRepo, catRepo)},
// 1275/AC3: the merge tool refuses the first member in order; the second is not merged, and the group is said
// urgent, naming the group, the refused member and the forge's words, as a stalled line and to the operator.
{"a refused first member stops the group and is said urgent", func(t *testing.T) {
w := approvedGroup(t)
told := hearConditions(w)
allGreen(w)
w.forge.refuseRepo = map[string]error{gCat: refusedWords("novox/mesh-catalog#2")}
w.op.answer(w.op.open()[0], OptionApprove)
w.settleAll()
if len(w.forge.merges) != 1 || !strings.Contains(w.forge.merges[0], "mesh-catalog#2") {
t.Fatalf("wanted only the first member asked: %v", w.forge.merges)
}
lines := urgentLines(w)
if len(lines) != 1 || lines[0].ID != IDOf(gCat, gHk) || !strings.Contains(lines[0].Says, "feat/g") ||
!strings.Contains(lines[0].Says, "mesh/repo-check (missing)") || !strings.Contains(lines[0].Says, "Needed") {
t.Fatalf("not said as one urgent line: %+v", lines)
}
if c := told.last(IDOf(gCat, gHk)); c == nil || c.State != asks.StateRaised || c.Severity != asks.SeverityUrgent ||
!strings.Contains(c.Summary, "feat/g") {
t.Fatalf("the operator was not told, urgent: %+v", c)
}
}},
// 1275/AC4: a group half merged (its pass stopped after the first member) raises one urgent condition, said once
// however many looks follow, and cleared once the group is whole.
{"a half-merged group is raised once and cleared once whole", func(t *testing.T) {
w := approvedGroup(t)
told := hearConditions(w)
allGreen(w)
w.merges.manual = true
w.op.answer(w.op.open()[0], OptionApprove)
w.settleAll()
w.h.PullMerged(merged(gCat, 2, gHk, gT1))
bridgeMerged(w, gCat, 2, gHk, gT1)
setChecks(w, gCtl, gHc, "success", "") // the second member's status went missing: the pass stops at it
w.settleAll()
for range 3 {
w.later(time30s)
w.settleAll()
}
id := IDOf(gCtl, gHc)
raised := 0
for _, c := range told.told {
if c.Subject.ID == id && c.State == asks.StateRaised {
raised++
}
}
if c := told.last(id); raised != 1 || c == nil || c.Severity != asks.SeverityUrgent || !strings.Contains(c.Summary, "feat/g") ||
!strings.Contains(c.Summary, "half merged (merged 1 of 2)") {
t.Fatalf("raised %d times, last %+v; wanted once, urgent, naming the group half merged", raised, c)
}
setChecks(w, gCtl, gHc, "success", "success")
w.h.CommitStatus(ForgeStatusEvent{Owner: "novox", Repo: "mesh-controller", SHA: gHc, Context: repoCheck, State: "success"})
w.settleAll() // the status is an event: the pass goes on with no timer
handed := false
for _, p := range w.merges.last("feat/g").Pulls {
handed = handed || (p.Repository == gCtl && p.State == sdk.MergeMerging)
}
if !handed {
t.Fatalf("the status turned success and the second member was not handed: %+v", w.merges.last("feat/g"))
}
w.h.PullMerged(merged(gCtl, 1, gHc, gT1))
bridgeMerged(w, gCtl, 1, gHc, gT1)
w.settleAll()
if c := told.last(id); c == nil || c.State != asks.StateCleared {
t.Fatalf("the group is whole and its condition is not cleared: %+v", c)
}
}},
// A condition the gate raised before it restarted, whose delivery stalls no more, is cleared at the first look.
{"a condition kept from before a restart is cleared", func(t *testing.T) {
w := newWorld(t)
told := hearConditions(w)
told.kept[conditionKeyOf("novox/app@0123456789ab")] = "urgent||"
w.settleAll()
if c := told.last("novox/app@0123456789ab"); c != nil {
t.Fatalf("cleared before the forge's open pull requests were read: %+v", c)
}
w.h.wantForge()
w.settleAll()
if c := told.last("novox/app@0123456789ab"); c == nil || c.State != asks.StateCleared || len(told.kept) != 0 {
t.Fatalf("not cleared: %+v, kept %v", c, told.kept)
}
}},
// Review round 1, blocker 1: a pull request closed or merged on the forge with no event heard (de-spiegel #4,
// closed 2026-10-17, ready here eight days later) settles at the next complete read of the open list, and no
// condition is raised for it; a merge learnt so does not move source.main.
{"a pull request gone from the open list settles and raises nothing", func(t *testing.T) {
for _, merged := range []bool{false, true} {
w := newWorld(t)
w.op.auto = false
const repo, head = "novox/de-spiegel", "fdcbaefb3123fdcbaefb3123fdcbaefb3123fdcb"
w.h.PullUpdated(pr(repo, 4, head, "feat/x", "src/a.go"))
w.h.Checked(judgedAt(repo, 4, head, "pass", "t1t1t1t1t1t1"))
w.settleAll()
w.forge.refuse = errors.New("novox/de-spiegel#4 merges into main, which no protection rule requiring the checks guards")
w.op.answer(w.op.open()[0], OptionApprove)
w.settleAll()
id := IDOf(repo, head)
if len(w.h.Stalled()) == 0 {
t.Fatalf("the refused head is not stalled to begin with")
}
p := ForgePull{Number: 4, State: "closed", HeadSHA: head, Base: "main"}
if merged {
p.Merged, p.MergeCommit = true, strings.Repeat("e", 40)
}
w.forge.pulls[repo+"#4"] = p
w.forge.open = []OpenPull{{Owner: "novox", Repo: "other", Number: 1, Base: "main", HeadSHA: "abab"}}
told := hearConditions(w)
moves := len(w.source.moves)
w.h.wantForge()
w.settleAll()
want := Closed
if merged {
want = Merged
}
if st := w.state(id); st != want {
t.Fatalf("merged %v: the delivery is %s, wanted %s", merged, st, want)
}
if c := told.last(id); c != nil {
t.Fatalf("merged %v: a condition was said for it: %+v", merged, c)
}
if len(w.source.moves) != moves {
t.Fatalf("merged %v: source.main moved for a merge learnt by its absence: %+v", merged, w.source.moves)
}
}
}},
// Review round 2: a merge learnt after the fact moves no other delivery's trunk (an approved group of the same
// repository keeps its trunk, its state and its composed check), takes the forge's time, and a catch-up after it
// does not move source.main.
{"a merge learnt late moves no trunk and no source", func(t *testing.T) {
w := approvedGroup(t)
allGreen(w)
w.merges.manual = true
w.op.answer(w.op.open()[0], OptionApprove)
w.settleAll()
const lateHead, lateMerge = "9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a", "8b8b8b8b8b8b8b8b8b8b8b8b8b8b8b8b8b8b8b8b"
w.h.PullUpdated(pr(gCat, 9, lateHead, "fix/old", "modules/app/x.go"))
w.h.Checked(judged(gCat, 9, lateHead, "pass", gTrunk0))
w.settleAll()
late := IDOf(gCat, lateHead)
trunkOf := func() map[string]string {
out := map[string]string{}
for _, id := range []string{IDOf(gCat, gHk), IDOf(gCtl, gHc)} {
out[id] = w.delivery(id).TrunkNow + " " + string(w.delivery(id).State)
}
return out
}
before, moves := trunkOf(), len(w.source.moves)
w.forge.pulls[gCat+"#9"] = ForgePull{Number: 9, State: "closed", Merged: true, MergeCommit: lateMerge,
MergedAt: "2026-10-01T09:00:00Z", HeadSHA: lateHead, Base: "main"}
w.forge.open = []OpenPull{{Owner: "novox", Repo: "mesh-catalog", Number: 2, Base: "main", HeadSHA: gHk},
{Owner: "novox", Repo: "mesh-controller", Number: 1, Base: "main", HeadSHA: gHc}}
w.h.wantForge()
w.settleAll()
d := w.delivery(late)
if d.State != Merged || !d.MergedAt.Equal(time.Date(2026, 10, 1, 9, 0, 0, 0, time.UTC)) || !d.LearntLate {
t.Fatalf("the late merge is %s at %v (learnt late %v)", d.State, d.MergedAt, d.LearntLate)
}
if after := trunkOf(); fmt.Sprint(after) != fmt.Sprint(before) {
t.Fatalf("the group's trunk or state moved: %v, was %v", after, before)
}
if g := w.h.groups["feat/g"]; g.Check == nil || g.Check.OutOfDate != "" {
t.Fatalf("the group's composed check went out of date: %+v", g.Check)
}
w.h.SourceCatchUp()
w.settleAll()
if len(w.source.moves) != moves {
t.Fatalf("source.main moved for a merge learnt late: %+v", w.source.moves[moves:])
}
}},
// mesh-delivery#51 (2026-10-25 21:43): the open list was read seconds after the bridge merged it and before the forge's
// merged event, so it was learnt late, moved no trunk, and its group's finish left it out of source.main. A merge
// learnt within replayLate of its landing is taken as heard as it happened; one a gate before the fix took late is
// written at the catch-up of the start.
{"a merge learnt from the open list seconds after it landed moves source.main", func(t *testing.T) {
const repo, head, merge = "novox/mesh-delivery", "4628ba26b41a4628ba26b41a4628ba26b41a4628", "db53ec975f01db53ec975f01db53ec975f01db53"
for _, oldGate := range []bool{false, true} {
w := newWorld(t)
w.merges.manual = true
w.h.PullUpdated(pr(repo, 51, head, "fix/1945", "modules/a/a.go"))
w.h.Checked(judgedAt(repo, 51, head, "pass", "t1t1t1t1t1t1"))
w.settleAll()
w.source.main = &sdk.Source{V: sdk.Version, Group: "before", Heads: map[string]string{repo: "73be3df28cd4"}}
w.forge.pulls[repo+"#51"] = ForgePull{Number: 51, State: "closed", Merged: true, MergeCommit: merge,
MergedAt: w.h.Now().Add(-3 * time.Second).UTC().Format(time.RFC3339), HeadSHA: head, Base: "main"}
w.forge.open = []OpenPull{{Owner: "novox", Repo: "other", Number: 1, Base: "main", HeadSHA: "abab"}}
w.h.wantForge()
w.settleAll()
if oldGate { // what the gate before the fix left
w.h.mu.Lock()
kept := w.h.deliveries[IDOf(repo, head)]
kept.LearntLate, kept.InSource, kept.Ready.SourceMoved = true, true, true
w.h.mu.Unlock()
w.source.main.Heads[repo] = "73be3df28cd4"
w.h.SourceCatchUp()
w.settleAll()
}
d := w.delivery(IDOf(repo, head))
if d.State != Merged || (!oldGate && d.LearntLate) || w.source.main.Heads[repo] != merge {
t.Fatalf("old gate %v: %s, learnt late %v, source.main %v", oldGate, d.State, d.LearntLate, w.source.main.Heads)
}
if heads, _ := w.h.SourceMain()["heads"].(map[string]string); heads[repo] != merge {
t.Fatalf("delivery_status's source-main says %v", w.h.SourceMain())
}
}
}},
// Review round 1, blocker 2 (1275/AC6): an out-of-date refusal at t0 with no other event is said urgent at t0+10
// minutes: the gate asks itself for a look then.
{"an out-of-date refusal turns urgent on the clock alone", func(t *testing.T) {
w := approvedGroup(t)
told := hearConditions(w)
allGreen(w)
w.forge.refuse = errAskedAgain
w.op.answer(w.op.open()[0], OptionApprove)
w.settleAll()
t0 := w.h.Now()
w.h.wanted.mu.Lock()
at := w.h.wanted.at
w.h.wanted.mu.Unlock()
if at.IsZero() || at.Sub(t0) > outOfDateFor {
t.Fatalf("no look is wanted by t0+%s: %v", outOfDateFor, at)
}
for i := 0; i < 5 && w.h.Now().Before(t0.Add(outOfDateFor)); i++ { // the timer fires a look, and nothing else happens
w.h.wanted.mu.Lock()
at = w.h.wanted.at
w.h.wanted.mu.Unlock()
if at.IsZero() {
break
}
w.later(at.Sub(w.h.Now()))
w.h.React()
}
c := told.last(IDOf(gCat, gHk))
if c == nil || c.State != asks.StateRaised || c.Severity != asks.SeverityUrgent {
t.Fatalf("at t0+%s with no event: %+v", w.h.Now().Sub(t0), c)
}
}},
// Review round 1, blocker 3: a gate that restarts while a head still waits on the forge does not clear its
// condition (the head's first sight is in memory alone, so its bound runs again), and does not say it again.
{"a restart keeps the condition of a head still waiting", func(t *testing.T) {
w := newWorld(t)
told := hearConditions(w)
const head = "bfe82315f42c8f4509d0598c32569a202d5eada6"
id := IDOf("novox/hq", head)
w.forge.open = []OpenPull{{Owner: "novox", Repo: "hq", Number: 243, Base: "main", HeadSHA: head, Required: []string{"mesh/repo-check"}}}
w.h.wantForge()
w.settleAll()
w.later(UncheckedAfter + time.Minute)
w.settleAll()
if c := told.last(id); c == nil || c.State != asks.StateRaised {
t.Fatalf("the unchecked head is not raised: %+v", c)
}
before := len(told.told)
again := newWorldOver(t, w.store, w.forge, w.h.Now()) // the gate restarts
again.h.Say, again.h.Conditions = told.say, told
again.h.wantForge()
again.settleAll()
again.later(time.Minute)
again.settleAll()
if len(told.told) != before {
t.Fatalf("the restart said something of a head still waiting: %+v", told.told[before:])
}
w.forge.open = nil // it closes
again.h.wantForge()
again.settleAll()
if c := told.last(id); c == nil || c.State != asks.StateCleared {
t.Fatalf("closed, and its condition is not cleared: %+v", c)
}
}},
// The kind behind the day's fault: a decision on a group's state named a word GroupState no longer says. Every word
// it says is one groupStatusState knows, and the merge goes on exactly in ready and merging.
{"every word of a group's state is decided on", func(t *testing.T) {
for _, word := range groupWords {
if groupStatusState(word) == "pending" && word != GroupChecking {
t.Errorf("groupStatusState does not know %q", word)
}
if may := groupMayMerge(word); may != (word == GroupReady || word == GroupMerging) {
t.Errorf("groupMayMerge(%q) = %v", word, may)
}
}
}},
}
for _, c := range cases {
t.Run(c.name, c.run)
}
}
// allGreenFor makes both merge checks a success at every member's head.
func allGreenFor(w *world, ms ...member) {
for _, m := range ms {
setChecks(w, m.repo, m.head, "success", "success")
}
}
// bridgeMerged is the forge bridge's word that it merged a member.
func bridgeMerged(w *world, repository string, number int, head, merge string) {
w.merges.say(sdk.Forge{Kind: sdk.ForgeMerged, Repository: repository, Number: number, Head: head, Merge: merge, Root: "root",
Trees: map[string]string{"app": "tree"}})
}
@@ -6,7 +6,7 @@ import "git.novox.be/novox/mesh-sdk/go/announce"
// issue 1790): the module says it itself, from its code, before it serves, and a reader is granted what it said
// (clause 3). A state is readable when another module's manifest reads it (keeps_test.go names the readers). keeps_test.go holds the list to the
// manifest's `state`, and keeps_integration_test.go that the module announces it on a bus.
var Kept = []announce.State{{Name: "asks", Readable: true}, {Name: "deliveries"}, {Name: "groups"}, {Name: "merges", Readable: true}, {Name: "owed", Readable: true}, {Name: "pulls", Readable: true}, {Name: "source", Readable: true}}
var Kept = []announce.State{{Name: "asks", Readable: true}, {Name: "conditions"}, {Name: "deliveries"}, {Name: "groups"}, {Name: "merges", Readable: true}, {Name: "owed", Readable: true}, {Name: "pulls", Readable: true}, {Name: "source", Readable: true}, {Name: "turns", Readable: true}}
// announceKept says Kept through keeps, which is stdio.Keeps.
func announceKept(keeps func(...announce.State) error) error { return keeps(Kept...) }
@@ -17,6 +17,9 @@ var readers = map[string][]string{
"pulls": {"build-agent"}, // in mesh-controller
"owed": {"mesh-delivery-forge-bridge"},
"source": {"mesh-delivery-indexer", "mesh-delivery-checker"},
// ADR 0548 §3, §4: the checker reads the merge turns and claims no check of a repository one names; the gate alone writes
// them (a state is written by its owner's account alone, ADR 0495 §3: announcing it readable grants a read, never a write).
"turns": {"mesh-delivery-checker"},
}
// 1790/S: the states the module announces are the states its manifest declares, no more, no fewer, and the readable
@@ -65,8 +65,12 @@ func main() {
Operator: kvOperator{},
Merges: kvMerges{},
Source: kvSource{},
Turns: kvTurns{},
Owed: kvOwed{},
Pulls: kvPulls{},
// The gate says what stalled to the operator itself (issue 1275, ADR 0437 §9).
Say: sayOnSeat,
Conditions: kvConditions{},
}
h.init()
l := &listening{now: "not yet: starting"}
@@ -354,11 +358,34 @@ func tools(h *Holder, l *listening) []stdio.Tool {
}
return h.Close(strArg(a, "id"), strArg(a, "why"))
}},
{Name: "delivery_turns",
Description: "Every merge turn (novox/hq ADR 0548 §3): per repository, the approved merges that wait to merge into it in the order of " +
"their Approve, the check of another delivery each waits for and since when, whether the gate wrote the turn to its bucket " +
"turns yet (the checker claims no check of a repository a turn names), and the bound past which a held merge is a stalled " +
"line. Read-only.",
Run: func(map[string]any) (any, error) { return h.TurnsRead(), nil }},
{Name: "delivery_hand_now",
Description: "On the operator's word, hand an approved merge held for another delivery's running check to the merge at once " +
"(novox/hq ADR 0548 §4): the check it waited for runs on and is carried to the new trunk after the merge. Refused for a " +
"delivery that is not an approved merge held by a check, naming why. The why is kept on the delivery.",
Input: map[string]any{"type": "object", "properties": map[string]any{"id": str("the delivery's id (owner/repository@commit)"),
"why": str("the operator's word, kept on the delivery")}, "required": []string{"id", "why"}},
Run: func(a map[string]any) (any, error) {
if err := need(a, "id", "why"); err != nil {
return nil, err
}
return h.HandNow(strArg(a, "id"), strArg(a, "why"))
}},
{Name: "delivery_status",
Description: "Whether this owner works: how many deliveries in each state, the groups, what is owed to " +
"the forge and the bus and not yet done, how many owed effects were tried since it started (done, failed " +
"and tried again, given up) and when the last was, whether its events arrive, and the last refusals.",
Run: func(map[string]any) (any, error) { return h.Status(l.get()), nil }},
"and tried again, given up) and when the last was, whether its events arrive, the last refusals, and " +
"source.main as the gate's bucket source holds it (each repository's trunk head the checker judges on).",
Run: func(map[string]any) (any, error) {
out := h.Status(l.get())
out["source-main"] = h.SourceMain()
return out, nil
}},
}
// A verb that changes a delivery asks for a look at what it makes possible (react.go).
for i := range list {
@@ -91,9 +91,10 @@ func TestItClaimsTheDeliverySeatAndFollowsWhatItHandles(t *testing.T) {
if !reflect.DeepEqual(m.Emits, []string{"transition", "group", "merged"}) {
t.Fatalf("emits %v", m.Emits)
}
// Seven buckets, and no `asked` (the teller asks, ADR 0437 §6): ADR 0450 §3.
// Nine buckets, and no `asked` (the teller asks, ADR 0437 §6): ADR 0450 §3, the conditions it raised (issue 1275) and the
// merge turns (ADR 0548 §3).
buckets := m.buckets()
if !reflect.DeepEqual(buckets, []string{"deliveries", "groups", "pulls", "asks", "merges", "owed", "source"}) {
if !reflect.DeepEqual(buckets, []string{"deliveries", "groups", "pulls", "asks", "merges", "owed", "source", "conditions", "turns"}) {
t.Fatalf("state %v", buckets)
}
// What it reads: the teller's answers and the checker's verdicts (issue 1302). The carry from today's module's buckets, the
@@ -102,10 +103,10 @@ func TestItClaimsTheDeliverySeatAndFollowsWhatItHandles(t *testing.T) {
t.Fatalf("reads %v", m.Reads)
}
// It calls no controller verb and reads the forge with three tools; the forge's writes are the bridge's (ADR 0437 §8). It
// uses no operator-channel: the teller asks (ADR 0437 §6).
// uses operator-channel only to say its conditions (ADR 0437 §9, issue 1275): the teller asks (ADR 0437 §6).
want := []string{"gitea.gitea_commit_statuses", "gitea.gitea_contains", "gitea.gitea_open_pulls"}
if len(m.Uses) != 0 {
t.Fatalf("uses %v: the teller asks, not the gate", m.Uses)
if !reflect.DeepEqual(m.Uses, []string{"operator-channel"}) {
t.Fatalf("uses %v: operator-channel, for its conditions alone", m.Uses)
}
if !reflect.DeepEqual(m.Invokes, want) {
t.Fatalf("invokes %v, wanted %v", m.Invokes, want)
@@ -105,6 +105,26 @@ func (kvSource) Move(update func(prev *sdk.Source) (sdk.Source, bool)) error {
return fmt.Errorf("source.main was written by another writer first %d times: %w", changeAttempts, stdio.ErrChanged)
}
// SourceMain is source.main as the bucket source holds it, read and never written: its group and each repository's trunk
// head, or why it cannot be read. Outside the lock.
func (h *Holder) SourceMain() map[string]any {
if h.Source == nil {
return map[string]any{"error": "no source bucket"}
}
var held *sdk.Source
err := h.Source.Move(func(prev *sdk.Source) (sdk.Source, bool) {
held = prev
return sdk.Source{}, false
})
switch {
case err != nil:
return map[string]any{"error": err.Error()}
case held == nil:
return map[string]any{"heads": map[string]string{}, "says": "source.main is not written yet"}
}
return map[string]any{"group": held.Group, "heads": held.Heads}
}
// approvedOf are the heads an Approve covers as a merge carries them: `<repository>@<number>` to the full head.
func (h *Holder) approvedOf(heads map[string]string) map[string]string {
out := map[string]string{}
@@ -356,6 +376,9 @@ func (h *Holder) settleMerges() []func() {
var members []string
for hid := range r.Heads {
m := h.deliveries[hid]
if m.LearntLate {
continue // learnt after the fact: never written to source.main (settleGone)
}
commit := m.MergedAs
if m.Ready != nil && m.Ready.MergeCommit != "" {
commit = m.Ready.MergeCommit
@@ -363,6 +386,9 @@ func (h *Holder) settleMerges() []func() {
heads[m.Repository] = commit
members = append(members, hid)
}
if len(members) == 0 {
continue // every member learnt after the fact: nothing to write
}
sort.Strings(members)
last := h.deliveries[members[len(members)-1]]
out = append(out, func() { h.finish(name, key, approved, last.ID, heads, members) })
@@ -428,6 +454,18 @@ func gateMerges(d *Delivery) bool {
return d.Ready != nil && d.Ready.Approved != nil && d.Ready.AskID != "" && len(d.Ready.Heads) > 0
}
// leftOutOfForgeSource is whether settleForgeMerges never writes a merge to source.main: one the gate handed on an Approve
// (finish writes it), and one learnt after the fact, long past or with no time the forge said. One learnt late within
// replayLate of its merge, as a gate before the fix took mesh-delivery#51 (2026-10-25, its group's finish left it out and
// source.main kept the trunk before it), is weighed like any merge heard from the forge at a catch-up, and never moves
// source.main back (olderHead).
func (d *Delivery) leftOutOfForgeSource() bool {
if d.LearntLate {
return d.MergedAt.IsZero() || d.Since.Sub(d.MergedAt) >= replayLate
}
return gateMerges(d)
}
// settleForgeMerges moves source.main for each merge into the trunk heard only from the forge (issue 888): a pull request
// merged at the forge directly, not handed by the gate on an Approve. Every other delivery's trunk moved with it (trunkMoved),
// and the checker judges on the trunk source.main names, so source.main moves too. A merge replayed as history, or older than
@@ -441,7 +479,7 @@ func (h *Holder) settleForgeMerges() []func() {
again = h.newestInSource()
}
for _, d := range h.deliveries {
if d.MergedAs == "" || (d.InSource && !again[d]) || gateMerges(d) {
if d.MergedAs == "" || (d.InSource && !again[d]) || d.leftOutOfForgeSource() {
continue
}
if d.Base != MergeBranch || d.replayed() || h.newerMergeOf(d) {
@@ -558,7 +596,7 @@ func (h *Holder) SourceCatchUp() {
func (h *Holder) newestInSource() map[*Delivery]bool {
newest := map[string]*Delivery{}
for _, d := range h.deliveries {
if d.MergedAs == "" || !d.InSource || gateMerges(d) || d.Base != MergeBranch || d.replayed() {
if d.MergedAs == "" || !d.InSource || d.leftOutOfForgeSource() || d.Base != MergeBranch || d.replayed() {
continue
}
if o := newest[d.Repository]; o == nil || mergedWhen(d).After(mergedWhen(o)) {
@@ -82,8 +82,11 @@ type Delivery struct {
MergedAt time.Time `json:"merged_at"`
// InSource says the merge was looked at for source.main: written there, or left as older than the trunk's head (issue 888).
// A merge the gate handed to the bridge on an Approve is moved there by finish instead (merges.go).
InSource bool `json:"in_source,omitempty"`
HeldWhy string `json:"held_why,omitempty"`
InSource bool `json:"in_source,omitempty"`
// LearntLate says its merge was learnt after the fact, from its absence among the forge's open pull requests
// (settleGone, issue 1275): it moved no trunk here, and it never moves source.main, not even at a catch-up.
LearntLate bool `json:"learnt_late,omitempty"`
HeldWhy string `json:"held_why,omitempty"`
// FromForge is a delivery made from the forge's word at its merge: its head was never heard by this owner.
FromForge bool `json:"from_forge,omitempty"`
@@ -72,8 +72,12 @@ func (h *Holder) CheckerVerdictChanged(key string, value []byte) {
if i := strings.LastIndex(key, "."); i >= 0 {
hash = key[i+1:]
}
if h.runningSince == nil {
h.runningSince = map[string]time.Time{}
}
if len(value) == 0 || string(value) == "null" {
delete(h.checker, hash)
delete(h.runningSince, hash)
h.Poke()
return
}
@@ -85,6 +89,11 @@ func (h *Holder) CheckerVerdictChanged(key string, value []byte) {
if v.Heads == "" {
v.Heads = hash
}
if prev, ok := h.checker[v.Heads]; v.State == checkerRunning && (!ok || prev.State != checkerRunning) {
h.runningSince[v.Heads] = h.Now() // a check starts: a merge it holds says since when (ADR 0548 §2)
} else if v.State != checkerRunning {
delete(h.runningSince, v.Heads)
}
h.checker[v.Heads] = v
h.Poke()
}
@@ -155,6 +164,13 @@ func (h *Holder) takeCheckStatus(repository, sha, state, desc string) {
}
h.moved(d, []Transition{t})
}
if d.State == Ready && !h.checkGates() {
// The check gates no merge (check-gates-merge false): its verdict is shown on the delivery, and moves nothing.
d.Check = v
h.Logf("[mesh-delivery] %s: its mesh/check is %s — %s (shown; the check gates no merge)", d.ID, state, oneLine(clip(desc, 120)))
h.keep(d)
return
}
if d.State != Proposed {
h.keep(d)
return // past its verdict: the merge reads the forge's status itself
@@ -199,14 +199,3 @@ func TestAClosedPullRequestAsksNoForgetChecks(t *testing.T) {
w.h.PullUpdated(p)
w.h.PullClosed(p)
}
// 1302: the mesh/check line gives the head and the trunk it judged.
func TestTheCheckLineGivesHeadAndTrunk(t *testing.T) {
head, trunk := checkLine("pass " + oneHead[:12] + " on " + oneTrunk[:12] + " | unit 4 run, 0 kept | group g of 2 pull requests")
if head != oneHead[:12] || trunk != oneTrunk[:12] {
t.Fatalf("head %q trunk %q", head, trunk)
}
if head, trunk = checkLine("failed: a test"); head != "" || trunk != "" {
t.Fatalf("a line that names no commits gave %q %q", head, trunk)
}
}
@@ -45,7 +45,9 @@ type ForgePull struct {
Head string `json:"head,omitempty"`
HeadSHA string `json:"head_sha,omitempty"`
MergeCommit string `json:"merge_commit_sha,omitempty"`
HTMLURL string `json:"html_url,omitempty"`
// MergedAt is when it merged, as the forge says it (RFC 3339); empty when not merged or not said.
MergedAt string `json:"merged_at,omitempty"`
HTMLURL string `json:"html_url,omitempty"`
}
// ForgeStatus is one status of a commit, the newest of its context.
@@ -57,7 +57,7 @@ func (h *Holder) Work() {
// React takes every step the facts allow, after an event that can change what they are: the forge's open pull requests
// read when a check may have answered, the steps that need another's word (look), what is owed (Flush), and then
// the time the next look is wanted at, if any step waits on the clock. Never concurrent with itself.
// the conditions of what stalled (conditions.go), and then the time the next look is wanted at, if any step waits on the clock. Never concurrent with itself.
func (h *Holder) React() {
h.reactMu.Lock()
defer h.reactMu.Unlock()
@@ -75,6 +75,7 @@ func (h *Holder) React() {
}
h.look()
h.Flush()
h.sayConditions()
h.arm()
}
@@ -29,7 +29,7 @@ import (
// - a group member only once the group's composed check passed for its members' heads as they are;
// - a push ends the Approve and says so once on the pull request (ADR 0421 §6); the new head is asked on its own;
// - a check that failed merges nothing: it is said once on the pull request, and as a stalled line, which the
// controller raises as the condition `delivery.<id>.stalled`. So is a merge the tool refused for an hour.
// gate says to the operator as a condition (conditions.go). So is a merge the tool refused for an hour.
//
// The review record is read and put in the ask; it is never a gate (ADR 0421 §5). What this owner does is kept on the
// delivery (Ready), said in its log, and the merge's transitions name the Approve as their warrant.
@@ -121,6 +121,13 @@ type ReadyRun struct {
Declined *time.Time `json:"declined,omitempty"`
Warrant string `json:"warrant,omitempty"`
Refused string `json:"refused,omitempty"`
// One check per delivery (novox/hq ADR 0548 §2, §4, turns.go): HeldSince is when the approved merge began to wait for the
// check of other heads HeldFor names (its verdict key), running on a repository it goes into; HandNow the operator's word
// that handed it at once past that check.
HeldSince *time.Time `json:"held_since,omitempty"`
HeldFor string `json:"held_for,omitempty"`
HandNow string `json:"hand_now,omitempty"`
}
// MergeBranch is the one branch this owner merges into (novox/hq ADR 0398 §4): a merge into any other is its author's.
@@ -157,6 +164,7 @@ func (h *Holder) runOf(d *Delivery, now time.Time) *ReadyRun {
func (h *Holder) readyAsks(now time.Time) []func() {
var asks []func()
h.endStaleAsks()
gates := h.checkGates()
ids := make([]string, 0, len(h.deliveries))
for id := range h.deliveries {
ids = append(ids, id)
@@ -167,6 +175,9 @@ func (h *Holder) readyAsks(now time.Time) []func() {
if d.Number == 0 || d.State.Final() || d.MergedAs != "" {
continue // not of a pull request, or past its merge
}
if !gates {
h.ungate(d, now)
}
if d.Ready == nil && d.State == Ready {
h.runOf(d, now)
h.Poke() // its check, if none is asked, is worth asking now (askCheck)
@@ -240,6 +251,9 @@ func (h *Holder) readyAsks(now time.Time) []func() {
if waits == "" {
waits, approvalAsk = h.approval(d, now)
}
if waits == "" && gates {
waits = h.turnWaits(d, now) // nothing merges into a repository while a check of other heads judges it (ADR 0548 §2)
}
if waits != r.Waits {
r.Waits = waits
h.keep(d)
@@ -263,6 +277,7 @@ func (h *Holder) readyAsks(now time.Time) []func() {
asks = append(asks, a)
}
}
asks = append(asks, h.turnWrites()...) // the merge turns, after the holds of this look (ADR 0548 §3)
asks = append(asks, h.settleMerges()...)
asks = append(asks, h.settleForgeMerges()...)
asks = append(asks, h.withdrawnKeys()...)
@@ -271,6 +286,30 @@ func (h *Holder) readyAsks(now time.Time) []func() {
return asks
}
// ungate stands a delivery where it is while mesh/check gates no merge (check-gates-merge false): a proposed one is ready
// with no verdict, and a rejected one, refused on a verdict before the setting changed, is proposed again and so ready. Its
// verdict stays on it, shown. Under the lock.
func (h *Holder) ungate(d *Delivery, now time.Time) {
f := h.factsAt(now)
if d.State == Rejected {
f.By, f.Why = byOwner, "the check gates no merge (check-gates-merge false): its verdict is shown, not waited on"
t, err := Apply(d, EvRecheck, true, f)
if err != nil {
h.refuse(err)
return
}
if d.Ready != nil {
d.Ready.Failed = ""
}
h.moved(d, []Transition{t})
f.By, f.Why = "", ""
}
if d.State == Proposed {
h.moved(d, settle(d, f))
h.closeGroupOf(d)
}
}
// sayAfterLine tells a pull request, once, that its description holds an `after: <repository>` line, which is not read
// (novox/hq ADR 0419 §2): the order of a delivery comes from the module graph alone, and no refusal asks for the line.
// Once per pull request, not per head. Under the lock; the comment is said outside it.
@@ -337,6 +376,9 @@ func (r *ReadyRun) noteRefusal(d *Delivery, answer string) {
// stays through a trunk move, ADR 0421 §3) and its check against the trunk's head is not on its way; one without an
// Approve waits for that check first.
func (h *Holder) readyWaits(d *Delivery) string {
if !h.checkGates() {
return h.ungatedWaits(d)
}
if d.judgedByCheck() && d.State == Proposed && d.outOfDate() != "" && h.covering(d) == nil {
return "its mesh/check is " + d.outOfDate() + ": the checker checks it again against the trunk's head"
}
@@ -345,6 +387,9 @@ func (h *Holder) readyWaits(d *Delivery) string {
case d.State == Proposed && d.outOfDate() != "" && d.AskedAgain != nil:
return "its check against the trunk's head is asked and has not answered"
case d.State == Proposed && d.outOfDate() != "" && d.Check.Passes() && h.covering(d) != nil:
// Approved, and its trunk moved: the checker carries its verdict to the trunk's head by its package digests (ADR 0548
// §5, §8). The Approve stands and is not asked again; the merge waits for the carried verdict, not a fresh check.
return "approved; its verdict is being carried to the trunk's head " + shortOf(d.TrunkNow, 12) + " (" + d.outOfDate() + ")"
case d.State == Proposed && d.outOfDate() != "":
return "its verdict is out of date: its check against the trunk's head is asked before the operator is"
case d.State == Proposed && d.AskedAgain != nil:
@@ -369,6 +414,27 @@ func (h *Holder) readyWaits(d *Delivery) string {
return h.groupTurnToMerge(d)
}
// ungatedWaits is readyWaits while mesh/check gates no merge (check-gates-merge false): a ready delivery waits only for its
// review at its head, and a group member for every member to be ready; no verdict, composed check or trunk is waited on.
func (h *Holder) ungatedWaits(d *Delivery) string {
if d.State != Ready {
return "it is " + string(d.State) + ", not ready yet"
}
if h.covering(d) == nil {
if w := h.reviewWaits(d); w != "" {
return w
}
}
if g := h.groups[d.Group]; d.Group != "" && g != nil {
for _, m := range h.membersOf(g) {
if m.State != Ready && m.State != Merged {
return "its group " + g.ID + " waits for " + m.ID + ", which is " + string(m.State)
}
}
}
return ""
}
// waitsForReview is how a hold for a review begins, as the delivery's state and its mesh/delivery status say it.
const waitsForReview = "waits for a review at "
@@ -401,13 +467,14 @@ func (d *Delivery) holdsForReview() bool {
// groupTurnToMerge is why a group member may not merge yet; empty when it may: the group's composed check passed for
// its members' heads as they are and is not out of date. No member waits for another's merge or delivery (novox/hq
// ADR 0418 §4): once the composed check passed and the Approve covers the heads, every member is merged in one pass.
// A group part way through that pass is merging, and its next member's turn is then (issue 1275).
func (h *Holder) groupTurnToMerge(d *Delivery) string {
g := h.groups[d.Group]
if g == nil {
return ""
}
members := h.membersOf(g)
if state, why := GroupState(g, members); state != "ready" && state != "delivering" {
if state, why := GroupState(g, members); !groupMayMerge(state) {
return fmt.Sprintf("its group %s is %s: %s", g.ID, state, why)
}
if c := g.Check; c == nil || (c.Verdict != "pass" && c.Verdict != "warning") || c.OutOfDate != "" ||
@@ -417,6 +484,16 @@ func (h *Holder) groupTurnToMerge(d *Delivery) string {
return ""
}
// groupMayMerge is whether a group in this state hands its next member to the forge bridge: one ready, and one merging (a
// member merged, the pass goes on). Every other word holds the pass.
func groupMayMerge(state string) bool {
switch state {
case GroupReady, GroupMerging:
return true
}
return false
}
// sayOn owes a comment on a delivery's pull request: an owed act numbered among the comments on its head, written as a key
// with the delivery's other effects (owed.go). Under the lock; what it returns has nothing left to do.
func (h *Holder) sayOn(d *Delivery, body string) func() {
@@ -427,7 +504,7 @@ func (h *Holder) sayOn(d *Delivery, body string) func() {
}
// readyLines are the approved heads that will not merge: a check that failed at once, a merge refused past its bound;
// as stalled lines, which the controller raises as conditions. Under the lock.
// as stalled lines, which the gate says as conditions (conditions.go). Under the lock.
func (h *Holder) readyLines(now time.Time) []StalledLine {
var out []StalledLine
for _, d := range h.deliveries {
@@ -64,6 +64,43 @@ func TestAnApprovedPullRequestIsMergedThroughTheMergeTool(t *testing.T) {
if len(w.forge.merges) != 1 {
t.Fatalf("merged again: %v", w.forge.merges)
}
// check-gates-merge false (the operator, 2026-10-25 22:56): a reviewed pull request is asked with no verdict, another
// merge moving its trunk puts it out of nothing, a failed verdict is shown on it and holds nothing, and the gate's
// status and show say the mode.
u := newWorld(t)
ungated(u)
u.op.auto = false
const other = "3333333333333333333333333333333333333333"
u.h.PullUpdated(pr("novox/app", 7, readyHead, "feat/x", "src/a.go"))
u.h.PullUpdated(pr("novox/app", 8, other, "feat/y", "src/b.go"))
u.settleAll()
if u.op.count() != 2 || u.state(IDOf("novox/app", readyHead)) != Ready {
t.Fatalf("ungated, two reviewed pull requests with no verdict: %d asks, %s", u.op.count(), u.state(IDOf("novox/app", readyHead)))
}
if gates, _ := u.h.Status("listening")["check-gates-merge"].(bool); gates {
t.Fatalf("delivery_status does not say the check gates no merge: %v", u.h.Status("listening"))
}
shown, _ := u.h.Show(IDOf("novox/app", other))
if gates, _ := shown.(map[string]any)["check-gates-merge"].(bool); gates {
t.Fatalf("show does not say the check gates no merge: %v", shown)
}
u.h.Checked(judgedAt("novox/app", 8, other, "fail", "t1t1t1t1t1t1"))
u.h.PullMerged(merged("novox/app", 7, readyHead, "m7m7m7m7m7m7"))
u.settleAll()
d = u.delivery(IDOf("novox/app", other))
if d.State != Ready || d.Check == nil || d.Check.Passes() {
t.Fatalf("ungated, after a failed verdict and a trunk's move: %s, verdict %+v", d.State, d.Check)
}
for _, a := range u.op.open() {
if strings.Contains(a, "feat/y") {
u.op.answer(a, OptionApprove)
}
}
u.settleAll()
if len(u.forge.merges) != 1 || u.forge.merges[0] != "novox/app#8@"+other {
t.Fatalf("ungated, the approved head is merged %v; wanted once, with no verdict waited on", u.forge.merges)
}
}
// 1144/AC3: a refused merge is not asked again by any timer; only an event (here, mergeAgainNow called directly, as a
@@ -207,6 +244,22 @@ func TestAGroupMergesEveryMemberInOnePassOnlyOnceComposedAndApproved(t *testing.
if w.op.count() != 1 {
t.Fatalf("a second Approve was asked: %d", w.op.count())
}
// check-gates-merge false: the group is asked once with no verdict and no composed check, and merges whole in one pass.
u := newWorld(t)
ungated(u)
u.op.auto = false
u.h.PullUpdated(pr("novox/mesh-controller", 1, ctlHead, "feat/x", "internal/a.go"))
u.h.PullUpdated(pr("novox/mesh-catalog", 2, catHead, "feat/x", "modules/app/module.json"))
u.settleAll()
if u.op.count() != 1 {
t.Fatalf("ungated, the group is asked %d times; wanted once", u.op.count())
}
u.op.answer(u.op.open()[0], OptionApprove)
u.settleAll()
if len(u.forge.merges) != 2 {
t.Fatalf("ungated, both members merge in one pass: %v", u.forge.merges)
}
}
// **A merge into another branch than main is its author's** (novox/hq ADR 0398 §4, issue 948): a green pull request
@@ -25,6 +25,10 @@ const (
type Settings struct {
RefusalFloorMinutes *int `json:"refusal-stall-floor-minutes"`
AuthorBoundMinutes *int `json:"author-new-head-minutes"`
// CheckGatesMerge is whether mesh/check gates the merge (the default, true). False (the operator, 2026-10-25 22:56:
// "run the check locally only, but not as a gate, it loses us too much time"): a reviewed pull request is ready and its
// Approve asked without a verdict, a trunk's move puts nothing out of date, and a verdict is shown, never waited on.
CheckGatesMerge *bool `json:"check-gates-merge"`
}
// readSettings reads the settings file; a value that is not a whole number of minutes, one or more, is refused.
@@ -55,9 +59,30 @@ func readSettings(path string) (Settings, error) {
return Settings{}, fmt.Errorf("%s is not a number", name)
}
}
switch v := loose["check-gates-merge"].(type) {
case nil:
case bool:
s.CheckGatesMerge = &v
default:
return Settings{}, errors.New("check-gates-merge is not true or false")
}
return s, nil
}
// checkGates is whether mesh/check gates the merge: true unless the setting check-gates-merge is false.
func (h *Holder) checkGates() bool {
if h.Settings == nil {
return true
}
v := h.Settings().CheckGatesMerge
return v == nil || *v
}
// factsAt are the facts a settle reads at now: the moment, and whether the check gates the merge.
func (h *Holder) factsAt(now time.Time) Facts {
return Facts{Now: now, Ungated: !h.checkGates()}
}
// minutesOr is a setting's minutes, or the default when it gives none.
func minutesOr(n *int, def time.Duration) time.Duration {
if n == nil {

Some files were not shown because too many files have changed in this diff Show More