Compare commits

...
Author SHA1 Message Date
jschoubben 1c55e5d44d Vault: take and hash a process's own bus account (broker-<process id>) as a covered account, so take-over is not held by every process user
mesh/delivery merged: source.main moved to novox/mesh-catalog@22381d75224f
mesh/check failed: integration: meshbuildnode/cmd/mesh-build-node.TestAFixtureBundleBuiltTwiceHasOneDigest: no toolchain of this lock is installed und…
The controller hands each process account of a core or moved module with origin issued (ADR 0436 §2); the vault took
only broker and provisioner-broker, refused these and never hashed them. Owed hashes are now said once per pass, and
account_hash says who asked when it refuses too (review round 1).

Issue: 1945
2026-10-26 02:35:57 +01:00
jschoubben 32b912cfe5 Vault: hash each bus account it is handed into bus-accounts.<user>, once per generation, so the grants part can compose the user list without the controller
The controller still mints each covered account and hands it to the vault (ADR 0374); ADR 0547 clause 9 has the vault
write the bcrypt hash of what it holds into its own state, readable by the grants part, and say account-hashed. It
hashes on keep, at start every held account with no key of its generation, and again by hand (account_hash); a refused
write is owed and written at the next pass. secret_own says per bus account whether this generation's hash is written.

Issue: 1945
2026-10-26 02:28:28 +01:00
jschoubben e572937d97 Vault: secret_own reads the module-own secrets it keeps, as fingerprints, so a generation can be told without the value
mesh/check pass e572937d97fa on 567736f7795a | unit 2143 run, 2198 kept, 5 skipped | integration 2165 run, 2217 kept, 9 skipped | over bound: unit, in…
mesh/delivery closed: a newer head of the same pull request
Issue: 1945
2026-10-26 02:09:03 +01:00
mesh-tools 567736f779 Merge pull request 'Build modules' step and toolchain lists read only the newest keys they answer, in parallel (issue 1827)' (#485) from fix/1827-bounded-reads into main 2026-10-25 23:56:12 +00:00
jschoubben e901769cb2 Read the newest steps first: sort the node's keys by id (start order) before the cut, and say older steps were not read
mesh/delivery merged: source.main moved to novox/mesh-catalog@567736f7795a
mesh/check failed: integration: meshbuildnode/cmd/mesh-build-node.TestAFixtureBundleBuiltTwiceHasOneDigest: no toolchain of this lock is installed und…
The review found a cut read took an arbitrary 1000 keys, so the newest steps could be missing. A step id is
base-36 milliseconds plus random digits, so its string order is its start order; the newest max(limit*4, 100)
keys are read now, 1000 at most. The toolchains read logs when it drops keys past its cap.
2026-10-26 00:43:53 +01:00
jschoubben 8f9c78bcba Read only the node's own steps, in parallel and bounded, so build_*_steps answers within the call's 30 s (issue 1827)
On a steps state of 1164 keys / 20 MB (mesh-build-node: 10034 keys) the read tools listed every key and
then read each value one round trip at a time, filtering by node only after the Get, so they timed out
at 30 s. Now the keys are filtered by the node's prefix before any Get, the values are read 16 at a
time, at most 1000 a call (the answer says it was cut), and one log line gives rows read and time.
build_*_toolchains reads its values the same parallel way. The kind: a read tool that reads a whole
state key by key; every read tool of the three modules is covered.
2026-10-26 00:39:33 +01:00
mesh-tools 82e7e3e326 Merge pull request 'mesh-provisioner-client: a module that leaves or is placed again asks the vault afresh (issue 1945)' (#484) from fix/1945-held-secret-on-leave into main 2026-10-25 23:21:44 +00:00
jschoubben 363a3d3f7e Provisioner client: drop a module's held vault secrets when it leaves or is placed again, so it asks afresh (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-catalog@82e7e3e32675
mesh/check pass 363a3d3f7e8f on 82e7e3e32675 | unit 2015 run, 2322 kept, 5 skipped | integration 2037 run, 2341 kept, 9 skipped | over bound: unit, in…
A re-assigned module was filled with the generation it held before the vault minted a new one; the bus refused the account.
2026-10-26 00:03:43 +01:00
jschoubben 609013a263 Merge pull request 'Gate: a merge learnt seconds late moves the trunk; and a setting check-gates-merge so the merge check can stop gating (operator 2026-10-25 22:56)' (#483) from fix/merge-without-check-gate into main
Reviewed-on: #483
2026-10-25 22:18:41 +00:00
jschoubben 32ad59fc64 mesh-delivery-gate and gitea: a setting check-gates-merge; false asks a reviewed pull request's Approve and merges it without waiting for mesh/check
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check pass 32ad59fc6402 on 609013a2638f | unit 2010 run, 2322 kept, 5 skipped | integration 2032 run, 2341 kept, 9 skipped | over bound: unit, in…
The operator, 2026-10-25 22:56: "run the check locally only, but not as a gate, it loses us too much time". Tonight each
merge ask waited 3-7 minutes for mesh/check, and every merge into main refused the next approved merge for a check judged
on the trunk before (mesh-delivery#52). Both settings default to true, so nothing changes until the operator sets them.
False: the gate makes a reviewed pull request ready and asks its Approve with no verdict, a trunk's move puts nothing out
of date, a verdict is kept and shown, and a group still merges in one pass; the merge tool merges a head the Approve
names whatever its mesh/check says. delivery_status and show say the mode.
2026-10-25 23:07:50 +01:00
jschoubben 93f655dac9 mesh-delivery-gate: a merge learnt from the open list seconds after it landed moves its trunk and source.main; delivery_status shows source.main
mesh-delivery#51 merged 21:43:11 and the gate read the open list at 21:43:14, before the forge's merged event: taken as
learnt late, it moved no trunk and its group's finish left it out of source.main, so the checker kept judging every
mesh-delivery head on 73be3df. A late merge within replayLate of its landing is now taken as heard as it happened, and
one a gate before this left is written at the catch-up of the start (never backwards).
2026-10-25 23:01:37 +01:00
mesh-tools 573a59e3cc Merge pull request 'mesh-provisioner-client: retry a refused write on the node-engine's wake and ask the vault again at once (issue 924)' (#482) from fix/provisioner-retries-on-declaration into main 2026-10-25 21:08:48 +00:00
jschoubben d0566f9ade Fill a refused file again once the node-engine's declaration may let it through (ace, 2026-10-25 21:17)
mesh/delivery merged: source.main moved to novox/mesh-catalog@573a59e3cc07
mesh/delivery-group group fix/provisioner-retries-on-declaration merged: every member is merged
mesh/check pass d0566f9ade9a on 573a59e3cc07 | unit 2301 run, 3410 kept, 10 skipped | integration 2326 run, 3431 kept, 13 skipped | over bound: unit, …
The node-engine refused mesh-build-go's files as not named as filled 32 s before
its want arrived, and nothing filled them after the apply that named them: the
runtime crash-looped on its missing broker credential. The client now fills
every module again when the node-engine touches its write socket after such an
apply (mesh-host fix/provisioner-retries-on-declaration), says each retry and
each refusal it waits on, and asks the vault again at once for a refused secret
when that wake or the module's own assignment may have ended the refusal.
2026-10-25 21:42:34 +01:00
jschoubben 9d9dfda19b Merge pull request 'Gate: hand every member of an approved group in pin order, settle pull requests closed on the forge, and say a stopped pass as a condition (issue 1275)' (#480) from fix/1275-group-hands-every-member into main
Reviewed-on: #480
2026-10-25 19:28:33 +00:00
jschoubben cdbe2945cf Merge pull request 'Build modules' steps live two days and their pause has a state of its own, so the steps bucket no longer fills (2026-10-25)' (#481) from fix/steps-bucket-ages into main
Reviewed-on: #481
2026-10-25 18:06:39 +00:00
jschoubben 4e82a824be Announce the pauses state in the three build modules, and pin mesh-sdk 518dae5 (an unreadable pause pulls as not paused)
mesh/delivery-group group fix/steps-bucket-ages held: a member is held: novox/mesh-sdk@518dae5ec9c8 (merged without a passing check: only a person decides what…
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check pass 4e82a824be7a on cdbe2945cf6e | unit 1994 run, 2322 kept, 5 skipped | integration 2016 run, 2341 kept, 9 skipped | over bound: unit, in…
Without it in Kept, stdio.KeptState refuses the state before the bus (mesh-sdk go/keeps.go
refusedUnannounced), so the pause could be neither read nor written.
2026-10-25 18:57:08 +01:00
jschoubben 78d26867d2 Give the build modules' steps state an age of two days, and keep a node's pause in a pauses state of its own
mesh-build-go's steps bucket filled to the bus's 64 MiB cap (27,370 keys, none ever removed) and
refused the end of a passed check, so the check read failed; mesh-build-node's (17.8 MB) and
mesh-build-image's grow the same way. A step's key is per step: its reader reads its end within
the step's bound, and a check's verdict is kept by the checker, so two days is ample. The node's
pause key lived in the same bucket and would have ended by itself; it moves to a pauses state
without an age (mesh-sdk fix/steps-bucket-ages, Puller.PausePut). No node is paused today, so
nothing is carried over.
2026-10-25 18:53:04 +01:00
jschoubben 0c216d99d0 Gate takes a merge learnt after the fact as past: no trunk move, the forge's time, never source.main (issue 1275, review round 2)
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check pass 0c216d99d0c0 on 9d9dfda19b7e | unit 2181 run, 2147 kept, 5 skipped | integration 2203 run, 2166 kept, 9 skipped | over bound: unit, in…
A pull request found merged only by its absence from the open list went through PullMerged, which moved every
other open delivery of its repository back to an old trunk (ready ones to proposed, composed checks out of
date), dated the merge now, and so let a later catch-up write the old commit into source.main. pullMerged now
takes such a merge as late: its own delivery is merged and its group closed, no other trunk moves, its time is
the forge's or none, and LearntLate keeps it out of source.main in finish, settleForgeMerges and the catch-up.

The gitea tool gitea_commit_statuses now answers the pull request's merged_at, and the gate reads a gone pull
request by its number alone, so a deleted branch does not fail the read.
2026-10-25 18:37:59 +01:00
jschoubben 145c54d19c Gate settles pull requests closed with no event, looks again at a line's bound, and keeps a waiting head's condition over a restart (issue 1275, review round 1)
The conditions the gate now says would otherwise be wrong three ways:

- A pull request closed or merged on the forge with no event heard stayed ready here for days (de-spiegel #4,
  closed 2026-10-17) and would raise a condition. After a complete read of the open list (shorter than the
  forge tool's 1000), each delivery not final whose pull request is missing is read from the forge and taken as
  its close or merge; a merge learnt so does not move source.main, as it is undated and could move it back.
- A line that comes due on the clock (a state's bound, the author's bound, an answered refusal, an
  out-of-date refusal turning urgent after 10 minutes) waited for an unrelated event. The gate now asks a look
  at the earliest such time.
- After a restart the heads waiting on the forge start their bound again, so their conditions were cleared and
  raised again later. A condition is cleared only once the open list was read since start, and not while the
  forge still shows its head open and waiting.

Also: an empty path list ranks with a truncated one in the merge order, and a seat that keeps refusing
conditions is retried on the clock 10 looks in a row, then at events only, said once in the log.
2026-10-25 18:32:19 +01:00
jschoubben 67092161bd Gate hands every member of an approved group, in pin order, and says a stopped pass (issue 1275)
Met again 2026-10-25: group fix/puller-wakes-on-capacity was merged half. Three causes in the gate:

- The next member's turn compared the group's state with "delivering", a word GroupState no longer says
  since release 6 renamed it "merging" (ready.go groupTurnToMerge). Once the first member's merge was heard
  from the forge the group read merging, and the second member waited for ever. Group states are now named
  constants, the turn goes on in ready and merging (groupMayMerge), and a test holds every word to it.
- Members merged in id order, so mesh-catalog went before the mesh-sdk commit it pins. membersOf now orders
  by mergeRank: mesh-sdk first, then members that change no go.mod or go.sum, then those that do.
- No condition was raised: the controller's self-check that read `stalled` went with release 6
  (mesh-controller dc456d56). The gate now says its stalled lines as conditions itself on the
  operator-channel seat (ADR 0437 §9) after every look, clears those listed no more, and keeps which it
  raised in its new bucket conditions. A held group's line says "half merged" when a member merged.

A pass held for a required status no longer retries on a timer: the status turning success is the event
that asks the merge again.
2026-10-25 18:20:58 +01:00
mesh-tools 2df0503034 Merge pull request 'Let the build modules follow their capacity on the bus, so a node raised from capacity 0 starts pulling (issue 1887)' (#479) from fix/puller-wakes-on-capacity into main 2026-10-25 17:02:40 +00:00
jschoubben 23c827e791 Take the mesh-sdk head that wakes a puller on its capacity, within the test budget, in the three build modules (issue 1887)
mesh/delivery merged: source.main moved to novox/mesh-catalog@2df0503034dc
mesh/check pass 23c827e7910c on a85d90374195 | judged by the candidate's mesh-build-go at eab02c80fe86
mesh/delivery-group group fix/puller-wakes-on-capacity merged: every member is merged
2026-10-25 17:44:50 +01:00
jschoubben 3138570c6b Let the build modules follow their capacity on the bus, so a node raised from capacity 0 starts pulling (issue 1887)
g14's Go builder sat idle at capacity 2 because nothing woke its parked
puller. mesh-build-go, mesh-build-node and mesh-build-image now pin mesh-sdk
b9911dc1, whose puller watches the config value capacity as mesh-config
resolved it and wakes on each change, and hand it that watch.
2026-10-25 17:28:00 +01:00
jschoubben a85d903741 Merge pull request 'nextcloud: the module makes its own mesh-admin account right on a 401 and backs off on a refusal, so its shares poll stops being refused every minute (issue 1253)' (#478) from fix/nextcloud-admin-401 into main
Reviewed-on: #478
2026-10-25 16:01:37 +00:00
jschoubben 1cc54959af nextcloud: repair the account only on a 401, never reset the throttle (issue 1253, review 2)
mesh/check pass 1cc54959afcf on a85d90374195 | unit 1994 run, 2322 kept, 5 skipped | integration 2016 run, 2341 kept, 9 skipped | over bound: unit, in…
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
Resetting Nextcloud's brute-force record for the docker gateway cleared it for
every client behind that address: someone guessing passwords could drive the
throttle to 429 and have the module's poll wipe it. A throttle is no evidence
the password is wrong, so a 429 is now backed off only, and the back-off
outlasts the throttle. The account's existence is read from user:info's exit
status, since user:list pages.
2026-10-25 16:57:46 +01:00
jschoubben 962bc359a7 nextcloud: repair only the module's own account, read its existence from the user list, no admin group (issue 1253, review 1)
occ's user:info says 'user not found' on stdout with an empty stderr, so the
error read as 'occ produced no output' and mesh-admin was never made on novox:
occ now says stderr and stdout both, and existence is read from user:list.
The repair could reset whatever account the user setting names, the
operator's included, and the 'admin' fallback is a hand-made install's human
administrator: only mesh-admin is repaired now, and it is the default. The
shares poll needs no admin rights, so the account gets no group. After a
repair the throttle of the caller's address is reset, and a failing repair
reaches the back-off's one loud line with occ's message.
2026-10-25 16:54:51 +01:00
jschoubben 67b13c42d3 nextcloud: repair the account on a throttle too, since a throttle follows refusals (issue 1253)
mesh/check checking 67b13c42d32a on f27f4c8cd58c
mesh/delivery closed: a newer head of the same pull request
Right after the delivery the first answer may be Nextcloud's 429, not a 401;
without a repair then the account would never be made and the back-off would
wait out a throttle that cannot end.
2026-10-25 16:40:54 +01:00
jschoubben 4021724ca4 nextcloud: make the mesh's own account match its secret when Nextcloud refuses it, and back off a refusal that stays (issue 1253)
The image creates NEXTCLOUD_ADMIN_USER only on a fresh install. novox's
instance was installed before the mesh adopted it, so mesh-admin was never
created there, and the shares poll asked Nextcloud every minute with a login
that cannot pass: 401, then Nextcloud's brute-force throttle (429), which
also held the backup before a delivery.

On a refused login the client now adds the account through occ (admin
group), or sets its password to the module's own secret, passing it by
OC_PASS so it is on no argv, and asks once more. A refusal that stays is
said once and the poll backs off from 5 minutes to an hour instead of nine
lines of XML every minute. An empty setting no longer becomes an empty name.
2026-10-25 16:40:01 +01:00
jschoubben f27f4c8cd5 Merge pull request 'mesh-build-go: a package that fails without an assertion carries its real cause, never "PASS" (issue 1901)' (#477) from fix/1901-package-cause into main
Reviewed-on: #477
2026-10-25 15:07:40 +00:00
jschoubben 155124b59a mesh-build-go: a cause is the runtime's line at column 0, and only go's exact summary lines are dropped (issue 1901, review round 1)
mesh/check pass 155124b59af4 on f27f4c8cd58c | unit 2015 run, 2883 kept, 5 skipped | integration 2037 run, 2978 kept, 9 skipped | over bound: unit, in…
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/delivery-group group fix/1901-package-cause held: a member is held: novox/mesh-catalog@155124b59af4 (merged without a passing check: only a person decides…
An indented log line of another running test that held 'panic:' won over the
real timeout panic, since the lines were trimmed before matching. Go prints
the runtime's lines at column 0; a test's log is indented.
2026-10-25 15:58:07 +01:00
jschoubben 6ec6fdaad2 mesh-build-go: a race the race detector reports is a failure's cause, in a test or after PASS (issue 1901)
mesh-controller's try was worded 'PASS' again because the race detector
reported after the tests passed and the program exited 66. The cause is now
the race: its access, the frame that made it and where its goroutine was
created, instead of PASS or testing's generic 'race detected during execution
of test'. Tests from captured go test -race -json (1901/AC4).
2026-10-25 15:52:20 +01:00
jschoubben 663f3de975 mesh-build-go: a package that fails without an assertion carries its real cause, never "PASS" (issue 1901)
A try of mesh-controller kept 'PASS' as its only failure: the package's test
program stopped after printing PASS and the package-level failure took its
first line. Now a failure that is not one test's assertion names the tests that
were running and carries the panic line (a timeout's with its running tests),
a signal or fatal error line, a build's first error, or words saying the
program exited during which tests or after they passed; go test's summary lines
are never kept. Tests from captured go test -json of eight real failures
(1901/AC4).
2026-10-25 15:38:28 +01:00
jschoubben 68f7431d98 Merge pull request 'The provisioner client fills every module again when the socket returns and when its node's own composition moves (issue 1944)' (#476) from fix/r6-node-composition into main
Reviewed-on: #476
2026-10-25 00:18:25 +00:00
jschoubben 53749833bf Fill every module again when the node-engine's socket returns and when the node's own composition moves, so a file an apply left empty is filled (issue 1945)
mesh/delivery-group group fix/r6-node-composition held: a member is held: novox/mesh-sdk@10b6e202320f (merged without a passing check: only a person decides wh…
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check checking 53749833bfbd on 2167b95a3699 | novox/mesh-catalog#476 in group fix/r6-node-composition
2026-10-25 02:02:02 +02:00
mesh-tools 2167b95a36 Merge pull request 'R6: the gate keeps only the merge's states; walk, plan and step code and their verbs go (issue 1946)' (#475) from feat/r6 into main 2026-10-24 23:00:25 +00:00
jschoubben d09cbf1523 Ignore the binaries a local go build leaves in three module roots (issue 1943)
mesh/delivery-group group feat/r6 merged: every member is merged
mesh/delivery delivering: 0 machine step(s) passed
mesh/check pass d09cbf152317 on 6fb0ceeef817 | unit 4378 run, 4719 kept, 15 skipped | integration 5060 run, 4917 kept, 23 skipped | over bound: unit, …
2026-10-25 00:02:45 +02:00
jschoubben ac6712cbb5 Pin mesh-sdk to feat/r6 748c564, which carries main's composer (issue 1943) 2026-10-25 00:02:45 +02:00
jschoubben 5b32342ac2 Merge main into feat/r6: keep R6's gate, which already carries #474's catch-up at start, and drop the seat and walk adoption R6 removed
The reads-outside markers, mesh-build-go's compose job and gitea's forge-account fix come in from main.
2026-10-24 23:03:42 +02:00
jschoubben 6fb0ceeef8 Merge pull request 'R4g finish: mesh-build-go's compose job, the reads-outside marker in the digest, merge-check.sh gone (issues 1295, 1621-1623, 1935)' (#473) from feat/r4g-finish into main
Reviewed-on: #473
2026-10-24 20:56:49 +00:00
mesh-tools c651161787 Merge pull request 'gitea's forge-account reads the forge's own host key file before scanning its sshd, so a forge restart no longer refuses every forge account (issue 1939)' (#472) from fix/1939-forge-host-key into main 2026-10-24 20:05:39 +00:00
jschoubben ee216e4e63 Merge pull request 'The gate keeps a merge it hears before it holds its seat and catches source.main up when it does, so a missed merge no longer leaves the checker on an old trunk (issue 1726)' (#474) from fix/1726-trunk-catch-up into main
Reviewed-on: #474
2026-10-24 19:47:47 +00:00
jschoubben b67b5af5e6 Pin the gate to mesh-sdk feat/r6 at d7a9b968 (issue 1946) 2026-10-24 21:18:26 +02:00
jschoubben d700ceca20 Pin the gate to mesh-sdk feat/r6 at 7f9cc0e5 (issue 1946) 2026-10-24 20:43:41 +02:00
jschoubben 3a1017f559 Pin the gate to mesh-sdk feat/r6 at e45bdc08 (issue 1946) 2026-10-24 20:22:07 +02:00
jschoubben a17d0c8e93 The messenger holds and tells what the delivery view says is in motion, not the controller's plans (issue 1946) 2026-10-24 18:25:38 +02:00
jschoubben 9fa6710296 Pin the gate to mesh-sdk feat/r6 at a523a8cb (issue 1946) 2026-10-24 18:18:06 +02:00
jschoubben 05f849979c Match a fix to a delivered revision by what the gate says of its merge; keep #474's source catch-up at start (issues 1946, 1726) 2026-10-24 18:13:40 +02:00
jschoubben 4dee77df23 Pin the gate to the head of mesh-sdk feat/r6 (issue 1946) 2026-10-24 17:54:30 +02:00
jschoubben 8a79877a15 A newer delivered revision covers the older merges of its repository, as the forge says ancestry (issue 1946) 2026-10-24 17:03:20 +02:00
jschoubben 2139e83a8a Remove a binary built by mistake 2026-10-24 16:58:33 +02:00
jschoubben e7ca8f7af1 The backup module says on the bus when a backup asked for now has ended, so no one polls backed-up (issue 1944, ADR 0539 §16) 2026-10-24 16:58:19 +02:00
jschoubben eb79996336 The tracker derives a fix's delivered from the delivery view's event, not the gate's gone states (issue 1946) 2026-10-24 16:50:50 +02:00
jschoubben 1a271a1fb1 The group's integration test hears the forge's merges once both members are handed (issue 1946, R6-C3) 2026-10-24 16:43:48 +02:00
jschoubben 30d04f6bef Pin the gate to mesh-sdk's release 6 verb tables and prove it holds the seat as they have it (issue 1946, R6-C3)
The integration tests hear the forge's own word that a pull request merged, as the real flow does, and fail on any call of the controller.
2026-10-24 16:35:15 +02:00
jschoubben 5ff337e0dc Gate keeps the merge's six states and asks the controller nothing (issue 1946, R6-C1, C2, C4)
Release 6 deletes the plans: the walk half of the holder, the controller port, the carry-over of the former module and the verbs times, what-if, release, stop and retire-history go; the table holds proposed, ready, rejected, merged, held and closed; the delivery's status and view name the source revision the merge moved.
2026-10-24 16:26:36 +02:00
jschoubben 2fabbe556e Pin to the group's new mesh-sdk head bfc3883a, whose composer reads a fallback to a predefined key as the indexer does (ADR 0348 §7)
mesh/delivery delivered
mesh/delivery-group group feat/r4g-finish merged: every member is merged
mesh/check checking 2fabbe556e73 on 6fb0ceeef817 | novox/mesh-catalog#473 in group feat/r4g-finish
2026-10-24 14:57:35 +02:00
jschoubben 89b9212c4d Pin to the group's new mesh-sdk head 19c1d359, whose composer names the reader's pinned SDK instead of a bare (devel) (ADR 0482 clause 5)
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: shanks: novox/mesh-catalog#473: modules/mesh-build-go/go.mod names mesh-sdk 19c1d35939bc, which is neither on mesh-sdk's trunk nor …
2026-10-24 14:12:09 +02:00
jschoubben 40da368ee3 Date an adopted merge once, by the walk it was adopted from, so a later walk it follows never makes it look newer than a forge merge heard after it (issue 1726, review)
mesh/check pass 40da368ee34e on ee216e4e63f1 | unit 246 run, 4189 kept | integration 228 run, 4249 kept | over bound: unit, integration (slowest unit …
mesh/delivery delivered
2026-10-24 13:55:10 +02:00
jschoubben 865f3b6455 Say what pacman-tools reads outside: the root predefined.json (issue 1935)
mesh/delivery-group group feat/r4g-finish rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: shanks: novox/mesh-catalog#473: modules/mesh-build-go/go.mod names mesh-sdk bc0c25e51052, which is neither on mesh-sdk's trunk nor …
2026-10-24 13:48:18 +02:00
jschoubben 03524ecd13 Mark pacman-tools as reading outside its directory: its upgrade-order test reads the root predefined.json, which its digest does not cover (issue 1935) 2026-10-24 13:48:09 +02:00
jschoubben 15101c9ce6 Move source.main for a merge the gate caught up after a gap, so the checker re-checks open heads on the trunk that is (issue 1726)
The checker takes a repository's trunk head only from the gate's source.main. On 2026-10-24 gitea's
restarted poll announced mesh-catalog #471's merge while the gate was starting and could not yet tell
whether it held the seat; the gate acknowledged the event unread. It then adopted the controller's walk
of the merge commit, which carries no merge time, so every older merge of the repository looked newer
and source.main was never moved: #472 was checked on the trunk before #471 and its merge refused.

- An event heard before the gate can tell whether it holds the seat is offered again, not taken.
- An adopted merge is dated by its walk's creation, so it settles into source.main as a forge merge does.
- Each time the gate comes to hold the seat it weighs the newest merge of each repository against
  source.main once more, so a merge marked there and never written (as #471) is written; never backwards.
2026-10-24 13:41:56 +02:00
jschoubben f910b0ceb8 Pin mesh-build-go to the group's mesh-sdk head after main's merge, so it builds on the SDK the group delivers 2026-10-24 13:28:16 +02:00
jschoubben 91c3e954ae Merge main into feat/r4g-finish (#471) 2026-10-24 13:26:24 +02:00
jschoubben 545c72bb59 Merge pull request 'gitea's watcher answers its handshake without waiting for the forge, so a slow forge at a tool-runner restart no longer kills its tools and events (issue 1726)' (#471) from fix/1726-gitea-handshake-not-on-forge into main
Reviewed-on: #471
2026-10-24 11:10:39 +00:00
jschoubben 7afcd0282d Set up the forge's webhook without holding the bundle's import, so a slow forge never costs gitea its handshake and its tools (issue 1726)
At 12:37 on 2026-10-24 node-tools on novox was replaced; gitea's watcher bundle awaited the webhook's set-up on the forge's admin API at the top of its import, outlasted the 10 s handshake and was killed, never to start again. gitea_commit_status, gitea_merge_pull_request and gitea_events were then served nowhere: the forge bridge's calls found no responders and no new pull request was announced.
2026-10-24 13:00:57 +02:00
jschoubben 75156fce00 gitea: forge-account reads the forge's own host key file before scanning its sshd (issue 1939)
mesh/delivery delivered
mesh/check pass 75156fce007e on c65116178720 | unit 246 run, 4192 kept | integration 228 run, 4252 kept | over bound: unit, integration (slowest unit …
2026-10-24 12:50:20 +02:00
jschoubben 16d7b1898d Mark the packages whose tests read files outside their directory, found by the review (issue 1935) 2026-10-24 04:38:49 +02:00
jschoubben f04465537b Say in realtree_test that a missing trunk fails, not skips (issue 1295) 2026-10-24 04:37:52 +02:00
jschoubben e870a7f65f Pin to the finish group's mesh-sdk head dd0d156b, so every member names one SDK (issue 1622) 2026-10-24 04:28:14 +02:00
jschoubben e8a59171c7 Merge remote-tracking branch 'origin/feat/r4g-finish-composer' into HEAD 2026-10-24 04:26:03 +02:00
jschoubben ebb25757a2 Merge remote-tracking branch 'origin/feat/r4g-finish-1935' into HEAD 2026-10-24 04:26:02 +02:00
jschoubben 6a70735489 Run a compose ask on the Go build seat: build mesh-sdk's composer at the asked commit once, fill its manifests from the indexer and run it shut off (issue 1622) 2026-10-24 04:14:21 +02:00
jschoubben a715c8821f mesh-build-go: a catalogue module's non-Go files enter its packages' digests; markers where tests read other modules (issue 1935) 2026-10-24 03:59:08 +02:00
jschoubben b6ed8599c8 mesh-build-go: a package marked reads-outside is never kept, so tests that read files outside their directory run on every check (issue 1935) 2026-10-24 03:46:45 +02:00
jschoubben ae6f7d5b54 Make a missing trunk copy a failure again, now the checker places it (issue 1295) 2026-10-24 03:43:49 +02:00
jschoubben 25176e1ccd Merge pull request 'mesh-build-go: lint the unit level's directories as many at once as its tests, so a large repository's check ends within its bound (issue 1930)' (#470) from fix/1930-parallel-lint into main
Reviewed-on: #470
2026-10-24 00:16:55 +00:00
jschoubben 7e1f3de77a mesh-build-go: lint directories side by side like the tests, so a large repository's check stays inside its bound (issue 1930)
mesh/delivery delivered
mesh/check failed: mesh-build-go: the check of novox/mesh-catalog#470: the ask mvlnph6fd84a7b's step on shanks did not end within 30m0s of its start |…
2026-10-24 02:01:36 +02:00
jschoubben 4b1bed4b9d Merge pull request 'mesh-mail-reader: read messenger's sent_on as a list, so an unchanged set of mails is signalled once, not every 160 s (issue 1926)' (#469) from fix/1926-mail-signal-once into main
Reviewed-on: #469
2026-10-23 22:36:10 +00:00
jschoubben 25d69406b7 Merge pull request 'mesh-build-go: a failed test keeps its last lines and its cause, not its first fifteen (issue 1901, with mesh-delivery)' (#466) from fix/1901-rebased into main
Reviewed-on: #466
2026-10-23 22:35:57 +00:00
jschoubben f836c021e1 Merge pull request 'gitea: the checker's owed results are taken at once and written in the background, so the runner keeps the watch and every mesh/check reaches the forge (issue 1726)' (#468) from fix/1726-owed-watch-not-blocked into main
Reviewed-on: #468
2026-10-23 22:35:33 +00:00
jschoubben 450f0bbe79 Merge pull request 'The build modules pin mesh-sdk's list-form level digest, so a check of a repository with a package named token ends (issue 1930, with mesh-sdk and mesh-delivery)' (#467) from fix/1930-digest-shape into main
Reviewed-on: #467
2026-10-23 22:08:24 +00:00
jschoubben 5a7076a6a2 Take the checker's owed without waiting on the forge, so a large owed never times its watch out (issue 1726)
mesh/delivery delivered
mesh/check checking 5a7076a6a25e on f836c021e14f
The runtime gives a watch's current values 10 s and waits for each to be taken; a forge write per key made ~200 keys outlast it at every attempt, so after node-tools restarted on novox at 23:25:58 the watch never held and no verdict reached mesh/check. Writes now run in the background, a few at a time, each key's in revision order.
2026-10-23 23:52:52 +02:00
jschoubben 30b22c198e Pin the build modules to mesh-sdk 403a5d7, whose level digest is a list, so a check of a repository with a package named token ends (issue 1930)
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792875397682979170 (novox/mesh-catalog at 6fb0ceee into main,…
mesh/delivery-group group fix/1930-digest-shape merged: every member is merged
mesh/check checking 30b22c198e6a on 450f0bbe7984 | novox/mesh-catalog#467 in group fix/1930-digest-shape
2026-10-23 23:36:31 +02:00
jschoubben 753e8fc202 mesh-mail-reader: read tell's sent_on as the list it is, so a told set is not sent again; leave past-due and duplicate mails out (issue 1926)
mesh/delivery delivered
mesh/check checking 753e8fc202c3 on 4b1bed4b9d5b
2026-10-23 23:27:23 +02:00
mesh-tools e5202445ba Merge pull request 'mesh-build-go: build_go_test takes packages as a typed list too, so checks run again (issue 1925)' (#465) from fix/1925-packages-shape into main 2026-10-23 20:44:09 +00:00
jschoubben dcae699323 mesh-build-go: a clipped failure starts with its cause, not the marker, for an older checker (issue 1901/AC3)
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-catalog@753e8fc202c3 (merged as 4b1bed4b into main, walk plan-17927949853…
mesh/delivery-group group fix/1901-rebased merged: every member is merged
mesh/check checking dcae699323c8 on 25d69406b740 | novox/mesh-catalog#466 in group fix/1901-rebased
2026-10-23 22:26:29 +02:00
jschoubben 70ef99595a mesh-build-go: build_go_test takes packages as a typed list too (issue 1925)
mesh/delivery delivered
mesh/check failed: shanks: the check of novox/mesh-catalog#465 failed: the unit level: packages is a list of package patterns | 70ef99595aa9 on e52024…
2026-10-23 22:21:41 +02:00
jschoubben 746eb885a3 Test 1901/AC3: an older pair of build module and checker still agree (issue 1901) 2026-10-23 22:14:28 +02:00
jschoubben 2cdb9b8fc4 mesh-build-go: a failed test keeps its last lines and its error line, not its first fifteen (issue 1901) 2026-10-23 22:13:41 +02:00
jschoubben e830045545 Merge pull request 'R4g+R5 (mesh-catalog): the build modules pull asks, scripts become tests, build-agent references go (design 47)' (#464) from feat/r4g into main
Reviewed-on: #464
2026-10-23 17:49:43 +00:00
jschoubben 454fb8a918 Write a pull request's check verdict into its key after the key is created: the status event can be taken first (R4g, issue 1887)
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792777718506798022 (novox/mesh-catalog at e8300455 into main,…
mesh/check checking 454fb8a918ad on 3bec357968da | novox/mesh-catalog#464 in group feat/r4g
mesh/delivery-group group feat/r4g merged: every member is merged
2026-10-23 19:42:54 +02:00
jschoubben c32474ee40 Judge the delivery-before-check order from the step records, not a poll: at capacity 1 the check starts the instant the delivery ends (R4g, issue 1887)
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: integration: mesh-delivery-gate/cmd/mesh-d… | c32474ee40f4 on 3bec357968da | judged by the candidate's mesh-build-go at 2e667ac15ce7
2026-10-23 19:19:45 +02:00
jschoubben f95f3f430e Give the delivery-ask integration test a bus of its own: the level shares one bus across directories, so a named stream there could be taken by another test (R4g, issue 1887) 2026-10-23 19:15:52 +02:00
253 changed files with 7458 additions and 10998 deletions
+5
View File
@@ -29,3 +29,8 @@ modules/mesh-build-go/cmd/mesh-build-go/mesh-build-go
modules/mesh-config/mesh-config
modules/mesh-build-image/cmd/mesh-build-image/mesh-build-image
*.test
# binaries a local go build leaves in a module root
/checks/paths/paths
/modules/mesh-build-go/mesh-build-go
/modules/mesh-delivery-gate/mesh-delivery-gate
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+2 -1
View File
@@ -251,7 +251,8 @@ func main() {
}
var grown []string
if baseDir == "" {
fmt.Println("NOT COMPARED: no base branch's lists were given, so additions to them were not judged")
fmt.Fprintln(os.Stderr, "NOT COMPARED: no base branch's lists were given (-base <dir>), so additions to them were not judged; that is a failure, not a pass (issue 1295)")
os.Exit(2)
} else {
baseShared, _, err1 := list(filepath.Join(baseDir, "shared"))
baseKnown, _, err2 := list(filepath.Join(baseDir, "known"))
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+2 -2
View File
@@ -32,8 +32,8 @@ func TestNoModuleNamesAnothersState(t *testing.T) {
}
}
// And neither list gains a line the trunk's copies at the merge base do not hold, unless the line says why after `# allow:`. Skipped,
// and said, where no trunk is beside.
// And neither list gains a line the trunk's copies at the merge base do not hold, unless the line says why after `# allow:`. Fails, saying
// NOT COMPARED, where no trunk is beside (issue 1295).
func TestTheListsOfPathsHoldNoLineTheTrunksDoNot(t *testing.T) {
sharedPath, okShared := trunk.Copy(t, "checks/paths/shared")
knownPath, okKnown := trunk.Copy(t, "checks/paths/known")
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+14 -18
View File
@@ -3,14 +3,14 @@
//
// The checker fetches the change's own repository at its base commit beside the change, as it fetches a sibling, and says where
// in MESH_CHECK_BESIDE; the copy is under the repository's name (the same convention the other repositories' tests use). With
// MESH_CHECK_BESIDE unset, or set without the catalogue's copy in it, the comparison is skipped and the test says NOT COMPARED
// WITH THE TRUNK, naming why: the checker that runs before R4g's delivery places siblings but not the change's own repository,
// so R4g's own check cannot hold the copy (issue 1295 notes that a missing copy becomes a failure once that checker is gone). A trunk that holds no such file had
// MESH_CHECK_BESIDE unset, or set without the catalogue's copy in it, the test fails saying NOT COMPARED WITH THE TRUNK and
// why: the checker places the change's own repository since R4g (issue 1295), so a missing copy is a fault, never a pass. A trunk that holds no such file had
// none: the list's first copy, which nothing was added to.
package trunk
import (
"errors"
"fmt"
"os"
"path/filepath"
"testing"
@@ -23,13 +23,11 @@ const Repository = "mesh-catalog"
const BesideEnv = "MESH_CHECK_BESIDE"
// Copy is the trunk's copy of the file at rel (a path from the catalogue's root), written to a file of the test's own: its path,
// and whether the trunk holds one (false: its first copy). It skips the test, saying so, when no copy of the catalogue is beside.
// and whether the trunk holds one (false: its first copy). It fails the test when no copy of the catalogue is beside.
func Copy(t testing.TB, rel string) (string, bool) {
t.Helper()
raw, found, skip, err := Read(rel)
raw, found, err := Read(rel)
switch {
case skip != "":
t.Skip(skip)
case err != nil:
t.Fatal(err)
case !found:
@@ -43,27 +41,25 @@ func Copy(t testing.TB, rel string) (string, bool) {
return path, true
}
// Read is the trunk's copy of rel and whether the trunk holds one. skip is the words to skip the comparison with, when no trunk is
// named; err is a trunk named and not there, or unreadable.
func Read(rel string) (content []byte, found bool, skip string, err error) {
// Read is the trunk's copy of rel and whether the trunk holds one. err is a trunk not beside the check (MESH_CHECK_BESIDE unset,
// or without the catalogue's directory), or a copy unreadable: a comparison that cannot be made is never a pass.
func Read(rel string) (content []byte, found bool, err error) {
beside := os.Getenv(BesideEnv)
if beside == "" {
return nil, false, "NOT COMPARED WITH THE TRUNK: " + BesideEnv + " is not set, so the trunk's copy of " + rel +
" is not beside this run; the checker places it", nil
return nil, false, fmt.Errorf("NOT COMPARED WITH THE TRUNK: %s is not set, so the trunk's copy of %s is not beside this run; the checker places the change's own repository there (issue 1295)", BesideEnv, rel)
}
dir := filepath.Join(beside, Repository)
if _, err := os.Stat(dir); errors.Is(err, os.ErrNotExist) {
return nil, false, "NOT COMPARED WITH THE TRUNK: " + Repository + " is not beside this check in " + BesideEnv + "=" + beside +
", so " + rel + " is not judged against the trunk's copy (the checker before R4g places no copy of the change's own repository)", nil
return nil, false, fmt.Errorf("NOT COMPARED WITH THE TRUNK: %s is not beside this check in %s=%s, so %s is not judged against the trunk's copy; the checker places the change's own repository there at its trunk (issue 1295)", Repository, BesideEnv, beside, rel)
} else if err != nil {
return nil, false, "", err
return nil, false, err
}
raw, err := os.ReadFile(filepath.Join(dir, rel))
if errors.Is(err, os.ErrNotExist) {
return nil, false, "", nil
return nil, false, nil
}
if err != nil {
return nil, false, "", err
return nil, false, err
}
return raw, true, "", nil
return raw, true, nil
}
+14 -14
View File
@@ -17,27 +17,27 @@ func TestTheCloneBesideTheCheckIsTheTrunk(t *testing.T) {
t.Fatal(err)
}
t.Setenv(BesideEnv, beside)
raw, found, skip, err := Read("checks/x/known")
if err != nil || skip != "" || !found || string(raw) != "a\nb\n" {
t.Fatalf("read %q found %v skip %q err %v", raw, found, skip, err)
raw, found, err := Read("checks/x/known")
if err != nil || !found || string(raw) != "a\nb\n" {
t.Fatalf("read %q found %v err %v", raw, found, err)
}
if _, found, skip, err := Read("checks/x/absent"); err != nil || skip != "" || found {
t.Fatalf("a file the trunk does not hold: found %v skip %q err %v; wanted its first copy", found, skip, err)
if _, found, err := Read("checks/x/absent"); err != nil || found {
t.Fatalf("a file the trunk does not hold: found %v err %v; wanted its first copy", found, err)
}
}
func TestWithNothingBesideTheComparisonIsSkippedAndSaidNotMade(t *testing.T) {
func TestWithNothingBesideTheComparisonFailsAndSaysNotMade(t *testing.T) {
t.Setenv(BesideEnv, "")
_, found, skip, err := Read("checks/x/known")
if found || err != nil || len(skip) < 12 || skip[:12] != "NOT COMPARED" {
t.Fatalf("found %v, skip %q, err %v; wanted NOT COMPARED", found, skip, err)
_, found, err := Read("checks/x/known")
if found || err == nil || !strings.Contains(err.Error(), "NOT COMPARED") {
t.Fatalf("found %v, err %v; wanted a failure saying NOT COMPARED", found, err)
}
}
func TestACloneBesideWithoutTheCatalogueIsSkippedAndSaidNotMade(t *testing.T) {
t.Setenv(BesideEnv, t.TempDir()) // the siblings are there, the catalogue's own base is not (the checker before R4g)
_, found, skip, err := Read("checks/x/known")
if found || err != nil || len(skip) < 12 || skip[:12] != "NOT COMPARED" || !strings.Contains(skip, "not beside") {
t.Fatalf("found %v, skip %q, err %v; wanted NOT COMPARED naming the missing copy", found, skip, err)
func TestACloneBesideWithoutTheCatalogueFailsAndSaysNotMade(t *testing.T) {
t.Setenv(BesideEnv, t.TempDir()) // the siblings are there, the catalogue's own base is not
_, found, err := Read("checks/x/known")
if found || err == nil || !strings.Contains(err.Error(), "NOT COMPARED") || !strings.Contains(err.Error(), "not beside") {
t.Fatalf("found %v, err %v; wanted a failure naming the missing copy", found, err)
}
}
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+28 -9
View File
@@ -83,6 +83,9 @@ export function checkStatusOf(change: OwedChange): { owner: string; repo: string
return { owner, repo, sha, revision, status };
}
/** How many mesh/check writes this module has at the forge at once. */
export const WRITES_AT_ONCE = 4;
/** The one writer of mesh/check: each owed revision of the checker's, in revision order per key. */
export class CheckStatuses {
/** The newest revision written of each key, and the newest seen; the oldest keys go past what any forge holds open. */
@@ -91,6 +94,9 @@ export class CheckStatuses {
/** The turn each key's writes wait on, and the asks again still running. */
private turns = new Map<string, Promise<unknown>>();
private again = new Set<Promise<void>>();
/** Writes at the forge at once, at most WRITES_AT_ONCE: a watch's start hands over every key together (issue 1726). */
private writing = 0;
private queued: (() => void)[] = [];
private forge: CheckForge;
private say: (line: string) => void;
private sleep: (ms: number) => Promise<void>;
@@ -123,7 +129,7 @@ export class CheckStatuses {
this.say(`[gitea] ${asked.owner}/${asked.repo}@${asked.sha.slice(0, 12)}: mesh/check at revision ${asked.revision} is older than ${newest}, already written, and is dropped`);
return "dropped";
}
await this.forge.setCommitStatus(asked.owner, asked.repo, asked.sha, asked.status);
await this.atTheForge(() => this.forge.setCommitStatus(asked.owner, asked.repo, asked.sha, asked.status));
this.remember(this.written, key, asked.revision);
return "written";
});
@@ -133,19 +139,32 @@ export class CheckStatuses {
return turn;
}
/** What the watch hands over: written, or when the forge refuses, asked again on a bound in the background until it is
* written or a newer revision of the key was seen. Never throws: a watch's handler that throws loses the change. */
/** What the watch hands over: taken at once and written in the background, or when the forge refuses, asked again on a
* bound until it is written or a newer revision of the key was seen. Never throws: a watch's handler that throws loses
* the change. It never waits on the forge (novox/hq issue 1726): the runtime hands a watch's current values one at a
* time, waits for each to be taken and gives the whole set 10 s; a forge write per key made a large owed outlast that
* bound at every attempt, so the watch never held and no verdict reached mesh/check. Each key's writes still take
* turns in revision order (apply). */
async take(change: OwedChange): Promise<void> {
const run = this.apply(change).then(() => {}, (err) => this.askAgain(change, err));
this.again.add(run);
void run.finally(() => this.again.delete(run));
}
/** Runs one forge write when fewer than WRITES_AT_ONCE are running, else after one ends. */
private async atTheForge<T>(write: () => Promise<T>): Promise<T> {
if (this.writing >= WRITES_AT_ONCE) await new Promise<void>((r) => this.queued.push(r));
else this.writing++;
try {
await this.apply(change);
} catch (err) {
const run = this.askAgain(change, err);
this.again.add(run);
void run.finally(() => this.again.delete(run));
return await write();
} finally {
const next = this.queued.shift();
if (next) next();
else this.writing--;
}
}
/** Resolves when no ask again is running. */
/** Resolves when no write or ask again is running. */
async settled(): Promise<void> {
while (this.again.size) await Promise.all([...this.again]);
}
@@ -65,6 +65,7 @@ func configFromEnv(getenv func(string) string) (Config, error) {
HostFile: getenv("MESH_FORGE_HOST_FILE"),
OriginsFile: getenv("MESH_GITEA_ORIGINS_FILE"),
Port: port,
HostKeyFile: getenv("MESH_GITEA_HOST_KEY_FILE"), // optional
Forge: forgehttp.Client{URL: strings.TrimRight(getenv("MESH_GITEA_URL"), "/"), User: getenv("MESH_GITEA_ADMIN_USER"),
PasswordFile: getenv("MESH_GITEA_ADMIN_PASSWORD_FILE"), Timeout: 20 * time.Second},
Runner: execRunner{},
@@ -44,10 +44,13 @@ type Config struct {
// origins, one per line.
HostFile, OriginsFile string
// Port is where the forge's sshd is published on the mesh.
Port int
Forge forgehttp.Client
Runner Runner
Log func(format string, a ...any)
Port int
// HostKeyFile is the forge's own ed25519 host key (the .pub file in its data directory). It is read first; when it
// is unset or unreadable the forge's sshd is asked with ssh-keyscan instead (issue 1939).
HostKeyFile string
Forge forgehttp.Client
Runner Runner
Log func(format string, a ...any)
}
// Service answers the consumers of the provision `forge-account` (novox/hq ADR 0399, ADR 0400; issue 955): for each
@@ -448,14 +451,30 @@ func (s *Service) httpsBase() (string, error) {
return "", fmt.Errorf("%s names no https origin", s.cfg.OriginsFile)
}
// scan reads the forge's ed25519 host key from its own sshd and writes it as the machines' known_hosts line.
// scan gives the forge's ed25519 host key as the machines' known_hosts line. The key the forge itself holds is read
// from its file first; only when that cannot be read is the forge's sshd asked over the network, and when that fails
// too (the forge's container is restarting) the key read before is kept, as a host key does not change (issue 1939).
func (s *Service) scan(ctx context.Context) (string, error) {
host, err := s.hostName()
if err != nil {
return "", err
}
if s.cfg.HostKeyFile != "" {
raw, ferr := os.ReadFile(s.cfg.HostKeyFile)
if ferr == nil {
if line, perr := knownHostLine("127.0.0.1 "+strings.TrimSpace(string(raw)), host, s.cfg.Port); perr == nil {
return line, nil
}
}
}
out, err := s.cfg.Runner.Run(ctx, "ssh-keyscan", "-t", "ed25519", "-p", fmt.Sprint(s.cfg.Port), "127.0.0.1")
if err != nil {
s.mu.Lock()
kept := s.known
s.mu.Unlock()
if kept != "" {
return kept, nil
}
return "", fmt.Errorf("the forge's ssh host key cannot be read: %w", err)
}
return knownHostLine(string(out), host, s.cfg.Port)
@@ -696,3 +696,42 @@ func TestNothingIsSaidForAPerNodeUserWithoutMeshKeys(t *testing.T) {
}
}
}
// The forge's own host key file is read first: no scan over the network is needed (issue 1939).
func TestTheHostKeyIsReadFromTheForgesOwnFile(t *testing.T) {
r := newRig(t)
f := filepath.Join(t.TempDir(), "ssh_host_ed25519_key.pub")
if err := os.WriteFile(f, []byte("ssh-ed25519 AAAAC3FILEKEY forge@host\n"), 0o644); err != nil {
t.Fatal(err)
}
r.svc.cfg.HostKeyFile = f
r.runner.err = errors.New("sshd down")
got, err := r.svc.scan(context.Background())
if err != nil || got != "[novox.internal]:222 ssh-ed25519 AAAAC3FILEKEY" {
t.Fatalf("scan = %q, %v", got, err)
}
if r.runner.args != nil {
t.Errorf("the forge's sshd was asked: %v", r.runner.args)
}
}
// A file that cannot be read falls back to the forge's sshd.
func TestAnUnreadableHostKeyFileFallsBackToTheScan(t *testing.T) {
r := newRig(t)
r.svc.cfg.HostKeyFile = filepath.Join(t.TempDir(), "missing.pub")
got, err := r.svc.scan(context.Background())
if err != nil || got != "[novox.internal]:222 ssh-ed25519 AAAAC3HOSTKEY" {
t.Fatalf("scan = %q, %v", got, err)
}
}
// While the forge restarts, a key read before is kept: the sshd is not up, the key has not changed.
func TestARestartingForgeKeepsTheKeyReadBefore(t *testing.T) {
r := newRig(t)
r.svc.known = "[novox.internal]:222 ssh-ed25519 AAAAC3HOSTKEY"
r.runner.err = errors.New("connection refused")
got, err := r.svc.scan(context.Background())
if err != nil || got != r.svc.known {
t.Fatalf("scan = %q, %v", got, err)
}
}
+5 -1
View File
@@ -589,7 +589,11 @@ if (gitea) {
};
// **The forge's events first, the polls as the safety net** (novox/hq issue 729): while verified deliveries arrive the
// polls slow to every five minutes and announce only what no event said (Pace); otherwise they keep 30 s.
await listenForForge(client);
// **Not awaited** (novox/hq issue 1726): the receiver's set-up asks the forge's admin API, and the runtime kills a bundle
// whose import has not ended in its 10 s handshake and never starts it again; a forge slow to answer while every bundle
// of the node starts left gitea_commit_status, gitea_merge_pull_request and gitea_events served nowhere. Until it is
// listening, Pace keeps the polls at 30 s, so nothing is missed meanwhile; listenForForge says its own failure.
void listenForForge(client);
// Main takes no push and only the tools' account merges, in every repository, kept and not only when a token is
// minted: at the start and every ten minutes after, so a repository made later gets it too (issue 948, ADR 0398 §2).
// Only while the setting keep-main-rules is true (mainrules.ts); false, the default, leaves every repository's rule as set.
+13
View File
@@ -15,6 +15,19 @@ export function keepMainRules(env: NodeJS.ProcessEnv = process.env): boolean {
} catch { return false; }
}
/** The setting check-gates-merge (the operator, 2026-10-25 22:56: "run the check locally only, but not as a gate, it loses
* us too much time"): whether the merge tool judges mesh/check before it merges. Absent, unreadable or anything but false
* is true; false lets a head the operator's Approve names merge with mesh/check pending, failed, missing or judged on an
* older trunk (mesh-delivery-gate's setting of the same name stops waiting for it before the Approve). */
export function checkGatesMerge(env: NodeJS.ProcessEnv = process.env): boolean {
const file = env.MESH_GITEA_CONFIG_FILE;
if (!file) return true;
try {
const v = (JSON.parse(readFileSync(file, "utf8")) as Record<string, unknown>)["check-gates-merge"];
return !(v === false || v === "false");
} catch { return true; }
}
/** The 10-minute step: puts the rules right only when kept; else says once that they are left and writes nothing. */
export function mainRulesStep(keep: boolean, put: () => Promise<string[]>, say: (line: string) => void): () => Promise<void> {
let said = false;
+8 -2
View File
@@ -19,6 +19,7 @@ import { mergeableUnder, newestByContext } from "./checks.js";
import { MERGE_METHODS, Refused, checkName } from "./guard.js";
import { CHECK_CONTEXT, CHECK_STATUS, REPO_CHECK_CONTEXT, maintained, requiredOf } from "./pulls.js";
import { approvedHeadsOf, groupRefusal, memberKey, namesSiblings, notApproved, readSiblings, siblingOutOfDate, siblingsFile } from "./siblings.js";
import { checkGatesMerge } from "./mainrules.js";
import { announcementOf, askedWords, outOfDateWords, sameCommit, tookFile, tookWords, twelve, verdictAtTrunk, waitsForComposed } from "./trunk.js";
/** A head named for a merge: at least 12 hex characters of its sha (novox/hq issue 381), up to a sha-256 commit's 64. */
@@ -31,11 +32,13 @@ export interface MergeDeps {
emit?: (type: string, body: unknown) => Promise<void>;
stateDir?: string;
now?: () => Date;
/** Whether mesh/check gates the merge: the setting check-gates-merge, read at each merge (mainrules.ts). */
checkGates?: () => boolean;
}
export function mergeTool(gitea: GiteaClient, given: MergeDeps): ToolDefinition {
const deps = { ...given, emit: given.emit ?? announce, stateDir: given.stateDir ?? process.env.MESH_GITEA_STATE_DIR,
now: given.now ?? (() => new Date()) };
now: given.now ?? (() => new Date()), checkGates: given.checkGates ?? (() => checkGatesMerge()) };
return {
name: "gitea_merge_pull_request",
description: "Merge a pull request at the head you name — the commit you reviewed — when that is still its head, " +
@@ -126,7 +129,10 @@ export function mergeTool(gitea: GiteaClient, given: MergeDeps): ToolDefinition
}
const kept = maintained(required);
const byCheck = required.includes(CHECK_STATUS) || kept.length === 0;
if (byCheck) {
// **check-gates-merge false** (the operator, 2026-10-25): mesh/check is information, not a gate. A head the operator's
// Approve names merges whatever its mesh/check says, pending, failed, missing or judged on an older trunk; without
// approved_heads the check still judges.
if (byCheck && (deps.checkGates() || !approved)) {
// **mesh/check is the checker's** (ADR 0447 §4): it is judged here against the trunk's head, and refused when it
// is not current; nothing is written or asked — the checker checks a head again when its trunk moves.
const check = checks[CHECK_STATUS];
+9 -1
View File
@@ -181,6 +181,12 @@
"trusted": true,
"default": {}
},
"check-gates-merge": {
"type": "boolean",
"description": "Whether gitea_merge_pull_request judges mesh/check before it merges. True (the default): a head merges only on a passing mesh/check judged against the trunk's head (ADR 0447, ADR 0534 §4). False (the operator, 2026-10-25 22:56: \"run the check locally only, but not as a gate, it loses us too much time\"): a head the operator's Approve names in approved_heads merges whatever its mesh/check says (pending, failed, missing or judged on an older trunk); a head that moved, a branch no rule guards and a merge git refuses are still refused. Set together with mesh-delivery-gate's check-gates-merge.",
"trusted": true,
"default": true
},
"main-mergers": {
"type": "list",
"description": "The forge logins that may merge into main beside the tools' account, on every repository of the organisation (ADR 0534 §2): the operator's own, so the operator can always merge by hand at the forge. The tools' account is always on main's merge whitelist; no agent's account belongs here.",
@@ -316,7 +322,8 @@
"takes-config": [
"url",
"keep-main-rules",
"main-mergers"
"main-mergers",
"check-gates-merge"
]
},
{
@@ -537,6 +544,7 @@
"MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret",
"MESH_GITEA_ORIGINS_FILE": "${dir:mesh-state}/origins",
"MESH_GITEA_SSH_PORT": "222",
"MESH_GITEA_HOST_KEY_FILE": "${dir:data}/ssh/ssh_host_ed25519_key.pub",
"MESH_FORGE_HOST_FILE": "/etc/gitea/forge-host"
}
}
+50
View File
@@ -109,3 +109,53 @@ test("a bundle whose read of the checker's owed is granted late writes the lates
assert.match(got.statuses[0], /"state":"success".*pass at 4/);
assert.match(err, /mesh-delivery-checker\.owed could not be bound/, "the refusal is said");
});
// **The bundle's import does not wait on the forge** (novox/hq issue 1726, 2026-10-24 12:37). The webhook receiver's set-up
// asks the forge's admin API for its system webhook; awaited at the top of index.ts, a forge slow to answer (busy while
// every bundle of the node starts) held the import past node-tools' 10 s handshake, the bundle was killed and never
// started again, and gitea_commit_status, gitea_merge_pull_request and gitea_events were served nowhere: the forge
// bridge's calls found no responders and no new pull request was announced. Here the forge stand-in takes every
// request and never answers: the import resolves at once all the same, with the bundle's tools registered.
test("importing the bundle resolves, its tools registered, while the forge never answers the webhook's set-up", async () => {
const index = pathToFileURL(join(here, "..", "dist", "index.js")).href;
const script = `
const http = await import("node:http");
const fs = await import("node:fs");
const os = await import("node:os");
const path = await import("node:path");
const forge = http.createServer(() => {});
await new Promise((r) => forge.listen(0, "127.0.0.1", r));
const dir = fs.mkdtempSync(path.join(os.tmpdir(), "gitea-1726-"));
fs.writeFileSync(path.join(dir, "admin.password"), "pw");
process.env.MESH_GITEA_URL = "http://127.0.0.1:" + forge.address().port;
process.env.MESH_GITEA_STATE_DIR = dir;
process.env.MESH_GITEA_ADMIN_PASSWORD_FILE = path.join(dir, "admin.password");
process.env.MESH_GITEA_HOOK_PORT = "0";
process.env.MESH_GITEA_HOOK_HOST = "127.0.0.1";
const { useBroker } = await import("@novox/mesh-sdk/messaging");
useBroker(() => ({
subscribe: async () => () => {}, publish: async () => {}, request: async () => { throw new Error("no responders"); },
handle: async () => () => {}, close: async () => {},
}));
await import(${JSON.stringify(index)});
const { collectTools } = await import("@novox/mesh-sdk/tools");
console.log(JSON.stringify({ imported: true, tools: collectTools().flatMap((r) => r.tools.map((t) => t.name)) }));
process.exit(0);`;
const env = { ...process.env, MESH_GITEA_TOKEN: "x", MESH_GITEA_ADMIN_USER: "admin", MESH_GITEA_HOOK_URL: "http://127.0.0.1:1" };
for (const k of ["MESH_SERVED_MODULE", "MESH_GITEA_ADMIN_PASSWORD", "MESH_GITEA_CONFIG_FILE"]) delete env[k];
const child = spawn(process.execPath, ["--input-type=module", "-e", script], { cwd: join(here, ".."), env, stdio: ["ignore", "pipe", "pipe"] });
let out = "";
let err = "";
child.stdout.on("data", (d) => (out += d));
child.stderr.on("data", (d) => (err += d));
const ended = new Promise<void>((r) => child.on("exit", () => r()));
const timer = setTimeout(() => child.kill("SIGKILL"), 3000);
await ended;
clearTimeout(timer);
const line = out.split("\n").find((l) => l.startsWith("{"));
assert.ok(line, `the import did not resolve within 3 s while the forge did not answer; stdout: ${out.slice(0, 400)} stderr: ${err.slice(0, 600)}`);
const got = JSON.parse(line!) as { tools: string[] };
for (const t of ["gitea_events", "gitea_commit_status", "gitea_merge_pull_request"]) {
assert.ok(got.tools.includes(t), `${t} is registered: ${got.tools}`);
}
});
+42
View File
@@ -133,3 +133,45 @@ test("no tool writes mesh/check: gitea_check_status is gone and gitea_commit_sta
const index = readFileSync(new URL("../index.ts", import.meta.url), "utf8");
assert.ok(!/gitea_check_status|checkStatusTool/.test(index), "index.ts serves no gitea_check_status");
});
test("the watch's current values are taken without waiting on the forge, so a slow forge never times the watch out (issue 1726)", async () => {
// The tool runtime hands a watch's current values one at a time and waits for each to be taken; it gives the whole set
// 10 s (mesh-tools StateTimeout) and otherwise ends the watch. On 2026-10-23 the owed held ~200 keys, each written to the
// forge before the next was handed over: the set took longer than 10 s, the watch ended at every attempt, and no
// verdict reached the forge's mesh/check again.
const { CheckStatuses, watchOwed } = await import("../check.ts");
const sent: string[] = [];
let release: () => void = () => {};
const slow = new Promise<void>((r) => { release = r; });
const f = { setCommitStatus: async (_o: string, _r: string, s: string, st: any) => { await slow; sent.push(`${s.slice(0, 4)} ${st.state}`); } };
const checks = new CheckStatuses(f, () => {}, noSleep);
const keys = Array.from({ length: 200 }, (_, i) => key(i.toString(16).padStart(4, "0") + sha.slice(4), "novox/mesh-sdk", i + 1));
let handedOver = 0;
let attempts = 0;
const watch = async (handler: (c: any) => Promise<void>) => {
if (++attempts > 1) return; // a second attempt is not what this test waits for: the first must hold
// The runtime's bound on the current values: the forge has answered none of them yet.
const timedOut = new Promise<"timeout">((r) => setTimeout(() => r("timeout"), 200));
const delivered = (async () => { for (const [i, k] of keys.entries()) { await handler(put(k, owed(), i + 1, true)); handedOver++; } return "delivered" as const; })();
if (await Promise.race([delivered, timedOut]) === "timeout") throw new Error("the current values of mesh-delivery-checker.owed did not arrive in 10s");
};
const said: string[] = [];
await watchOwed(watch, checks, noSleep, (l) => said.push(l));
assert.equal(handedOver, 200, "every current value was taken before the forge answered");
assert.ok(!said.some((l) => /could not be bound/.test(l)), said.join(" | "));
assert.equal(sent.length, 0, "nothing is written before the forge answers");
release();
await checks.settled();
assert.equal(sent.length, 200, "every owed status is written once the forge answers");
});
test("the writes a watch's start hands over go to the forge a few at a time, never all at once (issue 1726)", async () => {
const { CheckStatuses, WRITES_AT_ONCE } = await import("../check.ts");
let running = 0, most = 0, done = 0;
const f = { setCommitStatus: async () => { running++; most = Math.max(most, running); await new Promise((r) => setTimeout(r, 1)); running--; done++; } };
const checks = new CheckStatuses(f, () => {}, noSleep);
for (let i = 0; i < 50; i++) await checks.take(put(key(i.toString(16).padStart(4, "0") + sha.slice(4), "novox/mesh-sdk", i + 1), owed(), i + 1, true));
await checks.settled();
assert.equal(done, 50);
assert.equal(most, WRITES_AT_ONCE);
});
+12
View File
@@ -525,6 +525,18 @@ test("gitea_merge_pull_request judges mesh/check on a main requiring no status,
state.statuses = line(TRUNK);
assert.equal(((await merge()) as Record<string, unknown>).merged, true);
// check-gates-merge false (the operator, 2026-10-25): a head the Approve names merges whatever its mesh/check says;
// without approved_heads it is refused as before.
const ungated = mergeTool(client, { gates, stateDir, emit: async () => {}, checkGates: () => false });
const mergeUngated = (args: Record<string, unknown> = { approved_heads }) =>
ungated.run({ owner: "novox", repo: "mesh-catalog", number: 7, head: REVIEWED, ...args });
for (const statuses of [[], line(TRUNK, "pending", "checking"), line(TRUNK, "failure", "fail"), line(MOVED)]) {
state.merged = false;
state.statuses = statuses;
await assert.rejects(mergeUngated({}), /Approve/);
assert.equal(((await mergeUngated()) as Record<string, unknown>).merged, true, JSON.stringify(statuses));
}
});
test("a rule named for a branch is not made weaker than the glob rule that guards it now", async () => {
+2 -2
View File
@@ -508,11 +508,11 @@ function allGiteaTools(gitea: GiteaClient, admin?: () => GiteaAdmin): ToolDefini
if (!owner || !repo) throw new Error("name the repository: owner and repo");
if (!Number.isInteger(number) || number < 0) throw new Error(`${args.number} is not a pull request's number`);
let pull: { number: number; title: string; state: string; merged: boolean; base?: string; head?: string;
head_sha?: string; merge_commit_sha?: string; html_url: string } | undefined;
head_sha?: string; merge_commit_sha?: string; merged_at?: string; html_url: string } | undefined;
if (number > 0) {
const p = await gitea.getPullRequest(owner, repo, number);
pull = { number: p.number, title: p.title, state: p.state, merged: p.merged, base: p.base, head: p.head,
head_sha: p.head_sha, merge_commit_sha: p.merge_commit_sha, html_url: p.html_url };
head_sha: p.head_sha, merge_commit_sha: p.merge_commit_sha, merged_at: p.merged_at, html_url: p.html_url };
if (!ref) ref = p.head_sha ?? "";
if (!base) base = p.base ?? "";
if (!ref) throw new Error(`${owner}/${repo}#${number} names no head commit`);
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+5 -1
View File
@@ -102,7 +102,11 @@ integration level, the postgres major and the nats-server version; and, when the
workspace (`beside`), each one's repository and commit, which the answer echoes as `beside` (the checker keeps a verdict only
for the siblings it was judged with). The text's form is `DigestVersion` 3. The Go version is not in it: the toolchain enters a
fingerprint once, as the version the build nodes agree on (ADR 0477 §3), and the standard library's packages are that
version's. A package with no tests at the level has no digest and is named in `without_tests`. A directory that vendors
version's. A package with no tests at the level has no digest and is named in `without_tests`. A catalogue module's packages also hash the module's non-Go files (the nearest ancestor directory holding `module.json`: its manifest, README, `files/` and the rest, so `../../module.json` is covered; digest form 5). A package whose directory holds a file named `reads-outside` (any content; say what the tests read) is never kept (issue 1935): its tests read
files outside it, which a digest from `go list` cannot name, so its digest is a fresh random one on every call, the ask finds no kept
verdict for it and runs it, and the answer names it in `reads_outside` and the module's log says why. The checker still writes a
verdict under that digest, which nothing ever reads again. The marker is the plain way to say it until each verdict keeps the files
its tests opened (Go's test log). A directory that vendors
is listed and tested from its vendor directory (`-mod=vendor`).
## A change to this module is judged by its candidate
@@ -14,6 +14,7 @@ var Kept = []announce.State{
{Name: "toolchains", Readable: true},
{Name: "keys", Readable: true},
{Name: "steps", Readable: true},
{Name: "pauses", Readable: true},
}
// keepStates announces Kept as this module's (stdio.Keeps): main calls it before it serves.
@@ -80,19 +80,23 @@ func (m *Module) Puller(steps []StepTool, source buildstep.AskSource) *buildstep
source = buildstep.SeatAsks{Language: delivery.AskGo}
}
capacity := m.Capacity
// The capacity's own change wakes the puller (issue 1887): the runtime's watch of the config value as mesh-config
// resolved it; the settings file is read only when that watch cannot start. A test's Capacity is its own.
var watch func(string, func(stdio.ProvisionChange) error) error
if capacity == nil {
capacity = buildstep.NodeCapacity(m.Settings, buildstep.DefaultCapacities, m.Node, m.Say)
watch = buildstep.WatchConfigServed
}
ask := m.askSteps(steps)
var load func(string) ([]byte, error)
if m.ReadSteps.Get != nil {
if m.ReadPauses.Get != nil {
load = func(key string) ([]byte, error) {
raw, err := m.ReadSteps.Get(key)
raw, err := m.ReadPauses.Get(key)
return raw, err
}
}
return &buildstep.Puller{Source: source, Language: delivery.AskGo, Node: m.Node, Put: m.Steps, Load: load,
Capacity: capacity, Logf: m.Say,
return &buildstep.Puller{Source: source, Language: delivery.AskGo, Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load,
Capacity: capacity, WatchConfig: watch, Logf: m.Say,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
if err := m.Check(ctx); err != nil {
return nil, err
@@ -100,6 +104,9 @@ func (m *Module) Puller(steps []StepTool, source buildstep.AskSource) *buildstep
if err := m.room(); err != nil {
return nil, err
}
if a.Compose != nil {
return m.compose(ctx, ask.Fetch, a)
}
return buildstep.RunAsk(ctx, ask, a)
}}
}
@@ -4,7 +4,6 @@ package main
import (
"context"
"os"
"sync"
"testing"
"time"
@@ -14,16 +13,25 @@ import (
"git.novox.be/novox/mesh-sdk/go/buildstep"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/testbus"
)
// checkCommit is the commit the check ask fetches, apart from the delivery ask's, so the fetch order names which ran first.
const checkCommit = "89abcdef0123456789abcdef0123456789abcdef"
// 1887/AC-order, 1887/AC-pull against the real bus: a check ask published before a delivery ask runs after it, and at
// capacity 1 the second waits in the stream until the first ends.
//
// **A bus of its own.** The integration level runs every directory's tests at once on one bus, and this test names the
// build seat's stream and its fixed workers (SEAT_NODE_BUILD_GO and its workers) and deletes the stream at its end, so
// anything else on that bus, a puller or a leftover consumer, could hold or take its delivery ask and let the check
// ask run first. It starts from testbus (mesh-sdk, novox/hq ADR 0495 clause 5): a bus for this test alone, gone after it.
//
// **The order is read from the record.** At capacity 1 the puller starts the check the instant the delivery ask's step
// ends, well before any poll, so "the check key holds a state" is true on correct behaviour (issue 1887). The test
// waits for both asks to end and compares the step records' own Started and Ended times, and the fetch order by commit.
func TestTheModuleTakesADeliveryAskBeforeAnEarlierCheck(t *testing.T) {
url := os.Getenv("MESH_TEST_NATS")
if url == "" {
t.Skip("MESH_TEST_NATS names no bus: the integration level starts one")
}
nc, err := nats.Connect(url)
nc, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
@@ -40,6 +48,7 @@ func TestTheModuleTakesADeliveryAskBeforeAnEarlierCheck(t *testing.T) {
t.Cleanup(func() { _ = js.DeleteStream(context.Background(), "SEAT_NODE_BUILD_GO") })
check := deliveryAsk()
check.Kind = delivery.AskCheck
check.Build.Fetch["commit"] = checkCommit
first := deliveryAsk()
for _, a := range []delivery.StepAsk{check, first} {
raw, _ := delivery.EncodeStepAsk(a)
@@ -68,30 +77,36 @@ func TestTheModuleTakesADeliveryAskBeforeAnEarlierCheck(t *testing.T) {
p := m.Puller(steps, &buildstep.JetStreamAsks{JS: js, Language: delivery.AskGo})
p.PullWait = 200 * time.Millisecond
go p.Pull(ctx)
for _, a := range []delivery.StepAsk{first, check} {
key, _ := delivery.AskStepKey("ace", a.ID)
for deadline := time.Now().Add(10 * time.Second); ; time.Sleep(10 * time.Millisecond) {
states.mu.Lock()
v := states.steps[key]
states.mu.Unlock()
if len(v) > 0 && v[len(v)-1].State != delivery.StepRunning {
break
}
if time.Now().After(deadline) {
t.Fatalf("the ask %s (%s) never ended", a.ID, a.Kind)
}
}
var ran map[string][]delivery.Step
for deadline := time.Now().Add(10 * time.Second); ; time.Sleep(10 * time.Millisecond) {
ran = map[string][]delivery.Step{}
ended := 0
states.mu.Lock()
checkKey, _ := delivery.AskStepKey("ace", check.ID)
if a.ID == first.ID && len(states.steps[checkKey]) > 0 {
states.mu.Unlock()
t.Fatal("the check ask started before the delivery ask ended at capacity 1")
for _, a := range []delivery.StepAsk{first, check} {
key, _ := delivery.AskStepKey("ace", a.ID)
v := append([]delivery.Step(nil), states.steps[key]...)
ran[a.ID] = v
if len(v) > 0 && v[len(v)-1].State != delivery.StepRunning {
ended++
}
}
states.mu.Unlock()
if ended == 2 {
break
}
if time.Now().After(deadline) {
t.Fatalf("the asks never both ended: %v", ran)
}
}
// The check's step may start the moment the delivery's ends, so the order is read from the records' own times.
deliveryEnded := ran[first.ID][len(ran[first.ID])-1].Ended
checkStarted := ran[check.ID][0].Started
if checkStarted.Before(deliveryEnded) {
t.Fatalf("the check ask started at %v, before the delivery ask ended at %v at capacity 1", checkStarted, deliveryEnded)
}
mu.Lock()
defer mu.Unlock()
if len(order) != 2 {
t.Fatalf("fetched %v", order)
if len(order) != 2 || order[0] != askCommit || order[1] != checkCommit {
t.Fatalf("fetched %v, want the delivery ask's commit %s first, then the check's %s", order, askCommit, checkCommit)
}
}
@@ -174,7 +174,7 @@ func TestACheckAsksDigestIsTheToolsAnswerInTheCall(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if l := out.(delivery.CheckAnswer).Levels; len(l) != 1 || !strings.Contains(string(l[0].Digest), `"x/a":"d1"`) {
if l := out.(delivery.CheckAnswer).Levels; len(l) != 1 || l[0].Digest.Map()["x/a"] != "d1" {
t.Fatalf("levels = %+v, want the digest's packages", l)
}
}
@@ -116,7 +116,7 @@ func TestTheIntegrationLevelDigestsUnderBothTags(t *testing.T) {
t.Errorf("the integration digest listed %q", s.command)
}
}
if DigestVersion != "4" {
if DigestVersion != "5" {
t.Errorf("the digest's form changed in kind and is still version %s", DigestVersion)
}
}
@@ -0,0 +1,210 @@
package main
// **A compose ask runs mesh-sdk's composer at a commit** (novox/hq ADR 0482 clause 4, issues 1621 and 1622): the
// assigner asks it, to compose a group with its mesh-sdk member's head, and to read the group's manifests with the
// mesh-sdk the running controller and node-engine pin (ADR 0482 clause 5, issue 1623). The module fetches the commit
// (its fetch step: the forge key and the network, none of the change's code), builds the composer package in the Go
// module the ask names, in a run step shut off with nothing fetched, and keeps the program under its key
// (delivery.ComposerKey), so each commit is built once on a node. It fills the request's manifests from the delivery
// indexer's manifests state, by digest, and runs the program shut off: no network, none of the account, the request a
// file of a workspace of its own. The answer is the composer's verdict whole (delivery.ComposeAnswer); the module
// judges nothing of it.
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"sync"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"git.novox.be/novox/mesh-sdk/go/composer"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// ManifestsState is the delivery indexer's manifests by digest, which a compose ask's request names (the module reads
// it, module.json reads).
const ManifestsState = delivery.IndexerModule + ".manifests"
// composerLocks hold one mutex per composer key, so a composer two asks need at once is built once.
var composerLocks sync.Map
// composerPath is where the composer of key is kept: the cache directory, beside Go's own caches.
func (m *Module) composerPath(key string) string {
return filepath.Join(m.Cache, "composers", key, "mesh-compose")
}
// compose runs a compose ask: the composer of its commit, built once, over its request with the manifests filled.
func (m *Module) compose(ctx context.Context, fetch buildstep.StepFunc, a delivery.StepAsk) (any, error) {
job := *a.Compose
repository, _ := job.Fetch["repository"].(string)
commit, _ := job.Fetch["commit"].(string)
installed, err := m.Probe(ctx)
if err != nil {
return nil, err
}
key := delivery.ComposerKey(repository, commit, job.Dir, job.Package, delivery.ToolchainUsed{Version: installed, Recipe: m.Recipe.Version()})
exe, built, err := m.composerOf(ctx, fetch, job, key)
if err != nil {
return nil, err
}
req, err := composer.Decode(job.Request)
if err != nil {
return nil, buildstep.Refuse("the compose ask %s: %v", a.ID, err)
}
if err := composer.Fill(&req, m.manifestOf); err != nil {
return nil, err
}
verdict, err := m.runComposer(ctx, exe, req)
if err != nil {
return nil, err
}
return delivery.ComposeAnswer{Built: built, Key: key, Verdict: verdict}, nil
}
// manifestOf is the manifest the indexer keeps under digest, as its repository holds it.
func (m *Module) manifestOf(digest string) ([]byte, error) {
if m.ReadManifest == nil {
return nil, errors.New("this module reads no manifests state")
}
key, err := delivery.ManifestsKey(digest)
if err != nil {
return nil, err
}
raw, err := m.ReadManifest(key)
if err != nil {
return nil, err
}
if raw == nil {
return nil, fmt.Errorf("the indexer keeps no manifest of digest %s", digest)
}
man, err := delivery.DecodeManifest(raw)
if err != nil {
return nil, err
}
return []byte(man.JSON), nil
}
// composerOf is the composer kept under key, or, when none is, the one built now from the fetched commit (built true).
func (m *Module) composerOf(ctx context.Context, fetch buildstep.StepFunc, job delivery.ComposeJob, key string) (string, bool, error) {
exe := m.composerPath(key)
mu, _ := composerLocks.LoadOrStore(exe, &sync.Mutex{})
mu.(*sync.Mutex).Lock()
defer mu.(*sync.Mutex).Unlock()
if st, err := os.Stat(exe); err == nil && st.Mode().IsRegular() {
return exe, false, nil
}
if fetch == nil {
return "", false, buildstep.Refuse("this build module fetches nothing")
}
out, err := fetch(ctx, job.Fetch)
if err != nil {
return "", false, err
}
raw, err := json.Marshal(out)
if err != nil {
return "", false, err
}
var f struct {
Workspace string `json:"workspace"`
}
if err := json.Unmarshal(raw, &f); err != nil || f.Workspace == "" {
return "", false, fmt.Errorf("the fetch answered no workspace: %s", raw)
}
defer m.releaseSaid(f.Workspace)
done := m.spaces().Hold(f.Workspace)
defer done()
dir := filepath.Join(m.Work, f.Workspace, "tree", job.Dir)
if _, err := os.Stat(filepath.Join(dir, "go.mod")); err != nil {
return "", false, fmt.Errorf("%s at %s holds no go.mod in %s, so no composer can be built there", job.Fetch["repository"], job.Fetch["commit"], job.Dir)
}
into := filepath.Join(m.Work, f.Workspace, "composer", "mesh-compose")
if err := os.MkdirAll(filepath.Dir(into), 0o700); err != nil {
return "", false, err
}
run, err := m.stepRunner(f.Workspace, false)
if err != nil {
return "", false, err
}
env := append(m.goEnvFor(dir), "GOPROXY=off", "GOTOOLCHAIN=local", "CGO_ENABLED=0")
r := run(ctx, env, "go", "-C", dir, "build", "-trimpath", "-o", into, job.Package)
if r.Status != 0 || r.Err != "" {
// The words Go says when the module holds no such package (a mesh-sdk from before the composer) are kept whole,
// so the asker tells that apart from a fault.
return "", false, fmt.Errorf("the composer %s does not build at %s: %s", job.Package, job.Fetch["commit"], compileError(r.Err+"\n"+r.Stderr+"\n"+r.Stdout))
}
if err := copyExecutable(into, exe); err != nil {
return "", false, fmt.Errorf("the composer could not be kept: %w", err)
}
return exe, true, nil
}
// copyExecutable copies a built program into place through a file beside it, so a half-written program is never kept.
func copyExecutable(from, to string) error {
if err := os.MkdirAll(filepath.Dir(to), 0o700); err != nil {
return err
}
src, err := os.Open(from)
if err != nil {
return err
}
defer src.Close()
tmp := to + ".new"
dst, err := os.OpenFile(tmp, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o700)
if err != nil {
return err
}
if _, err := io.Copy(dst, src); err != nil {
dst.Close()
return err
}
if err := dst.Close(); err != nil {
return err
}
return os.Rename(tmp, to)
}
// releaseSaid removes a workspace once its steps ended, saying a failure: the sweep removes it later.
func (m *Module) releaseSaid(workspace string) {
if err := m.spaces().Release(workspace); err != nil && m.Say != nil {
m.Say("the compose ask's workspace %s: %v", workspace, err)
}
}
// runComposer runs the composer over req in a workspace of its own, shut off, and answers its verdict: the last line of
// its standard output, a composer.Verdict of the version this module reads.
func (m *Module) runComposer(ctx context.Context, exe string, req composer.Request) (json.RawMessage, error) {
id := buildstep.NewID()
done := m.spaces().Hold(id)
defer m.releaseSaid(id)
defer done()
ws := filepath.Join(m.Work, id)
if err := os.MkdirAll(ws, 0o700); err != nil {
return nil, err
}
raw, err := json.Marshal(req)
if err != nil {
return nil, err
}
file := filepath.Join(ws, "request.json")
if err := os.WriteFile(file, raw, 0o600); err != nil {
return nil, err
}
run := m.Run
if !m.Inside {
iso := m.isolation(id, false)
iso.ReadOnly = append(iso.ReadOnly, exe)
run = buildstep.Isolated(m.Run, iso)
}
r := run(ctx, nil, exe, "-request", file)
line := strings.TrimSpace(lastLine(r.Stdout))
var v composer.Verdict
if r.Status != 0 || r.Err != "" || json.Unmarshal([]byte(line), &v) != nil || v.V != composer.Version {
return nil, fmt.Errorf("the composer answered no verdict (status %d %s): %s", r.Status, r.Err, firstLine(r.Stderr+"\n"+r.Stdout))
}
return json.RawMessage(line), nil
}
@@ -0,0 +1,156 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"git.novox.be/novox/mesh-sdk/go/composer"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/forgekey"
"git.novox.be/novox/mesh-sdk/go/testbus"
)
// sdkPin is the mesh-sdk version this module's go.mod requires: the composer the test builds is that version's.
func sdkPin(t *testing.T) string {
t.Helper()
raw, err := os.ReadFile("../../go.mod")
if err != nil {
t.Fatal(err)
}
for _, l := range strings.Split(string(raw), "\n") {
if f := strings.Fields(l); len(f) >= 2 && f[0] == "git.novox.be/novox/mesh-sdk/go" {
return f[1]
}
}
t.Fatal("go.mod requires no mesh-sdk")
return ""
}
// goEnvOf is one value of this machine's go env.
func goEnvOf(t *testing.T, name string) string {
t.Helper()
out, err := exec.Command("go", "env", name).Output()
if err != nil {
t.Fatalf("go env %s: %v", name, err)
}
return strings.TrimSpace(string(out))
}
// 1622/AC1, AC3 against the real bus: a compose ask put on the Go build seat's check stream is pulled by the module
// started whole on that bus, which fetches the commit (here a tree of a module requiring mesh-sdk at this module's pin,
// as a reader repository is), builds mesh-sdk's composer there with nothing fetched, fills the request's manifests from
// the indexer's state and runs the composer; the step key <node>.<ask id> passes with the composer's verdict. A second
// ask of the same commit runs the kept composer: fetched and built once.
func TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus(t *testing.T) {
nc, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, err := jetstream.New(nc)
if err != nil {
t.Fatal(err)
}
ctx, stop := context.WithCancel(context.Background())
defer stop()
if err := buildstep.DeclareAskWorkers(ctx, js, delivery.AskGo, 2*time.Minute); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = js.DeleteStream(context.Background(), "SEAT_NODE_BUILD_GO") })
states := newStates()
m := moduleWith(states, "go1.26.0")
m.Work, m.Cache, m.Inside = t.TempDir(), t.TempDir(), true
m.Run = StepRunner(t.TempDir())
m.GoModCache = goEnvOf(t, "GOMODCACHE") // the module cache this module's own build filled, read and not fetched
run, gocache := m.Run, goEnvOf(t, "GOCACHE") // this machine's build cache, so the composer's packages compile once
m.Run = func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
return run(ctx, append(env, "GOCACHE="+gocache), name, args...)
}
read, digests := indexerHolds(t, `{"module":"demo","version":"1"}`)
m.ReadManifest = read
gosum, err := os.ReadFile("../../go.sum")
if err != nil {
t.Fatal(err)
}
fetches := 0
steps := fakeSteps(new([]string))
for i := range steps {
if steps[i].Name == "build_go_fetch" {
steps[i].Run = func(ctx context.Context, args map[string]any) (any, error) {
fetches++
ws := buildstep.NewID()
tree := filepath.Join(m.Work, ws, "tree")
if err := os.MkdirAll(tree, 0o700); err != nil {
return nil, err
}
mod := "module reader\n\ngo 1.26.0\n\nrequire git.novox.be/novox/mesh-sdk/go " + sdkPin(t) + "\n"
if err := os.WriteFile(filepath.Join(tree, "go.mod"), []byte(mod), 0o600); err != nil {
return nil, err
}
return map[string]any{"workspace": ws}, os.WriteFile(filepath.Join(tree, "go.sum"), gosum, 0o600)
}
}
}
p := m.Puller(steps, &buildstep.JetStreamAsks{JS: js, Language: delivery.AskGo})
p.PullWait = 200 * time.Millisecond
go p.Pull(ctx)
ask := func() delivery.Step {
a := composeAskOf(t, digests[0])
a.Compose.Fetch["repository"], a.Compose.Dir = "novox/mesh-controller", delivery.RootPath
raw, err := delivery.EncodeStepAsk(a)
if err != nil {
t.Fatal(err)
}
subject, _ := delivery.AskSubject(delivery.AskGo, delivery.AskCheck)
if _, err := js.Publish(ctx, subject, raw); err != nil {
t.Fatal(err)
}
key, _ := delivery.AskStepKey("ace", a.ID)
for deadline := time.Now().Add(90 * time.Second); time.Now().Before(deadline); time.Sleep(50 * time.Millisecond) {
states.mu.Lock()
v := states.steps[key]
states.mu.Unlock()
if len(v) > 0 && v[len(v)-1].State != delivery.StepRunning {
return v[len(v)-1]
}
}
t.Fatalf("the compose ask %s never ended", a.ID)
return delivery.Step{}
}
for n, wantBuilt := range []bool{true, false} {
step := ask()
if step.State != delivery.StepPassed {
t.Fatalf("ask %d: %s: %s", n+1, step.State, step.Reason)
}
var ans delivery.ComposeAnswer
var v composer.Verdict
if err := json.Unmarshal(step.Result, &ans); err != nil {
t.Fatal(err)
}
if err := json.Unmarshal(ans.Verdict, &v); err != nil {
t.Fatal(err)
}
if ans.Built != wantBuilt || !strings.HasPrefix(ans.Key, "composer-") || len(v.Cases) != 1 || !v.Cases[0].Manifests["demo"].Accepted ||
!strings.Contains(v.SDK, sdkPin(t)) {
t.Fatalf("ask %d: answer built %v key %s, verdict %+v; want built %v and demo accepted by mesh-sdk %s", n+1, ans.Built, ans.Key, v, wantBuilt, sdkPin(t))
}
}
if fetches != 1 {
t.Errorf("two asks of one commit fetched %d times, want once", fetches)
}
}
@@ -0,0 +1,125 @@
package main
import (
"context"
"encoding/json"
"os"
"path/filepath"
"slices"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"git.novox.be/novox/mesh-sdk/go/composer"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
const composeCommit = "4444444444444444444444444444444444444444"
// indexerHolds is the delivery indexer's manifests state holding each body under its digest, as the indexer writes it.
func indexerHolds(t *testing.T, bodies ...string) (func(string) (json.RawMessage, error), []string) {
t.Helper()
kept, digests := map[string]json.RawMessage{}, []string{}
for _, b := range bodies {
d := delivery.ManifestDigest([]byte(b))
key, _ := delivery.ManifestsKey(d)
raw, err := delivery.EncodeManifest(delivery.Manifest{Module: "demo", JSON: b})
if err != nil {
t.Fatal(err)
}
kept[key], digests = raw, append(digests, d)
}
return func(key string) (json.RawMessage, error) { return kept[key], nil }, digests
}
func composeAskOf(t *testing.T, digest string) delivery.StepAsk {
t.Helper()
req, _ := json.Marshal(composer.Request{V: composer.Version, Cases: []composer.Case{{Name: "change",
Manifests: map[string]composer.Manifest{"demo": {Digest: digest}}}}})
return delivery.StepAsk{ID: buildstep.NewID(), Kind: delivery.AskCheck, Language: delivery.AskGo, Asker: "mesh-delivery-assigner",
At: time.Now(), Compose: &delivery.ComposeJob{Fetch: map[string]any{"repository": "novox/mesh-sdk", "commit": composeCommit},
Dir: "go", Package: composer.Package, Request: req}}
}
// composeModule is a module with work and cache directories of its own, running its commands as they are (Inside: a
// unit test has no bubblewrap to shut them off in).
func composeModule(t *testing.T) *Module {
t.Helper()
m := moduleWith(newStates(), "go1.26.0")
m.Work, m.Cache, m.Inside, m.Run = t.TempDir(), t.TempDir(), true, StepRunner(t.TempDir())
return m
}
// 1622/AC3: a composer kept under the ask's key is run again, nothing fetched or built: the answer says it was not
// built, names the key, and carries the composer's verdict whole; the request it read had its manifests filled from the
// indexer's state.
func TestAKeptComposerIsRunWithTheManifestsFilled(t *testing.T) {
m := composeModule(t)
read, digests := indexerHolds(t, `{"module":"demo","version":"1"}`)
m.ReadManifest = read
a := composeAskOf(t, digests[0])
key := delivery.ComposerKey("novox/mesh-sdk", composeCommit, "go", composer.Package, delivery.ToolchainUsed{Version: "go1.26.0", Recipe: m.Recipe.Version()})
exe := m.composerPath(key)
if err := os.MkdirAll(filepath.Dir(exe), 0o700); err != nil {
t.Fatal(err)
}
// A composer that answers which manifests its request held, filled or not, as a verdict.
script := "#!/bin/sh\n" +
`if grep -q '"digest"' "$2"; then echo '{"v":1,"sdk":"stand-in","cases":[{"name":"unfilled","manifests":{}}]}'; ` +
`else echo '{"v":1,"sdk":"stand-in","cases":[{"name":"change","manifests":{"demo":{"accepted":true}}}]}'; fi` + "\n"
if err := os.WriteFile(exe, []byte(script), 0o700); err != nil {
t.Fatal(err)
}
fetched := 0
fetch := func(context.Context, map[string]any) (any, error) { fetched++; return nil, nil }
out, err := m.compose(context.Background(), fetch, a)
if err != nil {
t.Fatal(err)
}
ans := out.(delivery.ComposeAnswer)
var v composer.Verdict
if err := json.Unmarshal(ans.Verdict, &v); err != nil {
t.Fatal(err)
}
if ans.Built || ans.Key != key || fetched != 0 || len(v.Cases) != 1 || v.Cases[0].Name != "change" || !v.Cases[0].Manifests["demo"].Accepted {
t.Fatalf("answer = %+v (fetched %d), verdict %+v", ans, fetched, v)
}
if entries, _ := os.ReadDir(m.Work); len(entries) != 0 {
t.Errorf("the run's workspace was left: %v", entries)
}
}
// 1622: a manifest the indexer does not keep fails the ask, naming its digest; a fetched commit with no go.mod where
// the ask builds fails it naming the directory, and its workspace is released.
func TestAComposeAskFailsLoudly(t *testing.T) {
m := composeModule(t)
m.ReadManifest, _ = indexerHolds(t)
a := composeAskOf(t, "feedfeedfeedfeedfeedfeedfeedfeedfeedfeedfeedfeedfeedfeedfeedfeed")
key := delivery.ComposerKey("novox/mesh-sdk", composeCommit, "go", composer.Package, delivery.ToolchainUsed{Version: "go1.26.0", Recipe: m.Recipe.Version()})
os.MkdirAll(filepath.Dir(m.composerPath(key)), 0o700)
os.WriteFile(m.composerPath(key), []byte("#!/bin/sh\nexit 0\n"), 0o700)
if _, err := m.compose(context.Background(), nil, a); err == nil || !strings.Contains(err.Error(), "feedfeed") {
t.Errorf("an unkept manifest: %v", err)
}
m2 := composeModule(t)
fetch := func(context.Context, map[string]any) (any, error) {
if err := os.MkdirAll(filepath.Join(m2.Work, "ws0000000000a", "tree", "go"), 0o700); err != nil {
return nil, err
}
return map[string]any{"workspace": "ws0000000000a"}, nil
}
if _, err := m2.compose(context.Background(), fetch, composeAskOf(t, "x")); err == nil || !strings.Contains(err.Error(), "no go.mod in go") {
t.Errorf("a commit with no go.mod: %v", err)
}
if entries, _ := os.ReadDir(m2.Work); len(entries) != 0 {
t.Errorf("the fetched workspace was left: %v", entries)
}
}
// 1622: the module reads the delivery indexer's manifests, which a compose ask names by digest.
func TestTheModuleReadsTheIndexersManifests(t *testing.T) {
if m := readManifest(t); !slices.Contains(m.Reads, ManifestsState) {
t.Errorf("reads = %v, lacks %s", m.Reads, ManifestsState)
}
}
@@ -7,6 +7,7 @@ package main
import (
"bytes"
"context"
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"encoding/json"
@@ -24,12 +25,20 @@ import (
// 2: no Go version in the text; the toolchain enters a fingerprint as the agreed version (ADR 0477 §3). 3: the siblings'
// commits (the lead's decision L5) and the integration level's tags integration,isolation (the checker's agreement).
// 4: the siblings' commits enter only the digest of a package whose tests reach a file naming MESH_CHECK_BESIDE (ADR 0536 §1).
const DigestVersion = "4"
// 5: a catalogue module's non-Go files enter the digest of its packages (issue 1935).
const DigestVersion = "5"
// besideVariable is the name a file bears when it reads where the siblings lie; a package whose tests reach such a file
// depends on the siblings' commits.
const besideVariable = "MESH_CHECK_BESIDE"
// ReadsOutsideMarker is the name of a file in a package's directory that says its tests read files outside it (a
// repository's documents, a sibling directory): such a package is never kept (issue 1935). The digest hashes only what
// the Go toolchain lists, so it cannot name what a test opens elsewhere; a marker file, beside the Go files where a
// reader of the package sees it, is the plain way to say so until the checker keeps the files a test opened (the Go
// test log) with each verdict.
const ReadsOutsideMarker = "reads-outside"
// integrationTags are the build tags of the integration level, its run and its digest alike: integration, and isolation
// for the bubblewrap proofs (mesh-sdk's and the build modules'), which a build node can run.
var integrationTags = []string{"integration", "isolation"}
@@ -89,6 +98,9 @@ type DigestResult struct {
// only when they are the siblings it gave the fetch.
Beside []string `json:"beside,omitempty"`
WithoutTests []string `json:"without_tests,omitempty"`
// ReadsOutside names the packages that carry the marker: each has a digest that no kept verdict can have (a fresh
// random one), so the check runs it every time.
ReadsOutside []string `json:"reads_outside,omitempty"`
// JudgedBy is the program that ran the digest (ADR 0535 §5): "installed", or "candidate <merge commit>" under the
// runner workspace.
JudgedBy string `json:"judged_by,omitempty"`
@@ -199,6 +211,12 @@ func (m *Module) digestHere(ctx context.Context, workspace, level string) (Diges
}
}
sort.Strings(res.WithoutTests)
sort.Strings(res.ReadsOutside)
for _, p := range res.ReadsOutside {
if m.Say != nil {
m.Say("the %s level of %s always runs: its directory carries %s, so its tests read files outside it that no digest names", level, p, ReadsOutsideMarker)
}
}
return res, nil
}
@@ -256,6 +274,17 @@ func digestModule(res *DigestResult, pkgs map[string]*goPackage, tree string, he
if testMain == nil {
return fmt.Errorf("go list gave no test binary for %s", path)
}
if _, err := os.Stat(filepath.Join(root.Dir, ReadsOutsideMarker)); err == nil {
// The digest of a package that reads outside itself is one nothing can have kept: the key is a fresh random
// hex, so the ask finds no kept verdict and runs it, and what the checker writes under it is never read again.
var nonce [32]byte
if _, err := rand.Read(nonce[:]); err != nil {
return err
}
res.Packages[path] = hex.EncodeToString(nonce[:])
res.ReadsOutside = append(res.ReadsOutside, path)
continue
}
d, err := packageDigest(root, testMain, pkgs, tree, header, beside)
if err != nil {
return fmt.Errorf("%s: %w", path, err)
@@ -303,6 +332,15 @@ func packageDigest(root, testMain *goPackage, pkgs map[string]*goPackage, tree s
}
}
}
// A catalogue module's tests read its manifest, its README and its files from the package's directory (../../module.json):
// every non-Go file of the module's directory is an input (issue 1935). The module is the nearest ancestor with a module.json.
if modDir := enclosingModule(tree, root.Dir); modDir != "" {
more, err := moduleFileLines(tree, root, modDir)
if err != nil {
return "", err
}
lines = append(lines, more...)
}
data := filepath.Join(root.Dir, "testdata")
if _, err := os.Stat(data); err == nil {
err := filepath.WalkDir(data, func(path string, e fs.DirEntry, err error) error {
@@ -328,6 +366,41 @@ func packageDigest(root, testMain *goPackage, pkgs map[string]*goPackage, tree s
return hex.EncodeToString(h.Sum(nil)), nil
}
// enclosingModule is the nearest ancestor of dir, inside tree, that holds a module.json, or "" when there is none.
func enclosingModule(tree, dir string) string {
tree = filepath.Clean(tree)
for d := filepath.Clean(dir); d != tree && strings.HasPrefix(d, tree+string(filepath.Separator)); d = filepath.Dir(d) {
if st, err := os.Stat(filepath.Join(d, "module.json")); err == nil && !st.IsDir() {
return d
}
}
return ""
}
// moduleFileLines are the content lines of every file of a module's directory that is not Go source (Go source is the
// packages' own, listed by go list), skipping .git and node_modules.
func moduleFileLines(tree string, p *goPackage, modDir string) ([]string, error) {
var lines []string
err := filepath.WalkDir(modDir, func(path string, e fs.DirEntry, err error) error {
if err != nil {
return err
}
if e.IsDir() {
if n := e.Name(); n == ".git" || n == "node_modules" {
return filepath.SkipDir
}
return nil
}
if strings.HasSuffix(path, ".go") {
return nil
}
l, err := contentLine(tree, p, path)
lines = append(lines, l)
return err
})
return lines, err
}
func fileLine(tree string, p *goPackage, name string) (string, error) {
return contentLine(tree, p, filepath.Join(p.Dir, name))
}
@@ -189,3 +189,71 @@ func TestAModuleOutsideTheTreeCountsByVersionAndChecksum(t *testing.T) {
t.Errorf("a new version left the digest as it was")
}
}
// Issue 1935: a package whose directory carries the reads-outside marker is never kept: its digest is one no kept verdict
// can have (it differs on every call, though nothing changed), and the answer names it. A package without the marker
// keeps its digest.
func TestAPackageWithTheReadsOutsideMarkerIsNeverKept(t *testing.T) {
t.Parallel()
m, tree := digestFixture(t)
plain := digests(t, m, "unit")
if len(plain.ReadsOutside) != 0 {
t.Fatalf("a package carries the marker in the fixture: %v", plain.ReadsOutside)
}
edit(t, tree, "c/"+ReadsOutsideMarker, "the tests read ../docs\n")
first, second := digests(t, m, "unit"), digests(t, m, "unit")
c := "fixture/digest/c"
if len(first.Packages[c]) != 64 || first.Packages[c] == second.Packages[c] || first.Packages[c] == plain.Packages[c] {
t.Errorf("c carries the marker and its digest is stable: %s, %s, before %s", first.Packages[c], second.Packages[c], plain.Packages[c])
}
if strings.Join(first.ReadsOutside, ",") != c {
t.Errorf("the answer does not name the package that reads outside: %v", first.ReadsOutside)
}
a := "fixture/digest/a"
if first.Packages[a] != plain.Packages[a] || first.Packages[a] != second.Packages[a] {
t.Errorf("a has no marker and its digest moved: %s, %s, before %s", first.Packages[a], second.Packages[a], plain.Packages[a])
}
}
// Issue 1935: a catalogue module's packages read ../../module.json and the module's files: a change to them changes the
// digest of the module's package, and a change to another module's files does not.
func TestACatalogueModulesFilesChangeTheDigestOfItsPackagesOnly(t *testing.T) {
t.Parallel()
m, tree := digestFixture(t)
for _, mod := range []string{"mod1", "mod2"} {
dir := filepath.Join(tree, mod, "cmd", "p")
if err := os.MkdirAll(filepath.Join(tree, mod, "files"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
edit(t, tree, mod+"/module.json", `{"module":"`+mod+`"}`)
edit(t, tree, mod+"/README.md", "# "+mod+"\n")
edit(t, tree, mod+"/files/f.env", "A=1\n")
edit(t, tree, mod+"/cmd/p/p.go", "package p\n")
edit(t, tree, mod+"/cmd/p/p_test.go", "package p\n\nimport (\n\t\"os\"\n\t\"testing\"\n)\n\nfunc TestReads(t *testing.T) {\n\tif _, err := os.ReadFile(\"../../module.json\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n")
}
p1, p2 := "fixture/digest/mod1/cmd/p", "fixture/digest/mod2/cmd/p"
before := digests(t, m, "unit")
if len(before.Packages[p1]) != 64 || len(before.Packages[p2]) != 64 {
t.Fatalf("no digest for the module packages: %+v", before.Packages)
}
if before.Packages[p1] == before.Packages[p2] {
t.Errorf("two modules, one digest")
}
for _, file := range []string{"mod1/module.json", "mod1/README.md", "mod1/files/f.env"} {
edit(t, tree, file, "changed "+file+"\n")
after := digests(t, m, "unit")
if after.Packages[p1] == before.Packages[p1] {
t.Errorf("a change to %s did not change the module's digest", file)
}
if after.Packages[p2] != before.Packages[p2] {
t.Errorf("a change to %s changed the other module's digest", file)
}
before = after
}
if got := digests(t, m, "unit"); got.Packages["fixture/digest/c"] == "" {
t.Errorf("a package outside any module lost its digest")
}
}
@@ -0,0 +1,112 @@
package main
import (
"encoding/json"
"os"
"strings"
"testing"
)
// A failed test keeps the lines that say why it failed, not the lines it logged first (issue 1901). The fixture is
// the real `go test -json` of a test that logs a line and then fails with a multi-line t.Fatalf.
func TestAFailedTestKeepsItsFatalMessageAfterItsLogLines(t *testing.T) {
raw, err := os.ReadFile("testdata/captured/go-test-json-log-then-fatal.jsonl")
if err != nil {
t.Fatal(err)
}
res := &TestResult{}
rd := newReader(res, ".")
rd.read(string(raw))
rd.finish()
if len(res.Failed) != 1 {
t.Fatalf("failures = %+v, want one", res.Failed)
}
f := res.Failed[0]
for _, want := range []string{"3 subscription(s) derive to a subject no emitter owns:", "a.b.c", "d.e.f", "g.h.i"} {
if !strings.Contains(f.Output, want) {
t.Errorf("1901/AC1: output lacks %q: %q", want, f.Output)
}
}
if want := "3 subscription(s) derive to a subject no emitter owns:"; f.Cause != want && !strings.HasSuffix(f.Cause, want) {
t.Errorf("1901/AC1: cause = %q, want the t.Fatalf line, not the t.Logf line", f.Cause)
}
}
// A failure longer than the bound keeps its last lines, where the cause is, and says how many earlier lines it dropped.
func TestAFailureLongerThanTheBoundKeepsItsLastLines(t *testing.T) {
var lines []string
for i := 0; i < 400; i++ {
lines = append(lines, " x_test.go:1: log line "+strings.Repeat("y", 40))
}
lines = append(lines, " x_test.go:9: the real cause", "--- FAIL: TestX (0.00s)")
out := clip(lines, "")
if len(out) > outputBytes+100 {
t.Errorf("1901/AC1: kept %d bytes, bound is %d", len(out), outputBytes)
}
if !strings.HasSuffix(out, "the real cause\n--- FAIL: TestX (0.00s)") || !strings.Contains(out, "\n... (") {
t.Errorf("1901/AC1: out = %q...%q", out[:40], out[len(out)-60:])
}
}
// An older checker, which knows no "cause" field, still reads the new build module's answer, and the output it gets
// holds at least what the older build module kept (its first lines, for an output shorter than the bound).
func TestAnOlderCheckerReadsTheNewFailureAndGetsAtLeastWhatItKept(t *testing.T) { // 1901/AC3
raw, err := os.ReadFile("testdata/captured/go-test-json-log-then-fatal.jsonl")
if err != nil {
t.Fatal(err)
}
res := &TestResult{}
rd := newReader(res, ".")
rd.read(string(raw))
rd.finish()
b, err := json.Marshal(res.Failed)
if err != nil {
t.Fatal(err)
}
var old []struct {
Package string `json:"package"`
Test string `json:"test,omitempty"`
Output string `json:"output,omitempty"`
}
if err := json.Unmarshal(b, &old); err != nil || len(old) != 1 {
t.Fatalf("1901/AC3: an older checker refused the answer: %v, %d failures", err, len(old))
}
first := strings.SplitN(old[0].Output, "\n", 2)[0]
if first == "" || !strings.Contains(old[0].Output, "judged against") || !strings.Contains(old[0].Output, "3 subscription(s)") {
t.Errorf("1901/AC3: older checker's output = %q", old[0].Output)
}
}
// A long failure, whose earlier lines are left out, still starts with a real line for an older checker, which keeps only the
// first non-empty line of the output cut to 120 characters, and not with the marker.
func TestALongFailureStillStartsWithARealLineForAnOlderChecker(t *testing.T) { // 1901/AC3
var lines []string
for i := 0; i < 400; i++ {
lines = append(lines, " x_test.go:1: log line "+strings.Repeat("y", 40))
}
lines = append(lines, " x_test.go:9: the real cause", "--- FAIL: TestX (0.00s)")
oldFirst := func(out string) string { // the older checker's rule (judge.go firstLine)
for _, l := range strings.Split(out, "\n") {
if l = strings.TrimSpace(l); l != "" {
if len(l) > 120 {
l = l[:120]
}
return l
}
}
return ""
}
for _, cause := range []string{"the real cause", ""} {
out := clip(lines, cause)
got := oldFirst(out)
if strings.HasPrefix(got, "... (") || got == "" {
t.Errorf("1901/AC3: cause %q: an older checker would read %q", cause, got)
}
if cause != "" && got != cause {
t.Errorf("1901/AC3: older checker reads %q, want the cause", got)
}
if !strings.HasSuffix(out, "--- FAIL: TestX (0.00s)") {
t.Errorf("1901/AC3: the last lines are lost: %q", out[len(out)-40:])
}
}
}
@@ -288,10 +288,14 @@ func inTestdata(path string) bool {
// git, through the account's rewrite of the forge's address to ssh), and nothing that asks a question. For a directory
// that vendors, see goEnvFor.
func (m *Module) goEnv() []string {
modcache := filepath.Join(m.Cache, "gomod")
if m.GoModCache != "" {
modcache = m.GoModCache
}
return []string{
"GOTOOLCHAIN=local",
"GOFLAGS=-mod=mod",
"GOMODCACHE=" + filepath.Join(m.Cache, "gomod"),
"GOMODCACHE=" + modcache,
"GOCACHE=" + filepath.Join(m.Cache, "gobuild"),
"GOPRIVATE=git.novox.be",
"GIT_TERMINAL_PROMPT=0",
+248 -11
View File
@@ -9,6 +9,8 @@ import (
"bufio"
"encoding/json"
"fmt"
"regexp"
"slices"
"sort"
"strings"
"time"
@@ -49,11 +51,17 @@ type TestTime struct {
}
// Failure is a test that failed, or a package that failed with no test to name (a build that failed, a TestMain that
// panicked); Output is the first lines it wrote.
// panicked); Output is the last lines it wrote, up to outputBytes (issue 1901), and Cause the first line of what the
// test reported as its error (t.Error, t.Fatal), without its file and line, when go test said which lines those were.
// **A failure that is not one test's assertion carries its real cause too** (issue 1901, its third meeting): the
// "panic:" line (a timeout's with the tests it found running), a "fatal error:" or "signal:" line, a build's first
// error, or in words that the test program exited, while which tests ran or after its tests passed. Test then names the
// tests that were running when the program stopped. A "PASS" or "ok" line is never a failure's words.
type Failure struct {
Package string `json:"package"`
Test string `json:"test,omitempty"`
Output string `json:"output,omitempty"`
Cause string `json:"cause,omitempty"`
}
// TestResult is what a test step passes with.
@@ -96,13 +104,20 @@ type event struct {
Package string
Test string
Output string
OutputType string // "error" for a t.Error or t.Fatal message, "error-continue" for its further lines (go test -json)
ImportPath string
FailedBuild string
Elapsed float64
}
// outputLines is how many lines of output a failure keeps.
const outputLines = 15
// outputBytes is the most a failure keeps of its output, and outputLineBytes the most of one line. **A failure keeps its
// last lines** (issue 1901): a test logs before it fails, so the lines that say why are at the end, and the first
// fifteen lines kept before lost them (the verdict of group r4g kept a t.Logf line, not the t.Fatalf after it). Four
// kilobytes holds a long assertion message and stays far under a bus message's or a verdict's bound.
const (
outputBytes = 4096
outputLineBytes = 1024
)
// reader collects the events of one go.mod directory's `go test -json` into a result.
type reader struct {
@@ -113,13 +128,21 @@ type reader struct {
builds map[string][]string // build output by import path
pkgOut map[string][]string
testOut map[string][]string // by package + "\x00" + test
causes map[string]string // the first error line of a test, by the same key
times []TestTime
failedIn map[string]bool // packages with a failed test
// running is the tests that started and have not ended, by package, in the order they started: when a package
// fails with no failed test, these were running when its test program stopped (issue 1901).
running map[string][]string
// passed says a package's test program printed "PASS" (its tests passed) before it failed; built says its test
// program failed to build.
passed, built map[string]bool
}
func newReader(res *TestResult, dir string) *reader {
return &reader{res: res, dir: dir, pkgs: map[string]*PackageResult{}, builds: map[string][]string{},
pkgOut: map[string][]string{}, testOut: map[string][]string{}, failedIn: map[string]bool{}}
pkgOut: map[string][]string{}, testOut: map[string][]string{}, causes: map[string]string{}, failedIn: map[string]bool{},
running: map[string][]string{}, passed: map[string]bool{}, built: map[string]bool{}}
}
// read takes go test's output. Lines that are not events are kept for a failure that has nothing else to say.
@@ -167,12 +190,21 @@ func (r *reader) event(ev event) {
p := r.pkg(ev.Package)
switch ev.Action {
case "output":
r.pkgOut[ev.Package] = keep(r.pkgOut[ev.Package], ev.Output)
// go test's own summary of a package ("PASS", "FAIL", "ok <pkg>", "FAIL <pkg>") is never a failure's
// words: a "PASS" before a "FAIL <pkg>" says only that the tests passed and the program failed after them.
switch l := strings.TrimRight(ev.Output, "\r\n"); {
case l == "PASS":
r.passed[ev.Package] = true
case summaryLine(l):
default:
r.pkgOut[ev.Package] = keep(r.pkgOut[ev.Package], ev.Output)
}
case "pass":
p.Passed, p.Elapsed = true, ev.Elapsed
case "fail":
p.Passed, p.Elapsed = false, ev.Elapsed
if ev.FailedBuild != "" {
r.built[ev.Package] = true
r.pkgOut[ev.Package] = append(r.builds[ev.FailedBuild], r.pkgOut[ev.Package]...)
}
case "skip":
@@ -183,7 +215,13 @@ func (r *reader) event(ev event) {
switch ev.Action {
case "output":
r.testOut[key] = keep(r.testOut[key], ev.Output)
if ev.OutputType == "error" && r.causes[key] == "" {
r.causes[key] = errorWords(ev.Output)
}
case "run":
r.running[ev.Package] = append(r.running[ev.Package], ev.Test)
case "pass", "fail", "skip":
r.running[ev.Package] = slices.DeleteFunc(r.running[ev.Package], func(t string) bool { return t == ev.Test })
p := r.pkg(ev.Package)
p.Tests++
r.res.Tests++
@@ -193,17 +231,216 @@ func (r *reader) event(ev event) {
r.times = append(r.times, TestTime{Package: ev.Package, Test: ev.Test, Elapsed: ev.Elapsed})
if ev.Action == "fail" {
r.failedIn[ev.Package] = true
r.res.Failed = append(r.res.Failed, Failure{Package: ev.Package, Test: ev.Test, Output: clip(r.testOut[key])})
cause := r.causes[key]
if race := raceCause(r.testOut[key]); race != "" && (cause == "" || strings.HasPrefix(cause, "race detected")) {
cause = race // the race itself, not testing's "race detected during execution of test"
} else if cause == "" {
cause = causeOf(r.testOut[key])
}
r.res.Failed = append(r.res.Failed, Failure{Package: ev.Package, Test: ev.Test, Output: clip(r.testOut[key], cause), Cause: cause})
}
}
}
}
func clip(lines []string) string {
if len(lines) > outputLines {
lines = append(lines[:outputLines:outputLines], fmt.Sprintf("... (%d more lines)", len(lines)-outputLines))
// clip joins the last lines of a failure's output that fit outputBytes, each cut to outputLineBytes, and says how many
// earlier lines it left out. **Its first line is the failure's cause** (issue 1901/AC3, AC4), put ahead when the kept
// lines do not start with it, or, with no cause and lines left out, the first non-empty line of the output, then the
// marker. An older checker takes only the first non-empty line of the output, and a marker there would be less than the
// older pair kept.
func clip(lines []string, cause string) string {
size, from := 0, len(lines)
for from > 0 {
n := len(cutLine(lines[from-1])) + 1
if size+n > outputBytes && from < len(lines) {
break
}
size += n
from--
}
return strings.Join(lines, "\n")
kept := make([]string, 0, len(lines)-from+2)
head := strings.TrimSpace(cause)
for i := 0; head == "" && i < from; i++ {
head = strings.TrimSpace(lines[i])
}
if head != "" && (from > 0 || len(lines) == 0 || strings.TrimSpace(lines[from]) != head) {
kept = append(kept, cutLine(head))
}
if from > 0 {
kept = append(kept, fmt.Sprintf("... (%d earlier lines)", from))
}
for _, l := range lines[from:] {
kept = append(kept, cutLine(l))
}
return strings.Join(kept, "\n")
}
// summaryLine says a line is go test's summary of a package, in go's exact forms only: "PASS" or "FAIL" alone,
// "ok \t<pkg>\t<time>" and "FAIL\t<pkg>\t<time>" (review round 1: a line of the package's own that starts "ok " is kept).
func summaryLine(l string) bool {
return l == "PASS" || l == "FAIL" || strings.HasPrefix(l, "ok \t") || strings.HasPrefix(l, "FAIL\t")
}
// causeOf is the cause in a failure's output that is not a test's assertion (issue 1901): a race the race detector
// reported (raceCause); else the first "panic:" line, a timeout's with the tests it names as running; else the first
// "fatal error:" (the runtime's) or "signal:" line (a test program killed); "" when there is none. It reads every line,
// so a long goroutine dump after the panic cannot hide it. **It matches only lines at column 0**, where go prints the
// runtime's: a test's indented log line holding the same words is not the cause (review round 1).
func causeOf(lines []string) string {
if race := raceCause(lines); race != "" {
return race
}
for _, prefix := range []string{"panic: ", "fatal error: ", "signal: "} {
for i, l := range lines {
l = strings.TrimRight(l, "\r\n") // at column 0: an indented line is a test's own log, not the runtime's
if !strings.HasPrefix(l, prefix) {
continue
}
if strings.HasPrefix(l, "panic: test timed out after") {
var names []string
for _, n := range lines[i+1:] {
if !strings.HasPrefix(n, "\t\t") {
if strings.TrimSpace(n) == "running tests:" {
continue
}
break
}
names = append(names, strings.TrimSpace(n))
}
if len(names) > 0 {
l += "; running tests: " + strings.Join(names, ", ")
}
}
return cutLine(l)
}
}
return ""
}
// raceCause is the first race the race detector reported in a failure's output, in one line (issue 1901: a race after
// the tests' "PASS" was worded "PASS"): its access and the frame that made it, and where the goroutine that made it was
// created, as "WARNING: DATA RACE: Write at 0x... by goroutine 10 in pkg.F.func1() (f_test.go:13); goroutine created at
// pkg.F() (f_test.go:13)". "" when the output holds no race.
func raceCause(lines []string) string {
at := -1
for i, l := range lines {
if strings.TrimRight(l, "\r\n") == "WARNING: DATA RACE" { // at column 0, as causeOf
at = i
break
}
}
if at < 0 {
return ""
}
// frame is the function and the file's base name and line that follow a heading line at i.
frame := func(i int) string {
var fn, file string
for _, l := range lines[i+1:] {
if l = strings.TrimSpace(l); l == "" {
break
}
if fn == "" {
fn = l
continue
}
file = l
if j := strings.LastIndexByte(file, '/'); j >= 0 {
file = file[j+1:]
}
if j := strings.Index(file, " +0x"); j >= 0 {
file = file[:j]
}
break
}
if file != "" {
return fn + " (" + file + ")"
}
return fn
}
words := "WARNING: DATA RACE"
for i := at + 1; i < len(lines); i++ {
l := strings.TrimSpace(lines[i])
if l == "" || strings.HasPrefix(l, "===") {
continue
}
words += ": " + strings.TrimSuffix(l, ":")
if f := frame(i); f != "" {
words += " in " + f
}
break
}
for i := at + 1; i < len(lines); i++ {
l := strings.TrimSpace(lines[i])
if strings.HasPrefix(l, "==================") {
break
}
if strings.HasPrefix(l, "Goroutine ") && strings.HasSuffix(l, " created at:") {
if f := frame(i); f != "" {
words += "; its goroutine created at " + f
}
break
}
}
return cutLine(words)
}
// buildCause is the first error line of a failed build: the first line that is not go's "# <package>" heading.
func buildCause(lines []string) string {
for _, l := range lines {
if l = strings.TrimSpace(l); l != "" && !strings.HasPrefix(l, "# ") {
return cutLine(l)
}
}
return ""
}
// testNames names the tests that were running, at most three, and how many more.
func testNames(tests []string) string {
if len(tests) <= 3 {
return strings.Join(tests, ", ")
}
return fmt.Sprintf("%s and %d more", strings.Join(tests[:3], ", "), len(tests)-3)
}
// packageFailure is a package that failed with no failed test to name it (issue 1901): its test program did not build,
// panicked, timed out, was killed or exited. It names the tests that were running when the program stopped, keeps
// their output with the package's, and says the cause.
func (r *reader) packageFailure(name string) Failure {
running := r.running[name]
var lines []string
for _, t := range running {
lines = append(lines, r.testOut[name+"\x00"+t]...)
}
lines = append(lines, r.pkgOut[name]...)
cause := causeOf(lines)
switch {
case cause != "":
case r.built[name]:
cause = buildCause(lines)
case len(running) > 0:
cause = "the test program exited while " + testNames(running) + " ran, with no panic: an os.Exit or a signal (go test -json reports no exit status)"
case r.passed[name]:
cause = "the tests passed, then the test program exited with a failure: an os.Exit or a panic in TestMain or after the tests (go test -json reports no exit status)"
}
if cause == "" {
cause = "the package failed with no test to name and no panic, signal or build error in its output"
}
return Failure{Package: name, Test: testNames(running), Output: clip(lines, cause), Cause: cause}
}
// cutLine cuts a line to outputLineBytes, on a character boundary.
func cutLine(l string) string {
if len(l) <= outputLineBytes {
return l
}
return strings.ToValidUTF8(l[:outputLineBytes], "") + "..."
}
var fileAndLine = regexp.MustCompile(`^\S+\.go:\d+: `)
// errorWords is the words of an error line of a test: trimmed, without the file and line go test put before them.
func errorWords(out string) string {
return strings.TrimSpace(fileAndLine.ReplaceAllString(strings.TrimSpace(out), ""))
}
// finish moves what was read into the result: the packages in order, a failure for each failed package that has no
@@ -212,7 +449,7 @@ func (r *reader) finish() {
for _, name := range r.order {
p := r.pkgs[name]
if !p.Passed && !r.failedIn[name] {
r.res.Failed = append(r.res.Failed, Failure{Package: name, Output: clip(r.pkgOut[name])})
r.res.Failed = append(r.res.Failed, r.packageFailure(name))
}
if !p.NoTests && p.Elapsed > r.res.Elapsed {
r.res.Elapsed = p.Elapsed
@@ -12,6 +12,7 @@ import (
"regexp"
"sort"
"strings"
"sync"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
@@ -127,3 +128,45 @@ func (m *Module) lint(ctx context.Context, run forgekey.Runner, tree, d string,
}
return failed, nil
}
// lintAll lints each directory, up to m.parallel() at once like the level's tests; the failures join in directory order
// and the error is the first directory's, so the answer does not depend on which ended first (issue 1930).
func (m *Module) lintAll(ctx context.Context, run forgekey.Runner, tree string, dirs []string, env func(d string) []string) ([]Failure, error) {
type answer struct {
failed []Failure
err error
}
answers := make([]answer, len(dirs))
slots := make(chan struct{}, m.parallel())
var wg sync.WaitGroup
var crash any // a panic of a lint goes on in the caller, as a panic of a test run does
var crashMu sync.Mutex
for i, d := range dirs {
wg.Add(1)
slots <- struct{}{}
go func() {
defer wg.Done()
defer func() { <-slots }()
defer func() {
if p := recover(); p != nil {
crashMu.Lock()
crash = p
crashMu.Unlock()
}
}()
answers[i].failed, answers[i].err = m.lint(ctx, run, tree, d, env(d))
}()
}
wg.Wait()
if crash != nil {
panic(crash)
}
var failed []Failure
for _, a := range answers {
if a.err != nil {
return nil, a.err
}
failed = append(failed, a.failed...)
}
return failed, nil
}
@@ -5,7 +5,9 @@ import (
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
@@ -90,3 +92,43 @@ func TestAnUnformattedFileFailsTheUnitLevel(t *testing.T) {
}
}
}
// Issue 1930: the lint of many directories runs side by side up to the level's bound, not one after another, and the
// failures still join in directory order.
func TestTheLintOfManyDirectoriesRunsAtOnceUpToTheBound(t *testing.T) {
files := map[string]string{}
var dirs []string
for _, d := range []string{"a", "b", "c", "d", "e", "f"} {
files[d+"/go.mod"] = "module x/" + d + "\n"
files[d+"/x.go"] = "package x\n"
dirs = append(dirs, d)
}
tree := writeTree(t, files)
var mu sync.Mutex
running, most := 0, 0
run := func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
mu.Lock()
running++
most = max(most, running)
mu.Unlock()
time.Sleep(30 * time.Millisecond)
mu.Lock()
running--
mu.Unlock()
if strings.HasSuffix(args[1], "b") || strings.HasSuffix(args[1], "e") {
return forgekey.Ran{Status: 1, Stderr: "bad " + filepath.Base(args[1])}
}
return forgekey.Ran{}
}
m := &Module{Parallel: 3}
failed, err := m.lintAll(context.Background(), run, tree, dirs, func(string) []string { return nil })
if err != nil {
t.Fatal(err)
}
if most < 2 || most > 3 {
t.Errorf("%d lint commands ran at once, want 2 or 3 (the bound)", most)
}
if len(failed) != 2 || failed[0].Package != "b" || failed[1].Package != "e" {
t.Errorf("failed = %+v, want b's then e's", failed)
}
}
@@ -40,6 +40,8 @@ func main() {
Emit: func(t string, b any) error { return stdio.Emit(t, b) },
Steps: stdio.State(delivery.StepsState).Put,
ReadSteps: readOf(stdio.State(delivery.StepsState)),
Pauses: stdio.State(delivery.PausesState).Put,
ReadPauses: readOf(stdio.State(delivery.PausesState)),
ReadToolchainsState: readOf(stdio.State(delivery.ToolchainsState)),
Ctx: ctx,
Say: logf,
@@ -54,6 +56,7 @@ func main() {
m.RegistryFile = m.Home + "/registry"
m.Settings = m.Home + "/settings.json"
m.ReadKept = readOf(stdio.State(KeptState)).Get
m.ReadManifest = readOf(stdio.State(ManifestsState)).Get
m.Probe = ProbeGo(m.Home, m.Cache)
m.Bubblewrap = ProbeBubblewrap
// Outside a launch by the runtime — a build, a check — there is no node and no state to write.
@@ -92,9 +92,13 @@ func TestManifestKeepsItsStatesClaimsItsBuildSeatAndDeclaresItsHealthTool(t *tes
states := []string{}
for _, s := range m.State {
states = append(states, s.Name)
// A step's key is per step and dropped after its age, so the bucket cannot fill; a pause has no age.
if (s.Name == "steps") != (s.TTLSeconds > 0) {
t.Errorf("state %s has ttl-seconds %d: only steps has an age", s.Name, s.TTLSeconds)
}
}
slices.Sort(states)
if want := []string{"keys", "steps", "toolchains"}; !slices.Equal(states, want) {
if want := []string{"keys", "pauses", "steps", "toolchains"}; !slices.Equal(states, want) {
t.Errorf("state = %v, want %v", states, want)
}
if len(m.Claims) != 1 || m.Claims[0].Name != Seat {
@@ -29,6 +29,9 @@ type Module struct {
Toolchains, Steps buildstep.Put
// ReadSteps and ReadToolchainsState read those two states back for the read tools (issue 1562).
ReadSteps, ReadToolchainsState StateRead
// Pauses is the Put of the pauses state, which keeps the node's pause key without an age; ReadPauses reads it back.
Pauses buildstep.Put
ReadPauses StateRead
// Emit tells the bus an event (stdio.Emit); nil emits nothing. The guard emits toolchain.reported after each write of
// the toolchains key, so the indexer, which holds no timer, learns that the bucket exists (issue 1444).
Emit func(eventType string, body any) error
@@ -82,6 +85,10 @@ type Module struct {
Settings string
Capacity func() int
ReadKept func(key string) (json.RawMessage, error)
// ReadManifest reads a key of the delivery indexer's manifests state (ManifestsState), which a compose ask's request
// names its manifests by (compose.go); GoModCache, when set, is the Go module cache in place of Cache's (a test).
ReadManifest func(key string) (json.RawMessage, error)
GoModCache string
spacesOnce sync.Once
workspaces *buildstep.Workspaces
@@ -0,0 +1,103 @@
package main
import (
"os"
"strings"
"testing"
)
// A failure that is not one test's assertion carries its real cause (issue 1901, its third meeting: a try of
// mesh-controller kept only "PASS" for a package that stopped half-way). Each fixture is the real `go test -json` of a
// throwaway package (testdata/captured/README).
func TestAFailureThatIsNotAnAssertionCarriesItsRealCause(t *testing.T) { // 1901/AC4
cases := []struct {
fixture string
test string // the test the failure names; "" for none
cause []string // what the cause holds
}{
{"panicking", "TestPanics", []string{"panic: assignment to entry in nil map"}},
{"timeout", "TestWaitsForever", []string{"panic: test timed out after 2s", "TestWaitsForever"}},
{"deeptimeout", "TestHoldsTheSeat, TestWaitsForTheBus", []string{"panic: test timed out after 2s", "TestHoldsTheSeat", "TestWaitsForTheBus"}},
{"buildfail", "", []string{`cannot use "seat" (untyped string constant) as int value`}},
{"exiting", "TestExits", []string{"exited", "TestExits"}},
{"killed", "TestKilled", []string{"signal: killed"}},
{"passthenfail", "", []string{"panic: the bus closed after the tests: nats: connection closed"}},
{"mainexit", "", []string{"passed", "exited"}},
{"race", "TestRacesOnTheSeat", []string{"DATA RACE", "Write at", "example.com/race.TestRacesOnTheSeat.func1()", "p_test.go:13", "created at example.com/race.TestRacesOnTheSeat()"}},
{"racemain", "", []string{"DATA RACE", "Write at", "example.com/racemain.TestMain.func1()", "p_test.go:15", "created at example.com/racemain.TestMain()"}},
}
for _, c := range cases {
t.Run(c.fixture, func(t *testing.T) {
raw, err := os.ReadFile("testdata/captured/go-test-json-pkg-" + c.fixture + ".jsonl")
if err != nil {
t.Fatal(err)
}
res := &TestResult{}
rd := newReader(res, ".")
rd.read(string(raw))
rd.finish()
if len(res.Failed) != 1 {
t.Fatalf("failures = %+v, want one", res.Failed)
}
f := res.Failed[0]
if f.Test != c.test {
t.Errorf("1901/AC4: the failure names test %q, want %q", f.Test, c.test)
}
for _, want := range c.cause {
if !strings.Contains(f.Cause, want) {
t.Errorf("1901/AC4: cause = %q, want it to hold %q", f.Cause, want)
}
}
if len(f.Cause) > outputLineBytes+3 {
t.Errorf("1901/AC4: cause holds %d bytes, over the line bound %d", len(f.Cause), outputLineBytes)
}
// Never a "PASS" or "ok" line as a failure's words, and the first line of the output, all an older
// checker reads, is the cause.
for _, l := range strings.Split(f.Output, "\n") {
if l = strings.TrimSpace(l); l == "PASS" || strings.HasPrefix(l, "ok ") || strings.HasPrefix(l, "ok\t") {
t.Errorf("1901/AC4: the output keeps the line %q: %q", l, f.Output)
}
}
if first := strings.SplitN(f.Output, "\n", 2)[0]; strings.TrimSpace(first) != strings.TrimSpace(f.Cause) {
t.Errorf("1901/AC4: the output starts with %q, want the cause %q", first, f.Cause)
}
})
}
}
// A package that passes, and a test that fails on its own assertion, keep what they had: no cause is made up for them.
func TestAnAssertionKeepsItsOwnCause(t *testing.T) { // 1901/AC4
raw, err := os.ReadFile("testdata/captured/go-test-json-log-then-fatal.jsonl")
if err != nil {
t.Fatal(err)
}
res := &TestResult{}
rd := newReader(res, ".")
rd.read(string(raw))
rd.finish()
if len(res.Failed) != 1 || !strings.HasSuffix(res.Failed[0].Cause, "3 subscription(s) derive to a subject no emitter owns:") {
t.Errorf("1901/AC4: failures = %+v", res.Failed)
}
}
// go prints the runtime's lines ("panic:", "fatal error:", "signal:", "WARNING: DATA RACE") at column 0: a test's own
// log line that holds the same words is indented, and is not the cause (review round 1). The fixture is the deeptimeout
// capture with two log lines of the first running test added: a t.Logf and its indented continuation saying "panic:".
func TestALoggedPanicIsNotTheCause(t *testing.T) { // 1901/AC4
raw, err := os.ReadFile("testdata/captured/go-test-json-pkg-deeptimeout.jsonl")
if err != nil {
t.Fatal(err)
}
lines := strings.SplitAfter(string(raw), "\n")
added := `{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" p_test.go:9: the child said:\n"}` + "\n" +
`{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" panic: child retried and recovered\n"}` + "\n" +
`{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" WARNING: DATA RACE\n"}` + "\n"
out := strings.Join(lines[:10], "") + added + strings.Join(lines[10:], "")
res := &TestResult{}
rd := newReader(res, ".")
rd.read(out)
rd.finish()
if len(res.Failed) != 1 || !strings.HasPrefix(res.Failed[0].Cause, "panic: test timed out after 2s") {
t.Errorf("1901/AC4: failures = %+v, want the timeout's panic as the cause", res.Failed)
}
}
@@ -0,0 +1,79 @@
package main
import (
"context"
"reflect"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// 1925/AC1: packagesArg takes the list as the bus's JSON shape ([]any) and as the typed list an ask hands in process
// ([]string), and refuses a bad element in either.
func TestPackagesArgTakesBothShapes(t *testing.T) {
cases := []struct {
name string
in any
want []string
refused bool
}{
{"absent", nil, nil, false},
{"typed", []string{"x/a", "x/b/..."}, []string{"x/a", "x/b/..."}, false},
{"json", []any{"x/a", "x/b/..."}, []string{"x/a", "x/b/..."}, false},
{"typed empty", []string{}, []string{}, false},
{"typed dotdot", []string{"x/../y"}, nil, true},
{"json dotdot", []any{"x/../y"}, nil, true},
{"typed bad", []string{"x a"}, nil, true},
{"json bad", []any{"x a"}, nil, true},
{"json not a string", []any{3}, nil, true},
{"a string", "x/a", nil, true},
}
for _, c := range cases {
got, err := packagesArg(c.in)
if c.refused != (err != nil) {
t.Errorf("%s: err = %v, refused want %v", c.name, err, c.refused)
continue
}
if !c.refused && !reflect.DeepEqual(got, c.want) {
t.Errorf("%s: got %#v, want %#v", c.name, got, c.want)
}
}
}
// 1925/AC2: a check ask through the module's real build_go_test body gets past the packages argument: the digest names
// a package, buildstep.RunAsk hands it in as a typed list, and the step fails only later, on the workspace nobody fetched.
func TestACheckAskReachesTheTestStepPastItsPackages(t *testing.T) {
m := &Module{Node: "ace", Work: t.TempDir()}
var calls []string
steps := fakeSteps(&calls)
for i := range steps {
switch steps[i].Name {
case "build_go_digest":
steps[i].Run = func(ctx context.Context, args map[string]any) (any, error) {
return map[string]any{"level": "unit", "packages": map[string]string{"x/a": "d1"}}, nil
}
case "build_go_test":
for _, real := range m.StepTools() {
if real.Name == "build_go_test" {
steps[i].Run = real.Run
}
}
}
}
a := delivery.StepAsk{ID: buildstep.NewID(), Kind: delivery.AskCheck, Language: delivery.AskGo, Asker: "mesh-delivery-checker",
At: time.Now(), Check: &delivery.CheckJob{Fetch: map[string]any{"repository": "novox/x", "commit": askCommit}, Levels: []string{delivery.LevelUnit}}}
out, err := buildstep.RunAsk(context.Background(), m.askSteps(steps), a)
got := ""
if err != nil {
got = err.Error()
}
if strings.Contains(got, "packages is a list") {
t.Fatalf("the test step refused the ask's packages: %s", got)
}
if !strings.Contains(got, "no workspace") {
t.Fatalf("want the step to fail on the unfetched workspace, past its packages; err=%v out=%+v", err, out)
}
}
+97 -22
View File
@@ -7,9 +7,12 @@ package main
import (
"encoding/json"
"fmt"
"log"
"sort"
"strconv"
"strings"
"sync"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
@@ -39,6 +42,53 @@ func readOf(s stdio.KeptState) StateRead {
}}
}
// MaxStateReads is the most values one read tool reads from a state in a call, and stateReadParallel how many of them
// are in flight at once. Every value is a round trip from the tool through the runtime to the bus, so a call that read
// every key of a large state one after the other outlasted the call's 30 s (issue 1827: 1164 keys, 20 MB).
const (
MaxStateReads = 1000
stateReadParallel = 16
)
// readValues reads the values of keys, stateReadParallel at a time, in the order of keys (nil for a key with none). It
// stops starting reads after the first error and returns it.
func readValues(keys []string, get func(string) (json.RawMessage, error)) ([]json.RawMessage, error) {
out := make([]json.RawMessage, len(keys))
var (
wg sync.WaitGroup
mu sync.Mutex
fail error
next = make(chan int)
)
for w := 0; w < stateReadParallel && w < len(keys); w++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := range next {
raw, err := get(keys[i])
mu.Lock()
if err != nil && fail == nil {
fail = err
}
mu.Unlock()
out[i] = raw
}
}()
}
for i := range keys {
mu.Lock()
stop := fail != nil
mu.Unlock()
if stop {
break
}
next <- i
}
close(next)
wg.Wait()
return out, fail
}
// stepKey is the key of a step of this node: the id alone, or the whole key <node>.<id> as a step tool answered it.
func (m *Module) stepKey(id string) (string, error) {
id = strings.TrimSpace(id)
@@ -80,27 +130,41 @@ type StepLine struct {
Reason string `json:"reason,omitempty"`
}
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero).
func (m *Module) RecentSteps(limit int) ([]StepLine, error) {
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero). Only
// the node's own keys (<node>.) are listed and the newest max(limit*4, 100) of them read (at most MaxStateReads): a step's
// id is base-36 milliseconds and random digits, so its string order is its start order. cut says older steps were not
// read. The values still order the lines, since an ask step's id carries the ask time.
func (m *Module) RecentSteps(limit int) (lines []StepLine, cut bool, err error) {
if limit <= 0 {
limit = 10
}
if m.ReadSteps.Keys == nil || m.ReadSteps.Get == nil {
return nil, fmt.Errorf("the steps state cannot be read here")
return nil, false, fmt.Errorf("the steps state cannot be read here")
}
start := time.Now()
keys, err := m.ReadSteps.Keys()
if err != nil {
return nil, err
return nil, false, err
}
own := keys[:0:0]
for _, k := range keys {
if strings.HasPrefix(k, m.Node+".") {
if node, _, err := delivery.ParseStepKey(k); err == nil && node == m.Node {
own = append(own, k)
}
}
}
sort.Sort(sort.Reverse(sort.StringSlice(own)))
if n := min(max(limit*4, 100), MaxStateReads); len(own) > n {
own, cut = own[:n], true
}
raws, err := readValues(own, m.ReadSteps.Get)
log.Printf("steps read: %d of %d keys listed read in %v (cut %v)", len(own), len(keys), time.Since(start).Round(time.Millisecond), cut)
if err != nil {
return nil, cut, err
}
var steps []delivery.Step
for _, k := range keys {
if node, _, err := delivery.ParseStepKey(k); err != nil || node != m.Node {
continue
}
raw, err := m.ReadSteps.Get(k)
if err != nil {
return nil, err
}
for _, raw := range raws {
if len(raw) == 0 {
continue
}
@@ -122,7 +186,7 @@ func (m *Module) RecentSteps(limit int) ([]StepLine, error) {
}
out = append(out, l)
}
return out, nil
return out, cut, nil
}
// ReadToolchains is every key of the toolchains state (<node>.<language>) with its stored value.
@@ -130,24 +194,30 @@ func (m *Module) ReadToolchains() (map[string]delivery.Toolchain, error) {
if m.ReadToolchainsState.Keys == nil || m.ReadToolchainsState.Get == nil {
return nil, fmt.Errorf("the toolchains state cannot be read here")
}
start := time.Now()
keys, err := m.ReadToolchainsState.Keys()
if err != nil {
return nil, err
}
cut := len(keys) > MaxStateReads
if cut {
keys = keys[:MaxStateReads] // a node has a key per language: far fewer in practice
}
raws, err := readValues(keys, m.ReadToolchainsState.Get)
log.Printf("toolchains read: %d keys read in %v (cut %v: keys past %d were not read)", len(keys), time.Since(start).Round(time.Millisecond), cut, MaxStateReads)
if err != nil {
return nil, err
}
out := map[string]delivery.Toolchain{}
for _, k := range keys {
raw, err := m.ReadToolchainsState.Get(k)
if err != nil {
return nil, err
}
for i, raw := range raws {
if len(raw) == 0 {
continue
}
t, err := delivery.DecodeToolchain(raw)
if err != nil {
return nil, fmt.Errorf("%s: %w", k, err)
return nil, fmt.Errorf("%s: %w", keys[i], err)
}
out[k] = t
out[keys[i]] = t
}
return out, nil
}
@@ -171,8 +241,13 @@ func (m *Module) ReadTools() []stdio.Tool {
Input: map[string]any{"type": "object", "properties": map[string]any{
"limit": map[string]any{"type": "integer", "description": "how many, at most; 10 when left out"}}},
Run: func(args map[string]any) (any, error) {
steps, err := m.RecentSteps(intArg(args["limit"]))
return map[string]any{"node": m.Node, "steps": steps}, err
steps, cut, err := m.RecentSteps(intArg(args["limit"]))
ans := map[string]any{"node": m.Node, "steps": steps}
if cut {
ans["cut"] = true
ans["note"] = fmt.Sprintf("this node has more steps stored than one call reads; older steps were not read (ask for a larger limit, at most %d reads)", MaxStateReads)
}
return ans, err
}},
{Name: ToolchainsReadTool,
Description: "What the module's toolchains state holds: each key <node>.<language> with the version installed, the " +
@@ -3,8 +3,10 @@ package main
import (
"context"
"encoding/json"
"fmt"
"slices"
"strings"
"sync/atomic"
"testing"
"time"
@@ -145,3 +147,62 @@ func TestReadToolsAreInTheManifestAndNeedNoToolchain(t *testing.T) {
t.Errorf("the read refused: %v", err)
}
}
// 1827: on the live shape (4 nodes, 600 steps each; a steps state of 1164 keys / 20 MB timed out at 30 s) the steps tool
// lists, then reads only this node's keys, in parallel, at most MaxStateReads of them, and says when it cut.
func TestRecentStepsReadsOnlyTheNodesKeysInParallelAndBounded(t *testing.T) {
t0 := time.Date(2026, 10, 25, 9, 0, 0, 0, time.UTC)
for _, row := range []struct {
perNode, limit int
reads int64
steps int
cut bool
}{
{20, 0, 20, 10, false},
{600, 0, 100, 10, true},
{1500, 300, MaxStateReads, 300, true},
} {
var reads, inFlight, maxFlight atomic.Int64
stored := map[string][]byte{}
for _, n := range []string{"ace", "g14", "novox", "shanks"} {
for i := 0; i < row.perNode; i++ {
b, _ := delivery.EncodeStep(delivery.Step{ID: fmt.Sprintf("s%04d", i), Tool: "t", Node: n, State: delivery.StepPassed, Started: t0.Add(time.Duration(i) * time.Second)})
stored[n+".s"+fmt.Sprintf("%04d", i)] = b
}
}
m := &Module{Node: "shanks"}
m.ReadSteps = StateRead{
Keys: func() ([]string, error) {
var k []string
for key := range stored {
k = append(k, key)
}
return k, nil
},
Get: func(key string) (json.RawMessage, error) {
reads.Add(1)
now := inFlight.Add(1)
for o := maxFlight.Load(); now > o && !maxFlight.CompareAndSwap(o, now); o = maxFlight.Load() {
}
time.Sleep(time.Millisecond)
inFlight.Add(-1)
if !strings.HasPrefix(key, "shanks.") {
t.Errorf("read %s, a key of another node", key)
}
return stored[key], nil
},
}
start := time.Now()
lines, cut, err := m.RecentSteps(row.limit)
if err != nil || len(lines) != row.steps || cut != row.cut || reads.Load() != row.reads {
t.Errorf("%d a node: %d steps, cut %v, %d reads, err %v; want %d, %v, %d", row.perNode, len(lines), cut, reads.Load(), err, row.steps, row.cut, row.reads)
}
if row.cut && len(lines) > 0 && lines[0].ID != fmt.Sprintf("s%04d", row.perNode-1) {
t.Errorf("%d a node: newest answered %s", row.perNode, lines[0].ID)
}
if maxFlight.Load() < 2 {
t.Errorf("%d a node: values read one at a time", row.perNode)
}
t.Logf("%d a node: %d reads in %v (one after the other: %v)", row.perNode, reads.Load(), time.Since(start), time.Duration(reads.Load())*time.Millisecond)
}
}
@@ -0,0 +1 @@
These tests read other catalogue modules' files, outside this module (issue 1935): never kept.
+19 -13
View File
@@ -127,17 +127,25 @@ func (m *Module) besideEnv(workspace string) []string {
return []string{"MESH_CHECK_BESIDE=" + dir}
}
// packagesArg reads the packages argument: a list of package patterns, as []any when it came over the bus as JSON and
// as []string when an ask handed it in process (issue 1925).
func packagesArg(v any) ([]string, error) {
if v == nil {
var list []string
switch t := v.(type) {
case nil:
return nil, nil
}
list, ok := v.([]any)
if !ok {
case []string:
list = t
case []any:
for _, e := range t {
s, _ := e.(string)
list = append(list, s)
}
default:
return nil, fmt.Errorf("packages is a list of package patterns")
}
out := []string{}
for _, e := range list {
s, _ := e.(string)
for _, s := range list {
if !packageOK.MatchString(s) || strings.Contains(strings.TrimSuffix(s, "/..."), "..") {
return nil, fmt.Errorf("%q is not a package pattern", s)
}
@@ -201,13 +209,11 @@ func (m *Module) runLevel(ctx context.Context, tree string, o levelOpts) (*TestR
// gofmt and go vet of each directory the level runs come before its tests (ADR 0529 §1; issue 1887 D7-go): a
// directory that fails them fails the level at once, and their time is the result's Lint.
linted := m.clock()
var failed []Failure
for _, d := range todo {
f, err := m.lint(ctx, run, tree, d, append(m.goEnvFor(filepath.Join(tree, d)), o.env...))
if err != nil {
return nil, err
}
failed = append(failed, f...)
failed, err := m.lintAll(ctx, run, tree, todo, func(d string) []string {
return append(m.goEnvFor(filepath.Join(tree, d)), o.env...)
})
if err != nil {
return nil, err
}
res.Lint = m.clock().Sub(linted).Seconds()
if len(failed) > 0 {
@@ -24,3 +24,15 @@ The tar streams (the archives gunzipped) of the three archives above, captured t
gunzip and sha256) at origin/main 0205b8f7 on 2026-10-21: directory f8af1d94...d1b6, within 294e2bec...07fb, file
055cffc2...428e. The digests above hold the gzip bytes of go1.27.1. The first try of issue 1589 on the merge check's build node answered
other digests; the tests therefore compare the stream always and the digest only under go1.27.1.
go-test-json-pkg-*.jsonl are the standard output and error of `go test -json -count=1 ./...` (go1.27.3, the timeouts with
`-timeout 2s`), captured on 2026-10-25 for issue 1901 in throwaway one-file modules that each fail without a test's own
assertion: panicking (a nil map write in TestPanics), timeout (TestWaitsForever sleeps past the timeout), deeptimeout (two
parallel tests and sixty blocked goroutines past the timeout, so the goroutine dump pushes the panic line out of the last
4 KB), buildfail (a type error in a test file), exiting (os.Exit(3) in TestExits), killed (TestKilled sends itself SIGKILL),
passthenfail (TestMain panics in a goroutine after m.Run passed) and mainexit (TestMain exits 2 after m.Run passed: go test
prints "PASS" and then "FAIL <pkg>").
go-test-json-pkg-race.jsonl and go-test-json-pkg-racemain.jsonl are the same, with `go test -race -json -count=1 ./...`
(CGO_ENABLED=1, go1.27.3, 2026-10-25, issue 1901): race (TestRacesOnTheSeat writes a package variable from itself and a
goroutine) and racemain (TestMain does so after m.Run passed: go test prints "PASS", then the race report and "Found 1
data race(s)", then "FAIL <pkg>").
@@ -0,0 +1,13 @@
{"Action":"start","Package":"x"}
{"Action":"run","Package":"x","Test":"TestThree"}
{"Action":"output","Package":"x","Test":"TestThree","Output":"=== RUN TestThree\n","OutputType":"frame"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" a_test.go:4: judged against /var/lib/mesh-build-go-work/id/beside/mesh-catalog as cloned\n"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" a_test.go:5: 3 subscription(s) derive to a subject no emitter owns:\n","OutputType":"error"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" a.b.c\n","OutputType":"error-continue"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" d.e.f\n","OutputType":"error-continue"}
{"Action":"output","Package":"x","Test":"TestThree","Output":" g.h.i\n","OutputType":"error-continue"}
{"Action":"output","Package":"x","Test":"TestThree","Output":"--- FAIL: TestThree (0.00s)\n","OutputType":"frame"}
{"Action":"fail","Package":"x","Test":"TestThree","Elapsed":0}
{"Action":"output","Package":"x","Output":"FAIL\n","OutputType":"frame"}
{"Action":"output","Package":"x","Output":"FAIL\tx\t0.002s\n","OutputType":"frame"}
{"Action":"fail","Package":"x","Elapsed":0.002}
@@ -0,0 +1,6 @@
{"ImportPath":"example.com/buildfail [example.com/buildfail.test]","Action":"build-output","Output":"# example.com/buildfail [example.com/buildfail.test]\n"}
{"ImportPath":"example.com/buildfail [example.com/buildfail.test]","Action":"build-output","Output":"./p_test.go:5:40: cannot use \"seat\" (untyped string constant) as int value in variable declaration\n"}
{"ImportPath":"example.com/buildfail [example.com/buildfail.test]","Action":"build-fail"}
{"Time":"2026-10-25T15:34:33.261137737+01:00","Action":"start","Package":"example.com/buildfail"}
{"Time":"2026-10-25T15:34:33.261255019+01:00","Action":"output","Package":"example.com/buildfail","Output":"FAIL\texample.com/buildfail [build failed]\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.261268099+01:00","Action":"fail","Package":"example.com/buildfail","Elapsed":0,"FailedBuild":"example.com/buildfail [example.com/buildfail.test]"}
@@ -0,0 +1,417 @@
{"Time":"2026-10-25T15:35:18.991630993+01:00","Action":"start","Package":"example.com/deeptimeout"}
{"Time":"2026-10-25T15:35:18.993895188+01:00","Action":"run","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat"}
{"Time":"2026-10-25T15:35:18.993963909+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":"=== RUN TestHoldsTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.99399484+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":"=== PAUSE TestHoldsTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.9939982+01:00","Action":"pause","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat"}
{"Time":"2026-10-25T15:35:18.99400173+01:00","Action":"run","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus"}
{"Time":"2026-10-25T15:35:18.99400456+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"=== RUN TestWaitsForTheBus\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.99400838+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"=== PAUSE TestWaitsForTheBus\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.99401133+01:00","Action":"pause","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus"}
{"Time":"2026-10-25T15:35:18.99401441+01:00","Action":"cont","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat"}
{"Time":"2026-10-25T15:35:18.99401704+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":"=== CONT TestHoldsTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:18.994074141+01:00","Action":"cont","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus"}
{"Time":"2026-10-25T15:35:18.994102612+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"=== CONT TestWaitsForTheBus\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:21.00935195+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"panic: test timed out after 2s\n"}
{"Time":"2026-10-25T15:35:21.00938383+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\trunning tests:\n"}
{"Time":"2026-10-25T15:35:21.009394391+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t\tTestHoldsTheSeat (2s)\n"}
{"Time":"2026-10-25T15:35:21.009398021+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t\tTestWaitsForTheBus (2s)\n"}
{"Time":"2026-10-25T15:35:21.009401801+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009406481+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 81 [running]:\n"}
{"Time":"2026-10-25T15:35:21.009409911+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.(*M).startAlarm.func1()\n"}
{"Time":"2026-10-25T15:35:21.009413361+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2961 +0x34a\n"}
{"Time":"2026-10-25T15:35:21.009418341+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by time.goFunc\n"}
{"Time":"2026-10-25T15:35:21.009421781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/time/sleep.go:182 +0x2d\n"}
{"Time":"2026-10-25T15:35:21.009424991+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009428341+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 1 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009431721+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner.func1()\n"}
{"Time":"2026-10-25T15:35:21.009441811+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2144 +0x425\n"}
{"Time":"2026-10-25T15:35:21.009445511+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner(0x282e9702008, 0x282e9646bc8)\n"}
{"Time":"2026-10-25T15:35:21.009449641+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2201 +0x123\n"}
{"Time":"2026-10-25T15:35:21.009453471+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.runTests({0x55a45d, 0x17}, {0x55a45d, 0x17}, 0x282e95940f0, {0x6f19f0, 0x2, 0x2}, {0xc2afa2ca3b3c0c8d, 0x7737f168, ...})\n"}
{"Time":"2026-10-25T15:35:21.009462092+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2742 +0x510\n"}
{"Time":"2026-10-25T15:35:21.009467222+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.(*M).Run(0x282e96581e0)\n"}
{"Time":"2026-10-25T15:35:21.009470662+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2602 +0x6af\n"}
{"Time":"2026-10-25T15:35:21.009473692+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"main.main()\n"}
{"Time":"2026-10-25T15:35:21.009476942+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t_testmain.go:48 +0x9b\n"}
{"Time":"2026-10-25T15:35:21.009480092+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009483242+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 7 [sleep]:\n"}
{"Time":"2026-10-25T15:35:21.009486502+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"time.Sleep(0xdf8475800)\n"}
{"Time":"2026-10-25T15:35:21.009490682+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/runtime/time.go:368 +0x165\n"}
{"Time":"2026-10-25T15:35:21.009493812+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat(0x282e9702248?)\n"}
{"Time":"2026-10-25T15:35:21.009498842+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:14 +0x9e\n"}
{"Time":"2026-10-25T15:35:21.009502242+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner(0x282e9702248, 0x6d5e78)\n"}
{"Time":"2026-10-25T15:35:21.009505742+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:35:21.009509292+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:35:21.009512752+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:35:21.009515792+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009518623+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 8 [sleep]:\n"}
{"Time":"2026-10-25T15:35:21.009521303+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"time.Sleep(0xdf8475800)\n"}
{"Time":"2026-10-25T15:35:21.009524423+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/runtime/time.go:368 +0x165\n"}
{"Time":"2026-10-25T15:35:21.009531403+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestWaitsForTheBus(0x282e9702488?)\n"}
{"Time":"2026-10-25T15:35:21.009534943+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:19 +0x25\n"}
{"Time":"2026-10-25T15:35:21.009538403+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"testing.tRunner(0x282e9702488, 0x6d5e80)\n"}
{"Time":"2026-10-25T15:35:21.009541953+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:35:21.009545233+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:35:21.009548323+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:35:21.009551343+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009554263+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 9 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009557523+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009561063+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009564513+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009571113+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009574233+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009577433+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 10 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009580823+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009584174+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009587504+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009590974+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009594004+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009597184+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 11 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009600394+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009603874+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009610954+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009614714+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009618104+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009621364+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 12 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009624704+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009628324+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009631954+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009635404+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009638734+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009641904+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 13 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009645325+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009648645+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009652065+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009655625+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009658885+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009663385+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 14 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009666735+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009670235+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009676385+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009680205+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009683315+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009686475+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 15 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009689785+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009697235+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009700795+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009704985+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009708085+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009711346+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 16 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009714586+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009717946+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009721676+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009725626+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009728866+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009732106+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 17 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009735376+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009738956+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009742326+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009746436+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009750666+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009754126+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 18 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009758286+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009761846+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009765776+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009769816+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009773167+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009780507+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 19 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009784207+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009788007+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009791627+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009796347+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009799427+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009802937+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 20 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009806667+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009809987+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009813827+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009818087+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009821397+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009824227+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 21 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009827607+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009831307+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009835167+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009839528+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009842728+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009845888+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 22 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009849168+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009852548+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009855958+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009863648+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009866938+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009870088+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 23 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009879828+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009883728+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009887608+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009891548+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009895238+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009898748+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 24 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009902349+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009907109+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009911519+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009915609+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009918939+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009922329+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 25 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009928329+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009931909+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.009935489+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.009939429+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.009942699+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.009946019+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 26 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.009949559+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.009953489+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.00996244+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.00996647+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.00996987+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.00997314+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 27 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.00997664+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.00998035+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.00998418+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.00998793+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.00999131+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.00999534+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 28 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.00999894+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01000268+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.0100087+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01001258+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01001605+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01002007+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 29 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01002393+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010027531+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010031351+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010035521+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010038781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010042281+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 30 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010046091+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010054261+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010057881+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010061781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010064931+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010068261+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 31 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010072151+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010075981+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010079841+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010098802+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010103202+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010106382+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 32 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010109852+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010113242+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010116952+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010120672+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010124112+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010127162+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 33 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010130662+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010134442+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010138142+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010141972+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010145362+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010152262+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 34 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010156353+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010159913+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010163533+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010167403+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010170533+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010173953+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 35 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010177453+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010181033+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010184693+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010188583+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010191753+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010195033+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 36 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010198603+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010202393+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010206123+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010210583+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010213503+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010216914+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 37 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010220134+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010223644+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010227154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010235154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010238354+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010241754+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 38 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010270284+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010275144+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010278714+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010282825+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010285795+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010288935+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 39 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010292405+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010295915+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010299545+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010311835+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010315565+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010318975+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 40 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010322655+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010326225+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010329815+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010333745+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010337095+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010340515+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 41 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010344066+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010347706+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010355596+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010359046+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010362036+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010365076+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 42 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010367956+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010370906+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010375366+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010378386+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010381006+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010383666+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 43 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010386676+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010389786+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010392636+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010395746+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010398466+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010401166+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 44 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010404226+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010407186+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010410117+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010413237+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010416157+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010418897+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 45 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010423157+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010430107+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010433197+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010451207+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010454217+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010456967+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 46 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010459877+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010463967+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010467067+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010470597+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010473418+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010476108+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 47 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010478998+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010481958+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010484978+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010488258+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010490888+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010493628+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 48 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010496588+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010499498+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010502498+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010505768+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010508558+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010514148+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 49 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010517178+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010520238+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010523478+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010526818+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010529718+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010532508+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 50 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010535639+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010538749+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010541939+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010546689+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010549869+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010552989+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 51 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010556299+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010560079+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010563479+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010567189+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010570299+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010573079+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 52 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010576319+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010579599+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010583099+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010590519+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010594029+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010596989+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 53 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01059986+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01060433+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.01060789+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01061136+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01061393+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01061667+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 54 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01061984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.0106227+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.01062566+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01062876+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01063131+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01063428+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 55 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01063719+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01063997+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.01064278+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.01064582+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.01064836+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.01065114+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 56 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.01065437+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.01065724+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010671481+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010675671+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010678431+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010691171+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 57 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010694531+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010697641+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010700781+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010704001+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010706611+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010709421+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 58 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010712331+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010715351+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010718431+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010721691+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010724371+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010727362+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 59 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010730592+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010733662+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010736712+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010740162+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010742922+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010745772+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 60 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010752222+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010755402+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010758542+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010761822+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010764592+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010767992+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 61 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010771062+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010774282+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010779382+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010782942+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010785722+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010788573+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 62 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010791893+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010794943+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010798083+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010801683+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010804293+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010807283+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 63 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010810443+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010813553+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010816793+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010820233+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010823063+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010829493+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 64 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010832683+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010835953+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010839223+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010842463+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010845333+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010848423+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 65 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010851663+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010854964+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010858364+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010861714+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010864914+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010867984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 66 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010870944+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010874114+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010877464+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010880504+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010883144+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010886194+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 67 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010889154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010892234+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010895524+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010902154+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.010904984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\n"}
{"Time":"2026-10-25T15:35:21.010907984+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"goroutine 68 [chan receive]:\n"}
{"Time":"2026-10-25T15:35:21.010911064+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"example.com/deeptimeout.TestHoldsTheSeat.func1()\n"}
{"Time":"2026-10-25T15:35:21.010914074+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x19\n"}
{"Time":"2026-10-25T15:35:21.010917365+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"created by example.com/deeptimeout.TestHoldsTheSeat in goroutine 7\n"}
{"Time":"2026-10-25T15:35:21.010920505+01:00","Action":"output","Package":"example.com/deeptimeout","Test":"TestWaitsForTheBus","Output":"\t/tmp/fix-1901b/deeptimeout/p_test.go:12 +0x37\n"}
{"Time":"2026-10-25T15:35:21.013209581+01:00","Action":"output","Package":"example.com/deeptimeout","Output":"FAIL\texample.com/deeptimeout\t2.021s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:35:21.013258851+01:00","Action":"fail","Package":"example.com/deeptimeout","Elapsed":2.022}
@@ -0,0 +1,10 @@
{"Time":"2026-10-25T15:34:33.768742837+01:00","Action":"start","Package":"example.com/exiting"}
{"Time":"2026-10-25T15:34:33.773822287+01:00","Action":"run","Package":"example.com/exiting","Test":"TestFine"}
{"Time":"2026-10-25T15:34:33.773896608+01:00","Action":"output","Package":"example.com/exiting","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.773940319+01:00","Action":"output","Package":"example.com/exiting","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.773945359+01:00","Action":"pass","Package":"example.com/exiting","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:34:33.773952529+01:00","Action":"run","Package":"example.com/exiting","Test":"TestExits"}
{"Time":"2026-10-25T15:34:33.773956439+01:00","Action":"output","Package":"example.com/exiting","Test":"TestExits","Output":"=== RUN TestExits\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.773959829+01:00","Action":"output","Package":"example.com/exiting","Test":"TestExits","Output":" p_test.go:11: closing the store\n"}
{"Time":"2026-10-25T15:34:33.775158538+01:00","Action":"output","Package":"example.com/exiting","Output":"FAIL\texample.com/exiting\t0.006s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.775186509+01:00","Action":"fail","Package":"example.com/exiting","Elapsed":0.006}
@@ -0,0 +1,7 @@
{"Time":"2026-10-25T15:34:34.328258934+01:00","Action":"start","Package":"example.com/killed"}
{"Time":"2026-10-25T15:34:34.331465435+01:00","Action":"run","Package":"example.com/killed","Test":"TestKilled"}
{"Time":"2026-10-25T15:34:34.331699248+01:00","Action":"output","Package":"example.com/killed","Test":"TestKilled","Output":"=== RUN TestKilled\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.331853111+01:00","Action":"output","Package":"example.com/killed","Test":"TestKilled","Output":" p_test.go:10: the kernel is about to kill this\n"}
{"Time":"2026-10-25T15:34:34.334563213+01:00","Action":"output","Package":"example.com/killed","Test":"TestKilled","Output":"signal: killed\n"}
{"Time":"2026-10-25T15:34:34.334611114+01:00","Action":"output","Package":"example.com/killed","Output":"FAIL\texample.com/killed\t0.006s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.334627134+01:00","Action":"fail","Package":"example.com/killed","Elapsed":0.006}
@@ -0,0 +1,8 @@
{"Time":"2026-10-25T15:34:35.274953775+01:00","Action":"start","Package":"example.com/mainexit"}
{"Time":"2026-10-25T15:34:35.278129565+01:00","Action":"run","Package":"example.com/mainexit","Test":"TestFine"}
{"Time":"2026-10-25T15:34:35.278191516+01:00","Action":"output","Package":"example.com/mainexit","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.278341979+01:00","Action":"output","Package":"example.com/mainexit","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.278351739+01:00","Action":"pass","Package":"example.com/mainexit","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:34:35.278359039+01:00","Action":"output","Package":"example.com/mainexit","Output":"PASS\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.279516487+01:00","Action":"output","Package":"example.com/mainexit","Output":"FAIL\texample.com/mainexit\t0.004s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:35.279539298+01:00","Action":"fail","Package":"example.com/mainexit","Elapsed":0.005}
@@ -0,0 +1,27 @@
{"Time":"2026-10-25T15:34:33.031091436+01:00","Action":"start","Package":"example.com/panicking"}
{"Time":"2026-10-25T15:34:33.036353279+01:00","Action":"run","Package":"example.com/panicking","Test":"TestBefore"}
{"Time":"2026-10-25T15:34:33.036441781+01:00","Action":"output","Package":"example.com/panicking","Test":"TestBefore","Output":"=== RUN TestBefore\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.036568163+01:00","Action":"output","Package":"example.com/panicking","Test":"TestBefore","Output":"--- PASS: TestBefore (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.036609863+01:00","Action":"pass","Package":"example.com/panicking","Test":"TestBefore","Elapsed":0}
{"Time":"2026-10-25T15:34:33.036641304+01:00","Action":"run","Package":"example.com/panicking","Test":"TestPanics"}
{"Time":"2026-10-25T15:34:33.036645914+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"=== RUN TestPanics\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.036698995+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":" p_test.go:8: about to look up the seat\n"}
{"Time":"2026-10-25T15:34:33.036773996+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"--- FAIL: TestPanics (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.03897418+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"panic: assignment to entry in nil map [recovered, repanicked]\n"}
{"Time":"2026-10-25T15:34:33.040332172+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\n"}
{"Time":"2026-10-25T15:34:33.040414493+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"goroutine 8 [running]:\n"}
{"Time":"2026-10-25T15:34:33.040521475+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"testing.tRunner.func1.2({0x6b8120, 0x6eff80})\n"}
{"Time":"2026-10-25T15:34:33.040601826+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2125 +0x232\n"}
{"Time":"2026-10-25T15:34:33.040641947+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"testing.tRunner.func1()\n"}
{"Time":"2026-10-25T15:34:33.040720648+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2128 +0x329\n"}
{"Time":"2026-10-25T15:34:33.04083286+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"panic({0x6b8120?, 0x6eff80?})\n"}
{"Time":"2026-10-25T15:34:33.040912921+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/runtime/panic.go:859 +0x125\n"}
{"Time":"2026-10-25T15:34:33.040981652+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"example.com/panicking.TestPanics(0x228226cd0488?)\n"}
{"Time":"2026-10-25T15:34:33.041059903+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/tmp/fix-1901b/panicking/p_test.go:10 +0x53\n"}
{"Time":"2026-10-25T15:34:33.041132704+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"testing.tRunner(0x228226cd0488, 0x6d5bc8)\n"}
{"Time":"2026-10-25T15:34:33.041215586+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:34:33.041221966+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:34:33.041225556+01:00","Action":"output","Package":"example.com/panicking","Test":"TestPanics","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:34:33.041984088+01:00","Action":"fail","Package":"example.com/panicking","Test":"TestPanics","Elapsed":0}
{"Time":"2026-10-25T15:34:33.041996638+01:00","Action":"output","Package":"example.com/panicking","Output":"FAIL\texample.com/panicking\t0.010s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:33.042044079+01:00","Action":"fail","Package":"example.com/panicking","Elapsed":0.011}
@@ -0,0 +1,15 @@
{"Time":"2026-10-25T15:34:34.919624192+01:00","Action":"start","Package":"example.com/passthenfail"}
{"Time":"2026-10-25T15:34:34.922861224+01:00","Action":"run","Package":"example.com/passthenfail","Test":"TestFine"}
{"Time":"2026-10-25T15:34:34.922932965+01:00","Action":"output","Package":"example.com/passthenfail","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.922969205+01:00","Action":"output","Package":"example.com/passthenfail","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.922979956+01:00","Action":"pass","Package":"example.com/passthenfail","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:34:34.922989416+01:00","Action":"output","Package":"example.com/passthenfail","Output":"PASS\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.925022528+01:00","Action":"output","Package":"example.com/passthenfail","Output":"panic: the bus closed after the tests: nats: connection closed\n"}
{"Time":"2026-10-25T15:34:34.925043388+01:00","Action":"output","Package":"example.com/passthenfail","Output":"\n"}
{"Time":"2026-10-25T15:34:34.925046848+01:00","Action":"output","Package":"example.com/passthenfail","Output":"goroutine 20 [running]:\n"}
{"Time":"2026-10-25T15:34:34.925051338+01:00","Action":"output","Package":"example.com/passthenfail","Output":"example.com/passthenfail.TestMain.func1()\n"}
{"Time":"2026-10-25T15:34:34.925055878+01:00","Action":"output","Package":"example.com/passthenfail","Output":"\t/tmp/fix-1901b/passthenfail/p_test.go:13 +0x25\n"}
{"Time":"2026-10-25T15:34:34.925061428+01:00","Action":"output","Package":"example.com/passthenfail","Output":"created by example.com/passthenfail.TestMain in goroutine 1\n"}
{"Time":"2026-10-25T15:34:34.925065028+01:00","Action":"output","Package":"example.com/passthenfail","Output":"\t/tmp/fix-1901b/passthenfail/p_test.go:13 +0x25\n"}
{"Time":"2026-10-25T15:34:34.925390203+01:00","Action":"output","Package":"example.com/passthenfail","Output":"FAIL\texample.com/passthenfail\t0.005s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:34.925402904+01:00","Action":"fail","Package":"example.com/passthenfail","Elapsed":0.006}
@@ -0,0 +1,53 @@
{"Time":"2026-10-25T15:50:42.908557309+01:00","Action":"start","Package":"example.com/race"}
{"Time":"2026-10-25T15:50:42.924298679+01:00","Action":"run","Package":"example.com/race","Test":"TestBefore"}
{"Time":"2026-10-25T15:50:42.92438195+01:00","Action":"output","Package":"example.com/race","Test":"TestBefore","Output":"=== RUN TestBefore\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.924648842+01:00","Action":"output","Package":"example.com/race","Test":"TestBefore","Output":"--- PASS: TestBefore (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.924685133+01:00","Action":"pass","Package":"example.com/race","Test":"TestBefore","Elapsed":0}
{"Time":"2026-10-25T15:50:42.924728573+01:00","Action":"run","Package":"example.com/race","Test":"TestRacesOnTheSeat"}
{"Time":"2026-10-25T15:50:42.924733573+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"=== RUN TestRacesOnTheSeat\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.92637298+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"==================\n"}
{"Time":"2026-10-25T15:50:42.92641567+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"WARNING: DATA RACE\n"}
{"Time":"2026-10-25T15:50:42.926444061+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Write at 0x000000833468 by goroutine 10:\n"}
{"Time":"2026-10-25T15:50:42.926462621+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" example.com/race.TestRacesOnTheSeat.func1()\n"}
{"Time":"2026-10-25T15:50:42.926467121+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /tmp/fix-1901b/race/p_test.go:13 +0x24\n"}
{"Time":"2026-10-25T15:50:42.926483061+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"\n"}
{"Time":"2026-10-25T15:50:42.926509741+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Previous write at 0x000000833468 by goroutine 9:\n"}
{"Time":"2026-10-25T15:50:42.926530142+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" example.com/race.TestRacesOnTheSeat()\n"}
{"Time":"2026-10-25T15:50:42.926534952+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /tmp/fix-1901b/race/p_test.go:14 +0x30\n"}
{"Time":"2026-10-25T15:50:42.926550462+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.tRunner()\n"}
{"Time":"2026-10-25T15:50:42.926554612+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2195 +0x21c\n"}
{"Time":"2026-10-25T15:50:42.926571142+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*T).Run.gowrap1()\n"}
{"Time":"2026-10-25T15:50:42.926575812+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2260 +0x38\n"}
{"Time":"2026-10-25T15:50:42.926592202+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"\n"}
{"Time":"2026-10-25T15:50:42.926607792+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Goroutine 10 (running) created at:\n"}
{"Time":"2026-10-25T15:50:42.926624292+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" example.com/race.TestRacesOnTheSeat()\n"}
{"Time":"2026-10-25T15:50:42.926628923+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /tmp/fix-1901b/race/p_test.go:13 +0x24\n"}
{"Time":"2026-10-25T15:50:42.926643853+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.tRunner()\n"}
{"Time":"2026-10-25T15:50:42.926648103+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2195 +0x21c\n"}
{"Time":"2026-10-25T15:50:42.926674843+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*T).Run.gowrap1()\n"}
{"Time":"2026-10-25T15:50:42.926680673+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2260 +0x38\n"}
{"Time":"2026-10-25T15:50:42.926695433+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"\n"}
{"Time":"2026-10-25T15:50:42.926711063+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"Goroutine 9 (running) created at:\n"}
{"Time":"2026-10-25T15:50:42.926726614+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*T).Run()\n"}
{"Time":"2026-10-25T15:50:42.926730814+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2260 +0xb12\n"}
{"Time":"2026-10-25T15:50:42.926746484+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.runTests.func1()\n"}
{"Time":"2026-10-25T15:50:42.926750764+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2744 +0x84\n"}
{"Time":"2026-10-25T15:50:42.926765354+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.tRunner()\n"}
{"Time":"2026-10-25T15:50:42.926769844+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2195 +0x21c\n"}
{"Time":"2026-10-25T15:50:42.926791794+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.runTests()\n"}
{"Time":"2026-10-25T15:50:42.926796704+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2742 +0x9e9\n"}
{"Time":"2026-10-25T15:50:42.926811544+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.(*M).Run()\n"}
{"Time":"2026-10-25T15:50:42.926816694+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" /usr/lib/go/src/testing/testing.go:2602 +0xf44\n"}
{"Time":"2026-10-25T15:50:42.926831695+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" main.main()\n"}
{"Time":"2026-10-25T15:50:42.926835935+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" _testmain.go:50 +0x164\n"}
{"Time":"2026-10-25T15:50:42.926850905+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"==================\n"}
{"Time":"2026-10-25T15:50:42.935320241+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":" testing.go:1867: race detected during execution of test\n","OutputType":"error"}
{"Time":"2026-10-25T15:50:42.935520113+01:00","Action":"output","Package":"example.com/race","Test":"TestRacesOnTheSeat","Output":"--- FAIL: TestRacesOnTheSeat (0.01s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.935592973+01:00","Action":"fail","Package":"example.com/race","Test":"TestRacesOnTheSeat","Elapsed":0.01}
{"Time":"2026-10-25T15:50:42.935722225+01:00","Action":"run","Package":"example.com/race","Test":"TestAfter"}
{"Time":"2026-10-25T15:50:42.935728845+01:00","Action":"output","Package":"example.com/race","Test":"TestAfter","Output":"=== RUN TestAfter\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.935995928+01:00","Action":"output","Package":"example.com/race","Test":"TestAfter","Output":"--- PASS: TestAfter (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.936006328+01:00","Action":"pass","Package":"example.com/race","Test":"TestAfter","Elapsed":0}
{"Time":"2026-10-25T15:50:42.936009978+01:00","Action":"output","Package":"example.com/race","Output":"FAIL\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.937730745+01:00","Action":"output","Package":"example.com/race","Output":"FAIL\texample.com/race\t0.028s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:42.937802066+01:00","Action":"fail","Package":"example.com/race","Elapsed":0.029}
@@ -0,0 +1,27 @@
{"Time":"2026-10-25T15:50:43.551701745+01:00","Action":"start","Package":"example.com/racemain"}
{"Time":"2026-10-25T15:50:43.570709708+01:00","Action":"run","Package":"example.com/racemain","Test":"TestFine"}
{"Time":"2026-10-25T15:50:43.570806059+01:00","Action":"output","Package":"example.com/racemain","Test":"TestFine","Output":"=== RUN TestFine\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:43.57085525+01:00","Action":"output","Package":"example.com/racemain","Test":"TestFine","Output":"--- PASS: TestFine (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:43.57086149+01:00","Action":"pass","Package":"example.com/racemain","Test":"TestFine","Elapsed":0}
{"Time":"2026-10-25T15:50:43.57086974+01:00","Action":"output","Package":"example.com/racemain","Output":"PASS\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:43.572155623+01:00","Action":"output","Package":"example.com/racemain","Output":"==================\n"}
{"Time":"2026-10-25T15:50:43.572168083+01:00","Action":"output","Package":"example.com/racemain","Output":"WARNING: DATA RACE\n"}
{"Time":"2026-10-25T15:50:43.572171933+01:00","Action":"output","Package":"example.com/racemain","Output":"Write at 0x000000836428 by goroutine 9:\n"}
{"Time":"2026-10-25T15:50:43.572176703+01:00","Action":"output","Package":"example.com/racemain","Output":" example.com/racemain.TestMain.func1()\n"}
{"Time":"2026-10-25T15:50:43.572180053+01:00","Action":"output","Package":"example.com/racemain","Output":" /tmp/fix-1901b/racemain/p_test.go:15 +0x24\n"}
{"Time":"2026-10-25T15:50:43.572184923+01:00","Action":"output","Package":"example.com/racemain","Output":"\n"}
{"Time":"2026-10-25T15:50:43.572189613+01:00","Action":"output","Package":"example.com/racemain","Output":"Previous write at 0x000000836428 by main goroutine:\n"}
{"Time":"2026-10-25T15:50:43.572192993+01:00","Action":"output","Package":"example.com/racemain","Output":" example.com/racemain.TestMain()\n"}
{"Time":"2026-10-25T15:50:43.572198573+01:00","Action":"output","Package":"example.com/racemain","Output":" /tmp/fix-1901b/racemain/p_test.go:16 +0x44\n"}
{"Time":"2026-10-25T15:50:43.572202433+01:00","Action":"output","Package":"example.com/racemain","Output":" main.main()\n"}
{"Time":"2026-10-25T15:50:43.572205623+01:00","Action":"output","Package":"example.com/racemain","Output":" _testmain.go:48 +0x171\n"}
{"Time":"2026-10-25T15:50:43.572208443+01:00","Action":"output","Package":"example.com/racemain","Output":"\n"}
{"Time":"2026-10-25T15:50:43.572211573+01:00","Action":"output","Package":"example.com/racemain","Output":"Goroutine 9 (running) created at:\n"}
{"Time":"2026-10-25T15:50:43.572214803+01:00","Action":"output","Package":"example.com/racemain","Output":" example.com/racemain.TestMain()\n"}
{"Time":"2026-10-25T15:50:43.572218043+01:00","Action":"output","Package":"example.com/racemain","Output":" /tmp/fix-1901b/racemain/p_test.go:15 +0x37\n"}
{"Time":"2026-10-25T15:50:43.572225414+01:00","Action":"output","Package":"example.com/racemain","Output":" main.main()\n"}
{"Time":"2026-10-25T15:50:43.572228654+01:00","Action":"output","Package":"example.com/racemain","Output":" _testmain.go:48 +0x171\n"}
{"Time":"2026-10-25T15:50:43.572231664+01:00","Action":"output","Package":"example.com/racemain","Output":"==================\n"}
{"Time":"2026-10-25T15:50:44.581262249+01:00","Action":"output","Package":"example.com/racemain","Output":"Found 1 data race(s)\n"}
{"Time":"2026-10-25T15:50:44.584693204+01:00","Action":"output","Package":"example.com/racemain","Output":"FAIL\texample.com/racemain\t1.031s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:50:44.584737425+01:00","Action":"fail","Package":"example.com/racemain","Elapsed":1.033}
@@ -0,0 +1,43 @@
{"Time":"2026-10-25T15:34:36.049580618+01:00","Action":"start","Package":"example.com/timeout"}
{"Time":"2026-10-25T15:34:36.053250986+01:00","Action":"run","Package":"example.com/timeout","Test":"TestQuick"}
{"Time":"2026-10-25T15:34:36.053354018+01:00","Action":"output","Package":"example.com/timeout","Test":"TestQuick","Output":"=== RUN TestQuick\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:36.053459789+01:00","Action":"output","Package":"example.com/timeout","Test":"TestQuick","Output":"--- PASS: TestQuick (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:36.05348499+01:00","Action":"pass","Package":"example.com/timeout","Test":"TestQuick","Elapsed":0}
{"Time":"2026-10-25T15:34:36.0535114+01:00","Action":"run","Package":"example.com/timeout","Test":"TestWaitsForever"}
{"Time":"2026-10-25T15:34:36.05351549+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"=== RUN TestWaitsForever\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:36.053567451+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":" p_test.go:11: waiting for the bus\n"}
{"Time":"2026-10-25T15:34:38.064693307+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"panic: test timed out after 2s\n"}
{"Time":"2026-10-25T15:34:38.06487122+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\trunning tests:\n"}
{"Time":"2026-10-25T15:34:38.06491684+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t\tTestWaitsForever (2s)\n"}
{"Time":"2026-10-25T15:34:38.065034382+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\n"}
{"Time":"2026-10-25T15:34:38.065108573+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"goroutine 33 [running]:\n"}
{"Time":"2026-10-25T15:34:38.065154754+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.(*M).startAlarm.func1()\n"}
{"Time":"2026-10-25T15:34:38.065386208+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2961 +0x34a\n"}
{"Time":"2026-10-25T15:34:38.065431878+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"created by time.goFunc\n"}
{"Time":"2026-10-25T15:34:38.0655151+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/time/sleep.go:182 +0x2d\n"}
{"Time":"2026-10-25T15:34:38.06553471+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\n"}
{"Time":"2026-10-25T15:34:38.065847805+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"goroutine 1 [chan receive]:\n"}
{"Time":"2026-10-25T15:34:38.066038298+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.(*T).Run(0x3a0209bc8008, {0x5580aa?, 0x3a0209a9caa0?}, 0x6d5b10)\n"}
{"Time":"2026-10-25T15:34:38.066047488+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2268 +0x4f2\n"}
{"Time":"2026-10-25T15:34:38.066051368+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.runTests.func1(0x3a0209bc8008)\n"}
{"Time":"2026-10-25T15:34:38.066056128+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2744 +0x37\n"}
{"Time":"2026-10-25T15:34:38.066059478+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.tRunner(0x3a0209bc8008, 0x3a0209a9cbc8)\n"}
{"Time":"2026-10-25T15:34:38.066064958+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:34:38.066069599+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.runTests({0x559017, 0x13}, {0x559017, 0x13}, 0x3a0209b8e0d8, {0x6f19f0, 0x2, 0x2}, {0xc2afa2bf832a3f56, 0x773908b5, ...})\n"}
{"Time":"2026-10-25T15:34:38.066083309+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2742 +0x510\n"}
{"Time":"2026-10-25T15:34:38.066086879+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.(*M).Run(0x3a0209b8c1e0)\n"}
{"Time":"2026-10-25T15:34:38.066090269+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2602 +0x6af\n"}
{"Time":"2026-10-25T15:34:38.066093429+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"main.main()\n"}
{"Time":"2026-10-25T15:34:38.066097569+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t_testmain.go:48 +0x9b\n"}
{"Time":"2026-10-25T15:34:38.066100639+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\n"}
{"Time":"2026-10-25T15:34:38.066103769+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"goroutine 20 [sleep]:\n"}
{"Time":"2026-10-25T15:34:38.066106969+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"time.Sleep(0xdf8475800)\n"}
{"Time":"2026-10-25T15:34:38.066110279+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/runtime/time.go:368 +0x165\n"}
{"Time":"2026-10-25T15:34:38.066113639+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"example.com/timeout.TestWaitsForever(0x3a0209bc8488?)\n"}
{"Time":"2026-10-25T15:34:38.066117109+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/tmp/fix-1901b/timeout/p_test.go:12 +0x48\n"}
{"Time":"2026-10-25T15:34:38.066120239+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"testing.tRunner(0x3a0209bc8488, 0x6d5b10)\n"}
{"Time":"2026-10-25T15:34:38.066123509+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2195 +0xea\n"}
{"Time":"2026-10-25T15:34:38.066126839+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"created by testing.(*T).Run in goroutine 1\n"}
{"Time":"2026-10-25T15:34:38.066129989+01:00","Action":"output","Package":"example.com/timeout","Test":"TestWaitsForever","Output":"\t/usr/lib/go/src/testing/testing.go:2260 +0x4d4\n"}
{"Time":"2026-10-25T15:34:38.066986453+01:00","Action":"output","Package":"example.com/timeout","Output":"FAIL\texample.com/timeout\t2.017s\n","OutputType":"frame"}
{"Time":"2026-10-25T15:34:38.067002113+01:00","Action":"fail","Package":"example.com/timeout","Elapsed":2.017}
+1 -1
View File
@@ -3,7 +3,7 @@ module meshbuildgo
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023150321-a2c813a6b468
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261025175617-518dae5ec9c8
github.com/distribution/distribution/v3 v3.1.2
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023150321-a2c813a6b468 h1:UcMee+IV2ii5E4dRZdQUaSaeh9IedTFYumuQbBiB0k0=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023150321-a2c813a6b468/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261025175617-518dae5ec9c8 h1:x4mvY7yiGTP5lnzxbiwyj1o24p05hAmzZP7xHg3XX2M=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261025175617-518dae5ec9c8/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/alicebob/miniredis/v2 v2.35.0 h1:QwLphYqCEAo1eu1TqPRN2jgVMPBweeQcR21jeqDCONI=
github.com/alicebob/miniredis/v2 v2.35.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
+3 -2
View File
@@ -10,7 +10,7 @@
"state": [
"toolchains",
"keys",
"steps"
{"name": "steps", "ttl-seconds": 172800}, "pauses"
],
"claims": [
{
@@ -20,7 +20,8 @@
}
],
"reads": [
"mesh-delivery-checker.kept"
"mesh-delivery-checker.kept",
"mesh-delivery-indexer.manifests"
],
"config": {
"capacity": {
@@ -13,6 +13,7 @@ import (
var Kept = []announce.State{
{Name: "keys", Readable: true},
{Name: "steps", Readable: true},
{Name: "pauses", Readable: true},
}
// keepStates announces Kept as this module's (stdio.Keeps): main calls it before it serves.
@@ -37,16 +37,20 @@ func (m *Module) Puller(steps []StepTool, source buildstep.AskSource) *buildstep
source = buildstep.SeatAsks{Language: delivery.AskImage}
}
capacity := m.Capacity
// The capacity's own change wakes the puller (issue 1887): the runtime's watch of the config value as mesh-config
// resolved it; the settings file is read only when that watch cannot start. A test's Capacity is its own.
var watch func(string, func(stdio.ProvisionChange) error) error
if capacity == nil {
capacity = buildstep.NodeCapacity(m.Settings, buildstep.DefaultCapacities, m.Node, m.Say)
watch = buildstep.WatchConfigServed
}
ask := m.askSteps(steps)
var load func(string) ([]byte, error)
if m.ReadSteps.Get != nil {
load = func(key string) ([]byte, error) { return m.ReadSteps.Get(key) }
if m.ReadPauses.Get != nil {
load = func(key string) ([]byte, error) { return m.ReadPauses.Get(key) }
}
return &buildstep.Puller{Source: source, Language: delivery.AskImage, Node: m.Node, Put: m.Steps, Load: load,
Capacity: capacity, Logf: m.Say,
return &buildstep.Puller{Source: source, Language: delivery.AskImage, Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load,
Capacity: capacity, WatchConfig: watch, Logf: m.Say,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
if err := m.Check(ctx); err != nil {
return nil, err
@@ -29,13 +29,15 @@ func logf(format string, a ...any) { fmt.Fprintf(os.Stderr, "[mesh-build-image]
func main() {
ctx := context.Background()
m := &Module{
Node: strings.TrimSpace(os.Getenv("MESH_NODE")),
Steps: stdio.State(delivery.StepsState).Put,
ReadSteps: readOf(stdio.State(delivery.StepsState)),
Ctx: ctx,
Say: logf,
Work: WorkDir,
Store: StoreDir,
Node: strings.TrimSpace(os.Getenv("MESH_NODE")),
Steps: stdio.State(delivery.StepsState).Put,
ReadSteps: readOf(stdio.State(delivery.StepsState)),
Pauses: stdio.State(delivery.PausesState).Put,
ReadPauses: readOf(stdio.State(delivery.PausesState)),
Ctx: ctx,
Say: logf,
Work: WorkDir,
Store: StoreDir,
}
m.Floor = floorOf(m.Node)
if u, err := user.Current(); err == nil {
@@ -97,9 +97,13 @@ func TestManifestKeepsItsStatesClaimsItsBuildSeatAndDeclaresItsHealthTool(t *tes
states := []string{}
for _, s := range m.State {
states = append(states, s.Name)
// A step's key is per step and dropped after its age, so the bucket cannot fill; a pause has no age.
if (s.Name == "steps") != (s.TTLSeconds > 0) {
t.Errorf("state %s has ttl-seconds %d: only steps has an age", s.Name, s.TTLSeconds)
}
}
slices.Sort(states)
if want := []string{"keys", "steps"}; !slices.Equal(states, want) {
if want := []string{"keys", "pauses", "steps"}; !slices.Equal(states, want) {
t.Errorf("state = %v, want %v", states, want)
}
if len(m.Claims) != 1 || m.Claims[0].Name != Seat || !slices.Equal(m.Claims[0].Serves, []string{"current", "pause", "resume"}) {
@@ -24,6 +24,9 @@ type Module struct {
// Steps is the Put of the steps state; ReadSteps reads it back for the read tools.
Steps buildstep.Put
ReadSteps StateRead
// Pauses is the Put of the pauses state, which keeps the node's pause key without an age; ReadPauses reads it back.
Pauses buildstep.Put
ReadPauses StateRead
// Ctx is the process's context, which a step runs under (the tool call's is cut at 30 s, the step's is not).
Ctx context.Context
Say func(format string, a ...any)
@@ -7,9 +7,12 @@ package main
import (
"encoding/json"
"fmt"
"log"
"sort"
"strconv"
"strings"
"sync"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
@@ -38,6 +41,53 @@ func readOf(s stdio.KeptState) StateRead {
}}
}
// MaxStateReads is the most values one read tool reads from a state in a call, and stateReadParallel how many of them
// are in flight at once. Every value is a round trip from the tool through the runtime to the bus, so a call that read
// every key of a large state one after the other outlasted the call's 30 s (issue 1827: 1164 keys, 20 MB).
const (
MaxStateReads = 1000
stateReadParallel = 16
)
// readValues reads the values of keys, stateReadParallel at a time, in the order of keys (nil for a key with none). It
// stops starting reads after the first error and returns it.
func readValues(keys []string, get func(string) (json.RawMessage, error)) ([]json.RawMessage, error) {
out := make([]json.RawMessage, len(keys))
var (
wg sync.WaitGroup
mu sync.Mutex
fail error
next = make(chan int)
)
for w := 0; w < stateReadParallel && w < len(keys); w++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := range next {
raw, err := get(keys[i])
mu.Lock()
if err != nil && fail == nil {
fail = err
}
mu.Unlock()
out[i] = raw
}
}()
}
for i := range keys {
mu.Lock()
stop := fail != nil
mu.Unlock()
if stop {
break
}
next <- i
}
close(next)
wg.Wait()
return out, fail
}
// stepKey is the key of a step of this node: the id alone, or the whole key <node>.<id> as a step tool answered it.
func (m *Module) stepKey(id string) (string, error) {
id = strings.TrimSpace(id)
@@ -79,27 +129,41 @@ type StepLine struct {
Reason string `json:"reason,omitempty"`
}
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero).
func (m *Module) RecentSteps(limit int) ([]StepLine, error) {
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero). Only
// the node's own keys (<node>.) are listed and the newest max(limit*4, 100) of them read (at most MaxStateReads): a step's
// id is base-36 milliseconds and random digits, so its string order is its start order. cut says older steps were not
// read. The values still order the lines, since an ask step's id carries the ask time.
func (m *Module) RecentSteps(limit int) (lines []StepLine, cut bool, err error) {
if limit <= 0 {
limit = 10
}
if m.ReadSteps.Keys == nil || m.ReadSteps.Get == nil {
return nil, fmt.Errorf("the steps state cannot be read here")
return nil, false, fmt.Errorf("the steps state cannot be read here")
}
start := time.Now()
keys, err := m.ReadSteps.Keys()
if err != nil {
return nil, err
return nil, false, err
}
own := keys[:0:0]
for _, k := range keys {
if strings.HasPrefix(k, m.Node+".") {
if node, _, err := delivery.ParseStepKey(k); err == nil && node == m.Node {
own = append(own, k)
}
}
}
sort.Sort(sort.Reverse(sort.StringSlice(own)))
if n := min(max(limit*4, 100), MaxStateReads); len(own) > n {
own, cut = own[:n], true
}
raws, err := readValues(own, m.ReadSteps.Get)
log.Printf("steps read: %d of %d keys listed read in %v (cut %v)", len(own), len(keys), time.Since(start).Round(time.Millisecond), cut)
if err != nil {
return nil, cut, err
}
var steps []delivery.Step
for _, k := range keys {
if node, _, err := delivery.ParseStepKey(k); err != nil || node != m.Node {
continue
}
raw, err := m.ReadSteps.Get(k)
if err != nil {
return nil, err
}
for _, raw := range raws {
if len(raw) == 0 {
continue
}
@@ -121,7 +185,7 @@ func (m *Module) RecentSteps(limit int) ([]StepLine, error) {
}
out = append(out, l)
}
return out, nil
return out, cut, nil
}
// ReadTools are the module's read tools. All are read-only.
@@ -143,8 +207,13 @@ func (m *Module) ReadTools() []stdio.Tool {
Input: map[string]any{"type": "object", "properties": map[string]any{
"limit": map[string]any{"type": "integer", "description": "how many, at most; 10 when left out"}}},
Run: func(args map[string]any) (any, error) {
steps, err := m.RecentSteps(intArg(args["limit"]))
return map[string]any{"node": m.Node, "steps": steps}, err
steps, cut, err := m.RecentSteps(intArg(args["limit"]))
ans := map[string]any{"node": m.Node, "steps": steps}
if cut {
ans["cut"] = true
ans["note"] = fmt.Sprintf("this node has more steps stored than one call reads; older steps were not read (ask for a larger limit, at most %d reads)", MaxStateReads)
}
return ans, err
}},
}
}
@@ -2,7 +2,11 @@ package main
import (
"encoding/json"
"fmt"
"strings"
"sync/atomic"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
@@ -39,8 +43,67 @@ func TestTheReadToolsAnswerAStepsResult(t *testing.T) {
if err := json.Unmarshal(got.Result, &back); err != nil || back.Digest != res.Digest {
t.Errorf("result = %s, %v", got.Result, err)
}
lines, err := m.RecentSteps(0)
lines, _, err := m.RecentSteps(0)
if err != nil || len(lines) != 1 || lines[0].Tool != BuildTool {
t.Errorf("steps = %+v, %v", lines, err)
}
}
// 1827: on the live shape (4 nodes, 600 steps each; a steps state of 1164 keys / 20 MB timed out at 30 s) the steps tool
// lists, then reads only this node's keys, in parallel, at most MaxStateReads of them, and says when it cut.
func TestRecentStepsReadsOnlyTheNodesKeysInParallelAndBounded(t *testing.T) {
t0 := time.Date(2026, 10, 25, 9, 0, 0, 0, time.UTC)
for _, row := range []struct {
perNode, limit int
reads int64
steps int
cut bool
}{
{20, 0, 20, 10, false},
{600, 0, 100, 10, true},
{1500, 300, MaxStateReads, 300, true},
} {
var reads, inFlight, maxFlight atomic.Int64
stored := map[string][]byte{}
for _, n := range []string{"ace", "g14", "novox", "shanks"} {
for i := 0; i < row.perNode; i++ {
b, _ := delivery.EncodeStep(delivery.Step{ID: fmt.Sprintf("s%04d", i), Tool: "t", Node: n, State: delivery.StepPassed, Started: t0.Add(time.Duration(i) * time.Second)})
stored[n+".s"+fmt.Sprintf("%04d", i)] = b
}
}
m := &Module{Node: "shanks"}
m.ReadSteps = StateRead{
Keys: func() ([]string, error) {
var k []string
for key := range stored {
k = append(k, key)
}
return k, nil
},
Get: func(key string) (json.RawMessage, error) {
reads.Add(1)
now := inFlight.Add(1)
for o := maxFlight.Load(); now > o && !maxFlight.CompareAndSwap(o, now); o = maxFlight.Load() {
}
time.Sleep(time.Millisecond)
inFlight.Add(-1)
if !strings.HasPrefix(key, "shanks.") {
t.Errorf("read %s, a key of another node", key)
}
return stored[key], nil
},
}
start := time.Now()
lines, cut, err := m.RecentSteps(row.limit)
if err != nil || len(lines) != row.steps || cut != row.cut || reads.Load() != row.reads {
t.Errorf("%d a node: %d steps, cut %v, %d reads, err %v; want %d, %v, %d", row.perNode, len(lines), cut, reads.Load(), err, row.steps, row.cut, row.reads)
}
if row.cut && len(lines) > 0 && lines[0].ID != fmt.Sprintf("s%04d", row.perNode-1) {
t.Errorf("%d a node: newest answered %s", row.perNode, lines[0].ID)
}
if maxFlight.Load() < 2 {
t.Errorf("%d a node: values read one at a time", row.perNode)
}
t.Logf("%d a node: %d reads in %v (one after the other: %v)", row.perNode, reads.Load(), time.Since(start), time.Duration(reads.Load())*time.Millisecond)
}
}
@@ -0,0 +1 @@
These tests read other catalogue modules' files, outside this module (issue 1935): never kept.
+1 -1
View File
@@ -3,7 +3,7 @@ module meshbuildimage
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023150321-a2c813a6b468
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261025175617-518dae5ec9c8
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023150321-a2c813a6b468 h1:UcMee+IV2ii5E4dRZdQUaSaeh9IedTFYumuQbBiB0k0=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023150321-a2c813a6b468/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261025175617-518dae5ec9c8 h1:x4mvY7yiGTP5lnzxbiwyj1o24p05hAmzZP7xHg3XX2M=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261025175617-518dae5ec9c8/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
+1 -1
View File
@@ -9,7 +9,7 @@
],
"state": [
"keys",
"steps"
{"name": "steps", "ttl-seconds": 172800}, "pauses"
],
"claims": [
{
@@ -14,6 +14,7 @@ var Kept = []announce.State{
{Name: "toolchains", Readable: true},
{Name: "keys", Readable: true},
{Name: "steps", Readable: true},
{Name: "pauses", Readable: true},
}
// keepStates announces Kept as this module's (stdio.Keeps): main calls it before it serves.
@@ -45,16 +45,20 @@ func (m *Module) Puller(steps []StepTool, source buildstep.AskSource) *buildstep
source = buildstep.SeatAsks{Language: delivery.AskNode}
}
capacity := m.Capacity
// The capacity's own change wakes the puller (issue 1887): the runtime's watch of the config value as mesh-config
// resolved it; the settings file is read only when that watch cannot start. A test's Capacity is its own.
var watch func(string, func(stdio.ProvisionChange) error) error
if capacity == nil {
capacity = buildstep.NodeCapacity(m.Settings, buildstep.DefaultCapacities, m.Node, m.Say)
watch = buildstep.WatchConfigServed
}
ask := m.askSteps(steps)
var load func(string) ([]byte, error)
if m.ReadSteps.Get != nil {
load = func(key string) ([]byte, error) { return m.ReadSteps.Get(key) }
if m.ReadPauses.Get != nil {
load = func(key string) ([]byte, error) { return m.ReadPauses.Get(key) }
}
return &buildstep.Puller{Source: source, Language: delivery.AskNode, Node: m.Node, Put: m.Steps, Load: load,
Capacity: capacity, Logf: m.Say,
return &buildstep.Puller{Source: source, Language: delivery.AskNode, Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load,
Capacity: capacity, WatchConfig: watch, Logf: m.Say,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
if err := m.Prepare(ctx); err != nil {
return nil, err
@@ -46,6 +46,8 @@ func main() {
Emit: func(t string, b any) error { return stdio.Emit(t, b) },
Steps: stdio.State(delivery.StepsState).Put,
ReadSteps: readOf(stdio.State(delivery.StepsState)),
Pauses: stdio.State(delivery.PausesState).Put,
ReadPauses: readOf(stdio.State(delivery.PausesState)),
ReadToolchainsState: readOf(stdio.State(delivery.ToolchainsState)),
Work: WorkDir,
Run: forgekey.ExecRunnerClean(0),
@@ -94,9 +94,13 @@ func TestManifestKeepsItsStatesClaimsItsBuildSeatAndDeclaresItsHealthTool(t *tes
states := []string{}
for _, s := range m.State {
states = append(states, s.Name)
// A step's key is per step and dropped after its age, so the bucket cannot fill; a pause has no age.
if (s.Name == "steps") != (s.TTLSeconds > 0) {
t.Errorf("state %s has ttl-seconds %d: only steps has an age", s.Name, s.TTLSeconds)
}
}
slices.Sort(states)
if want := []string{"keys", "steps", "toolchains"}; !slices.Equal(states, want) {
if want := []string{"keys", "pauses", "steps", "toolchains"}; !slices.Equal(states, want) {
t.Errorf("state = %v, want %v", states, want)
}
if len(m.Claims) != 1 || m.Claims[0].Name != Seat || !slices.Equal(m.Claims[0].Serves, []string{"current", "pause", "resume"}) {
@@ -43,6 +43,9 @@ type Module struct {
Toolchains, Steps buildstep.Put
// ReadSteps and ReadToolchainsState read those two states back for the read tools (issue 1562).
ReadSteps, ReadToolchainsState StateRead
// Pauses is the Put of the pauses state, which keeps the node's pause key without an age; ReadPauses reads it back.
Pauses buildstep.Put
ReadPauses StateRead
// Emit tells the bus an event (stdio.Emit); nil emits nothing. The guard emits toolchain.reported after each write of
// the toolchains key, so the indexer, which holds no timer, learns that the bucket exists (issue 1444).
Emit func(eventType string, body any) error
@@ -7,9 +7,12 @@ package main
import (
"encoding/json"
"fmt"
"log"
"sort"
"strconv"
"strings"
"sync"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
@@ -39,6 +42,53 @@ func readOf(s stdio.KeptState) StateRead {
}}
}
// MaxStateReads is the most values one read tool reads from a state in a call, and stateReadParallel how many of them
// are in flight at once. Every value is a round trip from the tool through the runtime to the bus, so a call that read
// every key of a large state one after the other outlasted the call's 30 s (issue 1827: 1164 keys, 20 MB).
const (
MaxStateReads = 1000
stateReadParallel = 16
)
// readValues reads the values of keys, stateReadParallel at a time, in the order of keys (nil for a key with none). It
// stops starting reads after the first error and returns it.
func readValues(keys []string, get func(string) (json.RawMessage, error)) ([]json.RawMessage, error) {
out := make([]json.RawMessage, len(keys))
var (
wg sync.WaitGroup
mu sync.Mutex
fail error
next = make(chan int)
)
for w := 0; w < stateReadParallel && w < len(keys); w++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := range next {
raw, err := get(keys[i])
mu.Lock()
if err != nil && fail == nil {
fail = err
}
mu.Unlock()
out[i] = raw
}
}()
}
for i := range keys {
mu.Lock()
stop := fail != nil
mu.Unlock()
if stop {
break
}
next <- i
}
close(next)
wg.Wait()
return out, fail
}
// stepKey is the key of a step of this node: the id alone, or the whole key <node>.<id> as a step tool answered it.
func (m *Module) stepKey(id string) (string, error) {
id = strings.TrimSpace(id)
@@ -80,27 +130,41 @@ type StepLine struct {
Reason string `json:"reason,omitempty"`
}
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero).
func (m *Module) RecentSteps(limit int) ([]StepLine, error) {
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero). Only
// the node's own keys (<node>.) are listed and the newest max(limit*4, 100) of them read (at most MaxStateReads): a step's
// id is base-36 milliseconds and random digits, so its string order is its start order. cut says older steps were not
// read. The values still order the lines, since an ask step's id carries the ask time.
func (m *Module) RecentSteps(limit int) (lines []StepLine, cut bool, err error) {
if limit <= 0 {
limit = 10
}
if m.ReadSteps.Keys == nil || m.ReadSteps.Get == nil {
return nil, fmt.Errorf("the steps state cannot be read here")
return nil, false, fmt.Errorf("the steps state cannot be read here")
}
start := time.Now()
keys, err := m.ReadSteps.Keys()
if err != nil {
return nil, err
return nil, false, err
}
own := keys[:0:0]
for _, k := range keys {
if strings.HasPrefix(k, m.Node+".") {
if node, _, err := delivery.ParseStepKey(k); err == nil && node == m.Node {
own = append(own, k)
}
}
}
sort.Sort(sort.Reverse(sort.StringSlice(own)))
if n := min(max(limit*4, 100), MaxStateReads); len(own) > n {
own, cut = own[:n], true
}
raws, err := readValues(own, m.ReadSteps.Get)
log.Printf("steps read: %d of %d keys listed read in %v (cut %v)", len(own), len(keys), time.Since(start).Round(time.Millisecond), cut)
if err != nil {
return nil, cut, err
}
var steps []delivery.Step
for _, k := range keys {
if node, _, err := delivery.ParseStepKey(k); err != nil || node != m.Node {
continue
}
raw, err := m.ReadSteps.Get(k)
if err != nil {
return nil, err
}
for _, raw := range raws {
if len(raw) == 0 {
continue
}
@@ -122,7 +186,7 @@ func (m *Module) RecentSteps(limit int) ([]StepLine, error) {
}
out = append(out, l)
}
return out, nil
return out, cut, nil
}
// ReadToolchains is every key of the toolchains state (<node>.<language>) with its stored value.
@@ -130,24 +194,30 @@ func (m *Module) ReadToolchains() (map[string]delivery.Toolchain, error) {
if m.ReadToolchainsState.Keys == nil || m.ReadToolchainsState.Get == nil {
return nil, fmt.Errorf("the toolchains state cannot be read here")
}
start := time.Now()
keys, err := m.ReadToolchainsState.Keys()
if err != nil {
return nil, err
}
cut := len(keys) > MaxStateReads
if cut {
keys = keys[:MaxStateReads] // a node has a key per language: far fewer in practice
}
raws, err := readValues(keys, m.ReadToolchainsState.Get)
log.Printf("toolchains read: %d keys read in %v (cut %v: keys past %d were not read)", len(keys), time.Since(start).Round(time.Millisecond), cut, MaxStateReads)
if err != nil {
return nil, err
}
out := map[string]delivery.Toolchain{}
for _, k := range keys {
raw, err := m.ReadToolchainsState.Get(k)
if err != nil {
return nil, err
}
for i, raw := range raws {
if len(raw) == 0 {
continue
}
t, err := delivery.DecodeToolchain(raw)
if err != nil {
return nil, fmt.Errorf("%s: %w", k, err)
return nil, fmt.Errorf("%s: %w", keys[i], err)
}
out[k] = t
out[keys[i]] = t
}
return out, nil
}
@@ -171,8 +241,13 @@ func (m *Module) ReadTools() []stdio.Tool {
Input: map[string]any{"type": "object", "properties": map[string]any{
"limit": map[string]any{"type": "integer", "description": "how many, at most; 10 when left out"}}},
Run: func(args map[string]any) (any, error) {
steps, err := m.RecentSteps(intArg(args["limit"]))
return map[string]any{"node": m.Node, "steps": steps}, err
steps, cut, err := m.RecentSteps(intArg(args["limit"]))
ans := map[string]any{"node": m.Node, "steps": steps}
if cut {
ans["cut"] = true
ans["note"] = fmt.Sprintf("this node has more steps stored than one call reads; older steps were not read (ask for a larger limit, at most %d reads)", MaxStateReads)
}
return ans, err
}},
{Name: ToolchainsReadTool,
Description: "What the module's toolchains state holds: each key <node>.<language> with the version installed, the " +
@@ -3,8 +3,10 @@ package main
import (
"context"
"encoding/json"
"fmt"
"slices"
"strings"
"sync/atomic"
"testing"
"time"
@@ -145,3 +147,62 @@ func TestReadToolsAreInTheManifestAndNeedNoToolchain(t *testing.T) {
t.Errorf("the read refused: %v", err)
}
}
// 1827: on the live shape (4 nodes, 600 steps each; a steps state of 1164 keys / 20 MB timed out at 30 s) the steps tool
// lists, then reads only this node's keys, in parallel, at most MaxStateReads of them, and says when it cut.
func TestRecentStepsReadsOnlyTheNodesKeysInParallelAndBounded(t *testing.T) {
t0 := time.Date(2026, 10, 25, 9, 0, 0, 0, time.UTC)
for _, row := range []struct {
perNode, limit int
reads int64
steps int
cut bool
}{
{20, 0, 20, 10, false},
{600, 0, 100, 10, true},
{1500, 300, MaxStateReads, 300, true},
} {
var reads, inFlight, maxFlight atomic.Int64
stored := map[string][]byte{}
for _, n := range []string{"ace", "g14", "novox", "shanks"} {
for i := 0; i < row.perNode; i++ {
b, _ := delivery.EncodeStep(delivery.Step{ID: fmt.Sprintf("s%04d", i), Tool: "t", Node: n, State: delivery.StepPassed, Started: t0.Add(time.Duration(i) * time.Second)})
stored[n+".s"+fmt.Sprintf("%04d", i)] = b
}
}
m := &Module{Node: "shanks"}
m.ReadSteps = StateRead{
Keys: func() ([]string, error) {
var k []string
for key := range stored {
k = append(k, key)
}
return k, nil
},
Get: func(key string) (json.RawMessage, error) {
reads.Add(1)
now := inFlight.Add(1)
for o := maxFlight.Load(); now > o && !maxFlight.CompareAndSwap(o, now); o = maxFlight.Load() {
}
time.Sleep(time.Millisecond)
inFlight.Add(-1)
if !strings.HasPrefix(key, "shanks.") {
t.Errorf("read %s, a key of another node", key)
}
return stored[key], nil
},
}
start := time.Now()
lines, cut, err := m.RecentSteps(row.limit)
if err != nil || len(lines) != row.steps || cut != row.cut || reads.Load() != row.reads {
t.Errorf("%d a node: %d steps, cut %v, %d reads, err %v; want %d, %v, %d", row.perNode, len(lines), cut, reads.Load(), err, row.steps, row.cut, row.reads)
}
if row.cut && len(lines) > 0 && lines[0].ID != fmt.Sprintf("s%04d", row.perNode-1) {
t.Errorf("%d a node: newest answered %s", row.perNode, lines[0].ID)
}
if maxFlight.Load() < 2 {
t.Errorf("%d a node: values read one at a time", row.perNode)
}
t.Logf("%d a node: %d reads in %v (one after the other: %v)", row.perNode, reads.Load(), time.Since(start), time.Duration(reads.Load())*time.Millisecond)
}
}

Some files were not shown because too many files have changed in this diff Show More