Compare commits

...
Author SHA1 Message Date
mesh-tools 18136ce063 Merge pull request 'The builder re-reads released versions at start and waits the pause on a refused tag; one parse of every pipeline key form (H-A-484, H-A-485)' (#334) from fix/tags-and-again-tiers into main 2026-11-22 18:06:50 +00:00
jschoubben a5495bb15e Read every pipeline key form by one parse; the assigner moves an again pipeline's tiers (H-A-485)
The assigner took a tier progress for `pipeline.<branch>.<revision>` only (two dots), so
the progress of a pipeline started again at the head, `<branch>.<revision>.again.<n>`,
was never moved and read as an empty end: its later tiers were never wanted. One parse
of the runner's seat's key forms (mesh-delivery-sdk pipelinekey, over the contract's key
parsers) now reads a pipeline's key for the assigner's tier progress and the runner's
mover keys; a pipeline-ended whose key is of no form is refused loudly and nothing of
it is kept.

Builds: design 47 H2 H-A-485
2026-11-22 18:54:40 +01:00
jschoubben 5d3164b5b7 Builder: take a released version the catalog holds at the start; a refused tag waits the pause (H-A-484)
A version-released whose tag the seat git's holder refused was handed back at once,
spent its five deliveries in seconds and was dropped by the bus: its reservation stood
and its publish item asked again every 30 s for ever. At the start the builder now holds
every built, unreleased reservation against the catalog's verb versions and takes a
released one in as its version-released would (tag, release, bound and item ended,
findings cleared); a refused tag is handed back with the publish pause. The tag check
itself is the forge module's, already accepting a nested package path on the catalog's
trunk.

Builds: design 47 H2 H-A-484, H-A-484 addendum
2026-11-22 18:49:52 +01:00
mesh-tools 96d8136e2b Merge pull request 'The builder's publishes have one consumer per holder; publish_first retired (H-A-483)' (#333) from fix/publishes-one-consumer into main 2026-11-22 17:39:50 +00:00
jschoubben c45f1621fb Seat mesh-builds: publish-queue lists in stream order (H-A-483)
The verb's description and its answer's class and list say what the
builder now does: one consumer per holder, items listed in stream order,
no class pulled first. Description only, regenerated with seatgen; the
seat's format stays, as at B4's description changes of this seat.

Builds: design 47 H2 H-A-483 third addendum, §793
2026-11-22 18:30:30 +01:00
jschoubben fd2cb13961 Publish queue: retire builder_publish_first, list in stream order (H-A-483)
With one consumer per holder an item has no class to move ahead of, so
builder_publish_first, its mode and its act go. The verb publish-queue
lists the items in stream order, the order the one consumer gives them,
and its description says so. A holder is a module name, so one ending in
-live or -again (an earlier release's class durable) is refused at the
learn, said, instead of having another holder's durable deleted.

Builds: design 47 H2 H-A-483 third addendum, §793
2026-11-22 18:28:55 +01:00
jschoubben 915a54f136 Publish queue: one pull consumer per holder (H-A-483)
The two class consumers (live, again) could not order live before again
under every scheduling: the idle-dispatch row failed on the build node
after four review rounds. One consumer per learnt holder (publishes-<holder>)
gives items in order; a failed send is handed back with its pause, so an
item put meanwhile goes first by construction. The builder retires the
holder's class durables of the earlier release before making the one,
since a work queue refuses overlapping filters; their unacknowledged items
reach the one consumer. The gap read, the held again items and the live
in-flight check go.

Builds: design 47 H2 H-A-483, its addendum and second addendum, §793
2026-11-22 18:20:06 +01:00
mesh-tools 1c0309df23 Merge pull request 'The assigner's blocked-keeping test waits for the keeping (H-A-476)' (#332) from fix/assigner-test-race into main 2026-11-22 15:34:10 +00:00
mesh-tools 7dd2891f32 Merge pull request 'A failed mover is started again at the head; a reopening pins its revision again (H-A-479)' (#331) from fix/runner-start-modules into main 2026-11-22 15:34:08 +00:00
mesh-tools b7e364e47b Merge pull request 'grants-for answers from what it kept; the runner's widens bound 30 s (H-A-478)' (#330) from fix/grants-for-kept into main 2026-11-22 15:34:04 +00:00
jschoubben dca2545eb7 H-A-479 third addendum: a reopening never builds beside an open pipeline
A rerun that would reopen an ended pipeline while another open delivering
pipeline carries one of its movers at the same revision is refused with
"its pipeline <key> is running", nothing asked, so one entry is never built
side by side. Tests now cover the two paths review round 2 found untested:
the indexer keeps a revision's gathering while another pipeline's
reservation is left and removes it with the last, and a test step's rerun
reopening an ended pipeline says pipeline-started before its run is started
again.

Builds: design 47 H2 H-A-479 third addendum, H-A-479 second addendum, H-A-479 addendum (2); review round 2
2026-11-22 16:29:24 +01:00
jschoubben bd5852b817 H-A-479 second addendum: own movers withdrawn, no side-by-side start, unbounded n
A pipeline ending not passed withdrew every reservation of its revision, so
an again pipeline's failure could withdraw the head's take-in pipeline's
reservations and the other way round. The indexer now withdraws only the
reservations of the ended pipeline's own movers, read from the movers its
pipeline-ended carries, and withdraws the revision's gathering only when
none is left. A start with modules is refused for a module whose head entry
a pipeline still runs ("its pipeline <key> is running"), and takes the next
n without a bound. A rerun reopening an ended pipeline says its
pipeline-started before the step is asked again, so the revision is pinned
before anything reads it. Tests now cover the again form through the
branch's key forms and a release written into a passed again pipeline.

Builds: design 47 H2 H-A-479 second addendum, H-A-479 addendum (2); review round 1
2026-11-22 16:21:31 +01:00
jschoubben 1bfd54425b mesh-grants: kept grants are keyed by revision and inputs, bound tested
A revision's kept grants carry the digest of the inputs they were
composed with; an ask whose inputs differ composes again, so grants-for
never lists a change the candidate did not cause. TestGrantsForKeepsWhatItMay
now proves that a revision dropped beyond the bound has no grants kept
either (review round 1); the Logf comment says the tools program's
journal too.

Builds: design 47 H2 H-A-478 second addendum
2026-11-22 16:13:08 +01:00
jschoubben c6c86801ca H-A-479 (2) runner: a rerun reopening an ended pipeline pins its revision again
The indexer pins a pipeline's revision from pipeline-started to
pipeline-ended, so a rerun of a step in an ended pipeline (main.258's
composition at 14:41, after its end at 14:15) asked a step against a
revision the catalog had already removed. A rerun of an ended delivering
pipeline now reads its revision fresh through the catalog seat's verb
revision first and is refused by name, nothing asked, when it is gone
("its revision <r> is gone; start the mover again at the head"); once
reopened it says its pipeline-started again, so the indexer pins the
revision until its next end.

Builds: design 47 H2 H-A-479, H-A-479 addendum (2)
2026-11-22 16:08:57 +01:00
jschoubben 33feb148ce H-A-479 (1) runner: start with modules starts a failed mover again at the head
A module whose pipeline failed at an unchanged entry was never carried again
until its fingerprint moved, since start refused the head whose pipeline
existed. The start verb now takes modules: the named modules whose newest
entry has no passed pipeline are started at the head's revision, at the
take-in's tiers, keyed <branch>.<revision>.again.<n> (n from 1, written on
absence); a module whose newest entry passed, or that the head holds no
entry of, is refused by name, nothing started. Exactly one of module or
modules is given. The new key form is read by every key-form list (the
sweep's, a release's, the mover keys'). The runner seat's form is 8; the
contract is regenerated.

Builds: design 47 H2 H-A-479, H-A-479 addendum (1)
2026-11-22 16:05:27 +01:00
jschoubben f44442097c mesh-delivery-pipeline: widens-read-bound defaults to 30 s
A cold grants-for (one composing a revision's grants the grants part did
not keep yet) took 3 to 4 s against the 5 s bound and was cut three times
on 2026-11-22; the bound becomes 30 s in the manifest and the code.

Builds: design 47 H2 H-A-478
2026-11-22 16:01:46 +01:00
jschoubben 1f765ef17b mesh-grants: grants-for composes each revision's grants once
grants-for composed every module twice per ask (3 to 4 s against the
runner's 5 s bound, cut three times on 2026-11-22). The tools program
that serves it now keeps each kept revision's composed grants beside the
revision (the newest's and the candidate's), bounded by
grants-for-revisions-per-branch, so a second ask for a kept revision
composes nothing; it reads its inputs at each ask and journals each
stage's time on its standard error.

Builds: design 47 H2 H-A-478, H-A-478 addendum
2026-11-22 16:01:39 +01:00
jschoubben 203454b1a9 Assigner test: read the keeping's need only once the keeping took it
TestAPassWritesWantsWhileAKeepingIsBlocked read the graph's need right after
two passes, while the keeping takes it on its own goroutine; on a loaded build
node it read too early (pipeline main.251). The blocked object bucket now says
when a keeping meets its gate, and the test waits on that, bounded by the
level bound, as it does on the pass and the wake.

Builds: design 47 H2 H-A-476
2026-11-22 14:59:47 +01:00
mesh-tools b691428ffc Merge pull request 'Builder: the publish dispatch sends live first at one processor; judge test row; done-key test bound (H-A-465, H-A-469, H-A-473)' (#329) from fix/main-tests into main 2026-11-22 13:44:24 +00:00
mesh-tools a41a7c2862 Merge pull request 'Orphaned durables: removable by the bus module, left out of the behind check, not composed again (H-A-472)' (#328) from fix/orphan-durables into main 2026-11-22 13:42:31 +00:00
mesh-tools 314a996ed9 Merge pull request 'The delivery view shows pipelines live (issue 1983, H-A-467)' (#327) from feat/pipelines-view into main 2026-11-22 13:41:03 +00:00
jschoubben 7e103f3221 Builder: with again items held, fetch live once more before the gap read
A live item put after the turn's live fetch and before the again items were held
was not yet delivered at the gap read, so no gap showed and the later again item
went first (review round 3 on c37a7317, publishitem.go:792-809). With again items
held the dispatch now fetches live without waiting once more: what it returns runs
and the again items stay held; only an empty fetch leads to the gap read. The test's
idle-dispatch case is a subtest of its own, so it runs alone.

Builds: design 47 H2 §793, H-A-473 (third addendum)
2026-11-22 14:40:17 +01:00
jschoubben c37a7317b7 Builder: every again fetch and gap read follows a live fetch
On the turn after an again item ran, the dispatch fetched again and read the live gap
without fetching live first, so a live item the stream held but had not yet given to
the waiting live pull showed no gap and a later again item went first (review round 2
on b77b70a6, publishitem.go:791). The live consumer is now fetched without waiting on
every turn no live item is held. The test row adds that case and bounds its rounds by
the level's set-up share.

Builds: design 47 H2 §793, H-A-473 (third addendum)
2026-11-22 14:35:41 +01:00
jschoubben b2c9e60ac3 Grants: the bound of a current twin is the setting twin-current-hours
Review round 1: the day after which a twin is no longer current was a
constant here and again in the bus module. It is now the setting
twin-current-hours of the manifest (24 by default), written into a file
the part reads at its start, the field name the bus module's manifest
names too, so the operator sets it once per module alike; the
durable-orphaned condition names the bound. hasOwnProcess no longer sits
under ownServesOf's doc comment.

Builds: design 47 H2 H-A-472 (2) and its third addendum (2); review round 1 findings (2), (3)
2026-11-22 14:34:06 +01:00
jschoubben b77b70a66c Builder: an again item waits for a live one in flight to the idle dispatch's waiting pull
The idle dispatch keeps a waiting pull on each consumer, the bus's only wake for an
item put or delivered again. A live item the waiting pull was given but the dispatch
had not yet received let an again item go first (review finding on 1f33a4c0,
publishitem.go:700, 761, 797). An again item taken while the live pull waits now runs
only once the live consumer's delivered sequence is no further than the live items
received; a gap holds it until the live item arrives and runs.

Builds: design 47 H2 §793, H-A-473 (third addendum)
2026-11-22 14:30:22 +01:00
jschoubben 7f5f175b05 Builder whole test: done builds under the level's set-up bound; the hang at its own
The test ran the builder at a 1 s step bound throughout, so a loaded build
node could fail a build that must end done. Those builds now run under
testbus.SetupBound; the hung step's part starts the builder again at 1 s,
and the build again after it under the set-up bound.

Builds: design 47 H2 §793; H-A-473 (addendum)
2026-11-22 14:16:37 +01:00
jschoubben 1f33a4c04f Builder: the publish dispatch pulls the live consumer itself, then the again
The dispatch read the live consumer's pending count against items the
pull's callback had not yet handed it, so at one processor an again item
was sent before a live one already delivered. It now fetches the live
consumer without waiting and pulls the again consumer only when that
returns nothing; an item is received when its fetch returns it.

Builds: design 47 H2 §793; H-A-473
2026-11-22 14:14:15 +01:00
jschoubben 49c656a4ce Grants: compose a module's program consumer only where it names a program
The pass made <node>_<module> beside <node>_<module>_tools for every
consuming module, so a module whose events only the tool runner reads
got a durable nobody pulls, which piled up and came back after each
removal. The program's consumer is now composed only for a module with
an account of its own and a process resource with no bus of its own;
the tool runner's twin stays for every consuming module.

Builds: design 47 H2 H-A-472 addendum and second addendum, G A-397
2026-11-22 14:12:45 +01:00
jschoubben 9005a2ee8e Grants: the behind check leaves out an orphaned durable and names it apart
About 110 durables left by modules that moved into or out of the tool
runner pile up behind their stream's head and raised 'Messages pile up
for a listener' for no real backlog. A durable with no pulls waiting
whose twin on the stream (<node>_<module> and <node>_<module>_tools) has
pulls waiting and delivered within the day is now left out of that check
and named once in mesh-grants.durable-orphaned, cleared when gone.

Builds: design 47 H2 H-A-472 (2), G A-340
2026-11-22 14:03:55 +01:00
jschoubben b914974b1d Judge test: the unknown top-level word is one no SDK knows
The row proving that the judgement's strict read refuses an unknown
top-level word used class, which the H2 SDK knows, so it failed at main.
It now uses a word no SDK knows and proves the same refusal.

Builds: design 47 H2 H-A-465, H-A-469
2026-11-22 13:57:22 +01:00
jschoubben d151033dae View: the pipelines heard kept for the pages is the setting pipelines-heard
The view kept the last change of at most 1024 pipelines for its pages, a
cap written into the code. A cap is a setting with a default (principle
11): the manifest gives pipelines-heard, 1024 by default, written into a
file of the state directory the process reads at its start. The cap is
kept, not removed: without it the pipelines heard grow without end over
the view's life.

Issue 1983, review round 1 finding 3.
Builds: design 47 H2 H-A-467 (1)
2026-11-22 11:45:32 +01:00
jschoubben 3c742145d7 View: the pipelines page takes a limit, the newest 20 by default
The design gives list a limit with a default of 20; the view asked 20
always and neither the web page nor the terminal view could ask another.
The web list takes a limit query parameter and the terminal view a -limit
flag; a followed branch's list is read at the default, any other limit is
read at the asking.

Issue 1983, review round 1 finding 2.
Builds: design 47 H2 H-A-467 (1)
2026-11-22 11:42:27 +01:00
jschoubben d557c0c2af View: a pipelines list not read is a page shown unreadable, not a refusal
The socket answered a refusal and a list it could not read with the same
field, so the terminal view's -follow ended at a list not read yet (the
runner not answering at the view's start) instead of showing it unreadable
and reading it again at the runner's next event. A refusal now has its own
field; a list's page is handed on whatever it holds. The unit tests' waits
for a change are bounded, so a change never said fails instead of hanging.

Issue 1983, review round 1 finding 1.
Builds: design 47 H2 H-A-467 (1), (2)
2026-11-22 11:41:25 +01:00
jschoubben c11c90348e H-A-467 view: a pipelines page in the web view and the terminal view
The operator follows the pipelines in the delivery view: the newest
pipelines of a branch (main by default, the newest 20) with their steps
counted per outcome, one pipeline's mover rows and one mover's steps, read
through the seat mesh-pipeline-runner's list and show and read again on the
runner's events pipeline-started, step-ended, pipeline-ended and re-pinned,
heard on the module's one pull; the web pages reload on a stream of those
changes and the terminal view's -follow prints again, never on a timer. Each
delivered source revision links to the newest pipeline of its source. The
view keeps no state of its own for it.

Issue 1983.
Builds: design 47 H2 H-A-467 (1), (2), (3), H-A-376
2026-11-22 11:26:08 +01:00
jschoubben 1b177d5912 H-A-467 runner: list's row carries source, revision and steps per outcome
The delivery view's pipelines page reads the newest pipelines of a branch in
one read of list's short form, so each row now carries the pipeline's source
revision, its catalog graph revision and its steps counted per outcome. The
runner seat's form is 7; the contract is regenerated; the full form stays.

Issue 1983.
Builds: design 47 H2 H-A-467 (addendum), H-A-376
2026-11-22 11:12:34 +01:00
78 changed files with 3534 additions and 658 deletions
@@ -19,6 +19,7 @@ import (
mpr "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/testbus"
"mesh-delivery-assigner/internal/assign"
"mesh-delivery-assigner/internal/assigngraph"
@@ -170,15 +171,21 @@ func TestTheWantsAreSaidWantedFromTheKeptAssignmentGraphRevision(t *testing.T) {
}
// blockedObjects is an object bucket whose listing waits while a gate stands: a keeping that does not end until it is opened.
// A listing that meets the gate closes entered: the keeping took its need and runs it (H-A-476).
type blockedObjects struct {
memObjects
mu sync.Mutex
gate chan struct{}
mu sync.Mutex
gate chan struct{}
entered chan struct{}
}
func (b *blockedObjects) Names(ctx context.Context) ([]string, error) {
b.mu.Lock()
gate := b.gate
gate, entered := b.gate, b.entered
if gate != nil && entered != nil {
close(entered)
b.entered = nil
}
b.mu.Unlock()
if gate != nil {
<-gate
@@ -204,16 +211,33 @@ func (b *blockedObjects) Get(ctx context.Context, name string) ([]byte, error) {
func (b *blockedObjects) block() func() {
gate := make(chan struct{})
b.mu.Lock()
b.gate = gate
b.gate, b.entered = gate, make(chan struct{})
b.mu.Unlock()
return func() {
b.mu.Lock()
b.gate = nil
b.gate, b.entered = nil, nil
b.mu.Unlock()
close(gate)
}
}
// awaitBlocked returns once a keeping meets the standing gate, so what the keeping took is read after it took it, and fails the
// test when none did within bound (H-A-476: a test waits on the signal of the work it reads after, never on an order it assumes).
func (b *blockedObjects) awaitBlocked(t *testing.T, bound time.Duration, what string) {
t.Helper()
b.mu.Lock()
entered := b.entered
b.mu.Unlock()
if entered == nil {
return // met already
}
select {
case <-entered:
case <-time.After(bound):
t.Fatalf("no keeping met the gate within %s: %s", bound, what)
}
}
// Design 47 H-A-259, H-A-261: the keeping of the assignment graph runs off the pass loop. At the start, while the first keeping is
// blocked, a pass returns and writes no want; the keeping's end wakes a pass, which writes the passed pipeline's want. Later, while a
// keeping of new inputs is blocked, a pass writes the wants from the revision it holds, and a pass of the same inputs hands nothing
@@ -222,6 +246,7 @@ func TestAPassWritesWantsWhileAKeepingIsBlocked(t *testing.T) {
fp := mcc.SourcePrefix2 + strings.Repeat("a", 64)
wantsKV := &memKV{values: map[string][]byte{}}
p := newPasser(kvSet{buckets.Wants(): wantsKV, buckets.Asks(): &memKV{values: map[string][]byte{}}}, time.Now())
bound := testbus.SetupBound(t) // every wait on a goroutine's work, never a value in the test (H-A-476)
var mu sync.Mutex
var journal strings.Builder
p.logf = func(f string, a ...any) { mu.Lock(); fmt.Fprintf(&journal, f+"\n", a...); mu.Unlock() }
@@ -252,7 +277,7 @@ func TestAPassWritesWantsWhileAKeepingIsBlocked(t *testing.T) {
go func() { p.pass(context.Background()); close(done) }()
select {
case <-done:
case <-time.After(10 * time.Second):
case <-time.After(bound):
t.Fatal("the pass waits on the keeping of the assignment graph")
}
}
@@ -281,7 +306,7 @@ func TestAPassWritesWantsWhileAKeepingIsBlocked(t *testing.T) {
open()
select {
case <-woke:
case <-time.After(10 * time.Second):
case <-time.After(bound):
t.Fatal("the first keeping's end wakes no pass")
}
pass()
@@ -292,7 +317,8 @@ func TestAPassWritesWantsWhileAKeepingIsBlocked(t *testing.T) {
open = objects.block() // a keeping of new inputs is blocked
placeOn(t, p.w, "g14", "web")
pass()
pass() // the same inputs again
pass() // the same inputs again
objects.awaitBlocked(t, bound, "the keeping of new inputs never took its need") // H-A-476: read the need once it was taken
if w, err := wantOf("g14"); err != nil || w.Fp != fp || w.Version != "0.1.2" {
t.Errorf("the want of web on g14 = %+v (%v), want the passed version while a keeping is blocked", w, err)
}
@@ -17,6 +17,7 @@ import (
mprc "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshpipelinerunnerclient"
mpcat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
mpr "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/pipelinekey"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/keys"
@@ -166,6 +167,12 @@ func (e *pipelineEnds) take(p mpr.PipelineEnded) (bool, error) {
if p.Outcome != mpr.PipelineEndedOutcomePassed || p.Run != "" {
return false, nil
}
// The pipeline's key is read by the one parse of the runner's seat's key forms (design 47 H2 H-A-485): a key it does not
// know is refused loudly, and nothing of the pipeline is kept, never an empty tier progress.
if _, err := pipelinekey.Parse(p.Pipeline); err != nil {
e.logf("the pipeline-ended of %s is refused, and none of its movers' versions is kept: %v", p.Pipeline, err)
return false, nil
}
changed := false
var kept []tierMover
versioned := map[string]bool{}
@@ -96,6 +96,17 @@ func TestThePassedVersionIsKept(t *testing.T) {
ended("main", 5, pass, mover("web", "fp1", "0.1.2", 1, ok))}, &End{Pipeline: "main.6", Source: 6, Fp: "fp2", Version: "0.1.3", Rank: 1}},
{"a newer version replaces", []mpr.PipelineEnded{ended("main", 5, pass, mover("web", "fp1", "0.1.2", 1, ok)),
ended("main", 6, pass, mover("web", "fp2", "0.1.10", 1, ok))}, &End{Pipeline: "main.6", Source: 6, Fp: "fp2", Version: "0.1.10", Rank: 1}},
// H-A-485: an again pipeline's end is kept as a take-in's; a key of no form the runner's seat defines is refused, nothing kept.
{"an again pipeline keeps its version", []mpr.PipelineEnded{func() mpr.PipelineEnded {
p := ended("main", 5, pass, mover("web", "fp1", "0.1.2", 2, ok))
p.Pipeline = "main.5.again.2"
return p
}()}, &End{Pipeline: "main.5.again.2", Source: 5, Fp: "fp1", Version: "0.1.2", Rank: 2}},
{"a key of no form is refused", []mpr.PipelineEnded{func() mpr.PipelineEnded {
p := ended("main", 5, pass, mover("web", "fp1", "0.1.2", 2, ok))
p.Pipeline = "main.5.later.2"
return p
}()}, nil},
{"a run on request keeps nothing", []mpr.PipelineEnded{func() mpr.PipelineEnded {
p := ended("main", 5, pass, mover("web", "fp1", "0.1.2", 1, ok))
p.Run = "r1"
@@ -109,6 +120,9 @@ func TestThePassedVersionIsKept(t *testing.T) {
}
}
got, kept := e.endOf("web", "main")
if ks, _ := e.st.keys(); c.name == "a key of no form is refused" && len(ks) != 0 {
t.Errorf("%s: the state written holds %v, want nothing (no empty tier progress)", c.name, ks)
}
switch {
case c.want == nil && kept:
t.Errorf("%s: kept %+v", c.name, got)
@@ -6,12 +6,14 @@ import (
"fmt"
"slices"
"sort"
"strconv"
"strings"
"time"
mcc "git.novox.be/novox/mesh-contracts/health-and-repair/go/clients/meshconditionsclient"
mcseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/pipelinekey"
"git.novox.be/novox/mesh-sdk/go/keys"
"git.novox.be/novox/mesh-sdk/go/views"
@@ -58,10 +60,16 @@ type tierProgress struct {
// tierKey is the key of pipeline's tier progress; pipeline is the runner's key `<branch>.<source>`, its branch escaped.
func tierKey(pipeline string) string { return tierPrefix + pipeline }
// isTierKey says whether a key of the state written is a pipeline's tier progress: three parts, the first `pipeline` (a module's
// end has two).
// isTierKey says whether a key of the state written is a pipeline's tier progress: `pipeline.` and a pipeline's key of any form
// the runner's seat defines, read by the one parse (design 47 H2 H-A-485), so an again pipeline's progress is one as a take-in's
// is (a module's end, `<module>.<branch>`, is none).
func isTierKey(key string) bool {
return strings.HasPrefix(key, tierPrefix) && strings.Count(key, ".") == 2
rest, ok := strings.CutPrefix(key, tierPrefix)
if !ok {
return false
}
_, err := pipelinekey.Parse(rest)
return err == nil
}
// startTiers writes the tier progress of a passed pipeline, holding every passed mover, versioned or not, its first tier the
@@ -166,11 +174,11 @@ func (p *passer) tiers(n assign.Nodes, placedOn func(module, branch string) []st
continue
}
pipeline := strings.TrimPrefix(key, tierPrefix)
branchEsc, number, _ := strings.Cut(pipeline, ".")
branch, err := keys.UnescapeName(branchEsc)
pk, err := pipelinekey.Parse(pipeline)
if err != nil {
continue
}
branch, number := pk.Branch, strconv.FormatInt(pk.Revision, 10)
for {
t, rev, ok := p.ends.tiersOf(pipeline)
if !ok {
@@ -28,7 +28,12 @@ import (
// tier-bound: the finding tier-late names it and its machine, pipeline-wanted says tier-late, and no consumer want is written.
func TestAPipelinesWantsAreWrittenTierByTier(t *testing.T) {
db, web := meshcatalogclient.SourcePrefix2+strings.Repeat("d", 64), meshcatalogclient.SourcePrefix2+strings.Repeat("c", 64)
for _, runs := range []bool{true, false} {
// H-A-485: a pipeline started again at the head, keyed `<branch>.<revision>.again.<n>`, moves tier by tier as a take-in's does.
for _, c := range []struct {
runs bool
pipeline string
}{{true, trunkBranch + ".5"}, {false, trunkBranch + ".5"}, {true, trunkBranch + ".5.again.2"}, {false, trunkBranch + ".5.again.2"}} {
runs, pipeline := c.runs, c.pipeline
start := time.Date(2026, 11, 20, 4, 0, 0, 0, time.UTC)
now := start
wantsKV := &memKV{values: map[string][]byte{}}
@@ -41,7 +46,7 @@ func TestAPipelinesWantsAreWrittenTierByTier(t *testing.T) {
var said []meshwants.PipelineWanted
p.publishWants = func(event string, body any, _ string) error {
if b, ok := body.(meshwants.PipelineWantedEvent); ok {
if event != "pipeline-wanted."+trunkBranch+".5" {
if event != "pipeline-wanted."+pipeline {
t.Errorf("pipeline-wanted said as %s", event)
}
said = append(said, b)
@@ -65,7 +70,7 @@ func TestAPipelinesWantsAreWrittenTierByTier(t *testing.T) {
// The pipeline of revision 5 (its key's number) of source 4: a re-pin's revision (H-A-213, H-A-216).
e := ended(trunkBranch, 5, mpr.PipelineEndedOutcomePassed, mover("db", db, "0.2.0", 1, mpr.StepResultOutcomePassed),
mover("web", web, "0.1.2", 2, mpr.StepResultOutcomePassed))
e.Source = 4
e.Source, e.Pipeline = 4, pipeline
if _, err := p.ends.take(e); err != nil {
t.Fatal(err)
}
@@ -76,7 +81,7 @@ func TestAPipelinesWantsAreWrittenTierByTier(t *testing.T) {
}
p.wants(context.Background())
if !wanted("g14", "db") || !wanted("ace", "db") || wanted("g14", "web") || wanted("ace", "web") {
t.Fatalf("runs %v: tier 1 alone is not what is wanted first: %v", runs, wantsKV.values)
t.Fatalf("%s runs %v: tier 1 alone is not what is wanted first: %v", pipeline, runs, wantsKV.values)
}
if runs {
raw, _ := json.Marshal(nodeengine.Running{Node: "g14", Module: "db", Fingerprint: db})
@@ -86,15 +91,15 @@ func TestAPipelinesWantsAreWrittenTierByTier(t *testing.T) {
now = start.Add(time.Minute)
p.wants(context.Background())
if !wanted("g14", "web") || !wanted("ace", "web") {
t.Fatalf("the consumer's want is not written once its provider runs: %v", wantsKV.values)
t.Fatalf("%s: the consumer's want is not written once its provider runs: %v", pipeline, wantsKV.values)
}
raw, _ = json.Marshal(nodeengine.Running{Node: "g14", Module: "web", Fingerprint: web})
_ = p.w.hearRunning("running.g14.web", raw)
p.wants(context.Background())
if len(said) != 1 || said[0].Outcome != meshwants.PipelineWantedOutcomeRunning || said[0].Revision != 5 || said[0].Source != 4 || len(findings) != 0 {
t.Errorf("every tier runs: said %+v, findings %v", said, findings)
t.Errorf("%s: every tier runs: said %+v, findings %v", pipeline, said, findings)
}
if _, _, ok := p.ends.tiersOf(trunkBranch + ".5"); ok {
if _, _, ok := p.ends.tiersOf(pipeline); ok {
t.Error("the tier progress stands after every tier ran")
}
continue
@@ -107,7 +112,7 @@ func TestAPipelinesWantsAreWrittenTierByTier(t *testing.T) {
now = start.Add(30 * time.Minute)
p.wants(context.Background())
p.wants(context.Background())
f, ok := findings["delivery."+assign.KeyID(trunkBranch+".5")+".tier-late"]
f, ok := findings["delivery."+assign.KeyID(pipeline)+".tier-late"]
if !ok || !strings.Contains(f.Summary, "db on g14") || strings.Contains(f.Summary, "ace") {
t.Errorf("tier-late is not raised naming db on g14 alone: %v", findings)
}
@@ -116,7 +121,7 @@ func TestAPipelinesWantsAreWrittenTierByTier(t *testing.T) {
t.Errorf("pipeline-wanted tier-late is not said once, naming revision 5 of source 4: %+v", said)
}
// A tier progress kept before it held its source says the key's number as both (H-A-216).
p.sayPipelineWanted(trunkBranch+".5", trunkBranch, "5", 0, meshwants.PipelineWantedOutcomeRunning, nil)
p.sayPipelineWanted(pipeline, trunkBranch, "5", 0, meshwants.PipelineWantedOutcomeRunning, nil)
if len(said) != 2 || said[1].Revision != 5 || said[1].Source != 5 {
t.Errorf("a progress with no source: said %+v, want revision and source 5", said)
}
@@ -73,7 +73,7 @@ func publishQueueVia(ask func(key string, body any) (json.RawMessage, error)) fu
}
}
// publishActTools are builder_publish_drop and builder_publish_first (design 47 H2 §793): hand acts, recorded in the hand-act log
// publishActTools are builder_publish_drop (design 47 H2 §793; builder_publish_first is retired, H-A-483 third addendum): hand acts, recorded in the hand-act log
// first, then written to reasks for the builder, which owns the work queue, to perform and answer.
func publishActTools(ask queueAsker, now func() time.Time, acts meshhandactsclient.Recorder) []stdio.Tool {
str := func(args map[string]any, k string) string { s, _ := args[k].(string); return strings.TrimSpace(s) }
@@ -108,7 +108,6 @@ func publishActTools(ask queueAsker, now func() time.Time, acts meshhandactsclie
}
return []stdio.Tool{
tool(versions.PublishDropTool, build.ModePublishDrop, versions.PublishDropDescription, "dropped the publish of"),
tool(versions.PublishFirstTool, build.ModePublishFirst, versions.PublishFirstDescription, "put first the publish of"),
}
}
@@ -78,8 +78,8 @@ func TestAStateIsReadInOnePage(t *testing.T) {
}
})
// Design 47 H2 §793, H-A-454: builder_publish_queue asks the seat mesh-builds' verb publish-queue, which the builder serves from
// its own stream, and answers its counts, list and cap with the states beside them; builder_publish_drop and
// builder_publish_first record the hand act first, then write their reasks value.
// its own stream, and answers its counts, list and cap with the states beside them; builder_publish_drop records the hand act
// first, then writes its reasks value; builder_publish_first is retired (H-A-483, third addendum).
t.Run("the queue is the builder's verb publish-queue", func(t *testing.T) {
var asked []string
ask := func(key string, body any) (json.RawMessage, error) {
@@ -123,7 +123,7 @@ func TestAStateIsReadInOnePage(t *testing.T) {
t.Errorf("asked %v, answered %+v (%v): want each holder asked in turn at its node, answered together, the silent one with why", asked, hs, err)
}
})
t.Run("drop and first record the hand act, then ask the builder", func(t *testing.T) {
t.Run("drop records the hand act, then asks the builder; first is retired", func(t *testing.T) {
var order []string
ask := func(key string, value []byte) (json.RawMessage, error) {
order = append(order, "ask "+key)
@@ -141,8 +141,8 @@ func TestAStateIsReadInOnePage(t *testing.T) {
t.Fatalf("%s: %v", tool.Name, err)
}
}
if len(tools) != 2 || fmt.Sprint(order) != "[record ask publish-drop.5000000000 record ask publish-first.5000000000]" {
t.Errorf("tools %d, order %v: want each hand act recorded before its reasks value", len(tools), order)
if len(tools) != 1 || tools[0].Name != versions.PublishDropTool || fmt.Sprint(order) != "[record ask publish-drop.5000000000]" {
t.Errorf("tools %d, order %v: want builder_publish_drop alone, its hand act recorded before its reasks value", len(tools), order)
}
if _, err := tools[0].Run(map[string]any{"module": "m", "why": "no version"}); err == nil {
t.Error("a drop naming no version is not refused")
@@ -40,6 +40,7 @@ import (
"git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"git.novox.be/novox/mesh-sdk/go/testbus"
"git.novox.be/novox/mesh-sdk/go/views/viewstest"
"mesh-delivery-builder/internal/build"
@@ -578,7 +579,10 @@ func TestBuilderMakesADoneKeyWithTheDigests(t *testing.T) { // 1607/AC7
first := "src2:" + strings.Repeat("1", 64)
b.moduleFP(t, "netcheck", netcheck, "modules/netcheck", bothToolchains, commit, first, 1)
digest := b.moduleFP(t, "netcheck", netcheck, "modules/netcheck", bothToolchains, commit, fp, 2)
b.start(t, "n1", map[string]string{"MESH_BUILDER_STEP_BOUND": "1s"})
// H-A-473 (addendum): the builds that must end done run under the level bound's set-up share, never a bound a loaded
// node cannot meet; the hung step's part below starts the builder again at a short bound of its own.
doneBound := map[string]string{"MESH_BUILDER_STEP_BOUND": testbus.SetupBound(t).String()}
b.start(t, "n1", doneBound)
v := b.await(t, "netcheck", isState(builds.BuildDone))
for _, a := range []string{"tools-go", "tools-typescript"} {
es := reg.of(t, "netcheck/"+a)
@@ -693,6 +697,8 @@ func TestBuilderMakesADoneKeyWithTheDigests(t *testing.T) { // 1607/AC7
}
// DLV-B10: a build module's bound running out fails the trunk build, which is the builder's finding build-failed; the
// fingerprint built again clears it with its done build, numbering nothing of the builder's own (H-A-159).
b.stop()
b.start(t, "n1", map[string]string{"MESH_BUILDER_STEP_BOUND": "1s"}) // a step that never ends runs out of any bound
release := make(chan struct{})
b.fake.hang.Store(&release)
third := "src2:" + strings.Repeat("4", 64)
@@ -713,6 +719,8 @@ func TestBuilderMakesADoneKeyWithTheDigests(t *testing.T) { // 1607/AC7
reg.awaitFinding(t, failed, true)
b.fake.hang.Store(nil)
close(release)
b.stop() // the build again must end done
b.start(t, "n1", doneBound)
b.moduleFP(t, "netcheck", netcheck, "modules/netcheck", bothToolchains, "89abcdef0123456789abcdef0123456789abcdef", third, 3) // the entry moves to a later commit, same fingerprint
b.awaitFP(t, "netcheck", third, isState(builds.BuildDone))
reg.awaitFinding(t, failed, false)
@@ -273,7 +273,7 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
Entry: entryVia(calls.Call), Send: sendVia(calls.CallKeptAsk),
Unreleased: reserved.unreleasedWords, Logf: logger.Printf,
Holders: holders, // a registry seat's holder from the seat-holders view, from the start (design 47 G A-416, A-416a)
Started: time.Now()} // a build ended from now on is of the live class, sent first (H-A-290 (1))
Started: time.Now()} // a build ended from now on is of the live class (H-A-290 (1))
// A number a registry seat already holds is never reserved (H-A-275): read before reserving, through the seats' verb versions.
reserved.r.HeldNumbers = func(ctx context.Context, module, fp string, refs []string) ([]string, error) {
return republish.HeldNumbers(ctx, module, fp, refs, registryVersionsVia(calls.Call))
@@ -298,7 +298,8 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
if err != nil {
return err
}
pq.ackWait = publishSet.PauseMax // a sent item unheard is delivered again after the longest pause (H-A-440 (1))
pq.ackWait = publishSet.PauseMax // a sent item unheard is delivered again after the longest pause (H-A-440 (1))
reserved.pause = publishSet.Pause // a version-released whose tag the forge did not write waits the pause (H-A-484 addendum)
releasedKept, err := numberFile(getenv("MESH_BUILDER_RELEASED_KEPT_FILE"), versions.DefaultReleasedKept)
if err != nil {
return err
@@ -322,15 +323,17 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
// A version that never publishes (withdrawn or taken over) has its publish item ended by the queue too (H-A-290 (2)).
republish.Ended = items.Ended
pq.take = items.take
reasksKV.Ready(func(jetstream.KeyValue) { items.AskAllDone(ctx) })
// At the start, a built reservation the catalog holds released is taken in as its version-released would (H-A-484 addendum),
// before the publish items of the rest are asked.
reasksKV.Ready(func(jetstream.KeyValue) {
reserved.releasedAtStart(ctx)
items.AskAllDone(ctx)
})
wirePublishItems(reserved, items, awaitedReserved{awaitedVersions{b: actedKV, bucket: own.acted}}, releasedKept)
acts.Republish = republish.Republish
acts.PublishDrop = func(ctx context.Context, module, version, by, why string) (string, error) {
return pq.drop(ctx, items, module, version, by, why)
}
acts.PublishFirst = func(ctx context.Context, module, version, _, _ string) (string, error) {
return pq.first(ctx, items, module, version)
}
// builder_reserve_done: a done trunk build that reserved none is given a version, and its publish queued as publish again does.
acts.ReserveDone = reserveDoneAct(b.ReserveDone, republish.Republish, logger.Printf)
// The operator is asked once a key's event builds are spent (design 47 G §568): the builder's asker on the seat operator-channel.
@@ -22,19 +22,21 @@ import (
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshbuilds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/testbus"
"git.novox.be/novox/mesh-sdk/go/views"
"mesh-delivery-builder/internal/versions"
)
// Design 47 H2 §793 (whole, against the bus): the builder's work queue publishes, its pull consumers made as it learns a holder.
// A live publish enqueued behind 100 again items while the builder was away is sent first; a dropped item is never sent, and
// put again within the duplicates window is not stored; the move of the old state run twice puts each item once, and the
// builder removes the old state's bucket itself (H-A-461 (1)). The seat mesh-builds' verb publish-queue reads the stream: counts
// per holder and class, the oldest and newest, the live item listed first (H-A-454). Then §796's tag at another commit, its
// reservation's version-reserved waited on by name (§821), and
// §794's backlog held six minutes, as steps (H-A-453 (3)).
func TestThePublishQueueSendsLiveFirstAndNeverADroppedItem(t *testing.T) {
// Design 47 H2 §793 (whole, against the bus): the builder's work queue publishes, one pull consumer made as it learns a holder
// (H-A-483 and its addenda), the class durables of an earlier release retired. No item is lost, a failed send comes back after
// its pause behind an item put meanwhile, and a dropped item never comes back; 100 again items and a live one put while the
// builder was away are all sent, the dropped one never, and put again within the duplicates window it is not stored; the move
// of the old state run twice puts each item once, and the builder removes the old state's bucket itself (H-A-461 (1)). The seat
// mesh-builds' verb publish-queue reads the stream: counts per holder and class, the oldest and newest, the items listed in
// stream order (H-A-454; H-A-483, third addendum). Then §796's tag at another commit, its reservation's version-reserved waited on by name (§821), and §794's
// backlog held six minutes, as steps (H-A-453 (3)).
func TestThePublishQueueLosesNoItemAndNeverSendsADroppedItem(t *testing.T) {
b := startBed(t)
ctx := b.ctx
// The stream as the manifest's old streams section declares it: the module's own, retention work-queue, duplicates 24 hours.
@@ -43,281 +45,426 @@ func TestThePublishQueueSendsLiveFirstAndNeverADroppedItem(t *testing.T) {
Retention: jetstream.WorkQueuePolicy, Discard: jetstream.DiscardNew, Duplicates: 24 * time.Hour, MaxAge: 7 * 24 * time.Hour}); err != nil {
t.Fatal(err)
}
kv := &itemKV{vals: map[string][]byte{}, revs: map[string]uint64{}}
var mu sync.Mutex
var sent []string
published := map[string]bool{} // the modules whose publish is heard
sentCh := make(chan string, 200)
items := &publishItems{reasks: func() jetstream.KeyValue { return kv }, set: publishSettings{Pause: time.Second, PauseMax: time.Minute, Tries: 5},
now: time.Now, logf: t.Logf,
reserved: func(_ context.Context, m, v string) (versions.ReservedVersion, bool, error) {
// One consumer per holder (H-A-483 and its addenda): the holder's class durables of an earlier release are retired, the item
// one of them held unacknowledged reaches the one consumer; no item is lost; a failed send comes back after its pause, and a
// live item put while it waits is sent before it returns; a dropped item never comes back.
t.Run("one consumer", func(t *testing.T) {
const holder, pause = "paused", 3 * time.Second // the pause is far longer than a send of an item put now
bound := testbus.SetupBound(t)
kv := &itemKV{vals: map[string][]byte{}, revs: map[string]uint64{}}
type send struct {
module string
at time.Time
}
var mu sync.Mutex
published, failed := map[string]bool{}, map[string]bool{}
sentCh := make(chan send, 16)
items := &publishItems{reasks: func() jetstream.KeyValue { return kv }, set: publishSettings{Pause: pause, PauseMax: time.Minute, Tries: 5},
now: time.Now, logf: t.Logf,
reserved: func(_ context.Context, m, v string) (versions.ReservedVersion, bool, error) {
mu.Lock()
defer mu.Unlock()
return versions.ReservedVersion{Module: m, Version: v, Artifacts: map[string]string{m + ":a": "sha256:a"}, Published: map[string]bool{m + ":a": published[m]}, BuiltAt: time.Now()}, true, nil
},
send: func(_ context.Context, q versions.QueuedPublish) versions.SendFate {
sentCh <- send{q.Module, time.Now()}
mu.Lock()
defer mu.Unlock()
if (q.Module == "fail" || q.Module == "dropme") && !failed[q.Module] { // the first send of each fails
failed[q.Module] = true
return versions.SendFate{Failed: "the registry is down"}
}
published[q.Module] = true
return versions.SendFate{Sent: true}
}}
pq := &publishQueue{js: b.js, ackWait: time.Minute, logf: t.Logf, take: items.take, binding: publishesOf("mesh-delivery-builder")}
put := func(module, class string) {
q := versions.QueuedPublish{V: 1, Holder: holder, Module: module, Version: "0.1.0", Artifacts: map[string]string{module + ":a": "sha256:a"}, Class: class, Queued: time.Now()}
data, _ := json.Marshal(q)
if _, err := b.js.Publish(ctx, pq.subject(bind, q), data); err != nil {
t.Fatal(err)
}
}
hear := func() { // each publish sent is heard: its item acknowledged once it is held
mu.Lock()
defer mu.Unlock()
return versions.ReservedVersion{Module: m, Version: v, Artifacts: map[string]string{m + ":a": "sha256:a"}, Published: map[string]bool{m + ":a": published[m]}, BuiltAt: time.Now()}, true, nil
},
send: func(_ context.Context, q versions.QueuedPublish) versions.SendFate {
mu.Lock()
sent = append(sent, q.Module)
var done []string
for m, ok := range published {
if ok {
done = append(done, m)
}
}
mu.Unlock()
sentCh <- q.Module
return versions.SendFate{Sent: true}
}}
pq := &publishQueue{js: b.js, ackWait: time.Minute, logf: t.Logf, take: items.take, binding: publishesOf("mesh-delivery-builder")}
item := func(module, class string) versions.QueuedPublish {
return versions.QueuedPublish{V: 1, Holder: "oci", Module: module, Version: "0.1.0", Artifacts: map[string]string{module + ":a": "sha256:a"}, Class: class, Queued: time.Now()}
}
away := func(q versions.QueuedPublish) { // put while the builder was away: no holder learnt, nothing pulled
data, _ := json.Marshal(q)
if _, err := b.js.Publish(ctx, pq.subject(bind, q), data, jetstream.WithMsgID(versions.PublishMsgID(q.Holder, q.Class, q.Module, q.Version))); err != nil {
for _, m := range done {
items.heard(ctx, m, "0.1.0")
}
}
next := func() send {
t.Helper()
select {
case s := <-sentCh:
hear()
return s
case <-time.After(bound):
t.Fatal("no item was sent within the level's set-up share")
}
return send{}
}
// The two class durables an earlier release made, the live one holding an item given and not acknowledged.
for _, class := range []string{versions.PublishClassLive, versions.PublishClassStart} {
if _, err := b.js.CreateOrUpdateConsumer(ctx, bind.Stream, jetstream.ConsumerConfig{Durable: publishesConsumer(holder) + "-" + class,
FilterSubject: bind.Subject + ".publish." + holder + "." + class + ".>", AckPolicy: jetstream.AckExplicitPolicy, AckWait: time.Hour}); err != nil {
t.Fatal(err)
}
}
put("given", versions.PublishClassLive)
old, err := b.js.Consumer(ctx, bind.Stream, publishesConsumer(holder)+"-"+versions.PublishClassLive)
if err != nil {
t.Fatal(err)
}
}
for i := range 100 {
away(item(fmt.Sprintf("a%03d", i), versions.PublishClassStart))
}
away(item("dropme", versions.PublishClassStart))
away(item("live", versions.PublishClassLive))
if answer, err := pq.drop(ctx, items, "dropme", "0.1.0", "op@node", "a test drops it"); err != nil {
t.Fatalf("drop: %v", err)
} else {
t.Log(answer)
}
if dup, err := pq.put(ctx, item("dropme", versions.PublishClassStart)); err != nil || !dup {
t.Errorf("a dropped item put again within the window: duplicate %v (%v), want it not stored", dup, err)
}
raw, err := publishQueueVerb(ctx, pq)("op@node", json.RawMessage(`{}`))
if a, ok := raw.(meshbuilds.PublishQueueAnswer); err != nil || !ok || a.Total != 101 || len(a.Counts) != 2 || a.Counts[0].Class != versions.PublishClassStart ||
a.Counts[0].Count != 100 || a.Counts[1].Count != 1 || a.Counts[0].Oldest.After(a.Counts[0].Newest) || len(a.Queue) != versions.PublishQueueListCap ||
a.Queue[0].Module != "live" || a.Capped {
t.Errorf("publish-queue answered %+v (%v), want 101 items, oci's 100 again and 1 live, fifty listed, the live first", raw, err)
}
if err := pq.learnAll(ctx); err != nil {
t.Fatal(err)
}
select {
case first := <-sentCh:
if first != "live" {
t.Errorf("the first item sent is %s, want the live one, enqueued behind 100 again items", first)
if m, err := old.Next(); err != nil {
t.Fatal(err)
} else if md, _ := m.Metadata(); md.NumDelivered != 1 {
t.Fatalf("the old live durable gave %+v", md)
}
case <-time.After(30 * time.Second):
t.Fatal("nothing was sent")
}
for range 4 { // the again items in flight: at most four held unacknowledged, their publish unheard
select {
case <-sentCh:
case <-time.After(30 * time.Second):
t.Fatal("the again items were not sent")
pq.learn(ctx, holder)
stream, err := b.js.Stream(ctx, bind.Stream)
if err != nil {
t.Fatal(err)
}
var names []string
for n := range stream.ConsumerNames(ctx).Name() {
if strings.HasPrefix(n, publishesConsumer(holder)) {
names = append(names, n)
}
}
if !slices.Equal(names, []string{publishesConsumer(holder)}) {
t.Fatalf("the consumers of %s are %v, want only %s: the class durables retired", holder, names, publishesConsumer(holder))
}
if s := next(); s.module != "given" {
t.Fatalf("sent %s first, want the item the old durable held unacknowledged", s.module)
}
put("fail", versions.PublishClassStart)
put("dropme", versions.PublishClassStart)
var firstFail time.Time
for range 2 {
if s := next(); s.module == "fail" {
firstFail = s.at
}
}
put("live", versions.PublishClassLive) // put while both failed items wait out their pause
if s := next(); s.module != "live" {
t.Fatalf("sent %s, want the live item put while the failed ones wait their pause", s.module)
}
if answer, err := pq.drop(ctx, items, "dropme", "0.1.0", "op@node", "a test drops it"); err != nil {
t.Fatalf("drop: %v", err)
} else {
t.Log(answer)
}
if s := next(); s.module != "fail" || s.at.Sub(firstFail) < pause {
t.Fatalf("sent %s %s after the failed send, want fail again after its pause %s", s.module, s.at.Sub(firstFail), pause)
}
// Every item is acknowledged and the dropped one is gone: nothing is left to come back.
c, err := b.js.Consumer(ctx, bind.Stream, publishesConsumer(holder))
if err != nil {
t.Fatal(err)
}
st, err := b.js.Stream(ctx, bind.Stream)
if err != nil {
t.Fatal(err)
}
for deadline := time.Now().Add(bound); ; {
hear()
info, err := c.Info(ctx)
if err != nil {
t.Fatal(err)
}
sinfo, err := st.Info(ctx, jetstream.WithSubjectFilter(bind.Subject+".publish."+holder+".>"))
if err != nil {
t.Fatal(err)
}
if info.NumAckPending == 0 && info.NumPending == 0 && len(sinfo.State.Subjects) == 0 {
break
}
if time.Now().After(deadline) {
t.Fatalf("the queue of %s holds %v, its consumer %d pending and %d unacknowledged: want every item acknowledged and the dropped one gone", holder, sinfo.State.Subjects, info.NumPending, info.NumAckPending)
}
select {
case s := <-sentCh:
t.Fatalf("sent %s again, want nothing more", s.module)
case <-time.After(10 * time.Millisecond):
}
}
})
t.Run("the queue, the move, tags and backlog", func(t *testing.T) {
kv := &itemKV{vals: map[string][]byte{}, revs: map[string]uint64{}}
var mu sync.Mutex
var sent []string
published := map[string]bool{} // the modules whose publish is heard
sentCh := make(chan string, 200)
items := &publishItems{reasks: func() jetstream.KeyValue { return kv }, set: publishSettings{Pause: time.Second, PauseMax: time.Minute, Tries: 5},
now: time.Now, logf: t.Logf,
reserved: func(_ context.Context, m, v string) (versions.ReservedVersion, bool, error) {
mu.Lock()
defer mu.Unlock()
return versions.ReservedVersion{Module: m, Version: v, Artifacts: map[string]string{m + ":a": "sha256:a"}, Published: map[string]bool{m + ":a": published[m]}, BuiltAt: time.Now()}, true, nil
},
send: func(_ context.Context, q versions.QueuedPublish) versions.SendFate {
mu.Lock()
sent = append(sent, q.Module)
mu.Unlock()
sentCh <- q.Module
return versions.SendFate{Sent: true}
}}
pq := &publishQueue{js: b.js, ackWait: time.Minute, logf: t.Logf, take: items.take, binding: publishesOf("mesh-delivery-builder")}
item := func(module, class string) versions.QueuedPublish {
return versions.QueuedPublish{V: 1, Holder: "oci", Module: module, Version: "0.1.0", Artifacts: map[string]string{module + ":a": "sha256:a"}, Class: class, Queued: time.Now()}
}
away := func(q versions.QueuedPublish) { // put while the builder was away: no holder learnt, nothing pulled
data, _ := json.Marshal(q)
if _, err := b.js.Publish(ctx, pq.subject(bind, q), data, jetstream.WithMsgID(versions.PublishMsgID(q.Holder, q.Class, q.Module, q.Version))); err != nil {
t.Fatal(err)
}
}
for i := range 100 {
away(item(fmt.Sprintf("a%03d", i), versions.PublishClassStart))
}
away(item("dropme", versions.PublishClassStart))
away(item("live", versions.PublishClassLive))
if answer, err := pq.drop(ctx, items, "dropme", "0.1.0", "op@node", "a test drops it"); err != nil {
t.Fatalf("drop: %v", err)
} else {
t.Log(answer)
}
if dup, err := pq.put(ctx, item("dropme", versions.PublishClassStart)); err != nil || !dup {
t.Errorf("a dropped item put again within the window: duplicate %v (%v), want it not stored", dup, err)
}
raw, err := publishQueueVerb(ctx, pq)("op@node", json.RawMessage(`{}`))
if a, ok := raw.(meshbuilds.PublishQueueAnswer); err != nil || !ok || a.Total != 101 || len(a.Counts) != 2 || a.Counts[0].Class != versions.PublishClassStart ||
a.Counts[0].Count != 100 || a.Counts[1].Count != 1 || a.Counts[0].Oldest.After(a.Counts[0].Newest) || len(a.Queue) != versions.PublishQueueListCap ||
a.Queue[0].Module != "a000" || a.Capped {
t.Errorf("publish-queue answered %+v (%v), want 101 items, oci's 100 again and 1 live, fifty listed in stream order, a000 first", raw, err)
}
if err := pq.learnAll(ctx); err != nil {
t.Fatal(err)
}
for range 4 { // the items in flight: at most four held unacknowledged, their publish unheard
select {
case <-sentCh:
case <-time.After(30 * time.Second):
t.Fatal("the again items were not sent")
}
}
deadline := time.After(60 * time.Second)
for { // each publish sent is heard: its item acknowledged, the next sent
mu.Lock()
n, done := len(sent), slices.Clone(sent)
for _, m := range done {
published[m] = true
}
mu.Unlock()
if n >= 101 {
break
}
for _, m := range done {
items.heard(ctx, m, "0.1.0")
}
select {
case <-sentCh:
case <-deadline:
t.Fatalf("sent %d of 101", n)
}
}
}
deadline := time.After(60 * time.Second)
for { // each publish sent is heard: its item acknowledged, the next sent
mu.Lock()
n, done := len(sent), slices.Clone(sent)
for _, m := range done {
published[m] = true
if slices.Contains(sent, "dropme") || len(sent) != 101 {
t.Errorf("sent %d items (dropme among them: %v), want the 101 and never the dropped one", len(sent), slices.Contains(sent, "dropme"))
}
mu.Unlock()
if n >= 101 {
break
// The move of the old state, run twice, puts each item once (at another holder, so the pull of oci is not asked).
old, err := b.js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: builds.StateBucket("mesh-delivery-builder", versions.PublishQueueState)})
if err != nil {
t.Fatal(err)
}
for _, m := range done {
items.heard(ctx, m, "0.1.0")
var tagsWritten []string
call := func(tool string, body any) (json.RawMessage, error) {
switch tool {
case "seat:" + gitseat.Seat + "." + gitseat.VerbTags: // m4's 0.2.0 is tagged at another commit (§796)
return json.Marshal(gitseat.TagsAnswer{Form: 2, Tags: []gitseat.Tag{{Tag: "modules/m4/v0.2.0", Commit: "c-other"}}})
case "seat:" + gitseat.Seat + ".tag":
tagsWritten = append(tagsWritten, body.(gitseat.TagInput).Tag+"@"+body.(gitseat.TagInput).Commit)
return json.Marshal(gitseat.TagAnswer{Performed: true})
}
return json.Marshal(meshcatalog.VersionsAnswer{Module: "m"})
}
found := &memFindings{vals: map[string][]byte{}}
rv := newReservations(&memReserve{vals: map[string][]byte{}, revs: map[string]uint64{}}, call, func() jetstream.KeyValue { return &boundsKV{keys: map[string]bool{}} },
&findings{module: builds.BuilderModule, w: found, logf: t.Logf, now: time.Now}, nil, t.Logf)
rv.filled.Store(true)
type oldItem struct {
key string
raw []byte
}
var olds []oldItem
for _, m := range []string{"m1", "m2", "m3"} {
v, err := rv.Reserve(ctx, m, "0.1", []string{m + ":a"}, versions.Source{Fingerprint: "src2:" + m})
if err != nil {
t.Fatal(err)
}
if err := rv.Built(ctx, m, v.Version, map[string]string{m + ":a": "sha256:a"}, nil); err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(versions.QueuedPublish{V: 1, Holder: "go", Module: m, Version: v.Version, Artifacts: map[string]string{m + ":a": "sha256:a"}, Class: "asked-again-at-start"})
if _, err := old.Put(ctx, versions.QueueKey("go", m, v.Version), raw); err != nil {
t.Fatal(err)
}
olds = append(olds, oldItem{versions.QueueKey("go", m, v.Version), raw})
}
rp := &versions.Republisher{Reserved: rv.r, Put: func(ctx context.Context, q versions.QueuedPublish) (bool, error) {
data, _ := json.Marshal(q)
ack, err := b.js.Publish(ctx, pq.subject(bind, q), data, jetstream.WithMsgID(versions.PublishMsgID(q.Holder, q.Class, q.Module, q.Version)))
if err != nil {
return false, err
}
return ack.Duplicate, nil
}, Started: time.Now(), Logf: t.Logf}
oldBucket := builds.StateBucket("mesh-delivery-builder", versions.PublishQueueState)
moveOld(ctx, b.js, oldBucket, rp, t.Logf)
if _, err := b.js.KeyValue(ctx, oldBucket); !errors.Is(err, jetstream.ErrBucketNotFound) {
t.Errorf("after the move the old state's bucket answers %v, want it removed by the builder (H-A-461 (1))", err)
}
// A start that stopped before the removal moves the same items again: the duplicates window keeps each once.
again, err := b.js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: oldBucket})
if err != nil {
t.Fatal(err)
}
for _, e := range olds {
if _, err := again.Put(ctx, e.key, e.raw); err != nil {
t.Fatal(err)
}
}
moveOld(ctx, b.js, oldBucket, rp, t.Logf)
moveOld(ctx, b.js, oldBucket, rp, t.Logf) // no bucket left: nothing moves
if _, err := b.js.KeyValue(ctx, oldBucket); !errors.Is(err, jetstream.ErrBucketNotFound) {
t.Errorf("after the second move the old state's bucket answers %v, want it removed", err)
}
st, err := b.js.Stream(ctx, bind.Stream)
if err != nil {
t.Fatal(err)
}
info, err := st.Info(ctx, jetstream.WithSubjectFilter(bind.Subject+".publish.go.>"))
if err != nil {
t.Fatal(err)
}
if len(info.State.Subjects) != 3 || info.State.Subjects[bind.Subject+".publish.go.again.m1.0=2E1=2E0"] != 1 {
t.Errorf("the move run twice left %v on the work queue: want each of the three once, as again", info.State.Subjects)
}
// Design 47 H2 §796 (whole, H-A-453 (3)): a module's tags are read before it reserves; a number whose tag stands at another
// commit is skipped, the next number taken and version-tagged-elsewhere raised; the release tags <directory>/v<version> at
// the build's commit, and the module's version-released clears the finding.
rv.r.Names = func(context.Context, string, string) (string, bool, error) {
return "git.example/o/r/modules/m4", true, nil
}
// Design 47 H2 §821, §805: the reservation is said as the seat mesh-builds' version-reserved, said as the process says it
// (sayReservedOn), and waited on by the seat and the event's name.
rv.names, rv.sayReserved = rv.r.Names, sayReservedOn(b.js, process.Account{Module: builds.BuilderModule, Node: "n1"})
said, err := nats.Connect(b.srvURL, nats.UserInfo("ctl", "pw"))
if err != nil {
t.Fatal(err)
}
defer said.Close()
reservedCh := make(chan *nats.Msg, 4)
rs, err := said.ChanSubscribe(meshbuilds.SubjectVersionReserved, reservedCh)
if err != nil {
t.Fatal(err)
}
defer rs.Unsubscribe()
if err := said.Flush(); err != nil {
t.Fatal(err)
}
v4, err := rv.Reserve(ctx, "m4", "0.2", []string{"m4:lib"}, versions.Source{Fingerprint: "src2:m4", GoPackage: true, GoArtifact: "lib",
Repository: "o/r", Commit: "c-mine", Branch: "main", CatalogRevision: 1})
elsewhere := views.FindingKey(meshconditionsclient.ScopeModule, "m4", versions.VersionTaggedElsewhere)
if _, raised := found.vals[elsewhere]; err != nil || v4.Version != "0.2.1" || v4.Tag != "modules/m4/v0.2.1" || !raised {
t.Fatalf("m4 reserved %+v (%v), %s raised %v: want 0.2.1 past 0.2.0's tag at another commit, its tag modules/m4/v0.2.1, and the finding", v4, err, elsewhere, raised)
}
select {
case <-sentCh:
case <-deadline:
t.Fatalf("sent %d of 101", n)
case m := <-reservedCh:
ev, _, err := meshbuilds.DecodeVersionReservedEvent(m.Data)
if err != nil || ev.Module != "m4" || ev.Artifact != "m4:lib" || ev.Path != "git.example/o/r/modules/m4" || ev.Version != "0.2.1" ||
ev.Fingerprint != "src2:m4" || ev.Revision != "main.1" || m.Header.Get("x-source") != builds.BuilderModule {
t.Errorf("version-reserved = %+v (%v) from %q, want m4:lib at 0.2.1, its path, fingerprint src2:m4 and revision main.1, said by the builder", ev, err, m.Header.Get("x-source"))
}
case <-time.After(10 * time.Second):
t.Fatal("no version-reserved of m4's reservation within 10 s on " + meshbuilds.SubjectVersionReserved)
}
catalogEvents, err := b.js.StreamNameBySubject(ctx, meshcatalog.SubjectVersionReleased) // the bed's stream of the catalog's events
if err != nil {
t.Fatal(err)
}
evs, err := b.js.CreateOrUpdateConsumer(ctx, catalogEvents, jetstream.ConsumerConfig{Durable: "builder-events", AckPolicy: jetstream.AckExplicitPolicy,
FilterSubject: meshcatalog.SubjectVersionReleased, DeliverPolicy: jetstream.DeliverNewPolicy})
if err != nil {
t.Fatal(err)
}
next := func(module, version string) jetstream.Msg {
t.Helper()
data, _ := json.Marshal(meshcatalog.VersionReleasedEvent{Module: module, Version: version})
if _, err := b.js.Publish(ctx, meshcatalog.SubjectVersionReleased, data); err != nil {
t.Fatal(err)
}
m, err := evs.Next(jetstream.FetchMaxWait(10 * time.Second))
if err != nil {
t.Fatal(err)
}
return m
}
if !rv.heard(ctx, next("m4", "0.2.1")) {
t.Fatal("the reservations did not take version-released")
}
if _, raised := found.vals[elsewhere]; raised || !slices.Equal(tagsWritten, []string{"modules/m4/v0.2.1@c-mine"}) {
t.Errorf("after version-released %s raised %v, tags written %v: want it cleared and modules/m4/v0.2.1 written at c-mine, no tag moved", elsewhere, raised, tagsWritten)
}
}
mu.Lock()
if slices.Contains(sent, "dropme") || len(sent) != 101 {
t.Errorf("sent %d items (dropme among them: %v), want the 101 and never the dropped one", len(sent), slices.Contains(sent, "dropme"))
}
mu.Unlock()
// The move of the old state, run twice, puts each item once (at another holder, so the pull of oci is not asked).
old, err := b.js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: builds.StateBucket("mesh-delivery-builder", versions.PublishQueueState)})
if err != nil {
t.Fatal(err)
}
var tagsWritten []string
call := func(tool string, body any) (json.RawMessage, error) {
switch tool {
case "seat:" + gitseat.Seat + "." + gitseat.VerbTags: // m4's 0.2.0 is tagged at another commit (§796)
return json.Marshal(gitseat.TagsAnswer{Form: 2, Tags: []gitseat.Tag{{Tag: "modules/m4/v0.2.0", Commit: "c-other"}}})
case "seat:" + gitseat.Seat + ".tag":
tagsWritten = append(tagsWritten, body.(gitseat.TagInput).Tag+"@"+body.(gitseat.TagInput).Commit)
return json.Marshal(gitseat.TagAnswer{Performed: true})
}
return json.Marshal(meshcatalog.VersionsAnswer{Module: "m"})
}
found := &memFindings{vals: map[string][]byte{}}
rv := newReservations(&memReserve{vals: map[string][]byte{}, revs: map[string]uint64{}}, call, func() jetstream.KeyValue { return &boundsKV{keys: map[string]bool{}} },
&findings{module: builds.BuilderModule, w: found, logf: t.Logf, now: time.Now}, nil, t.Logf)
rv.filled.Store(true)
type oldItem struct {
key string
raw []byte
}
var olds []oldItem
for _, m := range []string{"m1", "m2", "m3"} {
v, err := rv.Reserve(ctx, m, "0.1", []string{m + ":a"}, versions.Source{Fingerprint: "src2:" + m})
// Design 47 H2 §794 (whole, H-A-453 (3), H-A-462): a message the builder's handler leaves unacknowledged is kept on its
// consumer's ledger in the state consumers on the bus; held six minutes, the health names that backlog, and within five
// minutes it does not.
consumers, err := b.js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: builds.StateBucket(builds.BuilderModule, process.ConsumersState)})
if err != nil {
t.Fatal(err)
}
if err := rv.Built(ctx, m, v.Version, map[string]string{m + ":a": "sha256:a"}, nil); err != nil {
t.Fatal(err)
held := time.Now()
hub := &oneHandler{}
aged := agedHub{hub: hub, ledgers: &ledgers{logf: t.Logf, now: func() time.Time { return held },
sink: consumersSink(process.AwaitBucket(ctx, b.js, nil, builds.StateBucket(builds.BuilderModule, process.ConsumersState), "", t.Logf))}}
_ = aged.Handle(meshcatalog.SubjectVersionReleased, func(m jetstream.Msg) { _ = m.Nak() }) // a handler standing still: asked again
hub.handle(next("m5", "0.1.0"))
state := versions.Reader{Keys: func() ([]string, error) {
l, err := consumers.ListKeys(ctx)
if err != nil {
return nil, err
}
var keys []string
for k := range l.Keys() {
keys = append(keys, k)
}
return keys, nil
}, Get: func(k string) (json.RawMessage, bool, error) {
e, err := consumers.Get(ctx, k)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return nil, false, nil
}
if err != nil {
return nil, false, err
}
return e.Value(), true, nil
}}
if a := versions.BuilderHealth(state, held.Add(6*time.Minute), 5*time.Minute); a.Healthy || !strings.Contains(a.Why, "builder-events (its oldest unacknowledged message waits 6m0s") {
t.Errorf("the health with a backlog held six minutes = %+v, want unhealthy naming builder-events", a)
}
raw, _ := json.Marshal(versions.QueuedPublish{V: 1, Holder: "go", Module: m, Version: v.Version, Artifacts: map[string]string{m + ":a": "sha256:a"}, Class: "asked-again-at-start"})
if _, err := old.Put(ctx, versions.QueueKey("go", m, v.Version), raw); err != nil {
t.Fatal(err)
if a := versions.BuilderHealth(state, held.Add(4*time.Minute), 5*time.Minute); !a.Healthy {
t.Errorf("the health four minutes in = %+v, want healthy", a)
}
olds = append(olds, oldItem{versions.QueueKey("go", m, v.Version), raw})
}
rp := &versions.Republisher{Reserved: rv.r, Put: func(ctx context.Context, q versions.QueuedPublish) (bool, error) {
data, _ := json.Marshal(q)
ack, err := b.js.Publish(ctx, pq.subject(bind, q), data, jetstream.WithMsgID(versions.PublishMsgID(q.Holder, q.Class, q.Module, q.Version)))
if err != nil {
return false, err
}
return ack.Duplicate, nil
}, Started: time.Now(), Logf: t.Logf}
oldBucket := builds.StateBucket("mesh-delivery-builder", versions.PublishQueueState)
moveOld(ctx, b.js, oldBucket, rp, t.Logf)
if _, err := b.js.KeyValue(ctx, oldBucket); !errors.Is(err, jetstream.ErrBucketNotFound) {
t.Errorf("after the move the old state's bucket answers %v, want it removed by the builder (H-A-461 (1))", err)
}
// A start that stopped before the removal moves the same items again: the duplicates window keeps each once.
again, err := b.js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: oldBucket})
if err != nil {
t.Fatal(err)
}
for _, e := range olds {
if _, err := again.Put(ctx, e.key, e.raw); err != nil {
t.Fatal(err)
}
}
moveOld(ctx, b.js, oldBucket, rp, t.Logf)
moveOld(ctx, b.js, oldBucket, rp, t.Logf) // no bucket left: nothing moves
if _, err := b.js.KeyValue(ctx, oldBucket); !errors.Is(err, jetstream.ErrBucketNotFound) {
t.Errorf("after the second move the old state's bucket answers %v, want it removed", err)
}
st, err := b.js.Stream(ctx, bind.Stream)
if err != nil {
t.Fatal(err)
}
info, err := st.Info(ctx, jetstream.WithSubjectFilter(bind.Subject+".publish.go.>"))
if err != nil {
t.Fatal(err)
}
if len(info.State.Subjects) != 3 || info.State.Subjects[bind.Subject+".publish.go.again.m1.0=2E1=2E0"] != 1 {
t.Errorf("the move run twice left %v on the work queue: want each of the three once, as again", info.State.Subjects)
}
// Design 47 H2 §796 (whole, H-A-453 (3)): a module's tags are read before it reserves; a number whose tag stands at another
// commit is skipped, the next number taken and version-tagged-elsewhere raised; the release tags <directory>/v<version> at
// the build's commit, and the module's version-released clears the finding.
rv.r.Names = func(context.Context, string, string) (string, bool, error) {
return "git.example/o/r/modules/m4", true, nil
}
// Design 47 H2 §821, §805: the reservation is said as the seat mesh-builds' version-reserved, said as the process says it
// (sayReservedOn), and waited on by the seat and the event's name.
rv.names, rv.sayReserved = rv.r.Names, sayReservedOn(b.js, process.Account{Module: builds.BuilderModule, Node: "n1"})
said, err := nats.Connect(b.srvURL, nats.UserInfo("ctl", "pw"))
if err != nil {
t.Fatal(err)
}
defer said.Close()
reservedCh := make(chan *nats.Msg, 4)
rs, err := said.ChanSubscribe(meshbuilds.SubjectVersionReserved, reservedCh)
if err != nil {
t.Fatal(err)
}
defer rs.Unsubscribe()
if err := said.Flush(); err != nil {
t.Fatal(err)
}
v4, err := rv.Reserve(ctx, "m4", "0.2", []string{"m4:lib"}, versions.Source{Fingerprint: "src2:m4", GoPackage: true, GoArtifact: "lib",
Repository: "o/r", Commit: "c-mine", Branch: "main", CatalogRevision: 1})
elsewhere := views.FindingKey(meshconditionsclient.ScopeModule, "m4", versions.VersionTaggedElsewhere)
if _, raised := found.vals[elsewhere]; err != nil || v4.Version != "0.2.1" || v4.Tag != "modules/m4/v0.2.1" || !raised {
t.Fatalf("m4 reserved %+v (%v), %s raised %v: want 0.2.1 past 0.2.0's tag at another commit, its tag modules/m4/v0.2.1, and the finding", v4, err, elsewhere, raised)
}
select {
case m := <-reservedCh:
ev, _, err := meshbuilds.DecodeVersionReservedEvent(m.Data)
if err != nil || ev.Module != "m4" || ev.Artifact != "m4:lib" || ev.Path != "git.example/o/r/modules/m4" || ev.Version != "0.2.1" ||
ev.Fingerprint != "src2:m4" || ev.Revision != "main.1" || m.Header.Get("x-source") != builds.BuilderModule {
t.Errorf("version-reserved = %+v (%v) from %q, want m4:lib at 0.2.1, its path, fingerprint src2:m4 and revision main.1, said by the builder", ev, err, m.Header.Get("x-source"))
}
case <-time.After(10 * time.Second):
t.Fatal("no version-reserved of m4's reservation within 10 s on " + meshbuilds.SubjectVersionReserved)
}
catalogEvents, err := b.js.StreamNameBySubject(ctx, meshcatalog.SubjectVersionReleased) // the bed's stream of the catalog's events
if err != nil {
t.Fatal(err)
}
evs, err := b.js.CreateOrUpdateConsumer(ctx, catalogEvents, jetstream.ConsumerConfig{Durable: "builder-events", AckPolicy: jetstream.AckExplicitPolicy,
FilterSubject: meshcatalog.SubjectVersionReleased, DeliverPolicy: jetstream.DeliverNewPolicy})
if err != nil {
t.Fatal(err)
}
next := func(module, version string) jetstream.Msg {
t.Helper()
data, _ := json.Marshal(meshcatalog.VersionReleasedEvent{Module: module, Version: version})
if _, err := b.js.Publish(ctx, meshcatalog.SubjectVersionReleased, data); err != nil {
t.Fatal(err)
}
m, err := evs.Next(jetstream.FetchMaxWait(10 * time.Second))
if err != nil {
t.Fatal(err)
}
return m
}
if !rv.heard(ctx, next("m4", "0.2.1")) {
t.Fatal("the reservations did not take version-released")
}
if _, raised := found.vals[elsewhere]; raised || !slices.Equal(tagsWritten, []string{"modules/m4/v0.2.1@c-mine"}) {
t.Errorf("after version-released %s raised %v, tags written %v: want it cleared and modules/m4/v0.2.1 written at c-mine, no tag moved", elsewhere, raised, tagsWritten)
}
// Design 47 H2 §794 (whole, H-A-453 (3), H-A-462): a message the builder's handler leaves unacknowledged is kept on its
// consumer's ledger in the state consumers on the bus; held six minutes, the health names that backlog, and within five
// minutes it does not.
consumers, err := b.js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: builds.StateBucket(builds.BuilderModule, process.ConsumersState)})
if err != nil {
t.Fatal(err)
}
held := time.Now()
hub := &oneHandler{}
aged := agedHub{hub: hub, ledgers: &ledgers{logf: t.Logf, now: func() time.Time { return held },
sink: consumersSink(process.AwaitBucket(ctx, b.js, nil, builds.StateBucket(builds.BuilderModule, process.ConsumersState), "", t.Logf))}}
_ = aged.Handle(meshcatalog.SubjectVersionReleased, func(m jetstream.Msg) { _ = m.Nak() }) // a handler standing still: asked again
hub.handle(next("m5", "0.1.0"))
state := versions.Reader{Keys: func() ([]string, error) {
l, err := consumers.ListKeys(ctx)
if err != nil {
return nil, err
}
var keys []string
for k := range l.Keys() {
keys = append(keys, k)
}
return keys, nil
}, Get: func(k string) (json.RawMessage, bool, error) {
e, err := consumers.Get(ctx, k)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return nil, false, nil
}
if err != nil {
return nil, false, err
}
return e.Value(), true, nil
}}
if a := versions.BuilderHealth(state, held.Add(6*time.Minute), 5*time.Minute); a.Healthy || !strings.Contains(a.Why, "builder-events (its oldest unacknowledged message waits 6m0s") {
t.Errorf("the health with a backlog held six minutes = %+v, want unhealthy naming builder-events", a)
}
if a := versions.BuilderHealth(state, held.Add(4*time.Minute), 5*time.Minute); !a.Healthy {
t.Errorf("the health four minutes in = %+v, want healthy", a)
}
})
}
@@ -7,7 +7,6 @@ import (
"fmt"
"io/fs"
"os"
"sort"
"strconv"
"strings"
"sync"
@@ -531,9 +530,9 @@ func builtOf(all []versions.ReservedVersion) []versions.ReservedVersion {
// the manifest's old streams section with the work-queue retention, the module's own, consumed by the module's one process on the
// module's account (H-A-464), which declares no bus and so is the stream's one reader by the module's shape; each item on
// `mesh.stream.<module>.event.publish.<holder>.<class>.<module>.<version>` under the message id
// `<holder>.<class>.<module>.<version>`. The builder makes the pull consumers at run time as it learns a holder,
// `publishes-<holder>-<class>`, with ack wait publish-pause-max, max ack pending four and max deliveries ten, and pulls the live
// class first, at most four sends at once per holder.
// `<holder>.<class>.<module>.<version>`. The builder makes one pull consumer at run time as it learns a holder,
// `publishes-<holder>`, with ack wait publish-pause-max, max ack pending four and max deliveries ten, and sends its items in the
// order the consumer gives them, at most four at once per holder (H-A-483 and its addenda).
type publishQueue struct {
js jetstream.JetStream
binding func() (publishesStream, error)
@@ -555,16 +554,16 @@ func publishesOf(module string) func() (publishesStream, error) {
return func() (publishesStream, error) { return b, nil }
}
// The consumers' numbers (H-A-446 (2)): at most four sends in flight per holder and class, and ten deliveries, the work queue's
// largest; the builder ends an item at publish-tries anyway.
// The consumer's numbers (H-A-446 (2)): at most four sends in flight per holder, and ten deliveries, the work queue's largest;
// the builder ends an item at publish-tries anyway.
const (
publishesMaxAckPending = versions.PublishSendsAtOnce
publishesMaxDeliver = 10
)
// publishesConsumer is the durable consumer of holder's items of class.
func publishesConsumer(holder, class string) string {
return versions.PublishesLocal + "-" + holder + "-" + class
// publishesConsumer is the one durable consumer of holder's items (H-A-483 addendum).
func publishesConsumer(holder string) string {
return versions.PublishesLocal + "-" + holder
}
// subject is the item's subject on the bus.
@@ -616,8 +615,15 @@ func (pq *publishQueue) learnAll(ctx context.Context) error {
return nil
}
// learn starts holder's pull once.
// learn starts holder's pull once. A holder is a module name, so one ending in -live or -again, the class durables an earlier
// release made, is refused, said (H-A-483, third addendum): its consumer would be another holder's old durable.
func (pq *publishQueue) learn(ctx context.Context, holder string) {
for _, class := range []string{versions.PublishClassLive, versions.PublishClassStart} {
if strings.HasSuffix(holder, "-"+class) {
pq.logf("the publishes of %s are refused: a holder is a module name, and one ending in -%s names the class durable of an earlier release (H-A-483)", holder, class)
return
}
}
pq.mu.Lock()
if pq.pulling == nil {
pq.pulling = map[string]bool{}
@@ -636,94 +642,71 @@ func (pq *publishQueue) learn(ctx context.Context, holder string) {
}
}
// pull makes holder's two consumers and dispatches their items, the live class first, at most four sends at once: an again item
// waits while a live one waits on the queue.
// pull makes holder's one consumer and dispatches its items (design 47 H2 §793, H-A-483 and its addenda): first it retires the
// holder's class durables an earlier release made (`-live`, `-again`), whose filters a work queue refuses beside the one; their
// unacknowledged items stay on the queue and reach the one consumer.
func (pq *publishQueue) pull(ctx context.Context, holder string) error {
b, err := pq.binding()
if err != nil {
return err
}
chans := map[string]chan jetstream.Msg{}
var live jetstream.Consumer
for _, class := range []string{versions.PublishClassStart, versions.PublishClassLive} { // the again first: the live is first by the dispatch, not by luck
c, err := pq.js.CreateOrUpdateConsumer(ctx, b.Stream, jetstream.ConsumerConfig{Durable: publishesConsumer(holder, class),
FilterSubject: b.Subject + ".publish." + versions.EscapeToken(holder) + "." + class + ".>", AckPolicy: jetstream.AckExplicitPolicy,
AckWait: pq.ackWait, MaxAckPending: publishesMaxAckPending, MaxDeliver: publishesMaxDeliver})
if err != nil {
return fmt.Errorf("the consumer %s: %w", publishesConsumer(holder, class), err)
}
ch := make(chan jetstream.Msg, publishesMaxAckPending)
cc, err := c.Consume(func(m jetstream.Msg) { ch <- m }, jetstream.PullMaxMessages(publishesMaxAckPending))
if err != nil {
return fmt.Errorf("the consumer %s is not pulled: %w", publishesConsumer(holder, class), err)
}
go func() { <-ctx.Done(); cc.Stop() }()
chans[class] = ch
if class == versions.PublishClassLive {
live = c
for _, class := range []string{versions.PublishClassLive, versions.PublishClassStart} {
old := publishesConsumer(holder) + "-" + class
err := pq.js.DeleteConsumer(ctx, b.Stream, old)
switch {
case err == nil:
pq.logf("the consumer %s of an earlier release is removed: the publishes of %s have one consumer (H-A-483)", old, holder)
case !errors.Is(err, jetstream.ErrConsumerNotFound):
return fmt.Errorf("the consumer %s of an earlier release is not removed: %w", old, err)
}
}
liveWaits := func() bool {
info, err := live.Info(ctx)
return err == nil && info.NumPending > 0
c, err := pq.js.CreateOrUpdateConsumer(ctx, b.Stream, jetstream.ConsumerConfig{Durable: publishesConsumer(holder),
FilterSubject: b.Subject + ".publish." + versions.EscapeToken(holder) + ".>", AckPolicy: jetstream.AckExplicitPolicy,
AckWait: pq.ackWait, MaxAckPending: publishesMaxAckPending, MaxDeliver: publishesMaxDeliver})
if err != nil {
return fmt.Errorf("the consumer %s: %w", publishesConsumer(holder), err)
}
pq.logf("pulling the publishes of %s, the live first", holder)
go pq.dispatch(ctx, chans[versions.PublishClassLive], chans[versions.PublishClassStart], liveWaits)
pq.logf("pulling the publishes of %s", holder)
go pq.dispatch(ctx, holder, c)
return nil
}
// dispatch runs the items of one holder, at most four at once, every live item before an again one: an again item pulled while
// a live one waits on the queue (liveWaits) runs after it.
func (pq *publishQueue) dispatch(ctx context.Context, live, again <-chan jetstream.Msg, liveWaits func() bool) {
// dispatch runs the items of one holder in the order its one consumer gives them, at most four at once (design 47 H2 §793,
// H-A-483): each free slot takes the next item through a pull that waits, so an item put or handed back after its pause is
// received the moment the bus gives it; an item handed back reappears only after its pause, so an item put meanwhile goes
// before it. A pull the bus refuses ends the dispatch, said; the holder is pulled again when an item of it is put.
func (pq *publishQueue) dispatch(ctx context.Context, holder string, c jetstream.Consumer) {
sem := make(chan struct{}, versions.PublishSendsAtOnce)
run := func(m jetstream.Msg) {
go func() {
defer func() { <-sem }()
pq.take(ctx, m)
}()
}
var spare jetstream.Msg // an again item pulled while a live one waited
for {
select {
case sem <- struct{}{}:
case <-ctx.Done():
return
}
select {
case m := <-live:
run(m)
continue
default:
}
m := spare
spare = nil
if m == nil {
select {
case lm := <-live:
run(lm)
continue
case m = <-again:
case <-ctx.Done():
return
var got jetstream.Msg
b, err := c.Fetch(1)
if err == nil {
for m := range b.Messages() {
got = m
}
err = b.Error()
}
if liveWaits() {
spare = m
select {
case lm := <-live:
run(lm)
case <-ctx.Done():
return
}
if err != nil && !errors.Is(err, jetstream.ErrNoMessages) && !errors.Is(err, nats.ErrTimeout) && ctx.Err() == nil {
<-sem
pq.logf("the publishes of %s are not pulled: %v; they are pulled when an item of it is put", holder, err)
pq.mu.Lock()
delete(pq.pulling, holder)
pq.mu.Unlock()
return
}
if got == nil {
<-sem
continue
}
select {
case lm := <-live: // came while the queue was asked
spare = m
run(lm)
default:
run(m)
}
go func(m jetstream.Msg) {
defer func() { <-sem }()
pq.take(ctx, m)
}(got)
}
}
@@ -827,45 +810,6 @@ func (pq *publishQueue) drop(ctx context.Context, items *publishItems, module, v
return fmt.Sprintf("the publish of %s %s is dropped: %d items terminated, and none is sent or put again", module, version, len(qs)), nil
}
// first is builder_publish_first (§793): each again item of the version is put again as live, under the message id
// `<holder>.live.<module>.<version>`, and the old terminated; a live id the queue holds within its window takes nothing, and the
// old stays, said.
func (pq *publishQueue) first(ctx context.Context, items *publishItems, module, version string) (string, error) {
qs, st, err := pq.itemsOf(ctx, module, version, versions.PublishClassStart)
if err != nil {
return "", fmt.Errorf("the work queue %s cannot be read: %w", versions.PublishesLocal, err)
}
if len(qs) == 0 {
return "", fmt.Errorf("the work queue %s holds no again item of %s %s", versions.PublishesLocal, module, version)
}
var moved, kept []string
for subj, q := range qs {
q.Class = versions.PublishClassLive
dup, err := pq.put(ctx, q)
if err != nil {
return "", fmt.Errorf("%s %s is not put again as live at %s: %w", module, version, q.Holder, err)
}
if dup {
kept = append(kept, q.Holder)
continue
}
if err := st.Purge(ctx, jetstream.WithPurgeSubject(subj)); err != nil {
return "", fmt.Errorf("the again item of %s %s at %s is not terminated: %w", module, version, q.Holder, err)
}
moved = append(moved, q.Holder)
}
for _, h := range items.settle(module, version, func(h heldItem) bool { return h.q.Class == versions.PublishClassStart }) {
_ = h.m.Term()
}
sort.Strings(moved)
answer := fmt.Sprintf("%s %s is put first as live at %s, its again item terminated", module, version, strings.Join(moved, ", "))
if len(kept) > 0 {
sort.Strings(kept)
answer += fmt.Sprintf("; at %s the work queue holds its live id within the duplicates window, so its again item stays", strings.Join(kept, ", "))
}
return answer, nil
}
// endItem writes the version's publish item ended with answer, made when none stands: Try and take then leave it, and a start
// asks it no more.
func (p *publishItems) endItem(ctx context.Context, module, version, answer string) error {
@@ -440,3 +440,17 @@ func TestAPublishIsNotAskedAgainWhileItsPublishedMayBeUnheard(t *testing.T) {
})
}
}
// Design 47 H2 H-A-483 (third addendum): a holder is a module name, so one ending in -live or -again, the class durables of an
// earlier release, is refused at the learn, said, and nothing of it is pulled.
func TestAHolderNamedLikeAClassDurableIsRefused(t *testing.T) {
for _, holder := range []string{"oci-live", "oci-again"} {
var said []string
pq := &publishQueue{logf: func(f string, a ...any) { said = append(said, fmt.Sprintf(f, a...)) },
binding: func() (publishesStream, error) { return publishesStream{}, errors.New("no bus in a unit test") }}
pq.learn(context.Background(), holder)
if len(said) != 1 || !strings.Contains(said[0], "the publishes of "+holder+" are refused") || pq.pulling[holder] {
t.Errorf("%s: said %q, pulling %v, want it refused at the learn, said once, nothing pulled", holder, said, pq.pulling[holder])
}
}
}
@@ -65,7 +65,7 @@ func reAsked(ctx context.Context, b askAgainer, acts versions.Acts, items *publi
released(ctx, b, kv, e, r, logf)
return
}
if r.Mode == build.ModeRepublish || r.Mode == build.ModePublishDrop || r.Mode == build.ModePublishFirst || r.Mode == build.ModeFloorAgain || r.Mode == build.ModeOldFormDrop || r.Mode == build.ModeReservedDrop || r.Mode == build.ModeReserveDone {
if r.Mode == build.ModeRepublish || r.Mode == build.ModePublishDrop || r.Mode == build.ModeFloorAgain || r.Mode == build.ModeOldFormDrop || r.Mode == build.ModeReservedDrop || r.Mode == build.ModeReserveDone {
// A version's hand act (design 51 *Tools of the builder*) may build again, so it is answered beside the watch.
go versionActed(ctx, acts, kv, e, r, logf)
return
@@ -208,11 +208,8 @@ func versionActed(ctx context.Context, acts versions.Acts, kv jetstream.KeyValue
r.Answer = "queued " + fmt.Sprint(len(r.Queued)) + " versions to publish again: " + strings.Join(r.Queued, ", ") + "; builder_publish_queue shows them"
}
}
} else if r.Mode == build.ModePublishDrop || r.Mode == build.ModePublishFirst {
} else if r.Mode == build.ModePublishDrop {
act := acts.PublishDrop
if r.Mode == build.ModePublishFirst {
act = acts.PublishFirst
}
if act == nil {
err = errors.New("this builder keeps no work queue of publishes")
} else {
@@ -59,6 +59,9 @@ type reservations struct {
withdraw func(module, fingerprint, why string)
// call asks a seat's verb: the git seat's tag (design 47 G §370).
call toolCaller
// pause is the publish pause a version-released whose tag was not written is handed back with (H-A-484 addendum); zero hands
// it back at once.
pause time.Duration
// rebuild builds a done key of module at fp again (the build-again path, A-490; H-A-279); nil builds none.
rebuild func(ctx context.Context, module, fp, why string) error
// say says package-published on the seat mesh-builds; nil says none.
@@ -310,32 +313,20 @@ func (rv *reservations) heard(ctx context.Context, m jetstream.Msg) bool {
rv.logf("%s holds what is no version-released, left: %v", m.Subject(), err)
break
}
if err := rv.tagged(ctx, ev); err != nil {
if refused, err := rv.release(ctx, ev); err != nil {
rv.logf("%v; heard again", err)
if nakErr := m.Nak(); nakErr != nil {
// A tag the seat git's holder did not write is handed back with the publish pause, so the deliveries the consumer
// allows span the holder's outage (design 47 H2 H-A-484 addendum).
nak := m.Nak
if refused && rv.pause > 0 {
nak = func() error { return m.NakWithDelay(rv.pause) }
}
if nakErr := nak(); nakErr != nil {
rv.logf("leaving %s for the bus to give again: %v", m.Subject(), nakErr)
}
givenUp(m, "the version-released of "+ev.Module+" "+ev.Version, err, rv.logf)
return true
}
gone, err := rv.r.Released(ctx, ev.Module, ev.Version)
if err != nil {
rv.logf("%v; heard again", err)
if nakErr := m.Nak(); nakErr != nil {
rv.logf("leaving %s for the bus to give again: %v", m.Subject(), nakErr)
}
givenUp(m, "the version-released of "+ev.Module+" "+ev.Version, err, rv.logf)
return true
}
for _, k := range gone {
rv.dropBound(ctx, k)
}
if rv.released != nil {
rv.released(ctx, ev.Module, ev.Version)
}
rv.found.clear(ctx, versions.VersionUnreleased, meshconditionsclient.ScopeModule, ev.Module)
rv.found.clear(ctx, versions.VersionHeldElsewhere, meshconditionsclient.ScopeModule, ev.Module)
rv.found.clear(ctx, versions.VersionTaggedElsewhere, meshconditionsclient.ScopeModule, ev.Module)
default:
return false
}
@@ -345,6 +336,62 @@ func (rv *reservations) heard(ctx context.Context, m jetstream.Msg) bool {
return true
}
// release takes a released version in as the catalog's version-released says it (design 47 G §369, §370): its tag written, its
// reservation released, its bound and publish item ended and its findings cleared. refused says the tag was not written (the
// seat git did not write it, or the reservation could not be read to tag it); the reservation then stands.
func (rv *reservations) release(ctx context.Context, ev meshcatalog.VersionReleasedEvent) (refused bool, err error) {
if err := rv.tagged(ctx, ev); err != nil {
return true, err
}
gone, err := rv.r.Released(ctx, ev.Module, ev.Version)
if err != nil {
return false, err
}
for _, k := range gone {
rv.dropBound(ctx, k)
}
if rv.released != nil {
rv.released(ctx, ev.Module, ev.Version)
}
rv.found.clear(ctx, versions.VersionUnreleased, meshconditionsclient.ScopeModule, ev.Module)
rv.found.clear(ctx, versions.VersionHeldElsewhere, meshconditionsclient.ScopeModule, ev.Module)
rv.found.clear(ctx, versions.VersionTaggedElsewhere, meshconditionsclient.ScopeModule, ev.Module)
return false, nil
}
// releasedAtStart takes in, at the builder's start, every built, unreleased reservation whose version the catalog's verb versions
// holds released, as its version-released would (design 47 H2 H-A-484 addendum): a version-released the bus gave up on, or one
// said while the builder was away, is taken all the same, its tag written, its reservation released and its publish item ended.
// A module whose versions cannot be read, and a release that is not taken, are said and left to its next version-released.
func (rv *reservations) releasedAtStart(ctx context.Context) {
all, err := rv.r.All(ctx)
if err != nil {
rv.logf("the reservations are not held against the catalog's released versions at the start: %v", err)
return
}
read := map[string][]string{}
for _, v := range builtOf(all) {
if versions.NotStanding(v, true) != "" {
continue
}
released, ok := read[v.Module]
if !ok {
if released, err = rv.r.ReleasedOf(ctx, v.Module); err != nil {
rv.logf("%s: the catalog's released versions cannot be read at the start, so its built reservations wait for its next version-released: %v", v.Module, err)
}
read[v.Module] = released
}
if !slices.Contains(released, v.Version) {
continue
}
if _, err := rv.release(ctx, meshcatalog.VersionReleasedEvent{Module: v.Module, Version: v.Version}); err != nil {
rv.logf("%s %s is released in the catalog and not taken in at the start: %v", v.Module, v.Version, err)
continue
}
rv.logf("%s %s is released in the catalog: taken in at the start, as its version-released", v.Module, v.Version)
}
}
// published marks the artifacts of the reservation a registry seat's published names (its version and digest). inProgress is told
// before a pass that may read many reservations, so the handler says in progress to the bus (design 47 H H-A-280); nil is told
// nothing.
@@ -164,14 +164,15 @@ func TestAReleasedVersionIsTaggedAtTheCommitItWasBuiltFrom(t *testing.T) {
ctx := context.Background()
var tags []git.TagInput
refuse := ""
var standing []git.Tag // the tags the seat git's tags answers (design 47 H2 §796)
var standing []git.Tag // the tags the seat git's tags answers (design 47 H2 §796)
var catalog []meshcatalog.VersionEntry // the released versions the catalog's verb versions answers
call := func(addr string, body any) (json.RawMessage, error) {
switch addr {
case "seat:git.tags":
return json.Marshal(map[string]any{"form": 2, "tags": standing, "a-field-of-a-later-form": true})
case "seat:git.tag":
default:
return json.Marshal(meshcatalog.VersionsAnswer{})
return json.Marshal(meshcatalog.VersionsAnswer{Versions: catalog})
}
tags = append(tags, body.(git.TagInput))
return json.Marshal(git.TagAnswer{Performed: refuse == "", Error: refuse})
@@ -203,11 +204,44 @@ func TestAReleasedVersionIsTaggedAtTheCommitItWasBuiltFrom(t *testing.T) {
if m := release("pkg", "0.1.0"); !m.naked || m.acked {
t.Fatalf("a tag the forge did not write: naked %v acked %v; want the release heard again", m.naked, m.acked)
}
// Design 47 H2 H-A-484 addendum: with a publish pause set, a refused tag hands the version-released back after the pause, so
// the deliveries its consumer allows span the holder's outage.
rv.pause = 30 * time.Second
if m := release("pkg", "0.1.0"); m.delayed != 30*time.Second || m.naked || m.acked {
t.Fatalf("a tag the forge did not write, with the pause: delayed %v naked %v acked %v; want handed back after 30s", m.delayed, m.naked, m.acked)
}
rv.pause = 0
if all, _ := rv.r.All(ctx); len(all) != 1 {
t.Fatalf("the reservation went with an unwritten tag: %v", all)
}
// Design 47 H2 H-A-484 addendum: at the builder's start, a built reservation the catalog holds released is tagged and released
// as its version-released would (a version-released the bus gave up on), and one the catalog does not hold, or not built,
// stands.
refuse = ""
tags = nil
var ended []string
rv.released = func(_ context.Context, module, version string) { ended = append(ended, module+" "+version) }
rv.releasedAtStart(ctx) // built nothing yet: nothing taken
if all, _ := rv.r.All(ctx); len(all) != 1 || len(tags) != 0 {
t.Fatalf("at the start with no build done: reservations %v, tags %+v; want the reservation standing, no tag", all, tags)
}
if _, err := rv.r.Built(ctx, "pkg", "0.1.0", "pkg:lib", "sha256:aa", nil); err != nil {
t.Fatal(err)
}
rv.releasedAtStart(ctx) // the catalog holds none released: it stands
if all, _ := rv.r.All(ctx); len(all) != 1 || len(tags) != 0 {
t.Fatalf("at the start with nothing released: reservations %v, tags %+v; want the reservation standing", all, tags)
}
catalog = []meshcatalog.VersionEntry{{Module: "pkg", Version: "0.1.0"}}
rv.releasedAtStart(ctx)
if all, _ := rv.r.All(ctx); len(all) != 0 || len(tags) != 1 || tags[0] != (git.TagInput{Repository: "o/r", Commit: "c0ffee", Tag: "go/v0.1.0"}) || !slices.Equal(ended, []string{"pkg 0.1.0"}) {
t.Fatalf("at the start with 0.1.0 released: reservations %v, tags %+v, items ended %v; want it tagged, released and its item ended", all, tags, ended)
}
catalog, rv.released = nil, nil
if _, err := rv.Reserve(ctx, "pkg", "0.1", []string{"pkg:lib"}, goPkg); err != nil { // the version is reserved again for the rows below
t.Fatal(err)
}
tags = nil
if m := release("pkg", "0.1.0"); !m.acked || len(tags) != 1 || tags[0] != (git.TagInput{Repository: "o/r", Commit: "c0ffee", Tag: "go/v0.1.0"}) {
t.Fatalf("the release: acked %v, tags %+v; want go/v0.1.0 at c0ffee of o/r", m.acked, tags)
}
@@ -74,14 +74,11 @@ const ModePublish = "publish"
// optional, name what to publish again; the builder answers with the versions it queued (Queued).
const ModeRepublish = "republish"
// ModePublishDrop and ModePublishFirst are the Modes of a reasks value of builder_publish_drop and builder_publish_first (design
// 47 H2 §793): Module and Version name the item; the builder terminates it, or puts it again as live and terminates the old.
const (
ModePublishDrop = "publish-drop"
ModePublishFirst = "publish-first"
)
// ModePublishDrop is the Mode of a reasks value of builder_publish_drop (design 47 H2 §793): Module and Version name the item;
// the builder terminates it.
const ModePublishDrop = "publish-drop"
// NewPublishAct is the value builder_publish_drop (mode ModePublishDrop) or builder_publish_first (ModePublishFirst) writes
// NewPublishAct is the value builder_publish_drop (mode ModePublishDrop) writes
// (§793): its key, `<mode>.<unix nanoseconds>` (a call is one value), and the value. Module, version and why are required.
func NewPublishAct(mode, module, version, why, by string, at time.Time) (string, []byte, error) {
module, version = strings.TrimSpace(module), strings.TrimSpace(version)
@@ -27,10 +27,8 @@ type Acts struct {
ReserveDone func(ctx context.Context, module, fp, by, why string, at time.Time) (string, error)
// DropReserved drops a module's reservation of one MAJOR.MINOR by hand (builder_reserved_drop): its answer, or why not.
DropReserved func(ctx context.Context, module, series, by, why string, at time.Time) (string, error)
// PublishDrop terminates one publish item, and PublishFirst puts it again as live and terminates the old (builder_publish_drop,
// builder_publish_first, design 47 H2 §793): the answer, or why not.
PublishDrop func(ctx context.Context, module, version, by, why string) (string, error)
PublishFirst func(ctx context.Context, module, version, by, why string) (string, error)
// PublishDrop terminates one publish item (builder_publish_drop, design 47 H2 §793): the answer, or why not.
PublishDrop func(ctx context.Context, module, version, by, why string) (string, error)
}
// FloorAgain is builder_floor_again of name: refused, since the builder numbers nothing of its own and no claim asks a floor
@@ -37,9 +37,8 @@ const (
PublishQueueTool = "builder_publish_queue"
// QueueFull is what version-unreleased says of a version the work queue's cap refused.
QueueFull = "queue full"
// PublishDropTool and PublishFirstTool are builder_publish_drop and builder_publish_first (§793).
PublishDropTool = "builder_publish_drop"
PublishFirstTool = "builder_publish_first"
// PublishDropTool is builder_publish_drop (§793); builder_publish_first is retired (H-A-483, third addendum).
PublishDropTool = "builder_publish_drop"
// NotStoredWords opens what a refused artifact is said as.
NotStoredWords = "not stored: "
// VerbPublish is the registry seats' verb a publish again sends, VerbVersions the one whose record says what a seat holds
@@ -48,7 +47,7 @@ const (
VerbVersions = "versions"
// PublishClassLive is the class of a publish whose build ended in this run, which a pipeline's build step waits on;
// PublishClassStart that of a publish asked again at a start, of a build that ended before it. A holder's items are pulled
// live first, each class oldest first (design 47 H H-A-290 (1), H2 §793).
// by its one consumer in the order they are put (design 47 H H-A-290 (1), H2 §793, H-A-483).
PublishClassLive = "live"
PublishClassStart = "again"
// PublishSendsAtOnce bounds the sends of one holder that run at once, its consumers' max ack pending (H-A-290 (3), §793).
@@ -97,21 +96,19 @@ type QueuedPublish struct {
Class string `json:"class,omitempty"`
}
// rank is the place of q's class in a holder's queue: the live first.
func (q QueuedPublish) rank() int {
if q.Class == PublishClassLive {
return 0
}
return 1
}
// sortQueued sorts a holder's queued versions in the order they are sent: the live class first, each class oldest first.
func sortQueued(qs []QueuedPublish) {
sort.SliceStable(qs, func(i, j int) bool {
if a, b := qs[i].rank(), qs[j].rank(); a != b {
// streamOrder sorts the keys of the work queue's items in stream order: each key is its item's stream sequence (the builder's
// reader, publishitem.go), compared as a number; a key that is no number sorts after them by its words.
func streamOrder(keys []string) {
sort.SliceStable(keys, func(i, j int) bool {
a, aerr := strconv.ParseUint(keys[i], 10, 64)
b, berr := strconv.ParseUint(keys[j], 10, 64)
switch {
case aerr == nil && berr == nil:
return a < b
case aerr == nil || berr == nil:
return aerr == nil
}
return qs[i].Queued.Before(qs[j].Queued)
return keys[i] < keys[j]
})
}
@@ -456,7 +453,7 @@ func (r *Republisher) Republish(ctx context.Context, module, version string) ([]
type Queued struct {
Total int `json:"total"`
Counts []QueueCount `json:"counts"`
// Queue are the first queued versions, per holder in the order they are sent (the live class first, each class oldest first);
// Queue are the first queued versions, in stream order, the order each holder's one consumer gives them (H-A-483);
// with a module named, only that module's.
Queue []QueuedPublish `json:"queue"`
// Listed says how many of Total the answer lists.
@@ -512,7 +509,8 @@ func PublishItemKey(module, version string) string {
// PublishQueueOf is the seat mesh-builds' verb publish-queue (design 47 H2 §793; H-A-454): the builder's read of its own work queue
// publishes (queue, its items read in bulk, capped past its bound), small at any size of the queue (H-A-290 (5)): the items' count
// per holder and class with the oldest and newest queue time, and the first PublishQueueListCap in the order they are sent; module
// per holder and class with the oldest and newest queue time, and the first PublishQueueListCap in stream order, the order the
// holder's one consumer gives them (H-A-483, third addendum); module
// (and version) narrow the list to one module's (one version's), so an item's class can be read.
func PublishQueueOf(queue Reader, module, version string) (meshbuilds.PublishQueueAnswer, error) {
out := meshbuilds.PublishQueueAnswer{Counts: []meshbuilds.PublishQueueAnswerCountsItem{}, Queue: []meshbuilds.PublishQueueAnswerQueueItem{}}
@@ -525,7 +523,13 @@ func PublishQueueOf(queue Reader, module, version string) (meshbuilds.PublishQue
}
counts := map[[2]string]*meshbuilds.PublishQueueAnswerCountsItem{}
var listed []QueuedPublish
for _, raw := range vals {
keys := make([]string, 0, len(vals))
for k := range vals {
keys = append(keys, k)
}
streamOrder(keys)
for _, k := range keys {
raw := vals[k]
var q QueuedPublish
if json.Unmarshal(raw, &q) != nil {
continue
@@ -559,25 +563,12 @@ func PublishQueueOf(queue Reader, module, version string) (meshbuilds.PublishQue
}
return out.Counts[i].Class < out.Counts[j].Class
})
sort.SliceStable(listed, func(i, j int) bool { return listed[i].Holder < listed[j].Holder })
byHolder := map[string][]QueuedPublish{}
var holders []string
for _, q := range listed {
if byHolder[q.Holder] == nil {
holders = append(holders, q.Holder)
}
byHolder[q.Holder] = append(byHolder[q.Holder], q)
}
for _, h := range holders {
qs := byHolder[h]
sortQueued(qs)
for _, q := range qs {
if len(out.Queue) == PublishQueueListCap {
break
}
out.Queue = append(out.Queue, meshbuilds.PublishQueueAnswerQueueItem{Holder: q.Holder, Module: q.Module, Version: q.Version,
Class: q.Class, Artifacts: q.Artifacts, Queued: q.Queued})
if len(out.Queue) == PublishQueueListCap {
break
}
out.Queue = append(out.Queue, meshbuilds.PublishQueueAnswerQueueItem{Holder: q.Holder, Module: q.Module, Version: q.Version,
Class: q.Class, Artifacts: q.Artifacts, Queued: q.Queued})
}
out.Listed = int64(len(out.Queue))
return out, nil
@@ -631,20 +622,17 @@ func AnswerQueue(q meshbuilds.PublishQueueAnswer, acted, reasks Reader) (Queued,
// PublishQueueDescription is builder_publish_queue's description.
const PublishQueueDescription = "The versions the builder will publish again (its work queue " + PublishesLocal + ", design 47 G section 588, H2 section 793): " +
"how many each registry seat holder's items hold per class (live, a build a pipeline's step waits on, pulled first, or again, asked again at a start) " +
"with the oldest and newest queue time; the first 50 in the order they are pulled, each with its class, its artifacts' digests and when it was " +
"how many each registry seat holder's items hold per class (live, a build a pipeline's step waits on, or again, asked again at a start) " +
"with the oldest and newest queue time; the first 50 in stream order, the order each holder's one consumer pulls them, each with its class, its artifacts' digests and when it was " +
"queued (module, and version, list only that module's, so one item's class can be read); the keys of the state " + ActedState + ", the last stream " +
"sequence the builder acted on for each subject; and the publish items (design 47 G A-457a), counted, the first 50 by next try, each version's " +
"tries and next try, and why one ended (its tries spent, or dropped). A read past its pages is answered capped, naming the items read and the " +
"queue's size (H-A-293). Read-only."
// PublishDropDescription and PublishFirstDescription are builder_publish_drop's and builder_publish_first's (§793).
// PublishDropDescription is builder_publish_drop's (§793).
const PublishDropDescription = "Drop one version's publish (design 47 H2 section 793): its items on the work queue " + PublishesLocal + " are " +
"terminated at every holder, and its publish item ended with who dropped it and why, so it is never sent or put again; builder_publish_again " +
"asks it anew. A hand act, recorded in the hand-act log first."
const PublishFirstDescription = "Put one version's publish first (design 47 H2 section 793): each of its again items on the work queue " +
PublishesLocal + " is put again as live, under the message id <holder>.live.<module>.<version>, pulled before every again item, and the old " +
"terminated. A hand act, recorded in the hand-act log first."
// PublishAgainDescription is builder_publish_again's description.
const PublishAgainDescription = "Publish reserved versions again (design 47 G section 588), which builds nothing: with module and version, that reserved " +
@@ -7,6 +7,7 @@ import (
"fmt"
"maps"
"slices"
"strconv"
"strings"
"sync"
"testing"
@@ -404,8 +405,9 @@ func answerOf(queue, acted, reasks Reader, module, version string) (Queued, erro
}
// TestThePublishQueueToolAnswersAtAnySize: design 47 H H-A-290 (5) — builder_publish_queue over two thousand queued versions answers
// within a second and small: counts per holder and class with the oldest and newest queue time, a list capped at fifty in the
// order they are sent, and one module's item with its class; each state read in bulk where the reader can.
// within a second and small: counts per holder and class with the oldest and newest queue time, a list capped at fifty in stream
// order (H-A-483, third addendum: each key is its item's stream sequence), and one module's item with its class; each state read
// in bulk where the reader can.
func TestThePublishQueueToolAnswersAtAnySize(t *testing.T) {
at := time.Date(2026, 11, 20, 17, 0, 0, 0, time.UTC)
vals := map[string]json.RawMessage{}
@@ -418,7 +420,7 @@ func TestThePublishQueueToolAnswersAtAnySize(t *testing.T) {
class = PublishClassLive
}
q := QueuedPublish{V: 1, Holder: holder, Module: fmt.Sprintf("m%d", i), Version: "0.1.0", Artifacts: map[string]string{"a": "sha256:aa"}, Queued: at.Add(time.Duration(i) * time.Second), Class: class}
vals[QueueKey(holder, q.Module, q.Version)], _ = json.Marshal(q)
vals[strconv.Itoa(i+1)], _ = json.Marshal(q) // the reader keys each item by its stream sequence
}
gets := 0
queue := Reader{All: func() (StatePage, error) { return StatePage{Values: vals}, nil },
@@ -435,8 +437,10 @@ func TestThePublishQueueToolAnswersAtAnySize(t *testing.T) {
if fmt.Sprintln(a.Counts[1].Holder, a.Counts[1].Class, a.Counts[1].Count, a.Counts[1].Oldest.Equal(at.Add(1991*time.Second)), a.Counts[1].Newest.Equal(at.Add(1999*time.Second))) != "go "+PublishClassLive+" 5 true true\n" || len(a.Counts) != 4 {
t.Errorf("counts %+v, want per holder and class with their oldest and newest", a.Counts)
}
if a.Queue[0].Class != PublishClassLive || a.Queue[0].Holder != "go" {
t.Errorf("the list opens with %+v, want the go holder's live class, sent first", a.Queue[0])
for i, q := range a.Queue {
if q.Module != fmt.Sprintf("m%d", i) {
t.Fatalf("the list holds %s at %d, want m%d: the items in stream order, whatever their holder or class", q.Module, i, i)
}
}
one, err := answerOf(queue, none, none, "m1995", "")
if err != nil || len(one.Queue) != 1 || one.Queue[0].Class != PublishClassLive || one.Total != 2000 {
@@ -57,7 +57,7 @@ func VerbDescription(verb string) string {
case VerbDrop:
return "End an ask on the seat's work queue for a caller that no longer waits for it (its step ended by its bound): one waiting is ended dropped when it is pulled, one taken is cancelled; both end the step could-not-run, its why naming the drop and who asked it, so no holder works on it after (design 47 H2 §778; H-A-401; ADR 0664 §4)."
case VerbPublishQueue:
return "What the builder's work queue publishes holds, read by the builder from its own stream (design 47 H2 §793; H-A-454): the items' count per registry seat holder and class with the oldest and newest queue time, and the first 50 in the order they are pulled (the live class first, each class oldest first); module, and version, list only that module's items. A queue read past 4,000 items is answered capped, naming the items read and the queue's size."
return "What the builder's work queue publishes holds, read by the builder from its own stream (design 47 H2 §793; H-A-454): the items' count per registry seat holder and class with the oldest and newest queue time, and the first 50 in stream order, the order each holder's one consumer pulls them (H-A-483); module, and version, list only that module's items. A queue read past 4,000 items is answered capped, naming the items read and the queue's size."
}
return ""
}
@@ -338,7 +338,7 @@ const (
type PublishQueueAnswerQueueItem struct {
// each artifact reference, <module>:<artifact>, with the digest to publish
Artifacts map[string]string `json:"artifacts,omitempty"`
// live, a build a pipeline's step waits on, pulled first; again, asked again at a start
// live, a build a pipeline's step waits on; again, asked again at a start
Class string `json:"class"`
// the registry seat's holder the item is sent to, a module
Holder string `json:"holder"`
@@ -368,7 +368,7 @@ type PublishQueueAnswer struct {
ItemsRead int64 `json:"items-read,omitempty"`
// how many of total the answer lists
Listed int64 `json:"listed"`
// the first 50 items listed, per holder in the order they are pulled
// the first 50 items listed, in stream order
Queue []PublishQueueAnswerQueueItem `json:"queue"`
// with capped: the queue's size in bytes
QueueBytes int64 `json:"queue-bytes,omitempty"`
@@ -14,13 +14,13 @@ const Seat = "mesh-pipeline-runner"
// Form is the form of the seat's values, events and answers this contract was generated at (design 47 H2 §780):
// a reader of a value of another form says "form N, this reader knows M".
const Form = 6
const Form = 8
// The forms and the digest of the fields this contract was generated from: the generator's compare fails a definition
// whose fields changed while its forms did not (design 47 H2 §780).
const (
generatedForms = "6"
generatedFieldsDigest = "AcceptAnswer=224df0394653 Accepted=d2b011e5090f AskAgainAnswer=224df0394653 CancelAnswer=a0dcc32b3f57 EndAgainAnswer=a0dcc32b3f57 FailedStep=cd7184342136 Fill=7a76c9ceb157 FillingEvent=5bd4df78ed84 FullEvent=5bd4df78ed84 KeptAnswer=dfe0d48bc169 KeptByLevelDigestValue=f06c90f7da7f KeptResult=af5f81f189b7 ListAnswer=284210b3c054 LogEntry=b059967b2ae4 Member=8a103baec01f Mover=5db376275bcf MoverEnded=4b35095f6829 MoverRow=b61f41d131a3 Pipeline=2663792125bf PipelineEnded=523f8fc0fe41 PipelineEndedEvent=cb44007f7173 PipelineEndedPhases=9d1c33ce4cc4 PipelineEndedPhasesBuild=a08d49692a09 PipelineEndedPhasesComposition=a08d49692a09 PipelineEndedPhasesIntegration=a08d49692a09 PipelineEndedPhasesLint=a08d49692a09 PipelineEndedPhasesUnit=a08d49692a09 PipelineEndedPhasesValidate=a08d49692a09 PipelineRow=64600be410e9 PipelineStarted=a3d000f095db PipelineStartedEvent=eaa401af4117 PipelinesByBranchRevisionUnfinishedModuleValue=693580eed498 PipelinesByBranchRevisionValue=693580eed498 PipelinesByBranchRunRunValue=693580eed498 PipelinesByBranchSourcePublishedPackageVersionValue=693580eed498 PipelinesByStepsPipelineMoverValue=94de99db7816 PlaceAnswer=a0dcc32b3f57 PlaceUnfinishedAnswer=ed94ced4513a Published=d9af3024dacc QueueByQueueNextValue=765a2918a728 QueueByRunningPipelineValue=ed94ced4513a QueueByWaitingSequenceValue=99605298ae13 RePinnedEvent=92fd04bfa6d3 RerunAnswer=9b7efd0df344 ReviewAnswer=f397b8965179 RoomEvent=5bd4df78ed84 RunAnswer=96bd45aa2e62 RunStarted=b280617410c3 RunStartedEvent=0767a9358fa3 Shared=c981cf6aa860 SharedAskedBy=c9b30cc4d0d0 ShowAnswer=48b8cecc529a StandsAnswer=0afa3693bf3b StartAnswer=ed94ced4513a StepEndedEvent=799020642c19 StepEvent=64f18c40ed5d StepLogByPipelineValue=ffdde090a101 StepResult=0dac0e15d017 StepStartedEvent=799020642c19 TestCounts=9c1332427159 TestRun=ca5a87f3b6b8 TestRunPackage=a7d30558470d TestRunReader=c9b30cc4d0d0 TestRunsAnswer=7ab81a45ca33 TestRunsDropAnswer=8ee70b35469d ToolchainUsed=43b6d42376fb Wait=21e4ee7a3ef1 WaitEndedEvent=2bf6d38041c0 WaitHeldEvent=2bf6d38041c0 Waiting=ed94ced4513a WhyCarriedAnswer=8e2fcea6479f Widening=e2790a307466"
generatedForms = "8"
generatedFieldsDigest = "AcceptAnswer=224df0394653 Accepted=d2b011e5090f AskAgainAnswer=224df0394653 CancelAnswer=a0dcc32b3f57 EndAgainAnswer=a0dcc32b3f57 FailedStep=cd7184342136 Fill=7a76c9ceb157 FillingEvent=5bd4df78ed84 FullEvent=5bd4df78ed84 KeptAnswer=dfe0d48bc169 KeptByLevelDigestValue=f06c90f7da7f KeptResult=af5f81f189b7 ListAnswer=284210b3c054 LogEntry=b059967b2ae4 Member=8a103baec01f Mover=5db376275bcf MoverEnded=4b35095f6829 MoverRow=b61f41d131a3 Pipeline=2663792125bf PipelineEnded=523f8fc0fe41 PipelineEndedEvent=cb44007f7173 PipelineEndedPhases=9d1c33ce4cc4 PipelineEndedPhasesBuild=a08d49692a09 PipelineEndedPhasesComposition=a08d49692a09 PipelineEndedPhasesIntegration=a08d49692a09 PipelineEndedPhasesLint=a08d49692a09 PipelineEndedPhasesUnit=a08d49692a09 PipelineEndedPhasesValidate=a08d49692a09 PipelineRow=f58dbdcaf2d4 PipelineStarted=a3d000f095db PipelineStartedEvent=eaa401af4117 PipelinesByBranchRevisionAgainNValue=693580eed498 PipelinesByBranchRevisionUnfinishedModuleValue=693580eed498 PipelinesByBranchRevisionValue=693580eed498 PipelinesByBranchRunRunValue=693580eed498 PipelinesByBranchSourcePublishedPackageVersionValue=693580eed498 PipelinesByStepsPipelineMoverValue=94de99db7816 PlaceAnswer=a0dcc32b3f57 PlaceUnfinishedAnswer=ed94ced4513a Published=d9af3024dacc QueueByQueueNextValue=765a2918a728 QueueByRunningPipelineValue=ed94ced4513a QueueByWaitingSequenceValue=99605298ae13 RePinnedEvent=92fd04bfa6d3 RerunAnswer=9b7efd0df344 ReviewAnswer=f397b8965179 RoomEvent=5bd4df78ed84 RunAnswer=96bd45aa2e62 RunStarted=b280617410c3 RunStartedEvent=0767a9358fa3 Shared=c981cf6aa860 SharedAskedBy=c9b30cc4d0d0 ShowAnswer=48b8cecc529a StandsAnswer=0afa3693bf3b StartAnswer=ed94ced4513a StepEndedEvent=799020642c19 StepEvent=64f18c40ed5d StepLogByPipelineValue=ffdde090a101 StepResult=0dac0e15d017 StepStartedEvent=799020642c19 TestCounts=9c1332427159 TestRun=ca5a87f3b6b8 TestRunPackage=a7d30558470d TestRunReader=c9b30cc4d0d0 TestRunsAnswer=7ab81a45ca33 TestRunsDropAnswer=8ee70b35469d ToolchainUsed=43b6d42376fb Wait=21e4ee7a3ef1 WaitEndedEvent=2bf6d38041c0 WaitHeldEvent=2bf6d38041c0 Waiting=ed94ced4513a WhyCarriedAnswer=8e2fcea6479f Widening=e2790a307466"
)
const (
@@ -69,11 +69,11 @@ func VerbDescription(verb string) string {
case VerbRun:
return "The run on request of one or more members, each a module at a named commit, through their pipelines; delivers nothing, replacing the checker's try (ADR 0666 section 18). Says run-started."
case VerbStart:
return "A delivering pipeline for a watched branch's head, by hand; written in the hand-act log."
return "A delivering pipeline for a watched branch's head, by hand; written in the hand-act log. With modules, a pipeline at the head keyed <branch>.<revision>.again.<n> carrying the named modules whose newest entry has no passed pipeline, at the take-in's tiers; a module whose newest entry passed is refused by name. Exactly one of module or modules (design 47 H2 H-A-479)."
case VerbShow:
return "One pipeline: naming a mover, its steps alone; with no mover, each mover's row (its module, outcome and failed step), never every step of every mover, or the steps of any mover the step and state filters hold; and its step log, with the stores' fill (design 47 H2 §789, H-A-376). Reads only."
case VerbList:
return "The pipelines and runs of a module, a branch or a commit, newest first: by default one short row per pipeline, the newest 20 (its key, call, start, end, outcome, movers' count and first failed or could-not-run step with its module and why); the whole pipelines, each record with its movers and steps counted, never its steps (show gives those), only with full and a limit (design 47 H2 H-A-376). Reads only."
return "The pipelines and runs of a module, a branch or a commit, newest first: by default one short row per pipeline, the newest 20 (its key, call, source, revision, start, end, outcome, movers' count, its steps counted per outcome and first failed or could-not-run step with its module and why); the whole pipelines, each record with its movers and steps counted, never its steps (show gives those), only with full and a limit (design 47 H2 H-A-376, H-A-467). Reads only."
case VerbStands:
return "What stands still, and since when: each step not ended and each wait held. Reads only."
case VerbWhyCarried:
@@ -354,6 +354,44 @@ func DecodePipelinesByBranchRevisionUnfinishedModuleValue(raw []byte) (Pipelines
return v, formRead(raw), nil
}
// PipelinesByBranchRevisionAgainNKey is the key of one value of the state pipelines under the form "<branch>.<revision>.again.<n>".
func PipelinesByBranchRevisionAgainNKey(branch, revision, n string) string {
return branch + "." + revision + ".again." + n
}
// ParsePipelinesByBranchRevisionAgainNKey reads the parts of a key of the form "<branch>.<revision>.again.<n>", in the order of the form; the last result is false for a key of another form.
func ParsePipelinesByBranchRevisionAgainNKey(key string) (string, string, string, bool) {
rest := key
v0, after0, found0 := strings.Cut(rest, ".")
if !found0 || v0 == "" {
return "", "", "", false
}
rest = after0
v2, after2, found2 := strings.Cut(rest, ".again.")
if !found2 || v2 == "" {
return "", "", "", false
}
rest = after2
if rest == "" {
return "", "", "", false
}
v4 := rest
rest = ""
if rest != "" {
return "", "", "", false
}
return v0, v2, v4, true
}
// DecodePipelinesByBranchRevisionAgainNValue reads one value of the state pipelines under the form "<branch>.<revision>.again.<n>" as another part wrote it: a field PipelinesByBranchRevisionAgainNValue does not know is left unread, never refused; the form read is answered, 1 when absent (design 47 H2 §780).
func DecodePipelinesByBranchRevisionAgainNValue(raw []byte) (PipelinesByBranchRevisionAgainNValue, int, error) {
var v PipelinesByBranchRevisionAgainNValue
if err := json.Unmarshal(raw, &v); err != nil {
return v, 0, err
}
return v, formRead(raw), nil
}
// PipelinesByStepsPipelineMoverKey is the key of one value of the state pipelines under the form "steps.<pipeline>.<mover>".
func PipelinesByStepsPipelineMoverKey(pipeline, mover string) string {
return "steps." + pipeline + "." + mover
@@ -879,7 +917,7 @@ const (
PipelineEndedOutcomeCancelled = "cancelled"
)
// PipelineRow is one pipeline in list's short form (design 47 H2 H-A-376).
// PipelineRow is one pipeline in list's short form, with its source, revision and its steps counted per outcome (design 47 H2 H-A-376, H-A-467).
type PipelineRow struct {
// the merge call whose source revision the take-in took in
Call string `json:"call,omitempty"`
@@ -889,8 +927,14 @@ type PipelineRow struct {
MoverCount int64 `json:"mover-count"`
Outcome string `json:"outcome"`
// the pipeline's key
Pipeline string `json:"pipeline"`
Started *time.Time `json:"started,omitempty"`
Pipeline string `json:"pipeline"`
// the catalog graph revision of the branch the pipeline runs for (design 47 H2 H-A-467)
Revision int64 `json:"revision,omitempty"`
// the source revision of the watched branch the pipeline runs for, its key (design 47 H2 H-A-467)
Source int64 `json:"source,omitempty"`
Started *time.Time `json:"started,omitempty"`
// how many of the pipeline's steps hold each outcome, by the step's outcome (design 47 H2 H-A-467)
Steps map[string]int64 `json:"steps,omitempty"`
}
// The values of PipelineRow's outcome.
@@ -1238,7 +1282,10 @@ type RunAnswer struct {
// StartInput is what start takes.
type StartInput struct {
Branch string `json:"branch"`
Module string `json:"module"`
// the one mover of the head's pipeline <branch>.<revision>; absent when modules is given
Module string `json:"module,omitempty"`
// the modules started again at the head, each whose newest entry has no passed pipeline (a failed, cut or superseded one); absent when module is given (design 47 H2 H-A-479)
Modules []string `json:"modules,omitempty"`
// why, written with the caller the runtime proves in the hand-act log
Why string `json:"why"`
}
@@ -1631,6 +1678,9 @@ type PipelinesByBranchRunRunValue = Pipeline
// PipelinesByBranchRevisionUnfinishedModuleValue is one value of the state pipelines under the key <branch>.<revision>.unfinished.<module>.
type PipelinesByBranchRevisionUnfinishedModuleValue = Pipeline
// PipelinesByBranchRevisionAgainNValue is one value of the state pipelines under the key <branch>.<revision>.again.<n>.
type PipelinesByBranchRevisionAgainNValue = Pipeline
// PipelinesByStepsPipelineMoverValue is one value of the state pipelines under the key steps.<pipeline>.<mover>.
type PipelinesByStepsPipelineMoverValue struct {
// is the form of what this value carries, left out while it is 1 (design 47 H2 §780).
@@ -156,7 +156,7 @@
{
"name": "publish-queue",
"class": "read",
"description": "What the builder's work queue publishes holds, read by the builder from its own stream (design 47 H2 §793; H-A-454): the items' count per registry seat holder and class with the oldest and newest queue time, and the first 50 in the order they are pulled (the live class first, each class oldest first); module, and version, list only that module's items. A queue read past 4,000 items is answered capped, naming the items read and the queue's size.",
"description": "What the builder's work queue publishes holds, read by the builder from its own stream (design 47 H2 §793; H-A-454): the items' count per registry seat holder and class with the oldest and newest queue time, and the first 50 in stream order, the order each holder's one consumer pulls them (H-A-483); module, and version, list only that module's items. A queue read past 4,000 items is answered capped, naming the items read and the queue's size.",
"input": {
"type": "object",
"properties": {
@@ -243,7 +243,7 @@
"live",
"again"
],
"description": "live, a build a pipeline's step waits on, pulled first; again, asked again at a start"
"description": "live, a build a pipeline's step waits on; again, asked again at a start"
},
"artifacts": {
"type": "object",
@@ -266,7 +266,7 @@
"queued"
]
},
"description": "the first 50 items listed, per holder in the order they are pulled"
"description": "the first 50 items listed, in stream order"
},
"listed": {
"type": "integer",
@@ -50,12 +50,20 @@
},
{
"name": "start",
"description": "A delivering pipeline for a watched branch's head, by hand; written in the hand-act log.",
"description": "A delivering pipeline for a watched branch's head, by hand; written in the hand-act log. With modules, a pipeline at the head keyed <branch>.<revision>.again.<n> carrying the named modules whose newest entry has no passed pipeline, at the take-in's tiers; a module whose newest entry passed is refused by name. Exactly one of module or modules (design 47 H2 H-A-479).",
"input": {
"type": "object",
"properties": {
"module": {
"type": "string"
"type": "string",
"description": "the one mover of the head's pipeline <branch>.<revision>; absent when modules is given"
},
"modules": {
"type": "array",
"items": {
"type": "string"
},
"description": "the modules started again at the head, each whose newest entry has no passed pipeline (a failed, cut or superseded one); absent when module is given (design 47 H2 H-A-479)"
},
"branch": {
"type": "string"
@@ -66,7 +74,6 @@
}
},
"required": [
"module",
"branch",
"why"
]
@@ -171,7 +178,7 @@
},
{
"name": "list",
"description": "The pipelines and runs of a module, a branch or a commit, newest first: by default one short row per pipeline, the newest 20 (its key, call, start, end, outcome, movers' count and first failed or could-not-run step with its module and why); the whole pipelines, each record with its movers and steps counted, never its steps (show gives those), only with full and a limit (design 47 H2 H-A-376). Reads only.",
"description": "The pipelines and runs of a module, a branch or a commit, newest first: by default one short row per pipeline, the newest 20 (its key, call, source, revision, start, end, outcome, movers' count, its steps counted per outcome and first failed or could-not-run step with its module and why); the whole pipelines, each record with its movers and steps counted, never its steps (show gives those), only with full and a limit (design 47 H2 H-A-376, H-A-467). Reads only.",
"input": {
"type": "object",
"properties": {
@@ -910,6 +917,13 @@
},
"description": "a pipeline of its own of one mover kept in the runner's state unfinished, put in it by place-unfinished; the revision the unfinished entry names (design 47 H2 §86, H-A-398)"
},
{
"key": "<branch>.<revision>.again.<n>",
"value": {
"$ref": "#/defs/Pipeline"
},
"description": "a pipeline started again at the head by start with modules, its movers the named modules whose newest entry has no passed pipeline; n from 1, taken by writing on absence (design 47 H2 H-A-479)"
},
{
"key": "steps.<pipeline>.<mover>",
"value": {
@@ -2275,6 +2289,14 @@
"type": "string",
"description": "the merge call whose source revision the take-in took in"
},
"source": {
"type": "integer",
"description": "the source revision of the watched branch the pipeline runs for, its key (design 47 H2 H-A-467)"
},
"revision": {
"type": "integer",
"description": "the catalog graph revision of the branch the pipeline runs for (design 47 H2 H-A-467)"
},
"started": {
"type": "string",
"format": "date-time"
@@ -2301,9 +2323,16 @@
},
"failed": {
"$ref": "#/defs/FailedStep"
},
"steps": {
"type": "object",
"additionalProperties": {
"type": "integer"
},
"description": "how many of the pipeline's steps hold each outcome, by the step's outcome (design 47 H2 H-A-467)"
}
},
"description": "one pipeline in list's short form (design 47 H2 H-A-376)",
"description": "one pipeline in list's short form, with its source, revision and its steps counted per outcome (design 47 H2 H-A-376, H-A-467)",
"required": [
"pipeline",
"outcome",
@@ -2338,5 +2367,5 @@
]
}
},
"form": 6
"form": 8
}
@@ -251,7 +251,7 @@ export interface PublishQueueAnswerCountsItem {
export interface PublishQueueAnswerQueueItem {
/** each artifact reference, <module>:<artifact>, with the digest to publish */
"artifacts"?: Record<string, string>;
/** live, a build a pipeline's step waits on, pulled first; again, asked again at a start */
/** live, a build a pipeline's step waits on; again, asked again at a start */
"class": "live" | "again";
/** the registry seat's holder the item is sent to, a module */
"holder": string;
@@ -275,7 +275,7 @@ export interface PublishQueueAnswer {
"items-read"?: number;
/** how many of total the answer lists */
"listed": number;
/** the first 50 items listed, per holder in the order they are pulled */
/** the first 50 items listed, in stream order */
"queue": PublishQueueAnswerQueueItem[];
/** with capped: the queue's size in bytes */
"queue-bytes"?: number;
@@ -121,6 +121,12 @@ export declare function pipelinesByBranchRevisionUnfinishedModuleKey(branch: str
/** The parts of a key of the form "<branch>.<revision>.unfinished.<module>", or undefined for a key of another form. */
export declare function parsePipelinesByBranchRevisionUnfinishedModuleKey(_key: string): { branch: string; revision: string; module: string } | undefined;
/** The key of one value of the state pipelines under the form "<branch>.<revision>.again.<n>". */
export declare function pipelinesByBranchRevisionAgainNKey(branch: string, revision: string, n: string): string;
/** The parts of a key of the form "<branch>.<revision>.again.<n>", or undefined for a key of another form. */
export declare function parsePipelinesByBranchRevisionAgainNKey(_key: string): { branch: string; revision: string; n: string } | undefined;
/** The key of one value of the state pipelines under the form "steps.<pipeline>.<mover>". */
export declare function pipelinesByStepsPipelineMoverKey(pipeline: string, mover: string): string;
@@ -420,7 +426,7 @@ export interface PipelineEnded {
"why"?: string;
}
/** one pipeline in list's short form (design 47 H2 H-A-376) */
/** one pipeline in list's short form, with its source, revision and its steps counted per outcome (design 47 H2 H-A-376, H-A-467) */
export interface PipelineRow {
/** the merge call whose source revision the take-in took in */
"call"?: string;
@@ -431,7 +437,13 @@ export interface PipelineRow {
"outcome": "waiting" | "running" | "passed" | "failed" | "refused" | "superseded" | "cancelled";
/** the pipeline's key */
"pipeline": string;
/** the catalog graph revision of the branch the pipeline runs for (design 47 H2 H-A-467) */
"revision"?: number;
/** the source revision of the watched branch the pipeline runs for, its key (design 47 H2 H-A-467) */
"source"?: number;
"started"?: string;
/** how many of the pipeline's steps hold each outcome, by the step's outcome (design 47 H2 H-A-467) */
"steps"?: Record<string, number>;
}
/** a pipeline's start, said at its write, a waiting one's too (design 47 H §746, H-A-52) */
@@ -693,7 +705,10 @@ export interface RunAnswer {
/** is what start takes. */
export interface StartInput {
"branch": string;
"module": string;
/** the one mover of the head's pipeline <branch>.<revision>; absent when modules is given */
"module"?: string;
/** the modules started again at the head, each whose newest entry has no passed pipeline (a failed, cut or superseded one); absent when module is given (design 47 H2 H-A-479) */
"modules"?: string[];
/** why, written with the caller the runtime proves in the hand-act log */
"why": string;
}
@@ -1025,6 +1040,9 @@ export type PipelinesByBranchRunRunValue = Pipeline;
/** is one value of the state pipelines under the key <branch>.<revision>.unfinished.<module>. */
export type PipelinesByBranchRevisionUnfinishedModuleValue = Pipeline;
/** is one value of the state pipelines under the key <branch>.<revision>.again.<n>. */
export type PipelinesByBranchRevisionAgainNValue = Pipeline;
/** is one value of the state pipelines under the key steps.<pipeline>.<mover>. */
export interface PipelinesByStepsPipelineMoverValue {
/** is the form of what this value carries, left out while it is 1 (design 47 H2 §780). */
@@ -199,6 +199,29 @@ export function parsePipelinesByBranchRevisionUnfinishedModuleKey(_key) {
return { branch, revision, module };
}
/** The key of one value of the state pipelines under the form "<branch>.<revision>.again.<n>". */
export function pipelinesByBranchRevisionAgainNKey(branch, revision, n) {
return branch + "." + revision + ".again." + n;
}
/** The parts of a key of the form "<branch>.<revision>.again.<n>", or undefined for a key of another form. */
export function parsePipelinesByBranchRevisionAgainNKey(_key) {
let _rest = _key;
const _at0 = _rest.indexOf(".");
if (_at0 <= 0) return undefined;
const branch = _rest.slice(0, _at0);
_rest = _rest.slice(_at0 + 1);
const _at2 = _rest.indexOf(".again.");
if (_at2 <= 0) return undefined;
const revision = _rest.slice(0, _at2);
_rest = _rest.slice(_at2 + 7);
if (_rest === "") return undefined;
const n = _rest;
_rest = "";
if (_rest !== "") return undefined;
return { branch, revision, n };
}
/** The key of one value of the state pipelines under the form "steps.<pipeline>.<mover>". */
export function pipelinesByStepsPipelineMoverKey(pipeline, mover) {
return "steps." + pipeline + "." + mover;
@@ -105,9 +105,14 @@ func (ix *Indexer) HeardPins(ctx context.Context, started *runner.PipelineStarte
}
}
}
// A pipeline that did not pass withdraws the reservations its revision holds (design 47 H2 §805, H-A-418 (1)).
// A pipeline that did not pass withdraws the reservations its revision holds of its own movers, never another open
// pipeline's at that revision (design 47 H2 §805, H-A-418 (1), H-A-479 second addendum).
if ended.Outcome != runner.PipelineEndedOutcomePassed {
if err := ix.Withdraw(ctx, ended.Branch, ended.Revision); err != nil {
movers := make([]string, 0, len(ended.Movers))
for _, m := range ended.Movers {
movers = append(movers, m.Module)
}
if err := ix.Withdraw(ctx, ended.Branch, ended.Revision, movers); err != nil {
return fmt.Errorf("the reservations of %s are not withdrawn: %w", BranchObject(ended.Branch, ended.Revision), err)
}
}
@@ -2,6 +2,7 @@ package index
import (
"context"
"encoding/json"
"fmt"
"slices"
"strings"
@@ -151,7 +152,14 @@ func TestAReleaseRepinsAnEntryNamingItsPackageWithNoPin(t *testing.T) {
if err := ix.Settle(ctx, next); err != nil || newest() != 4 {
t.Fatalf("the reservation of 0.1.29 settled (%v) keeping main's newest %d, want 4", err, newest())
}
if err := ix.Withdraw(ctx, "main", 3); err != nil || newest() != 5 {
// H-A-479 second addendum: a pipeline at the same revision whose movers are others withdraws none of mesh-sdk's.
if err := ix.Withdraw(ctx, "main", 3, []string{"app"}); err != nil || newest() != 4 {
t.Fatalf("another pipeline's withdrawal (%v) kept main's newest %d, want 4: nothing of mesh-sdk withdrawn", err, newest())
}
if _, _, kept, _ := heard.Get(ctx, ReservedKey(next, "mesh-sdk")); !kept {
t.Fatalf("another pipeline's withdrawal took mesh-sdk's reservation")
}
if err := ix.Withdraw(ctx, "main", 3, []string{"mesh-sdk"}); err != nil || newest() != 5 {
t.Fatalf("the withdrawal (%v) kept main's newest %d, want 5", err, newest())
}
if now, _ := moduleOf(t, mods, "g"); now.Packages["git.novox.be/novox/mesh-sdk/go"] != "v0.1.28" {
@@ -160,4 +168,31 @@ func TestAReleaseRepinsAnEntryNamingItsPackageWithNoPin(t *testing.T) {
if _, _, kept, _ := heard.Get(ctx, ReservedKey(next, "mesh-sdk")); kept {
t.Errorf("the withdrawn reservation is still kept")
}
// H-A-479 second addendum: two pipelines' movers reserve at one revision while its gathering is open; withdrawing one
// pipeline's mover keeps the gathering and the other's reservation, withdrawing the last removes the gathering too.
six := BranchObject("main", 6)
for _, m := range []string{"one", "two"} {
raw, _ := json.Marshal(Reserved{V: 1, Revision: six, Module: m})
if err := heard.Create(ctx, ReservedKey(six, m), raw); err != nil {
t.Fatal(err)
}
}
if err := heard.Create(ctx, SettlingKey(six), []byte(`{}`)); err != nil {
t.Fatal(err)
}
if err := ix.Withdraw(ctx, "main", 6, []string{"one"}); err != nil {
t.Fatal(err)
}
_, _, gathering, _ := heard.Get(ctx, SettlingKey(six))
_, _, one, _ := heard.Get(ctx, ReservedKey(six, "one"))
_, _, two, _ := heard.Get(ctx, ReservedKey(six, "two"))
if !gathering || one || !two {
t.Fatalf("withdrawing one: gathering %v, one kept %v, two kept %v; want the gathering and two kept, one gone", gathering, one, two)
}
if err := ix.Withdraw(ctx, "main", 6, []string{"two"}); err != nil {
t.Fatal(err)
}
if _, _, gathering, _ := heard.Get(ctx, SettlingKey(six)); gathering {
t.Error("withdrawing the last reservation kept the revision's gathering")
}
}
@@ -350,20 +350,35 @@ func (ix *Indexer) settleNow(ctx context.Context, revision string) error {
return err
}
// Withdraw takes a pipeline-ended not passed of branch's revision in: each reservation the revision holds is withdrawn, its
// series' numbered set back to the newest released version, its importers re-pinned to that, and its key deleted.
func (ix *Indexer) Withdraw(ctx context.Context, branch string, revision int64) error {
// Withdraw takes a pipeline-ended not passed of branch's revision in: each reservation the revision holds of one of the
// ended pipeline's own movers is withdrawn, its series' numbered set back to the newest released version, its importers
// re-pinned to that, and its key deleted. Another open pipeline's reservations at the same revision are kept, and the
// revision's gathering is withdrawn only when none of them is left (design 47 H2 H-A-479 second addendum).
func (ix *Indexer) Withdraw(ctx context.Context, branch string, revision int64, movers []string) error {
if ix.Heard == nil {
return nil
}
name := BranchObject(branch, revision)
if _, rev, found, err := ix.Heard.Get(ctx, SettlingKey(name)); err == nil && found {
_ = ix.Heard.Delete(ctx, SettlingKey(name), rev) // the gathering is withdrawn with it
}
held, revs, err := ix.reservedOf(ctx, name)
if err != nil || len(held) == 0 {
all, revs, err := ix.reservedOf(ctx, name)
if err != nil {
return err
}
held := map[string]Reserved{}
for key, r := range all {
if slices.Contains(movers, r.Module) {
held[key] = r
} else {
delete(revs, key)
}
}
if len(held) == len(all) {
if _, rev, found, err := ix.Heard.Get(ctx, SettlingKey(name)); err == nil && found {
_ = ix.Heard.Delete(ctx, SettlingKey(name), rev) // the gathering is withdrawn with it
}
}
if len(held) == 0 {
return nil
}
for _, r := range held {
for _, one := range r.Reservations {
if err := ix.withdrawNumbered(ctx, one); err != nil {
@@ -27,9 +27,10 @@ func TestTheJudge(t *testing.T) {
want: map[string]string{"a": `secret-`}},
{name: "an unknown field", raws: map[string]string{"a": `{"module":"a","version":"1.0","a-field-from-later":1}`},
want: map[string]string{"a": `"a-field-from-later"`}},
// H-A-465: the take-in sets a top-level word it does not know aside (ForInputs); the judgement still refuses it.
{name: "a word a newer SDK adds", raws: map[string]string{"a": `{"module":"a","version":"1.0","class":"core"}`},
want: map[string]string{"a": `"class"`}},
// H-A-465: the take-in sets a top-level word it does not know aside (ForInputs); the judgement still refuses it. The
// word is one no SDK knows: class, the word of H-A-465's night, is known since H2.
{name: "a word a newer SDK adds", raws: map[string]string{"a": `{"module":"a","version":"1.0","zzz-unknown-word":"core"}`},
want: map[string]string{"a": `"zzz-unknown-word"`}},
// Design 47 G §23, §28: a step's status is kept in the registration and never in a manifest.
{name: "a manifest naming a step's status", raws: map[string]string{"a": `{"module":"a","version":"1.0","policy":{"lint":"bypass"}}`},
want: map[string]string{"a": `"policy"`}},
@@ -19,11 +19,11 @@ func VerbTools(ask func(key string, body any) (json.RawMessage, error)) []stdio.
}
return []stdio.Tool{
verbTool[seat.StartInput](ask, "start", seat.VerbStart,
"Start by hand the delivering pipeline of the head of a watched branch for a module, by the seat verb start: the branch, the module and why. A head whose pipeline is written already is refused by the verb, naming it.",
obj([]string{"branch", "module", "why"}, map[string]any{"branch": str, "module": str, "why": str}),
"Start by hand the delivering pipeline of the head of a watched branch for a module, by the seat verb start: the branch, the module and why. A head whose pipeline is written already is refused by the verb, naming it. With modules instead of module, the named modules whose newest entry has no passed pipeline (a failed, cut or superseded one) are started again at the head, keyed <branch>.<revision>.again.<n>; a module whose newest entry passed is refused by name (design 47 H2 H-A-479).",
obj([]string{"branch", "why"}, map[string]any{"branch": str, "module": str, "modules": map[string]any{"type": "array", "items": str}, "why": str}),
func(in seat.StartInput) string {
if in.Branch == "" || in.Module == "" {
return "start names the watched branch and the module"
if in.Branch == "" || (in.Module == "") == (len(in.Modules) == 0) {
return "start names the watched branch and exactly one of the module or the modules"
}
return whyOf(in.Why)
}),
@@ -18,7 +18,11 @@ func TestVerbTools(t *testing.T) {
{"start", "runner_start", "seat:mesh-pipeline-runner.start", "", map[string]any{"branch": "main", "module": "i3", "why": "cancelled"},
seat.StartInput{Branch: "main", Module: "i3", Why: "cancelled"}},
{"start no why", "runner_start", "", "names why", map[string]any{"branch": "main", "module": "i3"}, nil},
{"start no module", "runner_start", "", "branch and the module", map[string]any{"branch": "main", "why": "x"}, nil},
{"start no module", "runner_start", "", "exactly one of the module or the modules", map[string]any{"branch": "main", "why": "x"}, nil},
// H-A-479: modules start the named modules again at the head; both together are refused.
{"start modules", "runner_start", "seat:mesh-pipeline-runner.start", "", map[string]any{"branch": "main", "modules": []any{"i3", "lib"}, "why": "failed"},
seat.StartInput{Branch: "main", Modules: []string{"i3", "lib"}, Why: "failed"}},
{"start module and modules", "runner_start", "", "exactly one of the module or the modules", map[string]any{"branch": "main", "module": "i3", "modules": []any{"lib"}, "why": "x"}, nil},
{"run", "runner_run", "seat:mesh-pipeline-runner.run", "", map[string]any{"members": []any{map[string]any{}}, "why": "x"},
seat.RunInput{Members: []seat.Member{{}}, Why: "x"}},
{"run no members", "runner_run", "", "members", map[string]any{}, nil},
@@ -96,3 +96,34 @@ func TestARevisionHeardIsHandedToItsStart(t *testing.T) {
t.Errorf("a body that is no event = %v, want it ended", err)
}
}
// H-A-479 addendum (2): a revision is held when the catalog seat's revision verb answers that revision, read fresh; none-yet,
// another revision or a refusal saying "no such object" is gone; another failure is said as it is.
func TestRevisionHeldReadsTheCatalogsRevisionFresh(t *testing.T) {
for _, c := range []struct {
name string
answer catalog.RevisionAnswer
err error
gone bool
fails bool
}{
{name: "held", answer: catalog.RevisionAnswer{Revision: &catalog.CatalogRevision{Revision: 258}}},
{name: "another revision", answer: catalog.RevisionAnswer{Revision: &catalog.CatalogRevision{Revision: 259}}, gone: true, fails: true},
{name: "none yet", answer: catalog.RevisionAnswer{NoneYet: true}, gone: true, fails: true},
{name: "removed", err: errors.New("revision main.258: no such object"), gone: true, fails: true},
{name: "unreachable", err: errors.New("no responders"), fails: true},
} {
var asked catalog.RevisionInput
call := func(_ string, body any) (json.RawMessage, error) {
asked = body.(catalog.RevisionInput)
if c.err != nil {
return nil, c.err
}
return json.Marshal(c.answer)
}
err := revisionHeld(call)("main", 258)
if (err != nil) != c.fails || errors.Is(err, pipeline.ErrRevisionGone) != c.gone || asked.Revision != 258 || asked.Branch != "main" {
t.Errorf("%s: %v (asked %+v)", c.name, err, asked)
}
}
}
@@ -4,10 +4,14 @@ import (
"encoding/json"
"errors"
"fmt"
"strings"
"git.novox.be/novox/mesh-contracts/health-and-repair/go/clients/meshhandactsclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
catalog "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"git.novox.be/novox/mesh-sdk/go/keys"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/seatlist"
@@ -65,5 +69,37 @@ func writeVerbs(h *pipeline.Hearer, s *pipeline.Starter, view func(string) (pipe
candidate := func(group, branch string, revision int64) (pipeline.Half, error) {
return readCandidate(calls.CallKept, catalog.CatalogRevisionEvent{Kind: "candidate", Group: group, Branch: branch, Revision: revision})
}
return pipeline.Verbs{Hearer: h, Starter: s, View: view, Candidate: candidate, Acts: meshhandactsclient.Recorder{Ask: calls.Call}}
return pipeline.Verbs{Hearer: h, Starter: s, View: view, Candidate: candidate, Acts: meshhandactsclient.Recorder{Ask: calls.Call},
RevisionHeld: revisionHeld(calls.CallKept)}
}
// revisionHeld reads the first page of a branch's catalog revision through the catalog seat's verb revision, never a kept
// copy, so a rerun reopening an ended pipeline knows whether the catalog still keeps its revision (design 47 H2 H-A-479
// addendum (2)): an answer of that revision is kept; none-yet, another revision, or a refusal saying the catalog's
// "no such object" is gone (pipeline.ErrRevisionGone); any other failure is said as it is.
func revisionHeld(call func(key string, body any) (json.RawMessage, error)) func(string, int64) error {
return func(branch string, revision int64) error {
raw, err := call(meshcatalogclient.RevisionVerb, catalog.RevisionInput{Branch: keys.EscapeName(branch), Revision: revision})
if err != nil {
if strings.Contains(err.Error(), "no such object") {
return fmt.Errorf("%w: %v", pipeline.ErrRevisionGone, err)
}
return err
}
answer, said, ok := buscalls.Unwrap(raw)
if !ok {
if strings.Contains(said, "no such object") {
return fmt.Errorf("%w: %s", pipeline.ErrRevisionGone, said)
}
return fmt.Errorf("%s answered: %s", meshcatalogclient.RevisionVerb, said)
}
var a catalog.RevisionAnswer
if err := json.Unmarshal(answer, &a); err != nil {
return fmt.Errorf("%s answered no revision: %w", meshcatalogclient.RevisionVerb, err)
}
if a.NoneYet || a.Revision == nil || a.Revision.Revision != revision {
return fmt.Errorf("%w: %s answered no revision %d of %s", pipeline.ErrRevisionGone, meshcatalogclient.RevisionVerb, revision, branch)
}
return nil
}
}
@@ -38,8 +38,9 @@ func widenerOf(call func(key string, body any) (json.RawMessage, error), bound t
const (
envWidensReadBound = "MESH_RUNNER_WIDENS_READ_BOUND_FILE"
envRevisionsKept = "MESH_RUNNER_REVISIONS_KEPT_FILE"
// defaultReadBound is how long one read of widens waits for its answer: a few seconds, never the call's 30.
defaultReadBound = 5 * time.Second
// defaultReadBound is how long one read of widens waits for its answer: 30 s, so a cold grants-for (one that composes a
// revision's grants the grants part did not keep yet) is not cut (design 47 H2 H-A-478).
defaultReadBound = 30 * time.Second
// defaultRevisionsKept is how many catalog revisions the runner keeps once read; a revision of the catalog graph is some 1.4 MB.
defaultRevisionsKept = 6
)
@@ -115,7 +115,7 @@ func TestWidensSettings(t *testing.T) {
kept int
ok bool
}{
{"defaults", nil, 5 * time.Second, 6, true},
{"defaults", nil, 30 * time.Second, 6, true},
{"set", map[string]string{envWidensReadBound: "2s\n", envRevisionsKept: "3\n"}, 2 * time.Second, 3, true},
{"no duration", map[string]string{envWidensReadBound: "5"}, 0, 0, false},
{"no number", map[string]string{envRevisionsKept: "0"}, 0, 0, false},
@@ -11,6 +11,7 @@ import (
"sync"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/pipelinekey"
"git.novox.be/novox/mesh-sdk/go/keys"
)
@@ -28,23 +29,20 @@ func StepsKey(pipeline, mover string) string {
// stepsPrefix is the start of every mover key of one pipeline.
func stepsPrefix(pipeline string) string { return StepsKey(pipeline, "") }
// IsStepsKey says whether a key of the state pipelines is a mover's steps rather than a pipeline's record: `steps.` and a
// pipeline key of a take-in's (4 parts), a run's (5 parts, its third `run`), an unfinished mover's own (6 parts, its fourth
// `unfinished`, H-A-398) or a publish's (7 parts) and one mover.
// IsStepsKey says whether a key of the state pipelines is a mover's steps rather than a pipeline's record: `steps.`, a pipeline's
// key of any form the seat defines (a take-in's, a run's, a publish's, an unfinished mover's own or one started again at the head,
// H-A-398, H-A-479), read by the one parse of the seat's key forms (design 47 H2 H-A-485), and one mover.
func IsStepsKey(key string) bool {
parts := strings.Split(key, ".")
if parts[0] != "steps" {
rest, ok := strings.CutPrefix(key, "steps.")
if !ok {
return false
}
switch len(parts) {
case 4, 7:
return true
case 5:
return parts[2] == runToken
case 6:
return parts[3] == unfinishedToken
i := strings.LastIndex(rest, ".")
if i <= 0 || i == len(rest)-1 {
return false
}
return false
_, err := pipelinekey.Parse(rest[:i])
return err == nil
}
// RawState is the state pipelines as bytes: a key's value and revision (nil, 0 and no error when it has none), a create by
@@ -286,12 +286,27 @@ func (r Reads) List(in seat.ListInput) (seat.ListAnswer, error) {
}
rows := make([]seat.PipelineRow, 0, len(out))
for _, p := range out {
rows = append(rows, seat.PipelineRow{Pipeline: p.Pipeline, Call: p.Call, Started: p.Started, Ended: p.Ended, Outcome: p.Outcome,
MoverCount: p.MoverCount, Failed: firstFailed(steps[p.Pipeline])})
// A row carries its source, its revision and its steps counted per outcome (design 47 H2 H-A-467): a page's list is
// one read of the short form.
rows = append(rows, seat.PipelineRow{Pipeline: p.Pipeline, Call: p.Call, Source: p.Source, Revision: p.Revision,
Started: p.Started, Ended: p.Ended, Outcome: p.Outcome, MoverCount: p.MoverCount, Steps: stepCounts(steps[p.Pipeline]),
Failed: firstFailed(steps[p.Pipeline])})
}
return seat.ListAnswer{Rows: rows}, nil
}
// stepCounts is how many of steps hold each outcome (design 47 H2 H-A-467), nil for none.
func stepCounts(steps []seat.StepResult) map[string]int64 {
if len(steps) == 0 {
return nil
}
out := map[string]int64{}
for _, s := range steps {
out[s.Outcome]++
}
return out
}
// Stands is every step that has started and not ended, with what it waits on and since when (§75, §83): the watchman
// seeds its open waits from it.
func (r Reads) Stands() (seat.StandsAnswer, error) {
@@ -67,7 +67,7 @@ func TestReads(t *testing.T) {
Step: seat.StepResultStepApproval, Status: "required", Outcome: seat.StepResultOutcomeWaiting, On: seat.WaitOnPerson, Started: &t2}},
Unlogged: []seat.LogEntry{{Pipeline: "next.8", Module: "delta", Step: seat.StepResultStepBuild, What: seat.LogEntryWhatEnded, At: t1}}})
// A pipeline that failed: list's row names its first failed step (design 47 H2 H-A-376).
put(t, pipes, "gone.2", seat.Pipeline{Pipeline: "gone.2", Branch: "gone", Call: "call-2", Outcome: seat.PipelineOutcomeFailed, Started: &t0, Ended: &t1,
put(t, pipes, "gone.2", seat.Pipeline{Pipeline: "gone.2", Branch: "gone", Call: "call-2", Source: 2, Revision: 5, Outcome: seat.PipelineOutcomeFailed, Started: &t0, Ended: &t1,
Movers: []seat.Mover{{Module: "zeta", Commit: "g2"}}, Steps: []seat.StepResult{
{Module: "zeta", Step: seat.StepResultStepLint, Artifact: "zeta:program", Status: "optional", Outcome: seat.StepResultOutcomeFailed},
{Module: "zeta", Step: seat.StepResultStepUnit, Artifact: "zeta:program", Status: "required", Outcome: seat.StepResultOutcomeCouldNotRun, Why: "its node lacks node-build-node"}}})
@@ -150,13 +150,19 @@ func TestReads(t *testing.T) {
a, err := r.List(seat.ListInput{Branch: "main", Limit: 1})
return names(a.Rows), err
}, "main.7.gamma"},
// Design 47 H2 H-A-376: a row is the pipeline's key, call, start, end, outcome, movers' count and first failed step;
// Design 47 H2 H-A-376, H-A-467: a row is the pipeline's key, call, source, revision, start, end, outcome, movers' count,
// its steps per outcome and first failed step;
// the whole pipelines are answered only with full and a limit; with no limit the rows are the newest 20.
{"list's row of a failed pipeline", func() (string, error) {
a, err := r.List(seat.ListInput{Branch: "gone"})
raw, _ := json.Marshal(a.Rows)
return fmt.Sprint(string(raw), " ", len(a.Pipelines)), err
}, `[{"call":"call-2","ended":"2026-11-11T10:01:00Z","failed":{"artifact":"zeta:program","module":"zeta","outcome":"could not run","step":"unit","why":"its node lacks node-build-node"},"mover-count":1,"outcome":"failed","pipeline":"gone.2","started":"2026-11-11T10:00:00Z"}] 0`},
}, `[{"call":"call-2","ended":"2026-11-11T10:01:00Z","failed":{"artifact":"zeta:program","module":"zeta","outcome":"could not run","step":"unit","why":"its node lacks node-build-node"},"mover-count":1,"outcome":"failed","pipeline":"gone.2","revision":5,"source":2,"started":"2026-11-11T10:00:00Z","steps":{"could not run":1,"failed":1}}] 0`},
// Design 47 H2 H-A-467: a row counts the steps per outcome, a mover's kept steps (H-A-187) with them.
{"list's row counts its steps per outcome", func() (string, error) {
a, err := r.List(seat.ListInput{Branch: "next"})
return fmt.Sprint(a.Rows[0].Source, " ", a.Rows[0].Steps), err
}, "8 map[passed:2 waiting:1]"},
{"list's full needs a limit", func() (string, error) {
_, err := r.List(seat.ListInput{Full: true})
return "", err
@@ -25,7 +25,7 @@ func (h *Hearer) Released(ev catalog.VersionReleasedEvent, branch string) (int,
// A pipeline is keyed by its revision (H-A-213 (4)), so the pipelines of a source are those naming it, of either key
// form a source's pipelines were kept under.
var pkeys []string
for _, f := range []string{b + ".*", b + ".*.published.*.*", b + ".*." + unfinishedToken + ".*"} {
for _, f := range []string{b + ".*", b + ".*.published.*.*", b + ".*." + unfinishedToken + ".*", b + ".*." + againToken + ".*"} {
k, err := h.Store.Keys(f)
if err != nil {
return 0, fmt.Errorf("the pipelines of source %d are not listed: %w", ev.Source, err)
@@ -69,8 +69,19 @@ func keyRevision(b Begin) int64 {
}
// keyForms are the filters of every key form of the state pipelines: a revision's, a run's, a publish's kept from before
// a release revision started one pipeline (H-A-213 (2)), and an unfinished mover's pipeline of its own (H-A-398).
var keyForms = []string{"*.*", "*." + runToken + ".*", "*.*.published.*.*", "*.*." + unfinishedToken + ".*"}
// a release revision started one pipeline (H-A-213 (2)), an unfinished mover's pipeline of its own (H-A-398), and a pipeline
// started again at the head (H-A-479).
var keyForms = []string{"*.*", "*." + runToken + ".*", "*.*.published.*.*", "*.*." + unfinishedToken + ".*", "*.*." + againToken + ".*"}
// againToken is the third part of the key of a pipeline started again at the head by start with modules,
// `<branch>.<revision>.again.<n>` (design 47 H2 H-A-479 addendum (1)).
const againToken = "again"
// AgainPipelineKey is the key of the n-th pipeline started again at revision of branch, `<branch>.<revision>.again.<n>`
// (H-A-479 addendum (1)).
func AgainPipelineKey(branch string, revision int64, n int) string {
return seat.PipelinesByBranchRevisionAgainNKey(keys.EscapeName(branch), strconv.FormatInt(revision, 10), strconv.Itoa(n))
}
// AllKeys are every pipeline's key, of each key form (§746, §747; H-A-83).
func AllKeys(store interface {
@@ -655,6 +655,50 @@ func TestNoWriteOfATestRunActsOnAReadBeforeItsLatestChange(t *testing.T) {
t.Error("the step's finding was written before its run's end")
}
})
// H-A-479 addendum (2): a test step's rerun reopening an ended pipeline says its pipeline-started before its run is
// started again, so its revision is pinned before the step is asked.
t.Run("a test step's rerun reopening an ended pipeline pins its revision first", func(t *testing.T) {
f := newRunFixture()
p := f.start(t, 4, 9)
unit := stepNamed(p, "app:program", "unit")
run := unit.Shared[0].Run
ended := time.Now()
if _, err := f.h.OnStepEnded(stepSeatOf(t, "node-build-go"), "mesh.seat.node-build-go.event.step-ended.ace", StepEnded{ID: unit.Ask,
Pipeline: p.Pipeline, Module: "app", Commit: "c1", Fingerprint: "src2:aa", Step: "unit", Node: "ace", Outcome: "failed", Ended: &ended}); err != nil {
t.Fatal(err)
}
q, rev, _ := f.store.Get(p.Pipeline)
q.Outcome, q.Ended = seat.PipelineOutcomeFailed, &ended
if _, err := f.store.Update(p.Pipeline, q, rev); err != nil {
t.Fatal(err)
}
before, _, _ := f.runs.Get(run)
var started []string
f.s.Say = func(_ string, body any) error {
if ps, ok := body.(seat.PipelineStarted); ok {
if r, _, _ := f.runs.Get(run); r.Ask != before.Ask || r.Ended == nil {
t.Errorf("pipeline-started said after the run was started again (ask %s, was %s)", r.Ask, before.Ask)
}
started = append(started, ps.Pipeline)
}
return nil
}
v := Verbs{Hearer: f.h, Starter: f.s, View: func(string) (Half, bool) {
return Half{Entries: map[string]catalog.CatalogEntry{
"app": {Module: "app", Path: "modules/app", Artifacts: []catalog.Artifact{{Ref: "app:program", Language: "go"}, {Ref: "app:tools", Language: "go"}}},
"mesh-build-go": {Module: "mesh-build-go", Acts: []catalog.ActingRule{{Seat: "node-build-go", Step: "build", On: "go"}}}}}, true
}, Acts: &actLog{}, RevisionHeld: func(string, int64) error { return nil }}
a, err := v.Rerun("op@shanks", seat.RerunInput{Pipeline: p.Pipeline, Step: "unit", Artifact: "app:program", Why: "fixed"})
if err != nil {
t.Fatal(err)
}
if len(started) != 1 || started[0] != p.Pipeline {
t.Errorf("pipeline-started at the reopening = %v, want %s once", started, p.Pipeline)
}
if r, _, _ := f.runs.Get(run); r.Ask != a.Result.Ask || r.Ended != nil {
t.Errorf("the rerun's run = %+v; want the new ask running", r)
}
})
// A rerun whose read of the key is changed before its write (a step joining the ended run) writes from the latest read,
// carrying that reader, never answering the change as an error nor putting back the earlier readers.
t.Run("a rerun writes its run from the latest read", func(t *testing.T) {
@@ -4,6 +4,7 @@ import (
"errors"
"fmt"
"slices"
"strconv"
"strings"
"time"
@@ -29,8 +30,15 @@ type Verbs struct {
Acts views.HandActs
// Runs answers the read verb test-runs, answered by the process that holds test-runs-cap-mib (A-470i).
Runs RunsRead
// RevisionHeld says whether a branch's catalog revision is still kept, read through the catalog seat's verb revision
// at the time of the ask: nil when it is, an error wrapping ErrRevisionGone when it is removed (design 47 H2 H-A-479
// addendum (2)).
RevisionHeld func(branch string, revision int64) error
}
// ErrRevisionGone is a catalog revision the catalog no longer keeps (H-A-479 addendum (2)).
var ErrRevisionGone = errors.New("the catalog revision is gone")
func (v Verbs) now() time.Time { return v.Hearer.now() }
func (v Verbs) record(verb, what, why string) error {
@@ -275,13 +283,89 @@ func (v Verbs) EndAgain(_ string, in seat.EndAgainInput) (seat.EndAgainAnswer, e
// the ask saying it is a rerun and the caller the bus proved; a failed pipeline runs again and ends once more with its end
// count one higher (§325), an approval never asked waiting again. A review waits again; an approval is asked again by
// `ask-again`.
//
// A rerun of a step in an ended delivering pipeline reopens it: its revision is read first and the rerun refused by name,
// nothing asked, when the catalog removed it; once reopened its pipeline-started is said again, so the indexer pins its
// revision again until its next end (design 47 H2 H-A-479 addendum (2)).
func (v Verbs) Rerun(caller string, in seat.RerunInput) (seat.RerunAnswer, error) {
if in.Step == seat.StepResultStepApproval {
return seat.RerunAnswer{}, errors.New("an approval is asked again by ask-again, not rerun")
}
reopens, err := v.reopening(in.Pipeline)
if err != nil {
return seat.RerunAnswer{}, err
}
return v.rerun(caller, in, reopens)
}
// sayReopened says pipeline-started again for p, written reopened by a rerun, before its step is asked again, so the
// indexer pins its revision again before anything reads it (H-A-479 addendum (2)).
func (v Verbs) sayReopened(p seat.Pipeline, reopens bool) {
if !reopens || v.Starter == nil || p.Ended != nil {
return
}
v.Starter.sayStartedPipeline(p, v.now())
v.Starter.logf("%s reopened by a rerun: its pipeline-started is said again, so its revision %d is pinned again (H-A-479)", p.Pipeline, p.Revision)
}
// reopening says whether a rerun of pipeline key reopens an ended delivering pipeline, refusing it when its revision is
// gone (H-A-479 addendum (2)). A pipeline not read, a run's, or one the rerun refuses anyway is left to the rerun.
func (v Verbs) reopening(key string) (bool, error) {
p, _, err := v.Hearer.Store.Get(key)
if err != nil || p.Ended == nil || p.Run != "" || p.Revision <= 0 ||
p.Outcome == seat.PipelineOutcomeCancelled || p.Outcome == seat.PipelineOutcomeSuperseded {
return false, nil
}
// A reopening never builds beside an open pipeline carrying one of its movers at the same revision (H-A-479 third
// addendum).
if open, err := v.openBeside(p); err != nil {
return false, err
} else if open != "" {
return false, fmt.Errorf("the pipeline %s ended and is not reopened: its pipeline %s is running: nothing was asked", key, open)
}
if v.RevisionHeld != nil {
if err := v.RevisionHeld(p.Branch, p.Revision); errors.Is(err, ErrRevisionGone) {
return false, fmt.Errorf("the pipeline %s ended and its revision %d is gone; start the mover again at the head: nothing was asked (%v)", key, p.Revision, err)
} else if err != nil {
return false, fmt.Errorf("the pipeline %s ended and its revision %d is not read, so it is not reopened: nothing was asked: %w", key, p.Revision, err)
}
}
return true, nil
}
// openBeside is the key of another delivering pipeline of p's branch and revision, not ended, carrying one of p's movers it
// did not leave; empty when none (H-A-479 third addendum).
func (v Verbs) openBeside(p seat.Pipeline) (string, error) {
pkeys, err := branchKeys(v.Hearer.Store, p.Branch)
if err != nil {
return "", fmt.Errorf("the pipelines of %s are not listed, so %s is not reopened: nothing was asked: %w", p.Branch, p.Pipeline, err)
}
for _, key := range pkeys {
if key == p.Pipeline {
continue
}
o, _, err := v.Hearer.Store.Get(key)
if err != nil {
return "", fmt.Errorf("the pipeline %s is not read, so %s is not reopened: nothing was asked: %w", key, p.Pipeline, err)
}
if o.Run != "" || o.Ended != nil || o.Revision != p.Revision ||
(o.Outcome != seat.PipelineOutcomeRunning && o.Outcome != seat.PipelineOutcomeWaiting) {
continue
}
for _, m := range p.Movers {
if Moves(o, m.Module) && !slices.Contains(o.Superseded, m.Module) {
return key, nil
}
}
}
return "", nil
}
// rerun asks one step again (Rerun).
func (v Verbs) rerun(caller string, in seat.RerunInput, reopens bool) (seat.RerunAnswer, error) {
if p, _, err := v.Hearer.Store.Get(in.Pipeline); err == nil {
if i, err := stepOf(p, in.Module, in.Step, in.Artifact); err == nil && v.Hearer.TestRuns != nil && sharedRun(p.Steps[i]) != "" && BoundSeat(p.Steps[i].Seat) {
return v.rerunTest(caller, in, p.Steps[i])
return v.rerunTest(caller, in, p.Steps[i], reopens)
}
}
var ask *Asked
@@ -332,6 +416,7 @@ func (v Verbs) Rerun(caller string, in seat.RerunInput) (seat.RerunAnswer, error
if err != nil {
return seat.RerunAnswer{}, err
}
v.sayReopened(p, reopens)
if ask != nil {
v.Starter.carry(in.Pipeline, []Asked{*ask})
if err := askAll(v.Starter.Steps, in.Pipeline, []Asked{*ask}, func(commit string) bool { return v.Starter.recovered(in.Pipeline, commit) }, v.Starter.Asks.arm, v.Starter.logf); err != nil {
@@ -349,7 +434,7 @@ func (v Verbs) Rerun(caller string, in seat.RerunInput) (seat.RerunAnswer, error
// rerunTest asks a test step of a test run again (design 47 G §727, A-469l): it joins its key's run while that runs, reads one
// that started after the step's last end, or else starts a new run under a new ask, replacing the ended key by compare-and-set
// and carrying its readers into the new value, so the new run's end moves every former reader that did not pass.
func (v Verbs) rerunTest(caller string, in seat.RerunInput, st seat.StepResult) (seat.RerunAnswer, error) {
func (v Verbs) rerunTest(caller string, in seat.RerunInput, st seat.StepResult, reopens bool) (seat.RerunAnswer, error) {
h := v.Hearer
run := sharedRun(st)
if open(st) {
@@ -430,6 +515,7 @@ func (v Verbs) rerunTest(caller string, in seat.RerunInput, st seat.StepResult)
if err != nil {
return seat.RerunAnswer{}, err
}
v.sayReopened(p, reopens)
switch {
case join || read:
v.Starter.joinRun(run, reader)
@@ -649,10 +735,17 @@ func (v Verbs) Run(caller string, in seat.RunInput) (seat.RunAnswer, error) {
// Start starts by hand a delivering pipeline for the head of a watched branch the runner's catalog half holds, recorded in
// the hand-act log first (§306, §394): the pipeline of that branch's source revision and the module, as a take-in would
// start it. A head whose pipeline is written already starts nothing and is refused, naming the pipeline.
// start it. A head whose pipeline is written already starts nothing and is refused, naming the pipeline. With modules it
// starts the named modules again at the head instead (startAgain, H-A-479); exactly one of module or modules is given.
func (v Verbs) Start(_ string, in seat.StartInput) (seat.StartAnswer, error) {
if in.Module != "" && len(in.Modules) > 0 {
return seat.StartAnswer{}, errors.New("start names the module or the modules, not both: nothing was started")
}
if in.Branch != "" && len(in.Modules) > 0 {
return v.startAgain(in)
}
if in.Branch == "" || in.Module == "" {
return seat.StartAnswer{}, errors.New("start names the watched branch and the module")
return seat.StartAnswer{}, errors.New("start names the watched branch and the module or the modules")
}
if v.Starter == nil || v.View == nil {
return seat.StartAnswer{}, errors.New("the runner cannot start a pipeline: it holds no starter or no catalog half")
@@ -681,6 +774,117 @@ func (v Verbs) Start(_ string, in seat.StartInput) (seat.StartAnswer, error) {
return seat.StartAnswer{Pipeline: key}, nil
}
// startAgain starts by hand, recorded in the hand-act log first, a pipeline at the head of a watched branch carrying the
// named modules whose newest entry, the head's, has no passed pipeline (a failed, cut or superseded one), with the head's
// revision as its candidate and the take-in's tiers (design 47 H2 H-A-479): keyed `<branch>.<revision>.again.<n>`, n from 1,
// taken by writing on absence (addendum (1)). Its pipeline-started pins the head's revision as a take-in's does, and its
// versions are reserved at its build steps as a take-in's are. A module the head holds no entry of, or whose newest entry
// passed in a pipeline, is refused by name, nothing started.
func (v Verbs) startAgain(in seat.StartInput) (seat.StartAnswer, error) {
if v.Starter == nil || v.View == nil || v.Hearer == nil || v.Hearer.Store == nil {
return seat.StartAnswer{}, errors.New("the runner cannot start a pipeline again: it holds no starter, no catalog half or no state pipelines")
}
half, ok := v.View(in.Branch)
if !ok || half.Source <= 0 {
return seat.StartAnswer{}, fmt.Errorf("the catalog half of %s is not held at a source revision yet: nothing was started", in.Branch)
}
movers := make([]seat.Mover, 0, len(in.Modules))
var absent []string
for _, m := range in.Modules {
e, ok := half.Entries[m]
if !ok {
absent = append(absent, m)
continue
}
if slices.ContainsFunc(movers, func(x seat.Mover) bool { return x.Module == m }) {
continue
}
movers = append(movers, seat.Mover{Module: m, Commit: e.Commit, Fingerprint: e.Fp, Repository: e.Repository})
}
if len(absent) > 0 {
return seat.StartAnswer{}, fmt.Errorf("the head of %s (revision %d) holds no entry of %s: nothing was started", in.Branch, half.Revision, strings.Join(absent, ", "))
}
passed, running, err := v.heldEntries(in.Branch, movers)
if err != nil {
return seat.StartAnswer{}, err
}
if len(passed) > 0 {
return seat.StartAnswer{}, fmt.Errorf("the newest entry of %s passed already: nothing was started", strings.Join(passed, ", "))
}
if len(running) > 0 {
return seat.StartAnswer{}, fmt.Errorf("the newest entry of %s: nothing was started", strings.Join(running, ", "))
}
movers = Tiers(movers, half.Entries, v.Starter.logf)
names := make([]string, 0, len(movers))
for _, m := range movers {
names = append(names, m.Module)
}
prefix := AgainPipelineKey(in.Branch, half.Revision, 0)
prefix = prefix[:len(prefix)-1]
held, err := v.Hearer.Store.Keys(prefix + "*")
if err != nil {
return seat.StartAnswer{}, fmt.Errorf("the pipelines started again at revision %d of %s are not listed: nothing was started: %w", half.Revision, in.Branch, err)
}
n := 1
for _, k := range held {
if i, err := strconv.Atoi(strings.TrimPrefix(k, prefix)); err == nil && i >= n {
n = i + 1
}
}
if err := v.record(seat.VerbStart, fmt.Sprintf("started %s again at the head of %s by hand", strings.Join(names, ", "), in.Branch), in.Why); err != nil {
return seat.StartAnswer{}, err
}
// The next n is taken without a bound: each refusal means the key exists (H-A-479 second addendum).
for ; ; n++ {
key := AgainPipelineKey(in.Branch, half.Revision, n)
started, err := v.Starter.Start(Begin{Branch: in.Branch, Source: half.Source, Revision: half.Revision, Entries: half.Entries,
Key: key, Movers: movers})
if err != nil {
return seat.StartAnswer{}, err
}
if started {
v.Starter.logf("%s starts %s again at the head by hand (H-A-479)", key, strings.Join(names, ", "))
return seat.StartAnswer{Pipeline: key}, nil
}
}
}
// heldEntries are the movers whose module, at the mover's fingerprint, a delivering pipeline of branch carries and did not
// leave: passed, those a passed pipeline holds (a newest entry that passed is not started again, H-A-479); running, those a
// pipeline not ended holds, each said "<module>: its pipeline <key> is running", so two pipelines never build one entry side
// by side (H-A-479 second addendum).
func (v Verbs) heldEntries(branch string, movers []seat.Mover) (passed, running []string, err error) {
pkeys, err := branchKeys(v.Hearer.Store, branch)
if err != nil {
return nil, nil, fmt.Errorf("the pipelines of %s are not listed, so nothing was started: %w", branch, err)
}
seen := map[string]bool{}
for _, key := range pkeys {
p, _, err := v.Hearer.Store.Get(key)
if err != nil {
return nil, nil, fmt.Errorf("the pipeline %s is not read, so nothing was started: %w", key, err)
}
open := p.Ended == nil && (p.Outcome == seat.PipelineOutcomeRunning || p.Outcome == seat.PipelineOutcomeWaiting)
if p.Run != "" || (p.Outcome != seat.PipelineOutcomePassed && !open) {
continue
}
for _, m := range movers {
if seen[m.Module] || slices.Contains(p.Superseded, m.Module) {
continue
}
if pm, ok := MoverOf(p, m.Module); ok && pm.Fingerprint == m.Fingerprint {
seen[m.Module] = true
if open {
running = append(running, m.Module+": its pipeline "+key+" is running")
} else {
passed = append(passed, m.Module+" (in "+key+")")
}
}
}
}
return passed, running, nil
}
// AskAgain asks a declined approval step once more (§306, §36), recorded in the hand-act log first: the step waits again,
// marked asked again by the caller the bus proved, and is asked at once when the pipeline's every other required step
// passed, under an ask of its own (the earlier ask's decision ended it).
@@ -1,7 +1,9 @@
package pipeline
import (
"encoding/json"
"errors"
"fmt"
"strings"
"testing"
"time"
@@ -423,3 +425,180 @@ func TestAValidateRerunMovesEveryMoversValidateAndReopensItsCompositions(t *test
}
}
}
// H-A-479: start with modules starts a pipeline at the head keyed <branch>.<revision>.again.<n> carrying the named modules
// whose newest entry has no passed pipeline, at the take-in's tiers; a module whose newest entry passed is refused by name,
// nothing started; exactly one of module or modules is given; a second start takes the next n.
func TestStartWithModulesStartsAFailedMoverAgainAtTheHead(t *testing.T) {
v, store, _, _, acts, _ := verbsFixture()
made := created{}
var started []seat.PipelineStarted
v.Starter.Pipelines = made
v.Starter.Say = func(_ string, body any) error {
if ps, ok := body.(seat.PipelineStarted); ok {
started = append(started, ps)
}
return nil
}
_, entries := placeFixture()
app := entries["app"]
app.Fp, app.Commit, app.Repository = "src2:aa", "c1", "o/app"
entries["app"] = app
entries["lib"] = catalog.CatalogEntry{Module: "lib", Fp: "src2:bb", Commit: "c2", Repository: "o/lib"}
v.View = func(string) (Half, bool) { return Half{Source: 7, Revision: 9, Entries: entries}, true }
// The fixture's main.5 moves app at src2:aa and failed; lib's newest entry passed in main.8.
e := store[pkey]
e.p.Outcome = seat.PipelineOutcomeFailed
store[pkey] = e
store["main.8"] = struct {
p seat.Pipeline
rev uint64
}{seat.Pipeline{Pipeline: "main.8", Branch: "main", Source: 6, Revision: 8, Outcome: seat.PipelineOutcomePassed,
Movers: []seat.Mover{{Module: "lib", Fingerprint: "src2:bb"}, {Module: "app", Fingerprint: "src2:aa"}}, Superseded: []string{"app"}}, 1}
if _, err := v.Start("operator", seat.StartInput{Branch: "main", Module: "app", Modules: []string{"app"}, Why: "x"}); err == nil {
t.Error("a start naming both module and modules was taken")
}
// A pipeline still running at app's head entry refuses the start: two pipelines never build one entry side by side.
store["main.7"] = struct {
p seat.Pipeline
rev uint64
}{seat.Pipeline{Pipeline: "main.7", Branch: "main", Source: 6, Revision: 7, Outcome: seat.PipelineOutcomeRunning,
Movers: []seat.Mover{{Module: "app", Fingerprint: "src2:aa"}}}, 1}
if _, err := v.Start("operator", seat.StartInput{Branch: "main", Modules: []string{"app"}, Why: "x"}); err == nil || !strings.Contains(err.Error(), "its pipeline main.7 is running") {
t.Errorf("a start of app while main.7 runs it = %v; want refused naming main.7", err)
}
delete(store, "main.7")
if _, err := v.Start("operator", seat.StartInput{Branch: "main", Modules: []string{"app", "lib"}, Why: "x"}); err == nil || !strings.Contains(err.Error(), "lib") {
t.Errorf("a start naming lib, whose newest entry passed, = %v; want refused naming lib", err)
}
if len(made) != 0 {
t.Fatalf("a refused start wrote %d pipelines", len(made))
}
a, err := v.Start("operator", seat.StartInput{Branch: "main", Modules: []string{"app"}, Why: "its pipeline failed at an unchanged entry"})
if err != nil || a.Pipeline != "main.9.again.1" || made["main.9.again.1"] == nil {
t.Fatalf("start with modules = %+v, %v; written %v", a, err, len(made))
}
var p seat.Pipeline
if err := json.Unmarshal(made["main.9.again.1"], &p); err != nil {
t.Fatal(err)
}
if p.Revision != 9 || p.Source != 7 || len(p.Movers) != 1 || p.Movers[0].Module != "app" || p.Movers[0].Fingerprint != "src2:aa" || p.Movers[0].Tier != 1 {
t.Errorf("the pipeline = %+v, want app at src2:aa, tier 1, at revision 9 of source 7", p)
}
if len(started) != 1 || started[0].Pipeline != "main.9.again.1" || started[0].Revision != 9 {
t.Errorf("pipeline-started = %+v, want main.9.again.1 at revision 9 (its revision pinned)", started)
}
// The next start at the same head, once main.9.again.1 failed, takes the next n.
at := time.Unix(3, 0)
p.Outcome, p.Ended = seat.PipelineOutcomeFailed, &at
store["main.9.again.1"] = struct {
p seat.Pipeline
rev uint64
}{p, 1}
if a, err := v.Start("operator", seat.StartInput{Branch: "main", Modules: []string{"app"}, Why: "again"}); err != nil || a.Pipeline != "main.9.again.2" {
t.Errorf("a second start with modules = %+v, %v; want main.9.again.2", a, err)
}
if len(*acts) != 2 {
t.Errorf("hand acts %v, want the two starts recorded", *acts)
}
// main.9.again.1 passed: its version is written into it as into any pipeline of its source, and a further start of app is
// refused naming it, read through the branch's key forms.
p.Outcome = seat.PipelineOutcomePassed
store["main.9.again.1"] = struct {
p seat.Pipeline
rev uint64
}{p, 1}
if wrote, err := v.Hearer.Released(catalog.VersionReleasedEvent{Module: "app", Version: "0.1.2", Source: 7}, "main"); err != nil || wrote != 1 ||
store["main.9.again.1"].p.Movers[0].Version != "0.1.2" {
t.Errorf("the release of app 0.1.2 wrote %d (%v), the again pipeline's mover %+v; want its version written", wrote, err, store["main.9.again.1"].p.Movers)
}
if _, err := v.Start("operator", seat.StartInput{Branch: "main", Modules: []string{"app"}, Why: "x"}); err == nil || !strings.Contains(err.Error(), "main.9.again.1") {
t.Errorf("a start of app after main.9.again.1 passed = %v; want refused naming it", err)
}
if !IsStepsKey(StepsKey("main.9.again.1", "app")) {
t.Error("a mover key of an again pipeline is not read as a mover's steps")
}
// H-A-485: the mover keys are read by the one parse of the seat's key forms; a pipeline key of no form (its n or its
// revision no number) is no mover's steps.
for _, k := range []string{"steps.main.9.again.x.app", "steps.main.r9.app", "steps.main.9.again.0.app"} {
if IsStepsKey(k) {
t.Errorf("%s, of no pipeline key form, is read as a mover's steps", k)
}
}
}
// H-A-479 addendum (2): a rerun of a step in an ended pipeline pins its revision again at the reopening (its pipeline-started
// said again), and is refused by name, nothing asked, when the revision is already removed.
func TestARerunOfAnEndedPipelinePinsItsRevisionAgainOrIsRefusedWhenItIsGone(t *testing.T) {
v, store, _, asks, _, _ := verbsFixture()
var started []seat.PipelineStarted
v.Starter.Say = func(_ string, body any) error {
if ps, ok := body.(seat.PipelineStarted); ok {
if len(asks.asks) != 0 {
t.Errorf("pipeline-started said after the step was asked again (%d asks): its revision is pinned too late", len(asks.asks))
}
started = append(started, ps)
}
return nil
}
e := store[pkey]
at := time.Unix(2, 0)
e.p.Revision, e.p.Outcome, e.p.Ended = 5, seat.PipelineOutcomeFailed, &at
for i := range e.p.Steps {
if e.p.Steps[i].Artifact == "app:program" && e.p.Steps[i].Step == "lint" {
e.p.Steps[i].Outcome = seat.StepResultOutcomeFailed
}
}
store[pkey] = e
gone := true
v.RevisionHeld = func(branch string, revision int64) error {
if gone {
return fmt.Errorf("revision %s.%d: no such object: %w", branch, revision, ErrRevisionGone)
}
return nil
}
in := seat.RerunInput{Pipeline: pkey, Step: "lint", Artifact: "app:program", Why: "retry"}
// H-A-479 third addendum: another open pipeline carrying app at the same revision refuses the reopening.
store["main.5.again.1"] = struct {
p seat.Pipeline
rev uint64
}{seat.Pipeline{Pipeline: "main.5.again.1", Branch: "main", Source: 5, Revision: 5, Outcome: seat.PipelineOutcomeRunning,
Movers: []seat.Mover{{Module: "app", Fingerprint: "src2:aa"}}}, 1}
gone = false
if _, err := v.Rerun("operator", in); err == nil || !strings.Contains(err.Error(), "its pipeline main.5.again.1 is running") {
t.Fatalf("a reopening beside main.5.again.1 = %v; want refused naming it", err)
}
if len(asks.asks) != 0 || store[pkey].p.Outcome != seat.PipelineOutcomeFailed {
t.Fatalf("a refused reopening asked %d, left the pipeline %s", len(asks.asks), store[pkey].p.Outcome)
}
delete(store, "main.5.again.1")
gone = true
if _, err := v.Rerun("operator", in); err == nil || !strings.Contains(err.Error(), "its revision 5 is gone; start the mover again at the head") {
t.Fatalf("a rerun of an ended pipeline whose revision is gone = %v; want refused by name", err)
}
if len(asks.asks) != 0 || store[pkey].p.Outcome != seat.PipelineOutcomeFailed || len(started) != 0 {
t.Fatalf("a refused rerun asked %d, left the pipeline %s, said %d starts", len(asks.asks), store[pkey].p.Outcome, len(started))
}
gone = false
if _, err := v.Rerun("operator", in); err != nil {
t.Fatal(err)
}
if len(started) != 1 || started[0].Pipeline != pkey || started[0].Revision != 5 {
t.Errorf("pipeline-started at the reopening = %+v, want %s at revision 5 (its revision pinned again)", started, pkey)
}
// A rerun of a running pipeline reopens nothing and says no start.
e = store[pkey]
for i := range e.p.Steps {
if e.p.Steps[i].Artifact == "app:program" && e.p.Steps[i].Step == "unit" {
e.p.Steps[i].Outcome = seat.StepResultOutcomeFailed
}
}
store[pkey] = e
if _, err := v.Rerun("operator", seat.RerunInput{Pipeline: pkey, Step: "unit", Artifact: "app:program", Why: "x"}); err != nil {
t.Fatal(err)
}
if len(started) != 1 {
t.Errorf("a rerun of a running pipeline said pipeline-started again: %+v", started)
}
}
@@ -237,7 +237,8 @@ func runSettingsOf(raw []byte) bool {
}
if len(m.Config) != len(want)+len(words) || m.Config["test-runs-cap-mib"]["maximum"] != float64(64) || m.Config["test-runs-cap-mib"]["default"] != float64(16) ||
m.Config["pipelines-at-once"]["minimum"] != float64(1) || m.Config["pipelines-at-once"]["maximum"] != float64(8) ||
m.Config["pipelines-at-once"]["default"] != float64(1) || m.Config["pipelines-waiting-cap"]["default"] != float64(64) {
m.Config["pipelines-at-once"]["default"] != float64(1) || m.Config["pipelines-waiting-cap"]["default"] != float64(64) ||
m.Config["widens-read-bound"]["default"] != "30s" {
return false
}
env := map[string]bool{}
+2 -2
View File
@@ -138,9 +138,9 @@
},
"widens-read-bound": {
"type": "string",
"description": "how long one read of a pipeline end's widens (the catalog's revision, the seat holders, the grants part) waits for its answer before it is cut and said could not run, a Go duration such as 5s (design 47 G A-472, A-472a).",
"description": "how long one read of a pipeline end's widens (the catalog's revision, the seat holders, the grants part) waits for its answer before it is cut and said could not run, a Go duration such as 30s (design 47 G A-472, A-472a, H2 H-A-478: long enough that a cold grants-for is not cut).",
"trusted": false,
"default": "5s"
"default": "30s"
},
"revisions-kept": {
"type": "number",
+1
View File
@@ -13,6 +13,7 @@ require (
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/containerimageregistry v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/gopackageregistry v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildgo v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildnode v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/npmpackageregistry v0.0.0-00010101000000-000000000000
@@ -0,0 +1,100 @@
// Package pipelinekey reads the key of a pipeline in the seat mesh-pipeline-runner's state pipelines, in every key form the
// seat defines, through the seat's contract (design 47 H2 H-A-485): one parse, used by the runner, the assigner and the view, so
// a pipeline started again at the head (`<branch>.<revision>.again.<n>`, H-A-479) or an unfinished mover's own pipeline
// (`<branch>.<revision>.unfinished.<module>`, H-A-398) is read as a take-in's is. A key of no form the seat defines is refused
// with an error, never read as an empty one.
package pipelinekey
import (
"fmt"
"strconv"
"strings"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
"git.novox.be/novox/mesh-sdk/go/keys"
)
// The key forms of a pipeline in the state pipelines (the seat's key forms, design 47 H §746, H-A-213 (2), H-A-398, H-A-479).
const (
FormRevision = "<branch>.<revision>"
FormRun = "<branch>.run.<run>"
FormPublished = "<branch>.<source>.published.<package>.<version>"
FormUnfinished = "<branch>.<revision>.unfinished.<module>"
FormAgain = "<branch>.<revision>.again.<n>"
)
// Key is a pipeline's key read: its form, its watched branch (unescaped), and the parts its form names: the revision (a
// published form's source), the run, the package and version (escaped, as the key holds them), the unfinished mover's
// module (unescaped) and the again form's n.
type Key struct {
Form string
Branch string
Revision int64
Run string
Package string
Version string
Module string
N int
}
// Parse reads key as a pipeline's key of one of the seat's forms; any other key is an error naming the forms.
func Parse(key string) (Key, error) {
parts := strings.Split(key, ".")
var k Key
var branch, revision, n, module string
ok := false
switch {
case len(parts) == 2:
branch, revision, ok = seat.ParsePipelinesByBranchRevisionKey(key)
k.Form = FormRevision
case len(parts) == 3 && parts[1] == "run":
branch, k.Run, ok = seat.ParsePipelinesByBranchRunRunKey(key)
k.Form = FormRun
case len(parts) == 4 && parts[2] == "unfinished":
branch, revision, module, ok = seat.ParsePipelinesByBranchRevisionUnfinishedModuleKey(key)
k.Form = FormUnfinished
case len(parts) == 4 && parts[2] == "again":
branch, revision, n, ok = seat.ParsePipelinesByBranchRevisionAgainNKey(key)
k.Form = FormAgain
case len(parts) == 5 && parts[2] == "published":
branch, revision, k.Package, k.Version, ok = seat.ParsePipelinesByBranchSourcePublishedPackageVersionKey(key)
k.Form = FormPublished
}
if !ok {
return Key{}, refused(key, "it is of no form the seat defines")
}
var err error
if k.Branch, err = keys.UnescapeName(branch); err != nil || k.Branch == "" {
return Key{}, refused(key, "its branch does not read")
}
if k.Form != FormRun {
if k.Revision, err = strconv.ParseInt(revision, 10, 64); err != nil || k.Revision <= 0 {
return Key{}, refused(key, "its revision is no number above 0")
}
}
switch k.Form {
case FormRun:
if k.Run == "" {
return Key{}, refused(key, "it names no run")
}
case FormUnfinished:
if k.Module, err = keys.UnescapeName(module); err != nil || k.Module == "" {
return Key{}, refused(key, "its module does not read")
}
case FormAgain:
if k.N, err = strconv.Atoi(n); err != nil || k.N < 1 {
return Key{}, refused(key, "its n is no number from 1")
}
case FormPublished:
if k.Package == "" || k.Version == "" {
return Key{}, refused(key, "it names no package or no version")
}
}
return k, nil
}
// refused is the error of a key Parse does not read.
func refused(key, why string) error {
return fmt.Errorf("%q is no key of a pipeline of the seat %s (%s): %s", key, seat.Seat,
strings.Join([]string{FormRevision, FormRun, FormPublished, FormUnfinished, FormAgain}, ", "), why)
}
@@ -0,0 +1,43 @@
package pipelinekey
import (
"strings"
"testing"
)
// H-A-485: every key form of a pipeline the seat mesh-pipeline-runner defines is read, the again form and the unfinished form as
// a take-in's, its branch and module unescaped; a key of no form, or whose parts do not read, is refused naming why.
func TestEveryPipelineKeyFormIsReadAndAnyOtherRefused(t *testing.T) {
for _, c := range []struct {
key string
want Key
why string
}{
{key: "main.265", want: Key{Form: FormRevision, Branch: "main", Revision: 265}},
{key: "main.265.again.2", want: Key{Form: FormAgain, Branch: "main", Revision: 265, N: 2}},
{key: "feat=2fx.12.unfinished.mesh-tools", want: Key{Form: FormUnfinished, Branch: "feat/x", Revision: 12, Module: "mesh-tools"}},
{key: "main.run.r7", want: Key{Form: FormRun, Branch: "main", Run: "r7"}},
{key: "main.40.published.lib.0=2e1=2e2", want: Key{Form: FormPublished, Branch: "main", Revision: 40, Package: "lib", Version: "0=2e1=2e2"}},
{key: "main", why: "no form"},
{key: "main.265.again", why: "no form"},
{key: "main.265.later.2", why: "no form"},
{key: "main.265.again.0", why: "its n"},
{key: "main.265.again.x", why: "its n"},
{key: "main.r1", why: "its revision"},
{key: "main.0", why: "its revision"},
{key: "ma=zz.5", why: "its branch"},
{key: "main.5.unfinished.=zz", why: "its module"},
{key: "main.5.6.7.8.9", why: "no form"},
} {
got, err := Parse(c.key)
if c.why != "" {
if err == nil || !strings.Contains(err.Error(), c.why) || !strings.Contains(err.Error(), FormAgain) {
t.Errorf("%s: read %+v (%v); want it refused, %s, naming the forms", c.key, got, err, c.why)
}
continue
}
if err != nil || got != c.want {
t.Errorf("%s: read %+v (%v); want %+v", c.key, got, err, c.want)
}
}
}
@@ -3,6 +3,8 @@
// to write the operator's answer to an ask that needs no proof of who answered (the view refuses the merge's Approve). `holds`
// lists every fingerprint the assigner holds with its node, cause and answer; `hold-release <module> <fingerprint> [words]` and
// `hold-keep <module> <fingerprint> [words]` answer its hold ask, again when the operator changes their mind (issue 1906).
// `pipelines [branch]` and `pipeline <pipeline> [mover]` are the pipelines page (design 47 H2 H-A-467): a branch's newest
// pipelines, one pipeline's mover rows or one mover's steps; with -follow printed again at each change the runner's events say.
package main
import (
@@ -13,7 +15,9 @@ import (
"os"
"strings"
"text/tabwriter"
"time"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
"mesh-delivery-view/internal/view"
)
@@ -31,9 +35,19 @@ func run(args []string, socket string, out, errOut io.Writer) int {
fs := flag.NewFlagSet("mesh-delivery-cli", flag.ContinueOnError)
fs.SetOutput(errOut)
asJSON := fs.Bool("json", false, "print the rows as JSON")
follow := fs.Bool("follow", false, "pipelines and pipeline: print again at each change the runner's events say, until interrupted")
next := fs.String("next", "", "pipeline: the cursor of the next page a pipeline's answer named")
limit := fs.Int("limit", view.DefaultPipelinesLimit, "pipelines: the newest this many pipelines of the branch")
if err := fs.Parse(args); err != nil {
return 2
}
if fs.NArg() > 0 && (fs.Arg(0) == "pipelines" || fs.Arg(0) == "pipeline") {
if *limit < 1 {
fmt.Fprintln(errOut, "mesh-delivery-cli: -limit is a number of at least one")
return 2
}
return pipelines(socket, fs.Args(), *next, *limit, *follow, *asJSON, out, errOut)
}
if fs.NArg() > 0 && fs.Arg(0) == "holds" {
return holds(socket, *asJSON, out, errOut)
}
@@ -53,7 +67,7 @@ func run(args []string, socket string, out, errOut io.Writer) int {
}
if fs.NArg() > 0 {
if fs.Arg(0) != "answer" || fs.NArg() < 3 {
fmt.Fprintln(errOut, "mesh-delivery-cli shows the rows, or answers: mesh-delivery-cli answer <ask key> <option> [words]; holds; hold-release|hold-keep <module> <fingerprint> [words]")
fmt.Fprintln(errOut, "mesh-delivery-cli shows the rows, or answers: mesh-delivery-cli answer <ask key> <option> [words]; holds; hold-release|hold-keep <module> <fingerprint> [words]; [-follow] [-limit <n>] pipelines [branch]; [-follow] [-next <cursor>] pipeline <pipeline> [mover]")
return 2
}
key, option := fs.Arg(1), fs.Arg(2)
@@ -82,6 +96,61 @@ func run(args []string, socket string, out, errOut io.Writer) int {
return 0
}
// pipelines prints the pipelines page (design 47 H2 H-A-467): `pipelines [branch]` the newest pipelines of a branch (main by
// default), `pipeline <pipeline> [mover]` one pipeline's mover rows or one mover's steps; with -follow again at each change.
func pipelines(socket string, args []string, next string, limit int, follow, asJSON bool, out, errOut io.Writer) int {
var branch, pipeline, mover string
switch {
case args[0] == "pipelines" && len(args) <= 2:
if len(args) == 2 {
branch = args[1]
}
case args[0] == "pipeline" && (len(args) == 2 || len(args) == 3):
pipeline = args[1]
if len(args) == 3 {
mover = args[2]
}
default:
fmt.Fprintln(errOut, "mesh-delivery-cli [-follow] [-limit <n>] pipelines [branch]; mesh-delivery-cli [-follow] [-next <cursor>] pipeline <pipeline> [mover]")
return 2
}
var failed error
err := view.AskPipelines(socket, branch, limit, pipeline, mover, next, follow, func(page view.PipelinesPage, show *seatShow) bool {
if follow {
fmt.Fprintf(out, "--- %s ---\n", time.Now().Format("15:04:05"))
}
if asJSON {
enc := json.NewEncoder(out)
enc.SetIndent("", " ")
var v any = page
if show != nil {
v = show
}
if failed = enc.Encode(v); failed != nil {
return false
}
return true
}
if show != nil {
view.PipelineTable(out, *show, mover)
} else {
view.PipelinesTable(out, page)
}
return true
})
if err == nil {
err = failed
}
if err != nil {
fmt.Fprintln(errOut, "mesh-delivery-cli:", err)
return 1
}
return 0
}
// seatShow is the seat's answer of show, as the view hands it on.
type seatShow = seat.ShowAnswer
// holds prints every hold: module, fingerprint, node, since, answer, words.
func holds(socket string, asJSON bool, out, errOut io.Writer) int {
rows, err := view.AskHolds(socket)
@@ -22,6 +22,7 @@ import (
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshdeliveryclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshwantsclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
pseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-view/viewstate"
)
@@ -304,3 +305,63 @@ func rawAsk(t *testing.T, a oc.Ask) json.RawMessage {
}
const testView = "acme-view"
// Design 47 H2 H-A-467: the terminal view's pipelines page: a branch's newest pipelines, and one pipeline's mover rows.
func TestThePipelinesCommands(t *testing.T) {
m := view.New(testView)
call := func(key string, body any) (json.RawMessage, error) {
switch key {
case view.PipelinesListVerb:
in := body.(pseat.ListInput)
if in.Branch != "next" {
return json.Marshal(pseat.ListAnswer{})
}
rows := []pseat.PipelineRow{{Pipeline: "next.3", Source: 2, Revision: 3, Outcome: "passed", MoverCount: 1, Steps: map[string]int64{"passed": 4}},
{Pipeline: "next.2", Source: 2, Revision: 2, Outcome: "superseded", MoverCount: 1}}
return json.Marshal(pseat.ListAnswer{Rows: rows[:min(int(in.Limit), len(rows))]})
case view.PipelinesShowVerb:
return json.Marshal(pseat.ShowAnswer{Pipeline: pseat.Pipeline{Pipeline: "next.3", Outcome: "passed", Movers: []pseat.Mover{{Module: "alpha", Fingerprint: "src2:aa", Version: "0.2.1"}}},
Movers: []pseat.MoverRow{{Module: "alpha", Outcome: "passed"}}})
}
return nil, nil
}
m.SetPipelines(view.NewPipelines(call, nil, 0, nil))
dir, err := os.MkdirTemp(os.Getenv("TMPDIR"), "p")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { os.RemoveAll(dir) })
l, err := view.ListenSocket(filepath.Join(dir, "cli.sock"))
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
t.Cleanup(cancel)
go view.ServeSocket(ctx, l, m)
path := filepath.Join(dir, "cli.sock")
for _, r := range []struct {
args []string
want string
}{
{[]string{"pipelines", "next"}, "next.3 3 2 passed 1 4 0 0 0 0 0 0 0"},
{[]string{"pipelines", "next"}, "next.2"}, // the newest 20 by default
{[]string{"-limit", "1", "pipelines", "next"}, "next.3"},
{[]string{"pipeline", "next.3"}, "alpha src2:aa 0.2.1 passed"},
{[]string{"-json", "pipeline", "next.3"}, `"fingerprint": "src2:aa"`},
} {
var out, errOut bytes.Buffer
if code := run(r.args, path, &out, &errOut); code != 0 || !strings.Contains(strings.Join(strings.Fields(out.String()), " "), strings.Join(strings.Fields(r.want), " ")) {
t.Errorf("%v = %d %q %q, want it to hold %q", r.args, code, out.String(), errOut.String(), r.want)
}
}
var out, errOut bytes.Buffer
if code := run([]string{"-limit", "1", "pipelines", "next"}, path, &out, &errOut); code != 0 || strings.Contains(out.String(), "next.2") {
t.Errorf("-limit 1 = %d %q, want only the newest", code, out.String())
}
for _, args := range [][]string{{"pipeline"}, {"-limit", "0", "pipelines"}} {
out.Reset()
if code := run(args, path, &out, &errOut); code != 2 {
t.Errorf("%v = %d, want 2", args, code)
}
}
}
@@ -27,9 +27,11 @@ import (
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-assigner/asks"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshdeliveryclient"
pclient "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshpipelinerunnerclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshwantsclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshdelivery"
pseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/buildstep"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-view/viewstate"
"git.novox.be/novox/mesh-sdk/go/process"
@@ -888,6 +890,78 @@ func TestProcessWholeSaysTheSourceRevisionIsDelivered(t *testing.T) {
if stored, gerr := kvs[viewstate.BucketsOf(testView).Reach].Get(ctx, re.Key); gerr != nil || string(stored.Value()) != string(re.Reach) {
t.Errorf("the event carries %s, the state reach holds %v (%v)", re.Reach, stored, gerr)
}
// Design 47 H2 H-A-467: the pipelines page reads the seat mesh-pipeline-runner's list, again at the runner's event heard on
// the module's one pull, and the delivered source revision links to the newest pipeline of its source.
pipelines := pipelinesRunner(t, nc)
pipelines.set(pseat.PipelineRow{Pipeline: "release.9", Source: rev, Revision: 9, Outcome: "passed", MoverCount: 1},
pseat.PipelineRow{Pipeline: "release.8", Source: rev, Revision: 8, Outcome: "superseded", MoverCount: 1})
var seen [][]string
err = view.AskPipelines(env["MESH_DELIVERY_SOCKET"], catalogBranch, 0, "", "", "", true, func(page view.PipelinesPage, _ *pseat.ShowAnswer) bool {
var keys []string
for _, r := range page.Rows {
keys = append(keys, r.Pipeline)
}
seen = append(seen, keys)
if len(seen) == 1 {
pipelines.set(pseat.PipelineRow{Pipeline: "release.10", Source: rev + 1, Revision: 10, Outcome: "running", MoverCount: 1},
pseat.PipelineRow{Pipeline: "release.9", Source: rev, Revision: 9, Outcome: "passed", MoverCount: 1})
body, _ := json.Marshal(pseat.PipelineEndedEvent{Pipeline: "release.9", Branch: catalogBranch, Source: rev, Revision: 9, Outcome: "passed"})
if _, err := js.Publish(ctx, pclient.RevisionSubject(pseat.EventPipelineEnded, catalogBranch, 9), body); err != nil {
t.Error(err)
return false
}
}
return len(seen) < 2 || !slices.Contains(keys, "release.10")
})
if err != nil || len(seen) < 2 || seen[len(seen)-1][0] != "release.10" {
t.Fatalf("the pipelines followed = %v, %v; want release.10 after the runner's event", seen, err)
}
snap, err := view.Ask(env["MESH_DELIVERY_SOCKET"])
if err != nil {
t.Fatal(err)
}
for _, src := range snap.Sources {
if src.Branch == catalogBranch && src.Pipeline != "release.9" {
t.Errorf("the delivery of %s's source revision %d links to %q, want release.9", src.Branch, src.Revision, src.Pipeline)
}
}
}
// fakeList is the seat mesh-pipeline-runner's verb list as its holder answers it on the bus, from the rows the test sets.
type fakeList struct {
mu sync.Mutex
rows []pseat.PipelineRow
}
func (f *fakeList) set(rows ...pseat.PipelineRow) {
f.mu.Lock()
defer f.mu.Unlock()
f.rows = rows
}
func pipelinesRunner(t *testing.T, nc *nats.Conn) *fakeList {
t.Helper()
f := &fakeList{}
subject := "mesh.seat." + pseat.Seat + ".call." + pseat.VerbList
for _, s := range []string{subject, subject + ".>"} {
sub, err := nc.Subscribe(s, func(m *nats.Msg) {
var in pseat.ListInput
_ = json.Unmarshal(m.Data, &in)
f.mu.Lock()
a := pseat.ListAnswer{}
if in.Branch == catalogBranch && in.Limit == view.DefaultPipelinesLimit {
a.Rows = f.rows
}
f.mu.Unlock()
raw, _ := json.Marshal(map[string]any{"result": a, "node": "n0"})
_ = m.Respond(raw)
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = sub.Unsubscribe() })
}
return f
}
// 1906/D10, whole on the bus (through run): a want that differs from its have is said `moving`, the have reaching it running and
@@ -15,6 +15,7 @@ import (
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"io"
"io/fs"
"log"
"maps"
"mesh-delivery-view/internal/view"
@@ -22,6 +23,7 @@ import (
"os"
"os/signal"
"slices"
"strconv"
"strings"
"sync"
"syscall"
@@ -191,6 +193,18 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
if err := hearWaits(ctx, hub, js, own.Waits, logger.Printf); err != nil {
return fmt.Errorf("the waits' events are not handled: %w", err)
}
// The pipelines page reads the seat mesh-pipeline-runner's list and show, and reads again on the runner's events heard on
// the module's one pull, never on a timer (design 47 H2 H-A-467); the watched branches' lists link each delivery.
heard, err := numberFile(getenv("MESH_DELIVERY_VIEW_PIPELINES_HEARD_FILE"), view.DefaultPipelinesHeard)
if err != nil {
return fmt.Errorf("the setting pipelines-heard: %w", err)
}
pipes := view.NewPipelines(calls.Call, branches.Followed, heard, logger.Printf)
model.SetPipelines(pipes)
if err := hearPipelines(hub, pipes, logger.Printf); err != nil {
return fmt.Errorf("the runner's events are not handled: %w", err)
}
go pipes.Run(ctx)
if err := hub.StartUntilBound(ctx, js, cred, nil, nil); err != nil {
logger.Printf("the consumer is not heard yet (tried again until it is): %v", err)
}
@@ -536,3 +550,27 @@ func (c *catalogModules) apply(module string, entry meshcatalog.CatalogEntry, li
e.History = true
c.model.Apply(e)
}
// numberFile reads a setting the node-engine writes into a file of the state directory: def when path is empty, the file
// missing or empty; an error when it holds no number of at least one.
func numberFile(path string, def int) (int, error) {
if path == "" {
return def, nil
}
raw, err := os.ReadFile(path)
if errors.Is(err, fs.ErrNotExist) {
return def, nil
}
if err != nil {
return 0, err
}
v := strings.TrimSpace(string(raw))
if v == "" {
return def, nil
}
n, err := strconv.Atoi(v)
if err != nil || n < 1 {
return 0, fmt.Errorf("%s holds %q, which is not a number of at least one", path, v)
}
return n, nil
}
@@ -4,6 +4,8 @@ import (
"context"
"encoding/json"
"io"
"os"
"path/filepath"
"strings"
"testing"
@@ -17,6 +19,37 @@ func TestRunNeedsItsCredential(t *testing.T) {
}
}
// The setting pipelines-heard is read from the file the node-engine writes, its default where none is written (principle 11).
func TestNumberFileReadsASetting(t *testing.T) {
dir := t.TempDir()
write := func(name, v string) string {
p := filepath.Join(dir, name)
if err := os.WriteFile(p, []byte(v), 0o600); err != nil {
t.Fatal(err)
}
return p
}
for _, r := range []struct {
name, path string
want int
fails bool
}{
{"no file named is the default", "", 1024, false},
{"a file not written is the default", filepath.Join(dir, "none"), 1024, false},
{"an empty file is the default", write("empty", "\n"), 1024, false},
{"a number is read", write("seven", "7\n"), 7, false},
{"zero is refused", write("zero", "0"), 0, true},
{"words are refused", write("words", "many"), 0, true},
} {
t.Run(r.name, func(t *testing.T) {
n, err := numberFile(r.path, 1024)
if (err != nil) != r.fails || n != r.want {
t.Errorf("numberFile = %d, %v; want %d, failing %v", n, err, r.want, r.fails)
}
})
}
}
// rawAsk is the router's ask as the JSON a gate's or an assigner's value carries (mesh-sdk keeps it raw, ADR 0627 section 4).
func rawAsk(t *testing.T, a oc.Ask) json.RawMessage {
t.Helper()
@@ -7,6 +7,8 @@ import (
"fmt"
"strings"
"mesh-delivery-view/internal/view"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-view/viewstate"
"git.novox.be/novox/mesh-sdk/go/process"
"github.com/nats-io/nats.go/jetstream"
@@ -92,3 +94,20 @@ func hearWaits(ctx context.Context, hub *process.EventHub, js jetstream.JetStrea
}
return nil
}
// hearPipelines registers the handlers of the runner's events the pipelines page reads again on (design 47 H2 H-A-467): each
// is acknowledged as heard, since the page reads the runner's verbs and keeps nothing of the event; one that names no
// pipeline is said and acknowledged, as no delivery of it again could name one.
func hearPipelines(hub *process.EventHub, pipes *view.Pipelines, logf func(string, ...any)) error {
for _, pattern := range view.PipelineEventPatterns() {
if err := hub.Handle(pattern, func(m jetstream.Msg) {
if err := pipes.Hear(m.Data()); err != nil {
logf("the runner's event %s is not read: %v", m.Subject(), err)
}
_ = m.Ack()
}); err != nil {
return err
}
}
return nil
}
+2
View File
@@ -13,9 +13,11 @@ require (
git.novox.be/novox/mesh-contracts/provisioning/go/clients/meshprovisionerclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshdeliveryclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshpipelinerunnerclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshwantsclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshdelivery v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-assigner/asks v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds v0.0.0-00010101000000-000000000000
@@ -0,0 +1,293 @@
package view
import (
"context"
"encoding/json"
"fmt"
"slices"
"strings"
"sync"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// The seat mesh-pipeline-runner's verbs list and show as the view's invokes names them: the pipelines page reads the
// pipelines through them and keeps no state of its own (design 47 H2 H-A-467 (1)).
const (
PipelinesListVerb = "seat:" + seat.Seat + "." + seat.VerbList
PipelinesShowVerb = "seat:" + seat.Seat + "." + seat.VerbShow
)
// The pipelines page's defaults (design 47 H2 H-A-467 (1)): the branch main and the newest 20.
const (
DefaultPipelinesBranch = "main"
DefaultPipelinesLimit = 20
// DefaultPipelinesHeard is the default of the setting pipelines-heard: how many pipelines' last changes the view keeps
// for its pages before it forgets them all (principle 11).
DefaultPipelinesHeard = 1024
)
// PipelineEvents are the runner's events the pipelines page refreshes on (design 47 H2 H-A-467 (1)), never a timer.
var PipelineEvents = []string{seat.EventPipelineStarted, seat.EventStepEnded, seat.EventPipelineEnded, seat.EventRePinned}
// PipelineEventPatterns are the subjects of PipelineEvents on the process's hub: each event, with and without the names
// its sayer adds after the event's name.
func PipelineEventPatterns() []string {
var out []string
for _, e := range PipelineEvents {
out = append(out, "mesh.seat."+seat.Seat+".event."+e, "mesh.seat."+seat.Seat+".event."+e+".>")
}
return out
}
// PipelinesPage is one branch's list as the page shows it: the rows list answered, or why it could not be read.
type PipelinesPage struct {
Branch string `json:"branch"`
Rows []seat.PipelineRow `json:"rows"`
Error string `json:"error,omitempty"`
}
// Pipelines reads the runner's pipelines through its seat's verbs and says when they changed, on the events it hears
// (design 47 H2 H-A-467). It holds the last list it read of each branch it follows, the answer of a bus call and no state
// of the view's own: what it read is read again at each event.
type Pipelines struct {
call func(key string, body any) (json.RawMessage, error)
branches func() []string // the watched branches followed beside the default; nil for none
logf func(string, ...any)
dirty chan struct{}
mu sync.Mutex
lists map[string]PipelinesPage // by branch, the branches followed
subs map[chan string]struct{}
version uint64
heard map[string]uint64 // the version of the last change of each pipeline heard
most int // the pipelines heard kept before heard is forgotten whole: the setting pipelines-heard
}
// NewPipelines is a reader of the pipelines through call; branches, when given, names the watched branches whose list is
// followed beside the default branch's, for the deliveries' links (design 47 H2 H-A-467 (3)); heard is the setting
// pipelines-heard, its default below one.
func NewPipelines(call func(key string, body any) (json.RawMessage, error), branches func() []string, heard int, logf func(string, ...any)) *Pipelines {
if logf == nil {
logf = func(string, ...any) {}
}
if heard < 1 {
heard = DefaultPipelinesHeard
}
return &Pipelines{call: call, branches: branches, logf: logf, dirty: make(chan struct{}, 1),
lists: map[string]PipelinesPage{}, subs: map[chan string]struct{}{}, heard: map[string]uint64{}, most: heard}
}
// List asks the seat's verb list for one branch's newest limit pipelines in its short form; a limit below one is the
// default's.
func (p *Pipelines) List(branch string, limit int) ([]seat.PipelineRow, error) {
if limit < 1 {
limit = DefaultPipelinesLimit
}
raw, err := p.call(PipelinesListVerb, seat.ListInput{Branch: branch, Limit: int64(limit)})
if err != nil {
return nil, fmt.Errorf("%s of %s: %w", PipelinesListVerb, branch, err)
}
answer, said, ok := buscalls.Unwrap(raw)
if !ok {
return nil, fmt.Errorf("%s of %s answered: %s", PipelinesListVerb, branch, said)
}
var a seat.ListAnswer
if err := json.Unmarshal(answer, &a); err != nil {
return nil, fmt.Errorf("the answer of %s of %s is not a list: %w", PipelinesListVerb, branch, err)
}
return a.Rows, nil
}
// Show asks the seat's verb show for one pipeline: naming no mover, its movers' rows; naming one, that mover's steps
// (design 47 H2 H-A-467 (2)); next is the cursor of a page show answered.
func (p *Pipelines) Show(pipeline, mover, next string) (seat.ShowAnswer, error) {
raw, err := p.call(PipelinesShowVerb, seat.ShowInput{Pipeline: pipeline, Mover: mover, Next: next})
if err != nil {
return seat.ShowAnswer{}, fmt.Errorf("%s of %s: %w", PipelinesShowVerb, pipeline, err)
}
answer, said, ok := buscalls.Unwrap(raw)
if !ok {
return seat.ShowAnswer{}, fmt.Errorf("%s of %s answered: %s", PipelinesShowVerb, pipeline, said)
}
var a seat.ShowAnswer
if err := json.Unmarshal(answer, &a); err != nil {
return a, fmt.Errorf("the answer of %s of %s is not a pipeline: %w", PipelinesShowVerb, pipeline, err)
}
return a, nil
}
// followed is the default branch and the watched branches, each once.
func (p *Pipelines) followed() []string {
out := []string{DefaultPipelinesBranch}
if p.branches != nil {
for _, b := range p.branches() {
if b != "" && !slices.Contains(out, b) {
out = append(out, b)
}
}
}
return out
}
// Page is one branch's newest limit pipelines (the default's below one): a followed branch's at the default limit as last
// read, any other branch or limit read now.
func (p *Pipelines) Page(branch string, limit int) PipelinesPage {
if branch = strings.TrimSpace(branch); branch == "" {
branch = DefaultPipelinesBranch
}
if limit < 1 {
limit = DefaultPipelinesLimit
}
if limit == DefaultPipelinesLimit {
p.mu.Lock()
page, ok := p.lists[branch]
p.mu.Unlock()
if ok {
return page
}
}
return p.read(branch, limit)
}
func (p *Pipelines) read(branch string, limit int) PipelinesPage {
rows, err := p.List(branch, limit)
page := PipelinesPage{Branch: branch, Rows: rows}
if err != nil {
page.Error = err.Error()
}
if page.Rows == nil {
page.Rows = []seat.PipelineRow{}
}
return page
}
// Refresh reads the list of every followed branch again and says the lists changed.
func (p *Pipelines) Refresh() {
lists := map[string]PipelinesPage{}
for _, b := range p.followed() {
page := p.read(b, DefaultPipelinesLimit)
if page.Error != "" {
p.logf("the pipelines of %s are not read (read again at the runner's next event): %s", b, page.Error)
}
lists[b] = page
}
p.mu.Lock()
p.lists = lists
p.mu.Unlock()
p.notify("")
}
// Of is the pipeline of a delivery: of the branch's followed list, the newest whose source is the delivery's source
// revision (design 47 H2 H-A-467 (3)); "" when none. A branch whose list is not read yet (a watched branch the reader
// learnt of after its last reading) has the lists read again, once.
func (p *Pipelines) Of(branch string, source uint64) string {
p.mu.Lock()
defer p.mu.Unlock()
page, ok := p.lists[branch]
if !ok {
p.owe()
}
for _, r := range page.Rows { // newest first, as list answers them
if source != 0 && r.Source == int64(source) {
return r.Pipeline
}
}
return ""
}
// Hear takes one event of the runner's (PipelineEvents): the pages of its pipeline are told at once, and the lists are
// read again by Run, which then tells the list pages.
func (p *Pipelines) Hear(data []byte) error {
var ev struct {
Pipeline string `json:"pipeline"`
}
if err := json.Unmarshal(data, &ev); err != nil || ev.Pipeline == "" {
return fmt.Errorf("not an event of a pipeline: %.200s", data)
}
p.notify(ev.Pipeline)
p.owe()
return nil
}
// owe has Run read the lists again.
func (p *Pipelines) owe() {
select {
case p.dirty <- struct{}{}:
default: // a reading is owed already: it reads what this change asks too
}
}
// Run reads the lists at its start and again after each event heard, until ctx ends; events heard while it reads are
// read at once after, in one reading.
func (p *Pipelines) Run(ctx context.Context) {
p.Refresh()
for {
select {
case <-ctx.Done():
return
case <-p.dirty:
p.Refresh()
}
}
}
// notify tells every subscriber what changed: a pipeline's key, or "" for the lists.
func (p *Pipelines) notify(what string) {
p.mu.Lock()
defer p.mu.Unlock()
p.version++
if what != "" {
if _, known := p.heard[what]; !known && len(p.heard) >= p.most {
clear(p.heard)
}
p.heard[what] = p.version
} else {
p.heard[""] = p.version
}
for ch := range p.subs {
select {
case ch <- what:
default: // the subscriber is told already and has not looked yet
}
}
}
// Version is the count of changes said so far, given to a page so its subscription knows what it missed.
func (p *Pipelines) Version() uint64 {
p.mu.Lock()
defer p.mu.Unlock()
return p.version
}
// Changed waits until what (a pipeline's key, or "" for the lists) changes after the version since, or ctx ends: it
// answers at once when it changed already.
func (p *Pipelines) Changed(ctx context.Context, what string, since uint64) bool {
ch := make(chan string, 1)
p.mu.Lock()
if p.heard[what] > since {
p.mu.Unlock()
return true
}
p.subs[ch] = struct{}{}
p.mu.Unlock()
defer func() {
p.mu.Lock()
delete(p.subs, ch)
p.mu.Unlock()
}()
for {
select {
case <-ctx.Done():
return false
case <-ch: // any change: a note dropped while one waited is read from heard, never lost
p.mu.Lock()
changed := p.heard[what] > since
p.mu.Unlock()
if changed {
return true
}
}
}
}
@@ -0,0 +1,298 @@
package view
import (
"bufio"
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"time"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
)
// fakeRunner answers the seat mesh-pipeline-runner's list and show as the runner's process does, from rows and pipelines
// the test sets; it counts the lists asked.
type fakeRunner struct {
mu sync.Mutex
rows map[string][]seat.PipelineRow // by branch
shows map[string]seat.ShowAnswer // by pipeline, or pipeline/mover
lists int
}
func (f *fakeRunner) call(key string, body any) (json.RawMessage, error) {
f.mu.Lock()
defer f.mu.Unlock()
switch key {
case PipelinesListVerb:
in := body.(seat.ListInput)
if in.Limit < 1 || in.Full {
return nil, fmt.Errorf("list asked with limit %d, full %v", in.Limit, in.Full)
}
f.lists++
rows := f.rows[in.Branch]
return json.Marshal(seat.ListAnswer{Rows: rows[:min(int(in.Limit), len(rows))]})
case PipelinesShowVerb:
in := body.(seat.ShowInput)
k := in.Pipeline
if in.Mover != "" {
k += "/" + in.Mover
}
a, ok := f.shows[k]
if !ok {
return json.Marshal(map[string]any{"isError": true, "content": []map[string]string{{"type": "text", "text": "no pipeline " + k}}})
}
return json.Marshal(a)
}
return nil, fmt.Errorf("the runner serves no %s", key)
}
func (f *fakeRunner) setRows(branch string, rows ...seat.PipelineRow) {
f.mu.Lock()
defer f.mu.Unlock()
f.rows[branch] = rows
}
// changedWithin is p.Changed bounded, so a change never said fails the test instead of hanging it.
func changedWithin(ctx context.Context, p *Pipelines, what string, since uint64) bool {
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
return p.Changed(ctx, what, since)
}
// Design 47 H2 H-A-467: the pipelines page reads the runner's list and show, refreshes on its events, never on a timer,
// and links each delivery to its pipeline by its source revision.
func TestThePipelinesPage(t *testing.T) {
started, ended := t0, t0.Add(3*60e9)
runner := &fakeRunner{rows: map[string][]seat.PipelineRow{}, shows: map[string]seat.ShowAnswer{
"main.12": {Pipeline: seat.Pipeline{Pipeline: "main.12", Branch: "main", Source: 9, Revision: 12, Outcome: seat.PipelineOutcomeFailed,
Started: &started, Ended: &ended, Movers: []seat.Mover{{Module: "alpha", Fingerprint: "src2:aa", Version: "0.1.4"}, {Module: "beta", Fingerprint: "src2:bb"}}},
Movers: []seat.MoverRow{{Module: "alpha", Outcome: seat.MoverRowOutcomePassed},
{Module: "beta", Outcome: seat.MoverRowOutcomeFailed, Failed: &seat.FailedStep{Step: "unit", Artifact: "beta:program", Outcome: "could not run", Why: "no build node"}}}},
"main.12/beta": {Pipeline: seat.Pipeline{Pipeline: "main.12", Branch: "main", Source: 9, Revision: 12, Outcome: seat.PipelineOutcomeFailed,
Steps: []seat.StepResult{{Module: "beta", Step: "unit", Artifact: "beta:program", Seat: "node-build-go", Node: "n1", Outcome: "could not run", Started: &started, Ended: &ended, Why: "no build node"}}}},
}}
runner.setRows("main", seat.PipelineRow{Pipeline: "main.12", Source: 9, Revision: 12, Started: &started, Outcome: seat.PipelineOutcomeFailed, MoverCount: 2,
Steps: map[string]int64{"passed": 5, "failed": 1, "could not run": 1, "waiting for a build node": 2}})
runner.setRows(testBranch, seat.PipelineRow{Pipeline: "release.8", Source: 4, Revision: 8, Outcome: seat.PipelineOutcomePassed},
seat.PipelineRow{Pipeline: "release.7", Source: 4, Revision: 7, Outcome: seat.PipelineOutcomeSuperseded})
p := NewPipelines(runner.call, func() []string { return []string{testBranch} }, 0, nil)
m := New(testView)
fill(m, sourceE(t, 4))
m.SetPipelines(p)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Run(ctx)
if !changedWithin(ctx, p, "", 0) { // the first reading at the start
t.Fatal("the lists were never read")
}
web := func(path string) string {
rec := httptest.NewRecorder()
Web(m).ServeHTTP(rec, httptest.NewRequest(http.MethodGet, path, nil))
if rec.Code != 200 {
t.Fatalf("%s = %d %s", path, rec.Code, rec.Body.String())
}
return rec.Body.String()
}
text := func(f func(*bytes.Buffer)) string {
var b bytes.Buffer
f(&b)
return strings.Join(strings.Fields(b.String()), " ")
}
rows := []struct {
name string
got func() string
want []string
}{
{"the terminal view's list counts the steps per outcome", func() string {
return text(func(b *bytes.Buffer) { PipelinesTable(b, p.Page("", 0)) })
}, []string{"PIPELINES OF main", "PASSED FAILED COULD NOT RUN NOT ASKED WAITING WAITING FOR A BUILD NODE BYPASSED NOTHING TO RUN",
"main.12 12 9 2026-10-18", "failed 2 5 1 1 0 0 2 0 0"}},
{"the web list links each pipeline and reloads on the view's changes, never a timer", func() string { return web("/pipelines") }, []string{
`<a href="/pipelines/show?pipeline=main.12">main.12</a>`, `new EventSource("/pipelines/changes?since=`}},
{"a pipeline's page is its mover rows, each linking its steps", func() string { return web("/pipelines/show?pipeline=main.12") }, []string{
`<a href="/pipelines/show?mover=alpha&amp;pipeline=main.12">alpha</a></td><td>src2:aa</td><td>0.1.4</td><td>passed</td>`,
"<td>unit beta:program: could not run: no build node</td>", `changes?pipeline=main.12\u0026since=`}},
{"a mover's page is its steps", func() string { return web("/pipelines/show?pipeline=main.12&mover=beta") }, []string{
"<td>unit beta:program</td><td>node-build-go</td><td>n1</td><td>could not run</td>", "<td>no build node</td>"}},
{"the terminal view's pipeline names each mover's fingerprint, version and failed step", func() string {
a, err := p.Show("main.12", "", "")
if err != nil {
return err.Error()
}
return text(func(b *bytes.Buffer) { PipelineTable(b, a, "") })
}, []string{"PIPELINE main.12: failed, source 9, revision 12", "alpha src2:aa 0.1.4 passed", "beta src2:bb failed unit beta:program: could not run: no build node"}},
{"the web list asks the newest limit of a branch", func() string { return web("/pipelines?branch=" + testBranch + "&limit=1") }, []string{
"release.8", `name="limit" value="1"`}},
{"the web list is the newest 20 by default", func() string { return web("/pipelines?branch=" + testBranch) }, []string{
"release.8", "release.7", `name="limit" value="20"`}},
{"a pipeline the runner does not hold says so", func() string { return web("/pipelines/show?pipeline=main.99") }, []string{"unreadable: ", "no pipeline main.99"}},
{"a delivery links to the newest pipeline of its source revision", func() string {
return text(func(b *bytes.Buffer) { Table(b, m.Snapshot()) }) + web("/")
}, []string{testBranch + ": source revision 4: delivered (pipeline release.8)", `(<a href="/pipelines/show?pipeline=release.8">pipeline</a>)`, `<a href="/pipelines">Pipelines</a>`}},
}
for _, r := range rows {
t.Run(r.name, func(t *testing.T) {
got := r.got()
for _, w := range r.want {
if !strings.Contains(got, w) {
t.Errorf("got %s\nwant it to hold %q", got, w)
}
}
})
}
if got := web("/pipelines?branch=" + testBranch + "&limit=1"); strings.Contains(got, "release.7") {
t.Errorf("the newest 1 shows release.7: %s", got)
}
for _, l := range []string{"0", "x"} {
rec := httptest.NewRecorder()
Web(m).ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/pipelines?limit="+l, nil))
if rec.Code != http.StatusBadRequest {
t.Errorf("limit %s = %d, want 400", l, rec.Code)
}
}
if strings.Contains(web("/pipelines"), "http-equiv=\"refresh\"") || strings.Contains(web("/pipelines/show?pipeline=main.12"), "http-equiv=\"refresh\"") {
t.Error("a pipelines page reloads on a timer")
}
t.Run("an event of the runner reads the lists again and tells the pages", func(t *testing.T) {
srv := httptest.NewServer(Web(m))
defer srv.Close()
v := p.Version()
res, err := http.Get(srv.URL + "/pipelines/changes?since=" + fmt.Sprint(v))
if err != nil {
t.Fatal(err)
}
defer res.Body.Close()
stream := bufio.NewReader(res.Body)
if line, _ := stream.ReadString('\n'); !strings.HasPrefix(line, ":") {
t.Fatalf("the stream starts %q", line)
}
shown := make(chan bool, 1)
go func() { shown <- changedWithin(ctx, p, "main.13", v) }()
runner.mu.Lock()
before := runner.lists
runner.mu.Unlock()
runner.setRows("main", seat.PipelineRow{Pipeline: "main.13", Source: 10, Revision: 13, Outcome: seat.PipelineOutcomeRunning, MoverCount: 1})
if err := p.Hear([]byte(`{"pipeline":"main.13","branch":"main","source":10}`)); err != nil {
t.Fatal(err)
}
if !<-shown {
t.Fatal("the page of the pipeline heard is not told")
}
for {
line, err := stream.ReadString('\n')
if err != nil {
t.Fatalf("the stream ended before the change: %v", err)
}
if line == "data: changed\n" {
break
}
}
runner.mu.Lock()
after := runner.lists
runner.mu.Unlock()
if !strings.Contains(web("/pipelines"), "main.13") || after <= before {
t.Errorf("the list was not read again (%d lists, %d before): %s", after, before, web("/pipelines"))
}
if err := p.Hear([]byte(`{"step":"unit"}`)); err == nil {
t.Error("an event naming no pipeline is taken")
}
})
t.Run("the terminal view follows the list on the socket, again at each event", func(t *testing.T) {
path, sm := served(t)
sm.SetPipelines(p)
var seen []string
heard := false
err := AskPipelines(path, "", 0, "", "", "", true, func(page PipelinesPage, _ *seat.ShowAnswer) bool {
seen = append(seen, page.Rows[0].Pipeline)
if !heard {
heard = true
runner.setRows("main", seat.PipelineRow{Pipeline: "main.14", Source: 11, Revision: 14, Outcome: seat.PipelineOutcomeRunning, MoverCount: 1})
if err := p.Hear([]byte(`{"pipeline":"main.14","mover":"alpha","revision":14}`)); err != nil { // a re-pinned names no branch
t.Error(err)
}
return true
}
return len(seen) < 2 || seen[len(seen)-1] != "main.14"
})
if err != nil || seen[len(seen)-1] != "main.14" {
t.Fatalf("followed %v, %v", seen, err)
}
var show *seat.ShowAnswer
if err := AskPipelines(path, "", 0, "main.12", "beta", "", false, func(_ PipelinesPage, a *seat.ShowAnswer) bool { show = a; return true }); err != nil {
t.Fatal(err)
}
if len(show.Pipeline.Steps) != 1 || show.Pipeline.Steps[0].Node != "n1" {
t.Fatalf("the mover's steps on the socket = %+v", show)
}
if err := AskPipelines(path, "", 0, "main.99", "", "", false, func(PipelinesPage, *seat.ShowAnswer) bool { return true }); err == nil || !strings.Contains(err.Error(), "no pipeline main.99") {
t.Fatalf("a pipeline the runner does not hold = %v", err)
}
})
t.Run("the pipelines heard are kept up to the setting pipelines-heard", func(t *testing.T) {
few := NewPipelines(runner.call, nil, 2, nil)
for _, k := range []string{"main.1", "main.2", "main.2", "main.3"} {
few.notify(k)
}
few.mu.Lock()
n, last := len(few.heard), few.heard["main.3"]
few.mu.Unlock()
if n > 2 || last == 0 {
t.Errorf("heard holds %d pipelines (main.3 at %d), want at most the setting's 2, the newest kept", n, last)
}
if d := NewPipelines(runner.call, nil, 0, nil); d.most != DefaultPipelinesHeard {
t.Errorf("the default keeps %d, want %d", d.most, DefaultPipelinesHeard)
}
})
t.Run("a list not read is shown unreadable and followed on, read again at the next event", func(t *testing.T) {
var mu sync.Mutex
away := true // the runner not answering yet, as the bus says when no holder listens
call := func(key string, body any) (json.RawMessage, error) {
mu.Lock()
defer mu.Unlock()
if away {
return nil, fmt.Errorf("nats: no responders available for request")
}
return runner.call(key, body)
}
up := NewPipelines(call, nil, 0, nil)
upCtx, upCancel := context.WithCancel(ctx)
defer upCancel()
go up.Run(upCtx)
if !changedWithin(upCtx, up, "", 0) {
t.Fatal("the lists were never read")
}
path, sm := served(t)
sm.SetPipelines(up)
var seen []PipelinesPage
err := AskPipelines(path, "", 0, "", "", "", true, func(page PipelinesPage, _ *seat.ShowAnswer) bool {
seen = append(seen, page)
if len(seen) == 1 {
mu.Lock()
away = false
mu.Unlock()
if err := up.Hear([]byte(`{"pipeline":"main.14"}`)); err != nil {
t.Error(err)
}
}
return page.Error != "" || len(page.Rows) == 0
})
if err != nil || len(seen) < 2 || !strings.Contains(seen[0].Error, "no responders") || seen[len(seen)-1].Rows[0].Pipeline != "main.14" {
t.Fatalf("followed %+v, %v; want the unreadable list, then main.14 after the event", seen, err)
}
var b bytes.Buffer
PipelinesTable(&b, seen[0])
if !strings.Contains(b.String(), "unreadable: ") {
t.Errorf("the terminal view of an unreadable list = %q", b.String())
}
})
}
@@ -0,0 +1,286 @@
package view
import (
"fmt"
"html/template"
"io"
"net/http"
"net/url"
"strconv"
"strings"
"text/tabwriter"
"time"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
)
// PipelineCountOutcomes are the step outcomes a list row counts, in the page's order (design 47 H2 H-A-467 (2)).
var PipelineCountOutcomes = []string{seat.StepResultOutcomePassed, seat.StepResultOutcomeFailed, seat.StepResultOutcomeCouldNotRun,
seat.StepResultOutcomeNotAsked, seat.StepResultOutcomeWaiting, seat.StepResultOutcomeWaitingForABuildNode,
seat.StepResultOutcomeBypassed, seat.StepResultOutcomeNothingToRun}
// PipelinesHead is the head of the list's rows.
var PipelinesHead = append([]string{"PIPELINE", "REVISION", "SOURCE", "STARTED", "OUTCOME", "MOVERS"}, upper(PipelineCountOutcomes)...)
// MoversHead is the head of one pipeline's mover rows, StepsHead of one mover's steps (design 47 H2 H-A-467 (2)).
var (
MoversHead = []string{"MOVER", "FINGERPRINT", "VERSION", "OUTCOME", "FAILED STEP"}
StepsHead = []string{"STEP", "SEAT", "NODE", "OUTCOME", "STARTED", "ENDED", "WHY"}
)
func upper(in []string) []string {
out := make([]string, len(in))
for i, s := range in {
out[i] = strings.ToUpper(s)
}
return out
}
func timeOf(t *time.Time) string {
if t == nil {
return ""
}
return since(*t)
}
func number(n int64) string {
if n == 0 {
return ""
}
return strconv.FormatInt(n, 10)
}
// PipelineRowText is one list row's cells, in PipelinesHead's order.
func PipelineRowText(r seat.PipelineRow) []string {
out := []string{r.Pipeline, number(r.Revision), number(r.Source), timeOf(r.Started), r.Outcome, strconv.FormatInt(r.MoverCount, 10)}
for _, o := range PipelineCountOutcomes {
out = append(out, strconv.FormatInt(r.Steps[o], 10))
}
return out
}
func failedText(f *seat.FailedStep) string {
if f == nil {
return ""
}
s := f.Step
if f.Artifact != "" {
s += " " + f.Artifact
}
s += ": " + f.Outcome
if f.Why != "" {
s += ": " + f.Why
}
return s
}
// MoverRowsText is one pipeline's mover rows as show without a mover answers them: each row's module, outcome and failed
// step, beside the pipeline's mover of that module's fingerprint and published version.
func MoverRowsText(a seat.ShowAnswer) [][]string {
movers := map[string]seat.Mover{}
for _, m := range a.Pipeline.Movers {
movers[m.Module] = m
}
var out [][]string
for _, r := range a.Movers {
m := movers[r.Module]
out = append(out, []string{r.Module, m.Fingerprint, m.Version, r.Outcome, failedText(r.Failed)})
}
return out
}
// StepsText is one mover's steps as show with the mover answers them.
func StepsText(a seat.ShowAnswer) [][]string {
var out [][]string
for _, s := range a.Pipeline.Steps {
step := s.Step
if s.Artifact != "" {
step += " " + s.Artifact
}
out = append(out, []string{step, s.Seat, s.Node, s.Outcome, timeOf(s.Started), timeOf(s.Ended), s.Why})
}
return out
}
func table(w io.Writer, head []string, rows [][]string) {
tw := tabwriter.NewWriter(w, 0, 4, 2, ' ', 0)
fmt.Fprintln(tw, strings.Join(head, "\t"))
for _, r := range rows {
fmt.Fprintln(tw, strings.Join(r, "\t"))
}
tw.Flush()
}
// PipelinesTable prints one branch's newest pipelines, the terminal view's pipelines page.
func PipelinesTable(w io.Writer, p PipelinesPage) {
fmt.Fprintf(w, "PIPELINES OF %s\n", p.Branch)
if p.Error != "" {
fmt.Fprintf(w, "unreadable: %s\n", p.Error)
return
}
var rows [][]string
for _, r := range p.Rows {
rows = append(rows, PipelineRowText(r))
}
table(w, PipelinesHead, rows)
}
// PipelineTable prints one pipeline as show answered it for mover: its mover rows, or that mover's steps.
func PipelineTable(w io.Writer, a seat.ShowAnswer, mover string) {
p := a.Pipeline
fmt.Fprintf(w, "PIPELINE %s: %s, source %d, revision %d, started %s, ended %s\n", p.Pipeline, p.Outcome, p.Source, p.Revision, timeOf(p.Started), timeOf(p.Ended))
if p.Why != "" {
fmt.Fprintf(w, "why: %s\n", p.Why)
}
if mover == "" {
table(w, MoversHead, MoverRowsText(a))
} else {
fmt.Fprintf(w, "STEPS OF %s\n", mover)
table(w, StepsHead, StepsText(a))
}
if a.Next != "" {
fmt.Fprintf(w, "more: -next %s\n", a.Next)
}
}
const pipelinesStyle = `<style>body{font:14px sans-serif;margin:1em}table{border-collapse:collapse}td,th{padding:2px 10px;text-align:left}</style>`
// changesScript reloads the page when the view says its part changed, on the runner's events (design 47 H2 H-A-467): never
// on a timer.
const changesScript = `<script>new EventSource({{.Changes}}).onmessage=function(){location.reload()}</script>`
var pipelinesPage = template.Must(template.New("pl").Parse(`<!doctype html><html><head><meta charset="utf-8">
<title>Pipelines of {{.Branch}}</title>` + pipelinesStyle + `</head>
<body><p><a href="/">What each node runs</a></p><h1>PIPELINES OF {{.Branch}}</h1>
<form method="get" action="/pipelines">branch <input name="branch" value="{{.Branch}}"> newest <input name="limit" value="{{.Limit}}" size="4"> <button>show</button></form>
{{if .Error}}<p>unreadable: {{.Error}}</p>{{else}}<table><tr>{{range .Head}}<th>{{.}}</th>{{end}}</tr>
{{range .Rows}}<tr><td><a href="{{.Link}}">{{index .Cells 0}}</a></td>{{range slice .Cells 1}}<td>{{.}}</td>{{end}}</tr>{{end}}</table>{{end}}
` + changesScript + `</body></html>`))
var pipelinePage = template.Must(template.New("p1").Parse(`<!doctype html><html><head><meta charset="utf-8">
<title>Pipeline {{.Pipeline}}</title>` + pipelinesStyle + `</head>
<body><p><a href="/">What each node runs</a> · <a href="{{.Back}}">Pipelines of {{.Branch}}</a>{{if .Mover}} · <a href="{{.Up}}">Pipeline {{.Pipeline}}</a>{{end}}</p>
<h1>PIPELINE {{.Pipeline}}{{if .Mover}}: {{.Mover}}{{end}}</h1>
{{if .Error}}<p>unreadable: {{.Error}}</p>{{else}}<p>{{.Line}}</p>{{if .Why}}<p>why: {{.Why}}</p>{{end}}
<table><tr>{{range .Head}}<th>{{.}}</th>{{end}}</tr>
{{range .Rows}}<tr>{{if .Link}}<td><a href="{{.Link}}">{{index .Cells 0}}</a></td>{{else}}<td>{{index .Cells 0}}</td>{{end}}{{range slice .Cells 1}}<td>{{.}}</td>{{end}}</tr>{{end}}</table>
{{if .More}}<p><a href="{{.More}}">more</a></p>{{end}}{{end}}
` + changesScript + `</body></html>`))
type linkedRow struct {
Cells []string
Link string
}
// PipelineLink is the page of one pipeline, or of one of its movers.
func PipelineLink(pipeline, mover string) string {
q := url.Values{"pipeline": {pipeline}}
if mover != "" {
q.Set("mover", mover)
}
return "/pipelines/show?" + q.Encode()
}
func changesLink(pipeline string, version uint64) string {
q := url.Values{"since": {strconv.FormatUint(version, 10)}}
if pipeline != "" {
q.Set("pipeline", pipeline)
}
return "/pipelines/changes?" + q.Encode()
}
// webPipelines serves the pipelines pages (design 47 H2 H-A-467): the list of a branch, one pipeline's mover rows, one
// mover's steps, and the changes each page reloads on.
func webPipelines(m *Model, w http.ResponseWriter, r *http.Request) {
p := m.Pipelines()
if p == nil {
http.Error(w, "the view reads no pipelines yet", http.StatusServiceUnavailable)
return
}
q := r.URL.Query()
switch r.URL.Path {
case "/pipelines/changes":
changes(p, w, r, q.Get("pipeline"))
case "/pipelines/show":
key, mover := q.Get("pipeline"), q.Get("mover")
if key == "" {
http.Error(w, "name a pipeline", http.StatusBadRequest)
return
}
v := p.Version()
a, err := p.Show(key, mover, q.Get("next"))
data := map[string]any{"Pipeline": key, "Mover": mover, "Changes": changesLink(key, v), "Up": PipelineLink(key, "")}
if err != nil {
data["Error"] = err.Error()
} else {
pl := a.Pipeline
data["Branch"] = pl.Branch
data["Back"] = "/pipelines?" + url.Values{"branch": {pl.Branch}}.Encode()
data["Line"] = fmt.Sprintf("%s, source %d, revision %d, started %s, ended %s", pl.Outcome, pl.Source, pl.Revision, timeOf(pl.Started), timeOf(pl.Ended))
data["Why"] = pl.Why
var rows []linkedRow
if mover == "" {
data["Head"] = MoversHead
for _, c := range MoverRowsText(a) {
rows = append(rows, linkedRow{Cells: c, Link: PipelineLink(key, c[0])})
}
} else {
data["Head"] = StepsHead
for _, c := range StepsText(a) {
rows = append(rows, linkedRow{Cells: c})
}
}
data["Rows"] = rows
if a.Next != "" {
more := url.Values{"pipeline": {key}, "next": {a.Next}}
if mover != "" {
more.Set("mover", mover)
}
data["More"] = "/pipelines/show?" + more.Encode()
}
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
pipelinePage.Execute(w, data)
default:
limit := DefaultPipelinesLimit
if l := strings.TrimSpace(q.Get("limit")); l != "" {
n, err := strconv.Atoi(l)
if err != nil || n < 1 {
http.Error(w, "limit is a number of at least one", http.StatusBadRequest)
return
}
limit = n
}
v := p.Version()
page := p.Page(q.Get("branch"), limit)
var rows []linkedRow
for _, row := range page.Rows {
rows = append(rows, linkedRow{Cells: PipelineRowText(row), Link: PipelineLink(row.Pipeline, "")})
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
pipelinesPage.Execute(w, map[string]any{"Branch": page.Branch, "Error": page.Error, "Head": PipelinesHead, "Rows": rows, "Limit": limit, "Changes": changesLink("", v)})
}
}
// changes is a stream of server-sent events: one message each time the page's part changed (a pipeline's key, or "" for
// the lists) after the version the page was made at, until the browser leaves.
func changes(p *Pipelines, w http.ResponseWriter, r *http.Request, what string) {
fl, ok := w.(http.Flusher)
if !ok {
http.Error(w, "this server cannot stream", http.StatusInternalServerError)
return
}
since, _ := strconv.ParseUint(r.URL.Query().Get("since"), 10, 64)
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-store")
fmt.Fprint(w, ": the page reloads at the runner's next event\n\n")
fl.Flush()
for {
if !p.Changed(r.Context(), what, since) {
return
}
since = p.Version()
fmt.Fprint(w, "data: changed\n\n")
fl.Flush()
}
}
@@ -15,6 +15,8 @@ import (
"strings"
"text/tabwriter"
"time"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
)
// ListenSocket listens on the unix socket at path, in a directory of mode 0755, with mode 0666 on the socket: the
@@ -48,6 +50,14 @@ type ask struct {
Words string `json:"words,omitempty"`
Module string `json:"module,omitempty"`
FP string `json:"fp,omitempty"`
// The pipelines page's asks (design 47 H2 H-A-467): pipelines of a branch, or one pipeline, of a mover when named; follow
// keeps the connection and answers again at each change the runner's events say.
Branch string `json:"branch,omitempty"`
Limit int `json:"limit,omitempty"` // the newest this many of the branch, the default's when left out
Pipeline string `json:"pipeline,omitempty"`
Mover string `json:"mover,omitempty"`
Next string `json:"next,omitempty"`
Follow bool `json:"follow,omitempty"`
}
// ServeSocket answers {"ask":"rows"} on l, line-delimited JSON, until ctx ends, and {"ask":"answer","key":…,"option":…,"words":…}
@@ -74,11 +84,14 @@ func answer(c net.Conn, m *Model) {
return
}
var a ask
if json.Unmarshal(line, &a) != nil || (a.Ask != "rows" && a.Ask != "answer" && a.Ask != "holds" && a.Ask != "hold") {
json.NewEncoder(c).Encode(map[string]string{"error": `refused: this socket answers {"ask":"rows"}, {"ask":"answer"}, {"ask":"holds"} and {"ask":"hold"} and nothing else`})
if json.Unmarshal(line, &a) != nil || (a.Ask != "rows" && a.Ask != "answer" && a.Ask != "holds" && a.Ask != "hold" && a.Ask != "pipelines" && a.Ask != "pipeline") {
json.NewEncoder(c).Encode(map[string]string{"error": `refused: this socket answers {"ask":"rows"}, {"ask":"answer"}, {"ask":"holds"}, {"ask":"hold"}, {"ask":"pipelines"} and {"ask":"pipeline"} and nothing else`})
return
}
switch a.Ask {
case "pipelines", "pipeline":
answerPipelines(c, m, a)
return
case "holds":
json.NewEncoder(c).Encode(m.Holds())
return
@@ -253,12 +266,38 @@ func wantText(s Snapshot) [][]string {
func sourceText(s Snapshot) []string {
var out []string
for _, src := range s.Sources {
out = append(out, fmt.Sprintf("%s: source revision %d: %s", src.Branch, src.Revision, src.State))
line := fmt.Sprintf("%s: source revision %d: %s", src.Branch, src.Revision, src.State)
if src.Pipeline != "" { // its pipeline (design 47 H2 H-A-467 (3))
line += " (pipeline " + src.Pipeline + ")"
}
out = append(out, line)
out = append(out, src.Waiting...)
}
return out
}
// sourceLine is one line of the source revisions on the page, with the link to its pipeline when it has one.
type sourceLine struct {
Line string
Link string
}
// sourceLines are sourceText's lines on the page: each delivery's line links to its pipeline (design 47 H2 H-A-467 (3)).
func sourceLines(s Snapshot) []sourceLine {
var out []sourceLine
for _, src := range s.Sources {
l := sourceLine{Line: fmt.Sprintf("%s: source revision %d: %s", src.Branch, src.Revision, src.State)}
if src.Pipeline != "" {
l.Link = PipelineLink(src.Pipeline, "")
}
out = append(out, l)
for _, w := range src.Waiting {
out = append(out, sourceLine{Line: w})
}
}
return out
}
// backlogText is one line per language and kind: its waiting asks, the asks held by a node, the oldest waiting ask's age.
func backlogText(s Snapshot) [][]string {
var out [][]string
@@ -380,12 +419,12 @@ func (o OfflineRow) name() string {
var page = template.Must(template.New("p").Parse(`<!doctype html><html><head><meta charset="utf-8"><meta http-equiv="refresh" content="5">
<title>Delivery</title><style>body{font:14px sans-serif;margin:1em}table{border-collapse:collapse}td,th{padding:2px 10px;text-align:left}</style></head>
<body><h1>What each node runs</h1><table><tr>{{range .Head}}<th>{{.}}</th>{{end}}</tr>
<body><p><a href="/pipelines">Pipelines</a></p><h1>What each node runs</h1><table><tr>{{range .Head}}<th>{{.}}</th>{{end}}</tr>
{{range .Rows}}<tr>{{range .}}<td>{{.}}</td>{{end}}</tr>{{end}}</table>
{{if .Groups}}<h2>CANDIDATES</h2><table><tr><th>GROUP</th><th>PULL REQUEST</th><th>HEAD</th><th>MODULES</th></tr>{{range .Groups}}<tr>{{range .}}<td>{{.}}</td>{{end}}</tr>{{end}}</table>{{end}}
{{if .Asks}}<h2>ASKS</h2><table><tr><th>ASK</th><th>HEADLINE</th><th>WORDS</th><th>PULL REQUESTS</th><th>ANSWER</th></tr>{{range .Asks}}<tr>{{range .Cells}}<td>{{.}}</td>{{end}}<td>{{if .Options}}<form method="post" action="/answer"><input type="hidden" name="key" value="{{.Key}}"><input name="words" placeholder="words (optional)">{{range .Options}} <button name="option" value="{{.}}">{{.}}</button>{{end}}</form>{{end}}</td></tr>{{end}}</table>{{end}}
{{if .Backlog}}<h2>BACKLOG OF BUILD ASKS</h2><table><tr>{{range .BacklogHead}}<th>{{.}}</th>{{end}}</tr>{{range .Backlog}}<tr>{{range .}}<td>{{.}}</td>{{end}}</tr>{{end}}</table>{{end}}
{{if .Source}}<h2>SOURCE REVISION</h2><p>{{range .Source}}{{.}}<br>{{end}}</p>{{end}}{{if .Wants}}<h2>WANTS AGAINST HAVES</h2><table><tr>{{range .WantHead}}<th>{{.}}</th>{{end}}</tr>{{range .Wants}}<tr>{{range .}}<td>{{.}}</td>{{end}}</tr>{{end}}</table>{{end}}
{{if .Source}}<h2>SOURCE REVISION</h2><p>{{range .Source}}{{.Line}}{{if .Link}} (<a href="{{.Link}}">pipeline</a>){{end}}<br>{{end}}</p>{{end}}{{if .Wants}}<h2>WANTS AGAINST HAVES</h2><table><tr>{{range .WantHead}}<th>{{.}}</th>{{end}}</tr>{{range .Wants}}<tr>{{range .}}<td>{{.}}</td>{{end}}</tr>{{end}}</table>{{end}}
{{if .Fingerprints}}<h2>FINGERPRINTS</h2><table><tr><th>MODULE</th><th>STATE</th><th>INDEXER AGAINST BUILD</th></tr>{{range .Fingerprints}}<tr>{{range .}}<td>{{.}}</td>{{end}}</tr>{{end}}</table>{{end}}
{{if .Offline}}<h2>OFFLINE</h2><table>{{range .Offline}}<tr>{{range .}}<td>{{.}}</td>{{end}}</tr>{{end}}</table>{{end}}
{{if .Stuck}}<h2>STUCK</h2><table><tr><th>BUCKET</th><th>KEY</th><th>WAITED FOR</th><th>SINCE</th></tr>{{range .Stuck}}<tr>{{range .}}<td>{{.}}</td>{{end}}</tr>{{end}}</table>{{end}}
@@ -414,6 +453,10 @@ func Web(m *Model) http.Handler {
http.Error(w, "this view is read-only", http.StatusMethodNotAllowed)
return
}
if r.URL.Path == "/pipelines" || strings.HasPrefix(r.URL.Path, "/pipelines/") {
webPipelines(m, w, r)
return
}
if r.URL.Path == "/wants" {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
wantsPage.Execute(w, map[string]any{"Head": wantHead, "Wants": wantText(m.Snapshot())})
@@ -439,7 +482,7 @@ func Web(m *Model) http.Handler {
for _, u := range snap.Unreadable {
bad = append(bad, []string{u.Bucket, u.Key, u.Error})
}
page.Execute(w, map[string]any{"Head": head, "Rows": rows, "Groups": groupText(snap), "Asks": askPage(snap), "Fingerprints": fingerprintText(snap), "Wants": wantText(snap), "Source": sourceText(snap), "Backlog": backlogText(snap), "BacklogHead": backlogHead, "WantHead": wantHead, "Offline": off, "Stuck": stuck, "Unreadable": bad})
page.Execute(w, map[string]any{"Head": head, "Rows": rows, "Groups": groupText(snap), "Asks": askPage(snap), "Fingerprints": fingerprintText(snap), "Wants": wantText(snap), "Source": sourceLines(snap), "Backlog": backlogText(snap), "BacklogHead": backlogHead, "WantHead": wantHead, "Offline": off, "Stuck": stuck, "Unreadable": bad})
})
}
@@ -498,3 +541,101 @@ func answerForm(m *Model, w http.ResponseWriter, r *http.Request) {
}
http.Redirect(w, r, "/", http.StatusSeeOther)
}
// answerPipelines answers the pipelines page's asks: a branch's list, or one pipeline as show answers it; with follow, again
// at each change of its part the runner's events say (design 47 H2 H-A-467), until the asker leaves.
func answerPipelines(c net.Conn, m *Model, a ask) {
p := m.Pipelines()
enc := json.NewEncoder(c)
if p == nil {
enc.Encode(pipelinesRefusal{Refused: "the view reads no pipelines yet"})
return
}
if a.Ask == "pipeline" && a.Pipeline == "" {
enc.Encode(pipelinesRefusal{Refused: "name a pipeline"})
return
}
what := ""
if a.Ask == "pipeline" {
what = a.Pipeline
}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
if a.Follow {
c.SetDeadline(time.Time{})
go func() { // the asker leaving ends the following
io.Copy(io.Discard, c)
cancel()
}()
}
for {
v := p.Version()
var err error
if a.Ask == "pipelines" {
err = enc.Encode(p.Page(a.Branch, a.Limit))
} else if sa, serr := p.Show(a.Pipeline, a.Mover, a.Next); serr != nil {
err = enc.Encode(pipelinesRefusal{Refused: serr.Error()})
} else {
err = enc.Encode(sa)
}
if err != nil || !a.Follow || !p.Changed(ctx, what, v) {
return
}
}
}
// pipelinesRefusal is the socket's answer to a pipelines ask it cannot answer: no reader, no pipeline named, or a pipeline
// show could not read. A list that could not be read is no refusal: it is a page whose Error says why, handed on, and a
// following reads it again at the runner's next event (design 47 H2 H-A-467).
type pipelinesRefusal struct {
Refused string `json:"refused"`
}
// AskPipelines asks the socket at path for one branch's newest limit pipelines (the default's below one), or with pipeline
// set for that pipeline (of mover when named); with follow, each answers again at each change, handed to each until it
// answers false or the view leaves.
func AskPipelines(path, branch string, limit int, pipeline, mover, next string, follow bool, each func(PipelinesPage, *seat.ShowAnswer) bool) error {
c, err := net.DialTimeout("unix", path, 5*time.Second)
if err != nil {
return fmt.Errorf("cannot reach the view at %s: %w", path, err)
}
defer c.Close()
a := ask{Ask: "pipelines", Branch: branch, Limit: limit, Follow: follow}
if pipeline != "" {
a = ask{Ask: "pipeline", Pipeline: pipeline, Mover: mover, Next: next, Follow: follow}
}
if !follow {
c.SetDeadline(time.Now().Add(10 * time.Second))
}
req, _ := json.Marshal(a)
if _, err := c.Write(append(req, '\n')); err != nil {
return err
}
dec := json.NewDecoder(c)
for {
var raw json.RawMessage
if err := dec.Decode(&raw); err != nil {
if errors.Is(err, io.EOF) && follow {
return errors.New("the view ended the following")
}
return fmt.Errorf("the view's answer is not pipelines: %w", err)
}
var refused pipelinesRefusal
if json.Unmarshal(raw, &refused) == nil && refused.Refused != "" {
return errors.New(refused.Refused)
}
var page PipelinesPage
var show *seat.ShowAnswer
if pipeline != "" {
show = &seat.ShowAnswer{}
if err := json.Unmarshal(raw, show); err != nil {
return fmt.Errorf("the view's answer is not a pipeline: %w", err)
}
} else if err := json.Unmarshal(raw, &page); err != nil {
return fmt.Errorf("the view's answer is not pipelines: %w", err)
}
if !each(page, show) || !follow {
return nil
}
}
}
@@ -145,6 +145,9 @@ type SourceRow struct {
// Away are the nodes the watchman says are offline, left out of the comparison (ADR 0603 §7): the revision counts for them
// once they return and run their wants.
Away []string `json:"away,omitempty"`
// Pipeline is the pipeline of the delivery: the newest of the branch whose source is this source revision, as the seat
// mesh-pipeline-runner's list answers it (design 47 H2 H-A-467 (3)); empty when none is listed.
Pipeline string `json:"pipeline,omitempty"`
}
// AskRow is an open ask (novox/hq ADR 0437 §10): the gate's ask with no answer under its key in the teller's answers.
@@ -271,6 +274,7 @@ type Model struct {
reachSaid map[string]uint64 // the reached head's source revision said last, by repository key
reachNotYet map[string]string // the merges said not yet on the machines with it, by repository key
reachPut func(ReachWrite) error
pipelines *Pipelines // the runner's pipelines, read through its seat's verbs (design 47 H2 H-A-467)
}
type presenceEntry struct {
@@ -710,15 +714,34 @@ func errText(err error) string {
func (m *Model) Snapshot() Snapshot {
s := m.snapshot()
m.mu.Lock()
b := m.backlog
b, p := m.backlog, m.pipelines
m.mu.Unlock()
s.Backlog = []BacklogRow{}
if b != nil {
s.Backlog = b.Rows() // read from the bus outside the model's lock
}
if p != nil {
for i, src := range s.Sources { // each delivery links to its pipeline (design 47 H2 H-A-467 (3))
s.Sources[i].Pipeline = p.Of(src.Branch, src.Revision)
}
}
return s
}
// SetPipelines gives the model the runner's pipelines, which the pipelines page shows and each delivery links to.
func (m *Model) SetPipelines(p *Pipelines) {
m.mu.Lock()
defer m.mu.Unlock()
m.pipelines = p
}
// Pipelines is the reader SetPipelines gave, nil before.
func (m *Model) Pipelines() *Pipelines {
m.mu.Lock()
defer m.mu.Unlock()
return m.pipelines
}
// SetBacklog gives the model the seats' workers to read the backlog of asks from.
// wantsHeld is the real bucket of mesh-wants' wants at the holder followed now, "" for none.
func (m *Model) wantsHeld() string {
+42 -22
View File
@@ -71,26 +71,33 @@ type claim struct {
}
type manifest struct {
Packages map[string]string `json:"packages"`
Claims []claim `json:"claims"`
Class string `json:"class"`
State []state `json:"state"`
Module string `json:"module"`
Moved bool `json:"moved"`
Version string `json:"version"`
Slug string `json:"slug"`
Reads []string `json:"reads"`
Consumes []string `json:"consumes"`
Invokes []invoke `json:"invokes"`
Uses []string `json:"uses"`
Tools []string `json:"tools"`
Emits []string `json:"emits"`
Streams []stream `json:"streams"`
OwnSecrets map[string]string `json:"own-secrets"`
SecretsOwner string `json:"secrets-owner"`
Resources []map[string]any `json:"resources"`
Requires []require `json:"requires"`
Items []item `json:"items"`
Packages map[string]string `json:"packages"`
Claims []claim `json:"claims"`
Class string `json:"class"`
State []state `json:"state"`
Module string `json:"module"`
Moved bool `json:"moved"`
Version string `json:"version"`
Slug string `json:"slug"`
Reads []string `json:"reads"`
Consumes []string `json:"consumes"`
Invokes []invoke `json:"invokes"`
Uses []string `json:"uses"`
Tools []string `json:"tools"`
Emits []string `json:"emits"`
Streams []stream `json:"streams"`
OwnSecrets map[string]string `json:"own-secrets"`
Config map[string]struct {
Type string `json:"type"`
Description string `json:"description"`
Trusted bool `json:"trusted"`
Minimum float64 `json:"minimum"`
Default any `json:"default"`
} `json:"config"`
SecretsOwner string `json:"secrets-owner"`
Resources []map[string]any `json:"resources"`
Requires []require `json:"requires"`
Items []item `json:"items"`
Build struct {
Artifacts []artifact `json:"artifacts"`
} `json:"build"`
@@ -186,7 +193,8 @@ func TestManifestIsTakenAndReadsOnly(t *testing.T) { // 1134/AC7
{Verb: "seat:mesh-wants.revision"}, // §512, A-242: view_tiers compares kept assignment revisions
{Verb: "seat:mesh-wants.difference"}, // §512, A-242: the tiers and each change's after are the assigner's
{Verb: "seat:node-build-go.current"}, {Verb: "seat:node-build-image.current"}, {Verb: "seat:node-build-node.current"}, // design 47 H2 §813 (9): build_usage reads each holder's work by its seat's verb current
{Verb: "seat:node-engine.manifest-reads"}}; !reflect.DeepEqual(m.Invokes, want) { // ADR 0618 §3: node_manifest_reads asks the node-engine seat's verb of every node; 1585/T24: view_tiers asks indexer_pending, a verb edge (ADR 0614 §2)
{Verb: "seat:node-engine.manifest-reads"},
{Verb: "seat:mesh-pipeline-runner.list"}, {Verb: "seat:mesh-pipeline-runner.show"}}; !reflect.DeepEqual(m.Invokes, want) { // design 47 H2 H-A-467: the pipelines page reads the runner's list and show // ADR 0618 §3: node_manifest_reads asks the node-engine seat's verb of every node; 1585/T24: view_tiers asks indexer_pending, a verb edge (ADR 0614 §2)
t.Errorf("invokes = %v, want %v", m.Invokes, want)
}
// design 47 G §387: the keeper's ask for the findings is heard, the finding stream declared (§383), and §368: version is MAJOR.MINOR.
@@ -207,7 +215,8 @@ func TestManifestIsTakenAndReadsOnly(t *testing.T) { // 1134/AC7
"mesh-catalog.catalog-revision", "mesh-catalog.registration-changed", "mesh-delivery.source-moved", "mesh-catalog.seat-definition-changed", "mesh-catalog.seat-definition-removed", "mesh-delivery-indexer.views.candidates", "mesh-delivery-indexer.views.taken", "mesh-delivery-teller.views.answers",
"mesh-wants.wait-held", "mesh-wants.wait-ended", "node-engine.wait-held", "node-engine.wait-ended", "mesh-pipeline-runner.wait-held", "mesh-pipeline-runner.wait-ended",
"mesh-grants.wait-held", "mesh-grants.wait-ended", "mesh-delivery-builder.wait-held", "mesh-delivery-builder.wait-ended",
"mesh-broker.recreated", "mesh-broker.dropped"}; !reflect.DeepEqual(m.Consumes, want) { // design 47 G §512, A-242: every part's wait-held and wait-ended
"mesh-broker.recreated", "mesh-broker.dropped",
"mesh-pipeline-runner.pipeline-started", "mesh-pipeline-runner.step-ended", "mesh-pipeline-runner.pipeline-ended", "mesh-pipeline-runner.re-pinned"}; !reflect.DeepEqual(m.Consumes, want) { // design 47 G §512, A-242: every part's wait-held and wait-ended; design 47 H2 H-A-467: the runner's events the pipelines page reads again on
t.Errorf("consumes = %v, want %v", m.Consumes, want)
}
// The one event the view says: a source revision is delivered (1906/D9, ADR 0539 §17), derived by it from the wants and the haves.
@@ -385,6 +394,17 @@ func TestManifestResources(t *testing.T) { // 1134/AC7
if env["MESH_BROKER_FILE"] != "${dir:state}/broker" || m.OwnSecrets["broker"] != "${dir:state}/broker" {
t.Errorf("the process reads %v, the own-secret is written at %v", env["MESH_BROKER_FILE"], m.OwnSecrets["broker"])
}
// The setting pipelines-heard reaches the process as a file of its state directory, read at its start (principle 11).
if c, ok := m.Config["pipelines-heard"]; !ok || c.Default != float64(view.DefaultPipelinesHeard) || c.Minimum != 1 || c.Type != "number" || len(m.Config) != 1 {
t.Errorf("config = %+v, want pipelines-heard, a number of at least 1, %d by default", m.Config, view.DefaultPipelinesHeard)
}
var heardFile bool
for _, r := range byType["file"] {
heardFile = heardFile || (r["id"] == "pipelines-heard" && r["path"] == "${dir:state}/pipelines-heard" && r["content"] == "${config:pipelines-heard}" && r["owner"] == user && r["mode"] == "0600")
}
if !heardFile || env["MESH_DELIVERY_VIEW_PIPELINES_HEARD_FILE"] != "${dir:state}/pipelines-heard" {
t.Errorf("the setting pipelines-heard is written to its file %v, the process reads %v", heardFile, env["MESH_DELIVERY_VIEW_PIPELINES_HEARD_FILE"])
}
if h, _ := p["health"].(map[string]any); h["kind"] != "unit" {
t.Errorf("health = %v", p["health"])
}
+27 -3
View File
@@ -102,7 +102,11 @@
"mesh-delivery-builder.wait-held",
"mesh-delivery-builder.wait-ended",
"mesh-broker.recreated",
"mesh-broker.dropped"
"mesh-broker.dropped",
"mesh-pipeline-runner.pipeline-started",
"mesh-pipeline-runner.step-ended",
"mesh-pipeline-runner.pipeline-ended",
"mesh-pipeline-runner.re-pinned"
],
"invokes": [
"mesh-delivery-assigner.assigner_read_differences",
@@ -128,7 +132,9 @@
"seat:node-build-go.current",
"seat:node-build-image.current",
"seat:node-build-node.current",
"seat:node-engine.manifest-reads"
"seat:node-engine.manifest-reads",
"seat:mesh-pipeline-runner.list",
"seat:mesh-pipeline-runner.show"
],
"reads": [
"seat:node-build-go.pauses",
@@ -148,6 +154,15 @@
"seat:mesh-vault.signers",
"node-engine.have"
],
"config": {
"pipelines-heard": {
"type": "number",
"description": "How many pipelines' last changes the view keeps for its pipelines pages before it forgets them all, read at its start (design 47 H2 H-A-467; principle 11).",
"trusted": false,
"minimum": 1,
"default": 1024
}
},
"own-secrets": {
"broker": "${dir:state}/broker"
},
@@ -185,13 +200,22 @@
"./mesh-delivery-view"
],
"env": {
"MESH_BROKER_FILE": "${dir:state}/broker"
"MESH_BROKER_FILE": "${dir:state}/broker",
"MESH_DELIVERY_VIEW_PIPELINES_HEARD_FILE": "${dir:state}/pipelines-heard"
},
"user": "mesh-delivery-view",
"health": {
"kind": "unit"
}
},
{
"id": "pipelines-heard",
"type": "file",
"path": "${dir:state}/pipelines-heard",
"owner": "mesh-delivery-view",
"mode": "0600",
"content": "${config:pipelines-heard}"
},
{
"id": "command-program",
"type": "archive",
@@ -110,7 +110,10 @@ func TestTheGrantsPartWhole(t *testing.T) {
}
manifest("nats", `{"module":"nats","version":"1","claims":[{"name":"mesh-broker"}],"own-secrets":{"broker":"/var/lib/nats/broker"},`+
`"emits":["created","done","dropped","recreated"],"streams":[{"name":"broker-events","events":["created","dropped","recreated"],"max-age-days":30,"max-mib":256}]}`)
manifest("mod-c", `{"module":"mod-c","version":"1","own-secrets":{"broker":"/var/lib/c/broker"},"consumes":["nats.done"]}`)
// mod-c names a program of its own, so its program's consumer a_mod-c is composed beside the tool runner's twin (design 47
// H2, H-A-472 second addendum).
manifest("mod-c", `{"module":"mod-c","version":"1","own-secrets":{"broker":"/var/lib/c/broker"},"consumes":["nats.done"],`+
`"resources":[{"id":"program","type":"process","run":["./mod-c"]}]}`)
// The part itself, placed as on the mesh, so its own consumer on EVENTS hears the seat mesh-broker's events (DLV-G1).
own, err := os.ReadFile("../../module.json")
if err != nil {
@@ -188,6 +188,11 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
return err
}
f.Part.RevisionsPerBranch, f.Part.Followed = perBranch, branches.Followed
// The setting twin-current-hours, read at the start: the bound of a current twin for the orphaned durables (design 47 H2,
// H-A-472 third addendum).
if f.Part.TwinCurrentHours, err = grants.TwinCurrentHoursSetting(strings.TrimSpace(getenv(grants.EnvTwinCurrentHours))); err != nil {
return err
}
f.Watch = fv
f.Part.ReadingRunning(nodeengine.StreamRunning)
// The catalog-revision event names each branch's newest revision, so grants-for asks the seat mesh-catalog's revision only
+5 -2
View File
@@ -314,11 +314,14 @@ func recordHandActAs(cause string) func(what, why string) error {
func serveTools() error {
tv := &toolViews{}
gate := newSeatGate()
p := &grants.Part{Read: runtimeReader{views: tv}, Call: runtimeBus{}, Store: grants.BigValues(runtimeStore{}, nil), Emit: runtimeBus{}, SeatList: gate}
// Its journal is its standard error, which the tool runner keeps: grants-for says each stage's time there (H-A-478).
logf := func(f string, a ...any) { fmt.Fprintf(os.Stderr, "mesh-grants: "+f+"\n", a...) }
p := &grants.Part{Read: runtimeReader{views: tv}, Call: runtimeBus{}, Store: grants.BigValues(runtimeStore{}, nil), Emit: runtimeBus{}, SeatList: gate,
Logf: logf}
// The reader asks through the runtime, which serves once Serve has its channel.
go func() {
<-stdio.Served()
gate.Follow(seatlist.StdioBus(), nil, func(f string, a ...any) { fmt.Fprintf(os.Stderr, "mesh-grants: "+f+"\n", a...) })
gate.Follow(seatlist.StdioBus(), nil, logf)
}()
tools := append(toolsOf(p), gate.Tools()...)
for i := range tools {
+132 -11
View File
@@ -547,6 +547,10 @@ func (b *fakeBus) setAssignments(pl meshmoduleassigner.Assignments) {
maps.Copy(b.states[grants.ReadAssignments], values)
}
// programResource names a program of the module's own, a process resource with no bus of its own: its consumer
// <node>_<module> is composed beside the tool runner's twin (design 47 H2, H-A-472 second addendum).
const programResource = `,"resources":[{"id":"program","type":"process","run":["./program"]}]`
const (
natsManifest = `{"module":"nats","version":"1","claims":[{"name":"mesh-broker"}],"own-secrets":{"broker":"/var/lib/nats/broker"}}`
xManifest = `{"module":"mod-x","version":"1","own-secrets":{"broker":"/var/lib/x/broker"},"state":[{"name":"kept","history":3}],` +
@@ -973,7 +977,8 @@ func TestGrants(t *testing.T) {
if again := p.Compose(); again.Withheld == nil || len(again.Withheld.Written) != 0 || again.Withheld.Withheld != 1 {
t.Errorf("a second pass wrote %+v, want nothing (on change only)", again.Withheld)
}
b.manifestOf("reader", `{"module":"reader","version":"1","runs-as":"reader","resources":[{"id":"account","type":"user","name":"reader","shell":"/usr/bin/nologin","home":"/var/lib/reader"}],`+
b.manifestOf("reader", `{"module":"reader","version":"1","runs-as":"reader","resources":[{"id":"account","type":"user","name":"reader","shell":"/usr/bin/nologin","home":"/var/lib/reader"},`+
`{"id":"program","type":"process","run":["./reader"]}],`+
`"own-secrets":{"broker":"/var/lib/r/broker"},"secrets-owner":"reader","consumes":["mesh-mail.mail.box.message.added"]}`)
if heard := p.Compose(); heard.Withheld == nil || !slices.Equal(heard.Withheld.Deleted, []string{key}) || b.own[grants.StateWithheld][key] != nil {
t.Errorf("once reader runs as its own account the pass did %+v (errors %v), want %s deleted", heard.Withheld, heard.Errors, key)
@@ -2244,7 +2249,7 @@ func TestGrants(t *testing.T) {
// A seat's event the seat list says is kept in a stream of its own (design 47): its stream is made once, and the module that
// consumes it has its durable on that stream, none on the shared one.
kb := fixture()
kb.manifestOf("mod-y", `{"module":"mod-y","version":"1","own-secrets":{"broker":"/var/lib/y/broker"},"consumes":["mesh-conditions.condition-raised"]}`)
kb.manifestOf("mod-y", `{"module":"mod-y","version":"1","own-secrets":{"broker":"/var/lib/y/broker"},"consumes":["mesh-conditions.condition-raised"]`+programResource+`}`)
kb.loaded = shadowOf(t, kb)
kp := partOn(kb)
kp.Create, kp.Consumers = kb, kb
@@ -2401,7 +2406,7 @@ func TestGrants(t *testing.T) {
})
t.Run("account consumers: a held consumer gains the filters of a new event, never loses one", func(t *testing.T) {
const cManifest = `{"module":"mod-c","version":"1","own-secrets":{"broker":"/var/lib/c/broker"},"consumes":["mod-y.done","mod-y.more"]}`
const cManifest = `{"module":"mod-c","version":"1","own-secrets":{"broker":"/var/lib/c/broker"},"consumes":["mod-y.done","mod-y.more"]` + programResource + `}`
b := fixture()
b.manifestOf("mod-c", cManifest)
b.assign("b", "mod-c")
@@ -3305,7 +3310,7 @@ func TestGrants(t *testing.T) {
// The part's own consumer on the bus module's stream, and every module's on a module stream it hears, is created by
// the part since the controller ended (ADR 0363 §5, ADR 0590): here mod-y's on mod-x's THINGS.
heard := fixture()
heard.manifestOf("mod-y", strings.Replace(yManifest, `"reads"`, `"consumes":["mod-x.thing.added"],"reads"`, 1))
heard.manifestOf("mod-y", strings.Replace(yManifest, `"reads"`, `"consumes":["mod-x.thing.added"],"resources":[{"id":"program","type":"process","run":["./y"]}],"reads"`, 1))
hp := partOn(heard)
hp.Consumers = heard
hp.Compose()
@@ -3769,7 +3774,8 @@ func TestGrants(t *testing.T) {
p := partOn(b)
p.Consumers = b
p.Compose()
for _, c := range []string{"INDEXER_VIEWS.a_node-tools", "ASSIGNER_VIEWS.a_node-tools"} {
// The runner names no process of its own, so its consumers are its twins alone (H-A-472 second addendum).
for _, c := range []string{"INDEXER_VIEWS.a_node-tools_tools", "ASSIGNER_VIEWS.a_node-tools_tools"} {
if !slices.Contains(b.consumerCreates, c) {
t.Errorf("the tool runner's consumer %s was not made: %v", c, b.consumerCreates)
}
@@ -4140,7 +4146,7 @@ func TestGrants(t *testing.T) {
// configuration, are created where the bus lacks them with no lease key at all (the controller ended by its
// unassignment, never by the lease write); one the bus holds is never changed.
t.Run("account consumers: created where the bus lacks them, with no lease key", func(t *testing.T) {
const cManifest = `{"module":"mod-c","version":"1","own-secrets":{"broker":"/var/lib/c/broker"},"consumes":["mod-y.done"]}`
const cManifest = `{"module":"mod-c","version":"1","own-secrets":{"broker":"/var/lib/c/broker"},"consumes":["mod-y.done"]` + programResource + `}`
b := fixture()
b.manifestOf("mod-c", cManifest)
b.assign("b", "mod-c")
@@ -4167,12 +4173,14 @@ func TestGrants(t *testing.T) {
}
// A held consumer that differs is never changed by a pass: it is raised as a condition naming it, and brought to
// the composed configuration only by hand (grants_consumer_match), with no lease key too.
var said, behind []oc.Condition
var said, behind, orphaned []oc.Condition
p.Say = func(c oc.Condition) error {
switch c.Key {
case grants.GrantMissingCondition, grants.RefusedCondition, grants.HeldModuleCondition, grants.FullCondition: // their own rows
case grants.ConsumerBehindCondition:
behind = append(behind, c)
case grants.DurableOrphanedCondition:
orphaned = append(orphaned, c)
default:
said = append(said, c)
}
@@ -4222,6 +4230,65 @@ func TestGrants(t *testing.T) {
if len(res.Consumers.Behind) != 0 || behind[len(behind)-1].State != occ.StateCleared {
t.Errorf("a consumer back within the bound: %+v, said %+v", res.Consumers, behind)
}
// Design 47 H2, H-A-472 (2): a durable far behind with no pulls waiting, whose twin on the stream (b_mod-c_tools) has
// pulls waiting and delivered within the day, is an orphan: left out of the behind check, named once in
// mesh-grants.durable-orphaned, and cleared when it is gone; with its twin not current, it is behind again.
var twin map[string]any
for _, c := range b.consumers["EVENTS"] {
if c["name"] == "b_mod-c_tools" {
twin = c
}
}
twin["waiting_pulls"], twin["last_delivered"] = 1, "2026-10-25T21:00:00.5Z"
made["pending"] = grants.BehindBound + 1
p.Compose()
o := len(orphaned)
if res = p.Compose(); len(res.Consumers.Behind) != 0 || len(orphaned) != o+1 || orphaned[o].State != occ.StateRaised ||
!strings.Contains(orphaned[o].Explanation, "EVENTS.b_mod-c") || strings.Contains(orphaned[o].Explanation, "EVENTS.b_mod-c_tools") {
t.Fatalf("an orphaned durable: %+v, behind said %+v, orphaned said %+v", res.Consumers, behind[len(behind)-1], orphaned)
}
if p.Compose(); len(orphaned) != o+1 {
t.Fatalf("an orphan was said again: %+v", orphaned)
}
twin["last_delivered"] = "2026-10-23T21:00:00Z"
if res = p.Compose(); len(res.Consumers.Behind) != 1 || len(orphaned) != o+2 || orphaned[o+1].State != occ.StateCleared {
t.Fatalf("a durable whose twin delivered two days ago: %+v, orphaned said %+v", res.Consumers, orphaned)
}
// The third addendum (2): the bound is the setting twin-current-hours; at 72 hours, a twin that delivered two days
// ago is current, so the durable is an orphan again; the setting's file gives its hours, its default when missing.
p.TwinCurrentHours = 72
if res = p.Compose(); len(res.Consumers.Behind) != 0 || len(orphaned) != o+3 || orphaned[o+2].State != occ.StateRaised {
t.Fatalf("with twin-current-hours 72, a durable whose twin delivered two days ago: %+v, orphaned said %+v", res.Consumers, orphaned)
}
p.TwinCurrentHours = 0
if res = p.Compose(); len(res.Consumers.Behind) != 1 || len(orphaned) != o+4 || orphaned[o+3].State != occ.StateCleared {
t.Fatalf("back at the default bound: %+v, orphaned said %+v", res.Consumers, orphaned)
}
for _, row := range []struct {
text string
hours int
fails bool
}{{"72\n", 72, false}, {"", grants.DefaultTwinCurrentHours, false}, {"0", 0, true}, {"x", 0, true}} {
path := filepath.Join(t.TempDir(), "twin-current-hours")
if err := os.WriteFile(path, []byte(row.text), 0o600); err != nil {
t.Fatal(err)
}
if n, err := grants.TwinCurrentHoursSetting(path); n != row.hours || (err != nil) != row.fails {
t.Errorf("twin-current-hours %q: %d, %v", row.text, n, err)
}
}
if n, err := grants.TwinCurrentHoursSetting(filepath.Join(t.TempDir(), "none")); n != 24 || err != nil {
t.Errorf("with no file twin-current-hours is %d, %v; want 24", n, err)
}
made["pending"] = grants.BehindBound
p.Compose()
// H-A-472 second addendum: a module with an account of its own but no process resource with no bus of its own has
// no program reading b_<module>, so only the tool runner's twin is made.
b.manifestOf("mod-t", `{"module":"mod-t","version":"1","own-secrets":{"broker":"/var/lib/t/broker"},"consumes":["mod-y.done"]}`)
b.assign("b", "mod-t")
if res = p.Compose(); !slices.Contains(res.Consumers.Created, "EVENTS.b_mod-t_tools") || slices.Contains(res.Consumers.Created, "EVENTS.b_mod-t") {
t.Errorf("a module with no program of its own: created %v", res.Consumers.Created)
}
})
// 774/AC13 (novox/hq ADR 0561 §7): after take-over a list in which more than one loaded user's hash changes, with no
@@ -5275,6 +5342,52 @@ func TestGrantsFor(t *testing.T) {
}
}
// grants-for composes each revision's grants once and keeps them with the revision (design 47 H2 H-A-478 and its addendum):
// a second ask for the same revisions composes nothing again; an ask of another candidate against the same newest composes
// the candidate alone; each stage's time is in the journal.
func TestGrantsForComposesARevisionOnce(t *testing.T) {
candidateX := strings.Replace(xManifest, `"state"`, `"emits":["hello"],"state"`, 1)
b := fixture()
b.revision = map[string]map[string]string{"newest": {"mod-x": xManifest, "mod-y": xManifest}, "candidate": {"mod-x": candidateX, "mod-y": xManifest}}
b.revisionNumber = map[string]int64{"newest": 9, "candidate": 8}
p := partOn(b)
var journal []string
p.Logf = func(f string, a ...any) { journal = append(journal, fmt.Sprintf(f, a...)) }
ask := func(in meshgrants.GrantsForInput) (meshgrants.GrantsFor, string) {
t.Helper()
journal = nil
got, err := p.GrantsFor(in)
if err != nil || got.Error != "" {
t.Fatalf("grants-for: %+v, %v", got, err)
}
if len(journal) != 1 || !strings.HasPrefix(journal[0], "grants-for main ") || !strings.Contains(journal[0], "the inputs ") {
t.Fatalf("grants-for said %q, want one line of its stages' times", journal)
}
return got, journal[0]
}
first, said := ask(meshgrants.GrantsForInput{Branch: "main", Source: "7"})
if len(first.Modules) != 1 || first.Modules[0].Module != "mod-x" || strings.Count(said, "(composed)") != 2 {
t.Errorf("the first ask answered %+v and said %q, want mod-x and both revisions' grants composed", first, said)
}
again, said := ask(meshgrants.GrantsForInput{Branch: "main", Source: "7"})
if strings.Contains(said, "composed") || strings.Count(said, "(kept)") != 2 || !reflect.DeepEqual(first, again) {
t.Errorf("a second ask for the same revisions said %q and answered %+v, want both kept and %+v", said, again, first)
}
// Inputs that differ compose both again (H-A-478 second addendum); the same inputs then keep them again.
b.put(grants.ReadResolved, "a.mod-x.greeting", map[string]any{"node": "a", "module": "mod-x", "key": "greeting", "value": "hello"})
if _, said = ask(meshgrants.GrantsForInput{Branch: "main", Source: "7"}); strings.Count(said, "(composed)") != 2 {
t.Errorf("an ask whose inputs differ said %q, want both revisions' grants composed again", said)
}
if _, said = ask(meshgrants.GrantsForInput{Branch: "main", Source: "7"}); strings.Count(said, "(kept)") != 2 {
t.Errorf("an ask with the same inputs again said %q, want both kept", said)
}
// A run's candidate is another revision: composed once, the newest's grants kept.
_, said = ask(meshgrants.GrantsForInput{Branch: "main", Run: "r1"})
if !strings.Contains(said, "the newest's grants (kept)") || !strings.Contains(said, "the candidate's grants (composed)") {
t.Errorf("an ask of another candidate said %q, want the newest's grants kept and the candidate's composed", said)
}
}
// grants-for keeps what H-A-195 says of the catalog: so many revisions to a branch as the setting
// grants-for-revisions-per-branch says, a run's candidate counted within its branch's; only the followed branches', freed when
// one leaves them; and an event below the newest held is the numbering started again, which drops what is kept.
@@ -5305,8 +5418,11 @@ func TestGrantsForKeepsWhatItMay(t *testing.T) {
perBranch int
then func(t *testing.T, b *fakeBus, p *grants.Part, followed *[]string)
pages int
// composed is how many of the two revisions' grants the grants-for after it composes again: a revision no longer
// kept has no grants kept either (H-A-478).
composed int
}{
{name: "the setting bounds a branch's revisions: with 2, the candidate is dropped once two newer are kept", pages: 2,
{name: "the setting bounds a branch's revisions: with 2, the candidate is dropped once two newer are kept", pages: 2, composed: 1,
then: func(t *testing.T, b *fakeBus, p *grants.Part, _ *[]string) {
p.RevisionsPerBranch = setting(t, "2\n")
hear(t, p, 10, 0)
@@ -5314,14 +5430,14 @@ func TestGrantsForKeepsWhatItMay(t *testing.T) {
_, _ = p.GrantsFor(meshgrants.GrantsForInput{Branch: "main", Source: "7"})
}},
{name: "a run's candidate counts within its branch's bound: with 1, the candidate and the newest each drop the other", run: true,
perBranch: 1, pages: 4, then: func(*testing.T, *fakeBus, *grants.Part, *[]string) {}},
{name: "a branch that leaves the watched branches frees its revisions", pages: 4,
perBranch: 1, pages: 4, composed: 2, then: func(*testing.T, *fakeBus, *grants.Part, *[]string) {}},
{name: "a branch that leaves the watched branches frees its revisions", pages: 4, composed: 2,
then: func(t *testing.T, _ *fakeBus, p *grants.Part, followed *[]string) {
*followed = nil
_, _ = p.GrantsFor(meshgrants.GrantsForInput{Branch: "main", Source: "7"})
*followed = []string{"main"}
}},
{name: "an event below the newest held drops what is kept of the branch", pages: 4,
{name: "an event below the newest held drops what is kept of the branch", pages: 4, composed: 2,
then: func(t *testing.T, _ *fakeBus, p *grants.Part, _ *[]string) { hear(t, p, 3, 0) }},
} {
t.Run(row.name, func(t *testing.T) {
@@ -5344,10 +5460,15 @@ func TestGrantsForKeepsWhatItMay(t *testing.T) {
}
row.then(t, b, p, &followed)
before := len(slices.DeleteFunc(slices.Clone(b.calls), func(c string) bool { return c != grants.CallRevision }))
var said []string
p.Logf = func(f string, a ...any) { said = append(said, fmt.Sprintf(f, a...)) }
_, _ = p.GrantsFor(in)
if got := len(slices.DeleteFunc(slices.Clone(b.calls), func(c string) bool { return c != grants.CallRevision })) - before; got != row.pages {
t.Errorf("grants-for read %d revision pages, want %d", got, row.pages)
}
if len(said) != 1 || strings.Count(said[0], "(composed)") != row.composed {
t.Errorf("grants-for said %q, want %d of the revisions' grants composed again", said, row.composed)
}
})
}
if n, err := grants.RevisionsPerBranchSetting(filepath.Join(t.TempDir(), "none")); n != grants.DefaultRevisionsPerBranch || err != nil {
@@ -618,7 +618,8 @@ func TestTheCopiedComposerComposesAsTheControllers(t *testing.T) {
}
// The grants part is granted the create of each account's consumer by its stream and name, and no other.
users, err := Users(Records{Nodes: []string{"a"}, Assigned: map[string][]Declared{"a": {
{Module: GrantsModule}, {Module: "m", Consumes: []string{"x.happened"}}, {Module: "quiet"}}}})
{Module: GrantsModule}, {Module: "m", Consumes: []string{"x.happened"}, HasProgram: true}, {Module: "quiet"},
{Module: "tooled", Consumes: []string{"x.happened"}}}}})
if err != nil {
t.Fatal(err)
}
@@ -639,6 +640,13 @@ func TestTheCopiedComposerComposesAsTheControllers(t *testing.T) {
MayPublish(perms, "$JS.API.CONSUMER.DELETE.EVENTS.a_m") || MayPublish(perms, "$JS.API.CONSUMER.CREATE.EVENTS.a_m.mesh.mod.x.event.happened") {
t.Errorf("the grants part creates consumers as %v", u.CreatesConsumers)
}
// Design 47 H2, H-A-472 second addendum: the program's consumer only for a module with a program of its own; the
// tool runner's twin for every consuming module.
for durable, want := range map[string]bool{"a_m": true, "a_m_tools": true, "a_tooled": false, "a_tooled_tools": true} {
if MayPublish(perms, "$JS.API.CONSUMER.CREATE.EVENTS."+durable) != want {
t.Errorf("the grants part creates EVENTS.%s: %v, want %v (%v)", durable, !want, want, u.CreatesConsumers)
}
}
}
// 1952 S5 (novox/hq ADR 0564 clauses 2, 5, 8; Limits 2, 4, 8): the health holders' credentials.
healthChecks(t)
@@ -191,8 +191,9 @@ func ConsumersOf(users []Principal) []ModuleConsumer {
}
seen[key] = true
if needed {
// Each with the tool runner's twin beside it (A-397).
for _, tc := range withToolsTwin(c) {
// Each with the tool runner's twin beside it (A-397), the program's only for a module with a program of its
// own (H-A-472 second addendum).
for _, tc := range twinsOf(c, p.HasProgram) {
out = append(out, ModuleConsumer{Node: p.Node, Module: p.Module, Consumer: tc})
}
// And one on each module stream it hears (novox/hq ADR 0363 §5).
@@ -200,7 +201,7 @@ func ConsumersOf(users []Principal) []ModuleConsumer {
if len(h.Filters) == 0 {
continue
}
for _, tc := range withToolsTwin(heardConsumer(p, h)) {
for _, tc := range twinsOf(heardConsumer(p, h), p.HasProgram) {
out = append(out, ModuleConsumer{Node: p.Node, Module: p.Module, Consumer: tc})
}
}
@@ -243,7 +244,7 @@ func ConsumersOf(users []Principal) []ModuleConsumer {
// 0436 §1, §2, §6), the same reason its ordinary consumer is (ADR 0198): mesh-mail has no
// account of its own (NoAccount) and is carried here, but its process does, and the consumer
// it reads through must exist and survive PruneHeard whether or not the module itself speaks.
add(Principal{Kind: KindModule, Node: p.Node, Module: d.Module, Emits: d.Emits,
add(Principal{Kind: KindModule, Node: p.Node, Module: d.Module, HasProgram: d.HasProgram, Emits: d.Emits,
Consumes: d.Consumes, Serves: d.Serves, Holds: d.Holds, Uses: d.Uses, Watches: d.Watches,
Streams: d.Streams, Hears: ComposedHears(d), Queues: d.Queues})
}
@@ -278,6 +278,17 @@ func withToolsTwin(c Consumer) []Consumer {
return []Consumer{c, twin}
}
// twinsOf are the consumers composed of a module's consumer c (design 47 H2, H-A-472 addendum and second addendum): the
// tool runner's twin <name>_tools for every consuming module, and c itself, the program's, only for a module with a
// program of its own (Declared.HasProgram), so no pass makes a durable nobody reads.
func twinsOf(c Consumer, hasProgram bool) []Consumer {
both := withToolsTwin(c)
if hasProgram {
return both
}
return both[1:]
}
// heardGrants are what a principal publishes to read and acknowledge the consumers it hears module streams
// through: asking about each, pulling from it, acknowledging it — its own name on each stream, no other's.
func heardGrants(p Principal, hears []Heard) []string {
@@ -156,6 +156,9 @@ type Principal struct {
// Process is the process resource's id, for a KindProcess principal alone (novox/hq ADR 0436 §2):
// Username and inbox are <node>.<module>.<process id>.
Process string
// HasProgram is Declared.HasProgram, for a KindModule principal: its program's consumer <node>_<module> is composed
// beside the tool runner's twin (H-A-472 second addendum).
HasProgram bool
// Objects are the bus names of its module's own object buckets, and ReadsObjects those of others it reads (novox/hq
// ADR 0592 §5, §9).
File diff suppressed because one or more lines are too long
@@ -88,6 +88,10 @@ type Declared struct {
// delivered to it and nothing can connect as it (novox/hq issue 195). Said in the negative so a
// record that does not say is composed as it always was.
NoAccount bool
// HasProgram says the module runs a program of its own that reads its consumer <node>_<module>: an account of its own
// (an own broker secret) and a process resource with no `bus` of its own (design 47 H2, H-A-472 second addendum).
// Without it, only the tool runner's twin <node>_<module>_tools is composed, so the pass makes no orphan.
HasProgram bool
// SnapshotsTheBus says the module holds mesh-broker — it is the bus — and so is the one module
// granted the snapshot API, to copy the bus's streams for the night's backup (novox/hq ADR 0235).
SnapshotsTheBus bool
@@ -349,7 +353,7 @@ func Users(r Records) ([]Principal, error) {
continue
}
out = append(out, Principal{
Kind: KindModule, Node: node, Module: d.Module,
Kind: KindModule, Node: node, Module: d.Module, HasProgram: d.HasProgram,
Emits: d.Emits, Consumes: d.Consumes, Serves: d.Serves,
Holds: d.Holds, Uses: d.Uses, Watches: d.Watches, Invokes: d.Invokes,
State: stateNames(d.State), Reads: d.Reads, ReadLogs: d.ReadLogs, SnapshotsTheBus: d.SnapshotsTheBus,
@@ -4,9 +4,13 @@ import (
"encoding/json"
"errors"
"fmt"
"io/fs"
"os"
"slices"
"sort"
"strconv"
"strings"
"time"
"github.com/nats-io/nats.go/jetstream"
@@ -60,6 +64,74 @@ type heldConsumer struct {
Filters []string `json:"filters"`
MaxDeliver int `json:"max_deliver"`
Pending int64 `json:"pending"`
// WaitingPulls and LastDelivered say whether it is read now: pulls waiting on it, and when it last delivered (H-A-472).
WaitingPulls int64 `json:"waiting_pulls"`
LastDelivered string `json:"last_delivered"`
}
// DefaultTwinCurrentHours is the setting twin-current-hours's default (design 47 H2, H-A-472 third addendum): how many
// hours since a twin's last delivery it is current, within the day; the bus module's manifest names the same setting for
// nats_consumer_remove, so "current" and "orphaned" share one bound.
const DefaultTwinCurrentHours = 24
// EnvTwinCurrentHours names the file the manifest writes the setting twin-current-hours into.
const EnvTwinCurrentHours = "MESH_GRANTS_TWIN_CURRENT_HOURS_FILE"
// TwinCurrentHoursSetting reads the setting twin-current-hours from its file, read at the start: a missing or empty file
// is its default, one that holds no positive whole number an error naming the file.
func TwinCurrentHoursSetting(path string) (int, error) {
if path == "" {
return DefaultTwinCurrentHours, nil
}
raw, err := os.ReadFile(path)
if errors.Is(err, fs.ErrNotExist) {
return DefaultTwinCurrentHours, nil
}
if err != nil {
return 0, err
}
v := strings.TrimSpace(string(raw))
if v == "" {
return DefaultTwinCurrentHours, nil
}
n, err := strconv.Atoi(v)
if err != nil || n < 1 {
return 0, fmt.Errorf("%s holds %q, which is not a positive whole number", path, v)
}
return n, nil
}
// twinCurrentWithin is the part's bound of a current twin: its setting twin-current-hours, else the default.
func (p *Part) twinCurrentWithin() time.Duration {
if p.TwinCurrentHours > 0 {
return time.Duration(p.TwinCurrentHours) * time.Hour
}
return DefaultTwinCurrentHours * time.Hour
}
// durableTwin names the twin of a module's durable: its program's <node>_<module> and its tool runner's
// <node>_<module>_tools (design 47 G, A-340), each the other's.
func durableTwin(name string) string {
if t, ok := strings.CutSuffix(name, "_tools"); ok {
return t
}
return name + "_tools"
}
// orphaned says whether a held durable is an orphan (design 47 H2, H-A-472 (2)): no pulls waiting on it, while its twin
// on the same stream is current, with pulls waiting and a delivery within the bound (the setting twin-current-hours,
// the third addendum). A module that moved into the tool runner left its program's durable so, and one that runs its own
// program again its tool runner's.
func orphaned(h heldConsumer, held map[string]heldConsumer, now time.Time, within time.Duration) bool {
if h.WaitingPulls > 0 {
return false
}
t, ok := held[durableTwin(h.Name)]
if !ok || t.WaitingPulls == 0 {
return false
}
last, err := time.Parse(time.RFC3339Nano, t.LastDelivered)
return err == nil && now.Sub(last) < within
}
// heldConsumers reads the consumers the bus holds on stream, by name.
@@ -123,6 +195,7 @@ func (p *Part) consumers(res *Result, recs buscompose.Records, match string) {
st.Wanted = len(wanted)
seen := map[string]bool{}
defer func() { p.behindSeen = seen }()
var orphans []string
byStream := map[string][]buscompose.Consumer{}
for _, mc := range wanted {
byStream[mc.Consumer.Stream] = append(byStream[mc.Consumer.Stream], mc.Consumer)
@@ -141,7 +214,13 @@ func (p *Part) consumers(res *Result, recs buscompose.Records, match string) {
}
if ok {
if h.Pending > BehindBound {
if p.behindSeen[key] {
// An orphan is left out of the behind check and named in a finding of its own (H-A-472 (2)), so the
// alarm says a real backlog only.
switch {
case !p.behindSeen[key]:
case orphaned(h, held, p.now(), p.twinCurrentWithin()):
orphans = append(orphans, key)
default:
st.Behind = append(st.Behind, fmt.Sprintf("%s: %d pending", key, h.Pending))
}
seen[key] = true
@@ -216,6 +295,7 @@ func (p *Part) consumers(res *Result, recs buscompose.Records, match string) {
}
p.sayDiffer(st)
p.sayBehind(st)
p.sayOrphaned(orphans)
}
// sayDiffer raises the warning condition mesh-grants.consumer-differs while an account consumer the part owns (from the
@@ -347,6 +427,41 @@ func (p *Part) sayBehind(st *ConsumersState) {
}
}
// DurableOrphanedCondition is the key of the warning raised while an account consumer is an orphaned durable: far
// behind, no pulls waiting, its twin on the stream current (design 47 H2, H-A-472 (2)).
const DurableOrphanedCondition = "mesh-grants.durable-orphaned"
// sayOrphaned raises the warning condition mesh-grants.durable-orphaned naming the orphaned durables the behind check
// left out, once per set, and clears it when none is left (they were removed, by nats_consumer_remove).
func (p *Part) sayOrphaned(orphans []string) {
if p.Say == nil {
return
}
why := strings.Join(orphans, ", ")
p.holdMu.Lock()
defer p.holdMu.Unlock()
if p.orphanedKnown && why == p.orphanedSaid {
return
}
c := oc.Condition{Key: DurableOrphanedCondition, Subject: oc.ConditionSubject{Scope: "module", ID: "mesh-grants"}}
if why == "" {
c.State = occ.StateCleared
} else {
c.Severity, c.State = occ.SeverityWarning, occ.StateRaised
c.Summary = fmt.Sprintf("%d durable consumers are orphans: nothing reads them, while their twin on the stream does", len(orphans))
c.Headline = "A listener's old consumer is left over"
needsYou(&c, "their removal, with the bus module's nats_consumer_remove", "These durables are more than the bound behind "+
"their stream's head with no pulls waiting, while their module's twin on the same stream (<node>_<module> and "+
"<node>_<module>_tools) has pulls waiting and delivered within "+p.twinCurrentWithin().String()+" (the setting "+
"twin-current-hours): a module that moved into or out of the tool runner left them, and they are not said as "+
"messages piling up: "+why)
c.Resolved = "no orphaned durable is left"
}
if p.Say(c) == nil {
p.orphanedSaid, p.orphanedKnown = why, true
}
}
// sayWorker logs one named seat worker's outcome (created, held already) once per change, so the journal names each worker the
// pass considers (A-413).
func (p *Part) sayWorker(line string) {
+114 -22
View File
@@ -2,6 +2,8 @@ package grants
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
@@ -13,6 +15,7 @@ import (
"strconv"
"strings"
"sync"
"time"
"git.novox.be/novox/mesh-contracts/identity-and-access/go/seats/meshgrants"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
@@ -94,9 +97,13 @@ func (p *Part) following(branch string) bool {
// revision was taken in as, so grants-for asks the seat mesh-catalog's revision only for a revision it does not hold, never
// to learn the newest.
type revisionsKept struct {
mu sync.Mutex
order map[string][]string
kept map[string]map[string]catalogue.Manifest
mu sync.Mutex
order map[string][]string
kept map[string]map[string]catalogue.Manifest
// grants are the grants each kept revision composes to, by module (grantsOfEntries), with the digest of the inputs they were
// composed with: composed once per revision and inputs, kept as long as the revision is (design 47 H2 H-A-478 and its
// addenda).
grants map[string]keptGrants
newest map[string]int64
sources map[string]map[int64]int64
sourced map[string][]int64
@@ -113,6 +120,7 @@ func revisionKey(branch string, r meshcatalog.CatalogRevision) string {
func (k *revisionsKept) init() {
if k.kept == nil {
k.kept, k.order, k.newest = map[string]map[string]catalogue.Manifest{}, map[string][]string{}, map[string]int64{}
k.grants = map[string]keptGrants{}
k.sources, k.sourced = map[string]map[int64]int64{}, map[string][]int64{}
}
}
@@ -124,6 +132,44 @@ func (k *revisionsKept) get(branch string, r meshcatalog.CatalogRevision) (map[s
return m, ok
}
// keptGrants are a revision's composed grants, by module, and the digest of the inputs they were composed with.
type keptGrants struct {
inputs string
grants map[string]map[string]bool
}
// grantsOf is the grants kept of the revision key names, composed with the inputs of the digest given; false when none is.
func (k *revisionsKept) grantsOf(key, inputs string) (map[string]map[string]bool, bool) {
k.mu.Lock()
defer k.mu.Unlock()
g, ok := k.grants[key]
if !ok || g.inputs != inputs {
return nil, false
}
return g.grants, true
}
// keepGrants keeps the grants the revision key names composes to with the inputs of the digest given, in place of any kept
// before, while that revision is kept: so the grants are bounded as the revisions are (H-A-195's setting).
func (k *revisionsKept) keepGrants(key, inputs string, g map[string]map[string]bool) {
k.mu.Lock()
defer k.mu.Unlock()
if _, ok := k.kept[key]; ok {
k.grants[key] = keptGrants{inputs: inputs, grants: g}
}
}
// inputsDigest is the sha256 of the inputs as JSON: what a revision's kept grants are composed with (H-A-478 second
// addendum). Empty when they do not marshal: nothing is kept then.
func inputsDigest(in Inputs) string {
raw, err := json.Marshal(in)
if err != nil {
return ""
}
sum := sha256.Sum256(raw)
return hex.EncodeToString(sum[:])
}
func (k *revisionsKept) put(branch string, r meshcatalog.CatalogRevision, manifests map[string]catalogue.Manifest, max int) {
k.mu.Lock()
defer k.mu.Unlock()
@@ -135,6 +181,7 @@ func (k *revisionsKept) put(branch string, r meshcatalog.CatalogRevision, manife
k.kept[key] = manifests
for len(k.order[branch]) > max {
delete(k.kept, k.order[branch][0])
delete(k.grants, k.order[branch][0])
k.order[branch] = k.order[branch][1:]
}
if r.Kind != "candidate" && r.Source != 0 {
@@ -167,6 +214,7 @@ func boolsOf[V any](m map[string]V) map[string]bool {
func (k *revisionsKept) dropLocked(branch string) {
for _, key := range k.order[branch] {
delete(k.kept, key)
delete(k.grants, key)
}
delete(k.order, branch)
delete(k.newest, branch)
@@ -250,14 +298,15 @@ func (p *Part) HearCatalogRevision(_ context.Context, data []byte) error {
// is answered from what it holds with no read; one it knows but does not hold is read by that number. Otherwise a revision
// read whole is kept: once the first page of an answer names a revision that is kept, it is answered from what is kept,
// without the pages after it. The branch's newest is read on by the number its first page names, so its pages are of one
// revision though a newer one is written between them; a revision whose pages name another is not kept.
func (p *Part) revisionEntries(in meshcatalog.RevisionInput) (manifests map[string]catalogue.Manifest, none bool, err error) {
// revision though a newer one is written between them; a revision whose pages name another is not kept. kept names the revision
// as it is kept (revisionKey), empty when it is not: what its grants are kept by (H-A-478).
func (p *Part) revisionEntries(in meshcatalog.RevisionInput) (manifests map[string]catalogue.Manifest, kept string, none bool, err error) {
branch := in.Branch
followed := p.following(branch)
newestAsked := in.Group == "" && in.Heads == "" && in.Source == 0 && in.Revision == 0
if n, ok := p.revisions.known(in); ok {
if m, held := p.revisions.get(branch, meshcatalog.CatalogRevision{Revision: n}); held {
return m, false, nil
return m, revisionKey(branch, meshcatalog.CatalogRevision{Revision: n}), false, nil
}
if newestAsked {
in.Revision = n
@@ -269,14 +318,14 @@ func (p *Part) revisionEntries(in meshcatalog.RevisionInput) (manifests map[stri
for {
raw, err := p.Call.Call(CallRevision, in)
if err != nil {
return nil, false, fmt.Errorf("%s does not answer: %w", CallRevision, err)
return nil, "", false, fmt.Errorf("%s does not answer: %w", CallRevision, err)
}
var a meshcatalog.RevisionAnswer
if err := json.Unmarshal(raw, &a); err != nil {
return nil, false, fmt.Errorf("%s answered what is not a revision: %w", CallRevision, err)
return nil, "", false, fmt.Errorf("%s answered what is not a revision: %w", CallRevision, err)
}
if a.NoneYet {
return nil, true, nil
return nil, "", true, nil
}
switch {
case a.Revision == nil:
@@ -286,7 +335,7 @@ func (p *Part) revisionEntries(in meshcatalog.RevisionInput) (manifests map[stri
p.revisions.asked(branch, a.Revision.Revision)
}
if m, ok := p.revisions.get(branch, *a.Revision); ok {
return m, false, nil
return m, revisionKey(branch, *a.Revision), false, nil
}
first = a.Revision
if newestAsked {
@@ -302,15 +351,16 @@ func (p *Part) revisionEntries(in meshcatalog.RevisionInput) (manifests map[stri
}
m, err := catalogue.Decode(body)
if err != nil {
return nil, false, fmt.Errorf("the manifest of %s in the revision does not decode: %w", e.Module, err)
return nil, "", false, fmt.Errorf("the manifest of %s in the revision does not decode: %w", e.Module, err)
}
manifests[e.Module] = m
}
if a.Next == "" || a.Next == in.After {
if keep && first != nil && followed {
p.revisions.put(branch, *first, manifests, p.revisionsPerBranch())
kept = revisionKey(branch, *first)
}
return manifests, false, nil
return manifests, kept, false, nil
}
in.After = a.Next
}
@@ -389,26 +439,41 @@ func (p *Part) GrantsFor(in meshgrants.GrantsForInput) (meshgrants.GrantsFor, er
default:
return meshgrants.GrantsFor{}, errors.New("grants-for names a source revision or a run")
}
cand, none, err := p.revisionEntries(candidate)
// Each stage's time in the journal (H-A-478): the inputs, each revision's read and each one's grants, composed or kept.
var stages []string
stage := func(what string, start time.Time) {
stages = append(stages, fmt.Sprintf("%s %s", what, time.Since(start).Round(time.Millisecond)))
}
defer func() {
if p.Logf != nil {
p.Logf("grants-for %s: %s", candidateName(candidate), strings.Join(stages, ", "))
}
}()
start := time.Now()
cand, candKept, none, err := p.revisionEntries(candidate)
stage("the candidate read", start)
if err != nil || none {
return meshgrants.GrantsFor{Error: orWhy(err, "the candidate's catalog half is not held: the branch has no revision of it")}, nil
}
newest, none, err := p.revisionEntries(meshcatalog.RevisionInput{Branch: in.Branch})
start = time.Now()
newest, newestKept, none, err := p.revisionEntries(meshcatalog.RevisionInput{Branch: in.Branch})
stage("the newest read", start)
if err != nil || none {
return meshgrants.GrantsFor{Error: orWhy(err, "the branch's newest catalog half is not held")}, nil
}
base := map[string]catalogue.Manifest{}
for module := range cand {
if m, ok := newest[module]; ok {
base[module] = m
}
}
start = time.Now()
input := p.inputs()
was, err := grantsOfEntries(input, base)
digest := inputsDigest(input)
stage("the inputs", start)
start = time.Now()
was, composed, err := p.grantsOfRevision(input, digest, newestKept, newest)
stage("the newest's grants ("+composed+")", start)
if err != nil {
return meshgrants.GrantsFor{Error: "the branch's newest does not compose: " + err.Error()}, nil
}
will, err := grantsOfEntries(input, cand)
start = time.Now()
will, composed, err := p.grantsOfRevision(input, digest, candKept, cand)
stage("the candidate's grants ("+composed+")", start)
if err != nil {
return meshgrants.GrantsFor{Error: "the candidate does not compose: " + err.Error()}, nil
}
@@ -435,6 +500,33 @@ func (p *Part) GrantsFor(in meshgrants.GrantsForInput) (meshgrants.GrantsFor, er
return out, nil
}
// candidateName names the candidate an ask is of, for the journal.
func candidateName(c meshcatalog.RevisionInput) string {
if c.Group != "" {
return c.Branch + " " + c.Group
}
return c.Branch + " source " + strconv.FormatInt(c.Source, 10)
}
// grantsOfRevision is the grants a revision's manifests compose to with in (of the digest inputs), by module: those kept of the
// revision kept names when they were composed with the same inputs, else composed and kept with it (H-A-478: once per revision
// and inputs). how says which.
func (p *Part) grantsOfRevision(in Inputs, inputs, kept string, manifests map[string]catalogue.Manifest) (grants map[string]map[string]bool, how string, err error) {
if kept != "" && inputs != "" {
if g, ok := p.revisions.grantsOf(kept, inputs); ok {
return g, "kept", nil
}
}
g, err := grantsOfEntries(in, manifests)
if err != nil {
return nil, "composed", err
}
if kept != "" && inputs != "" {
p.revisions.keepGrants(kept, inputs, g)
}
return g, "composed", nil
}
func orWhy(err error, why string) string {
if err != nil {
return err.Error()
+8 -1
View File
@@ -170,7 +170,8 @@ type Part struct {
Emit Emitter
// Create creates a declared stream the bus lacks: the process's connection; nil in the tools, which ask the process.
Create Creator
// Logf logs what no state or condition says (an error of acting on an answer): the process's; nil in the tools.
// Logf logs what no state or condition says (an error of acting on an answer): the process's journal, and in the tools
// program its standard error (grants-for's stage times, H-A-478).
Logf func(string, ...any)
// Objects is the part's object bucket of sealed lists: the process's connection; nil in the tools.
Objects Objects
@@ -197,6 +198,9 @@ type Part struct {
// RevisionsPerBranch is the setting grants-for-revisions-per-branch, read at the start (grantsfor.go, design 47 H H-A-195);
// 0 is its default.
RevisionsPerBranch int
// TwinCurrentHours is the setting twin-current-hours, read at the start (consumers.go, design 47 H2, H-A-472 third
// addendum): the bound of a current twin, by which a durable is orphaned; 0 is its default.
TwinCurrentHours int
// Followed is the watched branches the part follows, as named (the branch follower's): grants-for keeps the revisions
// of these alone, and frees a branch's once it leaves them; nil in the tools, where every branch is kept.
Followed func() []string
@@ -230,6 +234,9 @@ type Part struct {
behindSaid string
behindKnown bool
behindSeen map[string]bool
// orphanedSaid is the orphaned durables last said as a condition, once orphanedKnown (consumers.go, H-A-472).
orphanedSaid string
orphanedKnown bool
// fullCleared is whether the old history-full key was cleared since the start (full.go, ADR 0595 §6).
fullCleared bool
// filterRefused is, by consumer, the refusal of the last addition of filters, so a standing refusal is said once.
@@ -206,6 +206,7 @@ func declaredFor(m catalogue.Manifest, seats map[string]catalogue.SeatDeclaratio
if _, reads := m.OwnSecrets["broker"]; !reads {
d.NoAccount = true
}
d.HasProgram = !d.NoAccount && hasOwnProcess(m)
if _, reads := m.OwnSecrets[buscompose.ProvisionerSecret]; reads && m.ClaimsSeat(buscompose.NodeProvisionerSeat) {
d.ProvisionsNode = true
}
@@ -651,6 +652,21 @@ func queueOwnersOf(m catalogue.Manifest) []buscompose.QueueOwner {
return out
}
// hasOwnProcess says the manifest names a process resource with no `bus` of its own: one that runs on the module's own
// account and reads its consumer <node>_<module> (H-A-472 second addendum).
func hasOwnProcess(m catalogue.Manifest) bool {
buses := m.ProcessBuses()
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "process" {
continue
}
if _, bus := buses[fmt.Sprint(r["id"])]; !bus {
return true
}
}
return false
}
// ownServesOf is the seat verbs of the module's processes that declare no bus, which run on the module's own account (design
// 47 G A-115, A-127); a process with a bus carries its own serves (processesOf).
func ownServesOf(m catalogue.Manifest) []string {
+17 -1
View File
@@ -73,6 +73,13 @@
"maximum": 32,
"trusted": false,
"default": 4
},
"twin-current-hours": {
"type": "number",
"description": "how many hours since a module's durable's twin last delivered it is current (design 47 H2 H-A-472): the consumer-behind check leaves out a durable with no pulls waiting whose twin has pulls waiting and a delivery within these hours, and names it in durable-orphaned. The bus module's manifest names the same setting for nats_consumer_remove.",
"minimum": 1,
"trusted": false,
"default": 24
}
},
"emits": [
@@ -175,6 +182,14 @@
"mode": "0600",
"content": "${config:grants-for-revisions-per-branch}"
},
{
"id": "twin-current-hours",
"type": "file",
"path": "${dir:state}/twin-current-hours",
"owner": "mesh-grants",
"mode": "0600",
"content": "${config:twin-current-hours}"
},
{
"id": "grants",
"type": "process",
@@ -185,7 +200,8 @@
],
"env": {
"MESH_BROKER_FILE": "${dir:state}/broker",
"MESH_GRANTS_REVISIONS_PER_BRANCH_FILE": "${dir:state}/grants-for-revisions-per-branch"
"MESH_GRANTS_REVISIONS_PER_BRANCH_FILE": "${dir:state}/grants-for-revisions-per-branch",
"MESH_GRANTS_TWIN_CURRENT_HOURS_FILE": "${dir:state}/twin-current-hours"
},
"user": "mesh-grants",
"health": {