Compare commits

...
10 Commits
Author SHA1 Message Date
mesh-tools e196064e51 Merge pull request 'Design 47 H: the builder's publish queue: live asks first, newest per series, one listing per pass (H-A-290..293)' (#316) from fix/47-H1-builder-publish-queue into main 2026-11-21 18:00:25 +00:00
jschoubben 2eaa41a420 H-A-292, H-A-293: builder_publish_queue reads one page; a build is retired only within its series
The review found two faults. The tool paged on through a state of any size, one replay of the bucket per page; it now
reads one page of each of its own states and, when the state holds more, answers that page capped, with the keys read
and the state's size from its info, never asking a second page and never refusing. And the retiring of superseded done
builds keyed the newest reservation and release by module alone, so a standing 0.2 would retire a done 0.1.x; both are
now found per module and series (MAJOR.MINOR).

Builds: design 47 H H-A-292, H-A-293, H-A-291 (1), H-A-290 (5)
2026-11-21 18:59:58 +01:00
jschoubben 90ff867e7a lock the test fakes' maps
Two tests failed under -race on main too: the re-ask state fake's maps were written by a try on the watch's goroutine while
the test read and deleted them, and the operator-ask test read the build store unlocked and waited on the builder while
the build its answer starts could still be counting itself in. The fake's maps are locked and read through it, the store
is read under its lock, and the builder is waited on only once that build has claimed its key.

Builds: design 47 H H-A-290 (the module's tests pass with -race, as its checks ask)
2026-11-21 18:59:58 +01:00
jschoubben 77fb6c4ab1 H-A-291: a start publishes only each module's newest done build; a live enqueue cuts a start list short
A start asked a publish again for every unreleased done build, older ones whose digests a newer reservation or release
of the same module made dead included, and a live publish queued during a long start pass waited for its end. A done
build a newer reservation or release (the builder's own record) supersedes is now retired at the start and at the pass:
its key dropped, its publish item ended, said in the journal; the start asks only each module's newest. A live enqueue
marks its holder, and a pass about to send a start-class item stops there, lets the sends in flight end and lists again,
so the live one goes next; a start-class enqueue never does.

Builds: design 47 H H-A-291 (1), (2), H-A-290, H-A-275
2026-11-21 18:59:58 +01:00
jschoubben bda61ae64c H-A-290: the builder's publish queue sends live publishes first, in passes, never a withdrawn build
The tool runner's release waited forty minutes behind old versions: every start asked a publish again for every done
build, and each send relisted and reread the whole queue. A publish whose build ended in this run now goes before any
asked again at a start (the class kept on the key's value, a live key of a build before the start made a start key at
the start); a pass lists a holder's keys once and sends them four at a time; a withdrawn or taken-over build is dropped
with its publish item ended at enqueue, at the pass and at the start; a send that returns unsent says why, and every call
it makes honours its one bound; builder_publish_queue reads its states in pages and answers counts per holder and class,
the oldest and newest queue time, a list capped at fifty and one module's items with their class.

Builds: design 47 H H-A-290 (1)-(5), H-A-269, H-A-271, H-A-275
2026-11-21 18:59:58 +01:00
mesh-tools 5547fd028e Merge pull request 'Design 47 H: a pipeline with nothing open ends at once; the end's reads are made once (H-A-288, H-A-289)' (#315) from fix/47-H1-runner-end-when-empty into main 2026-11-21 17:22:44 +00:00
jschoubben d51d389c1b Runner: a pipeline with nothing open ends at once (47-H1-runner-end-when-empty)
main.117 judged its end at 17:45:19 and never wrote it: every mover's widens
was read inside the end's compare-and-set loop, 126 movers each cut on
grants-for, so each try took about eleven minutes and every write was refused
as changed until the restart, whose start pass judged no end.

The reads an end needs are made once, before the compare-and-set, shared by
every mover and every retry; the runner's start pass ends each running
pipeline with nothing open before the queue is placed; every end says its
outcome in the journal.

Builds: design 47 H H-A-288, H-A-289, H-A-223, H-A-243
2026-11-21 18:18:32 +01:00
mesh-tools c6edd5987f Merge pull request 'Design 47 H: a re-pin of the same source keeps a mover in its pipeline (H-A-283..286)' (#314) from fix/47-H1-runner-same-source-repin into main 2026-11-21 17:00:58 +00:00
jschoubben 9a49b0f4f7 Give a kept mover's asks ids of its kept revision and place it from that revision (H-A-286)
Review round 1: a kept mover's ask ids still came from the pipeline's start
time, so a pin-held ask or a composition took the id of an ask the runner
had just dropped for the old fingerprint, and a restart never matched the
steps placed again; and a kept mover placed later read the branch's held
view, so a lagging view gave the new fingerprint the old pins. Every ask id
of a kept mover's steps (build, tests, validate, pin-held asks,
composition, rerun, restart) now mixes its kept revision into the id's
sum, and every placing of a kept mover (at once, later, at a restart, a
rerun) reads its entries at its kept revision by number, waiting while
that revision cannot be read. The test's starter gets a clock that moves.

Builds: design 47 H H-A-286 (amending H-A-284; with H-A-283, H-A-285)
2026-11-21 17:58:07 +01:00
jschoubben 47f81051be Keep a same-source re-pin's mover in its pipeline and build it again there (H-A-283)
A module importing packages its own pipeline releases was re-pinned at
each publish; each re-pin revision of the same source took it out of the
running pipeline, and each newer build withdrew the older reservation, so
nothing ever published. A revision of the same source now moves no mover
out of the open pipeline of that source: the pipeline keeps it at its new
fingerprint, ends every step of it superseded (said at the old
fingerprint, its asks dropped, its runs left) and places them again from
the re-pin revision's entry under new asks, the build first, still naming
the pipeline's revision. The revision the mover is kept at rides in the
record's kept-revisions, beside the movers, where the pin check reads its
entry. A revision whose every mover is kept starts no pipeline; one of a
new source moves movers as before.

Builds: design 47 H H-A-283, H-A-284, H-A-285 (with H-A-223, H-A-244, ADR 0682)
2026-11-21 17:47:54 +01:00
27 changed files with 1943 additions and 172 deletions
@@ -14,7 +14,8 @@ import (
"mesh-delivery-builder/internal/versions"
)
// readerOf is a kept state as the versions package's reader.
// readerOf is a kept state as the versions package's reader: its values read in one page, one request for many (design 47 H
// H-A-290 (5), H-A-292), where a key-by-key read of a queue of two thousand keys outlasts the tool's call.
func readerOf(s stdio.KeptState) versions.Reader {
return versions.Reader{Keys: s.Keys, Get: func(key string) (json.RawMessage, bool, error) {
e, err := s.Get(key)
@@ -22,7 +23,34 @@ func readerOf(s stdio.KeptState) versions.Reader {
return nil, false, err
}
return e.Value, true, nil
}}
}, All: func() (versions.StatePage, error) { return onePage(s.Page, s.Info) }}
}
// statePageBytes and statePageBudget bound the one page of a state read, so its answer fits one bus message and its time.
const (
statePageBytes = 768 * 1024
statePageBudget = 10 * time.Second
)
// onePage is the first page of a state's values, and never a second (H-A-292, H-A-293): when the state holds more, the page says
// so with the state's size from its info. A runtime that answers no page says errors.ErrUnsupported.
func onePage(page func(after string, maxBytes int, budget time.Duration) (stdio.StatePage, error), info func() (*stdio.StateInfo, error)) (versions.StatePage, error) {
pg, err := page("", statePageBytes, statePageBudget)
if err != nil {
return versions.StatePage{}, err
}
out := versions.StatePage{Values: make(map[string]json.RawMessage, len(pg.Entries)), More: pg.Next != ""}
for _, e := range pg.Entries {
out.Values[e.Key] = e.Value
}
if out.More {
if in, err := info(); err != nil || in == nil {
out.InfoWords = fmt.Sprintf("the state's info cannot be read: %v", err)
} else {
out.Keys, out.Bytes = in.Keys, in.Bytes
}
}
return out, nil
}
// republishTools are builder_publish_again and builder_publish_queue (design 47 G §588). The publish again is a hand act:
@@ -59,7 +87,11 @@ func republishTools(queue, acted, reasks versions.Reader, ask queueAsker, now fu
return map[string]any{"key": key, "queued": a.Queued, "answer": a.Answer}, nil
}},
{Name: versions.PublishQueueTool, Description: versions.PublishQueueDescription,
Input: map[string]any{"type": "object", "properties": map[string]any{}},
Run: func(map[string]any) (any, error) { return versions.AnswerQueue(queue, acted, reasks) }},
Input: map[string]any{"type": "object", "properties": map[string]any{
"module": map[string]any{"type": "string", "description": "list only this module's queued versions, each with its class"},
"version": map[string]any{"type": "string", "description": "with module: only this version, MAJOR.MINOR.PATCH"}}},
Run: func(args map[string]any) (any, error) {
return versions.AnswerQueue(queue, acted, reasks, str(args, "module"), str(args, "version"))
}},
}
}
@@ -0,0 +1,44 @@
package main
import (
"encoding/json"
"errors"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// TestAStateIsReadInOnePage: design 47 H H-A-292, H-A-293 — builder_publish_queue reads each of its own states in one page and
// never asks a second: a state past the page answers that page with More and the state's size from its info; one the page holds
// answers whole without reading the info.
func TestAStateIsReadInOnePage(t *testing.T) {
for _, c := range []struct {
name string
next string
}{{"past one page", "k1"}, {"held by one page", ""}} {
t.Run(c.name, func(t *testing.T) {
var asked []string
infos := 0
page := func(after string, maxBytes int, budget time.Duration) (stdio.StatePage, error) {
asked = append(asked, after)
if after != "" {
return stdio.StatePage{Entries: []stdio.StateEntry{{Key: "k2", Value: json.RawMessage(`2`)}}}, nil
}
return stdio.StatePage{Entries: []stdio.StateEntry{{Key: "k1", Value: json.RawMessage(`1`)}}, Next: c.next}, nil
}
info := func() (*stdio.StateInfo, error) { infos++; return &stdio.StateInfo{Keys: 2, Bytes: 640}, nil }
pg, err := onePage(page, info)
more := c.next != ""
if err != nil || len(asked) != 1 || len(pg.Values) != 1 || pg.More != more || (more && (pg.Keys != 2 || pg.Bytes != 640 || infos != 1)) || (!more && infos != 0) {
t.Errorf("asked the pages after %q, read %v, more %v, size %d keys %d bytes, info read %d times (%v): want one page only, its rest said with the state's size", asked, pg.Values, pg.More, pg.Keys, pg.Bytes, infos, err)
}
})
}
unsupported := func(string, int, time.Duration) (stdio.StatePage, error) {
return stdio.StatePage{}, errors.ErrUnsupported
}
if _, err := onePage(unsupported, nil); !errors.Is(err, errors.ErrUnsupported) {
t.Errorf("a runtime with no pages answered %v, want ErrUnsupported so the reader reads key by key", err)
}
}
@@ -261,7 +261,8 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
republish := &versions.Republisher{Reserved: reserved.r, Queue: awaitedQueue{awaitedReserved{awaitedVersions{b: queueKV, bucket: own.queue}}},
Entry: entryVia(calls.Call), Send: sendVia(calls.Call),
Unreleased: reserved.unreleasedWords, Logf: logger.Printf,
Holders: holders} // a registry seat's holder from the seat-holders view, from the start (design 47 G A-416, A-416a)
Holders: holders, // a registry seat's holder from the seat-holders view, from the start (design 47 G A-416, A-416a)
Started: time.Now()} // a build ended from now on is of the live class, sent first (H-A-290 (1))
// A number a registry seat already holds is never reserved (H-A-275): read before reserving, through the seats' verb versions.
reserved.r.HeldNumbers = func(ctx context.Context, module, fp string, refs []string) ([]string, error) {
return republish.HeldNumbers(ctx, module, fp, refs, registryVersionsVia(calls.Call))
@@ -303,10 +304,10 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
return republish.Held(ctx, v, registryVersionsVia(calls.Call))
},
markHeld: reserved.markHeld, moveElsewhere: reserved.movedElsewhere,
allDone: func(ctx context.Context) ([]versions.ReservedVersion, error) {
all, err := reserved.r.All(ctx)
return builtOf(all), err
}}
allDone: func(ctx context.Context) ([]versions.ReservedVersion, error) { return reserved.r.All(ctx) },
retired: reserved.r.Retired} // only each module's newest done build is asked at the start (H-A-291 (1))
// A version that never publishes (withdrawn or taken over) has its publish item ended by the queue too (H-A-290 (2)).
republish.Ended = items.Done
reasksKV.Ready(func(jetstream.KeyValue) { items.AskAllDone(ctx) })
wirePublishItems(reserved, items, awaitedReserved{awaitedVersions{b: actedKV, bucket: own.acted}}, releasedKept)
acts.Republish = republish.Republish
@@ -46,7 +46,11 @@ type publishItems struct {
reserved func(ctx context.Context, module, version string) (versions.ReservedVersion, bool, error)
republish func(ctx context.Context, module, version string) ([]string, error)
endedWith func(ctx context.Context, v versions.ReservedVersion, why string) // raises the finding version-unreleased
allDone func(ctx context.Context) ([]versions.ReservedVersion, error)
// allDone are the reservations, done or not: those whose build is done are asked at the start (builtOf).
allDone func(ctx context.Context) ([]versions.ReservedVersion, error)
// retired are the done builds a newer reservation or release of the same module supersedes, with why (H-A-291 (1)); nil
// retires none.
retired func(ctx context.Context, all []versions.ReservedVersion) (map[string]string, error)
// unheard is how many messages the builder's durable on the events stream holds pending and unacknowledged, read before each
// try (design 47 H H-A-269 (2), H-A-272); nil reads none.
unheard func(ctx context.Context) (int, error)
@@ -98,12 +102,28 @@ func (p *publishItems) Ask(ctx context.Context, module, version string) error {
// AskAllDone asks the publish item of every reservation whose build is done and that has no release: at the builder's start, so
// what stood before this change moves too, whatever its artifacts.
func (p *publishItems) AskAllDone(ctx context.Context) {
vs, err := p.allDone(ctx)
all, err := p.allDone(ctx)
if err != nil {
p.logf("the reservations of done builds are not asked to publish at the start: %v", err)
return
}
for _, v := range vs {
var retired map[string]string
if p.retired != nil {
if retired, err = p.retired(ctx, all); err != nil {
p.logf("the superseded builds are not retired at the start: %v", err)
}
}
for _, v := range builtOf(all) {
if why := retired[v.Module+" "+v.Version]; why != "" { // only each module's newest is asked (H-A-291 (1), H-A-275)
p.logf("%s %s is not asked to publish at the start: %s; its publish item is ended", v.Module, v.Version, why)
p.Done(ctx, v.Module, v.Version)
continue
}
if why := versions.NotStanding(v, true); why != "" { // withdrawn: it never publishes (H-A-290 (2))
p.logf("%s %s is not asked to publish at the start: %s; its publish item is ended", v.Module, v.Version, why)
p.Done(ctx, v.Module, v.Version)
continue
}
if err := p.Ask(ctx, v.Module, v.Version); err != nil {
p.logf("%v", err)
}
@@ -137,7 +157,8 @@ func (p *publishItems) Try(ctx context.Context, kv jetstream.KeyValue, e jetstre
p.logf("%s %s: its reservation cannot be read, the publish item waits: %v", r.Module, r.Version, err)
return
}
if !held { // dropped, taken over or released: never because its artifacts are published (A-456b)
if why := versions.NotStanding(v, held); why != "" { // dropped, taken over, released or withdrawn: never because its artifacts are published (A-456b, H-A-290 (2))
p.logf("%s %s: the publish item is ended: %s", r.Module, r.Version, why)
_ = kv.Delete(ctx, e.Key())
return
}
@@ -4,9 +4,12 @@ import (
"context"
"encoding/json"
"errors"
"maps"
"os"
"path/filepath"
"slices"
"strings"
"sync"
"testing"
"time"
@@ -19,9 +22,10 @@ import (
"mesh-delivery-builder/internal/versions"
)
// itemKV is the re-ask state in memory: one revision per key.
// itemKV is the re-ask state in memory: one revision per key, its maps locked (a try writes from the re-ask watch's goroutine).
type itemKV struct {
jetstream.KeyValue
mu sync.Mutex
vals map[string][]byte
revs map[string]uint64
rev uint64
@@ -42,13 +46,25 @@ func (e itemEntry) Operation() jetstream.KeyValueOp { return jetstream.KeyValueP
func (k *itemKV) entry(key string) jetstream.KeyValueEntry {
return itemEntry{key: key, value: k.vals[key], rev: k.revs[key]}
}
// has says whether key holds a value.
func (k *itemKV) has(key string) bool {
k.mu.Lock()
defer k.mu.Unlock()
_, ok := k.vals[key]
return ok
}
func (k *itemKV) Get(_ context.Context, key string) (jetstream.KeyValueEntry, error) {
k.mu.Lock()
defer k.mu.Unlock()
if _, ok := k.vals[key]; !ok {
return nil, jetstream.ErrKeyNotFound
}
return k.entry(key), nil
}
func (k *itemKV) Create(_ context.Context, key string, v []byte, _ ...jetstream.KVCreateOpt) (uint64, error) {
k.mu.Lock()
defer k.mu.Unlock()
if _, ok := k.vals[key]; ok {
return 0, jetstream.ErrKeyExists
}
@@ -57,6 +73,8 @@ func (k *itemKV) Create(_ context.Context, key string, v []byte, _ ...jetstream.
return k.rev, nil
}
func (k *itemKV) Update(_ context.Context, key string, v []byte, rev uint64) (uint64, error) {
k.mu.Lock()
defer k.mu.Unlock()
if k.revs[key] != rev {
return 0, jetstream.ErrKeyExists
}
@@ -65,6 +83,8 @@ func (k *itemKV) Update(_ context.Context, key string, v []byte, rev uint64) (ui
return k.rev, nil
}
func (k *itemKV) Delete(_ context.Context, key string, _ ...jetstream.KVDeleteOpt) error {
k.mu.Lock()
defer k.mu.Unlock()
delete(k.vals, key)
return nil
}
@@ -140,21 +160,49 @@ func TestAPublishItemIsTriedWithAGrowingPauseAndEndsLoudly(t *testing.T) {
kv.vals, kv.revs = map[string][]byte{}, map[string]uint64{}
p.allDone = func(context.Context) ([]versions.ReservedVersion, error) {
published := versions.ReservedVersion{Module: "n", Version: "0.2.0", Artifacts: map[string]string{"n:a": "sha256:a"}, Published: map[string]bool{"n:a": true}, BuiltAt: now}
return builtOf([]versions.ReservedVersion{v, published, {Module: "o", Version: "0.1.0"}}), nil // the last build has not ended
withdrawn := versions.ReservedVersion{Module: "w", Version: "0.4.0", BuiltAt: now, Withdrawn: map[string]string{"0.4.0": "fp"}}
return builtOf([]versions.ReservedVersion{v, published, withdrawn, {Module: "o", Version: "0.1.0"}}), nil // the last build has not ended
}
_ = p.Ask(ctx, "w", "0.4.0") // asked before its reservation was withdrawn
p.AskAllDone(ctx)
if len(kv.vals) != 2 {
t.Errorf("the start asked %d items, want one per done unreleased reservation", len(kv.vals))
if _, ok := kv.vals[versions.PublishItemKey("w", "0.4.0")]; len(kv.vals) != 2 || ok {
t.Errorf("the start left %d items (withdrawn's kept: %v), want one per done unreleased reservation and none for a withdrawn one (H-A-290 (2))", len(kv.vals), ok)
}
// A start with three unreleased done versions of one module asks only the newest and retires the two (H-A-291 (1), H-A-275).
kv.vals, kv.revs = map[string][]byte{}, map[string]uint64{}
_ = p.Ask(ctx, "s", "0.1.3") // asked in an earlier run
p.allDone = func(context.Context) ([]versions.ReservedVersion, error) {
var out []versions.ReservedVersion
for _, ver := range []string{"0.1.3", "0.1.10", "0.1.2"} { // one series (H-A-293)
out = append(out, versions.ReservedVersion{Module: "s", Version: ver, Artifacts: map[string]string{"s:a": "sha256:a"}, BuiltAt: now})
}
return out, nil
}
p.retired = (&versions.Reserver{}).Retired
p.AskAllDone(ctx)
if _, ok := kv.vals[versions.PublishItemKey("s", "0.1.10")]; len(kv.vals) != 1 || !ok {
t.Errorf("the start left the items %v, want only the newest, s 0.1.10, asked and the two older retired", slices.Collect(maps.Keys(kv.vals)))
}
kv.vals, kv.revs = map[string][]byte{}, map[string]uint64{}
// A released version's item is deleted; a version no longer reserved has it deleted at its next try.
p.Done(ctx, "m", "0.1.2")
p.Done(ctx, "n", "0.2.0")
if len(kv.vals) != 0 {
t.Errorf("the release left %v", kv.vals)
}
// A withdrawn reservation has its item ended at its try, nothing queued (H-A-290 (2)).
queued = nil
_ = p.Ask(ctx, "m", "0.1.2")
v.Withdrawn = map[string]string{"0.1.2": "fp"}
e, _ := kv.Get(ctx, key)
p.Try(ctx, kv, e, decodeItem(t, e.Value()))
if len(kv.vals) != 0 || len(queued) != 0 {
t.Errorf("a withdrawn reservation left its item %v, queued %v", kv.vals, queued)
}
v.Withdrawn = nil
_ = p.Ask(ctx, "m", "0.1.2")
reserved = false
e, _ := kv.Get(ctx, key)
e, _ = kv.Get(ctx, key)
p.Try(ctx, kv, e, decodeItem(t, e.Value()))
if len(kv.vals) != 0 {
t.Errorf("a dropped reservation left its item %v", kv.vals)
@@ -220,7 +268,7 @@ func TestThePublishItemsAreWiredToTheReservationsAndTheReAskWatch(t *testing.T)
}
first := done("src2:one")
key := versions.PublishItemKey("m", first.Version)
if _, ok := kv.vals[key]; !ok {
if !kv.has(key) {
t.Fatalf("a done build has no publish item: %v", kv.vals)
}
// A build whose artifacts are all published still gets its item, until its release is written (A-456b).
@@ -231,7 +279,7 @@ func TestThePublishItemsAreWiredToTheReservationsAndTheReAskWatch(t *testing.T)
if err := rv.Built(ctx, "m", first.Version, map[string]string{"m:a": "sha256:a"}, nil); err != nil {
t.Fatal(err)
}
if _, ok := kv.vals[key]; !ok {
if !kv.has(key) {
t.Fatalf("a build with every artifact published has no item: %v", kv.vals)
}
// The watch hands the item to its try, which queues the publish.
@@ -247,11 +295,11 @@ func TestThePublishItemsAreWiredToTheReservationsAndTheReAskWatch(t *testing.T)
}
// A takeover by another build deletes the older's item and asks the newer's.
second := done("src2:two")
if _, ok := kv.vals[key]; ok {
if kv.has(key) {
t.Errorf("the taken-over reservation kept its item")
}
secondKey := versions.PublishItemKey("m", second.Version)
if _, ok := kv.vals[secondKey]; !ok {
if !kv.has(secondKey) {
t.Fatalf("the newer build has no item: %v", kv.vals)
}
// version-released deletes the item and keeps the release.
@@ -259,7 +307,7 @@ func TestThePublishItemsAreWiredToTheReservationsAndTheReAskWatch(t *testing.T)
if !rv.heard(ctx, msg) || !msg.acked {
t.Fatal("version-released was not taken in")
}
if _, ok := kv.vals[secondKey]; ok {
if kv.has(secondKey) {
t.Errorf("a released version kept its publish item")
}
if ok, _ := rv.ReleasedFor(ctx, "m", "src2:two"); !ok {
@@ -270,7 +318,7 @@ func TestThePublishItemsAreWiredToTheReservationsAndTheReAskWatch(t *testing.T)
if _, err := rv.dropped(ctx, "m", "0.1", "op", "never", time.Unix(0, 0)); err != nil {
t.Fatal(err)
}
if _, ok := kv.vals[versions.PublishItemKey("m", third.Version)]; ok {
if kv.has(versions.PublishItemKey("m", third.Version)) {
t.Errorf("a dropped reservation kept its item")
}
}
@@ -33,8 +33,8 @@ func (s awaitedQueue) Bytes(ctx context.Context) (uint64, error) {
// entryVia reads a catalog graph entry by its fingerprint through the seat mesh-catalog's verb entry.
func entryVia(call toolCaller) func(ctx context.Context, fingerprint string) (meshcatalog.CatalogEntry, error) {
return func(_ context.Context, fingerprint string) (meshcatalog.CatalogEntry, error) {
raw, err := call("seat:"+meshcatalog.Seat+"."+meshcatalog.VerbEntry, meshcatalog.EntryInput{Fingerprint: fingerprint})
return func(ctx context.Context, fingerprint string) (meshcatalog.CatalogEntry, error) {
raw, err := callWithin(ctx, call, "seat:"+meshcatalog.Seat+"."+meshcatalog.VerbEntry, meshcatalog.EntryInput{Fingerprint: fingerprint})
if err != nil {
return meshcatalog.CatalogEntry{}, err
}
@@ -50,8 +50,8 @@ func entryVia(call toolCaller) func(ctx context.Context, fingerprint string) (me
// artifact store, addressed by digest. The four seats' publish take one wire form; an answer that lacks parts, or a fault of
// the call, is the holder's refusal, said in its words (not stored when the holder lacks the bytes).
func sendVia(call toolCaller) func(ctx context.Context, p versions.Publish) error {
return func(_ context.Context, p versions.Publish) error {
raw, err := call("seat:"+p.Seat+"."+versions.VerbPublish, publishBody(p))
return func(ctx context.Context, p versions.Publish) error {
raw, err := callWithin(ctx, call, "seat:"+p.Seat+"."+versions.VerbPublish, publishBody(p))
if err != nil {
return fmt.Errorf("%s's publish of %s %s: %w", p.Seat, p.Name, p.Version, err)
}
@@ -66,6 +66,29 @@ func sendVia(call toolCaller) func(ctx context.Context, p versions.Publish) erro
}
}
// callWithin is call within ctx's bound (design 47 H H-A-290 (4)): its answer, or, once the bound ends first, that it was not
// answered within it. The call itself ends by its own bound (the SDK's call timeout), its answer then dropped.
func callWithin(ctx context.Context, call toolCaller, key string, body any) (json.RawMessage, error) {
if err := ctx.Err(); err != nil {
return nil, fmt.Errorf("%s is not called: %w", key, context.Cause(ctx))
}
type answer struct {
raw json.RawMessage
err error
}
done := make(chan answer, 1)
go func() {
raw, err := call(key, body)
done <- answer{raw, err}
}()
select {
case a := <-done:
return a.raw, a.err
case <-ctx.Done():
return nil, fmt.Errorf("%s is not answered within its bound: %w", key, context.Cause(ctx))
}
}
// publishBody is the seat's publish input for p: the digest, its kind and name and version, no parts.
func publishBody(p versions.Publish) gopackageregistry.PublishInput {
return gopackageregistry.PublishInput{Commit: p.Commit, Digest: p.Digest, Fingerprint: p.Fingerprint, Kind: p.Kind, Name: p.Name, Transfer: p.Transfer, Version: p.Version}
@@ -57,6 +57,26 @@ func TestRepublishSend(t *testing.T) {
t.Errorf("%s: asked %q, error %v, want %q", name, asked, err, c.want)
}
}
// A send and its entry read honour the bound they are given (design 47 H H-A-290 (4)): a call still unanswered when it ends
// is said as not answered within it, and the send goes on.
release := make(chan struct{})
defer close(release)
called := make(chan struct{}, 2)
slow := func(string, any) (json.RawMessage, error) {
called <- struct{}{}
<-release
return json.RawMessage(`{}`), nil
}
for name, call := range map[string]func(context.Context) error{
"the publish": func(ctx context.Context) error { return sendVia(slow)(ctx, p) },
"the entry": func(ctx context.Context) error { _, err := entryVia(slow)(ctx, "fp"); return err },
} {
ctx, cancel := context.WithCancel(context.Background())
go func() { <-called; cancel() }()
if err := call(ctx); err == nil || !strings.Contains(err.Error(), "within its bound") {
t.Errorf("%s past its bound answered %v, want it not answered within its bound", name, err)
}
}
if _, _, err := build.NewRepublish("", "0.1.3", "why", "me", versionsNow()); err == nil {
t.Error("a version with no module is accepted")
}
@@ -102,7 +102,10 @@ func seed(t *testing.T, st *memStore, b builds.BuilderBuild) string {
func valueOf(t *testing.T, st *memStore, key string) builds.BuilderBuild {
t.Helper()
b, err := builds.DecodeBuilderBuild(st.vals[key])
st.mu.Lock()
raw := st.vals[key]
st.mu.Unlock()
b, err := builds.DecodeBuilderBuild(raw)
if err != nil {
t.Fatal(err)
}
@@ -95,10 +95,14 @@ func theOperatorAnswers(t *testing.T, option string) {
if err != nil || h != occ.Acted {
t.Fatalf("%s: %v, %v", option, h, err)
}
for i := 0; i < 50 && option == OptionBuildOnceMore; i++ { // the build the answer starts runs on its own goroutine
r.b.Wait()
if valueOf(t, r.st, r.key).State == builds.BuildDone {
// The build the answer starts runs on its own goroutine: the builder is waited on only once that build has claimed the key
// (its start counted before its claim, read through the store's lock), never while its start may still be counted.
for i := 0; i < 50 && option == OptionBuildOnceMore; i++ {
if st := valueOf(t, r.st, r.key).State; st == builds.BuildDone {
break
} else if st != builds.BuildFailed {
r.b.Wait()
continue
}
time.Sleep(time.Millisecond)
}
@@ -7,6 +7,7 @@ import (
"fmt"
"slices"
"sort"
"strconv"
"strings"
"sync"
"time"
@@ -38,6 +39,15 @@ const (
// (H-A-272).
VerbPublish = "publish"
VerbVersions = "versions"
// PublishClassLive is the class of a publish whose build ended in this run, which a pipeline's build step waits on;
// PublishClassStart that of a publish asked again at a start, of a build that ended before it. A holder's queue sends every
// live one before any of the start, each class oldest first (design 47 H H-A-290 (1)).
PublishClassLive = "build-ended-this-run"
PublishClassStart = "asked-again-at-start"
// PublishSendsAtOnce bounds the sends of one holder's queue that run at once (H-A-290 (3)).
PublishSendsAtOnce = 4
// PublishQueueListCap caps the queued versions and the publish items builder_publish_queue lists (H-A-290 (5)).
PublishQueueListCap = 50
)
// RepublishTransfer is a publish again's transfer name, the builder's own (the registry part's publish: the caller names its
@@ -78,6 +88,26 @@ type QueuedPublish struct {
Artifacts map[string]string `json:"artifacts"`
Queued time.Time `json:"queued"`
Begun bool `json:"begun,omitempty"`
// Class is PublishClassLive or PublishClassStart (H-A-290 (1)); a value written before it had one is of the start.
Class string `json:"class,omitempty"`
}
// rank is the place of q's class in a holder's queue: the live first.
func (q QueuedPublish) rank() int {
if q.Class == PublishClassLive {
return 0
}
return 1
}
// sortQueued sorts a holder's queued versions in the order they are sent: the live class first, each class oldest first.
func sortQueued(qs []QueuedPublish) {
sort.SliceStable(qs, func(i, j int) bool {
if a, b := qs[i].rank(), qs[j].rank(); a != b {
return a < b
}
return qs[i].Queued.Before(qs[j].Queued)
})
}
// QueueFilter is the filter of every queued version: a queue key has three tokens, which AllFilter (two) never lists on the bus.
@@ -121,9 +151,18 @@ type Republisher struct {
// Holders answers a registry seat's holders: the catalog's SeatHolders, the part's one holder function that its FollowSeat
// of the assignments takes too, read from the builder's start (design 47 G A-416, A-416a); nil answers none.
Holders func(seat string) []string
// Started is when this run of the builder started: a version whose build ended at or after it is of the live class, one
// that ended before it of the start's (H-A-290 (1)). Zero makes every version live.
Started time.Time
// Ended ends the publish item of a version that never publishes, its reservation withdrawn or taken over (H-A-290 (2));
// nil ends none.
Ended func(ctx context.Context, module, version string)
mu sync.Mutex
busy map[string]bool
// relist marks a holder whose pass lists again before its next start-class send: a live publish was queued meanwhile
// (H-A-291 (2)).
relist map[string]bool
// lastQueued is the newest time a version was queued at: a clock that has not moved between two versions (a coarse clock,
// or a fixed one) must not make the queue's order the order of its keys, so each is queued after the last.
lastQueued time.Time
@@ -155,6 +194,127 @@ func (r *Republisher) logf(f string, a ...any) {
}
}
// classOf is the class of v's publish: live when its build ended in this run (H-A-290 (1)).
func (r *Republisher) classOf(v ReservedVersion) string {
if v.BuiltAt.Before(r.Started) {
return PublishClassStart
}
return PublishClassLive
}
// NotStanding says why the reservation of a version, as Of answers it (found), never publishes: it stands no more at the
// version (released, dropped or taken over by a newer build), or it was withdrawn (design 47 H H-A-225, H-A-279). Empty when it
// stands (H-A-290 (2)).
func NotStanding(v ReservedVersion, found bool) string {
switch {
case !found:
return "its reservation stands no more at the version (released, dropped or taken over)"
case hasKey(v.Withdrawn, v.Version):
return "its reservation was withdrawn"
}
return ""
}
// newerVersion says whether MAJOR.MINOR.PATCH a is above b, number by number.
func newerVersion(a, b string) bool {
pa, pb := strings.Split(a, "."), strings.Split(b, ".")
if len(pa) != 3 || len(pb) != 3 {
return false
}
for i := range 3 {
x, errx := strconv.Atoi(pa[i])
y, erry := strconv.Atoi(pb[i])
if errx != nil || erry != nil {
return false
}
if x != y {
return x > y
}
}
return false
}
// Retired are the done builds without a release among all that a newer reservation or a newer release of the same module and
// series (MAJOR.MINOR) supersedes, each with why (design 47 H H-A-291 (1), H-A-293; H-A-275: the older digest is dead): keyed
// `<module> <version>`. A standing newer series never retires an older one's build. The releases are the builder's own record
// of them (Releases), read once per module.
func (r *Reserver) Retired(ctx context.Context, all []ReservedVersion) (map[string]string, error) {
seriesOf := func(v ReservedVersion) string {
s, _, err := SplitVersion(v.Version)
if err != nil {
return ""
}
return v.Module + " " + s
}
newest := map[string]string{} // module and series -> its newest reserved version
for _, v := range all {
k := seriesOf(v)
if n, ok := newest[k]; k != "" && (!ok || newerVersion(v.Version, n)) {
newest[k] = v.Version
}
}
kept := map[string][]string{} // module -> the releases the record keeps
read := map[string]bool{}
out := map[string]string{}
for _, v := range all {
k := seriesOf(v)
if v.BuiltAt.IsZero() || k == "" {
continue
}
if n := newest[k]; newerVersion(n, v.Version) {
out[v.Module+" "+v.Version] = "the newer reservation " + n + " of " + v.Module + " supersedes it"
continue
}
if !read[v.Module] && r.Releases != nil {
read[v.Module] = true
raw, _, found, err := r.Releases.Get(ctx, ReleasedKey(v.Module))
if err != nil {
return out, fmt.Errorf("the releases of %s cannot be read: %w", v.Module, err)
}
var rec ReleasedRecord
if found && json.Unmarshal(raw, &rec) == nil {
for _, rb := range rec.Kept {
kept[v.Module] = append(kept[v.Module], rb.Version)
}
}
}
series, _, _ := SplitVersion(v.Version)
rel := ""
for _, kv := range kept[v.Module] {
if s, _, err := SplitVersion(kv); err == nil && s == series && newerVersion(kv, v.Version) && (rel == "" || newerVersion(kv, rel)) {
rel = kv
}
}
if rel != "" {
out[v.Module+" "+v.Version] = "the newer release " + rel + " of " + v.Module + " supersedes it"
}
}
return out, nil
}
// drop deletes every queued key of module's version and ends its publish item: it never publishes (H-A-290 (2)), said with why.
func (r *Republisher) drop(ctx context.Context, module, version, why string) {
ctx = context.WithoutCancel(ctx)
keys, err := r.Queue.KeysOf(ctx, "*."+EscapeToken(module)+"."+EscapeToken(version))
if err != nil {
r.logf("publish again: %s %s never publishes (%s), and its queued keys cannot be listed: %v", module, version, why, err)
}
for _, k := range keys {
if err := r.Queue.Delete(ctx, k); err != nil {
r.logf("%s cannot be deleted from %s: %v", k, PublishQueueState, err)
}
}
r.end(ctx, module, version, why)
}
// end ends module's version's publish item, said with why.
func (r *Republisher) end(ctx context.Context, module, version, why string) {
r.logf("publish again: %s %s is not published: %s; its queued key is dropped and its publish item ended", module, version, why)
if r.Ended != nil {
r.Ended(context.WithoutCancel(ctx), module, version)
}
}
// ActedKey is the acted key of a subject.
func ActedKey(subject string) string { return EscapeToken(subject) }
@@ -189,6 +349,10 @@ func (r *Republisher) queue(ctx context.Context, vs []ReservedVersion, holdersOf
if max > 0 && len(queued) >= max {
break
}
if why := NotStanding(v, true); why != "" {
r.drop(ctx, v.Module, v.Version, why)
continue
}
if v.Fingerprint == "" || v.BuiltAt.IsZero() {
r.logf("publish again: %s %s is not queued: it holds no fingerprint or its build has not ended with every digest", v.Module, v.Version)
continue
@@ -251,7 +415,7 @@ func (r *Republisher) enqueue(ctx context.Context, v ReservedVersion, holder str
r.logf("publish again: %s %s is queued at %s already", v.Module, v.Version, holder)
return false, nil
}
raw, _ := json.Marshal(QueuedPublish{V: 1, Holder: holder, Module: v.Module, Version: v.Version, Artifacts: arts, Queued: r.queuedAt()})
raw, _ := json.Marshal(QueuedPublish{V: 1, Holder: holder, Module: v.Module, Version: v.Version, Artifacts: arts, Queued: r.queuedAt(), Class: r.classOf(v)})
used, err := r.Queue.Bytes(ctx)
if err != nil {
return false, fmt.Errorf("the size of the state %s cannot be read: %w", PublishQueueState, err)
@@ -268,7 +432,15 @@ func (r *Republisher) enqueue(ctx context.Context, v ReservedVersion, holder str
}
return false, fmt.Errorf("%s cannot be queued: %w", key, err)
}
r.logf("publish again: %s %s is queued to publish at %s: %d artifacts", v.Module, v.Version, holder, len(arts))
r.logf("publish again: %s %s is queued to publish at %s: %d artifacts, class %s", v.Module, v.Version, holder, len(arts), r.classOf(v))
if r.classOf(v) == PublishClassLive { // a pass sending a start list lists again, so this one goes next (H-A-291 (2))
r.mu.Lock()
if r.relist == nil {
r.relist = map[string]bool{}
}
r.relist[holder] = true
r.mu.Unlock()
}
return true, nil
}
@@ -305,13 +477,43 @@ func (r *Republisher) Republish(ctx context.Context, module, version string) ([]
return r.queue(ctx, vs, r.holdersFn, 0)
}
// Queued are the queued versions, per holder in the order they are sent, and the acted keys with their sequences:
// builder_publish_queue.
// Queued is builder_publish_queue's answer, small at any size of the queue (design 47 H H-A-290 (5)): the queued versions'
// count per holder and class with the oldest and newest queue time, the first PublishQueueListCap of them in the order they are
// sent (each with its class), the acted keys with their sequences, and the publish items, counted and capped alike.
type Queued struct {
Queue []QueuedPublish `json:"queue"`
Acted map[string]uint64 `json:"acted"`
// Items are the publish items (design 47 G A-457a), each with its tries and its next try; an item that ended says why.
Items []PublishItem `json:"items"`
Total int `json:"total"`
Counts []QueueCount `json:"counts"`
// Queue are the first queued versions, per holder in the order they are sent (the live class first, each class oldest first);
// with a module named, only that module's.
Queue []QueuedPublish `json:"queue"`
// Listed says how many of Total the answer lists.
Listed int `json:"listed"`
Acted map[string]uint64 `json:"acted"`
// ItemCount is how many publish items stand; Items the first of them by their next try (design 47 G A-457a), each with its
// tries and its next try; an item that ended says why.
ItemCount int `json:"item-count"`
Items []PublishItem `json:"items"`
// Capped says a state held more than the one page read of it (H-A-293); Read names each such state, the keys read and its
// size. Every count above is then of the keys read.
Capped bool `json:"capped,omitempty"`
Read []StateCapped `json:"read,omitempty"`
}
// capped notes a state read only in part.
func (q *Queued) capped(c *StateCapped) {
if c != nil {
q.Capped = true
q.Read = append(q.Read, *c)
}
}
// QueueCount is how many versions one holder's queue holds of one class, and when the oldest and the newest were queued.
type QueueCount struct {
Holder string `json:"holder"`
Class string `json:"class"`
Count int `json:"count"`
Oldest time.Time `json:"oldest"`
Newest time.Time `json:"newest"`
}
// PublishItemKeyPrefix opens the key of a publish item in the builder's re-ask state.
@@ -335,69 +537,113 @@ func PublishItemKey(module, version string) string {
return PublishItemKeyPrefix + EscapeToken(module) + "." + EscapeToken(version)
}
// AnswerQueue is builder_publish_queue over the two sections as the tools bundle reads them.
func AnswerQueue(queue, acted, reasks Reader) (Queued, error) {
out := Queued{Queue: []QueuedPublish{}, Acted: map[string]uint64{}, Items: []PublishItem{}}
keys, err := queue.Keys()
// AnswerQueue is builder_publish_queue over the three states as the tools bundle reads them (H-A-290 (5)): each state read in
// bulk (Reader.All), the answer counted and capped at PublishQueueListCap whatever the queue holds; module (and version) narrow the
// listed queue to one module's (one version's), so an item's class can be read.
func AnswerQueue(queue, acted, reasks Reader, module, version string) (Queued, error) {
out := Queued{Counts: []QueueCount{}, Queue: []QueuedPublish{}, Acted: map[string]uint64{}, Items: []PublishItem{}}
vals, c, err := queue.values(PublishQueueState)
out.capped(c)
if err != nil {
return out, fmt.Errorf("the state %s cannot be listed: %w", PublishQueueState, err)
return out, fmt.Errorf("the state %s cannot be read: %w", PublishQueueState, err)
}
for _, k := range keys {
raw, ok, err := queue.Get(k)
if err != nil || !ok {
counts := map[[2]string]*QueueCount{}
var listed []QueuedPublish
for _, raw := range vals {
var q QueuedPublish
if json.Unmarshal(raw, &q) != nil {
continue
}
var q QueuedPublish
if json.Unmarshal(raw, &q) == nil {
if q.Class == "" {
q.Class = PublishClassStart
}
out.Total++
c := counts[[2]string{q.Holder, q.Class}]
if c == nil {
c = &QueueCount{Holder: q.Holder, Class: q.Class, Oldest: q.Queued, Newest: q.Queued}
counts[[2]string{q.Holder, q.Class}] = c
}
c.Count++
if q.Queued.Before(c.Oldest) {
c.Oldest = q.Queued
}
if q.Queued.After(c.Newest) {
c.Newest = q.Queued
}
if (module == "" || q.Module == module) && (version == "" || q.Version == version) {
listed = append(listed, q)
}
}
for _, c := range counts {
out.Counts = append(out.Counts, *c)
}
sort.Slice(out.Counts, func(i, j int) bool {
if out.Counts[i].Holder != out.Counts[j].Holder {
return out.Counts[i].Holder < out.Counts[j].Holder
}
return out.Counts[i].Class < out.Counts[j].Class
})
sort.SliceStable(listed, func(i, j int) bool { return listed[i].Holder < listed[j].Holder })
byHolder := map[string][]QueuedPublish{}
var holders []string
for _, q := range listed {
if byHolder[q.Holder] == nil {
holders = append(holders, q.Holder)
}
byHolder[q.Holder] = append(byHolder[q.Holder], q)
}
for _, h := range holders {
qs := byHolder[h]
sortQueued(qs)
for _, q := range qs {
if len(out.Queue) == PublishQueueListCap {
break
}
out.Queue = append(out.Queue, q)
}
}
sort.SliceStable(out.Queue, func(i, j int) bool {
if out.Queue[i].Holder != out.Queue[j].Holder {
return out.Queue[i].Holder < out.Queue[j].Holder
}
return out.Queue[i].Queued.Before(out.Queue[j].Queued)
})
akeys, err := acted.Keys()
out.Listed = len(out.Queue)
avals, c, err := acted.values(ActedState)
out.capped(c)
if err != nil {
return out, fmt.Errorf("the state %s cannot be listed: %w", ActedState, err)
return out, fmt.Errorf("the state %s cannot be read: %w", ActedState, err)
}
for _, k := range akeys {
raw, ok, err := acted.Get(k)
if err != nil || !ok {
continue
}
for k, raw := range avals {
var a Acted
if json.Unmarshal(raw, &a) == nil {
out.Acted[k] = a.Sequence
}
}
rkeys, err := reasks.Keys()
rvals, c, err := reasks.values("reasks")
out.capped(c)
if err != nil {
return out, fmt.Errorf("the re-ask state cannot be listed: %w", err)
return out, fmt.Errorf("the re-ask state cannot be read: %w", err)
}
for _, k := range rkeys {
for k, raw := range rvals {
if !strings.HasPrefix(k, PublishItemKeyPrefix) {
continue
}
raw, ok, err := reasks.Get(k)
if err != nil || !ok {
continue
}
var it PublishItem
if json.Unmarshal(raw, &it) == nil {
out.Items = append(out.Items, it)
}
}
out.ItemCount = len(out.Items)
sort.SliceStable(out.Items, func(i, j int) bool { return out.Items[i].NextTry.Before(out.Items[j].NextTry) })
if len(out.Items) > PublishQueueListCap {
out.Items = out.Items[:PublishQueueListCap]
}
return out, nil
}
// PublishQueueDescription is builder_publish_queue's description.
const PublishQueueDescription = "The versions the builder will publish again (its state " + PublishQueueState + ", design 47 G section 588): per registry " +
"seat holder, in the order they are sent, each with its artifacts' digests, when it was queued and whether its publish has begun; and the " +
"keys of the state " + ActedState + ", the last stream sequence the builder acted on for each subject; and the publish items (design 47 G " +
"A-457a), each version's tries and next try, and why one ended after its tries. Read-only."
const PublishQueueDescription = "The versions the builder will publish again (its state " + PublishQueueState + ", design 47 G section 588, H H-A-290): " +
"how many each registry seat holder's queue holds per class (build-ended-this-run, sent first, or asked-again-at-start) with the oldest and newest " +
"queue time; the first 50 in the order they are sent, each with its class, its artifacts' digests, when it was queued and whether its publish has " +
"begun (module, and version, list only that module's, so one item's class can be read); the keys of the state " + ActedState + ", the last stream " +
"sequence the builder acted on for each subject; and the publish items (design 47 G A-457a), counted, the first 50 by next try, each version's " +
"tries and next try, and why one ended after its tries. Each state is read in one page; a state past it is answered capped, naming the " +
"keys read and the state's size (H-A-293). Read-only."
// PublishAgainDescription is builder_publish_again's description.
const PublishAgainDescription = "Publish reserved versions again (design 47 G section 588), which builds nothing: with module and version, that reserved " +
@@ -406,22 +652,57 @@ const PublishAgainDescription = "Publish reserved versions again (design 47 G se
"with the versions it queued; each outcome is said as package-published, or left in version-unreleased with the holder's words. A hand act, " +
"recorded in the hand-act log."
// Resume takes up, at the builder's start, every queued version again, a begun one sent anew (§588).
// Resume takes up, at the builder's start, every queued version again, a begun one sent anew (§588): a version whose reservation
// was withdrawn or taken over is dropped with its publish item ended (H-A-290 (2)), and one of the live class whose build ended
// before this run is of the start's now (H-A-290 (1)). It reads the reservations once.
func (r *Republisher) Resume(ctx context.Context) error {
keys, err := r.Queue.KeysOf(ctx, QueueFilter)
if err != nil {
return fmt.Errorf("the state %s cannot be listed: %w", PublishQueueState, err)
}
all, err := r.Reserved.All(ctx)
if err != nil {
return fmt.Errorf("the queued publishes are not taken up: %w", err)
}
reserved := make(map[string]ReservedVersion, len(all))
for _, v := range all {
reserved[v.Module+" "+v.Version] = v
}
retired, err := r.Reserved.Retired(ctx, all)
if err != nil {
r.logf("publish again: the superseded builds are not retired at the start: %v", err)
}
holders := map[string]bool{}
for _, k := range keys {
raw, _, found, err := r.Queue.Get(ctx, k)
raw, rev, found, err := r.Queue.Get(ctx, k)
if err != nil || !found {
continue
}
var q QueuedPublish
if json.Unmarshal(raw, &q) == nil {
holders[q.Holder] = true
if json.Unmarshal(raw, &q) != nil {
continue
}
v, ok := reserved[q.Module+" "+q.Version]
why := NotStanding(v, ok)
if why == "" {
why = retired[q.Module+" "+q.Version]
}
if why != "" {
if err := r.Queue.Delete(ctx, k); err != nil {
r.logf("%s cannot be deleted from %s: %v", k, PublishQueueState, err)
}
r.end(ctx, q.Module, q.Version, why)
continue
}
if class := r.classOf(v); q.Class != class {
q.Class = class
if raw, err := json.Marshal(q); err == nil {
if _, err := r.Queue.Update(ctx, k, raw, rev); err != nil {
r.logf("%s keeps its class %s at the start: %v", k, q.Class, err)
}
}
}
holders[q.Holder] = true
}
for h := range holders {
r.drainLater(ctx, h)
@@ -451,16 +732,22 @@ func (r *Republisher) drainLater(ctx context.Context, holder string) {
go run()
}
// Drain sends holder's queued versions one at a time, oldest first, each bounded by UnreleasedBound; a key is deleted when
// its outcome is said.
// Drain sends holder's queued versions in passes (H-A-290 (3)): a pass lists the holder's keys once, reads each once, and sends
// them in their order (the live class first, each class oldest first), PublishSendsAtOnce at a time; the keys are listed again
// only when the pass is spent, or when a live publish was queued while it sends a start list: then the sends in flight end and
// the keys are listed again, so the live one goes next (H-A-291 (2)); a start-class one queued meanwhile waits for the pass's end.
// Each send is bounded by UnreleasedBound; a key is deleted when its outcome is said.
func (r *Republisher) Drain(ctx context.Context, holder string) {
for ctx.Err() == nil {
r.mu.Lock()
delete(r.relist, holder)
r.mu.Unlock()
keys, err := r.Queue.KeysOf(ctx, EscapeToken(holder)+".*.*")
if err != nil {
r.logf("the queue of %s cannot be listed: %v", holder, err)
return
}
var items []QueuedPublish
items := make([]QueuedPublish, 0, len(keys))
for _, k := range keys {
if raw, _, found, err := r.Queue.Get(ctx, k); err == nil && found {
var q QueuedPublish
@@ -472,12 +759,35 @@ func (r *Republisher) Drain(ctx context.Context, holder string) {
if len(items) == 0 {
return
}
sort.SliceStable(items, func(i, j int) bool { return items[i].Queued.Before(items[j].Queued) })
r.send(ctx, items[0])
sortQueued(items)
sem := make(chan struct{}, PublishSendsAtOnce)
var wg sync.WaitGroup
for _, q := range items {
if ctx.Err() != nil {
break
}
sem <- struct{}{}
r.mu.Lock()
again := q.Class != PublishClassLive && r.relist[holder]
r.mu.Unlock()
if again {
<-sem
r.logf("publish again: %s's pass lists again after its sends in flight: a live publish was queued", holder)
break
}
wg.Add(1)
go func() {
defer func() { <-sem; wg.Done() }()
r.send(ctx, q)
}()
}
wg.Wait()
}
}
// send publishes one queued version's artifacts and says the outcome, then deletes its key.
// send publishes one queued version's artifacts and says the outcome, then deletes its key. Every call it makes honours the one
// bound it is given, UnreleasedBound (a context's bound, no timer, ADR 0437 §8), and a send that returns without sending says why
// (H-A-290 (4)).
func (r *Republisher) send(ctx context.Context, q QueuedPublish) {
key := QueueKey(q.Holder, q.Module, q.Version)
defer func() {
@@ -485,18 +795,44 @@ func (r *Republisher) send(ctx context.Context, q QueuedPublish) {
r.logf("%s cannot be deleted from %s: %v", key, PublishQueueState, err)
}
}()
bound, cancel := context.WithTimeout(ctx, UnreleasedBound)
defer cancel()
if !q.Begun {
q.Begun = true
raw, _ := json.Marshal(q)
if _, rev, found, err := r.Queue.Get(ctx, key); err == nil && found {
_, _ = r.Queue.Update(ctx, key, raw, rev)
if _, rev, found, err := r.Queue.Get(bound, key); err == nil && found {
_, _ = r.Queue.Update(bound, key, raw, rev)
}
}
v, found, err := r.Reserved.Of(ctx, q.Module, q.Version)
if err != nil || !found {
return // released meanwhile, or dropped
all, err := r.Reserved.All(bound)
if err != nil {
r.logf("publish again: %s %s is not sent to %s: its reservation cannot be read: %v; its publish item asks it again", q.Module, q.Version, q.Holder, err)
return
}
e, err := r.Entry(ctx, v.Fingerprint)
var v ReservedVersion
found := false
var mine []ReservedVersion
for _, w := range all {
if w.Module == q.Module {
mine = append(mine, w)
if w.Version == q.Version {
v, found = w, true
}
}
}
why := NotStanding(v, found)
if why == "" {
retired, err := r.Reserved.Retired(bound, mine)
if err != nil {
r.logf("publish again: %s %s: whether a newer version supersedes it cannot be read, so it is sent: %v", q.Module, q.Version, err)
}
why = retired[q.Module+" "+q.Version]
}
if why != "" {
r.end(ctx, q.Module, q.Version, why)
return
}
e, err := r.Entry(bound, v.Fingerprint)
if err != nil {
r.say(ctx, v, fmt.Sprintf("the catalog entry %s cannot be read: %v", v.Fingerprint, err))
return
@@ -512,10 +848,8 @@ func (r *Republisher) send(ctx context.Context, q QueuedPublish) {
r.say(ctx, v, ref+": the catalog entry names no registry seat for it")
return
}
bound, cancel := context.WithTimeout(ctx, UnreleasedBound) // a context's bound, no timer (ADR 0437 §8)
err := r.Send(bound, Publish{Seat: seat, Kind: kind, Name: name, Version: v.Version, Digest: q.Artifacts[ref], Fingerprint: v.Fingerprint,
Commit: v.Commit, Transfer: RepublishTransfer(q.Module, v.Version, strings.TrimPrefix(ref, q.Module+":"))})
cancel()
if err == nil {
r.logf("publish again: %s %s: %s sent to %s as %s %s, digest %s; the package-published is said when the seat says published", q.Module, q.Version, ref, seat, name, v.Version, q.Artifacts[ref])
}
@@ -523,7 +857,7 @@ func (r *Republisher) send(ctx context.Context, q QueuedPublish) {
r.logf("publish again: %s %s: %s refused by %s: %v", q.Module, q.Version, ref, seat, err)
words := err.Error()
if strings.Contains(words, strings.TrimSuffix(NotStoredWords, ": ")) {
if ok, merr := r.Reserved.MarkUnstored(ctx, q.Module, q.Version, ref, words); merr != nil {
if ok, merr := r.Reserved.MarkUnstored(bound, q.Module, q.Version, ref, words); merr != nil {
r.logf("%s %s: the unstored %s is not kept: %v", q.Module, q.Version, ref, merr)
} else if ok {
r.say(ctx, v, ref+" "+NotStoredWords+words)
@@ -5,8 +5,13 @@ import (
"encoding/json"
"errors"
"fmt"
"maps"
"slices"
"strings"
"sync"
"sync/atomic"
"testing"
"testing/synctest"
"time"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
@@ -15,10 +20,19 @@ import (
type queueMem struct {
*reserveMem
bytes uint64
// listings counts the listings of one holder's keys (a filter `<holder>.*.*`), what a drain's pass lists.
listings atomic.Int32
}
func (q *queueMem) Bytes(context.Context) (uint64, error) { return q.bytes, nil }
func (q *queueMem) KeysOf(ctx context.Context, filter string) ([]string, error) {
if strings.HasSuffix(filter, ".*.*") && filter != QueueFilter {
q.listings.Add(1)
}
return q.reserveMem.KeysOf(ctx, filter)
}
// republishBed is a Republisher over memory states: reservations of module m (a bundle by the oci seat and a Go package by
// the Go seat), one holder module `oci` that claims the oci seat, and a Send and a Unreleased that record.
type republishBed struct {
@@ -26,8 +40,10 @@ type republishBed struct {
reserved *reserveMem
queue *queueMem
now time.Time
mu sync.Mutex // sent and said, which a holder's sends at once write
sent []Publish
said []string
ended []string
refuse string // the words Send fails with, empty for none
// holders are the registry seats' holders the seat-holders view names (design 47 G A-416).
holders map[string][]string
@@ -48,6 +64,8 @@ func newRepublishBed(t *testing.T) *republishBed {
{Ref: m + ":lib", Kind: "package", Language: "go", Package: &meshcatalog.ArtifactPackage{Ecosystem: "go", Name: "git.novox.be/novox/" + m + "/go"}}}}, nil
},
Send: func(_ context.Context, p Publish) error {
b.mu.Lock()
defer b.mu.Unlock()
b.sent = append(b.sent, p)
if b.refuse != "" {
return errors.New(b.refuse)
@@ -55,8 +73,15 @@ func newRepublishBed(t *testing.T) *republishBed {
return nil
},
Unreleased: func(_ context.Context, v ReservedVersion, why string) {
b.mu.Lock()
defer b.mu.Unlock()
b.said = append(b.said, v.Module+" "+v.Version+": "+why)
},
Ended: func(_ context.Context, module, version string) {
b.mu.Lock()
defer b.mu.Unlock()
b.ended = append(b.ended, module+" "+version)
},
Logf: t.Logf}
b.Holders = func(seat string) []string { return b.holders[seat] }
return b
@@ -190,7 +215,7 @@ func TestRepublish(t *testing.T) {
return json.RawMessage(`{"mode":"publish","module":"m","version":"0.1.0","tries":3,"next-try":"2026-11-17T01:00:00Z","why":"w"}`), true, nil
}}
none := Reader{Keys: func() ([]string, error) { return nil, nil }}
answer, err := AnswerQueue(none, none, items)
answer, err := AnswerQueue(none, none, items, "", "")
if err != nil || len(answer.Items) != 1 || answer.Items[0].Tries != 3 || answer.Items[0].Version != "0.1.0" {
t.Errorf("builder_publish_queue items = %+v (%v), want the one publish item with its tries", answer.Items, err)
}
@@ -235,3 +260,279 @@ func TestTheRegistriesRecordSaysWhetherAVersionIsHeld(t *testing.T) {
t.Errorf("a version with no missing artifact is judged %+v (%v), want all held without a read", j, err)
}
}
// queueAt writes a queued version of module m at holder oci directly, as a start finds it.
func (b *republishBed) queueAt(t *testing.T, v ReservedVersion, class string, at time.Time) {
t.Helper()
raw, _ := json.Marshal(QueuedPublish{V: 1, Holder: "oci", Module: v.Module, Version: v.Version, Artifacts: map[string]string{v.Module + ":prog": "sha256:aa"}, Queued: at, Class: class})
if _, err := b.queue.Create(context.Background(), QueueKey("oci", v.Module, v.Version), raw); err != nil {
t.Fatal(err)
}
}
// TestAHoldersQueueSendsLiveFirstInPasses: design 47 H H-A-290 (1)-(3) — a publish whose build ended in this run is sent before
// any of a build that ended before the start, each class oldest first; a pass lists the holder's keys once and sends them, at most
// four at a time; a build whose reservation was withdrawn or taken over never publishes, its key dropped and its publish item ended,
// at enqueue, at the pass and at the start.
func TestAHoldersQueueSendsLiveFirstInPasses(t *testing.T) {
ctx := context.Background()
t.Run("the live class first, at most four sends at once", func(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
b := newRepublishBed(t)
var old []ReservedVersion
for _, m := range []string{"o1", "o2", "o3", "o4", "o5"} {
old = append(old, b.reserve(t, m))
}
b.Started = b.now
b.reserve(t, "live")
for i, v := range old {
b.queueAt(t, v, PublishClassStart, b.now.Add(time.Duration(i)*time.Second))
}
b.now = b.now.Add(time.Hour) // the live one is queued last, after every old one
b.holders = map[string][]string{SeatArtifacts: {"oci"}}
release := make(chan struct{})
var arrived []string
var inflight, most int
send := b.Send
b.Send = func(ctx context.Context, p Publish) error {
b.mu.Lock()
arrived = append(arrived, p.Name)
inflight++
most = max(most, inflight)
b.mu.Unlock()
<-release
b.mu.Lock()
inflight--
b.mu.Unlock()
return send(ctx, p)
}
b.Go = func(f func()) { go f() }
if q, err := b.Republish(ctx, "live", ""); err != nil || len(q) != 1 {
t.Fatalf("queued %v (%v), want the live version", q, err)
}
synctest.Wait() // every send the bound lets run waits on release
b.mu.Lock()
first := slices.Clone(arrived)
b.mu.Unlock()
if len(first) != PublishSendsAtOnce || !slices.Contains(first, "live/prog") {
t.Errorf("the sends running at once are %v: want %d, the live one among them", first, PublishSendsAtOnce)
}
close(release)
synctest.Wait()
if len(b.sent) != 6 || most != PublishSendsAtOnce {
t.Errorf("sent %d, at most %d at once: want all six, %d at once", len(b.sent), most, PublishSendsAtOnce)
}
})
})
t.Run("a pass lists the keys once for all its sends", func(t *testing.T) {
b := newRepublishBed(t)
var vs []ReservedVersion
for i := range 6 {
vs = append(vs, b.reserve(t, fmt.Sprintf("m%d", i)))
}
for i, v := range vs {
b.queueAt(t, v, PublishClassStart, b.now.Add(time.Duration(i)*time.Second))
}
b.Drain(ctx, "oci")
if n := b.queue.listings.Load(); len(b.sent) != 6 || n != 2 {
t.Errorf("sent %d with %d listings, want six sends from one pass and one listing that finds it spent", len(b.sent), n)
}
})
t.Run("a start makes a live key of a build before it a start key", func(t *testing.T) {
b := newRepublishBed(t)
v := b.reserve(t, "m")
b.Started = b.now
b.queueAt(t, v, PublishClassLive, b.now)
b.Go = func(func()) {} // the start's drain is not run here
if err := b.Resume(ctx); err != nil {
t.Fatal(err)
}
raw, _, _, _ := b.queue.Get(ctx, QueueKey("oci", "m", v.Version))
var q QueuedPublish
if json.Unmarshal(raw, &q) != nil || q.Class != PublishClassStart {
t.Errorf("the key after the start is %s, want the class %s: its build did not end in this run", raw, PublishClassStart)
}
})
for _, at := range []string{"enqueue", "pass", "start"} {
t.Run("a withdrawn build never publishes, at the "+at, func(t *testing.T) {
b := newRepublishBed(t)
v := b.reserve(t, "m")
b.holders = map[string][]string{SeatArtifacts: {"oci"}}
if _, ok, err := b.Reserved.Withdraw(ctx, "m", v.Version); err != nil || !ok {
t.Fatalf("withdrawn %v (%v)", ok, err)
}
switch at {
case "enqueue":
b.queueAt(t, v, PublishClassLive, b.now)
b.Go = func(func()) {} // a key queued before the withdrawal is dropped by the enqueue itself
if q, _ := b.Republish(ctx, "m", v.Version); len(q) != 0 {
t.Errorf("queued %v", q)
}
case "pass":
b.queueAt(t, v, PublishClassLive, b.now)
b.Drain(ctx, "oci")
case "start":
b.queueAt(t, v, PublishClassLive, b.now)
b.Go = func(func()) {}
if err := b.Resume(ctx); err != nil {
t.Fatal(err)
}
}
keys, _ := b.queue.KeysOf(ctx, QueueFilter)
if len(b.sent) != 0 || len(keys) != 0 || fmt.Sprint(b.ended) != "[m "+v.Version+"]" {
t.Errorf("sent %v, keys %v, items ended %v: want nothing sent, the key dropped and m %s's item ended", b.sent, keys, b.ended, v.Version)
}
})
}
t.Run("a live publish queued mid-pass goes after the sends in flight, before the rest of the start list", func(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
b := newRepublishBed(t)
var old []ReservedVersion
for _, m := range []string{"o1", "o2", "o3", "o4", "o5", "o6"} {
old = append(old, b.reserve(t, m))
}
b.Started = b.now
b.reserve(t, "live")
for i, v := range old {
b.queueAt(t, v, PublishClassStart, b.now.Add(time.Duration(i)*time.Second))
}
b.now = b.now.Add(time.Hour)
b.holders = map[string][]string{SeatArtifacts: {"oci"}}
first, then := make(chan struct{}), make(chan struct{}) // the four in flight wait on first, every later send on then
var arrived []string
send := b.Send
b.Send = func(ctx context.Context, p Publish) error {
b.mu.Lock()
arrived = append(arrived, p.Name)
gate := map[bool]chan struct{}{true: first, false: then}[len(arrived) <= PublishSendsAtOnce]
b.mu.Unlock()
<-gate
return send(ctx, p)
}
b.Go = func(f func()) { go f() }
b.drainLater(ctx, "oci")
synctest.Wait() // four start sends in flight, the pass holding o5 and o6
if q, err := b.Republish(ctx, "live", ""); err != nil || len(q) != 1 {
t.Fatalf("queued %v (%v)", q, err)
}
close(first)
synctest.Wait() // the sends after the four: with the live one among them, or o5 and o6 alone when the pass goes on
b.mu.Lock()
next := slices.Clone(arrived[PublishSendsAtOnce:])
b.mu.Unlock()
if !slices.Contains(next, "live/prog") {
t.Errorf("after the four in flight the sends are %v, want the live one among them, the pass listed again", next)
}
close(then)
synctest.Wait()
if len(arrived) != 7 {
t.Errorf("sent %v, want all seven", arrived)
}
})
})
t.Run("a build a newer patch or release of its series supersedes is retired at the pass and at the start, never by a newer series", func(t *testing.T) {
b := newRepublishBed(t)
v := b.reserve(t, "m")
b.holders = map[string][]string{SeatArtifacts: {"oci"}}
if _, err := b.Reserved.Reserve(ctx, "m", "0.2", []string{"m:prog"}, Source{Fingerprint: "src2:m"}); err != nil {
t.Fatal(err)
}
b.queueAt(t, v, PublishClassStart, b.now)
b.Drain(ctx, "oci")
if len(b.sent) != 1 || len(b.ended) != 0 {
t.Errorf("with a standing 0.2: sent %v, ended %v, want m %s sent, its series not superseded (H-A-293)", b.sent, b.ended, v.Version)
}
rel := &reserveMem{mem: newMem()}
raw, _ := json.Marshal(ReleasedRecord{Kept: []ReleasedBuild{{Fingerprint: "f", Version: "0.2.3"}, {Fingerprint: "g", Version: "0.1.9"}}})
_, _ = rel.Create(ctx, ReleasedKey("m"), raw)
b.Reserved.Releases = rel
b.sent = nil
b.queueAt(t, v, PublishClassStart, b.now)
b.Drain(ctx, "oci")
if len(b.sent) != 0 || fmt.Sprint(b.ended) != "[m "+v.Version+"]" {
t.Errorf("at the pass: sent %v, ended %v, want m %s retired by the release 0.1.9", b.sent, b.ended, v.Version)
}
b.ended = nil
b.queueAt(t, v, PublishClassStart, b.now)
b.Go = func(func()) {}
if err := b.Resume(ctx); err != nil {
t.Fatal(err)
}
if keys, _ := b.queue.KeysOf(ctx, QueueFilter); len(keys) != 0 || fmt.Sprint(b.ended) != "[m "+v.Version+"]" {
t.Errorf("at the start: keys %v, ended %v, want m %s retired", keys, b.ended, v.Version)
}
only := &reserveMem{mem: newMem()}
raw, _ = json.Marshal(ReleasedRecord{Kept: []ReleasedBuild{{Fingerprint: "f", Version: "0.2.0"}}})
_, _ = only.Create(ctx, ReleasedKey("n"), raw)
got, err := (&Reserver{Releases: only}).Retired(ctx, []ReservedVersion{{Module: "n", Version: "0.1.4", BuiltAt: b.now}, {Module: "n", Version: "0.1.6"},
{Module: "n", Version: "0.3.0", BuiltAt: b.now}, {Module: "p", Version: "0.1.2", BuiltAt: b.now}})
if err != nil || len(got) != 1 || !strings.Contains(got["n 0.1.4"], "reservation 0.1.6") {
t.Errorf("retired %v (%v), want only n 0.1.4, by the newer patch 0.1.6; n 0.3.0 and p 0.1.2 kept, a release of 0.2 retiring neither", got, err)
}
})
t.Run("a version taken over is not sent, and its item ended", func(t *testing.T) {
b := newRepublishBed(t)
v := b.reserve(t, "m")
b.queueAt(t, v, PublishClassStart, b.now)
if _, err := b.Reserved.Reserve(ctx, "m", "0.1", []string{"m:prog"}, Source{Fingerprint: "src2:other"}); err != nil {
t.Fatal(err)
}
b.Drain(ctx, "oci")
if len(b.sent) != 0 || fmt.Sprint(b.ended) != "[m "+v.Version+"]" {
t.Errorf("sent %v, ended %v: want nothing sent and the item ended", b.sent, b.ended)
}
})
}
// TestThePublishQueueToolAnswersAtAnySize: design 47 H H-A-290 (5) — builder_publish_queue over two thousand queued versions answers
// within a second and small: counts per holder and class with the oldest and newest queue time, a list capped at fifty in the
// order they are sent, and one module's item with its class; each state read in bulk where the reader can.
func TestThePublishQueueToolAnswersAtAnySize(t *testing.T) {
at := time.Date(2026, 11, 20, 17, 0, 0, 0, time.UTC)
vals := map[string]json.RawMessage{}
for i := range 2000 {
holder, class := "oci", PublishClassStart
if i%2 == 1 {
holder = "go"
}
if i >= 1990 {
class = PublishClassLive
}
q := QueuedPublish{V: 1, Holder: holder, Module: fmt.Sprintf("m%d", i), Version: "0.1.0", Artifacts: map[string]string{"a": "sha256:aa"}, Queued: at.Add(time.Duration(i) * time.Second), Class: class}
vals[QueueKey(holder, q.Module, q.Version)], _ = json.Marshal(q)
}
gets := 0
queue := Reader{All: func() (StatePage, error) { return StatePage{Values: vals}, nil },
Keys: func() ([]string, error) { return slices.Collect(maps.Keys(vals)), nil },
Get: func(k string) (json.RawMessage, bool, error) { gets++; v, ok := vals[k]; return v, ok, nil }}
none := Reader{Keys: func() ([]string, error) { return nil, nil }}
start := time.Now()
a, err := AnswerQueue(queue, none, none, "", "")
took := time.Since(start)
raw, _ := json.Marshal(a)
if err != nil || took > time.Second || a.Total != 2000 || len(a.Queue) != PublishQueueListCap || a.Listed != PublishQueueListCap || len(raw) > 64<<10 || gets != 0 {
t.Fatalf("answered %d of %d in %s, %d bytes, %d single reads (%v): want all counted, fifty listed, within a second, small, in bulk", len(a.Queue), a.Total, took, len(raw), gets, err)
}
if fmt.Sprintln(a.Counts[1].Holder, a.Counts[1].Class, a.Counts[1].Count, a.Counts[1].Oldest.Equal(at.Add(1991*time.Second)), a.Counts[1].Newest.Equal(at.Add(1999*time.Second))) != "go "+PublishClassLive+" 5 true true\n" || len(a.Counts) != 4 {
t.Errorf("counts %+v, want per holder and class with their oldest and newest", a.Counts)
}
if a.Queue[0].Class != PublishClassLive || a.Queue[0].Holder != "go" {
t.Errorf("the list opens with %+v, want the go holder's live class, sent first", a.Queue[0])
}
one, err := AnswerQueue(queue, none, none, "m1995", "")
if err != nil || len(one.Queue) != 1 || one.Queue[0].Class != PublishClassLive || one.Total != 2000 {
t.Errorf("one module's answer %+v (%v), want its item with its class", one.Queue, err)
}
if a.Capped || len(a.Read) != 0 {
t.Errorf("a state one page holds is answered capped: %+v", a.Read)
}
queue.All = func() (StatePage, error) { return StatePage{Values: vals, More: true, Keys: 5000, Bytes: 3 << 20}, nil }
if c, err := AnswerQueue(queue, none, none, "", ""); err != nil || !c.Capped || len(c.Read) != 1 || c.Read[0].State != PublishQueueState ||
c.Read[0].KeysRead != 2000 || c.Read[0].StateKeys != 5000 || c.Read[0].StateBytes != 3<<20 || c.Total != 2000 {
t.Errorf("a state past its one page answered capped %v, read %+v (%v): want capped, 2000 keys read of 5000, its size said (H-A-293)", c.Capped, c.Read, err)
}
queue.All = func() (StatePage, error) { return StatePage{}, errors.ErrUnsupported }
if b, err := AnswerQueue(queue, none, none, "", ""); err != nil || b.Total != 2000 || gets != 2000 {
t.Errorf("a reader with no bulk read answered %d (%v) with %d single reads, want every key read one by one", b.Total, err, gets)
}
}
@@ -21,10 +21,59 @@ const (
OldFormClosedEvent = "version.old-form-closed"
)
// Reader reads the state as the tools bundle may: every key, and one.
// Reader reads the state as the tools bundle may: every key, and one; and, where All is set, the state's values in one page,
// one request for many (design 47 H H-A-290 (5), H-A-292).
type Reader struct {
Keys func() ([]string, error)
Get func(key string) (json.RawMessage, bool, error)
// All is one page of the state's values; nil, or an error satisfying errors.Is(err, errors.ErrUnsupported), reads key by key.
All func() (StatePage, error)
}
// StatePage is one page of a state (H-A-292, H-A-293): its values, and, when the state holds more than the page, More with the
// state's size as its info says it (Keys, Bytes; InfoWords when the info cannot be read). No second page is asked.
type StatePage struct {
Values map[string]json.RawMessage
More bool
Keys, Bytes uint64
InfoWords string
}
// StateCapped is what an answer says of a state it read only the first page of (H-A-293): how many keys it read, and the
// state's size from its info.
type StateCapped struct {
State string `json:"state"`
KeysRead int `json:"keys-read"`
StateKeys uint64 `json:"state-keys"`
StateBytes uint64 `json:"state-bytes"`
InfoWords string `json:"info,omitempty"`
}
// values is the state's values: one page where the reader can (capped says what a page past the state's end left out), else
// key by key.
func (r Reader) values(state string) (map[string]json.RawMessage, *StateCapped, error) {
if r.All != nil {
pg, err := r.All()
if !errors.Is(err, errors.ErrUnsupported) {
if err != nil || !pg.More {
return pg.Values, nil, err
}
return pg.Values, &StateCapped{State: state, KeysRead: len(pg.Values), StateKeys: pg.Keys, StateBytes: pg.Bytes, InfoWords: pg.InfoWords}, nil
}
}
keys, err := r.Keys()
if err != nil {
return nil, nil, err
}
out := make(map[string]json.RawMessage, len(keys))
for _, k := range keys {
raw, ok, err := r.Get(k)
if err != nil || !ok {
continue
}
out[k] = raw
}
return out, nil, nil
}
// readStore is a Store over a Reader that refuses every write: the tools bundle only reads versions.
@@ -150,7 +150,7 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
}
logger.Printf("widens: each read bound %s, %d catalog revisions kept", readBound, revisionsKept)
widener := widenerOf(calls.Call, readBound, revisionsKept, logger.Printf)
hearer.Widens = widener.Widens
hearer.Widens = widener.Reads
hearer.LogLast = stepLog{ctx: ctx, js: js, module: cred.Module}.Last
hearer.Where = routerRecordOf(cred.Module, calls.Call, calls.Ask)
hearer.Failures = &pipeline.Failures{Findings: rv.Findings, Observer: cred.Module, Author: commitAuthor(calls.Call), Logf: logger.Printf}
@@ -177,6 +177,8 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
clock := askClock(ctx, bounds, askBound, silentBound, rv.Findings, cred.Module, time.Now)
starter.Asks, hearer.Asks = clock, clock
starter.Supersede = func(newer seat.Pipeline) { hearer.SupersedeRunning(newer) }
starter.Left = hearer.LeftInPlace // a same-source re-pin's superseded steps leave their runs (H-A-284)
starter.Revision = widener.Revision // a kept mover is placed from its kept revision (H-A-286)
hearer.Unplaced = func(p seat.Pipeline, module string) string { return starter.Unplaced(p, module, rv.Catalog.View) }
hearer.AskPinHeld = func(key, module string, pins map[string]string) {
if _, err := starter.AskPinHeld(hearer.Store, key, module, pins, widener.Revision); err != nil {
@@ -272,6 +274,10 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
if err := placeLostRuns(starter.UnplacedRuns, calls.CallKept, placeHalf, logger.Printf); err != nil {
logger.Printf("the runs not placed at the start: %v", err)
}
// A running pipeline nothing of which is still open ends now, its turn freed, before the queue is placed (H-A-288 (c)).
if n, err := hearer.EndAtStart(); err != nil || n > 0 {
logger.Printf("the pipelines with nothing open at the start: %d ended (%v)", n, err)
}
// The queue is placed at the start from its own state (§748).
if n, err := starter.StartQueue(rv.Catalog.View); err != nil || n > 0 {
logger.Printf("the queue at the start: %d waiting pipelines placed (%v)", n, err)
@@ -206,6 +206,7 @@ func (h *Hearer) OnDecided(event oc.Warrant) (seat.Pipeline, error) {
return seat.Pipeline{}, err
}
for _, key := range found {
wr := h.widensReads() // made before the compare-and-set, kept for its retries (H-A-289)
for {
p, rev, err := h.Store.Get(key)
if err != nil {
@@ -219,7 +220,7 @@ func (h *Hearer) OnDecided(event oc.Warrant) (seat.Pipeline, error) {
if err != nil || !changed {
return p, err
}
ended := h.endPipeline(&p, now)
ended := h.endPipeline(&p, now, wr)
i := stepOfAsk(p, w.Ask)
if i >= 0 {
p.Unlogged = append(p.Unlogged, endEntry(key, p.Steps[i], now))
@@ -308,6 +308,7 @@ func markRerun(a *Asked, caller string) {
// other mover's validate reading the ask with it, a composition its validate held ended not asked, the pipeline ended when
// nothing else is open, said and logged as any step's end.
func (h *Hearer) endUnanswered(key, ask, why string) error {
wr := h.widensReads() // made before the compare-and-set, kept for its retries (H-A-289)
for {
p, rev, err := h.Store.Get(key)
if err != nil {
@@ -324,14 +325,14 @@ func (h *Hearer) endUnanswered(key, ask, why string) error {
var readers []int
if s.Step == seat.StepResultStepValidate {
readers = sameValidate(&p, i, now)
_, notAsked := releaseCompositions(&p, now)
_, notAsked := releaseCompositions(&p, now, h.kept(key))
readers = append(readers, notAsked...)
}
asked, err := askApproval(key, &p, now, h.Asker)
if err != nil {
return err
}
ended := h.endPipeline(&p, now)
ended := h.endPipeline(&p, now, wr)
if _, err := h.Store.Update(key, p, rev); errors.Is(err, ErrChanged) {
continue
} else if err != nil {
@@ -0,0 +1,129 @@
package pipeline
import (
"fmt"
"slices"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-contracts/identity-and-access/go/seats/meshgrants"
builds "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshbuilds"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
)
// refusedOnce is a store whose first write of a pipeline is refused as changed, another write having come between (the
// pipeline's key moving while its end is judged, as main.117's did).
type refusedOnce struct {
memStore
refused map[string]bool
}
func (r *refusedOnce) Update(key string, p seat.Pipeline, rev uint64) (uint64, error) {
if !r.refused[key] {
r.refused[key] = true
e := r.memStore[key]
e.rev++
r.memStore[key] = e
return 0, ErrChanged
}
return r.memStore.Update(key, p, rev)
}
// H-A-288, H-A-289 (met with main.117 and main.76): a pipeline ends, and the next waiting is placed, the moment nothing of it is
// still open: when its last mover leaves, when its last open step ends by the silent bound, and at the runner's start pass;
// its outcome is said in the journal. The reads its end needs are made once, before the compare-and-set, shared by every mover
// and every retry of the write: a write refused once is written at the retry without asking grants-for again.
func TestAPipelineWithNothingOpenEndsAtOnce(t *testing.T) {
t0 := time.Date(2026, 11, 21, 17, 45, 19, 0, time.UTC)
movers := []seat.Mover{{Module: "a", Commit: "c1", Fingerprint: "src2:a1"}, {Module: "b", Commit: "c1", Fingerprint: "src2:b1"},
{Module: "c", Commit: "c1", Fingerprint: "src2:c1"}}
step := func(module, outcome, ask string) seat.StepResult {
return seat.StepResult{Module: module, Step: seat.StepResultStepBuild, Status: seat.StepResultStatusRequired, Outcome: outcome, Ask: ask, Seat: builds.Seat}
}
rows := []struct {
name string
steps []seat.StepResult
end func(t *testing.T, h *Hearer)
outcome string
}{
{"its last mover leaves for a newer pipeline (H-A-223)",
[]seat.StepResult{step("a", seat.StepResultOutcomeRunning, "x1"), step("b", seat.StepResultOutcomePassed, "x2"), step("c", seat.StepResultOutcomePassed, "x3")},
func(t *testing.T, h *Hearer) {
h.SupersedeRunning(seat.Pipeline{Pipeline: "main.120", Branch: "main", Revision: 120, Movers: []seat.Mover{
{Module: "a", Commit: "c2", Fingerprint: "src2:a2"}, {Module: "b", Commit: "c2", Fingerprint: "src2:b2"}, {Module: "c", Commit: "c2", Fingerprint: "src2:c2"}}})
}, seat.PipelineOutcomeSuperseded},
{"its last open step ends could not run on the silent bound (H-A-243)",
[]seat.StepResult{step("a", seat.StepResultOutcomeRunning, "mwql5pio3390aa"), step("b", seat.StepResultOutcomePassed, "x2"), step("c", seat.StepResultOutcomePassed, "x3")},
func(t *testing.T, h *Hearer) {
if err := h.endUnanswered("main.117", "mwql5pio3390aa", SilentWhy("mwql5pio3390aa")); err != nil {
t.Fatal(err)
}
}, seat.PipelineOutcomeFailed},
{"the runner's start pass finds it with nothing open",
[]seat.StepResult{step("a", seat.StepResultOutcomePassed, "x1"), step("b", seat.StepResultOutcomePassed, "x2"), step("c", seat.StepResultOutcomePassed, "x3")},
func(t *testing.T, h *Hearer) {
if n, err := h.EndAtStart(); n != 1 || err != nil {
t.Fatalf("the start pass ended %d (%v), want 1", n, err)
}
}, seat.PipelineOutcomePassed},
}
for _, r := range rows {
t.Run(r.name, func(t *testing.T) {
store := &refusedOnce{memStore: memStore{}, refused: map[string]bool{}}
store.memStore["main.117"] = struct {
p seat.Pipeline
rev uint64
}{seat.Pipeline{V: 1, Pipeline: "main.117", Branch: "main", Source: 117, Revision: 117, Outcome: seat.PipelineOutcomeRunning,
Movers: slices.Clone(movers), Steps: slices.Clone(r.steps)}, 1}
// Another running pipeline with a step open, and one waiting its turn: neither ends.
store.memStore["main.118"] = struct {
p seat.Pipeline
rev uint64
}{seat.Pipeline{V: 1, Pipeline: "main.118", Branch: "other", Source: 118, Revision: 118, Outcome: seat.PipelineOutcomeRunning,
Movers: movers[:1], Steps: []seat.StepResult{step("a", seat.StepResultOutcomeRunning, "y1")}}, 1}
store.memStore["main.119"] = struct {
p seat.Pipeline
rev uint64
}{seat.Pipeline{V: 1, Pipeline: "main.119", Branch: "other", Source: 119, Revision: 119, Outcome: seat.PipelineOutcomeWaiting, Movers: movers[:1]}, 1}
store.refused["main.118"], store.refused["main.119"] = true, true
grants, revisions := 0, 0
w := &Widener{
Revision: func(string, int64) (Half, error) { revisions++; return Half{}, nil },
Holder: func(s string) (string, error) { return s, nil },
GrantsFor: func(string, int64) (meshgrants.GrantsFor, error) {
grants++
return meshgrants.GrantsFor{}, fmt.Errorf("cut after 5s")
},
}
var turned, journal []string
h := &Hearer{Store: store, Say: func(string, any) error { return nil }, Now: func() time.Time { return t0 }, Widens: w.Reads,
Turn: func(key string) { turned = append(turned, key) },
Logf: func(f string, a ...any) { journal = append(journal, fmt.Sprintf(f, a...)) }}
r.end(t, h)
p, _, _ := store.Get("main.117")
if p.Outcome != r.outcome {
t.Fatalf("main.117 = %s (%s), want %s", p.Outcome, p.Why, r.outcome)
}
if !slices.Equal(turned, []string{"main.117"}) {
t.Errorf("the turns freed = %v, want main.117's alone, so the next waiting is placed", turned)
}
if !slices.ContainsFunc(journal, func(l string) bool { return strings.HasPrefix(l, "main.117 ended "+r.outcome) }) {
t.Errorf("the journal does not say main.117's end %s: %q", r.outcome, journal)
}
if grants != 1 || revisions != 2 {
t.Errorf("the end asked grants-for %d times and a revision %d times over 3 movers and a refused write, want 1 and 2", grants, revisions)
}
for _, m := range p.Movers {
if len(m.Widens) != 1 || m.Widens[0].Name != "mesh-grants" {
t.Errorf("%s's widens = %+v, want the cut grants-for said", m.Module, m.Widens)
}
}
for _, k := range []string{"main.118", "main.119"} {
if q, _, _ := store.Get(k); q.Outcome == seat.PipelineOutcomeSuperseded || q.Outcome == seat.PipelineOutcomePassed || q.Outcome == seat.PipelineOutcomeFailed {
t.Errorf("%s ended %s with a step open or waiting its turn", k, q.Outcome)
}
}
})
}
}
@@ -288,6 +288,65 @@ func (s *MoverStore) Get(key string) (seat.Pipeline, uint64, error) {
// when the record or a mover key was written since the read revision gave. A record carrying its steps inline has its
// mover keys written before the record, so its steps are never lost between the two.
func (s *MoverStore) Update(key string, p seat.Pipeline, revision uint64) (uint64, error) {
return s.update(key, p, revision, nil)
}
// UpdateKept is Update with kept laid over the record's kept-revisions (H-A-285): the catalog revision each kept mover's
// fingerprint is from (H-A-283, H-A-284 (2)).
func (s *MoverStore) UpdateKept(key string, p seat.Pipeline, revision uint64, kept map[string]int64) (uint64, error) {
return s.update(key, p, revision, kept)
}
// KeptRevisions are pipeline key's kept-revisions, the catalog revision each mover a same-source re-pin kept in it is from
// (H-A-283, H-A-284 (2), H-A-285); none for a pipeline that kept none.
func (s *MoverStore) KeptRevisions(key string) (map[string]int64, error) {
defer s.lock(key)()
raw, _, err := s.Raw.Get(key)
if err != nil {
return nil, err
}
if raw == nil {
return nil, fmt.Errorf("%w: %s", ErrNoPipeline, key)
}
return keptOf(raw)
}
// keptRevisionsField is the pipeline record's property kept-revisions, beside its movers (H-A-285): written and read by the
// runner alone, never part of the seat's Go form of a pipeline, which drops it on every decode.
const keptRevisionsField = "kept-revisions"
// keptOf is the kept-revisions a record carries; none when it carries none.
func keptOf(raw []byte) (map[string]int64, error) {
var fields map[string]json.RawMessage
if err := json.Unmarshal(raw, &fields); err != nil {
return nil, fmt.Errorf("the record is no object: %w", err)
}
f, ok := fields[keptRevisionsField]
if !ok {
return nil, nil
}
var kept map[string]int64
if err := json.Unmarshal(f, &kept); err != nil {
return nil, fmt.Errorf("the record's %s is no map of module to revision: %w", keptRevisionsField, err)
}
return kept, nil
}
// withKept is record with kept as its kept-revisions.
func withKept(record []byte, kept map[string]int64) ([]byte, error) {
var fields map[string]json.RawMessage
if err := json.Unmarshal(record, &fields); err != nil {
return nil, err
}
raw, err := json.Marshal(kept)
if err != nil {
return nil, err
}
fields[keptRevisionsField] = raw
return json.Marshal(fields)
}
func (s *MoverStore) update(key string, p seat.Pipeline, revision uint64, kept map[string]int64) (uint64, error) {
defer s.lock(key)()
record, want, err := split(key, p)
if err != nil {
@@ -303,6 +362,20 @@ func (s *MoverStore) Update(key string, p seat.Pipeline, revision uint64) (uint6
if cur != revision {
return 0, ErrChanged
}
// The record's kept-revisions ride on through every write, kept laid over them (H-A-285).
merged, err := keptOf(raw)
if err != nil {
return 0, fmt.Errorf("the pipeline %s: %w", key, err)
}
if len(kept) > 0 && merged == nil {
merged = map[string]int64{}
}
maps.Copy(merged, kept)
if len(merged) > 0 {
if record, err = withKept(record, merged); err != nil {
return 0, err
}
}
vals, revs, err := s.moverKeys(key)
if err != nil {
return 0, err
@@ -61,7 +61,18 @@ var moduleSteps = []string{seat.StepResultStepValidate, seat.StepResultStepCompo
// placement makes the same id at every start and the bus dedupes on it, and each mover's step its own id (design 47 H
// §746); started is the pipeline's start time, or a rerun's own (design 47 G A-297).
func AskID(started time.Time, pipeline, module, artifact, step string) string {
sum := sha256.Sum256([]byte(pipeline + "\n" + module + "\n" + artifact + "\n" + step))
return keptAskID(started, pipeline, module, artifact, step, 0)
}
// keptAskID is AskID of a step of a mover a same-source re-pin kept at catalog revision kept (H-A-286): the kept revision
// is mixed into the id's sum, so no id equals one the pipeline asked for the mover's old fingerprint, and every placing,
// pin-held ask, composition, rerun and restart of the kept mover finds the same ids again; kept 0 is AskID's own form.
func keptAskID(started time.Time, pipeline, module, artifact, step string, kept int64) string {
in := pipeline + "\n" + module + "\n" + artifact + "\n" + step
if kept > 0 {
in += "\nkept " + strconv.FormatInt(kept, 10)
}
sum := sha256.Sum256([]byte(in))
return strconv.FormatInt(started.UnixMilli(), 36) + hex.EncodeToString(sum[:3])
}
@@ -80,14 +91,16 @@ func startOf(p seat.Pipeline, now time.Time) time.Time {
// could not run naming the missing seat; the module's steps wait. It returns the steps and the asks to put on the seats'
// work queues, one per running step.
func Place(p seat.Pipeline, m seat.Mover, reg catalog.Registration, entries map[string]catalog.CatalogEntry, now time.Time, asker string) ([]seat.StepResult, []Asked) {
return place(p, m, reg, entries, now, asker, true)
return place(p, m, reg, entries, now, asker, true, 0)
}
// place is Place; share false places each build step its own ask, as a step asked before the test runs is (§730, A-468k).
// With share, the build steps of one candidate, level and share digest are asked once (§724): the first artifact's step, in
// reference order, asks the run and names it in Shared; every other step of the key reads it, keeping no ask of its own and
// naming the run's ask and the step that asked it (A-469m, A-469q).
func place(p seat.Pipeline, m seat.Mover, reg catalog.Registration, entries map[string]catalog.CatalogEntry, now time.Time, asker string, share bool) ([]seat.StepResult, []Asked) {
// kept is the catalog revision a same-source re-pin kept m at, 0 for a mover not kept: its every ask id mixes it in
// (H-A-286).
func place(p seat.Pipeline, m seat.Mover, reg catalog.Registration, entries map[string]catalog.CatalogEntry, now time.Time, asker string, share bool, kept int64) ([]seat.StepResult, []Asked) {
entry := entries[m.Module]
candidate, named := CandidateOf(p)
share = share && named
@@ -99,16 +112,9 @@ func place(p seat.Pipeline, m seat.Mover, reg catalog.Registration, entries map[
// The build step is every mover's first, required whatever the policy names, asked of the seat mesh-builds for the
// mover's own entry only (A-99, A-97): each mover is built by its own step (§746).
build := seat.StepResult{Step: seat.StepResultStepBuild, Module: m.Module, Status: seat.StepResultStatusRequired, Fingerprint: m.Fingerprint,
Seat: builds.Seat, Outcome: seat.StepResultOutcomeRunning, Started: &now, Ask: AskID(startOf(p, now), p.Pipeline, m.Module, "", seat.StepResultStepBuild)}
Seat: builds.Seat, Outcome: seat.StepResultOutcomeRunning, Started: &now, Ask: keptAskID(startOf(p, now), p.Pipeline, m.Module, "", seat.StepResultStepBuild, kept)}
steps = append(steps, build)
// A run on request's build step ask names no source (the seat mesh-builds' contract; H-A-162): its candidate's
// fingerprints are never in the builder's picture of the branch, so it waits for no revision there.
source := p.Source
if p.Run != "" {
source = 0
}
asks = append(asks, Asked{Seat: builds.Seat, ID: build.Ask, Commit: m.Commit, Ask: builds.StepRun{Pipeline: p.Pipeline, Step: builds.StepRunStepBuild, Branch: p.Branch,
Revision: p.Revision, Source: source, Entries: []builds.BuildEntry{{Module: m.Module, Commit: m.Commit, Fingerprint: m.Fingerprint}}}})
asks = append(asks, buildAsk(p, m, build.Ask))
for _, a := range arts {
name := a.Ref
if _, after, ok := strings.Cut(a.Ref, ":"); ok {
@@ -151,7 +157,7 @@ func place(p seat.Pipeline, m seat.Mover, reg catalog.Registration, entries map[
continue
}
s.Seat, s.Outcome, s.Started = bs, seat.StepResultOutcomeRunning, &now
s.Ask = AskID(startOf(p, now), p.Pipeline, m.Module, a.Ref, step)
s.Ask = keptAskID(startOf(p, now), p.Pipeline, m.Module, a.Ref, step, kept)
body := stepAsk(p, m, entry, a, language, step, s.Ask, asker, now)
ask := Asked{Seat: bs, ID: s.Ask, Ask: body, Toolchain: a.Language, Commit: m.Commit}
if share && BoundSeat(bs) {
@@ -196,7 +202,7 @@ func place(p seat.Pipeline, m seat.Mover, reg catalog.Registration, entries map[
// validate is the assigner's, an accepted kind step of the seat mesh-wants, run once per take-in revision: every
// mover's validate step reads the one ask, its id naming no mover (§745, H-A-51).
s.Seat, s.Outcome, s.Started, s.Fingerprint = wants.Seat, seat.StepResultOutcomeRunning, &now, m.Fingerprint
s.Ask = ValidateAskID(p, now)
s.Ask = keptAskID(startOf(p, now), p.Pipeline, "", "", seat.StepResultStepValidate, kept)
asks = append(asks, wantsAsk(p, m, s.Ask, wants.StepRunStepValidate))
validating = true
case seat.StepResultStepComposition:
@@ -204,7 +210,7 @@ func place(p seat.Pipeline, m seat.Mover, reg catalog.Registration, entries map[
// runner's seat as its asker (H-A-88). It is asked once the mover's validate passed, at once when it has none
// (§745).
s.Seat, s.Fingerprint = wants.Seat, m.Fingerprint
s.Ask = AskID(startOf(p, now), p.Pipeline, m.Module, "", step)
s.Ask = keptAskID(startOf(p, now), p.Pipeline, m.Module, "", step, kept)
if validating {
s.Ask, s.Why = "", "asked once validate passed"
break
@@ -309,12 +315,13 @@ func (s *Starter) AskPinHeld(store Updater, key, module string, pins map[string]
return 0, nil
}
var entry catalog.CatalogEntry
unread := ""
switch half, err := readRevision(revision, p.Branch, p.Revision); {
at, kept, whose, unread := moverRevision(store, p, module)
switch half, err := readRevision(revision, p.Branch, at); {
case unread != "":
case err != nil:
unread = fmt.Sprintf("revision %d of %s, the pipeline's, cannot be read for its entry: %v", p.Revision, p.Branch, err)
unread = fmt.Sprintf("revision %d of %s, %s, cannot be read for its entry: %v", at, p.Branch, whose, err)
case half.Entries[module].Fp != m.Fingerprint:
unread = fmt.Sprintf("revision %d of %s holds %s at %q, not the mover's %s", p.Revision, p.Branch, module, half.Entries[module].Fp, m.Fingerprint)
unread = fmt.Sprintf("revision %d of %s holds %s at %q, not the mover's %s", at, p.Branch, module, half.Entries[module].Fp, m.Fingerprint)
default:
entry = half.Entries[module]
}
@@ -332,7 +339,7 @@ func (s *Starter) AskPinHeld(store Updater, key, module string, pins map[string]
if unread != "" || j < 0 || err != nil {
why := unread
if why == "" {
why = fmt.Sprintf("revision %d of %s names no artifact %s of %s to ask it of", p.Revision, p.Branch, st.Artifact, module)
why = fmt.Sprintf("revision %d of %s names no artifact %s of %s to ask it of", at, p.Branch, st.Artifact, module)
}
st.Outcome, st.Ended = seat.StepResultOutcomeCouldNotRun, &now
st.Why = "its build passed with its pins, but " + why + " (H-A-207, H-A-205)"
@@ -342,7 +349,7 @@ func (s *Starter) AskPinHeld(store Updater, key, module string, pins map[string]
}
a := entry.Artifacts[j]
st.Outcome, st.Why, st.Started = seat.StepResultOutcomeRunning, "", &now
st.Ask = AskID(startOf(p, now), p.Pipeline, module, a.Ref, st.Step)
st.Ask = keptAskID(startOf(p, now), p.Pipeline, module, a.Ref, st.Step, kept)
ask := Asked{Seat: st.Seat, ID: st.Ask, Ask: stepAsk(p, m, entry, a, language, st.Step, st.Ask, s.Name, now), Toolchain: a.Language, Commit: m.Commit}
if a.Package != nil {
ask.Package = a.Package.Name
@@ -363,6 +370,57 @@ func (s *Starter) AskPinHeld(store Updater, key, module string, pins map[string]
}
}
// keptRevisions is the state pipelines' kept-revisions (MoverStore, H-A-285): the catalog revision each mover a same-source
// re-pin kept in a pipeline is from (H-A-283, H-A-284 (2)).
type keptRevisions interface {
KeptRevisions(key string) (map[string]int64, error)
UpdateKept(key string, p seat.Pipeline, revision uint64, kept map[string]int64) (uint64, error)
}
// moverRevision is the catalog revision mover module's entry of p is read at (H-A-284 (2)): the one its pipeline kept it at
// when a same-source re-pin kept it, else the pipeline's; whose says which, and unread why it is not read when its record
// cannot be.
// It answers that revision, the kept revision (0 for a mover not kept), whose it is, and why it is not read.
func moverRevision(store Updater, p seat.Pipeline, module string) (int64, int64, string, string) {
kept, err := keptOfStore(store, p.Pipeline)
if err != nil {
return p.Revision, 0, "the pipeline's", fmt.Sprintf("the revision %s was kept at in %s cannot be read: %v", module, p.Pipeline, err)
}
if r := kept[module]; r > 0 {
return r, r, "the one the pipeline kept " + module + " at (H-A-284)", ""
}
return p.Revision, 0, "the pipeline's", ""
}
// keptOfStore is pipeline key's kept-revisions in store (H-A-285); none for a store that keeps none.
func keptOfStore(store any, key string) (map[string]int64, error) {
ks, ok := store.(keptRevisions)
if !ok || key == "" {
return nil, nil
}
kept, err := ks.KeptRevisions(key)
if errors.Is(err, ErrNoPipeline) {
return nil, nil // a pipeline not written yet keeps none
}
return kept, err
}
// keptEntries are the entries of branch's catalog revision rev, the one a same-source re-pin kept a mover at, read by its
// number (H-A-286): a kept mover is placed and pin-checked from them, never from the branch's held view; false while they
// cannot be read, and the mover waits.
func (s *Starter) keptEntries(branch string, rev int64) (map[string]catalog.CatalogEntry, bool) {
if s.Revision == nil {
s.logf("the runner reads no catalog revision, so a mover kept at revision %d of %s waits", rev, branch)
return nil, false
}
half, err := s.Revision(branch, rev)
if err != nil || half.Revision != rev {
s.logf("revision %d of %s, which a mover is kept at, is not read (%v): the mover waits for it (H-A-286)", rev, branch, err)
return nil, false
}
return half.Entries, true
}
func stepAsk(p seat.Pipeline, m seat.Mover, e catalog.CatalogEntry, a catalog.Artifact, language, step, id, asker string, now time.Time) nodebuildgo.StepAsk {
// A step is for its module alone (principle 10, A-447): the fetch always names the module's directory, and the
// artifact's named paths in it when the entry has them.
@@ -440,7 +498,8 @@ func wantsAsk(p seat.Pipeline, m seat.Mover, id, step string) Asked {
// releaseCompositions asks the composition of each mover whose validate ended passed, and ends `not asked` the composition
// of each whose validate did not pass, so a manifest the validator refuses is never composed (design 47 H §745). It answers
// the asks to put and the indexes of the steps it ended.
func releaseCompositions(p *seat.Pipeline, now time.Time) ([]Asked, []int) {
// kept is p's kept-revisions: a kept mover's composition id mixes its kept revision in (H-A-286).
func releaseCompositions(p *seat.Pipeline, now time.Time, kept map[string]int64) ([]Asked, []int) {
var asks []Asked
var ended []int
for _, m := range p.Movers {
@@ -466,7 +525,7 @@ func releaseCompositions(p *seat.Pipeline, now time.Time) ([]Asked, []int) {
ended = append(ended, c)
continue
}
s.Ask = AskID(startOf(*p, now), p.Pipeline, m.Module, "", seat.StepResultStepComposition)
s.Ask = keptAskID(startOf(*p, now), p.Pipeline, m.Module, "", seat.StepResultStepComposition, kept[m.Module])
s.Outcome, s.Started, s.Why = seat.StepResultOutcomeRunning, &now, ""
asks = append(asks, wantsAsk(*p, m, s.Ask, wants.StepRunStepComposition))
}
@@ -0,0 +1,213 @@
package pipeline
import (
"errors"
"fmt"
"slices"
"strings"
"time"
builds "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshbuilds"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
)
// keepSameSource keeps, in the open pipeline of the same source revision that holds it, each mover of the revision half a
// re-pin made (design 47 H H-A-283): a revision of the same source moves no mover out of a running or waiting pipeline (as
// H-A-223 and H-A-244 (1) move one for a revision of a new source); the pipeline keeps the mover and takes its new
// fingerprint, and the revision's own pipeline starts without it. It answers the movers no open pipeline of the source
// holds, which the revision's pipeline carries.
func (s *Starter) keepSameSource(branch string, half Half, movers []seat.Mover) []seat.Mover {
if s.Store == nil || len(movers) == 0 {
return movers
}
pkeys, err := branchKeys(s.Store, branch)
if err != nil {
s.logf("the pipelines of %s are not listed, so no open pipeline of source %d keeps the movers of revision %d: %v", branch, half.Source, half.Revision, err)
return movers
}
left := movers
for _, key := range pkeys {
if len(left) == 0 {
break
}
left = s.keepIn(key, half, left)
}
return left
}
// holdsOpen says whether the pipeline p, not ended and of no run on request, is of revision half's source, older than it, and
// still holds mover module as its own.
func holdsOpen(p seat.Pipeline, half Half, module string) bool {
return (p.Outcome == seat.PipelineOutcomeRunning || p.Outcome == seat.PipelineOutcomeWaiting) && p.Run == "" &&
p.Source == half.Source && p.Revision < half.Revision && Moves(p, module) && !slices.Contains(p.Superseded, module)
}
// keepIn keeps in pipeline key each of movers it holds (H-A-283), answering the movers it does not hold. A kept mover whose
// fingerprint the re-pin moved takes its new fingerprint and commit, and the pipeline keeps the revision it is from in its
// kept-revisions (H-A-285); when its steps were placed, every one of them, open or ended, ends superseded and its steps are
// placed again from the revision's entry, the build first (H-A-284 (1)), each under an ask id its kept revision is mixed
// into (H-A-286), so no ask of the old fingerprint is kept for the new. One not placed yet is placed when it is due, from
// the entries of the revision it is kept at.
func (s *Starter) keepIn(key string, half Half, movers []seat.Mover) []seat.Mover {
for {
p, rev, err := s.Store.Get(key)
if err != nil {
return movers
}
var kept, left []seat.Mover
for _, m := range movers {
if holdsOpen(p, half, m.Module) {
kept = append(kept, m)
} else {
left = append(left, m)
}
}
if len(kept) == 0 {
return movers
}
now := s.now()
why := fmt.Sprintf("superseded: the re-pin revision %d of the pipeline's own source %d moved its mover's fingerprint; its steps are placed again (H-A-283, H-A-284)", half.Revision, half.Source)
revisions := map[string]int64{}
olds := slices.Clone(p.Movers) // the superseded steps are said at the fingerprint they ran for
var moved, unregistered []string
var ended []seat.StepResult
var leave, drop []seat.StepResult
var asks []Asked
var notAsked []int
for _, m := range kept {
i := slices.IndexFunc(p.Movers, func(o seat.Mover) bool { return o.Module == m.Module })
old := p.Movers[i]
if old.Fingerprint == m.Fingerprint && old.Commit == m.Commit {
continue
}
p.Movers[i].Fingerprint, p.Movers[i].Commit = m.Fingerprint, m.Commit
revisions[m.Module] = half.Revision
moved = append(moved, fmt.Sprintf("%s %s to %s", m.Module, old.Fingerprint, m.Fingerprint))
if p.Outcome != seat.PipelineOutcomeRunning || !placed(p, m.Module) {
continue
}
rest := make([]seat.StepResult, 0, len(p.Steps))
for _, st := range p.Steps {
if st.Module != m.Module {
rest = append(rest, st)
continue
}
live := open(st) || st.Outcome == seat.StepResultOutcomeWaiting
l, d := supersedeStep(&p, &st, why, now)
if live {
leave, drop = append(leave, l...), append(drop, d...)
}
ended = append(ended, st)
}
p.Steps = rest
a, n, unreg := s.placeAgain(&p, p.Movers[i], half, now)
asks, notAsked = append(asks, a...), append(notAsked, n...)
if unreg {
unregistered = append(unregistered, m.Module)
}
}
if len(moved) == 0 {
s.logf("%s keeps %d movers of revision %d, of its own source %d, at the fingerprints it holds (H-A-283)", key, len(kept), half.Revision, half.Source)
return left
}
var holderEnded []int
if len(asks) > 0 {
asks, holderEnded = s.holderless(&p, asks, now)
}
ks, keeps := s.Store.(keptRevisions)
if keeps {
_, err = ks.UpdateKept(key, p, rev, revisions)
} else {
_, err = s.Store.Update(key, p, rev)
}
if errors.Is(err, ErrChanged) {
continue
} else if err != nil {
s.logf("%s is not written, so the movers revision %d re-pinned stay its own at their old fingerprints: %v", key, half.Revision, err)
return left
}
if !keeps {
s.logf("%s keeps no kept-revisions, so the pin check of its re-pinned movers reads the pipeline's revision (H-A-285)", key)
}
s.logf("%s keeps the movers revision %d of its own source %d re-pinned, %s: %d steps ended superseded, %d asked again (H-A-283, H-A-284)",
key, half.Revision, half.Source, strings.Join(moved, ", "), len(ended), len(asks))
for _, st := range ended {
if s.Say == nil {
break
}
was := p
was.Movers = olds
if err := s.Say(stepEvent(seat.EventStepEnded, was, st, now)); err != nil {
s.logf("step-ended of %s's %s %s, superseded, is not said: %v", key, st.Module, st.Step, err)
}
}
if s.Left != nil {
s.Left(key, leave, drop, why)
}
for _, m := range unregistered {
s.logf("%s: %s has no registration yet: its steps wait on its registration, which places them (A-327)", key, m)
s.registrationMissing(m, key)
}
s.sayNotAsked(p, notAsked)
if len(holderEnded) > 0 {
if err := s.endedAtPlacing(p, holderEnded); err != nil {
s.logf("%v", err)
}
return left
}
if s.Steps != nil && len(asks) > 0 {
if err := s.askSteps(key, asks); err != nil {
s.logf("%s: the steps placed again are not all put, they are asked at the next start: %v", key, err)
}
}
return left
}
}
// placeAgain places the steps of kept mover m of p from the re-pin revision half's entry (H-A-284 (1)), as a mover is placed
// (placeMovers), under ask ids its kept revision is mixed into (H-A-286): it answers the asks to put, the steps placed not asked because a
// dependency did not build, and whether m has no registration. A mover not due yet (it waits on a publish) is left
// unplaced, placed when the publish is heard.
func (s *Starter) placeAgain(p *seat.Pipeline, m seat.Mover, half Half, now time.Time) ([]Asked, []int, bool) {
if s.Registration == nil {
return nil, nil, false
}
r, ok := s.Registration(m.Module)
if !ok {
return nil, nil, true
}
due, waiting, failed := ready(*p, m.Module, half.Entries)
if !due && failed == "" {
s.logf("%s: %s waits for %s before its build", p.Pipeline, m.Module, strings.Join(waiting, ", "))
return nil, nil, false
}
steps, asks := place(*p, m, r, half.Entries, now, s.Name, true, half.Revision)
supersededWhy(*p, m.Module, half.Entries, steps)
var notAsked []int
if failed != "" {
why := "its dependency " + failed + " did not build"
for k := range steps {
st := &steps[k]
if st.Outcome == seat.StepResultOutcomeBypassed {
continue
}
st.Outcome, st.Why, st.Ask, st.Shared, st.Ended = seat.StepResultOutcomeNotAsked, why, "", nil, &now
p.Unlogged = append(p.Unlogged, endEntry(p.Pipeline, *st, now))
notAsked = append(notAsked, len(p.Steps)+k)
}
asks = nil
}
p.Steps = append(p.Steps, steps...)
return asks, notAsked, false
}
// buildAsk is mover m's build step ask under id, of the seat mesh-builds for the mover's own entry only (A-99, A-97): a run on
// request's names no source (H-A-162).
func buildAsk(p seat.Pipeline, m seat.Mover, id string) Asked {
source := p.Source
if p.Run != "" {
source = 0
}
return Asked{Seat: builds.Seat, ID: id, Commit: m.Commit, Ask: builds.StepRun{Pipeline: p.Pipeline, Step: builds.StepRunStepBuild, Branch: p.Branch,
Revision: p.Revision, Source: source, Entries: []builds.BuildEntry{{Module: m.Module, Commit: m.Commit, Fingerprint: m.Fingerprint}}}}
}
@@ -0,0 +1,191 @@
package pipeline
import (
"fmt"
"slices"
"strings"
"testing"
"time"
builds "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshbuilds"
catalog "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
nodebuildgo "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildgo"
"git.novox.be/novox/mesh-sdk/go/keys"
)
// ticking is a clock that moves a second at every read, so a start and a later re-pin never share a millisecond.
func ticking() func() time.Time {
at := time.Date(2026, 11, 21, 17, 0, 0, 0, time.UTC)
return func() time.Time { at = at.Add(time.Second); return at }
}
// repinFixture is a pipeline main.10 of source 41 moving app at src2:aa, started, with the re-pin revision 11 of the same
// source moving app to src2:ab (an entry naming a package with no version yet), and a revision 12 of a new source 42.
func repinFixture() (catalog.Registration, map[string]catalog.CatalogEntry, map[string]catalog.CatalogEntry, func(string, int64) (Half, error)) {
reg, entries := placeFixture()
repinned := map[string]catalog.CatalogEntry{}
for k, e := range entries {
repinned[k] = e
}
app := entries["app"]
app.Fp = "src2:aa"
entries["app"] = app
re := app
re.Fp, re.Packages = "src2:ab", map[string]string{"git.novox.be/novox/mesh-sdk/go": "v0.1.26", "git.novox.be/novox/x/go": ""}
repinned["app"] = re
read := func(branch string, revision int64) (Half, error) {
switch revision {
case 10:
return Half{Revision: 10, Source: 41, Entries: entries}, nil
case 11:
return Half{Revision: 11, Source: 41, Entries: repinned, Call: "release of mesh-sdk 0.1.26",
Movers: []catalog.TakeInEntry{{Module: "app", Commit: "c1", Fingerprint: "src2:ab", Repository: "o/app"}}}, nil
case 12:
return Half{Revision: 12, Source: 42, Entries: repinned, Call: "take-in of 42",
Movers: []catalog.TakeInEntry{{Module: "app", Commit: "c2", Fingerprint: "src2:ac", Repository: "o/app"}}}, nil
}
return Half{}, fmt.Errorf("no revision %d", revision)
}
return reg, entries, repinned, read
}
func revisionEvent(n int64) catalog.CatalogRevisionEvent {
return catalog.CatalogRevisionEvent{Kind: "trunk", Branch: keys.EscapeName("main"), Revision: n, CallKind: "release"}
}
// H-A-283, H-A-284, H-A-285, H-A-286: a re-pin revision of the same source moves no mover out of the running pipeline of that source:
// the pipeline keeps it at its new fingerprint, every step of it ends superseded (said at the old fingerprint, its asks
// dropped) and is placed again from the re-pin revision's entry under new asks, the build first, still naming the
// pipeline's revision; the revision the mover is kept at rides in the record's kept-revisions, where the pin check reads
// its entry; the re-pin starts no pipeline. A revision of a new source still starts its own and moves the mover out.
func TestASameSourceRepinKeepsTheMoverInItsPipeline(t *testing.T) {
reg, entries, _, read := repinFixture()
store := &MoverStore{Raw: newMemRaw()}
asks := &stepAsks{}
var said []seat.StepEvent
var left [][]seat.StepResult
s := &Starter{Pipelines: store, Store: store, Steps: asks, Name: "the-runner", Now: ticking(), Revision: read,
Registration: func(m string) (catalog.Registration, bool) { return reg, m == "app" },
Say: func(_ string, body any) error {
if ev, ok := body.(seat.StepEvent); ok && ev.Result.Outcome == seat.StepResultOutcomeSuperseded {
said = append(said, ev)
}
return nil
},
Left: func(_ string, leave, drop []seat.StepResult, _ string) { left = append(left, append(leave, drop...)) }}
h := &Hearer{Store: store, Say: func(string, any) error { return nil }}
s.Supersede = func(newer seat.Pipeline) { h.SupersedeRunning(newer) }
key := Key("main", 10)
if ok, err := s.Start(Begin{Branch: "main", Source: 41, Revision: 10, Entries: entries,
Movers: []seat.Mover{{Module: "app", Commit: "c1", Fingerprint: "src2:aa", Repository: "o/app"}}}); !ok || err != nil {
t.Fatalf("start = %v, %v", ok, err)
}
before, _, _ := store.Get(key)
oldAsks := map[string]bool{}
for _, st := range before.Steps {
if st.Ask != "" {
oldAsks[st.Ask] = true
}
}
asks.at, asks.ids, asks.asks = nil, nil, nil
ev := revisionEvent
if n, err := s.Revised(ev(11), read); n != 0 || err != nil {
t.Fatalf("the same-source re-pin started %d pipelines (%v), want none", n, err)
}
if _, _, err := store.Get(Key("main", 11)); err == nil {
t.Fatal("the same-source re-pin wrote a pipeline of its own")
}
p, _, err := store.Get(key)
if err != nil {
t.Fatal(err)
}
if m, _ := MoverOf(p, "app"); m.Fingerprint != "src2:ab" || slices.Contains(p.Superseded, "app") || p.Outcome != seat.PipelineOutcomeRunning {
t.Fatalf("the kept mover = %+v, superseded %v, outcome %s: want it kept at src2:ab, the pipeline running", m, p.Superseded, p.Outcome)
}
if len(p.Steps) != len(before.Steps) || p.Steps[0].Step != seat.StepResultStepBuild || p.Steps[0].Outcome != seat.StepResultOutcomeRunning ||
p.Steps[0].Fingerprint != "src2:ab" || oldAsks[p.Steps[0].Ask] {
t.Fatalf("the steps placed again = %+v, want the build first, running at src2:ab under a new ask", p.Steps)
}
for _, st := range p.Steps {
if st.Outcome == seat.StepResultOutcomeSuperseded || (st.Ask != "" && oldAsks[st.Ask]) {
t.Errorf("the step %s %s stands for the old fingerprint: %+v", st.Artifact, st.Step, st)
}
}
// Every old step ended superseded, said at the fingerprint it ran for; the ones open on a build node are dropped.
if len(said) != len(before.Steps) || said[0].Fingerprint != "src2:aa" || len(left) != 1 || len(left[0]) == 0 {
t.Errorf("said %d superseded steps (of %d), the first at %+v; left %v", len(said), len(before.Steps), said, left)
}
b, ok := asks.asks[0].(builds.StepRun)
if !ok || asks.ids[0] != p.Steps[0].Ask || b.Revision != 10 || b.Source != 41 || len(b.Entries) != 1 || b.Entries[0].Fingerprint != "src2:ab" {
t.Fatalf("the build asked again = %+v (%s), want the new fingerprint at the pipeline's revision", asks.asks[0], asks.ids[0])
}
if kept, err := store.KeptRevisions(key); err != nil || kept["app"] != 11 {
t.Errorf("kept-revisions = %v (%v), want app at 11", kept, err)
}
// The pin check reads the kept mover's entry at the revision it was kept at, never the pipeline's.
if n, err := s.AskPinHeld(store, key, "app", map[string]string{"git.novox.be/novox/mesh-sdk/go": "v0.1.26", "git.novox.be/novox/x/go": "v0.1.1"}, read); n == 0 || err != nil {
after, _, _ := store.Get(key)
t.Fatalf("the held steps asked %d (%v): %+v", n, err, after.Steps)
}
last := asks.asks[len(asks.asks)-1].(nodebuildgo.StepAsk)
if last.Step.Fingerprint != "src2:ab" || last.Step.Fetch.Packages["git.novox.be/novox/x/go"] != "v0.1.1" {
t.Errorf("a held step asked = %+v", last.Step)
}
// A revision of a new source starts its own pipeline and moves the mover out (H-A-223).
if n, err := s.Revised(ev(12), read); n != 1 || err != nil {
t.Fatalf("a new source's revision started %d (%v)", n, err)
}
if p, _, _ := store.Get(key); !slices.Contains(p.Superseded, "app") {
t.Errorf("the new source's revision left app in %s: %v", key, p.Superseded)
}
}
// H-A-286: a kept mover not placed at its keep (its registration arrived later) is placed from the revision it is kept at
// while the branch's held view still holds the older revision, under ask ids its kept revision is mixed into; and a restart
// (Resume) finds its running steps by those ids and asks them again.
func TestAKeptMoverIsPlacedLateAndResumedAtItsKeptRevision(t *testing.T) {
reg, entries, _, read := repinFixture()
store := &MoverStore{Raw: newMemRaw()}
asks := &stepAsks{}
registered := false
s := &Starter{Pipelines: store, Store: store, Steps: asks, Name: "the-runner", Now: ticking(), Revision: read,
Say: func(string, any) error { return nil },
Registration: func(m string) (catalog.Registration, bool) { return reg, registered && m == "app" }}
key := Key("main", 10)
if ok, err := s.Start(Begin{Branch: "main", Source: 41, Revision: 10, Entries: entries,
Movers: []seat.Mover{{Module: "app", Commit: "c1", Fingerprint: "src2:aa", Repository: "o/app"}}}); !ok || err != nil {
t.Fatalf("start = %v, %v", ok, err)
}
if n, err := s.Revised(revisionEvent(11), read); n != 0 || err != nil {
t.Fatalf("the re-pin started %d (%v)", n, err)
}
// The registration arrives while the runner's view still holds revision 10.
registered = true
lagging := func(string) (Half, bool) { return Half{Revision: 10, Source: 41, Entries: entries}, true }
if n, err := s.Registered(store, reg, lagging); n != 1 || err != nil {
t.Fatalf("the late registration placed in %d pipelines (%v)", n, err)
}
p, _, err := store.Get(key)
if err != nil {
t.Fatal(err)
}
var build seat.StepResult
held := 0
for _, st := range p.Steps {
switch {
case st.Step == seat.StepResultStepBuild:
build = st
case pinHeld(st) && strings.HasSuffix(st.Why, "git.novox.be/novox/x/go"):
held++
}
}
if want := keptAskID(*p.Started, key, "app", "", seat.StepResultStepBuild, 11); build.Ask != want || build.Fingerprint != "src2:ab" || held == 0 {
t.Fatalf("the late placing: build %+v (want ask %s), %d steps held on revision 11's unpinned package: %+v", build, want, held, p.Steps)
}
// A restart asks the kept mover's running steps again under the same ids.
asks.at, asks.ids, asks.asks = nil, nil, nil
if n, err := s.Resume(store, "main", Half{Revision: 10, Source: 41, Entries: entries}); n == 0 || err != nil || !slices.Contains(asks.ids, build.Ask) {
t.Fatalf("Resume asked %d (%v), ids %v: want the kept build %s again", n, err, asks.ids, build.Ask)
}
}
@@ -165,6 +165,13 @@ type Starter struct {
// Supersede takes out of every older running pipeline of newer's branch each mover whose fingerprint newer moved
// (H-A-223; Hearer.SupersedeRunning); nil leaves running pipelines as they are.
Supersede func(newer seat.Pipeline)
// Left takes the steps a same-source re-pin superseded in a pipeline that kept their mover out of the test runs they
// read and drops their own asks (H-A-283, H-A-284 (1); Hearer.LeftInPlace); nil leaves them.
Left func(key string, leave, drop []seat.StepResult, why string)
// Revision reads a catalog revision of a branch by its number through the catalog seat's revision verb: a mover a
// same-source re-pin kept is placed from the entries of the revision it is kept at (H-A-286); nil leaves such a mover
// waiting.
Revision func(branch string, revision int64) (Half, error)
// Fills refuses a start while a store of the runner is full (§366); nil refuses none.
Fills *Fills
// Recovered says whether a commit is named a recovery (§81, §86): its steps are asked on the build seats' stream delivery.
@@ -334,6 +341,11 @@ func (s *Starter) placeMovers(p *seat.Pipeline, reg func(module string) (catalog
var unregistered []string
var notAsked []int
var waits []seat.Mover
kept, err := keptOfStore(s.Store, p.Pipeline)
if err != nil {
s.logf("the kept-revisions of %s are not read, so its movers are placed as movers not kept: %v", p.Pipeline, err)
}
all := entries
for _, m := range p.Movers {
if placed(*p, m.Module) {
continue
@@ -343,6 +355,13 @@ func (s *Starter) placeMovers(p *seat.Pipeline, reg func(module string) (catalog
unregistered = append(unregistered, m.Module)
continue
}
// A mover a same-source re-pin kept is placed from the revision it is kept at, its ids its own (H-A-286).
entries := all
if rev := kept[m.Module]; rev > 0 {
if entries, ok = s.keptEntries(p.Branch, rev); !ok {
continue
}
}
// A mover's steps wait for the publish of each mover it builds on (§746, H-A-83 (2)): placed once each is heard,
// ended not asked when one's build ended without passing.
due, waiting, failed := ready(*p, m.Module, entries)
@@ -351,7 +370,7 @@ func (s *Starter) placeMovers(p *seat.Pipeline, reg func(module string) (catalog
waits = append(waits, m)
continue
}
steps, a := Place(*p, m, r, entries, now, s.Name)
steps, a := place(*p, m, r, entries, now, s.Name, true, kept[m.Module])
supersededWhy(*p, m.Module, entries, steps)
for _, st := range steps {
if pinHeld(st) {
@@ -390,13 +409,13 @@ func (s *Starter) placeMovers(p *seat.Pipeline, reg func(module string) (catalog
}
}
}
if steps, ended := cycled(p, waits, unregistered, reg, entries, now, s.Name); ended {
if steps, ended := cycled(p, waits, unregistered, reg, all, now, s.Name); ended {
return nil, unregistered, steps, notAsked
} else {
notAsked = append(notAsked, steps...)
}
// A mover placed after the validate ended has its composition asked, or not asked, at once (§745).
more, _ := releaseCompositions(p, now)
more, _ := releaseCompositions(p, now, kept)
asks = append(asks, more...)
asks, ended := s.holderless(p, asks, now)
return asks, unregistered, ended, notAsked
@@ -611,13 +630,24 @@ func (s *Starter) Resume(store Updater, branch string, half Half) (int, error) {
}
}
var due []Asked
kept, err := keptOfStore(store, key)
if err != nil {
s.logf("the kept-revisions of %s are not read, so its kept movers' steps are not asked again: %v", key, err)
}
for _, m := range p.Movers {
reg, ok := s.Registration(m.Module)
if !ok {
continue
}
_, shared := Place(p, m, reg, half.Entries, now, s.Name)
_, unshared := place(p, m, reg, half.Entries, now, s.Name, false)
// A kept mover's steps are found by the ids its kept revision gives, from that revision's entries (H-A-286).
entries, rev := half.Entries, kept[m.Module]
if rev > 0 {
if entries, ok = s.keptEntries(p.Branch, rev); !ok {
continue
}
}
_, shared := place(p, m, reg, entries, now, s.Name, true, rev)
_, unshared := place(p, m, reg, entries, now, s.Name, false, rev)
for _, a := range shared {
if running[a.ID] && !own[a.ID] {
due = append(due, a)
@@ -1023,6 +1053,12 @@ func (s *Starter) Revised(ev catalog.CatalogRevisionEvent, read func(branch stri
for _, e := range half.Movers {
movers = append(movers, seat.Mover{Module: e.Module, Commit: e.Commit, Fingerprint: e.Fingerprint, Repository: e.Repository})
}
// A revision of the same source moves no mover out of the open pipeline of that source holding it: that pipeline keeps
// it, and this revision starts no pipeline of its own for it (H-A-283); only a revision of a new source moves movers.
if movers = s.keepSameSource(branch, half, movers); len(movers) == 0 {
s.logf("the revision %d of %s (%s) moves only movers the open pipelines of its source %d keep: it starts no pipeline (H-A-283)", ev.Revision, branch, ev.Call, half.Source)
return 0, nil
}
movers = Tiers(movers, half.Entries, s.logf)
call := half.Call
if call == "" {
@@ -88,6 +88,9 @@ func TestARevisionOfAReleaseStartsOnePipeline(t *testing.T) {
return Half{}, fmt.Errorf("branch %q", branch)
}
r := Half{Revision: revision, Source: 41, Entries: entries, Call: "release of mesh-sdk 0.1.17"}
if revision > 10 {
r.Source = 41 + revision // a later source's revision: one of the same source keeps its movers in 10's (H-A-283)
}
if revision != 12 {
r.Movers = movers
}
@@ -154,8 +154,9 @@ type Hearer struct {
Ager *Ager
// Fills says the stores' fill at each step's end (§366); nil says none.
Fills *Fills
// Widens computes a mover's widens, kept in its value at the pipeline's end (§71, A-207; H-A-51); nil keeps none.
Widens func(p seat.Pipeline, module string) []seat.Widening
// Widens makes, for one end, the reader of each mover's widens, kept in its value at the pipeline's end (§71, A-207;
// H-A-51): made before the end's compare-and-set, its reads shared by every mover and every retry (H-A-289); nil keeps none.
Widens func() func(p seat.Pipeline, module string) []seat.Widening
// TestRuns is the state test-runs and Clock its bound keys (§725); Runs the settings of its runs; DropRun drops a
// run's ask when its run ends unfinished (§119). A nil TestRuns keeps no test run.
TestRuns TestRuns
@@ -199,6 +200,15 @@ func (h *Hearer) EndReaders(run string, readers []seat.TestRunReader, r seat.Tes
return first
}
// kept is pipeline key's kept-revisions (H-A-285, H-A-286); none when they cannot be read, said.
func (h *Hearer) kept(key string) map[string]int64 {
kept, err := keptOfStore(h.Store, key)
if err != nil {
h.logf("the kept-revisions of %s are not read, so its kept movers' asks take the ids of movers not kept: %v", key, err)
}
return kept
}
func (h *Hearer) now() time.Time {
if h.Now != nil {
return h.Now().UTC()
@@ -236,6 +246,7 @@ func (h *Hearer) OnStepEndedPinned(bs StepSeat, subject string, ev StepEnded, sh
if ev.Asker != "" && ev.Asker != h.Name {
return seat.Pipeline{}, nil
}
wr := h.widensReads() // made before the compare-and-set, kept for its retries (H-A-289)
for {
p, rev, err := h.Store.Get(ev.Pipeline)
if err != nil {
@@ -314,7 +325,7 @@ func (h *Hearer) OnStepEndedPinned(bs StepSeat, subject string, ev StepEnded, sh
if s.Step == seat.StepResultStepValidate {
readers = append(readers, sameValidate(&p, i, now)...)
var notAsked []int
due, notAsked = releaseCompositions(&p, now)
due, notAsked = releaseCompositions(&p, now, h.kept(ev.Pipeline))
readers = append(readers, notAsked...)
}
from := now
@@ -328,7 +339,7 @@ func (h *Hearer) OnStepEndedPinned(bs StepSeat, subject string, ev StepEnded, sh
// A mover's build passed is what a kind-3 edge waits for (§746, H-A-83 (2)); one that did not pass ends its
// dependents' steps not asked.
release := s.Step == seat.StepResultStepBuild && (!stepPassed(*s) || built(&p, s.Module))
ended := h.endPipeline(&p, now)
ended := h.endPipeline(&p, now, wr)
if _, err := h.Store.Update(ev.Pipeline, p, rev); errors.Is(err, ErrChanged) {
continue
} else if err != nil {
@@ -468,6 +479,12 @@ func (h *Hearer) said(p seat.Pipeline, asked []int, ended bool) error {
h.Turn(p.Pipeline)
}
if ended {
// The journal says which end it was (H-A-288).
if p.Why != "" {
h.logf("%s ended %s: %s", p.Pipeline, p.Outcome, p.Why)
} else {
h.logf("%s ended %s", p.Pipeline, p.Outcome)
}
h.Failures.Ended(p)
if h.RegistrationWaits != nil {
h.RegistrationWaits()
@@ -484,6 +501,51 @@ func (h *Hearer) said(p seat.Pipeline, asked []int, ended bool) error {
return nil
}
// EndAtStart judges, at the runner's start pass, the end of every running pipeline nothing of which is still open (H-A-288
// (c)): one whose last mover left or whose last open step ended while no runner wrote its end ends as at a step's end,
// superseded, failed or passed by the end judgement, said, and the next waiting placed. It answers how many it ended.
func (h *Hearer) EndAtStart() (int, error) {
pkeys, err := allKeys(h.Store)
if err != nil {
return 0, fmt.Errorf("the pipelines are not listed: %w", err)
}
n := 0
var first error
for _, key := range pkeys {
ended, err := h.endIfNothingOpen(key)
if err != nil && first == nil {
first = err
}
if ended {
n++
}
}
return n, first
}
// endIfNothingOpen ends the running pipeline key when nothing of it is still open (H-A-288), its reads made before the
// compare-and-set (H-A-289); true when it ended.
func (h *Hearer) endIfNothingOpen(key string) (bool, error) {
wr := h.widensReads()
for {
p, rev, err := h.Store.Get(key)
if err != nil || p.Outcome != seat.PipelineOutcomeRunning {
return false, nil
}
now := h.now()
if !h.endPipeline(&p, now, wr) {
return false, nil
}
if _, err := h.Store.Update(key, p, rev); errors.Is(err, ErrChanged) {
continue
} else if err != nil {
return false, fmt.Errorf("pipeline %s cannot be written: %w", key, err)
}
h.logf("%s had nothing open at the runner's start: its end is written now (H-A-288)", key)
return true, h.said(p, nil, true)
}
}
// endEntry is the step log's entry of a step's end, with the keys of the kept results it wrote or carried.
func endEntry(key string, s seat.StepResult, at time.Time, kept ...string) seat.LogEntry {
return seat.LogEntry{Pipeline: key, Module: s.Module, Step: s.Step, Artifact: s.Artifact, What: seat.LogEntryWhatEnded, Outcome: s.Outcome,
@@ -720,19 +782,28 @@ func endPipeline(p *seat.Pipeline, now time.Time, unplaced func(seat.Pipeline, s
return true
}
// endPipeline is endPipeline with each mover's widens computed into its value at the end (§71, A-207; H-A-51).
func (h *Hearer) endPipeline(p *seat.Pipeline, now time.Time) bool {
// endPipeline is endPipeline with each mover's widens computed into its value at the end (§71, A-207; H-A-51), by wr, the
// reads the caller made before its compare-and-set (H-A-289); nil computes none.
func (h *Hearer) endPipeline(p *seat.Pipeline, now time.Time, wr func(seat.Pipeline, string) []seat.Widening) bool {
if !endPipeline(p, now, h.Unplaced) {
return false
}
if h.Widens != nil {
if wr != nil {
for i := range p.Movers {
p.Movers[i].Widens = h.Widens(*p, p.Movers[i].Module)
p.Movers[i].Widens = wr(*p, p.Movers[i].Module)
}
}
return true
}
// widensReads is the reader of one end's widens, made before its compare-and-set and kept for every retry (H-A-289).
func (h *Hearer) widensReads() func(seat.Pipeline, string) []seat.Widening {
if h.Widens == nil {
return nil
}
return h.Widens()
}
// EndedAtPlacing says a pipeline its starter ended at placing, a seat with no holder (A-470g, A-470h): the step-ended of each
// step of steps, its step log entries appended, the head's finding pipeline-failed (§33), then its pipeline-ended.
func (h *Hearer) EndedAtPlacing(p seat.Pipeline, steps []int) error {
@@ -91,6 +91,7 @@ func (h *Hearer) SupersedeRunning(newer seat.Pipeline) []string {
}
func (h *Hearer) supersedeRunningOne(key string, newer seat.Pipeline) bool {
wr := h.widensReads() // made before the compare-and-set, kept for its retries (H-A-289)
for {
p, rev, err := h.Store.Get(key)
if err != nil || p.Outcome != seat.PipelineOutcomeRunning || p.Branch != newer.Branch || p.Run != "" || newer.Run != "" ||
@@ -118,18 +119,12 @@ func (h *Hearer) supersedeRunningOne(key string, newer seat.Pipeline) bool {
if !slices.Contains(moved, st.Module) || (!open(*st) && st.Outcome != seat.StepResultOutcomeWaiting) {
continue
}
switch {
case sharedRun(*st) != "" && st.Seat != wants.Seat:
leave = append(leave, *st)
case st.Ask != "" && BoundSeat(st.Seat):
drop = append(drop, *st) // its own ask on a build node: the seat's drop cancels it
}
st.Outcome, st.Why, st.Ended = seat.StepResultOutcomeSuperseded, why, &now
p.Unlogged = append(p.Unlogged, endEntry(key, *st, now))
l, d := supersedeStep(&p, st, why, now)
leave, drop = append(leave, l...), append(drop, d...)
ended = append(ended, i)
}
p.Superseded = append(p.Superseded, moved...)
done := h.endPipeline(&p, now)
done := h.endPipeline(&p, now, wr)
if _, err := h.Store.Update(key, p, rev); errors.Is(err, ErrChanged) {
continue
} else if err != nil {
@@ -142,20 +137,7 @@ func (h *Hearer) supersedeRunningOne(key string, newer seat.Pipeline) bool {
h.logf("step-ended of %s's %s %s, superseded, is not said: %v", key, p.Steps[i].Module, p.Steps[i].Step, err)
}
}
for _, st := range leave {
h.leaveRun(key, st, "the mover "+st.Module+" left "+key+" for the newer "+newer.Pipeline)
}
// Its running asks on the build nodes are cancelled through the seat's drop (H-A-223). The seat mesh-builds serves no
// drop, so a build ask there runs to its end, which the runner ignores: the step is no longer open.
for _, st := range drop {
if h.DropRun == nil {
h.logf("%s: the ask %s of %s's %s is not dropped: the runner has no drop", key, st.Ask, st.Module, st.Step)
continue
}
if err := h.DropRun(st.Seat, st.Node, st.Ask, why); err != nil {
h.logf("%s: the drop of the ask %s of %s's %s is not asked of %s: %v", key, st.Ask, st.Module, st.Step, st.Seat, err)
}
}
h.leaveSteps(key, leave, drop, "the mover %s left "+key+" for the newer "+newer.Pipeline, why)
if err := h.said(p, nil, done); err != nil {
h.logf("%v", err)
}
@@ -164,6 +146,45 @@ func (h *Hearer) supersedeRunningOne(key string, newer seat.Pipeline) bool {
}
}
// supersedeStep ends the open or waiting step st of p superseded, naming why, with its step log entry: a step reading a test
// run is answered to leave it, and one with its own ask on a build node to drop it (H-A-223).
func supersedeStep(p *seat.Pipeline, st *seat.StepResult, why string, now time.Time) (leave, drop []seat.StepResult) {
switch {
case sharedRun(*st) != "" && st.Seat != wants.Seat:
leave = append(leave, *st)
case st.Ask != "" && BoundSeat(st.Seat):
drop = append(drop, *st) // its own ask on a build node: the seat's drop cancels it
}
st.Outcome, st.Why, st.Ended = seat.StepResultOutcomeSuperseded, why, &now
p.Unlogged = append(p.Unlogged, endEntry(p.Pipeline, *st, now))
return leave, drop
}
// leaveSteps takes the superseded steps of pipeline key out of the test runs they read (leaving, its %s the step's mover)
// and drops their own asks on the build nodes, naming why (H-A-223). The seat mesh-builds serves no drop, so a build ask
// there runs to its end, which the runner ignores: the step is no longer open.
func (h *Hearer) leaveSteps(key string, leave, drop []seat.StepResult, leaving, why string) {
for _, st := range leave {
h.leaveRun(key, st, fmt.Sprintf(leaving, st.Module))
}
for _, st := range drop {
if h.DropRun == nil {
h.logf("%s: the ask %s of %s's %s is not dropped: the runner has no drop", key, st.Ask, st.Module, st.Step)
continue
}
if err := h.DropRun(st.Seat, st.Node, st.Ask, why); err != nil {
h.logf("%s: the drop of the ask %s of %s's %s is not asked of %s: %v", key, st.Ask, st.Module, st.Step, st.Seat, err)
}
}
}
// LeftInPlace takes the steps a same-source re-pin superseded in pipeline key, whose mover it kept (H-A-283, H-A-284 (1)),
// out of the test runs they read and drops their own asks, naming why, and appends their step log entries.
func (h *Hearer) LeftInPlace(key string, leave, drop []seat.StepResult, why string) {
h.leaveSteps(key, leave, drop, "the mover %s of "+key+" was re-pinned and its steps placed again (H-A-284)", why)
h.appendUnlogged(key)
}
// supersedeWaiting takes, in each older waiting pipeline of newer's branch, the movers newer also moves out of it (design 47 H
// §748, ADR 0682 §7): a waiting pipeline left with no mover ends superseded, its waiting key deleted, saying wait-ended and its
// pipeline-ended. newer is a pipeline just written, waiting or placed: a mover a newer revision moves leaves every pipeline not
@@ -52,6 +52,7 @@ func (v Verbs) record(verb, what, why string) error {
func (v Verbs) change(key, event string, record func() error, mutate func(p *seat.Pipeline, now time.Time) (int, error)) (seat.Pipeline, int, error) {
h := v.Hearer
recorded := false
wr := h.widensReads() // made before the compare-and-set, kept for its retries (H-A-289)
for {
p, rev, err := h.Store.Get(key)
if err != nil {
@@ -73,7 +74,7 @@ func (v Verbs) change(key, event string, record func() error, mutate func(p *sea
if err != nil {
return p, i, err
}
ended := h.endPipeline(&p, now)
ended := h.endPipeline(&p, now, wr)
if i >= 0 && event == seat.EventStepEnded {
p.Unlogged = append(p.Unlogged, endEntry(key, p.Steps[i], now))
}
@@ -530,9 +531,11 @@ func (v Verbs) passedReader(reader RunReader, run string) bool {
// under one id is kept once), saying it is a rerun and by whom. The validate ask is the pipeline's one, naming no mover
// (H-A-51): its rerun asks that one ask again; a composition's rerun builds the mover's composition ask (H-A-236).
func (v Verbs) askOf(p seat.Pipeline, s seat.StepResult, caller string, now time.Time) (*Asked, error) {
id := AskID(now, p.Pipeline, s.Module, s.Artifact, s.Step)
// A kept mover's rerun mixes its kept revision into the id (H-A-286).
rev := v.keptOf(p.Pipeline)[s.Module]
id := keptAskID(now, p.Pipeline, s.Module, s.Artifact, s.Step, rev)
if s.Seat == wants.Seat && s.Step == seat.StepResultStepValidate {
id = AskID(now, p.Pipeline, "", "", seat.StepResultStepValidate)
id = keptAskID(now, p.Pipeline, "", "", seat.StepResultStepValidate, rev)
}
a, err := v.placedAsk(p, s, id, now)
if err != nil {
@@ -580,8 +583,15 @@ func (v Verbs) placedAsk(p seat.Pipeline, s seat.StepResult, id string, now time
} else {
return nil, fmt.Errorf("the catalog half of %s is not held yet: the step is asked when it is", p.Branch)
}
_, placed := place(p, m, reg, half.Entries, now, v.Starter.Name, false)
base := AskID(startOf(p, now), p.Pipeline, m.Module, s.Artifact, s.Step)
// A kept mover's step is the one placed from the revision it is kept at, under its kept ids (H-A-286).
entries, rev := half.Entries, v.keptOf(p.Pipeline)[m.Module]
if rev > 0 {
if entries, ok = v.Starter.keptEntries(p.Branch, rev); !ok {
return nil, fmt.Errorf("revision %d of %s, which %s is kept at, is not read: the step is asked when it is", rev, p.Branch, m.Module)
}
}
_, placed := place(p, m, reg, entries, now, v.Starter.Name, false, rev)
base := keptAskID(startOf(p, now), p.Pipeline, m.Module, s.Artifact, s.Step, rev)
for _, a := range placed {
if a.ID != base {
continue
@@ -596,6 +606,14 @@ func (v Verbs) placedAsk(p seat.Pipeline, s seat.StepResult, id string, now time
return nil, fmt.Errorf("the %s step of %s is placed no ask now", s.Step, p.Pipeline)
}
// keptOf is pipeline key's kept-revisions (H-A-285); none when they cannot be read, said.
func (v Verbs) keptOf(key string) map[string]int64 {
if v.Hearer == nil {
return nil
}
return v.Hearer.kept(key)
}
// Run starts a run on request for the caller the bus proved (design 47 G §64; Starter.Run): its pipelines written, its
// run-started said, its steps placed when its candidate's revision is heard.
func (v Verbs) Run(caller string, in seat.RunInput) (seat.RunAnswer, error) {
@@ -31,20 +31,87 @@ func (w *Widener) Widens(p seat.Pipeline, module string) []seat.Widening {
if w == nil || w.Revision == nil {
return nil
}
return w.Reads()(p, module)
}
// Reads is Widens over reads made once and kept (H-A-289): the two revisions, each serving seat's holder and the grants part's
// grants-for are each read at their first need and shared by every mover of an end and every retry of its write, so a retry
// rereads only the pipeline's key. One end takes one Reads, made before its compare-and-set.
func (w *Widener) Reads() func(p seat.Pipeline, module string) []seat.Widening {
if w == nil || w.Revision == nil {
return func(seat.Pipeline, string) []seat.Widening { return nil }
}
r := &endReads{w: w, revisions: map[string]revisionRead{}, holders: map[string]holderRead{}, grants: map[string]grantsRead{}}
return r.widens
}
type revisionRead struct {
half Half
err error
}
type holderRead struct {
holder string
err error
}
type grantsRead struct {
g meshgrants.GrantsFor
err error
}
// endReads are the reads of one end, each made at its first need and kept.
type endReads struct {
w *Widener
revisions map[string]revisionRead
holders map[string]holderRead
grants map[string]grantsRead
}
func (r *endReads) revision(branch string, rev int64) (Half, error) {
k := fmt.Sprintf("%s.%d", branch, rev)
got, ok := r.revisions[k]
if !ok {
got.half, got.err = r.w.Revision(branch, rev)
r.revisions[k] = got
}
return got.half, got.err
}
func (r *endReads) holder(s string) (string, error) {
got, ok := r.holders[s]
if !ok {
got.holder, got.err = r.w.Holder(s)
r.holders[s] = got
}
return got.holder, got.err
}
func (r *endReads) grantsFor(branch string, source int64) (meshgrants.GrantsFor, error) {
k := fmt.Sprintf("%s.%d", branch, source)
got, ok := r.grants[k]
if !ok {
got.g, got.err = r.w.GrantsFor(branch, source)
r.grants[k] = got
}
return got.g, got.err
}
func (r *endReads) widens(p seat.Pipeline, module string) []seat.Widening {
could := func(name, said string) []seat.Widening {
return []seat.Widening{{What: seat.WideningWhatCouldNotRun, Name: name, Said: said}}
}
cand, err := w.Revision(p.Branch, p.Revision)
cand, err := r.revision(p.Branch, p.Revision)
if err != nil {
return could("mesh-catalog", fmt.Sprintf("the candidate's revision %d is not read: %v", p.Revision, err))
}
newest, err := w.Revision(p.Branch, 0)
newest, err := r.revision(p.Branch, 0)
if err != nil {
return could("mesh-catalog", fmt.Sprintf("the branch's newest revision is not read: %v", err))
}
var out []seat.Widening
for _, s := range servingSeats(p) {
holder, err := w.Holder(s)
holder, err := r.holder(s)
if err != nil {
out = append(out, seat.Widening{What: seat.WideningWhatCouldNotRun, Name: s, Said: fmt.Sprintf("the holder of %s is not read: %v", s, err)})
continue
@@ -57,8 +124,8 @@ func (w *Widener) Widens(p seat.Pipeline, module string) []seat.Widening {
}
out = append(out, ruleWidens(module, newest.Entries[module], cand.Entries[module])...)
out = append(out, credentialWidens(module, newest.Entries[module], cand.Entries[module])...)
if w.GrantsFor != nil {
g, err := w.GrantsFor(p.Branch, p.Source)
if r.w.GrantsFor != nil {
g, err := r.grantsFor(p.Branch, p.Source)
switch {
case err != nil:
out = append(out, seat.Widening{What: seat.WideningWhatCouldNotRun, Name: "mesh-grants", Said: err.Error()})
@@ -88,17 +88,19 @@ func TestWidensIsTheCandidateAgainstTheNewest(t *testing.T) {
// A pipeline's end keeps each mover's widens in its value, whether or not anything widens (design 47 G §71, A-207; H H-A-51).
func TestAnEndKeepsEachMoversWidens(t *testing.T) {
h := &Hearer{Widens: func(p seat.Pipeline, module string) []seat.Widening {
return []seat.Widening{{What: seat.WideningWhatGrant, Name: module + " s v"}}
h := &Hearer{Widens: func() func(seat.Pipeline, string) []seat.Widening {
return func(p seat.Pipeline, module string) []seat.Widening {
return []seat.Widening{{What: seat.WideningWhatGrant, Name: module + " s v"}}
}
}}
p := seat.Pipeline{Outcome: seat.PipelineOutcomeRunning, Movers: []seat.Mover{{Module: "app"}, {Module: "lib"}},
Steps: []seat.StepResult{{Module: "app", Step: "build", Outcome: "passed"}, {Module: "lib", Step: "build", Outcome: "passed"}}}
if !h.endPipeline(&p, time.Unix(1, 0)) || len(p.Movers[0].Widens) != 1 || p.Movers[0].Widens[0].Name != "app s v" || p.Movers[1].Widens[0].Name != "lib s v" {
if !h.endPipeline(&p, time.Unix(1, 0), h.widensReads()) || len(p.Movers[0].Widens) != 1 || p.Movers[0].Widens[0].Name != "app s v" || p.Movers[1].Widens[0].Name != "lib s v" {
t.Errorf("the ended movers = %+v", p.Movers)
}
h.Widens = nil
q := seat.Pipeline{Outcome: seat.PipelineOutcomeRunning, Movers: []seat.Mover{{Module: "app"}}, Steps: []seat.StepResult{{Module: "app", Step: "build", Outcome: "passed"}}}
if !h.endPipeline(&q, time.Unix(1, 0)) || len(q.Movers[0].Widens) != 0 {
if !h.endPipeline(&q, time.Unix(1, 0), h.widensReads()) || len(q.Movers[0].Widens) != 0 {
t.Errorf("a runner with no widener kept %v", q.Movers[0].Widens)
}
}