Gate hands every member of an approved group, in pin order, and says a stopped pass (issue 1275)

Met again 2026-10-25: group fix/puller-wakes-on-capacity was merged half. Three causes in the gate:

- The next member's turn compared the group's state with "delivering", a word GroupState no longer says
  since release 6 renamed it "merging" (ready.go groupTurnToMerge). Once the first member's merge was heard
  from the forge the group read merging, and the second member waited for ever. Group states are now named
  constants, the turn goes on in ready and merging (groupMayMerge), and a test holds every word to it.
- Members merged in id order, so mesh-catalog went before the mesh-sdk commit it pins. membersOf now orders
  by mergeRank: mesh-sdk first, then members that change no go.mod or go.sum, then those that do.
- No condition was raised: the controller's self-check that read `stalled` went with release 6
  (mesh-controller dc456d56). The gate now says its stalled lines as conditions itself on the
  operator-channel seat (ADR 0437 §9) after every look, clears those listed no more, and keeps which it
  raised in its new bucket conditions. A held group's line says "half merged" when a member merged.

A pass held for a required status no longer retries on a timer: the status turning success is the event
that asks the merge again.
This commit is contained in:
2026-10-25 18:20:58 +01:00
parent 2df0503034
commit 67092161bd
14 changed files with 563 additions and 64 deletions
+4 -1
View File
@@ -22,7 +22,10 @@ is gone is read as the state it came to (merged, closed or proposed) when the ga
## When a refused or rejected merge is said (ADR 0505 §4, §5)
`stalled` is what the controller raises as a condition for the operator. The gate's own list (`show`, `deliveries`)
`stalled` is what the gate says to the operator as conditions, itself, on the operator-channel seat (ADR 0437 §9, issue
1275): after every look it raises each line, raises it again when its severity or need changes, and clears each that is
listed no more; its bucket `conditions` keeps which it raised. (The controller's self-check that raised them went in
release 6.) The gate's own list (`show`, `deliveries`)
names every refusal and rejection at once, and the pull request is told at once; only `stalled` waits:
- a merge refused because its trunk moved, whose check the gate asked again, is stalled only once that check answered and
@@ -0,0 +1,166 @@
package main
import (
"encoding/json"
"fmt"
"sort"
"strings"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/asks"
)
// The gate says its own stalled lines to the operator as conditions (novox/hq issue 1275, met again 2026-10-25; ADR 0437
// §9: any module says a condition on the operator-channel seat's one subject, and the router alone chooses the channels).
// Until release 6 the controller's self-check read the `stalled` verb and raised each line as `delivery.<id>.stalled`;
// release 6 deleted that self-check (mesh-controller dc456d56) and nothing read the verb since, so a half-merged group was
// said to no one. Now, after every look, the gate raises each line `stalled` lists, raises it again when its severity or
// its need changes, and clears each it raised that is listed no more. What it raised is kept in its bucket conditions, so a
// gate that restarts clears what ended while it was away.
// ConditionStore keeps the conditions the gate has raised: the condition's key to what it said (conditionSaid).
type ConditionStore interface {
Said() (map[string]string, error)
PutSaid(said map[string]string) error
}
// saidKey is the one key of the bucket conditions.
const saidKey = "said"
// kvConditions is the bucket conditions over the runtime.
type kvConditions struct{}
func (kvConditions) Said() (map[string]string, error) {
e, err := stdio.State("conditions").Get(saidKey)
if err != nil || e == nil {
return map[string]string{}, err
}
said := map[string]string{}
if err := json.Unmarshal(e.Value, &said); err != nil {
return nil, fmt.Errorf("the conditions the gate raised are not readable: %w", err)
}
return said, nil
}
func (kvConditions) PutSaid(said map[string]string) error {
_, err := stdio.State("conditions").Put(saidKey, said)
return err
}
// sayOnSeat says a condition on the operator-channel seat under the gate's own name.
func sayOnSeat(c asks.Condition) error {
subject, err := asks.ConditionSubject("mesh-delivery-gate")
if err != nil {
return err
}
raw, err := asks.EncodeCondition(c)
if err != nil {
return err
}
return stdio.SeatPublish(subject, json.RawMessage(raw), "")
}
// conditionKeyOf is the key of a stalled line's condition: its last part is the condition's kind.
func conditionKeyOf(id string) string { return "mesh-delivery-gate.delivery." + id + ".stalled" }
// conditionSaid is what a raised condition said that, when it changes, raises it again: its severity and what it needs.
// The words of a line hold durations, which change at every look and are no reason to tell the operator again.
func conditionSaid(l StalledLine) string { return severityOfLine(l) + "|" + l.H2 + "|" + l.Needs }
func severityOfLine(l StalledLine) string {
if l.Severity == asks.SeverityUrgent {
return asks.SeverityUrgent
}
return asks.SeverityWarning
}
// conditionOf is a stalled line as the condition the operator is told.
func conditionOf(l StalledLine) asks.Condition {
headline := "stalled: " + l.ID
if strings.HasPrefix(l.Says, "group ") {
group, _, _ := strings.Cut(strings.TrimPrefix(l.Says, "group "), " ")
headline = "group " + group + " is not merged whole"
}
if r := []rune(headline); len(r) > asks.HeadlineLength {
headline = string(r[:asks.HeadlineLength-1]) + "…"
}
needs := "read mesh-delivery.show " + l.ID + " and decide"
return asks.Condition{Key: conditionKeyOf(l.ID), Severity: severityOfLine(l), State: asks.StateRaised,
Subject: asks.Scope{Scope: "delivery", ID: l.ID}, Summary: l.Says, Headline: headline, Needs: needs,
Explanation: asks.NeedsYou + needs + ". " + l.Says + " What ends it: " + l.H2 + ".",
Resolved: "The gate lists " + l.ID + " as stalled no more."}
}
// sayConditions raises and clears the conditions of the stalled lines (see above). Outside the lock; React runs it after
// the look, so it follows every event. A condition that could not be said is said again at the next look, after
// askAgainAfter at the latest.
func (h *Holder) sayConditions() {
if h.Say == nil || h.Conditions == nil {
return
}
if h.said == nil {
said, err := h.Conditions.Said()
if err != nil {
h.Logf("[mesh-delivery] the conditions raised before cannot be read, so none is said yet: %v; asked again in %s", err, askAgainAfter)
h.again("the conditions the gate raised, read")
return
}
h.said = said
}
lines := map[string]StalledLine{}
for _, l := range h.Stalled() {
// One condition for an id: the most severe of its lines.
if o, ok := lines[l.ID]; !ok || (severityOfLine(o) != asks.SeverityUrgent && severityOfLine(l) == asks.SeverityUrgent) {
lines[l.ID] = l
}
}
ids := make([]string, 0, len(lines))
for id := range lines {
ids = append(ids, id)
}
sort.Strings(ids)
changed, failed := false, false
for _, id := range ids {
l, key := lines[id], conditionKeyOf(id)
said := conditionSaid(l)
if h.said[key] == said {
continue
}
if err := h.Say(conditionOf(l)); err != nil {
h.Logf("[mesh-delivery] the condition of %s could not be said: %v", id, err)
failed = true
continue
}
h.Logf("[mesh-delivery] %s is said to the operator as %s: %s", id, severityOfLine(l), l.Says)
h.said[key], changed = said, true
}
var gone []string
for key := range h.said {
id := strings.TrimSuffix(strings.TrimPrefix(key, "mesh-delivery-gate.delivery."), ".stalled")
if _, ok := lines[id]; !ok {
gone = append(gone, key)
}
}
sort.Strings(gone)
for _, key := range gone {
id := strings.TrimSuffix(strings.TrimPrefix(key, "mesh-delivery-gate.delivery."), ".stalled")
c := asks.Condition{Key: key, Subject: asks.Scope{Scope: "delivery", ID: id}, State: asks.StateCleared,
Resolved: "The gate lists " + id + " as stalled no more."}
if err := h.Say(c); err != nil {
h.Logf("[mesh-delivery] the condition of %s could not be cleared: %v", id, err)
failed = true
continue
}
h.Logf("[mesh-delivery] the condition of %s is cleared: it is stalled no more", id)
delete(h.said, key)
changed = true
}
if changed {
if err := h.Conditions.PutSaid(h.said); err != nil {
h.Logf("[mesh-delivery] the conditions raised cannot be kept: %v", err)
}
}
if failed {
h.again("a condition that could not be said to the operator")
}
}
@@ -63,7 +63,8 @@ func (h *Holder) unmergedLive(g *Group) []*Delivery {
// groupMergeable reads every member's required statuses at the head the Approve covers, and says whether all are a success.
// When one is not, or cannot be read, nothing is handed to the bridge: the group is held, said urgent (ADR 0441 §2, §4), and
// the target's turn is given back so the next event that decides whether it may merge asks it again. Outside the lock.
// the target's turn is given back so the next event that decides whether it may merge asks it again. A read that could not
// be made is a failed ask, tried again after askAgainAfter. Outside the lock.
func (h *Holder) groupMergeable(gid, target string, reads []memberRead) bool {
release := func() {
if d := h.deliveries[target]; d != nil && d.Ready != nil {
@@ -98,8 +99,9 @@ func (h *Holder) groupMergeable(gid, target string, reads []memberRead) bool {
g.Hold = &GroupHold{Kind: "blocked", Member: m.id, Says: says, Since: since}
h.keepGroup(g)
}
// No timer: the status turning success is an event on the forge (CommitStatus, mergeAgainNow), and it asks the
// merge again at once (issue 1275: a pass that stops part way resumes on an event).
release()
h.again("a group's merge held for a required status")
h.mu.Unlock()
return false
}
@@ -112,7 +114,7 @@ func (h *Holder) groupMergeable(gid, target string, reads []memberRead) bool {
return true
}
// groupLines are the groups that are held or half merged, as urgent stalled lines the controller raises as conditions:
// groupLines are the groups that are held or half merged, as urgent stalled lines the gate says as conditions (conditions.go):
// one for each, on its first member not merged, until every member is merged or a person decides (issue 1275). A member
// the merge tool refused is said by readyLines, with its group; this adds a group held for a required status, and a
// group half merged with no refusal. Under the lock.
@@ -141,12 +143,16 @@ func (h *Holder) groupLines(now time.Time) []StalledLine {
switch {
// A held member the merge tool refused is said by readyLines already: one line for an id, not two.
case g.Hold != nil && g.Hold.Kind == "blocked" && !running && !(heldOn != nil && heldOn.Ready != nil && heldOn.Ready.RefusedSince != nil):
stands := "is approved and nothing is merged"
if merged > 0 {
stands = "is half merged" // a member merged before the pass stopped (issue 1275)
}
out = append(out, StalledLine{ID: g.Hold.Member, Number: numberOf(h.deliveries[g.Hold.Member]), State: first.State,
For: now.Sub(g.Hold.Since).Round(time.Second).String(), Bound: "0s", Severity: "urgent",
H2: "none: the required status turns success, which asks the merge again — or the operator's decision",
Says: fmt.Sprintf("group %s is approved and nothing is merged (merged %d of %d): %s waits on a required status "+
Says: fmt.Sprintf("group %s %s (merged %d of %d): %s waits on a required status "+
"at its head: %s. Needed: that status turns success (its check asked again), or the operator decides.",
g.ID, merged, len(h.membersOf(g)), g.Hold.Member, g.Hold.Says)})
g.ID, stands, merged, len(h.membersOf(g)), g.Hold.Member, g.Hold.Says)})
case merged > 0 && !refused && !running:
out = append(out, StalledLine{ID: first.ID, Number: first.Number, State: first.State,
For: now.Sub(g.Updated).Round(time.Second).String(), Bound: "0s", Severity: "urgent",
@@ -21,25 +21,44 @@ const (
// approvedGroup is a group of two pull requests on one branch, checks green, composed green and the operator's Approve
// asked and not yet answered (w.op.auto is off).
func approvedGroup(t *testing.T) *world {
return approvedGroupOf(t, member{gCtl, 1, gHc, "internal/a.go"}, member{gCat, 2, gHk, "modules/app/module.json"})
}
// member is a pull request of a group in feat/g: its repository, number, head and the paths it changes.
type member struct {
repo string
number int
head string
paths string
}
// approvedGroupOf is a group feat/g of these members, each checked, composed together and asked once.
func approvedGroupOf(t *testing.T, ms ...member) *world {
t.Helper()
w := newWorld(t)
w.op.auto = false
w.h.PullUpdated(pr(gCtl, 1, gHc, "feat/g", "internal/a.go"))
w.h.PullUpdated(pr(gCat, 2, gHk, "feat/g", "modules/app/module.json"))
w.h.Checked(judged(gCtl, 1, gHc, "pass", gTrunk0))
w.h.Checked(judged(gCat, 2, gHk, "pass", gTrunk0))
for _, m := range ms {
w.h.PullUpdated(pr(m.repo, m.number, m.head, "feat/g", strings.Split(m.paths, ",")...))
}
for _, m := range ms {
w.h.Checked(judged(m.repo, m.number, m.head, "pass", gTrunk0))
}
w.settleAll()
g := w.h.groups["feat/g"]
if g == nil || g.Check == nil {
t.Fatalf("no composed check was asked: %+v", g)
}
w.h.Checked(CheckedEvent{Group: "feat/g", ID: g.Check.Asked, Verdict: "pass", Summary: "composed",
Members: []struct {
c := CheckedEvent{Group: "feat/g", ID: g.Check.Asked, Verdict: "pass", Summary: "composed"}
for _, m := range ms {
owner, repo, _ := strings.Cut(m.repo, "/")
c.Members = append(c.Members, struct {
Owner string `json:"owner"`
Repo string `json:"repo"`
Commit string `json:"commit"`
Trunk string `json:"trunk,omitempty"`
}{{Owner: "novox", Repo: "mesh-controller", Commit: gHc, Trunk: gTrunk0[:12]},
{Owner: "novox", Repo: "mesh-catalog", Commit: gHk, Trunk: gTrunk0[:12]}}})
}{Owner: owner, Repo: repo, Commit: m.head, Trunk: gTrunk0[:12]})
}
w.h.Checked(c)
w.settleAll()
if w.op.count() != 1 || len(w.op.open()) != 1 {
t.Fatalf("the green group is asked once: %d asks, open %v", w.op.count(), w.op.open())
@@ -10,6 +10,8 @@ import (
"strings"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
)
// The holder of the mesh-delivery seat (novox/hq ADR 0239, to-be 47): the one writer of deliveries and
@@ -37,6 +39,11 @@ type Holder struct {
Owed OwedStore
// Pulls keeps the pull requests' keys on the bus (pullkey.go); nil keeps none.
Pulls PullKeys
// Say says a condition to the operator, and Conditions keeps which the gate raised (conditions.go); nil says none.
Say func(c asks.Condition) error
Conditions ConditionStore
// said is what Conditions keeps, read at the first look; touched by React alone.
said map[string]string
// cancels are asks to take back, gathered under the lock and run with the other asks of the look.
cancels []func()
@@ -921,7 +928,7 @@ func (h *Holder) askChecks(now time.Time) func() {
}
}
// membersOf are a group's current members, by id order.
// membersOf are a group's current members, in the order they merge (mergeRank, then id).
func (h *Holder) membersOf(g *Group) []*Delivery {
var out []*Delivery
for _, id := range g.Members {
@@ -929,9 +936,39 @@ func (h *Holder) membersOf(g *Group) []*Delivery {
out = append(out, d)
}
}
sort.SliceStable(out, func(i, j int) bool {
if ri, rj := mergeRank(out[i]), mergeRank(out[j]); ri != rj {
return ri < rj
}
return out[i].ID < out[j].ID
})
return out
}
// SDKRepository is the repository of mesh-sdk, the library the other repositories' modules pin by commit (novox/hq ADR 0482 §3).
const SDKRepository = "novox/mesh-sdk"
// mergeRank is a member's place in its group's merge (novox/hq issue 1275): a member another member pins merges before it, so
// that the trunk never holds a pin of a commit not on its own trunk. The gate knows a pin by the paths a pull request changes:
// mesh-sdk's member goes first (the members of the other repositories pin it, ADR 0482 §3); then the members that change no
// go.mod or go.sum (they pin nothing new); then those that do (they pin another member's commit). The id orders a tie. The
// module graph is the fuller source (which module requires which), and is not the gate's to read: this is what its own
// deliveries say.
func mergeRank(d *Delivery) int {
if strings.EqualFold(d.Repository, SDKRepository) {
return 0
}
if d.PathsTruncated {
return 2 // its paths are not all known: it may pin, so it goes after those that do not
}
for _, p := range d.Paths {
if base := p[strings.LastIndex(p, "/")+1:]; base == "go.mod" || base == "go.sum" {
return 2
}
}
return 1
}
// planGroup is what a group needs asked, and its turns.
func (h *Holder) planGroup(g *Group, members []*Delivery, now time.Time) []func() {
var asks []func()
@@ -956,7 +993,7 @@ func (h *Holder) planGroup(g *Group, members []*Delivery, now time.Time) []func(
if held {
status = "pending" // no Approve exists while a member waits for its review (novox/hq ADR 0496 §2)
}
if state == "merging" && !(g.Check != nil && (g.Check.Verdict == "pass" || g.Check.Verdict == "warning") &&
if state == GroupMerging && !(g.Check != nil && (g.Check.Verdict == "pass" || g.Check.Verdict == "warning") &&
sameMembers(g.Check.Commits, commitsOf(members)) && g.Check.OutOfDate == "") {
// Success there lets the merge tool merge its members: only for a composed check that passed for these heads.
status = "pending"
@@ -980,6 +1017,23 @@ func (h *Holder) planGroup(g *Group, members []*Delivery, now time.Time) []func(
return asks
}
// The words GroupState says a group's state with (ADR 0539 §17). Code that decides on a group's state compares with these,
// never with a literal: a word GroupState no longer says ("delivering", retired in release 6) left the second member of
// every approved group waiting for ever (novox/hq issue 1275, met again 2026-10-25). groupWords lists them all, and a test
// holds every decision on a group's state to that list.
const (
GroupChecking = "checking"
GroupRejected = "rejected"
GroupReady = "ready"
GroupMerging = "merging"
GroupMerged = "merged"
GroupHeld = "held"
GroupClosed = "closed"
)
// groupWords are every word GroupState says.
var groupWords = []string{GroupChecking, GroupRejected, GroupReady, GroupMerging, GroupMerged, GroupHeld, GroupClosed}
// GroupState is a group's state, derived from its members and its composed check — never set apart (novox/hq
// ADR 0239 decision 4, cut to the merge's states by ADR 0539 §17): checking, rejected, ready, merging (some members
// merged), merged, held and closed. Nothing after the merge is derived here: whether a merged change reached the machines is
@@ -992,36 +1046,36 @@ func GroupState(g *Group, members []*Delivery) (string, string) {
n := len(members)
switch {
case count[Held] > 0 && g.Closed:
return "held", "a member is held: " + heldWords(members)
return GroupHeld, "a member is held: " + heldWords(members)
case count[Closed] > 0 && g.Closed:
return "closed", "a member was closed before the group merged"
return GroupClosed, "a member was closed before the group merged"
case n > 0 && count[Merged] == n:
return "merged", "every member is merged"
return GroupMerged, "every member is merged"
case g.Closed && g.Check != nil && (g.Check.OutOfDate != "" || g.Check.Verdict == "" ||
!sameMembers(g.Check.Commits, commitsOf(members))):
// A member rebased or pushed once the group was merging, or its composed check out of date: it is composed
// again for these heads, and the group is checking until then, never held (novox/hq ADR 0323 §6, issue 406).
if g.Check.OutOfDate != "" {
return "checking", g.Check.OutOfDate
return GroupChecking, g.Check.OutOfDate
}
return "checking", "composed again for its members' new heads"
return GroupChecking, "composed again for its members' new heads"
case g.Closed && g.Check != nil && g.Check.Verdict != "pass" && g.Check.Verdict != "warning":
// Composed again for new heads once it was merging, and those do not compose: its members wait (issue 902).
return "rejected", "composed together: " + g.Check.Summary
return GroupRejected, "composed together: " + g.Check.Summary
case g.Closed:
return "merging", fmt.Sprintf("%d of %d merged", count[Merged], n)
return GroupMerging, fmt.Sprintf("%d of %d merged", count[Merged], n)
case count[Rejected] > 0:
return "rejected", "a member's own check failed"
return GroupRejected, "a member's own check failed"
case g.Check != nil && g.Check.OutOfDate != "":
// Out of date counts as outstanding (ADR 0239 §18, novox/hq ADR 0323 §7): checking, saying why.
return "checking", g.Check.OutOfDate
return GroupChecking, g.Check.OutOfDate
case g.Check != nil && g.Check.Verdict != "" && g.Check.Verdict != "pass" && g.Check.Verdict != "warning":
return "rejected", "composed together: " + g.Check.Summary
return GroupRejected, "composed together: " + g.Check.Summary
case count[Ready] == n && g.Check != nil && sameMembers(g.Check.Commits, commitsOf(members)) &&
(g.Check.Verdict == "pass" || g.Check.Verdict == "warning"):
return "ready", "every member ready, and composed together they pass"
return GroupReady, "every member ready, and composed together they pass"
}
return "checking", fmt.Sprintf("%d of %d member(s) ready", count[Ready], n)
return GroupChecking, fmt.Sprintf("%d of %d member(s) ready", count[Ready], n)
}
// heldWords says the held members and why.
@@ -1040,7 +1094,7 @@ func heldWords(members []*Delivery) string {
// says its mesh/delivery-group is pending, not a success. Under the lock.
func (h *Holder) groupState(g *Group, members []*Delivery) (state, why string, held bool) {
state, why = GroupState(g, members)
if state != "ready" || g.Closed {
if state != GroupReady || g.Closed {
return state, why, false
}
var waiting []string
@@ -1063,11 +1117,11 @@ func (h *Holder) groupState(g *Group, members []*Delivery) (state, why string, h
// heads (novox/hq ADR 0382), and a pending one there would hold them for ever (novox/hq issue 902).
func groupStatusState(s string) string {
switch s {
case "ready", "merging", "merged":
case GroupReady, GroupMerging, GroupMerged:
return "success"
case "rejected":
case GroupRejected:
return "failure"
case "held", "closed":
case GroupHeld, GroupClosed:
return "error"
}
return "pending"
@@ -3,8 +3,12 @@ package main
import (
"errors"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
sdk "git.novox.be/novox/mesh-sdk/go/delivery"
)
const time30s = 31 * time.Second
@@ -75,24 +79,6 @@ func TestAnApprovedGroupWithAMemberWhoseRequiredStatusIsNotASuccessMergesNothing
}
}
// 1275/AC3: the merge tool refuses the first member in order; the second is not merged, and the group is said urgent,
// naming the group, the refused member and the forge's words.
func TestARefusedFirstMemberStopsTheGroupAndIsSaidUrgent(t *testing.T) {
w := approvedGroup(t)
allGreen(w)
w.forge.refuseRepo = map[string]error{gCat: refusedWords("novox/mesh-catalog#2")}
w.op.answer(w.op.open()[0], OptionApprove)
w.settleAll()
if len(w.forge.merges) != 1 || !strings.Contains(w.forge.merges[0], "mesh-catalog#2") {
t.Fatalf("wanted only the first member asked: %v", w.forge.merges)
}
lines := urgentLines(w)
if len(lines) != 1 || lines[0].ID != IDOf(gCat, gHk) || !strings.Contains(lines[0].Says, "feat/g") ||
!strings.Contains(lines[0].Says, "mesh/repo-check (missing)") || !strings.Contains(lines[0].Says, "Needed") {
t.Fatalf("not said as one urgent line: %+v", lines)
}
}
// 1275/AC4: a group half merged (the second member refused after the first merged) stays raised, pass after pass, until
// the second is merged; then it goes.
func TestAHalfMergedGroupStaysRaisedUntilEveryMemberIsMerged(t *testing.T) {
@@ -344,3 +330,249 @@ func TestASeverityOnceUrgentNeverStepsDownToAWarning(t *testing.T) {
t.Fatalf("severity %v after an out of date refusal that followed an urgent one; wanted urgent", got)
}
}
// fakeConditions is the gate's bucket conditions and the operator-channel seat's subject for conditions: what it keeps,
// and every condition said, in order.
type fakeConditions struct {
mu sync.Mutex
kept map[string]string
told []asks.Condition
}
func (f *fakeConditions) Said() (map[string]string, error) {
f.mu.Lock()
defer f.mu.Unlock()
out := map[string]string{}
for k, v := range f.kept {
out[k] = v
}
return out, nil
}
func (f *fakeConditions) PutSaid(said map[string]string) error {
f.mu.Lock()
defer f.mu.Unlock()
f.kept = map[string]string{}
for k, v := range said {
f.kept[k] = v
}
return nil
}
func (f *fakeConditions) say(c asks.Condition) error {
if _, err := asks.EncodeCondition(c); err != nil {
return err // what the router would refuse
}
f.mu.Lock()
defer f.mu.Unlock()
f.told = append(f.told, c)
return nil
}
// last is the newest condition said for a delivery id, or nil.
func (f *fakeConditions) last(id string) *asks.Condition {
f.mu.Lock()
defer f.mu.Unlock()
for i := len(f.told) - 1; i >= 0; i-- {
if f.told[i].Subject.ID == id {
c := f.told[i]
return &c
}
}
return nil
}
// hearConditions has the gate say its conditions into a fake seat and bucket.
func hearConditions(w *world) *fakeConditions {
f := &fakeConditions{kept: map[string]string{}}
w.h.Say, w.h.Conditions = f.say, f
return f
}
// handedInOrder answers each member the gate hands the forge bridge as the forge does on 2026-10-25: the forge's word that
// it merged (gitea.pull.merged) first, the bridge's word after; or the bridge's first, as forgeFirst says. It returns the
// repositories in the order they were handed.
func handedInOrder(w *world, forgeFirst bool) []string {
var handed []string
numbers := map[string]int{}
for i := 0; i < 4; i++ {
var next *sdk.MergePull
for _, p := range w.merges.last("feat/g").Pulls {
if p.State == sdk.MergeMerging {
p := p
next = &p
}
}
if next == nil || numbers[next.Repository] > 0 {
break
}
handed = append(handed, next.Repository)
numbers[next.Repository] = next.Number
mergeCommit := strings.Repeat(string(rune('a'+i)), 40)
forge := func() { w.h.PullMerged(merged(next.Repository, next.Number, next.Head.Head, mergeCommit)) }
bridge := func() { bridgeMerged(w, next.Repository, next.Number, next.Head.Head, mergeCommit) }
if forgeFirst {
forge()
w.settleAll() // ten seconds pass between the two words on 2026-10-25: the gate looks in between
bridge()
} else {
bridge()
w.settleAll()
forge()
}
w.settleAll()
}
return handed
}
// Issue 1275, met again 2026-10-25 (group fix/puller-wakes-on-capacity): every member of an approved group is handed to
// the forge bridge, in the group's order, the next on the event that the one before merged; a group whose pass stops is
// said to the operator as a condition, which is cleared once it is whole; and every word a group's state is said in is
// one the merge decides on.
func TestAnApprovedGroupIsHandedWholeInItsOrderAndAStopIsSaid(t *testing.T) {
const (
sdkRepo, hS = "novox/mesh-sdk", "a8fbabbfc37369f9d1f6977c72e5523aff3d494c"
catRepo, hK = "novox/mesh-catalog", "23c827e7910c233608af4e4a8056cbc362dcb755"
ctlRepo, hC = "novox/mesh-controller", "c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0"
)
sdkMember := member{sdkRepo, 80, hS, "go/stdio.go,go/buildstep/pull.go"}
catMember := member{catRepo, 479, hK, "modules/mesh-build-go/asks.go,modules/mesh-build-go/go.mod,modules/mesh-build-go/go.sum"}
ctlMember := member{ctlRepo, 7, hC, "cmd/mesh-controller/main.go"}
inOrder := func(members []member, forgeFirst bool, want ...string) func(t *testing.T) {
return func(t *testing.T) {
w := approvedGroupOf(t, members...)
w.merges.manual = true
allGreenFor(w, members...)
w.op.answer(w.op.open()[0], OptionApprove)
w.settleAll()
if got := handedInOrder(w, forgeFirst); strings.Join(got, " ") != strings.Join(want, " ") {
t.Fatalf("handed %v, wanted %v, every member in the group's order", got, want)
}
if g := w.h.groups["feat/g"]; g == nil {
t.Fatal("the group is gone")
} else if state, why := GroupState(g, w.h.membersOf(g)); state != GroupMerged {
t.Fatalf("the group is %s (%s), not merged whole", state, why)
}
if w.source.main == nil || len(w.source.main.Heads) != len(want) {
t.Fatalf("source.main does not hold the group as one revision: %+v", w.source.main)
}
}
}
cases := []struct {
name string
run func(t *testing.T)
}{
// 1275/AC3 (2026-10-25): the case of the day. The catalogue member pins the sdk member's commit; the forge's word
// of a merge arrives before the bridge's. Before the fix the catalogue member went first, and the sdk member was
// never handed: its group read merging, a word the turn to merge did not know.
{"sdk first, then the member that pins it, the forge's word first", inOrder([]member{catMember, sdkMember}, true, sdkRepo, catRepo)},
{"sdk first, then the member that pins it, the bridge's word first", inOrder([]member{sdkMember, catMember}, false, sdkRepo, catRepo)},
{"a member that pins nothing before one that changes a go.mod", inOrder([]member{catMember, ctlMember}, true, ctlRepo, catRepo)},
{"three members", inOrder([]member{ctlMember, catMember, sdkMember}, true, sdkRepo, ctlRepo, catRepo)},
// 1275/AC3: the merge tool refuses the first member in order; the second is not merged, and the group is said
// urgent, naming the group, the refused member and the forge's words, as a stalled line and to the operator.
{"a refused first member stops the group and is said urgent", func(t *testing.T) {
w := approvedGroup(t)
told := hearConditions(w)
allGreen(w)
w.forge.refuseRepo = map[string]error{gCat: refusedWords("novox/mesh-catalog#2")}
w.op.answer(w.op.open()[0], OptionApprove)
w.settleAll()
if len(w.forge.merges) != 1 || !strings.Contains(w.forge.merges[0], "mesh-catalog#2") {
t.Fatalf("wanted only the first member asked: %v", w.forge.merges)
}
lines := urgentLines(w)
if len(lines) != 1 || lines[0].ID != IDOf(gCat, gHk) || !strings.Contains(lines[0].Says, "feat/g") ||
!strings.Contains(lines[0].Says, "mesh/repo-check (missing)") || !strings.Contains(lines[0].Says, "Needed") {
t.Fatalf("not said as one urgent line: %+v", lines)
}
if c := told.last(IDOf(gCat, gHk)); c == nil || c.State != asks.StateRaised || c.Severity != asks.SeverityUrgent ||
!strings.Contains(c.Summary, "feat/g") {
t.Fatalf("the operator was not told, urgent: %+v", c)
}
}},
// 1275/AC4: a group half merged (its pass stopped after the first member) raises one urgent condition, said once
// however many looks follow, and cleared once the group is whole.
{"a half-merged group is raised once and cleared once whole", func(t *testing.T) {
w := approvedGroup(t)
told := hearConditions(w)
allGreen(w)
w.merges.manual = true
w.op.answer(w.op.open()[0], OptionApprove)
w.settleAll()
w.h.PullMerged(merged(gCat, 2, gHk, gT1))
bridgeMerged(w, gCat, 2, gHk, gT1)
setChecks(w, gCtl, gHc, "success", "") // the second member's status went missing: the pass stops at it
w.settleAll()
for range 3 {
w.later(time30s)
w.settleAll()
}
id := IDOf(gCtl, gHc)
raised := 0
for _, c := range told.told {
if c.Subject.ID == id && c.State == asks.StateRaised {
raised++
}
}
if c := told.last(id); raised != 1 || c == nil || c.Severity != asks.SeverityUrgent || !strings.Contains(c.Summary, "feat/g") ||
!strings.Contains(c.Summary, "half merged (merged 1 of 2)") {
t.Fatalf("raised %d times, last %+v; wanted once, urgent, naming the group half merged", raised, c)
}
setChecks(w, gCtl, gHc, "success", "success")
w.h.CommitStatus(ForgeStatusEvent{Owner: "novox", Repo: "mesh-controller", SHA: gHc, Context: repoCheck, State: "success"})
w.settleAll() // the status is an event: the pass goes on with no timer
handed := false
for _, p := range w.merges.last("feat/g").Pulls {
handed = handed || (p.Repository == gCtl && p.State == sdk.MergeMerging)
}
if !handed {
t.Fatalf("the status turned success and the second member was not handed: %+v", w.merges.last("feat/g"))
}
w.h.PullMerged(merged(gCtl, 1, gHc, gT1))
bridgeMerged(w, gCtl, 1, gHc, gT1)
w.settleAll()
if c := told.last(id); c == nil || c.State != asks.StateCleared {
t.Fatalf("the group is whole and its condition is not cleared: %+v", c)
}
}},
// A condition the gate raised before it restarted, whose delivery stalls no more, is cleared at the first look.
{"a condition kept from before a restart is cleared", func(t *testing.T) {
w := newWorld(t)
told := hearConditions(w)
told.kept[conditionKeyOf("novox/app@0123456789ab")] = "urgent||"
w.settleAll()
if c := told.last("novox/app@0123456789ab"); c == nil || c.State != asks.StateCleared || len(told.kept) != 0 {
t.Fatalf("not cleared: %+v, kept %v", c, told.kept)
}
}},
// The kind behind the day's fault: a decision on a group's state named a word GroupState no longer says. Every word
// it says is one groupStatusState knows, and the merge goes on exactly in ready and merging.
{"every word of a group's state is decided on", func(t *testing.T) {
for _, word := range groupWords {
if groupStatusState(word) == "pending" && word != GroupChecking {
t.Errorf("groupStatusState does not know %q", word)
}
if may := groupMayMerge(word); may != (word == GroupReady || word == GroupMerging) {
t.Errorf("groupMayMerge(%q) = %v", word, may)
}
}
}},
}
for _, c := range cases {
t.Run(c.name, c.run)
}
}
// allGreenFor makes both merge checks a success at every member's head.
func allGreenFor(w *world, ms ...member) {
for _, m := range ms {
setChecks(w, m.repo, m.head, "success", "success")
}
}
// bridgeMerged is the forge bridge's word that it merged a member.
func bridgeMerged(w *world, repository string, number int, head, merge string) {
w.merges.say(sdk.Forge{Kind: sdk.ForgeMerged, Repository: repository, Number: number, Head: head, Merge: merge, Root: "root",
Trees: map[string]string{"app": "tree"}})
}
@@ -6,7 +6,7 @@ import "git.novox.be/novox/mesh-sdk/go/announce"
// issue 1790): the module says it itself, from its code, before it serves, and a reader is granted what it said
// (clause 3). A state is readable when another module's manifest reads it (keeps_test.go names the readers). keeps_test.go holds the list to the
// manifest's `state`, and keeps_integration_test.go that the module announces it on a bus.
var Kept = []announce.State{{Name: "asks", Readable: true}, {Name: "deliveries"}, {Name: "groups"}, {Name: "merges", Readable: true}, {Name: "owed", Readable: true}, {Name: "pulls", Readable: true}, {Name: "source", Readable: true}}
var Kept = []announce.State{{Name: "asks", Readable: true}, {Name: "conditions"}, {Name: "deliveries"}, {Name: "groups"}, {Name: "merges", Readable: true}, {Name: "owed", Readable: true}, {Name: "pulls", Readable: true}, {Name: "source", Readable: true}}
// announceKept says Kept through keeps, which is stdio.Keeps.
func announceKept(keeps func(...announce.State) error) error { return keeps(Kept...) }
@@ -67,6 +67,9 @@ func main() {
Source: kvSource{},
Owed: kvOwed{},
Pulls: kvPulls{},
// The gate says what stalled to the operator itself (issue 1275, ADR 0437 §9).
Say: sayOnSeat,
Conditions: kvConditions{},
}
h.init()
l := &listening{now: "not yet: starting"}
@@ -91,9 +91,9 @@ func TestItClaimsTheDeliverySeatAndFollowsWhatItHandles(t *testing.T) {
if !reflect.DeepEqual(m.Emits, []string{"transition", "group", "merged"}) {
t.Fatalf("emits %v", m.Emits)
}
// Seven buckets, and no `asked` (the teller asks, ADR 0437 §6): ADR 0450 §3.
// Eight buckets, and no `asked` (the teller asks, ADR 0437 §6): ADR 0450 §3, and the conditions it raised (issue 1275).
buckets := m.buckets()
if !reflect.DeepEqual(buckets, []string{"deliveries", "groups", "pulls", "asks", "merges", "owed", "source"}) {
if !reflect.DeepEqual(buckets, []string{"deliveries", "groups", "pulls", "asks", "merges", "owed", "source", "conditions"}) {
t.Fatalf("state %v", buckets)
}
// What it reads: the teller's answers and the checker's verdicts (issue 1302). The carry from today's module's buckets, the
@@ -102,10 +102,10 @@ func TestItClaimsTheDeliverySeatAndFollowsWhatItHandles(t *testing.T) {
t.Fatalf("reads %v", m.Reads)
}
// It calls no controller verb and reads the forge with three tools; the forge's writes are the bridge's (ADR 0437 §8). It
// uses no operator-channel: the teller asks (ADR 0437 §6).
// uses operator-channel only to say its conditions (ADR 0437 §9, issue 1275): the teller asks (ADR 0437 §6).
want := []string{"gitea.gitea_commit_statuses", "gitea.gitea_contains", "gitea.gitea_open_pulls"}
if len(m.Uses) != 0 {
t.Fatalf("uses %v: the teller asks, not the gate", m.Uses)
if !reflect.DeepEqual(m.Uses, []string{"operator-channel"}) {
t.Fatalf("uses %v: operator-channel, for its conditions alone", m.Uses)
}
if !reflect.DeepEqual(m.Invokes, want) {
t.Fatalf("invokes %v, wanted %v", m.Invokes, want)
@@ -57,7 +57,7 @@ func (h *Holder) Work() {
// React takes every step the facts allow, after an event that can change what they are: the forge's open pull requests
// read when a check may have answered, the steps that need another's word (look), what is owed (Flush), and then
// the time the next look is wanted at, if any step waits on the clock. Never concurrent with itself.
// the conditions of what stalled (conditions.go), and then the time the next look is wanted at, if any step waits on the clock. Never concurrent with itself.
func (h *Holder) React() {
h.reactMu.Lock()
defer h.reactMu.Unlock()
@@ -75,6 +75,7 @@ func (h *Holder) React() {
}
h.look()
h.Flush()
h.sayConditions()
h.arm()
}
@@ -29,7 +29,7 @@ import (
// - a group member only once the group's composed check passed for its members' heads as they are;
// - a push ends the Approve and says so once on the pull request (ADR 0421 §6); the new head is asked on its own;
// - a check that failed merges nothing: it is said once on the pull request, and as a stalled line, which the
// controller raises as the condition `delivery.<id>.stalled`. So is a merge the tool refused for an hour.
// gate says to the operator as a condition (conditions.go). So is a merge the tool refused for an hour.
//
// The review record is read and put in the ask; it is never a gate (ADR 0421 §5). What this owner does is kept on the
// delivery (Ready), said in its log, and the merge's transitions name the Approve as their warrant.
@@ -401,13 +401,14 @@ func (d *Delivery) holdsForReview() bool {
// groupTurnToMerge is why a group member may not merge yet; empty when it may: the group's composed check passed for
// its members' heads as they are and is not out of date. No member waits for another's merge or delivery (novox/hq
// ADR 0418 §4): once the composed check passed and the Approve covers the heads, every member is merged in one pass.
// A group part way through that pass is merging, and its next member's turn is then (issue 1275).
func (h *Holder) groupTurnToMerge(d *Delivery) string {
g := h.groups[d.Group]
if g == nil {
return ""
}
members := h.membersOf(g)
if state, why := GroupState(g, members); state != "ready" && state != "delivering" {
if state, why := GroupState(g, members); !groupMayMerge(state) {
return fmt.Sprintf("its group %s is %s: %s", g.ID, state, why)
}
if c := g.Check; c == nil || (c.Verdict != "pass" && c.Verdict != "warning") || c.OutOfDate != "" ||
@@ -417,6 +418,16 @@ func (h *Holder) groupTurnToMerge(d *Delivery) string {
return ""
}
// groupMayMerge is whether a group in this state hands its next member to the forge bridge: one ready, and one merging (a
// member merged, the pass goes on). Every other word holds the pass.
func groupMayMerge(state string) bool {
switch state {
case GroupReady, GroupMerging:
return true
}
return false
}
// sayOn owes a comment on a delivery's pull request: an owed act numbered among the comments on its head, written as a key
// with the delivery's other effects (owed.go). Under the lock; what it returns has nothing left to do.
func (h *Holder) sayOn(d *Delivery, body string) func() {
@@ -427,7 +438,7 @@ func (h *Holder) sayOn(d *Delivery, body string) func() {
}
// readyLines are the approved heads that will not merge: a check that failed at once, a merge refused past its bound;
// as stalled lines, which the controller raises as conditions. Under the lock.
// as stalled lines, which the gate says as conditions (conditions.go). Under the lock.
func (h *Holder) readyLines(now time.Time) []StalledLine {
var out []StalledLine
for _, d := range h.deliveries {
@@ -16,7 +16,7 @@ import (
// (novox/hq ADR 0406 §1, issue 975) — and once more at a head's bound, and one whose
// base requires the merge check — mesh/merge-gate or mesh/repo-check — with neither on its head for UncheckedAfter
// since this owner first saw it so is said as a delivery held: a `stalled` line in the state Unannounced, which the
// controller raises as `delivery.<id>.stalled` (D14) like any other delivery past its bound. The delivery domain's own
// gate says as a condition (conditions.go) like any other delivery past its bound. The delivery domain's own
// word, so no second condition and no second judge: the id is the one its delivery will have once announced.
//
// A head whose check started and never answered is said the same way (novox/hq issue 438). Counting any merge-check
@@ -129,7 +129,7 @@ func (h *Holder) Groups(all bool) []GroupLine {
continue
}
state, why, _ := h.groupState(g, members)
if !all && (state == "merged" || state == "closed") {
if !all && (state == GroupMerged || state == GroupClosed) {
continue
}
l := GroupLine{ID: g.ID, State: state, Why: why, ComposedGate: true}
@@ -185,7 +185,7 @@ type StalledLine struct {
// NeedsNewHead is what a stalled line needs when its cure is its author's new head (ADR 0505 §5).
const NeedsNewHead = "new-head"
// Stalled is the `stalled` verb: what the controller's self-check raises as `delivery.<id>.stalled`.
// Stalled is the `stalled` verb: what the gate says to the operator as conditions (conditions.go).
func (h *Holder) Stalled() []StalledLine {
h.mu.Lock()
defer h.mu.Unlock()
+5 -1
View File
@@ -33,6 +33,9 @@
"group",
"merged"
],
"uses": [
"operator-channel"
],
"invokes": [
"gitea.gitea_commit_statuses",
"gitea.gitea_contains",
@@ -57,7 +60,8 @@
{
"name": "source",
"history": 64
}
},
"conditions"
],
"reads": [
"mesh-delivery-teller.answers",