Compare commits

..
12 Commits
Author SHA1 Message Date
mesh-tools e4781edcd6 Merge pull request 'Design 47 H1: the Go packages other modules import are declared artifacts of their modules (H-A-186, H-A-188)' (#290) from fix/47-H1-declare-package-artifacts into main 2026-11-20 20:08:40 +00:00
jschoubben 1db38fcc72 Declare asks in the assigner's version series, and have the teller and the view accept it there (H-A-188)
The builder numbers every artifact of a release in its module's version, so the assigner's package asks is
published in 0.3, not the 0.1 eba4309b declared: asks now names series 0.3, and the teller and the view, which
import it, accept it at 0.3 in their manifests' packages. The other eight packages' modules are all at 0.1, so
their series stand. The gate in mesh-catalog, which imports asks too, follows with the catalog's next change.

Builds: design 47 G §370, H-A-186, H-A-188
2026-11-20 21:05:39 +01:00
jschoubben eba4309b19 Declare every Go package another module imports as a package artifact of its module (H-A-186)
mesh-build-go and other modules import packages of this repository that no manifest declared, so no release
published them and the registry holds no version: their importers stay unpinned and fail past the bound. Each
module now declares its imported packages in the form they had before issue 1966's bootstrap took them out:
asks (assigner), builds and buildrun (builder), forgemsg (forge bridge), takein (indexer), tellerevents
(teller), viewstate (view), grantsstate and busgrants (grants), each a Go package of series 0.1. Each
module's manifest test asserts its packages again.

Builds: design 47 G §370, H-A-186
2026-11-20 20:58:06 +01:00
mesh-tools a7fc37561e Merge pull request 'Design 47 H1: a pin is the version the registry published for the package path within the module's accepted series (H-A-183, H-A-184, H-A-185)' (#287) from fix/47-H1-pins-by-package-path into main 2026-11-20 19:15:04 +00:00
jschoubben e93575ffe7 mesh-delivery-indexer: the accepted series is the manifest's, contents or none
The pin fell back to the path's newest publish while no series value
held contents, which live none does, so a publish in a series a module
does not accept pinned it. The series a module accepts is the one its
manifest names, or the default; the pin is the newest version published
for the package path within it.

Builds: design 47 H H-A-185, H-A-184
2026-11-20 20:13:15 +01:00
jschoubben 859f88b2f0 mesh-delivery-indexer: a pin stays within the series the module accepts
A pin by the package path across every series moved a module to a
series it does not accept while its contents came from the accepted
one. The pin is the accepted series' newest published version where
that series gives the contents; only where none does yet is it the
newest published of the path. A publish in another series pins nothing
and moves nothing.

Builds: design 47 H H-A-184, H-A-183
2026-11-20 20:07:46 +01:00
jschoubben 517a4b3cf1 mesh-delivery-indexer: a pin is the version the registry published for the package's path
At main.39 every importer of a nested package was pinned to the release
number of the module holding it (v0.1.97 of the builder for its package
builds), and the Go seat found no tag <package dir>/v0.1.97. A Go
package's version is its tag: the pin is now the newest version the
registry published for that package path (the seats' published, kept in
the package's series), never a module's release; a package the
registry never published stays unpinned.

Builds: design 47 H H-A-183, H-A-167, G §370
2026-11-20 20:05:52 +01:00
mesh-tools 7f88916863 Merge pull request 'Design 47 H1: the forge bridge keeps only an act's outcome and pull request; the act stays in the gate's stream (H-A-181, H-A-182)' (#286) from fix/47-H1-bridge-performed-cap into main 2026-11-20 18:46:57 +00:00
jschoubben 5dee4cb263 mesh-delivery-forge-bridge: owed again refuses a performed value of the old form
A value kept before this change names no pull request, so bridge_owed_again read the subject owed..0 and said the
stream no longer held the act while it did. Every act the bridge decodes names its repository, so a value without
one is the old form: refused with what is true, never read again; it expires with the state's TTL.

Builds: design 47 G §163; H-A-182
2026-11-20 19:45:38 +01:00
jschoubben 77b683c1c1 mesh-delivery-forge-bridge: performed keeps the outcome and the pull request, not the act
The state performed filled to 77% of its template's 64 MiB (19,530 keys, 2.6 KB each): the bridge kept every owed
event whole beside its outcome. performed's value is now the outcome said as owed.ended, with when, and the
repository and number the act names (groups.Performed); bridge_owed_again reads the act from the gate's stream owed
on its pull request's subject through an ordered consumer, the stream being the act's one copy; bridge_owed reads
one act or one pull request's acts, never the whole state, whose answer had grown past what the bus carries.

Builds: design 47 G §163, §162, §168; H-A-181, H-A-182
2026-11-20 19:38:34 +01:00
mesh-tools 60e32772a5 Merge pull request 'Design 47 H1: the composition step of a module no machine holds ends nothing to run (H-A-179)' (#285) from fix/47-H1-composition-nothing-to-run into main 2026-11-20 18:08:15 +00:00
jschoubben fd2ef6c7e8 assigner: a mover no machine holds has nothing to compose (H-A-179)
A package module with a required composition step and no node placing
it ended could not run, so its pipeline could never pass. A step with
nothing to judge ends nothing to run, which counts as passed, saying why.

Builds: design 47 H H-A-179
2026-11-20 19:05:49 +01:00
29 changed files with 538 additions and 157 deletions
@@ -95,15 +95,17 @@ type machineEnd struct {
// run runs one composition step and answers what it came to: per machine not away it joins or starts the candidate's
// composition run (§718), waits for each run's end in its key, and judges each machine from it; failed when a machine does not
// compose, could not run when a machine could not be composed or was not judged, else passed with a warning per machine away.
// compose, could not run when a machine could not be composed or was not judged, nothing to run when no node places the module on
// the branch (H-A-179), else passed with a warning per machine away.
func (r *compositionRun) run(ctx context.Context, id string, step meshwants.StepRun) stepOutcome {
candidate, err := candidateKeyOf(step)
if err != nil {
return couldNotRun(err.Error())
}
nodes := r.inputs.machines(step.Module, step.Branch)
if len(nodes) == 0 {
return couldNotRun(fmt.Sprintf("no node places %s on the branch %q, so the composition step has no machine to compose", step.Module, step.Branch))
if len(nodes) == 0 { // a mover no machine holds on the branch has nothing to judge: nothing to run, passed (H-A-179, ADR 0682 §8)
return stepOutcome{Outcome: meshwants.StepEndedOutcomeNothingToRun,
Said: fmt.Sprintf("no node places %s on the branch %q, so the composition step has no machine to compose", step.Module, step.Branch)}
}
reader := readerOf(id, step)
prep := &runPrep{r: r, step: step}
@@ -259,7 +259,11 @@ func TestTheCompositionStepRunsOneAskPerMachine(t *testing.T) {
}
none := newRunStand(t, "src2:e1", "src2:e1", "d1")
none.r.inputs.machines = func(string, string) []string { return machines }
if out := none.r.run(context.Background(), "ask-step", step); out.Outcome != meshwants.StepEndedOutcomeCouldNotRun ||
outcome := meshwants.StepEndedOutcomeCouldNotRun
if len(machines) == 0 { // a mover no machine holds: nothing to run, saying why (H-A-179)
outcome = meshwants.StepEndedOutcomeNothingToRun
}
if out := none.r.run(context.Background(), "ask-step", step); out.Outcome != outcome ||
!strings.Contains(out.Said, want) || (len(machines) > 0 && !strings.Contains(out.Said, "shanks is away")) || len(none.asks) != 0 {
t.Errorf("%s came to %+v, %d asks", name, out, len(none.asks))
}
@@ -119,6 +119,13 @@ func TestManifest(t *testing.T) {
}
}
}
// H-A-186 (design 47 G §370): asks, which the gate, the teller and the view import, is a declared Go package of the
// module, so every release publishes it; H-A-188: in the module's version series, 0.3.
if !slices.ContainsFunc(m.Build.Artifacts, func(a catalogue.Artifact) bool {
return a.Name == "asks" && a.Kind == catalogue.ArtifactPackage && a.Language == "go" && slices.Equal(a.Paths, []string{"asks"}) && a.Series == m.Version && m.Version == "0.3"
}) {
t.Errorf("artifacts = %+v, want the Go package asks of series 0.3, the module's", m.Build.Artifacts)
}
// ADR 0635 §5: its views family is its ordinary events, `views.<state>` among its emits, on no stream of its own.
// The seat's four events are said on the seat's own subject (the seat's family), so the module keeps no stream for them
// and lists none among its emits.
@@ -370,6 +370,15 @@
"loads": [
"mesh-delivery-assigner"
]
},
{
"name": "asks",
"kind": "package",
"language": "go",
"paths": [
"asks"
],
"series": "0.3"
}
]
}
+10 -4
View File
@@ -266,10 +266,16 @@ func TestManifestProcess(t *testing.T) { // 1607/AC1
if _, bus := p["bus"]; bus {
t.Errorf("the process declares a bus: it runs on its own broker credential")
}
// Issue 1966's bootstrap: the package artifact builds is out of the manifest until the new builder runs; the follow-up
// puts it back with its assertion (ADR 0634 §1, ADR 0606 §4: three artifacts, the third the package builds).
if len(m.Build.Artifacts) != 2 {
t.Fatalf("artifacts = %+v, want the program and the tools", m.Build.Artifacts)
// H-A-186, design 47 G §370: a Go package another module imports is a declared package artifact of its module, so every
// release publishes it: builds (ADR 0634 §1, ADR 0606 §4), imported by the build modules and the node-engine, and
// buildrun, imported by the indexer, the view and the watchman.
if len(m.Build.Artifacts) != 4 {
t.Fatalf("artifacts = %+v, want the program, the tools and the packages builds and buildrun", m.Build.Artifacts)
}
for i, name := range []string{"builds", "buildrun"} {
if a := m.Build.Artifacts[2+i]; a.Name != name || a.Kind != "package" || a.Language != "go" || !reflect.DeepEqual(a.Paths, []string{name}) || a.Series != "0.1" {
t.Errorf("package artifact = %+v, want the Go package %s of series 0.1", a, name)
}
}
if a := m.Build.Artifacts[1]; a.Name != "tools" || a.Kind != "bundle" || a.Binary != "mesh-delivery-builder-tools" || !reflect.DeepEqual(a.Loads, []string{"mesh-delivery-builder-tools"}) {
t.Errorf("tools artifact = %+v", a) // 1945
+18
View File
@@ -334,6 +334,24 @@
"loads": [
"mesh-delivery-builder-tools"
]
},
{
"name": "builds",
"kind": "package",
"language": "go",
"paths": [
"builds"
],
"series": "0.1"
},
{
"name": "buildrun",
"kind": "package",
"language": "go",
"paths": [
"buildrun"
],
"series": "0.1"
}
]
}
@@ -141,7 +141,7 @@ func main() {
tools = append(tools, OwedTools(askState(owedAsks, groups.OwedAsksBucket(served), 25*time.Second, func(v []byte) bool {
a, err := groups.DecodeOwedAsk(v)
return err == nil && a.Done()
}), func() ([]OwedRow, error) { return performedRows(performed) }, func(what, why string) error {
}), func(f OwedFilter) ([]OwedRow, error) { return performedRows(performed, f) }, func(what, why string) error {
_, err := recorder.Record(meshhandactsclient.Request{What: what, Why: why, Cause: "owed-dropped"})
return err
}, time.Now)...)
@@ -94,17 +94,55 @@ func TestTheBridgeTools(t *testing.T) {
}
return json.Marshal(a)
}
rows := []OwedRow{{Of: seat.Seat, ID: "a", Outcome: forgemsg.OwedPerformed, At: at}}
rows := []OwedRow{
{OwedEnded: forgemsg.OwedEnded{Of: seat.Seat, ID: "a", Outcome: forgemsg.OwedPerformed, At: at}, Repository: "novox/mesh-sdk", Number: 178},
{OwedEnded: forgemsg.OwedEnded{Of: seat.Seat, ID: "b", Outcome: forgemsg.OwedPerformed, At: at}, Repository: "novox/mesh-sdk", Number: 179}}
var filters []OwedFilter
performed := func(f OwedFilter) ([]OwedRow, error) {
filters = append(filters, f)
var out []OwedRow
for _, r := range rows {
if f.holds(r) {
out = append(out, r)
}
}
return out, nil
}
var recorded []string
recordErr := error(nil)
record := func(what, why string) error { recorded = append(recorded, what+": "+why); return recordErr }
tools := map[string]func(map[string]any) (any, error){}
for _, tl := range OwedTools(ask, func() ([]OwedRow, error) { return rows, nil }, record, func() time.Time { return at }) {
for _, tl := range OwedTools(ask, performed, record, func() time.Time { return at }) {
tools[tl.Name] = tl.Run
}
out, err := tools[OwedTool](map[string]any{"of": seat.Seat})
if m, _ := out.(map[string]any); err != nil || len(m["performed"].([]OwedRow)) != 1 || len(m["backlog"].([]groups.Backlog)) != 1 {
t.Errorf("bridge_owed: %v %v", out, err)
// bridge_owed reads one act or one pull request's acts, never the whole state (H-A-181).
for _, c := range []struct {
name string
args map[string]any
ids []string
}{
{"one pull request", map[string]any{"repository": "novox/mesh-sdk", "number": float64(179)}, []string{"b"}},
{"one act", map[string]any{"of": seat.Seat, "id": "a"}, []string{"a"}},
{"the whole state", map[string]any{"of": seat.Seat}, nil},
{"a repository with no number", map[string]any{"repository": "novox/mesh-sdk"}, nil},
} {
n := len(filters)
out, err := tools[OwedTool](c.args)
if c.ids == nil {
if err == nil || len(filters) != n {
t.Errorf("bridge_owed of %s: %v %v, read %d; want refused, nothing read", c.name, out, err, len(filters)-n)
}
continue
}
m, _ := out.(map[string]any)
got, _ := m["performed"].([]OwedRow)
var ids []string
for _, r := range got {
ids = append(ids, r.ID)
}
if err != nil || !reflect.DeepEqual(ids, c.ids) || len(m["backlog"].([]groups.Backlog)) != 1 {
t.Errorf("bridge_owed of %s: %v %v, want %v", c.name, out, err, c.ids)
}
}
if out, err := tools[OwedAgainTool](map[string]any{"of": seat.Seat, "id": "a"}); err != nil || out.(map[string]any)["outcome"] != forgemsg.OwedPerformed ||
asked[len(asked)-1].Kind != groups.OwedAgain {
@@ -21,26 +21,50 @@ const (
OwedDropTool = "bridge_owed_drop"
)
// OwedRow is one act the bridge performed, as the state performed keeps it: the outcome said as owed.ended.
type OwedRow = forgemsg.OwedEnded
// OwedRow is one act the bridge performed, as the state performed keeps it: the outcome said as owed.ended, and the pull
// request the act names (groups.Performed).
type OwedRow struct {
forgemsg.OwedEnded
Repository string `json:"repository,omitempty"`
Number int `json:"number,omitempty"`
}
// performedRows reads every act the state performed holds; a key that is not an act (a group end's bookkeeping) is left.
func performedRows(s stdio.KeptState) ([]OwedRow, error) {
keys, err := s.Keys()
if err != nil {
return nil, fmt.Errorf("the bridge's performed state cannot be listed: %w", err)
// OwedFilter is what bridge_owed reads of performed: one act (its id, of narrowing it to one key) or one pull request's
// acts (repository and number), never the whole state (novox/hq design 47 H-A-181).
type OwedFilter struct {
Of, ID, Repository string
Number int
}
func (f OwedFilter) holds(r OwedRow) bool {
return (f.Of == "" || r.Of == f.Of) && (f.ID == "" || r.ID == f.ID) &&
(f.Repository == "" || (r.Repository == f.Repository && r.Number == f.Number))
}
// performedRows reads the acts of performed f names, newest first: one key when f names the seat and the id, else each
// key's value held to f. A key that is not an act (a group end's bookkeeping) is left.
func performedRows(s stdio.KeptState, f OwedFilter) ([]OwedRow, error) {
var keys []string
if f.Of != "" && f.ID != "" {
keys = []string{groups.PerformedKey(f.Of, f.ID)}
} else {
all, err := s.Keys()
if err != nil {
return nil, fmt.Errorf("the bridge's performed state cannot be listed: %w", err)
}
keys = all
}
var rows []OwedRow
rows := []OwedRow{}
for _, k := range keys {
e, err := s.Get(k)
if err != nil || e == nil {
continue
}
var p struct {
Ended OwedRow `json:"ended"`
}
var p groups.Performed
if json.Unmarshal(e.Value, &p) == nil && p.Ended.Of != "" {
rows = append(rows, p.Ended)
if r := (OwedRow{OwedEnded: p.Ended, Repository: p.Repository, Number: p.Number}); f.holds(r) {
rows = append(rows, r)
}
}
}
sort.Slice(rows, func(i, j int) bool { return rows[i].At.After(rows[j].At) })
@@ -50,7 +74,7 @@ func performedRows(s stdio.KeptState) ([]OwedRow, error) {
// OwedTools are bridge_owed, bridge_owed_again and bridge_owed_drop: ask writes an owed-asks value and waits for the
// bridge's answer; performed reads the state performed; record writes a hand act in the seat mesh-hand-acts' log before
// a drop is asked, so a drop is never made without its record.
func OwedTools(ask func(key string, value []byte) ([]byte, error), performed func() ([]OwedRow, error), record func(what, why string) error, now func() time.Time) []stdio.Tool {
func OwedTools(ask func(key string, value []byte) ([]byte, error), performed func(OwedFilter) ([]OwedRow, error), record func(what, why string) error, now func() time.Time) []stdio.Tool {
str := func(args map[string]any, k string) string { s, _ := args[k].(string); return strings.TrimSpace(s) }
act := map[string]any{
"of": map[string]any{"type": "string", "description": "the seat that said the act: mesh-delivery (the gate)"},
@@ -66,6 +90,12 @@ func OwedTools(ask func(key string, value []byte) ([]byte, error), performed fun
}
return groups.DecodeOwedAsk(v)
}
read := map[string]any{
"repository": map[string]any{"type": "string", "description": "the pull request's repository, as owner/name; with number"},
"number": map[string]any{"type": "integer", "description": "the pull request's number; with repository"}}
for k, v := range act {
read[k] = v
}
withWhy := map[string]any{"why": map[string]any{"type": "string", "description": "why it is dropped, in plain words"}}
for k, v := range act {
withWhy[k] = v
@@ -74,22 +104,24 @@ func OwedTools(ask func(key string, value []byte) ([]byte, error), performed fun
{Name: OwedTool,
Description: "What the forge bridge performed of what the gate and the checker owe the forge (design 47): each act it said as " +
"owed.ended, newest first, with its seat, id, pull request, outcome (performed, refused, could not be performed, " +
"undecodable, dropped), the forge's words and when, from its state performed (kept 7 days); and the backlog of its " +
"undecodable, dropped), the forge's words and when, from its state performed (kept 7 days), of one act (id, of " +
"narrowing it) or of one pull request (repository and number), never the whole state; and the backlog of its " +
"durable on each seat's stream owed: the last event applied, the events waiting and held, the age of the oldest. " +
"of or id narrow it. Read-only.",
Input: map[string]any{"type": "object", "properties": act},
"Read-only.",
Input: map[string]any{"type": "object", "properties": read},
Run: func(args map[string]any) (any, error) {
rows, err := performed()
f := OwedFilter{Of: str(args, "of"), ID: str(args, "id"), Repository: str(args, "repository")}
if n, ok := args["number"].(float64); ok {
f.Number = int(n)
}
if f.ID == "" && (f.Repository == "" || f.Number <= 0) {
return nil, fmt.Errorf("bridge_owed reads one act (id) or one pull request's acts (repository and number), never the whole state")
}
rows, err := performed(f)
if err != nil {
return nil, err
}
kept := []OwedRow{}
for _, r := range rows {
if (str(args, "of") == "" || r.Of == str(args, "of")) && (str(args, "id") == "" || r.ID == str(args, "id")) {
kept = append(kept, r)
}
}
out := map[string]any{"performed": kept}
out := map[string]any{"performed": rows}
if a, err := asked(groups.OwedBacklog, args); err != nil {
out["backlog-error"] = err.Error()
} else {
@@ -100,8 +132,8 @@ func OwedTools(ask func(key string, value []byte) ([]byte, error), performed fun
{Name: OwedAgainTool,
Description: "Perform one act owed to the forge again now, by hand, whatever its last outcome (a refusal the forge no longer " +
"makes, an act that could not be performed): the bridge calls the seat git's verb of the act under its id (safe to " +
"repeat) and says its outcome as owed.ended. The act is the one the bridge kept, or, for the gate's, the seat " +
"mesh-delivery's groups. Answers the outcome.",
"repeat) and says its outcome as owed.ended. The act is read from the stream owed of the seat that said it, on the " +
"subject of the pull request the bridge's state performed names (kept 7 days there). Answers the outcome.",
Input: map[string]any{"type": "object", "required": []string{"of", "id"}, "properties": act},
Run: func(args map[string]any) (any, error) {
a, err := asked(groups.OwedAgain, args)
@@ -0,0 +1,89 @@
package main
import (
"context"
"fmt"
"strconv"
"github.com/nats-io/nats.go/jetstream"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshdelivery"
"git.novox.be/novox/mesh-sdk/go/keys"
)
// The act bridge_owed_again performs again is read from the stream owed of the seat that said it, on the subject of the
// pull request performed names, the stream being the act's one copy (novox/hq design 47 G §162, §163, H-A-181).
// actReader reads the act the seat of said under id from its stream owed, on the subject of the pull request
// (repository and number) the act names, for decode to say each message's id; found false when the stream no longer
// holds it.
type actReader interface {
act(ctx context.Context, of, repository string, number int, id string, idOf func([]byte) string) (data []byte, found bool, err error)
}
// actOf is the act of to perform again, read from its seat's stream owed on the pull request p names.
func (o *owing) actOf(ctx context.Context, of string, p Performed, id string) ([]byte, error) {
read, err := readerOf(of)
if err != nil {
return nil, err
}
// A value of the form before H-A-182 names no pull request: every act the bridge decodes names its repository (a
// status may name no number, said on `.0` as the gate says it), so one with none is the old form, never read again.
if p.Repository == "" {
return nil, fmt.Errorf("performed names no pull request for the act %s of %s: kept in its old form, it expires with the state's TTL", id, of)
}
if o.acts == nil {
return nil, fmt.Errorf("the bridge reads no stream owed, so the act %s of %s cannot be performed again", id, of)
}
data, found, err := o.acts.act(ctx, of, p.Repository, p.Number, id, func(m []byte) string { c, _ := read(m); return c.ID })
if err != nil {
return nil, fmt.Errorf("the act %s of %s cannot be read from %s's stream owed: %w", id, of, of, err)
}
if !found {
return nil, fmt.Errorf("%s's stream owed no longer holds the act %s of %s#%d (kept 7 days)", of, id, p.Repository, p.Number)
}
return data, nil
}
// streamActs reads an act from the gate's stream owed (seat mesh-delivery's StreamOwed) by an ordered consumer filtered
// on the pull request's subject, as the gate says it (SubjectOwed with the repository escaped): every message the
// stream keeps there, the newest of the id kept.
type streamActs struct{ js jetstream.JetStream }
func (s streamActs) act(ctx context.Context, of, repository string, number int, id string, idOf func([]byte) string) ([]byte, bool, error) {
if of != seat.Seat {
return nil, false, fmt.Errorf("the seat %q keeps no stream owed the bridge reads", of)
}
subject := seat.SubjectOwed(keys.EscapeName(repository), strconv.Itoa(number))
st, err := s.js.Stream(ctx, seat.StreamOwed)
if err != nil {
return nil, false, err
}
info, err := st.Info(ctx, jetstream.WithSubjectFilter(subject))
if err != nil {
return nil, false, err
}
held := info.State.Subjects[subject]
if held == 0 {
return nil, false, nil
}
c, err := st.OrderedConsumer(ctx, jetstream.OrderedConsumerConfig{FilterSubjects: []string{subject}})
if err != nil {
return nil, false, err
}
var found []byte
for range held {
m, err := c.Next(jetstream.FetchContext(ctx))
if err != nil {
return nil, false, err
}
if idOf(m.Data()) == id {
found = m.Data()
}
if md, err := m.Metadata(); err == nil && md.NumPending == 0 {
break
}
}
return found, found != nil, nil
}
@@ -5,6 +5,7 @@ import (
"context"
"encoding/json"
"errors"
"fmt"
"log"
"strings"
"testing"
@@ -38,6 +39,24 @@ func (m memPerformed) Get(_ context.Context, key string) ([]byte, bool, error) {
}
func (m memPerformed) Put(_ context.Context, key string, v []byte) error { m[key] = v; return nil }
// memActs is a seat's stream owed by pull request subject (`<repository>#<number>`); asked records what was read.
type memActs struct {
held map[string][][]byte
asked []string
}
func (m *memActs) act(_ context.Context, of, repository string, number int, id string, idOf func([]byte) string) ([]byte, bool, error) {
pr := fmt.Sprintf("%s %s#%d", of, repository, number)
m.asked = append(m.asked, pr)
var found []byte
for _, d := range m.held[pr] {
if idOf(d) == id {
found = d
}
}
return found, found != nil, nil
}
// seatFake answers each seat verb from answers, by its key; one absent is not answered (an error, as a bound passed).
type seatFake struct {
calls []string
@@ -109,9 +128,12 @@ func TestWhatIsOwedIsPerformedSaidAndKept(t *testing.T) {
if v, ok := kept[performedKey(c.of, e.ID)]; ok != c.kept {
t.Errorf("%s: kept %v (%s), want %v", c.name, ok, v, c.kept)
} else if ok {
// The outcome and when, and the pull request the act names, nothing more (§163, H-A-181, H-A-182).
var p Performed
if err := json.Unmarshal(v, &p); err != nil || p.Ended.Outcome != c.outcome || !json.Valid(p.Owed) {
t.Errorf("%s: kept %s, want the outcome and the owed event", c.name, v)
var fields map[string]json.RawMessage
if err := json.Unmarshal(v, &p); err != nil || p.Ended.Outcome != c.outcome || p.Ended.At.IsZero() ||
p.Repository != "novox/mesh-sdk" || p.Number != 178 || json.Unmarshal(v, &fields) != nil || len(fields) != 3 {
t.Errorf("%s: kept %s, want the outcome, when, and the pull request novox/mesh-sdk#178, nothing more", c.name, v)
}
}
}
@@ -142,6 +164,9 @@ func TestWhatIsOwedIsPerformedSaidAndKept(t *testing.T) {
t.Run("asked again, dropped, and its backlog read by hand", func(t *testing.T) {
f := &seatFake{answers: statusOK}
o, pub, kept := newOwing(f)
// The act is read again from the gate's stream owed on its pull request's subject, never from performed (H-A-181).
acts := &memActs{held: map[string][][]byte{"mesh-delivery novox/mesh-sdk#178": {[]byte(`{`), gate("owed is said, a status")}}}
o.acts = acts
status := "status.novox/mesh-sdk#178.aaaaaaaaaaaa.mesh/merge-gate"
if _, err := o.heard(context.Background(), seat.Seat, gate("owed is said, a status")); err != nil {
t.Fatal(err)
@@ -153,11 +178,20 @@ func TestWhatIsOwedIsPerformedSaidAndKept(t *testing.T) {
calls, said int
outcome, answers string
}{
{"again, from performed", groups.OwedAsk{Kind: groups.OwedAgain, Of: seat.Seat, ID: status}, 2, 2, forgemsg.OwedPerformed, "performed"},
{"again, from the gate's stream owed", groups.OwedAsk{Kind: groups.OwedAgain, Of: seat.Seat, ID: status}, 2, 2, forgemsg.OwedPerformed, "performed"},
{"dropped", groups.OwedAsk{Kind: groups.OwedDrop, Of: seat.Seat, ID: status, Why: "set by hand", By: "lead@shanks"}, 2, 3, forgemsg.OwedDropped, "set by hand"},
{"again, an act nobody holds", groups.OwedAsk{Kind: groups.OwedAgain, Of: seat.Seat, ID: "x"}, 2, 3, "", "holds no owed act"},
{"the backlog", groups.OwedAsk{Kind: groups.OwedBacklog}, 2, 3, "", "backlog"},
{"again, the stream no longer holding it", groups.OwedAsk{Kind: groups.OwedAgain, Of: seat.Seat, ID: status}, 2, 3, "", "no longer holds"},
{"again, a value of the old form", groups.OwedAsk{Kind: groups.OwedAgain, Of: seat.Seat, ID: "old"}, 2, 3, "", "names no pull request"},
} {
switch c.name {
case "again, the stream no longer holding it":
acts.held = nil
case "again, a value of the old form":
// Kept before H-A-182: the outcome and the whole act, no pull request; the stream is not read for it.
kept[performedKey(seat.Seat, "old")] = []byte(`{"ended":{"v":1,"of":"mesh-delivery","id":"old","outcome":"performed","at":"2026-11-09T21:00:00Z"},"owed":{"id":"old"}}`)
}
a := o.answer(context.Background(), c.ask, backlog)
if !a.Done() || a.Outcome != c.outcome || !strings.Contains(a.Answer, c.answers) || len(f.calls) != c.calls || len(pub.data) != c.said {
t.Errorf("%s: %+v, called %v, said %d; want %q naming %q, %d calls, %d said", c.name, a, f.calls, len(pub.data), c.outcome, c.answers, c.calls, c.said)
@@ -167,8 +201,12 @@ func TestWhatIsOwedIsPerformedSaidAndKept(t *testing.T) {
}
}
var p Performed
if err := json.Unmarshal(kept[performedKey(seat.Seat, status)], &p); err != nil || p.Ended.Outcome != forgemsg.OwedDropped {
t.Errorf("performed after the drop: %+v %v", p, err)
if err := json.Unmarshal(kept[performedKey(seat.Seat, status)], &p); err != nil || p.Ended.Outcome != forgemsg.OwedDropped ||
p.Repository != "novox/mesh-sdk" || p.Number != 178 {
t.Errorf("performed after the drop: %+v %v; want dropped, its pull request kept", p, err)
}
if len(acts.asked) != 2 || acts.asked[0] != "mesh-delivery novox/mesh-sdk#178" {
t.Errorf("the stream owed was read on %v, want twice on the pull request performed names", acts.asked)
}
})
}
@@ -22,6 +22,7 @@ import (
git "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/git"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshdelivery"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-forge-bridge/forgemsg"
"git.novox.be/novox/mesh-sdk/go/keys"
"git.novox.be/novox/mesh-sdk/go/process"
"mesh-delivery-forge-bridge/internal/gitfixture"
@@ -33,7 +34,8 @@ import (
// no others, so a permission the bridge lacks fails a test here and not on the mesh (issue 1279).
//
// From module.json: the registrations and the gate's source-moved heard through its durable on EVENTS; branch-moved and
// what the gate owes heard through its durable on each seat's own stream (SEAT_<seat>_<event>); the seat git's verbs, the
// what the gate owes heard through its durable on each seat's own stream (SEAT_<seat>_<event>), and each seat's kept
// event also read through an ordered consumer of its own (mesh-grants heardGrants, A-233; bridge_owed_again, H-A-181); the seat git's verbs, the
// seat mesh-catalog's registrations, the seat mesh-delivery's source, the seat mesh-broker's show and the seat
// mesh-hand-acts' hand-act called on mesh.seat.<seat>.call.<verb>.<token> (or its plain fallback); its own states heads
// and owed-asks, and performed under the name its binding gives; its stream forge. `**` is written `>` on the wire. The
@@ -82,6 +84,11 @@ accounts {
"$JS.ACK.SEAT_mesh-delivery_owed.n1_mesh-delivery-forge-bridge.>"
"$JS.API.CONSUMER.INFO.SEAT_mesh-delivery_owed.n1_mesh-delivery-forge-bridge"
"$JS.API.CONSUMER.MSG.NEXT.SEAT_mesh-delivery_owed.n1_mesh-delivery-forge-bridge"
"$JS.API.CONSUMER.CREATE.SEAT_mesh-delivery_owed"
"$JS.API.CONSUMER.CREATE.SEAT_mesh-delivery_owed.>"
"$JS.API.CONSUMER.MSG.NEXT.SEAT_mesh-delivery_owed.*"
"$JS.API.CONSUMER.INFO.SEAT_mesh-delivery_owed.*"
"$JS.API.CONSUMER.DELETE.SEAT_mesh-delivery_owed.*"
"mesh.seat.mesh-broker.call.show.n1~mesh-delivery-forge-bridge"
"mesh.seat.mesh-broker.tool.show"
"mesh.seat.mesh-catalog.call.registrations.n1~mesh-delivery-forge-bridge"
@@ -518,7 +525,9 @@ func TestBridgePerformsWhatIsOwedOnce(t *testing.T) {
}
}
gateOwed := gitfixture.Seat(t, seat.Seat, "events", "owed is said, a status", "event:owed")[0]
subject := seat.SubjectOwed("novox~mesh-sdk", "178")
// On the pull request's subject as the gate says it (its repository escaped as a key part), where bridge_owed_again
// reads the act again (H-A-181).
subject := seat.SubjectOwed(keys.EscapeName("novox/mesh-sdk"), "178")
for range 2 {
if _, err := b.js.Publish(b.ctx, subject, gateOwed); err != nil {
t.Fatal(err)
@@ -597,12 +606,22 @@ func TestBridgePerformsWhatIsOwedOnce(t *testing.T) {
if a, err := groupkeys.DecodeOwedAsk(e.Value()); err == nil && a.Done() {
answered = true
if a.Outcome != forgemsg.OwedPerformed || count("git.status") != 3 {
t.Errorf("bridge_owed_again: %+v, status asked %d times; want performed again", a, count("git.status"))
t.Errorf("bridge_owed_again: %+v, status asked %d times; want performed again, read from the gate's stream owed", a, count("git.status"))
}
}
case <-bound:
t.Fatal("bridge_owed_again was not answered within 20 s")
}
}
// performed keeps the outcome, when and the pull request, never the act (§163, H-A-181, H-A-182).
kept, err := b.js.KeyValue(b.ctx, boundPerformed)
if err != nil {
t.Fatal(err)
}
if e, err := kept.Get(b.ctx, performedKey(seat.Seat, g.ID)); err != nil {
t.Errorf("performed of %s: %v", g.ID, err)
} else if fields := (map[string]json.RawMessage{}); json.Unmarshal(e.Value(), &fields) != nil || len(fields) != 3 || fields["owed"] != nil {
t.Errorf("performed of %s keeps %s, want the outcome, when and the pull request alone", g.ID, e.Value())
}
refusedRead(t, b, "mesh-delivery-gate_owed")
}
@@ -108,7 +108,7 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
}
performedKV := &boundBucket{binding: binding, local: performedLocal, js: js}
// What the gate owes the forge, performed through the seat git (design 47 *What the gate owes the forge*).
owe := &owing{call: call, pub: js, performed: kvPerformed{performedKV}, waits: binding.Waiting, now: time.Now, logger: logger}
owe := &owing{call: call, pub: js, performed: kvPerformed{performedKV}, acts: streamActs{js: js}, waits: binding.Waiting, now: time.Now, logger: logger}
keptCall := keptCallOf(ctx, call)
reader := &views.ProcessReader{Module: forgeBridgeModule(), Call: keptCall, Logf: logger.Printf, Hub: hub}
src := sourcebranches.New(sourcebranches.Call(keptCall))
@@ -2,7 +2,6 @@ package main
import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
@@ -22,22 +21,14 @@ import (
)
// The state performed (manifest `requires`, the template state, opened by its binding: bound.go) keeps one key per owed act it
// performed, the act's outcome and when, with the owed event as heard (novox/hq design 47 *What the gate owes the
// forge*). It is a saving and a read, never what keeps an act from happening twice: every act is safe to repeat at the
// forge.
// performed, in the form groups.Performed says (novox/hq design 47 G §163, H-A-181, H-A-182). It is a saving and a read,
// never what keeps an act from happening twice: every act is safe to repeat at the forge.
// performedKey is the key of one act in performed: the seat that said it and its id, the id in unpadded base64url, since
// an id may hold what a key may not (`#`, `/`).
func performedKey(of, id string) string {
return of + "." + base64.RawURLEncoding.EncodeToString([]byte(id))
}
// performedKey is the key of one act in performed (groups.PerformedKey).
func performedKey(of, id string) string { return groups.PerformedKey(of, id) }
// Performed is one value of the state performed: the outcome said as owed.ended, and the owed event it answers, so
// bridge_owed_again performs it again from here.
type Performed struct {
Ended forgemsg.OwedEnded `json:"ended"`
Owed json.RawMessage `json:"owed,omitempty"`
}
// Performed is one value of the state performed.
type Performed = groups.Performed
// performedStore is the state performed as the owing reads and writes it.
type performedStore interface {
@@ -70,6 +61,8 @@ type owing struct {
call caller
pub publisher
performed performedStore
// acts reads an act again from the stream owed of the seat that said it (H-A-181), for bridge_owed_again.
acts actReader
// waits, when set, says why the state performed cannot be used yet (its binding, design 47 G §577); empty when it can.
waits func() string
now func() time.Time
@@ -90,12 +83,12 @@ func readerOf(of string) (func([]byte) (bridge.OwedCall, error), error) {
func (o *owing) heard(ctx context.Context, of string, data []byte) (forgemsg.OwedEnded, error) {
read, err := readerOf(of)
if err != nil {
return o.end(ctx, forgemsg.OwedEnded{Of: of, Outcome: forgemsg.OwedUndecodable, Words: err.Error()}, data)
return o.end(ctx, forgemsg.OwedEnded{Of: of, Outcome: forgemsg.OwedUndecodable, Words: err.Error()}, "", 0)
}
c, err := read(data)
if err != nil {
return o.end(ctx, forgemsg.OwedEnded{Of: of, ID: c.ID,
Outcome: forgemsg.OwedUndecodable, Words: err.Error()}, data)
Outcome: forgemsg.OwedUndecodable, Words: err.Error()}, c.Repository, c.Number)
}
if w := o.waiting(); w != "" {
return forgemsg.OwedEnded{Of: c.Of, ID: c.ID}, fmt.Errorf("the bridge %s", w)
@@ -104,7 +97,7 @@ func (o *owing) heard(ctx context.Context, of string, data []byte) (forgemsg.Owe
// said twice, or said again while its outcome was on its way: performed once, the outcome said once
return forgemsg.OwedEnded{Of: c.Of, ID: c.ID}, nil
}
return o.perform(ctx, c, data)
return o.perform(ctx, c)
}
// hear is one message of a seat's stream owed: performed and acknowledged, or, when its outcome could not be said,
@@ -127,19 +120,20 @@ func (o *owing) waiting() string {
}
// perform calls the seat git's verb of one act and says what it came to.
func (o *owing) perform(ctx context.Context, c bridge.OwedCall, data []byte) (forgemsg.OwedEnded, error) {
func (o *owing) perform(ctx context.Context, c bridge.OwedCall) (forgemsg.OwedEnded, error) {
answer, callErr := o.call(ctx, "seat:"+git.Seat+"."+c.Verb, c.Input)
outcome, words := bridge.Outcome(answer, callErr)
if outcome != forgemsg.OwedPerformed {
o.logger.Printf("the owed %s %s of %s#%d (said by %s) is %s: %s", c.Act, c.ID, c.Repository, c.Number, c.Of, outcome, words)
}
return o.end(ctx, forgemsg.OwedEnded{Of: c.Of, ID: c.ID,
Outcome: outcome, Words: words}, data)
Outcome: outcome, Words: words}, c.Repository, c.Number)
}
// end says an act's outcome as owed.ended on the stream forge, then keeps it in performed. An outcome not kept is
// logged: the act may be performed again, which changes nothing at the forge.
func (o *owing) end(ctx context.Context, e forgemsg.OwedEnded, owed []byte) (forgemsg.OwedEnded, error) {
// end says an act's outcome as owed.ended on the stream forge, then keeps it in performed with the pull request the act
// names (repository and number). An outcome not kept is logged: the act may be performed again, which changes nothing at
// the forge.
func (o *owing) end(ctx context.Context, e forgemsg.OwedEnded, repository string, number int) (forgemsg.OwedEnded, error) {
e.At = o.now().UTC()
data, err := forgemsg.EncodeOwedEnded(e)
if err != nil {
@@ -151,10 +145,7 @@ func (o *owing) end(ctx context.Context, e forgemsg.OwedEnded, owed []byte) (for
if e.ID == "" {
return e, nil // an undecodable act with no id is said, and has no key to keep
}
if !json.Valid(owed) {
owed = nil
}
v, err := json.Marshal(Performed{Ended: e, Owed: owed})
v, err := json.Marshal(Performed{Ended: e, Repository: repository, Number: number})
if err == nil {
err = o.performed.Put(ctx, performedKey(e.Of, e.ID), v)
}
@@ -175,19 +166,17 @@ func (o *owing) held(ctx context.Context, of, id string) bool {
return found
}
// owedOf is the owed event of an act to perform again, as performed keeps it.
func (o *owing) owedOf(ctx context.Context, of, id string) ([]byte, error) {
// kept is what performed holds of an act: its outcome and its pull request.
func (o *owing) kept(ctx context.Context, of, id string) (Performed, error) {
v, found, err := o.performed.Get(ctx, performedKey(of, id))
if err != nil {
return nil, fmt.Errorf("performed cannot be read: %w", err)
return Performed{}, fmt.Errorf("performed cannot be read: %w", err)
}
if found {
var p Performed
if err := json.Unmarshal(v, &p); err == nil && len(p.Owed) > 0 {
return p.Owed, nil
}
var p Performed
if !found || json.Unmarshal(v, &p) != nil {
return Performed{}, fmt.Errorf("the bridge holds no owed act %s of %s in its state performed", id, of)
}
return nil, fmt.Errorf("the bridge holds no owed act %s of %s in its state performed", id, of)
return p, nil
}
// answer answers one owed ask: an act performed again now, whatever performed holds of it; an act dropped, said as
@@ -199,20 +188,22 @@ func (o *owing) answer(ctx context.Context, a groups.OwedAsk, backlog func(conte
a.Backlog, a.Answer = backlog(ctx), "the backlog of each seat's stream owed"
return a
}
data, err := o.owedOf(ctx, a.Of, a.ID)
p, err := o.kept(ctx, a.Of, a.ID)
if err != nil {
a.Answer = err.Error()
return a
}
var e forgemsg.OwedEnded
if a.Kind == groups.OwedDrop {
e, err = o.end(ctx, forgemsg.OwedEnded{Of: a.Of, ID: a.ID, Outcome: forgemsg.OwedDropped, Words: a.Why + " (dropped by " + a.By + ")"}, data)
e, err = o.end(ctx, forgemsg.OwedEnded{Of: a.Of, ID: a.ID, Outcome: forgemsg.OwedDropped, Words: a.Why + " (dropped by " + a.By + ")"}, p.Repository, p.Number)
} else if read, rerr := readerOf(a.Of); rerr != nil {
err = rerr
} else if data, rerr := o.actOf(ctx, a.Of, p, a.ID); rerr != nil {
err = rerr
} else if c, rerr := read(data); rerr != nil {
e, err = o.end(ctx, forgemsg.OwedEnded{Of: a.Of, ID: a.ID, Outcome: forgemsg.OwedUndecodable, Words: rerr.Error()}, data)
e, err = o.end(ctx, forgemsg.OwedEnded{Of: a.Of, ID: a.ID, Outcome: forgemsg.OwedUndecodable, Words: rerr.Error()}, p.Repository, p.Number)
} else {
e, err = o.perform(ctx, c, data)
e, err = o.perform(ctx, c)
}
if err != nil {
a.Answer = err.Error()
@@ -0,0 +1,26 @@
package groups
import (
"encoding/base64"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-forge-bridge/forgemsg"
)
// The state performed keeps one key per owed act the bridge performed, its outcome and when, and the pull request the
// act names, nothing more (novox/hq design 47 G §163, H-A-181, H-A-182): the act itself stays in the stream owed of the
// seat that said it, its one copy, where bridge_owed_again reads it again. The bridge writes it; its tools read it.
// PerformedKey is the key of one act in performed: the seat that said it and its id, the id in unpadded base64url, since
// an id may hold what a key may not (`#`, `/`).
func PerformedKey(of, id string) string {
return of + "." + base64.RawURLEncoding.EncodeToString([]byte(id))
}
// Performed is one value of the state performed (§163, H-A-182): the outcome said as owed.ended, with when, and the
// repository and number of the pull request the act names, so bridge_owed reads one pull request's acts and
// bridge_owed_again finds the act on its pull request's subject of the stream owed (H-A-181).
type Performed struct {
Ended forgemsg.OwedEnded `json:"ended"`
Repository string `json:"repository,omitempty"`
Number int `json:"number,omitempty"`
}
@@ -187,10 +187,13 @@ func TestManifestProcess(t *testing.T) { // 1235/AC1
if _, bus := p["bus"]; bus {
t.Errorf("the process declares a bus: it runs on its own broker credential")
}
// Issue 1966's bootstrap: the package artifact forgemsg is out of the manifest until the new builder runs; the follow-up
// puts it back with its assertion (ADR 0573 §1: three artifacts, the third the package forgemsg).
if len(m.Build.Artifacts) != 2 || m.Build.Artifacts[1].Name != "tools" || m.Build.Artifacts[1].From != "cmd/mesh-delivery-forge-bridge-tools" {
t.Fatalf("artifacts = %+v, want the program and the tools bundle", m.Build.Artifacts)
if len(m.Build.Artifacts) != 3 || m.Build.Artifacts[1].Name != "tools" || m.Build.Artifacts[1].From != "cmd/mesh-delivery-forge-bridge-tools" {
t.Fatalf("artifacts = %+v, want the program, the tools bundle and the package forgemsg", m.Build.Artifacts)
}
// ADR 0573 §1, H-A-186 (design 47 G §370): the gate, the builder and the indexer import forgemsg, so the module declares
// it as a package and every release publishes it.
if a := m.Build.Artifacts[2]; a.Name != "forgemsg" || a.Kind != "package" || a.Language != "go" || len(a.Paths) != 1 || a.Paths[0] != "forgemsg" || a.Series != "0.1" {
t.Errorf("package artifact = %+v", a)
}
a := m.Build.Artifacts[0]
if st, err := os.Stat(a.From); err != nil || !st.IsDir() || a.Binary != "mesh-delivery-forge-bridge" || !strings.HasSuffix(a.From, a.Binary) {
@@ -101,6 +101,15 @@
"loads": [
"mesh-delivery-forge-bridge-tools"
]
},
{
"name": "forgemsg",
"kind": "package",
"language": "go",
"paths": [
"forgemsg"
],
"series": "0.1"
}
]
}
@@ -123,9 +123,21 @@ func TestImportsReportedMoveTheFingerprintOnlyWithAnImportedPackage(t *testing.T
moved.Fp2, before.Fp2, moved.Inputs2.ImportForm, moved.Packages)
}
// mesh-sdk releases 0.1.20: the import form's fingerprint takes the pin too and moves, once (H-A-172).
// mesh-sdk's module releases 0.1.97 while the registry has published nothing of its Go package: the module's release
// number is never the package's pin (H-A-183), so avahi stays unpinned and where it was.
ix.Versions = newMem()
ix.Versions.(*memStore).put(meshcatalog.VersionsByModuleVersionKey("mesh-sdk", "0.1.20"), []byte(`{}`))
ix.Versions.(*memStore).put(meshcatalog.VersionsByModuleVersionKey("mesh-sdk", "0.1.97"), []byte(`{}`))
if err := ix.Recompute(ctx); err != nil {
t.Fatal(err)
}
if m, _ := moduleOf(t, mods, "avahi"); m.Fp2 != moved.Fp2 || m.Packages[meshSDK] != "" {
t.Fatalf("after mesh-sdk's module release alone: fp2 %q (was %q), versions %v; want it unpinned where it was", m.Fp2, moved.Fp2, m.Packages)
}
// The registry publishes its Go package at 0.1.20, another version than the module's release: the import form's
// fingerprint takes that pin and moves, once (H-A-172, H-A-183).
if _, err := ix.HearVersion(ctx, HeardVersion{Package: "mesh-sdk/go", Seat: "go-package-registry", Version: "0.1.20", Commit: c3, GoModule: meshSDK, Published: true}); err != nil {
t.Fatal(err)
}
for range 2 {
if err := ix.Recompute(ctx); err != nil {
t.Fatal(err)
@@ -283,17 +295,18 @@ func anAcceptedSeriesPinsItsNewest(t *testing.T) {
if pinned.Fp2 == before.Fp2 || pinned.Packages[meshSDK] != "v0.1.15" {
t.Fatalf("after 0.1.15: fp2 %q (was %q), versions %v; want moved by c3's announce, pinned to v0.1.15", pinned.Fp2, before.Fp2, pinned.Packages)
}
// A publish in a series avahi does not accept pins nothing and moves nothing (H-A-184).
if m := step("0.2.0 published from c1", hear("0.2.0", c1)); m.Fp2 != pinned.Fp2 || m.Packages[meshSDK] != "v0.1.15" {
t.Errorf("a publish of a series avahi does not accept moved it: %q %v", m.Fp2, m.Packages)
}
if m := step("trunk at c2", trunk(c2)); m.Fp2 != pinned.Fp2 || m.Packages[meshSDK] != "v0.1.15" {
t.Errorf("a trunk merge moved avahi off its accepted series' newest: %q %v", m.Fp2, m.Packages)
}
// mesh-sdk releases 0.1.20: an accepted series' older version never stands over a newer release (H-A-167).
// mesh-sdk's module releases 0.3.0: a module's release number is never a package's pin (H-A-183).
ix.Versions = newMem()
ix.Versions.(*memStore).put(meshcatalog.VersionsByModuleVersionKey("mesh-sdk", "0.1.20"), []byte(`{}`))
if m := step("0.1.20 released", func() error { return nil }); m.Packages[meshSDK] != "v0.1.20" {
t.Errorf("a newer release left avahi on its series' older version: %v", m.Packages)
ix.Versions.(*memStore).put(meshcatalog.VersionsByModuleVersionKey("mesh-sdk", "0.3.0"), []byte(`{}`))
if m := step("0.3.0 released", func() error { return nil }); m.Packages[meshSDK] != "v0.1.15" {
t.Errorf("a module's release pinned its package: %v", m.Packages)
}
}
@@ -218,8 +218,7 @@ func (ix *Indexer) recomputeLocked(ctx context.Context, overlay map[string]meshc
ix.logf("no fingerprint is written by this pass: %v", err)
return fmt.Errorf("no fingerprint is written: %w", err)
}
series := ix.SeriesAll(ctx) // each package's series, the newest a user accepting it takes (DLV-I6)
released := ix.releasedOf(ctx) // each module's newest released version, what a package is pinned to (H-A-161)
series := ix.SeriesAll(ctx) // each package's series, the newest a user accepting it takes (DLV-I6)
type result struct {
fp, none string
tcs map[string]meshcatalog.ToolchainUsed // by language, when there is a fingerprint
@@ -291,7 +290,7 @@ func (ix *Indexer) recomputeLocked(ctx context.Context, overlay map[string]meshc
r.none = strings.Join(missing, "; ")
case noToolchain != "":
r.none = noToolchain
if fp, pkgs, ok := stillComputed(module, kept, k, toolchains, held, pub, pins, ix.sourceHeads(), released); ok {
if fp, pkgs, ok := stillComputed(module, kept, k, toolchains, held, pub, pins, ix.sourceHeads(), series); ok {
r = result{fp: fp, tcs: k.held.Toolchains, pkgs: pkgs}
}
default:
@@ -299,7 +298,7 @@ func (ix *Indexer) recomputeLocked(ctx context.Context, overlay map[string]meshc
r.fp, r.none = f.fp, f.none
if r.fp != "" {
// a pin is a released version, never a commit (H-A-161), and the version line's fingerprint takes it (H-A-167)
r.fp, r.pkgs = pinnedOf(f, held, released)
r.fp, r.pkgs = pinnedOf(f, series, k.manifest)
r.tcs = toolchainsUsed(toolchains)
}
}
@@ -498,7 +497,7 @@ func hasContext(m catalogue.Manifest) bool {
// with the toolchains it was computed with in place of the languages agreed on none, give it again, in its form. Any
// other input that moved (a merge, a base, an agreed toolchain, an imported package) gives another fingerprint, and the
// module has none until they agree.
func stillComputed(module string, kept meshcatalog.ModulesByModuleValueInputs2, k *known, agreed map[string]meshcatalogclient.Toolchain2, held map[string]meshcatalog.ModulesByModuleValue, published map[string]meshcatalog.PackagesByModuleValue, pins sdkstep.PackageVersions, heads map[string]string, released map[string]string) (string, sdkstep.PackageVersions, bool) {
func stillComputed(module string, kept meshcatalog.ModulesByModuleValueInputs2, k *known, agreed map[string]meshcatalogclient.Toolchain2, held map[string]meshcatalog.ModulesByModuleValue, published map[string]meshcatalog.PackagesByModuleValue, pins sdkstep.PackageVersions, heads map[string]string, series map[string]SeriesPackage) (string, sdkstep.PackageVersions, bool) {
if k.held.Fp2 == "" {
return "", nil, false
}
@@ -518,7 +517,7 @@ func stillComputed(module string, kept meshcatalog.ModulesByModuleValueInputs2,
if f.none != "" || f.fp == "" {
return "", nil, false
}
fp, pkgs := pinnedOf(f, held, released)
fp, pkgs := pinnedOf(f, series, k.manifest)
return fp, pkgs, fp == k.held.Fp2
}
@@ -805,17 +805,34 @@ func TestAModuleKeepsTheToolchainsItsFingerprintWasComputedWith(t *testing.T) {
t.Fatal("a release whose re-pin failed was acknowledged")
}
mods.keysErr = nil
// The registry published mesh-sdk's Go package at 0.1.18, not at the module's release number 0.1.20, in the series
// avahi accepts (the default 0.1), with no contents read: the pin is that version (H-A-183, H-A-185).
ix.Packages = newMem()
if _, err := ix.HearVersion(context.Background(), HeardVersion{Package: "mesh-sdk/go", Seat: "go-package-registry", Version: "0.1.18", Commit: strings.Repeat("1", 40), GoModule: sdk, Published: true}); err != nil {
t.Fatal(err)
}
if err := ix.HearPackagePublished(context.Background(), release); err != nil {
t.Fatal(err)
}
ix.Revisions = revisions
if a, _ := moduleOf(t, mods, "avahi"); a.Fp2 == first.Fp2 || a.Fp2 == "" || !reflect.DeepEqual(a.Packages, map[string]string{sdk: "v0.1.20"}) {
t.Fatalf("after mesh-sdk's release: fp2 %q (was %q) packages %v, want a new fingerprint pinned to v0.1.20", a.Fp2, first.Fp2, a.Packages)
if a, _ := moduleOf(t, mods, "avahi"); a.Fp2 == first.Fp2 || a.Fp2 == "" || !reflect.DeepEqual(a.Packages, map[string]string{sdk: "v0.1.18"}) {
t.Fatalf("after mesh-sdk's release: fp2 %q (was %q) packages %v, want a new fingerprint pinned to v0.1.18", a.Fp2, first.Fp2, a.Packages)
} else {
// A 0.2.0 publish, of a series avahi does not accept, leaves it alone, contents or none (H-A-184, H-A-185).
if _, err := ix.HearVersion(context.Background(), HeardVersion{Package: "mesh-sdk/go", Seat: "go-package-registry", Version: "0.2.0", Commit: strings.Repeat("1", 40), GoModule: sdk, Published: true}); err != nil {
t.Fatal(err)
}
if _, err := ix.Repin(context.Background(), "mesh-sdk"); err != nil { // the pass for avahi alone: mesh-tools is held stale here
t.Fatal(err)
}
if b, _ := moduleOf(t, mods, "avahi"); b.Fp2 != a.Fp2 || b.Packages[sdk] != "v0.1.18" {
t.Fatalf("a publish of another series moved avahi: fp2 %q (was %q) packages %v", b.Fp2, a.Fp2, b.Packages)
}
}
if o, _ := moduleOf(t, mods, "mesh-tools"); o.Fp2 != tools.Fp2 {
t.Errorf("a re-pin moved mesh-tools, which holds no package of mesh-sdk: %q (was %q)", o.Fp2, tools.Fp2)
}
second := sdkMerge(strings.Repeat("2", 40), "tree-sdk-2", "v0.1.20")
second := sdkMerge(strings.Repeat("2", 40), "tree-sdk-2", "v0.1.18")
if second.Fp2 == first.Fp2 {
t.Errorf("a mesh-sdk trunk merge kept avahi's fingerprint %q", first.Fp2)
}
@@ -16,6 +16,7 @@ import (
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
sdkstep "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/buildstep"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/keys"
"mesh-delivery-indexer/internal/fingerprint"
@@ -366,51 +367,28 @@ func isFloor(raw []byte) bool {
return json.Unmarshal(raw, &v) == nil && len(v.Artifacts) == 0
}
// releasedOf is the newest released version of each module the state versions keeps (§349; design 47 H-A-161), by module;
// nil when the indexer keeps no such state or it cannot be listed, said.
func (ix *Indexer) releasedOf(ctx context.Context) map[string]string {
if ix.Versions == nil {
return nil
}
all, err := ix.Versions.Keys(ctx)
if err != nil {
ix.logf("versions: the state cannot be listed, so no package is pinned by this pass: %v", err)
return nil
}
out := map[string]string{}
for _, k := range all {
m, v, ok := meshcatalog.ParseVersionsByModuleVersionKey(k)
if _, parts := versionParts(v); !ok || !parts {
continue
}
if cur, has := out[m]; !has || compareVersions(v, cur) > 0 {
out[m] = v
}
}
return out
}
// releasedPins are a module's pins (its catalog entry's packages, which the builder builds it against) from the versions its
// fingerprint was computed with (design 47 H-A-161, G §370): a package the held module holding it has released is pinned
// to that module's newest released version, as Go names it (v<version>); a version already in that form (an accepted
// series' newest, DLV-I6) stands; any other, a commit, is no pin: its package is kept with no version, so its kind-1 edge
// stands (H §746) and no build is made against a commit. nil for none.
func releasedPins(versions sdkstep.PackageVersions, held map[string]meshcatalog.ModulesByModuleValue, released map[string]string) sdkstep.PackageVersions {
// fingerprint was computed with (design 47 H-A-161, H-A-183, H-A-184, H-A-185, G §370): each package is pinned to the
// newest version the registry published for its path (the seats' published, kept in the package's series by HearVersion)
// within the series the module accepts, the one its manifest m names or the default (Manifest.AcceptedSeries), whether
// or not that series holds contents yet; a publish in another series pins nothing and moves nothing, and a module's
// release number is never a pin. A package with none published in that series is kept with no version, so its kind-1
// edge stands (H §746) and its build waits for the publish (H-A-165). nil for none.
func releasedPins(versions sdkstep.PackageVersions, series map[string]SeriesPackage, m catalogue.Manifest) sdkstep.PackageVersions {
if len(versions) == 0 {
return nil
}
out := sdkstep.PackageVersions{}
for p, v := range versions {
rel := released[moduleOfKey(fingerprint.HolderOf(p, held))]
series := sdkstep.PackageSemver.MatchString(v)
_, plain := versionParts(strings.TrimPrefix(v, "v"))
switch {
case rel != "" && (!series || !plain || compareVersions(rel, strings.TrimPrefix(v, "v")) > 0):
out[p] = "v" + rel // never an older series' version over a newer release (H-A-167)
case series:
out[p] = v
default:
out[p] = ""
for p := range versions {
out[p] = ""
for _, s := range series {
if s.GoModule != p || s.Newest == nil || s.Series != m.AcceptedSeries(s.Name) {
continue
}
v := strings.TrimPrefix(s.Newest.Version, "v")
if _, ok := versionParts(v); ok && (out[p] == "" || compareVersions(v, strings.TrimPrefix(out[p], "v")) > 0) {
out[p] = "v" + v
}
}
}
return out
@@ -420,8 +398,8 @@ func releasedPins(versions sdkstep.PackageVersions, held map[string]meshcatalog.
// form's fingerprint takes them (design 47 H-A-167, H-A-172; ADR 0573 §4) and a re-pin moves it, the module a mover
// built again against the release; a fingerprint with no pin is the form's as it is. The held fingerprint of a build that
// stands is kept while its pins are the ones it was built with, else its import form takes the new pins.
func pinnedOf(f form, held map[string]meshcatalog.ModulesByModuleValue, released map[string]string) (string, sdkstep.PackageVersions) {
pins := releasedPins(f.versions, held, released)
func pinnedOf(f form, series map[string]SeriesPackage, m catalogue.Manifest) (string, sdkstep.PackageVersions) {
pins := releasedPins(f.versions, series, m)
if f.stands {
if maps.Equal(f.built, map[string]string(pins)) {
return f.fp, pins
@@ -42,6 +42,8 @@ type artifact struct {
Binary string `json:"binary"`
Loads []string `json:"loads"`
Env map[string]string `json:"env"`
Paths []string `json:"paths"`
Series string `json:"series"`
}
type stream struct {
@@ -344,8 +346,12 @@ func TestManifestProcess(t *testing.T) { // 1238/AC1
if _, bus := p["bus"]; bus {
t.Errorf("the process declares a bus: it runs on its own broker credential")
}
if len(m.Build.Artifacts) != 2 {
t.Fatalf("artifacts = %+v, want the program and the tools", m.Build.Artifacts)
if len(m.Build.Artifacts) != 3 {
t.Fatalf("artifacts = %+v, want the program, the tools and the package takein", m.Build.Artifacts)
}
// H-A-186 (design 47 G §370): takein, which the gate, the view and the watchman import, is a declared package.
if a := m.Build.Artifacts[2]; a.Name != "takein" || a.Kind != "package" || a.Language != "go" || !reflect.DeepEqual(a.Paths, []string{"takein"}) || a.Series != "0.1" {
t.Errorf("package artifact = %+v", a)
}
if a := m.Build.Artifacts[1]; a.Name != "tools" || a.Kind != "bundle" || a.Binary != "mesh-delivery-indexer-tools" || !reflect.DeepEqual(a.Loads, []string{"mesh-delivery-indexer-tools"}) {
t.Errorf("tools artifact = %+v", a) // 1945
@@ -223,6 +223,15 @@
"loads": [
"mesh-delivery-indexer-tools"
]
},
{
"name": "takein",
"kind": "package",
"language": "go",
"paths": [
"takein"
],
"series": "0.1"
}
]
}
+13 -2
View File
@@ -38,9 +38,12 @@ type artifact struct {
From string `json:"from"`
Binary string `json:"binary"`
Loads []string `json:"loads"`
Paths []string `json:"paths"`
Series string `json:"series"`
}
type manifest struct {
Packages map[string]string `json:"packages"`
Module string `json:"module"`
Version string `json:"version"`
Slug string `json:"slug"`
@@ -176,8 +179,16 @@ func TestManifestProcess(t *testing.T) { // 1240/AC1
if _, bus := p["bus"]; bus {
t.Errorf("the process declares a bus: it runs on its own broker credential")
}
if len(m.Build.Artifacts) != 2 {
t.Fatalf("artifacts = %+v, want the program and the tools", m.Build.Artifacts)
if len(m.Build.Artifacts) != 3 {
t.Fatalf("artifacts = %+v, want the program, the tools and the package tellerevents", m.Build.Artifacts)
}
// H-A-188: the assigner is at 0.3, so its package asks, which this module imports, is accepted in that series.
if got := m.Packages["git.novox.be/novox/mesh-delivery/modules/mesh-delivery-assigner/asks"]; got != "0.3" || len(m.Packages) != 1 {
t.Errorf("packages = %v, want asks accepted in the assigner's series 0.3", m.Packages)
}
// H-A-186 (design 47 G §370): tellerevents, which the assigner imports, is a declared package.
if a := m.Build.Artifacts[2]; a.Name != "tellerevents" || a.Kind != "package" || a.Language != "go" || !reflect.DeepEqual(a.Paths, []string{"tellerevents"}) || a.Series != "0.1" {
t.Errorf("package artifact = %+v", a)
}
a := m.Build.Artifacts[0]
if st, err := os.Stat(a.From); err != nil || !st.IsDir() || a.Binary != "mesh-delivery-teller" || !strings.HasSuffix(a.From, a.Binary) {
+12
View File
@@ -1,6 +1,9 @@
{
"module": "mesh-delivery-teller",
"version": "0.1",
"packages": {
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-assigner/asks": "0.3"
},
"slug": "teller",
"tools": [
"teller_end_ask",
@@ -123,6 +126,15 @@
"loads": [
"mesh-delivery-teller-tools"
]
},
{
"name": "tellerevents",
"kind": "package",
"language": "go",
"paths": [
"tellerevents"
],
"series": "0.1"
}
]
},
+7 -3
View File
@@ -70,6 +70,7 @@ type claim struct {
}
type manifest struct {
Packages map[string]string `json:"packages"`
Claims []claim `json:"claims"`
State []state `json:"state"`
Module string `json:"module"`
@@ -313,9 +314,12 @@ func TestManifestIsTakenAndReadsOnly(t *testing.T) { // 1134/AC7
func TestManifestIsWholeInItsRepository(t *testing.T) { // 1134/AC7
m, _ := load(t)
// Issue 1966's bootstrap: the package artifact viewstate is out of the manifest until the new builder runs; the
// follow-up puts the count back at 5.
if len(m.Build.Artifacts) != 4 {
// H-A-188: the assigner is at 0.3, so its package asks, which this module imports, is accepted in that series.
if got := m.Packages["git.novox.be/novox/mesh-delivery/modules/mesh-delivery-assigner/asks"]; got != "0.3" || len(m.Packages) != 1 {
t.Errorf("packages = %v, want asks accepted in the assigner's series 0.3", m.Packages)
}
// H-A-186 (design 47 G §370): viewstate, which the gate, the assigner and the teller import, is a declared package.
if len(m.Build.Artifacts) != 5 || m.Build.Artifacts[4].Name != "viewstate" {
t.Fatalf("artifacts = %+v", m.Build.Artifacts)
}
for _, a := range m.Build.Artifacts {
+12
View File
@@ -1,6 +1,9 @@
{
"module": "mesh-delivery-view",
"version": "0.1",
"packages": {
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-assigner/asks": "0.3"
},
"moved": true,
"slug": "view",
"tools": [
@@ -234,6 +237,15 @@
"name": "files",
"kind": "archive",
"from": "files"
},
{
"name": "viewstate",
"kind": "package",
"language": "go",
"paths": [
"viewstate"
],
"series": "0.1"
}
]
},
+11
View File
@@ -648,6 +648,17 @@ func TestGrants(t *testing.T) {
if _, ok := m.OwnSecrets["broker"]; !ok || len(m.OwnSecrets) != 1 || m.SecretsOwner != "mesh-grants" {
t.Errorf("own-secrets %v owner %q: a credential of its own, broker", m.OwnSecrets, m.SecretsOwner)
}
// H-A-186 (design 47 G §370): grantsstate (imported by the assigner, the vault and the node-engine) and busgrants
// (imported by the bus module and the watchman) are declared Go packages of the module, so every release publishes them.
var packages []string
for _, a := range m.Build.Artifacts {
if a.Kind == catalogue.ArtifactPackage && a.Language == "go" && len(a.Paths) == 1 && a.Paths[0] == a.Name && a.Series == "0.1" {
packages = append(packages, a.Name)
}
}
if !slices.Equal(packages, []string{"grantsstate", "busgrants"}) {
t.Errorf("Go packages %v, want grantsstate and busgrants of series 0.1", packages)
}
})
t.Run("no timer, no update or delete, a publish only of conditions, a create only of declared streams", func(t *testing.T) {
+18
View File
@@ -185,6 +185,24 @@
"loads": [
"mesh-grants"
]
},
{
"name": "grantsstate",
"kind": "package",
"language": "go",
"paths": [
"grantsstate"
],
"series": "0.1"
},
{
"name": "busgrants",
"kind": "package",
"language": "go",
"paths": [
"busgrants"
],
"series": "0.1"
}
]
}