Compare commits

...
18 Commits
Author SHA1 Message Date
mesh-admin 1513bbaac9 Merge pull request 'An older merge does not move a source' (#120) from fix/an-older-merge-does-not-move-a-source into main 2026-09-28 03:12:40 +00:00
jschoubben 0014984116 An older merge does not move a source
The forge announces what it finds merged, and an old merge surfacing late moved the recorded head
backwards and rebuilt everything built from that repository, once per old merge. A merge made
before the source was last seen is history; one that says nothing about when is taken as news.
The catalogue now carries when each source was last seen. The bus-records test follows #116:
a module's tools are every one under its own name.
2026-09-28 05:12:37 +02:00
mesh-admin d6e49dbd68 Merge pull request 'A base the registry already holds is not pulled from upstream again' (#119) from fix/a-mirrored-base-is-not-pulled-twice into main 2026-09-28 02:48:35 +00:00
jschoubben 3756bb3460 A base the registry already holds is not pulled from upstream again
A base is named by digest, and a digest the mesh's registry holds under the module's repository
is the same bytes whatever upstream would say. Asked on every build, the public hub's anonymous
pull limit was reached on the first merge that rebuilt a whole catalogue, and every module whose
base lives there failed on a copy it did not need.
2026-09-28 04:48:32 +02:00
mesh-admin 60be9c5360 Merge pull request 'A module hears what it consumes: its consumer is raised with the bus, and it pulls it' (#118) from fix/a-module-hears-what-it-consumes into main 2026-09-28 02:29:34 +00:00
jschoubben da31bcb11e A module hears what it consumes: its consumer is raised with the bus, and it pulls it
Every module moved onto the bus by the rollout was issued on the old one, so none had a consumer
waiting; and the grant named a push delivery a runtime's client never binds, while the pull it
does make — asking about its consumer, asking it for messages — was refused. The consumers a
module's declarations imply are now raised whenever the bus is, and the grant is the pull.
2026-09-28 04:29:32 +02:00
mesh-admin f03e7b33c9 Merge pull request 'The controller may ask any module's tool' (#117) from fix/the-controller-may-ask-a-tool into main 2026-09-28 02:21:26 +00:00
jschoubben 0baf727f36 The controller may ask any module's tool
The control plane is the way in for tool calls (novox/hq ADR 0095): a person or an agent asks
through it, so it alone may publish to every module's tool subject. The first ask on the new bus
was refused the publish.
2026-09-28 04:21:25 +02:00
mesh-admin 94dd49a968 Merge pull request 'A module serves every tool under its own name, and may answer' (#116) from fix/a-module-serves-its-own-namespace into main 2026-09-28 02:14:52 +00:00
jschoubben 83a298e7e0 A module serves every tool under its own name, and may answer
Every module that served a tool was refused the subscription on the new bus: the grant listed
tools from a manifest field no module fills, because the tools a module serves are what its code
answers and a second copy of that list would be a second source of truth. The grant is now the
module's own tool namespace; nothing else may subscribe it, a caller is still granted per tool by
name, and a module may answer what it was asked.
2026-09-28 04:14:47 +02:00
mesh-admin 220b79f5cd Merge pull request 'rollout check dials the bus the way the mesh does' (#115) from fix/the-check-dials-as-the-mesh-does into main 2026-09-28 02:05:35 +00:00
jschoubben e62201e227 rollout check dials the bus the way the mesh does
The probe connected bare, and a bus that requires TLS and a user refused it at the handshake —
so the check reported the standing server as absent. It now dials with the controller's own
credential and pin, which is the one fact the check is there to report.
2026-09-28 04:05:32 +02:00
mesh-admin 0ab9b86f0a Merge pull request 'An edge is recorded by path, like the artifact it points at' (#114) from fix/an-edge-is-recorded-by-path into main 2026-09-28 01:52:10 +00:00
jschoubben c7aabd3037 An edge is recorded by path, like the artifact it points at
The test pinned what a build stood on to the address the builder pulled from; the edge names
another module's artifact and is kept the way that artifact is (novox/hq 04-ISSUES/102).
2026-09-28 03:52:08 +02:00
mesh-admin c74d990cee Merge pull request 'A build records the bases it was handed, and the mesh reads its edges from builds' (#113) from feat/build-edges-are-recorded into main 2026-09-28 01:51:16 +00:00
jschoubben 35252af665 A build records the bases it was handed, and the mesh reads its edges from builds
Bases reach a recipe as build arguments, so the digest was never in the file the builder read
edges from: no build on the mesh recorded what it stood on, and 'build --on', the bases-first
order and the merge follow-up all walked a graph with no edges (novox/hq 04-ISSUES/131). The
builder now reports every base it resolved; the controller records them by artifact path and
reads the newest build's edges from the store, since a recorded manifest carries no build.on.
2026-09-28 03:51:14 +02:00
mesh-admin aab6ded41b Merge pull request 'One bus: the AMQP transport is gone from the controller' (#112) from feat/one-bus into main 2026-09-28 01:36:27 +00:00
mesh-admin 30362118a1 Merge pull request 'The controller follows the subject it decodes' (#111) from fix/the-controller-follows-what-it-decodes into main 2026-09-28 01:15:18 +00:00
17 changed files with 412 additions and 108 deletions
+7 -4
View File
@@ -17,8 +17,8 @@ import (
var aDigest = "sha256:" + strings.Repeat("e", 64) var aDigest = "sha256:" + strings.Repeat("e", 64)
// **A build is recorded by digest and path**, whatever address the builder pushed to — and only // **A build is recorded by digest and path**, whatever address the builder pushed to — what it
// what the build made is rewritten: an image the module runs from elsewhere is left where it says. // made and what it stood on both; an image the module runs from elsewhere is left where it says.
func TestABuildIsRecordedWithoutTheStoresAddress(t *testing.T) { func TestABuildIsRecordedWithoutTheStoresAddress(t *testing.T) {
manifest, _ := json.Marshal(map[string]any{ manifest, _ := json.Marshal(map[string]any{
"module": "gitea", "version": "1", "module": "gitea", "version": "1",
@@ -65,8 +65,11 @@ func TestABuildIsRecordedWithoutTheStoresAddress(t *testing.T) {
if strings.Contains(string(kept.Manifest), "anchor.internal:5100") { if strings.Contains(string(kept.Manifest), "anchor.internal:5100") {
t.Errorf("the recorded manifest still carries the store's address:\n%s", kept.Manifest) t.Errorf("the recorded manifest still carries the store's address:\n%s", kept.Manifest)
} }
if kept.Against[0] != "anchor.internal:5100/mesh-tools/runtime@"+aDigest { // What the build stood on is an edge to another module's artifact, and it is recorded the way
t.Errorf("what the build stood on was rewritten: %v", kept.Against) // that artifact is: by path in the store, so the edge still names the same thing when the
// store answers at another address.
if kept.Against[0] != catalogue.ArtifactStoreScheme+"mesh-tools/runtime@"+aDigest {
t.Errorf("what the build stood on was recorded by address: %v", kept.Against)
} }
} }
+18 -12
View File
@@ -42,23 +42,21 @@ func buildOn(ctx context.Context, base string, wait time.Duration) error {
if err != nil { if err != nil {
return err return err
} }
against, err := open.inventory.BuiltAgainst(ctx)
if err != nil {
return err
}
var on []inventory.Entry var on []inventory.Entry
for _, e := range held { for _, e := range held {
if e.Manifest.Build == nil { if standsOnModule(e, base, against) {
continue
}
for _, b := range e.Manifest.Build.On {
if standsOnModule(b, base) {
on = append(on, e) on = append(on, e)
break
}
} }
} }
if len(on) == 0 { if len(on) == 0 {
fmt.Printf("nothing the mesh holds stands on %s\n", base) fmt.Printf("nothing the mesh holds stands on %s\n", base)
return nil return nil
} }
on = orderByBases(on) on = orderByBases(on, against)
fmt.Printf("%d module(s) stand on %s:\n", len(on), base) fmt.Printf("%d module(s) stand on %s:\n", len(on), base)
var failed []string var failed []string
for _, e := range on { for _, e := range on {
@@ -134,8 +132,9 @@ func buildCommand(ctx context.Context, args []string) error {
// //
// **By digest and path, never by where it was pushed** (novox/hq 04-ISSUES/102). The builder // **By digest and path, never by where it was pushed** (novox/hq 04-ISSUES/102). The builder
// says `<registry>:<port>/<module>/<artifact>@sha256:…`; the mesh records the artifact-store // says `<registry>:<port>/<module>/<artifact>@sha256:…`; the mesh records the artifact-store
// reference and composes the store's address back in where a reference is used. `against` is kept // reference and composes the store's address back in where a reference is used. `against` — what
// as announced: it is what the build stood on as the builder saw it, and the catalogue's edge. // the build stood on, the catalogue's edge — is recorded the same way, so an edge names a module's
// artifact and not the machine it was pulled from.
func buildFrom(result link.BuildResult) inventory.Build { func buildFrom(result link.BuildResult) inventory.Build {
kept := inventory.Build{ kept := inventory.Build{
ID: result.ID, Repository: result.Repository, Ref: result.Ref, ID: result.ID, Repository: result.Repository, Ref: result.Ref,
@@ -145,7 +144,10 @@ func buildFrom(result link.BuildResult) inventory.Build {
// edges, and it is not always listening when a build happens — on a fresh mesh it cannot // edges, and it is not always listening when a build happens — on a fresh mesh it cannot
// be, for exactly the modules it needs most. Keeping them is what makes a replay able to // be, for exactly the modules it needs most. Keeping them is what makes a replay able to
// rebuild the graph rather than a list of names. // rebuild the graph rather than a list of names.
Path: result.Path, Against: result.Against, Path: result.Path,
}
for _, ref := range result.Against {
kept.Against = append(kept.Against, catalogue.Recorded(ref))
} }
var announced []inventory.Artifact var announced []inventory.Artifact
for _, made := range result.Made { for _, made := range result.Made {
@@ -344,7 +346,11 @@ func buildBehind(ctx context.Context, wait time.Duration) error {
// Bases first: a module built before the module it stands on is built against the old one // Bases first: a module built before the module it stands on is built against the old one
// and reports success (novox/hq 04-ISSUES/131). // and reports success (novox/hq 04-ISSUES/131).
stale = orderByBases(stale) against, err := inv.BuiltAgainst(ctx)
if err != nil {
return err
}
stale = orderByBases(stale, against)
var failed []string var failed []string
for _, e := range stale { for _, e := range stale {
+53 -8
View File
@@ -1,26 +1,37 @@
package main package main
import ( import (
"strings"
"testing" "testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue" "github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link" "github.com/novox/mesh-controller/internal/link"
) )
func entry(module string, on ...string) inventory.Entry { // entry is a module as the catalogue holds it: built, so its manifest carries no `build` any more.
b := &catalogue.Build{} func entry(module string, _ ...string) inventory.Entry {
for _, o := range on { return inventory.Entry{Manifest: catalogue.Manifest{Module: module}}
b.On = append(b.On, catalogue.BuildsOn{Arg: "X", Module: o, Artifact: "runtime"})
} }
return inventory.Entry{Manifest: catalogue.Manifest{Module: module, Build: b}}
// stoodOn is what each module's newest build recorded it was handed.
func stoodOn(edges map[string][]string) map[string][]string {
out := map[string][]string{}
for module, bases := range edges {
for _, b := range bases {
out[module] = append(out[module], catalogue.ArtifactStoreScheme+b+"/runtime@sha256:"+strings.Repeat("0", 64))
}
}
return out
} }
// A module built before the module it stands on is built against the old one and reports success // A module built before the module it stands on is built against the old one and reports success
// (novox/hq 04-ISSUES/131). So bases come first, however the set arrived. // (novox/hq 04-ISSUES/131). So bases come first, however the set arrived.
func TestBasesAreBuiltBeforeWhatStandsOnThem(t *testing.T) { func TestBasesAreBuiltBeforeWhatStandsOnThem(t *testing.T) {
in := []inventory.Entry{entry("app", "runtime"), entry("runtime", "base"), entry("other"), entry("base")} in := []inventory.Entry{entry("app"), entry("runtime"), entry("other"), entry("base")}
got := orderByBases(in) edges := stoodOn(map[string][]string{"app": {"runtime"}, "runtime": {"base"}})
got := orderByBases(in, edges)
pos := map[string]int{} pos := map[string]int{}
for i, e := range got { for i, e := range got {
pos[e.Manifest.Module] = i pos[e.Manifest.Module] = i
@@ -32,12 +43,31 @@ func TestBasesAreBuiltBeforeWhatStandsOnThem(t *testing.T) {
t.Fatalf("an entry was lost or doubled: %d", len(got)) t.Fatalf("an entry was lost or doubled: %d", len(got))
} }
// A base outside the set is not waited for: it is not being rebuilt. // A base outside the set is not waited for: it is not being rebuilt.
got = orderByBases([]inventory.Entry{entry("app", "elsewhere")}) got = orderByBases([]inventory.Entry{entry("app")}, stoodOn(map[string][]string{"app": {"elsewhere"}}))
if len(got) != 1 { if len(got) != 1 {
t.Fatalf("a dependency outside the set changed the set: %v", got) t.Fatalf("a dependency outside the set changed the set: %v", got)
} }
} }
// A module registered from its manifest and never built still names its bases there; once built,
// the recorded edge is what says so. Both are read, and a module never stands on itself.
func TestWhatStandsOnAModuleIsReadFromItsBuildOrItsManifest(t *testing.T) {
built := entry("gitea")
edges := stoodOn(map[string][]string{"gitea": {"mesh-tools"}})
if !standsOnModule(built, "mesh-tools", edges) {
t.Fatal("a recorded edge was not read")
}
if standsOnModule(built, "gitea", edges) || standsOnModule(built, "postgres", edges) {
t.Fatal("an edge was invented")
}
fresh := inventory.Entry{Manifest: catalogue.Manifest{Module: "plex", Build: &catalogue.Build{
On: []catalogue.BuildsOn{{Arg: "RUNTIME_BASE", Module: "mesh-tools", Artifact: "runtime"}},
}}}
if !standsOnModule(fresh, "mesh-tools", nil) {
t.Fatal("a manifest's own base was not read")
}
}
// A merge names a repository the way the forge does; a source is recorded the way a build was // A merge names a repository the way the forge does; a source is recorded the way a build was
// asked for. The two meet on owner/repo and branch, whichever form the record took. // asked for. The two meet on owner/repo and branch, whichever form the record took.
func TestAMergeMatchesTheSourcesBuiltFromIt(t *testing.T) { func TestAMergeMatchesTheSourcesBuiltFromIt(t *testing.T) {
@@ -61,3 +91,18 @@ func TestAMergeMatchesTheSourcesBuiltFromIt(t *testing.T) {
} }
} }
} }
// A merge made before the source was last seen is history: it does not move the source, and a
// merge that says nothing about when it was made is taken as news.
func TestAMergeOlderThanTheLastLookIsHistory(t *testing.T) {
seen := time.Date(2026, 9, 28, 3, 0, 0, 0, time.UTC)
if !isHistory("2026-09-28T02:00:00Z", seen) {
t.Fatal("an older merge was taken as news")
}
if isHistory("2026-09-28T04:00:00Z", seen) {
t.Fatal("a newer merge was taken as history")
}
if isHistory("", seen) || isHistory("2026-09-28T02:00:00Z", time.Time{}) {
t.Fatal("a merge or a source with no time on it was refused")
}
}
+25 -2
View File
@@ -753,8 +753,31 @@ func raiseTheBus(ctx context.Context, inv *inventory.Inventory, address string)
if err := broker.RaiseSeats(js, inventory.MeshSeats(), holders); err != nil { if err := broker.RaiseSeats(js, inventory.MeshSeats(), holders); err != nil {
return err return err
} }
fmt.Printf("the bus at %s has its streams, and %d machine(s) can hear a declaration\n", // And how every module hears what it consumes. Derived from the same records the user list is
broker.BareAddress(address), len(names)) // composed from, so a module the mesh grants a consumer's subjects has that consumer waiting.
// Done on every raise, not only when a credential is issued: every module moved onto this bus
// by the rollout was issued on the old one, and came up with nothing to bind to (2026-09-28).
records, err := inv.BusRecords(ctx)
if err != nil {
return err
}
users, err := broker.Users(records)
if err != nil {
return err
}
hearing := 0
for _, p := range users {
consumer, needed := broker.ConsumerFor(p)
if !needed {
continue
}
if err := js.EnsureConsumer(consumer); err != nil {
return fmt.Errorf("how %s on %s hears what it consumes: %w", p.Module, p.Node, err)
}
hearing++
}
fmt.Printf("the bus at %s has its streams, %d machine(s) can hear a declaration, and %d module(s) "+
"can hear what they consume\n", broker.BareAddress(address), len(names), hearing)
return nil return nil
} }
+6 -2
View File
@@ -127,9 +127,13 @@ func readinessOf(ctx context.Context, inv *inventory.Inventory) (broker.Readines
if address != "" { if address != "" {
// One dial, briefly. "Is it answering" is the one fact records cannot hold, and a mesh about // One dial, briefly. "Is it answering" is the one fact records cannot hold, and a mesh about
// to move onto a server that is not there should hear it here rather than afterwards. // to move onto a server that is not there should hear it here rather than afterwards.
if conn, err := nats.Connect(broker.BareAddress(address), nats.Timeout(5*time.Second)); err == nil { //
// **Dialled the way the mesh dials it** — credential and pin — because a bare connect to a
// bus that requires TLS and a user fails at the handshake, and the check then reported a
// standing server as absent (seen live, 2026-09-28).
if js, err := broker.Dial(address, nats.Timeout(5*time.Second)); err == nil {
state.ServerStanding = true state.ServerStanding = true
conn.Close() js.Close()
} }
} }
+54 -23
View File
@@ -240,6 +240,13 @@ func (f following) SourceMoved(ctx context.Context, m link.SourceMoved) error {
if e.Source.BuiltFrom == m.Commit { if e.Source.BuiltFrom == m.Commit {
continue continue
} }
// **A merge older than the last look at the source is history, not a move.** The forge
// announces what it finds merged, and an old merge surfacing late would otherwise move the
// recorded head backwards and rebuild everything built from that repository, once per old
// merge (2026-09-28).
if isHistory(m.MergedAt, e.Source.Seen) {
continue
}
if err := inv.SourceMoved(ctx, e.Manifest.Module, m.Commit); err != nil { if err := inv.SourceMoved(ctx, e.Manifest.Module, m.Commit); err != nil {
return notNow(err) return notNow(err)
} }
@@ -250,7 +257,11 @@ func (f following) SourceMoved(ctx context.Context, m link.SourceMoved) error {
m.Owner, m.Repo, m.Base, m.Commit) m.Owner, m.Repo, m.Base, m.Commit)
return nil return nil
} }
ordered := orderByBases(moved) against, err := inv.BuiltAgainst(ctx)
if err != nil {
return notNow(err)
}
ordered := orderByBases(moved, against)
names := make([]string, 0, len(ordered)) names := make([]string, 0, len(ordered))
for _, e := range ordered { for _, e := range ordered {
names = append(names, e.Manifest.Module) names = append(names, e.Manifest.Module)
@@ -265,7 +276,7 @@ func (f following) SourceMoved(ctx context.Context, m link.SourceMoved) error {
failed = append(failed, e.Manifest.Module) failed = append(failed, e.Manifest.Module)
// A base that failed is a reason to stop: what stands on it would be built against // A base that failed is a reason to stop: what stands on it would be built against
// the old one, and report success (novox/hq 04-ISSUES/131). // the old one, and report success (novox/hq 04-ISSUES/131).
if standsOn(ordered, e.Manifest.Module) { if standsOn(ordered, e.Manifest.Module, against) {
fmt.Printf(" stopping: %s is a base of what was still to build\n", e.Manifest.Module) fmt.Printf(" stopping: %s is a base of what was still to build\n", e.Manifest.Module)
break break
} }
@@ -292,10 +303,13 @@ func sourceIs(s inventory.Source, m link.SourceMoved) bool {
return s.Ref == "" || s.Ref == m.Base return s.Ref == "" || s.Ref == m.Base
} }
// orderByBases is the entries with every base before what stands on it: a module whose build names // orderByBases is the entries with every base before what stands on it: a module whose build stood
// another's artifact under build.on comes after that module. Entries outside the set are not // on another's artifact comes after that module. Entries outside the set are not waited for — they
// waited for — they are not being rebuilt. Stable for what has no order between it. // are not being rebuilt. Stable for what has no order between it.
func orderByBases(entries []inventory.Entry) []inventory.Entry { //
// `against` is what each module's newest build stood on (inventory.BuiltAgainst): the edges are
// derived from builds, not declared, because a recorded manifest no longer carries `build.on`.
func orderByBases(entries []inventory.Entry, against map[string][]string) []inventory.Entry {
inSet := map[string]bool{} inSet := map[string]bool{}
for _, e := range entries { for _, e := range entries {
inSet[e.Manifest.Module] = true inSet[e.Manifest.Module] = true
@@ -309,15 +323,11 @@ func orderByBases(entries []inventory.Entry) []inventory.Entry {
return return
} }
seen[name] = true seen[name] = true
if e.Manifest.Build != nil {
for _, on := range e.Manifest.Build.On {
for _, base := range entries { for _, base := range entries {
if base.Manifest.Module != name && inSet[base.Manifest.Module] && standsOnModule(on, base.Manifest.Module) { if base.Manifest.Module != name && inSet[base.Manifest.Module] && standsOnModule(e, base.Manifest.Module, against) {
place(base, seen) place(base, seen)
} }
} }
}
}
placed[name] = true placed[name] = true
out = append(out, e) out = append(out, e)
} }
@@ -328,26 +338,47 @@ func orderByBases(entries []inventory.Entry) []inventory.Entry {
} }
// standsOn is whether anything in the set is built on the named module's artifacts. // standsOn is whether anything in the set is built on the named module's artifacts.
func standsOn(entries []inventory.Entry, module string) bool { func standsOn(entries []inventory.Entry, module string, against map[string][]string) bool {
for _, e := range entries { for _, e := range entries {
if e.Manifest.Build == nil { if standsOnModule(e, module, against) {
continue
}
for _, on := range e.Manifest.Build.On {
if standsOnModule(on, module) {
return true return true
} }
} }
}
return false return false
} }
// standsOnModule is whether a base names the module: as written in a manifest (`module`), or as // standsOnModule is whether an entry's build stood on the named module: by what its newest build
// recorded after a build, when the mesh has replaced it with the artifact it resolved to // recorded it was handed (`artifact-store://<module>/<artifact>@…`, the module's own artifact), or
// (`artifact-store://<module>/<artifact>@…`). A recorded manifest is what the catalogue holds. // — for a module registered from a manifest and not yet built — by the base its manifest names.
func standsOnModule(on catalogue.BuildsOn, module string) bool { func standsOnModule(e inventory.Entry, module string, against map[string][]string) bool {
if e.Manifest.Module == module {
return false
}
if e.Manifest.Build != nil {
for _, on := range e.Manifest.Build.On {
if on.Module == module { if on.Module == module {
return true return true
} }
return strings.HasPrefix(on.Image, "artifact-store://"+module+"/") }
}
prefix := catalogue.ArtifactStoreScheme + module + "/"
for _, ref := range against[e.Manifest.Module] {
if strings.HasPrefix(ref, prefix) {
return true
}
}
return false
}
// isHistory is whether a merge made at mergedAt predates the last time the source was seen. A merge
// with no time on it is taken as news: refusing it would silence a forge that says less.
func isHistory(mergedAt string, seen time.Time) bool {
if mergedAt == "" || seen.IsZero() {
return false
}
at, err := time.Parse(time.RFC3339, mergedAt)
if err != nil {
return false
}
return at.Before(seen)
} }
+27 -10
View File
@@ -181,6 +181,11 @@ func PermissionsFor(p Principal) (Permissions, error) {
for _, seat := range meshSeatsTheControllerUses { for _, seat := range meshSeatsTheControllerUses {
pub = append(pub, "mesh.seat."+seat+".accept.>") pub = append(pub, "mesh.seat."+seat+".accept.>")
} }
// Every module's tools: **the control plane is the way in** (novox/hq ADR 0095). A person
// or an agent asks through it and every question passes one process where an audit
// belongs — so it, alone among principals, may call any tool by name. The first `ask` on
// the new bus was refused the publish (2026-09-28).
pub = append(pub, "mesh.mod.*.tool.>")
// The two events it reacts to, and its ack subject on the stream they arrive from // The two events it reacts to, and its ack subject on the stream they arrive from
// (streams.go). **Each named, not a pattern**: `mesh.mod.*.event.>` would make the // (streams.go). **Each named, not a pattern**: `mesh.mod.*.event.>` would make the
@@ -266,9 +271,13 @@ func PermissionsFor(p Principal) (Permissions, error) {
for _, e := range p.Emits { for _, e := range p.Emits {
pub = append(pub, own+".event."+e) pub = append(pub, own+".event."+e)
} }
for _, t := range p.Serves { // Every tool under its own name, not a list: the tools a module serves are what its code
sub = append(sub, own+".tool."+t) // answers, and a second copy of that list in the manifest would be a second source of
} // truth for the mesh to keep in step (2026-09-28: every module that served a tool was
// refused the subscription, because none had written the list twice). Nothing is given
// away — no other principal may subscribe this namespace, and a caller's authority is
// still granted per tool, by name, on the publish side.
sub = append(sub, own+".tool.>")
// 2. What it consumes, by the emitter's own subject — an event is addressed to its // 2. What it consumes, by the emitter's own subject — an event is addressed to its
// emitter, because the emitter's identity is the meaning (ADR 0118). // emitter, because the emitter's identity is the meaning (ADR 0118).
@@ -288,11 +297,18 @@ func PermissionsFor(p Principal) (Permissions, error) {
} }
} }
// 2c. Its own consumer, which it **pulls**: the runtime asks for the next message and is
// answered on its own inbox, so what it needs is to ask about the consumer and to ask it
// for messages — its own consumer's name, and no other's. Pulled rather than pushed
// because that is the one shape a runtime's client binds without creating anything; the
// controller and the hosts are pushed to. Named here rather than through ConsumerFor,
// which asks for these permissions to build the consumer and would ask forever. A
// subject for a consumer that turns out not to exist grants nothing anybody can use.
pub = append(pub,
"$JS.API.CONSUMER.INFO."+consumerStream(p)+"."+consumerDurable(p),
"$JS.API.CONSUMER.MSG.NEXT."+consumerStream(p)+"."+consumerDurable(p))
// 3. Seats it holds: full participation. // 3. Seats it holds: full participation.
// Its consumer's name, not ConsumerFor: that asks for these permissions to build the
// consumer, and would ask forever. A subject for a consumer that turns out not to exist
// grants nothing anybody can use.
sub = append(sub, "_DELIVER."+consumerDurable(p))
for _, s := range p.Holds { for _, s := range p.Holds {
// Taking work from the role's queue: the worker consumer it binds (asked about, // Taking work from the role's queue: the worker consumer it binds (asked about,
// delivered on, acknowledged), each on the seat's own stream. The first machine to // delivered on, acknowledged), each on the seat's own stream. The first machine to
@@ -348,9 +364,10 @@ func PermissionsFor(p Principal) (Permissions, error) {
return Permissions{ return Permissions{
Publish: pub, Publish: pub,
Subscribe: sub, Subscribe: sub,
// Only something that serves is ever answering. A pure consumer is granted nothing here. // A module answers what it was asked — a tool call reaches it on its own namespace, so the
AllowResponses: p.Kind == KindModule && (len(p.Serves) > 0 || len(p.Holds) > 0) || // authority is bounded by having been asked — and so does the controller. A node and a
p.Kind == KindController, // person are never asked anything, and are granted nothing here.
AllowResponses: p.Kind == KindModule || p.Kind == KindController,
}, nil }, nil
} }
+45 -10
View File
@@ -1,6 +1,7 @@
package broker package broker
import ( import (
"slices"
"strings" "strings"
"testing" "testing"
) )
@@ -89,17 +90,30 @@ func TestAnInboxIsScopedToItsOwner(t *testing.T) {
} }
// A responder answers on the caller's inbox, which it has no permission for. allow_responses is // A responder answers on the caller's inbox, which it has no permission for. allow_responses is
// what makes a scoped inbox workable at all — the authority is bounded by having been asked. // what makes a scoped inbox workable at all — the authority is bounded by having been asked. A
func TestOnlySomethingThatServesMayAnswer(t *testing.T) { // module is asked on its own namespace and may answer; a node and a person are never asked.
serving, _ := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "billing", func TestOnlyWhatCanBeAskedMayAnswer(t *testing.T) {
Serves: []string{"status"}, PasswordHash: "x"}) module, _ := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "audit",
if !serving.AllowResponses {
t.Fatal("a module serving a tool cannot answer the caller's inbox")
}
consumer, _ := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "audit",
Consumes: []string{"shop.order.placed"}, PasswordHash: "x"}) Consumes: []string{"shop.order.placed"}, PasswordHash: "x"})
if consumer.AllowResponses { if !module.AllowResponses {
t.Fatal("a pure consumer was granted the right to answer, which nothing asked it to do") t.Fatal("a module cannot answer a tool call on its own namespace")
}
node, _ := PermissionsFor(Principal{Kind: KindNode, Node: "one", PasswordHash: "x"})
if node.AllowResponses {
t.Fatal("a node was granted the right to answer, and nothing asks a node anything")
}
}
// A module serves every tool under its own name, and no other module's.
func TestAModuleServesItsOwnNamespaceAndNoOthers(t *testing.T) {
p, _ := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "gitea", PasswordHash: "x"})
if !slices.Contains(p.Subscribe, "mesh.mod.gitea.tool.>") {
t.Fatalf("a module may not serve its own tools: %v", p.Subscribe)
}
for _, s := range p.Subscribe {
if strings.HasPrefix(s, "mesh.mod.") && !strings.HasPrefix(s, "mesh.mod.gitea.") {
t.Fatalf("a module may subscribe another's namespace: %s", s)
}
} }
} }
@@ -324,3 +338,24 @@ func admits(pattern, subject []string) bool {
} }
return len(pattern) == len(subject) return len(pattern) == len(subject)
} }
// A module pulls its own consumer — asks about it, asks it for messages — and no other module's.
func TestAModulePullsItsOwnConsumerAndNoOthers(t *testing.T) {
p, _ := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "audit",
Consumes: []string{"shop.order.placed"}, PasswordHash: "x"})
for _, want := range []string{"$JS.API.CONSUMER.INFO.EVENTS.one_audit", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_audit"} {
if !slices.Contains(p.Publish, want) {
t.Errorf("a module cannot bind its own consumer: %v lacks %s", p.Publish, want)
}
}
for _, s := range p.Publish {
if strings.Contains(s, "CONSUMER.") && !strings.HasSuffix(s, ".one_audit") {
t.Errorf("a module may reach another consumer: %s", s)
}
}
for _, s := range p.Subscribe {
if strings.HasPrefix(s, "_DELIVER.") {
t.Errorf("a module is granted a push delivery it never binds: %s", s)
}
}
}
+9 -7
View File
@@ -24,7 +24,7 @@ accounts {
jetstream: enabled jetstream: enabled
users = [ users = [
{ user: "controller", password: "$2a$11$cccccccccccccccccccccc", permissions: { { user: "controller", password: "$2a$11$cccccccccccccccccccccc", permissions: {
publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "_INBOX.enrol.>", "mesh.control.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>"] } publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "_INBOX.enrol.>", "mesh.control.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>"] }
subscribe: { allow: ["$JS.API.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.gitea.event.pull.merged", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built"] } subscribe: { allow: ["$JS.API.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.gitea.event.pull.merged", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built"] }
allow_responses: { max: 1, ttl: "1m" } allow_responses: { max: 1, ttl: "1m" }
} } } }
@@ -37,17 +37,19 @@ accounts {
subscribe: { allow: ["_DELIVER.one", "_INBOX.node.one.>", "mesh.node.one.declare"] } subscribe: { allow: ["_DELIVER.one", "_INBOX.node.one.>", "mesh.node.one.declare"] }
} } } }
{ user: "one.telegram", password: "$2a$11$tttttttttttttttttttttt", permissions: { { user: "one.telegram", password: "$2a$11$tttttttttttttttttttttt", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.one_telegram.>", "$JS.ACK.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker.>", "$JS.API.CONSUMER.INFO.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "mesh.seat.telegram-sender.event.delivered", "mesh.seat.telegram-sender.event.failed"] } publish: { allow: ["$JS.ACK.EVENTS.one_telegram.>", "$JS.ACK.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker.>", "$JS.API.CONSUMER.INFO.EVENTS.one_telegram", "$JS.API.CONSUMER.INFO.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_telegram", "mesh.seat.telegram-sender.event.delivered", "mesh.seat.telegram-sender.event.failed"] }
subscribe: { allow: ["_DELIVER.SEAT_TELEGRAM_SENDER_worker", "_DELIVER.one_telegram", "_INBOX.one.telegram.>", "mesh.mod.telegram.tool.status", "mesh.seat.telegram-sender.accept.send"] } subscribe: { allow: ["_DELIVER.SEAT_TELEGRAM_SENDER_worker", "_INBOX.one.telegram.>", "mesh.mod.telegram.tool.>", "mesh.seat.telegram-sender.accept.send"] }
allow_responses: { max: 1, ttl: "1m" } allow_responses: { max: 1, ttl: "1m" }
} } } }
{ user: "two.audit", password: "$2a$11$aaaaaaaaaaaaaaaaaaaaaa", permissions: { { user: "two.audit", password: "$2a$11$aaaaaaaaaaaaaaaaaaaaaa", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_audit.>"] } publish: { allow: ["$JS.ACK.EVENTS.two_audit.>", "$JS.API.CONSUMER.INFO.EVENTS.two_audit", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_audit"] }
subscribe: { allow: ["_DELIVER.two_audit", "_INBOX.two.audit.>", "mesh.mod.shop.event.order.placed"] } subscribe: { allow: ["_INBOX.two.audit.>", "mesh.mod.audit.tool.>", "mesh.mod.shop.event.order.placed"] }
allow_responses: { max: 1, ttl: "1m" }
} } } }
{ user: "two.shop", password: "$2a$11$ssssssssssssssssssssss", permissions: { { user: "two.shop", password: "$2a$11$ssssssssssssssssssssss", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_shop.>", "mesh.mod.shop.event.order.placed", "mesh.seat.telegram-sender.accept.send"] } publish: { allow: ["$JS.ACK.EVENTS.two_shop.>", "$JS.API.CONSUMER.INFO.EVENTS.two_shop", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_shop", "mesh.mod.shop.event.order.placed", "mesh.seat.telegram-sender.accept.send"] }
subscribe: { allow: ["_DELIVER.two_shop", "_INBOX.two.shop.>"] } subscribe: { allow: ["_INBOX.two.shop.>", "mesh.mod.shop.tool.>"] }
allow_responses: { max: 1, ttl: "1m" }
} } } }
] ]
} }
+34 -15
View File
@@ -169,6 +169,8 @@ func Build(ctx context.Context, run Runner, publish Publisher,
} }
var built []catalogue.Built var built []catalogue.Built
// stoodOn is every base the build was handed, as resolved — the edges the catalogue derives.
var stoodOn []string
if manifest.Build != nil { if manifest.Build != nil {
// What this module said it stands on, answered with what this mesh actually holds. Done // What this module said it stands on, answered with what this mesh actually holds. Done
// before anything is built, so a missing base is refused in front of the person who can // before anything is built, so a missing base is refused in front of the person who can
@@ -186,11 +188,12 @@ func Build(ctx context.Context, run Runner, publish Publisher,
} }
return from, nil return from, nil
} }
args, err := standingOn(ctx, manifest, held, mirror) args, bases, err := standingOn(ctx, manifest, held, mirror)
if err != nil { if err != nil {
say("bases", "UNMET: %v", err) say("bases", "UNMET: %v", err)
return Result{}, err return Result{}, err
} }
stoodOn = bases
if len(args) > 0 { if len(args) > 0 {
say("bases", "%d resolved from what the mesh holds", len(args)/2) say("bases", "%d resolved from what the mesh holds", len(args)/2)
} }
@@ -217,7 +220,7 @@ func Build(ctx context.Context, run Runner, publish Publisher,
} }
say("done", "%s at %s — %d artifact(s) pinned", manifest.Module, short(commit), len(built)) say("done", "%s at %s — %d artifact(s) pinned", manifest.Module, short(commit), len(built))
return Result{Manifest: resolved, Commit: commit, Built: built, return Result{Manifest: resolved, Commit: commit, Built: built,
Against: against(within, manifest)}, nil Against: against(within, manifest, stoodOn)}, nil
} }
// Log is where a build says what it is doing, step by step. Nil is silent — the tests pass none, // Log is where a build says what it is doing, step by step. Nil is silent — the tests pass none,
@@ -339,14 +342,27 @@ func describe(path string) string {
// allowed to name — a tag is something somebody else can move under you. // allowed to name — a tag is something somebody else can move under you.
var pinnedImage = regexp.MustCompile(`[A-Za-z0-9][A-Za-z0-9._/:-]*@sha256:[0-9a-f]{64}`) var pinnedImage = regexp.MustCompile(`[A-Za-z0-9][A-Za-z0-9._/:-]*@sha256:[0-9a-f]{64}`)
// against reads what this module's image artifacts are built on top of, out of the files that // against is what this module's image artifacts are built on top of: every base the mesh resolved
// build them. Nothing is guessed: a reference that is not written down is not reported. // and handed the recipe as a build argument (`build.on`), and any image a recipe pins by digest
func against(within string, manifest catalogue.Manifest) []string { // itself. Nothing is guessed: a reference that was neither resolved nor written down is not
// reported.
//
// **The resolved bases are the edges.** A recipe reads its base from an argument (`FROM
// ${RUNTIME_BASE}`), so the digest is never in the file, and a derivation that read files alone
// recorded no edge for any module on the mesh — which is why nothing knew what a changed base
// meant to rebuild (novox/hq 04-ISSUES/131).
func against(within string, manifest catalogue.Manifest, resolved []string) []string {
if manifest.Build == nil { if manifest.Build == nil {
return nil return nil
} }
seen := map[string]bool{} seen := map[string]bool{}
var out []string var out []string
for _, r := range resolved {
if r != "" && !seen[r] {
seen[r] = true
out = append(out, r)
}
}
for _, a := range manifest.Build.Artifacts { for _, a := range manifest.Build.Artifacts {
if a.Kind != catalogue.ArtifactImage || a.From == "" { if a.Kind != catalogue.ArtifactImage || a.From == "" {
continue continue
@@ -704,40 +720,42 @@ var _ io.Writer = (*stringWriter)(nil)
// built cannot be built here yet, and the useful sentence names which module is missing — not the // built cannot be built here yet, and the useful sentence names which module is missing — not the
// one a container runtime produces when a recipe's first line refers to an image nobody has. // one a container runtime produces when a recipe's first line refers to an image nobody has.
// //
// The order is fixed so two builds of one commit invoke the same command. // The order is fixed so two builds of one commit invoke the same command. Returned alongside the
// arguments is every reference they resolved to, which is what the build stood on.
func standingOn(ctx context.Context, manifest catalogue.Manifest, held map[string]string, func standingOn(ctx context.Context, manifest catalogue.Manifest, held map[string]string,
mirror func(ctx context.Context, from, repository string) (string, error)) ([]string, error) { mirror func(ctx context.Context, from, repository string) (string, error)) ([]string, []string, error) {
if manifest.Build == nil || len(manifest.Build.On) == 0 { if manifest.Build == nil || len(manifest.Build.On) == 0 {
return nil, nil return nil, nil, nil
} }
on := append([]catalogue.BuildsOn{}, manifest.Build.On...) on := append([]catalogue.BuildsOn{}, manifest.Build.On...)
sort.Slice(on, func(i, j int) bool { return on[i].Arg < on[j].Arg }) sort.Slice(on, func(i, j int) bool { return on[i].Arg < on[j].Arg })
var args []string var args, resolved []string
for _, base := range on { for _, base := range on {
if base.Image != "" { if base.Image != "" {
// A vendor's image, declared (novox/hq 04-ISSUES/064, ADR 0097). Pinned, because a tag // A vendor's image, declared (novox/hq 04-ISSUES/064, ADR 0097). Pinned, because a tag
// is what somebody else can move; copied into the mesh's registry, because a build // is what somebody else can move; copied into the mesh's registry, because a build
// that reaches a public registry on its own is a build that works sometimes. // that reaches a public registry on its own is a build that works sometimes.
if base.Arg == "" || base.Module != "" || base.Artifact != "" { if base.Arg == "" || base.Module != "" || base.Artifact != "" {
return nil, fmt.Errorf( return nil, nil, fmt.Errorf(
"%s stands on the image %s, and a base is either a module's artifact or an "+ "%s stands on the image %s, and a base is either a module's artifact or an "+
"image — never both — read from one build argument", manifest.Module, base.Image) "image — never both — read from one build argument", manifest.Module, base.Image)
} }
if !strings.Contains(base.Image, "@sha256:") { if !strings.Contains(base.Image, "@sha256:") {
return nil, fmt.Errorf( return nil, nil, fmt.Errorf(
"%s stands on the image %q, which is not pinned by digest. A tag is what "+ "%s stands on the image %q, which is not pinned by digest. A tag is what "+
"somebody else can move; name it as <image>@sha256:…", manifest.Module, base.Image) "somebody else can move; name it as <image>@sha256:…", manifest.Module, base.Image)
} }
reference, err := mirror(ctx, base.Image, manifest.Module+"/on-"+strings.ToLower(base.Arg)) reference, err := mirror(ctx, base.Image, manifest.Module+"/on-"+strings.ToLower(base.Arg))
if err != nil { if err != nil {
return nil, fmt.Errorf("%s stands on %s: %w", manifest.Module, base.Image, err) return nil, nil, fmt.Errorf("%s stands on %s: %w", manifest.Module, base.Image, err)
} }
args = append(args, "--build-arg", base.Arg+"="+reference) args = append(args, "--build-arg", base.Arg+"="+reference)
resolved = append(resolved, reference)
continue continue
} }
if base.Arg == "" || base.Module == "" || base.Artifact == "" { if base.Arg == "" || base.Module == "" || base.Artifact == "" {
return nil, fmt.Errorf( return nil, nil, fmt.Errorf(
"%s says its build stands on something, and does not say all of what: a base "+ "%s says its build stands on something, and does not say all of what: a base "+
"needs the module, the artifact, and the build argument the recipe reads it "+ "needs the module, the artifact, and the build argument the recipe reads it "+
"from", manifest.Module) "from", manifest.Module)
@@ -745,14 +763,15 @@ func standingOn(ctx context.Context, manifest catalogue.Manifest, held map[strin
key := base.Module + "/" + base.Artifact key := base.Module + "/" + base.Artifact
reference, has := held[key] reference, has := held[key]
if !has { if !has {
return nil, fmt.Errorf( return nil, nil, fmt.Errorf(
"%s builds on %s, and this mesh has not built it. Build %s first — every module "+ "%s builds on %s, and this mesh has not built it. Build %s first — every module "+
"in this toolchain stands on it, so it is the thing to have before anything "+ "in this toolchain stands on it, so it is the thing to have before anything "+
"else", manifest.Module, key, base.Module) "else", manifest.Module, key, base.Module)
} }
args = append(args, "--build-arg", base.Arg+"="+reference) args = append(args, "--build-arg", base.Arg+"="+reference)
resolved = append(resolved, reference)
} }
return args, nil return args, resolved, nil
} }
// compile runs a module's own code through its toolchain, and says where the result is. // compile runs a module's own code through its toolchain, and says where the result is.
+14
View File
@@ -180,6 +180,20 @@ func (r Registry) MirrorImage(ctx context.Context, from, repository string) (str
if err != nil { if err != nil {
return "", err return "", err
} }
// **Already held is already mirrored.** A base is named by digest, and a digest this registry
// holds under the module's repository is the same bytes whatever upstream would say — so
// upstream is not asked. Asked every build, the public hub's anonymous pull limit was reached
// on the first merge that rebuilt a whole catalogue (2026-09-28), and every module whose base
// lives there failed on a copy it did not need.
if strings.HasPrefix(where.reference, "sha256:") {
held, err := r.has(ctx, "http://"+r.Address+"/v2/"+repository+"/manifests/"+where.reference)
if err != nil {
return "", fmt.Errorf("asking %s whether it holds %s: %w", r.Address, from, err)
}
if held {
return r.Address + "/" + repository + "@" + where.reference, nil
}
}
src := &source{client: r.client()} src := &source{client: r.client()}
digest, err := r.copyManifest(ctx, src, where, where.reference, repository) digest, err := r.copyManifest(ctx, src, where, where.reference, repository)
if err != nil { if err != nil {
+30
View File
@@ -103,6 +103,12 @@ func (m *theMeshsRegistry) handler() http.Handler {
m.mu.Lock() m.mu.Lock()
defer m.mu.Unlock() defer m.mu.Unlock()
switch { switch {
case r.Method == http.MethodHead && strings.Contains(r.URL.Path, "/manifests/"):
if _, ok := m.manifests[r.URL.Path[strings.LastIndex(r.URL.Path, "/")+1:]]; ok {
w.WriteHeader(http.StatusOK)
} else {
w.WriteHeader(http.StatusNotFound)
}
case r.Method == http.MethodHead && strings.Contains(r.URL.Path, "/blobs/"): case r.Method == http.MethodHead && strings.Contains(r.URL.Path, "/blobs/"):
if _, ok := m.blobs[r.URL.Path[strings.LastIndex(r.URL.Path, "/")+1:]]; ok { if _, ok := m.blobs[r.URL.Path[strings.LastIndex(r.URL.Path, "/")+1:]]; ok {
w.WriteHeader(http.StatusOK) w.WriteHeader(http.StatusOK)
@@ -219,3 +225,27 @@ func TestATagBeforeTheDigestIsNotPartOfTheRepository(t *testing.T) {
t.Fatalf("got %+v", got) t.Fatalf("got %+v", got)
} }
} }
// A base this registry already holds by digest is not asked of upstream at all: the public hub
// limits anonymous pulls, and a catalogue rebuilt on one merge asked it once per module.
func TestABaseAlreadyHeldIsNotAskedOfUpstream(t *testing.T) {
src, indexDigest, _ := anUpstreamRegistry(t)
dst := &theMeshsRegistry{blobs: map[string][]byte{}, manifests: map[string][]byte{}}
dstServer := httptest.NewServer(dst.handler())
defer dstServer.Close()
address := strings.TrimPrefix(dstServer.URL, "http://")
r := Registry{Address: address, HTTP: src.Client()}
host := strings.TrimPrefix(src.URL, "http://")
if _, err := r.MirrorImage(context.Background(), host+"/library/thing:latest", "hello-web/server"); err != nil {
t.Fatal(err)
}
// Upstream gone: the pinned base is answered from what the mesh holds.
src.Close()
reference, err := r.MirrorImage(context.Background(), host+"/library/thing@"+indexDigest, "hello-web/server")
if err != nil {
t.Fatalf("a base the registry holds was asked of an upstream that is gone: %v", err)
}
if reference != address+"/hello-web/server@"+indexDigest {
t.Fatalf("pinned as %q", reference)
}
}
+34 -6
View File
@@ -22,7 +22,7 @@ func TestABaseTheMeshHasNotBuiltIsRefused(t *testing.T) {
On: []catalogue.BuildsOn{{Arg: "RUNTIME_BASE", Module: "mesh-tools", Artifact: "runtime"}}, On: []catalogue.BuildsOn{{Arg: "RUNTIME_BASE", Module: "mesh-tools", Artifact: "runtime"}},
}, },
} }
_, err := standingOn(context.Background(), manifest, map[string]string{}, noMirror) _, _, err := standingOn(context.Background(), manifest, map[string]string{}, noMirror)
if err == nil { if err == nil {
t.Fatal("a base nothing has built was accepted; the build would have failed on its first line") t.Fatal("a base nothing has built was accepted; the build would have failed on its first line")
} }
@@ -42,7 +42,7 @@ func TestABaseTheMeshHoldsBecomesABuildArgument(t *testing.T) {
}, },
} }
held := map[string]string{"mesh-tools/runtime": "127.0.0.1:5000/mesh-tools/runtime@sha256:" + strings.Repeat("a", 64)} held := map[string]string{"mesh-tools/runtime": "127.0.0.1:5000/mesh-tools/runtime@sha256:" + strings.Repeat("a", 64)}
args, err := standingOn(context.Background(), manifest, held, noMirror) args, _, err := standingOn(context.Background(), manifest, held, noMirror)
if err != nil { if err != nil {
t.Fatalf("a base this mesh holds was refused: %v", err) t.Fatalf("a base this mesh holds was refused: %v", err)
} }
@@ -54,7 +54,7 @@ func TestABaseTheMeshHoldsBecomesABuildArgument(t *testing.T) {
// A module naming no base asks for nothing, which is most modules. // A module naming no base asks for nothing, which is most modules.
func TestAModuleNamingNoBaseAddsNoArguments(t *testing.T) { func TestAModuleNamingNoBaseAddsNoArguments(t *testing.T) {
args, err := standingOn(context.Background(), catalogue.Manifest{Module: "hello-web", Build: &catalogue.Build{}}, nil, noMirror) args, _, err := standingOn(context.Background(), catalogue.Manifest{Module: "hello-web", Build: &catalogue.Build{}}, nil, noMirror)
if err != nil || args != nil { if err != nil || args != nil {
t.Fatalf("a module naming no base produced %v, %v", args, err) t.Fatalf("a module naming no base produced %v, %v", args, err)
} }
@@ -66,7 +66,7 @@ func TestAnIncompleteBaseIsRefused(t *testing.T) {
Module: "postgres", Module: "postgres",
Build: &catalogue.Build{On: []catalogue.BuildsOn{{Module: "mesh-tools", Artifact: "runtime"}}}, Build: &catalogue.Build{On: []catalogue.BuildsOn{{Module: "mesh-tools", Artifact: "runtime"}}},
} }
if _, err := standingOn(context.Background(), manifest, map[string]string{"mesh-tools/runtime": "x"}, noMirror); err == nil { if _, _, err := standingOn(context.Background(), manifest, map[string]string{"mesh-tools/runtime": "x"}, noMirror); err == nil {
t.Fatal("a base with no build argument was accepted; nothing would have read it") t.Fatal("a base with no build argument was accepted; nothing would have read it")
} }
} }
@@ -86,7 +86,7 @@ func TestADeclaredVendorImageIsCopiedInAndHandedToTheRecipe(t *testing.T) {
}, },
} }
var asked []string var asked []string
args, err := standingOn(context.Background(), manifest, nil, func(_ context.Context, from, repository string) (string, error) { args, _, err := standingOn(context.Background(), manifest, nil, func(_ context.Context, from, repository string) (string, error) {
asked = append(asked, from+" -> "+repository) asked = append(asked, from+" -> "+repository)
return "127.0.0.1:5000/" + repository + "@sha256:" + strings.Repeat("d", 64), nil return "127.0.0.1:5000/" + repository + "@sha256:" + strings.Repeat("d", 64), nil
}) })
@@ -101,7 +101,7 @@ func TestADeclaredVendorImageIsCopiedInAndHandedToTheRecipe(t *testing.T) {
} }
// Unpinned, it is refused: a tag is what somebody else can move. // Unpinned, it is refused: a tag is what somebody else can move.
manifest.Build.On[0].Image = "quay.io/minio/mc:latest" manifest.Build.On[0].Image = "quay.io/minio/mc:latest"
if _, err := standingOn(context.Background(), manifest, nil, noMirror); err == nil || !strings.Contains(err.Error(), "not pinned") { if _, _, err := standingOn(context.Background(), manifest, nil, noMirror); err == nil || !strings.Contains(err.Error(), "not pinned") {
t.Fatalf("an unpinned vendor image was accepted: %v", err) t.Fatalf("an unpinned vendor image was accepted: %v", err)
} }
} }
@@ -151,3 +151,31 @@ func TestARecipeIsReadAsInstructions(t *testing.T) {
t.Fatalf("a heredoc line or a continued stage was read as a base: %v", bases) t.Fatalf("a heredoc line or a continued stage was read as a base: %v", bases)
} }
} }
// What a build was handed as its bases is what it stood on — recorded, so a changed base knows what
// to rebuild (novox/hq 04-ISSUES/131). A recipe reads the base from an argument, so nothing else
// could know.
func TestTheBasesABuildWasHandedAreWhatItStoodOn(t *testing.T) {
manifest := catalogue.Manifest{
Module: "gitea",
Build: &catalogue.Build{
On: []catalogue.BuildsOn{
{Arg: "RUNTIME_BASE", Module: "mesh-tools", Artifact: "runtime"},
{Arg: "BUILD_BASE", Module: "mesh-tools", Artifact: "build"},
},
Artifacts: []catalogue.Artifact{{Name: "runtime", Kind: catalogue.ArtifactImage, From: "Dockerfile"}},
},
}
held := map[string]string{
"mesh-tools/runtime": "127.0.0.1:5000/mesh-tools/runtime@sha256:" + strings.Repeat("a", 64),
"mesh-tools/build": "127.0.0.1:5000/mesh-tools/build@sha256:" + strings.Repeat("b", 64),
}
_, resolved, err := standingOn(context.Background(), manifest, held, noMirror)
if err != nil {
t.Fatal(err)
}
got := against(t.TempDir(), manifest, resolved)
if len(got) != 2 || got[0] != held["mesh-tools/build"] || got[1] != held["mesh-tools/runtime"] {
t.Fatalf("the bases the build was handed were not what it stood on: %v", got)
}
}
+35
View File
@@ -164,6 +164,41 @@ func (i *Inventory) Held(ctx context.Context) (map[string]string, error) {
return held, rows.Err() return held, rows.Err()
} }
// BuiltAgainst is what each module's newest successful build stood on, as recorded — the build
// edges (ADR 0009). A module whose last build recorded no bases is absent, which is also what a
// module standing on nothing looks like: an edge the mesh has not derived is not an edge.
func (i *Inventory) BuiltAgainst(ctx context.Context) (map[string][]string, error) {
rows, err := i.store.Pool().Query(ctx,
`select distinct on (module) module, built_against
from build
where module is not null and module <> '' and failed = ''
order by module, at desc`)
if err != nil {
return nil, err
}
defer rows.Close()
against := map[string][]string{}
for rows.Next() {
var module string
var raw []byte
if err := rows.Scan(&module, &raw); err != nil {
return nil, err
}
if len(raw) == 0 {
continue
}
var refs []string
if err := json.Unmarshal(raw, &refs); err != nil {
continue
}
if len(refs) > 0 {
against[module] = refs
}
}
return against, rows.Err()
}
// manifestOrNil keeps the difference between "declared nothing" and "predates this being kept". // manifestOrNil keeps the difference between "declared nothing" and "predates this being kept".
// //
// A build recorded before the mesh kept manifests has no manifest, and that is not the same as one // A build recorded before the mesh kept manifests has no manifest, and that is not the same as one
+3 -1
View File
@@ -86,9 +86,11 @@ func TestAnAssignedModuleBecomesAUserWithWhatItDeclared(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
// Its tools are every one under its own name — the list in the manifest is a person's
// vocabulary for asking, not the module's permission to answer.
if !granted(perms.Publish, "mesh.mod.shop.event.order.placed") || if !granted(perms.Publish, "mesh.mod.shop.event.order.placed") ||
!granted(perms.Publish, "mesh.seat.telegram-sender.accept.send") || !granted(perms.Publish, "mesh.seat.telegram-sender.accept.send") ||
!granted(perms.Subscribe, "mesh.mod.shop.tool.price") { !granted(perms.Subscribe, "mesh.mod.shop.tool.>") {
t.Fatalf("one.shop's authority is not what it declared: %+v", perms) t.Fatalf("one.shop's authority is not what it declared: %+v", perms)
} }
} }
+10 -2
View File
@@ -8,6 +8,7 @@ import (
"sort" "sort"
"strconv" "strconv"
"strings" "strings"
"time"
"github.com/jackc/pgx/v5" "github.com/jackc/pgx/v5"
"github.com/novox/mesh-controller/internal/catalogue" "github.com/novox/mesh-controller/internal/catalogue"
@@ -38,6 +39,9 @@ type Source struct {
BuiltFrom string BuiltFrom string
// Head is the newest commit the source is known to have. // Head is the newest commit the source is known to have.
Head string Head string
// Seen is when the source was last looked at — by a build, by hand, or by the forge saying it
// moved. What a late report of an older move is judged against.
Seen time.Time
} }
// Current reports whether what the mesh holds is what the source last had. // Current reports whether what the mesh holds is what the source last had.
@@ -936,12 +940,13 @@ func (i *Inventory) Catalogued(ctx context.Context) ([]Entry, error) {
`select m.name, m.manifest, `select m.name, m.manifest,
coalesce(m.source, ''), m.source_path, m.source_seat, coalesce(m.ref, ''), coalesce(m.source, ''), m.source_path, m.source_seat, coalesce(m.ref, ''),
coalesce(m.built_from, ''), coalesce(m.source_head, ''), coalesce(m.built_from, ''), coalesce(m.source_head, ''),
coalesce(m.source_seen, to_timestamp(0)),
coalesce(array_agg(n.name order by n.name) filter (where n.name is not null), '{}') coalesce(array_agg(n.name order by n.name) filter (where n.name is not null), '{}')
from module m from module m
left join assignment a on a.module = m.name left join assignment a on a.module = m.name
left join node n on n.id = a.node left join node n on n.id = a.node
group by m.name, m.manifest, m.source, m.source_path, m.source_seat, m.ref, m.built_from, group by m.name, m.manifest, m.source, m.source_path, m.source_seat, m.ref, m.built_from,
m.source_head m.source_head, m.source_seen
order by m.name`) order by m.name`)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -955,9 +960,12 @@ func (i *Inventory) Catalogued(ctx context.Context) ([]Entry, error) {
var source Source var source Source
var on []string var on []string
if err := rows.Scan(&name, &raw, &source.Repository, &source.Path, &source.Seat, &source.Ref, if err := rows.Scan(&name, &raw, &source.Repository, &source.Path, &source.Seat, &source.Ref,
&source.BuiltFrom, &source.Head, &on); err != nil { &source.BuiltFrom, &source.Head, &source.Seen, &on); err != nil {
return nil, err return nil, err
} }
if source.Seen.Unix() == 0 {
source.Seen = time.Time{}
}
var m catalogue.Manifest var m catalogue.Manifest
if err := json.Unmarshal(raw, &m); err != nil { if err := json.Unmarshal(raw, &m); err != nil {
return nil, err return nil, err
+2
View File
@@ -128,6 +128,8 @@ type SourceMoved struct {
Commit string `json:"merge_commit_sha"` Commit string `json:"merge_commit_sha"`
CloneURL string `json:"clone_url"` CloneURL string `json:"clone_url"`
HTMLURL string `json:"html_url"` HTMLURL string `json:"html_url"`
// MergedAt is when the forge merged it, RFC 3339. What decides whether this is news.
MergedAt string `json:"merged_at"`
} }
type Upgraded struct { type Upgraded struct {