Author SHA1 Message Date
jschoubben 71508cc962 Record the forge-made merge in the delivery gate, so the indexer takes it in (issue 1966, recovery)
mesh/check failed (2): ace: the check of novox/mesh-sdk#173 failed: the package registry's secret cannot be…; and 1 more | 71508cc96297 on 87bdc95a3d65
mesh/delivery-group group fix/1966-gate-record-2 rejected: composed together: the checker's verdict at the members' heads (13aa9af1d2d6): ace
mesh/delivery ready: it is merged on the Approve
mesh-host #213 (87bdc95a) was merged on the forge under recovery-fixes-merge-and-tell and has no gate merge record; the indexer holds its take-in. An empty merge through the gate covers it, and the forge-made wait is dropped (indexer_drop_pending).

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 19:23:06 +01:00
mesh-tools 87bdc95a3d Merge pull request 'The node-engine serves its seat's verbs from the bootstrapped definition until it holds the seat list (issue 1966, recovery)' (#213) from fix/1966-engine-seat-verbs-bootstrap into main 2026-11-08 17:34:51 +00:00
jschoubben 7b42c94a4d The node-engine serves its seat's verbs from the bootstrapped definition until it holds the seat list (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-host@87bdc95a3d65
mesh/check pass 7b42c94a4d78 on 87bdc95a3d65 | unit 1679 run, 333 kept, 31 skipped | integration 1692 run, 333 kept, 26 skipped | over bound: unit (sl…
The link serves the seat node-engine when it opens, before the node agent
starts the reader of the seat list; since ADR 0635 catalogue holds no seat
until then, so the engine registered no verb at all. Nobody answered
manifest-reads, compose, composition or the hand acts on novox, shanks and
now ace, and the assigner kept ace's old statement, holding its own
successor on seats the new engine states.

Quick and thorough fix are one here: the only seat-list read at start is
this one (the composer waits for the list, the reads statement restates).

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 18:34:20 +01:00
mesh-tools 6dd8e08a84 Merge pull request 'The node-engine passes seats with the SDK's one rule, by the build its node runs, and says the pass (ADR 0660, issue 1966, recovery)' (#212) from fix/1966-seat-pass-shared into main 2026-11-08 14:47:22 +00:00
jschoubben 60f413f9bc The node-engine reads each assignment by the build its node keeps and says a seat pass in its answer (issue 1966, ADR 0660)
mesh/delivery merged: source.main moved to novox/mesh-host@6dd8e08a84ed
mesh/delivery-group group fix/1966-seat-pass-shared merged: every member is merged
mesh/check checking 60f413f9bc6a on 6dd8e08a84ed | novox/mesh-host#212 in group fix/1966-seat-pass-shared
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 15:44:00 +01:00
jschoubben 5108c28e22 The node-engine composes seat passes with the SDK's one rule instead of its own copy (issue 1966, ADR 0660)
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 15:31:32 +01:00
mesh-tools 875b4ee5ce Merge pull request 'The node-engine states the seats of the list it holds, and again on every change (issue 1966, recovery)' (#211) from fix/1966-seat-reads-bootstrap into main 2026-11-08 03:45:15 +00:00
jschoubben 9ab85ef92e The node-engine states the seats of the seat list it holds, again at each change (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-host@875b4ee5ce38
mesh/delivery-group group fix/1966-seat-reads-bootstrap merged: every member is merged
mesh/check failed (2): mesh-delivery-checker: no published of the tagged version of go-sdk of mesh-delivery…; and 1 more | 9ab85ef92e6c on 875b4ee5ce38
manifestReads froze the statement at its first call, before the engine gave
catalogue even its bootstrapped seats (ADR 0635 empties mesh-sdk's table
until the list is read), so ace's engine stated no seat and its own next
build, naming mesh-catalog, was held on it.

The words and forms stay the build's, computed once; the seats are those of
the list held now. The reader of the seat list restates after each change:
the node record carries the new statement at once and the node seat's event
manifest-reads-changed says it (ADR 0618 §4); an unchanged statement writes
and says nothing; the verb manifest-reads answers when it was made.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 02:36:41 +01:00
mesh-tools 89475bb0cf Merge pull request 'The node-engine's bus watch never holds the client back (issue 1966, bootstrap recovery)' (#210) from fix/1966-watch-never-blocks into main 2026-11-07 23:40:48 +00:00
jschoubben 4f1e92160a Round 1: the pin tests merged (test budget), KVWatch's doc says what folds
mesh/delivery merged: source.main moved to novox/mesh-host@89475bb0cfec
The review of fe9533c1: a new test function raised a count already over the
budget, so two broker-pin tests are one; KVWatch no longer yields every
change, and its doc says so (issue 1966).

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 00:38:36 +01:00
jschoubben fe9533c186 The node-engine's bus watch never holds the client back: it keeps the newest update per key while its reader is busy (issue 1966)
Live fault 2026-11-08: node-engines on two machines logged only "nats: slow consumer, messages dropped" and applied nothing. The
client's watch sends into its channel from inside the subscription's callback; KVWatch forwarded that channel to its reader
blocking, and the reader (composing for minutes) stopped draining, so the subscription's pending limit overflowed.

Quick fix: a larger buffer on the watch's channel; it only moves the limit. Thorough fix (chosen): the forwarder always drains the
client's channel and, while its reader is busy, coalesces the pending updates by key (newest value and revision, first-seen
order, Caught kept in its place), so no reader's pace can back the client up, whatever the burst. A slow consumer error now
names its subscription and what it holds pending.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 00:26:33 +01:00
mesh-tools 70e70eda5a Merge pull request 'Registry milestone: mesh-host (issue 1966)' (#209) from feat/registry-milestone into main 2026-11-07 14:19:45 +00:00
jschoubben cb7a444a56 The check's composer says a node's refusal in the words of its composition with the lifted modules held (issue 1921, milestone 47/G)
mesh/delivery merged: source.main moved to novox/mesh-host@70e70eda5a24
mesh/delivery-group group feat/registry-milestone merged: every member is merged
mesh/check checking cb7a444a5634 on 70e70eda5a24 | novox/mesh-host#209 in group feat/registry-milestone
When a node refused also with the modules composed without their build held as its node-engine holds them, the
verdict kept the words of the composition from their unbuilt manifests, whose refusals the lift itself made (a tool
runner with no bundle, a resource whose artifact or template no build resolved). In the group check of c5d32074c945
that hid lab's real refusal on shanks among them. The node's refusal is now that of the held composition, named so.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 08:19:55 +01:00
jschoubben 53addd73c3 The check's composer composes a module at its build only where that build was made of the change's manifest (issue 1921, milestone 47/G)
mesh/delivery-group group feat/registry-milestone checking: 7 of 7 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 53addd73c358 on e73aae3686a6 | novox/mesh-host#209 in group feat/registry-milestone
The running indexer could not read 17 of the group's manifests, so the running node-engine composed those modules at
their trunk builds and said nothing; the head composer laid those builds over the change's other manifests and ace
failed on "nothing provides artifact-store, wanted by mesh-build-go" (group check 8a80e8e86858). The input a node
answers a check with now names, per module taken in from its build, the content digest of the manifest that build was
made of (ComposedHow.Manifests); the composer compares it with the case's manifest, and, for an input from a
node-engine before it, compares what the two manifests say beyond what a take-in resolves (the lead's ruling, §17).
Each module lifted either way is composed from the case's manifest, its build awaited, and named in NotJudged with
the comparison that lifted it; one that cannot compose without its build is held as its stub. The digest is taken
over one JSON form because a request through a build module arrives compacted.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 07:24:16 +01:00
jschoubben b6c174a324 A consumer beside a held provider is told what it serves without the names it does not compose, never their placeholders (issue 1921, milestone 47/G)
The provider held on its machine is composed apart, so no module of the composed set serves the provision; its
manifest's served values reached the consumer's binding raw and the binding refused the consumer for a
${bound:route:name-web} it does not require (the group check of 8a80e8e86858 on novox, mesh-build-node beside a held
gitea). Each such key is now left out and said on the need, as a provider elsewhere leaves it out (ADR 0346).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 07:19:58 +01:00
jschoubben 5b75e866d0 The composition verb says what holds each held module and the build it would be composed at (issue 1921, milestone 47/G H4)
mesh/delivery closed: a newer head of the same pull request
mesh/delivery-group group feat/registry-milestone checking: 6 of 7 member(s) ready
mesh/check failed (4): unit: git.novox.be/novox/mesh-catalog/checks/gitignored.TestNoProgramABuildLeavesIsC…; and 3 more | 5b75e866d079 on e73aae3686a6
The node's own composition named a held module's why only; it now also says whether its manifest's verdict or this
node-engine's decoder holds it, and at which build, as the input a check's composer reads says it. Said in the held
words, so the node-engine seat's answer keeps its shape.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:38:18 +01:00
jschoubben d0fb384f00 The check's composer composes a module the running indexer alone holds when the change's manifest is the check's to judge (issue 1921, milestone 47/G H2)
ADR 0578 §2 and §3: a case's accepted verdict, or none (the running checker drops the change's own), lifts the
input's hold on a manifest the composer reads whole; a case's refusal and bytes that do not decode stay held. It is
composed at the build the input carries (Input.Holds). Without that build the node is not judged and the verdict
names the module, why it was held and that its build is awaited; where it cannot compose without it, it is composed
held as its node-engine holds it, and said so. Never a silent pass (the lead's Q4 ruling). The issue 774 row now
gives the runner the refused verdict the checker gives a manifest the change does not bring.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:33:21 +01:00
jschoubben c114b786e8 The input a node answers a check with says what holds each held module and carries its build (issue 1921, milestone 47/G H3)
A module the running indexer refused stood in the input as its stub with no build, so a check's composer that
judges the change's manifest itself had nothing to compose it at. Each held module now says whether its verdict
or the decoder holds it; for a check's answer the node also takes in the build it would compose it at, when the
builder holds it done, beside the stub and never into what the node composes.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:33:21 +01:00
jschoubben e96a7ada50 The node-engine keeps no copy of a seat's verbs: the seat list comes from the bus (issue 1921, ADR 0635 §1)
Eight lists of seat verbs (checker, enrolment, artifact sweep, delivery, node-resolver, node-nfs-server,
node-mounts, mesh-config's input) and their schema helpers were read by nothing but one test, and drifted
from their seats (B0's verb input never reached the checker's copy). A guard row in the seats test finds
any list of the catalogue's verbs written in the node-engine's code.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:26:14 +01:00
jschoubben eec35a7c0c The sandbox composer composes against the seat list its request carries, and judges nothing without one (issue 1966)
The group try of 2026-11-06 10:07 failed its composition on all four
machines because mesh-compose, the node-engine's composer a check builds
at the head, loaded no seat list once the compiled one was deleted (ADR
0635 §1): the uplink's holder no longer stepped back from the resolver
file, and no holder of the registry seats was known. It now takes the
request's seats over the foundation's bootstrapped ones, as the
node-engine takes the bus's; without any it says each machine not judged
(ADR 0636 §1, ADR 0593 §9), never a refusal or pass made without them.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 12:01:00 +01:00
36 changed files with 1226 additions and 668 deletions
+47 -1
View File
@@ -23,11 +23,17 @@ import (
"flag"
"fmt"
"io"
"maps"
"os"
"path/filepath"
"slices"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"git.novox.be/novox/mesh-sdk/go/seatlist"
"github.com/novox/mesh-host/internal/composer"
)
@@ -105,7 +111,7 @@ func run(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
if *declarations {
req.Declarations = true
}
v, err := composer.Run(req)
v, err := runWithSeats(req)
if err != nil {
fmt.Fprintln(stderr, "mesh-compose:", err)
return 1
@@ -118,6 +124,46 @@ func run(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
return 0
}
// NoSeatList is what a machine reads, not judged, when the request carries no seat list (novox/hq ADR 0636 §1).
const NoSeatList = "the composer was handed no seat list, so it composes nothing (a reader of the seat list acts on nothing " +
"new until it has read the list, novox/hq ADR 0636 §1): the asker gives the seat definitions it judges against in the request's seats"
// runWithSeats composes req against the seat list the request carries (Request.Seats: the definitions the asker's seat list
// reader holds, the members' heads' laid over them in a check, ADR 0636 §4), over the foundation's bootstrapped ones
// (seatlist.Bootstrapped), as the node-engine composes against the list it read from the bus (ADR 0635 §1). The command
// compiles no seat list: without one in the request it reads every manifest and judges no machine, each said not judged
// with NoSeatList (ADR 0593 §9), never a refusal or a pass made without the seats' paths, deliveries and holders.
func runWithSeats(req composer.Request) (composer.Verdict, error) {
if len(req.Seats) == 0 {
machines := req.Machines
req.Machines = nil
v, err := composer.Run(req)
if err != nil || len(machines) == 0 {
return v, err
}
for i := range v.Cases {
v.Cases[i].Machines = map[string]composer.Machine{}
for _, node := range machines {
v.Cases[i].Machines[node] = composer.Machine{NotJudged: NoSeatList}
}
}
return v, nil
}
defs := map[string]seatdef.Definition{}
for _, d := range seatlist.Bootstrapped() {
defs[d.Seat] = d
}
for _, raw := range req.Seats {
d, err := seatdef.Parse(raw)
if err != nil {
return composer.Verdict{}, fmt.Errorf("a seat definition of the request: %w", err)
}
defs[d.Seat] = d
}
catalogue.UseSeats(catalogue.SeatsFromDefinitions(slices.Collect(maps.Values(defs))))
return composer.Run(req)
}
// readCatalogue reads every module.json of a catalogue checkout by its module's name: the root's and each of
// modules/*/. A manifest whose name cannot be read is kept under its directory's name, so the reader's refusal is said.
func readCatalogue(dir string) (map[string]composer.Manifest, error) {
+43 -3
View File
@@ -17,6 +17,7 @@ import (
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/seatlist/seatlisttest"
"github.com/novox/mesh-host/internal/beside"
"github.com/novox/mesh-host/internal/compose"
"github.com/novox/mesh-host/internal/composer"
@@ -49,11 +50,25 @@ func meshOf() compose.Input {
func request(t *testing.T, depth string) composer.Request {
t.Helper()
r := composer.Request{V: composer.Version, Depth: depth, HoldingsFromClaims: true,
Machines: []string{"anchor", "laptop", "ace"},
Cases: []composer.Case{{Name: "trunk", Manifests: captured(t)}}}
Machines: []string{"anchor", "laptop", "ace"}, Seats: seats(t),
Cases: []composer.Case{{Name: "trunk", Manifests: captured(t)}}}
return withInput(t, r, meshOf())
}
// seats is the seat list a request carries, as the checker hands the one its reader holds (ADR 0635 §1, ADR 0636 §4).
func seats(t *testing.T) []json.RawMessage {
t.Helper()
var out []json.RawMessage
for _, d := range seatlisttest.Definitions() {
raw, err := json.Marshal(d)
if err != nil {
t.Fatal(err)
}
out = append(out, raw)
}
return out
}
// withInput is r with in as its input, as a caller puts it on the wire: the request's shape carries the input as JSON.
func withInput(t *testing.T, r composer.Request, in compose.Input) composer.Request {
t.Helper()
@@ -124,6 +139,31 @@ func TestTheCommandWritesOneVerdictPerManifestAndMachine(t *testing.T) {
t.Errorf("%s does not compose and says no reason", name)
}
}
// Issue 1966, the group try of 2026-11-06 10:07: the composer judges against the seat list its request carries, the
// node-engine's from the bus (ADR 0635 §1). The uplink's holder steps back from the resolver file for the resolver's
// holder by the seat's definition; without the list nothing is judged (ADR 0636 §1), never "both declare the path".
resolverMachine := func(withSeats bool) composer.Machine {
r := composer.Request{V: composer.Version, Depth: composer.DepthCompose, HoldingsFromClaims: true,
Machines: []string{"laptop"}, Cases: []composer.Case{{Name: "trunk", Manifests: captured(t)}}}
if withSeats {
r.Seats = seats(t)
}
r = withInput(t, r, compose.Input{Nodes: []compose.Node{{Name: "laptop",
Assigned: []string{"networkmanager", "systemd-resolved", "pacman", "systemd", "dnsmasq"},
Capabilities: map[string]bool{"file-references": true, "privileged": true, "service-manager": true, "package-manager": true, "uplink-networkmanager": true}}}})
code, v, stderr := runJSON(t, nil, encode(t, r))
if code != 0 || len(v.Cases) != 1 {
t.Fatalf("exit %d: %s", code, stderr)
}
return v.Cases[0].Machines["laptop"]
}
if m := resolverMachine(true); strings.Contains(m.Reason+strings.Join(m.Problems, " "), "both declare") || m.NotJudged != "" {
t.Errorf("with the seat list, the uplink's and the resolver's holders on one machine: %+v, want the uplink's stepping back", m)
}
if m := resolverMachine(false); m.Composes || m.NotJudged != NoSeatList || m.Reason != "" {
t.Errorf("without a seat list: %+v, want not judged, saying why", m)
}
}
// 1621/AC3: over the captured catalogue and a mesh of three machines, the command's verdict equals, machine for
@@ -196,7 +236,7 @@ func TestACatalogueCheckoutIsComposedForANode(t *testing.T) {
if err := os.WriteFile(filepath.Join(dir, "modules", "demo", "module.json"), []byte(manifest), 0o644); err != nil {
t.Fatal(err)
}
in := encode(t, withInput(t, composer.Request{V: composer.Version}, compose.Input{Nodes: []compose.Node{{Name: "laptop", Assigned: []string{"demo"}}}}))
in := encode(t, withInput(t, composer.Request{V: composer.Version, Seats: seats(t)}, compose.Input{Nodes: []compose.Node{{Name: "laptop", Assigned: []string{"demo"}}}}))
code, v, stderr := runJSON(t, []string{"-catalogue", dir, "-node", "laptop", "-declarations"}, in)
if code != 0 {
t.Fatalf("exit %d: %s", code, stderr)
+7 -1
View File
@@ -135,7 +135,8 @@ func (w findingsOfTheNode) Delete(f views.Finding) error {
}
// startSeatList starts this run's reader: its bootstrapped seats are given to catalogue at once, and every change of what
// it holds again, saying so to the composer and to the waiter that writes the five-minute finding.
// it holds again, saying so to the composer and to the waiter that writes the five-minute finding, and stating again what
// the engine reads (link.Session.RestateReads, ADR 0618 §4): the seats it states are those of the list it holds.
func startSeatList(ctx context.Context, session *link.Session, node string, say func(string)) (*seatListState, error) {
st := &seatListState{changed: make(chan struct{}, 1)}
// Start returns at once and reads the seat on a goroutine, which may announce a change before the waiter exists.
@@ -143,6 +144,9 @@ func startSeatList(ctx context.Context, session *link.Session, node string, say
reader, err := seatlist.Start(seatlist.Config{Bus: sessionBus{ctx, session}, Bootstrap: seatlist.Bootstrapped(),
OnChange: func(seats []catalogue.Seat) {
catalogue.UseSeats(seats)
// What the engine reads changed with the list: its record and its event say it (ADR 0618 §4), off the
// reader's goroutine, which returns quickly.
go session.RestateReads(ctx)
if f := announce.Load(); f != nil { // the first read may end before the waiter exists
(*f)()
}
@@ -162,6 +166,7 @@ func startSeatList(ctx context.Context, session *link.Session, node string, say
}
announce.Store(&told)
catalogue.UseSeats(reader.Seats())
session.RestateReads(ctx) // its first statement holds the bootstrapped seats at least (issue 1966)
told()
// The first read ends whether the seat answered or not, and a read that held what the reader held already says no
// change: its end is told to the waiter and the composer, so a status that moved from reading to heard starts or ends
@@ -170,6 +175,7 @@ func startSeatList(ctx context.Context, session *link.Session, node string, say
select {
case <-reader.Ready():
catalogue.UseSeats(reader.Seats())
session.RestateReads(ctx)
told()
case <-ctx.Done():
}
@@ -1,6 +1,7 @@
package frombus
import (
"bytes"
"context"
"encoding/json"
"fmt"
@@ -218,6 +219,35 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
t.Errorf("spiegel in InOwn wanted at %s: %v, held %v, read as %s; want its build's image", sp.Fp2, err,
read.ComposedHow, raw)
}
// Issue 1921: the input a check is answered with names the manifest spiegel's build was made of (its content
// digest, the same for those bytes compacted inside a request), so the check's composer composes it at that
// build only where the change's manifest is that one.
var compact bytes.Buffer
_ = json.Compact(&compact, bb.manifests[sp.Manifest])
want, _ := compose.ContentDigest(bb.manifests[sp.Manifest])
if got, _ := compose.ContentDigest(compact.Bytes()); err == nil && (want == "" || read.ComposedHow.Manifests["spiegel"] != want || got != want) {
t.Errorf("the manifest spiegel's build was made of: %v, want %s (compacted %s)", read.ComposedHow.Manifests, want, got)
}
// Issue 1921: held by its manifest's verdict, spiegel is still held in what the node composes, and the input a check
// is answered with says what holds it and carries the build the node would compose it at, taken in while done; a
// node's own read takes in no build of it.
sp.Verdict, sp.Problems = meshcatalogclient.VerdictRefused, []string{"p"}
bb.modules["spiegel"] = sp
for _, done := range []bool{true, false} {
if !done {
bb.builds["spiegel "+sp.Fp2].State = builds.BuildBuilding
}
read, err = ComposingInput(ctx, bb, Mesh{}, Scope{Node: "n1"}, map[string]string{"lemurs": fp("e"), "spiegel": sp.Fp2})
hold := read.Holds["spiegel"]
raw, _ = json.Marshal(hold.Built)
if err != nil || !read.Catalogue["spiegel"].Stubbed || hold.By != compose.HeldByVerdict || hold.Build != sp.Fp2 ||
strings.Contains(string(raw), strings.Repeat("7", 64)) != done || (hold.Built == nil) == done {
t.Errorf("spiegel refused, its build done %v: %v, hold %+v (built %s)", done, err, hold, raw)
}
}
if own, err := Input(ctx, bb, Mesh{}, Scope{Node: "n1"}); err != nil || own.Holds["spiegel"] != (compose.Hold{By: compose.HeldByVerdict}) {
t.Errorf("a node's own read of spiegel refused: %v, hold %+v", err, own.Holds["spiegel"])
}
}
// (C) Issue 1126, C5: what the others read of a module carried as it runs is what runs, never its stub's: the filter
+60 -8
View File
@@ -167,6 +167,17 @@ type Scope struct {
// Kept is told, in words, each module composed as what runs shows because its manifest does not yet say it (the tool
// runner, the core's credentials; ADR 0601 §4), so the node says it. Nil: said nowhere.
Kept func(module, words string)
// takeInHeld, set for the input a node answers a check with (ComposingInput), takes in the build of each module placed
// on the node that its manifest's verdict or its decoder holds, into compose.Input.Holds, never into what the node composes: the
// check's composer, judging the change's manifest itself, composes it at that build (issue 1921).
takeInHeld bool
// manifests, set with takeInHeld, is filled with the ContentDigest of the manifest each module placed on the node was
// built from, as it is taken in from that build (compose.ComposedHow.Manifests).
manifests map[string]string
// Holds is told, after Compose read the input, what holds each module it holds by its manifest's verdict or its
// decoder (compose.Input.Holds), with the build it would compose it at, so the node says why it holds it (issue 1921,
// the composition verb). Nil: said nowhere.
Holds func(map[string]compose.Hold)
}
// InOwn is, of the modules p places on its node, those whose resources the node's own composition it runs
@@ -214,21 +225,23 @@ func Input(ctx context.Context, r Reader, mesh Mesh, s Scope) (compose.Input, er
// taken is a module's manifest as a reader takes it (novox/hq ADR 0578 §3): decoded leniently, with the indexer's verdict
// from its modules value. A refused one, or bytes that do not decode, stands as its stub (unusable) and why says so; a
// manifest not read past a field it set aside holds its name alone and is held the same.
func taken(name string, raw []byte, m meshcatalog.ModulesByModuleValue) (man catalogue.Manifest, why string) {
//
// by is what holds it (compose.HeldByVerdict or compose.HeldByDecoder), empty when nothing does.
func taken(name string, raw []byte, m meshcatalog.ModulesByModuleValue) (man catalogue.Manifest, why, by string) {
if v := meshcatalogclient.ModuleVerdict(name, m); v != nil {
if refused, _ := meshcatalogclient.TakeVerdict(v); refused {
return unusable(name, raw), meshcatalogclient.VerdictWhy(*v)
return unusable(name, raw), meshcatalogclient.VerdictWhy(*v), compose.HeldByVerdict
}
}
man, err := catalogue.Decode(raw)
switch {
case err != nil:
return unusable(name, raw), fmt.Sprintf("the manifest of %s does not decode: %v", name, err)
return unusable(name, raw), fmt.Sprintf("the manifest of %s does not decode: %v", name, err), compose.HeldByDecoder
case man.ReadNoFurther():
return unusable(name, raw), fmt.Sprintf("the manifest of %s could not be read past a field this reader does not "+
"know: %s", name, man.SetAside())
"know: %s", name, man.SetAside()), compose.HeldByDecoder
}
return man, ""
return man, "", ""
}
// input is Input; with refuse set, a module placed on the composing node whose build cannot be read or taken in, or whose
@@ -374,9 +387,13 @@ func readBase(ctx context.Context, r Reader, mesh Mesh, s Scope) (base, error) {
// follows (issue 774: power's bar item refused every node's composition, 2026-10-30; ADR 0578 §3: one refused
// manifest silenced disk-load for 45 hours). It stands in as its stub, so what it provides and requires still
// shapes the rest; it is held on the composing node where it is placed there, and elsewhere only its offer is read.
man, why := taken(name, raw, m)
man, why, by := taken(name, raw, m)
if why != "" {
named(&in.Refused, name, why)
if in.Holds == nil {
in.Holds = map[string]compose.Hold{}
}
in.Holds[name] = compose.Hold{By: by}
if ownPlaced[name] {
b.ownRefused = append(b.ownRefused, [2]string{name, why})
}
@@ -580,6 +597,16 @@ func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]s
if err != nil {
return nil, nil, held, err
}
if s.Holds != nil {
holds := map[string]compose.Hold{}
for name := range held {
if h, said := in.Holds[name]; said {
h.Build = fps[name]
holds[name] = h
}
}
s.Holds(holds)
}
// The pair credentials its composition reads, asked of the vault sealed to it (ADR 0550 §5).
asConsumer, asProvider, err := compose.CredentialsNeeded(in, s.Node)
if err != nil {
@@ -600,11 +627,12 @@ func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]s
// credential: the sandbox composes no pair value.
func ComposingInput(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]string) (compose.Input, error) {
held := map[string]string{}
s.takeInHeld, s.manifests = true, map[string]string{}
in, err := input(ctx, r, mesh, s, func(module string, err error) { held[module] = err.Error() }, fps)
if err != nil {
return compose.Input{}, err
}
c := &compose.ComposedHow{Node: s.Node, Builds: maps.Clone(fps), Held: held}
c := &compose.ComposedHow{Node: s.Node, Builds: maps.Clone(fps), Held: held, Manifests: s.manifests}
for _, run := range s.Running {
if _, isHeld := held[run.Module]; isHeld && run.Module != "" {
c.Running = append(c.Running, run)
@@ -909,7 +937,23 @@ func fromBuilds(ctx context.Context, r Reader, build func(ctx context.Context, m
continue
}
if _, refused := in.Refused[pm.Module]; refused {
continue // held as its stub: no build of a refused manifest is taken in (ADR 0578 §3)
// Held as its stub: no build of a refused manifest is composed (ADR 0578 §3). For a check, the build the node
// would compose it at is said beside it, and taken in when done, for the check's composer alone.
if hold, said := in.Holds[pm.Module]; s.takeInHeld && said {
hold.Build = m.Fp2
if w := wanted[pm.Module]; w != "" {
hold.Build = w
}
// Its stub names no build, so the builder is asked: a build not done, or not taken in, is awaited.
if b, err := build(ctx, pm.Module, hold.Build); err == nil && b != nil && b.State == builds.BuildDone {
if resolved, err := built(pm.Module, hold.Build, b); err == nil {
hold.Built = &resolved
}
}
in.Holds = maps.Clone(in.Holds)
in.Holds[pm.Module] = hold
}
continue
}
if man, ok := in.Catalogue[pm.Module]; !ok || (man.Build == nil && !man.Templated()) {
continue // nothing is built of it: its manifest is what it composes from
@@ -942,6 +986,14 @@ func fromBuilds(ctx context.Context, r Reader, build func(ctx context.Context, m
continue
}
in.Catalogue[pm.Module] = resolved
if s.manifests != nil {
// Unread or not JSON, it is left out: the check's composer then compares what the manifests say (issue 1921).
if raw, err := r.Manifest(ctx, b.Manifest); err == nil {
if digest, err := compose.ContentDigest(raw); err == nil {
s.manifests[pm.Module] = digest
}
}
}
}
for i := range in.Nodes {
if in.Nodes[i].Name != s.Node {
+50
View File
@@ -16,6 +16,9 @@
package compose
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
@@ -114,11 +117,36 @@ type Input struct {
// (compose.Stub), so it is held: nothing of its own is composed, while what it provides and requires still shapes the
// rest. A module not named is accepted, a missing verdict included.
Refused map[string]string `json:",omitempty"`
// Holds is, by module Refused names, what holds it and the build its node would compose it at (issue 1921): a check's
// composer that judges the change's manifest itself (novox/hq ADR 0578 §2) composes a module refused by the running
// indexer alone from that build, and names one whose build it is not given. Nil from a node-engine before it.
Holds map[string]Hold `json:",omitempty"`
// SetAside is each module whose manifest has a field the reader's decoder does not know, by module, with what was set
// aside (catalogue.Manifest.SetAside): the rest of it is composed, and the reader names it (ADR 0578 §6).
SetAside map[string]string `json:",omitempty"`
}
// What holds a module Refused names (Hold.By).
const (
// HeldByVerdict is a module the indexer's judge refused the manifest of (its verdict).
HeldByVerdict = "verdict"
// HeldByDecoder is a module whose bytes this reader's decoder does not decode, or cannot read past a field it does not
// know: no judge of the manifest overrules it.
HeldByDecoder = "decoder"
)
// Hold is what holds one module Refused names, and the build its node would compose it at (Input.Holds).
type Hold struct {
// By is HeldByVerdict or HeldByDecoder.
By string
// Build is the fingerprint the node would compose it at, its want's or else the indexer's newest; empty where it is not
// placed on the composing node or names no build.
Build string `json:",omitempty"`
// Built is its manifest as the node takes it in from that build, when the builder holds it done; nil while that build
// is awaited (or not asked: only a node's answer to a check asks it).
Built *catalogue.Manifest `json:",omitempty"`
}
// Issued is one pair credential made for a consumer of a provider (inv.Secret): ForProvider is its copy sealed to the
// provider, optional.
type Issued struct {
@@ -258,6 +286,28 @@ type ComposedHow struct {
// HeldBefore is each module the node holds composing without the change (the candidate's base), held by the read or
// blamed later for a refusal: the change refuses none of them (issue 774, review round 1).
HeldBefore map[string]string
// Manifests is, by module taken in from its build, the ContentDigest of the manifest that build was made of (its
// build record's manifest), filled in the input a node answers a check with: the check's composer composes a module
// at its build only where the change's manifest is that one (issue 1921). Nil from a node-engine before it.
Manifests map[string]string
}
// ContentDigest is the hex sha256 of a manifest's JSON in one form, its keys sorted and its spacing removed, its numbers
// as written: the same for the bytes its repository holds and for those bytes carried inside a request, which JSON
// encoding compacts on the way (a composition request through a build module), so the two are compared as one manifest.
func ContentDigest(raw []byte) (string, error) {
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
var v any
if err := dec.Decode(&v); err != nil {
return "", fmt.Errorf("compose: the manifest is not JSON: %w", err)
}
canonical, err := json.Marshal(v)
if err != nil {
return "", err
}
sum := sha256.Sum256(canonical)
return hex.EncodeToString(sum[:]), nil
}
// Place is where a node is, as its assignment says (inv.Overlay).
+4 -1
View File
@@ -60,7 +60,8 @@ type Declared struct {
ForeseenPorts []string
// CoreBoxesWanted are the core credentials whose box the send must ask the vault for and keep.
CoreBoxesWanted []CoreBoxWanted
// Passes are the seats the mesh would pass on, for the caller to record, and Said the lines found while composing.
// Passes are the seats composed as passed to their only claimant (ADR 0660), recorded by nobody here, and Said the lines found
// while composing, the passes among them.
Passes Passes
Said []string
}
@@ -228,6 +229,8 @@ func (c *composer) declaration(nodeName string, mode Mode) (Declared, error) {
return Declared{}, err
}
out.Passes = passes
// A pass is composed, never recorded (ADR 0660 §4): the answer says it, with what two claimants leave unsettled.
out.Said = append(out.Said, passes.Lines()...)
for _, l := range left {
out.Said = append(out.Said, l.String())
}
@@ -1,43 +0,0 @@
package resolution
import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// artifactSweepVerbs are the seat's verbs, with the controller's arguments so a caller changes only the seat it asks.
func artifactSweepVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "artifacts", Description: "Every artifact the mesh made that lives in its artifact store, each with its " +
"repository, digest, kind (image or archive) and state: kept, and why (a want names it, or one of the five " +
"newest builds of its module in the builder's log), eligible (made, kept for no reason, not yet let go), or " +
"unrecorded (in the store and named by nothing the mesh made: never removed). Reads only.",
Input: schema(map[string]string{
"repository": "only this repository's references, as <module>/<artifact> (the counts stay whole)",
"collected": "\"true\": list the collected references too (counted always)",
}, nil, "collected")},
{Name: "held", Description: "Whether the artifact store holds each reference given, asked with one HEAD each: " +
"held, gone (the store answered 404) or unknown with why. The builder asks it before it reuses a done key. " +
"Reads only.",
Input: listed(schema(map[string]string{
"references": "the artifacts' references, artifact-store://<module>/<artifact>/blobs/sha256:<hex>: a list, " +
"or one text separated by commas",
}, []string{"references"}), "references")},
{Name: "collect", Description: "The artifact store's sweep, now: what is eligible is let go of. Without confirm a " +
"dry run: what would be let go, nothing held or deleted. With confirm and why: every kept archive held first, " +
"then each eligible artifact let go of, oldest first, an index with the platform manifests no kept index " +
"names, each deletion recorded in the hand-act log. Bounded by most (500 by default, at most 5000) and 45 " +
"seconds; what is left is said. Never touches a digest the mesh did not make.",
Input: schema(map[string]string{
"confirm": "\"true\": let go of what is eligible (needs why); without it nothing is done",
"why": "with confirm: why, required and recorded",
"most": "let go of at most this many (500 by default, at most 5000)",
}, nil, "confirm")},
{Name: "mirrors", Description: "The bases the mesh copied into its artifact store, each kept (a kept build stood on " +
"it), eligible (only builds no longer kept stood on it) or unrecorded (no build the builder's log holds stood on " +
"it: never removed), by what the builder's log says each build stood on. Records nothing by hand.",
Input: schema(map[string]string{}, nil)},
{Name: "images", Description: "Every container image one machine's want and have name, each with the resources " +
"naming it: what the machine keeps when it prunes its images. Reads only.",
Input: schema(map[string]string{"node": "the machine's name"}, []string{"node"})},
}
}
@@ -1,33 +0,0 @@
package resolution
import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// checkerVerbs are the checker seat's verbs: ask a try, read one, end one by hand. Optional until the checker serves them
// everywhere.
func checkerVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "try", Optional: true, Description: "Try a pushed head before its pull request is opened: the checker " +
"judges it as it judges a pull request's head, merged onto its trunk's head, its unit and integration " +
"levels through the build modules, and owes the forge nothing. Answers the try's id at once; try-read " +
"reads where it stands. A try of the same head that waits or runs is the answer to a second ask; at most " +
"4 wait or run at once, mesh-wide.",
Input: schema(map[string]string{"repository": "<owner>/<name>",
"head": "the full commit to try, 40 or 64 hex digits",
"trunk": "the trunk commit to merge it onto, or the trunk's branch for its head now; the trunk's head when absent",
"tests": "a go test -run pattern: only those tests run, and nothing is kept",
"level": "unit or integration: only this level; both when absent"},
[]string{"repository", "head"}),
Replaces: []string{"sh merge-check.sh", "docker run postgres"}},
{Name: "try-read", Optional: true, Description: "Where a try stands, as stored: queued, running, or its end " +
"(pass, fail, or error when it could not run), with what was said of it, the head and the trunk commit " +
"it was merged onto, the time it took and what failed first. Read-only.",
Input: schema(map[string]string{"id": "the try's id, try-<digits>"}, []string{"id"})},
{Name: "try-end", Optional: true, Description: "End a try that waits or runs, as error, by hand: say why (kept in " +
"what it says). Its place among the tries is free at once and the next try of its head asks anew with a new " +
"id. A try that ended already is refused.",
Input: schema(map[string]string{"id": "the try's id, try-<digits>", "why": "why it is ended, in words"},
[]string{"id", "why"})},
}
}
@@ -4,6 +4,8 @@ import (
"encoding/json"
"errors"
"fmt"
"maps"
"slices"
"sort"
"strconv"
"strings"
@@ -240,7 +242,14 @@ func resolutionCompose(r mcat.Resolution, with mcat.Rendering, owner map[string]
}
if answered {
r.Needs[i].Serves = serves
continue
}
// No module of the composed set serves it: its provider is held on this machine (composed apart, as what runs of
// it), and offers what its manifest serves. None of its own names is composed here, so a key that names one is
// left out and said on the need, as a provider elsewhere leaves it out (ServedLeftOut, novox/hq ADR 0346 §3),
// never handed to the consumer as a placeholder its binding then refuses (issue 1921, novox: "mesh-build-node has
// a file that says ${bound:route:name-web}, and mesh-build-node does not require route").
r.Needs[i].Serves, r.Needs[i].LeftOut = withoutOwnNames(r.Needs[i], r.Node)
}
// Once, from every module's listens -- not per module. A module receiving only its own ports
@@ -2526,3 +2535,26 @@ func resolutionGeneratedHere(r mcat.Resolution, with mcat.Rendering) (map[string
}
return generated, nil
}
// withoutOwnNames is what a need's provider serves without each key whose value names one of the provider's own composed
// names (${bound:…}), and the need's left-out keys with each such key said: a provider not composed in this set
// composes none of its names here.
func withoutOwnNames(n mcat.Needed, node string) (map[string]any, []string) {
var asking []string
for _, key := range sortedAnyKeys(n.Serves) {
if text, ok := n.Serves[key].(string); ok && bound.MatchString(text) {
asking = append(asking, key)
}
}
if len(asking) == 0 {
return n.Serves, n.LeftOut
}
out := maps.Clone(n.Serves)
said := slices.Clone(n.LeftOut)
for _, key := range asking {
delete(out, key)
said = append(said, fmt.Sprintf("%s serves %s without %q: it names %s, and %s composes no names of %s, which it "+
"holds (novox/hq ADR 0346)", n.Module, n.Name, key, bound.FindString(n.Serves[key].(string)), orHere(node), n.Module))
}
return out, said
}
@@ -1,75 +0,0 @@
package resolution
import (
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliveryclient"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// deliveryRemovedInR6 is the row of a verb release 6 removed from the delivery seat.
func deliveryRemovedInR6(name string) mcat.Verb {
return mcat.Verb{Name: name, Description: meshdeliveryclient.RemovedPrefix + meshdeliveryclient.RemovedVerbs()[name], Input: schema(nil, nil), Optional: true}
}
// deliveryVerbs are the delivery seat's tools: what reads the merges and checks, and the acts of a person and healer H2
// on a merge (ADR 0539 clause 17).
func deliveryVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "deliveries", Description: "Every delivery not final, and those that ended in the last day, one line " +
"each: its id (owner/repository@commit), its state, its group, what it waits for and since when. Narrowed " +
"by state, repository or group.",
Input: schema(map[string]string{"state": "one state, e.g. delivering or held",
"repository": "owner/repository", "group": "a group's id (its branch name)",
"all": "\"true\": the final ones of the last thirty days too"}, nil, "all")},
deliveryRemovedInR6("times"),
{Name: "show", Description: "One delivery or group whole: its delivery plan (what it builds, what each " +
"machine receives, what is not an ordinary send), every transition with when and why, the machine " +
"steps of its walk, its group and its order.",
Input: schema(map[string]string{"id": "a delivery's id, or a group's"}, []string{"id"})},
{Name: "groups", Description: "Every delivery group: its members in order, why each pair is ordered " +
"(declared, built by, version skew, engine before controller, by name), its composed check and its " +
"state, derived from its members.",
Input: schema(map[string]string{"all": "\"true\": the groups that ended too"}, nil, "all")},
deliveryRemovedInR6("what-if"),
{Name: "checks", Description: "What the mesh's checks said of a pull request's head or of one commit: each " +
"of the commit's mesh statuses (mesh/check, the one a main requires from R4e, hq's mesh/repo-check, " +
"mesh/delivery, …) with its state, " +
"description and when it was set; the merge check's full verdict as the controller said it — each " +
"layer's summary, the machine that ran it, when, its build id and its report; and whether the branch's " +
"protection would let it merge, every required status being success.",
Input: schema(map[string]string{"repository": "owner/repository",
"number": "a pull request's number: its head is read",
"commit": "a commit's sha, or the start of one, instead of a pull request"}, []string{"repository"}),
// Added after the seat's first holder shipped: optional until mesh-delivery serves it everywhere.
Optional: true},
{Name: "table", Description: "The state table every delivery runs by: each transition with its guard, " +
"each state's bound and what healer H2 may do once it has passed; and the machine steps' table.",
Input: schema(map[string]string{}, nil)},
{Name: "stalled", Description: "Every delivery held past its state's bound, with the transition the table " +
"lets healer H2 take for it, or none.",
Input: schema(map[string]string{}, nil)},
{Name: "recheck", Description: "Check a rejected or ready delivery again: it goes back to proposed and the " +
"controller is asked for its check. With why.",
Input: schema(map[string]string{"id": "the delivery's id", "why": "why, kept with the transition"},
[]string{"id", "why"})},
deliveryRemovedInR6("release"),
deliveryRemovedInR6("stop"),
{Name: "close", Description: "Healer H2's verb: take the one transition the table names for a delivery held " +
"past its bound. Refused for any other.",
Input: schema(map[string]string{"id": "the delivery's id", "why": "the condition it answers"},
[]string{"id", "why"})},
deliveryRemovedInR6("retire-history"),
{Name: "settle", Description: "Ask the operator to settle deliveries held because their merge reached the " +
"trunk without a passing check: one ask on the operator's channel for all of them, as an Approve is asked. " +
"The verb moves nothing; on the operator's answer each goes to merged, and whether it is on the machines " +
"is derived as for any merge (novox/hq ADR 0556). Refused for a delivery not held, or held for another " +
"reason, naming why.",
Input: map[string]any{"type": "object", "additionalProperties": false, "required": []string{"ids", "why"},
"properties": map[string]any{
"ids": map[string]any{"type": "array", "items": map[string]any{"type": "string"}, "minItems": 1,
"description": "the held deliveries' ids (owner/repository@commit)"},
"why": map[string]any{"type": "string", "description": "why they may be settled, shown to the operator"},
}},
// Added after the seat's release 6 holder shipped: optional until the gate serves it everywhere.
Optional: true},
}
}
@@ -1,30 +0,0 @@
package resolution
import (
"testing"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliveryclient"
)
// Each verb release 6 removed from the delivery seat is a row refusing it by name with the replacement the seat's client
// gives (the list is the contract's, not the SDK's: novox/hq ADR 0634 §2), and the rows are exactly those verbs.
func TestTheDeliverySeatRefusesTheRemovedVerbsByNameWithTheClientsReplacement(t *testing.T) {
removed := meshdeliveryclient.RemovedVerbs()
rows := 0
for _, v := range deliveryVerbs() {
want, isRemoved := removed[v.Name]
if !isRemoved {
continue
}
rows++
if v.Description != meshdeliveryclient.RemovedPrefix+want || !v.Optional {
t.Errorf("%s = %q (optional %v), want %q", v.Name, v.Description, v.Optional, meshdeliveryclient.RemovedPrefix+want)
}
}
if rows != len(removed) {
t.Errorf("%d rows refuse a removed verb, the client names %d", rows, len(removed))
}
if len(removed) != 5 {
t.Errorf("the removed verbs are %v", removed)
}
}
@@ -1,67 +0,0 @@
package resolution
import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// enrolmentDecision is the record that made the seat.
const enrolmentDecision = "novox/hq ADR 0587"
// enrolmentVerbs are the seat's verbs: what reads all it holds and does by hand all it does (ADR 0587 §8). Not optional:
// the seat is new with its holder, so no holder of before serves fewer.
func enrolmentVerbs() []mcat.Verb {
str := func(description string) map[string]any {
return map[string]any{"type": "string", "description": description}
}
return []mcat.Verb{
{Name: mcat.EnrolmentVerbIssue, Description: "Issue a one-time token for a new machine, on the operator's Approve: " +
"refused at once, naming every finding, for a name that is not a node name or that a node record, an assignments " +
"key or an open enrolment holds, and for every call before the controller's lease is handed away. Otherwise " +
"reserves the lowest free private-network address, asks the operator under the seat's own name and, on the " +
"warrant, adds the node to the assignments with its tunnel key, asks the vault for the token and says enrolment-issued. " +
"The token itself is read only through tokens.",
Input: schema(map[string]string{
"node": "the new machine's name in the mesh",
"why": "why it joins, shown to the operator with the ask",
"expires": "how long the token lasts, a duration such as 1h or 24h: one hour when not given, at most 24 hours",
// The operator's decision of 2026-11-02 01:05: the tunnel key is given at issue, so on the Approve the node
// enters the assignments with it and the hub peers it before redeem.
"overlay_key": "the new machine's private network (tunnel) public key, as `nox-mesh-host key` prints it",
}, []string{"node", "why", "overlay_key"})},
{Name: mcat.EnrolmentVerbRedeem, Description: "Spend a token: asked by a joining node-engine dialled as enrol.<node> " +
"alone, with the public halves of its keys and its identity key's proof over them. Spends the token, asks the " +
"vault for the node's account and certificate, writes the node into the assignments, and answers the account sealed " +
"to the node, its certificate and the vault's public half, signed by the vault. Asked again with the same keys " +
"it finishes what is not done and answers the same; with other keys it is refused.",
Input: map[string]any{"type": "object", "additionalProperties": false,
"required": []string{"node", "public_key", "sealing_key", "proof"},
"properties": map[string]any{
"node": str("the node the token was issued for"),
"public_key": str("the identity key's public half, base64"),
"overlay_key": str("the private network key's public half"),
"sealing_key": str("the sealing key's public half, which the account is sealed to"),
"serving_key": str("the serving key's public half, which the certificate binds"),
"profile": map[string]any{"type": "object", "description": "what the machine is, as its record says it"},
"tunnel": map[string]any{"type": "object", "description": "a tunnel found on the machine: everything but its private key"},
"proof": str("the identity key's signature over the redeem's proof bytes (delivery.RedeemProof), base64"),
}}},
{Name: mcat.EnrolmentVerbConfirm, Description: "Confirm an enrolment: asked by the node-engine dialled as its own " +
"node login alone, the runtime's proof of that caller being the confirmation. Says node-enrolled and ends the " +
"enrolment.",
Input: schema(map[string]string{}, nil)},
{Name: mcat.EnrolmentVerbRevoke, Description: "End an enrolment at any stage, with why, without an Approve: withdraws " +
"an open ask, deletes the node's key and its address, has the vault retire an account it already minted for " +
"it, records the act as a hand act and says enrolment-revoked.",
Input: schema(map[string]string{"node": "the node", "why": "why, recorded with the hand act"},
[]string{"node", "why"})},
{Name: mcat.EnrolmentVerbTokens, Description: "Read every open enrolment, or one: its stage, expiry, ask and revision, " +
"the bucket's fill against its cap. For a node at live, its " +
"token sealed to the operator key, asked of the vault at that moment and relayed as the vault sealed it. With " +
"history, the ended enrolments from the seat's events, as far back as the events are kept.",
Input: schema(map[string]string{"node": "one node", "history": "\"true\": the ended enrolments too"},
nil, "history")},
{Name: mcat.EnrolmentVerbAgain, Description: "Raise an issue's ask again when the operator's channel does not hold it, " +
"and nothing more.",
Input: schema(map[string]string{"node": "the node whose ask is raised again"}, []string{"node"})},
}
}
@@ -6,36 +6,6 @@ import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// resolverVerbs is the contract every holder of node-resolver serves (novox/hq ADR 0247): what is routed
// where, route a set of domains, and take a route away. The same verbs are served on the machine itself to
// the modules there, so what a VPN pushed never crosses the bus to be routed; on the mesh they are the
// operator's way to read and correct the same.
func resolverVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "routes", Description: "What this machine's own resolver sends where: the mesh's resolvers, " +
"which answer every name not routed elsewhere, and each link given servers of its own with the " +
"domains routed to them. Also the resolver file's outside writes it kept, newest first: when, who " +
"wrote it as far as the file says, whether a module took it, and when the resolver's own file was " +
"put back.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"resolvectl status", "resolvectl dns", "resolvectl domain"}},
{Name: "route", Description: "Send these domains, and every name under them, to these servers over " +
"this link — and only them: the link is never the machine's default route for names, and the " +
"mesh's own domain is refused. Replaces whatever the link was given before. A link that goes away " +
"takes its route with it.",
Input: schema(map[string]string{
"link": "the network link the servers are reached over, by name (a VPN's tunnel interface)",
"domains": "the domains to route there, separated by spaces or commas",
"servers": "the servers' addresses, separated by spaces or commas",
}, []string{"link", "domains", "servers"}),
Replaces: []string{"resolvectl dns", "resolvectl domain", "resolvectl default-route"}},
{Name: "unroute", Description: "Take one link's route away: its domains go back to the mesh's " +
"resolvers. Nothing changes when the link has none.",
Input: schema(map[string]string{"link": "the network link, by name"}, []string{"link"}),
Replaces: []string{"resolvectl revert"}},
}
}
// holdsSeat says whether a module claims a seat, by its current name.
func holdsSeat(m mcat.Manifest, seat string) bool {
for _, c := range m.Claims {
-74
View File
@@ -1,74 +0,0 @@
package resolution
import (
"fmt"
"sort"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// SeatsToPassOn reads the mesh for seats held once whose holder on record has left (novox/hq ADR 0449 §4).
//
// **A holder on record that is assigned on no machine holds nothing, and a seat that answers nothing is
// the day a delivery removed its module.** Of such a seat, exactly one assignment in the mesh able to hold
// it (it claims the seat at its scope and CanHold accepts it) takes it over: that is a pass, and the caller
// records it and writes the hand act. With no such assignment the record stays; with two or more nothing is
// recorded and the answer says so, naming both and the `seat --to` that settles it. A holder still assigned
// anywhere changes nothing: the other claimants stand beside it, eligible and silent (ADR 0131). Only mesh
// seats held once are read: a replicated seat's holders are added by an act (ADR 0223), a kinded bench is held
// by its claims, and a seat with several holders on record is a store that disagrees with the mesh (refused
// by the resolver).
func SeatsToPassOn(holdings []mcat.Held, assignments []mcat.Assignment) ([]mcat.SeatPass, []string) {
assigned := map[string]bool{}
for _, a := range assignments {
assigned[a.Manifest.Module] = true
}
bySeat := map[string][]mcat.Held{}
var order []string
for _, h := range holdings {
s, ok := mcat.SeatNamed(h.Claim)
if !ok || s.Replicated || s.Scope != mcat.ScopeMesh || h.Scope != mcat.ScopeMesh || mcat.KindedBenches[s.Name] {
continue
}
if _, seen := bySeat[s.Name]; !seen {
order = append(order, s.Name)
}
bySeat[s.Name] = append(bySeat[s.Name], h)
}
sort.Strings(order)
var passes []mcat.SeatPass
var said []string
for _, name := range order {
recs := bySeat[name]
if len(recs) != 1 || assigned[recs[0].Module] {
continue
}
seat, _ := mcat.SeatNamed(name)
var claimants []mcat.Held
for _, a := range assignments {
if mcat.CanHold(a.Manifest, seat) != nil {
continue
}
claimants = append(claimants, mcat.Held{Claim: seat.Name, Scope: seat.Scope, Node: a.Node, Module: a.Manifest.Module})
}
sort.Slice(claimants, func(i, j int) bool {
if claimants[i].Node != claimants[j].Node {
return claimants[i].Node < claimants[j].Node
}
return claimants[i].Module < claimants[j].Module
})
switch len(claimants) {
case 0:
case 1:
passes = append(passes, mcat.SeatPass{Seat: seat, From: recs[0], To: claimants[0]})
default:
said = append(said, fmt.Sprintf(
"%s on %s is on record as the holder of %q and is assigned nowhere, and %s on %s and %s on %s both claim it "+
"— nothing is recorded; `seat %s --to <node>/<module>` records the one that holds it",
recs[0].Module, recs[0].Node, seat.Name, claimants[0].Module, claimants[0].Node,
claimants[1].Module, claimants[1].Node, seat.Name))
}
}
return passes, said
}
@@ -1,80 +0,0 @@
package resolution
import (
"strings"
"testing"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
func onRecord(node, module string) []mcat.Held {
return []mcat.Held{{Claim: "mesh-broker", Scope: mcat.ScopeMesh, Node: node, Module: module}}
}
// 1317/AC1
func TestASeatWhoseHolderIsAssignedNowhereIsPassedToItsOnlyClaimant(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
passes, said := SeatsToPassOn(onRecord("novox", "old-broker"),
[]mcat.Assignment{{Node: "novox", Manifest: newBroker()}})
if len(said) != 0 || len(passes) != 1 {
t.Fatalf("the only claimant was not given the seat: passes=%v said=%v", passes, said)
}
p := passes[0]
if p.Seat.Name != "mesh-broker" || p.From.Module != "old-broker" || p.From.Node != "novox" ||
p.To.Module != "new-broker" || p.To.Node != "novox" {
t.Fatalf("the pass does not name the seat, the former holder and the new one: %+v", p)
}
}
// 1317/AC2
func TestWithTwoClaimantsAndTheHolderGoneNothingIsPassedAndTheRefusalNamesBoth(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
passes, said := SeatsToPassOn(onRecord("novox", "gone"), []mcat.Assignment{
{Node: "novox", Manifest: newBroker()}, {Node: "ace", Manifest: oldBroker()}})
if len(passes) != 0 || len(said) != 1 {
t.Fatalf("two claimants: passes=%v said=%v", passes, said)
}
for _, want := range []string{"old-broker", "new-broker", "seat mesh-broker --to"} {
if !strings.Contains(said[0], want) {
t.Fatalf("the refusal does not say %q: %s", want, said[0])
}
}
}
// 1317/AC3
func TestAHolderStillAssignedSomewhereChangesNothing(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
// Assigned on another machine than the record names: still assigned somewhere.
passes, said := SeatsToPassOn(onRecord("novox", "old-broker"), []mcat.Assignment{
{Node: "ace", Manifest: oldBroker()}, {Node: "novox", Manifest: newBroker()}})
if len(passes) != 0 || len(said) != 0 {
t.Fatalf("a holder still assigned lost its seat: passes=%v said=%v", passes, said)
}
}
// 1317/AC4
func TestWithNoClaimantTheRecordStays(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
passes, said := SeatsToPassOn(onRecord("novox", "old-broker"), []mcat.Assignment{
{Node: "novox", Manifest: mcat.Manifest{Module: "other"}}})
if len(passes) != 0 || len(said) != 0 {
t.Fatalf("with no claimant: passes=%v said=%v", passes, said)
}
// A claimant that cannot hold the seat (it does not provide what the seat delivers) is no claimant.
bad := mcat.Manifest{Module: "no-bus", Claims: []mcat.Claim{{Name: "mesh-broker", Scope: mcat.ScopeMesh}}}
passes, said = SeatsToPassOn(onRecord("novox", "old-broker"), []mcat.Assignment{{Node: "novox", Manifest: bad}})
if len(passes) != 0 || len(said) != 0 {
t.Fatalf("a module that cannot hold the seat was passed it: passes=%v said=%v", passes, said)
}
}
// A replicated seat is never passed: its holders are added by an act (ADR 0223).
func TestAReplicatedSeatIsNeverPassed(t *testing.T) {
record := []mcat.Held{{Claim: "mesh-dns-resolver", Scope: mcat.ScopeMesh, Node: "novox", Module: "gone"}}
resolver := mcat.Manifest{Module: "resolver", Provides: []mcat.Offer{{Name: "wildcard-resolution", Scope: mcat.ScopeMesh}},
Claims: []mcat.Claim{{Name: "mesh-dns-resolver", Scope: mcat.ScopeMesh}}}
passes, _ := SeatsToPassOn(record, []mcat.Assignment{{Node: "novox", Manifest: resolver}})
if len(passes) != 0 {
t.Fatalf("a replicated seat was passed: %v", passes)
}
}
-15
View File
@@ -4,21 +4,6 @@ import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// configInput is the schema of mesh-config's verb: a value of the key's own type (a number for a number, never its
// text), and a limit that is a number; every other argument a string.
func configInput() map[string]any {
in := schema(map[string]string{"action": "list (when not given), set, clear or history",
"key": "the config key", "value": "for set: the value, of the key's type",
"module": "a module: its own value, with node", "node": "a node",
"why": "for set and clear: why", "by": "who gave the value at the controller's terminal",
"limit": "for history: at most this many, newest"}, nil)
props := in["properties"].(map[string]any)
props["value"] = map[string]any{"description": "for set: the value, of the key's type: a number, a string, " +
"a boolean, or for a trusted key an object or a list"}
props["limit"] = map[string]any{"type": "number", "description": "for history: at most this many, newest"}
return in
}
func providesAt(m mcat.Manifest, provision, scope string) bool {
for _, o := range m.Provides {
if o.Name == provision && o.At() == scope {
+94
View File
@@ -1,6 +1,12 @@
package resolution
import (
"go/ast"
"go/parser"
"go/token"
"io/fs"
"path/filepath"
"strconv"
"strings"
"testing"
@@ -70,4 +76,92 @@ func TestTheHolderIsTheModuleNotTheMachine(t *testing.T) {
t.Fatalf("the holder was not told apart from a neighbour: %+v", holder)
}
// A seat's verbs are its definition's, read from the bus (novox/hq ADR 0635 §1): the node-engine keeps no copy of
// them, so no copy can drift from the seat as the seven it kept did (issue 1921). A copy is a list of the catalogue's
// verbs written in the code; the guard proves it finds one, then finds none in the node-engine.
t.Run("no seat's verbs are copied into the node-engine", func(t *testing.T) {
copied := `package x
import mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
func someVerbs() []mcat.Verb { return []mcat.Verb{{Name: "routes"}} }
`
if got := verbCopies(t, "copied.go", copied); len(got) == 0 {
t.Fatal("the guard finds no copy in a file that holds one, so it proves nothing")
}
var found []string
root := filepath.Join("..", "..", "..")
err := filepath.WalkDir(root, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() && (d.Name() == "testdata" || strings.HasPrefix(d.Name(), ".")) && p != root {
return filepath.SkipDir
}
if d.IsDir() || !strings.HasSuffix(p, ".go") || strings.HasSuffix(p, "_test.go") {
return nil
}
found = append(found, verbCopies(t, p, nil)...)
return nil
})
if err != nil {
t.Fatal(err)
}
if len(found) > 0 {
t.Errorf("the node-engine copies a seat's verbs instead of reading the seat list from the bus: %v", found)
}
})
}
// verbCopies is where one Go file writes the catalogue's verbs itself: a function answering a list of them, or a verb
// written as a literal.
func verbCopies(t *testing.T, name string, src any) []string {
t.Helper()
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, name, src, parser.SkipObjectResolution)
if err != nil {
t.Fatal(err)
}
alias := ""
for _, imp := range f.Imports {
if p, _ := strconv.Unquote(imp.Path.Value); p == "git.novox.be/novox/mesh-sdk/go/catalogue" {
alias = "catalogue"
if imp.Name != nil {
alias = imp.Name.Name
}
}
}
if alias == "" {
return nil
}
isVerb := func(e ast.Expr) bool {
if a, ok := e.(*ast.ArrayType); ok {
e = a.Elt
}
sel, ok := e.(*ast.SelectorExpr)
if !ok || sel.Sel.Name != "Verb" {
return false
}
x, ok := sel.X.(*ast.Ident)
return ok && x.Name == alias
}
var out []string
ast.Inspect(f, func(n ast.Node) bool {
switch n := n.(type) {
case *ast.FuncDecl:
if n.Type.Results != nil {
for _, r := range n.Type.Results.List {
if isVerb(r.Type) {
out = append(out, fset.Position(n.Pos()).String()+" "+n.Name.Name)
}
}
}
case *ast.CompositeLit:
if n.Type != nil && isVerb(n.Type) {
out = append(out, fset.Position(n.Pos()).String())
}
}
return true
})
return out
}
@@ -1,6 +1,7 @@
package resolution
import (
"slices"
"strings"
"testing"
@@ -75,6 +76,29 @@ func TestAConsumerBesideTheProviderIsServedItsName(t *testing.T) {
if strings.Contains(binding, "${bound:") {
t.Fatalf("a placeholder reached the consumer:\n%s", binding)
}
// Issue 1921 (the group check of 8a80e8e86858, novox): the provider held on this machine (composed apart, as what runs
// of it) is not in the composed set, so none of its own names is composed here. The consumer beside it is told
// what it serves without the keys that name them, each said on the need, never the placeholder: its binding refused
// the consumer with "has a file that says ${bound:route:name-web}, and lab does not require route".
held, err := Resolve(shelf(minimalRouteProxy(), m, consumer), []string{"route-proxy", "forge", "lab"}, node, mcat.World{})
if err != nil {
t.Fatal(err)
}
held.Modules = slices.DeleteFunc(slices.Clone(held.Modules), func(m mcat.Manifest) bool { return m.Module == "forge" })
out, err = Declaration(held, mcat.Rendering{})
if err != nil {
t.Fatalf("a consumer beside a held provider: %v", err)
}
for _, r := range out {
if c, _ := r["content"].(string); r["path"] == "/var/lib/lab/package-registry.json" && (strings.Contains(c, "${bound:") || !strings.Contains(c, "npm-path")) {
t.Fatalf("the binding beside a held provider:\n%s", c)
}
}
for _, n := range held.Needs {
if n.Name == "npm-package-registry" && (len(n.LeftOut) != 1 || !strings.Contains(n.LeftOut[0], `"public-root"`)) {
t.Fatalf("the need beside a held provider says not what was left out: %+v", n)
}
}
}
// From another machine, the provider's served values carry what that machine composed, and a key it
-78
View File
@@ -1,78 +0,0 @@
package resolution
import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// nfsServerVerbs is the contract every holder of node-nfs-server serves (novox/hq ADR 0263).
func nfsServerVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "exports", Description: "Every share this machine exports: its name, its path, read-write or " +
"read-only, the owner every client is mapped to (uid and gid), each node it is exported to with that " +
"node's private address and access (only the nodes the server grants it to and that ask for it), " +
"what is granted and not asked for or asked for and not granted, and whether the kernel holds it " +
"now. Also the exports found that are not the mesh's: a dataset's sharenfs property, a line in " +
"/etc/exports.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"exportfs -v", "cat /etc/exports", "zfs get sharenfs"}},
{Name: "clients", Description: "Which machines have mounted which share now, as the NFS server " +
"knows its clients.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"ss -tn sport = :2049", "cat /proc/fs/nfsd/clients/*/info"}},
{Name: "test", Description: "Whether this machine exports one share for the mesh now, and whether its " +
"NFS service is up; with an address, also whether the share is exported to that address: a node's " +
"own private address, when the server grants it the share and the node asks for it. What a machine " +
"mounting the share asks before it mounts.",
Input: schema(map[string]string{
"share": "the share, by its name",
"address": "a node's private address, to ask whether the share is exported to it (optional)",
}, []string{"share"}),
Replaces: []string{"showmount -e"}},
{Name: "reload", Description: "Have the kernel read this machine's export files again now (exportfs " +
"-ra) and answer the shares as it then holds them. The module's own process writes its export " +
"file; this is for after a change made outside it. Changes no shared path.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"exportfs -ra"}},
{Name: "adopt", Description: "Take over an export a person made by hand: clear a dataset's sharenfs " +
"property for a path the mesh's own export now serves — only when that export is live. Without " +
"confirm, says what it would do and changes nothing.",
Input: schema(map[string]string{
"path": "the shared path whose hand-made export is taken over",
"confirm": "\"true\": change it (needs why); anything else is a dry run",
"why": "why, for the record",
}, []string{"path"}, "confirm"),
Replaces: []string{"zfs set sharenfs=off"}},
}
}
// mountsVerbs is the contract every holder of node-mounts serves (novox/hq ADR 0263).
func mountsVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "list", Description: "Every mount point on this machine: its fstab line, the mesh's mount and " +
"automount units for it, its state (armed, mounted, absent, unreachable, failed) and since when, " +
"and which settings asked for it. A password in a mount's options is never shown.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"cat /etc/fstab", "findmnt", "systemctl list-units --type=mount"}},
{Name: "test", Description: "Whether one share's or occasional source's server answers from this " +
"machine now, without touching its mount point.",
Input: schema(map[string]string{"name": "the share or source, by its name"}, []string{"name"}),
Replaces: []string{"showmount -e", "ping"}},
{Name: "mount", Description: "Mount one share or occasional source now, rather than at its first " +
"access.",
Input: schema(map[string]string{"name": "the share or source, by its name"}, []string{"name"}),
Replaces: []string{"mount"}},
{Name: "unmount", Description: "Release one share or occasional source now; its automount stays " +
"armed, so the next access mounts it again.",
Input: schema(map[string]string{"name": "the share or source, by its name"}, []string{"name"}),
Replaces: []string{"umount"}},
{Name: "adopt", Description: "Take over a mount a person wrote by hand: comment out the /etc/fstab " +
"line for a mount point the mesh's own units now serve, keeping a copy of the file — only when " +
"the mesh's automount for it is armed. Without confirm, says what it would do and changes nothing.",
Input: schema(map[string]string{
"mountpoint": "the mount point whose fstab line is taken over",
"confirm": "\"true\": change it (needs why); anything else is a dry run",
"why": "why, for the record",
}, []string{"mountpoint"}, "confirm"),
Replaces: []string{"sed -i /etc/fstab"}},
}
}
-59
View File
@@ -1,59 +0,0 @@
package resolution
import (
"maps"
"slices"
)
// schema is a JSON schema for an object of string properties, which is every argument the verbs
// above take. Kept small on purpose: a schema an agent cannot read is a tool it cannot call. The
// switches are the properties that are "true" or "false" and nothing else, said in the schema as an
// enum so a caller sees it and the verb can refuse any other word rather than read it as false.
//
// **The schema is the whole of what a verb takes** (novox/hq issue 244): an argument it does not
// name is refused when the verb is called, never passed over.
// withObject adds to a verb's input s the property name, an object of the string fields given, each required.
func withObject(s map[string]any, name, description string, fields map[string]string) map[string]any {
inner := schema(fields, slices.Sorted(maps.Keys(fields)))
inner["description"], inner["additionalProperties"] = description, false
s["properties"].(map[string]any)[name] = inner
return s
}
func schema(properties map[string]string, required []string, switches ...string) map[string]any {
isSwitch := map[string]bool{}
for _, s := range switches {
if _, declared := properties[s]; !declared {
panic("a switch that is not a property: " + s)
}
isSwitch[s] = true
}
props := map[string]any{}
for name, description := range properties {
p := map[string]any{"type": "string", "description": description}
if isSwitch[name] {
p["enum"] = []string{"true", "false"}
}
props[name] = p
}
out := map[string]any{"type": "object", "properties": props}
if len(required) > 0 {
out["required"] = required
}
return out
}
// listed makes the named properties of a schema lists of text: a caller gives them as a JSON array (or, as
// any argument, one text separated by commas).
func listed(in map[string]any, names ...string) map[string]any {
props, _ := in["properties"].(map[string]any)
for _, n := range names {
p, _ := props[n].(map[string]any)
if p == nil {
panic("a list that is not a property: " + n)
}
props[n] = map[string]any{"type": "array", "items": map[string]any{"type": "string"},
"description": p["description"]}
}
return in
}
+37 -14
View File
@@ -25,9 +25,10 @@ func unresolvable(err error) bool {
return errors.As(err, &n)
}
// Passes is what a composition found that the mesh must record, and did not: each seat held once whose holder on record
// is assigned nowhere that exactly one claimant takes over (novox/hq ADR 0449 clause 4), and the lines that say what was
// found. The caller records a pass and writes its hand act; this package does neither.
// Passes is what a composition found and composed with: each seat held once whose holder on record has left it (assigned
// nowhere, or none of its assignments' manifests claims the seat) that exactly one claimant takes over (novox/hq ADR 0449
// clause 4, widened by ADR 0660), and the lines that say what was found. A pass is composed and said, never recorded: neither
// this package nor its caller writes the holdings (ADR 0660 clause 4).
type Passes struct {
// Passes are the seats to pass, in the order they were found.
Passes []catalogue.SeatPass
@@ -92,8 +93,8 @@ func (c *composer) theRestOfTheMeshNow(exclude string) (catalogue.World, Passes,
pins map[string]catalogue.Chosen
}
var others []candidate
// Every machine's assignments, before the holdings are read by any resolution (novox/hq ADR 0449 clause 4): a seat
// held once whose holder on record is assigned on no machine passes to its only claimant here.
// Every machine's assignments, before the holdings are read by any resolution (novox/hq ADR 0449 clause 4, ADR 0660): a
// seat held once whose holder on record has left it passes to its only claimant here, composed and not recorded.
everyAssigned := map[string][]string{}
for _, n := range nodes {
if len(n.Assigned) > 0 {
@@ -101,7 +102,7 @@ func (c *composer) theRestOfTheMeshNow(exclude string) (catalogue.World, Passes,
}
}
var passes Passes
holdings, passes = passSeatsOn(holdings, assignmentsOf(shelf, everyAssigned))
holdings, passes = passSeatsOn(holdings, c.assignmentsOf(shelf, everyAssigned))
for _, n := range nodes {
if n.Name == exclude {
continue
@@ -175,11 +176,11 @@ func (c *composer) theRestOfTheMeshNow(exclude string) (catalogue.World, Passes,
return world, passes, left, nil
}
// passSeatsOn finds, in the composition that finds it, the new holder of each seat held once whose holder on record is
// assigned nowhere and which has exactly one claimant able to hold it (novox/hq ADR 0449 clause 4), and answers the
// passSeatsOn finds, in the composition that finds it, the new holder of each seat held once whose holder on record has left
// it and which has exactly one claimant able to hold it (catalogue.SeatsToPassOn, novox/hq ADR 0660), and answers the
// holdings as they are after it, with the passes. Nothing is recorded.
func passSeatsOn(holdings []catalogue.Held, assignments []catalogue.Assignment) ([]catalogue.Held, Passes) {
passes, said := resolution.SeatsToPassOn(holdings, assignments)
passes, said := catalogue.SeatsToPassOn(holdings, assignments)
out := Passes{Said: said}
for _, p := range passes {
out.Passes = append(out.Passes, p)
@@ -196,9 +197,11 @@ func passSeatsOn(holdings []catalogue.Held, assignments []catalogue.Assignment)
return holdings, out
}
// assignmentsOf are the assignments of every machine, as the passing of a seat reads them. A module assigned and
// missing from the shelf is still assigned: an assignment with no claims, so its holder is not taken for gone.
func assignmentsOf(shelf map[string]catalogue.Manifest, assigned map[string][]string) []catalogue.Assignment {
// assignmentsOf are the assignments of every machine, as the passing of a seat reads them (novox/hq ADR 0660 clause 2): each
// with the manifest of the build its node runs where the Input holds it (the node's Kept: what it runs of a module kept at
// that build), else the trunk's on the shelf, which for the composing node's own modules is already the build they are
// wanted at. A module assigned and missing from both is still assigned, marked unread, so its holder is not taken for gone.
func (c *composer) assignmentsOf(shelf map[string]catalogue.Manifest, assigned map[string][]string) []catalogue.Assignment {
var out []catalogue.Assignment
names := make([]string, 0, len(assigned))
for node := range assigned {
@@ -206,17 +209,37 @@ func assignmentsOf(shelf map[string]catalogue.Manifest, assigned map[string][]st
}
sort.Strings(names)
for _, node := range names {
var kept map[string]catalogue.Manifest
for _, n := range c.in.Nodes {
if n.Name == node {
kept = n.Kept
}
}
for _, module := range assigned[node] {
m, ok := shelf[module]
m, ok := kept[module]
if !ok {
m, ok = shelf[module]
}
if !ok {
m = catalogue.Manifest{Module: module}
}
out = append(out, catalogue.Assignment{Node: node, Manifest: m})
out = append(out, catalogue.Assignment{Node: node, Manifest: m, Unread: !ok})
}
}
return out
}
// Lines are what the passes say, one each, then what the finding itself says: a pass is composed so and recorded by nobody
// (ADR 0660 clause 4).
func (p Passes) Lines() []string {
var out []string
for _, x := range p.Passes {
out = append(out, fmt.Sprintf("%s: %s on %s is on record as its holder and holds it no more; composed as held by %s on %s, not recorded",
x.Seat.Name, x.From.Module, x.From.Node, x.To.Module, x.To.Node))
}
return append(out, p.Said...)
}
// servedOnNode is what a module on a node tells a consumer of one of its provisions, with THAT node's ports on it: the
// port the node was given over the one the mesh assigned over the manifest's own, settled with the node's settings
// layers. The one derivation for every reader of a provider's address.
+52 -3
View File
@@ -6,7 +6,6 @@ import (
"git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go/overlay"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-host/internal/compose/resolution"
)
// objects and files are the provider and consumer of mesh-controller's facts_test.go aMeshWithSecrets (cmd/mesh-controller at
@@ -184,9 +183,9 @@ func TestNoSeatIsPassedForTwoClaimantsAnAssignedHolderOrNoClaimant(t *testing.T)
// seat_pass_test.go TestAHolderAssignedButNotOnTheShelfIsNotTakenForGone.
func TestAHolderAssignedButNotOnTheShelfIsNotTakenForGone(t *testing.T) {
withSeat(t)
as := assignmentsOf(map[string]catalogue.Manifest{"gate": aBrokerClaiming("gate")},
as := (&composer{}).assignmentsOf(map[string]catalogue.Manifest{"gate": aBrokerClaiming("gate")},
map[string][]string{"novox": {"gate", "old-broker"}})
passes, _ := resolution.SeatsToPassOn(
passes, _ := catalogue.SeatsToPassOn(
[]catalogue.Held{{Claim: "mesh-broker", Scope: catalogue.ScopeMesh, Node: "novox", Module: "old-broker"}}, as)
if len(passes) != 0 {
t.Fatalf("a holder missing from the shelf lost its seat: %+v", passes)
@@ -230,3 +229,53 @@ func TestEachNodeIsResolvedWithItsOwnPinsInTheRestOfTheMesh(t *testing.T) {
t.Fatalf("a pin to a provider that does not answer did not leave the consumer out: %v %v", left, err)
}
}
// ADR 0660: a holder on record that is still assigned, whose manifest no longer claims the seat, has left it; the seat is
// composed with the one claimant, the pass is returned and said, and Input's holdings (the record) are not written.
func TestAHolderWhoseManifestDroppedTheSeatGoesToItsOnlyClaimant(t *testing.T) {
withSeat(t)
mk := func(claimants ...string) Input {
cat := map[string]catalogue.Manifest{"old-broker": {Module: "old-broker"}}
assigned := []string{"old-broker"}
for _, c := range claimants {
cat[c] = aBrokerClaiming(c)
assigned = append(assigned, c)
}
return Input{Catalogue: cat,
Nodes: []Node{{Name: "novox", Place: Place{Site: "here"}, Assigned: assigned}},
Holdings: []catalogue.Held{{Claim: "mesh-broker", Scope: catalogue.ScopeMesh, Node: "novox", Module: "old-broker", Site: "here"}}}
}
in := mk("gate")
w, passes, _, err := TheRestOfTheMesh(in, "")
if err != nil || len(passes.Passes) != 1 || passes.Passes[0].To.Module != "gate" || passes.Passes[0].From.Module != "old-broker" {
t.Fatalf("%v %+v", err, passes)
}
if len(w.Holdings) != 1 || w.Holdings[0].Module != "gate" {
t.Fatalf("the composition does not hold the pass: %+v", w.Holdings)
}
unchanged(t, func() Input { return mk("gate") }, func(in Input) {
if _, _, _, err := TheRestOfTheMesh(in, ""); err != nil {
t.Fatal(err)
}
})
_, passes, _, err = TheRestOfTheMesh(mk("gate", "other"), "")
said := strings.Join(passes.Said, "\n")
if err != nil || len(passes.Passes) != 0 || !strings.Contains(said, "gate") || !strings.Contains(said, "other") ||
!strings.Contains(said, "seat mesh-broker --to <node>/<module>") {
t.Fatalf("two claimants: %v %+v", err, passes)
}
// The build the holder's node runs still claims the seat, though the trunk dropped it: the holder keeps it (ADR 0660 §2).
running := mk("gate")
running.Nodes[0].Kept = map[string]catalogue.Manifest{"old-broker": aBrokerClaiming("old-broker")}
_, passes, _, err = TheRestOfTheMesh(running, "")
if err != nil || len(passes.Passes) != 0 || len(passes.Said) != 0 {
t.Fatalf("a holder whose running build claims the seat lost it: %v %+v", err, passes)
}
// And a pass is said in the composition's answer, not recorded.
var d Declared
d, _, err = Compose(mk("gate"), "novox")
said = strings.Join(d.Said, "\n")
if err != nil || !strings.Contains(said, "mesh-broker: old-broker on novox") || !strings.Contains(said, "held by gate on novox, not recorded") {
t.Fatalf("the answer does not say the pass: %v %q", err, said)
}
}
+230 -3
View File
@@ -15,6 +15,7 @@ import (
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go/overlay"
"maps"
"slices"
"sort"
"strings"
@@ -162,6 +163,8 @@ func runCase(r Request, c Case, depth string) (CaseVerdict, error) {
if err != nil {
return CaseVerdict{}, err
}
held := base
base, awaited := overrule(base, c, cv.Manifests, shelf)
assignments := map[string]*meshmoduleassigner.Assignments{}
for node, raw := range c.Assignments {
if string(raw) == "null" {
@@ -189,9 +192,17 @@ func runCase(r Request, c Case, depth string) (CaseVerdict, error) {
shelf[name] = m
}
// A module the input carries as its node-engine took it in from its build (SentInput) is composed so: the bus
// reader composes it from its build, never from the indexer's manifest a case gives.
// reader composes it from its build, never from the indexer's manifest a case gives. One the case overrules the
// node's hold on is composed at the build the node would compose it at (Input.Holds), else from the case's manifest.
for name, m := range base.Catalogue {
shelf[name] = m
if _, over := awaited[name]; !over {
shelf[name] = m
}
}
for name, hold := range base.Holds {
if _, over := awaited[name]; over && hold.Built != nil {
shelf[name] = *hold.Built
}
}
// A module the input's node-engine holds as refused (compose.Input.Refused: its verdict, or bytes that do not decode) is
// held here as there, and its read says why, whatever the case's own verdicts: the check then names the cause the node
@@ -225,11 +236,202 @@ func runCase(r Request, c Case, depth string) (CaseVerdict, error) {
}
cv.Machines = map[string]Machine{}
for _, node := range r.Machines {
cv.Machines[node] = machine(in, node, depth, r.Declarations)
m := machine(in, node, depth, r.Declarations)
if words := awaitedOn(in, node, awaited); words != "" {
// Composed without the build it runs at: named per node and module, never a silent pass (issue 1921). The
// machine is then not judged, as for a node whose running state its engine does not send (ADR 0593 §9). Where
// it does not compose without that build (a tool runner's bundle), it is composed with the module held as its
// node-engine holds it, and named so; a node that refuses then too is the change's refusal, in the words of that
// composition, never of the one from the unbuilt manifests, whose refusals the lift itself made (a tool runner
// with no bundle, a resource whose artifact or template no build resolved: the group check of c5d32074c945 on
// shanks).
how := "composed from the change's manifest without its build: "
if !m.Composes && m.NotJudged == "" {
switch again := machine(heldAgain(in, held, awaited), node, depth, r.Declarations); {
case again.Composes:
m, how = again, "composed with it held as its node-engine holds it, since it does not compose without its build: "
case again.NotJudged == "":
m, how = again, "refused with it held as its node-engine holds it, since it does not compose without its build: "
}
}
words = how + words
if m.Composes {
m = Machine{NotJudged: words, Declared: m.Declared}
} else if m.NotJudged == "" {
m.Reason += "; " + words
}
}
cv.Machines[node] = m
}
return cv, nil
}
// overrule lifts the hold of the input's node-engine on each module whose manifest the case carries and judges itself
// (novox/hq ADR 0578 §2): a module the input holds (Refused: the running indexer's verdict, or the running node-engine's
// decoder), whose bytes this composer reads whole, and that the case gives an accepted verdict or none (a reader takes a
// manifest with no verdict as accepted, ADR 0578 §3; the running checker drops the verdicts of the change's manifests,
// issue 1921). It is then no longer Refused nor held in how the node composes. A case's own refusal and bytes this
// composer does not decode stay held. awaited is each module lifted with no build the input carries: the words for the verdict, or
// "" where it is composed at its build.
func overrule(base compose.Input, c Case, reads map[string]ManifestRead, shelf map[string]catalogue.Manifest) (compose.Input, map[string]string) {
awaited := map[string]string{}
for name, why := range base.Refused {
hold := base.Holds[name]
if _, carried := c.Manifests[name]; !carried || !reads[name].Accepted || shelf[name].ReadNoFurther() {
continue
}
held := "held by the running node-engine"
switch hold.By {
case compose.HeldByVerdict:
held = "refused only by the running indexer"
case compose.HeldByDecoder:
held = "not read by the running node-engine's decoder"
}
judged := "accepted by the check's own validator"
if _, given := c.Verdicts[name]; !given {
judged = "given no verdict by the check, so taken as accepted (novox/hq ADR 0578 §3)"
}
awaited[name] = ""
if hold.Built == nil && compose.NamesBuild(shelf[name]) {
build := "its build"
if hold.Build != "" {
build += " " + hold.Build
}
awaited[name] = fmt.Sprintf("%s is %s (%s) and %s, and %s is awaited", name, held, why, judged, build)
}
}
// The same case where the running indexer cannot read the change's manifest at all (issue 1921, the group check of
// 8a80e8e86858): it gives that manifest no fingerprint, so the running node-engine composes the module at the build
// it runs, of the trunk's manifest, holds nothing and says nothing. Composing that build over the case's other
// manifests mixed two revisions (mesh-build-go's trunk build requiring a provision the change's manifests no longer
// name). A module the case carries that the input composes at a build of another manifest is composed from the
// case's manifest, its build awaited, named with the comparison that found it: the content digest of the manifest
// its build was made of where the input names one (ComposedHow.Manifests), else, from a node-engine before it, what
// the two manifests say once what a take-in resolves is left out (sameContract).
provided := map[string]bool{}
for _, name := range overlay.ProvidedModules() {
provided[name] = true // its manifest comes with mesh-sdk and always wins (runCase)
}
for name, built := range base.Catalogue {
if _, lifted := awaited[name]; lifted || provided[name] {
continue
}
if _, carried := c.Manifests[name]; !carried || !reads[name].Accepted || shelf[name].ReadNoFurther() {
continue
}
how := base.ComposedHow
var compared string
if digest := manifestOf(how, name); digest != "" {
if carried, err := compose.ContentDigest(c.Manifests[name].Raw); err == nil && carried == digest {
continue
}
compared = "its build was made of the manifest " + digest + ", not the change's"
} else {
if sameContract(built, shelf[name]) {
continue
}
compared = "its node-engine names no manifest its build was made of, and what the two manifests say differs " +
"beyond what a take-in resolves"
}
judged := "accepted by the check's own validator"
if _, given := c.Verdicts[name]; !given {
judged = "given no verdict by the check, so taken as accepted (novox/hq ADR 0578 §3)"
}
at := ""
if how != nil && how.Builds[name] != "" {
at = " " + how.Builds[name]
}
awaited[name] = ""
if compose.NamesBuild(shelf[name]) {
awaited[name] = fmt.Sprintf("%s is composed by the running node-engine at its build%s, of another manifest than the "+
"change's (%s), and %s, and its build of the change's manifest is awaited", name, at, compared, judged)
}
}
if len(awaited) == 0 {
return base, awaited
}
base.Refused = maps.Clone(base.Refused)
for name := range awaited {
delete(base.Refused, name)
}
if how := base.ComposedHow; how != nil {
copied := *how
copied.Held = maps.Clone(how.Held)
for name := range awaited {
delete(copied.Held, name)
}
base.ComposedHow = &copied
}
return base, awaited
}
// heldAgain is in with each module overrule lifted and gave no build held again as held, the input before overrule,
// holds it: its stub, Refused and how the node composes.
func heldAgain(in, held compose.Input, awaited map[string]string) compose.Input {
in.Catalogue = maps.Clone(in.Catalogue)
in.Refused = maps.Clone(in.Refused)
if in.Refused == nil {
in.Refused = map[string]string{}
}
if how := in.ComposedHow; how != nil {
copied := *how
copied.Held = maps.Clone(how.Held)
in.ComposedHow = &copied
}
for name, words := range awaited {
if words == "" {
continue
}
why, was := held.Refused[name]
if !was {
// Composed by the running node-engine at a build of another manifest, not held (overrule): held here as a node
// holds a module whose build has not landed, the change's manifest as its stub, so that build is not laid over
// the change's other manifests again.
in.Catalogue[name] = compose.Stub(in.Catalogue[name])
in.Refused[name] = words
if in.ComposedHow != nil {
if in.ComposedHow.Held == nil {
in.ComposedHow.Held = map[string]string{}
}
in.ComposedHow.Held[name] = words
}
continue
}
if stub, ok := held.Catalogue[name]; ok {
in.Catalogue[name] = stub
} else {
in.Catalogue[name] = compose.Stub(in.Catalogue[name])
}
in.Refused[name] = why
if in.ComposedHow != nil && held.ComposedHow != nil {
if why, was := held.ComposedHow.Held[name]; was {
if in.ComposedHow.Held == nil {
in.ComposedHow.Held = map[string]string{}
}
in.ComposedHow.Held[name] = why
}
}
}
return in
}
// awaitedOn is the words of each module placed on node that is composed without its build (overrule), joined, sorted.
func awaitedOn(in compose.Input, node string, awaited map[string]string) string {
var out []string
for _, n := range in.Nodes {
if n.Name != node {
continue
}
for _, m := range n.Assigned {
if words := awaited[m]; words != "" {
out = append(out, words)
}
}
}
sort.Strings(out)
return strings.Join(out, "; ")
}
// HoldingsOf is every seat an assigned module claims, held on the node it is assigned to, in node order: the holdings
// of an input that has none of record (the assigner's, mesh-delivery-assigner internal/assign input).
func HoldingsOf(shelf map[string]catalogue.Manifest, nodes []compose.Node) []catalogue.Held {
@@ -383,3 +585,28 @@ func SentInput(in compose.Input, node string) compose.Input {
}
return in
}
// manifestOf is the content digest of the manifest module's build was made of, as the input names it; "" when it names
// none (a node-engine before ComposedHow.Manifests, or a build it did not read).
func manifestOf(how *compose.ComposedHow, module string) string {
if how == nil {
return ""
}
return how.Manifests[module]
}
// sameContract says whether the manifest a module was taken in from its build at (built) and the manifest a case carries
// for it (carried) compose alike: equal once what a take-in resolves (the build, its bundles, the templates filled into
// resources and items, what is handed) is left out of both.
func sameContract(built, carried catalogue.Manifest) bool {
contract := func(m catalogue.Manifest) string {
m.Handed, m.Build, m.Templates, m.Resources, m.Bundles, m.Items = nil, nil, "", nil, nil, nil
raw, err := json.Marshal(m)
if err != nil {
return ""
}
return string(raw)
}
b, c := contract(built), contract(carried)
return b != "" && b == c
}
+146 -2
View File
@@ -4,6 +4,7 @@ import (
"context"
"encoding/json"
"errors"
"fmt"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
@@ -188,7 +189,8 @@ func TestARequestItCannotRunIsRefused(t *testing.T) {
})
// Issue 774 (the group check of feat/l3-final 278aa5f0ccbe): a tool runner its node-engine holds by its manifest's
// verdict is sent as its stub, and the input says it holds it: the sandbox composes the node with it held, never refuses
// it for its stub's missing bundle. An input from an engine before the mark is read by Refused alone, the same way.
// it for its stub's missing bundle. An input from an engine before the mark is read by Refused alone, the same way. The
// case gives the runner its refused verdict, as the checker does for a manifest the change does not bring.
t.Run("a refused tool runner the engine sends as its stub", func(t *testing.T) {
raw := `{"module": "runner", "version": "1", "tool-runner": true, ` +
`"build": {"artifacts": [{"name": "runtime", "kind": "bundle", "language": "go", "from": "cmd/runner", "binary": "runner"}]}}`
@@ -211,7 +213,9 @@ func TestARequestItCannotRunIsRefused(t *testing.T) {
for name, sent := range map[string]string{"with the mark": string(wire), "from an engine before it": older} {
var r Request
r = Request{V: Version, Machines: []string{"a"}, Depth: DepthCompose,
Cases: []Case{{Name: "change", Manifests: map[string]Manifest{"runner": {Raw: json.RawMessage(raw)}}}}}
Cases: []Case{{Name: "change", Manifests: map[string]Manifest{"runner": {Raw: json.RawMessage(raw)}},
Verdicts: bucketstest.Raw(map[string]meshcatalog.VerdictsByDigestValue{"runner": {V: 1, Module: "runner",
State: meshcatalogclient.VerdictRefused, Problems: []string{"p"}}})}}}
if err := json.Unmarshal([]byte(sent), &r.Input); err != nil {
t.Fatal(err)
}
@@ -227,6 +231,146 @@ func TestARequestItCannotRunIsRefused(t *testing.T) {
}
}
})
// ADR 0578 §2, §3 (issue 1921): a module its node-engine holds by the running indexer's verdict gives way when the case,
// judging the change's manifest itself, gives it an accepted verdict or none (the running checker drops the change's
// own), and this composer reads its bytes whole. It is composed at the build the input carries for it (Input.Holds);
// without one it is composed from the case's manifest and the node is not judged, naming it (never a silent pass), or,
// where it cannot compose without its build (a tool runner's bundle), composed held as its node-engine holds it, named
// so. Bytes that do not decode here, and a case's own refusal, stay held.
t.Run("a held module the case judges itself", func(t *testing.T) {
app := `{"module": "app", "version": "1", "resources": [{"id": "server", "type": "container", "name": "app-server", "artifact": "server"}], ` +
`"build": {"artifacts": [{"name": "server", "kind": "upstream", "from": "nats:2.11@sha256:` + strings.Repeat("a", 64) + `"}]}}`
// A module that names a build and composes from its manifest without it.
plain := `{"module": "app", "version": "1", "resources": [{"id": "d", "type": "directory", "path": "/var/lib/app"}], ` +
`"build": {"artifacts": [{"name": "server", "kind": "upstream", "from": "nats:2.11@sha256:` + strings.Repeat("a", 64) + `"}]}}`
runner := `{"module": "app", "version": "1", "tool-runner": true, ` +
`"build": {"artifacts": [{"name": "runtime", "kind": "bundle", "language": "go", "from": "cmd/runner", "binary": "runner"}]}}`
built, err := catalogue.Decode([]byte(`{"module": "app", "version": "1", "resources": [{"id": "built", "type": "directory", "path": "/var/lib/app-built"}]}`))
if err != nil {
t.Fatal(err)
}
verdict := func(state string) map[string]json.RawMessage {
return bucketstest.Raw(map[string]meshcatalog.VerdictsByDigestValue{"app": {V: 1, Module: "app", State: state, Problems: []string{"p"}}})
}
const why = "the indexer's judge refused the manifest of app"
for _, row := range []struct {
name string
raw string
verdicts map[string]json.RawMessage
hold *compose.Hold
composes bool // the machine composes, judged
path string // a path declared on it
words string // in NotJudged, else in the read's refusal
}{
{"an accepted verdict, its build carried", app, verdict(meshcatalogclient.VerdictAccepted),
&compose.Hold{By: compose.HeldByVerdict, Build: "fp1", Built: &built}, true, "/var/lib/app-built", ""},
{"no verdict, its build carried", app, nil, &compose.Hold{By: compose.HeldByVerdict, Build: "fp1", Built: &built}, true, "/var/lib/app-built", ""},
{"an accepted verdict, its build awaited", plain, verdict(meshcatalogclient.VerdictAccepted), &compose.Hold{By: compose.HeldByVerdict, Build: "fp1"}, false, "",
"composed from the change's manifest without its build: app is refused only by the running indexer (" + why + ") and accepted by the check's own validator, and its build fp1 is awaited"},
{"no verdict, from an engine before holds", plain, nil, nil, false, "",
"app is held by the running node-engine (" + why + ") and given no verdict by the check, so taken as accepted"},
{"a container, its image awaited", app, nil, &compose.Hold{By: compose.HeldByVerdict}, false, "",
"composed with it held as its node-engine holds it, since it does not compose without its build: app is refused only by the running indexer"},
{"a tool runner, its bundle awaited", runner, nil, &compose.Hold{By: compose.HeldByVerdict}, false, "",
"composed with it held as its node-engine holds it, since it does not compose without its build: app is refused only by the running indexer"},
{"bytes that do not decode", `{"module": "app", "version": "1", "resources": "none"}`, nil, &compose.Hold{By: compose.HeldByVerdict, Built: &built}, true, "", "does not decode"},
{"the case's own refusal", app, verdict(meshcatalogclient.VerdictRefused), &compose.Hold{By: compose.HeldByVerdict, Built: &built}, true, "", "refused the manifest of app"},
} {
man, _ := catalogue.Decode([]byte(app))
in := compose.Input{Catalogue: map[string]catalogue.Manifest{"app": compose.Stub(man)}, Refused: map[string]string{"app": why},
Nodes: []compose.Node{{Name: "a", Assigned: []string{"app"}}}}
if row.hold != nil {
in.Holds = map[string]compose.Hold{"app": *row.hold}
}
v, err := Run(withInput(t, Request{V: Version, Machines: []string{"a"}, Depth: DepthCompose, Declarations: true,
Cases: []Case{{Name: "change", Manifests: map[string]Manifest{"app": {Raw: json.RawMessage(row.raw)}}, Verdicts: row.verdicts}}}, in))
if err != nil {
t.Fatal(err)
}
m, read := v.Cases[0].Machines["a"], v.Cases[0].Manifests["app"]
declared := false
for _, p := range m.Declared {
declared = declared || p.Path == row.path
}
switch {
case m.Composes != row.composes || (row.path != "" && !declared):
t.Errorf("%s: machine %+v", row.name, m)
case !row.composes && (m.NotJudged == "" || !strings.Contains(m.NotJudged, row.words)):
t.Errorf("%s: not judged %q, want it naming %q", row.name, m.NotJudged, row.words)
case row.composes && row.path == "" && (read.Accepted || !strings.Contains(read.Refused, row.words)):
t.Errorf("%s: read %+v, want held naming %q", row.name, read, row.words)
}
}
})
// Issue 1921 (the group check of 8a80e8e86858, ace: "nothing provides artifact-store, wanted by mesh-build-go"): a
// module the input composes at its build, taken in by a node-engine whose indexer cannot read the change's manifest,
// is composed at that build only where the build was made of the case's manifest: by the content digest the input
// names, else, from a node-engine that names none, by what the two manifests say beyond what a take-in resolves.
// Otherwise it is composed from the case's manifest, never silently: the node is not judged, naming it and how.
t.Run("a module composed at a build of another manifest", func(t *testing.T) {
plain := `{"module": "app", "version": "1", "resources": [{"id": "d", "type": "directory", "path": "/var/lib/app"}], ` +
`"build": {"artifacts": [{"name": "server", "kind": "upstream", "from": "nats:2.11@sha256:` + strings.Repeat("a", 64) + `"}]}}`
built, err := catalogue.Decode([]byte(`{"module": "app", "version": "1", "resources": [{"id": "built", "type": "directory", "path": "/var/lib/app-built"}]}`))
if err != nil {
t.Fatal(err)
}
same, err := compose.ContentDigest([]byte(plain))
if err != nil {
t.Fatal(err)
}
runner := `{"module": "app", "version": "1", "tool-runner": true, ` +
`"build": {"artifacts": [{"name": "runtime", "kind": "bundle", "language": "go", "from": "cmd/runner", "binary": "runner"}]}}`
for _, row := range []struct {
name, raw, digest string
words string // in NotJudged, its build's paths not declared; "" composes at its build
beside string // a module the change refuses on the same node: words are then in the refusal
}{
{"the digest of the case's manifest", plain, same, "", ""},
{"another digest, the same contract", plain, strings.Repeat("b", 64), "of another manifest than the change's (its build was made of the manifest " + strings.Repeat("b", 64), ""},
{"no digest, the same contract", plain, "", "", ""},
{"no digest, another contract", strings.Replace(plain, `"version": "1"`, `"version": "2"`, 1), "",
"app is composed by the running node-engine at its build fp1, of another manifest than the change's (its node-engine names no manifest", ""},
{"a tool runner, its bundle awaited", runner, "",
"composed with it held as its node-engine holds it, since it does not compose without its build: app is composed by the running node-engine at its build fp1", ""},
// The group check of c5d32074c945, shanks: the change refused lab there, and the words were those of the
// composition from the change's manifests unbuilt ("mesh-build-go runs as its own account and its build
// produced no bundle to serve"), not of the one with them held, so the lift's own refusals failed the node.
{"a tool runner awaited beside a module the change refuses", runner, "",
"refused with it held as its node-engine holds it, since it does not compose without its build: app is composed by the running node-engine at its build fp1",
`{"module": "office", "version": "1", "resources": [{"id": "c", "type": "file", "path": "/etc/office.conf", "template": "x"}]}`},
} {
in := compose.Input{Catalogue: map[string]catalogue.Manifest{"app": built}, Nodes: []compose.Node{{Name: "a", Assigned: []string{"app"}}},
ComposedHow: &compose.ComposedHow{Node: "a", Builds: map[string]string{"app": "fp1"}}}
if row.digest != "" {
in.ComposedHow.Manifests = map[string]string{"app": row.digest}
}
manifests := map[string]Manifest{"app": {Raw: json.RawMessage(row.raw)}}
if row.beside != "" {
manifests["office"] = Manifest{Raw: json.RawMessage(row.beside)}
in.Nodes[0].Assigned = append(in.Nodes[0].Assigned, "office")
}
v, err := Run(withInput(t, Request{V: Version, Machines: []string{"a"}, Depth: DepthCompose, Declarations: true,
Cases: []Case{{Name: "change", Manifests: manifests}}}, in))
if err != nil {
t.Fatal(err)
}
m := v.Cases[0].Machines["a"]
if row.beside != "" {
said := fmt.Sprint(m.Reason, m.Problems, m.Refusals)
if m.Composes || m.NotJudged != "" || !strings.Contains(said, "office") || strings.Contains(said, "bundle") || !strings.Contains(m.Reason, row.words) {
t.Errorf("%s: machine %+v, want the change's refusal of office with app held, naming %q", row.name, m, row.words)
}
continue
}
declared := false
for _, p := range m.Declared {
declared = declared || p.Path == "/var/lib/app-built"
}
if row.words == "" && (!m.Composes || !declared) || row.words != "" && (m.Composes || declared || !strings.Contains(m.NotJudged, row.words)) {
t.Errorf("%s: machine %+v, want it naming %q", row.name, m, row.words)
}
}
})
v, err := Run(Request{V: Version, Cases: []Case{{Name: "only-read", Manifests: map[string]Manifest{"demo": {Raw: json.RawMessage(demo)}}}}})
if err != nil || len(v.Cases) != 1 || !v.Cases[0].Manifests["demo"].Accepted || v.Cases[0].Machines != nil || v.SDK == "" {
t.Fatalf("a request with no machines: %+v, %v", v, err)
+18 -1
View File
@@ -13,6 +13,7 @@ import (
oc "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/seats/operatorchannel"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/seatlist"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/micro"
)
@@ -99,6 +100,22 @@ func AnswerEngineCall(handle EngineFunc, verb, node, subject string, body []byte
return say()
}
// ServedEngineSeat is the engine's seat as it serves it: the seat list's definition once the engine holds it, and the
// bootstrapped definition before (novox/hq ADR 0636 §2: every reader holds the bootstrapped seats from its start). The
// link serves the seat when it opens, before the node agent starts the reader of the list; read from the list alone, the
// engine served no verb, so nobody answered manifest-reads and the assigner kept an engine's old statement (issue 1966).
func ServedEngineSeat() catalogue.Seat {
if s, ok := catalogue.SeatNamed(EngineSeat); ok && len(s.Serves) > 0 {
return s
}
for _, s := range catalogue.SeatsFromDefinitions(seatlist.Bootstrapped()) {
if s.Name == EngineSeat {
return s
}
}
return catalogue.Seat{Name: EngineSeat}
}
// EngineSeats is a link that serves this node's seat. Optional, as Asked is.
type EngineSeats interface {
ServeEngineSeat(EngineFunc)
@@ -118,7 +135,7 @@ func (l *natsLink) ServeEngineSeat(handle EngineFunc) {
return
}
apart := make(chan struct{}, apartAtOnce)
for _, v := range catalogue.EngineVerbs(EngineSeat) {
for _, v := range ServedEngineSeat().Serves {
verb := v.Name
answer := EngineAnswer(handle, verb, l.node, apart)
meta := EngineEndpointMetadata(v, l.node)
+42
View File
@@ -38,6 +38,17 @@ type haveLink struct {
// stated is what the tool runner's PING says of itself (Stater); statedErr is the runner not answering.
stated map[string]string
statedErr error
// readsSaid are the node seat's manifest-reads-changed events said on the link (ADR 0618 §4).
readsSaid []nodeengine.ManifestReadsChangedEvent
}
func (l *haveLink) SayEngine(event string, body any) error {
if e, ok := body.(nodeengine.ManifestReadsChangedEvent); ok && event == nodeengine.EventManifestReadsChanged {
l.hmu.Lock()
l.readsSaid = append(l.readsSaid, e)
l.hmu.Unlock()
}
return nil
}
func (l *haveLink) Stated(context.Context, string, string) (map[string]string, error) {
@@ -330,6 +341,37 @@ func TestTheNodeKeyCarriesTheNodesFacts(t *testing.T) {
if got, _ = nodeengineclient.DecodeHaveNode(l.kv["n1.node"].value); got.Record.Contract != NodeKeyContract {
t.Errorf("a report without a health statement dropped the contract: %+v", got.Record)
}
// Issue 1966 (ADR 0618 §4): the seats the engine states are those of the seat list it holds now. A change of the list
// writes the record once and says manifest-reads-changed with the new statement; a restatement that changes nothing
// writes and says nothing; the verb answers the statement and when it was made.
all := catalogue.Seats()
defer catalogue.UseSeats(all)
q.RestateReads(context.Background()) // the statement as the list stands, whatever an earlier test left
_, before := q.StatedReads()
at, saidBefore := l.kv["n1.node"].rev, len(l.readsSaid)
catalogue.UseSeats(slices.DeleteFunc(slices.Clone(all), func(s catalogue.Seat) bool { return s.Name == "mesh-vault" }))
q.RestateReads(context.Background())
got, _ = nodeengineclient.DecodeHaveNode(l.kv["n1.node"].value)
if l.kv["n1.node"].rev != at+1 || slices.Contains(got.Record.ManifestReads, "seat:mesh-vault") ||
!slices.Contains(got.Record.ManifestReads, "seat:mesh-catalog") || !slices.Contains(got.Record.ManifestReads, "word:claims") {
t.Errorf("a seat list without mesh-vault: revision %d (was %d), the record states %v", l.kv["n1.node"].rev, at, got.Record.ManifestReads)
}
if said := l.readsSaid[saidBefore:]; len(said) != 1 || said[0].Why != ReadsStarted || !slices.Equal(said[0].ManifestReads, got.Record.ManifestReads) {
t.Errorf("a changed statement said %+v, want one manifest-reads-changed carrying the record's", said)
}
if reads, made := q.StatedReads(); !slices.Equal(reads, got.Record.ManifestReads) || made.Before(before) {
t.Errorf("the verb answers %v made at %v (was %v), want the record's", reads, made, before)
}
q.RestateReads(context.Background())
if l.kv["n1.node"].rev != at+1 || len(l.readsSaid) != saidBefore+1 {
t.Errorf("an unchanged statement wrote the record or said it again")
}
catalogue.UseSeats(all)
q.RestateReads(context.Background())
if got, _ = nodeengineclient.DecodeHaveNode(l.kv["n1.node"].value); !slices.Equal(got.Record.ManifestReads, catalogue.ManifestReads()) {
t.Errorf("the list back whole: the record states %v, want its contract's", got.Record.ManifestReads)
}
}
// 1132/AC7: a write the bucket refuses is said in the log, and the next apply writes again.
+15 -2
View File
@@ -257,11 +257,11 @@ func (l *natsLink) HearMoveSecrets(handle MoveSecretFunc) {
}
// asyncError is said by the client as before, and a subscription the bus refused is handed to serve (Refused).
func (l *natsLink) asyncError(_ *nats.Conn, _ *nats.Subscription, err error) {
func (l *natsLink) asyncError(_ *nats.Conn, sub *nats.Subscription, err error) {
if err == nil {
return
}
fmt.Fprintf(os.Stderr, "%v\n", err)
fmt.Fprintf(os.Stderr, "%v%s\n", err, subscriptionDetail(sub))
if subject, ok := RefusedSubscription(err); ok && l.refused != nil {
select {
case l.refused <- subject:
@@ -368,3 +368,16 @@ func RefusedSubscription(err error) (string, bool) {
// User is the bus user this link authenticated as.
func (l *natsLink) User() string { return l.user }
// subscriptionDetail names the subscription an asynchronous error came from and how much it holds undelivered, so a slow
// consumer says which one it is (issue 1966).
func subscriptionDetail(sub *nats.Subscription) string {
if sub == nil {
return ""
}
msgs, bytes, err := sub.Pending()
if err != nil {
return fmt.Sprintf(" (subscription %s)", sub.Subject)
}
return fmt.Sprintf(" (subscription %s: %d messages, %d bytes pending)", sub.Subject, msgs, bytes)
}
+3 -7
View File
@@ -61,8 +61,8 @@ func server(t *testing.T) (address string, fingerprint string) {
return listener.Addr().String(), Fingerprint(der)
}
// The right broker is accepted; and a malformed pin is caught before connecting, rather than at the handshake, so a mistyped
// token fails while a person is looking at it (two tests merged).
// The right broker is accepted, a different one refused, and a malformed pin is caught before connecting, rather than at
// the handshake, so a mistyped token fails while a person is looking at it (three tests merged).
func TestTheRightBrokerIsAcceptedAndAMalformedPinRefused(t *testing.T) {
address, pin := server(t)
conn, err := dialPinned(address, pin, 5*time.Second)
@@ -76,15 +76,11 @@ func TestTheRightBrokerIsAcceptedAndAMalformedPinRefused(t *testing.T) {
t.Errorf("%q was accepted as a fingerprint", bad)
}
}
}
func TestADifferentBrokerIsRefused(t *testing.T) {
// The case the pin exists for: something else answering at that address. Since the host
// applies whatever the link delivers, connecting to the wrong mesh is the whole machine.
address, _ := server(t)
_, other := server(t)
_, err := dialPinned(address, other, 5*time.Second)
_, err = dialPinned(address, other, 5*time.Second)
if err == nil {
t.Fatal("a broker presenting a different certificate was accepted")
}
+62 -6
View File
@@ -6,6 +6,7 @@ import (
sdkkeys "git.novox.be/novox/mesh-sdk/go/keys"
"slices"
"sort"
"strings"
"sync"
"time"
@@ -34,13 +35,68 @@ type RecordKeys struct {
Stated []string
}
// manifestReads is what this engine reads of a manifest (novox/hq ADR 0614 §4): the reader capabilities of the mesh-sdk it
// is built with, stated in its record so the assigner moves a want only to an engine that reads its manifest. Computed
// once: it is the build's, and never changes while it runs.
var manifestReads = sync.OnceValue(catalogue.ManifestReads)
// buildReads are the words and forms this engine reads of a manifest (novox/hq ADR 0614 §4): the reader capabilities of
// the mesh-sdk it is built with, its seats left out. Computed once: they are the build's, and never change while it runs.
var buildReads = sync.OnceValue(func() []string {
return slices.DeleteFunc(catalogue.ManifestReads(), func(r string) bool { return strings.HasPrefix(r, catalogue.ReadSeat) })
})
// readsStatedAt is when this engine computed what it reads: its start (the verb manifest-reads, ADR 0618 §3).
var readsStatedAt = time.Now().UTC()
// readsNow is what this engine reads of a manifest now: its build's words and forms, and the seats of the seat list it
// holds now (catalogue.Seats, which the engine's reader of the list gives catalogue: bootstrapped or read; ADR 0636).
func readsNow() []string {
set := map[string]bool{}
for _, r := range buildReads() {
set[r] = true
}
for _, seat := range catalogue.Seats() {
set[catalogue.ReadSeat+seat.Name] = true
}
out := make([]string, 0, len(set))
for r := range set {
out = append(out, r)
}
sort.Strings(out)
return out
}
// readsStatement is what this engine states it reads and since when (the verb manifest-reads, ADR 0618 §3): made at its
// first reading, and made again by Restate whenever the seat list it holds changes what it reads (ADR 0618 §4). The seat
// list is the process's, so the statement is too.
type readsStatement struct {
mu sync.Mutex
reads []string
at time.Time
}
var stated readsStatement
// now is the statement, made at the first reading.
func (s *readsStatement) now() ([]string, time.Time) {
s.mu.Lock()
defer s.mu.Unlock()
if s.reads == nil {
s.reads, s.at = readsNow(), time.Now().UTC()
}
return slices.Clone(s.reads), s.at
}
// restate makes the statement again from what the engine reads now, and says whether it changed.
func (s *readsStatement) restate() ([]string, bool) {
cur := readsNow()
s.mu.Lock()
defer s.mu.Unlock()
if s.reads != nil && slices.Equal(cur, s.reads) {
return slices.Clone(cur), false
}
s.reads, s.at = cur, time.Now().UTC()
return slices.Clone(cur), true
}
// manifestReads is what this engine states it reads of a manifest now, which its record carries.
func manifestReads() []string {
reads, _ := stated.now()
return reads
}
// recordOf is the node's record from one facts report and its keys; nil for a report that carries no profile (one that
// says nothing of the machine, such as a rekey), so the record last written stands.
+25
View File
@@ -1,6 +1,9 @@
package link
import (
"slices"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/seatlist/seatlisttest"
)
@@ -8,3 +11,25 @@ import (
// The seats the tests judge against are given to catalogue as the node-engine gives the ones it read from the bus
// (novox/hq ADR 0635 §1); catalogue compiles none.
func init() { catalogue.UseSeats(catalogue.SeatsFromDefinitions(seatlisttest.Definitions())) }
// Before the engine holds a seat list it serves its seat's verbs from the bootstrapped definition (issue 1966: served from
// the empty list, it answered no verb and the assigner kept an old statement).
func TestServedEngineSeatBeforeTheList(t *testing.T) {
held := catalogue.Seats()
defer catalogue.UseSeats(held)
for _, tc := range []struct {
name string
seats []catalogue.Seat
}{{"no list held", nil}, {"list held", held}} {
catalogue.UseSeats(tc.seats)
var names []string
for _, v := range ServedEngineSeat().Serves {
names = append(names, v.Name)
}
for _, want := range []string{ActManifestReads, ActComposition, ActHandOver} {
if !slices.Contains(names, want) {
t.Errorf("%s: the engine serves %v, without %s", tc.name, names, want)
}
}
}
}
+38 -4
View File
@@ -93,6 +93,8 @@ type Session struct {
sync.Mutex
linked bool
}
// restating takes RestateReads one at a time.
restating sync.Mutex
// haves is what this node-engine writes beside its facts (novox/hq ADR 0429 §4, issue 1132).
haves haveState
@@ -390,10 +392,42 @@ func (q *Session) RouterRecord(key string) (*stdio.StateEntry, error) {
return asking.RouterRecord(key)
}
// StatedReads is what this engine states it reads of a manifest and when it computed them, which is its start (the
// node seat's verb manifest-reads, novox/hq ADR 0618 §3): the record is written only when it differs, so its put says
// nothing of when an unchanged statement was made.
func (q *Session) StatedReads() ([]string, time.Time) { return manifestReads(), readsStatedAt }
// StatedReads is what this engine states it reads of a manifest and when it made that statement, at its start or at the
// last change of its seat list that changed it (the node seat's verb manifest-reads, novox/hq ADR 0618 §3): the record is
// written only when it differs, so its put says nothing of when an unchanged statement was made.
func (q *Session) StatedReads() ([]string, time.Time) { return stated.now() }
// RestateReads states what this engine reads again, after the seat list it holds changed (novox/hq ADR 0618 §4, ADR
// 0636): when the statement changed, its node record carries the new one at once, written where it differs, and the node
// seat's event manifest-reads-changed says it ("started": what it states changed), so the assigner and the builder ask
// the verb again. A statement that did not change writes and says nothing. Calls are taken one at a time, so the last
// write is the newest statement.
func (q *Session) RestateReads(ctx context.Context) {
q.init()
q.restating.Lock()
defer q.restating.Unlock()
reads, changed := stated.restate()
if !changed {
return
}
s := &q.haves
s.mu.Lock()
if s.node != nil && s.node.Record != nil {
node, rec := *s.node, *s.node.Record
rec.ManifestReads = reads
node.Record = &rec
s.node = &node
}
s.mu.Unlock()
q.mu.Lock()
bus := q.bus
q.mu.Unlock()
if bus != nil {
q.writeHaves(ctx, bus)
}
q.SayEngine(nodeengine.EventManifestReadsChanged, nodeengine.ManifestReadsChangedEvent{V: sdkkeys.Version,
Node: q.Membership.Node, ManifestReads: reads, At: time.Now().UTC().Format(time.RFC3339Nano), Why: ReadsStarted})
}
// The reasons the engine says its reads, in the event's why (novox/hq ADR 0618 §4): it started, or its node returned
// after being away (a link after the first).
+62 -24
View File
@@ -23,8 +23,11 @@ type KVUpdate struct {
// KVBus is a link that can watch a bucket's keys: the node agent follows its wants and its compositions with it, and
// never polls.
type KVBus interface {
// KVWatch yields the keys of the bucket matching the pattern now, a Caught update, then every change, until ctx ends
// or the link does (the channel is then closed). A bucket the node's account may not read is an error. The pattern
// KVWatch yields the keys of the bucket matching the pattern now, a Caught update, then the changes, until ctx ends
// or the link does (the channel is then closed). It never holds the bus client back: while the reader does not take,
// a key's changes fold into its newest (a delete included), so a reader sees each key's latest state, not every
// change, and a key still queued from before Caught is handed before Caught with its newest value; across keys the
// order is the order each key was first queued. A bucket the node's account may not read is an error. The pattern
// may be several, separated by spaces (WatchPatterns), watched as one: a node's assignment in either form of its
// keys (research 072 row 37).
KVWatch(ctx context.Context, bucket, pattern string) (<-chan KVUpdate, error)
@@ -52,32 +55,67 @@ func (b OverNATS) KVWatch(ctx context.Context, bucket, pattern string) (<-chan K
go func() {
defer close(out)
defer func() { _ = w.Stop() }()
for {
select {
case <-ctx.Done():
return
case e, ok := <-w.Updates():
if !ok {
return
}
var u KVUpdate
if e == nil {
u.Caught = true
} else {
u = KVUpdate{Key: e.Key(), Value: e.Value(), Revision: e.Revision(),
Deleted: e.Operation() == nats.KeyValueDelete || e.Operation() == nats.KeyValuePurge}
}
select {
case out <- u:
case <-ctx.Done():
return
}
}
}
forwardCoalescing(ctx, w.Updates(), entryUpdate, out)
}()
return out, nil
}
// entryUpdate is the update a client's entry is; the nil entry the client sends after the last key there was is Caught.
func entryUpdate(e nats.KeyValueEntry) KVUpdate {
if e == nil {
return KVUpdate{Caught: true}
}
return KVUpdate{Key: e.Key(), Value: e.Value(), Revision: e.Revision(),
Deleted: e.Operation() == nats.KeyValueDelete || e.Operation() == nats.KeyValuePurge}
}
// forwardCoalescing hands in's items, as updates, to out, and never lets out hold in back: the client's watch sends into
// its channel from inside the subscription's callback, so a reader that is not draining (a node-engine composing for
// minutes) overflows the subscription's pending limit and the bus drops messages (issue 1966). While out is not taken,
// the pending updates are kept by key (the newest value and revision of each, in the order the keys were first seen) and
// the Caught marker keeps its place among the keys queued before it. It returns when ctx ends, or when in is closed and
// what is pending is handed over.
func forwardCoalescing[T any](ctx context.Context, in <-chan T, conv func(T) KVUpdate, out chan<- KVUpdate) {
var queue []*KVUpdate
byKey := map[string]*KVUpdate{}
add := func(u KVUpdate) {
if !u.Caught {
if q, ok := byKey[u.Key]; ok {
*q = u
return
}
}
q := &u
queue = append(queue, q)
if !u.Caught {
byKey[u.Key] = q
}
}
source := in
for source != nil || len(queue) > 0 {
var send chan<- KVUpdate
var head KVUpdate
if len(queue) > 0 {
send, head = out, *queue[0]
}
select {
case <-ctx.Done():
return
case x, ok := <-source:
if !ok {
source = nil
continue
}
add(conv(x))
case send <- head:
if !head.Caught {
delete(byKey, head.Key)
}
queue = queue[1:]
}
}
}
func (l *natsLink) KVWatch(ctx context.Context, bucket, pattern string) (<-chan KVUpdate, error) {
return OverNATS{Conn: l.conn, JS: l.js}.KVWatch(ctx, bucket, pattern)
}
+68
View File
@@ -0,0 +1,68 @@
package link
import (
"context"
"testing"
"time"
)
func TestForwardCoalescing(t *testing.T) {
u := func(k string, rev uint64) KVUpdate { return KVUpdate{Key: k, Value: []byte{byte(rev)}, Revision: rev} }
caught := KVUpdate{Caught: true}
tests := []struct {
name string
in []KVUpdate
want []KVUpdate
}{
{"newest per key, first-seen order", []KVUpdate{u("a", 1), u("b", 2), u("a", 3), u("b", 4), u("a", 5)}, []KVUpdate{u("a", 5), u("b", 4)}},
{"caught keeps its place", []KVUpdate{u("a", 1), u("b", 2), caught, u("c", 3), u("a", 4)}, []KVUpdate{u("a", 4), u("b", 2), caught, u("c", 3)}},
{"delete is the newest", []KVUpdate{u("a", 1), {Key: "a", Revision: 2, Deleted: true}}, []KVUpdate{{Key: "a", Revision: 2, Deleted: true}}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
in := make(chan KVUpdate) // unbuffered: every send must be taken at once, as the client's callback needs
out := make(chan KVUpdate) // unbuffered and not read yet
done := make(chan struct{})
go func() { forwardCoalescing(ctx, in, func(x KVUpdate) KVUpdate { return x }, out); close(done) }()
for _, x := range tt.in {
select {
case in <- x:
case <-time.After(5 * time.Second):
t.Fatal("the producer was held back by a reader that does not read")
}
}
close(in)
var got []KVUpdate
for x := range out2(out, done) {
got = append(got, x)
}
if len(got) != len(tt.want) {
t.Fatalf("got %d updates %v, want %d %v", len(got), got, len(tt.want), tt.want)
}
for i := range got {
if got[i].Key != tt.want[i].Key || got[i].Revision != tt.want[i].Revision || got[i].Caught != tt.want[i].Caught || got[i].Deleted != tt.want[i].Deleted {
t.Fatalf("update %d is %+v, want %+v", i, got[i], tt.want[i])
}
}
})
}
}
// out2 reads out until the forwarder has ended.
func out2(out <-chan KVUpdate, done <-chan struct{}) <-chan KVUpdate {
r := make(chan KVUpdate)
go func() {
defer close(r)
for {
select {
case x := <-out:
r <- x
case <-done:
return
}
}
}()
return r
}
+31 -4
View File
@@ -235,6 +235,9 @@ type Pass struct {
Refused string `json:"refused,omitempty"`
// Held is each module refused alone, with why.
Held map[string]string `json:"held,omitempty"`
// holds is what holds each module Held names by its manifest's verdict or its decoder, with the build it would be
// composed at (frombus.Scope.Holds), for LastHeld's words.
holds map[string]compose.Hold
// Handed is how many compositions the pass handed the agent that differ from what it handed before.
Handed int `json:"handed"`
// Again is when the node composes again on its own, after a seat or a read that did not answer; zero when it does
@@ -247,11 +250,34 @@ type Pass struct {
Queued string `json:"queued,omitempty"`
}
// LastHeld is what the last pass held, each module with why; nil before a pass or when it held none.
// LastHeld is what the last pass held, each module with why: for one its manifest's verdict or its decoder holds, also
// that and the build it would be composed at, as the input a check's composer reads says it (compose.Input.Holds,
// issue 1921); nil before a pass or when it held none.
func (c *Composer) LastHeld() map[string]string {
c.mu.Lock()
defer c.mu.Unlock()
return maps.Clone(c.last.Held)
out := maps.Clone(c.last.Held)
for name, h := range c.last.holds {
if why, held := out[name]; held {
out[name] = why + " (" + holdWords(h) + ")"
}
}
return out
}
// holdWords is what holds a module and the build it would be composed at, in words.
func holdWords(h compose.Hold) string {
by := "held by " + h.By
switch h.By {
case compose.HeldByVerdict:
by = "held by its manifest's verdict"
case compose.HeldByDecoder:
by = "held by this node-engine's decoder"
}
if h.Build == "" {
return by + ", at no build"
}
return by + ", at the build " + h.Build
}
// askedWithin is how recent a pass answers an ask of the verb compose without a new one: the first wait after a seat
@@ -919,6 +945,7 @@ func (c *Composer) compose(ctx context.Context) []agent.Update {
// module's have, from which the watchman raises its condition.
keptRuns := map[string][]string{}
var keptMu sync.Mutex
var holds map[string]compose.Hold
scope := frombus.Scope{Node: c.env.Node, KeptAt: kept, InOwn: inOwn, Running: running, Kept: func(module, words string) {
keptMu.Lock()
defer keptMu.Unlock()
@@ -926,7 +953,7 @@ func (c *Composer) compose(ctx context.Context) []agent.Update {
keptRuns[module] = append(keptRuns[module], words)
}
c.sayOnceEach(words)
}}
}, Holds: func(h map[string]compose.Hold) { holds = h }}
c.mu.Lock()
c.dataScope = frombus.Scope{Node: c.env.Node, KeptAt: kept, InOwn: inOwn} // the data items are worked out as composed
c.mu.Unlock()
@@ -985,7 +1012,7 @@ func (c *Composer) compose(ctx context.Context) []agent.Update {
return nil
}
c.paid()
pass.Held = held
pass.Held, pass.holds = held, holds
// The provisioner client's account the node runs with, dropped by this pass, is said: every filled write is refused
// under a declaration that names none (filling.JudgeAccount), and no module is handed to it (issue 774, ace,
// 2026-11-04 01:29). A placed client keeps its account while it is held (compose.Node.RunningProvisioner).
+6
View File
@@ -508,6 +508,12 @@ func TestTheNodeComposesItsWantsFromTheBus(t *testing.T) {
if len(*told4) != 1 || !strings.HasPrefix((*told4)[0], "held power: ") || !strings.Contains((*told4)[0], `"shows" is none of them`) {
t.Errorf("the agent was told %q, want power held alone, naming its item", *told4)
}
// Issue 1921: the composition verb's held says what holds power and the build it would be composed at, as the input a
// check's composer reads says it.
if why := c4.LastHeld()["power"]; !strings.Contains(why, `"shows" is none of them`) ||
!strings.Contains(why, "held by its manifest's verdict, at the build "+powerFP) {
t.Errorf("the last pass held power for %q, want its verdict and its build named", why)
}
// Review of issue 774 (the kept read): mesh-wireguard, which the mesh provides and which never has a fingerprint, is
// folded in the node's own and composed from the manifest mesh-sdk provides (overlay.ProvidedManifest), not from
// what runs nor the kept declaration: with nothing running (the kept read retired, Running empty) the node's own