43 Commits
Author SHA1 Message Date
mesh-tools d0b208b668 Merge pull request 'node-engine: the golden node grant holds every verb the manifest invokes (A-461c)' (#234) from fix/47-G-node-engine-integration into main 2026-11-17 08:49:22 +00:00
jschoubben 85ad1eef74 nodegrant: the A-461c check of the golden grant runs in an existing test, no test added (test budget)
Builds: design 47 G-22 A-461c
2026-11-17 09:46:55 +01:00
jschoubben 51b89964c5 nodegrant: the golden grant is checked against the manifest's invokes and holds every invoked verb's call subject (A-461c)
Builds: design 47 G-22 A-461c, A-461b
2026-11-17 09:41:36 +01:00
jschoubben 1a6a380507 nodegrant: the node's grant holds the catalog verb module the composer asks (A-461b)
The composer asks the trunk build of another node's module through the seat mesh-catalog's verb module (declared in the manifest); the golden grant lacked its call subject, so the ask hung to the node's bound in every pass and held the integration tests.

Builds: design 47 G-22 A-461b
2026-11-17 09:39:03 +01:00
mesh-tools 0612916af0 Merge pull request 'Node-engine: holds carry their wait; upstream images are no part; the store's address from another node's placed or trunk build (A-453, A-454, A-454a, A-461, A-461a, A-461b)' (#233) from fix/47-G-hold-carries-its-wait into main 2026-11-17 00:23:00 +00:00
jschoubben 10a6824abe Compose: a fault of finding the store other than its listen refusal still refuses the node, tested (A-461b)
Builds: design 47 G A-461b
2026-11-17 01:22:03 +01:00
jschoubben bb285f9c1b Selfcompose, compose: the trunk build stands where a placement names none; the fallback is another node's only; only the store listen refusal holds fetchers (A-461b)
Builds: design 47 G A-461b
2026-11-17 01:19:21 +01:00
jschoubben cbed32524a Selfcompose: another node's module whose running build has no entry is laid from its placed build's entry (A-461, A-461a)
Builds: design 47 G A-461 (first sentence), A-461a
2026-11-17 01:12:37 +01:00
jschoubben b132653fc0 Compose: an artifact store with no known address holds the modules that fetch from it, not the node (A-461, second part)
Builds: design 47 G A-461 (second sentence; the first, laying another node's module from its want's entry, waits on a lead answer)
2026-11-17 01:10:49 +01:00
jschoubben 523ad97e28 Node-engine: a digest the want's parts name always counts as fetched (A-454a)
Builds: design 47 G A-454a
2026-11-17 01:06:57 +01:00
jschoubben 5b4484efd8 Node-engine: a hold after an interrupted backup waits as the others do (A-453)
Builds: design 47 G A-453 (§541)
2026-11-17 01:00:57 +01:00
jschoubben 0a43489a74 Node-engine: an upstream copy served through the registry seat is no part of a want (A-454)
Builds: design 47 G A-454
2026-11-17 00:49:44 +01:00
jschoubben dbb77efab1 Node-engine: only an image the mesh built is a part of a want (A-454)
A container's image counts as fetched for the parts check only when the composition
names the seat it came through; an upstream image its manifest pins by digest is no
part and is never 'fetched and not in the want's parts'.

Builds: design 47 G A-454 (§421's parts check, A-432)
2026-11-17 00:48:46 +01:00
jschoubben 1b9f9355b5 Node-engine: a hold of the composition carries its wait, step compose (A-453)
A module not composed for want of its entry (entry-asked, entry-missing) and a want
whose parts differ write their have with a Wait of step compose and that reason, as
the kept data's holds do; the hold for a failed backup carries none.

Builds: design 47 G A-453 (ADR 0539 §16, §541)
2026-11-17 00:39:39 +01:00
mesh-tools 72385fd84a Merge pull request 'Node-engine: an unanswered requirement costs its consumer, not the machine (A-452)' (#232) from fix/47-G-unanswered-requirement-holds-consumer into main 2026-11-16 23:28:45 +00:00
jschoubben 3bb895152f Resolution: an optional consumer of an unanswered requirement is left absent and listed (A-452, ADR 0330 §3, §7)
Builds: design 47 G A-452
2026-11-17 00:27:47 +01:00
jschoubben c6d893e141 Resolution: a requirement nothing in the mesh answers holds its consumers too; optional ones stay absent (A-452)
The mesh-wide refusal ("nothing in this mesh provides") holds its consumers as the
node-wide one does; a module that wants the name only optionally is left absent
(ADR 0330 §3), never held. A held why still names the nodes that offer nothing
here. The two tests of the first commit are folded as rows into one table test
that replaces two existing ones.

Builds: design 47 G A-452
2026-11-17 00:25:30 +01:00
jschoubben 310ff0cabc Resolution: an unanswered requirement holds its consumers, not the node (A-452)
A requirement nothing answers no longer refuses the whole composition: each
module that wants it is held with the "nothing provides" words, as a refused
claim's claimant is, and the node composes the rest. Held modules stay in the
set, so what they provide still counts. The node is refused when every module
is held or no module of the set wants the name. Tests that used an unanswered
requirement to make a node's own set fail now use an ambiguous one.

Builds: design 47 G A-452 (extends A-405, A-407; ADR 0636 §3)
2026-11-17 00:18:25 +01:00
mesh-tools 13fcfae051 Merge pull request 'Node-engine: judge a move's data forms from the entry kept of the running build; no endless handing again' (#231) from fix/47-G-engine-own-manifest into main 2026-11-16 19:00:07 +00:00
jschoubben f8f33e54a7 Node-engine: judge a move's data forms from the entry kept of the running build; no endless handing again
The running build's entry stays held (kept in the composition state) while the node runs it, though the catalog half dropped it; the agent no longer tells the composer it waits for a composition it holds, and the composer hands again when the entry answers.

Builds: design 47 G §296, §297, A-113, §614
2026-11-16 19:59:20 +01:00
mesh-tools 869d15e6d3 Merge pull request 'Design 47 G start: node-engine fixes placed by hand (A-226b/c, A-237b/c, manifest digest)' (#230) from fix/47-G-start-2-host into main 2026-11-16 14:15:10 +00:00
jschoubben 292aa54bbb node-engine: compose-candidate's chunked input is tested through HandActs.Call and past the inline bound, folded into the compose integration test; comments cite A-237c
Builds: design 47 G (G-22 lead answers 2) A-237c; A-237b
2026-11-16 14:49:39 +01:00
jschoubben f539cdb7ec node-engine: compose-candidate sends its composition input in chunks over the link's connection when asked with a call token (A-237b)
Builds: design 47 G (G-22 lead answers 2) A-237b; A-237, A-249
2026-11-16 14:38:31 +01:00
jschoubben 3b365ba7cc node-engine: a catalog revision's manifest digest is made the candidate's hex form (A-226b)
Builds: design 47 G (G-22 lead answers 2) A-226b
2026-11-16 14:26:03 +01:00
jschoubben 6eb41a17e1 node-engine: a run's compose-candidate key reads candidates.run-<id> by its group, with no heads compared (A-226c)
Builds: design 47 G (G-22 lead answers 2) A-226c; A-226b
2026-11-16 14:00:30 +01:00
jschoubben a1cd4a1a2f node-engine: compose-candidate reads a branch pipeline's catalog revision through mesh-catalog.revision and composes it as the candidate (A-226b)
Builds: design 47 G (G-22 lead answers 2) A-226b; ADR 0593 §7
2026-11-16 13:58:36 +01:00
jschoubben 7a30c0d54c node-engine: fold the three new tests into existing tests as cases, so the branch adds no test function (test budget)
Builds: design 47 G §524, §525, A-405, A-407, A-430, A-430a, A-432 (hq main 3723b2ea), ADR 0259 §8
2026-11-16 09:40:29 +01:00
jschoubben a9eb4372d9 Merge remote-tracking branch 'origin/main' into fix/47-G-start-2-host 2026-11-16 09:25:14 +01:00
jschoubben c025bb5f1e node-engine: at a start a hold for differing parts or a missing entry is judged again, not kept as a failed backup waiting for an answer (§524, §421, ADR 0568 §4) 2026-11-16 01:04:07 +01:00
jschoubben 5a3510b115 node-engine: a runtime of its own runs the tool runner's program, which the module's parts check leaves out (design 47 G §524, A-432) 2026-11-16 01:01:01 +01:00
jschoubben 7f892a757c node-engine: a held tool runner's kept program is not replaced by the tools bundles it serves, so novox's new indexer is run by it (ADR 0259 §8) 2026-11-16 00:50:33 +01:00
jschoubben 40ed0633e0 node-engine: a held tool runner keeps its running program, so a module of its own account is not held for want of a runtime that runs (ADR 0259 §8) 2026-11-16 00:34:50 +01:00
jschoubben 5bbde5db40 node-engine: the composer judges a changed want by its fingerprint, version, parts and keeping node, so a want that gains its parts is composed (design 47 G §525) 2026-11-16 00:28:41 +01:00
mesh-tools 21e452df28 Merge pull request 'Node-engine invokes the registries' versions verbs (design 47 G A-430a)' (#229) from fix/47-G-engine-invokes-versions into main 2026-11-15 16:08:13 +00:00
jschoubben dc32afdd7e Node-engine invokes the registries' versions verbs, so its grant admits resolving a want's version (design 47 G A-430a)
The grants part admits a seat verb's invokers by the manifest on the trunk; a node-engine that resolves a want's
version through oci-artifact-registry.versions and container-image-registry.versions is refused on the bus until
its manifest on main names both. Manifest only: a node-engine that does not resolve versions never calls them.
2026-11-15 17:04:37 +01:00
jschoubben 8a0c39280d Node-engine: a want naming its version is resolved to its digests through the registry seats (A-430a)
A want of a released build names the module's version and no parts. Each artifact of the build's entry is
resolved at that version through the verb versions of its kind's registry seat: oci-artifact-registry for a
bundle or an archive, container-image-registry for an image, under <module>/<artifact> or a package's own
name. An upstream copy, which is not numbered, takes the digest its manifest pins. partsMissing, buildOf and
KeptBuildOf (and the shadow's added modules) read those digests; a want with parts is read as before.

A lookup that fails is said in the journal, naming the module, the version and the seat, and holds the module
whole: nothing of it is composed. It is asked again after a minute. The manifest invokes both verbs.

Builds: design 47 G A-430, A-430a; lead answer A-430b
2026-11-15 16:52:59 +01:00
jschoubben a00cccfe0d Node-engine: a running module the assignment names is held, never removed, whatever composed nothing of it (A-410)
Builds: design 47 G A-410
2026-11-14 23:38:29 +01:00
jschoubben 592b4002bd Node-engine: a running module with no fingerprint is held in its answer, never removed (A-410)
Builds: design 47 G A-410
2026-11-14 23:18:36 +01:00
jschoubben 0adc5ab4e6 Node-engine: a running module without an entry resolves as the newest build its want names
Builds: design 47 G §421, lead answers A-405 and A-407
2026-11-14 22:47:33 +01:00
jschoubben 6079f634d6 Node-engine: say whether its seat is served and how long the first looks of what runs and of the accounts take (A-405)
Live on novox 2026-11-14 root-free answered 'judged no account yet' minutes after the start and nothing was logged:
the seat's serving and the first looks were silent.
2026-11-14 15:59:28 +01:00
jschoubben eaf011e971 Node-engine: a first start whose composing never ends is named by MESH_ENGINE_OWN_MODULE (A-404, live start recovery)
A-403 named the engine from the base's catalogue, but on novox the base stops at the first manifest it cannot read,
before mesh-host, so the engine stayed unnamed and its seat unserved.
2026-11-14 15:53:44 +01:00
jschoubben 1c3b2597ba Node-engine: named from the catalogue the base read, before the rest of the input (A-403)
Live on novox 2026-11-14 the G node-engine never ended a composing pass, so it never named its own module and never
served its seat node-engine: grants got no responders for root-free, judged novox not free, and the operator channel
held every Approve ask (the seed, hq#609) as no channel could prove who answered.
2026-11-14 15:48:45 +01:00
mesh-tools fd023f7473 Merge pull request 'Milestone 47 G: mesh-host's part of the one release' (#228) from feat/47-G-join12 into main 2026-11-13 13:09:29 +00:00
38 changed files with 1652 additions and 167 deletions
+38 -8
View File
@@ -6,6 +6,7 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"log"
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
@@ -17,6 +18,7 @@ import (
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine" "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
occ "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/clients/operatorchannelclient" occ "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/clients/operatorchannelclient"
stdio "git.novox.be/novox/mesh-sdk/go" stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"github.com/novox/mesh-host/internal/accounts" "github.com/novox/mesh-host/internal/accounts"
"github.com/novox/mesh-host/internal/agent" "github.com/novox/mesh-host/internal/agent"
"github.com/novox/mesh-host/internal/apply" "github.com/novox/mesh-host/internal/apply"
@@ -378,9 +380,9 @@ func packagesFrom(node string, session *link.Session, composer *atomic.Pointer[s
// nec.AnswerBound, selfcompose candidateWithin). The answer is // nec.AnswerBound, selfcompose candidateWithin). The answer is
// the seat's shape (nodeengine.ComposeCandidateAnswer), checked as it is encoded; a node whose shadow does not run yet // the seat's shape (nodeengine.ComposeCandidateAnswer), checked as it is encoded; a node whose shadow does not run yet
// answers unready, which the checker takes as no answer. // answers unready, which the checker takes as no answer.
func composeCandidateFrom(node string, shadow *atomic.Pointer[selfcompose.Shadow]) func(candidate string, input bool) (any, error) { func composeCandidateFrom(node string, shadow *atomic.Pointer[selfcompose.Shadow]) func(candidate string, input bool, call string, line buscalls.ChunkLine) (any, error) {
return func(candidate string, input bool) (any, error) { return func(candidate string, input bool, call string, line buscalls.ChunkLine) (any, error) {
ask := nodeengine.ComposeCandidateInput{Candidate: candidate, Input: input} ask := nodeengine.ComposeCandidateInput{Candidate: candidate, Input: input, Call: call}
if _, err := nec.EncodeAsk(ask); err != nil { if _, err := nec.EncodeAsk(ask); err != nil {
return nil, err return nil, err
} }
@@ -394,11 +396,7 @@ func composeCandidateFrom(node string, shadow *atomic.Pointer[selfcompose.Shadow
defer cancel() defer cancel()
a = s.ComposeCandidate(ctx, ask) a = s.ComposeCandidate(ctx, ask)
} }
raw, err := nec.EncodeAnswer(a) return answerComposeCandidate(a, candidate, input, call, line)
if err != nil {
return nil, fmt.Errorf("this node's answer to %s does not fit the seat's shape: %w", candidate, err)
}
return json.RawMessage(raw), nil
} }
} }
@@ -425,3 +423,35 @@ func entryAgainFrom(composer *atomic.Pointer[selfcompose.Composer]) func(ctx con
return c.EntryAgain(ctx, fingerprint) return c.EntryAgain(ctx, fingerprint)
} }
} }
// answerComposeCandidate encodes the answer of compose-candidate. The verb is marked chunks (A-237b, A-237c): asked with input and a
// call token, the composition input goes in chunks over the link's connection, whatever its size, and the answer names them;
// asked with no token (the checker's, until G removes it, G-6 §321, A-237c) it stays inline in the answer.
func answerComposeCandidate(a nodeengine.ComposeCandidateAnswer, candidate string, input bool, call string, line buscalls.ChunkLine) (any, error) {
var chunked []byte
if input && call != "" && a.Input != nil {
if line == nil {
return nil, errors.New("this engine has no bus connection to send the composition input on")
}
body, err := json.Marshal(a.Input)
if err != nil {
return nil, fmt.Errorf("the composition input of %s cannot be encoded: %w", candidate, err)
}
chunked, a.Input = body, nil
a.Chunks = int64((len(body) + buscalls.SeatChunkBytes - 1) / buscalls.SeatChunkBytes)
}
raw, err := nec.EncodeAnswer(a)
if err != nil {
return nil, fmt.Errorf("this node's answer to %s does not fit the seat's shape: %w", candidate, err)
}
if chunked != nil {
send := buscalls.SeatChunks{Seat: nodeengine.Seat, Verb: nodeengine.VerbComposeCandidate, Call: call}
// The answer names the chunks first; the send runs after it and ends at a chunk the caller does not acknowledge.
go func() {
if err := send.Send(line, chunked, buscalls.SeatAckWithin, log.Printf); err != nil {
log.Printf("the send %s of the composition input of %s: %v", call, candidate, err)
}
}()
}
return json.RawMessage(raw), nil
}
+20
View File
@@ -992,6 +992,11 @@ func runLink(ctx context.Context, opts options) error {
// The engine's own module, first from its applied record of the build whose fingerprint is its own (design 47 G §527, // The engine's own module, first from its applied record of the build whose fingerprint is its own (design 47 G §527,
// §634, A-149); on a first start with no such record, from its first composing. // §634, A-149); on a first start with no such record, from its first composing.
own.Set(ownModuleApplied(appliedStoreFor(opts.state), os.Getenv(compositions.FingerprintVar))) own.Set(ownModuleApplied(appliedStoreFor(opts.state), os.Getenv(compositions.FingerprintVar)))
// The live start's recovery (A-404): a first start whose composing never ends is named by its unit's environment, so it
// serves its seat (root-free); the name the assignments place on this node. Ended by the lookup from the bus after G.
if own.Get() == "" {
own.Set(os.Getenv("MESH_ENGINE_OWN_MODULE"))
}
queue := &link.Session{ queue := &link.Session{
Own: own, Own: own,
Membership: membership, Membership: membership,
@@ -1257,6 +1262,7 @@ const (
// anyway. A statement the bus did not take is said at the next look. It reads; it never acts (ADR 0240 // anyway. A statement the bus did not take is said at the next look. It reads; it never acts (ADR 0240
// rule 6) — but for going on with a take act stopped at its bound (ADR 0328 §7), which runs beside it. // rule 6) — but for going on with a take act stopped at its bound (ADR 0328 §7), which runs beside it.
func judgeWhatRuns(ctx context.Context, j *liveness.Judge, queue *link.Session, say link.Announce, reapply, reconcile func(modules ...string)) { func judgeWhatRuns(ctx context.Context, j *liveness.Judge, queue *link.Session, say link.Announce, reapply, reconcile func(modules ...string)) {
say("the looks of what runs and of the accounts start (A-405)")
var missingBefore, inactiveBefore map[string]bool var missingBefore, inactiveBefore map[string]bool
ticker := time.NewTicker(liveness.LookEvery) ticker := time.NewTicker(liveness.LookEvery)
defer ticker.Stop() defer ticker.Stop()
@@ -1279,8 +1285,16 @@ func judgeWhatRuns(ctx context.Context, j *liveness.Judge, queue *link.Session,
case <-nudged: case <-nudged:
owed = true owed = true
} }
first := lastAccounts.Load() == nil
began := time.Now()
if first {
say("the first look of what runs begins (A-405)")
}
st, changed := j.Look(ctx) st, changed := j.Look(ctx)
owed = owed || changed owed = owed || changed
if first {
say(fmt.Sprintf("the first look of what runs took %s; units and accounts next (A-405)", time.Since(began).Round(time.Millisecond)))
}
var unitSt *units.Statement var unitSt *units.Statement
if u := unitJudge; u != nil { if u := unitJudge; u != nil {
us, unitsChanged := u.Look(ctx) us, unitsChanged := u.Look(ctx)
@@ -1294,9 +1308,15 @@ func judgeWhatRuns(ctx context.Context, j *liveness.Judge, queue *link.Session,
} }
var accountSt *accounts.Statement var accountSt *accounts.Statement
if a := accountJudge; a != nil { if a := accountJudge; a != nil {
if first {
say(fmt.Sprintf("the first look of the accounts begins at %s (A-405)", time.Since(began).Round(time.Millisecond)))
}
as, accountsChanged := a.Look(ctx) as, accountsChanged := a.Look(ctx)
accountSt = &as accountSt = &as
lastAccounts.Store(&as) lastAccounts.Store(&as)
if first {
say(fmt.Sprintf("the first look of the accounts ended at %s: %d accounts (A-405)", time.Since(began).Round(time.Millisecond), len(as.Accounts)))
}
owed = owed || accountsChanged owed = owed || accountsChanged
if accountsChanged { if accountsChanged {
for _, v := range as.Accounts { for _, v := range as.Accounts {
+2
View File
@@ -1200,6 +1200,8 @@ func composerEnv(session *link.Session, node string, say func(string), hooks com
// And the shadow too: the catalog half read by the fingerprints its wants and runnings name, over its own calls of the seat mesh-catalog's // And the shadow too: the catalog half read by the fingerprints its wants and runnings name, over its own calls of the seat mesh-catalog's
// verb entry on its link session (design 47 G §421): neither the indexer's nor the builder's states are read. // verb entry on its link session (design 47 G §421): neither the indexer's nor the builder's states are read.
env.Entry = selfcompose.EntryCall(session.AskMeshSeat) env.Entry = selfcompose.EntryCall(session.AskMeshSeat)
// And the registry seats' verb versions, to resolve a want that names its version to its artifacts' digests (A-430a).
env.Registry = selfcompose.EntryCall(session.AskMeshSeat)
return env return env
} }
+74 -2
View File
@@ -29,6 +29,7 @@ import (
"time" "time"
nec "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient" nec "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"git.novox.be/novox/mesh-sdk/go/buscompose" "git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/declaration" "git.novox.be/novox/mesh-sdk/go/declaration"
"git.novox.be/novox/mesh-sdk/go/seatlist" "git.novox.be/novox/mesh-sdk/go/seatlist"
@@ -702,7 +703,10 @@ func h8candidate(t *testing.T, b *itBed, fp, running string) {
t.Fatal(err) t.Fatal(err)
} }
path := filepath.Join(b.dir, "web.conf") path := filepath.Join(b.dir, "web.conf")
man := `{"module":"web","version":"2","config":{"greeting":{"type":"string","description":"what the file says","default":"hello"}},` + // A description past what one answer carries (900 KiB, and under the bus's 1 MiB for one value), so the composition input this
// node sends is over the inline bound (A-237b, A-237c).
pad := strings.Repeat("x", 930_000)
man := `{"module":"web","version":"2","config":{"greeting":{"type":"string","description":"what the file says ` + pad + `","default":"hello"}},` +
`"resources":[{"id":"conf","type":"file","path":"` + path + `","mode":"0640","content":"greeting=${config:greeting}\n"},` + `"resources":[{"id":"conf","type":"file","path":"` + path + `","mode":"0640","content":"greeting=${config:greeting}\n"},` +
`{"id":"tools","type":"package","package":"web-tools"}]}` `{"id":"tools","type":"package","package":"web-tools"}]}`
digest := meshcatalogclient.ManifestDigest([]byte(man)) digest := meshcatalogclient.ManifestDigest([]byte(man))
@@ -728,7 +732,43 @@ func h8candidate(t *testing.T, b *itBed, fp, running string) {
t.Fatal(err) t.Fatal(err)
} }
before, _ := b.composed.Status() before, _ := b.composed.Status()
got, err := composeCandidateFrom(itNode, &shadowNow)(key, false) // A-237c: asked with input and no call token (the checker's, until G removes it, G-6 §321) an input past what one answer carries
// is refused; asked with a token it is sent in chunks through the engine's call, whatever its size (A-237b).
if over, err := composeCandidateFrom(itNode, &shadowNow)(key, true, "", nil); err != nil {
t.Fatal(err)
} else if a, err := nec.DecodeAnswer(over.(json.RawMessage)); err != nil || a.Verdict != nec.Refused || !strings.Contains(a.Refused, "an answer on the bus carries") || a.Input != nil {
t.Errorf("an input over the inline bound asked with no token answered %+v (%v)", a, err)
}
line := &ackingLine{chunks: map[string][]byte{}, done: make(chan struct{})}
const call = "0123456789abcdef0123456789abcdef"
acts := &link.HandActs{Node: itNode, ComposeCandidate: composeCandidateFrom(itNode, &shadowNow)}
chunked, err := acts.Call(link.EngineCall{Verb: link.ActComposeCandidate, Line: line,
Args: map[string]any{"candidate": key, "input": true, "call": call}})
if err != nil {
t.Fatal(err)
}
ca, err := nec.DecodeAnswer(chunked.(json.RawMessage))
if err != nil || ca.Verdict != nec.Composed || ca.Chunks < 2 || ca.Input != nil {
t.Fatalf("the input asked with a token answered %+v (%v), want the chunks and no inline input", ca, err)
}
line.want = int(ca.Chunks)
select {
case <-line.done:
case <-time.After(10 * time.Second):
t.Fatalf("%d of %d chunks were sent", line.sent(), ca.Chunks)
}
var file []byte
for n := int64(1); n <= ca.Chunks; n++ {
var c buscalls.SeatChunk
if err := json.Unmarshal(line.chunks[buscalls.SeatChunks{Seat: nodeengine.Seat, Verb: nodeengine.VerbComposeCandidate, Call: call}.Chunk(n)], &c); err != nil {
t.Fatal(err)
}
file = append(file, c.Data...)
}
if len(file) <= 900<<10 || !json.Valid(file) {
t.Errorf("the chunks held %d bytes, want a JSON input over 900 KiB", len(file))
}
got, err := composeCandidateFrom(itNode, &shadowNow)(key, false, "", nil)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -1237,3 +1277,35 @@ func TestNatsTheFirstReadOfTheSeatListEndsTheNodesWait(t *testing.T) {
t.Error("the list was read by the reader of a run begun since, not by the one that met no responders") t.Error("the list was read by the reader of a run begun since, not by the one that met no responders")
} }
} }
// ackingLine is a bus connection that acknowledges every chunk published on it, as a caller does, and keeps the chunks.
type ackingLine struct {
mu sync.Mutex
onAck nats.MsgHandler
chunks map[string][]byte
done chan struct{}
want int
closed bool
}
func (l *ackingLine) Subscribe(_ string, cb nats.MsgHandler) (*nats.Subscription, error) {
l.mu.Lock()
l.onAck = cb
l.mu.Unlock()
return &nats.Subscription{}, nil
}
func (l *ackingLine) sent() int { l.mu.Lock(); defer l.mu.Unlock(); return len(l.chunks) }
func (l *ackingLine) Publish(subject string, data []byte) error {
l.mu.Lock()
l.chunks[subject] = data
cb := l.onAck
if l.want > 0 && len(l.chunks) >= l.want && !l.closed {
l.closed = true
close(l.done)
}
l.mu.Unlock()
go cb(&nats.Msg{Subject: strings.Replace(subject, ".chunk.", ".ack.", 1)})
return nil
}
+19 -5
View File
@@ -660,16 +660,25 @@ func (a *Agent) NotComposedEach(ctx context.Context, why string, held map[string
a.notComposed[module] = mark a.notComposed[module] = mark
a.mu.Unlock() a.mu.Unlock()
phase, detail := string(nodeengineclient.PhaseFailed), NotComposedDetail+why phase, detail := string(nodeengineclient.PhaseFailed), NotComposedDetail+why
var wait *nodeengine.Wait
if heldForEntry(why) { if heldForEntry(why) {
// The entry its want names is not held: the phase held, its detail opening entry-asked: or entry-missing: and // The entry its want names is not held: the phase held, its detail opening entry-asked: or entry-missing: and
// naming the fingerprint (design 47 G section 421). // naming the fingerprint (design 47 G section 421).
phase, detail = string(nodeengineclient.PhaseHeld), why phase, detail = string(nodeengineclient.PhaseHeld), why
// The hold says what it waits on, as every hold but a failed backup does (design 47 G A-453, section 541).
wait = a.composeWait(strings.TrimSuffix(heldReason(why), ":"), why)
} }
a.writeHave(ctx, Have{Module: module, FP: mark.prior.FP, Composed: mark.prior.Composed, Sealed: mark.prior.Sealed, a.writeHave(ctx, Have{Module: module, FP: mark.prior.FP, Composed: mark.prior.Composed, Sealed: mark.prior.Sealed,
Phase: phase, Detail: detail}) Phase: phase, Detail: detail, Wait: wait})
} }
} }
// composeWait is the wait a hold of the composition carries in its have: step compose, with its reason and its words (design 47 G
// A-453, section 541). Only a hold for a failed backup carries none.
func (a *Agent) composeWait(reason, words string) *nodeengine.Wait {
return &nodeengine.Wait{Owner: nodeengine.Seat, Step: nodeengine.WaitStepCompose, Reason: reason, Since: a.now(), Words: words}
}
// writeHave writes a module's have (the node's own key for NodeModule); on a node composing its own, with the sealed // writeHave writes a module's have (the node's own key for NodeModule); on a node composing its own, with the sealed
// values of the composition it names when the caller gave none (Have.Sealed): those of the composition the agent applied or holds at that digest, else // values of the composition it names when the caller gave none (Have.Sealed): those of the composition the agent applied or holds at that digest, else
// those the module's last have said at that digest. // those the module's last have said at that digest.
@@ -1530,7 +1539,7 @@ func (a *Agent) baseline(ctx context.Context, module string, w wantNow, comp com
a.mu.Unlock() a.mu.Unlock()
return false return false
} }
if h.Phase == string(nodeengineclient.PhaseHeld) { if h.Phase == string(nodeengineclient.PhaseHeld) && !judgedAgain(h.Detail) {
// Held for a failed backup, and the operator's answer is still to come: the hold is kept, nothing is // Held for a failed backup, and the operator's answer is still to come: the hold is kept, nothing is
// backed up again or applied, and the next revision of the want is the answer (ADR 0568 §4). Said, so // backed up again or applied, and the next revision of the want is the answer (ADR 0568 §4). Said, so
// why it holds is read from the journal and not guessed (issue 1585: this branch was silent). // why it holds is read from the journal and not guessed (issue 1585: this branch was silent).
@@ -1634,7 +1643,7 @@ func (a *Agent) apply(ctx context.Context, module string, w wantNow, comp compNo
} }
a.env.Say(fmt.Sprintf("%s is held: %s", module, words)) a.env.Say(fmt.Sprintf("%s is held: %s", module, words))
a.writeHave(ctx, Have{Module: module, FP: prev.Fp, Composed: prev.Composed, Phase: string(nodeengineclient.PhaseHeld), a.writeHave(ctx, Have{Module: module, FP: prev.Fp, Composed: prev.Composed, Phase: string(nodeengineclient.PhaseHeld),
Detail: words}) Detail: words, Wait: a.composeWait("parts-differ", words)})
return return
} }
@@ -1671,9 +1680,14 @@ func (a *Agent) apply(ctx context.Context, module string, w wantNow, comp compNo
a.mu.Lock() a.mu.Lock()
a.attempt[module] = attempt{FP: fp, Composed: dig, WantRev: w.Rev, CompRev: comp.Rev, Held: true, AnsweredRev: w.Rev} a.attempt[module] = attempt{FP: fp, Composed: dig, WantRev: w.Rev, CompRev: comp.Rev, Held: true, AnsweredRev: w.Rev}
a.mu.Unlock() a.mu.Unlock()
// It keeps running what it has: the have says what it holds and why it waits. // It keeps running what it has: the have says what it holds and why it waits. A backup interrupted is no failure of it, so
// its hold waits as kept.go's do, with the words as the reason; only a failed backup carries no wait (A-453).
var wait *nodeengine.Wait
if words == BackupInterrupted {
wait = a.composeWait(words, words)
}
a.writeHave(ctx, Have{Module: module, FP: prev.Fp, Composed: prev.Composed, Phase: string(nodeengineclient.PhaseHeld), a.writeHave(ctx, Have{Module: module, FP: prev.Fp, Composed: prev.Composed, Phase: string(nodeengineclient.PhaseHeld),
Detail: words}) Detail: words, Wait: wait})
return return
} }
} }
+47 -9
View File
@@ -357,15 +357,42 @@ func TestAWantAndItsCompositionApplyTheModuleAndTheHaveNamesBoth(t *testing.T) {
if got := len(b.appliedList()) - before; (got == 1) != row.applies { if got := len(b.appliedList()) - before; (got == 1) != row.applies {
t.Fatalf("parts %q: applied %d times, want applies=%v", row.name, got, row.applies) t.Fatalf("parts %q: applied %d times, want applies=%v", row.name, got, row.applies)
} }
if h := b.haves.last("web"); !row.applies && (h.Phase != string(nodeengineclient.PhaseHeld) || h.Detail != row.detail) { if h := b.haves.last("web"); !row.applies && (h.Phase != string(nodeengineclient.PhaseHeld) || h.Detail != row.detail ||
t.Fatalf("parts %q: the have is %s %q, want held %q", row.name, h.Phase, h.Detail, row.detail) h.Wait == nil || h.Wait.Step != nodeengine.WaitStepCompose || h.Wait.Reason != "parts-differ" || h.Wait.Words != row.detail) {
t.Fatalf("parts %q: the have is %s %q wait %+v, want held %q waiting on parts-differ (A-453)", row.name, h.Phase, h.Detail, h.Wait, row.detail)
} }
} }
// A container's digest is the one after its image's @. // A container's digest is the one after its image's @.
container := json.RawMessage(`{"id":"web.c","type":"container","name":"web-c","image":"store/web@` + d3 + `","module":"web"}`) container := json.RawMessage(`{"id":"web.c","type":"container","name":"web-c","image":"store/web@` + d3 + `","image-seat":"mesh-artifact-store","image-holder":"n1","module":"web"}`)
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{container}); got != "" { if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{container}); got != "" {
t.Fatalf("a container whose digest is in the parts differs: %q", got) t.Fatalf("a container whose digest is in the parts differs: %q", got)
} }
// A-454: an image the manifest pins from outside, by its digest, is no part of the want's parts, and a mesh-built one still is.
upstream := json.RawMessage(`{"id":"web.u","type":"container","name":"web-u","image":"registry@` + d2 + `","module":"web"}`)
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{container, upstream}); got != "" {
t.Fatalf("an upstream image pinned by its digest is a part: %q", got)
}
// And neither is the registry seat's copy of an upstream image, though it is routed through the seat.
mirrored := json.RawMessage(`{"id":"web.m","type":"container","name":"web-m","image":"store:5000/upstream/docker.io/library/registry@` + d1 + `","image-seat":"container-image-registry","image-holder":"n1","module":"web"}`)
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{container, mirrored}); got != "" {
t.Fatalf("an upstream copy served through the registry seat is a part: %q", got)
}
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{upstream}); !strings.Contains(got, "in the want's parts and not fetched: "+d3) || strings.Contains(got, d2) {
t.Fatalf("with only an upstream image the mesh-built one is not missing, or the upstream one is named: %q", got)
}
// A-454a: a mesh-built image whose seat holder is not on record carries no image-seat; its digest is in the parts, so it is fetched.
noSeat := json.RawMessage(`{"id":"web.n","type":"container","name":"web-n","image":"store/web@` + d3 + `","module":"web"}`)
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{noSeat}); got != "" {
t.Fatalf("a mesh-built image with no image-seat, named by the parts, is not fetched: %q", got)
}
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{noSeat, upstream}); got != "" {
t.Fatalf("an upstream image beside it is a part: %q", got)
}
// A-432: a runtime of its own runs the tool runner's program, which the module's parts do not name.
ownRuntime := json.RawMessage(`{"id":"web.own-runtime","type":"process","name":"web-runtime","source":"store/node-tools","digest":"` + d1 + `","run":["./node-tools"],"module":"web"}`)
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{container, ownRuntime}); got != "" {
t.Fatalf("a runtime of its own differs from the module's parts: %q", got)
}
} }
// R6-A AC2: given a kept declaration and wants and compositions equal to it, when the agent starts, nothing is applied // R6-A AC2: given a kept declaration and wants and compositions equal to it, when the agent starts, nothing is applied
@@ -749,7 +776,8 @@ func TestADataModuleIsBackedUpFirstAndHeldWhenItFails(t *testing.T) {
}) })
eventually(t, "the have does not say held", func() bool { eventually(t, "the have does not say held", func() bool {
h := b.haves.last("db") h := b.haves.last("db")
return h.Phase == string(nodeengineclient.PhaseHeld) && strings.Contains(h.Detail, "repository is locked") && h.FP == fp1 // A hold for a failed backup carries no wait: the operator is asked about it (A-453).
return h.Phase == string(nodeengineclient.PhaseHeld) && strings.Contains(h.Detail, "repository is locked") && h.FP == fp1 && h.Wait == nil
}) })
if got := b.appliedList(); len(got) != 0 { if got := b.appliedList(); len(got) != 0 {
t.Fatalf("something was applied behind a failed backup: %v", got) t.Fatalf("something was applied behind a failed backup: %v", got)
@@ -808,8 +836,9 @@ func TestADataModuleIsBackedUpFirstAndHeldWhenItFails(t *testing.T) {
stopI := ib.run() stopI := ib.run()
<-began <-began
stopI() stopI()
if h := ib.haves.last("db"); h.Phase != string(nodeengineclient.PhaseHeld) || h.FP != fp1 || h.Composed != dig1 || h.Detail != BackupInterrupted { if h := ib.haves.last("db"); h.Phase != string(nodeengineclient.PhaseHeld) || h.FP != fp1 || h.Composed != dig1 || h.Detail != BackupInterrupted ||
t.Errorf("held after an interrupted backup: %+v", h) h.Wait == nil || h.Wait.Step != nodeengine.WaitStepCompose || h.Wait.Reason != BackupInterrupted {
t.Errorf("held after an interrupted backup, with its wait (A-453): %+v", h)
} }
// At a start: a backup the node agent's own stop interrupted (postgres on novox, 2026-10-31) is no failure, so it is // At a start: a backup the node agent's own stop interrupted (postgres on novox, 2026-10-31) is no failure, so it is
@@ -829,6 +858,10 @@ func TestADataModuleIsBackedUpFirstAndHeldWhenItFails(t *testing.T) {
{"restic: the repository is locked", meshwants.WantsByWantNodeModuleValueBackupAgain, 1, 1, nodeengineclient.PhaseRunning, false}, {"restic: the repository is locked", meshwants.WantsByWantNodeModuleValueBackupAgain, 1, 1, nodeengineclient.PhaseRunning, false},
{"restic: the repository is locked", meshwants.WantsByWantNodeModuleValueBackupSkip, 0, 1, nodeengineclient.PhaseRunning, false}, {"restic: the repository is locked", meshwants.WantsByWantNodeModuleValueBackupSkip, 0, 1, nodeengineclient.PhaseRunning, false},
{"restic: the repository is locked", "", 0, 0, nodeengineclient.PhaseHeld, false}, {"restic: the repository is locked", "", 0, 0, nodeengineclient.PhaseHeld, false},
// A hold the node-engine judges again (novox's indexer, 2026-11-16): its parts differed or its entry was not
// held, which no answer of the operator's releases; at a start it is judged again, here backed up and applied.
{PartsDiffer + " fetched and not in the want's parts: sha256:a03131c6", "", 1, 1, nodeengineclient.PhaseRunning, false},
{EntryMissing + " the catalog half holds no entry of the build", "", 1, 1, nodeengineclient.PhaseRunning, false},
} { } {
r := newBed(t) r := newBed(t)
r.comps.before(ck1, craw1) r.comps.before(ck1, craw1)
@@ -1775,9 +1808,14 @@ func TestTheNodesOwnCompositionIsAppliedWhenItDiffersAndARecordMoveWaitsForAYes(
} }
// Design 47 G §421: a module whose want's entry the catalog half holds none of is the phase held, its detail opening // Design 47 G §421: a module whose want's entry the catalog half holds none of is the phase held, its detail opening
// entry-missing: and naming the fingerprint; composed again, its have is what it runs. // entry-missing: and naming the fingerprint; composed again, its have is what it runs.
b.ag.NotComposedEach(context.Background(), "", map[string]string{"idle": EntryMissing + " the catalog half holds no entry of the build src2:x"}) // A-453: the hold carries its wait, step compose with the reason, as every hold but a failed backup does.
if h := b.haves.last("idle"); h.Phase != string(nodeengineclient.PhaseHeld) || h.Composed != idleDig || h.Detail != EntryMissing+" the catalog half holds no entry of the build src2:x" { for _, row := range []struct{ opening, reason string }{{EntryMissing, "entry-missing"}, {EntryAsked, "entry-asked"}} {
t.Errorf("idle's have while its entry is missing: %+v", h) words := row.opening + " the catalog half holds no entry of the build src2:x"
b.ag.NotComposedEach(context.Background(), "", map[string]string{"idle": words})
if h := b.haves.last("idle"); h.Phase != string(nodeengineclient.PhaseHeld) || h.Composed != idleDig || h.Detail != words ||
h.Wait == nil || h.Wait.Step != nodeengine.WaitStepCompose || h.Wait.Reason != row.reason || h.Wait.Words != words || h.Wait.Since.IsZero() {
t.Errorf("idle's have while its entry is %s: %+v", row.reason, h)
}
} }
b.ag.NotComposedEach(context.Background(), "", nil) b.ag.NotComposedEach(context.Background(), "", nil)
if h := b.haves.last("idle"); h.Phase != string(nodeengineclient.PhaseRunning) || h.Composed != idleDig { if h := b.haves.last("idle"); h.Phase != string(nodeengineclient.PhaseRunning) || h.Composed != idleDig {
+3 -3
View File
@@ -239,9 +239,9 @@ func (a *Agent) keptStep(ctx context.Context, module string, w wantNow, comp com
since := r.now() since := r.now()
a.writeHave(ctx, Have{Module: module, FP: prev.Fp, Composed: prev.Composed, Phase: string(nodeengineclient.PhaseHeld), Detail: words, a.writeHave(ctx, Have{Module: module, FP: prev.Fp, Composed: prev.Composed, Phase: string(nodeengineclient.PhaseHeld), Detail: words,
Wait: &nodeengine.Wait{Owner: nodeengine.Seat, Step: nodeengine.WaitStepCompose, Reason: "entry-missing", Since: since, Words: words}}) Wait: &nodeengine.Wait{Owner: nodeengine.Seat, Step: nodeengine.WaitStepCompose, Reason: "entry-missing", Since: since, Words: words}})
if a.env.Waiting != nil { // The composition of the want is held: the wait is for an entry, not for a composition, so the composer is not told it
a.env.Waiting(module, fp) // is waited for (that handed the same composition again every pass, and the move failed the same way again); it
} // hands the composition again when the entry answers (Composer.entryChanged).
return true, nil return true, nil
} }
take, restore := keptMoves(changes, prev.Fp, fp) take, restore := keptMoves(changes, prev.Fp, fp)
+7
View File
@@ -30,6 +30,7 @@ type recoveryFake struct {
catchUp []meshrecovery.CatchUp catchUp []meshrecovery.CatchUp
expired chan struct{} expired chan struct{}
unknown error // FormChanges answers it unknown error // FormChanges answers it
waited int // times the agent told the composer it waits for a composition
} }
type fakeAnswer struct { type fakeAnswer struct {
@@ -110,6 +111,7 @@ func moveDB(t *testing.T, f *recoveryFake, ranFrom, to string, settle func(b *be
} }
f.events, f.expired = make(chan Ending, 16), make(chan struct{}) f.events, f.expired = make(chan Ending, 16), make(chan struct{})
b.env.Recovery = f.env() b.env.Recovery = f.env()
b.env.Waiting = func(string, string) { f.mu.Lock(); f.waited++; f.mu.Unlock() }
b.env.Stop = func(context.Context, string) Result { b.env.Stop = func(context.Context, string) Result {
f.mu.Lock() f.mu.Lock()
f.calls = append(f.calls, "stop") f.calls = append(f.calls, "stop")
@@ -289,6 +291,11 @@ func testAMoveAcrossAFormChange(t *testing.T) {
t.Run("a form change not known moves nothing and waits for the entries (A-113)", func(t *testing.T) { t.Run("a form change not known moves nothing and waits for the entries (A-113)", func(t *testing.T) {
f := &recoveryFake{unknown: fmt.Errorf("the entry is asked")} f := &recoveryFake{unknown: fmt.Errorf("the entry is asked")}
b := moveDB(t, f, "a", "b", heldHave) b := moveDB(t, f, "a", "b", heldHave)
f.mu.Lock()
if f.waited != 0 {
t.Errorf("the composer was told the agent waits for a composition %d times; it waits for an entry and holds the composition", f.waited)
}
f.mu.Unlock()
if h := b.haves.last("db"); h.FP != fp("a") || !contains(h.Detail, "is not known") || h.Wait == nil || h.Wait.Reason != "entry-missing" { if h := b.haves.last("db"); h.FP != fp("a") || !contains(h.Detail, "is not known") || h.Wait == nil || h.Wait.Reason != "entry-missing" {
t.Fatalf("the have %+v", h) t.Fatalf("the have %+v", h)
} }
+49 -4
View File
@@ -6,6 +6,8 @@ import (
"sort" "sort"
"strings" "strings"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/containerimageregistryclient/reference"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/declaration" "git.novox.be/novox/mesh-sdk/go/declaration"
) )
@@ -21,11 +23,38 @@ const (
EntryMissing = "entry-missing:" EntryMissing = "entry-missing:"
) )
// judgedAgain says a held have's detail is a hold the node-engine judges again at a start, not one that waits for the
// operator's answer: its want's parts differ from its composition (§524) or its entry is not held (§421). Only a failed
// backup or restore point waits for an answer (ADR 0568 §4); novox, 2026-11-16: the indexer held "parts-differ" was kept
// at a start as "held for a failed backup", waiting for an answer nobody was asked.
func judgedAgain(why string) bool {
return strings.HasPrefix(why, PartsDiffer) || heldForEntry(why)
}
// heldReason is the opening of a detail held for an entry: EntryAsked or EntryMissing.
func heldReason(why string) string {
if strings.HasPrefix(why, EntryAsked) {
return EntryAsked
}
return EntryMissing
}
// heldForEntry says whether why is a hold for the entry a want names. // heldForEntry says whether why is a hold for the entry a want names.
func heldForEntry(why string) bool { func heldForEntry(why string) bool {
return strings.HasPrefix(why, EntryAsked) || strings.HasPrefix(why, EntryMissing) return strings.HasPrefix(why, EntryAsked) || strings.HasPrefix(why, EntryMissing)
} }
// upstreamCopy is whether a routed image is the registry seat's copy of an upstream image, `<address>/upstream/<host>/<image>@<digest>`:
// the mesh mirrors it, but it built nothing, so it is no part of a want (design 47 G A-454).
func upstreamCopy(image string) bool {
_, path, ok := strings.Cut(image, "/")
if !ok {
return false
}
_, _, upstream := reference.InUpstream(path)
return upstream
}
// partsDifference checks a want's `parts` against the digests of the archive, process and container resources of the // partsDifference checks a want's `parts` against the digests of the archive, process and container resources of the
// composition, which are the digests the apply fetches and checks (design 47 G section 524). A want with no `parts` (an // composition, which are the digests the apply fetches and checks (design 47 G section 524). A want with no `parts` (an
// assigner of before G, or a node whose record does not state `want-parts`) is applied as today: it returns "". With // assigner of before G, or a node whose record does not state `want-parts`) is applied as today: it returns "". With
@@ -35,6 +64,10 @@ func partsDifference(parts map[string]string, resources []json.RawMessage) strin
if len(parts) == 0 { if len(parts) == 0 {
return "" return ""
} }
inParts := map[string]bool{}
for _, d := range parts {
inParts[d] = true
}
fetched := map[string]bool{} fetched := map[string]bool{}
for _, raw := range resources { for _, raw := range resources {
r, err := declaration.ParseResource(raw) r, err := declaration.ParseResource(raw)
@@ -47,15 +80,27 @@ func partsDifference(parts map[string]string, resources []json.RawMessage) strin
fetched[res.Digest] = true fetched[res.Digest] = true
} }
case *declaration.Process: case *declaration.Process:
if res.Source != "" { // A runtime of its own runs the tool runner's program, which the tool runner's want checks (A-432).
if res.Source != "" && !strings.HasSuffix(res.ID, "."+catalogue.OwnRuntimeID()) {
fetched[res.Digest] = true fetched[res.Digest] = true
} }
case *declaration.Container: case *declaration.Container:
// A digest the want's parts name always counts as fetched, whatever marks the resource carries; one they do not
// name counts only when it is the mesh's own: an image the manifest pins from outside, or the registry seat's
// copy of an upstream image, is no part (design 47 G A-454, A-454a).
digest := ""
if strings.HasPrefix(res.Image, "sha256:") { if strings.HasPrefix(res.Image, "sha256:") {
fetched[res.Image] = true digest = res.Image
} else if _, digest, ok := strings.Cut(res.Image, "@"); ok { } else if _, d, ok := strings.Cut(res.Image, "@"); ok {
fetched[digest] = true digest = d
} }
if digest == "" {
continue
}
if !inParts[digest] && (res.ImageSeat == "" || upstreamCopy(res.Image)) {
continue
}
fetched[digest] = true
} }
} }
named := map[string]bool{} named := map[string]bool{}
+51 -1
View File
@@ -161,7 +161,7 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
} }
kept := make(map[string]catalogue.Manifest, len(n.Kept)) kept := make(map[string]catalogue.Manifest, len(n.Kept))
for m, man := range n.Kept { for m, man := range n.Kept {
if man.Stubbed && man.RunsAs == "" { if man.Stubbed && man.RunsAs == "" && !keepsProgram(man) {
man.Bundles = runningBundles(n.RunsTools[m], reading.Catalogue[m]) man.Bundles = runningBundles(n.RunsTools[m], reading.Catalogue[m])
} }
kept[m] = man kept[m] = man
@@ -489,6 +489,56 @@ func Stub(m catalogue.Manifest) catalogue.Manifest {
// run). What it provides, requires, listens on and grants is its manifest's. A module whose resources runs holds none // run). What it provides, requires, listens on and grants is its manifest's. A module whose resources runs holds none
// of is its Stub, as before. // of is its Stub, as before.
func Carried(m catalogue.Manifest, module string, runs *engcomp.Composed) catalogue.Manifest { func Carried(m catalogue.Manifest, module string, runs *engcomp.Composed) catalogue.Manifest {
stub := carried(m, module, runs)
if catalogue.IsToolRunner(m) && (len(stub.Bundles) != 1 || stub.Bundles[0].Binary == "") {
if b, ok := runningProgram(module, runs); ok {
stub.Bundles = []catalogue.Bundle{b}
}
}
return stub
}
// keepsProgram says a held tool runner carries the program it runs (Carried, runningProgram): its one bundle is that
// program, which the tools bundles it serves never replace, or the modules of their own account it runs are held for
// want of it (lead-47 2026-11-16: 40ed063 kept novox's, and the loop over what the runtime serves dropped it again).
func keepsProgram(m catalogue.Manifest) bool {
return catalogue.IsToolRunner(m) && len(m.Bundles) == 1 && m.Bundles[0].Binary != ""
}
// runningProgram is the program bundle of the tool runner as it runs on the node: its process's source, digest and binary.
// A held tool runner keeps it, so a module of its own account is run by the same program the node runs (novox/hq ADR 0259
// §8) and is not held for want of a runtime that runs (lead-47 2026-11-16: novox's tool runner was held for a build the
// catalog half held no entry of, and held the new indexer with it).
func runningProgram(module string, runs *engcomp.Composed) (catalogue.Bundle, bool) {
if runs == nil {
return catalogue.Bundle{}, false
}
for _, raw := range runs.Resources {
var r struct {
ID string `json:"id"`
Module string `json:"module"`
Type string `json:"type"`
Source string `json:"source"`
Digest string `json:"digest"`
Run []string `json:"run"`
}
if json.Unmarshal(raw, &r) != nil || r.Type != "process" || r.Source == "" || r.Digest == "" || len(r.Run) == 0 {
continue
}
owner := r.Module
if owner == "" {
owner, _ = declaration.ModuleGuess(r.ID)
}
binary, ok := strings.CutPrefix(r.Run[0], "./")
if owner != module || !ok || binary == "" || strings.Contains(binary, "/") {
continue
}
return catalogue.Bundle{Name: "program", Source: registryforms.Forms.Recorded(r.Source), Digest: r.Digest, Binary: binary}, true
}
return catalogue.Bundle{}, false
}
func carried(m catalogue.Manifest, module string, runs *engcomp.Composed) catalogue.Manifest {
if runs == nil { if runs == nil {
return Stub(m) return Stub(m)
} }
+52
View File
@@ -19,6 +19,7 @@ func fpFor(c byte) string { return meshcatalogclient.SourcePrefix2 + strings.Rep
// to: every resource once, each in its module's composition at the fingerprint given, the module's ownership said on // to: every resource once, each in its module's composition at the fingerprint given, the module's ownership said on
// it; a module given no fingerprint is named as skipped and composed nowhere. // it; a module given no fingerprint is named as skipped and composed nowhere.
func TestCompositionsAreTheDeclarationSplitByModule(t *testing.T) { func TestCompositionsAreTheDeclarationSplitByModule(t *testing.T) {
t.Run("a held tool runner keeps its program for the modules of their own account", aHeldToolRunnerKeepsItsProgramForTheModulesOfTheirOwnAccount)
in := aSmallMesh(t) in := aSmallMesh(t)
d, _, err := Compose(in, "laptop") d, _, err := Compose(in, "laptop")
if err != nil { if err != nil {
@@ -191,3 +192,54 @@ func TestCompositionsAreTheDeclarationSplitByModule(t *testing.T) {
} }
} }
} }
// lead-47 2026-11-16: a held tool runner keeps the program it runs (Carried, runningProgram) after what the runtime serves
// is laid on the stubs (Node.RunsTools), so a module of its own account on its node is run by that program, not held
// "the tool runner is not here to run it from" (novox, the new indexer).
func aHeldToolRunnerKeepsItsProgramForTheModulesOfTheirOwnAccount(t *testing.T) {
dg := "sha256:" + strings.Repeat("d", 64)
source := catalogue.ArtifactStoreScheme + "node-tools/runtime/blobs/" + dg
runtime := catalogue.Manifest{Module: "node-tools", Version: "1", ToolRunner: true,
OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/var/lib/mesh/node-tools/broker"}}}
own := catalogue.Manifest{Module: "mesh-self-check", Version: "1", RunsAs: "mesh-self-check",
OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/var/lib/mesh/mesh-self-check/broker", Taken: catalogue.TakenAtStart}},
Bundles: []catalogue.Bundle{{Name: "tools", Source: catalogue.ArtifactStoreScheme + "mesh-self-check/tools/blobs/" + dg,
Digest: dg, Language: "go", Loads: []string{"mesh-self-check-tools"}}}}
store := catalogue.Manifest{Module: "artifacts", Version: "1",
Claims: []catalogue.Claim{{Name: "container-image-registry", Scope: catalogue.ScopeMesh}},
Listens: []catalogue.Listening{{Name: "registry", Port: 5100, From: catalogue.FromMesh}}}
runs := engcomp.Composed{Node: "anchor", Module: "node-tools", Resources: []json.RawMessage{json.RawMessage(
`{"id":"node-tools.runtime","module":"node-tools","type":"process","name":"node-tools","source":"` + source +
`","digest":"` + dg + `","run":["./node-tools"],"env":{"MESH_TOOL_MODULES":""}}`)}}
in := aMesh(t)
in.Catalogue["node-tools"], in.Catalogue["artifacts"], in.Catalogue["mesh-self-check"] = runtime, store, own
anchor := nodeOf(t, &in, "anchor")
anchor.Assigned = append(anchor.Assigned, "artifacts", "node-tools", "mesh-self-check")
in.BusMinted = []string{"anchor.node-tools", "anchor.mesh-self-check"}
anchor.SealingKey = keyOf(t, "anchor")
anchor.Held = map[string]bool{"node-tools": true}
anchor.Kept = map[string]catalogue.Manifest{"node-tools": Carried(runtime, "node-tools", &runs)}
anchor.RunsTools = ToolsRunning([]engcomp.Composed{runs})
fps := map[string]string{overlay.Name: fpFor('a'), "artifacts": fpFor('5'), "node-tools": fpFor('e'), "mesh-self-check": fpFor('6')}
out, _, _, err := CompositionsIn(in, "anchor", fps, Foreseeing)
if err != nil {
t.Fatalf("a module of its own account beside a held tool runner: %v", err)
}
ran := false
for _, c := range out {
for _, raw := range c.Resources {
var r struct {
Type string `json:"type"`
Digest string `json:"digest"`
Run []string `json:"run"`
User string `json:"user"`
}
if json.Unmarshal(raw, &r) == nil && r.Type == "process" && r.User == "mesh-self-check" {
ran = r.Digest == dg && len(r.Run) == 1 && r.Run[0] == "./node-tools"
}
}
}
if !ran {
t.Errorf("mesh-self-check is not run by the program node-tools runs: %v", out)
}
}
+3 -1
View File
@@ -201,7 +201,9 @@ func TestAGrantIsCreatedForEachCredentialMadeForAConsumer(t *testing.T) {
} }
// A consumer whose set does not resolve is skipped, not fatal. // A consumer whose set does not resolve is skipped, not fatal.
nodeOf(t, &in, "laptop").Assigned = append(nodeOf(t, &in, "laptop").Assigned, "files") nodeOf(t, &in, "laptop").Assigned = append(nodeOf(t, &in, "laptop").Assigned, "files")
in.Catalogue["files"] = catalogue.Manifest{Module: "files", Version: "1", Requires: []string{"s3-bucket", "nothing-gives-this"}} in.Catalogue["files"] = catalogue.Manifest{Module: "files", Version: "1", Requires: []string{"s3-bucket", "two-give-this"}}
in.Catalogue["giver-a"] = catalogue.Manifest{Module: "giver-a", Version: "1", Provides: catalogue.Offers("two-give-this")}
in.Catalogue["giver-b"] = catalogue.Manifest{Module: "giver-b", Version: "1", Provides: catalogue.Offers("two-give-this")}
if got, err := Render(in, "anchor", Foreseeing); err != nil || len(got.Rendering.Grants) != 0 { if got, err := Render(in, "anchor", Foreseeing); err != nil || len(got.Rendering.Grants) != 0 {
t.Errorf("a consumer that does not resolve: %v %+v", err, got.Rendering.Grants) t.Errorf("a consumer that does not resolve: %v %+v", err, got.Rendering.Grants)
} }
+39
View File
@@ -141,6 +141,45 @@ func Answer(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]st
a.Differs = slices.DeleteFunc(cmp.Differs, func(d nodeengine.CandidateDiffers) bool { return !notFailed(d.Module) }) a.Differs = slices.DeleteFunc(cmp.Differs, func(d nodeengine.CandidateDiffers) bool { return !notFailed(d.Module) })
cmp.Added = slices.DeleteFunc(cmp.Added, func(d nodeengine.CandidateAdded) bool { return !notFailed(d.Module) }) cmp.Added = slices.DeleteFunc(cmp.Added, func(d nodeengine.CandidateAdded) bool { return !notFailed(d.Module) })
skipped = slices.DeleteFunc(skipped, func(m string) bool { return !notFailed(m) }) skipped = slices.DeleteFunc(skipped, func(m string) bool { return !notFailed(m) })
// A module the node runs and an assignment names, but that no build can be given to compose at (what runs of it carries
// no fingerprint, it having started before the node-engine wrote them, and the declaration gives none), is held: kept as
// it runs, said as held with why, never removed and never stopped for it. Only a module no assignment names is removed
// (A-410).
for _, m := range skipped {
if !runs[m] {
continue
}
a.Removed = slices.DeleteFunc(a.Removed, func(x string) bool { return x == m })
a.Held = append(a.Held, nodeengine.CandidateRefusal{Module: m,
Why: "what runs of " + m + " has no fingerprint and none is given to compose it at, so it is kept as it runs",
Today: "running, kept as it runs"})
}
skipped = slices.DeleteFunc(skipped, func(m string) bool { return runs[m] })
// And whatever the candidate's assignment of this node names that runs and was composed to nothing for any other reason
// (the module is in no catalogue the composition reads, so it has no resources and is not even skipped): held the same
// way. Only a module no assignment names is removed (A-410).
placedHere := map[string]bool{}
if p, given := s.Assignments[s.Node]; given && p != nil {
for _, m := range p.Modules {
placedHere[m.Module] = true
}
} else if !given {
for _, m := range trunk[s.Node].Modules {
placedHere[m.Module] = true
}
}
a.Removed = slices.DeleteFunc(a.Removed, func(m string) bool {
if !placedHere[m] {
return false
}
a.Held = append(a.Held, nodeengine.CandidateRefusal{Module: m,
Why: "the assignment names " + m + " on this node, and the composition made nothing of it, so it is kept as it runs",
Today: "running, kept as it runs"})
return true
})
if len(a.Removed) == 0 {
a.Removed = nil
}
// What the proposal adds, by node and module: placed by a candidate and not by the trunk on that node. // What the proposal adds, by node and module: placed by a candidate and not by the trunk on that node.
added := map[string]bool{} added := map[string]bool{}
+45 -1
View File
@@ -12,6 +12,7 @@ import (
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
engcomp "github.com/novox/mesh-host/internal/compositions" engcomp "github.com/novox/mesh-host/internal/compositions"
"maps"
"os" "os"
"reflect" "reflect"
"slices" "slices"
@@ -193,6 +194,47 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
if a = Answer(ctx, b, Mesh{}, Scope{Node: "n1", Assignments: given(b), InOwn: []string{"spiegel"}}, fps, withMembership, "p-h1", 3); a.Verdict != nodeengine.CandidateAnswerVerdictSame { if a = Answer(ctx, b, Mesh{}, Scope{Node: "n1", Assignments: given(b), InOwn: []string{"spiegel"}}, fps, withMembership, "p-h1", 3); a.Verdict != nodeengine.CandidateAnswerVerdictSame {
t.Errorf("the node's own with its bus membership is answered %+v, want the same", a) t.Errorf("the node's own with its bus membership is answered %+v, want the same", a)
} }
// (C) A module the node runs and an assignment names, given no fingerprint to compose at (it started before the node
// wrote them): held as it runs, never removed (A-410). One no assignment names is removed.
noFp := maps.Clone(fps)
delete(noFp, "lemurs")
unmarked := make([]engcomp.Composed, 0, len(running))
for _, c := range running {
if c.Module == "lemurs" {
c.FP = ""
}
unmarked = append(unmarked, c)
}
unmarked = append(unmarked, engcomp.Composed{Node: "n1", Module: "gone", FP: fp("1")})
a = Answer(ctx, b, Mesh{}, Scope{Node: "n1", Assignments: given(b), InOwn: []string{"spiegel"}}, noFp, unmarked, "p-h1", 3)
if !reflect.DeepEqual(a.Removed, []string{"gone"}) {
t.Errorf("removed %v, want only the module no assignment names", a.Removed)
}
heldIt := false
for _, h := range a.Held {
heldIt = heldIt || h.Module == "lemurs" && h.Why != ""
}
if !heldIt || len(a.Added) != 0 {
t.Errorf("a running module with no fingerprint is answered held %+v added %+v, want held with why and not added", a.Held, a.Added)
}
// (D) One the assignment names whose manifest says its name alone (no resources: what stands in for a module whose entry
// is not held) composes to nothing, and is not even skipped: held too, not removed (A-410).
nameOnly := []byte(`{"module":"ghost","version":"0"}`)
ghostDigest := meshcatalogclient.ManifestDigest(nameOnly)
b.manifests[ghostDigest] = nameOnly
b.modules["ghost"] = meshcatalog.ModulesByModuleValue{Manifest: ghostDigest}
pGhost := b.assignments["n1"]
trunkMods := pGhost.Modules
pGhost.Modules = append(append([]meshmoduleassigner.PlacedModule{}, trunkMods...), meshmoduleassigner.PlacedModule{Module: "ghost"})
b.assignments["n1"] = pGhost
ghostRuns := append(append([]engcomp.Composed{}, unmarked...), engcomp.Composed{Node: "n1", Module: "ghost", FP: fp("2")})
a = Answer(ctx, b, Mesh{}, Scope{Node: "n1", Assignments: given(b), InOwn: []string{"spiegel"}}, noFp, ghostRuns, "p-h1", 3)
if !reflect.DeepEqual(a.Removed, []string{"gone"}) || !slices.ContainsFunc(a.Held, func(h nodeengine.CandidateRefusal) bool { return h.Module == "ghost" }) {
t.Errorf("a named module the composition made nothing of: removed %v held %+v, want it held and only gone removed (%.600s)", a.Removed, a.Held, fmt.Sprintf("%+v", a))
}
pGhost.Modules = trunkMods
b.assignments["n1"] = pGhost
delete(b.modules, "ghost")
// Without what runs, the fold composes spiegel from its manifest, as before, and the composition refuses it. // Without what runs, the fold composes spiegel from its manifest, as before, and the composition refuses it.
if _, _, held, err := Compose(ctx, b, Mesh{}, Scope{Node: "n1", InOwn: []string{"spiegel"}}, fps); err != nil || if _, _, held, err := Compose(ctx, b, Mesh{}, Scope{Node: "n1", InOwn: []string{"spiegel"}}, fps); err != nil ||
!strings.Contains(held["spiegel"], "spiegel has not been built for this mesh") { !strings.Contains(held["spiegel"], "spiegel has not been built for this mesh") {
@@ -633,7 +675,9 @@ func otherNodeNotResolvedIsNamed(t *testing.T) {
given[n] = &p given[n] = &p
} }
a := Answer(ctx, b, Mesh{}, Scope{Node: "n1", Assignments: given}, map[string]string{"cons": "src2:" + strings.Repeat("1", 64)}, nil, "p-h1", 1) a := Answer(ctx, b, Mesh{}, Scope{Node: "n1", Assignments: given}, map[string]string{"cons": "src2:" + strings.Repeat("1", 64)}, nil, "p-h1", 1)
if a.Verdict != nodeengine.CandidateAnswerVerdictRefused || !strings.Contains(a.Refused, "prov on n2 lacks container-runtime") { // cons is held for the requirement nothing answers (A-452) and its why names n2's provider.
if a.Verdict != nodeengine.CandidateAnswerVerdictRefused || len(a.Refusals) != 1 || a.Refusals[0].Module != "cons" ||
!strings.Contains(a.Refusals[0].Why, "prov on n2 lacks container-runtime") {
t.Errorf("a refusal for want of n2's provider = %+v", a) t.Errorf("a refusal for want of n2's provider = %+v", a)
} }
} }
+5
View File
@@ -271,6 +271,11 @@ func taken(name string, raw []byte, m meshcatalog.ModulesByModuleValue) (man cat
func input(ctx context.Context, r Reader, mesh Mesh, s Scope, refuse func(module string, err error), wanted map[string]string) (compose.Input, error) { func input(ctx context.Context, r Reader, mesh Mesh, s Scope, refuse func(module string, err error), wanted map[string]string) (compose.Input, error) {
b, err := readBase(ctx, r, mesh, s) b, err := readBase(ctx, r, mesh, s)
b.refuseOwn(refuse) b.refuseOwn(refuse)
// The engine is named from the catalogue the base read, before the rest of the input, so a node whose input fails
// still serves its seat (root-free): live on novox 2026-11-14 no pass ended, so the seat was never served (A-403).
if s.Engine != nil && len(b.in.Catalogue) > 0 {
s.Engine(EngineModule(b.in, s.Node, s.OwnFingerprint))
}
if err != nil { if err != nil {
return compose.Input{}, err return compose.Input{}, err
} }
+9 -3
View File
@@ -39,11 +39,17 @@ func aNode(t *testing.T, manifests ...string) (*bus, map[string]string) {
// refusal can be of the held module's stub (node-tools held, then "its build produced 0 bundle(s)", its own why unsaid). // refusal can be of the held module's stub (node-tools held, then "its build produced 0 bundle(s)", its own why unsaid).
func aNodeRefusedWholeSaysWhatItHeld(t *testing.T) { func aNodeRefusedWholeSaysWhatItHeld(t *testing.T) {
refused := `{"module":"power","version":"1","items":[{"collection":"contribution:node-bar:block","attributes":{"shows":"state"},"template":"files/a"}]}` refused := `{"module":"power","version":"1","items":[{"collection":"contribution:node-bar:block","attributes":{"shows":"state"},"template":"files/a"}]}`
needs := `{"module":"lamp","version":"1","requires":["nothing-provides-this"]}` needs := `{"module":"lamp","version":"1","requires":["two-provide-this"]}`
b, fps := aNode(t, refused, needs) giverA := `{"module":"giver-a","version":"1","provides":[{"name":"two-provide-this"}]}`
giverB := `{"module":"giver-b","version":"1","provides":[{"name":"two-provide-this"}]}`
b, fps := aNode(t, refused, needs, giverA, giverB)
// The two givers are in the catalogue and placed on no node, so lamp's requirement has two answers and nobody chose.
placed := b.assignments["n1"]
placed.Modules = placed.Modules[:2]
b.assignments["n1"] = placed
refuse(b, "power", `"shows" is none of them`) refuse(b, "power", `"shows" is none of them`)
_, _, held, err := Compose(context.Background(), b, Mesh{}, Scope{Node: "n1"}, fps) _, _, held, err := Compose(context.Background(), b, Mesh{}, Scope{Node: "n1"}, fps)
if err == nil || !strings.Contains(err.Error(), "nothing provides") || !strings.Contains(err.Error(), `held before: power: `) || if err == nil || !strings.Contains(err.Error(), "choose one") || !strings.Contains(err.Error(), `held before: power: `) ||
!strings.Contains(err.Error(), `"shows" is none of them`) { !strings.Contains(err.Error(), `"shows" is none of them`) {
t.Errorf("the node refused whole with %v (held %v), want lamp's refusal and power held before, with its why", err, held) t.Errorf("the node refused whole with %v (held %v), want lamp's refusal and power held before, with its why", err, held)
} }
+6 -4
View File
@@ -11,12 +11,14 @@ import (
// fourOnTheNetwork is aMesh and two more machines on the private network at the site `there`: `ace`, dialled at // fourOnTheNetwork is aMesh and two more machines on the private network at the site `there`: `ace`, dialled at
// ace.example:51820, at 10.77.0.3, and `box`, behind no endpoint, at 10.77.0.4. A module `office` requires // ace.example:51820, at 10.77.0.3, and `box`, behind no endpoint, at 10.77.0.4. A module `office` requires
// `document-server`, which nothing in the catalogue provides: the shape of issue 1833 (nextcloud on the hub wanting a // `document-server`, which two modules of the catalogue provide and nobody chose: a refusal of the node's own set, as
// document server nobody offered). // issue 1833 was (nextcloud on the hub wanting a document server; an unanswered requirement now costs only its consumer, A-452).
func fourOnTheNetwork(t *testing.T) Input { func fourOnTheNetwork(t *testing.T) Input {
t.Helper() t.Helper()
in := aMesh(t) in := aMesh(t)
in.Catalogue["office"] = catalogue.Manifest{Module: "office", Version: "1", Requires: []string{"document-server"}} in.Catalogue["office"] = catalogue.Manifest{Module: "office", Version: "1", Requires: []string{"document-server"}}
in.Catalogue["docs-a"] = catalogue.Manifest{Module: "docs-a", Version: "1", Provides: catalogue.Offers("document-server")}
in.Catalogue["docs-b"] = catalogue.Manifest{Module: "docs-b", Version: "1", Provides: catalogue.Offers("document-server")}
caps := map[string]bool{"container-runtime": true, "wireguard": true, "systemd": true} caps := map[string]bool{"container-runtime": true, "wireguard": true, "systemd": true}
in.Nodes = append(in.Nodes, in.Nodes = append(in.Nodes,
Node{Name: "ace", Place: Place{Key: keyOf(t, "ace"), Endpoint: "ace.example:51820", Site: "there", Address: "10.77.0.3"}, Node{Name: "ace", Place: Place{Key: keyOf(t, "ace"), Endpoint: "ace.example:51820", Site: "there", Address: "10.77.0.3"},
@@ -26,7 +28,7 @@ func fourOnTheNetwork(t *testing.T) Input {
return in return in
} }
// unresolvableBy makes one machine's own set fail to resolve: it is also given `office`, whose requirement nothing provides. // unresolvableBy makes one machine's own set fail to resolve: it is also given `office`, whose requirement two modules provide.
func unresolvableBy(t *testing.T, in *Input, name string) { func unresolvableBy(t *testing.T, in *Input, name string) {
t.Helper() t.Helper()
n := nodeOf(t, in, name) n := nodeOf(t, in, name)
@@ -70,7 +72,7 @@ func TestOneMachineThatDoesNotResolveIsRefusedAlone(t *testing.T) {
if err == nil || !IsUnresolvable(err) { if err == nil || !IsUnresolvable(err) {
t.Fatalf("the machine that does not resolve is not refused as its own set failing: %v", err) t.Fatalf("the machine that does not resolve is not refused as its own set failing: %v", err)
} }
if !strings.Contains(err.Error(), `nothing provides "document-server", wanted by office`) { if !strings.Contains(err.Error(), `"document-server" is wanted by office and 2 modules provide it`) {
t.Errorf("the refusal does not name what could not be resolved: %v", err) t.Errorf("the refusal does not name what could not be resolved: %v", err)
} }
} }
+10
View File
@@ -125,6 +125,16 @@ func (c *composer) render(nodeName string, mode Mode) (Rendered, error) {
} }
} }
// A module that fetches from an artifact store with no known address waits for it; the machine composes the rest (A-461).
for module, why := range placed.storeHeld {
if r.NotPlaced == nil {
r.NotPlaced = map[string]string{}
}
if _, said := r.NotPlaced[module]; !said {
r.NotPlaced[module] = why
}
}
// And a certificate for this machine's name inside the mesh, when anything on it asks. // And a certificate for this machine's name inside the mesh, when anything on it asks.
for _, m := range plan.Modules { for _, m := range plan.Modules {
if m.Certificate == nil { if m.Certificate == nil {
+43 -4
View File
@@ -10,6 +10,7 @@ import (
"git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go/overlay" "git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go/overlay"
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-host/internal/compose/resolution" "github.com/novox/mesh-host/internal/compose/resolution"
"github.com/novox/mesh-host/internal/registryforms"
) )
// The contracts of the node-engine's statement that gate what a machine is sent, as the controller's link package names them // The contracts of the node-engine's statement that gate what a machine is sent, as the controller's link package names them
@@ -41,6 +42,8 @@ type Placed struct {
// manifest's own number as the Reading mode composes them, and never invented. The controller's adapter allocates and // manifest's own number as the Reading mode composes them, and never invented. The controller's adapter allocates and
// composes again. // composes again.
ForeseenPorts []string ForeseenPorts []string
// storeHeld are the modules left out of the declaration because the artifact store has no known address, and why (A-461).
storeHeld map[string]string
} }
// Places composes the place half of the rendering for a node (the controller's renderingFor in the Reading and Foreseeing modes, // Places composes the place half of the rendering for a node (the controller's renderingFor in the Reading and Foreseeing modes,
@@ -92,7 +95,15 @@ func (c *composer) places(nodeName string, resolved Resolved) (Placed, error) {
r.MeshRange = c.overlayRange() r.MeshRange = c.overlayRange()
// The artifact store as this node reaches it now, with what the mesh has built. // The artifact store as this node reaches it now, with what the mesh has built.
// Where no address is known (the holder opens no listen to the mesh, or two), only the modules that fetch from the store
// wait, with the words of an unanswered requirement; the node composes the rest (design 47 G A-461, A-452).
store, err := c.artifactStoreAddress(nodeName) store, err := c.artifactStoreAddress(nodeName)
var noAddress *storeListenError
if errors.As(err, &noAddress) {
// Only the store holder's listen refusal holds its fetchers; any other error still refuses the node (A-461b).
out.storeHeld = modulesNeedingTheStore(plan.Modules, err)
store, err = "", nil
}
if err != nil { if err != nil {
return out, err return out, err
} }
@@ -327,6 +338,29 @@ func (c *composer) namesInTheMesh() map[string]string {
return out return out
} }
// modulesNeedingTheStore are the modules with a resource whose image or source is a kept reference into the artifact store, each
// with why it waits: nothing provides the store's address (A-461).
func modulesNeedingTheStore(modules []catalogue.Manifest, why error) map[string]string {
out := map[string]string{}
for _, m := range modules {
for _, res := range m.Resources {
needs := false
for _, key := range []string{"image", "source"} {
if written, ok := res[key].(string); ok {
if _, kept := registryforms.Forms.InArtifactStore(written); kept {
needs = true
}
}
}
if needs {
out[m.Module] = fmt.Sprintf("nothing provides %q, wanted by %s: %v", "the artifact store's address", m.Module, why)
break
}
}
}
return out
}
// StoreSeat is the seat whose holder is the artifact store (novox/hq ADR 0632 §1): the store's address is that holder's // StoreSeat is the seat whose holder is the artifact store (novox/hq ADR 0632 §1): the store's address is that holder's
// listen open to the mesh, on the machine assignment puts it on. The provision artifact-store it was found by before is // listen open to the mesh, on the machine assignment puts it on. The provision artifact-store it was found by before is
// retired; no module serves it. // retired; no module serves it.
@@ -363,6 +397,11 @@ func (c *composer) storeHolders() map[string]catalogue.Manifest {
return out return out
} }
// storeListenError is the refusal of a holder that opens no listen to the mesh, or two: the store has no address (A-461).
type storeListenError struct{ said string }
func (e *storeListenError) Error() string { return e.said }
// storeListen is the store holder's listen every machine pulls from: its one listen open to the mesh. A holder opening // storeListen is the store holder's listen every machine pulls from: its one listen open to the mesh. A holder opening
// none, or more than one, says no address, by name: which of two ports is the store is no guess the composer makes. // none, or more than one, says no address, by name: which of two ports is the store is no guess the composer makes.
func storeListen(m catalogue.Manifest) (catalogue.Listening, error) { func storeListen(m catalogue.Manifest) (catalogue.Listening, error) {
@@ -376,15 +415,15 @@ func storeListen(m catalogue.Manifest) (catalogue.Listening, error) {
case 1: case 1:
return open[0], nil return open[0], nil
case 0: case 0:
return catalogue.Listening{}, fmt.Errorf("%s holds %s and opens no tcp listen to the mesh: the artifact store has no "+ return catalogue.Listening{}, &storeListenError{fmt.Sprintf("%s holds %s and opens no tcp listen to the mesh: the artifact store has no "+
"address (novox/hq ADR 0632 §1)", m.Module, StoreSeat) "address (novox/hq ADR 0632 §1)", m.Module, StoreSeat)}
} }
var said []string var said []string
for _, l := range open { for _, l := range open {
said = append(said, fmt.Sprintf("%s (%d)", l.Name, l.Port)) said = append(said, fmt.Sprintf("%s (%d)", l.Name, l.Port))
} }
return catalogue.Listening{}, fmt.Errorf("%s holds %s and opens %s to the mesh: the artifact store's address is its one "+ return catalogue.Listening{}, &storeListenError{fmt.Sprintf("%s holds %s and opens %s to the mesh: the artifact store's address is its one "+
"listen open to the mesh (novox/hq ADR 0632 §1)", m.Module, StoreSeat, strings.Join(said, ", ")) "listen open to the mesh (novox/hq ADR 0632 §1)", m.Module, StoreSeat, strings.Join(said, ", "))}
} }
// artifactStoreOnNetwork is the machine on this network that holds the store's seat, and the port that machine put the // artifactStoreOnNetwork is the machine on this network that holds the store's seat, and the port that machine put the
+24 -2
View File
@@ -351,8 +351,30 @@ func TestTheStoreIsFoundByTheImageSeatsHolderAndItsListen(t *testing.T) {
m := in.Catalogue["registry"] m := in.Catalogue["registry"]
m.Listens = append(m.Listens, catalogue.Listening{Name: "other", Port: 5300, From: catalogue.FromMesh}) m.Listens = append(m.Listens, catalogue.Listening{Name: "other", Port: 5300, From: catalogue.FromMesh})
in.Catalogue["registry"] = m in.Catalogue["registry"] = m
if _, err := Places(in, "laptop"); err == nil || !strings.Contains(err.Error(), "opens registry (5000), other (5300) to the mesh") { // A-461: a store with no known address (two listens open to the mesh, so none is a guess) costs the modules that fetch from
t.Errorf("a holder opening two listens to the mesh: %v", err) // it, not the node: those wait with the words of an unanswered requirement, and the node composes the rest.
in.Catalogue["app"] = catalogue.Manifest{Module: "app", Version: "1",
Resources: []map[string]any{{"id": "c", "type": "container", "name": "app", "image": "artifact-store://app/c@sha256:" + strings.Repeat("a", 64)}}}
in.Catalogue["plain"] = catalogue.Manifest{Module: "plain", Version: "1"}
laptop := nodeOf(t, &in, "laptop")
laptop.Assigned = append(laptop.Assigned, "app", "plain")
got, err := Render(in, "laptop", Reading)
if err != nil {
t.Fatalf("a holder opening two listens to the mesh refuses the whole node: %v", err)
}
if why := got.Rendering.NotPlaced["app"]; !strings.Contains(why, "nothing provides") || !strings.Contains(why, "opens registry (5000), other (5300) to the mesh") {
t.Errorf("the module that fetches from the store waits with %q", why)
}
if why, held := got.Rendering.NotPlaced["plain"]; held {
t.Errorf("a module that fetches nothing from the store waits: %q", why)
}
// A-461b: only the listen refusal holds the store's fetchers; any other fault of finding the store (here the holder's machine is
// on the network as first read but unknown when its assignment is read) still refuses the node.
broken := newComposer(in)
broken.onTheNetwork()
broken.in.Nodes = []Node{*nodeOf(t, &in, "laptop")}
if _, err := broken.places("laptop", Resolved{}); err == nil || !strings.Contains(err.Error(), "reading what anchor is assigned") {
t.Errorf("a fault of finding the store other than its listen refusal did not refuse the node: %v", err)
} }
} }
+76 -7
View File
@@ -181,6 +181,22 @@ func resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.No
absent[want] = true absent[want] = true
reported[want] = true reported[want] = true
} }
// A requirement nothing answers costs the modules that want it, not the machine (A-452, ADR 0636 §3): said once per name,
// each consumer is held after the walk, when the set is known.
// The value is the remedy the mesh-wide refusal adds, empty for a requirement nothing on the node answers.
unanswered := map[string]string{}
var unansweredOrder []string
holdWhenUnanswered := func(want, remedy string) {
if !wantedHard(want) && wantedOptionally(want) {
// Only optional consumers: left absent, never held (ADR 0330 §3).
leaveAbsent(want)
return
}
if _, ok := unanswered[want]; !ok {
unansweredOrder = append(unansweredOrder, want)
}
unanswered[want] = remedy
}
for { for {
if len(queue) == 0 { if len(queue) == 0 {
@@ -373,9 +389,8 @@ func resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.No
} else if len(answers) > 1 { } else if len(answers) > 1 {
remedy = "— assign one of these to a node: " + strings.Join(answers, ", ") remedy = "— assign one of these to a node: " + strings.Join(answers, ", ")
} }
problems = append(problems, fmt.Sprintf( // Not answered on the node nor in the rest of the mesh: costs its consumers only (A-452).
"nothing in this mesh provides %q, wanted by %s %s", holdWhenUnanswered(want, remedy)
want, because[want], remedy))
case len(where) == 1: case len(where) == 1:
if c, pinned := world.Pinned[want]; pinned && !chosenMatches(c, where[0]) { if c, pinned := world.Pinned[want]; pinned && !chosenMatches(c, where[0]) {
// One provider, and it is not the one this machine was told to use. Silently // One provider, and it is not the one this machine was told to use. Silently
@@ -483,8 +498,7 @@ func resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.No
continue continue
} }
reported[want] = true reported[want] = true
problems = append(problems, fmt.Sprintf( holdWhenUnanswered(want, "")
"nothing provides %q, wanted by %s", want, because[want]))
continue continue
case 1: case 1:
// No choice to make, so none is made. This is the case that lets `install i3` bring // No choice to make, so none is made. This is the case that lets `install i3` bring
@@ -591,14 +605,17 @@ func resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.No
resolution := mcat.Resolution{Node: node.Name, At: node.At, PublicDomain: node.PublicDomain, resolution := mcat.Resolution{Node: node.Name, At: node.At, PublicDomain: node.PublicDomain,
Account: node.Account, AccountHome: node.AccountHome, AgentAccount: node.AgentAccount, Account: node.Account, AccountHome: node.AccountHome, AgentAccount: node.AgentAccount,
AgentAccountHome: node.AgentAccountHome, Capabilities: node.Capabilities, AgentAccountHome: node.AgentAccountHome, Capabilities: node.Capabilities,
Because: because, Needs: needs, Unhostable: unhostable, Kept: kept, Because: because, Needs: needs, Unhostable: unhostable, Kept: kept}
Absent: absentOf(absent, order, catalogue)}
for _, n := range providersFirst(order, catalogue) { for _, n := range providersFirst(order, catalogue) {
resolution.Modules = append(resolution.Modules, catalogue[n]) resolution.Modules = append(resolution.Modules, catalogue[n])
} }
problems = append(problems, checkCapabilities(resolution.Modules, node)...) problems = append(problems, checkCapabilities(resolution.Modules, node)...)
claims, held := holdClaims(resolution.Modules, node, elsewhere, world.Holdings) claims, held := holdClaims(resolution.Modules, node, elsewhere, world.Holdings)
held, unheld := holdUnanswered(resolution.Modules, unansweredOrder, unanswered, because, held, absent)
// Listed after the holding: an optional consumer of a requirement nothing answers is left absent there (ADR 0330 §3, §7).
resolution.Absent = absentOf(absent, order, catalogue)
problems = append(problems, unheld...)
problems = append(problems, checkResources(resolution.Modules)...) problems = append(problems, checkResources(resolution.Modules)...)
resolution.Claims = claims resolution.Claims = claims
@@ -680,6 +697,58 @@ func checkCapabilities(modules []mcat.Manifest, node mcat.Node) []string {
return problems return problems
} }
// unansweredWords is why a consumer is held for a requirement nothing answers, naming the provision.
func unansweredWords(want, consumer, remedy string) string {
if remedy == "" {
return fmt.Sprintf("nothing provides %q, wanted by %s", want, consumer)
}
return fmt.Sprintf("nothing in this mesh provides %q, wanted by %s %s", want, consumer, remedy)
}
// holdUnanswered holds each module of the set that wants a requirement nothing answers, with the words "nothing provides" naming the
// provision (A-452): it stays in the set, so what it provides still counts for the others. A requirement none of the set's modules
// wants (an assigned name the catalogue does not know) has no consumer to hold and comes back as a problem of the node; so does a
// set whose every module is held, since then nothing could be composed.
func holdUnanswered(modules []mcat.Manifest, unanswered []string, remedies map[string]string, because map[string]string, held map[string]string, absent map[string]bool) (map[string]string, []string) {
var problems []string
for _, want := range unanswered {
found := false
for _, m := range modules {
wants := false
for _, w := range m.Wants() {
if w == want {
wants = true
break
}
}
if !wants {
continue
}
found = true
if m.IsOptional(want) {
// Optional for this module: not held, left absent and listed (ADR 0330 §3, §7).
absent[want] = true
continue
}
if held == nil {
held = map[string]string{}
}
why := unansweredWords(want, m.Module, remedies[want])
if prior, ok := held[m.Module]; ok {
why = prior + "; " + why
}
held[m.Module] = why
}
if !found {
problems = append(problems, unansweredWords(want, because[want], remedies[want]))
}
}
if len(unanswered) > 0 && len(held) >= len(modules) {
problems = append(problems, "no module of the node could be composed: every one is held")
}
return held, problems
}
// holdClaims refuses two modules holding one singular thing, within this node's own set and against what is held elsewhere for // holdClaims refuses two modules holding one singular thing, within this node's own set and against what is held elsewhere for
// the wider scopes: the claims that hold, and for each module that cannot hold what it claims, why (every reason, joined). // the wider scopes: the claims that hold, and for each module that cannot hold what it claims, why (every reason, joined).
func holdClaims(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held, holdings []mcat.Held) ([]mcat.Held, map[string]string) { func holdClaims(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held, holdings []mcat.Held) ([]mcat.Held, map[string]string) {
+69 -25
View File
@@ -53,31 +53,75 @@ func TestARequirementWithOneAnswerIsTakenSilently(t *testing.T) {
} }
} }
func TestARequirementWithSeveralAnswersIsRefusedAndNamed(t *testing.T) { // A requirement nothing answers costs the modules that want it, not the machine (A-452; ace, 2026-11-16 23:11: ten unanswered
// The mesh does not pick. A default would be a choice made for somebody who finds out later, // requirements refused the whole composition), whether nothing on the node or nothing in the whole mesh answers it; one that
// and naming the candidates is the whole remedy. // only optional consumers want is left absent (ADR 0330 §3); several answers are still the node's refusal, naming them (the
_, err := Resolve(shelf( // mesh does not pick: a default would be a choice made for somebody who finds out later); and a node whose every module is held,
mod("editor", nil, []string{"shell"}, nil), // or an assigned name nobody wants, is still refused.
mod("bash", []string{"shell"}, nil, nil), func TestARequirementNoOneAnswersHoldsItsConsumersAndSeveralAnswersRefuseTheNode(t *testing.T) {
mod("zsh", []string{"shell"}, nil, nil), optional := mod("viewer", nil, []string{"codec"}, nil)
mod("fish", []string{"shell"}, nil, nil), optional.OptionalRequires = map[string]bool{"codec": true}
), []string{"editor"}, workstation(), mcat.World{}) hard := mod("player", nil, []string{"codec"}, nil)
meshDB := mcat.Manifest{Module: "postgres", Provides: []mcat.Offer{{Name: "postgres-database", Scope: mcat.ScopeMesh}}}
if err == nil { for _, c := range []struct {
t.Fatal("a requirement with three answers was resolved without asking") name string
} shelf map[string]mcat.Manifest
for _, want := range []string{"bash", "fish", "zsh", "choose one"} { assigned []string
if !strings.Contains(err.Error(), want) { held map[string]string // module -> words in its why
t.Errorf("the refusal does not mention %q: %v", want, err) refused []string // words of the node's refusal
} absent bool
} }{
} {name: "on the node", assigned: []string{"ca-trust", "baserow", "sheet-viewer", "editor"},
shelf: shelf(mod("ca-trust", nil, []string{"internal-acme-ca"}, nil),
func TestARequirementWithNoAnswerIsRefused(t *testing.T) { mod("baserow", []string{"spreadsheet"}, []string{"route"}, nil),
_, err := Resolve(shelf(mod("i3", nil, []string{"xorg"}, nil)), []string{"i3"}, workstation(), mcat.World{}) mod("sheet-viewer", nil, []string{"spreadsheet"}, nil), mod("editor", nil, nil, nil)),
held: map[string]string{"ca-trust": `nothing provides "internal-acme-ca"`, "baserow": `nothing provides "route"`}},
if err == nil || !strings.Contains(err.Error(), "nothing provides") { {name: "in the whole mesh", assigned: []string{"baserow", "editor"},
t.Fatalf("a requirement nothing satisfies gave %v", err) shelf: shelf(meshDB, mod("baserow", nil, []string{"postgres-database"}, nil), mod("editor", nil, nil, nil)),
held: map[string]string{"baserow": `nothing in this mesh provides "postgres-database", wanted by baserow`}},
{name: "wanted hard by one and optionally by another", assigned: []string{"player", "viewer"},
shelf: shelf(hard, optional), held: map[string]string{"player": `nothing provides "codec"`}, absent: true},
{name: "every module held", assigned: []string{"ca-trust"},
shelf: shelf(mod("ca-trust", nil, []string{"internal-acme-ca"}, nil)),
refused: []string{"nothing provides", "every one is held"}},
{name: "nobody wants it", assigned: []string{"ghost"}, shelf: shelf(mod("editor", nil, nil, nil)),
refused: []string{"nothing provides"}},
{name: "several answers", assigned: []string{"editor"},
shelf: shelf(mod("editor", nil, []string{"shell"}, nil), mod("bash", []string{"shell"}, nil, nil),
mod("zsh", []string{"shell"}, nil, nil), mod("fish", []string{"shell"}, nil, nil)),
refused: []string{"bash", "fish", "zsh", "choose one"}},
} {
t.Run(c.name, func(t *testing.T) {
got, held, err := ResolveHolding(c.shelf, c.assigned, workstation(), mcat.World{})
if c.refused != nil {
if err == nil {
t.Fatal("the node was composed, not refused")
}
for _, w := range c.refused {
if !strings.Contains(err.Error(), w) {
t.Errorf("the refusal does not mention %q: %v", w, err)
}
}
return
}
if err != nil {
t.Fatalf("the node was refused whole: %v", err)
}
if len(held) != len(c.held) {
t.Errorf("held %v, want modules %v", held, c.held)
}
for m, w := range c.held {
if !strings.Contains(held[m], w) {
t.Errorf("%s is held with %q, want %q", m, held[m], w)
}
}
if _, heldToo := held["viewer"]; c.absent && (heldToo || len(got.Absent) != 1 || got.Absent[0] != (mcat.Absent{Consumer: "viewer", Requirement: "codec"})) {
t.Errorf("the optional consumer was held or not listed absent: held %v absent %v", held, got.Absent)
}
if len(got.Modules) != len(c.assigned) {
t.Errorf("the node does not keep every module of its set: %v", names(got))
}
})
} }
} }
+18 -9
View File
@@ -291,22 +291,31 @@ func (c *composer) resolutionHolding(nodeName string) (catalogue.Resolution, map
At: onNetwork[nodeName], PublicDomain: node.PublicDomain, At: onNetwork[nodeName], PublicDomain: node.PublicDomain,
Account: node.Account, AccountHome: node.AccountHome, Account: node.Account, AccountHome: node.AccountHome,
AgentAccount: node.AgentAccount, AgentAccountHome: node.AgentAccountHome}, world) AgentAccount: node.AgentAccount, AgentAccountHome: node.AgentAccountHome}, world)
if err != nil { // What another node's set would have offered is missing when that set does not resolve here: said with the refusal, and with
// What another node's set would have offered is missing when that set does not resolve here: said with the // each module held for a requirement nothing answers, so "nothing provides" is read beside why (issue 1126, probe #532).
// refusal, so "nothing provides" is read beside why (issue 1126, probe #532). why := func() string {
var out string
if len(left) > 0 { if len(left) > 0 {
var why []string var parts []string
for _, l := range left { for _, l := range left {
why = append(why, l.String()) parts = append(parts, l.String())
} }
err = fmt.Errorf("%w\n\nand %d node(s) offer nothing here, as they could not be resolved: %s", err, len(left), strings.Join(why, "; ")) out += fmt.Sprintf("\n\nand %d node(s) offer nothing here, as they could not be resolved: %s", len(left), strings.Join(parts, "; "))
} }
if len(c.unhostable) > 0 { if len(c.unhostable) > 0 {
err = fmt.Errorf("%w\n\nand these modules offer nothing here, as their node cannot run them in this composition: %s", out += fmt.Sprintf("\n\nand these modules offer nothing here, as their node cannot run them in this composition: %s",
err, strings.Join(c.unhostable, "; ")) strings.Join(c.unhostable, "; "))
} }
return out
}
if err != nil {
// The node's own set does not compose: the only failure a mesh-wide gatherer may pass over. // The node's own set does not compose: the only failure a mesh-wide gatherer may pass over.
return catalogue.Resolution{}, nil, notResolvable{err} return catalogue.Resolution{}, nil, notResolvable{fmt.Errorf("%w%s", err, why())}
}
for m, w := range held {
if strings.Contains(w, "nothing provides") || strings.Contains(w, "nothing in this mesh provides") {
held[m] = w + why()
}
} }
return resolved, held, nil return resolved, held, nil
} }
+3 -1
View File
@@ -295,7 +295,9 @@ func TestARecordAnsweredNeedGetsItsKeyAndTheManagerItsPublicKey(t *testing.T) {
// 1362/AC5: a node whose own set does not compose returns an error callers can tell from an unreadable Input. // 1362/AC5: a node whose own set does not compose returns an error callers can tell from an unreadable Input.
func TestANodeWhoseOwnSetDoesNotComposeIsToldApartFromAnUnreadableInput(t *testing.T) { func TestANodeWhoseOwnSetDoesNotComposeIsToldApartFromAnUnreadableInput(t *testing.T) {
in := aMesh(t) in := aMesh(t)
in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Requires: []string{"nothing-gives-this"}} in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Requires: []string{"two-give-this"}}
in.Catalogue["giver-a"] = catalogue.Manifest{Module: "giver-a", Version: "1", Provides: catalogue.Offers("two-give-this")}
in.Catalogue["giver-b"] = catalogue.Manifest{Module: "giver-b", Version: "1", Provides: catalogue.Offers("two-give-this")}
laptop := nodeOf(t, &in, "laptop") laptop := nodeOf(t, &in, "laptop")
laptop.Assigned = append(laptop.Assigned, "rival-one") laptop.Assigned = append(laptop.Assigned, "rival-one")
if _, err := Resolve(in, "laptop"); err == nil || !IsUnresolvable(err) { if _, err := Resolve(in, "laptop"); err == nil || !IsUnresolvable(err) {
+12 -1
View File
@@ -7,6 +7,7 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"log"
"sort" "sort"
"strings" "strings"
"time" "time"
@@ -14,6 +15,7 @@ import (
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine" "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
oc "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/seats/operatorchannel" oc "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/seats/operatorchannel"
stdio "git.novox.be/novox/mesh-sdk/go" stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"git.novox.be/novox/mesh-sdk/go/busretry" "git.novox.be/novox/mesh-sdk/go/busretry"
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/seatlist" "git.novox.be/novox/mesh-sdk/go/seatlist"
@@ -52,6 +54,9 @@ type EngineCall struct {
Verb string Verb string
Caller string Caller string
Args map[string]any Args map[string]any
// Line is the link's bus connection, which a verb marked chunks sends its answer's file on (buscalls.SeatChunks.Send); nil
// where the call did not come over a live link.
Line buscalls.ChunkLine
} }
// EngineFunc answers one call: its result, or why it is refused. // EngineFunc answers one call: its result, or why it is refused.
@@ -157,12 +162,17 @@ func (l *natsLink) serveEngineSeat(handle EngineFunc, module string) {
Description: "the node seat node-engine of " + l.node + " (novox/hq ADR 0563)", QueueGroupDisabled: true, Description: "the node seat node-engine of " + l.node + " (novox/hq ADR 0563)", QueueGroupDisabled: true,
Metadata: map[string]string{"seat": EngineSeat, "scope": "node", "node": l.node}}) Metadata: map[string]string{"seat": EngineSeat, "scope": "node", "node": l.node}})
if err != nil { if err != nil {
log.Printf("the node seat %s of %s is not served: %v (A-405)", EngineSeat, l.node, err)
return return
} }
apart := make(chan struct{}, apartAtOnce) apart := make(chan struct{}, apartAtOnce)
withLine := func(c EngineCall) (any, error) {
c.Line = l.conn // a verb marked chunks sends its file on the link's own connection
return handle(c)
}
for _, v := range ServedEngineSeat().Serves { for _, v := range ServedEngineSeat().Serves {
verb := v.Name verb := v.Name
answer := EngineAnswer(handle, verb, l.node, apart) answer := EngineAnswer(withLine, verb, l.node, apart)
meta := EngineEndpointMetadata(v, l.node, module) meta := EngineEndpointMetadata(v, l.node, module)
_ = svc.AddEndpoint(verb, answer, micro.WithEndpointSubject(EngineToolSubject(verb, l.node)), _ = svc.AddEndpoint(verb, answer, micro.WithEndpointSubject(EngineToolSubject(verb, l.node)),
micro.WithEndpointMetadata(meta)) micro.WithEndpointMetadata(meta))
@@ -172,6 +182,7 @@ func (l *natsLink) serveEngineSeat(handle EngineFunc, module string) {
l.engineMu.Lock() l.engineMu.Lock()
l.engine = svc l.engine = svc
l.engineMu.Unlock() l.engineMu.Unlock()
log.Printf("the node seat %s of %s is served as %s (A-405)", EngineSeat, l.node, module)
} }
// answeredApart are the verbs answered off the subscription that delivered them: each endpoint's subscription hands its // answeredApart are the verbs answered off the subscription that delivered them: each endpoint's subscription hands its
+4 -2
View File
@@ -8,6 +8,7 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"git.novox.be/novox/mesh-sdk/go/buscalls"
sdkkeys "git.novox.be/novox/mesh-sdk/go/keys" sdkkeys "git.novox.be/novox/mesh-sdk/go/keys"
"sort" "sort"
"strconv" "strconv"
@@ -175,7 +176,8 @@ type HandActs struct {
// ComposeCandidate composes a check's candidate (the indexer's candidate key) on this node in shadow and answers per // ComposeCandidate composes a check's candidate (the indexer's candidate key) on this node in shadow and answers per
// module, applying and writing nothing; with input, also the composition input it composed from (the verb // module, applying and writing nothing; with input, also the composition input it composed from (the verb
// compose-candidate, ADR 0593 §7). // compose-candidate, ADR 0593 §7).
ComposeCandidate func(candidate string, input bool) (any, error) // call is the caller's token for the chunks the input is sent in (A-237b) and line the connection they go on; with no token the input is inline.
ComposeCandidate func(candidate string, input bool, call string, line buscalls.ChunkLine) (any, error)
// ManifestReads is what this engine states it reads of a manifest and when its record last carried them (the verb // ManifestReads is what this engine states it reads of a manifest and when its record last carried them (the verb
// manifest-reads, novox/hq ADR 0618 §3). // manifest-reads, novox/hq ADR 0618 §3).
ManifestReads func() ([]string, time.Time) ManifestReads func() ([]string, time.Time)
@@ -289,7 +291,7 @@ func (h *HandActs) Call(c EngineCall) (any, error) {
if b, ok := c.Args["input"].(bool); ok { if b, ok := c.Args["input"].(bool); ok {
input = b input = b
} }
return h.ComposeCandidate(arg("candidate"), input) return h.ComposeCandidate(arg("candidate"), input, arg("call"), c.Line)
case ActManifestReads: case ActManifestReads:
if h.ManifestReads == nil { if h.ManifestReads == nil {
return nil, errors.New("this engine states no manifest reads yet: its node agent has not started") return nil, errors.New("this engine states no manifest reads yet: its node agent has not started")
+9
View File
@@ -116,6 +116,15 @@
"$SRV.PING.node-tools.NODE", "$SRV.PING.node-tools.NODE",
"mesh.control.NODE.>", "mesh.control.NODE.>",
"mesh.seat.mesh-catalog.call.entry.node~NODE", "mesh.seat.mesh-catalog.call.entry.node~NODE",
"mesh.seat.mesh-catalog.call.module.node~NODE",
"mesh.seat.container-image-registry.call.versions.node~NODE",
"mesh.seat.mesh-catalog.call.revision.node~NODE",
"mesh.seat.mesh-grants.call.withheld.node~NODE",
"mesh.seat.mesh-module-assigner.call.assignments.node~NODE",
"mesh.seat.mesh-recovery.call.kept-points.node~NODE",
"mesh.seat.mesh-recovery.call.restore-kept.node~NODE",
"mesh.seat.mesh-recovery.call.snapshot-kept.node~NODE",
"mesh.seat.oci-artifact-registry.call.versions.node~NODE",
"mesh.seat.mesh-catalog.call.seat-definitions.node~NODE", "mesh.seat.mesh-catalog.call.seat-definitions.node~NODE",
"mesh.seat.node-backup.call.backed-up.NODE.node~NODE", "mesh.seat.node-backup.call.backed-up.NODE.node~NODE",
"mesh.seat.node-backup.call.now.NODE.node~NODE", "mesh.seat.node-backup.call.now.NODE.node~NODE",
+117 -23
View File
@@ -9,11 +9,14 @@ import (
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"git.novox.be/novox/mesh-sdk/go/keys"
"github.com/novox/mesh-host/internal/buckets" "github.com/novox/mesh-host/internal/buckets"
engcomp "github.com/novox/mesh-host/internal/compositions" engcomp "github.com/novox/mesh-host/internal/compositions"
"maps" "maps"
"slices" "slices"
"sort" "sort"
"strings"
"sync" "sync"
"time" "time"
@@ -143,13 +146,7 @@ func (s *Shadow) composeCandidate(ctx context.Context, c *Composer, ask nodeengi
a.Verdict = nec.Unready a.Verdict = nec.Unready
return a return a
} }
group, heads, err := nec.ParseCompositionKey(ask.Candidate) key, err := nec.ParseCandidateKey(ask.Candidate)
if err != nil {
return refuse(err.Error())
}
// The ask names the check (the composition key, `<group>.<heads hash>`); the indexer keys the group's candidate by
// the group alone (buckets.CandidateKey) and holds its heads hash in the value.
ckey, err := buckets.CandidateKey(group)
if err != nil { if err != nil {
return refuse(err.Error()) return refuse(err.Error())
} }
@@ -157,21 +154,48 @@ func (s *Shadow) composeCandidate(ctx context.Context, c *Composer, ask nodeengi
return unready(errNotReady.Error()) return unready(errNotReady.Error())
} }
base := run.add(c.scratch()) base := run.add(c.scratch())
raw, err := base.get(ctx, buckets.Candidates, ckey) var cand meshcatalog.CandidatesByGroupValue
if err != nil { var group, heads string
return unready(fmt.Sprintf("the indexer's candidate %s cannot be read: %v", ask.Candidate, err)) if key.Kind == nec.CandidateRevision {
} // A branch pipeline's catalog graph revision (A-226b): its entries are read through the seat mesh-catalog's verb
if raw == nil { // revision and composed as the candidate, each module at the revision's fingerprint.
return refuse("the indexer holds no candidate " + ask.Candidate) var answered bool
} cand, answered, err = s.revisionCandidate(ctx, base, key)
cand, err := meshcatalog.DecodeCandidatesByGroupValue(raw) if err != nil {
if err != nil { return unready(err.Error())
return refuse(fmt.Sprintf("the indexer's candidate %s cannot be read: %v", ask.Candidate, err)) }
} if !answered {
if cand.Heads != heads { return refuse(fmt.Sprintf("the catalog holds no revision %d of the branch %s", key.Revision, key.Branch))
// The group moved to other heads: the check asked is replaced, and this node composes not a candidate it was }
// not asked about. No answer (ADR 0593 §9), never a verdict on the wrong heads. } else {
return unready(fmt.Sprintf("the indexer's candidate of the group %s is at heads %s, not %s: the check asked is replaced", group, cand.Heads, heads)) group, heads = key.Group, key.HeadsHash
if key.Kind == nec.CandidateRun {
// A run on request's candidate is the indexer's `candidates.run-<id>`, read by its group as a check's is, with no
// heads to compare: the run is asked for by its id alone (A-226c).
group, heads = "run-"+key.Run, ""
}
// The ask names the check (the composition key, `<group>.<heads hash>`); the indexer keys the group's candidate by
// the group alone (buckets.CandidateKey) and holds its heads hash in the value.
ckey, err := buckets.CandidateKey(group)
if err != nil {
return refuse(err.Error())
}
raw, err := base.get(ctx, buckets.Candidates, ckey)
if err != nil {
return unready(fmt.Sprintf("the indexer's candidate %s cannot be read: %v", ask.Candidate, err))
}
if raw == nil {
return refuse("the indexer holds no candidate " + ask.Candidate)
}
cand, err = meshcatalog.DecodeCandidatesByGroupValue(raw)
if err != nil {
return refuse(fmt.Sprintf("the indexer's candidate %s cannot be read: %v", ask.Candidate, err))
}
if key.Kind == nec.CandidateCheck && cand.Heads != heads {
// The group moved to other heads: the check asked is replaced, and this node composes not a candidate it was
// not asked about. No answer (ADR 0593 §9), never a verdict on the wrong heads.
return unready(fmt.Sprintf("the indexer's candidate of the group %s is at heads %s, not %s: the check asked is replaced", group, cand.Heads, heads))
}
} }
// Every module's manifest is read before either composition, several at once, into the shadow's own: a composition // Every module's manifest is read before either composition, several at once, into the shadow's own: a composition
// reads every manifest in the mesh one by one, which off the control-node took about 35 s each (issue 1126). // reads every manifest in the mesh one by one, which off the control-node took about 35 s each (issue 1126).
@@ -214,7 +238,9 @@ func (s *Shadow) composeCandidate(ctx context.Context, c *Composer, ask nodeengi
} }
} }
}) })
if sentSize > maxSentInput { // An ask with a call token takes the input in chunks, whatever its size (A-237b); one with none is the checker's, answered
// inline until G removes the checker (G-6 §321, A-237c), so past what the bus carries in one answer it is refused.
if sentSize > maxSentInput && ask.Call == "" {
// Past what the bus carries in one answer, the checker would hear nothing: refused, saying so. // Past what the bus carries in one answer, the checker would hear nothing: refused, saying so.
size := sentSize size := sentSize
a.Input = nil a.Input = nil
@@ -481,3 +507,71 @@ func (c *Composer) composeShadow(ctx context.Context, running []engcomp.Composed
// candidateNoneNoModule is the words the indexer writes in a candidate's module that the merge preview names no manifest // candidateNoneNoModule is the words the indexer writes in a candidate's module that the merge preview names no manifest
// for: the change removes the module. It is the indexer's text, kept here until the catalogue client names it. // for: the change removes the module. It is the indexer's text, kept here until the catalogue client names it.
const candidateNoneNoModule = "the merge preview names no manifest for the module's directory" const candidateNoneNoModule = "the merge preview names no manifest for the module's directory"
// revisionCandidate reads the catalog graph revision key names, whole, page by page, through the seat mesh-catalog's verb
// revision (the Entry call, the engine's own name), and makes the candidate it composes as: one pull per repository and
// commit with each module's fingerprint and manifest digest, and a module the base holds that the revision does not marked as
// removed. answered is false when the catalog holds no such revision.
func (s *Shadow) revisionCandidate(ctx context.Context, base *Composer, key nec.Candidate) (cand meshcatalog.CandidatesByGroupValue, answered bool, err error) {
call := base.env.Entry
if call == nil {
return cand, false, errors.New("this node's engine cannot ask the catalog for a revision")
}
in := meshcatalog.RevisionInput{Branch: keys.EscapeName(key.Branch), Revision: key.Revision}
type group struct{ repository, commit string }
pulls := map[group]map[string]meshcatalog.CandidatesByGroupValuePullsItemFpsValue{}
seen := map[string]bool{}
for {
raw, err := call(meshcatalogclient.RevisionVerb, in)
if err != nil {
return cand, false, fmt.Errorf("the catalog's revision %d of %s cannot be read: %w", key.Revision, key.Branch, err)
}
body, said, ok := buscalls.Unwrap(raw)
if !ok {
return cand, false, fmt.Errorf("the catalog's revision %d of %s cannot be read: %s", key.Revision, key.Branch, said)
}
var a meshcatalog.RevisionAnswer
if err := json.Unmarshal(body, &a); err != nil {
return cand, false, fmt.Errorf("the catalog's revision %d of %s is not a revision answer: %w", key.Revision, key.Branch, err)
}
if a.NoneYet || a.Revision == nil || a.Revision.Revision != key.Revision {
return cand, false, nil
}
for _, e := range a.Entries {
seen[e.Module] = true
if e.Fp == "" {
continue // no build at the revision: composed as the trunk holds it
}
g := group{e.Repository, e.Commit}
if pulls[g] == nil {
pulls[g] = map[string]meshcatalog.CandidatesByGroupValuePullsItemFpsValue{}
}
// The catalog entry's manifest-digest is the `sha256:`-prefixed form (the seat's definition, entryManifest); a candidate's
// module and the manifests and verdicts keys carry the 64 hex digits alone (ADR 0429 clause 16, meshcatalogclient).
pulls[g][e.Module] = meshcatalog.CandidatesByGroupValuePullsItemFpsValue{Fp2: e.Fp, Manifest: strings.TrimPrefix(e.ManifestDigest, "sha256:")}
}
if a.Next == "" {
break
}
in.After = a.Next
}
removed := map[string]meshcatalog.CandidatesByGroupValuePullsItemFpsValue{}
base.mu.Lock()
for module := range base.modules {
if !seen[module] {
removed[module] = meshcatalog.CandidatesByGroupValuePullsItemFpsValue{None2: candidateNoneNoModule}
}
}
base.mu.Unlock()
cand.Group = key.Branch
for g, fps := range pulls {
cand.Pulls = append(cand.Pulls, meshcatalog.CandidatesByGroupValuePullsItem{Repository: g.repository, Head: g.commit, Fps: fps})
}
sort.Slice(cand.Pulls, func(i, j int) bool {
return cand.Pulls[i].Repository+cand.Pulls[i].Head < cand.Pulls[j].Repository+cand.Pulls[j].Head
})
if len(removed) > 0 {
cand.Pulls = append(cand.Pulls, meshcatalog.CandidatesByGroupValuePullsItem{Fps: removed})
}
return cand, true, nil
}
+33 -1
View File
@@ -85,7 +85,18 @@ func TestTheNodeComposesACandidateOnAsk(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
entry := meshcatalog.CatalogEntry{Module: "lemurs", Fp: fp, Manifest: manifest, Repository: "novox/x", Commit: "c", ManifestDigest: digest} entry := meshcatalog.CatalogEntry{Module: "lemurs", Fp: fp, Manifest: manifest, Repository: "novox/x", Commit: "c", ManifestDigest: digest}
env.Entry = EntryCall(func(_ context.Context, _, _ string, _ any) (json.RawMessage, error) { // A-226b: the revision a branch pipeline names is read through the verb revision: revision 362 of main holds lemurs at its
// new build and manifest; no other revision is held.
env.Entry = EntryCall(func(_ context.Context, _, verb string, args any) (json.RawMessage, error) {
if verb == meshcatalog.VerbRevision {
in, _ := args.(meshcatalog.RevisionInput)
if in.Revision != 362 || in.Branch != "main" {
return json.Marshal(meshcatalog.RevisionAnswer{NoneYet: true})
}
moved := entry
moved.Fp, moved.ManifestDigest = fpNew, "sha256:"+digest2 // the entry carries the prefixed form
return json.Marshal(meshcatalog.RevisionAnswer{Revision: &meshcatalog.CatalogRevision{Branch: "main", Revision: 362}, Entries: []meshcatalog.CatalogEntry{moved}})
}
return json.Marshal(meshcatalog.EntryAnswer{Entry: entry}) return json.Marshal(meshcatalog.EntryAnswer{Entry: entry})
}) })
var handed atomic.Int64 var handed atomic.Int64
@@ -141,6 +152,27 @@ func TestTheNodeComposesACandidateOnAsk(t *testing.T) {
if _, err := nec.EncodeAnswer(r); err != nil { if _, err := nec.EncodeAnswer(r); err != nil {
t.Errorf("the refused answer does not fit the seat's shape: %v", err) t.Errorf("the refused answer does not fit the seat's shape: %v", err)
} }
// A-226b: a branch pipeline's revision composes as the candidate, each module at the revision's build; one the catalog does
// not hold, and a run on request's key, are refused in words.
revKey, _ := nec.RevisionKey("main", 362)
rv := s.ComposeCandidate(ctx, nodeengine.ComposeCandidateInput{Candidate: revKey})
if rv.Verdict != nec.Composed || len(rv.Modules) != 1 || rv.Modules[0].Fp != fpNew || !rv.Modules[0].Changed {
t.Errorf("a catalog revision answered %+v", rv)
}
noRev, _ := nec.RevisionKey("main", 999)
if m := s.ComposeCandidate(ctx, nodeengine.ComposeCandidateInput{Candidate: noRev}); m.Verdict != nec.Refused || !strings.Contains(m.Refused, "holds no revision 999") {
t.Errorf("a revision the catalog does not hold answered %+v", m)
}
// A-226c: a run's key reads the indexer's candidates.run-<id> by its group, with no heads compared; one it holds none of is refused.
candidateOf("run-r1", digest2)
runKey, _ := nec.RunKey("r1")
if m := s.ComposeCandidate(ctx, nodeengine.ComposeCandidateInput{Candidate: runKey}); m.Verdict != nec.Composed || len(m.Modules) != 1 || m.Modules[0].Fp != fpNew || !m.Modules[0].Changed {
t.Errorf("a run's key answered %+v", m)
}
noRun, _ := nec.RunKey("r2")
if m := s.ComposeCandidate(ctx, nodeengine.ComposeCandidateInput{Candidate: noRun}); m.Verdict != nec.Refused || !strings.Contains(m.Refused, "holds no candidate") {
t.Errorf("a run the indexer holds none of answered %+v", m)
}
missing, _ := nec.CompositionKey("novox/x#8", "0123456789ab") missing, _ := nec.CompositionKey("novox/x#8", "0123456789ab")
if m := s.ComposeCandidate(ctx, nodeengine.ComposeCandidateInput{Candidate: missing}); !strings.Contains(m.Refused, "holds no candidate") { if m := s.ComposeCandidate(ctx, nodeengine.ComposeCandidateInput{Candidate: missing}); !strings.Contains(m.Refused, "holds no candidate") {
t.Errorf("a candidate the indexer does not hold answered %+v", m) t.Errorf("a candidate the indexer does not hold answered %+v", m)
+203 -42
View File
@@ -21,6 +21,7 @@ import (
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-host/internal/agent" "github.com/novox/mesh-host/internal/agent"
engcomp "github.com/novox/mesh-host/internal/compositions"
) )
// **The catalog half, read by fingerprint** (design 47 G §421, §683, §698; the lead's answers A-47 and A-50). The node-engine // **The catalog half, read by fingerprint** (design 47 G §421, §683, §698; the lead's answers A-47 and A-50). The node-engine
@@ -150,9 +151,42 @@ func (c *Composer) entryChanged(row meshcatalogclient.EntryRow) {
c.keep(entryKey(row.Fingerprint), meshcatalog.EntryAnswer{Entry: *row.Entry}) c.keep(entryKey(row.Fingerprint), meshcatalog.EntryAnswer{Entry: *row.Entry})
} }
c.tellNotFound(row.Fingerprint, row.State == meshcatalogclient.EntryNotFound) c.tellNotFound(row.Fingerprint, row.State == meshcatalogclient.EntryNotFound)
if row.State == meshcatalogclient.EntryAnswered {
c.handAgainFor(row.Fingerprint)
}
c.askRehand() c.askRehand()
} }
// handAgainFor forgets what was handed of each module of this node whose want or running build is fp, so the next pass hands it
// again and the node agent, which waits for this entry to judge a move (agent.keptStep), settles it again.
func (c *Composer) handAgainFor(fp string) {
var running []engcomp.Composed
if c.env.Running != nil {
running = c.env.Running()
}
c.mu.Lock()
defer c.mu.Unlock()
for module, w := range c.wants {
if module == agent.NodeModule || w.Fp == "" {
continue
}
// Only a move is judged by the entries (agent.keptStep): a module that runs the build its want names has none to wait on.
moves, runs := false, false
for _, r := range running {
if r.Module == module {
moves = moves || r.FP != w.Fp
runs = runs || r.FP == fp
}
}
if !moves || (w.Fp != fp && !runs) {
continue
}
if key, err := engcomp.Key(c.env.Node, module, w.Fp); err == nil {
delete(c.handed, key)
}
}
}
// tellNotFound tells the finding entry-not-found once when fp's entry is not found, and once when it no longer is (design 47 G §614, A-117). // tellNotFound tells the finding entry-not-found once when fp's entry is not found, and once when it no longer is (design 47 G §614, A-117).
func (c *Composer) tellNotFound(fp string, notFound bool) { func (c *Composer) tellNotFound(fp string, notFound bool) {
c.mu.Lock() c.mu.Lock()
@@ -222,20 +256,46 @@ func (c *Composer) entryOf(fp string) (meshcatalog.CatalogEntry, meshcatalogclie
// wantedBuilds is each fingerprint a want of this node or a running heard names, of the new form, with whether a want names it. // wantedBuilds is each fingerprint a want of this node or a running heard names, of the new form, with whether a want names it.
func (c *Composer) wantedBuilds() map[string]bool { func (c *Composer) wantedBuilds() map[string]bool {
out := map[string]bool{} out := map[string]bool{}
// Read before the lock: what the node runs is the agent's, and a read of it may wait.
var running []engcomp.Composed
if c.env.Running != nil {
running = c.env.Running()
}
c.mu.Lock() c.mu.Lock()
defer c.mu.Unlock() defer c.mu.Unlock()
for module, w := range c.wants { for module, w := range c.wants {
if module != agent.NodeModule && strings.HasPrefix(w.Fp, meshcatalogclient.SourcePrefix2) { if module != agent.NodeModule && strings.HasPrefix(w.Fp, meshcatalogclient.SourcePrefix2) {
out[w.Fp] = true out[w.Fp] = true
} }
// The build its want says is being built describes, for resolution alone, a module whose wanted build has no entry (A-405, A-407).
if module != agent.NodeModule && strings.HasPrefix(w.Newest, meshcatalogclient.SourcePrefix2) {
if _, named := out[w.Newest]; !named {
out[w.Newest] = false
}
}
} }
for _, mods := range c.heard { for node, mods := range c.heard {
for _, run := range mods { for module, run := range mods {
if !run.stopped && strings.HasPrefix(run.fingerprint, meshcatalogclient.SourcePrefix2) { if !run.stopped && strings.HasPrefix(run.fingerprint, meshcatalogclient.SourcePrefix2) {
if _, named := out[run.fingerprint]; !named { if _, named := out[run.fingerprint]; !named {
out[run.fingerprint] = false out[run.fingerprint] = false
} }
} }
// The build the assignment places for another node's module stands for its want where the running build has no entry (A-461).
if built := c.placedBuild(node, module); c.lacksEntry(node, module, run) && strings.HasPrefix(built, meshcatalogclient.SourcePrefix2) {
if _, named := out[built]; !named {
out[built] = false
}
}
}
}
// The build each module runs here, as the node's own record of it says: its entry, kept in the composition state, stays held
// for the judgement of a move's data forms though the catalog half dropped it and no event of the stream names it.
for _, r := range running {
if r.Module != agent.NodeModule && strings.HasPrefix(r.FP, meshcatalogclient.SourcePrefix2) {
if _, named := out[r.FP]; !named {
out[r.FP] = false
}
} }
} }
return out return out
@@ -245,6 +305,7 @@ func (c *Composer) wantedBuilds() map[string]bool {
// composes from, and lays the picture frombus reads (c.modules, c.manifests) from the entries it holds. It runs at the start of // composes from, and lays the picture frombus reads (c.modules, c.manifests) from the entries it holds. It runs at the start of
// each pass. // each pass.
func (c *Composer) syncEntries(ctx context.Context) { func (c *Composer) syncEntries(ctx context.Context) {
c.askTrunkBuilds()
need := c.wantedBuilds() need := c.wantedBuilds()
for fp := range need { for fp := range need {
c.entryOf(fp) c.entryOf(fp)
@@ -309,18 +370,24 @@ func (c *Composer) syncEntries(ctx context.Context) {
// layModules lays c.modules and c.manifests from the entries held: what each node was heard to run first, then each module this // layModules lays c.modules and c.manifests from the entries held: what each node was heard to run first, then each module this
// node's want names, at the want's fingerprint (a want wins). // node's want names, at the want's fingerprint (a want wins).
func (c *Composer) layModules() { func (c *Composer) layModules() {
var said []string
defer func() {
for _, line := range said {
c.sayOnceEach(line)
}
}()
c.mu.Lock() c.mu.Lock()
defer c.mu.Unlock() defer c.mu.Unlock()
modules := map[string]meshcatalog.ModulesByModuleValue{} modules := map[string]meshcatalog.ModulesByModuleValue{}
lay := func(module, fp string) { lay := func(module, fp string) bool {
k, ok := c.keptEnt[fp] k, ok := c.keptEnt[fp]
if !ok || k.entry.Module == "" { if !ok || k.entry.Module == "" {
return return false
} }
e := k.entry e := k.entry
digest, raw, err := entryManifest(e) digest, raw, err := entryManifest(e)
if err != nil { if err != nil {
return return false
} }
m := meshcatalog.ModulesByModuleValue{Fp2: fp, Manifest: digest, Repository: e.Repository, Commit: e.Commit, Path: e.Path, m := meshcatalog.ModulesByModuleValue{Fp2: fp, Manifest: digest, Repository: e.Repository, Commit: e.Commit, Path: e.Path,
Needs: e.Needs, Packages: e.Packages, Verdict: e.Verdict} Needs: e.Needs, Packages: e.Packages, Verdict: e.Verdict}
@@ -329,6 +396,7 @@ func (c *Composer) layModules() {
} }
modules[module] = m modules[module] = m
c.manifests[digest] = raw c.manifests[digest] = raw
return true
} }
nodes := make([]string, 0, len(c.heard)) nodes := make([]string, 0, len(c.heard))
for node := range c.heard { for node := range c.heard {
@@ -338,14 +406,37 @@ func (c *Composer) layModules() {
for _, node := range nodes { for _, node := range nodes {
for module, run := range c.heard[node] { for module, run := range c.heard[node] {
if !run.stopped && run.fingerprint != "" { if !run.stopped && run.fingerprint != "" {
lay(module, run.fingerprint) // Where the running build has no entry, the module is laid from the entry of the build the assignment places
// for that node, the mesh's truth of what it runs there (A-461); with neither it stays name-only below.
if !lay(module, run.fingerprint) && node != c.env.Node {
if built := c.placedBuild(node, module); built != "" && built != run.fingerprint {
lay(module, built)
}
}
} }
} }
} }
for module, w := range c.wants { for module, w := range c.wants {
if module != agent.NodeModule && w.Fp != "" { if module == agent.NodeModule || w.Fp == "" {
lay(module, w.Fp) continue
} }
if k, ok := c.keptEnt[w.Fp]; ok && k.entry.Module != "" {
lay(module, w.Fp)
continue
}
// A module the node runs whose wanted build has no entry stands as the build its want says is being built, for resolution
// alone: what that entry provides, claims and requires stands for the running build. Its want stays held (entryHolds), at
// the want's fingerprint, and nothing new of it is composed (A-405, A-407).
if k, ok := c.keptEnt[w.Newest]; ok && w.Newest != "" && k.entry.Module != "" && c.runsHere(module) {
lay(module, w.Newest)
if m, laid := modules[module]; laid {
m.Fp2 = w.Fp
modules[module] = m
said = append(said, fmt.Sprintf("%s runs a build whose entry the catalog half does not hold, so the node resolves around it as the build %s its want says is being built; its want stays held", module, w.Newest))
}
continue
}
lay(module, w.Fp)
} }
// A module an assignment places whose entry is not held (not wanted here, not heard running, or its entry asked and not // A module an assignment places whose entry is not held (not wanted here, not heard running, or its entry asked and not
// answered) stands in the catalogue as its name alone, so the placement still finds it: a module held for its entry offers // answered) stands in the catalogue as its name alone, so the placement still finds it: a module held for its entry offers
@@ -366,6 +457,60 @@ func (c *Composer) layModules() {
c.modules = modules c.modules = modules
} }
// placedBuild is the build the assignment places for module on node (c.mu held), the mesh's truth of what it runs there; where the
// placement names none, the module's trunk build as syncEntries asked it (A-461b). Empty where neither is known.
func (c *Composer) placedBuild(node, module string) string {
for _, pm := range c.assignments[node].Modules {
if pm.Module == module {
if pm.Build != "" {
return pm.Build
}
return c.trunkBuilds[module].fp
}
}
return ""
}
// lacksEntry says whether the build another node's module runs has no entry held (c.mu held): the only case the placed build stands
// in (A-461b). It is never so for this node's own modules.
func (c *Composer) lacksEntry(node, module string, run heardRun) bool {
if node == c.env.Node || run.stopped || run.fingerprint == "" {
return false
}
k, ok := c.keptEnt[run.fingerprint]
return !ok || k.entry.Module == ""
}
// askTrunkBuilds asks, for each module another node runs at a build with no entry and whose placement names no build, the trunk
// build by the seat's verb `module`, once per running build (A-461b). Called before the lock is taken: the verb is a call.
func (c *Composer) askTrunkBuilds() {
type ask struct{ node, module, running string }
var asks []ask
c.mu.Lock()
for node, mods := range c.heard {
for module, run := range mods {
if !c.lacksEntry(node, module, run) {
continue
}
for _, pm := range c.assignments[node].Modules {
if pm.Module == module && pm.Build == "" && c.trunkBuilds[module].running != run.fingerprint {
asks = append(asks, ask{node, module, run.fingerprint})
}
}
}
}
c.mu.Unlock()
for _, a := range asks {
fp, _ := c.trunkFp(a.module)
c.mu.Lock()
if c.trunkBuilds == nil {
c.trunkBuilds = map[string]trunkBuild{}
}
c.trunkBuilds[a.module] = trunkBuild{running: a.running, fp: fp}
c.mu.Unlock()
}
}
// entryManifest is an entry's manifest as a reader takes it, with its digest: the one the indexer set at take-in, or, where it set // entryManifest is an entry's manifest as a reader takes it, with its digest: the one the indexer set at take-in, or, where it set
// none, the digest of the manifest's bytes. // none, the digest of the manifest's bytes.
func entryManifest(e meshcatalog.CatalogEntry) (digest string, raw []byte, err error) { func entryManifest(e meshcatalog.CatalogEntry) (digest string, raw []byte, err error) {
@@ -404,7 +549,7 @@ func (c *Composer) entryHolds() map[string]string {
out[module] = fmt.Sprintf("%s the catalog half holds no entry of the build %s, so the node composes nothing new of %s", out[module] = fmt.Sprintf("%s the catalog half holds no entry of the build %s, so the node composes nothing new of %s",
agent.EntryMissing, w.Fp, module) agent.EntryMissing, w.Fp, module)
default: default:
if missing := partsMissing(e, w); missing != "" { if _, missing := c.partsOf(module, e, w); missing != "" {
out[module] = fmt.Sprintf("the want of %s names no digest for %s, so the node composes nothing new of it", module, missing) out[module] = fmt.Sprintf("the want of %s names no digest for %s, so the node composes nothing new of it", module, missing)
} }
} }
@@ -412,19 +557,7 @@ func (c *Composer) entryHolds() map[string]string {
return out return out
} }
// partsMissing names the artifacts of e whose digest w's parts do not give; "" when each has one. // referenceOf is an artifact's reference made from the digest its want's part names, or its version resolves (A-430a), in the form the artifact store is
func partsMissing(e meshcatalog.CatalogEntry, w meshwants.WantsByWantNodeModuleValue) string {
var out []string
for _, a := range e.Artifacts {
if w.Parts[a.Ref] == "" {
out = append(out, a.Ref)
}
}
sort.Strings(out)
return strings.Join(out, ", ")
}
// referenceOf is an artifact's reference made from the digest its want's part names, in the form the artifact store is
// addressed by (the lead's answer A-50): artifact-store://<module>/<artifact>/blobs/<digest>. // addressed by (the lead's answer A-50): artifact-store://<module>/<artifact>/blobs/<digest>.
func referenceOf(module, artifact, digest string) string { func referenceOf(module, artifact, digest string) string {
return "artifact-store://" + module + "/" + artifact + "/blobs/" + digest return "artifact-store://" + module + "/" + artifact + "/blobs/" + digest
@@ -439,7 +572,11 @@ func (c *Composer) buildOf(module, fp string) *builds.BuilderBuild {
w, wanted := c.wants[module] w, wanted := c.wants[module]
k, ok := c.keptEnt[fp] k, ok := c.keptEnt[fp]
c.mu.Unlock() c.mu.Unlock()
if !wanted || w.Fp != fp || !ok || partsMissing(k.entry, w) != "" { if !wanted || w.Fp != fp || !ok {
return nil
}
parts, missing := c.partsOf(module, k.entry, w)
if missing != "" {
return nil return nil
} }
digest, raw, err := entryManifest(k.entry) digest, raw, err := entryManifest(k.entry)
@@ -452,7 +589,7 @@ func (c *Composer) buildOf(module, fp string) *builds.BuilderBuild {
b := builds.BuilderBuild{State: builds.BuildDone, Manifest: digest} b := builds.BuilderBuild{State: builds.BuildDone, Manifest: digest}
for _, a := range k.entry.Artifacts { for _, a := range k.entry.Artifacts {
_, name, _ := strings.Cut(a.Ref, ":") _, name, _ := strings.Cut(a.Ref, ":")
digest := w.Parts[a.Ref] digest := parts[a.Ref]
b.Artifacts = append(b.Artifacts, builds.BuiltArtifact{Name: name, Kind: a.Kind, Digest: digest, b.Artifacts = append(b.Artifacts, builds.BuiltArtifact{Name: name, Kind: a.Kind, Digest: digest,
Reference: referenceOf(module, name, digest)}) Reference: referenceOf(module, name, digest)})
} }
@@ -582,30 +719,38 @@ func (c *Composer) syncShadowEntries(ctx context.Context) (pending bool) {
// moduleVerb is the seat mesh-catalog's verb `module`: one module's trunk fingerprint and manifest by name. // moduleVerb is the seat mesh-catalog's verb `module`: one module's trunk fingerprint and manifest by name.
const moduleVerb = "seat:" + meshcatalog.Seat + "." + meshcatalog.VerbModule const moduleVerb = "seat:" + meshcatalog.Seat + "." + meshcatalog.VerbModule
// trunkFp is the fingerprint of a module's trunk build, by the seat's verb `module`; empty with why where the catalog half gives none.
func (c *Composer) trunkFp(module string) (fp, why string) {
if c.env.Entry == nil {
return "", "this node reads no module of the catalog half"
}
raw, err := c.env.Entry(moduleVerb, meshcatalog.ModuleInput{Module: module})
if err != nil {
return "", fmt.Sprintf("the catalog half's answer for %s: %v", module, err)
}
answer, said, ok := buscalls.Unwrap(raw)
if !ok {
return "", fmt.Sprintf("the catalog half holds no module %s: %s", module, said)
}
var a meshcatalog.ModuleAnswer
if err := json.Unmarshal(answer, &a); err != nil {
return "", fmt.Sprintf("the catalog half's answer for %s cannot be read: %v", module, err)
}
if a.Module.Fp == "" {
return "", fmt.Sprintf("the catalog half gives %s no fingerprint: %s", module, a.Module.None)
}
return a.Module.Fp, ""
}
// addedModule is the trunk's build of a module an assignment candidate adds to this node when no want here names it yet: the // addedModule is the trunk's build of a module an assignment candidate adds to this node when no want here names it yet: the
// fingerprint the seat's verb `module` answers, then its entry by the verb `entry` as for a wanted module (design 47 G §421, A-312). // fingerprint the seat's verb `module` answers, then its entry by the verb `entry` as for a wanted module (design 47 G §421, A-312).
// A call, not a watch: nothing of the catalog is kept. It answers the entry when held; otherwise why not: a module with no // A call, not a watch: nothing of the catalog is kept. It answers the entry when held; otherwise why not: a module with no
// fingerprint says the verb's reason, an entry asked says so (its answer wakes the shadow, which composes again). // fingerprint says the verb's reason, an entry asked says so (its answer wakes the shadow, which composes again).
func (c *Composer) addedModule(ctx context.Context, module string) (fp string, entry meshcatalog.CatalogEntry, held, pending bool, why string) { func (c *Composer) addedModule(ctx context.Context, module string) (fp string, entry meshcatalog.CatalogEntry, held, pending bool, why string) {
if c.env.Entry == nil { fp, why = c.trunkFp(module)
return "", entry, false, false, "this node reads no module of the catalog half" if fp == "" {
return "", entry, false, false, why
} }
raw, err := c.env.Entry(moduleVerb, meshcatalog.ModuleInput{Module: module})
if err != nil {
return "", entry, false, false, fmt.Sprintf("the catalog half's answer for %s: %v", module, err)
}
answer, said, ok := buscalls.Unwrap(raw)
if !ok {
return "", entry, false, false, fmt.Sprintf("the catalog half holds no module %s: %s", module, said)
}
var a meshcatalog.ModuleAnswer
if err := json.Unmarshal(answer, &a); err != nil {
return "", entry, false, false, fmt.Sprintf("the catalog half's answer for %s cannot be read: %v", module, err)
}
if a.Module.Fp == "" {
return "", entry, false, false, fmt.Sprintf("the catalog half gives %s no fingerprint: %s", module, a.Module.None)
}
fp = a.Module.Fp
c.mu.Lock() c.mu.Lock()
if c.addedFps == nil { if c.addedFps == nil {
c.addedFps = map[string]bool{} c.addedFps = map[string]bool{}
@@ -651,3 +796,19 @@ func (c *Composer) askBuildsEntries(fps map[string]string) (pending string) {
} }
return pending return pending
} }
// runsHere says whether this node runs the module: heard to, or in what the node runs now (c.mu held; env.Running reads the
// agent's own record).
func (c *Composer) runsHere(module string) bool {
if run, ok := c.heard[c.env.Node][module]; ok && !run.stopped {
return true
}
if c.env.Running != nil {
for _, r := range c.env.Running() {
if r.Module == module {
return true
}
}
}
return false
}
+131
View File
@@ -259,3 +259,134 @@ func (c *Composer) holdEntry(t *testing.T, module, fp, body string) {
} }
c.keptEnt[fp] = keptEntry{entry: half[fp]} c.keptEnt[fp] = keptEntry{entry: half[fp]}
} }
// A-405, A-407: a module the node runs whose wanted build has no entry stands, for resolution alone, as the build its want says
// is being built, at the want's fingerprint, so its want stays held; one not running here, or whose want names no newer build the
// half holds, stands as its name alone.
func runningModuleWithoutEntryStandsAsTheNewestBuild(t *testing.T) {
old := meshcatalogclient.SourcePrefix2 + strings.Repeat("1", 64)
newest := meshcatalogclient.SourcePrefix2 + strings.Repeat("2", 64)
body := `{"module":"postgres","version":"2"}`
for _, tc := range []struct {
name string
runs bool
newest string
holds bool
laid bool
wantFp string
}{
{"runs, newest entry held", true, newest, true, true, old},
{"not running here", false, newest, true, false, old},
{"names no newer build", true, "", false, false, old},
{"newest entry not held", true, newest, false, false, old},
} {
t.Run(tc.name, func(t *testing.T) {
env := newFakeBus().env()
env.Node = "n1"
c := New(env)
c.mu.Lock()
c.wants["postgres"] = meshwants.WantsByWantNodeModuleValue{Fp: old, Newest: tc.newest}
c.assignments["n1"] = meshmoduleassigner.Assignments{Node: "n1", Modules: []meshmoduleassigner.PlacedModule{{Module: "postgres"}}}
if tc.runs {
c.heard = map[string]map[string]heardRun{"n1": {"postgres": {fingerprint: old}}}
}
c.mu.Unlock()
if tc.holds {
c.holdEntry(t, "postgres", newest, body)
}
if got := c.wantedBuilds()[newest]; tc.newest != "" && got {
t.Errorf("the newest build is named by a want, want only asked")
}
c.layModules()
c.mu.Lock()
m, ok := c.modules["postgres"]
raw := c.manifests[m.Manifest]
c.mu.Unlock()
if !ok || m.Fp2 != tc.wantFp {
t.Fatalf("modules[postgres] = %+v (%v), want fingerprint %s", m, ok, tc.wantFp)
}
if got := strings.Contains(string(raw), `"version":"2"`); got != tc.laid {
t.Errorf("manifest of the newest build laid = %v, want %v (%s)", got, tc.laid, raw)
}
})
}
}
// The node judges a move's data forms from the entry it kept of the build it runs, though the catalog half dropped that entry and no
// running event names it; and an answered entry has the composition handed again, which the agent waits on (novox, mesh-hand-acts).
func TestRunningBuildEntryKeptAndHandedAgain(t *testing.T) {
old := meshcatalogclient.SourcePrefix2 + strings.Repeat("1", 64)
next := meshcatalogclient.SourcePrefix2 + strings.Repeat("2", 64)
env := newFakeBus().env()
env.Node = "n1"
env.Running = func() []compositions.Composed {
return []compositions.Composed{{Node: "n1", Module: "hand", FP: old}}
}
c := New(env)
c.mu.Lock()
c.wants["hand"] = meshwants.WantsByWantNodeModuleValue{Fp: next}
c.mu.Unlock()
if named, ok := c.wantedBuilds()[old]; !ok || named {
t.Errorf("the running build %s is asked of no want yet kept: %v, %v", old, named, ok)
}
key, _ := compositions.Key("n1", "hand", next)
c.mu.Lock()
c.handed[key] = "d"
c.mu.Unlock()
c.entryChanged(meshcatalogclient.EntryRow{Fingerprint: old, State: meshcatalogclient.EntryAnswered,
Entry: &meshcatalog.CatalogEntry{Module: "hand", Fp: old}})
c.mu.Lock()
_, still := c.handed[key]
c.mu.Unlock()
if still {
t.Error("an entry of the running build was answered, and the composition was not handed again")
}
// A-461, A-461a, A-461b: another node's module whose running build has no entry is laid from the entry of the build the assignment
// places for it on that node, or, where the placement names none, of its trunk build; with neither entry it stays as its name
// alone, and this node's own module is never laid from a placement.
placedFp := meshcatalogclient.SourcePrefix2 + strings.Repeat("3", 64)
for _, tc := range []struct {
name string
node string
build, trunk string
holds, laid bool
}{
{"placed build has an entry", "n2", placedFp, "", true, true},
{"neither build has an entry", "n2", placedFp, "", false, false},
{"placement names no build, trunk build has an entry", "n2", "", placedFp, true, true},
{"placement names no build, trunk names none", "n2", "", "", false, false},
{"this node's own module is not laid from a placement", "n1", placedFp, "", true, false},
} {
t.Run(tc.name, func(t *testing.T) {
env := newFakeBus().env()
env.Node = "n1"
env.Entry = EntryCall(func(_ context.Context, _, verb string, _ any) (json.RawMessage, error) {
if verb == meshcatalog.VerbModule {
return json.Marshal(meshcatalog.ModuleAnswer{Module: meshcatalog.ModuleAnswerModule{Fp: tc.trunk}})
}
return nil, errors.New("not held")
})
c := New(env)
c.mu.Lock()
c.assignments[tc.node] = meshmoduleassigner.Assignments{Node: tc.node, Modules: []meshmoduleassigner.PlacedModule{{Module: "distribution", Build: tc.build}}}
c.heard = map[string]map[string]heardRun{tc.node: {"distribution": {fingerprint: old}}}
c.mu.Unlock()
if tc.holds {
c.holdEntry(t, "distribution", placedFp, `{"module":"distribution","version":"2"}`)
}
c.askTrunkBuilds()
if _, asked := c.wantedBuilds()[placedFp]; asked != (tc.node == "n2" && (tc.build != "" || tc.trunk != "")) {
t.Errorf("the placed build asked = %v", asked)
}
c.layModules()
c.mu.Lock()
m, ok := c.modules["distribution"]
raw := c.manifests[m.Manifest]
c.mu.Unlock()
if got := ok && strings.Contains(string(raw), `"version":"2"`); got != tc.laid {
t.Errorf("laid from the placed build = %v, want %v (%+v %s)", got, tc.laid, m, raw)
}
})
}
}
+13 -2
View File
@@ -16,9 +16,20 @@ func (c *Composer) KeptBuildOf(_ context.Context, module, digest string) (node,
if !ok || w.Fp == "" || w.KeptOn == "" { if !ok || w.Fp == "" || w.KeptOn == "" {
return "", "", false, nil return "", "", false, nil
} }
if len(w.Parts) > 0 { parts := w.Parts
if len(parts) == 0 && w.Version != "" {
// A want naming its version names its digests through the registry seats (A-430a): the resolution the composing made.
c.mu.Lock()
r := c.resolved[resolvedKey{module, w.Fp, w.Version}]
c.mu.Unlock()
if r.parts == nil {
return "", "", false, nil
}
parts = r.parts
}
if len(parts) > 0 {
named := false named := false
for _, d := range w.Parts { for _, d := range parts {
if strings.TrimPrefix(d, "sha256:") == strings.TrimPrefix(digest, "sha256:") { if strings.TrimPrefix(d, "sha256:") == strings.TrimPrefix(digest, "sha256:") {
named = true named = true
} }
+19 -5
View File
@@ -136,6 +136,9 @@ type Env struct {
// Set, the composer reads the catalog half by the fingerprints its wants and the runnings it heard name and nothing else of // Set, the composer reads the catalog half by the fingerprints its wants and the runnings it heard name and nothing else of
// the indexer's or the builder's states (design 47 G §421, §628); a composer is made with it. // the indexer's or the builder's states (design 47 G §421, §628); a composer is made with it.
Entry meshcatalogclient.Call Entry meshcatalogclient.Call
// Registry asks the registry seats' verb versions (oci-artifact-registry, container-image-registry) in the engine's own
// name: a want naming its version is resolved to its artifacts' digests through it (design 47 G A-430a).
Registry meshcatalogclient.Call
} }
// ManifestOf is the digest of the manifest of the build a module runs at fp, from the composer's picture of the catalog half; // ManifestOf is the digest of the manifest of the build a module runs at fp, from the composer's picture of the catalog half;
@@ -193,7 +196,9 @@ type Composer struct {
entries *meshcatalogclient.Entries entries *meshcatalogclient.Entries
keptEnt map[string]keptEntry keptEnt map[string]keptEntry
// notFound is each fingerprint whose entry-not-found the composer told, so it is told once and cleared once. // notFound is each fingerprint whose entry-not-found the composer told, so it is told once and cleared once.
notFound map[string]bool notFound map[string]bool
// resolved is each wanted build's digests as its version resolved them through the registry seats (versions.go, A-430a).
resolved map[resolvedKey]versionParts
keptWantVal map[string]keptWant keptWantVal map[string]keptWant
keptWants map[string]bool keptWants map[string]bool
keptWrite sync.Mutex keptWrite sync.Mutex
@@ -204,8 +209,10 @@ type Composer struct {
addedFps map[string]bool addedFps map[string]bool
// heard is, by node and module, what each node's event running last said it runs, kept from the seat's stream by this // heard is, by node and module, what each node's event running last said it runs, kept from the seat's stream by this
// node's own consumer (NoteRunning), under mu; heardRead is whether the first read of the stream ended (RunningRead). // node's own consumer (NoteRunning), under mu; heardRead is whether the first read of the stream ended (RunningRead).
heard map[string]map[string]heardRun heard map[string]map[string]heardRun
heardRead bool // trunkBuilds is, by module, the trunk build asked for another node's module whose placement names none, with the running build it was asked for (A-461b).
trunkBuilds map[string]trunkBuild
heardRead bool
// ends is, by module, the last build end naming no branch the builder said of it (NoteBuildEnded), its newest build that // ends is, by module, the last build end naming no branch the builder said of it (NoteBuildEnded), its newest build that
// ended, under mu; endsRead is whether the first read of the builder's stream ended (BuildEndsRead) (design 47 G A-293). // ended, under mu; endsRead is whether the first read of the builder's stream ended (BuildEndsRead) (design 47 G A-293).
ends map[string]builds.BuildEnded ends map[string]builds.BuildEnded
@@ -718,7 +725,9 @@ func (c *Composer) entry(bucket, key string) (v any, has, known bool) {
defer c.mu.Unlock() defer c.mu.Unlock()
switch bucket { switch bucket {
case buckets.Wants: case buckets.Wants:
// A want is composed by its fingerprint alone (fingerprints); its words, rule and newest change nothing composed. // A want is composed by its fingerprint, version, parts and keeping node (design 47 G §525); its words, rule and newest
// change nothing composed. Judged by its fingerprint alone, a want that gained its parts at the same fingerprint asked no
// pass, and the agent waited for a composition never made (lead-47 2026-11-16, the indexer on novox).
module := agent.NodeModule module := agent.NodeModule
if _, ok := meshwants.ParseWantsByWantNodeNodeKey(key); !ok { if _, ok := meshwants.ParseWantsByWantNodeNodeKey(key); !ok {
node, m, ok := meshwants.ParseWantsByWantNodeModuleKey(key) node, m, ok := meshwants.ParseWantsByWantNodeModuleKey(key)
@@ -728,7 +737,10 @@ func (c *Composer) entry(bucket, key string) (v any, has, known bool) {
module = m module = m
} }
w, ok := c.wants[module] w, ok := c.wants[module]
return w.Fp, ok, true return struct {
Fp, Version, KeptOn string
Parts map[string]string
}{w.Fp, w.Version, w.KeptOn, w.Parts}, ok, true
case buckets.Assignments: case buckets.Assignments:
// Any key of a node, its node key, a module key or its legacy key, stands for the node's whole assignment: a // Any key of a node, its node key, a module key or its legacy key, stands for the node's whole assignment: a
// module key written before its node key names it changes nothing composed. // module key written before its node key names it changes nothing composed.
@@ -2301,6 +2313,8 @@ func (c *Composer) WhyUnmade(module string) string {
return c.last.Unmade[module] return c.last.Unmade[module]
} }
type trunkBuild struct{ running, fp string }
// heardRun is one running event as the composer keeps it, with when it was said. // heardRun is one running event as the composer keeps it, with when it was said.
type heardRun struct { type heardRun struct {
at time.Time at time.Time
+55
View File
@@ -96,6 +96,58 @@ func (b *fakeBus) put(t *testing.T, bucket, key string, value []byte) {
b.send(bucket, agent.Update{Key: key, Value: value}) b.send(bucket, agent.Update{Key: key, Value: value})
} }
// Design 47 G A-461c: the golden node grant (called from the test of what the node reads under it) is checked against the node-engine's manifest. Every seat verb it invokes has its
// call subject in the grant, in the form the grants part writes: a grant to call a seat's verb is `mesh.seat.<seat>.tool.<verb>`
// and `.tool.<verb>.*` (mesh-grants internal/buscompose nats.go:1960-1969, invokedSubjects), and each is also written under the
// `call` kind with the caller's own token last (callers.go:75-91, calledPublish; the token is the user with each dot a `~`):
// `mesh.seat.<seat>.call.<verb>.node~<node>` and `mesh.seat.<seat>.call.<verb>.*.node~<node>`. A verb of a seat held per node
// may be granted for the one node, `.call.<verb>.<node>.node~<node>`, which is the same grant in the node's own form. A verb the
// composer asks that the golden grant lacks is refused in the real bus and hangs the pass to its bound (the verb module, #233).
func checkGrantHoldsEveryInvokedVerb(t *testing.T) {
t.Helper()
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
var m struct {
Invokes []json.RawMessage `json:"invokes"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
p := nodegrant.For("it1")
seen := 0
for _, r := range m.Invokes {
var verb string
if json.Unmarshal(r, &verb) != nil {
var o struct {
Verb string `json:"verb"`
}
if err := json.Unmarshal(r, &o); err != nil {
t.Fatal(err)
}
verb = o.Verb
}
rest, isSeat := strings.CutPrefix(verb, "seat:")
if !isSeat {
continue
}
seat, name, ok := strings.Cut(rest, ".")
if !ok {
t.Fatalf("the invoke %q names no verb", verb)
}
seen++
base := "mesh.seat." + seat + ".call." + name + "."
forms := []string{base + "node~it1", base + "*.node~it1", base + "it1.node~it1"}
if !slices.ContainsFunc(forms, func(f string) bool { return slices.Contains(p.Publish, f) }) {
t.Errorf("the node's grant holds no call subject for %s: want one of %v", verb, forms)
}
}
if seen == 0 {
t.Fatal("the manifest invokes no seat verb: the test reads nothing")
}
}
// grantAllows says whether the node's grant (the grants part's, nodegrant.For) lets it watch or read pattern of // grantAllows says whether the node's grant (the grants part's, nodegrant.For) lets it watch or read pattern of
// bucket: the consumer a watch or a get makes (`$JS.API.CONSUMER.CREATE.KV_<bucket>.<name>.$KV.<bucket>.<pattern>`) // bucket: the consumer a watch or a get makes (`$JS.API.CONSUMER.CREATE.KV_<bucket>.<name>.$KV.<bucket>.<pattern>`)
// and the stream's information, matched as the bus matches a permission against a subject. // and the stream's information, matched as the bus matches a permission against a subject.
@@ -158,6 +210,8 @@ func quiet(t *testing.T, out <-chan agent.Update) {
// names any more is taken back; a node that cannot compose says so once and hands nothing. // names any more is taken back; a node that cannot compose says so once and hands nothing.
func TestTheNodeComposesItsWantsFromTheBus(t *testing.T) { func TestTheNodeComposesItsWantsFromTheBus(t *testing.T) {
t.Run("it reads the catalog half by the fingerprints its wants name", composeFromEntries) t.Run("it reads the catalog half by the fingerprints its wants name", composeFromEntries)
t.Run("a running module without an entry stands as the newest build", runningModuleWithoutEntryStandsAsTheNewestBuild)
t.Run("a want naming its version is resolved through the registry seats", aWantNamingItsVersionIsResolvedThroughTheRegistrySeats)
b := newFakeBus() b := newFakeBus()
fp := meshcatalogclient.SourcePrefix2 + strings.Repeat("a", 64) fp := meshcatalogclient.SourcePrefix2 + strings.Repeat("a", 64)
digest := meshcatalogclient.ManifestDigest([]byte(lemurs)) digest := meshcatalogclient.ManifestDigest([]byte(lemurs))
@@ -1551,6 +1605,7 @@ func TestTheNodeComposesItsWantsFromTheBus(t *testing.T) {
if got, err := r.Served(ctx, "n2", "mailer", []string{"domain"}); err != nil || fmt.Sprint(got) != "map[domain:example.org]" { if got, err := r.Served(ctx, "n2", "mailer", []string{"domain"}); err != nil || fmt.Sprint(got) != "map[domain:example.org]" {
t.Errorf("what n2's mailer serves: %v, %v", got, err) t.Errorf("what n2's mailer serves: %v, %v", got, err)
} }
checkGrantHoldsEveryInvokedVerb(t) // A-461c
served := nodegrant.For("n1", "n2.mailer.domain") served := nodegrant.For("n1", "n2.mailer.domain")
b.mu.Lock() b.mu.Lock()
for _, read := range b.reads { for _, read := range b.reads {
+1 -1
View File
@@ -630,7 +630,7 @@ func (s *Shadow) answerProposal(ctx context.Context, proposal string) (nodeengin
} }
// The module is built from its entry and the parts the candidate gives (A-316); an artifact it gives no part for holds it. // The module is built from its entry and the parts the candidate gives (A-316); an artifact it gives no part for holds it.
w := meshwants.WantsByWantNodeModuleValue{Fp: fp, Parts: own.c.Added[pm.Module].Parts, Version: own.c.Added[pm.Module].Version} w := meshwants.WantsByWantNodeModuleValue{Fp: fp, Parts: own.c.Added[pm.Module].Parts, Version: own.c.Added[pm.Module].Version}
if missing := partsMissing(entry, w); missing != "" { if _, missing := c.partsOf(pm.Module, entry, w); missing != "" {
heldAdded[pm.Module] = fmt.Sprintf("the candidate gives no part for %s of %s's build %s, so the node composes nothing new of it", missing, pm.Module, fp) heldAdded[pm.Module] = fmt.Sprintf("the candidate gives no part for %s of %s's build %s, so the node composes nothing new of it", missing, pm.Module, fp)
continue continue
} }
+217
View File
@@ -0,0 +1,217 @@
package selfcompose
import (
"encoding/json"
"fmt"
"sort"
"strings"
"time"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/containerimageregistry"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/ociartifactregistry"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// **A want names its version; the node-engine resolves the digests** (design 47 G A-430, A-430a, the lead's answer A-430b).
// A want of a released build names the module's version and no parts: each artifact of the build's entry is looked up at that
// version through the verb `versions` of the registry seat of its kind (oci-artifact-registry for a bundle or an archive,
// container-image-registry for an image), under the name the builder numbered it by: `<module>/<artifact>`, or the entry's
// package name for a package. An upstream copy is not numbered (it is mirrored by its pinned digest): its digest is the one its
// manifest's `from` pins. A want that names parts (a build no version is known of) is read by its parts as before. A lookup that
// fails is said in the journal, naming the module, the version and the seat, and the module is held whole: never composed partly.
// resolveAgain is how long a failed lookup stands before it is asked again; a pass is asked then.
const resolveAgain = time.Minute
// versionsVerb is the verb a lookup asks.
const versionsVerb = "versions"
// resolvedKey names one resolution: a module's build at a version.
type resolvedKey struct{ module, fp, version string }
// versionParts is a module's build at a version as resolved: its digests by artifact reference, or why not, and when that was found.
type versionParts struct {
parts map[string]string
failed string
at time.Time
}
// registryVersions is the answer of either seat's verb `versions`: the two seats answer the same shape.
type registryVersions struct {
Versions []struct {
Digest string `json:"digest"`
Tag string `json:"tag,omitempty"`
Version string `json:"version,omitempty"`
} `json:"versions"`
}
// seatOfKind is the registry seat an artifact of kind is numbered in, and whether one is (design 51 *The seats*).
func seatOfKind(kind string) (string, bool) {
switch kind {
case catalogue.ArtifactImage:
return containerimageregistry.Seat, true
case catalogue.ArtifactBundle, catalogue.ArtifactArchive:
return ociartifactregistry.Seat, true
}
return "", false
}
// registryName is the name an artifact of module is numbered under: `<module>/<artifact>`, or a package's name in its
// ecosystem as the entry gives it.
func registryName(module string, a meshcatalog.Artifact) string {
_, artifact, _ := strings.Cut(a.Ref, ":")
if a.Kind == catalogue.ArtifactPackage && a.Package != nil && a.Package.Name != "" {
return a.Package.Name
}
return module + "/" + artifact
}
// partsOf is the digest of each artifact of e for want w, by artifact reference, and the artifacts it names no digest for ("" when
// each has one). A want with parts gives them; a want with a version and no parts is resolved through the registry seats.
func (c *Composer) partsOf(module string, e meshcatalog.CatalogEntry, w meshwants.WantsByWantNodeModuleValue) (map[string]string, string) {
parts := w.Parts
if len(parts) == 0 && w.Version != "" && len(e.Artifacts) > 0 {
var why string
if parts, why = c.resolve(module, e, w); why != "" {
return nil, why
}
}
return parts, partsMissing(e, parts)
}
// partsMissing names the artifacts of e whose digest parts does not give; "" when each has one.
func partsMissing(e meshcatalog.CatalogEntry, parts map[string]string) string {
var out []string
for _, a := range e.Artifacts {
if parts[a.Ref] == "" {
out = append(out, a.Ref)
}
}
sort.Strings(out)
return strings.Join(out, ", ")
}
// resolve is the digests of module's build at w's version, from what is resolved already or asked of the seats now. why names
// what failed; a failure stands resolveAgain, then is asked again.
func (c *Composer) resolve(module string, e meshcatalog.CatalogEntry, w meshwants.WantsByWantNodeModuleValue) (map[string]string, string) {
key := resolvedKey{module, w.Fp, w.Version}
now := c.now()
c.mu.Lock()
r, ok := c.resolved[key]
c.mu.Unlock()
if ok && (r.failed == "" || now.Sub(r.at) < resolveAgain) {
return r.parts, r.failed
}
parts, failed := c.lookUp(module, e, w)
c.mu.Lock()
if c.resolved == nil {
c.resolved = map[resolvedKey]versionParts{}
}
// Only the resolutions of the builds a want names now are kept.
for k := range c.resolved {
if cur, wanted := c.wants[k.module]; k.module != module && (!wanted || cur.Fp != k.fp || cur.Version != k.version) {
delete(c.resolved, k)
}
}
c.resolved[key] = versionParts{parts: parts, failed: failed, at: now}
c.mu.Unlock()
if failed != "" {
time.AfterFunc(resolveAgain, c.askRehand)
return nil, failed
}
return parts, ""
}
// lookUp asks each artifact's digest at w's version of the seat of its kind. Every failure is said, naming the module, the version
// and the seat; any failure resolves nothing of the build.
func (c *Composer) lookUp(module string, e meshcatalog.CatalogEntry, w meshwants.WantsByWantNodeModuleValue) (map[string]string, string) {
parts := map[string]string{}
var failed []string
fail := func(seat, ref, why string) {
line := fmt.Sprintf("%s at version %s: the seat %s's verb versions resolves no digest of %s: %s, so the node holds %s and composes nothing of it",
module, w.Version, seat, ref, why, module)
c.sayOnceEach(line)
failed = append(failed, fmt.Sprintf("%s (seat %s: %s)", ref, seat, why))
}
var manifest *catalogue.Manifest
for _, a := range e.Artifacts {
_, artifact, _ := strings.Cut(a.Ref, ":")
if a.Kind == catalogue.ArtifactUpstream {
if manifest == nil {
_, raw, err := entryManifest(e)
m, derr := catalogue.Decode(raw)
if err != nil || derr != nil {
fail("(none)", a.Ref, "the entry's manifest cannot be read for the digest its upstream copy pins")
continue
}
manifest = &m
}
if d := pinnedDigest(*manifest, artifact); d != "" {
parts[a.Ref] = d
} else {
fail("(none)", a.Ref, "the manifest pins no digest of the upstream copy")
}
continue
}
seat, numbered := seatOfKind(a.Kind)
if !numbered {
fail("(none)", a.Ref, "no registry seat the node-engine reads numbers an artifact of kind "+a.Kind)
continue
}
digest, why := c.versionDigest(seat, registryName(module, a), w)
if why != "" {
fail(seat, a.Ref, why)
continue
}
parts[a.Ref] = digest
}
if len(failed) > 0 {
sort.Strings(failed)
return nil, "no digest is resolved at version " + w.Version + " for " + strings.Join(failed, ", ")
}
return parts, ""
}
// versionDigest is the digest the seat's verb versions answers for name at w's version (its version or its tag), the newest
// first: a version names one build (A-430a), so the version alone is matched.
func (c *Composer) versionDigest(seat, name string, w meshwants.WantsByWantNodeModuleValue) (string, string) {
if c.env.Registry == nil {
return "", "this node-engine reads no registry seat"
}
raw, err := c.env.Registry("seat:"+seat+"."+versionsVerb, map[string]string{"name": name})
if err != nil {
return "", fmt.Sprintf("the call for %s failed: %v", name, err)
}
answer, said, ok := buscalls.Unwrap(raw)
if !ok {
return "", fmt.Sprintf("the seat refused %s: %s", name, said)
}
var v registryVersions
if err := json.Unmarshal(answer, &v); err != nil {
return "", fmt.Sprintf("its answer for %s cannot be read: %v", name, err)
}
for _, a := range v.Versions {
if (a.Version == w.Version || a.Tag == w.Version) && a.Digest != "" {
return a.Digest, ""
}
}
return "", fmt.Sprintf("it holds no version %s of %s", w.Version, name)
}
// pinnedDigest is the digest the manifest's upstream artifact pins in its from (`<image>@sha256:<hex>`); "" when none.
func pinnedDigest(m catalogue.Manifest, artifact string) string {
if m.Build == nil {
return ""
}
for _, a := range m.Build.Artifacts {
if a.Name == artifact {
if _, d, ok := strings.Cut(a.From, "@"); ok && strings.HasPrefix(d, "sha256:") {
return d
}
}
}
return ""
}
+123
View File
@@ -0,0 +1,123 @@
package selfcompose
import (
"context"
"encoding/json"
"errors"
"strings"
"sync"
"testing"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
)
// Design 47 G A-430, A-430a (the lead's answer A-430b): a want naming its version and no parts is resolved to its artifacts'
// digests through the verb versions of the registry seat of each artifact's kind, under `<module>/<artifact>`; an upstream copy
// by the digest its manifest pins. The build is made from those digests. A lookup that fails is said, naming the module, the
// version and the seat, and holds the module whole; a want with parts is read by its parts as before.
func aWantNamingItsVersionIsResolvedThroughTheRegistrySeats(t *testing.T) {
fp := meshcatalogclient.SourcePrefix2 + strings.Repeat("a", 64)
d := func(c string) string { return "sha256:" + strings.Repeat(c, 64) }
manifest := map[string]any{"module": "mail", "version": "1", "build": map[string]any{"artifacts": []any{
map[string]any{"name": "server", "kind": "upstream", "from": "alpine@" + d("9")},
map[string]any{"name": "files", "kind": "archive", "from": "files"},
map[string]any{"name": "web", "kind": "image", "from": "Dockerfile"},
}}}
entry := meshcatalog.CatalogEntry{Module: "mail", Fp: fp, Manifest: manifest, Artifacts: []meshcatalog.Artifact{
{Ref: "mail:server", Kind: "upstream"}, {Ref: "mail:files", Kind: "archive"}, {Ref: "mail:web", Kind: "image"}}}
// The seats hold mail/files and mail/web; versions newest first.
held := map[string]string{
"oci-artifact-registry.mail/files": `{"name":"mail/files","versions":[{"name":"mail/files","kind":"archive","digest":"` + d("2") + `","version":"1.3.0"},{"name":"mail/files","kind":"archive","digest":"` + d("1") + `","version":"1.2.3"}]}`,
"container-image-registry.mail/web": `{"name":"mail/web","versions":[{"name":"mail/web","kind":"image","digest":"` + d("3") + `","version":"1.2.3"}]}`,
"container-image-registry.mail/other": `{"name":"mail/other","versions":[]}`,
}
for name, row := range map[string]struct {
want meshwants.WantsByWantNodeModuleValue
down string // a seat whose call fails
parts map[string]string
missing string // a word the reason holds
said []string
}{
"resolved": {want: meshwants.WantsByWantNodeModuleValue{Fp: fp, Version: "1.2.3", Self: true},
parts: map[string]string{"mail:server": d("9"), "mail:files": d("1"), "mail:web": d("3")}},
"a version a seat does not hold": {want: meshwants.WantsByWantNodeModuleValue{Fp: fp, Version: "2.0.0", Self: true},
missing: "2.0.0", said: []string{"mail at version 2.0.0", "oci-artifact-registry", "container-image-registry"}},
"a seat that does not answer": {want: meshwants.WantsByWantNodeModuleValue{Fp: fp, Version: "1.2.3", Self: true}, down: "container-image-registry",
missing: "mail:web", said: []string{"mail at version 1.2.3: the seat container-image-registry"}},
"parts as before": {want: meshwants.WantsByWantNodeModuleValue{Fp: fp, Version: "1.2.3", Self: true,
Parts: map[string]string{"mail:server": d("9"), "mail:files": d("5"), "mail:web": d("6")}},
parts: map[string]string{"mail:server": d("9"), "mail:files": d("5"), "mail:web": d("6")}},
} {
var mu sync.Mutex
var said []string
env := Env{Say: func(line string) { mu.Lock(); said = append(said, line); mu.Unlock() }}
calls := 0
env.Registry = EntryCall(func(_ context.Context, seat, verb string, args any) (json.RawMessage, error) {
calls++
if verb != "versions" {
t.Errorf("%s: asked %s.%s", name, seat, verb)
}
if seat == row.down {
return nil, errors.New("no responders")
}
answer, ok := held[seat+"."+args.(map[string]string)["name"]]
if !ok {
return nil, errors.New("not held")
}
return json.RawMessage(answer), nil
})
env.Entry = EntryCall(func(context.Context, string, string, any) (json.RawMessage, error) {
return nil, errors.New("not asked")
})
c := New(env)
c.wants["mail"] = row.want
c.keptEnt[fp] = keptEntry{entry: entry}
parts, missing := c.partsOf("mail", entry, row.want)
b := c.buildOf("mail", fp)
if row.missing != "" {
if parts != nil || b != nil || !strings.Contains(missing, row.missing) {
t.Errorf("%s: parts %v, build %v, missing %q: want the module held whole, naming %s", name, parts, b, missing, row.missing)
}
if c.entryHolds()["mail"] == "" {
t.Errorf("%s: the module is not held", name)
}
for _, w := range row.said {
if !strings.Contains(strings.Join(said, "\n"), w) {
t.Errorf("%s: said %q, want a line naming %q", name, said, w)
}
}
continue
}
if missing != "" || len(parts) != len(row.parts) {
t.Fatalf("%s: parts %v, missing %q", name, parts, missing)
}
for ref, digest := range row.parts {
if parts[ref] != digest {
t.Errorf("%s: %s at %s, want %s", name, ref, parts[ref], digest)
}
}
if b == nil || len(b.Artifacts) != 3 {
t.Fatalf("%s: build %+v", name, b)
}
for _, a := range b.Artifacts {
if a.Digest != row.parts["mail:"+a.Name] || a.Reference != referenceOf("mail", a.Name, a.Digest) {
t.Errorf("%s: artifact %+v", name, a)
}
}
// A resolution is asked once, and it names the node keeping the build its digests are of.
before := calls
c.partsOf("mail", entry, row.want)
if calls != before {
t.Errorf("%s: resolved again: %d calls, then %d", name, before, calls)
}
c.wants["mail"] = meshwants.WantsByWantNodeModuleValue{Fp: fp, Version: row.want.Version, Parts: row.want.Parts, KeptOn: "g14", Self: true}
if node, _, kept, _ := c.KeptBuildOf(context.Background(), "mail", row.parts["mail:files"]); !kept || node != "g14" {
t.Errorf("%s: kept on %q %v, want g14", name, node, kept)
}
if _, _, kept, _ := c.KeptBuildOf(context.Background(), "mail", d("7")); kept {
t.Errorf("%s: a digest the want does not resolve to is said kept", name)
}
}
}
+3 -1
View File
@@ -40,6 +40,7 @@
} }
], ],
"invokes": [ "invokes": [
"seat:container-image-registry.versions",
"seat:mesh-catalog.entry", "seat:mesh-catalog.entry",
"seat:mesh-catalog.module", "seat:mesh-catalog.module",
{"verb": "seat:mesh-catalog.revision", "fields": ["none-yet"]}, {"verb": "seat:mesh-catalog.revision", "fields": ["none-yet"]},
@@ -48,7 +49,8 @@
"seat:mesh-module-assigner.assignments", "seat:mesh-module-assigner.assignments",
"seat:mesh-recovery.kept-points", "seat:mesh-recovery.kept-points",
"seat:mesh-recovery.restore-kept", "seat:mesh-recovery.restore-kept",
"seat:mesh-recovery.snapshot-kept" "seat:mesh-recovery.snapshot-kept",
"seat:oci-artifact-registry.versions"
], ],
"consumes": [ "consumes": [
"mesh-catalog.seat-definition-changed", "mesh-catalog.seat-definition-changed",