Compare commits

..
Author SHA1 Message Date
jschoubben 3ae3519714 The assigner pins mesh-sdk at c77d17f: a follower joining a watched state loses no change
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:52:33 +01:00
jschoubben 04aa1a1ace mesh-grants and the builder pin mesh-sdk at 8311ecb
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:36:58 +01:00
jschoubben b3a74e0bdc Merge builderA3's upgrade compare and builderA2's watched reader into group A 2026-11-01 03:35:05 +01:00
jschoubben b8d4a77e75 mesh-wants upgrade reads through mesh-sdk's watched reader, keeping no mirror of its own
One helper serves every part that reads whole states (ADR 0580 clause 3): the assigner's own watched mirror goes, its
compare reads the large states from stdio.WatchedStates and each manifest once per digest. mesh-sdk pinned at 8311ecb.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:34:21 +01:00
jschoubben 8112fe4b13 mesh-grants and the assigner read the grants part's key alone, root-free is each engine's word, and the hand-act log is the seat's alone (ADR 0571 clauses 9 and 11, ADR 0565 clause 11, issues 1953 and 1955)
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:26:59 +01:00
jschoubben b74c371ed6 The builder asks the artifact sweep's seat for held and no longer the controller (ADR 0571 clause 2, issue 1955)
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:26:59 +01:00
jschoubben f95d5b4d23 mesh-wants upgrade: read the large states by one watch each, started once, not a get per key, so compare answers in time (issue 1953 L5f)
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:22:12 +01:00
jschoubben edf7b3c0bf Answer review round 1: fresh is only what a take-in newly wrote, a merge keeps the verdict, the check compares with the trunk (ADR 0578, 0579)
mesh/delivery-group group feat/judge-once checking: 6 of 6 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking edf7b3c0bf3a on b1f262d7d06e | novox/mesh-delivery#129 in group feat/judge-once
Issue 1585, R1: fresh holds the digests a take-in newly wrote (a new module, a changed manifest); a registered or
taken-in module is judged too; a start pass that fails is judged again at the next message heard, no timer.
R2: a module's verdict, problems and warnings are carried while its digest stands and are no part of the compare.
R3: the checker judges the trunk beside the change and fails on every problem the change case adds, an untouched
manifest's included. R4: the judge reads the indexer's kept predefined keys, or the catalogue member's when it
changes them. A log line names a package no held module publishes. Every module pins mesh-sdk 62cddf4.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:12:09 +01:00
jschoubben 0e819eea4d Never refuse what the judge accepted before: warn, and refuse only a new or changed manifest (ADR 0579)
Issue 1585: the indexer keeps an accepted manifest accepted when a stricter judge re-judges it, its problems as
warnings carried in its value; a merge's new manifests are judged in full. The watchman raises manifest-outgrown as
a warning until mended. The checker fails only on a manifest the change adds or changes and names the rest as
warnings. Every module pins mesh-sdk 19c2042.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:45:18 +01:00
jschoubben f4ecc6fa64 Pin every mesh-delivery module to mesh-sdk feat/judge-once a20bdf7, the judge out and package versions in
Issue 1585: builder-1585b's package versions folded onto the judge once; one mesh-sdk head for the group.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:42:05 +01:00
jschoubben fa918318b2 Test the checker's latest package versions
The checker builds every Go check against them (issue 1585, S14); a row
in the repoModules test proves they come from the modules that hold each
package and leave out a package no module holds.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:41:28 +01:00
jschoubben 3f2303a086 Build checks against the group's package heads, not go.mod pins
ADR 0573 §4 makes refusing a member for its mesh-sdk pin a fault of the
check (issue 1585, S14). Every Go check ask now carries the latest
published version of each package, or the head of the group member that
publishes it, so the pin check, its forge reads and its tests go, and
the controller's tests read mesh-sdk at the version it is built with.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:41:22 +01:00
jschoubben ae6a54bca2 Take each mesh package's latest version into the fingerprint
ADR 0573 §2: a module builds against the latest published version of
each mesh package it uses, and a package's trunk merge moves every
dependent (issue 1585, S13). The bridge reads the packages each go.sum
names, the indexer keeps them and takes each one's trunk merge into the
fingerprint, a candidate takes a group member's head instead, and the
builder's ask carries the versions so the build module writes them.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:41:04 +01:00
jschoubben d6807233f2 Keep the reserved stream prefixes in the judge, the one part that reads them (ADR 0578 §4)
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:38:00 +01:00
jschoubben 3ae308426a Merge four of the judge's moved tests into one, so the move adds no test to the budget
Issue 1585: three tests are split between mesh-sdk (their composition half) and the judge (their judging half).

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:36:23 +01:00
jschoubben 8a98271680 Move the judge's rules and their tests from mesh-sdk into the indexer's judge (ADR 0578 §4)
Issue 1585, S6-S8: 192 judge-only declarations (the strict parser, every manifest problem function, the tables)
moved whole by an AST mover, methods becoming functions; 163 helpers composition keeps too are copied. 245 tests
move with them, reading the captured catalogue through a copy of internal/beside. judge.WithTemplatesFrom and
ReadTreeManifest judge what mesh-sdk's readers now only fill. mesh-delivery's own tests decode. Pinned to mesh-sdk 25a2cec.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:34:26 +01:00
jschoubben 0469641b9d Pin mesh-delivery's modules to mesh-sdk feat/judge-once 7cab079
Issue 1585: the group's mesh-sdk head (lenient Decode, verdict contract, frombus holding a refused module).

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:21:32 +01:00
jschoubben 4f6f38e16a Keep every strict read of a manifest in the indexer's judge, and add no test to the budget
Issue 1585, S5-S8: the indexer's fingerprint reads go through judge.ForInputs, so judge/ is the one place in
mesh-delivery that calls mesh-sdk's rules; the command's request judging moves into judge.Request, tested in the
judge's table; two predefined tests become one.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:21:32 +01:00
jschoubben 5b6602544e Prove the judge once whole: verdicts, manifest-refused, and a bus reader that holds and goes on (ADR 0578)
Issue 1585: the indexer's merge test takes in three manifests more against the real bus; it writes their
verdicts, says each refusal, and mesh-sdk's bus reader composes the valid one and holds the refused, named; before
any verdict it takes each as accepted and names the field set aside. One test extended, no test added.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:17:57 +01:00
jschoubben efef7183ef Let grants, builder and checker decode manifests, and the checker judge a change with the indexer's judge (ADR 0578 §2, §3)
Issue 1585, S12: grants, builder and checker read with catalogue.Decode and grants holds a module the indexer
refused; the checker asks the Go build seat to run mesh-judge, built at mesh-delivery's trunk or at the group's
mesh-delivery head when it changes the judge, over the catalogue as the change leaves it, and fails on a refusal.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:15:54 +01:00
jschoubben a4772a5c1d Let the assigner decode manifests and take the indexer's verdicts, never judging (ADR 0578 §3, §5, §6)
Issue 1585, S11: world and wants read with catalogue.Decode; a trunk refusal is held, named and handed to the
composer in its input; a proposal's composition read that fails is said, not dropped; a reader that cannot read a
changed field is a warning, no longer a failure.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:11:54 +01:00
jschoubben 26852f4a9d Raise a refused manifest as the watchman's finding, so the operator hears it once (ADR 0578 §1)
Issue 1585, S10: the watchman watches the indexer's modules and finds manifest-refused for a module whose value
carries a refusal, ending it once accepted. Its mesh-sdk pin moves to the group head, so it takes health.Finding's
Ref (issue 1952 renamed Token).

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:09:50 +01:00
jschoubben 82776466b2 Let the indexer write a verdict on every manifest it holds and say each refusal (ADR 0578 §1)
Issue 1585, S9: verdicts keyed by digest, written only when they differ; a module's value carries a refusal; a
new refusal is said as manifest-refused. Judged at start (a new judge judges again) and after every merge.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:08:21 +01:00
jschoubben 4837548349 Give the mesh one judge of manifests, in the indexer, and a command the checker runs (ADR 0578 §1, §2)
Issue 1585, S5: judge.Shelf judges each manifest alone and the rules between them, giving a problem between
manifests to each module whose removal ends it (naming alone blamed the module mounts for f1-circuits' problem).
cmd/mesh-judge answers a composer request so a Go build module runs it as it runs the composer.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:08:21 +01:00
537 changed files with 42277 additions and 301 deletions
@@ -167,7 +167,7 @@ func saysWhatItKeeps(t *testing.T, b *bed, done <-chan error) {
if err != nil {
t.Fatal(err)
}
m, err := catalogue.ParseManifest(raw)
m, err := catalogue.Decode(raw)
if err != nil {
t.Fatal(err)
}
@@ -407,7 +407,7 @@ func TestWantsAreWrittenAndNoDifferenceIs(t *testing.T) {
for _, fp := range []string{fNew, fOld} {
gkey, _ := delivery.ControllerGrantKey("web", fp)
grant, _ := delivery.EncodeControllerGrant(delivery.ControllerGrant{})
b.putJSON(t, delivery.ControllerGrantsBucket, gkey, grant)
b.putJSON(t, delivery.PartGrantsBucket, gkey, grant)
}
runs := map[string]string{"ace": fOld, "g14": "build-9", "shanks": fOld}
for _, node := range []string{"ace", "g14", "shanks"} {
@@ -617,7 +617,7 @@ func TestTheAssignerReplacedResumesWithNothingLostOrRepeated(t *testing.T) {
b.putJSON(t, delivery.BuilderBuildsBucket, bkey, built)
gkey, _ := delivery.ControllerGrantKey(name, fp)
grant, _ := delivery.EncodeControllerGrant(delivery.ControllerGrant{})
b.putJSON(t, delivery.ControllerGrantsBucket, gkey, grant)
b.putJSON(t, delivery.PartGrantsBucket, gkey, grant)
b.compose(t, "ace", name, fp, 'c', delivery.JudgedNeither)
b.putJSON(t, provider.AssignmentsBucket, "ace."+name, []byte(fmt.Sprintf(`{"node":"ace","module":%q,"build":"b-%d"}`, name, i)))
}
@@ -663,7 +663,7 @@ func TestAWantWaitsForItsComposition(t *testing.T) {
b.putJSON(t, delivery.BuilderBuildsBucket, bkey, built)
gkey, _ := delivery.ControllerGrantKey("web", fNew)
grant, _ := delivery.EncodeControllerGrant(delivery.ControllerGrant{})
b.putJSON(t, delivery.ControllerGrantsBucket, gkey, grant)
b.putJSON(t, delivery.PartGrantsBucket, gkey, grant)
on, _ := delivery.EncodePresence(delivery.Presence{Online: true})
b.putJSON(t, delivery.PresenceBucket, "ace", on)
b.putJSON(t, provider.AssignmentsBucket, "ace.web", []byte(`{"node":"ace","module":"web","build":"b-1"}`))
@@ -712,7 +712,7 @@ func (b *bed) fixture(t *testing.T, name, fp, manifest string) {
b.putJSON(t, delivery.BuilderBuildsBucket, bkey, built)
gkey, _ := delivery.ControllerGrantKey(name, fp)
grant, _ := delivery.EncodeControllerGrant(delivery.ControllerGrant{})
b.putJSON(t, delivery.ControllerGrantsBucket, gkey, grant)
b.putJSON(t, delivery.PartGrantsBucket, gkey, grant)
on, _ := delivery.EncodePresence(delivery.Presence{Online: true})
b.putJSON(t, delivery.PresenceBucket, "ace", on)
b.putJSON(t, provider.AssignmentsBucket, "ace."+name, []byte(`{"node":"ace","module":"`+name+`","build":"b-1"}`))
@@ -59,7 +59,6 @@ func main() {
var readBuckets = []string{
buscompose.ResolvedBucket,
provider.AssignmentsBucket,
delivery.ControllerGrantsBucket,
delivery.PartGrantsBucket,
buscompose.LeaseBucket,
delivery.ControllerComposedBucket,
@@ -219,8 +218,8 @@ func startsWithout(name string) bool {
return slices.Contains(seatBuckets, name) || sourceOfGrants(name)
}
// sourceOfGrants says the bucket is one of the two that choose the grants' source, the grants part's grants and the
// controller's lease: neither stops the assigner when it is not there (delivery.GrantSource).
// sourceOfGrants says the bucket is the grants part's grants or the controller's lease: neither stops the assigner when it
// is not there.
func sourceOfGrants(name string) bool {
return name == delivery.PartGrantsBucket || name == buscompose.LeaseBucket
}
@@ -333,7 +332,6 @@ func (e emitting) Emit(eventType string, body any) error { return e.Conn.Emit(ev
var watched = []string{
buscompose.ResolvedBucket,
provider.AssignmentsBucket,
delivery.ControllerGrantsBucket,
delivery.PartGrantsBucket,
buscompose.LeaseBucket,
delivery.ControllerComposedBucket,
@@ -419,11 +417,10 @@ func follow(ctx context.Context, js jetstream.JetStream, nc process.Subscriber,
if e.Operation() == jetstream.KeyValuePut {
value = e.Value()
}
before := w.grantsBucket()
before := w.handedAway()
w.putEntry(name, e.Key(), value, e.Revision(), e.Created())
if name == buscompose.LeaseBucket && w.grantsBucket() == before {
// The controller renews its lease every few seconds: only a change of where grants are read from
// makes a pass (delivery.GrantSource).
if name == buscompose.LeaseBucket && w.handedAway() == before {
// The controller renews its lease every few seconds: only the lease's being handed away makes a pass.
continue
}
wake()
@@ -36,7 +36,7 @@ func TestOpensWhatItsManifestNames(t *testing.T) {
if err != nil {
t.Fatal(err)
}
m, err := catalogue.ParseManifest(raw)
m, err := catalogue.Decode(raw)
if err != nil {
t.Fatal(err)
}
@@ -272,7 +272,7 @@ func putOfTheBus(t *testing.T) {
world(delivery.ControllerComposedBucket, ckey, cd)
gkey, _ := delivery.ControllerGrantKey("nats", fp)
g, _ := delivery.EncodeControllerGrant(delivery.ControllerGrant{})
world(delivery.ControllerGrantsBucket, gkey, g)
world(delivery.PartGrantsBucket, gkey, g)
got := step()
akey, _ := delivery.RecordAskKey("novox", "nats", digest)
if got.Step != delivery.PutPerson || got.Ask != akey {
@@ -664,12 +664,25 @@ func (p *passer) compute(prop delivery.PlacementProposal, current map[string]del
}
get := kvGetter(p.kvs[delivery.ManifestsBucket])
r := worldReader{w: p.w, manifests: get}
// A read that fails is said, never dropped silently (novox/hq ADR 0578 §3); a refused manifest no longer fails it, and
// its module is held, named in the input's Refused.
var now, with *compose.Input
if in, err := frombus.Input(context.Background(), r, frombus.Mesh{}, frombus.Scope{Every: true}); err == nil {
now = &in
} else if p.logf != nil {
p.logf("proposal %s: the current composition input cannot be read, so what it changes is judged without it: %v", prop.ID, err)
}
if in, err := frombus.Input(context.Background(), r, frombus.Mesh{}, frombus.Scope{Every: true, Placements: laid}); err == nil {
with = &in
} else if p.logf != nil {
p.logf("proposal %s: the composition input with the proposal cannot be read, so what it changes is judged without it: %v", prop.ID, err)
}
if now != nil && len(now.Refused) > 0 && p.logf != nil {
var held []string
for _, m := range slices.Sorted(maps.Keys(now.Refused)) {
held = append(held, m+" ("+now.Refused[m]+")")
}
p.logf("proposal %s: held as refused by the indexer's judge: %s", prop.ID, strings.Join(held, "; "))
}
nodes := map[string]bool{}
for n := range current {
@@ -6,6 +6,7 @@ import (
"errors"
"fmt"
"os"
"reflect"
"slices"
"strings"
"testing"
@@ -15,6 +16,7 @@ import (
"git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/health"
"git.novox.be/novox/mesh-sdk/go/provisions"
"github.com/nats-io/nats.go/jetstream"
@@ -400,6 +402,47 @@ func wantsVerbs(t *testing.T) {
if splits := out.(map[string]any)["splits"].([]assign.Split); len(splits) != 1 || splits[0].Module != "web" || out.(map[string]any)["findings"] == nil {
t.Errorf("splits = %+v", splits)
}
// Read by watch (issue 1953 L5f): the same answer, each watched state started once, and no get per key of the
// large states (the modules, their manifests, the composed record).
watches := map[string]int{}
byWatch := newWantsTools(st, nil, nil)
byWatch.watched = stdio.WatchedStatesOver(func(state string, handle func(stdio.StateChange) error) error {
watches[state]++
for k, v := range st[state].values {
_ = handle(stdio.StateChange{Key: k, Op: stdio.OpPut, Value: v, Current: true})
}
return nil
}, func(state, key string) ([]byte, bool, error) {
watches[state+" by key"]++
v, ok := st[state].values[key]
return v, ok, nil
})
for range 2 {
wout, werr := byWatch.doUpgrade(map[string]any{"compare": "true"}, "op")
if werr != nil {
t.Fatal(werr)
}
if ws := wout.(map[string]any)["splits"].([]assign.Split); len(ws) != 1 || ws[0].Module != "web" {
t.Errorf("splits by watch = %+v", ws)
}
}
if n := watches[manifestsState+" by key"]; n == 0 || n > len(st[manifestsState].values) {
t.Errorf("the manifests were read by key %d times, want each once", n)
}
for _, state := range []string{modulesState, composedState, health.FindingsState} {
if watches[state] != 1 {
t.Errorf("%s was watched %d times, want once", state, watches[state])
}
}
// ADR 0580 clause 3: the compare reads through mesh-sdk's watched reader and the seat keeps no mirror of its own.
for f := range reflect.TypeFor[wantsTools]().Fields() {
if f.Type.Kind() == reflect.Map || (f.Type.Kind() == reflect.Pointer && f.Type.Elem().Kind() == reflect.Map) {
t.Errorf("wantsTools keeps %s, a copy of its own beside the watched reader", f.Name)
}
}
if watches[policiesState] != 0 {
t.Errorf("the seat's own small policies state was watched: %v", watches)
}
// The process says the split as its finding, and deletes it once they agree.
findings := map[string][]byte{}
p := &passer{w: newWorld(), logf: func(string, ...any) {}, findings: assign.Findings{
@@ -415,7 +458,7 @@ func wantsVerbs(t *testing.T) {
mod, _ := delivery.DecodeModule(raw)
mraw, _, _, _ := st.get(manifestsState, mod.Manifest)
kept, _ := delivery.DecodeManifest(mraw)
man[name], _ = catalogue.ParseManifest([]byte(kept.JSON))
man[name], _ = catalogue.Decode([]byte(kept.JSON))
}
findings["module.gone.policy-split"] = []byte("{}")
p.policySplits(man, nil)
@@ -464,7 +507,7 @@ func readManifest() (catalogue.Manifest, error) {
if err != nil {
return catalogue.Manifest{}, err
}
return catalogue.ParseManifest(raw)
return catalogue.Decode(raw)
}
// countingKV is a memKV that counts its puts.
@@ -352,10 +352,12 @@ func serveTools() error {
}
}()
tools := append(s.tools(), newSeatTools(runtimeStore{}, stdio.Ask).tools()...)
tools = append(tools, newWantsTools(runtimeStore{}, stdio.Ask, func(e, id string, b any) error {
wants := newWantsTools(runtimeStore{}, stdio.Ask, func(e, id string, b any) error {
// The seat's event, said on the seat: a claimed seat's event is never the module's own.
return stdio.SeatPublish(wantsEventSubject(e), b, id)
}).tools()...)
})
wants.watched = stdio.NewWatchedStates() // the large states read whole from one watched copy (ADR 0580 clause 3)
tools = append(tools, wants.tools()...)
return stdio.Serve("", tools)
}
@@ -41,6 +41,10 @@ type wantsTools struct {
ask func(key string, body any) (json.RawMessage, error)
emit func(event, id string, body any) error
now func() time.Time
// watched is the copy of the large states it reads whole, each by one watch started on its first read and kept by
// its changes after (mesh-sdk's WatchedStates, novox/hq ADR 0580 clause 3), never a get per key, which timed out on
// a large state (issue 1953 L5f). Nil reads key by key.
watched *stdio.WatchedStates
}
// wantsEventSubject is the subject of one of the seat's events.
@@ -59,8 +63,16 @@ func (s *wantsTools) leased() bool {
return away
}
// values reads every key of a state.
// values reads every key of a state: from the watched copy when there is one, else key by key.
func (s *wantsTools) values(state string) (map[string][]byte, error) {
if s.watched != nil {
return s.watched.Values(state)
}
return s.direct(state)
}
// direct reads a state key by key, for a small state of the seat's own.
func (s *wantsTools) direct(state string) (map[string][]byte, error) {
keys, err := s.st.keys(state)
if err != nil {
return nil, fmt.Errorf("%s cannot be listed: %w", state, err)
@@ -78,6 +90,16 @@ func (s *wantsTools) values(state string) (map[string][]byte, error) {
return out, nil
}
// manifest reads a manifest by its digest: kept once read when there is a watched copy, a manifest never changing
// under its digest.
func (s *wantsTools) manifest(digest string) ([]byte, bool, error) {
if s.watched != nil {
return s.watched.Once(manifestsState, digest)
}
raw, _, found, err := s.st.get(manifestsState, digest)
return raw, found, err
}
// manifests is every module the indexer holds, with its trunk manifest and fingerprint.
func (s *wantsTools) manifests() (map[string]catalogue.Manifest, map[string]delivery.Module, error) {
mods, err := s.values(modulesState)
@@ -94,7 +116,7 @@ func (s *wantsTools) manifests() (map[string]catalogue.Manifest, map[string]deli
if m.Manifest == "" {
continue
}
mraw, _, found, err := s.st.get(manifestsState, m.Manifest)
mraw, found, err := s.manifest(m.Manifest)
if err != nil || !found {
continue
}
@@ -102,7 +124,8 @@ func (s *wantsTools) manifests() (map[string]catalogue.Manifest, map[string]deli
if err != nil {
continue
}
if man, err := catalogue.ParseManifest([]byte(kept.JSON)); err == nil {
// Decoded, never judged (ADR 0578 §3); one that does not decode is left out of the answer, as before.
if man, err := catalogue.Decode([]byte(kept.JSON)); err == nil {
out[name] = man
}
}
@@ -125,7 +148,7 @@ func (s *wantsTools) doUpgrade(args map[string]any, by string) (any, error) {
if err != nil {
return nil, err
}
raw, err := s.values(policiesState)
raw, err := s.direct(policiesState) // the seat's own small state, read whole after its own writes
if err != nil {
return nil, err
}
@@ -197,10 +197,10 @@ func (w *world) manifest(ctx context.Context, get getter, digest string) parsed
m, err := delivery.DecodeManifest(value)
if err != nil {
p = parsed{err: err.Error()}
} else if man, err := catalogue.ParseManifestForInputs([]byte(m.JSON)); err != nil {
// Read as the indexer reads it (issue 1751): a value falling back to a predefined key is judged by the merge gate
// in the catalogue, against the predefined.json at its root that this process never holds (ADR 0348 §7).
p = parsed{module: m.Module, err: err.Error()}
} else if man, err := catalogue.Decode([]byte(m.JSON)); err != nil {
// Decoded, never judged (novox/hq ADR 0578 §3): the indexer's judge judged it, and its verdict is taken from the
// module's value where it is read (world.mesh).
p = parsed{module: m.Module, err: "it does not decode: " + err.Error()}
} else {
p = parsed{module: m.Module, manifest: man}
}
@@ -313,6 +313,18 @@ func (w *world) mesh(ctx context.Context, manifests getter) assign.Mesh {
continue
}
m.Digests[name] = mod.Manifest
// The indexer's verdict on it, carried in its value (ADR 0578 §3, §5): a refused module is held and named as one
// that cannot be read, and its verdict goes into the composer's input.
if v := mod.VerdictOf(name); v != nil {
if refused, _ := delivery.TakeManifestVerdict(v); refused {
if m.Verdicts == nil {
m.Verdicts = map[string]delivery.ManifestVerdict{}
}
m.Verdicts[name] = *v
m.Unreadable[name] = v.Why()
continue
}
}
p := w.manifest(ctx, manifests, mod.Manifest)
if p.err != "" {
m.Unreadable[name] = p.err
@@ -572,15 +584,9 @@ func (w *world) value(bucket, key string) []byte {
return w.values[bucket][key]
}
// grantsBucket is the bucket the grants are read from now (delivery.GrantSource): the grants part's while the controller's
// lease is handed away, as the lease bucket's key says it (buscompose.LeaseHandedAway), the controller's in every other
// case, a lease not held here included.
func (w *world) grantsBucket() string {
raw := w.value(buscompose.LeaseBucket, buscompose.ControllerLeaseKey)
away, _ := buscompose.LeaseHandedAway(raw, raw != nil, nil)
bucket, _ := delivery.GrantSource(away)
return bucket
}
// grantsBucket is the bucket the grants are read from: the grants part's state `grants`, always (ADR 0571 clause 9); no
// manifest but the controller's reads the controller's grants.
func (w *world) grantsBucket() string { return delivery.PartGrantsBucket }
// has says whether a bucket holds a key now.
func (w *world) has(bucket, key string) bool {
@@ -73,10 +73,11 @@ func TestTheChangeOfACandidate(t *testing.T) {
}
// 1297/B3b: a candidate whose module has no manifest digest yet, or whose manifest the indexer does not hold yet, waits,
// saying for what; an unreadable manifest is the change's, not a wait.
// saying for what; a manifest that does not decode is the change's, not a wait (one with a field the assigner does not
// know is decoded with it set aside, novox/hq ADR 0578 §6).
func TestAChangeWaitsForItsManifests(t *testing.T) {
f := fakeManifests{}
bad := f.add(t, "web", `{"module":"web","version":"2","scope":"x"}`)
bad := f.add(t, "web", `{"module":"web","version":"2","requires":"x"}`)
for _, c := range []delivery.CandidatePull{
pull("novox/web", 1, map[string]delivery.CandidateModule{"web": {None2: "no preview yet"}}),
pull("novox/web", 1, map[string]delivery.CandidateModule{"web": {FP2: fp('b'), Manifest: strings.Repeat("9", 64)}}),
@@ -127,24 +128,22 @@ func theWorldCountsTheValuesAKeyTakes(t *testing.T) {
if got := w.changed(delivery.HaveBucket, "gone"); got != 0 {
t.Errorf("changed = %d for a key never seen", got)
}
// Issue 774: grants are read from the grants part's bucket only while the controller's lease is handed away: a grant
// only the part holds lets a node's put through then, and with a controller's lease (or none, or an unreadable one) it
// holds it, a grant only the controller holds being the one that counts instead.
// ADR 0571 clause 9: grants are read from the grants part's bucket in every case, a lease held, handed away, absent or
// unreadable alike, and the controller's own grants count for nothing.
fp := delivery.SourcePrefix2 + strings.Repeat("d", 64)
gkey, _ := delivery.ControllerGrantKey("web", fp)
grant, _ := delivery.EncodeControllerGrant(delivery.ControllerGrant{Accounts: []string{"ace.web"}})
for _, row := range []struct {
name, lease string
inPart, inController bool
bucket string
granted bool
}{
{"handed away, the part's grant", `{"handed-to":"the operator's hand-over"}`, true, false, delivery.PartGrantsBucket, true},
{"handed away, only the controller's grant", `{"handed-to":"x"}`, false, true, delivery.PartGrantsBucket, false},
{"a controller holds it, only the part's grant", `{"instance":"novox/1","epoch":3}`, true, false, delivery.ControllerGrantsBucket, false},
{"a controller holds it, its own grant", `{"instance":"novox/1"}`, false, true, delivery.ControllerGrantsBucket, true},
{"no lease read", ``, true, false, delivery.ControllerGrantsBucket, false},
{"an unreadable lease", `[1]`, true, false, delivery.ControllerGrantsBucket, false},
{"handed away, the part's grant", `{"handed-to":"the operator's hand-over"}`, true, false, true},
{"handed away, only the controller's grant", `{"handed-to":"x"}`, false, true, false},
{"a controller holds it, the part's grant", `{"instance":"novox/1","epoch":3}`, true, false, true},
{"a controller holds it, only its own grant", `{"instance":"novox/1"}`, false, true, false},
{"no lease read", ``, true, false, true},
{"an unreadable lease", `[1]`, true, false, true},
} {
p := newPasser(buckets{}, time.Now())
if row.lease != "" {
@@ -156,8 +155,8 @@ func theWorldCountsTheValuesAKeyTakes(t *testing.T) {
if row.inController {
p.w.put(delivery.ControllerGrantsBucket, gkey, grant)
}
if got := p.w.grantsBucket(); got != row.bucket {
t.Errorf("%s: grants are read from %s, want %s", row.name, got, row.bucket)
if got := p.w.grantsBucket(); got != delivery.PartGrantsBucket {
t.Errorf("%s: grants are read from %s, want the part's", row.name, got)
}
if _, granted := p.composedFor("ace", "web", fp); granted != row.granted {
t.Errorf("%s: granted = %v, want %v", row.name, granted, row.granted)
+1 -1
View File
@@ -3,7 +3,7 @@ module mesh-delivery-assigner
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031084708-acbf80ad910a
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101025153-c77d17fb42e3
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+18
View File
@@ -1,5 +1,23 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031084708-acbf80ad910a h1:DcSvfEmdPBKqCg5OmXRNb269EkkPe7BsEp+RykEhkgE=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031084708-acbf80ad910a/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005736-7b09de1e5502 h1:XndvkzyAFNde/821lEfYObqMTURzmve/w9Ks+QrBBJ8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005736-7b09de1e5502/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101012031-7cab079e2ec3 h1:an2f0MLx9OhaDLu5i51OMPBbuGwX+aoTzqOkFLMpVfE=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101012031-7cab079e2ec3/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101013319-25a2cecd989c h1:rlUCZ5ifLojqDUTkcCCcPrugTzV2RT6F6JbIVTJkaC8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101013319-25a2cecd989c/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014012-a20bdf795236 h1:JEkSnrmHs5QIRJGhtrghLC1+C0AxqMpef8a32d/BLPg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014012-a20bdf795236/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014310-19c204273bdc h1:kVq7sS4I0EnL8DcAA2a3YstnXS6WmYYzJz7eK7IZ9RY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014310-19c204273bdc/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101020750-62cddf4efc62 h1:R5C8ICjiaDITHRFR/uIATwARPPXKxssUC70m0svstb0=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101020750-62cddf4efc62/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101022905-ba8b8f28a364 h1:UUDPWfUjyerptar+wPwI891jTUAtbbuXori0j3phTE4=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101022905-ba8b8f28a364/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101023317-8311ecb84911 h1:+82ygf2hiY5Mhwfmc1xfCe9e/KY/5ObBuE2LecM37qI=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101023317-8311ecb84911/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101025153-c77d17fb42e3 h1:lb6hz+SBsoCAzsiCCvdtPKyE66Y5Ng/DBAkonAexPzA=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101025153-c77d17fb42e3/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -43,6 +43,9 @@ type Mesh struct {
// InputUnread says why that input could not be read from the bus, when it could not: the check then composes the
// nodes and their assignments alone, and says so in its warnings.
InputUnread string
// Verdicts are the indexer's refusals of trunk manifests, by module, as their values carry them (novox/hq ADR 0578
// §3): each such module is in Unreadable too, and the composer's input carries the verdict (§5).
Verdicts map[string]delivery.ManifestVerdict
}
// Change is a group's change as the indexer's candidate holds it: the manifest of every module it moves or adds, those
@@ -42,7 +42,7 @@ func TestAChangeOfOneModuleComposesItsMachine(t *testing.T) {
}
// 1953/D7 (ADR 0565 §5, ADR 0502 §4): the group's own pull request's held value at its head is laid and named, judged
// against the head's declaration; one of another pull request or another head is not laid.
web, err := catalogue.ParseManifest([]byte(`{"module":"web","version":"2","config":{"port":{"type":"number","default":80,"description":"the port, a number","runtime":true}}}`))
web, err := catalogue.Decode([]byte(`{"module":"web","version":"2","config":{"port":{"type":"number","default":80,"description":"the port, a number","runtime":true}}}`))
if err != nil {
t.Fatal(err)
}
@@ -49,10 +49,18 @@ func HeadJob(m Mesh, c Change) (delivery.ComposeJob, error) {
in.Catalogue = nil
fromClaims = len(in.Holdings) == 0
}
changeCase := composer.Case{Name: CaseChange, Manifests: change}
// Each trunk manifest the indexer refused goes in with its verdict, so the composer holds and names it (ADR 0578 §5);
// a manifest the change brings has no verdict of the indexer yet, and the checker's judge judges it (§2).
changeVerdicts := map[string]delivery.ManifestVerdict{}
for name, v := range m.Verdicts {
if _, changed := c.Digests[name]; !changed {
changeVerdicts[name] = v
}
}
changeCase := composer.Case{Name: CaseChange, Manifests: change, Verdicts: changeVerdicts}
req := composer.Request{V: composer.Version, Input: in, Depth: composer.DepthResolve,
HoldingsFromClaims: fromClaims, Machines: Touched(m, c),
Cases: []composer.Case{{Name: CaseTrunk, Manifests: trunk}, changeCase}}
Cases: []composer.Case{{Name: CaseTrunk, Manifests: trunk, Verdicts: m.Verdicts}, changeCase}}
return job(SDKRepository, c.SDKHead, SDKDir, req, "the mesh-sdk head of "+c.Group)
}
@@ -139,10 +147,10 @@ type Reader struct {
// FromComposer is the composition check of a group with a mesh-sdk member, judged from the composer built at its head
// (ADR 0482 clauses 3 to 5, issues 1622 and 1623): the same failures as Compose, read from the composer's two cases, the
// verdict naming the mesh-sdk the composer was built with; a trunk manifest the head's mesh-sdk refuses fails too (the
// group would deliver a reader that refuses it); and each reader that refuses a changed manifest the head reads fails
// it, naming the module and its words, unless the reader's mesh-sdk has no composer to ask (before issue 1621), which is
// said and judged nothing.
// verdict naming the mesh-sdk the composer was built with; a trunk manifest the head's mesh-sdk cannot decode fails too
// (the group would deliver a reader that cannot read it). A reader that cannot read a changed manifest is a warning,
// never a failure: it decodes leniently and sets a field it does not know aside (novox/hq ADR 0578 §6, which changes
// ADR 0482 §5).
func FromComposer(m Mesh, c Change, head Ran, readers []Reader, took time.Duration) delivery.CompositionVerdict {
v := delivery.CompositionVerdict{Group: c.Group, Heads: c.Heads, SDK: "mesh-sdk " + c.SDKHead, Took: took.Milliseconds()}
machines := Touched(m, c)
@@ -245,8 +253,10 @@ func judgeReaders(c Change, head composer.CaseVerdict, readers []Reader) (fails,
if !head.Manifests[name].Accepted || read.Manifests[name].Accepted {
continue
}
fails = append(fails, fmt.Sprintf("the manifest of %s uses what %s (mesh-sdk %s) cannot read: %s; the group must move %s onto "+
"mesh-sdk %s", name, at, r.Ran.Verdict.SDK, read.Manifests[name].Refused, r.Module, c.SDKHead))
// ADR 0578 §6: no longer a failure; the reader sets what it does not know aside, and the part that needs it takes
// the newer package (ADR 0573 §2).
warnings = append(warnings, fmt.Sprintf("the manifest of %s uses what %s (mesh-sdk %s) cannot read: %s; it reads the rest "+
"once it takes mesh-sdk %s", name, at, r.Ran.Verdict.SDK, read.Manifests[name].Refused, c.SDKHead))
}
}
return fails, warnings
@@ -40,9 +40,10 @@ func digestedMesh() Mesh {
// webShares is a group of mesh-sdk and mesh-catalog whose catalogue member gives web the field only the SDK head reads:
// the assigner's own reader refuses it (Unreadable), as the indexer's manifest is read in world.change.
func webShares(members map[string]string) Change {
_, err := catalogue.ParseManifest([]byte(raws["d-web2"]))
// As an older reader refused it; the field set aside by the lenient decoder is named the same (ADR 0578 §6).
m, _ := catalogue.Decode([]byte(raws["d-web2"]))
return Change{Group: "g", Heads: "0123456789ab", SDKHead: sdkHead, Manifests: map[string]catalogue.Manifest{},
Unreadable: map[string]string{"web": err.Error()}, Digests: map[string]string{"web": "d-web2"}, Members: members}
Unreadable: map[string]string{"web": m.SetAside()}, Digests: map[string]string{"web": "d-web2"}, Members: members}
}
// run is the composer's step for a job, as the build module answers it: the request filled from the indexer's digests
@@ -141,10 +142,11 @@ func TestAFieldTheGroupAddsPassesWithTheHeadsComposer(t *testing.T) {
}
}
// 1623/AC1, AC2: the group of mesh-sdk and mesh-catalog adding and using a field, without a controller member, fails
// naming the field and mesh-controller, whose running build pins an older mesh-sdk; with a controller member pinned to
// the SDK head (its composer reads what the head reads) it passes.
func TestAFieldTheRunningControllerCannotReadFailsTheGroup(t *testing.T) {
// 1623/AC1, AC2 as ADR 0578 §6 changes them: the group of mesh-sdk and mesh-catalog adding and using a field, without a
// controller member, passes with a warning naming the field and mesh-controller, whose running build pins an older
// mesh-sdk that cannot read it: a reader sets a field it does not know aside, so it fails nothing. With a controller
// member pinned to the SDK head it passes with no such warning.
func TestAFieldTheRunningControllerCannotReadIsAWarning(t *testing.T) {
c := webShares(nil)
job, _ := HeadJob(digestedMesh(), c)
head := run(t, job, asTheHeadReads)
@@ -156,10 +158,12 @@ func TestAFieldTheRunningControllerCannotReadFailsTheGroup(t *testing.T) {
if rjob.Fetch["commit"] != running || rjob.Dir != delivery.RootPath {
t.Fatalf("reader job = %+v", rjob)
}
old := run(t, rjob, nil) // the running controller's mesh-sdk reads web2 as the assigner's own does: refused
// The running controller's mesh-sdk, older than the lenient decoder, refuses web2 for its field.
old := Ran{State: delivery.StepPassed, Verdict: &composer.Verdict{V: composer.Version, SDK: "v0.1.13", Cases: []composer.CaseVerdict{
{Name: CaseChange, Manifests: map[string]composer.ManifestRead{"web": {Refused: `json: unknown field "shares"`}}}}}}
v := FromComposer(digestedMesh(), c, head, []Reader{{Module: "mesh-controller", Commit: running, Ran: old}}, 0)
if v.State != delivery.CompositionFail || !strings.Contains(v.Words, "shares") || !strings.Contains(v.Words, "move mesh-controller onto mesh-sdk "+sdkHead) {
t.Fatalf("without a controller member: %+v, want a fail naming the field and mesh-controller", v)
if v.State != delivery.CompositionPass || !hasWarning(v, "shares") || !hasWarning(v, "mesh-controller at "+running) {
t.Fatalf("without a controller member: %+v, want a pass warning of the field and mesh-controller", v)
}
member := "2222222222222222222222222222222222222222"
pinned := run(t, rjob, asTheHeadReads)
@@ -196,9 +200,26 @@ func TestTheHeadsComposerFailingOrRefusingTheTrunkFails(t *testing.T) {
t.Fatalf("a head composer that did not run: %+v", v)
}
job, _ := HeadJob(digestedMesh(), c)
strict := map[string]string{"d-web2": asTheHeadReads["d-web2"], "d-mail": `{"module": "mail", "version": "1", "gone": true}`}
strict := map[string]string{"d-web2": asTheHeadReads["d-web2"], "d-mail": `{"module": "mail", "version": "1", "requires": "x"}`}
v = FromComposer(digestedMesh(), c, run(t, job, strict), nil, 0)
if v.State != delivery.CompositionFail || !strings.Contains(v.Words, "refuses the trunk's manifest of mail") {
t.Fatalf("a head that refuses a trunk manifest: %+v", v)
}
// ADR 0578 §5: a trunk manifest the indexer refused goes into the composer's input with its verdict, in both cases;
// the composer holds and names it, and the group is not blamed for it.
m := digestedMesh()
m.Verdicts = map[string]delivery.ManifestVerdict{"mail": {Module: "mail", State: delivery.VerdictRefused, Problems: []string{"it is wrong"}}}
m.Unreadable = map[string]string{"mail": m.Verdicts["mail"].Why()}
job, _ = HeadJob(m, c)
var req composer.Request
if err := json.Unmarshal(job.Request, &req); err != nil || !req.Cases[0].Verdicts["mail"].Refused() || !req.Cases[1].Verdicts["mail"].Refused() {
t.Fatalf("the request carries no verdict: %+v %v", req.Cases, err)
}
head := run(t, job, asTheHeadReads)
if trunk, _ := caseOf(head.Verdict, CaseTrunk); !strings.Contains(trunk.Manifests["mail"].Refused, "refused the manifest of mail: it is wrong") {
t.Fatalf("the composer did not name the refusal: %+v", trunk.Manifests["mail"])
}
if v = FromComposer(m, c, head, nil, 0); v.State != delivery.CompositionPass {
t.Fatalf("a refusal on the trunk blamed the group: %+v", v)
}
}
@@ -16,9 +16,9 @@ func load(t *testing.T) catalogue.Manifest {
if err != nil {
t.Fatal(err)
}
m, err := catalogue.ParseManifest(raw)
m, err := catalogue.Decode(raw)
if err != nil {
t.Fatalf("mesh-sdk's manifest reader refuses the manifest: %v", err)
t.Fatalf("mesh-sdk cannot decode the manifest: %v", err)
}
return m
}
@@ -33,7 +33,7 @@ func TestManifest(t *testing.T) {
}
reads := slices.Clone(m.Reads)
slices.Sort(reads)
want := []string{"mesh-build-go.steps", "mesh-conditions.takeover", "mesh-config.kept", "mesh-config.resolved", "mesh-controller.assignments", "mesh-controller.composed", "mesh-controller.grants", "mesh-controller.lease",
want := []string{"mesh-build-go.steps", "mesh-conditions.takeover", "mesh-config.kept", "mesh-config.resolved", "mesh-controller.assignments", "mesh-controller.composed", "mesh-controller.lease",
"mesh-delivery-builder.build-log", "mesh-delivery-builder.builds", "mesh-delivery-indexer.candidates", "mesh-delivery-indexer.manifests",
"mesh-delivery-indexer.modules", "mesh-delivery-teller.answers", "mesh-delivery-watchman.presence", "mesh-grants.grants", "mesh-provisioner.provisions", "node-engine.have"}
if !reflect.DeepEqual(reads, want) {
@@ -59,7 +59,7 @@ func TestManifest(t *testing.T) {
t.Errorf("uses = %v, want exactly node-build-go and operator-channel", m.Uses)
}
// ADR 0546: it serves its own tools (put, release, puts) from its bundle through the tool runner (the artifact loads the
// module), says each put's step as the event put, and calls the controller's warranted and the bus node's backup holder.
// module), says each put's step as the event put, and calls the hand-act seat and the bus node's backup holder.
// ADR 0567 §1, §6-§10: it holds the seat mesh-placement, serving its seven verbs; keeps the seat's events on its stream
// placement for 30 days; calls the controller's export, its four verbs that keep its assignments in step and its
// hand-act log; and reads mesh-config's resolved values to compose every node a proposal touches.
@@ -82,9 +82,9 @@ func TestManifest(t *testing.T) {
!slices.Equal(slices.Sorted(slices.Values(m.Streams[0].Events)), slices.Sorted(slices.Values(delivery.PlacementEvents))) {
t.Errorf("streams = %+v, want placement carrying the seat's four events for %d days", m.Streams, delivery.PlacementStreamDays)
}
if want := []string{"seat:mesh-controller.assign", "seat:mesh-controller.hand-act", "seat:mesh-controller.pin",
if want := []string{"seat:mesh-controller.assign", "seat:mesh-controller.pin",
"seat:mesh-controller.placement-export", "seat:mesh-controller.unassign", "seat:mesh-controller.unpin",
"seat:mesh-controller.warranted", "seat:mesh-hand-acts.hand-act", "seat:mesh-hand-acts.hand-acts", "seat:mesh-hand-acts.warranted", // ADR 0565 §9, §11: the seat, the controller's only while it has no holder
"seat:mesh-hand-acts.hand-act", "seat:mesh-hand-acts.hand-acts", "seat:mesh-hand-acts.warranted", // ADR 0565 §9, §11: the seat alone
"seat:node-backup.backed-up", "seat:node-backup.now"}; !slices.Equal(slices.Sorted(slices.Values(m.Invokes)), want) {
t.Errorf("invokes = %v, want %v", m.Invokes, want)
}
@@ -32,7 +32,6 @@
"mesh-config.resolved",
"mesh-controller.assignments",
"mesh-controller.composed",
"mesh-controller.grants",
"mesh-controller.lease",
"mesh-delivery-builder.build-log",
"mesh-delivery-builder.builds",
@@ -51,12 +50,10 @@
],
"invokes": [
"seat:mesh-controller.assign",
"seat:mesh-controller.hand-act",
"seat:mesh-controller.pin",
"seat:mesh-controller.placement-export",
"seat:mesh-controller.unassign",
"seat:mesh-controller.unpin",
"seat:mesh-controller.warranted",
"seat:mesh-hand-acts.hand-act",
"seat:mesh-hand-acts.hand-acts",
"seat:mesh-hand-acts.warranted",
@@ -123,5 +123,6 @@ func (m manifestReader) Manifest(ctx context.Context, digest string) (catalogue.
if err != nil {
return catalogue.Manifest{}, err
}
return catalogue.ParseManifestForInputs([]byte(kept.JSON))
// Decoded, never judged (novox/hq ADR 0578 §3): what a build needs of it is read, a field it does not know set aside.
return catalogue.Decode([]byte(kept.JSON))
}
@@ -11,7 +11,6 @@ import (
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/handacts"
"mesh-delivery-builder/internal/build"
)
@@ -26,19 +25,14 @@ const SeatHeld = "seat:" + catalogue.ArtifactSweepSeat + ".held"
// toolCaller calls a tool or a seat verb as the builder's own account (mesh-sdk process.Conn.Call).
type toolCaller func(key string, body any) (json.RawMessage, error)
// storedVia is build.Builder's Stored: it asks the controller's verb held (mesh-sdk delivery.HeldVerb; novox/hq issue
// 1937) and answers the references the store no longer holds. One it could not ask about, with none gone, is an error:
// the key is then left done, and the controller's take-in still refuses a gone blob.
//
// It asks the seat mesh-artifact-sweep's `held` (novox/hq ADR 0571 §2), and the controller's while that seat has no holder that
// answers (a named exception of clause 10, until the lease write).
// storedVia is build.Builder's Stored: it asks the seat mesh-artifact-sweep's `held` (novox/hq ADR 0571 §2, issue 1937)
// and answers the references the store no longer holds. One it could not ask about, with none gone, is an error: the key
// is then left done, and the controller's take-in still refuses a gone blob. The controller's `held` is not asked: it
// is no longer a fallback (issue 1955).
func storedVia(call toolCaller) func(ctx context.Context, references []string) ([]string, error) {
return func(_ context.Context, references []string) ([]string, error) {
body := map[string]any{"references": references}
raw, err := call(SeatHeld, body)
if handacts.NoResponder(err) {
raw, err = call(delivery.HeldVerb, body)
}
if err != nil {
return nil, fmt.Errorf("asking held: %w", err)
}
+1 -1
View File
@@ -3,7 +3,7 @@ module mesh-delivery-builder
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005726-af4e4e60a3a0
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101023317-8311ecb84911
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+2 -4
View File
@@ -1,7 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031084708-acbf80ad910a h1:DcSvfEmdPBKqCg5OmXRNb269EkkPe7BsEp+RykEhkgE=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031084708-acbf80ad910a/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005726-af4e4e60a3a0 h1:q6ywchpTM9fwaJ1UNwDSuOjb5FYDFcsaeiTrKoB4scc=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005726-af4e4e60a3a0/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101023317-8311ecb84911 h1:+82ygf2hiY5Mhwfmc1xfCe9e/KY/5ObBuE2LecM37qI=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101023317-8311ecb84911/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -61,8 +61,8 @@ type ClaimOptions struct {
// a package's reference. A key queued waiting for a version (BuilderBuild.Waits) is taken only once it is (ADR 0500
// clause 6). Nil means none is.
Published func(ref string) bool
// Stored answers which of the references given the artifact store no longer holds (the controller's verb held,
// mesh-sdk delivery.HeldVerb; novox/hq issue 1937). A done key of the trunk's fingerprint is asked before it is left
// Stored answers which of the references given the artifact store no longer holds (the seat mesh-artifact-sweep's verb held,
// novox/hq ADR 0571 §2, issue 1937). A done key of the trunk's fingerprint is asked before it is left
// done: the store's nightly collector deletes a blob nothing holds, and a key the controller never took in holds
// nothing, so a key whose blob is gone is claimed building as a new revision and built again. Nil asks nothing.
Stored func(ctx context.Context, references []string) (gone []string, err error)
@@ -51,7 +51,7 @@ type Builder struct {
// Answered is told the build module whose step answered a reference (issue 1886): what the builder says it writes
// rests on it. Nil tells nobody.
Answered func(buildModule string)
// Stored answers which references the artifact store no longer holds (the controller's verb held; issue 1937): a done
// Stored answers which references the artifact store no longer holds (the seat mesh-artifact-sweep's verb held; issue 1937): a done
// key of the trunk's fingerprint is asked before it is left done (ClaimOptions.Stored). Nil asks nothing.
Stored func(ctx context.Context, references []string) (gone []string, err error)
// Trunk answers the head of a repository in the gate's source revision rev (source.main at that revision): the trunk a
@@ -107,8 +107,8 @@ func TestManifestDeclarations(t *testing.T) { // 1607/AC1
}
// 1937/AC3: the builder asks the controller whether the artifact store still holds a done key's blobs (held), and no
// other tool; it holds no capacity.
// 1955/L7 (ADR 0571 §2): the seat mesh-artifact-sweep's held, the controller's only while the seat has no holder (clause 10).
if want := []string{"seat:mesh-artifact-sweep.held", "seat:mesh-controller.held"}; !reflect.DeepEqual(sorted(m.Invokes), want) || len(m.Config) != 0 {
// 1955/L7 (ADR 0571 §2): the seat mesh-artifact-sweep's held, the controller's `held` is no longer asked (issue 1955).
if want := []string{"seat:mesh-artifact-sweep.held"}; !reflect.DeepEqual(sorted(m.Invokes), want) || len(m.Config) != 0 {
t.Errorf("invokes = %v, config = %v, want %v and no config", m.Invokes, m.Config, want)
}
// The builder is woken by a build module's report of a toolchain, as the indexer is: the bucket may not exist when it
+1 -2
View File
@@ -20,8 +20,7 @@
"mesh-controller.artifacts-gone"
],
"invokes": [
"seat:mesh-artifact-sweep.held",
"seat:mesh-controller.held"
"seat:mesh-artifact-sweep.held"
],
"state": [
{ "name": "builds", "history": 5 },
@@ -125,11 +125,11 @@ func (m manifestReader) Manifest(ctx context.Context, digest string) (catalogue.
return parseManifest(kept.JSON)
}
// parseManifest reads a manifest the indexer kept as the indexer and the assigner read it (issues 1751, 1906): a config value that
// falls back to a predefined key is judged by the merge gate in the catalogue, against the predefined.json at its root that this
// process never holds, so it is not unreadable here (novox/hq ADR 0348 §7).
// parseManifest decodes a manifest the indexer kept, as every reader does (novox/hq ADR 0578 §3): judged by nothing here,
// a field it does not know set aside. The checker judges a change's manifests with the indexer's judge, run as a step
// (judge.go).
func parseManifest(raw string) (catalogue.Manifest, error) {
return catalogue.ParseManifestForInputs([]byte(raw))
return catalogue.Decode([]byte(raw))
}
// awaitedKV is a bucket that may appear after the start (process.AwaitBucket) as a check.KV: a call while it is not on the
@@ -2,6 +2,8 @@ package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"sort"
"strings"
@@ -31,7 +33,31 @@ type judgeDeps struct {
compositions check.CompositionReader
// nodes names every node with a record; nil hands no names.
nodes *nodeNames
logf func(format string, args ...any)
// predefined is the indexer's kept list of predefined keys; nil reads none.
predefined check.PredefinedList
logf func(format string, args ...any)
}
// keptPredefined reads the indexer's key predefined.list (delivery.PredefinedKept), bound once its state appears.
type keptPredefined struct{ b *process.AwaitedBucket }
func (k keptPredefined) PredefinedList(ctx context.Context) (json.RawMessage, error) {
kv := k.b.KV()
if kv == nil {
return nil, process.ErrNotYet
}
e, err := kv.Get(ctx, delivery.PredefinedListKey)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return nil, nil
}
if err != nil {
return nil, err
}
var kept delivery.PredefinedKept
if err := json.Unmarshal(e.Value(), &kept); err != nil {
return nil, err
}
return kept.List, nil
}
// nodeNames reads every node's name from the node-engine's have, one key `<node>.node` per node with a record (novox/hq
@@ -69,7 +95,7 @@ func newJudge(d judgeDeps) check.Judge {
nodes = d.nodes
}
return &check.Examiner{Nodes: nodes, Compositions: d.compositions, Asks: d.asks, Toolchains: d.tools,
Kept: check.Kept{Store: d.kept}, Previews: d.previews, Modules: d.repos, Packages: d.repos, Published: d.repos, Forge: d.forge, Runs: d.repos, Builds: d.builds, Announce: d.announce, Trunks: d.trunks, Logf: d.logf}
Kept: check.Kept{Store: d.kept}, Previews: d.previews, Modules: d.repos, Packages: d.repos, Published: d.repos, Forge: d.forge, Runs: d.repos, Builds: d.builds, Announce: d.announce, Trunks: d.trunks, Manifests: d.repos, Predefined: d.predefined, Logf: d.logf}
}
// manifestKV reads a manifest by its digest.
@@ -250,6 +276,20 @@ func (r buildReader) Build(ctx context.Context, module, fp2 string) (delivery.Bu
return b, err == nil, err
}
// ManifestDigests is the manifest digest of every module the indexer holds, by module: the catalogue the indexer's judge
// judges a change's manifests in (novox/hq ADR 0578 §2).
func (r *repoModules) ManifestDigests() map[string]string {
r.mu.Lock()
defer r.mu.Unlock()
out := map[string]string{}
for module, digest := range r.typed {
if digest != "" {
out[module] = digest
}
}
return out
}
// CommitOf is the commit the indexer's record of the module names: the commit its fingerprint first appeared at, so the source
// of the build the mesh runs (a later merge that leaves the module alone writes no record, the operator 2026-10-25).
func (r *repoModules) CommitOf(module string) string {
@@ -202,6 +202,8 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
repos: repos, builds: builds, forge: forgeReads{call: caller.Call}, logf: logger.Printf,
announce: announce, compositions: compositions,
nodes: openNodeNames(ctx, js, logger.Printf),
predefined: keptPredefined{process.AwaitBucket(ctx, js, conn, delivery.IndexerModule+"_"+delivery.IndexerPredefinedState,
"a change's manifests are judged with no predefined keys, each fallback as its value declares it", logger.Printf)},
})
ch.Announce = announce
tryer.Judge = ch.Judge
+1 -1
View File
@@ -3,7 +3,7 @@ module mesh-delivery-checker
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005726-af4e4e60a3a0
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101020750-62cddf4efc62
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+12 -2
View File
@@ -1,7 +1,17 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031084708-acbf80ad910a h1:DcSvfEmdPBKqCg5OmXRNb269EkkPe7BsEp+RykEhkgE=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031084708-acbf80ad910a/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005726-af4e4e60a3a0 h1:q6ywchpTM9fwaJ1UNwDSuOjb5FYDFcsaeiTrKoB4scc=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005726-af4e4e60a3a0/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005736-7b09de1e5502 h1:XndvkzyAFNde/821lEfYObqMTURzmve/w9Ks+QrBBJ8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005736-7b09de1e5502/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101012031-7cab079e2ec3 h1:an2f0MLx9OhaDLu5i51OMPBbuGwX+aoTzqOkFLMpVfE=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101012031-7cab079e2ec3/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101013319-25a2cecd989c h1:rlUCZ5ifLojqDUTkcCCcPrugTzV2RT6F6JbIVTJkaC8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101013319-25a2cecd989c/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014012-a20bdf795236 h1:JEkSnrmHs5QIRJGhtrghLC1+C0AxqMpef8a32d/BLPg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014012-a20bdf795236/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014310-19c204273bdc h1:kVq7sS4I0EnL8DcAA2a3YstnXS6WmYYzJz7eK7IZ9RY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014310-19c204273bdc/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101020750-62cddf4efc62 h1:R5C8ICjiaDITHRFR/uIATwARPPXKxssUC70m0svstb0=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101020750-62cddf4efc62/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -17,6 +17,7 @@ import (
"time"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"git.novox.be/novox/mesh-sdk/go/composer"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/delivery/buildrun"
)
@@ -928,3 +929,81 @@ func TestASecondCheckOfTheSameHeadsKeepsWhatTheFirstPassed(t *testing.T) {
t.Errorf("%d test steps ran on the second check", n)
}
}
// keptList is the indexer's kept predefined list as a test gives it.
type keptList string
func (k keptList) PredefinedList(context.Context) (json.RawMessage, error) {
return json.RawMessage(k), nil
}
// fixedDigests are the indexer's manifest digests as a test gives them.
type fixedDigests map[string]string
func (f fixedDigests) ManifestDigests() map[string]string { return f }
// ADR 0578 §2: a change's manifests are judged by the indexer's judge, built at mesh-delivery's trunk and run as the
// composer is (a compose ask of its command); a refusal fails the check naming the manifest and each problem; a group
// whose mesh-delivery member changes the judge is judged by that member's head.
func TestTheIndexersJudgeJudgesTheChangesManifests(t *testing.T) {
web, trunk := strings.Repeat("e", 64), strings.Repeat("d", 40)
for _, row := range []struct {
name string
refused string
fails string
// untouched is the judge's refusal of mail, which the change leaves as it is, and trunkMail its refusal on the trunk:
// a problem the trunk had is a warning, one the change causes fails (ADR 0579 §2)
untouched, trunkMail string
}{
{name: "accepted"},
{name: "an untouched manifest's problem the trunk had", untouched: "mail says no credential form", trunkMail: "mail says no credential form"},
{name: "a problem the change causes in an untouched manifest", untouched: "mail reads a state web no longer keeps",
fails: "refuses the manifest of mail: mail reads a state web no longer keeps"},
{name: "refused", refused: "web requires \"amqp\"\nweb has no id", fails: "the indexer's judge at mesh-delivery's trunk " + strings.Repeat("d", 40) + ` refuses the manifest of web: web requires "amqp"; web has no id`},
} {
b := newBench(t)
b.ex.Manifests = fixedDigests{"mail": strings.Repeat("a", 64), "web": strings.Repeat("b", 64)}
b.ex.Trunks = fixedTrunks{heads: map[string]string{DeliveryRepository: trunk}}
b.ex.Predefined = keptList(`{"font-size": {"type": "number", "description": "the size"}}`)
var judged []delivery.ComposeJob
answer := b.asks.Answer
b.asks.Answer = func(a delivery.StepAsk) delivery.Step {
if a.Compose == nil {
return answer(a)
}
judged = append(judged, *a.Compose)
read := composer.ManifestRead{Accepted: row.refused == "", Refused: row.refused}
verdict, _ := json.Marshal(composer.Verdict{V: composer.Version, SDK: "exe:1", Cases: []composer.CaseVerdict{{Name: JudgeCase,
Manifests: map[string]composer.ManifestRead{"web": read, "mail": {Accepted: row.untouched == "", Refused: row.untouched}}},
{Name: JudgeTrunk, Manifests: map[string]composer.ManifestRead{"web": {Accepted: true}, "mail": {Accepted: row.trunkMail == "", Refused: row.trunkMail}}}}})
result, _ := json.Marshal(delivery.ComposeAnswer{Key: "k", Verdict: verdict})
return delivery.Step{ID: a.ID, State: delivery.StepPassed, Result: result}
}
p := pullOf(repoA, 7, head1)
p.Modules = []string{"web"}
p.Fps = map[string]delivery.CandidateModule{"web": {FP2: "src2:" + strings.Repeat("f", 64), Manifest: web}}
v := b.judge(t, caseOf(t, p)).Verdict
if len(judged) != 1 || judged[0].Fetch["repository"] != DeliveryRepository || judged[0].Fetch["commit"] != trunk ||
judged[0].Package != JudgePackage || judged[0].Dir != JudgeDir {
t.Fatalf("%s: judged %+v (verdict %+v)", row.name, judged, v)
}
var req composer.Request
if err := json.Unmarshal(judged[0].Request, &req); err != nil || req.Cases[0].Manifests["web"].Digest != web ||
req.Cases[1].Name != JudgeTrunk || req.Cases[1].Manifests["web"].Digest != strings.Repeat("b", 64) ||
req.Predefined["font-size"].Type != "number" ||
req.Cases[0].Manifests["mail"].Digest != strings.Repeat("a", 64) {
t.Fatalf("%s: the request %+v: %v", row.name, req, err)
}
switch {
case row.fails == "" && v.State != delivery.VerdictPass:
t.Errorf("%s: %+v", row.name, v)
case row.fails != "" && (v.State != delivery.VerdictFail || !strings.Contains(v.Failed.Words, row.fails)):
t.Errorf("%s: %+v, want a fail saying %q", row.name, v.Failed, row.fails)
}
}
if !changesJudge(DeliveryRepository, []Fetched{{Changed: []string{JudgeDir + "/judge/judge.go"}}}) ||
changesJudge(DeliveryRepository, []Fetched{{Changed: []string{JudgeDir + "/internal/index/x.go"}}}) ||
changesJudge("novox/mesh-catalog", []Fetched{{Changed: []string{JudgeDir + "/judge/judge.go"}}}) {
t.Error("which merge changes the judge")
}
}
@@ -111,6 +111,11 @@ type Examiner struct {
}
// Trunks are the trunk heads of every repository, for the siblings a repository's tests read.
Trunks interface{ Heads() map[string]string }
// Manifests are the indexer's manifest digests by module: the catalogue a change's manifests are judged in by the
// indexer's judge (manifests_judge.go, novox/hq ADR 0578 §2); nil judges no manifest.
Manifests TrunkManifests
// Predefined reads the catalogue's predefined keys the indexer keeps (manifests_judge.go); nil reads none.
Predefined PredefinedList
// Compositions answers the assigner's composition check of a check by the check's key (delivery.CompositionKey),
// waiting until the assigner wrote a verdict that does not wait, or ctx ends (ADR 0528 clause 5, issue 1297); nil
// leaves the composition not run.
@@ -219,6 +224,11 @@ type state struct {
slow []string
// beside are the siblings read at their trunk's head, by repository (ADR 0548 §6).
beside map[string]string
// judgeHead is the group's mesh-delivery member head when its merge changes the judge (ADR 0578 §2); empty judges
// with mesh-delivery's trunk.
judgeHead string
// predefinedHead is the catalogue member's head when its merge changes predefined.json: the list is read there.
predefinedHead string
}
// Judge runs the case (it is the Judge of a Checker). A failure of the change, of a merge or of a step is a verdict; an error is only a case that cannot
@@ -229,6 +239,10 @@ func (j *Examiner) Judge(ctx context.Context, c *Case) (Outcome, error) {
if v := st.judgeAll(ctx); v != nil {
return st.finish(ctx, *v), nil
}
// Every manifest the change touches, judged by the indexer's judge (ADR 0578 §2): a refusal fails the check.
if v := st.judgeManifests(ctx); v != nil {
return st.finish(ctx, *v), nil
}
return st.finish(ctx, delivery.Verdict{State: delivery.VerdictPass}), nil
}
@@ -388,6 +402,7 @@ func (st *state) judgeAll(ctx context.Context) *delivery.Verdict {
if st.c.Try == nil { // a try owes the indexer no preview: no pull request holds its head
st.j.writePreview(ctx, p, merged)
}
st.judgedByMember(p, merged)
if p.Repository == SDKRepository && len(st.c.Pulls) > 1 && st.j.changesPackage(p, merged) {
groupPackage = map[string]any{"repository": p.Repository, "commit": p.Head, "ref": refOf(p.Number)}
}
@@ -0,0 +1,259 @@
package check
import (
"context"
"encoding/json"
"fmt"
"maps"
"slices"
"strings"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/composer"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// A change's manifests are judged by the indexer's judge (novox/hq ADR 0578 §2): the checker asks the Go build seat to
// build the judge's command (mesh-delivery's modules/mesh-delivery-indexer, cmd/mesh-judge) at mesh-delivery's trunk, or
// at the group's mesh-delivery member head when that member changes the judge, and to run it in its sandbox over the
// catalogue as the change leaves it, exactly as the composer is run (delivery.ComposeJob, ADR 0482 §4). A manifest the
// change touches that the judge refuses fails the check, naming the manifest and each problem. The checker links no
// judge of its own.
// The judge the checker runs.
const (
DeliveryRepository = "novox/mesh-delivery"
JudgeDir = "modules/mesh-delivery-indexer"
JudgePackage = "mesh-delivery-indexer/cmd/mesh-judge"
// JudgeCase is the case of a judge request holding the catalogue as the change leaves it, and JudgeTrunk the one
// holding it as the trunk holds it: a problem the change case has and the trunk case lacks is the change's (ADR 0579 §2).
JudgeCase = "change"
JudgeTrunk = "trunk"
// judgeBetween is how the judge names the problems between manifests that no one module's removal ends.
judgeBetween = "(between manifests)"
)
// judgePaths are the paths of mesh-delivery whose change changes the judge: a group whose mesh-delivery member changes
// one is judged by that member's head.
var judgePaths = []string{JudgeDir + "/judge/", JudgeDir + "/cmd/mesh-judge/", JudgeDir + "/go.mod", JudgeDir + "/go.sum"}
// changesJudge says whether a merge of mesh-delivery with these changed paths changes the judge.
func changesJudge(repository string, merged []Fetched) bool {
if repository != DeliveryRepository {
return false
}
for _, f := range merged {
for _, p := range f.Changed {
for _, j := range judgePaths {
if p == j || strings.HasPrefix(p, j) {
return true
}
}
}
}
return false
}
// TrunkManifests says the manifest digest of every module the indexer holds (its modules state), by module.
type TrunkManifests interface {
ManifestDigests() map[string]string
}
// PredefinedList is the catalogue's list of predefined keys the indexer keeps (its state predefined, key list), the one a
// config value falling back to a key is judged against (ADR 0348 §7); nil when it keeps none.
type PredefinedList interface {
PredefinedList(ctx context.Context) (json.RawMessage, error)
}
// judgeRequest is the judge's request over the catalogue as the case leaves it: every trunk manifest by its digest, each
// one a pull request touches replaced by the digest its candidate names; touched are the modules whose manifest the
// change adds or changes (a digest the trunk does not hold for it), sorted (novox/hq ADR 0579 §2).
func judgeRequest(trunk map[string]string, pulls []Pull) (req composer.Request, touched []string) {
manifests, before := map[string]composer.Manifest{}, map[string]composer.Manifest{}
for module, digest := range trunk {
if digest != "" {
manifests[module] = composer.Manifest{Digest: digest}
before[module] = composer.Manifest{Digest: digest}
}
}
seen := map[string]bool{}
for _, p := range pulls {
for module, cm := range p.Fps {
if cm.Manifest == "" {
continue
}
manifests[module] = composer.Manifest{Digest: cm.Manifest}
if trunk[module] != cm.Manifest {
seen[module] = true
}
}
}
return composer.Request{V: composer.Version, Cases: []composer.Case{{Name: JudgeCase, Manifests: manifests},
{Name: JudgeTrunk, Manifests: before}}}, slices.Sorted(maps.Keys(seen))
}
// judgeManifests runs the indexer's judge over the case's manifests and fails the check for each touched manifest it
// refuses. A case that touches no manifest, or an examiner that reads no trunk manifests, judges nothing; a judge that
// cannot be asked or does not answer fails the check, said, never passes it.
func (st *state) judgeManifests(ctx context.Context) *delivery.Verdict {
if st.j.Manifests == nil || st.c.Try != nil {
return nil
}
req, touched := judgeRequest(st.j.Manifests.ManifestDigests(), st.c.Pulls)
if len(touched) == 0 {
return nil
}
commit, at := st.judgeHead, "the group's mesh-delivery head "+st.judgeHead
if commit == "" {
if st.j.Trunks != nil {
commit = st.j.Trunks.Heads()[DeliveryRepository]
}
at = "mesh-delivery's trunk " + commit
}
if commit == "" {
return fail(goModule, "the change's manifests cannot be judged: the trunk head of "+DeliveryRepository+" is not known")
}
// The list of predefined keys the manifests are judged against (ADR 0578 §2): the catalogue member's predefined.json
// when the change alters it, else the one the indexer keeps.
if list, from := st.predefinedList(ctx); list != nil {
req.Predefined = list
st.c.note("the manifests are judged against the predefined keys of " + from)
}
raw, err := json.Marshal(req)
if err != nil {
return fail(goModule, err.Error())
}
job := delivery.ComposeJob{Fetch: map[string]any{"repository": DeliveryRepository, "commit": commit}, Dir: JudgeDir,
Package: JudgePackage, Request: raw, Reader: "the indexer's judge at " + at}
a := delivery.StepAsk{ID: buildstep.NewID(), Kind: delivery.AskCheck, Language: delivery.AskGo, Asker: delivery.CheckerModule,
At: st.j.now().UTC(), Compose: &job}
st.c.Progress.Set(fmt.Sprintf("the judge ask %s of the manifests of %s", a.ID, strings.Join(touched, ", ")), "")
st.c.Asked(a.ID)
if err := st.j.Asks.Put(ctx, a); err != nil {
return fail(goModule, "the judge of the change's manifests could not be asked: "+err.Error())
}
step, err := st.j.Asks.Wait(ctx, a)
if err != nil {
return fail(goModule, "the judge of the change's manifests: "+err.Error())
}
cv, trunkCase, why := judgeAnswer(step)
if why != "" {
return fail(goModule, fmt.Sprintf("the indexer's judge at %s did not judge the change's manifests: %s", at, why))
}
// A new or changed manifest is judged in full; of one the change leaves as it is, a problem the trunk already had is a
// warning (it is not refused for a rule it already broke), and a problem the change causes in it, which the trunk case
// lacks, fails the check as the change's (ADR 0579 §2).
var refused, warned []string
for _, module := range slices.Sorted(maps.Keys(cv.Manifests)) {
r := cv.Manifests[module]
if r.Accepted || r.Refused == "" {
continue
}
had := map[string]bool{}
for _, p := range strings.Split(trunkCase.Manifests[module].Refused, "\n") {
had[p] = true
}
var caused, old []string
for _, p := range strings.Split(r.Refused, "\n") {
if slices.Contains(touched, module) || !had[p] {
caused = append(caused, p)
} else {
old = append(old, p)
}
}
if len(caused) > 0 {
what := "the manifest of " + module
if module == judgeBetween {
what = "the catalogue"
}
refused = append(refused, fmt.Sprintf("the indexer's judge at %s refuses %s: %s", at, what, strings.Join(caused, "; ")))
}
if len(old) > 0 {
warned = append(warned, module+": "+strings.Join(old, "; "))
}
}
if len(warned) > 0 {
st.c.note("warnings of the indexer's judge on manifests the change leaves as they are (ADR 0579 §2): " + strings.Join(warned, " | "))
}
if len(refused) > 0 {
return fail(goModule, strings.Join(refused, "\n"))
}
st.c.note(fmt.Sprintf("the manifests of %s are accepted by the indexer's judge at %s", strings.Join(touched, ", "), at))
return nil
}
// judgeAnswer reads the judge's change and trunk cases from its step, or why it cannot.
func judgeAnswer(step delivery.Step) (change, trunk composer.CaseVerdict, why string) {
if step.State != delivery.StepPassed {
return change, trunk, fmt.Sprintf("its step %s: %s", step.State, step.Reason)
}
var a delivery.ComposeAnswer
if err := json.Unmarshal(step.Result, &a); err != nil {
return change, trunk, "its step's answer cannot be read: " + err.Error()
}
var v composer.Verdict
if err := json.Unmarshal(a.Verdict, &v); err != nil || v.V != composer.Version {
return change, trunk, fmt.Sprintf("its verdict cannot be read (version %d): %v", v.V, err)
}
found := 0
for _, c := range v.Cases {
switch c.Name {
case JudgeCase:
change, found = c, found+1
case JudgeTrunk:
trunk, found = c, found+1
}
}
if found != 2 {
return change, trunk, "its verdict does not hold both the cases " + JudgeCase + " and " + JudgeTrunk
}
return change, trunk, ""
}
// predefinedList is the list of predefined keys a case's manifests are judged against, and where it comes from: the
// catalogue member's predefined.json at its head when its merge changes it, else the one the indexer keeps; nil when
// neither can be read, said, and the judge then takes each fallback as its value declares it.
func (st *state) predefinedList(ctx context.Context) (catalogue.PredefinedList, string) {
read := func(raw []byte, from string) (catalogue.PredefinedList, string) {
list, err := catalogue.ParsePredefined(raw)
if err != nil {
st.c.note("the predefined keys of " + from + " cannot be read, so fallbacks are judged as their values declare them: " + err.Error())
return nil, ""
}
return list, from
}
if st.predefinedHead != "" && st.j.Forge != nil {
text, err := st.j.Forge.File(ctx, catalogueRepository, catalogue.PredefinedFile, st.predefinedHead)
if err == nil {
return read([]byte(text), "the catalogue member's head "+st.predefinedHead)
}
st.c.note("the catalogue member's predefined.json cannot be read: " + err.Error())
}
if st.j.Predefined == nil {
return nil, ""
}
raw, err := st.j.Predefined.PredefinedList(ctx)
if err != nil || raw == nil {
if err != nil {
st.c.note("the indexer's predefined keys cannot be read: " + err.Error())
}
return nil, ""
}
return read(raw, "the indexer's kept list")
}
// judgedByMember notes the group's mesh-delivery member head as the judge's when its merge changes the judge, and the
// catalogue member's head as where the predefined keys are read when its merge changes predefined.json.
func (st *state) judgedByMember(p Pull, merged []Fetched) {
if changesJudge(p.Repository, merged) {
st.judgeHead = p.Head
}
if p.Repository == catalogueRepository {
for _, f := range merged {
if slices.Contains(f.Changed, catalogue.PredefinedFile) {
st.predefinedHead = p.Head
}
}
}
}
@@ -103,7 +103,7 @@ func TestManifestDeclarations(t *testing.T) { // 1678/AC1
}
if want := []string{"mesh-build-go.steps", "mesh-build-go.toolchains", "mesh-build-node.steps", "mesh-build-node.toolchains",
"mesh-delivery-assigner.compositions", "mesh-delivery-assigner.placement", "mesh-delivery-builder.builds", "mesh-delivery-gate.source", "mesh-delivery-gate.turns", "mesh-delivery-indexer.candidates",
"mesh-delivery-indexer.manifests", "mesh-delivery-indexer.modules", "mesh-delivery-watchman.presence", "node-engine.have"}; !reflect.DeepEqual(sorted(m.Reads), want) {
"mesh-delivery-indexer.manifests", "mesh-delivery-indexer.modules", "mesh-delivery-indexer.predefined", "mesh-delivery-watchman.presence", "node-engine.have"}; !reflect.DeepEqual(sorted(m.Reads), want) {
t.Errorf("reads = %v, want %v", m.Reads, want)
}
// 1887/A5b: the checker reads nothing of the controller's plans-building (ADR 0483 superseded by ADR 0525 §7), calls no
@@ -42,6 +42,7 @@
"mesh-delivery-indexer.candidates",
"mesh-delivery-indexer.manifests",
"mesh-delivery-indexer.modules",
"mesh-delivery-indexer.predefined",
"mesh-delivery-watchman.presence",
"node-engine.have"
],
+1 -1
View File
@@ -3,7 +3,7 @@ module mesh-delivery-forge-bridge
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005726-af4e4e60a3a0
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101020750-62cddf4efc62
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
@@ -2,6 +2,12 @@ git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030201806-eff81f1220e2 h1:RF73nVI1
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030201806-eff81f1220e2/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005726-af4e4e60a3a0 h1:q6ywchpTM9fwaJ1UNwDSuOjb5FYDFcsaeiTrKoB4scc=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005726-af4e4e60a3a0/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014012-a20bdf795236 h1:JEkSnrmHs5QIRJGhtrghLC1+C0AxqMpef8a32d/BLPg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014012-a20bdf795236/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014310-19c204273bdc h1:kVq7sS4I0EnL8DcAA2a3YstnXS6WmYYzJz7eK7IZ9RY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014310-19c204273bdc/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101020750-62cddf4efc62 h1:R5C8ICjiaDITHRFR/uIATwARPPXKxssUC70m0svstb0=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101020750-62cddf4efc62/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -4,6 +4,9 @@ package main
import (
"context"
"crypto/ecdh"
"crypto/rand"
"encoding/base64"
"encoding/json"
"fmt"
"io"
@@ -20,6 +23,8 @@ import (
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/compose/frombus"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/process"
)
@@ -229,7 +234,12 @@ func TestIndexerWritesModulesAtAMerge(t *testing.T) { // 1239/AC9; I1, I3: the m
// TestIndexerKeepsManifestsAtAMerge: a merged message of mesh-catalog carrying the real manifest of the module avahi and the
// hash of its go.sum (the bridge's Manifests and Locks, issue 1448) makes, within 5 s, modules.avahi with its manifest digest,
// its inputs and the reason it has no fingerprint of the new form, and manifests.<digest> with the manifest byte for byte.
func TestIndexerKeepsManifestsAtAMerge(t *testing.T) { // 1444/AC4, 1449/AC4
//
// And it judges them (novox/hq ADR 0578, issue 1585): with three manifests more, one valid, one refused and one with a
// field the decoder does not know, mesh-sdk's bus reader read before the indexer judged composes all three, the field set
// aside and named (a missing verdict is accepted, §3); the indexer writes the three verdicts, says manifest-refused for
// each refusal, and the reader then composes the valid one and holds the others, named, never ending its read.
func TestIndexerKeepsManifestsAtAMerge(t *testing.T) { // 1444/AC4, 1449/AC4, 1585
// The manifest is avahi's module.json from mesh-catalog at a67f01963f0d9ef0913a63720b89e573ca7a16ef (copied into the
// index package's testdata); the lock hash is the sha256 of modules/avahi/go.sum there.
manifest, err := os.ReadFile("../../internal/index/testdata/catalogue-a67f0196-avahi.module.json")
@@ -237,14 +247,31 @@ func TestIndexerKeepsManifestsAtAMerge(t *testing.T) { // 1444/AC4, 1449/AC4
t.Fatal(err)
}
const goSum = "3b5028f0c1e9bde04dd766abd63eb5931bc516bae695cf46f00abcd8510b0381"
judged := map[string]string{
"good": `{"module":"good","version":"1","resources":[{"id":"d","type":"directory","path":"/srv/good","mode":"0700"}],"data":{"own":[{"id":"d","path":"${dir:d}","class":"valuable"}]}}`,
"bad": `{"module":"bad","version":"1","provides":[{"name":"amqp"}],"resources":[{"id":"d","type":"directory","path":"/srv/bad","mode":"0700"}],"data":{"own":[{"id":"d","path":"${dir:d}","class":"valuable"}]}}`,
"later": `{"module":"later","version":"1","a-field-from-later":1,"resources":[{"id":"d","type":"directory","path":"/srv/later","mode":"0700"}],"data":{"own":[{"id":"d","path":"${dir:d}","class":"valuable"}]}}`,
}
js, ctx, _, modules := startIndexer(t, func(ctx context.Context, builds buildLog) {
builds.end(t, "novox/mesh-catalog", "avahi", "modules/avahi")
}, func(ctx context.Context, js jetstream.JetStream) {
// The verdicts state and the stream of the indexer's events, as the controller makes them from its manifest.
if _, err := js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: delivery.VerdictsBucket, History: delivery.VerdictsHistory}); err != nil {
t.Fatal(err)
}
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: "INDEXER_EVENTS", Subjects: []string{"mesh.mod.mesh-delivery-indexer.event.>"}}); err != nil {
t.Fatal(err)
}
})
data, err := delivery.EncodeForge(delivery.Forge{Kind: delivery.ForgeMerged, Repository: "novox/mesh-catalog", Branch: "feat/x", Number: 301,
forge := delivery.Forge{Kind: delivery.ForgeMerged, Repository: "novox/mesh-catalog", Branch: "feat/x", Number: 301,
Head: "0123456789abcdef0123456789abcdef01234567", Merge: "89abcdef0123456789abcdef0123456789abcdef", Root: "r0",
Trees: map[string]string{"modules/avahi": "newtree"},
Manifests: map[string]string{"modules/avahi": string(manifest)},
Locks: map[string]map[string]string{"modules/avahi": {"go.sum": goSum}}})
Locks: map[string]map[string]string{"modules/avahi": {"go.sum": goSum}}}
for name, raw := range judged {
forge.Manifests["modules/"+name], forge.Trees["modules/"+name] = raw, "tree-"+name
}
data, err := delivery.EncodeForge(forge)
if err != nil {
t.Fatal(err)
}
@@ -275,6 +302,156 @@ func TestIndexerKeepsManifestsAtAMerge(t *testing.T) { // 1444/AC4, 1449/AC4
if got, err := delivery.DecodeManifest(e.Value()); err != nil || got.Module != "avahi" || got.JSON != string(manifest) {
t.Errorf("manifest = %+v err %v, want avahi and the file's bytes", got, err)
}
verdicts, err := js.KeyValue(ctx, delivery.VerdictsBucket)
if err != nil {
t.Fatal(err)
}
want := map[string]string{"good": delivery.VerdictAccepted, "bad": delivery.VerdictRefused, "later": delivery.VerdictRefused}
got := map[string]delivery.ManifestVerdict{}
for deadline := time.Now().Add(10 * time.Second); time.Now().Before(deadline) && len(got) < len(want); time.Sleep(50 * time.Millisecond) {
for name := range want {
if e, err := verdicts.Get(ctx, delivery.ManifestDigest([]byte(judged[name]))); err == nil {
if v, err := delivery.DecodeManifestVerdict(e.Value()); err == nil {
got[name] = v
}
}
}
}
for name, state := range want {
if v := got[name]; v.State != state || v.Module != name || !strings.HasPrefix(v.Judge, "exe:") {
t.Errorf("the verdict on %s = %+v within 10 s, want %s by the running judge", name, v, state)
}
}
if !strings.Contains(strings.Join(got["bad"].Problems, " "), `provides "amqp"`) || !strings.Contains(strings.Join(got["later"].Problems, " "), "a-field-from-later") {
t.Errorf("the refusals name no problem: %+v", got)
}
// manifest-refused, said once for each refusal.
events, err := js.Stream(ctx, "INDEXER_EVENTS")
if err != nil {
t.Fatal(err)
}
said := map[string]bool{}
for seq := uint64(1); ; seq++ {
msg, err := events.GetMsg(ctx, seq)
if err != nil {
break
}
var r delivery.ManifestRefused
if msg.Subject == "mesh.mod.mesh-delivery-indexer.event."+delivery.ManifestRefusedEvent && json.Unmarshal(msg.Data, &r) == nil {
said[r.Module] = true
}
}
if !said["bad"] || !said["later"] || said["good"] {
t.Errorf("manifest-refused said for %v, want bad and later", said)
}
// mesh-sdk's bus reader over the indexer's modules and manifests: the refused modules held and named, the valid one
// composed, the read whole.
r := judgedReader{ctx: ctx, js: js, t: t}
items, _, held, err := frombus.DataItemsHeld(ctx, r, frombus.Mesh{}, frombus.Scope{Node: "n1"})
if err != nil || len(items["good"]) != 1 || items["bad"] != nil || items["later"] != nil ||
!strings.Contains(held["bad"], `provides "amqp"`) || !strings.Contains(held["later"], "a-field-from-later") {
t.Errorf("the bus reader after the judge: items %v held %v: %v", items, held, err)
}
// Before any verdict (a value without one, as an older indexer wrote it), the reader takes each as accepted and names
// the field it set aside.
for _, name := range []string{"bad", "later"} {
e, err := modules.Get(ctx, name)
if err != nil {
t.Fatal(err)
}
m, _ := delivery.DecodeModule(e.Value())
m.Verdict, m.Problems = "", nil
raw, _ := delivery.EncodeModule(m)
r.unjudged = append(r.unjudged, raw)
if r.values == nil {
r.values = map[string][]byte{}
}
r.values[name] = raw
}
in, err := frombus.Input(ctx, r, frombus.Mesh{}, frombus.Scope{Node: "n1"})
if err != nil || len(in.Refused) != 0 || !strings.Contains(in.SetAside["later"], "a-field-from-later") || in.Catalogue["later"].Stubbed {
t.Errorf("the bus reader before the judge: refused %v, set aside %v: %v", in.Refused, in.SetAside, err)
}
}
// judgedReader is mesh-sdk's bus reader over the indexer's modules and manifests on the bus, with good, bad and later placed
// on n1 and n1's record; values stand in for a module's value where set. It reads nothing else.
type judgedReader struct {
ctx context.Context
js jetstream.JetStream
t *testing.T
values map[string][]byte
unjudged [][]byte
}
func (r judgedReader) Placements(context.Context) (map[string]delivery.Placement, error) {
p := delivery.Placement{Node: "n1", Decided: delivery.NodeDecided{Hub: true, Address: "10.77.0.1"}}
for _, m := range []string{"good", "bad", "later"} {
p.Modules = append(p.Modules, delivery.PlacedModule{Module: m})
}
return map[string]delivery.Placement{"n1": p}, nil
}
func (r judgedReader) Records(context.Context) (map[string]delivery.NodeRecord, error) {
k, err := ecdh.X25519().GenerateKey(rand.Reader)
if err != nil {
return nil, err
}
return map[string]delivery.NodeRecord{"n1": {Node: "n1", TunnelKey: base64.StdEncoding.EncodeToString(k.PublicKey().Bytes()),
SealingKey: "sk", Capabilities: []string{"systemd"}}}, nil
}
func (r judgedReader) Modules(ctx context.Context) (map[string]delivery.Module, error) {
kv, err := r.js.KeyValue(ctx, delivery.ModulesBucket)
if err != nil {
return nil, err
}
out := map[string]delivery.Module{}
for _, name := range []string{"good", "bad", "later"} {
e, err := kv.Get(ctx, name)
if err != nil {
return nil, err
}
raw := e.Value()
if v, ok := r.values[name]; ok {
raw = v
}
m, err := delivery.DecodeModule(raw)
if err != nil {
return nil, err
}
out[name] = m
}
return out, nil
}
func (r judgedReader) Manifest(ctx context.Context, digest string) ([]byte, error) {
kv, err := r.js.KeyValue(ctx, delivery.ManifestsBucket)
if err != nil {
return nil, err
}
e, err := kv.Get(ctx, digest)
if err != nil {
return nil, err
}
m, err := delivery.DecodeManifest(e.Value())
return []byte(m.JSON), err
}
func (judgedReader) Values(context.Context, string, string) (map[string]any, error) { return nil, nil }
func (judgedReader) OwnSecrets(context.Context, string) ([]compose.OwnSecretRow, error) {
return nil, nil
}
func (judgedReader) Credentials(context.Context, string, []compose.CredentialKey, []compose.CredentialKey) ([]compose.Credential, []compose.Issued, error) {
return nil, nil, nil
}
func (judgedReader) Build(context.Context, string, string) (*delivery.BuilderBuild, error) {
return nil, nil
}
func (judgedReader) Archive(context.Context, string, string) ([]byte, error) { return nil, nil }
func (judgedReader) Served(context.Context, string, string, []string) (map[string]any, error) {
return nil, nil
}
// TestIndexerWritesAGoModuleAgainWhenTheBuildModuleReportsItsToolchain: the bucket mesh-build-go_toolchains exists (the state
@@ -291,7 +291,10 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
return err
}
defer calls.Close()
// The verdicts on manifests (ADR 0578 §1): a state this delivery brings, bound when it appears.
verdicts := process.AwaitBucket(ctx, js, conn, index.VerdictsBucket, "no verdict on a manifest is written", logger.Printf)
ix := &index.Indexer{Store: kvStore{candidates}, Modules: kvStore{modules}, Manifests: kvStore{manifests}, Taken: awaitedStore{taken}, Builds: known,
Verdicts: awaitedStore{verdicts}, SayRefused: sayRefused(js, cred, logger.Printf),
Predefined: awaitedStore{predefined}, ConfigCandidates: awaitedStore{configCandidates}, Logf: logger.Printf,
Named: namedVia(calls.Call), Trunk: trunkVia(calls.Call), Contains: containsVia(calls.Call), Read: readVia(calls.Call),
ReadOpen: openVia(calls.Call)}
@@ -385,6 +388,19 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
if err := ix.Reconcile(ctx); err != nil {
logger.Printf("dropping the modules the trunk does not hold at start: %v", err)
}
// Every manifest judged at the start, so a judge that changed with this program judges every one again (ADR 0578
// §1), once the verdicts state is there; each merge judges them again (index.ApplyMessage).
verdicts.Ready(func(jetstream.KeyValue) {
// A pass that fails (a read the bus did not answer) is judged again at the next message or source revision the
// indexer hears (index.kickJudge): no timer.
j, err := ix.JudgeAtStart(ctx)
if err != nil {
logger.Printf("judging the manifests at start: %v; judged again at the next message heard", err)
return
}
logger.Printf("judged %d manifests at start: %d verdicts written, %d with problems, of which outgrown and kept accepted: %v",
j.Judged, j.Written, len(j.Refused), j.Outgrown)
})
// A build module says each time it wrote its toolchains key: its stream, and the indexer's durable consumer on it, are
// made by the controller as the forge's are. Where the stream is not there yet the indexer says so and is started again
// with the build module, as with the gate's source.
@@ -0,0 +1,38 @@
package main
import (
"context"
"encoding/json"
"fmt"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/process"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
)
// sayRefused says the indexer's event manifest-refused (novox/hq ADR 0578 §1): mesh.mod.<module>.event.manifest-refused,
// with the SDK's envelope headers and the digest and judge as the bus's dedup id, so one refusal is said once.
func sayRefused(js jetstream.JetStream, cred process.Account, logf func(string, ...any)) func(context.Context, delivery.ManifestRefused) {
return func(ctx context.Context, r delivery.ManifestRefused) {
data, err := json.Marshal(r)
if err != nil {
logf("judge: %s for %s is not said: %v", delivery.ManifestRefusedEvent, r.Module, err)
return
}
msg := nats.NewMsg(cred.EventSubject(delivery.ManifestRefusedEvent))
id := "manifest-refused-" + r.Digest + "-" + r.Judge
msg.Header.Set("x-event-id", id)
msg.Header.Set("x-source", cred.Module)
msg.Header.Set("x-node", cred.Node)
msg.Header.Set("x-time", time.Now().UTC().Format(time.RFC3339Nano))
msg.Header.Set("content-type", "application/json")
msg.Data = data
pub, cancel := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
defer cancel()
if _, err := js.PublishMsg(pub, msg, jetstream.WithMsgID(id)); err != nil {
logf("judge: %s", fmt.Errorf("saying %s for %s: %w", delivery.ManifestRefusedEvent, r.Module, err))
}
}
}
@@ -0,0 +1,61 @@
// Command mesh-judge is the indexer's judge as a program the checker runs (novox/hq ADR 0578 §2): built at the trunk, or
// at a group's mesh-delivery head when that head changes the judge, and run by a Go build module's compose step in its
// sandbox exactly as the composer is (ADR 0482 §4; delivery.ComposeJob with this package). It reads a composer request
// (mesh-sdk composer.Request; `-request <file>`, or its standard input) and writes one line, a composer verdict, whose
// cases say of each manifest whether the judge accepts it and, when not, every problem. It reads nothing else: no bus,
// store, network or environment.
package main
import (
"encoding/json"
"flag"
"fmt"
"io"
"os"
"git.novox.be/novox/mesh-sdk/go/composer"
"mesh-delivery-indexer/judge"
)
// Package is this command's import path in the indexer's Go module (the directory delivery.ComposeJob's Dir names).
const Package = "mesh-delivery-indexer/cmd/mesh-judge"
func main() { os.Exit(run(os.Args[1:], os.Stdin, os.Stdout, os.Stderr)) }
func run(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
flags := flag.NewFlagSet("mesh-judge", flag.ContinueOnError)
flags.SetOutput(stderr)
file := flags.String("request", "", "the request's file; without it, the standard input")
if err := flags.Parse(args); err != nil {
return 2
}
var raw []byte
var err error
if *file != "" {
raw, err = os.ReadFile(*file)
} else {
raw, err = io.ReadAll(stdin)
}
if err != nil {
fmt.Fprintln(stderr, "mesh-judge:", err)
return 1
}
r, err := composer.Decode(raw)
if err != nil {
fmt.Fprintln(stderr, "mesh-judge:", err)
return 1
}
v, err := judge.Request(r)
if err != nil {
fmt.Fprintln(stderr, "mesh-judge:", err)
return 1
}
out, err := json.Marshal(v)
if err != nil {
fmt.Fprintln(stderr, "mesh-judge:", err)
return 1
}
fmt.Fprintln(stdout, string(out))
return 0
}
+1 -1
View File
@@ -3,7 +3,7 @@ module mesh-delivery-indexer
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005726-af4e4e60a3a0
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101020750-62cddf4efc62
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+12 -2
View File
@@ -1,7 +1,17 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031084708-acbf80ad910a h1:DcSvfEmdPBKqCg5OmXRNb269EkkPe7BsEp+RykEhkgE=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031084708-acbf80ad910a/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005726-af4e4e60a3a0 h1:q6ywchpTM9fwaJ1UNwDSuOjb5FYDFcsaeiTrKoB4scc=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005726-af4e4e60a3a0/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005736-7b09de1e5502 h1:XndvkzyAFNde/821lEfYObqMTURzmve/w9Ks+QrBBJ8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101005736-7b09de1e5502/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101012031-7cab079e2ec3 h1:an2f0MLx9OhaDLu5i51OMPBbuGwX+aoTzqOkFLMpVfE=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101012031-7cab079e2ec3/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101013319-25a2cecd989c h1:rlUCZ5ifLojqDUTkcCCcPrugTzV2RT6F6JbIVTJkaC8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101013319-25a2cecd989c/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014012-a20bdf795236 h1:JEkSnrmHs5QIRJGhtrghLC1+C0AxqMpef8a32d/BLPg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014012-a20bdf795236/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014310-19c204273bdc h1:kVq7sS4I0EnL8DcAA2a3YstnXS6WmYYzJz7eK7IZ9RY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101014310-19c204273bdc/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101020750-62cddf4efc62 h1:R5C8ICjiaDITHRFR/uIATwARPPXKxssUC70m0svstb0=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101020750-62cddf4efc62/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -0,0 +1,134 @@
// Package beside is where a test finds another repository of the mesh it reads: the catalogue's manifests,
// the node-engine's genesis template (novox/hq issue 432), the SDK's conformance fixtures (issue 449).
//
// Copied from mesh-controller internal/beside at commit 6c2a8d62 (R4b, issue 1379); the one change is that Dir
// lays out the captured copy of a repository once per process, not once per test (see laidOut). In mesh-sdk the
// catalogue's tests read mesh-catalog and mesh-host only; the captured copy holds just what they read.
//
// A test used to read the checkout beside this one, `../../../mesh-catalog`, so its verdict depended on
// whatever sat on the machine running it: a stale or dirty checkout failed it on a desktop, and where none
// was beside it skipped and said nothing. Now there are two inputs, both named, and no third:
//
// - In a merge check, the build seat clones each core repository beside the one checked (the catalogue
// at its main, the node-engine at the commit the mesh runs) and says where in MESH_CHECK_BESIDE. A
// test judges against those clones, so agreement with the other repository is checked where
// `mesh/repo-check` runs; a repository missing there fails the test, never skips it. Which clones a
// check gets is chosen from the inventory: mesh-catalog by the source of the `nats` module, mesh-host
// by the source of `mesh-host`, and mesh-sdk as the controller's sibling at the commit the controller's
// go.mod pins. Each repository's tests read these siblings, and the checker places exactly these
// (mesh-delivery's checker, besideRules, follows this list): mesh-controller's read mesh-catalog,
// mesh-host and mesh-sdk; mesh-host's read mesh-catalog, mesh-sdk and mesh-controller (its step-nine
// test reads the controller's own manifest, novox/hq issue 1295); mesh-sdk's read mesh-catalog and
// mesh-host; mesh-catalog's read mesh-sdk and mesh-controller; mesh-delivery's read mesh-sdk and
// mesh-catalog. In a mesh where either module has no source repository nothing is
// cloned, and these tests fail loudly with "not beside this check": a cause in the setup, not in the
// change. And in a delivery group that holds a mesh-catalog pull request, these tests read the
// catalogue's main, not the group's head.
// - Anywhere else, the test judges against the copy captured in this repository's testdata/beside, at the
// commit testdata/beside/CAPTURED names. Never against a developer's own checkout: to judge one, set
// MESH_CHECK_BESIDE to the directory holding it, as the check does.
//
// The captured manifests are named module.json.captured, and put back as module.json in a directory of
// the test's own: a module.json anywhere in this repository is a module of it to the forge's announcer and
// the planner, and a merge would build and register a hundred copies of the catalogue's.
package beside
import (
"io/fs"
"os"
"path/filepath"
"runtime"
"strings"
"sync"
"testing"
)
// Env is where a merge check says the repositories cloned beside the one checked are (internal/builder's
// EnvBeside, repeated here so a test does not import the builder).
const Env = "MESH_CHECK_BESIDE"
// CapturedSuffix is what a captured file's name carries that the repository's own does not.
const CapturedSuffix = ".captured"
// Dir is the named repository a test reads — the clone beside a merge check, or the captured copy — and
// says which in the test's log.
func Dir(t testing.TB, repository string) string {
t.Helper()
if root := os.Getenv(Env); root != "" {
dir := filepath.Join(root, repository)
if _, err := os.Stat(dir); err != nil {
t.Fatalf("%s is not beside this check in %s=%s, so its agreement with this repository cannot be "+
"judged here: %v", repository, Env, root, err)
}
t.Logf("judged against %s as cloned beside this check", dir)
return dir
}
from := filepath.Join(Captured(), repository)
if _, err := os.Stat(from); err != nil {
t.Fatalf("no captured copy of %s at %s: %v", repository, from, err)
}
dir, err := layOut(repository, from)
if err != nil {
t.Fatalf("the captured copy of %s could not be laid out: %v", repository, err)
}
t.Logf("judged against the copy of %s captured in testdata/beside (see CAPTURED); a merge check "+
"judges it against the repository's own clone", repository)
return dir
}
// laidOut is the captured copies already put back in place in this process, by repository. A copy is
// read-only to every test, so it is laid out once per process and not once per test: with a copy per test
// the catalogue's tests alone took 2.14 s, over mesh-sdk's unit bound of 2 s, and with one 0.81 s (measured
// 2026-10-20). The directories are in the temp directory, which the system clears; a process cannot remove
// them while another test may still read them.
var (
laidOutMu sync.Mutex
laidOut = map[string]string{}
)
func layOut(repository, from string) (string, error) {
laidOutMu.Lock()
defer laidOutMu.Unlock()
if dir, ok := laidOut[repository]; ok {
return dir, nil
}
root, err := os.MkdirTemp("", "beside-")
if err != nil {
return "", err
}
dir := filepath.Join(root, repository)
err = filepath.WalkDir(from, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
rel, _ := filepath.Rel(from, path)
into := filepath.Join(dir, strings.TrimSuffix(rel, CapturedSuffix))
if d.IsDir() {
return os.MkdirAll(into, 0o755)
}
raw, err := os.ReadFile(path)
if err != nil {
return err
}
return os.WriteFile(into, raw, 0o644)
})
if err != nil {
os.RemoveAll(root)
return "", err
}
laidOut[repository] = dir
return dir, nil
}
// Catalogue is the catalogue's modules directory, as Dir finds the catalogue.
func Catalogue(t testing.TB) string {
t.Helper()
return filepath.Join(Dir(t, "mesh-catalog"), "modules")
}
// Captured is this repository's testdata/beside, found from this file rather than from the working
// directory, so a test in any package reaches it.
func Captured() string {
_, file, _, _ := runtime.Caller(0)
return filepath.Join(filepath.Dir(file), "..", "..", "testdata", "beside")
}
@@ -52,6 +52,12 @@ type Indexer struct {
Predefined, ConfigCandidates Store
// Taken is the taken bucket: per repository the last merge taken in (taken.go, ADR 0556); nil writes none.
Taken Store
// Verdicts is the verdicts state (verdicts.go, novox/hq ADR 0578 §1); nil judges nothing.
Verdicts Store
// JudgeID names the judge a verdict says made it; empty is judge.Identity, the running program's.
JudgeID string
// SayRefused says the event manifest-refused for a refusal newly written; nil says nothing.
SayRefused func(ctx context.Context, r delivery.ManifestRefused)
// Now is the clock of a taken value; nil is time.Now.
Now func() time.Time
Builds *Builds
@@ -72,19 +78,20 @@ type Indexer struct {
// closed message the indexer missed heals at the next one (issue 1126); nil asks none.
ReadOpen func(ctx context.Context) ([]delivery.OpenPull, error)
mu sync.Mutex // guards source, toolchains and previews, which the watches and the forge messages all reach
source sourceNow
previews map[string]delivery.Preview // by previews key (previews.go)
toolchains map[string]map[string]delivery.Toolchain // language, then node (toolchains.go)
tried map[string]bool // repository, commit and directory a registered module's manifest was tried at (registered.go)
unread map[string]bool // registered modules whose last read the forge could not answer, read again by RetryUnread
retrying atomic.Bool // a RetryUnread runs in the background (kickRetry)
retryAgain atomic.Bool // an event asked for one more RetryUnread after the running one
retryWG sync.WaitGroup // the background RetryUnread, for tests to wait on
recompute sync.Mutex // one Recompute at a time
untaken map[string]string // repository to the latest merge applied while the taken state was missing (taken.go)
filling atomic.Bool // a FillTaken runs in the background (KickFill)
fillAgain atomic.Bool // a kick arrived while it ran
mu sync.Mutex // guards source, toolchains and previews, which the watches and the forge messages all reach
source sourceNow
previews map[string]delivery.Preview // by previews key (previews.go)
toolchains map[string]map[string]delivery.Toolchain // language, then node (toolchains.go)
tried map[string]bool // repository, commit and directory a registered module's manifest was tried at (registered.go)
unread map[string]bool // registered modules whose last read the forge could not answer, read again by RetryUnread
retrying atomic.Bool // a RetryUnread runs in the background (kickRetry)
retryAgain atomic.Bool // an event asked for one more RetryUnread after the running one
retryWG sync.WaitGroup // the background RetryUnread, for tests to wait on
recompute sync.Mutex // one Recompute at a time
untaken map[string]string // repository to the latest merge applied while the taken state was missing (taken.go)
filling atomic.Bool // a FillTaken runs in the background (KickFill)
fillAgain atomic.Bool // a kick arrived while it ran
judgePending atomic.Bool // the start's judge pass failed: judged again at the next message (verdicts.go)
}
func (ix *Indexer) logf(format string, args ...any) {
@@ -105,6 +112,7 @@ func (ix *Indexer) Apply(ctx context.Context, f delivery.Forge) error {
// ApplyMessage is Apply, and answers which modules a merged message wrote and kept (for the indexer's heard state). Each
// applied merged message is said in one line: its repository, commit and pull request, and the modules written.
func (ix *Indexer) ApplyMessage(ctx context.Context, f delivery.Forge) (Applied, error) {
ix.kickJudge(ctx)
var applied Applied
if f.Kind == delivery.ForgeOpen {
return applied, ix.applyOpen(ctx, f)
@@ -131,6 +139,11 @@ func (ix *Indexer) ApplyMessage(ctx context.Context, f delivery.Forge) (Applied,
if err := ix.configCandidatesAfterMerge(ctx, f, writtenModules(w.written)); err != nil {
return applied, err
}
// Every manifest judged again: a rule between manifests can turn on the one this merge moved (ADR 0578 §1).
// What this take-in newly wrote is fresh: judged in full (ADR 0579 §1).
if _, err := ix.JudgeAll(ctx, w.fresh); err != nil {
ix.logf("judge: the manifests are not judged after %s#%d: %v", f.Repository, f.Number, err)
}
}
ix.kickRetry(ctx) // a registered module the forge could not read is read again on a merge, in the background (issue 1945)
}
@@ -9,6 +9,8 @@ import (
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
"mesh-delivery-indexer/judge"
)
// The reasons a module has no fingerprint of the new form while the indexer writes none for any module (novox/hq
@@ -70,7 +72,7 @@ func inputsOf(module, raw, tree string, locks map[string]string, base baseFP) mo
// packages are the mesh packages of other repositories the module's go.sum names (novox/hq ADR 0573 §2): a module with a
// Go artifact keeps them, and the fingerprint takes the latest published version of each (Recompute).
func inputsOfNamed(module, raw, tree string, locks, objects map[string]string, packages []string, base baseFP) moduleInputs {
m, err := catalogue.ParseManifestForInputs([]byte(raw))
m, err := judge.ForInputs([]byte(raw))
if err != nil {
return moduleInputs{None2: None2Unreadable + err.Error()}
}
@@ -7,8 +7,9 @@ import (
"slices"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
"mesh-delivery-indexer/judge"
)
// applyMerged writes, for each module whose manifest the merged message carries (each directory of the merge commit that
@@ -154,7 +155,7 @@ func (ix *Indexer) namedObjects(ctx context.Context, f delivery.Forge) (map[stri
objects := map[string]map[string]string{}
need := map[string][]string{}
for dir, raw := range f.Manifests {
m, err := catalogue.ParseManifestForInputs([]byte(raw))
m, err := judge.ForInputs([]byte(raw))
if err != nil {
continue // inputsOfNamed says why
}
@@ -34,6 +34,16 @@ type mergeWrite struct {
late string // the taken merge this one is older than, or ""
asked string // the taken merge the order was asked against, or "" when none was asked
written []string
// fresh are the manifest digests this take-in newly wrote: a new module's, or a changed one's (ADR 0579 §1).
fresh map[string]bool
}
// taken marks a digest this take-in newly wrote.
func (w *mergeWrite) taken(digest string) {
if w.fresh == nil {
w.fresh = map[string]bool{}
}
w.fresh[digest] = true
}
func newMergeWrite(f delivery.Forge) *mergeWrite {
@@ -19,9 +19,7 @@ func TestAManifestFallingBackToAPredefinedKeyIsRead(t *testing.T) { // 1751
t.Errorf("%s: inputs = %+v, want the manifest read and its inputs kept", module, in)
}
}
}
func TestAPredefinedKeyOfAnotherTypeThanItsValueIsStillRefused(t *testing.T) { // 1751: the read judges what it can
// A predefined key of another type than its value is still refused: the read judges what it can.
raw := strings.Replace(fixture(t, "catalogue-835bf22b-xterm"), `"predefined": "font-size"`, `"predefined": "font-size", "trusted": "yes"`, 1)
if in := inputsOf("xterm", raw, "newtree", nil, func(string) (string, string) { return "", "none" }); in.Kept != nil {
t.Errorf("inputs = %+v, want a manifest the catalogue refuses still refused", in)
@@ -134,9 +134,14 @@ func (ix *Indexer) indexAt(ctx context.Context, r Registration, force, reread bo
if p := read.Packages[dir]; len(p) > 0 {
f.Packages = map[string][]string{dir: p}
}
if _, err := ix.applyManifests(ctx, f, false); err != nil {
w, err := ix.applyManifests(ctx, f, false)
if err != nil {
return t, err
}
// A module registered or taken in is judged as a merge's are (ADR 0578 §1, ADR 0579 §1).
if _, err := ix.JudgeAll(ctx, w.fresh); err != nil {
ix.logf("judge: the manifests are not judged after taking in %s: %v", r.Module, err)
}
if _, _, found, err := ix.Modules.Get(ctx, key); err != nil {
return t, fmt.Errorf("modules.%s: %w", key, err)
} else if !found {
@@ -37,6 +37,7 @@ func (ix *Indexer) srcOf(repository, merge string) uint64 {
// when it is written, applyMerged), and a repository none of the indexer's modules is built from writes nothing. The
// modules of a repository are those the modules bucket holds of it (ADR 0501 clause 6).
func (ix *Indexer) ApplySource(ctx context.Context, rev uint64, s delivery.Source) error {
ix.kickJudge(ctx)
ix.mu.Lock()
if rev < ix.source.rev {
ix.mu.Unlock()
@@ -10,6 +10,8 @@ import (
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
"mesh-delivery-indexer/judge"
)
// The new-form fingerprint of a module is written, and written again with no merge, when what it is of changes
@@ -254,7 +256,10 @@ func (ix *Indexer) recomputeModules(ctx context.Context, overlay map[string]deli
}
// The latest published version of each mesh package the module uses: the trunk merge of the module that holds it,
// as this recompute holds it, so a package's merge moves every dependent in the same recompute (ADR 0573 §2).
versions, _ := delivery.PackageVersionsOf(kept.Packages, held)
versions, unpublished := delivery.PackageVersionsOf(kept.Packages, held)
if len(unpublished) > 0 {
ix.logf("%s: no module the indexer holds publishes %s, so its fingerprint takes no version of it", module, strings.Join(unpublished, ", "))
}
var r result
switch {
case k.contexts:
@@ -346,10 +351,19 @@ func (ix *Indexer) writeOverlay(ctx context.Context, overlay, finals map[string]
wrote := false
err := write(ctx, ix.Modules, key, func(value []byte, found bool) ([]byte, bool, error) {
wrote = false
if found {
if cur, err := delivery.DecodeModule(value); err == nil && sameButWhere(cur, next) {
return nil, false, nil
}
next := next
cur, err := delivery.DecodeModule(value)
held := found && err == nil
if held && cur.Manifest == next.Manifest {
// The judge's verdict stands while the digest does (ADR 0578 §3, ADR 0579 §1): a recompute never writes a
// refused or outgrown module as accepted; only the judge changes it.
next.Verdict, next.Problems, next.Warnings = cur.Verdict, cur.Problems, cur.Warnings
}
if held && sameButWhere(cur, next) {
return nil, false, nil
}
if next.Manifest != "" && (!held || cur.Manifest != next.Manifest) {
w.taken(next.Manifest) // a new module, or a changed manifest: fresh, judged in full (ADR 0579 §1)
}
data, err := delivery.EncodeModule(next)
wrote = err == nil
@@ -365,10 +379,12 @@ func (ix *Indexer) writeOverlay(ctx context.Context, overlay, finals map[string]
return nil
}
// sameButWhere says the held value cur is next but for where it was computed (Commit and Src): the module is unchanged.
// sameButWhere says the held value cur is next but for where it was computed (Commit and Src) and the judge's verdict,
// which only the judge writes: the module is unchanged.
func sameButWhere(cur, next delivery.Module) bool {
next = withV(next)
next.Commit, next.Src = cur.Commit, cur.Src
next.Verdict, next.Problems, next.Warnings = cur.Verdict, cur.Problems, cur.Warnings
return reflect.DeepEqual(cur, next)
}
@@ -389,7 +405,7 @@ func (ix *Indexer) manifestRaw(ctx context.Context, digest string) (string, cata
if err != nil {
return "", catalogue.Manifest{}, err
}
m, err := catalogue.ParseManifestForInputs([]byte(kept.JSON))
m, err := judge.ForInputs([]byte(kept.JSON))
return kept.JSON, m, err
}
@@ -0,0 +1,228 @@
package index
import (
"context"
"encoding/json"
"errors"
"fmt"
"maps"
"slices"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/process"
"mesh-delivery-indexer/judge"
)
// The indexer judges every manifest it holds, once, with its own judge (novox/hq ADR 0578 §1): it writes each verdict in
// its state verdicts (delivery.VerdictsBucket), one key per manifest digest, only when the verdict differs from the one
// held; a module's value in the modules state carries a refusal (Verdict, Problems), so a reader of modules takes it
// without a second read; a new refusal is said as the event manifest-refused. It judges at its start, so a judge that
// changed (a new indexer) judges every manifest again, and after every merge it takes in, since a rule between manifests
// can turn on any of them. Nothing polls.
// VerdictsBucket is the indexer's state of verdicts.
var VerdictsBucket = delivery.VerdictsBucket
// Judged is what one pass of the judge did: the manifests judged, the verdicts written, and the modules refused.
type Judged struct {
Judged, Written int
Refused map[string][]string
// Outgrown are the modules accepted before whose problems are now warnings (ADR 0579 §1).
Outgrown []string
}
// JudgeAll judges every manifest the modules state names, as one catalogue with the predefined list the indexer keeps,
// and writes what differs. A manifest the manifests state does not hold is left unjudged and said; a write that fails is
// said and the pass goes on, so one key never costs the others their verdict.
//
// **A stricter judge never refuses what it accepted before** (novox/hq ADR 0579 §1): a manifest whose verdict under its
// digest is accepted, or which the indexer held before any verdict (taken as accepted, ADR 0578 §3) and is not in fresh,
// stays accepted, the new problems written as the verdict's Warnings and carried in its module's value, which the
// watchman raises as manifest-outgrown. A manifest in fresh (the digests a merge just took in) with no verdict yet, and
// one already refused, is judged in full.
func (ix *Indexer) JudgeAll(ctx context.Context, fresh map[string]bool) (Judged, error) {
var out Judged
if ix.Verdicts == nil || ix.Modules == nil || ix.Manifests == nil {
return out, nil
}
held, err := ix.heldModules(ctx)
if err != nil {
return out, err
}
raws := map[string][]byte{}
digests := map[string]string{}
for _, name := range slices.Sorted(maps.Keys(held)) {
digest := held[name].Manifest
if digest == "" {
continue
}
value, _, found, err := ix.Manifests.Get(ctx, digest)
if err != nil {
return out, fmt.Errorf("reading manifests.%s of %s: %w", digest, name, err)
}
if !found {
ix.logf("judge: the manifest of %s (digest %s) is not in the manifests state, so it is not judged", name, digest)
continue
}
kept, err := delivery.DecodeManifest(value)
if err != nil {
ix.logf("judge: manifests.%s of %s cannot be read, so it is not judged: %v", digest, name, err)
continue
}
raws[name] = []byte(kept.JSON)
digests[name] = digest
}
refused, unnamed := judge.Shelf(raws, ix.predefinedList(ctx), 0)
for _, p := range unnamed {
ix.logf("judge: a problem between the manifests that no one module's removal ends, so it refuses none: %s", p)
}
out.Refused = refused
id := ix.JudgeID
if id == "" {
id = judge.Identity()
}
for _, name := range slices.Sorted(maps.Keys(digests)) {
out.Judged++
v := delivery.ManifestVerdict{Module: name, State: delivery.VerdictAccepted, Judge: id, At: ix.now()}
if problems := refused[name]; len(problems) > 0 {
if ix.acceptedBefore(ctx, digests[name], fresh) {
v.Warnings = problems
out.Outgrown = append(out.Outgrown, name)
} else {
v.State, v.Problems = delivery.VerdictRefused, problems
}
}
wrote, err := ix.writeVerdict(ctx, digests[name], v)
if errors.Is(err, process.ErrNotYet) {
return out, fmt.Errorf("the state %s is not on the bus yet: %w", VerdictsBucket, err)
}
if err != nil {
ix.logf("judge: the verdict on %s (digest %s) is not written: %v", name, digests[name], err)
continue
}
if wrote {
out.Written++
if v.Refused() {
ix.logf("judge: %s", v.Why())
if ix.SayRefused != nil {
ix.SayRefused(ctx, delivery.RefusedOf(digests[name], v))
}
}
}
if err := ix.carryVerdict(ctx, name, digests[name], v); err != nil {
ix.logf("judge: modules.%s does not carry its verdict: %v", name, err)
}
}
return out, nil
}
// JudgeAtStart is JudgeAll of every manifest held, none fresh, as the indexer starts (a judge that changed judges every
// one again); one that fails is judged again at the next message or source revision heard (kickJudge).
func (ix *Indexer) JudgeAtStart(ctx context.Context) (Judged, error) {
j, err := ix.JudgeAll(ctx, nil)
ix.judgePending.Store(err != nil)
return j, err
}
// kickJudge judges again a start pass that failed, when the indexer hears something: never on a timer.
func (ix *Indexer) kickJudge(ctx context.Context) {
if !ix.judgePending.Load() {
return
}
if j, err := ix.JudgeAtStart(ctx); err != nil {
ix.logf("judge: the manifests are still not judged: %v", err)
} else {
ix.logf("judge: judged %d manifests again after a failed start: %d verdicts written", j.Judged, j.Written)
}
}
// acceptedBefore says whether the manifest of digest was accepted before this judgement: its verdict says accepted, or
// it has none and is not fresh (held before any verdict, so taken in under the judges that ran until then).
func (ix *Indexer) acceptedBefore(ctx context.Context, digest string, fresh map[string]bool) bool {
value, _, found, err := ix.Verdicts.Get(ctx, digest)
if err != nil {
return false // unread: judged in full, said by the write that follows
}
if !found {
return !fresh[digest]
}
prev, err := delivery.DecodeManifestVerdict(value)
return err == nil && !prev.Refused()
}
// writeVerdict writes v under the digest's key unless the verdict held there says the same (ManifestVerdict.Same); wrote
// says it was written.
func (ix *Indexer) writeVerdict(ctx context.Context, digest string, v delivery.ManifestVerdict) (wrote bool, err error) {
key, err := delivery.VerdictsKey(digest)
if err != nil {
return false, err
}
err = write(ctx, ix.Verdicts, key, func(value []byte, found bool) ([]byte, bool, error) {
wrote = false
if found {
if cur, err := delivery.DecodeManifestVerdict(value); err == nil && cur.Same(v) {
return nil, false, nil
}
}
wrote = true
next, err := delivery.EncodeManifestVerdict(v)
return next, false, err
})
return wrote, err
}
// carryVerdict puts a refusal into the module's value, and takes one out once the manifest is accepted, while the value
// still names the digest judged; an accepted module's value carries nothing, so judging writes no value of a module the
// judge accepts (a reader takes a value without a verdict as accepted, delivery.TakeManifestVerdict).
func (ix *Indexer) carryVerdict(ctx context.Context, name, digest string, v delivery.ManifestVerdict) error {
key, err := delivery.ModulesKey(name)
if err != nil {
return err
}
return write(ctx, ix.Modules, key, func(value []byte, found bool) ([]byte, bool, error) {
if !found {
return nil, false, nil
}
cur, err := delivery.DecodeModule(value)
if err != nil || cur.Manifest != digest {
return nil, false, nil // gone, unreadable, or moved to another manifest since: the next pass judges it
}
state := ""
if v.Refused() {
state = delivery.VerdictRefused
}
if cur.Verdict == state && slices.Equal(cur.Problems, v.Problems) && slices.Equal(cur.Warnings, v.Warnings) {
return nil, false, nil
}
cur.Verdict, cur.Problems, cur.Warnings = state, v.Problems, v.Warnings
if state == "" {
cur.Problems = nil
}
next, err := delivery.EncodeModule(cur)
return next, false, err
})
}
// predefinedList is the catalogue's predefined list the indexer keeps (writePredefined), or nil when it keeps none or
// it cannot be read: the judge then takes each fallback as its value declares it.
func (ix *Indexer) predefinedList(ctx context.Context) catalogue.PredefinedList {
if ix.Predefined == nil {
return nil
}
value, _, found, err := ix.Predefined.Get(ctx, delivery.PredefinedListKey)
if err != nil || !found {
return nil
}
var kept delivery.PredefinedKept
if json.Unmarshal(value, &kept) != nil {
return nil
}
list, err := catalogue.ParsePredefined(kept.List)
if err != nil {
ix.logf("judge: the predefined list kept cannot be read, so fallbacks are judged as their values declare them: %v", err)
return nil
}
return list
}
@@ -0,0 +1,126 @@
package index
import (
"context"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// The indexer judges every manifest it holds and writes each verdict only when it differs; a module's value carries a
// refusal, which is said once as manifest-refused; a new judge writes again only what it judges otherwise; a manifest
// mended clears the module's refusal (novox/hq ADR 0578 §1).
func TestTheIndexerJudgesEveryManifestOnce(t *testing.T) {
ctx := context.Background()
ix := &Indexer{Modules: newMem(), Manifests: newMem(), Verdicts: newMem(), JudgeID: "judge-1",
Now: func() time.Time { return time.Date(2026, 11, 1, 3, 0, 0, 0, time.UTC) }}
var said []delivery.ManifestRefused
ix.SayRefused = func(_ context.Context, r delivery.ManifestRefused) { said = append(said, r) }
hold := func(module, raw string) string {
digest, err := ix.keepManifest(ctx, module, raw)
if err != nil {
t.Fatal(err)
}
value, err := delivery.EncodeModule(delivery.Module{FP2: "src2:" + strings.Repeat("a", 64), Manifest: digest,
Repository: "novox/mesh-catalog", Commit: "c"})
if err != nil {
t.Fatal(err)
}
ix.Modules.(*memStore).put(module, value)
return digest
}
good := hold("good", `{"module":"good","version":"1"}`)
bad := hold("bad", `{"module":"bad","version":"1","requires":["amqp"]}`)
// Both taken in by a merge: judged in full.
j, err := ix.JudgeAll(ctx, map[string]bool{good: true, bad: true})
if err != nil || j.Judged != 2 || j.Written != 2 || len(j.Refused) != 1 {
t.Fatalf("the first pass: %+v %v", j, err)
}
verdict := func(digest string) delivery.ManifestVerdict {
value, _, found, _ := ix.Verdicts.Get(ctx, digest)
v, err := delivery.DecodeManifestVerdict(value)
if !found || err != nil {
t.Fatalf("no verdict under %s: %v", digest, err)
}
return v
}
if v := verdict(good); v.Refused() || v.Judge != "judge-1" {
t.Fatalf("good: %+v", v)
}
if v := verdict(bad); !v.Refused() || !strings.Contains(v.Problems[0], `requires "amqp"`) {
t.Fatalf("bad: %+v", v)
}
module := func(name string) delivery.Module {
value, _, _, _ := ix.Modules.Get(ctx, name)
m, err := delivery.DecodeModule(value)
if err != nil {
t.Fatal(err)
}
return m
}
if m := module("bad"); m.Verdict != delivery.VerdictRefused || len(m.Problems) != 1 {
t.Fatalf("modules.bad carries %+v", m)
}
if m := module("good"); m.Verdict != "" {
t.Fatalf("modules.good carries %q", m.Verdict)
}
if len(said) != 1 || said[0].Module != "bad" || said[0].Digest != bad || said[0].Judge != "judge-1" {
t.Fatalf("said %+v", said)
}
// A new judge that judges alike writes nothing and says nothing again.
ix.JudgeID = "judge-2"
if j, err := ix.JudgeAll(ctx, nil); err != nil || j.Written != 0 || len(said) != 1 {
t.Fatalf("a judge that judges alike: %+v %v, said %d", j, err, len(said))
}
// The manifest mended: its new digest is accepted and the module's refusal goes.
mended := hold("bad", `{"module":"bad","version":"2"}`)
if j, err := ix.JudgeAll(ctx, map[string]bool{mended: true}); err != nil || j.Written != 1 || len(j.Refused) != 0 {
t.Fatalf("mended: %+v %v", j, err)
}
if m := module("bad"); m.Verdict != "" || m.Problems != nil {
t.Fatalf("modules.bad still carries %+v", m)
}
// ADR 0579 §1: a manifest held before any verdict (taken in under the judges until then), which this judge finds a
// problem in, stays accepted with the problem as a warning, carried in its value and said nowhere as a refusal; the
// same problem in a manifest a merge takes in now, or in one whose digest changed, is refused in full.
old := hold("old", `{"module":"old","version":"1","requires":["amqp"]}`)
if j, err := ix.JudgeAll(ctx, nil); err != nil || len(j.Outgrown) != 1 || j.Outgrown[0] != "old" {
t.Fatalf("an outgrown manifest: %+v %v", j, err)
}
if v := verdict(old); v.Refused() || len(v.Warnings) != 1 || !strings.Contains(v.Warnings[0], `requires "amqp"`) {
t.Fatalf("old: %+v", v)
}
if m := module("old"); m.Verdict != "" || len(m.Warnings) != 1 || len(said) != 1 {
t.Fatalf("modules.old carries %+v, said %d", m, len(said))
}
changed := hold("old", `{"module":"old","version":"2","requires":["amqp"]}`)
fresh := hold("new", `{"module":"new","version":"1","requires":["amqp"]}`)
if j, err := ix.JudgeAll(ctx, map[string]bool{changed: true, fresh: true}); err != nil || len(j.Outgrown) != 0 {
t.Fatalf("a changed and a new manifest: %+v %v", j, err)
}
if !verdict(changed).Refused() || !verdict(fresh).Refused() || module("old").Verdict != delivery.VerdictRefused || len(module("old").Warnings) != 0 {
t.Fatalf("the changed and the new manifest are not refused: %+v %+v", verdict(changed), verdict(fresh))
}
// A merge's write of a module whose digest it leaves as it is carries the verdict over, and nothing is fresh; a changed
// digest is fresh, judged in full (review R1, R2).
w := &mergeWrite{}
same := delivery.Module{FP2: "src2:" + strings.Repeat("c", 64), Manifest: changed, Repository: "novox/mesh-catalog", Commit: "c2"}
if err := ix.writeOverlay(ctx, map[string]delivery.Module{"old": same}, nil, w); err != nil {
t.Fatal(err)
}
if m := module("old"); m.Verdict != delivery.VerdictRefused || len(m.Problems) == 0 || len(w.fresh) != 0 {
t.Fatalf("a merge rewrote a refused module as %+v, fresh %v", m, w.fresh)
}
other := strings.Repeat("d", 64)
same.Manifest = other
if err := ix.writeOverlay(ctx, map[string]delivery.Module{"old": same}, nil, w); err != nil || !w.fresh[other] {
t.Fatalf("a changed digest is not fresh: %v %v", w.fresh, err)
}
// Without the verdicts state the indexer judges nothing.
ix.Verdicts = nil
if j, err := ix.JudgeAll(ctx, nil); err != nil || j.Judged != 0 {
t.Fatalf("without the state: %+v %v", j, err)
}
}
@@ -0,0 +1,279 @@
// Package judge is the mesh's one judge of module manifests (novox/hq ADR 0578 §1, §2): strict decoding, the seat and
// namespace rules, the bus and stream limits, and the rules between the manifests of one catalogue. Only the indexer,
// which judges every manifest it takes in and writes the verdicts (delivery.VerdictsBucket), and the checker, which
// judges every manifest a change touches by running this judge built at the trunk or a group's head (cmd/mesh-judge,
// as the composer is run), use it. Every other part decodes a manifest leniently (catalogue.Decode) and takes the
// verdict; it never judges (ADR 0578 §3).
//
// Its rules are still mesh-sdk's catalogue functions it calls; they leave mesh-sdk for this package in the group's last
// mesh-sdk commit, once no reader imports them (ADR 0578 §7, issue 1585 S19). Callers depend only on what this package
// answers, so that move changes nothing for them.
package judge
import (
"crypto/sha256"
"encoding/hex"
json "encoding/json"
"fmt"
"io"
"maps"
"os"
filepath "path/filepath"
"regexp"
"slices"
"sort"
"strings"
"sync"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/composer"
)
// Manifest judges one manifest alone: decoded strictly, and every rule a manifest is held to by itself. The list of
// predefined keys is the one a config value falling back to a key is judged against (ADR 0348 §7); nil judges each
// fallback as its value declares it, where no list is at hand. Problems is empty when the manifest is accepted, and
// then m is the manifest as the judge read it.
func Manifest(raw []byte, predefined catalogue.PredefinedList) (m catalogue.Manifest, problems []string) {
var err error
if predefined != nil {
m, err = ParseManifestWith(raw, predefined)
} else {
m, err = ParseManifestForInputs(raw)
}
if err != nil {
return catalogue.Manifest{}, problemsOf(err)
}
return m, nil
}
// ForInputs is Manifest worded as the strict parser words a refusal: the indexer's read of a manifest for the inputs of its
// fingerprint, which keeps a refused manifest's reason in its value (None2) as it always has.
func ForInputs(raw []byte) (catalogue.Manifest, error) {
return ParseManifestForInputs(raw)
}
// listedRefusal is how the strict parser words a refusal of several problems, one per line after its header.
const listedRefusal = "this manifest cannot be used:\n - "
// problemsOf splits the strict parser's refusal into its problems, one each.
func problemsOf(err error) []string {
text := err.Error()
if rest, ok := strings.CutPrefix(text, listedRefusal); ok {
return strings.Split(rest, "\n - ")
}
return []string{text}
}
// Shelf judges every manifest of one catalogue, by the name it is given under (a module's name): each alone (Manifest),
// then the rules between the manifests accepted alone (Between). A problem between manifests is a problem of each module
// whose removal from the catalogue ends it (a rule two modules break together refuses both); one no removal ends is in
// unnamed, for the caller to say. A module not in the answer is accepted.
func Shelf(raws map[string][]byte, predefined catalogue.PredefinedList, longestMachine int) (refused map[string][]string, unnamed []string) {
refused = map[string][]string{}
shelf := catalogue.Shelf{}
for name, raw := range raws {
m, problems := Manifest(raw, predefined)
if len(problems) > 0 {
refused[name] = problems
continue
}
if m.Module != name {
refused[name] = []string{fmt.Sprintf("the manifest is of module %q, not %q", m.Module, name)}
continue
}
shelf[name] = m
}
between := Between(shelf, longestMachine)
without := map[string]map[string]bool{} // by module: the problems between the rest, once asked
ends := func(module, problem string) bool {
if without[module] == nil {
rest := maps.Clone(shelf)
delete(rest, module)
without[module] = map[string]bool{}
for _, p := range Between(rest, longestMachine) {
without[module][p] = true
}
}
return !without[module][problem]
}
for _, p := range between {
// The modules it names are asked first; only when none of them ends it is every module asked.
var of []string
for _, name := range Named(p, shelf) {
if ends(name, p) {
of = append(of, name)
}
}
if len(of) == 0 {
for _, name := range slices.Sorted(maps.Keys(shelf)) {
if ends(name, p) {
of = append(of, name)
}
}
}
if len(of) == 0 {
unnamed = append(unnamed, p)
continue
}
for _, name := range of {
refused[name] = append(refused[name], p)
}
}
for name := range refused {
sort.Strings(refused[name])
}
return refused, unnamed
}
// Between is every problem between the manifests of shelf, sorted: catalogue.CatalogueProblems, IdentityProblems over
// the longest machine name (catalogue.DefaultLongestMachine when not above 0) and DataProblems.
func Between(shelf catalogue.Shelf, longestMachine int) []string {
if longestMachine <= 0 {
longestMachine = catalogue.DefaultLongestMachine
}
between := CatalogueProblems(shelf)
between = append(between, IdentityProblems(shelf, longestMachine)...)
between = append(between, DataProblems(shelf)...)
sort.Strings(between)
return slices.Compact(between)
}
// nameChar is a character a module's name may hold; a name is named in a text only where none borders it.
var nameChar = regexp.MustCompile(`[a-z0-9.-]`)
// Named is every module of shelf a problem names, as a word of its own (not part of a longer name), sorted: the modules
// Shelf asks first whether their removal ends the problem. A module named only as a word ("mounts" in "f1-circuits
// mounts its directory") is not blamed for it, since its removal ends nothing.
func Named(problem string, shelf catalogue.Shelf) []string {
var out []string
for name := range shelf {
for at := 0; ; {
i := strings.Index(problem[at:], name)
if i < 0 {
break
}
i += at
end := i + len(name)
before := i == 0 || !nameChar.MatchString(problem[i-1:i])
after := end == len(problem) || !nameChar.MatchString(problem[end:end+1]) ||
// A sentence's full stop after a name ends it.
(problem[end] == '.' && (end+1 == len(problem) || problem[end+1] == ' '))
if before && after {
out = append(out, name)
break
}
at = i + 1
}
}
sort.Strings(out)
return out
}
// Unnamed names, in a request's case verdict, the problems between manifests that no one module's removal ends.
const Unnamed = "(between manifests)"
// Request judges every case of a composer request (mesh-sdk composer.Request), what cmd/mesh-judge answers the checker:
// each case's manifests as one catalogue (Shelf), every manifest accepted or refused with its problems, one per line; the
// problems no module's removal ends under Between. The verdict's SDK names this judge (Identity). A manifest left as a
// digest, or a case named twice, is a request it cannot run.
func Request(r composer.Request) (composer.Verdict, error) {
v := composer.Verdict{V: composer.Version, SDK: Identity()}
seen := map[string]bool{}
for _, c := range r.Cases {
if seen[c.Name] {
return composer.Verdict{}, fmt.Errorf("the case %q is named twice", c.Name)
}
seen[c.Name] = true
raws := map[string][]byte{}
for name, m := range c.Manifests {
if len(m.Raw) == 0 {
return composer.Verdict{}, fmt.Errorf("%w: %s (digest %s)", composer.ErrUnfilled, name, m.Digest)
}
raws[name] = m.Raw
}
refused, unnamed := Shelf(raws, r.Predefined, 0)
cv := composer.CaseVerdict{Name: c.Name, Manifests: map[string]composer.ManifestRead{}}
for name := range raws {
if problems := refused[name]; len(problems) > 0 {
cv.Manifests[name] = composer.ManifestRead{Refused: strings.Join(problems, "\n")}
continue
}
cv.Manifests[name] = composer.ManifestRead{Accepted: true}
}
if len(unnamed) > 0 {
cv.Manifests[Unnamed] = composer.ManifestRead{Refused: strings.Join(unnamed, "\n")}
}
v.Cases = append(v.Cases, cv)
}
return v, nil
}
var (
identityOnce sync.Once
identity string
)
// Identity names this judge: the sha256 of the running program, "exe:" and 16 hex digits, so a verdict says which judge
// made it and the indexer knows its judge changed when the program it runs does. "unknown" when the program cannot be
// read.
func Identity() string {
identityOnce.Do(func() {
identity = "unknown"
path, err := os.Executable()
if err != nil {
return
}
f, err := os.Open(path)
if err != nil {
return
}
defer f.Close()
h := sha256.New()
if _, err := io.Copy(h, f); err != nil {
return
}
identity = "exe:" + hex.EncodeToString(h.Sum(nil)[:8])
})
return identity
}
// WithTemplatesFrom is catalogue.WithTemplatesFrom judged: m with each template's text read from dir, then judged again
// as filled, against the list of predefined keys m was read with (a template's text can ask for what the manifest
// does not declare). The catalogue's reader only fills it (novox/hq ADR 0578 §3); judging the result is this judge's.
func WithTemplatesFrom(m catalogue.Manifest, dir string) (catalogue.Manifest, error) {
filled, err := catalogue.WithTemplatesFrom(m, dir)
if err != nil {
return catalogue.Manifest{}, err
}
// Judged as it composes without its build's templates archive, as the catalogue judged it before (template.go).
filled.Moved = false
raw, err := json.Marshal(filled)
if err != nil {
return catalogue.Manifest{}, err
}
again, err := parseManifest(raw, m.Predefined())
if err != nil {
return catalogue.Manifest{}, err
}
again.Moved = m.Moved
return again, nil
}
// ReadTreeManifest is the manifest in a module's directory of a repository tree judged as the merge gate judges it:
// strictly, against the list of predefined keys at the tree's root, then with each template's text read from its
// directory and judged again (WithTemplatesFrom).
func ReadTreeManifest(root, dir string) (catalogue.Manifest, error) {
list, err := catalogue.ReadPredefined(root)
if err != nil {
return catalogue.Manifest{}, err
}
raw, err := os.ReadFile(filepath.Join(dir, "module.json"))
if err != nil {
return catalogue.Manifest{}, err
}
m, err := ParseManifestWith(raw, list)
if err != nil {
return catalogue.Manifest{}, err
}
return WithTemplatesFrom(m, dir)
}
@@ -0,0 +1,93 @@
package judge
import (
"encoding/json"
"reflect"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/composer"
"git.novox.be/novox/mesh-sdk/go/overlay"
)
// The judge (novox/hq ADR 0578 §1): a manifest alone is accepted, or refused with each problem; an unknown field is
// refused (strict decoding); a rule between manifests refuses every module it names, and a word inside a longer name
// is no naming.
func TestTheJudge(t *testing.T) {
for _, row := range []struct {
name string
raws map[string]string
want map[string]string // module: a word of its first problem; absent: accepted
}{
{name: "a valid manifest", raws: map[string]string{"a": `{"module":"a","version":"1"}`}},
{name: "a refused manifest", raws: map[string]string{"a": `{"module":"a","version":"1","provides":[{"name":"amqp"}]}`},
want: map[string]string{"a": `provides "amqp", which is not a provision`}},
{name: "a provision named as a credential's", raws: map[string]string{"a": `{"module":"a","version":"1","provides":[{"name":"secret-db","scope":"mesh"}]}`},
want: map[string]string{"a": `secret-`}},
{name: "an unknown field", raws: map[string]string{"a": `{"module":"a","version":"1","a-field-from-later":1}`},
want: map[string]string{"a": `"a-field-from-later"`}},
{name: "a manifest under another name", raws: map[string]string{"b": `{"module":"a","version":"1"}`},
want: map[string]string{"b": `of module "a", not "b"`}},
{name: "a seat declared twice",
raws: map[string]string{
"one": `{"module":"one","version":"1","seats":[{"name":"thing","scope":"mesh"}]}`,
"two": `{"module":"two","version":"1","seats":[{"name":"thing","scope":"mesh"}]}`,
"ok": `{"module":"ok","version":"1"}`},
want: map[string]string{"one": "already declares", "two": "already declares"}},
} {
raws := map[string][]byte{}
for k, v := range row.raws {
raws[k] = []byte(v)
}
refused, unnamed := Shelf(raws, nil, 0)
if len(unnamed) > 0 {
t.Errorf("%s: unnamed %v", row.name, unnamed)
}
if len(refused) != len(row.want) {
t.Errorf("%s: refused %v, want %v", row.name, refused, row.want)
continue
}
for m, word := range row.want {
if len(refused[m]) == 0 || !strings.Contains(strings.Join(refused[m], "\n"), word) {
t.Errorf("%s: %s refused with %v, want %q", row.name, m, refused[m], word)
}
}
}
// The private network's manifest, which the mesh provides itself, is accepted (moved from mesh-sdk overlay's test).
if raw, err := json.Marshal(overlay.Manifest()); err != nil {
t.Fatal(err)
} else if _, problems := Manifest(raw, nil); len(problems) > 0 {
t.Errorf("the provided private network is refused: %v", problems)
}
shelf := catalogue.Shelf{"nats": {}, "mesh-nats": {}, "a.b": {}}
for problem, want := range map[string][]string{
"nats removes a stream": {"nats"},
"mesh-nats reads nats.": {"mesh-nats", "nats"},
"the state nats.modules and mesh-natsy": nil,
"a.b and a.bc": {"a.b"},
`"nats" is quoted, (mesh-nats) bracketed`: {"mesh-nats", "nats"},
} {
if got := Named(problem, shelf); !reflect.DeepEqual(got, want) {
t.Errorf("Named(%q) = %v, want %v", problem, got, want)
}
}
if id := Identity(); !strings.HasPrefix(id, "exe:") || id != Identity() {
t.Errorf("identity %q", id)
}
// What cmd/mesh-judge answers the checker (ADR 0578 §2): a composer request in, a verdict out, each manifest accepted
// or refused; a manifest left as a digest is a request it cannot run.
r := composer.Request{V: composer.Version, Cases: []composer.Case{{Name: "change", Manifests: map[string]composer.Manifest{
"a": {Raw: json.RawMessage(`{"module":"a","version":"1"}`)},
"b": {Raw: json.RawMessage(`{"module":"b","version":"1","requires":["amqp"]}`)},
}}}}
v, err := Request(r)
if err != nil || v.V != composer.Version || len(v.Cases) != 1 || v.SDK != Identity() || !v.Cases[0].Manifests["a"].Accepted ||
v.Cases[0].Manifests["b"].Accepted || !strings.Contains(v.Cases[0].Manifests["b"].Refused, `requires "amqp"`) {
t.Fatalf("request: %+v %v", v, err)
}
r.Cases[0].Manifests["c"] = composer.Manifest{Digest: strings.Repeat("a", 64)}
if _, err := Request(r); err == nil {
t.Fatal("a manifest left as a digest was judged")
}
}
@@ -0,0 +1 @@
These tests read files outside this directory: the catalogue and the node-engine captured in the module's testdata/beside (internal/beside).
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,249 @@
{
"module": "claude-code",
"version": "1",
"moved": true,
"slug": "agent",
"capabilities": [
"package-manager"
],
"requires": [
"mcp-endpoint",
{
"name": "gitea-read",
"optional": true
}
],
"binds": {
"mcp-endpoint": "${dir:state}/mcp-endpoint.json"
},
"secrets": {
"gitea-read": "${dir:state}/gitea-read.secret"
},
"uses": [
"operator-channel"
],
"state": [
"servers",
"holdings",
"config",
"proposals",
"questions",
"memory",
"waits"
],
"reads": [
"claude-licence-manager.bindings"
],
"tools": [
"claude_code_status",
"claude_code_render",
"claude_code_guard",
"claude_code_pull",
"claude_code_grant",
"claude_code_add_api_key",
"claude_code_registrations",
"claude_code_memory",
"claude_code_mcp_list",
"claude_code_mcp_register",
"claude_code_mcp_unregister",
"claude_code_skill_list",
"claude_code_skill_register",
"claude_code_skill_unregister",
"claude_code_agent_list",
"claude_code_agent_register",
"claude_code_agent_unregister",
"claude_code_command_list",
"claude_code_command_register",
"claude_code_command_unregister",
"claude_code_hook_list",
"claude_code_hook_register",
"claude_code_hook_unregister",
"claude_code_output_style_list",
"claude_code_output_style_register",
"claude_code_output_style_unregister",
"claude_code_instructions_list",
"claude_code_instructions_register",
"claude_code_instructions_unregister",
"claude_code_instructions_propose",
"claude_code_proposals",
"claude_code_ask",
"claude_code_answer",
"claude_code_wait",
"claude_code_settings_get",
"claude_code_settings_set",
"claude_code_settings_clear",
"claude_code_permission_add",
"claude_code_permission_remove",
"claude_code_config_list",
"claude_code_config_show",
"claude_code_config_status",
"claude_code_config_import",
"claude_code_home_show",
"claude_code_home_remove",
"claude_code_home_removed",
"claude_code_home_restore",
"claude_code_judge",
"claude_code_judged"
],
"data": {
"own": [
{
"id": "agent-home",
"path": "${dir:agent-home}",
"class": "valuable",
"why": "the operator's agent's own files: its memory, history and projects"
}
]
},
"config": {
"role": {
"type": "string",
"description": "What this machine is for, in a sentence, written into the instructions every Claude Code session on it reads; empty says nothing.",
"trusted": true,
"default": ""
},
"mcp_servers": {
"type": "object",
"description": "The tool servers every session on this machine is given beside the mesh's own, by name, each as Claude Code's managed MCP configuration takes it; empty adds none.",
"trusted": true,
"default": {}
},
"managed_settings": {
"type": "object",
"description": "Keys of Claude Code's managed settings every session on this machine obeys, such as spinnerTipsEnabled, laid over the mesh's own; empty adds none. Falls back to agent-managed-settings, the mesh's or this node's.",
"trusted": true,
"predefined": "agent-managed-settings",
"default": {}
},
"instructions": {
"type": "object",
"description": "The instructions every session on this machine reads, by name, each a text as one line or a list of its lines; empty adds none.",
"trusted": true,
"default": {}
},
"output_styles": {
"type": "object",
"description": "The output styles offered to every session on this machine, by name, each a text as one line or a list of its lines; empty adds none.",
"trusted": true,
"default": {}
},
"skills": {
"type": "object",
"description": "The skills offered to every session on this machine, by name, each its files by path, a text as one line or a list of its lines; a skill without its SKILL.md is left out; empty adds none.",
"trusted": true,
"default": {}
},
"commands": {
"type": "object",
"description": "The slash commands offered to every session on this machine, by name, each a text as one line or a list of its lines; empty adds none.",
"trusted": true,
"default": {}
},
"agents": {
"type": "object",
"description": "The agents offered to every session on this machine, by name, each a text as one line or a list of its lines; empty adds none.",
"trusted": true,
"default": {}
}
},
"resources": [
{
"id": "package",
"type": "package",
"package": "claude-code"
},
{
"id": "managed",
"type": "directory",
"path": "/etc/claude-code",
"mode": "0755",
"unknown": "program"
},
{
"id": "start",
"type": "file",
"path": "/usr/local/bin/claude-agent",
"mode": "0755",
"template": "files/claude-agent"
},
{
"id": "agent-account-name",
"type": "file",
"path": "/etc/claude-code/agent-account",
"mode": "0644",
"template": "files/agent-account"
},
{
"id": "agent-account",
"type": "user",
"name": "${machine:agent-account}",
"home": "${machine:agent-home}",
"root": "${machine:agent-root}"
},
{
"id": "agent-home",
"type": "directory",
"path": "${machine:agent-home}/.claude",
"mode": "0700",
"owner": "${machine:agent-account}",
"unknown": "module"
},
{
"id": "state",
"type": "directory",
"mode": "0700",
"owner": "${machine:account}",
"place": ".",
"unknown": "program"
},
{
"id": "facts",
"type": "file",
"path": "${dir:state}/facts.json",
"mode": "0600",
"owner": "${machine:account}",
"template": "files/facts.json"
},
{
"id": "settings",
"type": "file",
"path": "${dir:state}/settings.json",
"mode": "0600",
"owner": "${machine:account}",
"takes-config": ["role", "mcp_servers", "managed_settings", "instructions", "output_styles", "skills", "commands", "agents"],
"merge": "json",
"template": "files/settings.json"
}
],
"items": [
{
"collection": "shell:zsh:normal",
"template": "files/zsh-normal"
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/claude-code",
"binary": "claude-code",
"loads": [
"claude-code"
],
"env": {
"MESH_CLAUDE_CODE_STATE": "${dir:state}",
"MESH_CLAUDE_CODE_FACTS": "${dir:state}/facts.json",
"MESH_CLAUDE_CODE_SETTINGS": "${dir:state}/settings.json"
}
},
{
"name": "files",
"kind": "archive",
"from": "files"
}
]
}
}
@@ -0,0 +1,123 @@
{
"module": "dunst",
"version": "1",
"moved": true,
"capabilities": [
"package-manager"
],
"claims": [
{
"name": "node-notifier",
"scope": "node",
"serves": [
"send",
"history"
]
}
],
"tools": [
"dunst_pause",
"dunst_resume",
"dunst_close_all",
"dunst_rules",
"dunst_count",
"dunst_reload"
],
"config": {
"font-size": {
"type": "number",
"description": "The size of a notification's text, in points of Inter: 10 reads well on a screen of about 100 DPI, and a denser screen needs a larger size.",
"minimum": 6,
"maximum": 72,
"predefined": "font-size",
"default": 10
},
"width": {
"type": "number",
"description": "The width of a notification, in pixels: 250 fits a title of about forty characters at 10 points, and a larger font needs a wider notification.",
"minimum": 50,
"maximum": 4000,
"default": 250
}
},
"resources": [
{
"id": "package",
"type": "package",
"package": "dunst"
},
{
"id": "client",
"type": "package",
"package": "libnotify"
},
{
"id": "configuration-dir",
"type": "directory",
"path": "${machine:account-home}/.config/dunst",
"owner": "${machine:account}",
"mode": "0755",
"unknown": "move-out"
},
{
"id": "dropins",
"type": "directory",
"path": "${machine:account-home}/.config/dunst/dunstrc.d",
"owner": "${machine:account}",
"mode": "0755",
"unknown": "move-out",
"knows": ["50-slack.conf"]
},
{
"id": "configuration",
"trusted": false,
"type": "file",
"path": "${machine:account-home}/.config/dunst/dunstrc",
"owner": "${machine:account}",
"mode": "0644",
"template": "files/dunstrc"
},
{
"id": "notifier",
"type": "service",
"unit": "dunst.service",
"scope": "user",
"user": "${machine:account}",
"reload-on": [
"configuration"
]
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/dunst-tools",
"binary": "dunst-tools",
"loads": [
"dunst-tools"
]
},
{
"name": "files",
"kind": "archive",
"from": "files"
}
]
},
"emits": [
"action-chosen",
"started",
"paused"
],
"items": [
{
"collection": "contribution:node-display-session:config",
"name": "display-session",
"template": "files/i3/50-dunst.conf"
}
]
}
@@ -0,0 +1,236 @@
{
"module": "xterm",
"version": "1",
"moved": true,
"capabilities": [
"package-manager"
],
"requires": [
"x11-display"
],
"claims": [
{
"name": "node-terminal-emulator",
"scope": "node",
"serves": [
"open"
]
}
],
"tools": [
"xterm_font",
"xterm_colours"
],
"config": {
"face": {
"type": "string",
"description": "The font family xterm draws with, as fc-list names it: JetBrainsMono Nerd Font, because research 026/04 chose it for its glyphs the prompt uses. A family the machine does not have falls back to the nearest installed face (xterm_font says which).",
"pattern": "[A-Za-z0-9 ._-]{1,64}",
"default": "JetBrainsMono Nerd Font"
},
"size": {
"type": "number",
"description": "The size of xterm's text, in points: 12 reads well on a screen of about 100 DPI, and a denser screen needs a larger size.",
"minimum": 6,
"maximum": 72,
"predefined": "font-size",
"default": 12
},
"save-lines": {
"type": "number",
"description": "How many lines of scrollback xterm keeps per terminal: 10000 holds a long build log, and each line costs memory in every open terminal.",
"minimum": 0,
"maximum": 1000000,
"default": 10000
},
"term-name": {
"type": "string",
"description": "The TERM value xterm sets for the programs it runs: xterm-256color, because the palette has 256 colours and every program knows the name.",
"pattern": "[A-Za-z0-9._-]{1,64}",
"default": "xterm-256color"
},
"render-font": {
"type": "boolean",
"description": "Whether xterm draws with fontconfig faces (true) or the old X core fonts (false): true, because the face is a fontconfig family and core fonts cannot show it.",
"default": true
},
"bold-mode": {
"type": "boolean",
"description": "Whether xterm draws bold text by overstriking the normal face (true): false, because the face has a real bold style and overstriking blurs it.",
"default": false
},
"always-bold-mode": {
"type": "boolean",
"description": "Whether xterm overstrikes bold even where the face has a bold style (true): false, for the same reason as bold-mode.",
"default": false
},
"allow-bold-fonts": {
"type": "boolean",
"description": "Whether xterm may use the face's bold style for bold text (true): true, so bold text is drawn with the real bold weight.",
"default": true
},
"background": {
"type": "string",
"description": "The terminal's background colour, as #rrggbb: black, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#000000"
},
"foreground": {
"type": "string",
"description": "The terminal's text colour, as #rrggbb: a light grey, easier on the eyes than white on black.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#a8a8a8"
},
"color0": {
"type": "string",
"description": "Palette entry 0, the normal black, as #rrggbb: #000000, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#000000"
},
"color1": {
"type": "string",
"description": "Palette entry 1, the normal red, as #rrggbb: #9e1828, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#9e1828"
},
"color2": {
"type": "string",
"description": "Palette entry 2, the normal green, as #rrggbb: #008800, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#008800"
},
"color3": {
"type": "string",
"description": "Palette entry 3, the normal yellow, as #rrggbb: #d2bb4b, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#d2bb4b"
},
"color4": {
"type": "string",
"description": "Palette entry 4, the normal blue, as #rrggbb: #414171, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#414171"
},
"color5": {
"type": "string",
"description": "Palette entry 5, the normal magenta, as #rrggbb: #963c59, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#963c59"
},
"color6": {
"type": "string",
"description": "Palette entry 6, the normal cyan, as #rrggbb: #418179, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#418179"
},
"color7": {
"type": "string",
"description": "Palette entry 7, the normal white, as #rrggbb: #bebebe, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#bebebe"
},
"color8": {
"type": "string",
"description": "Palette entry 8, the bright black, as #rrggbb: #666666, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#666666"
},
"color9": {
"type": "string",
"description": "Palette entry 9, the bright red, as #rrggbb: #bc5766, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#bc5766"
},
"color10": {
"type": "string",
"description": "Palette entry 10, the bright green, as #rrggbb: #61a171, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#61a171"
},
"color11": {
"type": "string",
"description": "Palette entry 11, the bright yellow, as #rrggbb: #e7db52, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#e7db52"
},
"color12": {
"type": "string",
"description": "Palette entry 12, the bright blue, as #rrggbb: #5085af, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#5085af"
},
"color13": {
"type": "string",
"description": "Palette entry 13, the bright magenta, as #rrggbb: #a97a99, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#a97a99"
},
"color14": {
"type": "string",
"description": "Palette entry 14, the bright cyan, as #rrggbb: #6ba4a4, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#6ba4a4"
},
"color15": {
"type": "string",
"description": "Palette entry 15, the bright white, as #rrggbb: #ffffff, as both workstations ran it.",
"pattern": "#[0-9a-fA-F]{6}",
"default": "#ffffff"
},
"copy-binding": {
"type": "string",
"description": "The key combination that copies the selection to the clipboard, in X translation syntax: Ctrl Shift ~Alt <Key> C, so a plain Ctrl+C still interrupts the program. It holds no colon, backslash or line break, which would end the resource.",
"pattern": "[^:\\\\\\n]+",
"default": "Ctrl Shift ~Alt <Key> C"
},
"paste-binding": {
"type": "string",
"description": "The key combination that pastes the clipboard, in X translation syntax: Ctrl Shift ~Alt <Key> V, the pair of copy-binding. It holds no colon, backslash or line break, which would end the resource.",
"pattern": "[^:\\\\\\n]+",
"default": "Ctrl Shift ~Alt <Key> V"
}
},
"items": [
{
"collection": "environment",
"name": "terminal",
"attributes": {
"kind": "variable",
"name": "TERMINAL"
},
"template": "files/env-terminal"
},
{
"collection": "shell:xresources:normal",
"name": "xresources-normal",
"template": "files/xresources-normal"
}
],
"resources": [
{
"id": "package",
"type": "package",
"package": "xterm"
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/xterm-tools",
"binary": "xterm-tools",
"loads": [
"xterm-tools"
]
},
{
"name": "files",
"kind": "archive",
"from": "files"
}
]
}
}
@@ -81,12 +81,13 @@ func TestManifestDeclarations(t *testing.T) { // 1238/AC1, 1444 (reads the build
if m.Module != "mesh-delivery-indexer" {
t.Errorf("module = %q", m.Module)
}
// 1126: placement is the seat mesh-placement's (ADR 0567): the indexer holds no placement state and says no event.
if len(m.Emits) != 0 {
t.Errorf("emits = %v, want none", m.Emits)
// 1126: placement is the seat mesh-placement's (ADR 0567): the indexer holds no placement state. ADR 0578 §1: it says
// a refused manifest as manifest-refused, its one event.
if !reflect.DeepEqual(m.Emits, []string{"manifest-refused"}) {
t.Errorf("emits = %v, want manifest-refused alone", m.Emits)
}
// 1126 note 76: heard keeps the last 64 FORGE messages the indexer handled, for indexer_heard.
if want := []state{{Name: "candidates", History: 10}, {Name: "heard", History: 64}, {Name: "manifests", History: 1}, {Name: "modules", History: 20}, {Name: "predefined", History: 5}, {Name: "config-candidates", History: 5}, {Name: "taken", History: 1}, {Name: "takeins", History: 1}}; !reflect.DeepEqual(m.State, want) {
if want := []state{{Name: "candidates", History: 10}, {Name: "heard", History: 64}, {Name: "manifests", History: 1}, {Name: "modules", History: 20}, {Name: "predefined", History: 5}, {Name: "config-candidates", History: 5}, {Name: "taken", History: 1}, {Name: "takeins", History: 1}, {Name: "verdicts", History: 5}}; !reflect.DeepEqual(m.State, want) {
t.Errorf("state = %+v, want %+v", m.State, want)
}
if want := []string{"mesh-build-go.toolchains", "mesh-build-node.toolchains", "mesh-delivery-assigner.placement", "mesh-delivery-builder.build-log", "mesh-delivery-checker.previews", "mesh-delivery-gate.source"}; !reflect.DeepEqual(sorted(m.Reads), want) {
+5 -1
View File
@@ -19,7 +19,11 @@
{ "name": "predefined", "history": 5 },
{ "name": "config-candidates", "history": 5 },
{ "name": "taken", "history": 1 },
{ "name": "takeins", "history": 1 }
{ "name": "takeins", "history": 1 },
{ "name": "verdicts", "history": 5 }
],
"emits": [
"manifest-refused"
],
"reads": [
"mesh-build-go.toolchains",
+27
View File
@@ -0,0 +1,27 @@
What a test reads of another repository when no merge check has cloned it beside this one (internal/beside,
novox/hq issue 432). Copied from the repositories at the commits below, never written by hand. Each
module.json is kept as module.json.captured: a module.json in this repository is a module of it to the
forge and the planner, and a merge would build and register it.
The copy carries the catalogue's private details: domains, first names in module names, a node name and
private addresses. That is acceptable while mesh-sdk is private; if it ever goes public, this copy is
a second place to clean besides the catalogue itself.
Only what go/catalogue's tests read is captured (R4b, issue 1379). Refreshed 2026-10-21 from the mains of both
repositories; before that it was mesh-controller's copy, which still held two modules that had left the catalogue.
mesh-catalog b5dbfbb784507e211372a7c9640d91f7d6dadd84 modules/*/module.json, modules/*/files/**, modules/*/templates/**, predefined.json
mesh-host cdcba9ca73755854d21e4e657d7e46c32b9b5265 internal/apply/placement_guard.go
To move them, from this repository's go/ directory, with the two repositories checked out beside mesh-sdk:
rm -rf testdata/beside/mesh-catalog testdata/beside/mesh-host
mkdir -p testdata/beside/mesh-catalog testdata/beside/mesh-host/internal/apply
git -C ../../mesh-catalog archive <commit> -- predefined.json ':(glob)modules/*/module.json' \
':(glob)modules/*/files/**' ':(glob)modules/*/templates/**' | tar -x -C testdata/beside/mesh-catalog
find testdata/beside -name module.json -exec mv {} {}.captured \;
git -C ../../mesh-host show <commit>:internal/apply/placement_guard.go \
> testdata/beside/mesh-host/internal/apply/placement_guard.go.captured
and write the commits here. The placement guard is kept as .captured so no Go tool reads it as this
repository's code.
@@ -0,0 +1,5 @@
# Written by the mesh (module adwaita, novox/hq ADR 0208): the default cursor theme, for programs that
# read neither XCURSOR_THEME nor the X resources.
[Icon Theme]
Name=Default
Inherits=Adwaita
@@ -0,0 +1 @@
Adwaita:dark
@@ -0,0 +1 @@
/usr/share/themes/Adwaita-dark/gtk-2.0/gtkrc
@@ -0,0 +1,9 @@
# Written by the mesh (module adwaita, novox/hq ADR 0208), for GTK 3 and GTK 4 alike. Replaced at
# every push; adwaita_appearance switches dark and light for the running session.
[Settings]
gtk-theme-name=Adwaita
gtk-icon-theme-name=Adwaita
gtk-cursor-theme-name=Adwaita
gtk-cursor-theme-size=24
gtk-font-name=Inter 11
gtk-application-prefer-dark-theme=1
@@ -0,0 +1,11 @@
# Written by the mesh (module adwaita, novox/hq ADR 0208). Replaced at every push.
#
# Which portal backend answers each interface. i3 is not a desktop xdg-desktop-portal knows, so with
# no preference it uses whichever backend happens to be installed: fine while gtk is the only one,
# wrong the day another arrives as somebody else's dependency. Named instead. gtk also serves
# org.freedesktop.appearance (dark or light) from GSettings, which the session's start sets.
[preferred]
default=gtk
# Secrets for sandboxed programs come from the keyring's backend. Without this line no backend answers
# the interface: gtk does not implement it, and gnome-keyring's names only GNOME as its desktop.
org.freedesktop.impl.portal.Secret=gnome-keyring
@@ -0,0 +1,29 @@
[Appearance]
color_scheme_path=/usr/share/qt6ct/colors/darker.conf
custom_palette=true
icon_theme=Adwaita
standard_dialogs=default
style=Fusion
[Fonts]
fixed="JetBrainsMono Nerd Font,11,-1,5,50,0,0,0,0,0"
general="Inter,11,-1,5,50,0,0,0,0,0"
[Interface]
activate_item_on_single_click=1
buttonbox_layout=0
cursor_flash_time=1000
dialog_buttons_have_icons=1
double_click_interval=400
gui_effects=@Invalid()
keyboard_scheme=2
menus_have_icons=true
show_shortcuts_in_context_menus=true
stylesheets=@Invalid()
toolbutton_style=4
underline_shortcut=1
wheel_scroll_lines=3
[Troubleshooting]
force_raster_widgets=1
ignored_applications=@Invalid()
@@ -0,0 +1,10 @@
# The appearance (module adwaita): GSettings is where the portal reads dark or light, and the portal is
# the only way it reaches Electron, Chromium, Firefox and flatpaks. Set at every session start to the
# module's default; adwaita_appearance switches it for a session.
gsettings set org.gnome.desktop.interface color-scheme 'prefer-dark' || true
gsettings set org.gnome.desktop.interface gtk-theme 'Adwaita' || true
gsettings set org.gnome.desktop.interface icon-theme 'Adwaita' || true
gsettings set org.gnome.desktop.interface cursor-theme 'Adwaita' || true
gsettings set org.gnome.desktop.interface cursor-size 24 || true
gsettings set org.gnome.desktop.interface font-name 'Inter 11' || true
gsettings set org.gnome.desktop.interface monospace-font-name 'JetBrainsMono Nerd Font 11' || true
@@ -0,0 +1,3 @@
! adwaita: the cursor, for X programs that take it from the resources.
Xcursor.theme: Adwaita
Xcursor.size: 24
@@ -0,0 +1,176 @@
{
"module": "adwaita",
"version": "1",
"moved": true,
"capabilities": [
"package-manager"
],
"tools": [
"adwaita_appearance",
"adwaita_cursor",
"adwaita_icons",
"adwaita_portal_check"
],
"items": [
{
"collection": "environment",
"name": "gtk-theme",
"attributes": {
"kind": "variable",
"name": "GTK_THEME"
},
"template": "files/env-gtk-theme"
},
{
"collection": "environment",
"name": "gtk2-rc-files",
"attributes": {
"kind": "variable",
"name": "GTK2_RC_FILES"
},
"template": "files/env-gtk2-rc-files"
},
{
"collection": "environment",
"name": "qt-qpa-platformtheme",
"attributes": {
"kind": "variable",
"name": "QT_QPA_PLATFORMTHEME"
},
"template": "files/env-qt-qpa-platformtheme"
},
{
"collection": "environment",
"name": "qt-style-override",
"attributes": {
"kind": "variable",
"name": "QT_STYLE_OVERRIDE"
},
"template": "files/env-qt-style-override"
},
{
"collection": "environment",
"name": "qt-select",
"attributes": {
"kind": "variable",
"name": "QT_SELECT"
},
"template": "files/env-qt-select"
},
{
"collection": "environment",
"name": "xcursor-theme",
"attributes": {
"kind": "variable",
"name": "XCURSOR_THEME"
},
"template": "files/env-xcursor-theme"
},
{
"collection": "environment",
"name": "xcursor-size",
"attributes": {
"kind": "variable",
"name": "XCURSOR_SIZE"
},
"template": "files/env-xcursor-size"
},
{
"collection": "shell:xresources:normal",
"name": "xresources-normal",
"template": "files/xresources-normal"
},
{
"collection": "shell:xinitrc:normal",
"name": "xinitrc-normal",
"template": "files/xinitrc-normal"
}
],
"resources": [
{
"id": "package-gnome-themes-extra",
"type": "package",
"package": "gnome-themes-extra"
},
{
"id": "package-adwaita-icon-theme",
"type": "package",
"package": "adwaita-icon-theme"
},
{
"id": "package-adwaita-cursors",
"type": "package",
"package": "adwaita-cursors"
},
{
"id": "package-qt6ct",
"type": "package",
"package": "qt6ct"
},
{
"id": "package-xdg-desktop-portal-gtk",
"type": "package",
"package": "xdg-desktop-portal-gtk"
},
{
"id": "gtk3",
"type": "file",
"path": "${machine:account-home}/.config/gtk-3.0/settings.ini",
"owner": "${machine:account}",
"mode": "0644",
"template": "files/gtk-settings.ini"
},
{
"id": "gtk4",
"type": "file",
"path": "${machine:account-home}/.config/gtk-4.0/settings.ini",
"owner": "${machine:account}",
"mode": "0644",
"template": "files/gtk-settings.ini"
},
{
"id": "qt6ct",
"type": "file",
"path": "${machine:account-home}/.config/qt6ct/qt6ct.conf",
"owner": "${machine:account}",
"mode": "0644",
"template": "files/qt6ct.conf"
},
{
"id": "portals",
"type": "file",
"path": "${machine:account-home}/.config/xdg-desktop-portal/portals.conf",
"owner": "${machine:account}",
"mode": "0644",
"template": "files/portals.conf"
},
{
"id": "cursor",
"type": "file",
"path": "${machine:account-home}/.icons/default/index.theme",
"owner": "${machine:account}",
"mode": "0644",
"template": "files/cursor-index.theme"
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/adwaita-tools",
"binary": "adwaita-tools",
"loads": [
"adwaita-tools"
]
},
{
"name": "files",
"kind": "archive",
"from": "files"
}
]
}
}
@@ -0,0 +1,4 @@
# The names the artifact store had before, from the setting older-store-names, written by the mesh.
# artifact_store_references reads every definition under these too (novox/hq issue 323).
# Replaced on every push; change the setting, never this file.
older=${config:older-store-names}
@@ -0,0 +1,65 @@
{
"module": "artifact-store-tools",
"version": "1",
"invokes": [
"seat:mesh-controller.artifacts",
"seat:mesh-controller.collect",
"mesh-catalog.catalog_modules",
"mesh-catalog.catalog_module"
],
"tools": [
"artifact_store_repositories",
"artifact_store_usage",
"artifact_store_references",
"artifact_store_collect"
],
"config": {
"older-store-names": {
"type": "string",
"description": "The names the artifact store had before, as host names or host:port separated by spaces, or none when it had no other. artifact_store_references reads every definition under these as well as under the store's current name, so an image a definition pins under one is said to be named by a definition (novox/hq issue 323).",
"pattern": "none|[A-Za-z0-9.:-]+( [A-Za-z0-9.:-]+)*",
"default": "none"
}
},
"resources": [
{
"id": "state",
"type": "directory",
"mode": "0700",
"place": ".",
"unknown": "program"
},
{
"id": "store-names",
"type": "file",
"path": "${dir:state}/store-names",
"mode": "0600",
"trusted": false,
"template": "files/store-names"
}
],
"build": {
"artifacts": [
{
"name": "tools-go",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/artifact-store-tools",
"binary": "artifact-store-tools",
"loads": [
"artifact-store-tools"
],
"env": {
"MESH_ARTIFACT_STORE_CONTAINER": "mesh-registry",
"MESH_ARTIFACT_STORE_NAMES_FILE": "${dir:state}/store-names"
}
},
{
"name": "files",
"kind": "archive",
"from": "files"
}
]
}
}
@@ -0,0 +1,33 @@
#!/bin/bash
# zephyrus-backlight + | - | PERCENT — step or set the internal panel's backlight, never below 1 %.
# Shipped by the mesh's asus-zephyrus-g14 module; edit the catalogue.
#
# The panel is the backlight beneath the eDP connector, not a name: in hybrid mode this model also
# registers the discrete GPU's backlight (nvidia_0), which moves nothing. Writable by the video group
# through the module's udev rule, so the vendor-key trigger needs no root.
set -u
STEP=5
dev=""
for d in /sys/class/backlight/*; do
[ -e "$d" ] || continue
case "$(readlink -f "$d")" in *-eDP-*) dev=$d; break ;; esac
done
if [ -z "$dev" ]; then
for d in /sys/class/backlight/*; do [ -e "$d" ] && { dev=$d; break; }; done
fi
[ -n "$dev" ] || { echo "zephyrus-backlight: no backlight" >&2; exit 1; }
cur=$(cat "$dev/brightness")
max=$(cat "$dev/max_brightness")
pct=$(( cur * 100 / max ))
case "${1:-}" in
+|up|Up) pct=$(( pct + STEP )) ;;
-|down|Down) pct=$(( pct - STEP )) ;;
''|*[!0-9]*) echo "usage: zephyrus-backlight + | - | PERCENT" >&2; exit 2 ;;
*) pct=$1 ;;
esac
(( pct < 1 )) && pct=1
(( pct > 100 )) && pct=100
new=$(( max * pct / 100 ))
(( new < 1 )) && new=1
printf '%s' "$new" >"$dev/brightness" || exit 1
exec "$(dirname "$0")/zephyrus-notify" 5555 "Brightness: ${pct}%"
@@ -0,0 +1,41 @@
#!/bin/bash
# zephyrus-display primary | order — the laptop's internal panel among the outputs.
# Shipped by the mesh's asus-zephyrus-g14 module; edit the catalogue.
#
# primary makes the internal panel X's primary output, where the bars' tray goes.
# order the display key (Fn+F9, which the firmware sends as Super+P): odd workspaces to the
# internal panel, even ones to the first external output, or all to the panel when it
# is alone. The predecessor's orden-workspaces.
#
# The panel is found, not named: the connected output whose name starts with eDP. The predecessor
# wrote eDP-1, which is what this model calls it today and not a promise.
set -u
here=$(dirname "$0")
panel=$(xrandr --query 2>/dev/null | awk '$2 == "connected" && $1 ~ /^eDP/ { print $1; exit }')
[ -n "$panel" ] || { echo "zephyrus-display: no internal panel connected" >&2; exit 1; }
case "${1:-}" in
primary)
exec xrandr --output "$panel" --primary
;;
order)
external=$(xrandr --listmonitors 2>/dev/null | awk -v p="$panel" 'NR > 1 && $NF != p { print $NF; exit }')
[ -n "$external" ] || external=$panel
# i3's workspace list, one object per workspace; num and output read from each. No jq: the
# fields are flat strings and numbers, and rect, the only nested value, holds neither name.
i3-msg -t get_workspaces 2>/dev/null | sed 's/},{"id"/}\n{"id"/g' |
while read -r ws; do
num=$(printf '%s' "$ws" | grep -o '"num":-\?[0-9]*' | cut -d: -f2)
out=$(printf '%s' "$ws" | grep -o '"output":"[^"]*"' | cut -d'"' -f4)
[ -n "$num" ] && [ "$num" -ge 0 ] || continue
if [ $((num % 2)) -eq 0 ]; then dest=$external; else dest=$panel; fi
[ "$out" = "$dest" ] && continue
i3-msg "workspace number $num; move workspace to output $dest" >/dev/null
done
if [ "$external" = "$panel" ]; then
"$here/zephyrus-notify" 7780 "Workspaces: all on the panel"
else
"$here/zephyrus-notify" 7780 "Workspaces: odd on the panel, even on $external"
fi
;;
*) echo "usage: zephyrus-display primary | order" >&2; exit 2 ;;
esac
@@ -0,0 +1,20 @@
#!/bin/bash
# zephyrus-kbd-notify — shows the keyboard backlight's level when it changes. The level itself is set
# by the firmware and asusd (Fn+F2/F3), which tell UPower; this only listens and notifies. Started once
# per session from the module's i3 fragment. Shipped by the mesh's asus-zephyrus-g14 module.
#
# One instance per session: i3 runs its `exec` lines again on an in-place restart, and the
# predecessor's listener ran twice after one, showing every change twice.
set -u
here=$(dirname "$0")
lock="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}/zephyrus-kbd-notify.lock"
exec 9>"$lock"
flock -n 9 || exit 0
levels=(Off Low Med High)
dbus-monitor --system "type='signal',interface='org.freedesktop.UPower.KbdBacklight',member='BrightnessChanged'" 2>/dev/null |
while read -r line; do
if [[ $line =~ int32\ ([0-9]+) ]]; then
v=${BASH_REMATCH[1]}
"$here/zephyrus-notify" 5556 "Keyboard: ${levels[$v]:-$v}"
fi
done
@@ -0,0 +1,27 @@
#!/bin/bash
# zephyrus-media play-pause | next | previous — the media keys, through MPRIS (playerctl), with a short
# notification of what happened. Shipped by the mesh's asus-zephyrus-g14 module; edit the catalogue.
#
# From the predecessor's media-control, kept: the lock against a double fire (the vendor keys can
# repeat), and the notification. Dropped: its fallback to a media server's local API, which needed a
# token from a file of secrets (novox/hq research 027 question 2). A player that speaks MPRIS is
# reached; one that does not says so.
set -u
here=$(dirname "$0")
action=${1:-play-pause}
case "$action" in play-pause | next | previous) ;; *) echo "usage: zephyrus-media play-pause | next | previous" >&2; exit 2 ;; esac
lock="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}/zephyrus-media.lock"
exec 9>"$lock"
flock -n 9 || exit 0
if ! playerctl status >/dev/null 2>&1; then
"$here/zephyrus-notify" 7777 "Media: no player"
exit 0
fi
playerctl "$action" 2>/dev/null
if [ "$action" = play-pause ]; then
sleep 0.2
[ "$(playerctl status 2>/dev/null)" = Playing ] && label=Playing || label=Paused
else
label=${action^}
fi
"$here/zephyrus-notify" 7777 "Media: $label"
@@ -0,0 +1,12 @@
#!/bin/bash
# zephyrus-notify ID SUMMARY — a short desktop notification that replaces the previous one with the
# same ID, through the session's notification service on its bus. busctl is the service manager's
# own client, so nothing is installed for it. Shipped by the mesh's asus-zephyrus-g14 module.
set -u
id=${1:-0}
summary=${2:-}
uid=$(id -u)
DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/${uid}/bus" \
busctl --user call org.freedesktop.Notifications /org/freedesktop/Notifications \
org.freedesktop.Notifications Notify susssasa{sv}i \
asus-zephyrus-g14 "$id" "" "$summary" "" 0 1 urgency y 0 1500 >/dev/null 2>&1 || true
@@ -0,0 +1,46 @@
#!/bin/bash
# zephyrus-session COMMAND [ARG...] — run a command in the operator's graphical session from outside
# it: from a vendor-key trigger, which triggerhappy runs as the operator's account but with none of
# the session's environment. Shipped by the mesh's asus-zephyrus-g14 module; edit the catalogue.
#
# What it replaces: the predecessor's `as-user`, which triggerhappy ran as root and which `su`-ed to
# a named person with a hard-coded user id and display, and sourced a file of secrets on the way.
# Here the account is whoever runs it, the bus is that account's, and the display is the one the
# account's own session uses. Nothing is sourced.
set -u
uid=$(id -u)
export XDG_RUNTIME_DIR="/run/user/${uid}"
export DBUS_SESSION_BUS_ADDRESS="unix:path=${XDG_RUNTIME_DIR}/bus"
home=$(getent passwd "$uid" | cut -d: -f6)
[ -n "$home" ] && export HOME="$home"
# The session's own window manager says best which display and authority the session uses: read
# them from its environment, as the desktop modules' session finder does. Then logind, then any
# process of this account that has a display.
from_environ() {
env=$(tr '\0' '\n' <"/proc/$1/environ" 2>/dev/null) || return 1
d=$(printf '%s\n' "$env" | sed -n 's/^DISPLAY=//p' | head -n1)
[ -n "$d" ] || return 1
export DISPLAY="$d"
a=$(printf '%s\n' "$env" | sed -n 's/^XAUTHORITY=//p' | head -n1)
[ -n "$a" ] && export XAUTHORITY="$a"
return 0
}
if [ -z "${DISPLAY:-}" ]; then
for pid in $(pgrep -xu "$uid" i3 2>/dev/null); do
from_environ "$pid" && break
done
fi
if [ -z "${DISPLAY:-}" ]; then
for s in $(loginctl list-sessions --no-legend 2>/dev/null | awk -v u="$uid" '$2 == u { print $1 }'); do
d=$(loginctl show-session "$s" -p Display --value 2>/dev/null)
if [ -n "$d" ]; then export DISPLAY="$d"; break; fi
done
fi
if [ -z "${DISPLAY:-}" ]; then
for pid in $(pgrep -u "$uid" 2>/dev/null); do
from_environ "$pid" && break
done
fi
: "${XAUTHORITY:=${HOME}/.Xauthority}"
export XAUTHORITY
exec "$@"
@@ -0,0 +1,29 @@
#!/bin/bash
# zephyrus-touchpad reset | toggle — apply the touchpad's settings again, or switch it on or off.
# Shipped by the mesh's asus-zephyrus-g14 module; edit the catalogue.
#
# The settings themselves are an X input class (/etc/X11/xorg.conf.d/30-asus-zephyrus-g14-touchpad.conf),
# which X applies every time the device appears — after a resume too, which is what the predecessor's
# sleep hook existed for. This is the manual form, bound to the touchpad key.
set -u
here=$(dirname "$0")
name=$("$here/zephyrus-session" xinput list --name-only 2>/dev/null | grep -m1 -i 'touchpad')
[ -n "$name" ] || { echo "zephyrus-touchpad: no touchpad in this session" >&2; exit 1; }
x() { "$here/zephyrus-session" xinput "$@"; }
case "${1:-reset}" in
reset)
x set-prop "$name" "libinput Tapping Enabled" 1
x set-prop "$name" "libinput Natural Scrolling Enabled" 1
x set-prop "$name" "libinput Accel Speed" 0.15
x enable "$name"
"$here/zephyrus-session" "$here/zephyrus-notify" 7779 "Touchpad: reset"
;;
toggle)
if x list-props "$name" | grep -q 'Device Enabled ([0-9]*):[[:space:]]*1'; then
x disable "$name"; "$here/zephyrus-session" "$here/zephyrus-notify" 7779 "Touchpad: off"
else
x enable "$name"; "$here/zephyrus-session" "$here/zephyrus-notify" 7779 "Touchpad: on"
fi
;;
*) echo "usage: zephyrus-touchpad reset | toggle" >&2; exit 2 ;;
esac
@@ -0,0 +1,229 @@
{
"module": "asus-zephyrus-g14",
"version": "1",
"moved": true,
"templates": "templates",
"capabilities": [
"package-manager",
"service-manager"
],
"requires": [
"x11-display"
],
"emits": [
"profile.switched"
],
"tools": [
"zephyrus_brightness",
"zephyrus_battery",
"zephyrus_charge_limit",
"zephyrus_gpu_mode",
"zephyrus_profile",
"zephyrus_thermals",
"zephyrus_power_draw",
"zephyrus_profile_policy",
"zephyrus_fan_curves",
"zephyrus_keys",
"zephyrus_check"
],
"resources": [
{
"id": "asusctl",
"type": "package",
"package": "asusctl"
},
{
"id": "playerctl",
"type": "package",
"package": "playerctl"
},
{
"id": "asusd",
"type": "service",
"unit": "asusd.service",
"state": "running"
},
{
"id": "supergfxd",
"type": "service",
"unit": "supergfxd.service",
"state": "running",
"boot": "enabled"
},
{
"id": "scripts",
"type": "archive",
"path": "/usr/local/lib/asus-zephyrus-g14",
"artifact": "scripts",
"unknown": "program"
},
{
"id": "nvidia-options",
"type": "file",
"path": "/etc/modprobe.d/g14-nvidia-power.conf",
"mode": "0644",
"template": "templates/modprobe.d/g14-nvidia-power.conf"
},
{
"id": "video-options",
"type": "file",
"path": "/etc/modprobe.d/video-brightness-switch.conf",
"mode": "0644",
"template": "templates/modprobe.d/video-brightness-switch.conf"
},
{
"id": "suspend-drop-ins",
"type": "directory",
"path": "/etc/systemd/system/systemd-suspend.service.d",
"mode": "0755",
"unknown": "program"
},
{
"id": "nvidia-on-suspend",
"type": "file",
"path": "/etc/systemd/system/systemd-suspend.service.d/asus-zephyrus-g14-nvidia.conf",
"mode": "0644",
"template": "templates/systemd/system/systemd-suspend.service.d/asus-zephyrus-g14-nvidia.conf"
},
{
"id": "hibernate-drop-ins",
"type": "directory",
"path": "/etc/systemd/system/systemd-hibernate.service.d",
"mode": "0755",
"unknown": "program"
},
{
"id": "nvidia-on-hibernate",
"type": "file",
"path": "/etc/systemd/system/systemd-hibernate.service.d/asus-zephyrus-g14-nvidia.conf",
"mode": "0644",
"template": "templates/systemd/system/systemd-hibernate.service.d/asus-zephyrus-g14-nvidia.conf"
},
{
"id": "suspend-then-hibernate-drop-ins",
"type": "directory",
"path": "/etc/systemd/system/systemd-suspend-then-hibernate.service.d",
"mode": "0755",
"unknown": "program"
},
{
"id": "nvidia-on-suspend-then-hibernate",
"type": "file",
"path": "/etc/systemd/system/systemd-suspend-then-hibernate.service.d/asus-zephyrus-g14-nvidia.conf",
"mode": "0644",
"template": "templates/systemd/system/systemd-suspend-then-hibernate.service.d/asus-zephyrus-g14-nvidia.conf"
},
{
"id": "powerd-drop-ins",
"type": "directory",
"path": "/etc/systemd/system/nvidia-powerd.service.d",
"mode": "0755",
"unknown": "program"
},
{
"id": "powerd-opt-in",
"type": "file",
"path": "/etc/systemd/system/nvidia-powerd.service.d/asus-zephyrus-g14.conf",
"mode": "0644",
"template": "templates/systemd/system/nvidia-powerd.service.d/asus-zephyrus-g14.conf"
},
{
"id": "backlight-rule",
"type": "file",
"path": "/etc/udev/rules.d/90-backlight.rules",
"mode": "0644",
"template": "templates/udev/rules.d/90-backlight.rules"
},
{
"id": "udev",
"type": "service",
"unit": "systemd-udevd.service",
"reload-on": [
"backlight-rule"
]
},
{
"id": "upower-package",
"type": "package",
"package": "upower"
},
{
"id": "upower-drop-ins",
"type": "directory",
"path": "/etc/UPower/UPower.conf.d",
"mode": "0755",
"unknown": "program"
},
{
"id": "low-battery",
"type": "file",
"path": "/etc/UPower/UPower.conf.d/50-asus-zephyrus-g14.conf",
"mode": "0644",
"template": "templates/UPower/UPower.conf.d/50-asus-zephyrus-g14.conf"
},
{
"id": "upower",
"type": "service",
"unit": "upower.service",
"state": "running",
"boot": "enabled",
"restart-on": [
"low-battery"
]
},
{
"id": "xorg-drop-ins",
"type": "directory",
"path": "/etc/X11/xorg.conf.d",
"mode": "0755",
"unknown": "program"
},
{
"id": "touchpad",
"type": "file",
"path": "/etc/X11/xorg.conf.d/30-asus-zephyrus-g14-touchpad.conf",
"mode": "0644",
"template": "templates/X11/xorg.conf.d/30-asus-zephyrus-g14-touchpad.conf"
}
],
"items": [
{
"collection": "shell:after-wake:normal",
"template": "templates/after-wake"
},
{
"collection": "contribution:node-hotkeys:trigger",
"template": "templates/hotkeys-triggers"
},
{
"collection": "contribution:node-display-session:config",
"name": "display-session",
"template": "templates/display-session"
}
],
"build": {
"artifacts": [
{
"name": "tools-go",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/zephyrus",
"binary": "zephyrus",
"loads": [
"zephyrus"
]
},
{
"name": "scripts",
"kind": "archive",
"from": "files"
},
{
"name": "templates",
"kind": "archive",
"from": "templates"
}
]
}
}
@@ -0,0 +1,11 @@
# Managed by the mesh (module asus-zephyrus-g14). Replaced on every push; edit the catalogue instead.
#
# On low battery the machine suspends rather than powering off, at 7 % — s2idle still draws a little,
# so it leaves headroom. A drop-in over the package's own UPower.conf, which stays the package's.
[UPower]
UsePercentageForPolicy=true
PercentageLow=15.0
PercentageCritical=10.0
PercentageAction=7.0
CriticalPowerAction=Suspend
AllowRiskyCriticalPowerAction=true
@@ -0,0 +1,12 @@
# Managed by the mesh (module asus-zephyrus-g14). Replaced on every push; edit the catalogue instead.
#
# The touchpad's settings, applied by X every time the device appears — at login and after every
# resume, when the device is initialised again. This replaces the predecessor's sleep hook, which ran
# xinput after a resume as a named person on a guessed display.
Section "InputClass"
Identifier "asus-zephyrus-g14 touchpad"
MatchIsTouchpad "on"
Option "Tapping" "on"
Option "NaturalScrolling" "true"
Option "AccelSpeed" "0.15"
EndSection
@@ -0,0 +1,7 @@
# The touchpad's settings again after waking, in the operator's session: X applies the module's
# input class when the device appears, and this covers a wake that does not initialise it again.
# Runs as root from the power module; the reset itself runs as the session's owner.
sleep 2
owner=$(ps -o user= -C i3 | head -n 1)
[ -n "$owner" ] && runuser -u "$owner" -- /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-touchpad reset
true
@@ -0,0 +1,30 @@
# The laptop's own lines in i3 (module asus-zephyrus-g14, novox/hq ADR 0208, ADR 0210). Owned by the
# mesh: replaced at every push. Once the controller places contributions to node-display-session
# (ADR 0210), these become the module's contribution instead of a file in i3's directory.
#
# The keys the firmware turns into ordinary key presses. The vendor keys that reach no X client are
# triggerhappy's (/etc/triggerhappy/triggers.d/asus-g14.conf): M4 and Fn+F4/F5 for media, Fn+F7/F8 for
# the panel, Fn+F10 for the touchpad. The keyboard backlight (Fn+F2/F3) is the firmware's and asusd's.
# Fn+F6, the screenshot key: the firmware sends Super+Shift+S. Released before it runs, because the
# screenshot grabs the pointer to select a region, which fails while the key is still held.
bindsym --release $mod+Shift+s exec --no-startup-id $XDG_CONFIG_HOME/i3/scripts/screenshot.sh
# Fn+F9, the display key: the firmware sends Super+P. Odd workspaces to the panel, even ones to the
# external output.
bindsym $mod+p exec --no-startup-id /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-display order
# The keyboard backlight's level, shown when it changes.
exec --no-startup-id /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-kbd-notify
# The laptop's own lines in i3 (module asus-zephyrus-g14, novox/hq ADR 0208, ADR 0210). Owned by the
# mesh: replaced at every push. Once the controller places contributions to node-display-session
# (ADR 0210), these become the module's contribution instead of a file in i3's directory.
#
# The model's panel and touchpad.
# The internal panel is the primary output, where the bars' tray goes. Which monitors are on and where
# is the display server's (autorandr, run at every session start).
exec --no-startup-id /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-display primary
# The touchpad's settings again, by hand. X applies them itself whenever the device appears.
bindsym $mod+Shift+x exec --no-startup-id /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-touchpad reset
@@ -0,0 +1,10 @@
# The ROG Zephyrus G14's vendor keys, which reach no X client: media (the M-keys), panel brightness,
# and the touchpad key. Each runs this module's own script, as the operator's account.
KEY_PROG1 1 /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-session /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-media play-pause
KEY_PROG3 1 /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-session /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-media previous
KEY_PROG4 1 /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-session /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-media next
KEY_BRIGHTNESSDOWN 1 /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-backlight -
KEY_BRIGHTNESSDOWN 2 /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-backlight -
KEY_BRIGHTNESSUP 1 /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-backlight +
KEY_BRIGHTNESSUP 2 /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-backlight +
KEY_F21 1 /usr/local/lib/asus-zephyrus-g14/bin/zephyrus-touchpad reset
@@ -0,0 +1,16 @@
# Managed by the mesh (module asus-zephyrus-g14). Replaced on every push; edit the catalogue instead.
#
# The discrete GPU's driver options on the ROG Zephyrus G14 (GA403, RTX 40 series, hybrid graphics).
#
# NVreg_DynamicPowerManagement=0x00 turns runtime D3 off. With it on, a change of power source sends
# the driver an ACPI notification it fails to handle on this model ("RmHandleDNotifierEvent: Failed to
# handle ACPI D-Notifier event, status=0x62"), and the GPU stops making progress until the machine is
# powered off. Off costs a few idle watts in hybrid mode and keeps the machine up.
#
# NVreg_PreserveVideoMemoryAllocations=1 saves video memory across suspend, so what used the GPU still
# works after waking. It needs nvidia-suspend, -hibernate and -resume to run around a sleep, which this
# module's drop-ins on the sleep services ask for.
#
# A change here applies when the driver next loads: at the next boot.
options nvidia NVreg_PreserveVideoMemoryAllocations=1
options nvidia NVreg_DynamicPowerManagement=0x00
@@ -0,0 +1,6 @@
# Managed by the mesh (module asus-zephyrus-g14). Replaced on every push; edit the catalogue instead.
#
# The ACPI video driver does not change the backlight itself on the brightness keys: on this model it
# moves the wrong one. The keys are triggerhappy's (see /etc/triggerhappy/triggers.d/asus-g14.conf).
# Applies when the module next loads: at the next boot.
options video brightness_switch_enabled=0
@@ -0,0 +1,7 @@
# Managed by the mesh (module asus-zephyrus-g14). Replaced on every push; edit the catalogue instead.
#
# nvidia-powerd (Dynamic Boost) was the first error in the chain that hung this model's GPU on a change
# of power source, and asusd starts it on mains. It runs only when the kernel command line says
# zephyrus.nvidia-powerd — an explicit opt-in, at boot.
[Unit]
ConditionKernelCommandLine=zephyrus.nvidia-powerd
@@ -0,0 +1,7 @@
# Managed by the mesh (module asus-zephyrus-g14). Replaced on every push; edit the catalogue instead.
#
# The NVIDIA driver's own sleep actions, asked for by the sleep itself rather than enabled as
# links: the mesh declares files and never makes links (novox/hq ADR 0012), and the host's service
# shape must not start these units by hand, which would put the GPU to sleep with the machine awake.
[Unit]
Wants=nvidia-hibernate.service nvidia-resume.service
@@ -0,0 +1,7 @@
# Managed by the mesh (module asus-zephyrus-g14). Replaced on every push; edit the catalogue instead.
#
# The NVIDIA driver's own sleep actions, asked for by the sleep itself rather than enabled as
# links: the mesh declares files and never makes links (novox/hq ADR 0012), and the host's service
# shape must not start these units by hand, which would put the GPU to sleep with the machine awake.
[Unit]
Wants=nvidia-suspend-then-hibernate.service nvidia-resume.service

Some files were not shown because too many files have changed in this diff Show More