Compare commits

...
13 Commits
Author SHA1 Message Date
jochen 1cdd9df581 Let platform manifests go only on a confirmed collect, and copy again what a sweep took
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
An unrecorded index or a copy in progress can name a platform the records do not see, so
only a person's collect, after its dry run, takes an index's platforms, and only once every
kept index of each repository it touches was read. A copy missing a platform is copied
again, and a copy a build holds again is no longer recorded as collected (review of #144).
2026-10-08 15:32:10 +02:00
jochen 7edfb56e7a Record the bases a build copies, keep them by the builds that stood on them, and copy each image once
A copied base was named only in what a build stood on, and nowhere when the build failed,
so the store's sweep could never let one go (hq issue 321). One repository per upstream
image stops each module asking the public registry for the same image again, and letting
an index go now takes its own platform manifests, which otherwise kept every byte. A
person can record the copies no record names through the new mirrors verb (hq ADR 0257).

The forge test fix is the same commit as on feat/plain-notifications: main fails without it.
2026-10-08 15:32:10 +02:00
mesh-admin 34611c8e38 Merge pull request 'Let a seat receive blocks as data its holder renders, placed where the machine has the hardware (hq ADR 0255)' (#143) from feat/the-bar-takes-blocks into main 2026-10-08 13:31:48 +00:00
jochen 8984c3437f Leave out a block its holder cannot render, and keep if-capability to known names on offered kinds (hq ADR 0255)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer delivery to the same trunk took over its walk
A piece whose shows the holder's template does not know rendered as nothing and failed
the whole machine's declaration; it is now left out and named, for push, plan and the
merge gate. quote escapes DEL, which TOML refuses bare. An if-capability nothing
detects, or on a kind a holder depends on, would drop a piece silently, so both are
refused.
2026-10-08 15:20:59 +02:00
jochen 9766338368 Build every artifact the forge declares in the forge tests
The catalogue's forge gained an npm-registry bundle (hq ADR 0251 §4), so resolving it
against its code bundle alone failed three tests on main and on every pull request.
2026-10-08 15:20:59 +02:00
jochen b1acb3d9de Let a seat receive blocks as data its holder renders, placed where the machine has the hardware (hq ADR 0255)
A module adding a battery to the bar had to write i3status-rust's TOML, so a second
bar could not take its place. node-bar now receives a bar-neutral block: the
contributor says what it shows, the holder renders it with its own template, places
every bar and place once, and a contribution may name a capability the machine must
report. The block is offered: the power module runs on servers without a bar.
2026-10-08 15:20:59 +02:00
mesh-admin a759ac65a5 Merge pull request 'Say where an answer no notification can give is given; never offer to silence data loss (hq ADR 0258)' (#147) from fix/needs-you-from-the-console into main 2026-10-08 13:20:42 +00:00
jochen 2e6a9b1efc Say where an answer no notification can give is given, and never offer to silence data loss
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer delivery to the same trunk took over its walk
mesh/delivery-group group fix/needs-you-from-the-console closed: a member was closed before the group merged
A "Needs you" with no button left the operator guessing where to act, and a click
could silence a condition that says data is gone. The place is named in the
glossary's word, the mesh MCP server (hq ADR 0258).
2026-10-08 15:03:11 +02:00
mesh-admin 45ae1d0112 Merge pull request 'Offer only acknowledgements, keep a refused verdict, say a change of words (hq ADR 0258, issue 324)' (#145) from fix/notifications-after-review into main 2026-10-08 13:02:29 +00:00
jochen 76cfbac7a1 Offer only acknowledgements as answers, keep a refused verdict, and say a change of words
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/notifications-after-review merged: every member is merged
Review found that a desk click proves nothing about who chose, that refused words
could turn "Needs you" into "Nothing for you to do", that sound words were refused,
and that a quiet warning whose words came to need the operator was never said
(hq ADR 0258).
2026-10-08 14:53:36 +02:00
mesh-admin 28712eaea1 Merge pull request 'Pass a wait for a person's new login, and keep the pass of a module healthy beside a failure (hq ADR 0254, issue 318)' (#142) from fix/318-a-wait-for-a-person-is-not-a-failure into main 2026-10-08 12:41:08 +00:00
jochen 51d7bbfdc8 Give relogin-needed its plain words: it needs the operator and offers no button, since only their new login can end it (hq ADR 0253, ADR 0254)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/318-a-wait-for-a-person-is-not-a-failure merged: every member is merged
2026-10-08 14:25:20 +02:00
jochen e3b5c224e8 Pass a wait for a person's new login with the wait carried, and keep the pass of a module healthy beside a failure (hq ADR 0254, issue 318) 2026-10-08 14:20:55 +02:00
58 changed files with 3899 additions and 162 deletions
+2
View File
@@ -264,6 +264,8 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc,
forgeFrom(), say, request.Seats)
}
// What it copied into the store, whatever became of the build (novox/hq ADR 0257).
result.Mirrored = built.Mirrored
// Only a build the kill ended: the kill came before its work did. One that finished — built, or
// failed on its own — in the moment the kill arrived says what it did, and the kill is refused.
killed := false
+3
View File
@@ -160,6 +160,9 @@ func buildFrom(result link.BuildResult) inventory.Build {
for _, ref := range result.Against {
kept.Against = append(kept.Against, catalogue.Recorded(ref))
}
for _, ref := range result.Mirrored {
kept.Mirrored = append(kept.Mirrored, catalogue.Recorded(ref))
}
for _, r := range result.Read {
kept.Read = append(kept.Read, inventory.ReadRepository{Repository: r.Repository, Ref: r.Ref})
}
+28
View File
@@ -31,6 +31,11 @@ type sweepBounds struct {
most int
// budget is the longest it keeps its caller waiting.
budget time.Duration
// platforms is whether an index is let go of with its own platform manifests (novox/hq ADR 0257).
// Only a sweep a person confirmed, after its dry run listed what stays and names a platform: the
// records cannot see an unrecorded index, or a copy in progress, naming the same platform, and the
// store's tools can.
platforms bool
}
// afterBuild are the bounds of the sweep inside somebody's build.
@@ -76,6 +81,29 @@ func sweep(ctx context.Context, inv *inventory.Inventory, store artifacts.Store,
// as builds come, and is two HEADs each once done. A kept archive that could not be held stops
// the sweep before it deletes anything: "everything kept is held" is the precondition the
// collector's safety rests on, and a store refusing a hold would refuse the deletes too.
// **An index goes with its platform manifests only when a person confirmed it** (novox/hq ADR
// 0257), and a kept index keeps its own: which platform manifests the kept indexes name is read
// from the store for every repository the sweep will touch, before the first delete. A single
// read that fails stops the sweep before it deletes anything, as a kept archive that cannot be
// held does. The sweep after a build lets an index go alone, as it always has.
store.Spare = nil
if bounds.platforms {
if inv == nil {
r.Stopped = "no records to read which platform manifests a kept index names, so nothing was let go"
r.Left = len(references)
return r
}
images, err := inv.KeptImages(ctx)
if err == nil {
store.Spare, err = store.SpareKeptIndexes(within, images, references)
}
if err != nil {
r.Stopped = fmt.Sprintf("which platform manifests a kept index names could not be read, so nothing was let go: %v", err)
r.Left = len(references)
return r
}
}
wrote, missing, err := holdKept(within, store, kept)
r.HoldersWritten, r.Missing = wrote, missing
if err != nil {
+93 -6
View File
@@ -77,6 +77,10 @@ type health int
const (
healthGood health = iota
// healthPerson is a wait for a person (novox/hq ADR 0254): everything unhealthy of the module waits for
// one person's new login (personWait). The build did what it should; no bound a machine can meet ends
// the wait. It counts as a pass, and the gate carries the reading along in its verdict.
healthPerson
// healthWaiting is not a pass and not a fault: what the module's checks find waits on an unhealthy
// provider (ADR 0240 rule 5), so the judging waits — past the bound too — rather than putting back a
// build for something it did not do.
@@ -209,7 +213,9 @@ func judgeHealth(module, component string, m catalogue.Manifest, machine string,
firstLine(f.openErr.Error())
}
for _, c := range f.open {
if c.Source == gateProbe || c.Raised.Before(since) {
// A wait for a person's new login is the module's reading, not a fault raised since the send: the
// gate reads it from the statement below (ADR 0254).
if c.Source == gateProbe || c.Raised.Before(since) || c.Kind == kindReloginNeeded {
continue
}
onIt := c.Subject.Machine == machine || slices.Contains(c.Subject.Also, machine) ||
@@ -435,13 +441,19 @@ func judgeMoves(ctx context.Context, open *stores, g *inventory.PlanGate, pairs
for node, moved := range byMachine {
words[node] = aboutTheMachine(node, moved, *g.Since, facts)
}
// **Each module is judged on its own** (novox/hq ADR 0254, issue 318): its reading is the worst of its
// machines', its passes are counted apart, and the send's verdict is still one — but a module that was
// healthy on its own for the passes the gate asks keeps a pass when another beside it fails.
worst, why := healthGood, ""
var failing []string
broken := map[string]bool{}
reading := map[string]health{}
waits := map[string]string{}
var modules []string
for _, j := range pairs {
w := words[j.node]
h, said := judgeHealth(j.module, coreComponent(j.module), shelf[j.module], j.node, *g.Since, w.facts)
if h == healthGood {
if h == healthGood || h == healthPerson {
if on, named := w.on[j.module]; named {
h, said = healthNotYet, on
} else if w.whole != "" {
@@ -450,7 +462,16 @@ func judgeMoves(ctx context.Context, open *stores, g *inventory.PlanGate, pairs
h, said = healthWaiting, w.waiting
}
}
if h != healthGood && !slices.Contains(failing, j.module) {
if _, seen := reading[j.module]; !seen {
modules = append(modules, j.module)
}
if h > reading[j.module] {
reading[j.module] = h
}
if h == healthPerson {
waits[j.module] = joinSaid(waits[j.module], said)
}
if h > healthPerson && !slices.Contains(failing, j.module) {
failing = append(failing, j.module)
}
if h == healthBroken {
@@ -458,14 +479,45 @@ func judgeMoves(ctx context.Context, open *stores, g *inventory.PlanGate, pairs
}
if h > worst {
worst, why = h, said
} else if h == worst && h != healthGood && why == "" {
} else if h == worst && h > healthPerson && why == "" {
why = said
}
}
if g.Healthy == nil {
g.Healthy = map[string]int{}
}
g.Waits = nil
for _, m := range modules {
switch {
case reading[m] > healthPerson:
g.Healthy[m] = 0
default:
g.Healthy[m]++
if w := waits[m]; w != "" {
if g.Waits == nil {
g.Waits = map[string]string{}
}
g.Waits[m] = w
}
}
}
// passedAlone is every module that passed on its own while the send as a whole did not.
passedAlone := func() []string {
var out []string
if now.Sub(*g.Since) < gateSettle {
return nil
}
for _, m := range modules {
if reading[m] <= healthPerson && g.Healthy[m] >= gatePasses {
out = append(out, m)
}
}
return out
}
switch {
case worst == healthBroken:
// What broke is put back; what was only not yet healthy beside it is too — they moved together.
g.Failing = failing
g.Failing, g.Passing = failing, passedAlone()
decide(g, inventory.GateFailed, why, now)
case worst == healthWaiting:
// Waiting on a provider that is unhealthy: not a pass, and not a failure at the bound either —
@@ -474,19 +526,48 @@ func judgeMoves(ctx context.Context, open *stores, g *inventory.PlanGate, pairs
case worst == healthNotYet:
g.Passes, g.LastPass, g.Last, g.Failing = 0, nil, why, failing
if now.Sub(*g.Since) > gateBound {
g.Passing = passedAlone()
decide(g, inventory.GateFailed, fmt.Sprintf("not healthy within %s of its apply: %s", gateBound, why), now)
}
default:
// Healthy, or waiting for a person (ADR 0254): a pass, the wait carried along in the verdict.
g.Passes++
g.LastPass, g.Last, g.Failing = &now, "", nil
if g.Passes >= gatePasses && now.Sub(*g.Since) >= gateSettle {
decide(g, inventory.GatePassed, fmt.Sprintf("healthy %d times over %s", g.Passes,
now.Sub(*g.Since).Round(time.Second)), now)
now.Sub(*g.Since).Round(time.Second))+waitsSaid(g.Waits), now)
}
}
return g.Verdict, nil
}
// waitsSaid is the waits for a person a passing gate carries, as its verdict says them.
func waitsSaid(waits map[string]string) string {
if len(waits) == 0 {
return ""
}
modules := make([]string, 0, len(waits))
for m := range waits {
modules = append(modules, m)
}
sort.Strings(modules)
var said []string
for _, m := range modules {
said = append(said, waits[m])
}
return "; and it waits for a person: " + strings.Join(said, "; ")
}
func joinSaid(a, b string) string {
switch {
case a == "":
return b
case b == "" || strings.Contains(a, b):
return a
}
return a + "; " + b
}
// decide sets a gate's verdict.
func decide(g *inventory.PlanGate, verdict, why string, now time.Time) {
g.Verdict, g.Why, g.JudgedAt = verdict, why, &now
@@ -959,6 +1040,12 @@ func gateLine(g *inventory.PlanGate) string {
} else if g.Verdict == "" {
line += fmt.Sprintf(" (%d of %d passes)", g.Passes, gatePasses)
}
if g.Verdict == "" && len(g.Waits) > 0 {
line += waitsSaid(g.Waits)
}
if len(g.Passing) > 0 {
line += "; passed on their own and kept: " + strings.Join(g.Passing, ", ")
}
if g.Rollback != "" {
line += "; " + g.Rollback
}
+9 -2
View File
@@ -15,6 +15,7 @@ import (
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
@@ -65,8 +66,10 @@ var handActVerbs = []handActVerb{
// not trusted with it — either is a repair the owner should have made.
{Verb: "plans go"},
{Verb: "broker consumer-reset"},
// Silencing the same condition twice says the condition, or what it watches, wants mending.
{Verb: "conditions silence"},
// Silencing the same condition twice says the condition, or what it watches, wants mending — unless
// it is the operator's answer on a notification: a decision to live with it (novox/hq ADR 0258).
{Verb: "conditions silence", Decision: "the operator's answer on a notification is their decision, " +
"not a repair (ADR 0258)", DecidedFor: []string{conditions.CauseOperatorAnswer}},
// An act done outside the mesh: the mesh cannot tell a repair from a decision there, so it counts —
// except a drill recorded through it before `hand-act drill` existed (2026-10-07). It refuses the
// cause since, so no act recorded through it now carries it.
@@ -83,6 +86,10 @@ var handActVerbs = []handActVerb{
// a moment the person chose (ADR 0251) — never a repair.
{Verb: "collect", Decision: "letting the store go of what the records keep for no reason, now rather " +
"than at the next build, is a person's word (ADR 0251)"},
// Recording a copy no record names as the mesh's: a person's reading of the store, never a repair
// (ADR 0257).
{Verb: "mirrors", Decision: "which copies in the store no record names are the mesh's is a person's " +
"word (ADR 0257)"},
{Verb: "bus upgrade", Decision: "the bus is never rolled by the mesh: replacing it is a planned step a " +
"person starts (ADR 0236)"},
{Verb: "upgrade release-backlog", Decision: "after a release plan failed, the next opens only when a " +
+2
View File
@@ -100,6 +100,8 @@ func run() error {
return collectCommand(ctx, args[1:])
case "images":
return imagesCommand(ctx, args[1:])
case "mirrors":
return mirrorsCommand(ctx, args[1:])
case "plans":
return plansCommand(ctx, args[1:])
case "delivery":
+8
View File
@@ -135,6 +135,7 @@ type mergeComposed struct {
Problems []string `json:"problems,omitempty"`
Withheld []string `json:"withheld,omitempty"`
Unbound []string `json:"unbound,omitempty"`
Unplaced []string `json:"unplaced,omitempty"`
LeftOut map[string]string `json:"left-out,omitempty"`
Resources []string `json:"resources,omitempty"`
}
@@ -372,6 +373,12 @@ func judgeChange(ctx context.Context, in mergeCheckInput) (mergeVerdict, error)
v.Failures = append(v.Failures, fmt.Sprintf("%s: the change leaves a credential bound elsewhere — %s",
gm.Described, u))
}
// A contribution its holder's template renders nothing for (novox/hq ADR 0255): the machine
// would be sent without it, so a change that adds one is refused, naming it.
for _, u := range newOnly(gm.Change.Unplaced, gm.Base.Unplaced) {
v.Failures = append(v.Failures, fmt.Sprintf("%s: the change leaves a contribution unplaced — %s",
gm.Described, u))
}
for module, why := range gm.Change.LeftOut {
if _, was := gm.Base.LeftOut[module]; !was {
v.Failures = append(v.Failures, fmt.Sprintf("%s: the change leaves %s out of its declaration — %s",
@@ -780,6 +787,7 @@ func composeEveryMachine(ctx context.Context, in mergeCheckInput, shelf map[stri
for _, u := range declared.unbound {
c.Unbound = append(c.Unbound, u.String())
}
c.Unplaced = declared.unplaced
sort.Strings(c.Withheld)
sort.Strings(c.Unbound)
out[m.Name] = c
+237
View File
@@ -0,0 +1,237 @@
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"os"
"regexp"
"strings"
"github.com/novox/mesh-controller/internal/artifacts"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// The bases the mesh copied into its artifact store, and the copies no record names (novox/hq ADR 0257).
//
// A build that stands on an image published elsewhere copies it in first (ADR 0096, 0097). Since ADR 0257
// each copy is recorded where it is made, and a copy is kept while a kept build stood on it. Copies made
// before then by a build that failed — or before the records kept what a build stood on — are in the store
// with no record naming them, and ADR 0189 §3 keeps the sweep away from anything no record names. This
// verb is how a person says such a copy is the mesh's: it records it, and the sweep then decides as it
// does for every other copy. It deletes nothing.
// mirrorRepository is a repository only the mirror writes: one image's repository, or a module's own
// repository of a base from before ADR 0257.
var mirrorRepository = regexp.MustCompile(`^(upstream/[a-z0-9][a-z0-9._/-]*|[a-z0-9][a-z0-9._-]*/on-[a-z0-9_]+)$`)
// mirrorReference reads a reference a person gave into its recorded form, refusing anything that is not
// a manifest in a repository the mirror writes.
func mirrorReference(given string) (string, error) {
reference := catalogue.Recorded(strings.TrimSpace(given))
path, ours := catalogue.InArtifactStore(reference)
if !ours {
return "", fmt.Errorf("%q is not a reference into the artifact store (artifact-store://<repository>@sha256:<hex>)", given)
}
repository, digest, ok := strings.Cut(path, "@sha256:")
if !ok || len(digest) != 64 || strings.Trim(digest, "0123456789abcdef") != "" {
return "", fmt.Errorf("%q names no manifest by digest", given)
}
if !mirrorRepository.MatchString(repository) {
return "", fmt.Errorf("%q is in %s, which is not a repository the mirror writes "+
"(upstream/<host>/<path>, or <module>/on-<argument>)", given, repository)
}
return reference, nil
}
type mirrorEntry struct {
Reference string `json:"reference"`
State string `json:"state"`
Why []string `json:"why,omitempty"`
}
type mirrorsAnswer struct {
DryRun bool `json:"dry_run,omitempty"`
// Mirrors is every copy the records hold that is not yet let go of.
Mirrors []mirrorEntry `json:"mirrors"`
Counts struct {
Kept int `json:"kept"`
Eligible int `json:"eligible"`
Collected int `json:"collected"`
} `json:"counts"`
// Recorded, AlreadyRecorded and Refused answer a record: what was (or, a dry run, would be)
// recorded, what the records already held, and what was refused, with why.
Recorded []string `json:"recorded,omitempty"`
// Then says what recording does: a recorded copy is eligible, and the next sweep lets it go.
Then string `json:"then,omitempty"`
AlreadyRecorded []string `json:"already_recorded,omitempty"`
Refused map[string]string `json:"refused,omitempty"`
}
// mirrorsOf is the copies among the recorded states.
func mirrorsOf(states []inventory.ArtifactState, mirrored map[string]bool) mirrorsAnswer {
a := mirrorsAnswer{Mirrors: []mirrorEntry{}}
for _, s := range states {
if !mirrored[s.Reference] {
continue
}
switch s.State {
case inventory.ArtifactKept:
a.Counts.Kept++
case inventory.ArtifactEligible:
a.Counts.Eligible++
case inventory.ArtifactCollected:
a.Counts.Collected++
continue
}
a.Mirrors = append(a.Mirrors, mirrorEntry{Reference: s.Reference, State: s.State, Why: s.Why})
}
return a
}
// splitReferences reads references separated by spaces or commas.
func splitReferences(given string) []string {
return strings.FieldsFunc(given, func(r rune) bool { return r == ',' || r == ' ' || r == '\n' || r == '\t' })
}
// recordMirrors decides, for each reference given, whether it may be recorded: in a repository the
// mirror writes, not already recorded, and held by the store. A real run records those.
func recordMirrors(ctx context.Context, inv *inventory.Inventory, store artifacts.Store, given []string,
known map[string]bool, why string, real bool) (mirrorsAnswer, error) {
a := mirrorsAnswer{DryRun: !real, Mirrors: []mirrorEntry{}, Refused: map[string]string{}}
var record []string
seen := map[string]bool{}
for _, g := range given {
reference, err := mirrorReference(g)
if err != nil {
a.Refused[g] = err.Error()
continue
}
if seen[reference] {
continue
}
seen[reference] = true
if known[reference] {
a.AlreadyRecorded = append(a.AlreadyRecorded, reference)
continue
}
if store.Address == "" {
a.Refused[g] = "this mesh has no artifact store on its network to ask whether it holds this"
continue
}
held, err := store.HoldsManifest(ctx, reference)
switch {
case err != nil:
a.Refused[g] = err.Error()
continue
case !held:
a.Refused[g] = "the artifact store does not hold it"
continue
}
record = append(record, reference)
}
a.Recorded = record
if len(record) > 0 {
verb := "would become"
if real {
verb = "became"
}
a.Then = fmt.Sprintf("%d cop%s %s eligible: the next sweep (after any build, or collect) lets each go "+
"unless one of the five kept builds of a module the mesh holds stood on it", len(record),
map[bool]string{true: "y", false: "ies"}[len(record) == 1], verb)
}
if real && len(record) > 0 {
if err := inv.RecordMirrored(ctx, "", why, record); err != nil {
return a, fmt.Errorf("recording %d copies: %w", len(record), err)
}
}
return a, nil
}
func mirrorsCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("mirrors", flag.ContinueOnError)
asJSON := set.Bool("json", false, "answer as JSON")
record := set.String("record", "", "references of copies no record names, separated by spaces or commas, to record as the mesh's")
confirm := set.Bool("confirm", false, "record them, rather than only say what would be recorded")
f := addHandActFlags(set)
positionals, err := parseAround(set, args)
if err != nil {
return err
}
if len(positionals) != 0 {
return errors.New("mirrors [--json] [--record <references> [--confirm --why <text>]]")
}
if *confirm {
if strings.TrimSpace(*record) == "" {
return errors.New("mirrors: --confirm records what --record names, and it names nothing. Nothing was done")
}
if err := f.require("mirrors"); err != nil {
return err
}
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
inv := open.inventory
states, err := inv.Artifacts(ctx)
if err != nil {
return err
}
known, err := inv.Mirrored(ctx)
if err != nil {
return err
}
var answer mirrorsAnswer
if strings.TrimSpace(*record) == "" {
answer = mirrorsOf(states, known)
} else {
shelf, err := inv.Catalogue(ctx)
if err != nil {
return err
}
address, err := artifactStoreAddress(ctx, inv, shelf, "")
if err != nil {
return err
}
answer, err = recordMirrors(ctx, inv, artifacts.Store{Address: address}, splitReferences(*record), known,
strings.TrimSpace(*f.why), *confirm)
if err != nil {
return err
}
// The hand act is logged once the records say what it did, never before: an act that
// failed half-way is not logged as done.
if *confirm && len(answer.Recorded) > 0 {
f.record(ctx, "mirrors", append([]string{"--record"}, answer.Recorded...))
}
}
if *asJSON {
encoder := json.NewEncoder(os.Stdout)
encoder.SetIndent("", " ")
return encoder.Encode(answer)
}
if answer.DryRun && len(answer.Recorded) > 0 {
fmt.Println("a dry run: nothing was recorded (--confirm --why <text> to record)")
}
if answer.Then != "" {
fmt.Println(answer.Then)
}
for _, r := range answer.Recorded {
fmt.Printf(" record %s\n", r)
}
for _, r := range answer.AlreadyRecorded {
fmt.Printf(" already %s\n", r)
}
for g, why := range answer.Refused {
fmt.Printf(" refused %s: %s\n", g, why)
}
for _, m := range answer.Mirrors {
fmt.Printf(" %-9s %s %s\n", m.State, m.Reference, strings.Join(m.Why, ", "))
}
return nil
}
+280
View File
@@ -0,0 +1,280 @@
package main
import (
"fmt"
"net/http"
"net/http/httptest"
"slices"
"strings"
"sync"
"testing"
"time"
"github.com/novox/mesh-controller/internal/artifacts"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The bases the mesh copied in, and recording a copy no record names (novox/hq ADR 0257).
func TestTheMirrorsVerbComposesItsCommandAndRefusesWhatItDoesNotTake(t *testing.T) {
r := ref("route-proxy", "on-go_base", 1)
for _, c := range []struct {
args map[string]any
want string
}{
{map[string]any{}, "mirrors --json"},
{map[string]any{"record": r}, "mirrors --json --record " + r},
{map[string]any{"record": r, "confirm": "true", "why": "copied before copies were recorded"},
"mirrors --json --record " + r + " --confirm --why copied before copies were recorded"},
} {
argv, err := argvFor("mirrors", c.args)
if err != nil || strings.Join(argv, " ") != c.want {
t.Errorf("mirrors %v: %q %v, want %q", c.args, argv, err, c.want)
}
}
for _, args := range []map[string]any{
{"record": r, "confirm": "true"}, // recorded without a why
{"record": r, "why": "x"}, // a why for a dry run
{"confirm": "true", "why": "x"}, // nothing to record
{"record": r, "confirm": "yes", "why": "x"}, // a switch is true or false
{"delete": "true"},
} {
if argv, err := argvFor("mirrors", args); err == nil {
t.Errorf("mirrors %v was composed as %q", args, argv)
}
}
if repairingCommand([]string{"mirrors", "--json", "--record", r, "--confirm", "--why", "x"}) != "mirrors" {
t.Error("recording a copy through the generic verb would go unrecorded")
}
if repairingCommand([]string{"mirrors", "--json", "--record", r}) != "" {
t.Error("a dry run was taken for an act by hand")
}
if !personsDecision(link.HandAct{Verb: "mirrors"}) {
t.Error("recording a copy would count toward a healer the mesh lacks")
}
}
func TestOnlyACopyInARepositoryTheMirrorWritesIsTaken(t *testing.T) {
for given, ok := range map[string]bool{
ref("route-proxy", "on-go_base", 1): true,
catalogue.ArtifactStoreScheme + "upstream/docker.io/library/golang@sha256:" + strings.Repeat("a", 64): true,
ref("route-proxy", "server", 1): false, // a module's own image
catalogue.ArtifactStoreScheme + "novox/invoicing-api@sha256:" + strings.Repeat("a", 64): false,
archiveRef("web", "on-tools", 1): false, // a blob
catalogue.ArtifactStoreScheme + "web/on-x@sha256:abc": false, // not a whole digest
"docker.io/library/golang@sha256:" + strings.Repeat("a", 64): false,
} {
if _, err := mirrorReference(given); (err == nil) != ok {
t.Errorf("%s: taken %v, want %v (%v)", given, err == nil, ok, err)
}
}
}
// heldStore answers a manifest HEAD with 200 for the digests it holds, and records every request.
func heldStore(t *testing.T, holds ...string) (artifacts.Store, *[]string) {
t.Helper()
var asked []string
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
asked = append(asked, r.Method+" "+r.URL.Path)
if r.Method == http.MethodHead && slices.ContainsFunc(holds, func(d string) bool { return strings.HasSuffix(r.URL.Path, d) }) {
w.WriteHeader(http.StatusOK)
return
}
w.WriteHeader(http.StatusNotFound)
}))
t.Cleanup(server.Close)
return artifacts.Store{Address: strings.TrimPrefix(server.URL, "http://")}, &asked
}
func TestRecordingACopyTakesWhatTheStoreHoldsAndDeletesNothing(t *testing.T) {
inv := inventory.ForTest(t)
held := ref("route-proxy", "on-go_base", 1)
absent := ref("route-proxy", "on-go_base", 2)
known := ref("nats", "on-nats_base", 3)
notACopy := ref("route-proxy", "server", 4)
if err := inv.RecordMirrored(t.Context(), "b1", "", []string{known}); err != nil {
t.Fatal(err)
}
store, asked := heldStore(t, fmt.Sprintf("%064x", 1))
given := []string{held, absent, known, notACopy}
mirrored, err := inv.Mirrored(t.Context())
if err != nil {
t.Fatal(err)
}
// A dry run says, and records nothing.
dry, err := recordMirrors(t.Context(), inv, store, given, mirrored, "", false)
if err != nil {
t.Fatal(err)
}
if !dry.DryRun || !slices.Equal(dry.Recorded, []string{held}) || !slices.Equal(dry.AlreadyRecorded, []string{known}) ||
len(dry.Refused) != 2 || dry.Refused[absent] == "" || dry.Refused[notACopy] == "" {
t.Fatalf("a dry run answered %+v", dry)
}
if !strings.Contains(dry.Then, "would become eligible") || !strings.Contains(dry.Then, "next sweep") {
t.Fatalf("a dry run did not say what recording does: %q", dry.Then)
}
if again, _ := inv.Mirrored(t.Context()); again[held] {
t.Fatal("a dry run recorded a copy")
}
// A real one records what the store holds, with the person's why, and the sweep may now decide.
real, err := recordMirrors(t.Context(), inv, store, given, mirrored, "copied before copies were recorded", true)
if err != nil {
t.Fatal(err)
}
if !slices.Equal(real.Recorded, []string{held}) {
t.Fatalf("recorded %v", real.Recorded)
}
left, err := inv.ToCollect(t.Context())
if err != nil {
t.Fatal(err)
}
if !slices.Contains(left, held) {
t.Fatalf("a recorded copy no kept build stood on is not offered to collect: %v", left)
}
for _, r := range *asked {
if !strings.HasPrefix(r, "HEAD ") {
t.Fatalf("recording a copy asked the store %s", r)
}
}
}
// indexRegistry holds indexes and platforms of one image's repository, answers GETs with their
// documents, refuses GETs for the digests in fail, and records every delete.
type indexRegistry struct {
mu sync.Mutex
indexes map[string][]string
held map[string]bool
fail map[string]bool
deleted []string
}
func (f *indexRegistry) serve(t *testing.T) artifacts.Store {
t.Helper()
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
digest := r.URL.Path[strings.LastIndex(r.URL.Path, "/")+1:]
switch r.Method {
case http.MethodGet:
if f.fail[digest] {
w.WriteHeader(http.StatusInternalServerError)
return
}
if children, ok := f.indexes[digest]; ok && f.held[digest] {
var named []string
for _, c := range children {
named = append(named, `{"mediaType":"application/vnd.oci.image.manifest.v1+json","digest":"`+c+`"}`)
}
_, _ = w.Write([]byte(`{"schemaVersion":2,"mediaType":"application/vnd.oci.image.index.v1+json","manifests":[` +
strings.Join(named, ",") + `]}`))
return
}
if f.held[digest] {
_, _ = w.Write([]byte(`{"schemaVersion":2,"layers":[]}`))
return
}
w.WriteHeader(http.StatusNotFound)
case http.MethodHead:
if f.held[digest] {
w.WriteHeader(http.StatusOK)
return
}
w.WriteHeader(http.StatusNotFound)
case http.MethodDelete:
if !f.held[digest] {
w.WriteHeader(http.StatusNotFound)
return
}
delete(f.held, digest)
f.deleted = append(f.deleted, digest)
w.WriteHeader(http.StatusAccepted)
default:
w.WriteHeader(http.StatusBadRequest)
}
}))
t.Cleanup(server.Close)
return artifacts.Store{Address: strings.TrimPrefix(server.URL, "http://")}
}
func copyDigest(n int) string { return fmt.Sprintf("sha256:%064x", n) }
// twoCopies records, in one image's repository, a kept copy (stood on by a held module's build) naming
// platforms 11 and 12, and an eligible one (a failed build's) naming 12 and 13.
func twoCopies(t *testing.T, inv *inventory.Inventory) (kept, eligible string, reg *indexRegistry) {
t.Helper()
const repository = "upstream/docker.io/library/golang"
kept = catalogue.ArtifactStoreScheme + repository + "@" + copyDigest(1)
eligible = catalogue.ArtifactStoreScheme + repository + "@" + copyDigest(2)
if err := inv.RegisterModule(t.Context(), catalogue.Manifest{Module: "proxy", Version: "1"},
inventory.Source{Repository: "https://forge.invalid/proxy.git"}); err != nil {
t.Fatal(err)
}
failed := inventory.Build{ID: "f1", Repository: "https://forge.invalid/proxy.git", On: "a-build-machine",
Failed: "a recipe refused", Mirrored: []string{eligible}}
if err := inv.RecordBuild(t.Context(), failed); err != nil {
t.Fatal(err)
}
ok := inventory.Build{ID: "b1", Repository: "https://forge.invalid/proxy.git", Module: "proxy", On: "a-build-machine",
Commit: "c0ffee", Made: []inventory.Artifact{{Name: "app", Kind: "image", Reference: ref("proxy", "app", 7)}},
Against: []string{kept}, Mirrored: []string{kept}}
if err := inv.RecordBuild(t.Context(), ok); err != nil {
t.Fatal(err)
}
reg = &indexRegistry{
indexes: map[string][]string{copyDigest(1): {copyDigest(11), copyDigest(12)}, copyDigest(2): {copyDigest(12), copyDigest(13)}},
held: map[string]bool{copyDigest(1): true, copyDigest(2): true, copyDigest(11): true, copyDigest(12): true, copyDigest(13): true},
fail: map[string]bool{},
}
return kept, eligible, reg
}
// Through the records: a confirmed collect lets the eligible index go with the platform only it names,
// and leaves the platform the kept index names.
func TestAConfirmedCollectLetsAnIndexGoWithItsOwnPlatformsOnly(t *testing.T) {
inv := inventory.ForTest(t)
_, eligible, reg := twoCopies(t, inv)
store := reg.serve(t)
references, err := inv.ToCollect(t.Context())
if err != nil {
t.Fatal(err)
}
if !slices.Equal(references, []string{eligible}) {
t.Fatalf("offered %v, want the failed build's copy", references)
}
a := runCollect(t.Context(), inv, store, references, nil, true,
sweepBounds{most: 10, budget: 5 * time.Second, platforms: true})
if !slices.Equal(a.LetGo, []string{eligible}) || a.Stopped != "" {
t.Fatalf("let go %v, stopped %q", a.LetGo, a.Stopped)
}
if !slices.Equal(reg.deleted, []string{copyDigest(13), copyDigest(2)}) {
t.Fatalf("deleted %v; want its own platform, then the index", reg.deleted)
}
}
// The sweep after a build lets an index go alone: its platforms wait for a person's collect.
func TestTheSweepAfterABuildLetsAnIndexGoAlone(t *testing.T) {
inv := inventory.ForTest(t)
_, eligible, reg := twoCopies(t, inv)
store := reg.serve(t)
r := sweep(t.Context(), inv, store, []string{eligible}, nil, afterBuild)
if !slices.Equal(r.LetGo, []string{eligible}) || !slices.Equal(reg.deleted, []string{copyDigest(2)}) {
t.Fatalf("after a build: let go %v, deleted %v; want the index alone", r.LetGo, reg.deleted)
}
}
// A kept index the store will not answer for stops a confirmed collect before its first delete.
func TestASpareListThatCannotBeReadStopsTheSweepBeforeAnyDelete(t *testing.T) {
inv := inventory.ForTest(t)
_, eligible, reg := twoCopies(t, inv)
reg.fail[copyDigest(1)] = true
store := reg.serve(t)
a := runCollect(t.Context(), inv, store, []string{eligible}, nil, true,
sweepBounds{most: 10, budget: 5 * time.Second, platforms: true})
if len(a.LetGo) != 0 || len(reg.deleted) != 0 || !strings.Contains(a.Stopped, "nothing was let go") {
t.Fatalf("let go %v, deleted %v, stopped %q", a.LetGo, reg.deleted, a.Stopped)
}
}
+115 -4
View File
@@ -73,7 +73,7 @@ func stateHealth(ctx context.Context, inv *inventory.Inventory, k *conditions.Ke
for _, r := range h.Resources {
kept := inventory.ResourceHealth{Module: r.Module, Resource: r.Resource, Kind: r.Kind, Target: r.Target,
State: r.State, Reason: r.Reason, Since: r.Since, Streak: r.Streak, Restarts: r.Restarts,
Check: r.Check, Needs: r.Needs}
Check: r.Check, Needs: r.Needs, Account: r.Account}
resources = append(resources, kept)
if r.State == link.StateUnhealthy && r.Module != "" {
unhealthy[r.Module] = append(unhealthy[r.Module], kept)
@@ -135,7 +135,7 @@ func judgeModuleHealth(ctx context.Context, inv *inventory.Inventory, k *conditi
}
standing := map[string]conditions.Condition{}
for _, c := range open {
if c.Kind == kindModuleUnhealthy && c.Subject.Machine == node {
if (c.Kind == kindModuleUnhealthy || c.Kind == kindReloginNeeded) && c.Subject.Machine == node {
standing[c.Key] = c
}
}
@@ -155,6 +155,19 @@ func judgeModuleHealth(ctx context.Context, inv *inventory.Inventory, k *conditi
heldOn := map[string]string{}
providers := map[catalogue.Chosen]bool{}
for _, m := range modules {
// **A wait for a person's new login is said as that** (novox/hq ADR 0254): one plain sentence to the
// operator, never urgent, cleared on the first statement that no longer says it.
if said, waits := personWait(m, node, unhealthy[m]); waits {
o := reloginObservation(m, node, said, unhealthy[m])
seen[o.Key()] = true
if _, isOpen := standing[o.Key()]; streaks[m] < moduleUnhealthyAfter && !isOpen {
continue
}
if _, err := k.Observe(ctx, o); err != nil {
problems = append(problems, err.Error())
}
continue
}
o := moduleUnhealthyObservation(m, node, unhealthy[m])
if hold != nil {
if p, held := hold.heldUnder(node, m, unhealthy[m]); held {
@@ -188,6 +201,9 @@ func judgeModuleHealth(ctx context.Context, inv *inventory.Inventory, k *conditi
}
module := strings.TrimSuffix(c.Subject.ID, "."+node)
why := fmt.Sprintf("%s says no resource of %s is unhealthy", node, module)
if c.Kind == kindReloginNeeded {
why = fmt.Sprintf("%s says %s no longer waits for a new login", node, module)
}
if on, held := heldOn[key]; held {
why = fmt.Sprintf("what %s finds on %s waits on %s, which is unhealthy: held under its condition", module, node, on)
}
@@ -239,11 +255,33 @@ func sayWaiters(ctx context.Context, k *conditions.Keeper, hold *holding, p cata
return err
}
// reloginObservation is a module waiting for a person's new login on a machine (novox/hq ADR 0254): the
// operator's, a warning, its summary the one sentence that says what to do; the resources are evidence. Its
// plain words (ADR 0253) are the kind's: it needs the operator, and offers no answer — no verb can log a
// person in again, and a restart of the module's service would start it in the same session.
func reloginObservation(module, node, said string, rs []inventory.ResourceHealth) conditions.Observation {
o := moduleUnhealthyObservation(module, node, rs)
o.Token, o.Kind, o.Resolver, o.Summary = kindReloginNeeded, kindReloginNeeded, conditions.ResolverOperator, said
w := reloginWords(module, node)
o.Headline, o.Explanation, o.Resolved, o.Needs, o.Actions = w.Headline, w.Explanation, w.Resolved, w.Needs, nil
return o
}
// reloginWords is what the operator reads of a module waiting for their new login on a machine (ADR 0253,
// ADR 0254): never quiet, since only the operator can do it, and no button, since nothing else can.
func reloginWords(module, node string) words {
return words{Headline: fmt.Sprintf("%s waits for a new login on %s", module, node),
Needs: fmt.Sprintf("log out of %s completely and log in again, or restart it.", node),
Explanation: fmt.Sprintf("%s put your account in a group it needs. You logged in before that, so %s "+
"cannot run until you log in again. Its update is in place and nothing was undone.", conditions.Capital(module), module),
Resolved: fmt.Sprintf("%s runs on %s after your new login", module, node)}
}
// moduleUnhealthyObservation is a module unhealthy on a machine, in words: the summary names the module,
// the machine and what is wrong with each resource; the detail — targets, streaks, since — is evidence.
func moduleUnhealthyObservation(module, node string, rs []inventory.ResourceHealth) conditions.Observation {
var words, said, plain []string
needs, actions := moduleNeeds(node, rs)
needs := moduleNeeds(node, rs)
for _, r := range rs {
plain = append(plain, resourcePlainWords(r))
if r.Kind == link.KindUnit {
@@ -267,7 +305,6 @@ func moduleUnhealthyObservation(module, node string, rs []inventory.ResourceHeal
Explanation: fmt.Sprintf("%s on %s is not healthy: %s. It clears as soon as it runs again.", module, node,
namesWords(plain, 3)),
Needs: needs,
Actions: actions,
Resolved: fmt.Sprintf("%s works again on %s", module, node)}
}
@@ -296,9 +333,76 @@ func orNotSaid(s string) string {
return s
}
// kindReloginNeeded is a module's condition while it waits for a person's new login on a machine (novox/hq
// ADR 0254): its own kind, so that neither the gate nor anyone reading the conditions takes it for a fault.
const kindReloginNeeded = "relogin-needed"
// personWait is whether everything unhealthy of a module on a machine waits for one person's new login, and
// that wait in one plain sentence (novox/hq ADR 0254, issue 318). Narrow on purpose, so that a fault is never
// excused:
//
// - at least one of the module's resources is its account (kind account), unhealthy with a reason that
// starts "relogin needed" (ADR 0252): the database lists the account in the group, and the session
// running began before;
// - every other unhealthy resource of the module runs in the own service manager of one of those very
// accounts (Account names it): the manager that began before the group and does not hold it.
//
// Anything else unhealthy of the module — a container, a unit of the machine's own manager, a unit in
// another account's manager, a resource whose engine does not say whose manager it is in, an account
// not in its group or that could not be read — is not a wait, and the module is judged as before. A
// resource still starting is not unhealthy and does not make a wait either. Pure.
func personWait(module, machine string, rs []inventory.ResourceHealth) (string, bool) {
waiting := map[string]bool{}
var accounts []string
for _, r := range rs {
if r.Module == module && r.Kind == link.KindAccount && r.State == link.StateUnhealthy &&
strings.HasPrefix(r.Reason, link.ReasonRelogin) && accountOf(r) != "" && !waiting[accountOf(r)] {
waiting[accountOf(r)] = true
accounts = append(accounts, accountOf(r))
}
}
if len(accounts) == 0 {
return "", false
}
var units []string
for _, r := range rs {
if r.Module != module || r.State != link.StateUnhealthy {
continue
}
switch {
case r.Kind == link.KindAccount && strings.HasPrefix(r.Reason, link.ReasonRelogin) && waiting[accountOf(r)]:
case r.Kind != link.KindAccount && r.Account != "" && waiting[r.Account]:
units = append(units, r.Target)
default:
return "", false
}
}
sort.Strings(accounts)
said := fmt.Sprintf("relogin needed on %s: %s waits for a new login of %s, which is in its group and whose "+
"running session began before it was; log out of every session and in again, or reboot", machine, module,
strings.Join(accounts, ", "))
if len(units) > 0 {
sort.Strings(units)
said += fmt.Sprintf(" (until then %s cannot run in that session)", strings.Join(units, ", "))
}
return said, true
}
// accountOf is the account a resource of kind account is: named by the engine, or its target.
func accountOf(r inventory.ResourceHealth) string {
if r.Account != "" {
return r.Account
}
return r.Target
}
// moduleHealthWord is the gate's reading of a module's stated health on a machine (ADR 0240 §4, ADR 0236
// §2 as amended): good when every long-running resource of it is stated healthy in a statement heard since
// the send; not yet otherwise, saying which. A machine that never stated health is judged as before.
//
// **A wait for a person is its own reading** (novox/hq ADR 0254): when everything unhealthy of the module
// waits for one person's new login (personWait), the reading is healthPerson — not a fault of the build,
// and not something a machine can meet within a bound.
func moduleHealthWord(module, machine string, since time.Time, f gateFacts) (health, string) {
if f.healthErr != nil {
return healthNotYet, "what " + machine + " says of its resources' health cannot be read: " + firstLine(f.healthErr.Error())
@@ -310,10 +414,14 @@ func moduleHealthWord(module, machine string, since time.Time, f gateFacts) (hea
if h.HeardAt.Before(since) {
return healthNotYet, fmt.Sprintf("%s has not said how what %s runs is since it was sent", machine, module)
}
wait, waits := personWait(module, machine, h.Resources)
for _, r := range h.Resources {
if r.Module != module {
continue
}
if waits && r.State == link.StateUnhealthy {
continue
}
switch r.State {
case link.StateHealthy:
case link.StateStarting:
@@ -329,6 +437,9 @@ func moduleHealthWord(module, machine string, since time.Time, f gateFacts) (hea
reasonAfter(r.Reason))
}
}
if waits {
return healthPerson, wait
}
return healthGood, ""
}
+214
View File
@@ -0,0 +1,214 @@
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A wait for a person is not a failure, and one module's verdict is not every module's (novox/hq ADR 0254,
// issue 318).
func relogin(module, account string) inventory.ResourceHealth {
return inventory.ResourceHealth{Module: module, Resource: module + "." + account, Kind: link.KindAccount, Target: account,
Account: account, State: link.StateUnhealthy, Reason: link.ReasonRelogin + ": " + account + " is in the group " + module +
", and its running session began before it was; log out of every session and in again, or reboot"}
}
func userUnit(module, account string) inventory.ResourceHealth {
return inventory.ResourceHealth{Module: module, Resource: module + ".daemon", Kind: link.KindUnit,
Target: module + "-daemon.service", Account: account, State: link.StateUnhealthy,
Reason: "failed in the account's own service manager (exit-code)"}
}
// Only what depends on the new login alone is a wait; anything else unhealthy of the module is judged as
// before, so that a fault is never excused.
func TestOnlyWhatDependsOnTheNewLoginIsAWait(t *testing.T) {
notInGroup := relogin("lights", "operator")
notInGroup.Reason = "not in the group lights: the apply has not put operator there; its outcome says why"
systemUnit := userUnit("lights", "operator")
systemUnit.Account = ""
otherAccount := userUnit("lights", "guest")
container := inventory.ResourceHealth{Module: "lights", Resource: "lights.web", Kind: "container", Target: "lights",
State: link.StateUnhealthy, Reason: "down"}
otherModule := userUnit("sound", "operator")
starting := userUnit("lights", "operator")
starting.State = link.StateStarting
byTarget := relogin("lights", "operator")
byTarget.Account = "" // an account said by its target alone is still that account
for _, c := range []struct {
name string
rs []inventory.ResourceHealth
waits bool
}{
{"the account alone", []inventory.ResourceHealth{relogin("lights", "operator")}, true},
{"the account and its unit in that account's manager", []inventory.ResourceHealth{relogin("lights", "operator"),
userUnit("lights", "operator")}, true},
{"an account named by its target", []inventory.ResourceHealth{byTarget, userUnit("lights", "operator")}, true},
{"another module's unit is not this module's", []inventory.ResourceHealth{relogin("lights", "operator"), otherModule}, true},
{"a unit still starting is no fault", []inventory.ResourceHealth{relogin("lights", "operator"), starting}, true},
{"a unit and no account", []inventory.ResourceHealth{userUnit("lights", "operator")}, false},
{"an account not in its group", []inventory.ResourceHealth{notInGroup, userUnit("lights", "operator")}, false},
{"a unit whose manager is not said", []inventory.ResourceHealth{relogin("lights", "operator"), systemUnit}, false},
{"a unit in another account's manager", []inventory.ResourceHealth{relogin("lights", "operator"), otherAccount}, false},
{"a container beside the wait", []inventory.ResourceHealth{relogin("lights", "operator"), container}, false},
} {
said, waits := personWait("lights", "laptop", c.rs)
if waits != c.waits {
t.Errorf("%s: waits %v; want %v", c.name, waits, c.waits)
continue
}
if waits && !strings.HasPrefix(said, "relogin needed on laptop: lights waits for a new login of operator") {
t.Errorf("%s: said %q", c.name, said)
}
}
}
// The gate reads a wait for a person as its own reading, never a fault, and passes with it; the same module
// with a container down beside it is not yet healthy, as before.
func TestAWaitForAPersonIsAPassCarriedAlong(t *testing.T) {
now := time.Now()
since := now.Add(-time.Minute)
f := gateFacts{now: now, health: map[string]inventory.NodeHealth{"laptop": {Node: "laptop", HeardAt: now,
Resources: []inventory.ResourceHealth{relogin("lights", "operator"), userUnit("lights", "operator")}}}}
if h, why := moduleHealthWord("lights", "laptop", since, f); h != healthPerson || !strings.Contains(why, "relogin needed on laptop") {
t.Fatalf("a wait for a new login reads %v %q; want a wait for a person", h, why)
}
h := f.health["laptop"]
h.Resources = append(h.Resources, inventory.ResourceHealth{Module: "lights", Resource: "lights.web", Kind: "container",
Target: "lights", State: link.StateUnhealthy, Reason: "down"})
f.health["laptop"] = h
if got, why := moduleHealthWord("lights", "laptop", since, f); got != healthNotYet {
t.Fatalf("a container down beside the wait reads %v %q; want not yet", got, why)
}
}
// The module's condition says the wait in one sentence for a person, and clears on the first statement that
// no longer says it — said as the module's own fault when its unit still fails after the new login.
func TestTheReloginConditionSaysTheWaitAndClearsAfterTheLogin(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
k, _ := withConditionsInMemory(t)
at := h0
say := func(rs ...link.ResourceHealth) {
t.Helper()
at = at.Add(time.Second)
for i := range rs {
rs[i].Since = at
}
if err := stateHealth(ctx, open.inventory, k, "laptop", link.Health{Contract: link.ReadinessContract, At: at,
Resources: rs}, at); err != nil {
t.Fatal(err)
}
}
asLink := func(r inventory.ResourceHealth) link.ResourceHealth {
return link.ResourceHealth{Module: r.Module, Resource: r.Resource, Kind: r.Kind, Target: r.Target, State: r.State,
Reason: r.Reason, Account: r.Account}
}
account, unit := asLink(relogin("lights", "operator")), asLink(userUnit("lights", "operator"))
openNow := func() []conditions.Condition {
t.Helper()
list, err := k.Open(ctx)
if err != nil {
t.Fatal(err)
}
return list
}
say(account, unit)
say(account, unit)
list := openNow()
if len(list) != 1 || list[0].Key != "module.lights.laptop.relogin-needed" {
t.Fatalf("raised %+v; want lights' relogin-needed alone", list)
}
c := list[0]
if c.Severity != conditions.Warning || c.Resolver != conditions.ResolverOperator ||
!strings.HasPrefix(c.Summary, "relogin needed on laptop:") {
t.Fatalf("the condition is %s %s %q", c.Severity, c.Resolver, c.Summary)
}
// After the new login the account is healthy, and the unit, still failing, is the module's own fault.
account.State, account.Reason = link.StateHealthy, ""
say(account, unit)
list = openNow()
if len(list) != 1 || list[0].Key != "module.lights.laptop.unhealthy" {
t.Fatalf("after the login: %+v; want lights' own unhealthy condition alone", list)
}
unit.State, unit.Reason = link.StateHealthy, ""
say(account, unit)
if list = openNow(); len(list) != 0 {
t.Fatalf("after the unit runs: %+v; want nothing open", list)
}
}
// A module that is healthy on its own keeps its pass when another module of the same send fails at the
// bound: it is neither put back nor left without a verdict for every other walk to wait on.
func TestAModuleHealthyOnItsOwnKeepsItsPassWhenItsSendFails(t *testing.T) {
b := aBacklog(t)
ctx := t.Context()
inv := b.open.inventory
releaseHeard = func(context.Context, *stores) (map[string]bool, error) {
return map[string]bool{"laptop": true}, nil
}
backlogFacts := gatherGateFacts
gatherGateFacts = func(ctx context.Context, open *stores, component string) (gateFacts, error) {
f, err := backlogFacts(ctx, open, component)
f.health = map[string]inventory.NodeHealth{"laptop": {Node: "laptop", HeardAt: time.Now(),
Resources: []inventory.ResourceHealth{
{Module: "app", Resource: "app.web", Kind: "container", Target: "app", State: link.StateHealthy},
{Module: "late", Resource: "late.web", Kind: "container", Target: "late", State: link.StateUnhealthy,
Reason: "down"}}}}
return f, err
}
gateEvery, gateBound = 0, 300*time.Millisecond
deadline := time.Now().Add(5 * time.Second)
for time.Now().Before(deadline) {
advancePlans(ctx, b.open)
if p := b.release(t); p.State != inventory.PlanRolling {
break
}
time.Sleep(20 * time.Millisecond)
}
p := b.release(t)
if p.State != inventory.PlanFailed {
t.Fatalf("the walk is %s: %s; want it failed on late", p.State, p.Note)
}
if v, found, err := inv.GateOf(ctx, "build-app-c2"); err != nil || !found || v.Verdict != inventory.GatePassed ||
!strings.Contains(v.Why, "on its own") {
t.Fatalf("app's verdict: %+v (found %v, %v); want its own pass kept", v, found, err)
}
if failed, _ := inv.GateFailed(ctx, "build-late-c2"); !failed {
t.Fatal("late's build is not marked failed at its gate")
}
current, _ := inv.CurrentBuilds(ctx)
if current["app"].Commit != "c2" || current["late"].Commit != "c1" {
t.Fatalf("registered: app %s, late %s; want app kept at c2 and late put back to c1", current["app"].Commit,
current["late"].Commit)
}
if !strings.Contains(p.Note, "passed on their own and kept: app") {
t.Errorf("the walk does not say what kept its pass: %s", p.Note)
}
}
// What the operator reads of the wait (ADR 0253, ADR 0254): it needs them, so it is never quiet, and it offers
// no button, since nothing but their own new login can do it.
func TestTheReloginConditionNeedsTheOperatorAndOffersNoButton(t *testing.T) {
o := reloginObservation("openrazer", "g14", "relogin needed on g14: …", []inventory.ResourceHealth{
relogin("openrazer", "operator"), userUnit("openrazer", "operator")})
plainExample(t, o, "openrazer waits for a new login on g14",
"Needs you: log out of g14 completely and log in again, or restart it. Openrazer put your account in a "+
"group it needs. You logged in before that, so openrazer cannot run until you log in again. Its "+
"update is in place and nothing was undone.")
if len(o.Actions) != 0 || o.Needs == "" {
t.Errorf("relogin: needs %q, actions %+v; want needs and no button", o.Needs, o.Actions)
}
// The kind's own wording, for a condition raised without words, says the same.
if w := plainWordings[kindReloginNeeded](conditions.Observation{Scope: conditions.ScopeModule, ID: "openrazer.g14", Machine: "g14"}); w.Needs != o.Needs ||
w.Headline != o.Headline || len(w.Actions) != 0 {
t.Errorf("the kind's wording: %+v", w)
}
}
+46 -41
View File
@@ -183,6 +183,13 @@ var plainWordings = map[string]func(conditions.Observation) words{
Explanation: fmt.Sprintf("%s is not healthy. It clears as soon as it runs again.", conditions.Capital(thing)),
Resolved: conditions.Capital(thing) + " works again"}
}),
kindReloginNeeded: worded(func(o conditions.Observation) words {
module := ""
if o.Scope == conditions.ScopeModule {
module = idPart(o, 0)
}
return reloginWords(orModule(module), machineOr(o, "a machine"))
}),
kindProviderFailing: worded(func(o conditions.Observation) words {
thing, consumer := conditions.ThingWords(o), idPart(o, 2)
if consumer == "" {
@@ -247,9 +254,10 @@ var plainWordings = map[string]func(conditions.Observation) words{
kindDataMissing: worded(func(o conditions.Observation) words {
what, _ := dataWords(o)
return words{Headline: conditions.Capital(what) + " is gone",
Needs: "restore it from a backup, or silence this if you removed it.",
// No silence from a click: data loss is never waved away from a notification (ADR 0258).
Needs: "restore it from a backup, or silence this " + FromMeshMCPServer,
Explanation: fmt.Sprintf("Where %s is kept on %s, nothing exists any more.", what, machineOr(o, "its machine")),
Resolved: conditions.Capital(what) + " is back", Actions: []conditions.Action{conditions.SilenceAction(o.Key())}}
Resolved: conditions.Capital(what) + " is back"}
}),
kindDataShrank: worded(func(o conditions.Observation) words {
what, _ := dataWords(o)
@@ -258,9 +266,10 @@ var plainWordings = map[string]func(conditions.Observation) words{
what = "a dataset"
}
return words{Headline: conditions.Capital(what) + " shrank on " + m,
Needs: "if you meant it, silence this; if not, restore the last good copy from a backup.",
// No silence from a click: data loss is never waved away from a notification (ADR 0258).
Needs: "restore the last good copy from a backup, or silence this " + FromMeshMCPServer,
Explanation: "More than half of what it held is gone within a week.",
Resolved: "Resolved: the shrinking on " + m + " is explained", Actions: []conditions.Action{conditions.SilenceAction(o.Key())}}
Resolved: "Resolved: the shrinking on " + m + " is explained"}
}),
kindDataQuiet: worded(func(o conditions.Observation) words {
what, _ := dataWords(o)
@@ -630,48 +639,39 @@ func walkWaitingWords(w waitFacts, in time.Duration, severity conditions.Severit
// waitingNeeds is what the operator does about a walk waiting past its urgent bound: nothing before it.
func waitingNeeds(severity conditions.Severity) string {
if severity == conditions.Urgent {
return "start it, or stop it."
return "start it, or stop it, " + FromMeshMCPServer
}
return ""
}
// waitingActions are the answers to a walk waiting past its urgent bound: the controller's own verb, since
// the module that should have said go is the one not answering.
func waitingActions(w waitFacts, severity conditions.Severity) []conditions.Action {
if severity != conditions.Urgent {
return nil
}
return []conditions.Action{
{Label: "Start", Verb: "mesh-controller.plans", Arguments: map[string]string{"go": w.id, "why": ""}},
{Label: "Stop", Verb: "mesh-controller.plans", Arguments: map[string]string{"stop": w.id, "why": ""}},
}
}
// moduleNeeds is what the operator can do about a module unhealthy on a machine: log in again where its
// account's groups wait for it (ADR 0252), restart a failed service, or nothing where the mesh restarts it.
func moduleNeeds(node string, rs []inventory.ResourceHealth) (string, []conditions.Action) {
var actions []conditions.Action
// No answer is offered for a restart: a desk click performs only an acknowledgement (ADR 0258).
func moduleNeeds(node string, rs []inventory.ResourceHealth) string {
unit := ""
for _, r := range rs {
if strings.Contains(r.Reason, "relogin needed") {
return fmt.Sprintf("log out of every session on %s and log in again.", node), nil
return reloginNeeds(node)
}
if r.Kind == link.KindUnit && len(actions) < 2 {
scope := "system"
if strings.Contains(r.Reason, "account's own") {
scope = "user"
}
label := "Restart"
if len(actions) > 0 {
label = "Restart " + unitPlainWords(r.Target)
}
actions = append(actions, conditions.Action{Label: label, Verb: "node-service-manager.restart",
Machine: node, Arguments: map[string]string{"unit": r.Target, "scope": scope}})
if r.Kind == link.KindUnit && unit == "" {
unit = unitPlainWords(r.Target)
}
}
if len(actions) > 0 {
return "restart it; if it fails again, the details say why.", actions
if unit != "" {
return fmt.Sprintf("restart its service %s on %s %s", unit, node, FromMeshMCPServer)
}
return "", nil
return ""
}
// FromMeshMCPServer ends what the operator needs when no notification can do it (ADR 0258), naming the mesh MCP
// server (the glossary's word; "console" is retired): the answer is not an
// acknowledgement, so it is given where the operator is known to be the one asking, until answers are
// authorised (to-be 46 phases 5 and 6).
const FromMeshMCPServer = "from the mesh MCP server; this notification cannot do it."
// reloginNeeds is what an account waiting for its groups needs (ADR 0252).
func reloginNeeds(node string) string {
return fmt.Sprintf("log out of every session on %s and log in again.", node)
}
// stalledWords are the plain words of a delivery held past its bound, as mesh-delivery says it.
@@ -688,18 +688,15 @@ func stalledWords(l stalledLine, o conditions.Observation) (headline, explanatio
long = "for " + humanDuration(d)
}
if o.Resolver == conditions.ResolverOperator {
// Words only: releasing or stopping a delivery is not an acknowledgement, so no desk click
// performs it (ADR 0258).
switch held {
case "held":
needs = "release it, or stop it."
actions = []conditions.Action{
{Label: "Release", Verb: "mesh-delivery.release", Arguments: map[string]string{"id": l.ID, "why": ""}},
{Label: "Stop", Verb: "mesh-delivery.stop", Arguments: map[string]string{"id": l.ID, "why": ""}},
}
needs = "release it, or stop it, " + FromMeshMCPServer
case "ready", "checked":
needs = "merge its pull request, or close it."
default:
needs = "stop it, or read the details to see what it waits for."
actions = []conditions.Action{{Label: "Stop", Verb: "mesh-delivery.stop", Arguments: map[string]string{"id": l.ID, "why": ""}}}
needs = "stop it " + FromMeshMCPServer
}
}
return fmt.Sprintf("Delivery of %s %s %s", name, held, long),
@@ -759,3 +756,11 @@ func unitReasonWords(reason string) string {
}
return "failed"
}
// orModule is a module's name, or "a module" when none is known.
func orModule(m string) string {
if m == "" {
return "a module"
}
return m
}
+58 -23
View File
@@ -68,15 +68,9 @@ func TestADeliveryWaitingNeedsNothingUntilItsBoundThenOffersStartAndStop(t *test
f.waits[0].since = now.Add(-5 * time.Hour)
got = watchWaits(f)
plainExample(t, got[0], "openrazer delivery waiting to start",
"Needs you: start it, or stop it. The change to openrazer is merged and built, and mesh-delivery (the "+
"Needs you: start it, or stop it, from the mesh MCP server; this notification cannot do it. The change to openrazer is merged and built, and mesh-delivery (the "+
"module that decides when a delivery goes out) has not let it start for 5 hours, so mesh-delivery may "+
"be stuck.", "Start", "Stop")
if a := got[0].Actions[0]; a.Verb != "mesh-controller.plans" || a.Arguments["go"] != "plan-1791454185265004861" {
t.Errorf("start: %+v", a)
}
if a := got[0].Actions[1]; a.Verb != "mesh-controller.plans" || a.Arguments["stop"] != "plan-1791454185265004861" {
t.Errorf("stop: %+v", a)
}
"be stuck.")
// Many modules are counted, not listed in the headline.
f.waits[0].modules = []string{"a", "b", "c", "d"}
@@ -87,24 +81,32 @@ func TestADeliveryWaitingNeedsNothingUntilItsBoundThenOffersStartAndStop(t *test
}
// **A module unhealthy**: "openrazer on g14 is not healthy: its unit openrazer-daemon.service failed in the
// account's own service manager (exit-code)". The operator restarts it from the notification.
func TestAModuleUnhealthyOffersARestartOfItsService(t *testing.T) {
// account's own service manager (exit-code)". Restarting is not an acknowledgement, so it is said in words
// and offered as no answer (ADR 0258).
func TestAModuleUnhealthyAsksForARestartInWords(t *testing.T) {
o := moduleUnhealthyObservation("openrazer", "g14", []inventory.ResourceHealth{{Kind: link.KindUnit,
Resource: "openrazer-daemon", Target: "openrazer-daemon.service",
Reason: "failed in the account's own service manager (exit-code)", Since: time.Now()}})
plainExample(t, o, "openrazer not working on g14",
"Needs you: restart it; if it fails again, the details say why. openrazer on g14 is not healthy: its "+
"service openrazer-daemon stopped with an error. It clears as soon as it runs again.", "Restart")
if a := o.Actions[0]; a.Verb != "node-service-manager.restart" || a.Machine != "g14" ||
a.Arguments["unit"] != "openrazer-daemon.service" || a.Arguments["scope"] != "user" {
t.Errorf("restart: %+v", a)
}
// An account waiting for a new login (ADR 0252) asks for the login, which no button can give.
"Needs you: restart its service openrazer-daemon on g14 from the mesh MCP server; this notification cannot do it. "+
"openrazer on g14 is not healthy: its service openrazer-daemon stopped with an error. It clears as soon "+
"as it runs again.")
// An account waiting for a new login (ADR 0252) asks for the login, held to the plain rule.
o = moduleUnhealthyObservation("openrazer", "g14", []inventory.ResourceHealth{{Kind: "account",
Resource: "operator-in-group", Target: "jochen", Reason: "relogin needed: the account is in the group"}})
if o.Needs != "log out of every session on g14 and log in again." || len(o.Actions) != 0 {
t.Errorf("relogin: %q %+v", o.Needs, o.Actions)
}
w := conditions.Words{Headline: o.Headline, Explanation: o.Explanation, Resolved: o.Resolved, Needs: o.Needs}
if why, ok := conditions.PlainWords(w, "g14"); !ok {
t.Errorf("the relogin words are not plain: %s", why)
}
// And the kind issue 318 raises for it (ADR 0254).
rw := plainWordings["relogin-needed"](conditions.Observation{Scope: conditions.ScopeModule, ID: "openrazer.g14",
Machine: "g14", Kind: "relogin-needed", Severity: conditions.Warning})
if why, ok := conditions.PlainWords(rw, "g14"); !ok || rw.Needs == "" || len(rw.Actions) != 0 {
t.Errorf("relogin-needed: %s %+v", why, rw)
}
}
// **Failed units on a machine**: "shanks's service manager is degraded: 3 failed unit(s) no module places —
@@ -143,15 +145,12 @@ func TestAHealerWantedNeedsNothingFromTheOperator(t *testing.T) {
// **A delivery held past its bound**, as mesh-delivery says it: "the delivery novox/hq@055550802096 has been
// held for 36h2m6s, past its bound of 24h0m0s (it waits for the operator): healer H2 may none: …".
func TestADeliveryHeldOffersReleaseAndStop(t *testing.T) {
func TestADeliveryHeldAsksForReleaseOrStopInWords(t *testing.T) {
got := stalledObservations([]stalledLine{{ID: "novox/hq@055550802096", State: "held", For: "36h2m6s",
Bound: "24h0m0s", H2: "none: the state is the operator's", Says: "it waits for the operator"}})
plainExample(t, got[0], "Delivery of hq held for 36 hours",
"Needs you: release it, or stop it. A delivery of hq has been held for 36 hours, past its limit.",
"Release", "Stop")
if a := got[0].Actions[0]; a.Verb != "mesh-delivery.release" || a.Arguments["id"] != "novox/hq@055550802096" {
t.Errorf("release: %+v", a)
}
"Needs you: release it, or stop it, from the mesh MCP server; this notification cannot do it. A delivery of hq has been held for 36 hours, past its limit.",
)
}
// **Every kind the controller raises has plain words**, and its words are plain for a subject of every
@@ -184,3 +183,39 @@ func TestEveryWordingIsPlain(t *testing.T) {
}
}
}
// **An answer on a notification is no repair** (novox/hq ADR 0258): silencing chosen by the operator on the
// desk does not count toward a healer wanted; the same silence by hand for another cause still does.
func TestAnOperatorsAnswerIsNoHandRepair(t *testing.T) {
now := time.Date(2026, 10, 8, 12, 0, 0, 0, time.UTC)
f := calm(now)
for i := 0; i < 3; i++ {
a := actByHand(now.Add(-time.Duration(i+1)*time.Hour), conditions.CauseOperatorAnswer)
a.Verb = "conditions silence"
f.handActs = append(f.handActs, a)
}
if got := watchHandActs(f); len(got) != 0 {
t.Fatalf("an answer counted as a repair: %+v", got)
}
for i := range f.handActs {
f.handActs[i].Cause = "machine-units"
}
if got := watchHandActs(f); len(got) != 1 {
t.Fatalf("a silence by hand stopped counting: %+v", got)
}
}
// **Data loss is never silenced from a notification** (ADR 0258): data missing or shrunk offers no answer,
// and says where it is silenced.
func TestDataLossOffersNoSilence(t *testing.T) {
for _, kind := range []string{kindDataMissing, kindDataShrank} {
w := plainWordings[kind](conditions.Observation{Scope: conditions.ScopeMachine, ID: "ace.immich.library",
Machine: "ace", Kind: kind, Severity: conditions.Urgent})
if len(w.Actions) != 0 || !strings.HasSuffix(w.Needs, FromMeshMCPServer) {
t.Errorf("%s: %+v", kind, w)
}
if why, ok := conditions.PlainWords(w, "ace"); !ok {
t.Errorf("%s: %s", kind, why)
}
}
}
+6 -1
View File
@@ -419,7 +419,7 @@ func declarationWith(ctx context.Context, open *stores, node string,
out := sendable{Resources: composed.Resources, Adoption: adoption,
Received: composed.Received, Mesh: with.Mesh, BusUsers: with.BusUsers,
LeftOut: sortedKeysOf(composed.LeftOut), leftOutWhy: composed.LeftOut, withheld: with.Withheld,
unbound: with.Unbound, foreseen: composed.Foreseen}
unbound: with.Unbound, foreseen: composed.Foreseen, unplaced: composed.Unplaced}
// And which build of each module it carries, for the send to record (novox/hq issue 259, ADR
// 0221). Read only on the send path: a question about what would be sent records nothing.
if choosing == Allocating {
@@ -528,6 +528,11 @@ func reportLeftOut(node string, declared sendable) {
for _, u := range declared.unbound {
fmt.Printf("%s: %s\n", node, u)
}
// And every contribution its holder could not render, which the machine is sent without (novox/hq
// ADR 0255).
for _, u := range declared.unplaced {
fmt.Printf("%s: %s\n", node, u)
}
}
// busCredentialIssued refuses an own secret called `broker` whose bus account nobody issued.
+1 -1
View File
@@ -209,7 +209,7 @@ func collectCommand(ctx context.Context, args []string) error {
if *most < 1 {
return fmt.Errorf("collect: --most is at least 1, not %d", *most)
}
bounds := sweepBounds{most: min(*most, collectMostAtMost), budget: collectBudget}
bounds := sweepBounds{most: min(*most, collectMostAtMost), budget: collectBudget, platforms: true}
open, err := openStores(ctx)
if err != nil {
@@ -133,6 +133,10 @@ func (f *fakeStore) serve(t *testing.T) artifacts.Store {
w.WriteHeader(http.StatusOK)
case r.Method == http.MethodHead:
w.WriteHeader(http.StatusNotFound)
case r.Method == http.MethodGet && strings.Contains(r.URL.Path, "/manifests/"):
// An image, not an index: it names no platform manifests.
w.Header().Set("Content-Type", "application/vnd.oci.image.manifest.v1+json")
_, _ = w.Write([]byte(`{"schemaVersion":2,"mediaType":"application/vnd.oci.image.manifest.v1+json","layers":[]}`))
case r.Method == http.MethodPost:
w.Header().Set("Location", "/upload/x?state=1")
w.WriteHeader(http.StatusAccepted)
+37
View File
@@ -334,6 +334,15 @@ func failCarried(ctx context.Context, open *stores, p *inventory.Plan, g *invent
notes = append(notes, p.Note)
}
done := map[string]bool{except: true}
// **What passed on its own keeps its pass** (novox/hq ADR 0254, issue 318): healthy for the passes the
// gate asks while another module of the send failed, it is neither put back nor left unjudged — a build
// left without a verdict is one more move every other walk would wait for.
if kept := keepPassing(ctx, open, p, g, except); len(kept) > 0 {
notes = append(notes, "passed on their own and kept: "+strings.Join(kept, ", "))
for _, m := range kept {
done[m] = true
}
}
for _, c := range g.Carried {
if done[c.Module] || (len(g.Failing) > 0 && !slices.Contains(g.Failing, c.Module)) {
continue
@@ -356,6 +365,34 @@ func failCarried(ctx context.Context, open *stores, p *inventory.Plan, g *invent
p.Note = strings.Join(notes, "; ")
}
// keepPassing keeps a pass as the verdict of every build the gate carried that passed on its own when the
// send failed (ADR 0254), except the plan's own module; answers the modules kept.
func keepPassing(ctx context.Context, open *stores, p *inventory.Plan, g *inventory.PlanGate, except string) []string {
var kept []string
seen := map[string]bool{}
for _, c := range g.Carried {
if c.Module == except || c.Build == "" || seen[c.Build] || !slices.Contains(g.Passing, c.Module) {
continue
}
seen[c.Build] = true
why := fmt.Sprintf("healthy %d times on its own while the send failed: %s", g.Healthy[c.Module], g.Why)
if w := g.Waits[c.Module]; w != "" {
why += "; and it waits for a person: " + w
}
if err := open.inventory.RecordGate(ctx, inventory.GateVerdict{Build: c.Build, Module: c.Module, Commit: c.To,
Previous: c.From, Plan: p.ID, Machines: []string{c.Node}, Verdict: inventory.GatePassed, Why: why,
Component: coreComponent(c.Module), JudgingFrom: g.Since}); err != nil {
fmt.Printf("%s: %s passed its gate on %s on its own, and the verdict could not be kept: %v\n", p.ID, c.Module,
c.Node, err)
continue
}
if !slices.Contains(kept, c.Module) {
kept = append(kept, c.Module)
}
}
return kept
}
// sentTheBuild is every machine running a module that was last sent this build of it.
func sentTheBuild(ctx context.Context, open *stores, module, commit string) ([]string, error) {
running, err := open.inventory.Running(ctx, module)
+244
View File
@@ -0,0 +1,244 @@
package main
import (
"context"
"encoding/json"
"fmt"
"reflect"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/lease"
"github.com/novox/mesh-controller/internal/link"
)
// novox/hq issue 318, replayed with only what the controller had before its fix, so it can be laid over the
// older commit: the statements are read from the node-engine's JSON, as the controller hears them.
//
// 2026-10-08, 11:08 UTC: the walk of the builds waiting for a gate sent the laptop three moves in one send —
// docker ba6058ab → c78b5fc9, node-tools 9a4140b4 → d4845f7c, openrazer 88135ad0 → c78b5fc9 — and judged
// them together. openrazer's build put the operator's account in the group `openrazer` (ADR 0252); the
// laptop said the account "relogin needed", and the daemon's unit failed in the account's own service
// manager, which began before the group. The gate read "not yet healthy" for ten minutes, failed at its
// bound, put openrazer back (which by ADR 0252 also took the group back), and the walk failed: docker and
// node-tools, healthy at every judging, never reached the workstation, and every other walk was refused there.
//
// The rule's outcome: a wait for a person is not a failure. The send passes with the wait carried along,
// nothing is put back, the walk goes on to the workstation, and openrazer's condition says, in one sentence
// for the operator, that a new login is needed on the laptop.
func TestReplay318(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
keeper, _ := withConditionsInMemory(t)
was := doctorFrom
doctorFrom = nil
t.Cleanup(func() { doctorFrom = was })
build := func(module, repository, commit string, asked time.Time) {
manifest, _ := json.Marshal(catalogue.Manifest{Module: module, Version: "1"})
if err := inv.RecordBuild(ctx, inventory.Build{ID: "build-" + module + "-" + commit, Module: module, Commit: commit,
Repository: repository, Path: "modules/" + module, Manifest: manifest, Asked: asked, At: asked,
Made: []inventory.Artifact{{Name: "x", Kind: "bundle", Reference: "sha256:" + module + "-" + commit}}}); err != nil {
t.Fatal(err)
}
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: module, Version: "1"}, inventory.Source{
Repository: repository, Seat: "git", Path: "modules/" + module, BuiltFrom: commit, Head: commit,
Asked: asked}); err != nil {
t.Fatal(err)
}
}
type move struct{ module, repository, from, to string }
moves := []move{
{"docker", "novox/mesh-catalog", "ba6058ab", "c78b5fc9"},
{"node-tools", "novox/mesh-tools", "9a4140b4", "d4845f7c"},
{"openrazer", "novox/mesh-catalog", "88135ad0", "c78b5fc9"},
}
machines := []string{"laptop", "workstation"}
if _, err := inv.AddNode(ctx, "workstation"); err != nil {
t.Fatal(err)
}
old, recent := time.Now().Add(-3*time.Hour), time.Now().Add(-time.Minute)
for _, mv := range moves {
build(mv.module, mv.repository, mv.from, old)
for _, n := range machines {
if _, err := inv.Assign(ctx, n, mv.module); err != nil {
t.Fatal(err)
}
}
}
for _, n := range machines {
sent := map[string]string{}
for _, mv := range moves {
sent[mv.module] = mv.from
}
if err := inv.RecordSent(ctx, nodeID(t, open, n), "d-"+n, sent); err != nil {
t.Fatal(err)
}
}
for _, mv := range moves {
build(mv.module, mv.repository, mv.to, recent)
}
// What each machine says, as its node-engine words it: openrazer's account waits for a new login, and the
// daemon's unit failed in that account's own manager; docker's container is healthy.
statement := func(at time.Time) link.Health {
raw := fmt.Sprintf(`{"contract": %d, "at": %q, "resources": [
{"module": "docker", "resource": "docker.engine", "kind": "service", "target": "docker.service",
"state": "healthy", "since": %q},
{"module": "openrazer", "resource": "openrazer.operator", "kind": "account", "target": "operator",
"account": "operator", "state": "unhealthy", "since": %q, "streak": 3,
"reason": "relogin needed: operator is in the group openrazer, and its running session began before it was; log out of every session and in again, or reboot"},
{"module": "openrazer", "resource": "openrazer.daemon", "kind": "unit", "target": "openrazer-daemon.service",
"account": "operator", "state": "unhealthy", "since": %q, "streak": 3,
"reason": "failed in the account's own service manager (exit-code)"}]}`,
link.ReadinessContract, at.Format(time.RFC3339Nano), at.Format(time.RFC3339Nano), at.Format(time.RFC3339Nano),
at.Format(time.RFC3339Nano))
var h link.Health
if err := json.Unmarshal([]byte(raw), &h); err != nil {
t.Fatal(err)
}
return h
}
wasMoves, wasHeard, wasSend, wasGather := machineMoves, releaseHeard, sendRollout, gatherGateFacts
wasSettle, wasEvery, wasBound := gateSettle, gateEvery, gateBound
t.Cleanup(func() {
machineMoves, releaseHeard, sendRollout, gatherGateFacts = wasMoves, wasHeard, wasSend, wasGather
gateSettle, gateEvery, gateBound = wasSettle, wasEvery, wasBound
})
machineMoves = func(ctx context.Context, open *stores, f moveFacts, node string, all bool) ([]inventory.CarriedMove, error) {
modules, err := open.inventory.Assigned(ctx, node)
if err != nil {
return nil, err
}
sent, known, err := open.inventory.SentBuilds(ctx, node)
if err != nil {
return nil, err
}
return f.moves(node, modules, sent, known, all), nil
}
releaseHeard = func(context.Context, *stores) (map[string]bool, error) {
return map[string]bool{"laptop": true, "workstation": true}, nil
}
var sends [][]string
n := 0
sendRollout = func(ctx context.Context, open *stores, names []string) ([]string, error) {
sends = append(sends, append([]string(nil), names...))
current, err := open.inventory.CurrentBuilds(ctx)
if err != nil {
return nil, err
}
for _, node := range names {
modules, _ := open.inventory.Assigned(ctx, node)
carried := map[string]string{}
for _, m := range modules {
carried[m] = current[m].Commit
}
n++
digest := fmt.Sprintf("d-%s-%d", node, n)
if err := open.inventory.RecordSent(ctx, nodeID(t, open, node), digest, carried); err != nil {
return nil, err
}
if _, err := open.inventory.RecordDoing(ctx, nodeID(t, open, node), inventory.Doing{Node: node,
Outcome: inventory.OutcomeApplied, Declared: digest, Applied: 1, At: time.Now()}); err != nil {
return nil, err
}
}
return names, nil
}
// Every judging hears each machine's newest statement first, as the controller does between judgings:
// the account's wait and the failed unit are raised as openrazer's condition, after the send.
gatherGateFacts = func(ctx context.Context, open *stores, component string) (gateFacts, error) {
now := time.Now()
f := gateFacts{now: now, reports: map[string]inventory.Reported{}, engines: map[string]string{},
rolledBack: map[string][]lease.Rollback{}, served: map[string]served{}, health: map[string]inventory.NodeHealth{},
judged: true}
for _, m := range machines {
if err := stateHealth(ctx, open.inventory, keeper, m, statement(now), now); err != nil {
return f, err
}
f.served[m] = served{runtime: true, tools: map[string]bool{}}
}
reports, err := open.inventory.LastReports(ctx)
if err != nil {
return f, err
}
for _, r := range reports {
f.reports[r.Node] = r
}
if f.health, err = open.inventory.Healths(ctx); err != nil {
return f, err
}
f.open, f.openErr = keeper.Open(ctx)
return f, nil
}
// The bound is reached at the first judging that is not a pass: what happened at 11:21 happens at once.
gateSettle, gateEvery, gateBound = 0, 0, 0
for i := 0; i < 12; i++ {
advancePlans(ctx, open)
time.Sleep(5 * time.Millisecond)
}
var p inventory.Plan
plans, err := inv.RecentPlans(ctx, 10)
if err != nil {
t.Fatal(err)
}
for _, q := range plans {
if q.Release != nil {
p = q
break
}
}
if p.Release == nil {
t.Fatal("no walk of the builds waiting for a gate was opened")
}
if p.State != inventory.PlanDone || !reflect.DeepEqual(p.Release.Done, machines) {
t.Fatalf("the walk is %s on %v: %s; want it done on the laptop and then the workstation", p.State,
p.Release.Done, p.Note)
}
if !reflect.DeepEqual(sends, [][]string{{"laptop"}, {"workstation"}}) {
t.Fatalf("sent %v; want the laptop, then the workstation, and nothing put back", sends)
}
current, err := inv.CurrentBuilds(ctx)
if err != nil {
t.Fatal(err)
}
if current["openrazer"].Commit != "c78b5fc9" {
t.Fatalf("openrazer is registered at %s: its build was put back for a wait for a person", current["openrazer"].Commit)
}
for _, mv := range moves {
v, found, err := inv.GateOf(ctx, "build-"+mv.module+"-"+mv.to)
if err != nil || !found || v.Verdict != inventory.GatePassed {
t.Fatalf("%s's build %s: verdict %+v (found %v, %v); want a pass", mv.module, mv.to, v, found, err)
}
if mv.module == "openrazer" && !strings.Contains(v.Why, "relogin needed on ") {
t.Errorf("openrazer's pass does not carry its wait for a person: %q", v.Why)
}
}
// What the operator reads: one condition per machine for openrazer, saying a new login is needed there,
// a warning for a person, and no fault of the build.
list, err := keeper.Open(ctx)
if err != nil {
t.Fatal(err)
}
var said []string
for _, c := range list {
said = append(said, c.Key+": "+c.Summary)
if c.Subject.ID == "openrazer.laptop" {
if !strings.HasPrefix(c.Summary, "relogin needed on laptop") || c.Severity == conditions.Urgent ||
c.Resolver != conditions.ResolverOperator {
t.Errorf("openrazer's condition on the laptop: %s %s %s %q", c.Key, c.Severity, c.Resolver, c.Summary)
}
}
}
if !strings.Contains(strings.Join(said, "\n"), "relogin needed on laptop") {
t.Errorf("nothing says a new login is needed on the laptop:\n%s", strings.Join(said, "\n"))
}
}
+27 -1
View File
@@ -605,6 +605,30 @@ func (a *verbArguments) commandLine() ([]string, error) {
return nil, err
}
return []string{"images", str("node"), "--json"}, nil
case "mirrors":
// novox/hq ADR 0257.
argv := []string{"mirrors", "--json"}
record := str("record")
why := str("why")
if record == "" {
if on("confirm") || why != "" {
return nil, errors.New("mirrors takes confirm and why only with record: there is nothing " +
"else it records. Nothing was done")
}
return argv, nil
}
argv = append(argv, "--record", record)
if !on("confirm") {
if why != "" {
return nil, errors.New("mirrors takes why only with confirm: without confirm it is a dry run, " +
"and a reason for nothing would be recorded nowhere. Nothing was done")
}
return argv, nil
}
if err := need("why"); err != nil {
return nil, fmt.Errorf("%w: recording a copy as the mesh's is a hand act, which says why. Nothing was done", err)
}
return append(argv, "--confirm", "--why", why), nil
case "data":
argv := []string{"data", "--json"}
if m := str("machine"); m != "" {
@@ -804,7 +828,7 @@ func (a *verbArguments) commandLine() ([]string, error) {
var jsonVerbs = map[string]bool{"status": true, "seats": true, "plan": true, "collection": true,
"hand-acts": true, "durations": true, "conditions": true, "doctor": true, "retire": true, "cleanup": true, "data": true,
// What the records say the registries may keep (novox/hq ADR 0251).
"artifacts": true, "collect": true, "images": true,
"artifacts": true, "collect": true, "images": true, "mirrors": true,
// The delivery's owner's verbs answer JSON where they read (plan, order, check, walks) — novox/hq ADR 0239.
"delivery": true}
@@ -835,6 +859,8 @@ func repairingCommand(argv []string) string {
return "cleanup delete"
case argv[0] == "collect" && slices.Contains(argv, "--confirm"):
return "collect"
case argv[0] == "mirrors" && slices.Contains(argv, "--confirm"):
return "mirrors"
}
return ""
}
@@ -284,6 +284,7 @@ var accountedFlags = map[string]map[string]string{
"artifacts": {"json": "set by the verb: the answer is data"},
"collect": {"json": "set by the verb: the answer is data"},
"images": {"json": "set by the verb: the answer is data"},
"mirrors": {"json": "set by the verb: the answer is data"},
"conditions history": {"json": "set by the verb: the answer is data"},
"conditions show": {"json": "set by the verb: the answer is data"},
"healers": {"json": "set by the verb: the answer is data"},
+3
View File
@@ -58,6 +58,9 @@ type sendable struct {
// make (Foreseeing, novox/hq issue 275). Never on the wire, and a declaration that has any is never
// sent.
foreseen []string
// unplaced is every contribution its holder's template could not render, naming its module and
// why (novox/hq ADR 0255): left out of this declaration, for push and plan to say, never on the wire.
unplaced []string
// Builds is the build of each module this declaration carries — module to the commit its build
// was made from — recorded with the send and never on the wire (novox/hq issue 259, ADR 0221).
// Composed only on the send path; nil records that it is not known.
-1
View File
@@ -361,7 +361,6 @@ func watchWaits(f *signalFacts) []conditions.Observation {
Headline: deliveryName(w.modules, w.repository) + " waiting to start",
Explanation: walkWaitingWords(w, in, severity),
Needs: waitingNeeds(severity),
Actions: waitingActions(w, severity),
Resolved: deliveryName(w.modules, w.repository) + " no longer waiting"})
}
return out
+176
View File
@@ -0,0 +1,176 @@
package artifacts
import (
"context"
"errors"
"fmt"
"net/http"
"net/http/httptest"
"slices"
"strings"
"sync"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// An index is let go of with the platform manifests it names, except those a kept index of the same
// repository names too (novox/hq ADR 0257). Each platform manifest is a manifest of the repository in
// its own right, and the store's collector keeps every manifest a repository holds — so an index let go
// of alone frees nothing of the images it names.
func digestN(n int) string { return fmt.Sprintf("sha256:%064x", n) }
// indexStore holds indexes and images in one repository, answers GETs with their documents, and records
// every delete.
type indexStore struct {
mu sync.Mutex
indexes map[string][]string // digest → the platform manifests it names
images map[string]bool
deleted []string
}
func (s *indexStore) serve(t *testing.T) Store {
t.Helper()
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
s.mu.Lock()
defer s.mu.Unlock()
digest := r.URL.Path[strings.LastIndex(r.URL.Path, "/")+1:]
switch r.Method {
case http.MethodGet:
if children, ok := s.indexes[digest]; ok {
var named []string
for _, c := range children {
named = append(named, `{"mediaType":"application/vnd.oci.image.manifest.v1+json","digest":"`+c+`","size":1}`)
}
w.Header().Set("Content-Type", "application/vnd.oci.image.index.v1+json")
_, _ = w.Write([]byte(`{"schemaVersion":2,"mediaType":"application/vnd.oci.image.index.v1+json","manifests":[` +
strings.Join(named, ",") + `]}`))
return
}
if s.images[digest] {
w.Header().Set("Content-Type", "application/vnd.oci.image.manifest.v1+json")
_, _ = w.Write([]byte(`{"schemaVersion":2,"mediaType":"application/vnd.oci.image.manifest.v1+json","layers":[]}`))
return
}
w.WriteHeader(http.StatusNotFound)
case http.MethodHead:
if _, ok := s.indexes[digest]; ok || s.images[digest] {
w.WriteHeader(http.StatusOK)
return
}
w.WriteHeader(http.StatusNotFound)
case http.MethodDelete:
if _, ok := s.indexes[digest]; !ok && !s.images[digest] {
w.WriteHeader(http.StatusNotFound)
return
}
s.deleted = append(s.deleted, digest)
delete(s.indexes, digest)
delete(s.images, digest)
w.WriteHeader(http.StatusAccepted)
default:
w.WriteHeader(http.StatusBadRequest)
}
}))
t.Cleanup(server.Close)
return Store{Address: strings.TrimPrefix(server.URL, "http://")}
}
func TestAnIndexGoesWithItsPlatformsAndAKeptIndexKeepsItsOwn(t *testing.T) {
// Two indexes of one image's repository: the old one names platforms 11 and 12, the kept one 12 and 13.
s := &indexStore{
indexes: map[string][]string{digestN(1): {digestN(11), digestN(12)}, digestN(2): {digestN(12), digestN(13)}},
images: map[string]bool{digestN(11): true, digestN(12): true, digestN(13): true},
}
store := s.serve(t)
repository := "upstream/docker.io/library/golang"
old := catalogue.ArtifactStoreScheme + repository + "@" + digestN(1)
kept := catalogue.ArtifactStoreScheme + repository + "@" + digestN(2)
spare, err := store.SpareKeptIndexes(context.Background(), []string{kept}, []string{old})
if err != nil {
t.Fatal(err)
}
store.Spare = spare
if err := store.LetGo(context.Background(), old); err != nil {
t.Fatal(err)
}
// Its own platform first, the index last; the platform the kept index names is left in place.
if !slices.Equal(s.deleted, []string{digestN(11), digestN(1)}) {
t.Fatalf("deleted %v; want the old index's own platform, then the index", s.deleted)
}
if !s.images[digestN(12)] || !s.images[digestN(13)] {
t.Fatal("a platform a kept index names was let go of")
}
}
func TestWithoutSparingAnIndexGoesAloneAndAnImageNamesNoPlatforms(t *testing.T) {
s := &indexStore{indexes: map[string][]string{digestN(1): {digestN(11)}}, images: map[string]bool{digestN(11): true, digestN(5): true}}
store := s.serve(t)
// No Spare: as before ADR 0257, the index alone.
if err := store.LetGo(context.Background(), catalogue.ArtifactStoreScheme+"web/base@"+digestN(1)); err != nil {
t.Fatal(err)
}
if !slices.Equal(s.deleted, []string{digestN(1)}) {
t.Fatalf("without a Spare, deleted %v", s.deleted)
}
// An image is one delete, with a Spare or without.
s.deleted = nil
spare, err := store.SpareKeptIndexes(context.Background(), nil, []string{catalogue.ArtifactStoreScheme + "web/app@" + digestN(5)})
if err != nil {
t.Fatal(err)
}
store.Spare = spare
if err := store.LetGo(context.Background(), catalogue.ArtifactStoreScheme+"web/app@"+digestN(5)); err != nil {
t.Fatal(err)
}
if !slices.Equal(s.deleted, []string{digestN(5)}) {
t.Fatalf("an image was let go of as %v", s.deleted)
}
// An index the store no longer holds is Gone, and nothing is deleted on its account.
s.deleted = nil
err = store.LetGo(context.Background(), catalogue.ArtifactStoreScheme+"web/base@"+digestN(9))
if !errors.Is(err, Gone) || len(s.deleted) != 0 {
t.Fatalf("an index the store does not hold: %v, deleted %v", err, s.deleted)
}
}
func TestASpareThatCannotBeReadKeepsTheIndex(t *testing.T) {
s := &indexStore{indexes: map[string][]string{digestN(1): {digestN(11)}}, images: map[string]bool{digestN(11): true}}
store := s.serve(t)
store.Spare = func(context.Context, string) (map[string]bool, error) { return nil, fmt.Errorf("the store went away") }
err := store.LetGo(context.Background(), catalogue.ArtifactStoreScheme+"web/base@"+digestN(1))
if err == nil || !strings.Contains(err.Error(), "was kept") {
t.Fatalf("a spare that could not be read: %v", err)
}
if len(s.deleted) != 0 {
t.Fatalf("deleted %v without knowing what a kept index names", s.deleted)
}
}
// The spare list is read whole before the sweep: a kept index that cannot be read is an error before
// anything is deleted, and a repository not read before keeps its index.
func TestASpareListIsReadWholeBeforeAnythingGoes(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
}))
t.Cleanup(server.Close)
store := Store{Address: strings.TrimPrefix(server.URL, "http://")}
kept := catalogue.ArtifactStoreScheme + "web/base@" + digestN(2)
old := catalogue.ArtifactStoreScheme + "web/base@" + digestN(1)
if _, err := store.SpareKeptIndexes(context.Background(), []string{kept}, []string{old}); err == nil {
t.Fatal("a kept index the store would not answer for was taken as naming nothing")
}
s := &indexStore{indexes: map[string][]string{digestN(1): {digestN(11)}}, images: map[string]bool{digestN(11): true}}
good := s.serve(t)
spare, err := good.SpareKeptIndexes(context.Background(), nil, []string{catalogue.ArtifactStoreScheme + "other/repo@" + digestN(3)})
if err != nil {
t.Fatal(err)
}
good.Spare = spare
if err := good.LetGo(context.Background(), old); err == nil || len(s.deleted) != 0 {
t.Fatalf("an index in a repository not read before the sweep was let go of: %v, deleted %v", err, s.deleted)
}
}
+155
View File
@@ -9,8 +9,10 @@ package artifacts
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"strings"
@@ -24,6 +26,10 @@ type Store struct {
Address string
// HTTP is the client used; nil is a client with a modest timeout.
HTTP *http.Client
// Spare says, for one repository, which manifests a kept index there names: the platform
// manifests an index being let go of must leave in place (novox/hq ADR 0257). Nil spares every
// platform manifest — an index is then let go of alone, as it was before.
Spare func(ctx context.Context, repository string) (map[string]bool, error)
}
// Gone is the answer when the store does not hold it: the outcome wanted, already true.
@@ -77,9 +83,142 @@ func (s Store) LetGo(ctx context.Context, reference string) error {
return err
}
}
if kind == "manifests" && s.Spare != nil {
// **An index's platform manifests go before the index** (novox/hq ADR 0257). Each was put
// under its own digest, so each is a manifest of the repository in its own right, and the
// store's collector keeps every manifest a repository holds: an index let go of alone frees
// no byte of the images it names. Before, not after, so a sweep cut short leaves the index
// to be asked about again, and with it the list of what is still to go.
if err := s.letGoOfPlatforms(ctx, repository, digest, reference); err != nil {
return err
}
}
return s.remove(ctx, s.url(repository, kind, digest), reference)
}
// letGoOfPlatforms deletes the manifests an index names, except those a kept index of the same
// repository also names. A manifest that is not an index names none.
func (s Store) letGoOfPlatforms(ctx context.Context, repository, digest, reference string) error {
children, err := s.Platforms(ctx, repository, digest)
if err != nil || len(children) == 0 {
// Gone: there is no index to read, and the delete that follows says so.
if errors.Is(err, Gone) {
return nil
}
return err
}
spared, err := s.Spare(ctx, repository)
if err != nil {
return fmt.Errorf("cannot say which platform manifests of %s a kept index names, so %s was kept: %w",
repository, reference, err)
}
for _, child := range children {
if spared[child] {
continue
}
if err := s.remove(ctx, s.url(repository, "manifests", child), reference+" (its "+child+")"); err != nil && !errors.Is(err, Gone) {
return err
}
}
return nil
}
// anyManifest is every manifest the store may hold, indexes included, so it answers with the
// document as it is.
var anyManifest = []string{
"application/vnd.oci.image.index.v1+json",
"application/vnd.docker.distribution.manifest.list.v2+json",
"application/vnd.oci.image.manifest.v1+json",
"application/vnd.docker.distribution.manifest.v2+json",
}
// Platforms is the manifests an index names, by digest; none for a manifest that is not an index.
// Gone when the store does not hold it.
func (s Store) Platforms(ctx context.Context, repository, digest string) ([]string, error) {
request, err := http.NewRequestWithContext(ctx, http.MethodGet, s.url(repository, "manifests", digest), nil)
if err != nil {
return nil, err
}
request.Header.Set("Accept", strings.Join(anyManifest, ", "))
response, err := s.client().Do(request)
if err != nil {
return nil, err
}
defer response.Body.Close()
switch response.StatusCode {
case http.StatusOK:
case http.StatusNotFound:
return nil, Gone
default:
return nil, fmt.Errorf("the artifact store answered %s when asked for %s@%s", response.Status, repository, digest)
}
var document struct {
Manifests []struct {
Digest string `json:"digest"`
} `json:"manifests"`
}
if err := json.NewDecoder(io.LimitReader(response.Body, 4<<20)).Decode(&document); err != nil {
return nil, fmt.Errorf("%s@%s is not a manifest: %w", repository, digest, err)
}
var out []string
for _, m := range document.Manifests {
if strings.HasPrefix(m.Digest, "sha256:") {
out = append(out, m.Digest)
}
}
return out, nil
}
// SpareKeptIndexes is a Spare for a sweep, read whole before it starts: for every repository an
// image reference of `touching` is in, each kept manifest there and every platform manifest a kept
// index there names. Any read that fails is an error, and the sweep lets nothing go. A repository not
// read here is answered with an error, so an index in it is kept. A kept index the store no longer
// holds names nothing.
func (s Store) SpareKeptIndexes(ctx context.Context, kept, touching []string) (func(context.Context, string) (map[string]bool, error), error) {
imageIn := func(reference string) (repository, digest string, ok bool) {
path, ours := catalogue.InArtifactStore(reference)
if !ours {
return "", "", false
}
repository, kind, digest, err := split(path)
if err != nil || kind != "manifests" {
return "", "", false
}
return repository, digest, true
}
read := map[string]map[string]bool{}
for _, reference := range touching {
if repository, _, ok := imageIn(reference); ok {
read[repository] = map[string]bool{}
}
}
for _, reference := range kept {
repository, digest, ok := imageIn(reference)
spared, touched := read[repository]
if !ok || !touched {
continue
}
spared[digest] = true
children, err := s.Platforms(ctx, repository, digest)
if errors.Is(err, Gone) {
continue
}
if err != nil {
return nil, fmt.Errorf("reading the kept %s@%s: %w", repository, digest, err)
}
for _, c := range children {
spared[c] = true
}
}
return func(_ context.Context, repository string) (map[string]bool, error) {
spared, done := read[repository]
if !done {
return nil, fmt.Errorf("%s was not read before the sweep began", repository)
}
return spared, nil
}, nil
}
// remove asks the store to delete what is at url. Gone when it has no such thing.
func (s Store) remove(ctx context.Context, url, what string) error {
request, err := http.NewRequestWithContext(ctx, http.MethodDelete, url, nil)
@@ -121,3 +260,19 @@ func split(path string) (repository, kind, digest string, err error) {
}
return "", "", "", fmt.Errorf("%w: %q names nothing the store holds by digest", ErrNotOurs, path)
}
// HoldsManifest is whether the store holds the manifest a recorded image reference names.
func (s Store) HoldsManifest(ctx context.Context, reference string) (bool, error) {
path, ours := catalogue.InArtifactStore(reference)
if !ours {
return false, fmt.Errorf("%w: %s", ErrNotOurs, reference)
}
repository, kind, digest, err := split(path)
if err != nil {
return false, err
}
if kind != "manifests" {
return false, fmt.Errorf("%w: %s names a blob, not a manifest", ErrNotOurs, reference)
}
return s.has(ctx, s.url(repository, kind, digest), anyManifest...)
}
+42 -5
View File
@@ -77,6 +77,11 @@ type Result struct {
// the default, and that branch is its trunk.
Branches []string
// Mirrored is every base this build copied into the artifact store, as pinned (novox/hq ADR 0257):
// said whether the build worked or not, because the copy is in the store either way, and the
// records are what decide whether it stays.
Mirrored []string
// Source is the build's source fingerprint (source.go): what it was made from — the module's tree,
// the contexts' trees, the bases and toolchains by digest — hashed. Empty where the source does not
// pin the build. Two builds with one fingerprint are one build, whatever digests they made
@@ -106,6 +111,19 @@ type GitCredential struct {
func Build(ctx context.Context, run Runner, publish Publisher,
repository, path, ref, workspace string, held map[string]string, npmrc Npmrc,
forge GitCredential, log Log, seats ...map[string]string) (Result, error) {
// **What was copied is said whether the build worked or not** (novox/hq ADR 0257): a base is in
// the store from the moment it is copied, and a build that failed after copying it is the
// one record that it is there.
var mirrored []string
result, err := build(ctx, run, publish, repository, path, ref, workspace, held, npmrc, forge, log,
&mirrored, seats...)
result.Mirrored = mirrored
return result, err
}
func build(ctx context.Context, run Runner, publish Publisher,
repository, path, ref, workspace string, held map[string]string, npmrc Npmrc,
forge GitCredential, log Log, mirrored *[]string, seats ...map[string]string) (Result, error) {
// The clone base of each seat a context may name (novox/hq ADR 0155); variadic so the callers
// that hand none — tests of everything but contexts — read as they did.
var seatBases map[string]string
@@ -222,10 +240,22 @@ func Build(ctx context.Context, run Runner, publish Publisher,
// An image published elsewhere that the build stands on is copied into the mesh's own
// registry first, like an upstream artifact (ADR 0096), and the recipe is handed the copy.
// Genesis has nowhere to copy to and pulls it into this machine's store instead.
mirror := func(ctx context.Context, from, repository string) (string, error) {
mirror := func(ctx context.Context, from, repository, former string) (string, error) {
if m, can := publish.(BaseMirrorer); can {
say("bases", "copying %s into the mesh's registry as %s", from, repository)
reference, err := m.MirrorBase(ctx, from, repository, former)
if err == nil {
*mirrored = append(*mirrored, reference)
}
return reference, err
}
if m, can := publish.(Mirrorer); can {
say("bases", "copying %s into the mesh's registry", from)
return m.MirrorImage(ctx, from, repository)
say("bases", "copying %s into the mesh's registry as %s", from, repository)
reference, err := m.MirrorImage(ctx, from, repository)
if err == nil {
*mirrored = append(*mirrored, reference)
}
return reference, err
}
if _, err := run(ctx, tree, "docker", "pull", from); err != nil {
return "", fmt.Errorf("cannot fetch %s: %w", from, err)
@@ -901,7 +931,7 @@ var _ io.Writer = (*stringWriter)(nil)
// The order is fixed so two builds of one commit invoke the same command. Returned alongside the
// arguments is every reference they resolved to, which is what the build stood on.
func standingOn(ctx context.Context, manifest catalogue.Manifest, held map[string]string,
mirror func(ctx context.Context, from, repository string) (string, error)) ([]string, []string, error) {
mirror func(ctx context.Context, from, repository, former string) (string, error)) ([]string, []string, error) {
if manifest.Build == nil || len(manifest.Build.On) == 0 {
return nil, nil, nil
}
@@ -924,7 +954,14 @@ func standingOn(ctx context.Context, manifest catalogue.Manifest, held map[strin
"%s stands on the image %q, which is not pinned by digest. A tag is what "+
"somebody else can move; name it as <image>@sha256:…", manifest.Module, base.Image)
}
reference, err := mirror(ctx, base.Image, manifest.Module+"/on-"+strings.ToLower(base.Arg))
// **One copy per upstream image, whichever modules stand on it** (novox/hq ADR 0257). Its
// repository is named for the image, not the module; the module's own repository of
// before is offered as a source, so the move to one copy asks upstream for nothing.
repository, err := MirrorRepository(base.Image)
if err != nil {
return nil, nil, fmt.Errorf("%s stands on %s: %w", manifest.Module, base.Image, err)
}
reference, err := mirror(ctx, base.Image, repository, FormerMirrorRepository(manifest.Module, base.Arg))
if err != nil {
return nil, nil, fmt.Errorf("%s stands on %s: %w", manifest.Module, base.Image, err)
}
+117 -10
View File
@@ -27,6 +27,40 @@ type Mirrorer interface {
MirrorImage(ctx context.Context, from, repository string) (string, error)
}
// BaseMirrorer copies a base a build stands on into the one repository the mesh keeps for that
// upstream image, taking what an earlier copy under `former` already holds rather than asking
// upstream for it again (novox/hq ADR 0257).
type BaseMirrorer interface {
MirrorBase(ctx context.Context, from, repository, former string) (string, error)
}
// MirrorPrefix is the namespace of the repositories a base is mirrored into: `upstream/<host>/<path>`,
// one repository per upstream image, whichever modules stand on it (novox/hq ADR 0257).
const MirrorPrefix = "upstream/"
// MirrorRepository is the repository the mesh keeps an upstream image's copy in: the image's own
// host and path under MirrorPrefix, so two modules standing on one image stand on one copy, and two
// images that share a path on different hosts are never confused. Lower case and without a port's
// colon, because a repository name allows neither.
func MirrorRepository(from string) (string, error) {
where, err := parseReference(from)
if err != nil {
return "", err
}
host := strings.TrimPrefix(strings.TrimPrefix(where.base, "https://"), "http://")
if host == "registry-1.docker.io" {
host = "docker.io"
}
host = strings.ReplaceAll(host, ":", "-")
return strings.ToLower(MirrorPrefix + host + "/" + where.repository), nil
}
// FormerMirrorRepository is where a module's base was copied before ADR 0257: under the module's own
// repository, one copy per module (ADR 0097). Read only, as a source of what is already held.
func FormerMirrorRepository(module, arg string) string {
return module + "/on-" + strings.ToLower(arg)
}
const (
mediaIndexOCI = "application/vnd.oci.image.index.v1+json"
mediaIndexDocker = "application/vnd.docker.distribution.manifest.list.v2+json"
@@ -86,6 +120,9 @@ func parseReference(ref string) (upstream, error) {
type source struct {
client *http.Client
token string
// mountFrom is a repository of the mesh's own registry the source is, so a blob is mounted from
// it rather than read and written again.
mountFrom string
}
// get fetches a registry URL, answering a bearer challenge once with an anonymous token — which is
@@ -176,6 +213,15 @@ type descriptor struct {
// under `repository`, and returns the reference the mesh will pin: this registry, the repository,
// and the digest of the document that was put last, which is the index where there is one.
func (r Registry) MirrorImage(ctx context.Context, from, repository string) (string, error) {
return r.MirrorBase(ctx, from, repository, "")
}
// MirrorBase is MirrorImage, and where `former` — a repository of this registry — already holds the
// image by its digest, the copy is made from there: manifests read from this registry, blobs mounted
// across rather than moved (novox/hq ADR 0257). Upstream is asked only for what no repository here
// holds, which is what kept the public hub's anonymous pull limit out of reach when every module's
// base moved into one shared repository.
func (r Registry) MirrorBase(ctx context.Context, from, repository, former string) (string, error) {
where, err := parseReference(from)
if err != nil {
return "", err
@@ -186,7 +232,11 @@ func (r Registry) MirrorImage(ctx context.Context, from, repository string) (str
// on the first merge that rebuilt a whole catalogue (2026-09-28), and every module whose base
// lives there failed on a copy it did not need.
if strings.HasPrefix(where.reference, "sha256:") {
held, err := r.has(ctx, "http://"+r.Address+"/v2/"+repository+"/manifests/"+where.reference, manifestAccept)
// **Held whole, not only its index** (novox/hq ADR 0257): every platform manifest the index
// names is asked about too. A sweep that stopped half-way, or that ran while this build was
// copying, may have left an index naming a platform the store no longer holds; such a copy
// is made again, and the copy puts back what is missing.
held, err := r.holdsWhole(ctx, repository, where.reference)
if err != nil {
return "", fmt.Errorf("asking %s whether it holds %s: %w", r.Address, from, err)
}
@@ -195,6 +245,17 @@ func (r Registry) MirrorImage(ctx context.Context, from, repository string) (str
}
}
src := &source{client: r.client()}
if former != "" && former != repository && strings.HasPrefix(where.reference, "sha256:") {
held, err := r.holdsWhole(ctx, former, where.reference)
if err != nil {
return "", fmt.Errorf("asking %s whether %s holds %s: %w", r.Address, former, from, err)
}
if held {
// The same bytes, already here: copied from this registry, each blob mounted.
where = upstream{base: "http://" + r.Address, repository: former, reference: where.reference}
src.mountFrom = former
}
}
digest, err := r.copyManifest(ctx, src, where, where.reference, repository)
if err != nil {
return "", fmt.Errorf("copying %s into %s/%s: %w", from, r.Address, repository, err)
@@ -202,6 +263,42 @@ func (r Registry) MirrorImage(ctx context.Context, from, repository string) (str
return r.Address + "/" + repository + "@" + digest, nil
}
// holdsWhole is whether this registry holds the manifest under the repository and, for an index, every
// manifest it names.
func (r Registry) holdsWhole(ctx context.Context, repository, digest string) (bool, error) {
url := "http://" + r.Address + "/v2/" + repository + "/manifests/"
held, err := r.has(ctx, url+digest, manifestAccept)
if err != nil || !held {
return false, err
}
request, err := http.NewRequestWithContext(ctx, http.MethodGet, url+digest, nil)
if err != nil {
return false, err
}
request.Header.Set("Accept", manifestAccept)
response, err := r.client().Do(request)
if err != nil {
return false, fmt.Errorf("cannot reach the registry at %s: %w", r.Address, err)
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
return false, nil
}
var document struct {
Manifests []descriptor `json:"manifests"`
}
if err := json.NewDecoder(io.LimitReader(response.Body, 4<<20)).Decode(&document); err != nil {
return false, fmt.Errorf("%s@%s is not a manifest: %w", repository, digest, err)
}
for _, m := range document.Manifests {
held, err := r.has(ctx, url+m.Digest, manifestAccept)
if err != nil || !held {
return false, err
}
}
return true, nil
}
// copyManifest copies one manifest document and everything it names, and returns its digest. An
// index is copied by copying each manifest it names first, so the index never points at something
// the registry does not hold yet.
@@ -286,16 +383,14 @@ func (r Registry) copyBlob(ctx context.Context, src *source, where upstream, dig
} else if there {
return nil
}
response, err := src.get(ctx, where.base+"/v2/"+where.repository+"/blobs/"+digest, "")
if err != nil {
return err
// **Mounted where this registry already holds it** (novox/hq ADR 0257): a blob is stored once
// whichever repositories link it, so a mount moves no bytes. A registry that cannot mount answers
// with an ordinary upload's location, and the blob is moved as before.
uploads := base + "/blobs/uploads/"
if src.mountFrom != "" {
uploads += "?mount=" + digest + "&from=" + src.mountFrom
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
return fmt.Errorf("%s/%s: blob %s: %s", where.base, where.repository, digest, response.Status)
}
start, err := http.NewRequestWithContext(ctx, http.MethodPost, base+"/blobs/uploads/", nil)
start, err := http.NewRequestWithContext(ctx, http.MethodPost, uploads, nil)
if err != nil {
return err
}
@@ -304,9 +399,21 @@ func (r Registry) copyBlob(ctx context.Context, src *source, where upstream, dig
return fmt.Errorf("cannot start an upload to %s: %w", base, err)
}
begun.Body.Close()
if begun.StatusCode == http.StatusCreated && src.mountFrom != "" {
return nil
}
if begun.StatusCode != http.StatusAccepted {
return fmt.Errorf("%s answered %s when asked where to put a blob", base, begun.Status)
}
response, err := src.get(ctx, where.base+"/v2/"+where.repository+"/blobs/"+digest, "")
if err != nil {
return err
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
return fmt.Errorf("%s/%s: blob %s: %s", where.base, where.repository, digest, response.Status)
}
location := begun.Header.Get("Location")
if location == "" {
return fmt.Errorf("%s accepted an upload and said nowhere to put it", base)
+375
View File
@@ -0,0 +1,375 @@
package builder
import (
"context"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
)
// One copy per upstream image, whichever modules stand on it (novox/hq ADR 0257).
func TestAnUpstreamImageHasOneRepositoryNamedForIt(t *testing.T) {
for from, want := range map[string]string{
"golang@sha256:abc": "upstream/docker.io/library/golang",
"n8nio/n8n@sha256:abc": "upstream/docker.io/n8nio/n8n",
"quay.io/minio/mc@sha256:abc": "upstream/quay.io/minio/mc",
"ghcr.io/Mailu/Admin@sha256:abc": "upstream/ghcr.io/mailu/admin",
"localhost:5000/x/y@sha256:abc": "upstream/localhost-5000/x/y",
"docker.io/library/alpine:3.20@sha256:ab": "upstream/docker.io/library/alpine",
} {
got, err := MirrorRepository(from)
if err != nil {
t.Fatalf("%s: %v", from, err)
}
if got != want {
t.Errorf("%s: got %q want %q", from, got, want)
}
}
if got := FormerMirrorRepository("route-proxy", "GO_BASE"); got != "route-proxy/on-go_base" {
t.Fatalf("the former repository is %q", got)
}
}
// aRegistryOfRepositories is a registry the way the real one is: blobs stored once, and each
// repository linking the blobs and manifests it holds. It mounts a blob across repositories.
type aRegistryOfRepositories struct {
mu sync.Mutex
blobs map[string][]byte // digest → bytes, stored once
links map[string]map[string]bool // repository → blob digests it links
manifests map[string][]byte // repository@digest → document
uploads int
mounts int
gets int
// noMount answers every mount with an ordinary upload's location, as a registry that cannot
// mount across repositories does.
noMount bool
}
func newRegistryOfRepositories() *aRegistryOfRepositories {
return &aRegistryOfRepositories{blobs: map[string][]byte{}, links: map[string]map[string]bool{},
manifests: map[string][]byte{}}
}
func (m *aRegistryOfRepositories) link(repository, digest string) {
if m.links[repository] == nil {
m.links[repository] = map[string]bool{}
}
m.links[repository][digest] = true
}
// split reads /v2/<repository>/<kind>/<rest> with a repository of any depth.
func splitPath(path string) (repository, kind, rest string) {
path = strings.TrimPrefix(path, "/v2/")
for _, k := range []string{"/manifests/", "/blobs/uploads/", "/blobs/"} {
if i := strings.Index(path, k); i >= 0 {
return path[:i], strings.Trim(k, "/"), path[i+len(k):]
}
}
return "", "", ""
}
func (m *aRegistryOfRepositories) handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
m.mu.Lock()
defer m.mu.Unlock()
repository, kind, rest := splitPath(r.URL.Path)
switch {
case kind == "manifests" && (r.Method == http.MethodHead || r.Method == http.MethodGet):
body, ok := m.manifests[repository+"@"+rest]
if !ok {
w.WriteHeader(http.StatusNotFound)
return
}
if r.Method == http.MethodGet {
m.gets++
w.Header().Set("Content-Type", mediaTypeOf(body))
_, _ = w.Write(body)
return
}
w.WriteHeader(http.StatusOK)
case kind == "manifests" && r.Method == http.MethodPut:
body, _ := readAll(r)
m.manifests[repository+"@"+digestOf(body)] = body
w.WriteHeader(http.StatusCreated)
case kind == "blobs" && (r.Method == http.MethodHead || r.Method == http.MethodGet):
if !m.links[repository][rest] {
w.WriteHeader(http.StatusNotFound)
return
}
if r.Method == http.MethodGet {
m.gets++
_, _ = w.Write(m.blobs[rest])
return
}
w.WriteHeader(http.StatusOK)
case kind == "manifests" && r.Method == http.MethodDelete:
delete(m.manifests, repository+"@"+rest)
w.WriteHeader(http.StatusAccepted)
case kind == "blobs/uploads" && r.Method == http.MethodPost:
if digest, from := r.URL.Query().Get("mount"), r.URL.Query().Get("from"); digest != "" && m.links[from][digest] && !m.noMount {
m.link(repository, digest)
m.mounts++
w.WriteHeader(http.StatusCreated)
return
}
w.Header().Set("Location", "/v2/"+repository+"/blobs/uploads/one")
w.WriteHeader(http.StatusAccepted)
case kind == "blobs/uploads" && r.Method == http.MethodPut:
body, _ := readAll(r)
digest := r.URL.Query().Get("digest")
if digestOf(body) != digest {
w.WriteHeader(http.StatusBadRequest)
return
}
m.blobs[digest] = body
m.link(repository, digest)
m.uploads++
w.WriteHeader(http.StatusCreated)
default:
w.WriteHeader(http.StatusNotFound)
}
})
}
func mediaTypeOf(body []byte) string {
switch {
case strings.Contains(string(body), mediaIndexOCI):
return mediaIndexOCI
default:
return mediaManifestOCI
}
}
// A base a module's own repository already holds is copied into the image's repository from there:
// every blob mounted, nothing asked of upstream — which may be gone, or rationing anonymous pulls.
func TestABaseHeldUnderTheModulesFormerRepositoryIsMountedNotFetchedAgain(t *testing.T) {
src, indexDigest, srcBlobs := anUpstreamRegistry(t)
dst := newRegistryOfRepositories()
dstServer := httptest.NewServer(dst.handler())
defer dstServer.Close()
address := strings.TrimPrefix(dstServer.URL, "http://")
r := Registry{Address: address, HTTP: src.Client()}
host := strings.TrimPrefix(src.URL, "http://")
// Before: copied the old way, under the module's repository.
if _, err := r.MirrorImage(context.Background(), host+"/library/thing:latest", "hello-web/on-thing_base"); err != nil {
t.Fatal(err)
}
uploaded := dst.uploads
if uploaded != len(srcBlobs) {
t.Fatalf("the first copy uploaded %d of %d blobs", uploaded, len(srcBlobs))
}
src.Close()
from := host + "/library/thing@" + indexDigest
repository, err := MirrorRepository(from)
if err != nil {
t.Fatal(err)
}
reference, err := r.MirrorBase(context.Background(), from, repository, "hello-web/on-thing_base")
if err != nil {
t.Fatalf("a base the registry holds was asked of an upstream that is gone: %v", err)
}
if reference != address+"/"+repository+"@"+indexDigest {
t.Fatalf("pinned as %q", reference)
}
if dst.uploads != uploaded {
t.Fatalf("the move to one repository uploaded %d blobs again", dst.uploads-uploaded)
}
if dst.mounts != len(srcBlobs) {
t.Fatalf("%d of %d blobs mounted", dst.mounts, len(srcBlobs))
}
// The index and both platform manifests are in the image's repository, and every blob is linked.
for key := range dst.manifests {
if strings.HasPrefix(key, "hello-web/") {
continue
}
if !strings.HasPrefix(key, repository+"@") {
t.Fatalf("a manifest went to %s", key)
}
}
if n := countPrefix(dst.manifests, repository+"@"); n != 3 {
t.Fatalf("%d manifests in %s, want the index and two platforms", n, repository)
}
for digest := range srcBlobs {
if !dst.links[repository][digest] {
t.Fatalf("blob %s is not linked into %s", digest, repository)
}
}
// A second module standing on the same image: already held, nothing copied, the same reference.
again, err := r.MirrorBase(context.Background(), from, repository, "other-module/on-thing_base")
if err != nil || again != reference {
t.Fatalf("a second module's copy: %q %v", again, err)
}
}
func countPrefix(m map[string][]byte, prefix string) int {
n := 0
for k := range m {
if strings.HasPrefix(k, prefix) {
n++
}
}
return n
}
// What a build copied is said whether it worked or not: the copy is in the store either way, and a
// build that failed after copying is the one record that it is there.
func TestABuildSaysWhatItMirroredEvenWhenItFails(t *testing.T) {
// A recipe that reaches for an image nobody declared is refused — after the base was copied.
r, workspace := aRepository(t, anImage, map[string]string{
"modules/bus/Dockerfile": "ARG BASE\nFROM ${BASE}\nCOPY --from=vendor/other:1 /x /x"})
r.contents["modules/bus/module.json"] = anImage
r.tree = "aaaa"
m := &mirroring{recorded: r, at: "registry-a:5000"}
got, err := Build(context.Background(), r.run, m, "https://forge.invalid/catalogue.git", "modules/bus", "", workspace,
nil, Npmrc{}, GitCredential{}, nil)
if err == nil {
t.Fatal("a recipe fetching an undeclared image was built")
}
want := "registry-a:5000/upstream/docker.io/vendor/server@sha256:" + strings.Repeat("1", 64)
if len(got.Mirrored) != 1 || got.Mirrored[0] != want {
t.Fatalf("a failed build said it mirrored %v, want [%s]", got.Mirrored, want)
}
// And a build that works says it too.
r2, workspace2 := aRepository(t, anImage, map[string]string{"modules/bus/Dockerfile": "ARG BASE\nFROM ${BASE}"})
r2.contents["modules/bus/module.json"] = anImage
r2.tree = "aaaa"
ok, err := Build(context.Background(), r2.run, &mirroring{recorded: r2, at: "registry-a:5000"},
"https://forge.invalid/catalogue.git", "modules/bus", "", workspace2, nil, Npmrc{}, GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
if len(ok.Mirrored) != 1 || ok.Mirrored[0] != want {
t.Fatalf("a build said it mirrored %v, want [%s]", ok.Mirrored, want)
}
}
// held puts one image in a repository the old way and answers its index digest and the registry.
func heldUnderAModule(t *testing.T) (*aRegistryOfRepositories, Registry, string, string, map[string][]byte, func()) {
t.Helper()
src, indexDigest, srcBlobs := anUpstreamRegistry(t)
t.Cleanup(src.Close)
dst := newRegistryOfRepositories()
dstServer := httptest.NewServer(dst.handler())
t.Cleanup(dstServer.Close)
r := Registry{Address: strings.TrimPrefix(dstServer.URL, "http://"), HTTP: src.Client()}
host := strings.TrimPrefix(src.URL, "http://")
if _, err := r.MirrorImage(context.Background(), host+"/library/thing:latest", "hello-web/on-thing_base"); err != nil {
t.Fatal(err)
}
return dst, r, host + "/library/thing@" + indexDigest, indexDigest, srcBlobs, src.Close
}
// An index whose platform manifests are not all held is not "already held": it is copied again, and the
// copy puts back what is missing — what a sweep that stopped half-way, or ran while a build was copying,
// leaves behind.
func TestAnIndexMissingAPlatformIsCopiedAgain(t *testing.T) {
dst, r, from, indexDigest, _, _ := heldUnderAModule(t)
repository, _ := MirrorRepository(from)
if _, err := r.MirrorBase(context.Background(), from, repository, ""); err != nil {
t.Fatal(err)
}
// A sweep lets one platform go, between this build's copy and the next.
var platform string
for key := range dst.manifests {
if strings.HasPrefix(key, repository+"@") && !strings.HasSuffix(key, indexDigest) {
platform = key
break
}
}
delete(dst.manifests, platform)
if _, err := r.MirrorBase(context.Background(), from, repository, ""); err != nil {
t.Fatal(err)
}
if _, back := dst.manifests[platform]; !back {
t.Fatalf("an index missing %s was taken as held", platform)
}
}
// The same, for the module's former copy: a former copy missing a platform is not a source; upstream
// is asked instead.
func TestAFormerCopyMissingAPlatformIsNotTheSource(t *testing.T) {
dst, r, from, indexDigest, _, _ := heldUnderAModule(t)
for key := range dst.manifests {
if strings.HasPrefix(key, "hello-web/on-thing_base@") && !strings.HasSuffix(key, indexDigest) {
delete(dst.manifests, key)
break
}
}
repository, _ := MirrorRepository(from)
if _, err := r.MirrorBase(context.Background(), from, repository, "hello-web/on-thing_base"); err != nil {
t.Fatal(err)
}
if dst.mounts != 0 {
t.Fatalf("a former copy missing a platform was mounted from (%d mounts)", dst.mounts)
}
if n := countPrefix(dst.manifests, repository+"@"); n != 3 {
t.Fatalf("%d manifests copied from upstream, want 3", n)
}
}
// A registry that answers a mount with an upload's location (202) gets the blob moved as before.
func TestAMountRefusedFallsBackToAnUpload(t *testing.T) {
dst, r, from, _, srcBlobs, _ := heldUnderAModule(t)
dst.noMount = true
uploaded := dst.uploads
repository, _ := MirrorRepository(from)
reference, err := r.MirrorBase(context.Background(), from, repository, "hello-web/on-thing_base")
if err != nil {
t.Fatal(err)
}
if !strings.HasSuffix(reference, repository+"@"+strings.SplitN(from, "@", 2)[1]) {
t.Fatalf("pinned as %s", reference)
}
if dst.mounts != 0 || dst.uploads-uploaded != len(srcBlobs) {
t.Fatalf("%d mounts, %d uploads; want every blob uploaded from the former copy", dst.mounts, dst.uploads-uploaded)
}
for digest := range srcBlobs {
if !dst.links[repository][digest] {
t.Fatalf("blob %s is not linked into %s", digest, repository)
}
}
}
// A sweep and a build at once: while builds copy the base, platforms are let go of under them. Each
// build that returns has the copy whole.
func TestABuildCopyingWhileASweepLetsGoEndsWhole(t *testing.T) {
dst, r, from, indexDigest, _, _ := heldUnderAModule(t)
repository, _ := MirrorRepository(from)
if _, err := r.MirrorBase(context.Background(), from, repository, ""); err != nil {
t.Fatal(err)
}
done := make(chan struct{})
go func() {
defer close(done)
for i := 0; i < 50; i++ {
dst.mu.Lock()
for key := range dst.manifests {
if strings.HasPrefix(key, repository+"@") && !strings.HasSuffix(key, indexDigest) {
delete(dst.manifests, key)
break
}
}
dst.mu.Unlock()
}
}()
for i := 0; i < 20; i++ {
if _, err := r.MirrorBase(context.Background(), from, repository, ""); err != nil {
t.Fatal(err)
}
}
<-done
// The sweep is over; the next build finds what it left and makes the copy whole.
if _, err := r.MirrorBase(context.Background(), from, repository, ""); err != nil {
t.Fatal(err)
}
if n := countPrefix(dst.manifests, repository+"@"); n != 3 {
t.Fatalf("after the sweep and the builds, %d manifests in %s, want 3", n, repository)
}
}
+7
View File
@@ -117,6 +117,13 @@ func (m *theMeshsRegistry) handler() http.Handler {
} else {
w.WriteHeader(http.StatusNotFound)
}
case r.Method == http.MethodGet && strings.Contains(r.URL.Path, "/manifests/"):
body, ok := m.manifests[r.URL.Path[strings.LastIndex(r.URL.Path, "/")+1:]]
if !ok {
w.WriteHeader(http.StatusNotFound)
return
}
_, _ = w.Write(body)
case r.Method == http.MethodHead && strings.Contains(r.URL.Path, "/blobs/"):
if _, ok := m.blobs[r.URL.Path[strings.LastIndex(r.URL.Path, "/")+1:]]; ok {
w.WriteHeader(http.StatusOK)
+9 -6
View File
@@ -72,7 +72,7 @@ func TestAnIncompleteBaseIsRefused(t *testing.T) {
}
// noMirror is a mirror for tests whose bases are all the mesh's own.
func noMirror(context.Context, string, string) (string, error) {
func noMirror(context.Context, string, string, string) (string, error) {
return "", fmt.Errorf("nothing to copy in this test")
}
@@ -86,17 +86,20 @@ func TestADeclaredVendorImageIsCopiedInAndHandedToTheRecipe(t *testing.T) {
},
}
var asked []string
args, _, err := standingOn(context.Background(), manifest, nil, func(_ context.Context, from, repository string) (string, error) {
asked = append(asked, from+" -> "+repository)
args, _, err := standingOn(context.Background(), manifest, nil, func(_ context.Context, from, repository, former string) (string, error) {
asked = append(asked, from+" -> "+repository+" (from "+former+")")
return "127.0.0.1:5000/" + repository + "@sha256:" + strings.Repeat("d", 64), nil
})
if err != nil {
t.Fatal(err)
}
if len(asked) != 1 || asked[0] != "quay.io/minio/mc@sha256:"+strings.Repeat("c", 64)+" -> minio/on-mc_base" {
t.Fatalf("the image was not copied under the module's repository: %v", asked)
// One repository per upstream image, named for the image (novox/hq ADR 0257); the module's
// repository of before offered as where it may already be held.
if len(asked) != 1 || asked[0] != "quay.io/minio/mc@sha256:"+strings.Repeat("c", 64)+
" -> upstream/quay.io/minio/mc (from minio/on-mc_base)" {
t.Fatalf("the image was not copied into the image's own repository: %v", asked)
}
if strings.Join(args, " ") != "--build-arg MC_BASE=127.0.0.1:5000/minio/on-mc_base@sha256:"+strings.Repeat("d", 64) {
if strings.Join(args, " ") != "--build-arg MC_BASE=127.0.0.1:5000/upstream/quay.io/minio/mc@sha256:"+strings.Repeat("d", 64) {
t.Fatalf("the recipe was not handed the copy: %v", args)
}
// Unpinned, it is refused: a tag is what somebody else can move.
+1 -1
View File
@@ -32,7 +32,7 @@ func TestHandWrittenBackupLinesOfAnOlderModuleArePlaced(t *testing.T) {
if err != nil {
t.Fatal(err)
}
placed, err := seatContributions([]Manifest{pg, mail}, BackupSeat, "backup", Rendering{}, nil)
placed, _, err := seatContributions([]Manifest{pg, mail}, Manifest{}, BackupSeat+":"+"backup", Rendering{}, nil, nil)
if err != nil {
t.Fatal(err)
}
+3 -3
View File
@@ -174,7 +174,7 @@ func TestTheBackupHoldersLinesAreDerivedFromTheData(t *testing.T) {
"resources":[{"id":"meta","type":"directory","mode":"0700"}]}`)
facts := map[string]string{"account-home": "/home/op"}
with := Rendering{Settings: SettingsBy{"media": {{From: "node", Values: map[string]any{AccessesSetting: map[string]any{"films": "/tank/films"}}}}}}
backup, err := seatContributions([]Manifest{pg, agent, media}, BackupSeat, BackupKindBackup, with, facts)
backup, _, err := seatContributions([]Manifest{pg, agent, media}, Manifest{}, BackupSeat+":"+BackupKindBackup, with, facts, nil)
if err != nil {
t.Fatal(err)
}
@@ -182,7 +182,7 @@ func TestTheBackupHoldersLinesAreDerivedFromTheData(t *testing.T) {
if backup != want {
t.Fatalf("backup lines:\n%s\nwant:\n%s", backup, want)
}
data, err := seatContributions([]Manifest{pg, agent, media}, BackupSeat, BackupKindData, with, facts)
data, _, err := seatContributions([]Manifest{pg, agent, media}, Manifest{}, BackupSeat+":"+BackupKindData, with, facts, nil)
if err != nil {
t.Fatal(err)
}
@@ -192,7 +192,7 @@ func TestTheBackupHoldersLinesAreDerivedFromTheData(t *testing.T) {
if data != want {
t.Fatalf("data lines:\n%s\nwant:\n%s", data, want)
}
if _, err := seatContributions([]Manifest{agent}, BackupSeat, BackupKindData, Rendering{}, nil); err == nil {
if _, _, err := seatContributions([]Manifest{agent}, Manifest{}, BackupSeat+":"+BackupKindData, Rendering{}, nil, nil); err == nil {
t.Fatal("a home directory with no account on the machine reached the holder as a placeholder")
}
// What keeps something irreplaceable depends on the machine's backup holder — it backs it up or
+10 -5
View File
@@ -291,6 +291,10 @@ type Composed struct {
// as `module/name`, sorted: what the next send will make. Never set on a declaration that is
// sent — a placeholder in a sealed file is a credential the process cannot read.
Foreseen []string
// Unplaced is every contribution the holder's template could not render into something, each
// naming its module and why (novox/hq ADR 0255): that piece is left out and the machine is told
// everything else. Push and plan say it; the merge gate refuses a change that adds one.
Unplaced []string
}
// ForeseenSealed is what stands for an own secret a send will make, in a composition asked ahead of
@@ -326,8 +330,8 @@ func (r Resolution) Compose(with Rendering) (Composed, error) {
owner := map[string]string{}
received := map[string]map[string][]Contribution{}
leftOut := map[string]string{}
var foreseen []string
resources, err := r.compose(with, owner, received, leftOut, &foreseen)
var foreseen, unplaced []string
resources, err := r.compose(with, owner, received, leftOut, &foreseen, &unplaced)
if err != nil {
return Composed{}, err
}
@@ -340,8 +344,9 @@ func (r Resolution) Compose(with Rendering) (Composed, error) {
"sealed": with.BusMembership, "mode": "0600",
})
}
sort.Strings(unplaced)
return Composed{Resources: resources, Owner: owner, Received: received, LeftOut: leftOut,
Foreseen: foreseen}, nil
Foreseen: foreseen, Unplaced: unplaced}, nil
}
// BusMembershipID names the resource carrying a machine's membership for the new bus, and
@@ -352,7 +357,7 @@ const BusMembershipPath = "/var/lib/mesh/membership-next.json"
func (r Resolution) compose(with Rendering, owner map[string]string,
received map[string]map[string][]Contribution, leftOut map[string]string,
foreseen *[]string) ([]map[string]any, error) {
foreseen, unplaced *[]string) ([]map[string]any, error) {
// **A setting is judged where it is stored, and an impossible one costs a module, not a
// machine** (novox/hq ADR 0163, rule 6). A definition that moved under a stored setting makes
// this module uncomposable; it is left out of the declaration — its held things kept, its
@@ -970,7 +975,7 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
// seat that places them (novox/hq ADR 0203, ADR 0204). Gathered from every module on
// the node, as the jails are, and **last of every placeholder pass**: shell code is a
// shell's own syntax, full of `${…}` no pass above should ever be shown.
if err := contributionsInto(copied, m, r.Modules, thisMachine, with); err != nil {
if err := contributionsInto(copied, m, r.Modules, thisMachine, with, r.Capabilities, unplaced); err != nil {
return nil, err
}
copied["id"] = m.Module + "." + fmt.Sprint(resource["id"])
+5 -2
View File
@@ -547,7 +547,7 @@ func shellCode(modules []Manifest, shell, slot string) string {
// `${machine:…}` some module wrote for its shell to see. So nothing runs after them, the environment
// is filled before the shell's code is, and each is replaced in a single pass over what the holder
// wrote, so a contributed piece is never scanned again.
func contributionsInto(resource map[string]any, m Manifest, modules []Manifest, facts map[string]string, with Rendering) error {
func contributionsInto(resource map[string]any, m Manifest, modules []Manifest, facts map[string]string, with Rendering, caps map[string]bool, unplaced *[]string) error {
if problems := append(placeholderProblems(m, resource), seatPlaceholderProblems(m, resource)...); len(problems) > 0 {
return fmt.Errorf("%s", problems[0])
}
@@ -576,10 +576,13 @@ func contributionsInto(resource map[string]any, m Manifest, modules []Manifest,
found := ofContributed.FindStringSubmatch(placeholder)
first, second, _ := strings.Cut(found[2], ":")
if found[1] == "contribution" {
placed, err := seatContributions(modules, first, second, with, facts)
placed, left, err := seatContributions(modules, m, found[2], with, facts, caps)
if err != nil && failed == nil {
failed = err
}
if unplaced != nil {
*unplaced = append(*unplaced, left...)
}
return placed
}
return shellCode(modules, first, second)
+14 -13
View File
@@ -177,19 +177,8 @@ func TestTheForgeHoldsTheNpmAndGitSeats(t *testing.T) {
// wrong on every node whose assignment differs, and wrong for a second reason on a node given the
// port (ADR 0100). Composed through the whole path, because what proves the placeholder resolves
// in an `env` at all is a declaration, not a manifest.
// forgeBuilt is every artifact the forge's manifest builds: its code, and since hq ADR 0251 the npm
// registry it serves beside it. A manifest resolves only against all of what it asked to be built.
func forgeBuilt() []Built {
var out []Built
for _, name := range []string{"code", "npm-registry"} {
out = append(out, Built{Name: name, Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "gitea/" + name + "/blobs/" + bundleDigest, Digest: bundleDigest})
}
return out
}
func TestTheForgesOwnAddressFollowsThePortTheNodeGaveIt(t *testing.T) {
forge, err := catalogueManifest(t, "gitea").Resolve(forgeBuilt())
forge, err := catalogueManifest(t, "gitea").Resolve(forgeBuilt(catalogueManifest(t, "gitea")))
if err != nil {
t.Fatalf("the forge's manifest does not resolve against its own build: %v", err)
}
@@ -242,7 +231,7 @@ func TestTheForgesOwnAddressFollowsThePortTheNodeGaveIt(t *testing.T) {
func declaredGiteaSsh(t *testing.T, given map[int]int) map[string]any {
t.Helper()
forge := catalogueManifest(t, "gitea")
resolved, err := forge.Resolve(forgeBuilt())
resolved, err := forge.Resolve(forgeBuilt(forge))
if err != nil {
t.Fatalf("the forge's manifest does not resolve against its own build: %v", err)
}
@@ -314,3 +303,15 @@ func TestANodeMayGiveTheForgesSshPortADifferentNumber(t *testing.T) {
t.Fatalf("the forge is published on %v, not the port this node gave it", server["ports"])
}
}
// forgeBuilt is a build of every artifact the forge's manifest declares: its code bundle and, since the
// catalogue added it (novox/hq ADR 0251 §4), its npm registry's. Named from the manifest, so a third
// artifact does not break every forge test again.
func forgeBuilt(forge Manifest) []Built {
var out []Built
for _, a := range forge.Build.Artifacts {
out = append(out, Built{Name: a.Name, Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "gitea/" + a.Name + "/blobs/" + bundleDigest, Digest: bundleDigest})
}
return out
}
+100 -2
View File
@@ -1,5 +1,10 @@
package catalogue
import (
"fmt"
"strings"
)
// The graphical session's seats (novox/hq ADR 0208): one module per piece of software, each piece's
// role a node seat in the mesh's own set, so i3 and sway, xterm and foot, rofi and dmenu compete for
// a role rather than each inventing one — and a machine running two of one role is refused at
@@ -19,7 +24,8 @@ const (
)
// graphicalSessionSeats are the eleven, in the order ADR 0208's table reads, each with the verbs
// research 026/05 starts it with. Three have none yet: the bar, the compositor and the secret
// research 026/05 starts it with. Three have none yet: the bar (which receives blocks instead,
// ADR 0255), the compositor and the secret
// service are roles a second holder competes for, and nothing has needed to ask them anything.
func graphicalSessionSeats() []Seat {
const decided = "novox/hq ADR 0208"
@@ -93,7 +99,11 @@ func graphicalSessionSeats() []Seat {
{Name: "copy", Description: "Put text on the operator's clipboard.",
Input: schema(map[string]string{"text": "the text"}, []string{"text"})},
}},
{Name: BarSeat, Scope: ScopeNode, Decision: decided},
// It receives blocks as data (novox/hq ADR 0255): what a block shows, said for any bar, which
// the holder renders in its own grammar. A module that knows something about the machine — the
// power module its battery — gives the block; the bar knows no machine's hardware.
{Name: BarSeat, Scope: ScopeNode, Decision: decided,
Receives: []Receivable{{Kind: BarKindBlock, Comment: "#", Offered: true, Shape: barBlockShape()}}},
{Name: CompositorSeat, Scope: ScopeNode, Decision: decided},
{Name: SecretServiceSeat, Scope: ScopeNode, Decision: decided},
}
@@ -107,3 +117,91 @@ func withEnum(s map[string]any, property string, values ...string) map[string]an
p["enum"] = values
return s
}
// BarKindBlock is the bar's one received kind: a block (novox/hq ADR 0255).
const BarKindBlock = "block"
// barShows is what a contributed block may show, and the options each takes — the bar-neutral
// vocabulary every holder of node-bar renders. A new entry is a decision, and every holder's template
// renders it (the shape's examples) before it is accepted.
var barShows = map[string]map[string]barOption{
// The machine's battery: its charge, and how long it lasts or takes to fill.
"battery": {"device": {kind: "text"}},
// One line a command prints, run every interval seconds: what the bar cannot know itself.
"command": {"command": {kind: "text", required: true}, "interval": {kind: "seconds", required: true}},
}
type barOption struct {
kind string // text, or seconds (a whole number from 1 to 86400)
required bool
}
// barBlockShape is the shape of a block (novox/hq ADR 0255): which bar, where on it, in what order,
// what it shows, and that kind's options.
//
// - bar: top or bottom;
// - place: resources (beside the processor, memory and disks) or status (beside the sound, before the
// clock) — the two places every holder keeps for contributed blocks;
// - order: 0–99 within the place, 50 when absent;
// - shows: battery or command; options: what that kind takes.
func barBlockShape() *Shape {
return &Shape{
Places: []PlaceField{{Field: "bar", Values: []string{"bottom", "top"}},
{Field: "place", Values: []string{"resources", "status"}}},
Order: "order",
Check: barBlockProblems,
Examples: []map[string]any{
{"bar": "bottom", "place": "status", "shows": "battery"},
{"bar": "bottom", "place": "status", "shows": "battery", "options": map[string]any{"device": "BAT0"}},
{"bar": "bottom", "place": "status", "shows": "command",
"options": map[string]any{"command": "echo \"a line\"", "interval": float64(5)}},
},
}
}
func barBlockProblems(data map[string]any) []string {
var problems []string
for field := range data {
if !oneOf([]string{"bar", "place", "order", "shows", "options"}, field) {
problems = append(problems, fmt.Sprintf("a block has no field %q; it has bar, place, order, shows and options", field))
}
}
shows, _ := data["shows"].(string)
takes, known := barShows[shows]
if !known {
return append(problems, fmt.Sprintf("a block shows %v; it shows one of %s", data["shows"],
strings.Join(sortedKeys(barShows), ", ")))
}
options := map[string]any{}
if raw, has := data["options"]; has {
o, ok := raw.(map[string]any)
if !ok {
return append(problems, "a block's options are an object")
}
options = o
}
for name, v := range options {
opt, ok := takes[name]
if !ok {
problems = append(problems, fmt.Sprintf("a %s block has no option %q; it takes %s", shows, name,
orNothing(sortedKeys(takes))))
continue
}
switch opt.kind {
case "text":
if t, isText := v.(string); !isText || strings.TrimSpace(t) == "" {
problems = append(problems, fmt.Sprintf("a %s block's %s is text", shows, name))
}
case "seconds":
if _, whole := wholeIn(v, 1, 86400); !whole {
problems = append(problems, fmt.Sprintf("a %s block's %s is a whole number of seconds from 1 to 86400", shows, name))
}
}
}
for _, name := range sortedKeys(takes) {
if _, has := options[name]; takes[name].required && !has {
problems = append(problems, fmt.Sprintf("a %s block needs the option %s", shows, name))
}
}
return problems
}
+5
View File
@@ -60,6 +60,10 @@ type Claim struct {
// module's own `tools` must list every verb the seat promises, which is how a module named
// like its seat — the catalogue, the records — says they are one and the same.
Serves []string `json:"serves,omitempty"`
// Renders is how this holder writes each kind its seat receives as data, by kind: a Go
// text/template over one piece — its fields and `module` — in the tool's own grammar (novox/hq ADR
// 0255). The data is the mesh's, the format the holder's, as a module's facts template is.
Renders map[string]string `json:"renders,omitempty"`
}
// ServesFor is what this claim offers a seat's protocol: the verbs it names, else the module's
@@ -2030,6 +2034,7 @@ func ParseManifest(raw []byte) (Manifest, error) {
problems = append(problems, m.shellProblems()...)
problems = append(problems, m.contributionPlaceholderProblems()...)
problems = append(problems, m.seatContributionProblems()...)
problems = append(problems, m.placementProblems()...)
problems = append(problems, m.dataProblems()...)
problems = append(problems, m.upgradeProblems()...)
+5 -1
View File
@@ -134,6 +134,10 @@ type Resolution struct {
// here without a store lookup.
Account string
AccountHome string
// Capabilities are the machine's, as its profile reported them, carried from the node so a
// contribution placed only where the machine has something (`if-capability`, novox/hq ADR 0255)
// is decided here without a store lookup.
Capabilities map[string]bool
// Modules in the order they were resolved: assigned first, then what they pulled in.
Modules []Manifest
@@ -699,7 +703,7 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
}
resolution := Resolution{Node: node.Name, At: node.At, PublicDomain: node.PublicDomain,
Account: node.Account, AccountHome: node.AccountHome,
Account: node.Account, AccountHome: node.AccountHome, Capabilities: node.Capabilities,
Because: because, Needs: needs, Unhostable: unhostable, Kept: kept}
for _, n := range providersFirst(order, catalogue) {
resolution.Modules = append(resolution.Modules, catalogue[n])
+61 -8
View File
@@ -30,7 +30,14 @@ type SeatContribution struct {
// Kind is which of the seat's receivable kinds it is.
Kind string `json:"kind"`
// Content is the text, in the tool's own grammar. Never interpreted.
Content string `json:"content"`
Content string `json:"content,omitempty"`
// Data is the piece in the seat's own shape, for a kind the seat receives as data (novox/hq ADR
// 0255): the holder renders it with its template, and Content is then empty.
Data map[string]any `json:"data,omitempty"`
// IfCapability names a capability the machine must report for this contribution to be placed
// there (novox/hq ADR 0255): a battery's block only where the node-engine found a battery. The
// composition decides, from the machine's facts, not the tool at run time.
IfCapability string `json:"if-capability,omitempty"`
}
// ofContribution is where a holder places a kind: ${contribution:<seat>:<kind>}. Loose inside the
@@ -92,7 +99,30 @@ func (m Manifest) seatContributionProblems() []string {
"%s's contribution %d to %s is of the kind %q; %s (novox/hq ADR 0212)",
m.Module, i+1, s.Name, c.Kind, kindsOf(s)))
}
if strings.TrimSpace(c.Content) == "" {
// Only on a kind the seat offers (novox/hq ADR 0255): a piece shown where the hardware is. On a
// kind a holder depends on — what a backup keeps, a key's trigger — a machine missing the
// capability would lose the piece without a word.
if c.IfCapability != "" {
switch {
case !capabilityName.MatchString(c.IfCapability):
problems = append(problems, fmt.Sprintf("%s's contribution %d is if-capability %q, which is not a "+
"capability's name", m.Module, i+1, c.IfCapability))
case !oneOf(KnownCapabilities, c.IfCapability):
problems = append(problems, fmt.Sprintf("%s's contribution %d is if-capability %q, which no machine "+
"reports; the node-engine detects %s", m.Module, i+1, c.IfCapability, strings.Join(KnownCapabilities, ", ")))
case ok && !r.Offered:
problems = append(problems, fmt.Sprintf("%s's contribution %d to %s (%s) is if-capability %s; only a "+
"kind the seat offers may be left out where a machine lacks something (novox/hq ADR 0255)",
m.Module, i+1, s.Name, c.Kind, c.IfCapability))
}
}
switch {
case ok && r.Shape != nil:
problems = append(problems, shapedProblems(m.Module, i, r, c)...)
case len(c.Data) > 0:
problems = append(problems, fmt.Sprintf("%s's contribution %d has data; %s receives %s as text, "+
"given as `content`", m.Module, i+1, s.Name, c.Kind))
case strings.TrimSpace(c.Content) == "":
problems = append(problems, fmt.Sprintf("%s's contribution %d has no content", m.Module, i+1))
}
}
@@ -120,8 +150,15 @@ func seatPlaceholderProblems(m Manifest, r map[string]any) []string {
continue
}
for _, f := range found {
seat, kind, two := strings.Cut(f[1], ":")
s, _, ok := receivable(seat, kind)
seat, rest, two := strings.Cut(f[1], ":")
// A kind received as data may be narrowed by its shape's placing fields (novox/hq ADR 0255).
kind, where, _ := strings.Cut(rest, ":")
s, rcv, ok := receivable(seat, kind)
if two && ok {
if _, err := placeholderWhere(rcv, where); err != nil {
problems = append(problems, fmt.Sprintf("%s's resource %v names %s: %v", m.Module, r["id"], f[0], err))
}
}
if !two || !ok {
detail := "the mesh defines no seat " + fmt.Sprintf("%q", seat)
if s.Name != "" {
@@ -152,17 +189,33 @@ func seatPlaceholderProblems(m Manifest, r map[string]any) []string {
// A directory placed in a person's home is named `${machine:account-home}/…` (novox/hq to-be 29); in a
// kind that takes directories that is filled from the machine's facts as well, so the holder reads a
// path, never a placeholder (novox/hq ADR 0233: an item in the operator's home is backed up too).
func seatContributions(modules []Manifest, seat, kind string, with Rendering, facts map[string]string) (string, error) {
//
// A kind received as data is rendered through the holder's template instead, narrowed by where
// (novox/hq ADR 0255); and a contribution naming a capability this machine did not report is left
// out, of either form.
//
// What it could not render is answered as unplaced, each naming its module and why, and left out: one
// piece the holder's template does not know is that piece's fault, not the machine's.
func seatContributions(modules []Manifest, holder Manifest, placeholder string, with Rendering, facts map[string]string, caps map[string]bool) (string, []string, error) {
seat, rest, _ := strings.Cut(placeholder, ":")
kind, whereText, _ := strings.Cut(rest, ":")
s, r, ok := receivable(seat, kind)
if !ok {
return "", nil
return "", nil, nil
}
if r.Shape != nil {
where, err := placeholderWhere(r, whereText)
if err != nil {
return "", nil, err
}
return shapedContributions(modules, holder, s, r, where, caps)
}
var failed error
var b strings.Builder
for _, m := range inModuleOrder(modules) {
named := false
for _, c := range m.allContributions() {
if c.Kind != kind {
if c.Kind != kind || !capable(c, caps) {
continue
}
if cs, known := SeatNamed(c.Seat); !known || cs.Name != s.Name {
@@ -207,5 +260,5 @@ func seatContributions(modules []Manifest, seat, kind string, with Rendering, fa
}
}
}
return b.String(), failed
return b.String(), nil, failed
}
+402
View File
@@ -0,0 +1,402 @@
package catalogue
import (
"bytes"
"encoding/json"
"fmt"
"math"
"regexp"
"sort"
"strings"
"text/template"
)
// A kind a seat receives as data, rendered by its holder (novox/hq ADR 0255).
//
// ADR 0212 made a contribution text in the tool's own grammar, which the controller places and never
// reads. That is right where the tool is the seat's for good — a hotkey daemon's trigger lines — and
// wrong where the seat's whole point is that its tool can be replaced: a module adding a battery to the
// bar would write i3status-rust's TOML, and a second bar would have to read it. So a seat may define a
// kind as data instead. The seat says the data's shape; a contributor gives data in that shape; the
// holder gives a template that renders one piece of it into its tool's grammar, as a module's facts
// template renders the roster (`facts`): the data is the mesh's, the format is the holder's.
//
// What the holder places is narrowed by the shape's placing fields — the bar's `bar` and `place` — in
// the placeholder, `${contribution:<seat>:<kind>:<field>=<value>,…}`, and every combination of them is
// placed by the holder exactly once, so nothing a module contributes is dropped by a holder that forgot
// a corner of the shape.
// Shape is what a kind received as data looks like.
type Shape struct {
// Places are the fields a holder narrows its placeholders by, each with every value it takes.
Places []PlaceField
// Order is the whole-number field that orders the pieces within one placeholder (lowest first,
// then module order); absent in a piece means OrderDefault.
Order string
// Check is what else is wrong with one piece's data, beyond its placing fields and its order.
Check func(data map[string]any) []string `json:"-"`
// Examples are pieces every holder's template must render, one for each variant the shape has,
// so a holder that cannot render one is refused at registration and not found on a machine.
Examples []map[string]any
}
// PlaceField is one field a holder places by, and the values it takes.
type PlaceField struct {
Field string
Values []string
}
// OrderDefault is a piece's order when it gives none: the middle of 0–99.
const OrderDefault = 50
// capabilityName is the shape of a capability's name, as a node's profile reports it.
var capabilityName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
// KnownCapabilities is every capability the node-engine detects (mesh-host internal/profile), the
// names an `if-capability` may give (novox/hq ADR 0255). A name nothing detects would leave its
// contribution out on every machine, silently, so it is refused. A detector added there is added here.
var KnownCapabilities = []string{
"battery", "container-runtime", "firewall", "graphical-session", "overlay", "package-manager",
"power-meter", "privileged", "seat", "service-manager", "uplink-dhcpcd", "uplink-networkmanager",
"uplink-systemd-networkd", "virtualisation",
}
// renderFuncs are the functions a holder's template may call. `quote` writes a value as a JSON
// string with DEL escaped as well — JSON leaves it bare, and TOML refuses it — which makes it a valid
// basic string in TOML and in most tools' grammars, so a contributed command with quotes or control
// characters in it cannot break out of the holder's line.
var renderFuncs = template.FuncMap{"quote": quote}
func quote(v any) (string, error) {
b, err := json.Marshal(fmt.Sprint(v))
return strings.ReplaceAll(string(b), "\x7f", `\u007f`), err
}
// shapedProblems is what is wrong with one contribution of a kind received as data.
func shapedProblems(module string, i int, r Receivable, c SeatContribution) []string {
var problems []string
at := fmt.Sprintf("%s's contribution %d to %s (%s)", module, i+1, c.Seat, c.Kind)
if strings.TrimSpace(c.Content) != "" {
problems = append(problems, at+" has content; this kind is data, given as `data` (novox/hq ADR 0255)")
}
if len(c.Data) == 0 {
return append(problems, at+" has no data")
}
for _, p := range r.Shape.Places {
v, _ := c.Data[p.Field].(string)
if !oneOf(p.Values, v) {
problems = append(problems, fmt.Sprintf("%s says %s %q; it is one of %s",
at, p.Field, fmt.Sprint(c.Data[p.Field]), strings.Join(p.Values, ", ")))
}
}
if r.Shape.Order != "" {
if v, has := c.Data[r.Shape.Order]; has {
if _, ok := wholeIn(v, 0, 99); !ok {
problems = append(problems, fmt.Sprintf("%s says %s %v; it is a whole number from 0 to 99",
at, r.Shape.Order, v))
}
}
}
if r.Shape.Check != nil {
for _, p := range r.Shape.Check(c.Data) {
problems = append(problems, at+": "+p)
}
}
return problems
}
// wholeIn is v as a whole number within [lo, hi]; JSON gives every number as a float.
func wholeIn(v any, lo, hi int) (int, bool) {
f, ok := v.(float64)
if !ok {
if i, isInt := v.(int); isInt {
f, ok = float64(i), true
}
}
if !ok || f != math.Trunc(f) || f < float64(lo) || f > float64(hi) {
return 0, false
}
return int(f), true
}
// placeholderWhere parses the narrowing part of a placeholder, `bar=bottom,place=status`, against the
// kind's shape. Empty narrows nothing.
func placeholderWhere(r Receivable, where string) (map[string]string, error) {
out := map[string]string{}
if where == "" {
return out, nil
}
if r.Shape == nil {
return nil, fmt.Errorf("%s is text, which a placeholder does not narrow", r.Kind)
}
for _, pair := range strings.Split(where, ",") {
field, value, ok := strings.Cut(pair, "=")
var p *PlaceField
for i := range r.Shape.Places {
if r.Shape.Places[i].Field == field {
p = &r.Shape.Places[i]
}
}
switch {
case !ok || p == nil:
return nil, fmt.Errorf("%q narrows by %s, and %s is placed by %s", where, field, r.Kind, placeFields(r))
case !oneOf(p.Values, value):
return nil, fmt.Errorf("%q says %s=%s; %s is one of %s", where, field, value, field, strings.Join(p.Values, ", "))
case out[field] != "":
return nil, fmt.Errorf("%q names %s twice", where, field)
}
out[field] = value
}
return out, nil
}
func placeFields(r Receivable) string {
var names []string
for _, p := range r.Shape.Places {
names = append(names, p.Field)
}
return strings.Join(names, ", ")
}
// holderTemplate is the holder's template for a kind, parsed.
func holderTemplate(holder Manifest, seat Seat, kind string) (*template.Template, error) {
for _, c := range holder.Claims {
if s, known := SeatNamed(c.Name); !known || s.Name != seat.Name {
continue
}
text, has := c.Renders[kind]
if !has || strings.TrimSpace(text) == "" {
break
}
t, err := template.New(seat.Name + ":" + kind).Funcs(renderFuncs).Parse(text)
if err != nil {
return nil, fmt.Errorf("%s's template for %s:%s does not parse: %v", holder.Module, seat.Name, kind, err)
}
return t, nil
}
return nil, fmt.Errorf("%s places %s:%s, which is data, and its claim gives no template for it in `renders` "+
"(novox/hq ADR 0255)", holder.Module, seat.Name, kind)
}
// renderPiece is one piece of data in the holder's grammar. The template sees the piece's fields and
// `module`, the contributor. A value the piece does not have reads as nothing in a `with` or an `if`;
// printed bare it would write `<no value>` into the tool's file, which is refused here instead.
func renderPiece(t *template.Template, module string, data map[string]any) (string, error) {
view := map[string]any{"module": module}
for k, v := range data {
view[k] = v
}
var b bytes.Buffer
if err := t.Execute(&b, view); err != nil {
return "", fmt.Errorf("rendering %s's piece: %v", module, err)
}
out := b.String()
if strings.Contains(out, "<no value>") {
return "", fmt.Errorf("rendering %s's piece printed a value it does not have: %q", module, out)
}
return out, nil
}
// placedPiece is one contributed piece on its way into a holder's file.
type placedPiece struct {
module string
data map[string]any
order int
}
// shapedContributions is every module's data of one kind to one seat, narrowed by where and by the
// machine's capabilities, ordered, and rendered through the holder's template — each piece under a
// comment line naming its module.
//
// A piece the template renders to nothing — a `shows` the holder does not know yet — or fails on is
// left out and answered as unplaced, naming its module: the other pieces and the rest of the machine's
// declaration go on (novox/hq ADR 0255).
func shapedContributions(modules []Manifest, holder Manifest, s Seat, r Receivable, where map[string]string, caps map[string]bool) (string, []string, error) {
var pieces []placedPiece
for _, m := range inModuleOrder(modules) {
for _, c := range m.allContributions() {
if c.Kind != r.Kind || !capable(c, caps) {
continue
}
if cs, known := SeatNamed(c.Seat); !known || cs.Name != s.Name {
continue
}
match := true
for field, value := range where {
if v, _ := c.Data[field].(string); v != value {
match = false
}
}
if !match {
continue
}
order := OrderDefault
if v, ok := wholeIn(c.Data[r.Shape.Order], 0, 99); ok {
order = v
}
pieces = append(pieces, placedPiece{module: m.Module, data: c.Data, order: order})
}
}
if len(pieces) == 0 {
return "", nil, nil
}
t, err := holderTemplate(holder, s, r.Kind)
if err != nil {
return "", nil, err
}
sort.SliceStable(pieces, func(a, b int) bool { return pieces[a].order < pieces[b].order })
var b strings.Builder
var unplaced []string
for _, p := range pieces {
out, err := renderPiece(t, p.module, p.data)
if err == nil && strings.TrimSpace(out) == "" {
err = fmt.Errorf("%s's template renders nothing for it", holder.Module)
}
if err != nil {
unplaced = append(unplaced, fmt.Sprintf("%s's %s:%s %s is not placed by %s: %v",
p.module, s.Name, r.Kind, describePiece(p.data), holder.Module, err))
continue
}
fmt.Fprintf(&b, "%s %s\n", r.Comment, p.module)
b.WriteString(out)
if !strings.HasSuffix(out, "\n") {
b.WriteString("\n")
}
}
return b.String(), unplaced, nil
}
// describePiece is a piece in one line, its fields in order, for a person reading why it was left out.
func describePiece(data map[string]any) string {
var parts []string
for _, k := range sortedKeys(data) {
if _, nested := data[k].(map[string]any); nested {
continue
}
parts = append(parts, fmt.Sprintf("%s=%v", k, data[k]))
}
return "(" + strings.Join(parts, " ") + ")"
}
// capable is whether a contribution applies on a machine with these capabilities: always, unless it
// names one with `if-capability` the machine did not report (novox/hq ADR 0255). A machine whose
// capabilities are not known has none, so a conditional piece is left out rather than guessed in.
func capable(c SeatContribution, caps map[string]bool) bool {
return c.IfCapability == "" || caps[c.IfCapability]
}
// placementProblems is what is wrong with how a holder places the kinds its seats receive as data:
// once it places one at all, it needs a template that renders every example of the shape, and every combination of the placing
// fields placed exactly once across its files — or something contributed would be dropped, or written
// twice (novox/hq ADR 0255).
func (m Manifest) placementProblems() []string {
var problems []string
for _, c := range m.Claims {
s, known := SeatNamed(c.Name)
if !known {
continue
}
for _, r := range s.Receives {
// A holder that places none of the kind is older than it (novox/hq ADR 0255): refusing it
// would refuse every bar registered before the kind existed, and the controller that
// defines the kind ships first. Its contributions wait, unplaced, until it places them.
if r.Shape == nil || !placesKind(m, s, r.Kind) {
continue
}
t, err := holderTemplate(m, s, r.Kind)
if err != nil {
problems = append(problems, err.Error())
continue
}
for _, ex := range r.Shape.Examples {
if out, err := renderPiece(t, "example", ex); err != nil || strings.TrimSpace(out) == "" {
problems = append(problems, fmt.Sprintf("%s's template for %s:%s renders nothing for %v: %v",
m.Module, s.Name, r.Kind, ex, err))
}
}
placed := map[string]int{}
for _, res := range m.Resources {
content, _ := res["content"].(string)
for _, f := range ofContribution.FindAllStringSubmatch(content, -1) {
seat, rest, _ := strings.Cut(f[1], ":")
kind, whereText, _ := strings.Cut(rest, ":")
if cs, ok := SeatNamed(seat); !ok || cs.Name != s.Name || kind != r.Kind {
continue
}
where, err := placeholderWhere(r, whereText)
if err != nil {
continue // said by seatPlaceholderProblems
}
for _, combo := range combinations(r.Shape.Places) {
if covers(where, combo) {
placed[comboKey(combo)]++
}
}
}
}
for _, combo := range combinations(r.Shape.Places) {
switch n := placed[comboKey(combo)]; {
case n == 0:
problems = append(problems, fmt.Sprintf("%s never places %s:%s for %s; every combination of %s is "+
"placed once, or a contribution to it is lost (novox/hq ADR 0255)",
m.Module, s.Name, r.Kind, comboKey(combo), placeFields(r)))
case n > 1:
problems = append(problems, fmt.Sprintf("%s places %s:%s for %s %d times; once, or a contribution "+
"is written twice (novox/hq ADR 0255)", m.Module, s.Name, r.Kind, comboKey(combo), n))
}
}
}
}
return problems
}
// placesKind is whether any of a holder's files names the kind's placeholder.
func placesKind(m Manifest, s Seat, kind string) bool {
for _, res := range m.Resources {
content, _ := res["content"].(string)
for _, f := range ofContribution.FindAllStringSubmatch(content, -1) {
seat, rest, _ := strings.Cut(f[1], ":")
k, _, _ := strings.Cut(rest, ":")
if cs, ok := SeatNamed(seat); ok && cs.Name == s.Name && k == kind {
return true
}
}
}
return false
}
// combinations is every assignment of a value to each placing field.
func combinations(places []PlaceField) []map[string]string {
out := []map[string]string{{}}
for _, p := range places {
var next []map[string]string
for _, partial := range out {
for _, v := range p.Values {
c := map[string]string{p.Field: v}
for k, w := range partial {
c[k] = w
}
next = append(next, c)
}
}
out = next
}
return out
}
func covers(where, combo map[string]string) bool {
for field, value := range where {
if combo[field] != value {
return false
}
}
return true
}
func comboKey(combo map[string]string) string {
var parts []string
for _, k := range sortedKeys(combo) {
parts = append(parts, k+"="+combo[k])
}
return strings.Join(parts, ",")
}
+354
View File
@@ -0,0 +1,354 @@
package catalogue
import (
"encoding/json"
"fmt"
"strconv"
"strings"
"testing"
)
// Defends novox/hq ADR 0255: a seat may receive a kind as data in its own shape, which the holder
// renders with its template; the holder places every combination of the shape's placing fields once;
// a contribution may be placed only where the machine reports a capability; and an offered kind makes
// no dependency on its seat.
const barTemplate = `{{if eq .shows "battery"}}[[block]]
block = "battery"
{{with .options}}{{with .device}}device = {{quote .}}
{{end}}{{end}}{{else if eq .shows "command"}}[[block]]
block = "custom"
command = {{quote .options.command}}
interval = {{.options.interval}}
{{end}}`
func barHolder() Manifest {
place := func(bar string) string {
return "own-first\n${contribution:node-bar:block:bar=" + bar + ",place=resources}" +
"own-middle\n${contribution:node-bar:block:bar=" + bar + ",place=status}own-last\n"
}
return Manifest{Module: "a-bar", Claims: []Claim{{Name: BarSeat, Renders: map[string]string{BarKindBlock: barTemplate}}},
Resources: []map[string]any{
{"id": "bottom", "type": "file", "path": "/b", "content": place("bottom")},
{"id": "top", "type": "file", "path": "/t", "content": place("top")},
}}
}
func block(data string) SeatContribution {
var d map[string]any
if err := json.Unmarshal([]byte(data), &d); err != nil {
panic(err)
}
return SeatContribution{Seat: BarSeat, Kind: BarKindBlock, Data: d}
}
func composedFile(t *testing.T, r Resolution, id string) string {
t.Helper()
out, err := r.Declaration(Rendering{})
if err != nil {
t.Fatal(err)
}
for _, res := range out {
if res["id"] == id {
return res["content"].(string)
}
}
t.Fatalf("%s was not composed", id)
return ""
}
func TestABlockIsRenderedByTheHolderInItsPlaceAndOrderNamedByModule(t *testing.T) {
power := Manifest{Module: "power", Contributions: []SeatContribution{
block(`{"bar":"bottom","place":"status","shows":"battery"}`),
block(`{"bar":"bottom","place":"status","order":20,"shows":"command","options":{"command":"draw \"now\"","interval":5}}`),
}}
other := Manifest{Module: "another", Contributions: []SeatContribution{
block(`{"bar":"bottom","place":"status","order":70,"shows":"battery","options":{"device":"BAT1"}}`),
block(`{"bar":"top","place":"resources","shows":"command","options":{"command":"up","interval":60}}`),
}}
r := Resolution{Node: "laptop", Account: "op", Modules: []Manifest{barHolder(), power, other}}
// Ordered by order (20, 50, 70), then module order; each piece under its module's name; the
// command quoted so its own quotes cannot end the line.
want := "own-first\nown-middle\n" +
"# power\n[[block]]\nblock = \"custom\"\ncommand = \"draw \\\"now\\\"\"\ninterval = 5\n" +
"# power\n[[block]]\nblock = \"battery\"\n" +
"# another\n[[block]]\nblock = \"battery\"\ndevice = \"BAT1\"\n" +
"own-last\n"
if got := composedFile(t, r, "a-bar.bottom"); got != want {
t.Fatalf("the bottom bar is\n%s\nnot\n%s", got, want)
}
want = "own-first\n# another\n[[block]]\nblock = \"custom\"\ncommand = \"up\"\ninterval = 60\nown-middle\nown-last\n"
if got := composedFile(t, r, "a-bar.top"); got != want {
t.Fatalf("the top bar is\n%s\nnot\n%s", got, want)
}
}
func TestAContributionIfACapabilityIsPlacedOnlyWhereTheMachineReportsIt(t *testing.T) {
c := block(`{"bar":"bottom","place":"status","shows":"battery"}`)
c.IfCapability = "battery"
power := Manifest{Module: "power", Contributions: []SeatContribution{c}}
for _, c := range []struct {
caps map[string]bool
has bool
}{{map[string]bool{"battery": true}, true}, {map[string]bool{"seat": true}, false}, {nil, false}} {
r := Resolution{Node: "n", Account: "op", Capabilities: c.caps, Modules: []Manifest{barHolder(), power}}
if bar := composedFile(t, r, "a-bar.bottom"); strings.Contains(bar, `block = "battery"`) != c.has {
t.Errorf("with %v: the battery placed is not %v:\n%s", c.caps, c.has, bar)
}
}
}
func TestIfACapabilityNamesOneTheNodeEngineDetectsOnAnOfferedKindOnly(t *testing.T) {
cases := map[string]string{
// A name nothing detects would leave the piece out on every machine.
`{"seat":"node-bar","kind":"block","if-capability":"batery","data":{"bar":"top","place":"status","shows":"battery"}}`: "which no machine reports",
// A kind the holder depends on is never left out for want of something.
`{"seat":"node-hotkeys","kind":"trigger","if-capability":"battery","content":"KEY_BATTERY 1 x"}`: "only a kind the seat offers",
`{"seat":"node-backup","kind":"backup","if-capability":"seat","content":"path /x"}`: "only a kind the seat offers",
}
for c, want := range cases {
raw := `{"module":"power","contributions":[` + c + `]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("%s: accepted, or refused without %q: %v", c, want, err)
}
}
for _, name := range []string{"battery", "power-meter", "seat"} {
if !oneOf(KnownCapabilities, name) {
t.Errorf("%s is not a known capability", name)
}
}
}
func TestResolveCarriesTheMachinesCapabilities(t *testing.T) {
node := workstation()
node.Capabilities["battery"] = true
r, err := Resolve(shelf(Manifest{Module: "a"}), []string{"a"}, node, World{})
if err != nil {
t.Fatal(err)
}
if !r.Capabilities["battery"] || !r.Capabilities["seat"] {
t.Fatalf("the resolution carries %v, not the node's %v", r.Capabilities, node.Capabilities)
}
}
func TestAPieceTheHolderCannotRenderIsLeftOutAndNamedNotTheMachine(t *testing.T) {
// A shows the shape gained after the holder's template was written: built here, past the check.
power := Manifest{Module: "power", Contributions: []SeatContribution{
block(`{"bar":"bottom","place":"status","shows":"gpu"}`),
block(`{"bar":"bottom","place":"status","shows":"battery"}`),
}}
r := Resolution{Node: "n", Account: "op", Modules: []Manifest{barHolder(), power}}
c, err := r.Compose(Rendering{})
if err != nil {
t.Fatalf("one piece failed the whole machine: %v", err)
}
if len(c.Unplaced) != 1 || !strings.Contains(c.Unplaced[0], "power's node-bar:block") ||
!strings.Contains(c.Unplaced[0], "shows=gpu") || !strings.Contains(c.Unplaced[0], "renders nothing") {
t.Fatalf("unplaced: %v", c.Unplaced)
}
for _, res := range c.Resources {
if res["id"] == "a-bar.bottom" && !strings.Contains(res["content"].(string), `block = "battery"`) {
t.Fatalf("the piece beside it was lost too: %s", res["content"])
}
}
}
func TestTheCataloguesBarRendersEveryExampleOfTheShape(t *testing.T) {
holder := catalogueManifest(t, "i3status-rust")
s, _ := SeatNamed(BarSeat)
if !placesKind(holder, s, BarKindBlock) {
t.Skip("the catalogue beside this checkout has a bar that places no blocks yet")
}
tmpl, err := holderTemplate(holder, s, BarKindBlock)
if err != nil {
t.Fatal(err)
}
for _, ex := range s.Receives[0].Shape.Examples {
out, err := renderPiece(tmpl, "example", ex)
if err != nil || !strings.Contains(out, "[[block]]") {
t.Errorf("the catalogue's bar renders %v as %q: %v", ex, out, err)
}
}
for shows := range barShows {
covered := false
for _, ex := range s.Receives[0].Shape.Examples {
covered = covered || ex["shows"] == shows
}
if !covered {
t.Errorf("the shape has no example that shows %s, so no holder is made to render it", shows)
}
}
}
func TestQuoteIsATomlBasicStringOfTheSameText(t *testing.T) {
for _, in := range []string{`plain`, `say "hi" \ there`, "tab\tnew\nline", "del\x7fend", "nul\x00 esc\x1b", "<&> é ☃"} {
q, err := quote(in)
if err != nil {
t.Fatal(err)
}
got, err := tomlBasicString(q)
if err != nil || got != in {
t.Errorf("quote(%q) = %s, which TOML reads as %q: %v", in, q, got, err)
}
}
}
// tomlBasicString reads one TOML basic string, as the TOML specification (1.0, "String") defines it:
// any character but a quote, a backslash and the control characters U+0000–U+0008, U+000A–U+001F
// and U+007F, which are written as escapes.
func tomlBasicString(s string) (string, error) {
if len(s) < 2 || s[0] != '"' || s[len(s)-1] != '"' {
return "", fmt.Errorf("not quoted")
}
var b strings.Builder
rs := []rune(s[1 : len(s)-1])
for i := 0; i < len(rs); i++ {
r := rs[i]
switch {
case r == '"':
return "", fmt.Errorf("a bare quote at %d", i)
case r == 0x7f || (r < 0x20 && r != '\t'):
return "", fmt.Errorf("a bare control character %U at %d", r, i)
case r != '\\':
b.WriteRune(r)
continue
}
i++
if i >= len(rs) {
return "", fmt.Errorf("an escape at the end")
}
switch rs[i] {
case 'b':
b.WriteRune('\b')
case 't':
b.WriteRune('\t')
case 'n':
b.WriteRune('\n')
case 'f':
b.WriteRune('\f')
case 'r':
b.WriteRune('\r')
case '"':
b.WriteRune('"')
case '\\':
b.WriteRune('\\')
case 'u', 'U':
n := 4
if rs[i] == 'U' {
n = 8
}
if i+n > len(rs)-1 {
return "", fmt.Errorf("a short \\u escape")
}
v, err := strconv.ParseUint(string(rs[i+1:i+1+n]), 16, 32)
if err != nil {
return "", err
}
b.WriteRune(rune(v))
i += n
default:
return "", fmt.Errorf("the escape \\%c is not TOML's", rs[i])
}
}
return b.String(), nil
}
func TestABlockOutsideTheShapeIsRefused(t *testing.T) {
cases := map[string]string{
`{"seat":"node-bar","kind":"block","content":"[[block]]"}`: "has no data",
`{"seat":"node-bar","kind":"block","content":"x","data":{"bar":"top","place":"status","shows":"battery"}}`: "has content",
`{"seat":"node-hotkeys","kind":"trigger","data":{"a":"b"}}`: "receives trigger as text",
`{"seat":"node-bar","kind":"block","data":{"bar":"left","place":"status","shows":"battery"}}`: `bar "left"`,
`{"seat":"node-bar","kind":"block","data":{"bar":"top","place":"middle","shows":"battery"}}`: `place "middle"`,
`{"seat":"node-bar","kind":"block","data":{"bar":"top","place":"status","shows":"gpu"}}`: "shows one of battery, command",
`{"seat":"node-bar","kind":"block","data":{"bar":"top","place":"status","shows":"battery","colour":"red"}}`: `no field "colour"`,
`{"seat":"node-bar","kind":"block","data":{"bar":"top","place":"status","shows":"battery","order":100}}`: "whole number from 0 to 99",
`{"seat":"node-bar","kind":"block","data":{"bar":"top","place":"status","shows":"command"}}`: "needs the option command",
`{"seat":"node-bar","kind":"block","data":{"bar":"top","place":"status","shows":"battery","options":{"x":"y"}}}`: `no option "x"`,
`{"seat":"node-bar","kind":"block","data":{"bar":"top","place":"status","shows":"command","options":{"command":"c","interval":0}}}`: "seconds",
`{"seat":"node-bar","kind":"block","if-capability":"Has Battery","data":{"bar":"top","place":"status","shows":"battery"}}`: "not a capability",
}
for c, want := range cases {
raw := `{"module":"power","contributions":[` + c + `]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("%s: accepted, or refused without %q: %v", c, want, err)
}
}
ok := `{"module":"power","contributions":[{"seat":"node-bar","kind":"block","if-capability":"battery",` +
`"data":{"bar":"bottom","place":"status","order":10,"shows":"command","options":{"command":"c","interval":5}}}]}`
if _, err := ParseManifest([]byte(ok)); err != nil {
t.Errorf("a block in the shape was refused: %v", err)
}
}
func TestAHolderPlacesEveryCombinationOnceWithATemplateThatRendersEveryExample(t *testing.T) {
holder := func(template string, contents ...string) string {
var res []string
for i, c := range contents {
b, _ := json.Marshal(c)
res = append(res, `{"id":"f`+string(rune('a'+i))+`","type":"file","path":"/f`+string(rune('a'+i))+`","content":`+string(b)+`}`)
}
tb, _ := json.Marshal(template)
renders := ""
if template != "" {
renders = `,"renders":{"block":` + string(tb) + `}`
}
return `{"module":"a-bar","claims":[{"name":"node-bar"` + renders + `}],"resources":[` + strings.Join(res, ",") + `]}`
}
all := "${contribution:node-bar:block:bar=bottom,place=resources}${contribution:node-bar:block:bar=bottom,place=status}" +
"${contribution:node-bar:block:bar=top,place=resources}${contribution:node-bar:block:bar=top,place=status}"
cases := map[string]string{
holder(barTemplate, "${contribution:node-bar:block:bar=bottom}"): "never places node-bar:block for bar=top,place=resources",
holder(barTemplate, all, "${contribution:node-bar:block:place=status}"): "for bar=bottom,place=status 2 times",
holder("", all): "gives no template",
holder(`{{if eq .shows "battery"}}b{{end}}`, all): "renders nothing",
holder(`{{.nothing}}`, all): "renders nothing",
holder(`{{if}`, all): "does not parse",
holder(barTemplate, "${contribution:node-bar:block:bar=left}"+all): "bar is one of bottom, top",
holder(barTemplate, "${contribution:node-bar:block:colour=red}"+all): "placed by bar, place",
holder(barTemplate, "${contribution:node-hotkeys:trigger:bar=top}"): "does not claim node-hotkeys",
}
for raw, want := range cases {
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("%s:\naccepted, or refused without %q: %v", raw, want, err)
}
}
for _, raw := range []string{holder(barTemplate, all), holder(barTemplate, "${contribution:node-bar:block}"),
// A holder that places no block at all is older than the kind, and is not refused for it.
holder("", "no blocks here")} {
if _, err := ParseManifest([]byte(raw)); err != nil {
t.Errorf("%s: refused: %v", raw, err)
}
}
}
func TestAnOfferedBlockMakesNoDependencyOnTheBar(t *testing.T) {
power := Manifest{Module: "power", Contributions: []SeatContribution{
block(`{"bar":"bottom","place":"status","shows":"battery"}`),
{Seat: HotkeysSeat, Kind: "trigger", Content: "x"},
}}
for _, seat := range DependsOn(power) {
if seat == BarSeat {
t.Fatalf("a block made the power module depend on the bar: %v", DependsOn(power))
}
}
// And on a machine with no bar it is placed nowhere, and nothing fails.
r := Resolution{Node: "server", Account: "op", Capabilities: map[string]bool{"battery": true},
Modules: []Manifest{hotkeysHolder(), power}}
if _, err := r.Declaration(Rendering{}); err != nil {
t.Fatal(err)
}
}
func TestTheBarReceivesBlocksAsData(t *testing.T) {
s, _ := SeatNamed(BarSeat)
if len(s.Receives) != 1 || s.Receives[0].Kind != BarKindBlock || s.Receives[0].Shape == nil || !s.Receives[0].Offered {
t.Fatalf("node-bar receives %+v", s.Receives)
}
for _, ex := range s.Receives[0].Shape.Examples {
if p := barBlockProblems(ex); len(p) > 0 {
t.Errorf("the shape's own example %v is refused: %v", ex, p)
}
}
}
+4 -1
View File
@@ -118,8 +118,11 @@ func contributedTo(m Manifest) []string {
}
// Any other seat's contribution (novox/hq ADR 0212 §4), by the seat's canonical name; one the
// mesh does not define is refused at registration and depends on nothing here.
// A kind the seat only offers to place makes no dependency (novox/hq ADR 0255).
for _, c := range m.Contributions {
if s, known := SeatNamed(c.Seat); known {
if s, r, ok := receivable(c.Seat, c.Kind); ok && r.Offered {
continue
} else if s.Name != "" {
out = append(out, s.Name)
}
}
+7
View File
@@ -68,6 +68,13 @@ type Receivable struct {
// only the mesh knows where. Off for every kind written in a tool's grammar that has its own
// `${…}` — a shell's — where the mesh filling one would change what the tool reads.
Dirs bool
// Shape, when set, makes the kind data rather than text (novox/hq ADR 0255): a contributor gives
// `data` in this shape, and the holder renders each piece with the template its claim gives.
Shape *Shape
// Offered says a contribution of this kind does not depend on the seat (novox/hq ADR 0255): where
// nobody holds it nothing is placed, and the contributor runs there all the same. A battery's bar
// block is offered — the power module is on every machine, and a server has no bar.
Offered bool
}
// defaultSeats is the set the mesh ships with — the seed for the control plane's seat table and the
+17 -1
View File
@@ -449,7 +449,9 @@ var ControllerVerbs = []Verb{
{Name: "collect", Description: "The artifact store's sweep, now: what the mesh made and keeps for no reason " +
"is let go of. Without confirm a dry run — every kept archive asked about with HEADs, what would be let go " +
"listed, nothing held or deleted. With confirm and why: every kept archive held first, then each eligible " +
"artifact let go of, oldest first, and recorded collected — a hand act, recorded with its why. Bounded by " +
"artifact let go of, oldest first, and recorded collected — a hand act, recorded with its why. A confirmed " +
"collect lets an index go with the platform manifests no kept index names (read first; any read that " +
"fails lets nothing go); the sweep after a build lets an index go alone (novox/hq ADR 0257). Bounded by " +
"most (500 by default, at most 5000) and 45 seconds; what is left is said. Bytes are reclaimed by the " +
"store's nightly collector. Never touches a digest the mesh did not record (novox/hq ADR 0189, ADR 0251).",
Input: schema(map[string]string{
@@ -457,6 +459,20 @@ var ControllerVerbs = []Verb{
"why": "with confirm: why — required, and recorded in the hand-act log",
"most": "let go of at most this many (500 by default, at most 5000)",
}, nil, "confirm")},
{Name: "mirrors", Description: "The bases the mesh copied into its artifact store for builds to stand on, as " +
"the records hold them: each with its state — kept because a kept build stood on it, or eligible — and " +
"which build or person recorded it. With record: references of copies no record names (a copy made " +
"before copies were recorded, or by a build that failed before then) to record as the mesh's, so the " +
"sweep keeps or lets go of them like any other; only a repository the mirror writes is taken " +
"(upstream/<host>/<path>, or a module's <module>/on-<argument>), and each must be in the store. Without " +
"confirm a dry run; with confirm and why, recorded — a hand act. A recorded copy is eligible: the next " +
"sweep lets it go unless a kept build stood on it. Records only: this verb deletes nothing " +
"(novox/hq ADR 0257).",
Input: schema(map[string]string{
"record": "references to record, artifact-store://<repository>@sha256:<hex>, separated by spaces or commas",
"confirm": "\"true\": record them (needs why); without it nothing is recorded",
"why": "with confirm: why — required, and recorded with each copy and in the hand-act log",
}, nil, "confirm")},
{Name: "images", Description: "Every container image one machine's declaration names — the declaration the " +
"mesh would send it now and the one it was last sent — each with the resources naming it: what the " +
"machine keeps when it prunes its images. sent_known is false when the last one sent is not kept with " +
+3
View File
@@ -36,6 +36,9 @@ const (
ChangeSilenced = "silenced"
ChangeUnsilenced = "silence-ended"
ChangeCleared = "cleared"
// ChangeWords is what the operator needs, or the answers offered, said differently (novox/hq ADR
// 0258): a warning kept quiet as needing nothing is said once its words need the operator.
ChangeWords = "words"
)
// Event is the body of every condition event, and the shape a history entry keeps: **the condition
+69 -10
View File
@@ -62,10 +62,37 @@ const (
)
// SilenceAction is the action that stops a condition's messages for a week, with the operator's why: the
// answer to a condition the operator decided to live with.
// answer to a condition the operator decided to live with. It is an acknowledgement (ADR 0234 §8): the
// only kind of answer a desk click performs until answers are authorised (novox/hq ADR 0258). Its cause
// marks it as an answer, which the hand-act log does not count as a repair.
func SilenceAction(key string) Action {
return Action{Label: "Silence for a week", Verb: "mesh-controller.conditions",
Arguments: map[string]string{"silence": key, "for": "7d", "why": ""}}
Arguments: map[string]string{"silence": key, "for": "7d", "why": "", "cause": CauseOperatorAnswer}}
}
// CauseOperatorAnswer is the cause an answer chosen on a notification is recorded with.
const CauseOperatorAnswer = "operator-answer"
// FallbackNeeds is what a condition needs when its producer's words needed the operator and could not
// be said: the verdict is kept, never turned into "Nothing for you to do." (ADR 0258).
const FallbackNeeds = "read the details to see what to do."
func sameActions(a, b []Action) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i].Label != b[i].Label || a[i].Verb != b[i].Verb || a[i].Machine != b[i].Machine ||
len(a[i].Arguments) != len(b[i].Arguments) {
return false
}
for k, v := range a[i].Arguments {
if b[i].Arguments[k] != v {
return false
}
}
}
return true
}
// Bounds of the plain words: a headline fits a notification's title line, an explanation two sentences.
@@ -108,7 +135,9 @@ var (
goDuration = regexp.MustCompile(`\b[0-9]+(h[0-9]+m|m[0-9]+s|h[0-9]+m[0-9]+s)\b|\b[0-9]+(\.[0-9]+)?(ns|ms|µs)\b`)
clockTime = regexp.MustCompile(`\b[0-9]{1,2}:[0-9]{2}\b|\b[0-9]{4}-[0-9]{2}-[0-9]{2}\b|\bUTC\b`)
flag = regexp.MustCompile(`(^|\s)--?[a-z]`)
agentWord = regexp.MustCompile(`(?i)\bagents?\b|\bby hand\b|\bsession\b`)
agentWord = regexp.MustCompile(`(?i)\bagents?\b`)
byHand = regexp.MustCompile(`(?i)\bby hand\b`)
latinAbbr = regexp.MustCompile(`\b(e\.g|i\.e|etc)\.`)
markup = regexp.MustCompile("[`*<>{}\\[\\]|#]|(^|\\s)_|_(\\s|$)")
)
@@ -126,14 +155,15 @@ func Plain(text string, machines ...string) (string, bool) {
if m := markup.FindString(text); m != "" {
return "markup (" + m + ")", false
}
text = latinAbbr.ReplaceAllString(text, "eg")
for _, shape := range []struct {
re *regexp.Regexp
what string
}{{hexID, "a hash"}, {numberedID, "a numbered id"}, {dotted, "a dotted name"}, {goDuration, "a duration in code"},
{clockTime, "a clock time or date"}, {flag, "a command's flag"}} {
if m := shape.re.FindString(text); m != "" {
if shape.re == hexID && !strings.ContainsAny(m, "0123456789") {
continue // a word of letters a to f only
if shape.re == hexID && (!strings.ContainsAny(m, "0123456789") || !strings.ContainsAny(m, "abcdef")) {
continue // letters a to f only, or a number: words and counts, not a hash
}
return shape.what + " (" + strings.TrimSpace(m) + ")", false
}
@@ -158,16 +188,14 @@ func PlainWords(w Words, machines ...string) (string, bool) {
if len(w.Explanation) > ExplanationMax {
return fmt.Sprintf("explanation: longer than %d characters", ExplanationMax), false
}
for _, m := range agentWord.FindAllString(w.Explanation, -1) {
if !strings.EqualFold(m, "by hand") { // "repaired by hand" is a fact; telling to act by hand is not
return "explanation: sends the operator elsewhere (" + m + ")", false
}
if m := agentWord.FindString(w.Explanation); m != "" {
return "explanation: sends the operator elsewhere (" + m + ")", false
}
if w.Needs != "" {
if why, ok := Plain(w.Needs, machines...); !ok {
return "needs: " + why, false
}
if m := agentWord.FindString(w.Needs); m != "" {
if m := agentWord.FindString(w.Needs) + byHand.FindString(w.Needs); m != "" {
return "needs: not something the operator does themselves (" + m + ")", false
}
if len(w.Needs) > NeedsMax || !strings.HasSuffix(w.Needs, ".") {
@@ -221,6 +249,8 @@ func plainly(o Observation) Observation {
if w.Headline != "" && w.Resolved == "" {
w.Resolved = "Resolved: " + lowerFirst(w.Headline)
}
// An explanation too long is cut, never refused: what it says first is what matters.
w.Explanation = cut(w.Explanation, ExplanationMax-len(Verdict(w.Needs, ""))-1)
if from == "" {
if Unworded != nil {
Unworded(o, "the kind "+o.Kind+" has no plain words")
@@ -230,13 +260,42 @@ func plainly(o Observation) Observation {
if Unworded != nil {
Unworded(o, from+" is not plain: "+why)
}
// The scope's words, but never a weaker verdict: what needed the operator still does, and its
// answers are kept where they are themselves sound.
needed, actions := w.Needs != "", soundActions(w.Actions)
w = scopeWords(o)
if needed {
w.Needs, w.Actions = FallbackNeeds, actions
}
}
o.Headline, o.Resolved, o.Needs, o.Actions = w.Headline, w.Resolved, w.Needs, w.Actions
o.Explanation = Verdict(w.Needs, w.Explanation)
return o
}
// soundActions are the actions with a label and a verb.
func soundActions(list []Action) []Action {
var out []Action
for _, a := range list {
if a.Label != "" && len(a.Label) <= 24 && a.Verb != "" {
out = append(out, a)
}
}
return out
}
// cut shortens text to at most n bytes at a word, ending it with an ellipsis.
func cut(text string, n int) string {
if len(text) <= n || n < 10 {
return text
}
t := text[:n-3]
if i := strings.LastIndexByte(t, ' '); i > n/2 {
t = t[:i]
}
return strings.TrimRight(t, " ,;:") + "…"
}
// Verdict is an explanation opened by its verdict.
func Verdict(needs, explanation string) string {
if needs == "" {
+86 -2
View File
@@ -11,7 +11,7 @@ import (
func TestThePlainRuleRefusesIdentifiersCodeAndTimes(t *testing.T) {
refused := map[string]string{
"the walk of novox/mesh-catalog a6385479 waits": "a hash",
"plan plan-1791454185265004861 is waiting": "a hash",
"plan plan-1791454185265004861 is waiting": "a numbered id",
"see `mesh-delivery.show` for why": "markup",
"call mesh-delivery.show for why": "a dotted name",
"key plan.x.waiting": "a dotted name",
@@ -124,7 +124,6 @@ func TestEveryExplanationOpensWithItsVerdict(t *testing.T) {
for _, w := range []Words{
{Headline: "x waits", Explanation: "Have an agent start it.", Resolved: "x started"},
{Headline: "x waits", Explanation: "It waits.", Needs: "start it by hand.", Resolved: "x started"},
{Headline: "x waits", Explanation: "It waits.", Needs: "open a session and start it.", Resolved: "x started"},
{Headline: "x waits", Explanation: "It waits.", Needs: "start it", Resolved: "x started"},
{Headline: "x waits", Explanation: "It waits.", Resolved: "x started", Actions: []Action{{Label: "Start"}}},
} {
@@ -161,3 +160,88 @@ func TestAnEscalatedConditionSaysItNeedsTheOperator(t *testing.T) {
t.Errorf("seen again after escalation: %q", c.Explanation)
}
}
// **What the plain rule refused wrongly** (novox/hq ADR 0258): a login session is the operator's own word,
// "e.g." is not a dotted name, a long count is not a hash, and a long explanation is cut, not refused.
func TestThePlainRuleRefusesNoPlainWords(t *testing.T) {
for _, text := range []string{
"log out of every session on g14 and log in again.",
"A module, e.g. the lighting daemon, waits.",
"It holds 12345678 files.",
"A person repaired push by hand 35 times in 14 days.",
} {
if why, ok := Plain(text, "g14"); !ok {
t.Errorf("%q refused: %s", text, why)
}
}
if _, ok := Plain("commit 0123abc landed"); ok {
t.Error("a hash passed")
}
k, _, _, _ := keeper(t)
long := strings.Repeat("The bus refused a message. ", 30)
Wording("test-long", func(o Observation) Words {
return Words{Headline: "A long one", Explanation: long, Resolved: "Done"}
})
before := Unworded
var unworded []string
Unworded = func(o Observation, why string) { unworded = append(unworded, why) }
t.Cleanup(func() { Unworded = before })
c, err := k.Observe(t.Context(), Observation{Scope: ScopeBus, ID: "x", Kind: "test-long", Severity: Warning,
Source: "test", Summary: "long"})
if err != nil {
t.Fatal(err)
}
if c.Headline != "A long one" || len(c.Explanation) > ExplanationMax || !strings.HasSuffix(c.Explanation, "…") ||
len(unworded) != 0 {
t.Errorf("a long explanation was not cut: %d %q %v", len(c.Explanation), c.Headline, unworded)
}
}
// **Words refused keep their verdict and their answers** (ADR 0258): a producer that needed the operator
// still does, said as "read the details", and its sound answers stay.
func TestRefusedWordsKeepTheirVerdictAndAnswers(t *testing.T) {
k, _, _, _ := keeper(t)
before := Unworded
Unworded = func(Observation, string) {}
t.Cleanup(func() { Unworded = before })
silence := SilenceAction("module.openrazer.g14.x")
c, err := k.Observe(t.Context(), Observation{Scope: ScopeModule, ID: "openrazer.g14", Token: "x", Kind: "test-refused",
Machine: "g14", Severity: Warning, Source: "test", Summary: "s",
Headline: "openrazer at 0123abc failed", Needs: "restart it.", Explanation: "It failed.",
Actions: []Action{silence}})
if err != nil {
t.Fatal(err)
}
if c.Needs != FallbackNeeds || !strings.HasPrefix(c.Explanation, NeedsYou+" "+FallbackNeeds) ||
len(c.Actions) != 1 || c.Actions[0].Label != silence.Label || strings.Contains(c.Headline, "0123abc") {
t.Errorf("the verdict or the answers were lost: %+v", c)
}
// Words that needed nothing stay "Nothing for you to do.".
c, _ = k.Observe(t.Context(), Observation{Scope: ScopeModule, ID: "openrazer.g14", Token: "y", Kind: "test-refused",
Machine: "g14", Severity: Warning, Source: "test", Summary: "s", Headline: "at 0123abc", Explanation: "x."})
if c.Needs != "" || !strings.HasPrefix(c.Explanation, NothingToDo) {
t.Errorf("%+v", c)
}
}
// **Words that now need the operator are said as a change** (ADR 0258): a condition whose needs or answers
// change emits condition-changed, so a warning kept quiet is said.
func TestAChangeOfWordsIsSaid(t *testing.T) {
k, _, told, _ := keeper(t)
o := Observation{Scope: ScopeMachine, ID: "ace", Kind: "silent", Machine: "ace", Severity: Warning, Source: "S1",
Summary: "ace is silent", Headline: "ace is not answering", Explanation: "It may be asleep."}
if _, err := k.Observe(t.Context(), o); err != nil {
t.Fatal(err)
}
if _, err := k.Observe(t.Context(), o); err != nil { // the same words: no event
t.Fatal(err)
}
o.Needs = "check that ace is on."
if _, err := k.Observe(t.Context(), o); err != nil {
t.Fatal(err)
}
said := settled(t, told, 2)
if len(said) != 2 || said[1].Change != ChangeWords || said[1].Event != EventChanged || said[1].Needs != o.Needs {
t.Fatalf("%+v", said)
}
}
+4
View File
@@ -240,8 +240,12 @@ func (k *Keeper) Observe(ctx context.Context, o Observation) (Condition, error)
// The kind as the source says it now: a source that gave the same key a kind of its own since
// (a probe's finding split out for a healer) is read by that kind from its next observation.
c.Kind, c.Summary, c.Source, c.LastObserved = o.Kind, o.Summary, o.Source, now
wasNeeds, wasActions := c.Needs, c.Actions
c.Headline, c.Explanation, c.Resolved, c.Needs, c.Actions = o.Headline, o.Explanation, o.Resolved, o.Needs, o.Actions
escalatedWords(&c)
if c.Needs != wasNeeds || !sameActions(c.Actions, wasActions) {
changes = append(changes, Event{Change: ChangeWords, Was: wasNeeds})
}
if o.Machine != "" {
c.Subject.Machine = o.Machine
}
+38 -1
View File
@@ -39,6 +39,9 @@ type Build struct {
// Against is every artifact this build stood on, as references rather than module names —
// what makes a build edge derived rather than declared (ADR 0009).
Against []string
// Mirrored is every base this build copied into the artifact store (novox/hq ADR 0257), said
// whether the build worked or not: the copy is the mesh's, recorded in artifact_mirrored.
Mirrored []string
// Read is every repository this build read source from besides the module's own (novox/hq
// 04-ISSUES/131), at the ref it read.
Read []ReadRepository
@@ -120,7 +123,41 @@ func (i *Inventory) RecordBuild(ctx context.Context, b Build) error {
on conflict (id) do nothing`,
b.ID, b.Repository, b.Ref, module, b.Commit, b.On, b.Failed, made,
b.Path, manifestOrNil(b.Manifest), against, read, asked, b.SourceFingerprint)
return err
if err != nil {
return err
}
return i.RecordMirrored(ctx, b.ID, "", b.Mirrored)
}
// RecordMirrored keeps that the mesh copied these bases into the artifact store (novox/hq ADR 0257):
// by the build that said so, or by a person who says why. The first record of a copy stands.
//
// **A copy a build says it holds is held again.** The builder answers a copy only once the store holds
// it whole, copying again what was let go of; so a copy the records say was collected, and a build now
// says it copied, is no longer collected. Left collected, it would stay in the store and out of every
// sweep for ever.
func (i *Inventory) RecordMirrored(ctx context.Context, build, why string, references []string) error {
var by *string
if build != "" {
by = &build
}
for _, reference := range references {
if reference == "" {
continue
}
if _, err := i.store.Pool().Exec(ctx,
`insert into artifact_mirrored (reference, build, why) values ($1, $2, $3)
on conflict (reference) do nothing`, reference, by, why); err != nil {
return err
}
if build != "" {
if _, err := i.store.Pool().Exec(ctx,
`delete from artifact_collected where reference = $1`, reference); err != nil {
return err
}
}
}
return nil
}
// Builds is what has happened lately, newest first.
+118 -10
View File
@@ -37,6 +37,10 @@ const (
// KeptUnaddressable: it names no digest, so nothing here can speak for it, and it is kept rather
// than guessed about.
KeptUnaddressable = "unaddressable"
// KeptStoodOn: it is a base the mesh copied in, and one of the KeptBuilds most recent successful
// builds of a module the mesh still holds stood on it (novox/hq ADR 0257). A copy has no builds of
// its own; it is kept by the builds that used it.
KeptStoodOn = "stood-on"
)
// The states an artifact the mesh recorded making is in (novox/hq ADR 0251).
@@ -77,7 +81,7 @@ func (i *Inventory) Artifacts(ctx context.Context) ([]ArtifactState, error) {
if err != nil {
return nil, err
}
all, err := i.everyReferenceMade(ctx)
all, err := i.everyReferenceRecorded(ctx)
if err != nil {
return nil, err
}
@@ -156,8 +160,9 @@ func (i *Inventory) KeepSet(ctx context.Context) (map[string][]string, error) {
// join every module ever built kept five builds' artifacts for ever — and once the store's
// collector runs for real, what the keep set says is what the disk holds.
recent, err := i.store.Pool().Query(ctx,
`select made from (
select b.made, row_number() over (partition by b.module order by b.at desc, b.id desc) as back
`select made, built_against from (
select b.made, b.built_against,
row_number() over (partition by b.module order by b.at desc, b.id desc) as back
from build b
join module m on m.name = b.module
where b.failed = '' and b.module is not null and b.module <> ''
@@ -166,18 +171,26 @@ func (i *Inventory) KeepSet(ctx context.Context) (map[string][]string, error) {
return nil, err
}
defer recent.Close()
var stoodOn []string
for recent.Next() {
var raw []byte
if err := recent.Scan(&raw); err != nil {
var raw, against []byte
if err := recent.Scan(&raw, &against); err != nil {
return nil, err
}
var made []Artifact
if err := json.Unmarshal(raw, &made); err != nil {
continue
if err := json.Unmarshal(raw, &made); err == nil {
for _, a := range made {
if reference := asRecorded(a.Reference); reference != "" {
because(reference, KeptByRecentBuild)
}
}
}
for _, a := range made {
if reference := asRecorded(a.Reference); reference != "" {
because(reference, KeptByRecentBuild)
var bases []string
if err := json.Unmarshal(against, &bases); err == nil {
for _, b := range bases {
if reference := asRecorded(b); reference != "" {
stoodOn = append(stoodOn, reference)
}
}
}
}
@@ -185,6 +198,27 @@ func (i *Inventory) KeepSet(ctx context.Context) (map[string][]string, error) {
return nil, err
}
// **A copied base is kept by the kept builds that stood on it, and by nothing else** (novox/hq
// ADR 0257). It has no builds of its own to count five of, and no machine is ever handed a base,
// so a definition is never its reason: a definition names the image upstream, by the digest every
// copy of it shares, and read as a reason it would keep every module's copy for as long as any
// module stood on that image. A copy already let go of stays let go of.
mirrored, err := i.mirroredSet(ctx)
if err != nil {
return nil, err
}
if len(mirrored) > 0 {
collected, err := i.alreadyCollected(ctx)
if err != nil {
return nil, err
}
for _, reference := range stoodOn {
if mirrored[reference] && !collected[reference] {
because(reference, KeptStoodOn)
}
}
}
// And anything a manifest mentions — every reference ever made, the recent ones included, so a
// kept artifact carries each of its reasons and the operator reads why, not only that (novox/hq
// ADR 0251). A manifest holds a reference composed with the store's address or kept bare, so the
@@ -195,6 +229,9 @@ func (i *Inventory) KeepSet(ctx context.Context) (map[string][]string, error) {
return nil, err
}
for _, reference := range all {
if mirrored[reference] {
continue
}
digest := digestIn(reference)
if digest == "" {
// Not something the store holds by digest; nothing here can speak for it, so it
@@ -237,6 +274,72 @@ func (i *Inventory) KeptArchives(ctx context.Context) ([]string, error) {
return out, nil
}
// KeptImages is every image the mesh keeps, in a stated order: the manifests whose platform manifests
// a sweep spares when it lets an index of the same repository go (novox/hq ADR 0257).
func (i *Inventory) KeptImages(ctx context.Context) ([]string, error) {
keep, err := i.KeepSet(ctx)
if err != nil {
return nil, err
}
var out []string
for reference := range keep {
if path, ours := catalogue.InArtifactStore(reference); ours && strings.Contains(path, "@sha256:") {
out = append(out, reference)
}
}
sort.Strings(out)
return out, nil
}
// everyReferenceRecorded is every artifact the mesh recorded putting in its store: what each
// successful build made, and every base a build — or a person — recorded copying in (novox/hq ADR
// 0257). Oldest first, a copy placed at the moment it was recorded.
func (i *Inventory) everyReferenceRecorded(ctx context.Context) ([]string, error) {
made, err := i.everyReferenceMade(ctx)
if err != nil {
return nil, err
}
rows, err := i.store.Pool().Query(ctx, `select reference from artifact_mirrored order by at asc, reference asc`)
if err != nil {
return nil, err
}
defer rows.Close()
seen := map[string]bool{}
for _, r := range made {
seen[r] = true
}
out := made
for rows.Next() {
var reference string
if err := rows.Scan(&reference); err != nil {
return nil, err
}
if reference = asRecorded(reference); reference != "" && !seen[reference] {
seen[reference] = true
out = append(out, reference)
}
}
return out, rows.Err()
}
// mirroredSet is every base the records say the mesh copied in, as recorded.
func (i *Inventory) mirroredSet(ctx context.Context) (map[string]bool, error) {
rows, err := i.store.Pool().Query(ctx, `select reference from artifact_mirrored`)
if err != nil {
return nil, err
}
defer rows.Close()
out := map[string]bool{}
for rows.Next() {
var reference string
if err := rows.Scan(&reference); err != nil {
return nil, err
}
out[asRecorded(reference)] = true
}
return out, rows.Err()
}
// everyReferenceMade is every artifact reference any successful build recorded, oldest build first.
func (i *Inventory) everyReferenceMade(ctx context.Context) ([]string, error) {
rows, err := i.store.Pool().Query(ctx,
@@ -331,3 +434,8 @@ func (i *Inventory) MarkCollected(ctx context.Context, references []string) erro
}
return nil
}
// Mirrored is every base the records say the mesh copied into its store (novox/hq ADR 0257).
func (i *Inventory) Mirrored(ctx context.Context) (map[string]bool, error) {
return i.mirroredSet(ctx)
}
+5
View File
@@ -115,6 +115,11 @@ func dependenciesOf(entries []Entry, against map[string][]string, read map[strin
}
for _, ref := range against[name] {
if rest, ok := strings.CutPrefix(ref, catalogue.ArtifactStoreScheme); ok {
// A copy of an upstream image is no module's artifact (novox/hq ADR 0257): its
// repository is named for the image, and its first segment names no module.
if strings.HasPrefix(rest, builder.MirrorPrefix) {
continue
}
if base, _, found := strings.Cut(rest, "/"); found {
add(name, base, EdgeStandsOn)
}
+17
View File
@@ -154,3 +154,20 @@ func TestNothingStandsOnTheBuildAgentOnlyIsBuiltByIt(t *testing.T) {
t.Errorf("everything source-built is built by the agent: %d edges", builtBy)
}
}
// A copy of an upstream image is no module's artifact (novox/hq ADR 0257), even when a module happens to
// carry the name its repository starts with.
func TestACopyOfAnUpstreamImageIsNoEdge(t *testing.T) {
entries := []Entry{
{Manifest: catalogue.Manifest{Module: "route-proxy"}},
{Manifest: catalogue.Manifest{Module: "upstream"}},
}
against := map[string][]string{
"route-proxy": {catalogue.ArtifactStoreScheme + "upstream/docker.io/library/golang@sha256:a"},
}
for _, e := range dependenciesOf(entries, against, nil) {
if e.From == "route-proxy" && e.To == "upstream" {
t.Fatalf("a copy of an upstream image was read as a module's artifact: %+v", e)
}
}
}
+3
View File
@@ -28,6 +28,9 @@ type ResourceHealth struct {
// (ADR 0240 Phase B, to-be 48 §6).
Check string `json:"check,omitempty"`
Needs string `json:"needs,omitempty"`
// Account is the account whose own service manager runs it, or the account a resource of kind account
// is (novox/hq ADR 0254).
Account string `json:"account,omitempty"`
}
// NodeHealth is a machine's newest statement, as kept.
@@ -0,0 +1,30 @@
-- A base a build copies into the artifact store is recorded where it is copied (novox/hq ADR 0257).
--
-- A build that stands on an image published elsewhere copies it into the store first (ADR 0096, 0097),
-- and until now nothing recorded the copy as something the mesh made: the build named it only in what
-- it stood on, and a build that failed after copying named it nowhere. So on 2026-10-08 the store held
-- 374 manifests of copied bases, in 27 repositories, that the sweep could never let go of, because no
-- record said the mesh had put them there (ADR 0189 §3).
--
-- One row per copy, by the reference as the mesh records it. `build` is the build that first said it
-- copied it, whether that build worked or not; `why` is a person's words when the copy was recorded by
-- hand (the `mirrors` verb), and empty otherwise. Not a foreign key to build: a build's row may be
-- written after its copy is, and a copy recorded by hand has no build.
create table artifact_mirrored (
reference text primary key,
build text,
at timestamptz not null default now(),
why text not null default ''
);
-- The copies the records already name. A build that worked kept what it stood on, and every reference
-- into a module's own `on-<argument>` repository there is a copy that build's mirror made: only the
-- mirror ever wrote a repository so named (ADR 0097). The earliest build to name each is its maker.
insert into artifact_mirrored (reference, build, at)
select distinct on (stood_on) stood_on, b.id, b.at
from build b,
jsonb_array_elements_text(case when jsonb_typeof(b.built_against) = 'array'
then b.built_against else '[]'::jsonb end) as stood_on
where stood_on ~ '^artifact-store://[a-z0-9][a-z0-9._-]*/on-[a-z0-9_]+@sha256:[0-9a-f]{64}$'
order by stood_on, b.at asc, b.id asc
on conflict (reference) do nothing;
+211
View File
@@ -0,0 +1,211 @@
package inventory
import (
"context"
"fmt"
"os"
"slices"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// A base a build copies into the store is recorded where it is copied, and kept while a kept build stood
// on it (novox/hq ADR 0257).
func mirrorRef(repository string, n int) string {
return fmt.Sprintf("%s%s@sha256:%064x", catalogue.ArtifactStoreScheme, repository, n)
}
// builtOn records a successful build of a module that made one image and stood on these bases.
func builtOn(t *testing.T, inv *Inventory, id, module string, n int, mirrored []string, against ...string) {
t.Helper()
b := aBuild(id, module, "")
b.Made = []Artifact{{Name: "app", Kind: "image", Reference: ref(module, "app", n)}}
b.Against = against
b.Mirrored = mirrored
if err := inv.RecordBuild(context.Background(), b); err != nil {
t.Fatal(err)
}
}
func stateOf(t *testing.T, inv *Inventory, reference string) ArtifactState {
t.Helper()
states, err := inv.Artifacts(context.Background())
if err != nil {
t.Fatal(err)
}
for _, s := range states {
if s.Reference == reference {
return s
}
}
return ArtifactState{Reference: reference, State: "unrecorded"}
}
func TestACopyAFailedBuildMadeIsRecordedAndNothingKeepsIt(t *testing.T) {
inv := fresh(t)
golang := mirrorRef("upstream/docker.io/library/golang", 1)
failed := aBuild("f1", "", "the recipe fetched an undeclared image")
failed.Mirrored = []string{golang}
if err := inv.RecordBuild(context.Background(), failed); err != nil {
t.Fatal(err)
}
if s := stateOf(t, inv, golang); s.State != ArtifactEligible {
t.Fatalf("a copy only a failed build made reads %+v; want eligible", s)
}
}
func TestACopyIsKeptByTheKeptBuildsThatStoodOnItAndByNothingElse(t *testing.T) {
inv := fresh(t)
ctx := context.Background()
holding(t, inv, "proxy")
old := mirrorRef("upstream/docker.io/library/golang", 1)
current := mirrorRef("upstream/docker.io/library/golang", 2)
// The first build stood on the old copy; six more on the current one. The old copy falls out of
// the five kept builds; the current one is stood on by all five.
builtOn(t, inv, "b01", "proxy", 1, []string{old}, old)
for i := 2; i <= 7; i++ {
builtOn(t, inv, fmt.Sprintf("b%02d", i), "proxy", i, []string{current}, current)
}
if s := stateOf(t, inv, old); s.State != ArtifactEligible {
t.Fatalf("a copy no kept build stood on reads %+v", s)
}
if s := stateOf(t, inv, current); s.State != ArtifactKept || !slices.Equal(s.Why, []string{KeptStoodOn}) {
t.Fatalf("a copy the kept builds stood on reads %+v", s)
}
// A definition naming the image upstream, by the digest every copy shares, keeps no copy.
m := catalogue.Manifest{Module: "proxy", Version: "2",
Build: &catalogue.Build{On: []catalogue.BuildsOn{{Arg: "GO_BASE", Image: "golang@" + strings.SplitN(old, "@", 2)[1]}}}}
if err := inv.RegisterModule(ctx, m, Source{Repository: "https://forge.invalid/proxy.git"}); err != nil {
t.Fatal(err)
}
if s := stateOf(t, inv, old); s.State != ArtifactEligible {
t.Fatalf("a definition naming the upstream image kept the old copy: %+v", s)
}
go_, err := inv.ToCollect(ctx)
if err != nil {
t.Fatal(err)
}
if !slices.Contains(go_, old) || slices.Contains(go_, current) {
t.Fatalf("offered %v to collect", go_)
}
// Once let go of, a copy stays let go of, even should a kept build still name it.
if err := inv.MarkCollected(ctx, []string{current}); err != nil {
t.Fatal(err)
}
if s := stateOf(t, inv, current); s.State != ArtifactCollected {
t.Fatalf("a collected copy reads %+v", s)
}
}
func TestAModulesBaseIsNotKeptAsACopy(t *testing.T) {
// A base that is another module's artifact is kept by that module's own builds, as before: the
// stood-on reason is a copy's alone.
inv := fresh(t)
holding(t, inv, "app")
base := ref("go-base", "build", 1)
b := aBuild("g1", "go-base", "")
b.Made = []Artifact{{Name: "build", Kind: "image", Reference: base}}
if err := inv.RecordBuild(context.Background(), b); err != nil {
t.Fatal(err)
}
builtOn(t, inv, "a1", "app", 1, nil, base)
if s := stateOf(t, inv, base); s.State != ArtifactEligible {
t.Fatalf("a forgotten module's artifact read %+v because a build stood on it", s)
}
}
func TestAPersonRecordsACopyAndTheFirstRecordStands(t *testing.T) {
inv := fresh(t)
ctx := context.Background()
copyRef := mirrorRef("route-proxy/on-go_base", 3)
if err := inv.RecordMirrored(ctx, "", "copied before copies were recorded", []string{copyRef}); err != nil {
t.Fatal(err)
}
if err := inv.RecordMirrored(ctx, "b9", "", []string{copyRef}); err != nil {
t.Fatal(err)
}
known, err := inv.Mirrored(ctx)
if err != nil || !known[copyRef] {
t.Fatalf("recorded copies %v %v", known, err)
}
var why string
if err := inv.store.Pool().QueryRow(ctx, `select why from artifact_mirrored where reference = $1`, copyRef).Scan(&why); err != nil {
t.Fatal(err)
}
if why != "copied before copies were recorded" {
t.Fatalf("the first record did not stand: %q", why)
}
if s := stateOf(t, inv, copyRef); s.State != ArtifactEligible {
t.Fatalf("a copy a person recorded reads %+v", s)
}
}
// The migration records the copies the build records already name: only a module's own on-<argument>
// repository, and only once, by the earliest build naming it.
func TestTheCopiesTheRecordsAlreadyNameAreRecorded(t *testing.T) {
inv := fresh(t)
ctx := context.Background()
copyRef := mirrorRef("route-proxy/on-go_base", 1)
notACopy := ref("go-base", "build", 2)
builtOn(t, inv, "b1", "route-proxy", 1, nil, copyRef, notACopy)
builtOn(t, inv, "b2", "route-proxy", 2, nil, copyRef)
// The rows a mesh had before the table: written by builds that did not say what they copied.
if _, err := inv.store.Pool().Exec(ctx, `truncate artifact_mirrored`); err != nil {
t.Fatal(err)
}
sql, err := os.ReadFile("migrations/0081-a-mirrored-base-is-recorded.sql")
if err != nil {
t.Fatal(err)
}
_, backfill, _ := strings.Cut(string(sql), "insert into artifact_mirrored")
if _, err := inv.store.Pool().Exec(ctx, "insert into artifact_mirrored"+backfill); err != nil {
t.Fatal(err)
}
var reference, build string
var n int
if err := inv.store.Pool().QueryRow(ctx, `select count(*) from artifact_mirrored`).Scan(&n); err != nil || n != 1 {
t.Fatalf("%d copies recorded (%v); want the one", n, err)
}
if err := inv.store.Pool().QueryRow(ctx, `select reference, build from artifact_mirrored`).Scan(&reference, &build); err != nil {
t.Fatal(err)
}
if reference != copyRef || build != "b1" {
t.Fatalf("recorded %s by %s", reference, build)
}
}
// A copy let go of and copied again by a later build is held again: its record is back, and the sweep
// decides about it as before.
func TestACopyLetGoOfAndCopiedAgainIsHeldAgain(t *testing.T) {
inv := fresh(t)
ctx := context.Background()
holding(t, inv, "proxy")
golang := mirrorRef("upstream/docker.io/library/golang", 1)
builtOn(t, inv, "b01", "proxy", 1, []string{golang}, golang)
if err := inv.MarkCollected(ctx, []string{golang}); err != nil {
t.Fatal(err)
}
if s := stateOf(t, inv, golang); s.State != ArtifactCollected {
t.Fatalf("a let-go copy reads %+v", s)
}
builtOn(t, inv, "b02", "proxy", 2, []string{golang}, golang)
if s := stateOf(t, inv, golang); s.State != ArtifactKept || !slices.Equal(s.Why, []string{KeptStoodOn}) {
t.Fatalf("a copy copied again reads %+v; want kept again", s)
}
// A person's record never undoes a collection: only a build that copied it says it is there.
if err := inv.MarkCollected(ctx, []string{golang}); err != nil {
t.Fatal(err)
}
if err := inv.RecordMirrored(ctx, "", "again", []string{golang}); err != nil {
t.Fatal(err)
}
if s := stateOf(t, inv, golang); s.State != ArtifactCollected {
t.Fatalf("a person's record brought a collected copy back: %+v", s)
}
}
+9
View File
@@ -143,6 +143,15 @@ type PlanGate struct {
Carried []CarriedMove `json:"carried,omitempty"`
// Failing names the modules the last judging found wanting.
Failing []string `json:"failing,omitempty"`
// Healthy counts, per module, the consecutive judgings that found it healthy on every machine judged,
// or waiting for a person (novox/hq ADR 0254): each module's passes, apart from the send's.
Healthy map[string]int `json:"healthy,omitempty"`
// Waits is, per module, the wait for a person the newest judging read (ADR 0254): carried along, never
// a reason to fail.
Waits map[string]string `json:"waits,omitempty"`
// Passing names the modules that passed on their own when the send failed (ADR 0254): each keeps its
// pass, and is not put back.
Passing []string `json:"passing,omitempty"`
}
// CarriedMove is one module's build moving on a machine with a gated send.
+4
View File
@@ -215,6 +215,10 @@ type BuildResult struct {
// (novox/hq ADR 0009). The catalogue turns these into edges; nothing else need care.
Against []string `json:"against,omitempty"`
// Mirrored is every base this build copied into the artifact store, as pinned (novox/hq ADR
// 0257). Carried on a failed build too: the copy is in the store whether the build worked or not.
Mirrored []string `json:"mirrored,omitempty"`
// Read is every repository this build read source from besides the module's own. A module whose
// recipe packages source that lives elsewhere is affected when that repository moves, and the
// manifest the mesh keeps says nothing about it (novox/hq 04-ISSUES/131).
+13
View File
@@ -441,6 +441,15 @@ type Health struct {
// file it writes, a service whose lifecycle is the machine's — said unhealthy while it stays failed.
const KindUnit = "unit"
// KindAccount is an account a module puts in a group (novox/hq ADR 0252): healthy when the account's running
// session has every group the module declares, or nobody is logged in; unhealthy otherwise, its reason
// starting ReasonRelogin when only a new login is missing.
const KindAccount = "account"
// ReasonRelogin starts the reason of an account whose running session began before it was put in a group
// (ADR 0252): the build did what it should, and a person has one step left (novox/hq ADR 0254).
const ReasonRelogin = "relogin needed"
// The states of a machine's service managers (issue 315).
const (
UnitsRunning = "running"
@@ -529,6 +538,10 @@ type ResourceHealth struct {
// alone; Needs is the provision the check exercises (to-be 48 §6).
Check string `json:"check,omitempty"`
Needs string `json:"needs,omitempty"`
// Account is the account whose own service manager runs it, for a unit or service in an account's
// manager, and the account itself for a resource of kind KindAccount (novox/hq ADR 0254). Empty from an
// engine older than that, and for anything the machine's own manager or runtime runs.
Account string `json:"account,omitempty"`
}
// HealthSaid is the health event's body: the machine and its statement. The machine is read from the
+2 -1
View File
@@ -75,7 +75,8 @@
"build",
"artifacts",
"collect",
"images"
"images",
"mirrors"
],
"resources": [
{