Merge pull request 'artifact-store-tools: say what the artifact store holds and what the records name (hq ADR 0251)' (#123) from feat/store-tools into main

modules/artifact-store-tools/README.md

@@ -0,0 +1,76 @@
+# artifact-store-tools
+
+Tools that say what the mesh's artifact store holds, and what the controller's records say of it
+(novox/hq ADR 0251 §1–3, to-be 51). It declares no resources and changes nothing on its machine.
+
+## Why a module beside the store
+
+The store is the `distribution` module. A module that provides the artifact store cannot also build an
+artifact: building publishes to the store, so the store would be needed to create itself, and the
+controller's module check refuses that manifest. So the store's tools are a second module, as the
+controller's refusal says to do. **Assign it to the machine that holds the store**: its tools read the
+store's files through the store's own container, and anywhere else they answer that the container is not
+on that machine.
+
+## Tools
+
+| tool | | what |
+|---|---|---|
+| `artifact_store_repositories` | r | every repository: tags and what each names, how many manifests (tagged or not), size |
+| `artifact_store_usage` | r | the store's size, the largest repositories, bytes shared between repositories, bytes no manifest marks (what the nightly collector frees next) |
+| `artifact_store_references` | r | what the controller's records say of each manifest, counted and sized by state and repository; each manifest listed when one repository is asked; what the records keep that the store does not hold |
+| `artifact_store_collect` | a | a dry run unless `dry_run` is false: what the controller would let go of, and the bytes the nightly collector would free then and now. A real run needs `why` and asks the controller's `collect` |
+
+### How the store is read
+
+The store's door lists repositories and tags, but not a manifest no tag names, and the mesh pins every
+machine by digest, so that is most of them. So the bundle reads the store's own files through its own
+container (`docker exec mesh-registry`, busybox `find`, `stat` and `cat`), and only reads them: every
+blob with its size, every manifest each repository holds, what each tag names, and each manifest's
+content, four hundred to an exec. What it read it keeps (a manifest is named by its content, so it never
+changes), and the next call reads only what is new. A manifest that could not be read is counted and
+said; what it marks beyond itself is then unknown, so sizes may read low and freed bytes high, and the
+answer says so.
+
+docker runs as the tool runner's account; a socket that refuses it is asked again through `sudo -n`,
+never with a prompt, as the container runtime's own tools do.
+
+A manifest *marks* its own content, its configuration and its layers, and an index marks the manifests
+it lists. The store's collector removes every blob no manifest marks, so these numbers are its own.
+
+### The states of a manifest
+
+| state | what | removable |
+|---|---|---|
+| `kept` | a definition names it, or one of the five most recent builds of a module the mesh holds; `why` says which | no |
+| `holder-of-kept-archive` | the manifest that keeps a kept archive's blob (hq issue 253) | no |
+| `eligible` | the mesh made it and keeps it for no reason | through the controller's `collect` |
+| `holder-of-eligible-archive` | the manifest that keeps an eligible archive's blob | with its archive |
+| `let-go-yet-present` | the controller recorded letting go of it, and the store still holds it | no — a finding |
+| `named-document` | a one-layer manifest the controller keeps under a tag (hq to-be 45 §9) | no |
+| `unrecorded` | no record names it | **never**, by any tool (ADR 0189 §3) |
+
+The records come from the controller's `artifacts` verb, asked with the references already let go of;
+when that answer is too large to carry, it is asked without them and the answer says so.
+
+### Collecting
+
+`artifact_store_collect` never deletes anything itself. The controller decides and records what it
+lets go of (ADR 0189 §2), so a real run asks its `collect` with `why` and `confirm`; the controller holds
+every kept archive first and records the run as a hand-act. Bytes come back at the store's nightly
+collection, which runs with the store held still.
+
+## Tests
+
+```
+go test ./...
+```
+
+Against a fake container that prints what the two scripts would: the listing parsed (a repository name
+with slashes, a cut listing refused), manifests read with one absent or unreadable said as unread, a
+digest never becoming a path outside the blobs, the mark (an index, a shared blob, a blob nothing marks),
+every state, what the records keep that the store lacks, bytes freed counting a blob shared with a kept
+manifest as kept, a dry run never confirming, a real run without `why` refused before anything is asked,
+the fall-back when the references let go of cannot be carried, escalation through `sudo -n`, and that the
+tools served are exactly the manifest's `tools` and its `invokes` exactly the two controller verbs.
+Any command other than the two read-only scripts fails the test.

modules/artifact-store-tools/cmd/artifact-store-tools/classify.go

@@ -0,0 +1,203 @@
+package main
+
+import (
+	"sort"
+)
+
+// The states a manifest the store holds can be in, as the records see it (novox/hq to-be 51).
+const (
+	StateKept              = "kept"
+	StateHolderKept        = "holder-of-kept-archive"
+	StateEligible          = "eligible"
+	StateHolderEligible    = "holder-of-eligible-archive"
+	StateCollectedPresent  = "let-go-yet-present"
+	StateNamedDocument     = "named-document"
+	StateUnrecorded        = "unrecorded"
+	collectionRemovesThese = "eligible and holder-of-eligible-archive, through the controller's collect; nothing else"
+)
+
+// Entry is one manifest the store holds, and what the records say of it.
+type Entry struct {
+	Repository string   `json:"repository"`
+	Digest     string   `json:"digest"`
+	State      string   `json:"state"`
+	Why        []string `json:"why,omitempty"`
+	// Archive is the archive a holder keeps.
+	Archive string   `json:"archive,omitempty"`
+	Tags    []string `json:"tags,omitempty"`
+	// Bytes are the blobs it marks, its own content included.
+	Bytes int64 `json:"bytes"`
+	// Unread says its content could not be read, so what it marks is not known beyond itself.
+	Unread string `json:"unread,omitempty"`
+}
+
+// recordIndex finds a record by repository and digest.
+type recordIndex map[string]*Record
+
+func indexRecords(r *Records) recordIndex {
+	idx := recordIndex{}
+	for i := range r.References {
+		rec := &r.References[i]
+		if rec.Repository == "" || rec.Digest == "" {
+			continue
+		}
+		idx[rec.Kind+" "+Key(rec.Repository, rec.Digest)] = rec
+	}
+	return idx
+}
+
+// Classify gives every manifest the store holds one state.
+func Classify(v *View, recs *Records) []Entry {
+	idx := indexRecords(recs)
+	var out []Entry
+	for _, name := range v.L.RepoNames() {
+		r := v.L.Repos[name]
+		tagsAt := map[string][]string{}
+		for t, d := range r.Tags {
+			tagsAt[d] = append(tagsAt[d], t)
+		}
+		digests := make([]string, 0, len(r.Revisions))
+		for d := range r.Revisions {
+			digests = append(digests, d)
+		}
+		sort.Strings(digests)
+		for _, d := range digests {
+			e := Entry{Repository: name, Digest: d, State: StateUnrecorded, Tags: tagsAt[d]}
+			sort.Strings(e.Tags)
+			blobs := map[string]bool{}
+			for _, b := range v.Marks(name, d) {
+				blobs[b] = true
+			}
+			e.Bytes = v.Bytes(blobs)
+			e.Unread = v.Unread[Key(name, d)]
+			m := v.M[Key(name, d)]
+			if rec := idx["image "+Key(name, d)]; rec != nil {
+				e.State, e.Why = stateOf(rec, false), rec.Why
+			} else if m.Holder() {
+				archive := m.Layers[0]
+				if rec := idx["archive "+Key(name, archive)]; rec != nil {
+					e.State, e.Why, e.Archive = stateOf(rec, true), rec.Why, archive
+				} else if len(e.Tags) > 0 {
+					e.State = StateNamedDocument
+				}
+			}
+			out = append(out, e)
+		}
+	}
+	return out
+}
+
+func stateOf(rec *Record, holder bool) string {
+	switch rec.State {
+	case "kept":
+		if holder {
+			return StateHolderKept
+		}
+		return StateKept
+	case "eligible":
+		if holder {
+			return StateHolderEligible
+		}
+		return StateEligible
+	case "collected":
+		return StateCollectedPresent
+	}
+	// A state this bundle does not know is kept: never offered as removable.
+	return StateKept
+}
+
+// Missing are the references the records keep that the store does not hold: an image whose manifest
+// is not in its repository, an archive whose blob is not in the store.
+func Missing(v *View, recs *Records) []string {
+	var out []string
+	for _, rec := range recs.References {
+		if rec.State != "kept" || rec.Repository == "" || rec.Digest == "" {
+			continue
+		}
+		switch rec.Kind {
+		case "image":
+			if r := v.L.Repos[rec.Repository]; r == nil || !r.Revisions[rec.Digest] {
+				out = append(out, rec.Reference)
+			}
+		case "archive":
+			if _, ok := v.L.Blobs[rec.Digest]; !ok {
+				out = append(out, rec.Reference)
+			}
+		}
+	}
+	sort.Strings(out)
+	return out
+}
+
+// StateSum is the manifests of one state: how many, the bytes they mark together, and the bytes only
+// they mark — what the store would give back if they alone went.
+type StateSum struct {
+	Manifests      int   `json:"manifests"`
+	Bytes          int64 `json:"bytes"`
+	OnlyTheirBytes int64 `json:"only_their_bytes"`
+}
+
+// Summarise sums entries by state, over the whole store or one repository's entries, the "only theirs"
+// always judged against every manifest in the store.
+func Summarise(v *View, entries []Entry, all []Entry) map[string]*StateSum {
+	statesOf := map[string]map[string]bool{}
+	for _, e := range all {
+		for _, b := range v.Marks(e.Repository, e.Digest) {
+			if statesOf[b] == nil {
+				statesOf[b] = map[string]bool{}
+			}
+			statesOf[b][e.State] = true
+		}
+	}
+	sums := map[string]*StateSum{}
+	blobs := map[string]map[string]bool{}
+	for _, e := range entries {
+		s := sums[e.State]
+		if s == nil {
+			s = &StateSum{}
+			sums[e.State] = s
+			blobs[e.State] = map[string]bool{}
+		}
+		s.Manifests++
+		for _, b := range v.Marks(e.Repository, e.Digest) {
+			blobs[e.State][b] = true
+		}
+	}
+	for state, set := range blobs {
+		sums[state].Bytes = v.Bytes(set)
+		only := map[string]bool{}
+		for b := range set {
+			if len(statesOf[b]) == 1 {
+				only[b] = true
+			}
+		}
+		sums[state].OnlyTheirBytes = v.Bytes(only)
+	}
+	return sums
+}
+
+// LetGoKeys are the manifests the store would no longer hold once the controller lets go of these
+// references: an image's manifest, and an archive's holder.
+func LetGoKeys(v *View, refs []string) map[string]bool {
+	out := map[string]bool{}
+	for _, ref := range refs {
+		repo, digest, kind := splitReference(ref)
+		r := v.L.Repos[repo]
+		if r == nil {
+			continue
+		}
+		switch kind {
+		case "image":
+			if r.Revisions[digest] {
+				out[Key(repo, digest)] = true
+			}
+		case "archive":
+			for d := range r.Revisions {
+				if m := v.M[Key(repo, d)]; m.Holder() && m.Layers[0] == digest {
+					out[Key(repo, d)] = true
+				}
+			}
+		}
+	}
+	return out
+}

modules/artifact-store-tools/cmd/artifact-store-tools/main.go

@@ -0,0 +1,31 @@
+// The artifact-store-tools module's Go bundle (novox/hq ADR 0251 §1–3, to-be 51): a process the node's
+// runtime launches and speaks MCP over stdio to. It says what the artifact store holds — its
+// repositories, its size, and what the controller's records say of each manifest — read from the
+// store's own files through the store's own container, and writing nothing. A collection is asked of
+// the controller, which decides and records what it lets go of (ADR 0189).
+//
+// **A module beside the store, not the store's own.** A module that provides the artifact store cannot
+// also build an artifact: building publishes to the store, so the store would be needed to create
+// itself, and the controller refuses that manifest. So these tools are a second module, assigned to the
+// machine that holds the store. stdout is the protocol; what this bundle says, it says on stderr.
+package main
+
+import (
+	"fmt"
+	"os"
+
+	stdio "git.novox.be/novox/mesh-sdk/go"
+)
+
+func main() {
+	s := &Store{
+		Container: os.Getenv("MESH_ARTIFACT_STORE_CONTAINER"),
+		Run:       ExecRunner,
+		UID:       os.Getuid(),
+	}
+	// An empty name serves as the module the runtime names (MESH_SERVED_MODULE): artifact-store-tools.
+	if err := stdio.Serve("", Tools(s, Controller{Ask: stdio.Ask})); err != nil {
+		fmt.Fprintln(os.Stderr, err)
+		os.Exit(1)
+	}
+}

modules/artifact-store-tools/cmd/artifact-store-tools/read.go

@@ -0,0 +1,175 @@
+package main
+
+import (
+	"context"
+	"fmt"
+	"sort"
+	"strings"
+	"sync"
+)
+
+// readScript prints each manifest named on its command line, each after a line naming its path. A
+// manifest is JSON, which holds no raw line break inside a string, so no line of one starts with the
+// marker. One the store does not have is said, not skipped.
+const readScript = `cd ` + storageRoot + `
+for p in "$@"; do
+  echo "#manifest $p"
+  if [ -f "$p" ]; then cat "$p"; echo; else echo "#absent"; fi
+done
+echo '#end'`
+
+// readBatch is how many manifests one exec reads: well under any command line's limit.
+const readBatch = 400
+
+// Store reaches the artifact store's own files, through its own container, and only reads them.
+type Store struct {
+	Container string
+	Run       Runner
+	UID       int
+
+	mu sync.Mutex
+	// cache holds every manifest read: a manifest is named by its content's digest, so it never changes.
+	cache map[string]*Manifest
+}
+
+// List reads the store's files.
+func (s *Store) List(ctx context.Context) (*Listing, error) {
+	if s.Container == "" {
+		return nil, fmt.Errorf("this bundle was not told the artifact store's container (MESH_ARTIFACT_STORE_CONTAINER)")
+	}
+	out, err := docker(ctx, s.Run, s.UID, "exec", s.Container, "sh", "-c", listScript)
+	if err != nil {
+		return nil, err
+	}
+	return ParseListing(out)
+}
+
+// Key names one manifest in one repository.
+func Key(repo, digest string) string { return repo + "@" + digest }
+
+// blobPath is where the store keeps a blob, under storageRoot.
+func blobPath(digest string) string {
+	algorithm, hex, _ := strings.Cut(digest, ":")
+	if algorithm == "" || len(hex) < 2 || strings.ContainsAny(digest, "/ \n") {
+		return ""
+	}
+	return "blobs/" + algorithm + "/" + hex[:2] + "/" + hex + "/data"
+}
+
+// Manifests reads every manifest the listing names, from the cache or the store's files. Answers each
+// one read, and each one not read with why.
+func (s *Store) Manifests(ctx context.Context, l *Listing) (map[string]*Manifest, map[string]string) {
+	need := map[string]bool{}
+	s.mu.Lock()
+	if s.cache == nil {
+		s.cache = map[string]*Manifest{}
+	}
+	for _, name := range l.RepoNames() {
+		for d := range l.Repos[name].Revisions {
+			if _, ok := s.cache[d]; !ok {
+				need[d] = true
+			}
+		}
+	}
+	s.mu.Unlock()
+
+	digests := make([]string, 0, len(need))
+	for d := range need {
+		digests = append(digests, d)
+	}
+	sort.Strings(digests)
+	failed := map[string]string{}
+	for start := 0; start < len(digests); start += readBatch {
+		batch := digests[start:min(start+readBatch, len(digests))]
+		read, err := s.read(ctx, batch)
+		s.mu.Lock()
+		for _, d := range batch {
+			switch m := read[d]; {
+			case err != nil:
+				failed[d] = err.Error()
+			case m == nil:
+				failed[d] = "the store holds no manifest content for it"
+			default:
+				s.cache[d] = m
+			}
+		}
+		s.mu.Unlock()
+	}
+
+	got := map[string]*Manifest{}
+	unread := map[string]string{}
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	for _, name := range l.RepoNames() {
+		for d := range l.Repos[name].Revisions {
+			if m, ok := s.cache[d]; ok {
+				got[Key(name, d)] = m
+			} else if why, ok := failed[d]; ok {
+				unread[Key(name, d)] = why
+			} else {
+				unread[Key(name, d)] = "not read"
+			}
+		}
+	}
+	return got, unread
+}
+
+// read reads one batch of manifests. A blob that is not a manifest, or not there, is answered as nil;
+// an exec that fails fails the batch.
+func (s *Store) read(ctx context.Context, digests []string) (map[string]*Manifest, error) {
+	args := []string{"exec", s.Container, "sh", "-c", readScript, "sh"}
+	byPath := map[string]string{}
+	for _, d := range digests {
+		if p := blobPath(d); p != "" {
+			byPath[p] = d
+			args = append(args, p)
+		}
+	}
+	out, err := docker(ctx, s.Run, s.UID, args...)
+	if err != nil {
+		return nil, err
+	}
+	return parseManifests(out, byPath)
+}
+
+// parseManifests reads readScript's output.
+func parseManifests(out string, byPath map[string]string) (map[string]*Manifest, error) {
+	got := map[string]*Manifest{}
+	current := ""
+	var body strings.Builder
+	flush := func() {
+		if d := byPath[current]; d != "" {
+			m, err := ParseManifest([]byte(body.String()))
+			if err != nil {
+				m = nil
+			}
+			got[d] = m
+		}
+		current = ""
+		body.Reset()
+	}
+	ended := false
+	for _, line := range strings.Split(out, "\n") {
+		switch {
+		case strings.HasPrefix(line, "#manifest "):
+			flush()
+			current = strings.TrimPrefix(line, "#manifest ")
+		case line == "#absent":
+			if d := byPath[current]; d != "" {
+				got[d] = nil
+			}
+			current = ""
+			body.Reset()
+		case line == "#end":
+			flush()
+			ended = true
+		default:
+			body.WriteString(line)
+			body.WriteString("\n")
+		}
+	}
+	if !ended {
+		return nil, fmt.Errorf("reading the store's manifests was cut short (it never reached its end)")
+	}
+	return got, nil
+}

modules/artifact-store-tools/cmd/artifact-store-tools/read_test.go

@@ -0,0 +1,38 @@
+package main
+
+import "testing"
+
+func TestManifestsAreReadFromTheStoresFilesAndAMissingOneIsSaid(t *testing.T) {
+	got, err := parseManifests("#manifest blobs/sha256/aa/aa1/data\n"+image(d("cfg"), d("l"))+"\n\n#manifest blobs/sha256/bb/bb1/data\n#absent\n#manifest blobs/sha256/cc/cc1/data\nnot json\n#end\n",
+		map[string]string{"blobs/sha256/aa/aa1/data": "sha256:aa1", "blobs/sha256/bb/bb1/data": "sha256:bb1", "blobs/sha256/cc/cc1/data": "sha256:cc1"})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if m := got["sha256:aa1"]; m == nil || len(m.Layers) != 1 || m.Layers[0] != d("l") {
+		t.Errorf("aa1 %+v", m)
+	}
+	if got["sha256:bb1"] != nil || got["sha256:cc1"] != nil {
+		t.Error("an absent or unreadable manifest was read as one")
+	}
+	if _, err := parseManifests("#manifest x\n{}\n", map[string]string{"x": "sha256:x"}); err == nil {
+		t.Error("a cut read was accepted")
+	}
+
+	f := world()
+	delete(f.manifests, d("m-old"))
+	v := view(t, f)
+	if v.Unread[Key("app/server", d("m-old"))] == "" {
+		t.Error("a manifest whose content is gone was not said as unread")
+	}
+	if out := Usage(v, 3); out["unread"] == nil {
+		t.Error("the answer did not say a manifest was unread")
+	}
+}
+
+func TestADigestNeverBecomesAPathOutsideTheBlobs(t *testing.T) {
+	for _, bad := range []string{"sha256:", "nocolon", "sha256:a/../../etc", "sha256:ab cd"} {
+		if p := blobPath(bad); p != "" {
+			t.Errorf("%q became %q", bad, p)
+		}
+	}
+}

modules/artifact-store-tools/cmd/artifact-store-tools/records.go

@@ -0,0 +1,209 @@
+package main
+
+import (
+	"encoding/json"
+	"errors"
+	"fmt"
+	"strconv"
+	"strings"
+)
+
+// What the records say, asked of the controller and never copied (novox/hq ADR 0251 §2). The
+// controller decides what the mesh keeps; this bundle only sets that beside what the store holds.
+
+// Asking is how a tool on the mesh is asked: stdio.Ask, or a test's.
+type Asking func(key string, body any) (json.RawMessage, error)
+
+// ControllerSeat is the seat whose verbs the controller serves.
+const ControllerSeat = "mesh-controller"
+
+// Record is one reference the mesh recorded making, as the controller's `artifacts` answers it.
+type Record struct {
+	Reference  string   `json:"reference"`
+	Repository string   `json:"repository,omitempty"`
+	Digest     string   `json:"digest,omitempty"`
+	Kind       string   `json:"kind,omitempty"`
+	State      string   `json:"state"`
+	Why        []string `json:"why,omitempty"`
+}
+
+// Records is the controller's `artifacts` answer.
+type Records struct {
+	Store      string         `json:"store"`
+	KeptBuilds int            `json:"kept_builds"`
+	Counts     map[string]int `json:"counts"`
+	References []Record       `json:"references"`
+	// CollectedAsked is whether the references already let go of were asked for too.
+	CollectedAsked bool `json:"-"`
+	// Note says what could not be asked, when something could not.
+	Note string `json:"-"`
+}
+
+// CollectAnswer is the controller's `collect` answer.
+type CollectAnswer struct {
+	DryRun         bool     `json:"dry_run"`
+	Store          string   `json:"store"`
+	KeptArchives   int      `json:"kept_archives"`
+	Held           int      `json:"held"`
+	HoldersWritten int      `json:"holders_written"`
+	Missing        int      `json:"missing"`
+	Eligible       int      `json:"eligible"`
+	WouldLetGo     []string `json:"would_let_go"`
+	LetGo          []string `json:"let_go"`
+	Skipped        int      `json:"skipped"`
+	Left           int      `json:"left"`
+	Stopped        string   `json:"stopped"`
+	// Rest is whatever else the controller said, passed on as it said it.
+	Rest map[string]json.RawMessage `json:"-"`
+}
+
+// Controller asks the controller's seat through the runtime.
+type Controller struct{ Ask Asking }
+
+// answerOf reads a verb's answer whatever wraps it: the controller's {output, ok, answer}, a text the
+// runtime handed over, or the protocol's content list.
+func answerOf(raw json.RawMessage) (json.RawMessage, string, bool) {
+	var s string
+	if json.Unmarshal(raw, &s) == nil {
+		return answerOf(json.RawMessage(s))
+	}
+	var m map[string]json.RawMessage
+	if json.Unmarshal(raw, &m) != nil {
+		return raw, string(raw), true
+	}
+	if content, has := m["content"]; has {
+		var items []struct {
+			Text string `json:"text"`
+		}
+		var e struct {
+			IsError bool `json:"isError"`
+		}
+		_ = json.Unmarshal(raw, &e)
+		if json.Unmarshal(content, &items) == nil && len(items) > 0 {
+			inner, out, ok := answerOf(json.RawMessage(items[0].Text))
+			return inner, out, ok && !e.IsError
+		}
+	}
+	var v struct {
+		Output string          `json:"output"`
+		OK     *bool           `json:"ok"`
+		Answer json.RawMessage `json:"answer"`
+	}
+	if json.Unmarshal(raw, &v) == nil && v.OK != nil {
+		return v.Answer, v.Output, *v.OK
+	}
+	return raw, string(raw), true
+}
+
+func (c Controller) verb(name string, args map[string]any) (json.RawMessage, error) {
+	if c.Ask == nil {
+		return nil, errors.New("this bundle has no way to ask the controller")
+	}
+	raw, err := c.Ask("seat:"+ControllerSeat+"."+name, args)
+	if err != nil {
+		return nil, fmt.Errorf("the controller's %s could not be asked: %w", name, err)
+	}
+	answer, output, ok := answerOf(raw)
+	if !ok {
+		return nil, fmt.Errorf("the controller refused %s: %s", name, lastLine(output))
+	}
+	if len(answer) == 0 || string(answer) == "null" {
+		return nil, fmt.Errorf("the controller's %s answered no data: %s", name, lastLine(output))
+	}
+	return answer, nil
+}
+
+func lastLine(s string) string {
+	lines := strings.Split(strings.TrimSpace(s), "\n")
+	return strings.TrimSpace(lines[len(lines)-1])
+}
+
+// Artifacts asks what the records say, with the references already let go of when the answer can carry
+// them; when it cannot, without, and says so.
+func (c Controller) Artifacts(repository string) (*Records, error) {
+	args := map[string]any{"collected": "true"}
+	if repository != "" {
+		args["repository"] = repository
+	}
+	raw, err := c.verb("artifacts", args)
+	collected := true
+	note := ""
+	if err != nil {
+		delete(args, "collected")
+		var again error
+		raw, again = c.verb("artifacts", args)
+		if again != nil {
+			return nil, again
+		}
+		collected = false
+		note = "the references the mesh already let go of could not be asked for (" + err.Error() +
+			"), so a manifest the mesh let go of that the store still holds is counted as unrecorded"
+	}
+	var r Records
+	if err := json.Unmarshal(raw, &r); err != nil {
+		return nil, fmt.Errorf("the controller's artifacts answer is not readable: %v", err)
+	}
+	for i := range r.References {
+		fill(&r.References[i])
+	}
+	r.CollectedAsked, r.Note = collected, note
+	return &r, nil
+}
+
+// Collect asks the controller's sweep: a dry run unless confirm, which needs why.
+func (c Controller) Collect(why string, confirm bool, most int) (*CollectAnswer, error) {
+	args := map[string]any{}
+	if confirm {
+		if strings.TrimSpace(why) == "" {
+			return nil, errors.New("a real collection needs why: nothing was asked")
+		}
+		args["why"] = why
+		args["confirm"] = "true"
+	}
+	if most > 0 {
+		args["most"] = strconv.Itoa(most)
+	}
+	raw, err := c.verb("collect", args)
+	if err != nil {
+		return nil, err
+	}
+	var a CollectAnswer
+	if err := json.Unmarshal(raw, &a); err != nil {
+		return nil, fmt.Errorf("the controller's collect answer is not readable: %v", err)
+	}
+	_ = json.Unmarshal(raw, &a.Rest)
+	if !confirm && !a.DryRun && len(a.LetGo) > 0 {
+		return nil, fmt.Errorf("the controller let go of %d artifacts when only a dry run was asked", len(a.LetGo))
+	}
+	return &a, nil
+}
+
+// fill reads the repository, digest and kind out of a reference when the answer left them out.
+func fill(r *Record) {
+	repo, digest, kind := splitReference(r.Reference)
+	if r.Repository == "" {
+		r.Repository = repo
+	}
+	if r.Digest == "" {
+		r.Digest = digest
+	}
+	if r.Kind == "" {
+		r.Kind = kind
+	}
+}
+
+// splitReference reads `artifact-store://<repo>@sha256:…` (an image) and
+// `artifact-store://<repo>/blobs/sha256:…` (an archive).
+func splitReference(ref string) (repo, digest, kind string) {
+	path := ref
+	if _, after, ok := strings.Cut(ref, "://"); ok {
+		path = after
+	}
+	if before, after, ok := strings.Cut(path, "/blobs/sha256:"); ok {
+		return before, "sha256:" + after, "archive"
+	}
+	if before, after, ok := strings.Cut(path, "@sha256:"); ok {
+		return before, "sha256:" + after, "image"
+	}
+	return "", "", ""
+}

modules/artifact-store-tools/cmd/artifact-store-tools/runner.go

@@ -0,0 +1,96 @@
+package main
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"fmt"
+	"os/exec"
+	"regexp"
+	"strings"
+	"time"
+)
+
+// Ran is what a command did: its output, its exit status, and why it never ran to an answer.
+type Ran struct {
+	Stdout string
+	Stderr string
+	Status int
+	// Err is "ENOENT" when the program is not installed, or says it was ended for taking too long.
+	Err string
+}
+
+// Runner runs one command, so every tool can be tested without a daemon.
+type Runner func(ctx context.Context, name string, args ...string) Ran
+
+// ListTimeout is how long listing the store's files may take: below the runtime's thirty-second call
+// limit, so a store that hangs is answered as such rather than as a call the runtime gave up on.
+const ListTimeout = 15 * time.Second
+
+// ExecRunner runs a command on this machine, bounded by ListTimeout.
+func ExecRunner(ctx context.Context, name string, args ...string) Ran {
+	ctx, cancel := context.WithTimeout(ctx, ListTimeout)
+	defer cancel()
+	cmd := exec.CommandContext(ctx, name, args...)
+	var out, errb bytes.Buffer
+	cmd.Stdout, cmd.Stderr = &out, &errb
+	err := cmd.Run()
+	r := Ran{Stdout: out.String(), Stderr: errb.String()}
+	var exitErr *exec.ExitError
+	switch {
+	case errors.Is(ctx.Err(), context.DeadlineExceeded):
+		r.Status, r.Err = 124, fmt.Sprintf("no answer within %d s", int(ListTimeout/time.Second))
+	case errors.Is(err, exec.ErrNotFound):
+		r.Status, r.Err = 127, "ENOENT"
+	case errors.As(err, &exitErr):
+		r.Status = exitErr.ExitCode()
+	case err != nil:
+		r.Status, r.Err = 1, err.Error()
+	}
+	return r
+}
+
+var socketRefused = regexp.MustCompile(`(?i)permission denied.*docker.*sock|docker\.sock.*permission denied`)
+
+// docker runs one docker command as this account and answers its stdout. A socket that refuses the
+// account is asked again through `sudo -n`, as the container runtime's own tools do; never as root
+// otherwise, and never with a prompt.
+func docker(ctx context.Context, run Runner, uid int, args ...string) (string, error) {
+	r := run(ctx, "docker", args...)
+	program := "docker"
+	if r.Status != 0 && r.Err == "" && uid != 0 && socketRefused.MatchString(r.Stderr) {
+		program = "sudo"
+		r = run(ctx, "sudo", append([]string{"-n", "docker"}, args...)...)
+	}
+	if r.Status == 0 && r.Err == "" {
+		return r.Stdout, nil
+	}
+	said := strings.TrimSpace(r.Stderr + "\n" + r.Stdout)
+	switch {
+	case r.Err == "ENOENT" && program == "sudo":
+		return "", errors.New("the runtime's socket refused this account, and sudo is not installed here to escalate with")
+	case r.Err == "ENOENT":
+		return "", errors.New("docker is not installed on this machine, so the store's files cannot be listed")
+	case r.Err != "":
+		return "", fmt.Errorf("listing the store's files did not answer: %s", r.Err)
+	case program == "sudo" && strings.HasPrefix(said, "sudo:"):
+		return "", fmt.Errorf("the runtime's socket refused this account and it may not escalate without a prompt: %s", firstLine(said))
+	case strings.Contains(said, "No such container"):
+		return "", fmt.Errorf("the store's container is not on this machine: %s", firstLine(said))
+	case strings.Contains(said, "is not running"):
+		return "", fmt.Errorf("the store's container is not running: %s", firstLine(said))
+	}
+	if l := firstLine(said); l != "" {
+		return "", fmt.Errorf("listing the store's files failed (%d): %s", r.Status, l)
+	}
+	return "", fmt.Errorf("listing the store's files failed with status %d", r.Status)
+}
+
+func firstLine(s string) string {
+	for _, l := range strings.Split(s, "\n") {
+		if l = strings.TrimSpace(l); l != "" {
+			return l
+		}
+	}
+	return ""
+}

modules/artifact-store-tools/cmd/artifact-store-tools/store.go

@@ -0,0 +1,180 @@
+package main
+
+import (
+	"encoding/json"
+	"fmt"
+	"sort"
+	"strconv"
+	"strings"
+)
+
+// What the artifact store holds, read from its own files, and nothing written to them.
+//
+// **Why its files.** The store's door lists repositories and the tags in each, but not a manifest no tag
+// names — and since the mesh pins every machine by digest, that is almost every manifest the store
+// holds. The files list them all, with every blob's size, and hold every manifest's content. They are
+// read through the store's own container, which is the only process that has them, and only read
+// (novox/hq ADR 0251 §1). So this module runs on the machine that holds the store; anywhere else its
+// tools say the container is not there.
+
+// storageRoot is where the registry keeps its files, inside its container.
+const storageRoot = "/var/lib/registry/docker/registry/v2"
+
+// listScript lists, in three sections, every blob with its size, every manifest each repository holds,
+// and what each tag names now. Busybox's find and stat (the store's image is Alpine): no -printf. An
+// upload in progress is not listed, and nor are a tag's former targets.
+const listScript = `set -e
+cd ` + storageRoot + `
+echo '#blobs'
+if [ -d blobs ]; then find blobs -type f -name data | xargs -r stat -c '%s %n'; fi
+echo '#revisions'
+if [ -d repositories ]; then find repositories -type f -path '*/_manifests/revisions/sha256/*/link'; fi
+echo '#tags'
+if [ -d repositories ]; then find repositories -type f -path '*/_manifests/tags/*/current/link' -exec grep -H . {} + || true; fi
+echo '#end'`
+
+// Repo is one repository as the store's files list it.
+type Repo struct {
+	Name string
+	// Revisions are the digests of every manifest it holds, tagged or not.
+	Revisions map[string]bool
+	// Tags are what each tag names now.
+	Tags map[string]string
+}
+
+// Listing is the store's files: every blob with its size, and every repository.
+type Listing struct {
+	Blobs map[string]int64
+	Repos map[string]*Repo
+}
+
+func (l *Listing) repo(name string) *Repo {
+	r := l.Repos[name]
+	if r == nil {
+		r = &Repo{Name: name, Revisions: map[string]bool{}, Tags: map[string]string{}}
+		l.Repos[name] = r
+	}
+	return r
+}
+
+// RepoNames are the repositories in order.
+func (l *Listing) RepoNames() []string {
+	out := make([]string, 0, len(l.Repos))
+	for n := range l.Repos {
+		out = append(out, n)
+	}
+	sort.Strings(out)
+	return out
+}
+
+// ParseListing reads listScript's output. A section that never ended means the listing was cut short,
+// which is an error: a partial listing would make held things look absent.
+func ParseListing(out string) (*Listing, error) {
+	l := &Listing{Blobs: map[string]int64{}, Repos: map[string]*Repo{}}
+	section := ""
+	ended := false
+	for _, line := range strings.Split(out, "\n") {
+		line = strings.TrimSpace(line)
+		if line == "" {
+			continue
+		}
+		if strings.HasPrefix(line, "#") {
+			section = line
+			ended = line == "#end"
+			continue
+		}
+		switch section {
+		case "#blobs":
+			size, path, ok := strings.Cut(line, " ")
+			n, err := strconv.ParseInt(size, 10, 64)
+			if !ok || err != nil {
+				return nil, fmt.Errorf("the store's listing has a blob line it cannot read: %q", line)
+			}
+			// blobs/sha256/ab/<hex>/data
+			parts := strings.Split(strings.TrimPrefix(path, "./"), "/")
+			if len(parts) != 5 || parts[0] != "blobs" || parts[4] != "data" {
+				return nil, fmt.Errorf("the store's listing has a blob at an unexpected place: %q", path)
+			}
+			l.Blobs[parts[1]+":"+parts[3]] = n
+		case "#revisions":
+			// repositories/<repo>/_manifests/revisions/sha256/<hex>/link
+			name, rest, ok := strings.Cut(strings.TrimPrefix(strings.TrimPrefix(line, "./"), "repositories/"), "/_manifests/revisions/")
+			parts := strings.Split(rest, "/")
+			if !ok || len(parts) != 3 || parts[2] != "link" {
+				return nil, fmt.Errorf("the store's listing has a manifest at an unexpected place: %q", line)
+			}
+			l.repo(name).Revisions[parts[0]+":"+parts[1]] = true
+		case "#tags":
+			// repositories/<repo>/_manifests/tags/<tag>/current/link:sha256:<hex>
+			path, digest, ok := strings.Cut(line, ":sha256:")
+			name, rest, ok2 := strings.Cut(strings.TrimPrefix(strings.TrimPrefix(path, "./"), "repositories/"), "/_manifests/tags/")
+			tag, _, ok3 := strings.Cut(rest, "/current/link")
+			if !ok || !ok2 || !ok3 || tag == "" {
+				return nil, fmt.Errorf("the store's listing has a tag line it cannot read: %q", line)
+			}
+			l.repo(name).Tags[tag] = "sha256:" + digest
+		default:
+			return nil, fmt.Errorf("the store's listing has a line outside any section: %q", line)
+		}
+	}
+	if !ended {
+		return nil, fmt.Errorf("the store's listing was cut short (it never reached its end)")
+	}
+	return l, nil
+}
+
+// Manifest is what one manifest names.
+type Manifest struct {
+	MediaType   string
+	ConfigMedia string
+	// Refs are the blobs it names: its configuration and its layers.
+	Refs []string
+	// Layers are its layers alone, in order.
+	Layers []string
+	// Children are the manifests an index names.
+	Children []string
+}
+
+// Holder is the shape of a manifest that keeps one blob: an empty configuration and one layer. The
+// controller holds every archive the mesh keeps this way (novox/hq issue 253), and keeps a named
+// document the same way, under a tag.
+func (m *Manifest) Holder() bool {
+	return m != nil && m.ConfigMedia == "application/vnd.oci.empty.v1+json" && len(m.Layers) == 1
+}
+
+// ParseManifest reads a manifest's content, of any of the shapes a registry holds.
+func ParseManifest(raw []byte) (*Manifest, error) {
+	type desc struct {
+		MediaType string `json:"mediaType"`
+		Digest    string `json:"digest"`
+	}
+	var v struct {
+		MediaType string `json:"mediaType"`
+		Config    *desc  `json:"config"`
+		Layers    []desc `json:"layers"`
+		Manifests []desc `json:"manifests"`
+		FSLayers  []struct {
+			BlobSum string `json:"blobSum"`
+		} `json:"fsLayers"`
+	}
+	if err := json.Unmarshal(raw, &v); err != nil {
+		return nil, fmt.Errorf("not a manifest: %v", err)
+	}
+	m := &Manifest{MediaType: v.MediaType}
+	if v.Config != nil && v.Config.Digest != "" {
+		m.ConfigMedia = v.Config.MediaType
+		m.Refs = append(m.Refs, v.Config.Digest)
+	}
+	for _, d := range v.Layers {
+		m.Refs = append(m.Refs, d.Digest)
+		m.Layers = append(m.Layers, d.Digest)
+	}
+	for _, d := range v.FSLayers {
+		m.Refs = append(m.Refs, d.BlobSum)
+		m.Layers = append(m.Layers, d.BlobSum)
+	}
+	for _, d := range v.Manifests {
+		m.Children = append(m.Children, d.Digest)
+	}
+	return m, nil
+}

modules/artifact-store-tools/cmd/artifact-store-tools/store_test.go

@@ -0,0 +1,427 @@
+package main
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"os"
+	"reflect"
+	"sort"
+	"strings"
+	"sync"
+	"testing"
+)
+
+// d is a digest made from a short name, so a test reads as names.
+func d(name string) string {
+	return fmt.Sprintf("sha256:%064x", []byte(name))[:71]
+}
+
+func hexOf(digest string) string { return strings.TrimPrefix(digest, "sha256:") }
+
+// fakeStore is a store's files and its manifests: what listScript and readScript would print.
+type fakeStore struct {
+	blobs     map[string]int64
+	revisions map[string][]string // repo -> digests
+	tags      map[string]map[string]string
+	manifests map[string]string // digest -> content
+}
+
+func (f *fakeStore) listing() string {
+	var b strings.Builder
+	b.WriteString("#blobs\n")
+	for dg, n := range f.blobs {
+		fmt.Fprintf(&b, "%d blobs/sha256/%s/%s/data\n", n, hexOf(dg)[:2], hexOf(dg))
+	}
+	b.WriteString("#revisions\n")
+	for repo, ds := range f.revisions {
+		for _, dg := range ds {
+			fmt.Fprintf(&b, "repositories/%s/_manifests/revisions/sha256/%s/link\n", repo, hexOf(dg))
+		}
+	}
+	b.WriteString("#tags\n")
+	for repo, ts := range f.tags {
+		for t, dg := range ts {
+			fmt.Fprintf(&b, "repositories/%s/_manifests/tags/%s/current/link:%s\n", repo, t, dg)
+		}
+	}
+	b.WriteString("#end\n")
+	return b.String()
+}
+
+// run is the store's container as docker exec reaches it: the listing, or the manifests asked for, as
+// the two scripts print them. Anything else is a test failure: these tools only read.
+func (f *fakeStore) run(t *testing.T) Runner {
+	return func(_ context.Context, name string, args ...string) Ran {
+		if name != "docker" || len(args) < 5 || args[0] != "exec" || args[1] != "mesh-registry" || args[2] != "sh" || args[3] != "-c" {
+			t.Fatalf("ran %s %v", name, args)
+		}
+		switch args[4] {
+		case listScript:
+			return Ran{Stdout: f.listing()}
+		case readScript:
+			var b strings.Builder
+			for _, p := range args[6:] {
+				fmt.Fprintf(&b, "#manifest %s\n", p)
+				found := false
+				for dg, body := range f.manifests {
+					if blobPath(dg) == p {
+						b.WriteString(body + "\n")
+						found = true
+					}
+				}
+				if !found {
+					b.WriteString("#absent\n")
+				}
+			}
+			b.WriteString("#end\n")
+			return Ran{Stdout: b.String()}
+		}
+		t.Fatalf("ran a script these tools do not have: %q", args[4])
+		return Ran{}
+	}
+}
+
+func image(config string, layers ...string) string {
+	var ls []string
+	for _, l := range layers {
+		ls = append(ls, fmt.Sprintf(`{"mediaType":"application/vnd.oci.image.layer.v1.tar+gzip","digest":%q,"size":1}`, l))
+	}
+	return fmt.Sprintf(`{"schemaVersion":2,"mediaType":"application/vnd.oci.image.manifest.v1+json","config":{"mediaType":"application/vnd.oci.image.config.v1+json","digest":%q,"size":1},"layers":[%s]}`,
+		config, strings.Join(ls, ","))
+}
+
+func holder(blob string) string {
+	return fmt.Sprintf(`{"schemaVersion":2,"mediaType":"application/vnd.oci.image.manifest.v1+json","config":{"mediaType":"application/vnd.oci.empty.v1+json","digest":%q,"size":2},"layers":[{"mediaType":"application/vnd.oci.image.layer.v1.tar+gzip","digest":%q,"size":1}]}`,
+		d("empty"), blob)
+}
+
+func index(children ...string) string {
+	var cs []string
+	for _, c := range children {
+		cs = append(cs, fmt.Sprintf(`{"mediaType":"application/vnd.oci.image.manifest.v1+json","digest":%q,"size":1}`, c))
+	}
+	return fmt.Sprintf(`{"schemaVersion":2,"mediaType":"application/vnd.oci.image.index.v1+json","manifests":[%s]}`, strings.Join(cs, ","))
+}
+
+// world is one store with every state in it:
+//
+//	app/server: m-kept (definition), m-old (eligible), m-stranger (unrecorded, tagged), an index naming m-kept
+//	app/tools: h-kept holds a-kept (kept archive), h-old holds a-old (eligible archive)
+//	mesh/facts: doc (a named document under the tag latest)
+//	base layer "shared" is marked by m-kept and m-old; "old-only" only by m-old.
+func world() *fakeStore {
+	f := &fakeStore{
+		blobs:     map[string]int64{},
+		revisions: map[string][]string{},
+		tags:      map[string]map[string]string{},
+		manifests: map[string]string{},
+	}
+	put := func(repo, name, content string) {
+		f.manifests[d(name)] = content
+		f.blobs[d(name)] = int64(len(content))
+		f.revisions[repo] = append(f.revisions[repo], d(name))
+	}
+	for name, size := range map[string]int64{"cfg": 10, "shared": 1000, "old-only": 500, "stranger-layer": 300, "a-kept": 2000, "a-old": 4000, "empty": 2, "doc-body": 50, "orphan": 7000} {
+		f.blobs[d(name)] = size
+	}
+	put("app/server", "m-kept", image(d("cfg"), d("shared")))
+	put("app/server", "m-old", image(d("cfg"), d("shared"), d("old-only")))
+	put("app/server", "m-stranger", image(d("cfg"), d("stranger-layer")))
+	put("app/server", "idx", index(d("m-kept")))
+	put("app/tools", "h-kept", holder(d("a-kept")))
+	put("app/tools", "h-old", holder(d("a-old")))
+	put("mesh/facts", "doc", holder(d("doc-body")))
+	f.tags["app/server"] = map[string]string{"latest": d("m-stranger")}
+	f.tags["mesh/facts"] = map[string]string{"latest": d("doc")}
+	return f
+}
+
+func records() *Records {
+	return &Records{Store: "store:5000", KeptBuilds: 5, Counts: map[string]int{"kept": 3, "eligible": 2},
+		References: []Record{
+			{Reference: "artifact-store://app/server@" + d("m-kept"), State: "kept", Why: []string{"definition"}},
+			{Reference: "artifact-store://app/server@" + d("m-old"), State: "eligible"},
+			{Reference: "artifact-store://app/tools/blobs/" + d("a-kept"), State: "kept", Why: []string{"recent-build"}},
+			{Reference: "artifact-store://app/tools/blobs/" + d("a-old"), State: "eligible"},
+			{Reference: "artifact-store://app/server@" + d("m-gone"), State: "kept", Why: []string{"definition"}},
+		}}
+}
+
+func view(t *testing.T, f *fakeStore) *View {
+	t.Helper()
+	s := &Store{Container: "mesh-registry", UID: 1000, Run: f.run(t)}
+	v, err := s.View(context.Background())
+	if err != nil {
+		t.Fatal(err)
+	}
+	return v
+}
+
+func withFill(r *Records) *Records {
+	for i := range r.References {
+		fill(&r.References[i])
+	}
+	return r
+}
+
+func TestTheListingIsRead(t *testing.T) {
+	l, err := ParseListing("#blobs\n12 blobs/sha256/ab/abcd/data\n#revisions\nrepositories/a/b/c/_manifests/revisions/sha256/ff/link\n#tags\nrepositories/a/b/c/_manifests/tags/v1/current/link:sha256:ff\n#end\n")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if l.Blobs["sha256:abcd"] != 12 {
+		t.Errorf("blobs %v", l.Blobs)
+	}
+	r := l.Repos["a/b/c"]
+	if r == nil || !r.Revisions["sha256:ff"] || r.Tags["v1"] != "sha256:ff" {
+		t.Errorf("repository with slashes misread: %+v", r)
+	}
+}
+
+func TestACutListingIsRefused(t *testing.T) {
+	for _, out := range []string{"#blobs\n12 blobs/sha256/ab/abcd/data\n", "#blobs\nnot a line\n#end\n", "stray\n#end\n"} {
+		if _, err := ParseListing(out); err == nil {
+			t.Errorf("accepted %q", out)
+		}
+	}
+}
+
+func TestTheMarkIsTheCollectorsIncludingAnIndexAndSharedBlobs(t *testing.T) {
+	v := view(t, world())
+	marked := v.Marked(nil)
+	for _, name := range []string{"cfg", "shared", "old-only", "stranger-layer", "a-kept", "a-old", "doc-body", "empty", "m-kept", "idx"} {
+		if !marked[d(name)] {
+			t.Errorf("%s not marked", name)
+		}
+	}
+	if marked[d("orphan")] {
+		t.Error("a blob no manifest names is marked")
+	}
+	freed, n := v.Unmarked(nil)
+	if freed != 7000 || n != 1 {
+		t.Errorf("the collector frees %d in %d, want 7000 in 1", freed, n)
+	}
+	// The index alone gone frees its own content and nothing m-kept still marks.
+	_, n = v.Unmarked(map[string]bool{Key("app/server", d("idx")): true})
+	if n != 2 {
+		t.Errorf("without the index %d blobs unmarked, want 2 (orphan and the index itself)", n)
+	}
+	if sh := v.Shared(); !sh[d("empty")] || sh[d("cfg")] {
+		t.Error("shared misjudged")
+	}
+}
+
+func TestEveryStateIsGiven(t *testing.T) {
+	v := view(t, world())
+	got := map[string]string{}
+	for _, e := range Classify(v, withFill(records())) {
+		got[e.Repository+" "+e.Digest] = e.State
+	}
+	want := map[string]string{
+		"app/server " + d("m-kept"):     StateKept,
+		"app/server " + d("m-old"):      StateEligible,
+		"app/server " + d("m-stranger"): StateUnrecorded,
+		"app/server " + d("idx"):        StateUnrecorded,
+		"app/tools " + d("h-kept"):      StateHolderKept,
+		"app/tools " + d("h-old"):       StateHolderEligible,
+		"mesh/facts " + d("doc"):        StateNamedDocument,
+	}
+	if !reflect.DeepEqual(got, want) {
+		t.Errorf("got %v\nwant %v", got, want)
+	}
+	if m := Missing(v, withFill(records())); len(m) != 1 || !strings.Contains(m[0], hexOf(d("m-gone"))) {
+		t.Errorf("missing %v", m)
+	}
+}
+
+func TestALetGoReferenceStillPresentIsSaidSo(t *testing.T) {
+	v := view(t, world())
+	r := records()
+	r.References = append(r.References, Record{Reference: "artifact-store://app/server@" + d("m-stranger"), State: "collected"})
+	for _, e := range Classify(v, withFill(r)) {
+		if e.Digest == d("m-stranger") && e.State != StateCollectedPresent {
+			t.Errorf("state %s", e.State)
+		}
+	}
+}
+
+func TestAStateThisBundleDoesNotKnowIsKept(t *testing.T) {
+	if stateOf(&Record{State: "something-new"}, false) != StateKept {
+		t.Fatal("an unknown state was not kept")
+	}
+}
+
+func TestBytesFreedCountASharedBlobAsKept(t *testing.T) {
+	v := view(t, world())
+	a := &CollectAnswer{DryRun: true, Eligible: 2, WouldLetGo: []string{
+		"artifact-store://app/server@" + d("m-old"), "artifact-store://app/tools/blobs/" + d("a-old")}}
+	out := Collected(v, a, true)
+	now, after := out["collector_frees_now_bytes"].(int64), out["collector_frees_after_bytes"].(int64)
+	// m-old's own content, old-only and a-old and its holder go; shared and cfg stay with m-kept.
+	want := int64(7000) + 500 + 4000 + v.L.Blobs[d("m-old")] + v.L.Blobs[d("h-old")]
+	if now != 7000 || after != want {
+		t.Errorf("now %d after %d, want 7000 and %d", now, after, want)
+	}
+	if out["manifests_gone"].(int) != 2 {
+		t.Errorf("gone %v", out["manifests_gone"])
+	}
+}
+
+func TestOnlyTheirBytesLeaveOutWhatAnotherStateMarks(t *testing.T) {
+	v := view(t, world())
+	entries := Classify(v, withFill(records()))
+	sums := Summarise(v, entries, entries)
+	el := sums[StateEligible]
+	if el.OnlyTheirBytes != 500+v.L.Blobs[d("m-old")] {
+		t.Errorf("eligible only-theirs %d", el.OnlyTheirBytes)
+	}
+}
+
+// asked records every ask, and answers as the controller would.
+type asked struct {
+	mu    sync.Mutex
+	calls []string
+	args  []map[string]any
+	fail  map[string]bool
+}
+
+func (a *asked) ask(key string, body any) (json.RawMessage, error) {
+	a.mu.Lock()
+	defer a.mu.Unlock()
+	args, _ := body.(map[string]any)
+	a.calls = append(a.calls, key)
+	a.args = append(a.args, args)
+	if args["collected"] == "true" && a.fail["collected"] {
+		return nil, fmt.Errorf("maximum payload exceeded")
+	}
+	var answer any
+	switch key {
+	case "seat:mesh-controller.artifacts":
+		answer = records()
+	case "seat:mesh-controller.collect":
+		if args["confirm"] == "true" {
+			answer = CollectAnswer{LetGo: []string{"artifact-store://app/server@" + d("m-old")}}
+		} else {
+			answer = CollectAnswer{DryRun: true, Eligible: 1, WouldLetGo: []string{"artifact-store://app/server@" + d("m-old")}}
+		}
+	default:
+		return nil, fmt.Errorf("no such verb %s", key)
+	}
+	raw, _ := json.Marshal(answer)
+	return json.Marshal(map[string]any{"ok": true, "output": "", "answer": json.RawMessage(raw)})
+}
+
+func tool(t *testing.T, a *asked, name string) func(map[string]any) (any, error) {
+	f := world()
+	s := &Store{Container: "mesh-registry", Run: f.run(t)}
+	for _, tl := range Tools(s, Controller{Ask: a.ask}) {
+		if tl.Name == name {
+			return tl.Run
+		}
+	}
+	t.Fatalf("no tool %s", name)
+	return nil
+}
+
+func TestADryRunNeverConfirms(t *testing.T) {
+	a := &asked{}
+	for _, args := range []map[string]any{{}, {"dry_run": true}, {"dry_run": true, "why": "tidy"}} {
+		if _, err := tool(t, a, "artifact_store_collect")(args); err != nil {
+			t.Fatal(err)
+		}
+	}
+	for _, args := range a.args {
+		if args["confirm"] != nil || args["why"] != nil {
+			t.Errorf("a dry run asked %v", args)
+		}
+	}
+}
+
+func TestARealRunWithoutWhyIsRefusedAndAsksNothing(t *testing.T) {
+	a := &asked{}
+	if _, err := tool(t, a, "artifact_store_collect")(map[string]any{"dry_run": false}); err == nil {
+		t.Fatal("a real run without why was accepted")
+	}
+	if len(a.calls) != 0 {
+		t.Errorf("asked %v", a.calls)
+	}
+	out, err := tool(t, a, "artifact_store_collect")(map[string]any{"dry_run": false, "why": "the store is full", "most": float64(10)})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if a.args[0]["confirm"] != "true" || a.args[0]["why"] != "the store is full" || a.args[0]["most"] != "10" {
+		t.Errorf("asked %v", a.args[0])
+	}
+	if out.(map[string]any)["dry_run"] != false {
+		t.Error("a real run answered as a dry run")
+	}
+}
+
+func TestReferencesFallBackWhenCollectedCannotBeCarried(t *testing.T) {
+	a := &asked{fail: map[string]bool{"collected": true}}
+	out, err := tool(t, a, "artifact_store_references")(map[string]any{"repository": "app/server"})
+	if err != nil {
+		t.Fatal(err)
+	}
+	m := out.(map[string]any)
+	if m["note"] == nil || m["manifests"] == nil {
+		t.Errorf("answer %v", m)
+	}
+	if _, err := tool(t, a, "artifact_store_references")(map[string]any{"repository": "nope"}); err == nil {
+		t.Error("an unknown repository was answered")
+	}
+}
+
+func TestTheToolsServedAreTheToolsTheManifestNames(t *testing.T) {
+	raw, err := os.ReadFile("../../module.json")
+	if err != nil {
+		t.Fatal(err)
+	}
+	var m struct {
+		Tools   []string `json:"tools"`
+		Invokes []string `json:"invokes"`
+	}
+	if err := json.Unmarshal(raw, &m); err != nil {
+		t.Fatal(err)
+	}
+	var served []string
+	for _, tl := range Tools(&Store{}, Controller{}) {
+		if !strings.HasPrefix(tl.Name, "artifact_store_") || tl.Description == "" || tl.Run == nil {
+			t.Errorf("tool %q", tl.Name)
+		}
+		served = append(served, tl.Name)
+	}
+	sort.Strings(served)
+	listed := append([]string{}, m.Tools...)
+	sort.Strings(listed)
+	if !reflect.DeepEqual(served, listed) {
+		t.Fatalf("served %v, manifest %v", served, listed)
+	}
+	want := []string{"seat:mesh-controller.artifacts", "seat:mesh-controller.collect"}
+	if !reflect.DeepEqual(m.Invokes, want) {
+		t.Errorf("invokes %v", m.Invokes)
+	}
+}
+
+func TestASocketRefusalIsAskedAgainThroughSudo(t *testing.T) {
+	var ran []string
+	run := func(_ context.Context, name string, args ...string) Ran {
+		ran = append(ran, name)
+		if name == "docker" {
+			return Ran{Status: 1, Stderr: "permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock"}
+		}
+		return Ran{Stdout: "#end\n"}
+	}
+	if _, err := docker(context.Background(), run, 1000, "exec", "x"); err != nil {
+		t.Fatal(err)
+	}
+	if !reflect.DeepEqual(ran, []string{"docker", "sudo"}) {
+		t.Errorf("ran %v", ran)
+	}
+	if _, err := docker(context.Background(), func(context.Context, string, ...string) Ran {
+		return Ran{Status: 1, Stderr: "Error: No such container: mesh-registry"}
+	}, 1000, "exec"); err == nil || !strings.Contains(err.Error(), "not on this machine") {
+		t.Errorf("err %v", err)
+	}
+}

modules/artifact-store-tools/cmd/artifact-store-tools/tools.go

@@ -0,0 +1,351 @@
+package main
+
+import (
+	"context"
+	"fmt"
+	"math"
+	"sort"
+	"strconv"
+	"strings"
+
+	stdio "git.novox.be/novox/mesh-sdk/go"
+)
+
+// Tools are the artifact store's tools (novox/hq ADR 0251 §1). The first three only read; artifact_store_collect
+// only reads too, unless it is a real run, and then it asks the controller, which decides and deletes.
+func Tools(s *Store, c Controller) []stdio.Tool {
+	ctx := context.Background()
+	repoArg := map[string]any{"type": "string", "description": "one repository, as the store names it (<module>/<artifact>)"}
+	return []stdio.Tool{
+		{
+			Name: "artifact_store_repositories",
+			Description: "Every repository the artifact store holds: its tags and what each names, how many manifests it holds " +
+				"(tagged or not — the mesh pins by digest, so most are untagged) and its size, the blobs its manifests mark. " +
+				"A blob two repositories share is counted in each, and shared_bytes says how much of a size that is. Read from " +
+				"the store's own files; changes nothing. Replaces curl /v2/_catalog and /v2/<name>/tags/list. (r)",
+			Input: map[string]any{"repository": repoArg},
+			Run: func(args map[string]any) (any, error) {
+				v, err := s.View(ctx)
+				if err != nil {
+					return nil, err
+				}
+				return Repositories(v, text(args, "repository"))
+			},
+		},
+		{
+			Name: "artifact_store_usage",
+			Description: "How large the artifact store is: every blob's bytes together, the largest repositories, the bytes " +
+				"more than one repository's manifests mark, and the bytes no manifest marks — what the store's nightly collector " +
+				"frees next. Read from the store's own files; changes nothing. Replaces du on the store's directory. (r)",
+			Input: map[string]any{"top": map[string]any{"type": "integer", "description": "how many of the largest repositories to name (default 10, at most 200)"}},
+			Run: func(args map[string]any) (any, error) {
+				top, err := count(args, "top", 10, 1, 200)
+				if err != nil {
+					return nil, err
+				}
+				v, err := s.View(ctx)
+				if err != nil {
+					return nil, err
+				}
+				return Usage(v, top), nil
+			},
+		},
+		{
+			Name: "artifact_store_references",
+			Description: "What the controller's records say of each manifest the artifact store holds: kept (a definition " +
+				"names it, or one of the five most recent builds of a module the mesh holds), the holder of a kept archive, " +
+				"eligible (the mesh made it and keeps it for no reason), the holder of an eligible archive, let go yet present, " +
+				"a named document the controller keeps under a tag, or unrecorded — no record names it. Counted and sized by " +
+				"state and by repository; each manifest listed when one repository is asked. Then what the records keep that " +
+				"the store does not hold. Unrecorded manifests are never removed by any tool (novox/hq ADR 0189 §3). (r)",
+			Input: map[string]any{"repository": repoArg},
+			Run: func(args map[string]any) (any, error) {
+				repo := text(args, "repository")
+				recs, err := c.Artifacts("")
+				if err != nil {
+					return nil, err
+				}
+				v, err := s.View(ctx)
+				if err != nil {
+					return nil, err
+				}
+				return References(v, recs, repo)
+			},
+		},
+		{
+			Name: "artifact_store_collect",
+			Description: "Collect what the mesh made and keeps for no reason (novox/hq ADR 0189, ADR 0251 §3). A dry run unless " +
+				"dry_run is false: it asks the controller's collect what it would let go of, and works out how many bytes the " +
+				"store's nightly collector would free once those are gone, beside what it frees tonight anyway. A real run needs " +
+				"why: it asks the controller's collect to hold every kept archive and let go of the eligible ones, recorded as a " +
+				"hand-act with the why; the bytes come back at the nightly collection. Never removes an unrecorded manifest, and " +
+				"never deletes through the store's door itself. (a)",
+			Input: map[string]any{
+				"dry_run": map[string]any{"type": "boolean", "description": "true (the default): say what would go, change nothing"},
+				"why":     map[string]any{"type": "string", "description": "why — required for a real run, and recorded as the hand-act's reason"},
+				"most":    map[string]any{"type": "integer", "description": "the most artifacts one real run lets go of (the controller's default when absent)"},
+			},
+			Run: func(args map[string]any) (any, error) {
+				dry := true
+				if b, given := args["dry_run"].(bool); given {
+					dry = b
+				} else if t := text(args, "dry_run"); t == "false" {
+					dry = false
+				}
+				why := text(args, "why")
+				if !dry && why == "" {
+					return nil, fmt.Errorf("a real collection needs why: nothing was asked of the controller")
+				}
+				most, err := count(args, "most", 0, 1, 5000)
+				if err != nil {
+					return nil, err
+				}
+				answer, err := c.Collect(why, !dry, most)
+				if err != nil {
+					return nil, err
+				}
+				v, err := s.View(ctx)
+				if err != nil {
+					return map[string]any{"controller": answer, "bytes_not_worked_out": err.Error()}, nil
+				}
+				return Collected(v, answer, dry), nil
+			},
+		},
+	}
+}
+
+// View reads the store now.
+func (s *Store) View(ctx context.Context) (*View, error) {
+	l, err := s.List(ctx)
+	if err != nil {
+		return nil, err
+	}
+	m, unread := s.Manifests(ctx, l)
+	return &View{L: l, M: m, Unread: unread}, nil
+}
+
+// unreadNote says what manifests that could not be read do to the numbers, when there are any.
+func unreadNote(v *View) map[string]any {
+	if len(v.Unread) == 0 {
+		return nil
+	}
+	keys := make([]string, 0, len(v.Unread))
+	for k := range v.Unread {
+		keys = append(keys, k)
+	}
+	sort.Strings(keys)
+	first := keys[0] + ": " + v.Unread[keys[0]]
+	return map[string]any{
+		"manifests": len(keys),
+		"first":     first,
+		"means": "what these mark beyond themselves is not known, so sizes may be low and the bytes said to be " +
+			"freed may be high; ask again — what was read is kept, so the next call reads only the rest",
+	}
+}
+
+// Repositories answers artifact_store_repositories.
+func Repositories(v *View, only string) (map[string]any, error) {
+	shared := v.Shared()
+	var repos []map[string]any
+	for _, name := range v.L.RepoNames() {
+		if only != "" && name != only {
+			continue
+		}
+		r := v.L.Repos[name]
+		blobs := v.RepoBlobs(name)
+		sharedHere := map[string]bool{}
+		for b := range blobs {
+			if shared[b] {
+				sharedHere[b] = true
+			}
+		}
+		repos = append(repos, map[string]any{
+			"repository": name, "tags": tagsOf(r), "manifests": len(r.Revisions),
+			"bytes": v.Bytes(blobs), "size": human(v.Bytes(blobs)), "shared_bytes": v.Bytes(sharedHere),
+		})
+	}
+	if only != "" && len(repos) == 0 {
+		return nil, fmt.Errorf("the store holds no repository %q", only)
+	}
+	out := map[string]any{"count": len(repos), "repositories": repos,
+		"note": "a repository's size is the blobs its manifests mark; a blob shared with another repository is counted in each (shared_bytes)"}
+	if u := unreadNote(v); u != nil {
+		out["unread"] = u
+	}
+	return out, nil
+}
+
+// Usage answers artifact_store_usage.
+func Usage(v *View, top int) map[string]any {
+	type sized struct {
+		name  string
+		bytes int64
+		n     int
+	}
+	var all []sized
+	manifests := 0
+	for _, name := range v.L.RepoNames() {
+		n := len(v.L.Repos[name].Revisions)
+		manifests += n
+		all = append(all, sized{name, v.Bytes(v.RepoBlobs(name)), n})
+	}
+	sort.SliceStable(all, func(i, j int) bool { return all[i].bytes > all[j].bytes })
+	var largest []map[string]any
+	for i, s := range all {
+		if i >= top {
+			break
+		}
+		largest = append(largest, map[string]any{"repository": s.name, "bytes": s.bytes, "size": human(s.bytes), "manifests": s.n})
+	}
+	total := v.Total()
+	sharedBytes := v.Bytes(v.Shared())
+	freed, freedBlobs := v.Unmarked(nil)
+	out := map[string]any{
+		"total_bytes": total, "total": human(total), "blobs": len(v.L.Blobs), "manifests": manifests,
+		"repositories": len(v.L.Repos), "largest": largest,
+		"shared_bytes": sharedBytes, "collector_frees_next_bytes": freed, "collector_frees_next_blobs": freedBlobs,
+		"said": []string{
+			fmt.Sprintf("the store holds %s in %d blobs, %d manifests in %d repositories", human(total), len(v.L.Blobs), manifests, len(v.L.Repos)),
+			fmt.Sprintf("%s is marked by more than one repository's manifests", human(sharedBytes)),
+			fmt.Sprintf("the nightly collector frees %s in %d blobs no manifest marks", human(freed), freedBlobs),
+		},
+	}
+	if u := unreadNote(v); u != nil {
+		out["unread"] = u
+	}
+	return out
+}
+
+// References answers artifact_store_references.
+func References(v *View, recs *Records, only string) (map[string]any, error) {
+	if only != "" && v.L.Repos[only] == nil {
+		return nil, fmt.Errorf("the store holds no repository %q", only)
+	}
+	entries := Classify(v, recs)
+	byRepo := map[string][]Entry{}
+	for _, e := range entries {
+		byRepo[e.Repository] = append(byRepo[e.Repository], e)
+	}
+	var repos []map[string]any
+	for _, name := range v.L.RepoNames() {
+		if only != "" && name != only {
+			continue
+		}
+		repos = append(repos, map[string]any{"repository": name, "states": Summarise(v, byRepo[name], entries)})
+	}
+	whole := Summarise(v, entries, entries)
+	said := []string{}
+	for _, state := range []string{StateKept, StateHolderKept, StateEligible, StateHolderEligible, StateCollectedPresent, StateNamedDocument, StateUnrecorded} {
+		if s := whole[state]; s != nil {
+			said = append(said, fmt.Sprintf("%s: %d manifests marking %s, %s of it marked by nothing else", state, s.Manifests, human(s.Bytes), human(s.OnlyTheirBytes)))
+		}
+	}
+	missing := Missing(v, recs)
+	if len(missing) > 0 {
+		said = append(said, fmt.Sprintf("%d references the records keep are not in the store", len(missing)))
+	}
+	out := map[string]any{
+		"store": recs.Store, "kept_builds": recs.KeptBuilds, "records": recs.Counts,
+		"states": whole, "repositories": repos, "missing": nonNil(missing), "said": said,
+		"removable": collectionRemovesThese,
+	}
+	if only != "" {
+		out["manifests"] = byRepo[only]
+	}
+	if recs.Note != "" {
+		out["note"] = recs.Note
+	}
+	if u := unreadNote(v); u != nil {
+		out["unread"] = u
+	}
+	return out, nil
+}
+
+// Collected answers artifact_store_collect: the controller's answer, and what the store's collector frees.
+func Collected(v *View, a *CollectAnswer, dry bool) map[string]any {
+	refs := a.LetGo
+	if dry {
+		refs = a.WouldLetGo
+	}
+	gone := LetGoKeys(v, refs)
+	now, nowBlobs := v.Unmarked(nil)
+	after, afterBlobs := v.Unmarked(gone)
+	said := []string{}
+	if dry {
+		said = append(said, fmt.Sprintf("dry run: the controller would let go of %d of %d eligible artifacts; nothing was changed", len(a.WouldLetGo), a.Eligible))
+	} else {
+		said = append(said, fmt.Sprintf("the controller let go of %d artifacts (%d left, %d skipped)", len(a.LetGo), a.Left, a.Skipped))
+		if a.Stopped != "" {
+			said = append(said, "it stopped: "+a.Stopped)
+		}
+	}
+	said = append(said,
+		fmt.Sprintf("the nightly collector frees %s tonight as the store is now", human(now)),
+		fmt.Sprintf("and %s once those %d manifests are gone (%s more)", human(after), len(gone), human(after-now)))
+	if dry && a.Eligible > len(a.WouldLetGo) {
+		said = append(said, fmt.Sprintf("the controller listed %d of %d eligible: the bytes cover only those listed", len(a.WouldLetGo), a.Eligible))
+	}
+	out := map[string]any{
+		"dry_run": dry, "controller": a,
+		"collector_frees_now_bytes": now, "collector_frees_now_blobs": nowBlobs,
+		"collector_frees_after_bytes": after, "collector_frees_after_blobs": afterBlobs,
+		"manifests_gone": len(gone), "said": said,
+		"note": "a manifest let go of frees no bytes until the store's nightly collector runs with the store held still (ADR 0189 §4)",
+	}
+	if u := unreadNote(v); u != nil {
+		out["unread"] = u
+	}
+	return out
+}
+
+func nonNil(s []string) []string {
+	if s == nil {
+		return []string{}
+	}
+	return s
+}
+
+func human(n int64) string {
+	switch {
+	case n >= 1<<30:
+		return fmt.Sprintf("%.1f GiB", float64(n)/(1<<30))
+	case n >= 1<<20:
+		return fmt.Sprintf("%.1f MiB", float64(n)/(1<<20))
+	case n >= 1<<10:
+		return fmt.Sprintf("%.1f KiB", float64(n)/(1<<10))
+	}
+	return fmt.Sprintf("%d B", n)
+}
+
+func text(args map[string]any, key string) string {
+	s, _ := args[key].(string)
+	return strings.TrimSpace(s)
+}
+
+// count is a whole-number argument, defaulted, refused below least, held at most.
+func count(args map[string]any, key string, fallback, least, most int) (int, error) {
+	v, given := args[key]
+	if !given || v == nil {
+		return fallback, nil
+	}
+	var n int
+	switch x := v.(type) {
+	case float64:
+		if x != math.Trunc(x) {
+			return 0, fmt.Errorf("%s must be a whole number, not %v", key, x)
+		}
+		n = int(x)
+	case string:
+		i, err := strconv.Atoi(strings.TrimSpace(x))
+		if err != nil {
+			return 0, fmt.Errorf("%s must be a whole number, not %q", key, x)
+		}
+		n = i
+	default:
+		return 0, fmt.Errorf("%s must be a whole number", key)
+	}
+	if n < least {
+		return 0, fmt.Errorf("%s must be at least %d", key, least)
+	}
+	return min(n, most), nil
+}

modules/artifact-store-tools/cmd/artifact-store-tools/view.go

@@ -0,0 +1,129 @@
+package main
+
+import (
+	"sort"
+)
+
+// View is the store at one moment: its files, and every manifest's content that could be read.
+type View struct {
+	L *Listing
+	// M is each manifest read, by Key.
+	M map[string]*Manifest
+	// Unread is each manifest that could not be read, by Key, with why.
+	Unread map[string]string
+}
+
+// Marks are the blobs one manifest keeps from the store's collector: its own content, its
+// configuration and its layers, and for an index the manifests it lists. A manifest that could not be
+// read marks only itself here, and the answers that rest on it say so.
+func (v *View) Marks(repo, digest string) []string {
+	out := []string{digest}
+	if m := v.M[Key(repo, digest)]; m != nil {
+		out = append(out, m.Refs...)
+		out = append(out, m.Children...)
+	}
+	return out
+}
+
+// Marked is every blob some manifest marks, leaving out the manifests named in without — the
+// collector's own mark, and the mark it would make once those are gone.
+func (v *View) Marked(without map[string]bool) map[string]bool {
+	marked := map[string]bool{}
+	for name, r := range v.L.Repos {
+		for d := range r.Revisions {
+			if without[Key(name, d)] {
+				continue
+			}
+			for _, b := range v.Marks(name, d) {
+				marked[b] = true
+			}
+		}
+	}
+	return marked
+}
+
+// Bytes is the size of these blobs, as the store holds them; a blob the store does not hold counts
+// nothing.
+func (v *View) Bytes(blobs map[string]bool) int64 {
+	var n int64
+	for b := range blobs {
+		n += v.L.Blobs[b]
+	}
+	return n
+}
+
+// Total is the size of every blob the store holds.
+func (v *View) Total() int64 {
+	var n int64
+	for _, s := range v.L.Blobs {
+		n += s
+	}
+	return n
+}
+
+// Unmarked is the bytes and count of the blobs no manifest marks, once the manifests in without are
+// gone: what the nightly collector frees.
+func (v *View) Unmarked(without map[string]bool) (int64, int) {
+	marked := v.Marked(without)
+	var n int64
+	count := 0
+	for b, s := range v.L.Blobs {
+		if !marked[b] {
+			n += s
+			count++
+		}
+	}
+	return n, count
+}
+
+// RepoBlobs are the blobs one repository's manifests mark.
+func (v *View) RepoBlobs(name string) map[string]bool {
+	out := map[string]bool{}
+	if r := v.L.Repos[name]; r != nil {
+		for d := range r.Revisions {
+			for _, b := range v.Marks(name, d) {
+				out[b] = true
+			}
+		}
+	}
+	return out
+}
+
+// Shared are the blobs that the manifests of more than one repository mark.
+func (v *View) Shared() map[string]bool {
+	seen := map[string]string{}
+	shared := map[string]bool{}
+	for _, name := range v.L.RepoNames() {
+		for b := range v.RepoBlobs(name) {
+			if first, ok := seen[b]; ok && first != name {
+				shared[b] = true
+			} else {
+				seen[b] = name
+			}
+		}
+	}
+	return shared
+}
+
+// tagsOf is a repository's tags, in order, with what each names.
+func tagsOf(r *Repo) []map[string]string {
+	names := make([]string, 0, len(r.Tags))
+	for t := range r.Tags {
+		names = append(names, t)
+	}
+	sort.Strings(names)
+	out := make([]map[string]string, 0, len(names))
+	for _, t := range names {
+		out = append(out, map[string]string{"tag": t, "digest": r.Tags[t]})
+	}
+	return out
+}
+
+// taggedDigests are the manifests some tag names, in one repository.
+func taggedDigests(r *Repo) map[string]bool {
+	out := map[string]bool{}
+	for _, d := range r.Tags {
+		out[d] = true
+	}
+	return out
+}

modules/artifact-store-tools/go.mod

@@ -0,0 +1,5 @@
+module artifactstoretools
+
+go 1.22
+
+require git.novox.be/novox/mesh-sdk/go v0.1.7

modules/artifact-store-tools/go.sum

@@ -0,0 +1,2 @@
+git.novox.be/novox/mesh-sdk/go v0.1.7 h1:C0sTQmtTiyYH7bnqZb7PusXnqA37gKuT7Nqjn9gG47w=
+git.novox.be/novox/mesh-sdk/go v0.1.7/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=

modules/artifact-store-tools/module.json

@@ -0,0 +1,32 @@
+{
+  "module": "artifact-store-tools",
+  "version": "1",
+  "invokes": [
+    "seat:mesh-controller.artifacts",
+    "seat:mesh-controller.collect"
+  ],
+  "tools": [
+    "artifact_store_repositories",
+    "artifact_store_usage",
+    "artifact_store_references",
+    "artifact_store_collect"
+  ],
+  "build": {
+    "artifacts": [
+      {
+        "name": "tools-go",
+        "kind": "bundle",
+        "language": "go",
+        "system": "arch",
+        "from": "cmd/artifact-store-tools",
+        "binary": "artifact-store-tools",
+        "loads": [
+          "artifact-store-tools"
+        ],
+        "env": {
+          "MESH_ARTIFACT_STORE_CONTAINER": "mesh-registry"
+        }
+      }
+    ]
+  }
+}

modules/distribution/client.ts

@@ -1,81 +0,0 @@
-// The Docker Registry v2 client — registry's own code, living in the module (novox/hq ADR 0039).
-// Ported from the shared hal sdk, where a change to the registry API rebuilt everything; here it
-// rebuilds only registry. Both this module's tools and its events entrypoint import it.
-
-export interface RegistryImage {
-  repo: string;
-  tag: string;
-  digest: string;
-}
-
-export class RegistryClient {
-  readonly baseUrl: string;
-
-  // Auth is optional: a mesh-internal registry often runs open on the node, so a Basic header is
-  // sent only when credentials were configured — an empty one would look like a failed login.
-  constructor(
-    url: string,
-    private readonly authHeader?: string,
-  ) {
-    this.baseUrl = url.replace(/\/+$/, "");
-  }
-
-  /**
-   * Build from the module's resolved environment. The URL is MESH_REGISTRY_URL (or the local
-   * registry port), and credentials — if the registry requires them — are MESH_REGISTRY_USER and
-   * MESH_REGISTRY_PASSWORD. Throws when no URL is configured, so a misconfigured module exposes
-   * nothing rather than talking to the wrong place.
-   */
-  static fromEnv(env: NodeJS.ProcessEnv = process.env): RegistryClient {
-    const url = env.MESH_REGISTRY_URL ?? `http://127.0.0.1:${env.REGISTRY_PORT ?? "5000"}`;
-    if (!url) throw new Error("no registry URL — set MESH_REGISTRY_URL");
-    const user = env.MESH_REGISTRY_USER;
-    const password = env.MESH_REGISTRY_PASSWORD;
-    const authHeader =
-      user && password ? `Basic ${Buffer.from(`${user}:${password}`).toString("base64")}` : undefined;
-    return new RegistryClient(url, authHeader);
-  }
-
-  private headers(extra: Record<string, string> = {}): Record<string, string> {
-    return { ...(this.authHeader ? { Authorization: this.authHeader } : {}), ...extra };
-  }
-
-  private async getJson<T>(path: string): Promise<T> {
-    const res = await fetch(`${this.baseUrl}${path}`, { headers: this.headers() });
-    if (!res.ok) throw new Error(`Registry ${path}: ${res.status} ${await res.text()}`);
-    return res.json() as Promise<T>;
-  }
-
-  /** The catalog — every repository the registry holds. */
-  async listRepositories(): Promise<string[]> {
-    const data = await this.getJson<{ repositories: string[] | null }>("/v2/_catalog");
-    return data.repositories ?? [];
-  }
-
-  /** The tags of one repository. */
-  async listTags(repo: string): Promise<string[]> {
-    const data = await this.getJson<{ tags: string[] | null }>(`/v2/${repo}/tags/list`);
-    return data.tags ?? [];
-  }
-
-  /** The content digest of a repo:tag — the stable identity a tag currently points at. */
-  async getManifestDigest(repo: string, tag: string): Promise<string> {
-    const res = await fetch(`${this.baseUrl}/v2/${repo}/manifests/${tag}`, {
-      method: "HEAD",
-      headers: this.headers({ Accept: "application/vnd.docker.distribution.manifest.v2+json" }),
-    });
-    if (!res.ok) throw new Error(`Registry manifest ${repo}:${tag}: ${res.status} ${await res.text()}`);
-    const digest = res.headers.get("docker-content-digest");
-    if (!digest) throw new Error(`no Docker-Content-Digest for ${repo}:${tag}`);
-    return digest;
-  }
-
-  /** Delete a manifest by digest. Garbage collection reclaims the storage later. */
-  async deleteManifest(repo: string, digest: string): Promise<void> {
-    const res = await fetch(`${this.baseUrl}/v2/${repo}/manifests/${digest}`, {
-      method: "DELETE",
-      headers: this.headers({ Accept: "application/vnd.docker.distribution.manifest.v2+json" }),
-    });
-    if (!res.ok) throw new Error(`Registry delete ${repo}@${digest}: ${res.status} ${await res.text()}`);
-  }
-}

modules/distribution/index.ts

@@ -1,51 +0,0 @@
-// registry's events. The tool runtime imports this once the broker is bound.
-//
-// Emits (novox/hq ADR 0041/0042):
-//   module.registry.image.pushed   — a new image (repo:tag) was published to the registry
-//
-// This is a genuinely useful signal: a build finished and its image is now pullable, so anything
-// on the mesh that redeploys, mirrors or announces releases can react without polling the registry
-// itself. It is discovered by diffing the catalog and each repo's tags — the registry has no push
-// webhook of its own, so the module watches for it.
-//
-// The polling is deliberately unhurried: a new image a minute late is still the event, whereas
-// hammering the registry's catalog for immediacy nobody asked for is not.
-
-import { emit } from "@novox/mesh-sdk/events";
-import { RegistryClient } from "./client.js";
-
-const registry = RegistryClient.fromEnv();
-
-// Every repo:tag we have already accounted for. Primed silently on the first look so a registry
-// that was already full when this started does not announce its whole history as freshly pushed.
-const seen = new Set<string>();
-let primed = false;
-
-async function pollCatalog(): Promise<void> {
-  const repos = await registry.listRepositories();
-  for (const repo of repos) {
-    let tags: string[];
-    try {
-      tags = await registry.listTags(repo);
-    } catch {
-      continue; // a repo can vanish between catalog and tag read — skip it, catch it next tick
-    }
-    for (const tag of tags) {
-      const id = `${repo}:${tag}`;
-      if (!seen.has(id)) {
-        if (primed) await emit("image.pushed", { repo, tag });
-        seen.add(id);
-      }
-    }
-  }
-  primed = true;
-}
-
-const tick = (fn: () => Promise<void>, everyMs: number): void => {
-  const run = (): void => void fn().catch((err) => console.error(`[registry] ${err}`));
-  setInterval(run, everyMs);
-  run();
-};
-tick(pollCatalog, 60_000);
-
-console.log("[registry] watching the catalog for newly pushed images");

modules/distribution/module.json

@@ -16,9 +16,6 @@
   "capabilities": [
     "container-runtime"
   ],
-  "emits": [
-    "image.pushed"
-  ],
   "own-secrets": {
     "broker": "/var/lib/mesh/registry/broker"
   },

modules/distribution/package.json

@@ -1,14 +0,0 @@
-{
-  "name": "@novox/module-registry",
-  "version": "0.1.0",
-  "description": "registry — private Docker image registry. Its API client, tools and events live here (novox/hq ADR 0039).",
-  "type": "module",
-  "private": true,
-  "dependencies": {
-    "@novox/mesh-sdk": "^0.1.0"
-  },
-  "devDependencies": {
-    "@types/node": "^22.0.0",
-    "typescript": "^5.6.0"
-  }
-}

modules/distribution/tools/index.ts

@@ -1,59 +0,0 @@
-// registry's tools — moved here from the shared sdk (novox/hq ADR 0039), importing registry's own
-// client. They return structured data; the mesh serves them through the sdk's tool harness.
-
-import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
-import { RegistryClient } from "../client.js";
-
-export function getRegistryTools(registry: RegistryClient): ToolDefinition[] {
-  return [
-    {
-      name: "registry_list",
-      description: "List every repository in the Docker registry (the catalog).",
-      input: {},
-      run: async () => {
-        const repositories = await registry.listRepositories();
-        return { count: repositories.length, repositories };
-      },
-    },
-    {
-      name: "registry_tags",
-      description: "List the tags of one repository in the Docker registry.",
-      input: { repo: { type: "string", description: "the repository name, e.g. 'novox/mesh'" } },
-      run: async (args) => {
-        const repo = String(args.repo);
-        const tags = await registry.listTags(repo);
-        return { repo, count: tags.length, tags };
-      },
-    },
-    {
-      name: "registry_delete_image",
-      description:
-        "Delete an image tag from the registry (DESTRUCTIVE). Removes the manifest; storage is reclaimed by garbage collection later. Requires confirm: true.",
-      input: {
-        repo: { type: "string", description: "the repository name, e.g. 'novox/mesh'" },
-        tag: { type: "string", description: "the tag to delete, e.g. 'latest'" },
-        confirm: { type: "boolean", description: "must be true to actually delete" },
-      },
-      run: async (args) => {
-        const repo = String(args.repo);
-        const tag = String(args.tag);
-        if (args.confirm !== true) {
-          return { deleted: false, reason: "confirm must be true to delete an image" };
-        }
-        const digest = await registry.getManifestDigest(repo, tag);
-        await registry.deleteManifest(repo, digest);
-        return { deleted: true, repo, tag, digest, note: "run registry garbage collection to reclaim storage" };
-      },
-    },
-  ];
-}
-
-// The tools exist only when a registry URL is configured; otherwise registry contributes none
-// rather than failing the whole runtime.
-registerModuleTools("registry", (env) => {
-  try {
-    return getRegistryTools(RegistryClient.fromEnv(env));
-  } catch {
-    return [];
-  }
-});

modules/distribution/tsconfig.json

@@ -1,12 +0,0 @@
-{
-  "compilerOptions": {
-    "target": "ES2022",
-    "module": "NodeNext",
-    "moduleResolution": "NodeNext",
-    "strict": true,
-    "esModuleInterop": true,
-    "skipLibCheck": true,
-    "noEmit": true
-  },
-  "include": ["client.ts", "index.ts", "tools/index.ts"]
-}