diff --git a/modules/mesh-knowledge/README.md b/modules/mesh-knowledge/README.md index e1f0a255..bf746317 100644 --- a/modules/mesh-knowledge/README.md +++ b/modules/mesh-knowledge/README.md @@ -56,8 +56,14 @@ A clause is `proposed`, `in force`, `changed` (a later clause replaces part of i - `confirmed`: drafted, then confirmed by a second, separate judgement that saw only the record and the drafts. The links are `supersedes`, `changes` (to a clause: `0276 §4`), `extends`, `became` (from a research effort to a -decision record or design) and `cites` (to any record, an issue, or a design). Each link is stored once, on the -record it goes out from. +decision record or design) and `cites` (to any record, an issue, or a design). Two more hold what hq's records +write and to-be 55 names no kind for (hq issue 487): `rests-on`, from a decision to one it stands on (held to the +rules `extends` is), and `touches`, from a research effort to a record, issue or design it bears on. Each link is +stored once, on the record it goes out from. + +Besides the fields every record has, a decision holds `reconstructed` (true or false, where it says) and `amended` +(the days), and a research effort holds what it `touches` that is no record, issue or design ("the tool runtime") +as text. `get` serves each, and any other field a record of hq wrote under `fields`. ## Writing a record @@ -109,8 +115,20 @@ no counter moves. A second run changes nothing and says so. `dry-run` writes not `mesh-knowledge dry-run ` gives the same dry run over a checkout on disk, against an empty knowledge base. -A decision whose Decision section writes numbered statements with a bold lead gets them as `written` clauses. One -written as prose (226 of main's 278 on 2026-10-11) gets none at the import. A background worker then takes one +**`refresh`** (the import's third argument) repairs what an earlier import lost or misread (hq issue 487). For a record held whose every version +was written by the import or a refresh (no one has changed it since, and nothing waits over it), it reads the +record again at the commit and, when what it holds differs, writes it as a new version: "refreshed from hq at +: ", naming each field that differs. A record anyone changed after its import (drafted, accepted +over, corrected) is never touched, and its row says so. A record read again and found the same is left as it is, +so a second refresh changes nothing. With `dry-run` it says, record by record, what it would change. + +A decision whose Decision section's own top-level list writes numbered statements with a bold lead gets them as +`written` clauses. A numbered list inside a `###` subsection of the Decision section is that subsection's prose, +never the record's clauses (ADR 0259, 0272, 0273, 0278 and 0288 number steps so; hq issue 487). A decision with no +such list (245 of 278 at hq f0892019, 196 written clauses in the rest) gets none at the import, and so does one +with no Decision section at all (11): it is drafted from what it decides across its sections, all but its +preamble, consequences and references. One too large for one judgement (48 KiB) is marked as having no clauses, +and `get` says why. A background worker then takes one judgement at a time (`claude-code.claude_code_judge` on this node): it drafts the clauses, then confirms them in a separate call. A clause the second judgement rejects stays `drafted`, with the reason. hq's `supersedes-in-part` named a record, not a clause: which clause it changes is judged and then confirmed the same way, and only then is diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/answers.go b/modules/mesh-knowledge/cmd/mesh-knowledge/answers.go index 9a21a34e..7b2aab27 100644 --- a/modules/mesh-knowledge/cmd/mesh-knowledge/answers.go +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/answers.go @@ -304,6 +304,13 @@ type Got struct { Cut string `json:"cut,omitempty"` Bound int `json:"bound"` Read string `json:"read,omitempty"` + + // Reconstructed, Amended and Touches are the record's fields of those names; Fields, any other it holds as + // written (hq issue 487). + Reconstructed *bool `json:"reconstructed,omitempty"` + Amended []string `json:"amended,omitempty"` + Touches []string `json:"touches,omitempty"` + Fields map[string]string `json:"fields,omitempty"` } // Get is one record: its clauses, its links both ways, what changed it since; its reasoning only when asked. @@ -334,7 +341,8 @@ func (k *KB) Get(a GetAsk) (*Got, error) { bound := boundOf(a.Bound) g := &Got{Record: key.String(), Title: r.Title, Status: r.Status, Topic: r.Topic, Date: r.Date, Deciders: r.Deciders, Version: version, Versions: e.Versions, Insights: r.Insights, Reviews: r.Reviews, Drafting: r.Drafting, Source: r.Source, - NeedsOperat: r.NeedsOperator, Bound: bound, Since: k.since(ctx, key)} + NeedsOperat: r.NeedsOperator, Bound: bound, Since: k.since(ctx, key), + Reconstructed: r.Reconstructed, Amended: r.Amended, Touches: r.Touches, Fields: r.Extra} if r.Status == StatusReserved || r.Status == StatusWithdrawn { g.Reserved = fmt.Sprintf("reserved by %s at %s for %q", r.ReservedBy, r.ReservedAt.Format("2006-01-02 15:04 UTC"), r.ReservedFor) } diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/draft.go b/modules/mesh-knowledge/cmd/mesh-knowledge/draft.go index 2dda7547..4bcbe82e 100644 --- a/modules/mesh-knowledge/cmd/mesh-knowledge/draft.go +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/draft.go @@ -78,9 +78,50 @@ var whichConfirmSchema = map[string]any{ "required": []string{"exact", "reason"}, } -// DraftQuestion is the first judgement's question: the record's Decision section, nothing else of the mesh. -func DraftQuestion(r *Record, section, returned string) string { +// MostDraftSource is the most text a record is drafted from in one judgement, which takes at most 64 KiB. +const MostDraftSource = 48 << 10 + +// draftSource is what a record's clauses are drafted from: its Decision section, or, for a record that has none +// (hq issue 487), what it decides across its sections — all but its preamble, consequences and references. +func draftSource(r *Record) (text string, whole bool) { + if section, ok := decisionSection(r); ok && strings.TrimSpace(section) != "" { + return section, false + } var b strings.Builder + for _, s := range r.Sections { + name := strings.ToLower(strings.TrimSpace(s.Name)) + if name == "preamble" || strings.HasPrefix(name, "consequences") || strings.HasPrefix(name, "references") || strings.TrimSpace(s.Text) == "" { + continue + } + fmt.Fprintf(&b, "## %s\n\n%s\n\n", s.Name, strings.TrimSpace(s.Text)) + } + return strings.TrimSpace(b.String()), true +} + +// sourceName is how a question names what it hands the judgement. +func sourceName(whole bool) string { + if whole { + return "The record's sections" + } + return "The Decision section" +} + +// DraftQuestion is the first judgement's question: the record's Decision section — or, where it has none, its +// sections — and nothing else of the mesh. +func DraftQuestion(r *Record, section, returned string, whole bool) string { + var b strings.Builder + if whole { + fmt.Fprintf(&b, "Below are the sections of a decision record, %s: %q. It has no Decision section: what it decides is written "+ + "as prose across its sections. Write what it decides as numbered clauses: each one statement, one sentence, with a lead of a "+ + "few words naming what is decided. A clause says nothing the sections do not say: no inference, no context from elsewhere, "+ + "nothing added. Every decision the sections make is in one clause; a sentence that only explains, or describes what was, is "+ + "in none. Keep the record's own words where you can.\n", r.Key(), r.Title) + if returned != "" { + fmt.Fprintf(&b, "\nAn earlier draft was sent back, with this reason: %s\n", returned) + } + b.WriteString("\n" + sourceName(true) + ":\n\n" + section) + return b.String() + } fmt.Fprintf(&b, "Below is the Decision section of a decision record, %s: %q. It is written as prose. Write what it decides as numbered "+ "clauses: each one statement, one sentence, with a lead of a few words naming what is decided. A clause says nothing the section "+ "does not say: no inference, no context from elsewhere, nothing added. Every decision the section makes is in one clause; a "+ @@ -93,12 +134,16 @@ func DraftQuestion(r *Record, section, returned string) string { } // ConfirmQuestion is the second judgement's question: the record and the drafts, nothing else. -func ConfirmQuestion(r *Record, section string, drafts []Clause) string { +func ConfirmQuestion(r *Record, section string, drafts []Clause, whole bool) string { var b strings.Builder - fmt.Fprintf(&b, "Below are the Decision section of a decision record, %s: %q, and clauses drafted from it by someone else. For each "+ - "clause, answer whether the section says exactly that: exact is true only when the clause says nothing the section does not, "+ - "and leaves out nothing that changes its meaning. When it is not exact, say in reason what the section says instead.\n\n"+ - "The Decision section:\n\n%s\n\nThe drafted clauses:\n\n", r.Key(), r.Title, section) + what := "the Decision section of a decision record" + if whole { + what = "the sections of a decision record that has no Decision section" + } + fmt.Fprintf(&b, "Below are %s, %s: %q, and clauses drafted from it by someone else. For each "+ + "clause, answer whether the text says exactly that: exact is true only when the clause says nothing the text does not, "+ + "and leaves out nothing that changes its meaning. When it is not exact, say in reason what the text says instead.\n\n"+ + "%s:\n\n%s\n\nThe drafted clauses:\n\n", what, r.Key(), r.Title, sourceName(whole), section) for _, c := range drafts { fmt.Fprintf(&b, "%d. %s\n", c.N, c.Text) } @@ -155,10 +200,15 @@ func (k *KB) draftOne(ctx context.Context, key Key) (string, error) { return "", err } r := e.Current.Clone() - section, ok := decisionSection(r) - if !ok { - r.Drafting = &Drafting{State: DraftDone, Reason: "it has no Decision section", At: k.now()} - return k.saveDraft(ctx, e, r, EventDrafted, "no Decision section to draft from") + section, whole := draftSource(r) + if section == "" || len(section) > MostDraftSource { + why := fmt.Sprintf("no Decision section, and its sections hold %d bytes of what it decides, more than one judgement takes (%d): "+ + "it has no clauses", len(section), MostDraftSource) + if section == "" { + why = "no Decision section, and no section saying what it decides: it has no clauses" + } + r.Drafting = &Drafting{State: DraftDone, Reason: why, At: k.now()} + return k.saveDraft(ctx, e, r, EventDrafted, why) } model := k.judgeModel() fail := func(step string, err error) (string, error) { @@ -177,7 +227,7 @@ func (k *KB) draftOne(ctx context.Context, key Key) (string, error) { // A failed record goes back to the step it failed at: confirming when it holds drafts, drafting otherwise. confirming := r.Drafting.State == DraftDrafted || (r.Drafting.State == DraftFailed && hasDrafts(r)) if !confirming { - answer, by, err := k.Judge(DraftQuestion(r, section, r.Drafting.Returned), draftSchema, model) + answer, by, err := k.Judge(DraftQuestion(r, section, r.Drafting.Returned, whole), draftSchema, model) if err != nil { return fail("drafting", err) } @@ -202,7 +252,7 @@ func (k *KB) draftOne(ctx context.Context, key Key) (string, error) { r.Drafting = &Drafting{State: DraftDrafted, Drafter: by, Returned: r.Drafting.Returned, Tries: r.Drafting.Tries, At: k.now()} return k.saveDraft(ctx, e, r, EventDrafted, fmt.Sprintf("%d clauses drafted by %s", len(r.Clauses), by)) } - answer, by, err := k.Judge(ConfirmQuestion(r, section, r.Clauses), confirmSchema, model) + answer, by, err := k.Judge(ConfirmQuestion(r, section, r.Clauses, whole), confirmSchema, model) if err != nil { return fail("confirming", err) } @@ -338,13 +388,14 @@ func (k *KB) judgeChange(ctx context.Context, from Key, i int) (string, error) { return "", nil // the older record's clauses are not confirmed yet: judged once they are } model := k.judgeModel() - newer, _ := decisionSection(r) + newer, whole := draftSource(r) var b strings.Builder - fmt.Fprintf(&b, "%s says it replaces part of an older record, %s. Below are the clauses of %s and the Decision section of %s.\n\n", r.Key(), older.Key(), older.Key(), r.Key()) + fmt.Fprintf(&b, "%s says it replaces part of an older record, %s. Below are the clauses of %s and %s of %s.\n\n", r.Key(), older.Key(), older.Key(), + strings.ToLower(sourceName(whole)), r.Key()) for _, c := range older.Clauses { fmt.Fprintf(&b, "%d. %s\n", c.N, c.Text) } - fmt.Fprintf(&b, "\nThe Decision section of %s:\n\n%s\n", r.Key(), newer) + fmt.Fprintf(&b, "\n%s of %s:\n\n%s\n", sourceName(whole), r.Key(), newer) if l.Source == LinkPending { answer, by, err := k.Judge(b.String()+"\nWhich of the older record's clauses does the newer record replace part of? Name their numbers.", whichSchema, model) if err != nil { diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/fields_test.go b/modules/mesh-knowledge/cmd/mesh-knowledge/fields_test.go new file mode 100644 index 00000000..9c13b870 --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/fields_test.go @@ -0,0 +1,351 @@ +package main + +import ( + "context" + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +// hq issue 487: the import held every record of hq at f08920199d25 and lost some of what they say — `reconstructed`, +// `amended`, a research effort's `touches`, `rests-on` as its own link — took clauses from a numbered list inside a +// `###` subsection of the Decision section (ADR 0259, 0272, 0273, 0278, 0288, and 0040, 0114, 0148 the same way), +// and left the 11 records with no Decision section clause-less with no drafting queued. The fixture is those real +// records, copied from hq at that commit (testdata/hq-f0892019/COMMIT). + +func at487() dirRecords { return dirRecords{dir: filepath.Join("testdata", "hq-f0892019")} } + +func imported487(t *testing.T, s Store) *world { + t.Helper() + w := newWorld(t, s, at487()) + out, err := w.k.Import("import@test", ImportAsk{}, at487()) + if err != nil { + t.Fatal(err) + } + if out.Refused != 0 || out.Created != 13 { + t.Fatalf("%s %+v", out.Said, out.Rows) + } + return w +} + +func linkTo(r *Record, kind, to string) *Link { + for i, l := range r.Links { + if l.Kind == kind && l.To.String() == to { + return &r.Links[i] + } + } + return nil +} + +func TestTheImportHoldsEveryFieldOfARecord(t *testing.T) { + eachStore(t, func(t *testing.T, s Store) { + w := imported487(t, s) + get := func(k Key) *Record { return mustGet(t, s, k).Current } + + // reconstructed: true and false are both held, as a field of their own. + if r := get(Key{KindDecision, 2}); r.Reconstructed == nil || !*r.Reconstructed || r.Extra["reconstructed"] != "" { + t.Fatalf("0002 reconstructed: %v, extra %v", r.Reconstructed, r.Extra) + } + if r := get(Key{KindDecision, 4}); r.Reconstructed == nil || *r.Reconstructed { + t.Fatalf("0004 reconstructed: %v", r.Reconstructed) + } + // amended: ADR 0085's day. + if r := get(Key{KindDecision, 85}); len(r.Amended) != 1 || r.Amended[0] != "2026-09-20" || r.Extra["amended"] != "" { + t.Fatalf("0085 amended: %v, extra %v", r.Amended, r.Extra) + } + // rests-on: its own typed link, not a citation. + for _, n := range []int{37, 38} { + r := get(Key{KindDecision, n}) + if linkTo(r, LinkRestsOn, "ADR 0009") == nil || linkTo(r, LinkCites, "ADR 0009") != nil || r.Extra["rests-on"] != "" { + t.Fatalf("%04d's links: %v, extra %v", n, r.Links, r.Extra) + } + } + // touches: a link where it names a record, an issue or a design; text otherwise, in order. + r21 := get(Key{KindResearch, 21}) + if linkTo(r21, LinkTouches, "design 03-DESIGN/01-to-be/34-the-console.md") == nil || + strings.Join(r21.Touches, "|") != "the console|the tool runtime|seats|assignments" || r21.Extra["touches"] != "" { + t.Fatalf("research 021 touches: links %v, text %v, extra %v", r21.Links, r21.Touches, r21.Extra) + } + r30 := get(Key{KindResearch, 30}) + for _, to := range []string{"issue 242", "ADR 0030", "ADR 0053"} { + if linkTo(r30, LinkTouches, to) == nil { + t.Errorf("research 030 does not touch %s: %v", to, r30.Links) + } + } + // get serves them, and links reads them from both ends. + g, err := w.k.Get(GetAsk{Kind: KindResearch, Number: 21, Bound: 16 << 10}) + if err != nil { + t.Fatal(err) + } + if strings.Join(g.Touches, "|") != "the console|the tool runtime|seats|assignments" || !contains(g.Links, "touches design 03-DESIGN/01-to-be/34-the-console.md") { + t.Fatalf("get of research 021: %+v", g) + } + g2, _ := w.k.Get(GetAsk{Number: 2}) + g85, _ := w.k.Get(GetAsk{Number: 85}) + if g2.Reconstructed == nil || !*g2.Reconstructed || len(g85.Amended) != 1 { + t.Fatalf("get: 0002 reconstructed %v, 0085 amended %v", g2.Reconstructed, g85.Amended) + } + g37, _ := w.k.Get(GetAsk{Number: 37}) + if !contains(g37.Links, "rests-on ADR 0009") { + t.Fatalf("get of 0037: %v", g37.Links) + } + in, _ := s.Inbound(context.Background(), Key{KindDecision, 30}) + found := false + for _, l := range in { + found = found || (l.From == Key{KindResearch, 30} && l.Link.Kind == LinkTouches) + } + if !found { + t.Fatalf("0030 does not read research 030's touch from its end: %+v", in) + } + }) +} + +// Clauses are taken only from the Decision section's own top-level numbered list with bold leads, never from a list +// inside a `###` subsection of it. +func TestClausesComeOnlyFromTheDecisionSectionsOwnList(t *testing.T) { + eachStore(t, func(t *testing.T, s Store) { + imported487(t, s) + for _, n := range []int{259, 272, 273, 278, 288, 40} { + r := mustGet(t, s, Key{KindDecision, n}).Current + if len(r.Clauses) != 0 || r.Drafting == nil || r.Drafting.State != DraftWaiting { + t.Errorf("ADR %04d: %d clause(s) %v, drafting %+v", n, len(r.Clauses), r.Clauses, r.Drafting) + } + } + }) + // A Decision section with its own list, then a subsection with a list of its own: only the first is clauses. + section := "Some words.\n\n1. **The first.** It holds.\n And goes on.\n2. **The second.** It holds too.\n\n" + + "### A subsection\n\n1. **Not a clause.** A step of the subsection.\n2. **Nor this.** Another.\n" + got := WrittenClauses(section) + if len(got) != 2 || !strings.HasPrefix(got[0].Text, "**The first.**") || !strings.HasPrefix(got[1].Text, "**The second.**") { + t.Fatalf("%+v", got) + } + // A list inside a fenced block is not the section's either. + if got := WrittenClauses("```\n### not a heading\n```\n1. **One.** x\n"); len(got) != 1 { + t.Fatalf("a fence: %+v", got) + } +} + +// A record with no Decision section (11 in hq) is queued for drafting from what it decides, never silently +// clause-less; one too large to draft from in one judgement is marked as having none. +func TestARecordWithNoDecisionSectionIsDraftedFromWhatItDecides(t *testing.T) { + eachStore(t, func(t *testing.T, s Store) { + w := imported487(t, s) + for _, n := range []int{4, 37, 38} { + r := mustGet(t, s, Key{KindDecision, n}).Current + if r.Drafting == nil || r.Drafting.State != DraftWaiting || !strings.Contains(r.Drafting.Reason, "no Decision section") { + t.Fatalf("ADR %04d: %+v", n, r.Drafting) + } + } + k := Key{KindDecision, 4} + r := draftUntil(t, w, k, DraftDone) + if len(r.Clauses) == 0 || r.Clauses[0].Source != SourceConfirmed { + t.Fatalf("0004's clauses: %+v", r.Clauses) + } + var drafting string + for _, q := range w.judge.questions { + if strings.Contains(q, "ADR 0004") && strings.Contains(q, "no Decision section") { + drafting = q + break + } + } + if drafting == "" || !strings.Contains(drafting, "## The link is the security boundary") || strings.Contains(drafting, "## Consequences") { + t.Fatalf("0004 was drafted from: %.400s", drafting) + } + }) + // Too large to draft from in one judgement: marked, said, never clause-less in silence. + w := newWorld(t, newMemStore(), at487()) + r := &Record{Kind: KindDecision, Number: 9, Title: "huge", Status: StatusAccepted, Topic: "the mesh", + Sections: []Section{{Name: "What it is", Text: strings.Repeat("A node is a machine the mesh adopted. ", 2000)}}, + Drafting: &Drafting{State: DraftWaiting, Reason: NoDecisionSection}} + if err := w.k.Store.Create(context.Background(), Version{By: "test", Why: "made", Record: r}); err != nil { + t.Fatal(err) + } + if _, err := w.k.DraftStep(); err != nil { + t.Fatal(err) + } + got := mustGet(t, w.k.Store, Key{KindDecision, 9}).Current + if got.Drafting.State != DraftDone || !strings.Contains(got.Drafting.Reason, "no Decision section") || len(w.judge.questions) != 0 { + t.Fatalf("%+v", got.Drafting) + } +} + +// legacy is a record as the import before hq issue 487 wrote it: clauses from any numbered list of the Decision +// section, reconstructed, amended and touches only as unserved text, rests-on as a citation, and no drafting for a +// record with no Decision section. +func legacy(t *testing.T, path string, commit string) *Record { + t.Helper() + var p *ParsedRecord + var err error + if strings.HasPrefix(path, DecisionsFolder) { + p, err = ParseDecision(path, readFile(t, filepath.Join("testdata", "hq-f0892019", filepath.FromSlash(path)))) + } else { + docs := map[string]string{} + entries, _ := os.ReadDir(filepath.Join("testdata", "hq-f0892019", filepath.FromSlash(path))) + for _, e := range entries { + docs[e.Name()] = readFile(t, filepath.Join("testdata", "hq-f0892019", filepath.FromSlash(path), e.Name())) + } + p, err = ParseResearch(filepath.Base(path), docs) + } + if err != nil { + t.Fatal(err) + } + r := p.Record + if r.Extra == nil { + r.Extra = map[string]string{} + } + if r.Reconstructed != nil { + r.Extra["reconstructed"] = map[bool]string{true: "true", false: "false"}[*r.Reconstructed] + } + if len(r.Amended) > 0 { + r.Extra["amended"] = strings.Join(r.Amended, ", ") + } + r.Reconstructed, r.Amended = nil, nil + var links []Link + var touched []string + for _, l := range r.Links { + switch l.Kind { + case LinkRestsOn: + l.Kind = LinkCites + case LinkTouches: + touched = append(touched, l.To.String()) + continue + } + links = append(links, l) + } + r.Links = links + if len(touched)+len(r.Touches) > 0 { + r.Extra["touches"] = strings.Join(append(touched, r.Touches...), ", ") + } + r.Touches = nil + if r.Kind == KindDecision { + r.Drafting = nil + if s, ok := decisionSection(r); ok { + r.Clauses = writtenClauses(s) + for i := range r.Clauses { + r.Clauses[i].Source, r.Clauses[i].Status = SourceWritten, clauseStatusOf(r.Status) + } + if len(r.Clauses) == 0 { + r.Drafting = &Drafting{State: DraftWaiting} + } + } + } + r.Source = "hq@" + commit + ":" + path + return r +} + +func TestARefreshRepairsOnlyRecordsNobodyChangedSinceTheImportAndChangesNothingTwice(t *testing.T) { + eachStore(t, func(t *testing.T, s Store) { + ctx := context.Background() + w := newWorld(t, s, at487()) + commit, _ := at487().Standing() + paths := []string{} + for _, dir := range []string{DecisionsFolder, ResearchFolder} { + entries, _ := os.ReadDir(filepath.Join("testdata", "hq-f0892019", dir)) + for _, e := range entries { + paths = append(paths, dir+"/"+e.Name()) + } + } + at := time.Date(2026, 10, 11, 9, 35, 7, 0, time.UTC) + for _, p := range paths { + r := legacy(t, p, commit) + if err := s.Create(ctx, Version{By: Unproven, At: at, Why: "imported from hq at " + short(commit) + ": " + p, Record: r}); err != nil { + t.Fatal(err) + } + } + // ADR 0085 was drafted since its import: someone changed it, and the refresh never touches it. + e85 := mustGet(t, s, Key{KindDecision, 85}) + d := e85.Current.Clone() + d.Clauses = []Clause{{N: 1, Text: "**Secret-holding is a module.** It is.", Status: ClauseInForce, Source: SourceDrafted}} + d.Drafting = &Drafting{State: DraftDrafted} + if err := s.Save(ctx, Version{By: "the agent on test (sonnet)", At: at, Why: "1 clauses drafted", Record: d}, true, false, e85.Revision); err != nil { + t.Fatal(err) + } + before := countVersions(t, s) + + // The dry run says what it would change, and writes nothing. + dry, err := w.k.Import("import@test", ImportAsk{DryRun: true, Refresh: true}, at487()) + if err != nil { + t.Fatal(err) + } + if countVersions(t, s) != before || dry.Refreshed != 12 || dry.ChangedSince != 1 || dry.Created != 0 { + t.Fatalf("dry run: %s", dry.Said) + } + says := map[string]ImportRow{} + for _, row := range dry.Rows { + says[row.Record] = row + } + if row := says["ADR 0259"]; row.Action != ImportWouldRefresh || !strings.Contains(row.Why, "clauses") { + t.Fatalf("0259 in the dry run: %+v", row) + } + if row := says["ADR 0002"]; !strings.Contains(row.Why, "reconstructed") { + t.Fatalf("0002 in the dry run: %+v", row) + } + if row := says["ADR 0085"]; row.Action != ImportChangedSince { + t.Fatalf("0085 in the dry run: %+v", row) + } + + out, err := w.k.Import("import@test", ImportAsk{Refresh: true, Commit: commit[:12]}, at487()) + if err != nil { + t.Fatal(err) + } + if out.Refreshed != 12 || out.ChangedSince != 1 || countVersions(t, s) != before+12 { + t.Fatalf("refresh: %s", out.Said) + } + e := mustGet(t, s, Key{KindDecision, 259}) + vs, _ := s.Versions(ctx, Key{KindDecision, 259}) + if e.Versions != 2 || len(e.Current.Clauses) != 0 || e.Current.Drafting == nil || e.Current.Drafting.State != DraftWaiting || + !strings.HasPrefix(vs[1].Why, "refreshed from hq at "+short(commit)) || !strings.Contains(vs[1].Why, "clauses") { + t.Fatalf("0259 after the refresh: %+v, %q", e.Current.Clauses, vs[1].Why) + } + if r := mustGet(t, s, Key{KindDecision, 2}).Current; r.Reconstructed == nil || !*r.Reconstructed { + t.Fatal("0002 not repaired") + } + if r := mustGet(t, s, Key{KindDecision, 37}).Current; linkTo(r, LinkRestsOn, "ADR 0009") == nil || r.Drafting == nil { + t.Fatalf("0037 not repaired: %v %+v", r.Links, r.Drafting) + } + if r := mustGet(t, s, Key{KindResearch, 21}).Current; len(r.Touches) != 4 { + t.Fatalf("research 021 not repaired: %v", r.Touches) + } + if e := mustGet(t, s, Key{KindDecision, 85}); e.Versions != 2 || len(e.Current.Clauses) != 1 { + t.Fatalf("0085, changed since its import, was touched: %d versions", e.Versions) + } + // Run again: nothing changes, and it says so. + after := countVersions(t, s) + again, err := w.k.Import("import@test", ImportAsk{Refresh: true}, at487()) + if err != nil { + t.Fatal(err) + } + if again.Refreshed != 0 || again.Unchanged != 12 || countVersions(t, s) != after || !strings.Contains(again.Said, "Nothing changed") { + t.Fatalf("a second refresh: %s", again.Said) + } + // Once drafted, a repaired record is someone's: a later refresh leaves it alone. + draftUntil(t, w, Key{KindDecision, 259}, DraftDone) + third, _ := w.k.Import("import@test", ImportAsk{Refresh: true}, at487()) + changed := false + for _, row := range third.Rows { + changed = changed || (row.Record == "ADR 0259" && row.Action == ImportChangedSince) + } + if third.Refreshed != 0 || !changed || mustGet(t, s, Key{KindDecision, 259}).Current.Drafting.State != DraftDone { + t.Fatalf("a third refresh: %s", third.Said) + } + }) +} + +// A refresh without a commit's records refuses, and an import without refresh never rewrites what it holds. +func TestAnImportWithoutRefreshLeavesAHeldRecordAsItIs(t *testing.T) { + w := newWorld(t, newMemStore(), at487()) + commit, _ := at487().Standing() + p := DecisionsFolder + "/0259-an-ask-is-a-message-on-the-bus-any-channel-answers-it-and-the-asker-acts-on-a-warrant.md" + if err := w.k.Store.Create(context.Background(), Version{By: Unproven, Why: "imported from hq at " + short(commit) + ": " + p, Record: legacy(t, p, commit)}); err != nil { + t.Fatal(err) + } + out, err := w.k.Import("import@test", ImportAsk{}, at487()) + if err != nil { + t.Fatal(err) + } + if out.Held != 1 || out.Refreshed != 0 || len(mustGet(t, w.k.Store, Key{KindDecision, 259}).Current.Clauses) != 7 { + t.Fatalf("%s", out.Said) + } +} diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/import.go b/modules/mesh-knowledge/cmd/mesh-knowledge/import.go index 10973c07..aac7e26d 100644 --- a/modules/mesh-knowledge/cmd/mesh-knowledge/import.go +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/import.go @@ -9,6 +9,7 @@ package main import ( "context" + "encoding/json" "errors" "fmt" "regexp" @@ -29,10 +30,13 @@ const LinkPending = "pending" var commitPrefix = regexp.MustCompile(`^[0-9a-f]{7,40}$`) -// ImportAsk is what an import takes. +// ImportAsk is what an import takes. Refresh also repairs a record held whose only versions are the imports' own +// (hq issue 487): it is read again at the commit and written as a new version naming what was lost and why. A +// record anyone changed after its import is never touched. type ImportAsk struct { - Commit string - DryRun bool + Commit string + DryRun bool + Refresh bool } // ImportRow is what the import did, or would do, with one record. @@ -50,24 +54,33 @@ const ( ImportHeld = "held" ImportRefused = "refused" ImportWould = "would-create" + // A refresh's: a record repaired, or one that would be; and one changed here since its import, left alone. + ImportRefresh = "refresh" + ImportWouldRefresh = "would-refresh" + ImportChangedSince = "changed-since" ) // Imported is the answer to an import. type Imported struct { - DryRun bool `json:"dry-run"` - Commit string `json:"commit"` - Read int `json:"read"` - Created int `json:"created"` - Held int `json:"held"` - Refused int `json:"refused"` - Clauses int `json:"written-clauses"` - ToDraft int `json:"prose-decisions"` - Rows []ImportRow `json:"rows"` - Odd []string `json:"odd,omitempty"` - Cut string `json:"cut,omitempty"` - Said string `json:"said"` - Findings []Finding `json:"whole-set-findings,omitempty"` - Unresolved int `json:"unresolved-links"` + DryRun bool `json:"dry-run"` + Commit string `json:"commit"` + Read int `json:"read"` + Created int `json:"created"` + Held int `json:"held"` + Refused int `json:"refused"` + // A refresh's counts: repaired (or would be), read again and found the same, and changed here since their + // import, which a refresh never touches. + Refreshed int `json:"refreshed,omitempty"` + Unchanged int `json:"unchanged,omitempty"` + ChangedSince int `json:"changed-since,omitempty"` + Clauses int `json:"written-clauses"` + ToDraft int `json:"prose-decisions"` + Rows []ImportRow `json:"rows"` + Odd []string `json:"odd,omitempty"` + Cut string `json:"cut,omitempty"` + Said string `json:"said"` + Findings []Finding `json:"whole-set-findings,omitempty"` + Unresolved int `json:"unresolved-links"` } // MostImportRows is how many rows an import answers; the counts are always whole. @@ -199,9 +212,9 @@ func (k *KB) Import(caller string, a ImportAsk, records Records) (*Imported, err if err != nil { return nil, fmt.Errorf("the knowledge base cannot be read (%v); nothing was imported", err) } - held := map[Key]bool{} + held := map[Key]*Entry{} for _, e := range all { - held[e.Current.Key()] = true + held[e.Current.Key()] = e known[e.Current.Key()] = true } at := k.now() @@ -219,22 +232,31 @@ func (k *KB) Import(caller string, a ImportAsk, records Records) (*Imported, err row.Record, row.Status = r.Key().String(), r.Status out.Read++ out.Odd = append(out.Odd, c.parsed.Odd...) - if held[r.Key()] { - row.Action = ImportHeld - out.Held++ - continue - } for i, l := range r.Links { if to, ok := l.To.Record(); ok && !known[to] { r.Links[i].Unresolved = true - out.Unresolved++ - out.Odd = append(out.Odd, fmt.Sprintf("%s %s %s, which hq does not hold at this commit", r.Key(), l.Kind, to)) + if held[r.Key()] == nil { + out.Unresolved++ + out.Odd = append(out.Odd, fmt.Sprintf("%s %s %s, which hq does not hold at this commit", r.Key(), l.Kind, to)) + } } if l.Kind == LinkChanges && l.To.Clause == 0 { r.Links[i].Source = LinkPending } } r.Source = "hq@" + commit + ":" + c.path + if e := held[r.Key()]; e != nil { + if !a.Refresh { + row.Action = ImportHeld + out.Held++ + continue + } + k.refresh(ctx, a, e, r, c.path, commit, caller, &row, out) + if row.Action != ImportHeld { + rows = append(rows, row) + } + continue + } out.Clauses += len(r.Clauses) if r.Drafting != nil { out.ToDraft++ @@ -267,23 +289,194 @@ func (k *KB) Import(caller string, a ImportAsk, records Records) (*Imported, err if len(out.Odd) > 40 { out.Odd = append(out.Odd[:40], fmt.Sprintf("and %d more", len(out.Odd)-40)) } - verb := "created" + verb, again := "created", "refreshed" if a.DryRun { - verb = "would create" + verb, again = "would create", "would refresh" } out.Said = fmt.Sprintf("Read %d records of hq at commit %s: %s %d, already held %d, refused %d; %d clauses written as such, "+ "%d decisions written as prose wait for their clauses to be drafted and confirmed.", - out.Read, short(commit), verb, out.Created, out.Held, out.Refused, out.Clauses, out.ToDraft) + out.Read, short(commit), verb, out.Created, out.Held+out.Refreshed+out.Unchanged+out.ChangedSince, out.Refused, out.Clauses, out.ToDraft) + if a.Refresh { + out.Said += fmt.Sprintf(" Of those held: %s %d that only the import ever wrote, %d read again and the same, %d changed here "+ + "since their import and left as they are.", again, out.Refreshed, out.Unchanged, out.ChangedSince) + } switch { case a.DryRun: out.Said += " Nothing was written: a dry run." - case out.Created == 0 && out.Refused == 0: - out.Said += " Nothing changed: every record is already held." - case out.Created == 0: + case out.Created == 0 && out.Refreshed == 0 && out.Refused == 0: + out.Said += " Nothing changed: every record is already held as hq says it." + case out.Created == 0 && out.Refreshed == 0: out.Said += " Nothing changed: every record not refused is already held." } - if !a.DryRun && out.Created > 0 { + if !a.DryRun && out.Created+out.Refreshed > 0 { out.Findings = k.judgeSet(ctx) } return out, nil } + +// importedOnly says whether every version of a record was written by the import or a refresh of it: nobody has +// changed it since it was read from hq. +func importedOnly(vs []Version) bool { + if len(vs) == 0 { + return false + } + for _, v := range vs { + if v.Insight || v.Record == nil || !strings.HasPrefix(v.Record.Source, "hq@") || + !(strings.HasPrefix(v.Why, "imported from hq at ") || strings.HasPrefix(v.Why, "refreshed from hq at ")) { + return false + } + } + return true +} + +// refresh reads a held record again (r, as hq has it at the commit) and, when only the import ever wrote it and +// what it holds differs, writes r as its new version naming what changed and why. A record changed here since +// its import is never touched; a second refresh finds the same and changes nothing. +func (k *KB) refresh(ctx context.Context, a ImportAsk, e *Entry, r *Record, path, commit, caller string, row *ImportRow, out *Imported) { + vs, err := k.Store.Versions(ctx, r.Key()) + if err != nil { + row.Action, row.Why = ImportRefused, "its versions cannot be read: "+err.Error() + out.Refused++ + return + } + if e.Pending != nil || !importedOnly(vs) { + last := vs[len(vs)-1] + row.Action = ImportChangedSince + row.Why = fmt.Sprintf("changed here since its import (version %d by %s: %s); a refresh never touches it", last.N, last.By, last.Why) + if e.Pending != nil { + row.Why = "a version waits for its review over it; a refresh never touches it" + } + out.ChangedSince++ + return + } + diffs := differences(e.Current, r) + if len(diffs) == 0 { + row.Action = ImportHeld + out.Unchanged++ + return + } + row.Why = strings.Join(diffs, "; ") + row.Action = ImportWouldRefresh + out.Refreshed++ + if a.DryRun { + return + } + why := fmt.Sprintf("refreshed from hq at %s: %s; the import before hq issue 487 had lost or misread %s", short(commit), path, row.Why) + if len(why) > 1000 { + why = why[:997] + "..." + } + if err := k.Store.Save(ctx, Version{By: caller, At: k.now(), Why: why, Record: r}, true, false, e.Revision); err != nil { + row.Action, row.Why = ImportRefused, "it could not be written: "+err.Error() + out.Refused++ + out.Refreshed-- + return + } + row.Action = ImportRefresh + k.emit(EventChanged, r, caller, why, e.Versions+1) +} + +// differences names what a record read again from hq holds that the one held does not, field by field; none when +// the two hold the same. +func differences(old, now *Record) []string { + var out []string + o, n := old.Clone(), now.Clone() + o.Source, n.Source = "", "" + if a, b := renderJSON(o), renderJSON(n); a == b { + return nil + } + field := func(name, a, b string) { + if a != b { + if a == "" { + a = "unset" + } + if b == "" { + b = "unset" + } + out = append(out, fmt.Sprintf("%s: %s → %s", name, a, b)) + } + } + boolOf := func(p *bool) string { + if p == nil { + return "" + } + return fmt.Sprint(*p) + } + field("title", o.Title, n.Title) + field("status", o.Status, n.Status) + field("topic", o.Topic, n.Topic) + field("date", o.Date, n.Date) + field("deciders", strings.Join(o.Deciders, ", "), strings.Join(n.Deciders, ", ")) + field("reconstructed", boolOf(o.Reconstructed), boolOf(n.Reconstructed)) + field("amended", strings.Join(o.Amended, ", "), strings.Join(n.Amended, ", ")) + field("touches", strings.Join(o.Touches, ", "), strings.Join(n.Touches, ", ")) + var gone []string + for key := range o.Extra { + if _, ok := n.Extra[key]; !ok { + gone = append(gone, key) + } + } + sort.Strings(gone) + if len(gone) > 0 { + out = append(out, "held as a field of its own now: "+strings.Join(gone, ", ")) + } + if renderJSON(o.Clauses) != renderJSON(n.Clauses) { + s := fmt.Sprintf("clauses: %d → %d", len(o.Clauses), len(n.Clauses)) + if len(n.Clauses) == 0 && n.Drafting != nil { + s += ", to be drafted" + } + out = append(out, s) + } + if (o.Drafting == nil) != (n.Drafting == nil) || (n.Drafting != nil && o.Drafting != nil && o.Drafting.Reason != n.Drafting.Reason) { + if n.Drafting != nil { + s := "drafting queued" + if n.Drafting.Reason != "" { + s += " (" + n.Drafting.Reason + ")" + } + out = append(out, s) + } else { + out = append(out, "no drafting: its clauses are written") + } + } + ol, nl := map[string]bool{}, map[string]bool{} + for _, l := range o.Links { + ol[l.String()] = true + } + for _, l := range n.Links { + nl[l.String()] = true + } + var added, removed []string + for l := range nl { + if !ol[l] { + added = append(added, l) + } + } + for l := range ol { + if !nl[l] { + removed = append(removed, l) + } + } + sort.Strings(added) + sort.Strings(removed) + if len(added)+len(removed) > 0 { + s := "links:" + if len(added) > 0 { + s += " + " + strings.Join(added, ", + ") + } + if len(removed) > 0 { + s += " − " + strings.Join(removed, ", − ") + } + out = append(out, s) + } + if renderJSON(o.Sections) != renderJSON(n.Sections) { + out = append(out, "its sections") + } + if len(out) == 0 { + out = append(out, "what it holds otherwise (as hq writes it)") + } + return out +} + +func renderJSON(v any) string { + raw, _ := json.Marshal(v) + return string(raw) +} diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/kit_test.go b/modules/mesh-knowledge/cmd/mesh-knowledge/kit_test.go index 98cea051..0330b42e 100644 --- a/modules/mesh-knowledge/cmd/mesh-knowledge/kit_test.go +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/kit_test.go @@ -271,7 +271,11 @@ func (j *judge) judge(question string, schema map[string]any, model string) (jso case props["exact"] != nil: return json.RawMessage(`{"exact":true,"reason":"it does"}`), "the agent on test (" + model + ")", nil } - section := question[strings.Index(question, "The Decision section:")+len("The Decision section:"):] + marker := "The Decision section:" + if !strings.Contains(question, marker) { + marker = "The record's sections:" + } + section := question[strings.Index(question, marker)+len(marker):] var clauses []map[string]string for _, s := range regexp.MustCompile(`[^.]+\.`).FindAllString(strings.Join(strings.Fields(section), " "), -1) { s = strings.TrimSpace(s) diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/main.go b/modules/mesh-knowledge/cmd/mesh-knowledge/main.go index 62564673..70f92520 100644 --- a/modules/mesh-knowledge/cmd/mesh-knowledge/main.go +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/main.go @@ -881,12 +881,15 @@ func Tools(k *KB, caller func(map[string]any) string) []stdio.Tool { Description: "Import hq's decision records and research efforts, read through the records module at the commit its checkout " + "stands at, each whole at its own number as its first version naming that commit. It only creates a record that is not " + "held, never overwrites one, never moves a number counter, and run again changes nothing and says so. A decision written " + - "as prose waits for its clauses to be drafted and confirmed. With dry-run it writes nothing and says what it would do; " + - "with commit it refuses a checkout standing elsewhere.", + "as prose, or with no Decision section, waits for its clauses to be drafted and confirmed. With refresh it also reads " + + "again every record it holds that nobody changed since its import, and writes what it lost or misread as a new version " + + "naming why; a record changed since is never touched, and a second refresh changes nothing. With dry-run it writes " + + "nothing and says what it would do, record by record; with commit it refuses a checkout standing elsewhere.", Input: schema(nil, map[string]any{"dry-run": flag("read everything, write nothing, say what would be done"), - "commit": text("the commit of hq the record must stand at (a prefix is enough)")}), + "commit": text("the commit of hq the record must stand at (a prefix is enough)"), + "refresh": flag("also repair what it holds that only the import ever wrote, from hq at this commit")}), Run: func(m map[string]any) (any, error) { - a, err := readArgs("import", m, "dry-run", "commit") + a, err := readArgs("import", m, "dry-run", "commit", "refresh") if err != nil { return nil, err } @@ -894,6 +897,9 @@ func Tools(k *KB, caller func(map[string]any) string) []stdio.Tool { if ask.DryRun, err = a.flag("dry-run"); err != nil { return nil, err } + if ask.Refresh, err = a.flag("refresh"); err != nil { + return nil, err + } if ask.Commit, err = a.str("commit"); err != nil { return nil, err } diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/parse.go b/modules/mesh-knowledge/cmd/mesh-knowledge/parse.go index dc6bfe26..6b9260e7 100644 --- a/modules/mesh-knowledge/cmd/mesh-knowledge/parse.go +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/parse.go @@ -107,9 +107,32 @@ func splitSections(body string) []Section { return out } -// WrittenClauses are the numbered statements with a bold lead of a Decision section, each with the lines that -// continue it, in order. None when the section is prose. -func WrittenClauses(section string) []Clause { +// WrittenClauses are the numbered statements with a bold lead of a Decision section's own top-level list, each +// with the lines that continue it, in order: never a list inside a `###` subsection of it, which is a part of +// that subsection's prose (ADR 0259, 0272, 0273, 0278 and 0288 number the steps of a subsection so; hq issue +// 487). None when the section is prose. +func WrittenClauses(section string) []Clause { return writtenClauses(ownText(section)) } + +// subsection is a heading below the section's own level: `###` and deeper. +var subsection = regexp.MustCompile(`^#{3,}\s`) + +// ownText is a section's text before its first subsection, outside a fenced block: what the section says itself. +func ownText(section string) string { + lines := strings.Split(section, "\n") + fence := false + for i, line := range lines { + if strings.HasPrefix(strings.TrimSpace(line), "```") { + fence = !fence + } + if !fence && subsection.MatchString(line) { + return strings.Join(lines[:i], "\n") + } + } + return section +} + +// writtenClauses are the numbered statements with a bold lead at the margin of a text, as records.py reads them. +func writtenClauses(section string) []Clause { lines := strings.Split(section, "\n") var out []Clause var cur []string @@ -187,7 +210,8 @@ func ParseDecision(file, text string) (*ParsedRecord, error) { after := body[strings.Index(body, h[0])+len(h[0]):] r.Sections = splitSections(after) known := map[string]bool{"topic": true, "status": true, "date": true, "deciders": true, "extends": true, - "supersedes": true, "superseded-by": true, "supersedes-in-part": true} + "supersedes": true, "superseded-by": true, "supersedes-in-part": true, "reconstructed": true, "amended": true, + "rests-on": true} keys := make([]string, 0, len(front)) for k := range front { keys = append(keys, k) @@ -198,6 +222,20 @@ func ParseDecision(file, text string) (*ParsedRecord, error) { r.Extra[k] = strings.Join(front[k], ", ") } } + if v, ok := front["reconstructed"]; ok { + switch strings.Join(v, ", ") { + case "true": + t := true + r.Reconstructed = &t + case "false": + f := false + r.Reconstructed = &f + default: + r.Extra["reconstructed"] = strings.Join(v, ", ") + p.Odd = append(p.Odd, fmt.Sprintf("%s says reconstructed %q, which is neither true nor false; it is kept as written", file, strings.Join(v, ", "))) + } + } + r.Amended = append(r.Amended, front["amended"]...) if len(r.Extra) == 0 { r.Extra = nil } @@ -222,16 +260,12 @@ func ParseDecision(file, text string) (*ParsedRecord, error) { if by := first(front, "superseded-by"); by != "" { r.Extra = addExtra(r.Extra, "superseded-by", by) } - for _, rest := range front["rests-on"] { - add(LinkCites, []string{rest}, "rests-on") - } + add(LinkRestsOn, front["rests-on"], "rests-on") r.Links = append(r.Links, citations(r, after)...) - // Clauses: written, where the Decision section writes them. - for _, s := range r.Sections { - if strings.HasPrefix(s.Name, "Decision") { - r.Clauses = WrittenClauses(s.Text) - break - } + // Clauses: written, where the Decision section's own list writes them. + section, hasDecision := decisionSection(r) + if hasDecision { + r.Clauses = WrittenClauses(section) } for i := range r.Clauses { r.Clauses[i].Source = SourceWritten @@ -240,8 +274,14 @@ func ParseDecision(file, text string) (*ParsedRecord, error) { p.Odd = append(p.Odd, fmt.Sprintf("%s numbers its clauses out of order (clause %d is the %d.)", file, r.Clauses[i].N, i+1)) } } - if len(r.Clauses) == 0 && r.Section("Decision") != "" { + // A record without written clauses is drafted: from its Decision section, or, where it has none (11 of hq's + // records), from what it decides across its sections. Never clause-less in silence. + switch { + case len(r.Clauses) > 0: + case hasDecision && strings.TrimSpace(section) != "": r.Drafting = &Drafting{State: DraftWaiting} + default: + r.Drafting = &Drafting{State: DraftWaiting, Reason: NoDecisionSection} } p.Record = r return p, nil @@ -324,10 +364,22 @@ func ParseResearch(folder string, docs map[string]string) (*ParsedRecord, error) p.Odd = append(p.Odd, folder+"/00-overview.md has no '# ' heading; the folder's name is its title") } for k, v := range front { - if k != "status" && k != "initiated" && k != "became" { + if k != "status" && k != "initiated" && k != "became" && k != "touches" { r.Extra[k] = strings.Join(v, ", ") } } + // What it touches: a link where it names a record, an issue or a design; as written otherwise. + touched := map[Ref]bool{} + for _, v := range front["touches"] { + if ref, ok := refOfPath(v); ok && !(ref.Kind == KindResearch && ref.Number == number) { + if !touched[ref] { + touched[ref] = true + r.Links = append(r.Links, Link{Kind: LinkTouches, To: ref, Source: SourceWritten}) + } + continue + } + r.Touches = append(r.Touches, v) + } if len(r.Extra) == 0 { r.Extra = nil } @@ -361,5 +413,27 @@ func ParseResearch(folder string, docs map[string]string) (*ParsedRecord, error) return p, nil } +// NoDecisionSection is the drafting reason of a decision record that has no Decision section. +const NoDecisionSection = "no Decision section: its clauses are drafted from what it decides across its sections" + +// refOfPath is the record, issue or design a path of hq names, if it names one. +func refOfPath(v string) (Ref, bool) { + v = strings.TrimSpace(v) + switch { + case issueFolder.MatchString(v): + n, _ := strconv.Atoi(issueFolder.FindStringSubmatch(v)[1]) + return Ref{Kind: TargetIssue, Number: n}, true + case researchFolder.MatchString(v): + n, _ := strconv.Atoi(researchFolder.FindStringSubmatch(v)[1]) + return Ref{Kind: KindResearch, Number: n}, true + case adrFile.MatchString(v) && (strings.Contains(v, DecisionsFolder) || !strings.Contains(v, "/")): + n, _ := strconv.Atoi(adrFile.FindStringSubmatch(v)[1]) + return Ref{Kind: KindDecision, Number: n}, true + case designPath.MatchString(v): + return Ref{Kind: TargetDesign, Path: designPath.FindString(v)}, true + } + return Ref{}, false +} + // docName is a research document's name within its folder. func docName(p string) string { return path.Base(p) } diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/record.go b/modules/mesh-knowledge/cmd/mesh-knowledge/record.go index 2569f65c..6faa8a49 100644 --- a/modules/mesh-knowledge/cmd/mesh-knowledge/record.go +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/record.go @@ -57,17 +57,21 @@ const ( SourceConfirmed = "confirmed" ) -// The kinds of a link (to-be 55 §2). +// The kinds of a link (to-be 55 §2), and the two hq's records write that it has no other word for (hq issue +// 487): a decision's `rests-on` (a record it stands on, which may not be superseded under it), and a research +// effort's `touches` (a record, issue or design its effort bears on). const ( LinkSupersedes = "supersedes" LinkChanges = "changes" LinkExtends = "extends" LinkBecame = "became" LinkCites = "cites" + LinkRestsOn = "rests-on" + LinkTouches = "touches" ) // LinkKinds are the kinds a link may be. -var LinkKinds = []string{LinkSupersedes, LinkChanges, LinkExtends, LinkBecame, LinkCites} +var LinkKinds = []string{LinkSupersedes, LinkChanges, LinkExtends, LinkBecame, LinkCites, LinkRestsOn, LinkTouches} // What a link may point at besides a record of this knowledge base: an issue in the tracker, a design in hq. const ( @@ -266,6 +270,14 @@ type Record struct { Status string `json:"status"` Date string `json:"date,omitempty"` Deciders []string `json:"deciders,omitempty"` + // Reconstructed says whether a decision was written down after it was made, from what was built (hq's + // `reconstructed`); nil where the record does not say. + Reconstructed *bool `json:"reconstructed,omitempty"` + // Amended are the days a decision record says it was amended on (hq's `amended`). + Amended []string `json:"amended,omitempty"` + // Touches are what a research effort bears on that is no record, issue or design — "the tool runtime" — as + // written; what is one is a touches link. + Touches []string `json:"touches,omitempty"` // Effort is the status a proposed research effort takes when it is accepted. Effort string `json:"effort,omitempty"` NeedsOperator bool `json:"needs-operator,omitempty"` @@ -406,6 +418,11 @@ func Render(r *Record) string { field("date", r.Date) field("deciders", strings.Join(r.Deciders, ", ")) field("effort", r.Effort) + if r.Reconstructed != nil { + field("reconstructed", strconv.FormatBool(*r.Reconstructed)) + } + field("amended", strings.Join(r.Amended, ", ")) + field("touches", strings.Join(r.Touches, ", ")) if r.NeedsOperator { field("needs operator", "yes") } diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/rules.go b/modules/mesh-knowledge/cmd/mesh-knowledge/rules.go index e3b62980..86c812b6 100644 --- a/modules/mesh-knowledge/cmd/mesh-knowledge/rules.go +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/rules.go @@ -232,6 +232,14 @@ func JudgeRecord(r *Record, held *Entry, lookup Lookup, ref *Reference) []Refusa if r.Kind != l.To.Kind || !isRecord || l.To.Clause > 0 { refuse("link-fits", "%s goes from a record to a record of its kind: not %s", l.Kind, l) } + case l.Kind == LinkRestsOn: + if !decision || l.To.Kind != KindDecision || l.To.Clause > 0 { + refuse("link-fits", "rests-on goes from a decision to a decision it stands on: not %s", l) + } + case l.Kind == LinkTouches: + if r.Kind != KindResearch || l.To.Clause > 0 { + refuse("link-fits", "touches goes from a research effort to a record, an issue or a design: not %s", l) + } } if target == nil { continue @@ -241,9 +249,9 @@ func JudgeRecord(r *Record, held *Entry, lookup Lookup, ref *Reference) []Refusa continue } switch l.Kind { - case LinkExtends: + case LinkExtends, LinkRestsOn: if target.Status == StatusSuperseded && !superseders[to] { - refuse("rests-on", "%s extends %s, which is superseded", self, to) + refuse("rests-on", "%s %s %s, which is superseded", self, l.Kind, to) } case LinkSupersedes: if !target.Accepted() || target.Status == StatusSuperseded { @@ -509,14 +517,14 @@ func JudgeSet(records []*Record) []Finding { if c := target.Clause(l.To.Clause); c != nil && c.Status != ClauseChanged && c.Status != ClauseSuperseded { add("partial", fmt.Sprintf("%s changes %s §%d, which is %s", k, to, l.To.Clause, c.Status), k, to) } - case LinkExtends: + case LinkExtends, LinkRestsOn: if target.Status == StatusSuperseded { superseder := false for _, in := range inbound[to] { superseder = superseder || (in.l.Kind == LinkSupersedes && in.from.Key() == k) } if !superseder { - add("rests-on-live", fmt.Sprintf("%s extends %s, which is superseded", k, to), k, to) + add("rests-on-live", fmt.Sprintf("%s %s %s, which is superseded", k, l.Kind, to), k, to) } } } diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/01-RESEARCH/021-finding-a-tool-in-the-mesh/00-overview.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/01-RESEARCH/021-finding-a-tool-in-the-mesh/00-overview.md new file mode 100644 index 00000000..3f8d7747 --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/01-RESEARCH/021-finding-a-tool-in-the-mesh/00-overview.md @@ -0,0 +1,48 @@ +--- +status: graduated +initiated: 2026-10-03 +touches: [the console, 03-DESIGN/01-to-be/34-the-console.md, the tool runtime, seats, assignments] +became: [02-DECISIONS/0195-the-meshs-tools-are-found-by-address-not-announced-whole.md, 03-DESIGN/01-to-be/34-the-console.md] +--- + +# 021 — Finding a tool in the mesh + +## What was investigated + +How an agent finds the one tool it needs among everything the mesh answers, and how a call names +exactly what it asks — a role the mesh holds once, a role every machine holds, or one assignment of a +module on one machine — rather than receiving the whole catalogue and a name that can mean several +things. + +## Why + +The operator's observation on 2026-10-03: *Claude should not see all tools at once; they should be +discoverable — and `postgres.list_databases` is wrong, asking one machine's postgres is not asking +another's.* Measured the same day from the console's own answer: + +| | | +|---|---| +| tools announced to every session at its start | 228, in 110 KB | +| names (module or seat prefixes) | 43 | +| node seats' verbs, which require `node` | 22 | +| modules with tools on more than one machine | 4 — fail2ban, nftables (every machine), postgres, mssql (two each) | +| modules reported "not answering", most with no tools and several retired | 47 | + +The two stateful modules on two machines are listed **once**, with `node` optional and *whichever +answers* when it is left out — though their two instances hold different databases. Design 34 §3 says +such a module is listed once per machine; the live console does not do that. The list is taken once +per session, so a tool that arrives later is invisible until the client reconnects. And only Claude +Code's own deferral of long tool lists keeps the 228 from the model's context; another MCP client +would receive them whole. + +## Options + +1. **Keep the flat list; rely on the client to defer it.** Rejected: a property of one client, and it + leaves the ambiguity and the stale list. +2. **One flat tool per assignment** (`ace_postgres_list_databases`). Removes the ambiguity, multiplies + the list, and runs into the API's tool-name limit (letters, digits, `_`, `-`, 64 characters). +3. **A small fixed set of tools that walk the mesh's own structure**, with the full address as an + argument: the mesh's seats; a machine's node seats and assignments; a search; a description; a + call. Chosen — see [ADR 0195](../../02-DECISIONS/0195-the-meshs-tools-are-found-by-address-not-announced-whole.md). +4. **MCP resources or prompts for discovery.** Clients support them unevenly, and an agent acts + through tools; a resource it cannot be relied on to read is not a discovery path. diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/01-RESEARCH/030-backups-against-our-own-mistakes/00-overview.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/01-RESEARCH/030-backups-against-our-own-mistakes/00-overview.md new file mode 100644 index 00000000..25468bf4 --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/01-RESEARCH/030-backups-against-our-own-mistakes/00-overview.md @@ -0,0 +1,29 @@ +--- +status: graduated +became: [03-DESIGN/01-to-be/43-backups-against-mistakes.md, 02-DECISIONS/0214-backups-guard-against-mistakes-and-stay-on-the-machine.md] +initiated: 2026-10-05 +touches: [04-ISSUES/242-the-mesh-has-no-backups, 02-DECISIONS/0030-data-outlives-the-mesh-that-declared-it.md, 02-DECISIONS/0053-a-step-that-runs-on-a-schedule.md, 03-DESIGN/01-to-be/31-a-module-declares-its-fail2ban-jail.md] +--- + +# 030 — Backups against our own mistakes + +**What.** How the mesh keeps restore points of its data: what is copied, how often, how long it is +kept, full or incremental, where it lives, who runs it and how it is proven to restore. + +**Why.** Issue 242: nothing in the mesh backs anything up, and when one misread file dropped every +database on the control node (issue 241), the newest copies were migration leftovers nine to twelve +days old, found by searching a disk. + +**The scope is set by the operator, and it is narrow on purpose: mistakes, not disasters.** A +backup here protects against what a person, an agent or the mesh itself does wrong — a dropped +database, a deleted bucket, a bad migration, a file overwritten — and not against a disk dying or a +building burning. Losing data to a disaster is an accepted risk. That removes the off-site copy, the +cross-site transfer and the second key-holder from the problem, and leaves the part that would have +saved the night of issue 241. + +**What it touches.** The store providers (each knows how to dump its own store consistently), the +host's scheduled steps (ADR 0053, built), the node-wide composition pattern a module's jail already +uses (to-be 31), and the operator's output channel (research 028) for saying a backup failed. + +Documents: [01 — what the mesh holds](01-what-the-mesh-holds.md), +[02 — options and a proposal](02-options-and-proposal.md). diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/01-RESEARCH/030-backups-against-our-own-mistakes/01-what-the-mesh-holds.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/01-RESEARCH/030-backups-against-our-own-mistakes/01-what-the-mesh-holds.md new file mode 100644 index 00000000..2c787e1f --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/01-RESEARCH/030-backups-against-our-own-mistakes/01-what-the-mesh-holds.md @@ -0,0 +1,55 @@ +# 01 — What the mesh holds, measured 2026-10-05 + +Four machines: the control node (hosted, holds every public service), the home server (media, home +automation, a large ZFS pool), a workstation and a laptop. Sizes are apparent sizes, rounded. + +## Nothing backs anything up + +On every machine: no backup tool other than `rsync` and `pg_dump` is installed, no systemd timer and +no cron line mentions a backup, dump or snapshot. Every live data directory is on ext4 except the +home server's pool (ZFS), so a filesystem snapshot is available only there. + +## The control node — the data that cannot be recreated + +| what | size | how it changes | +|---|---|---| +| object store (file-sync service's files, photos) | 183 GB | slowly; files added, rarely rewritten | +| forge (repositories, attachments, its database) | 7 GB | daily | +| MS SQL Server databases | 5 GB | daily | +| mail (mailboxes; accounts in postgres) | 2 GB | continuously | +| postgres (forge, mail admin, identity, file-sync index, analytics, catalogue, licence manager) | ~2 GB | continuously | +| file-sync service's own directory, website, analytics | ~4 GB | slowly | +| MongoDB | 0.4 GB | daily | +| the mesh's own records (controller, vault, module state) | ~1.5 GB | continuously | + +Recreatable and not worth copying: container images (210 GB), the artifact registry (40 GB — every +artifact is rebuilt from git), a 115 GB speed-test bucket and a 7 GB pre-migration object-store copy. + +Free space: 833 GB on the filesystem holding the data, 2.9 TB on a second one. + +## The home server + +MS SQL Server 80 GB, postgres and a self-hosted backend platform ~3 GB, chat server 6 GB, home +automation, network controller and time-series data each under 2 GB, and the media services' +libraries (tens of GB, mostly cover art and metadata they re-fetch). The 89 TB media library is +replaceable by its nature and out of scope. Free: 31 TB on the pool, 453 GB on the system disk. + +## The workstation and the laptop + +The workstation has 142 GB under its services directory and 31 GB of container volumes; the laptop +3 GB. Mostly development; what among it is data nobody can regenerate is for each module to say. + +## Between the sites + +Control node to home server ~285 Mbit/s, home server to control node ~19 Mbit/s. Irrelevant now that +backups stay on the machine whose data they hold, recorded because it is why an off-site copy would +have been expensive. + +## What issue 241 says about the requirement + +- The mistake was noticed within hours. A restore point a day old would have lost a day. +- The restore had to go *beside* the live database, not over it, and that worked well. +- A copy of a live postgres data directory needed a throwaway server of the right version to read; + a logical dump would have restored directly. +- The data that survived was the data outside the dropped stores. A backup that lives inside the + store it protects — a database's own snapshot table, a bucket's own versions — dies with a drop. diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/01-RESEARCH/030-backups-against-our-own-mistakes/02-options-and-proposal.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/01-RESEARCH/030-backups-against-our-own-mistakes/02-options-and-proposal.md new file mode 100644 index 00000000..0a40a01e --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/01-RESEARCH/030-backups-against-our-own-mistakes/02-options-and-proposal.md @@ -0,0 +1,74 @@ +# 02 — Options and a proposal + +## Who decides what is backed up + +1. **A central list** on the backup holder. Rejected: it is the attentiveness rule ADR 0030 + rejected — a store added and not listed is silently unprotected. +2. **Each module declares its own data, a node-wide holder composes them.** The pattern of to-be 31 + (a module declares its jail; the mesh composes them per node). A store provider declares *how* + to take a consistent copy (a dump command), a module with plain files declares *which* paths. The + holder composes every declaration on the node into one schedule. **Proposed.** + +The data a module keeps in a database it gets from a provider is backed up by the provider, which +dumps every database it serves — so a consumer declares nothing, and a new consumer is covered the +day it is provisioned. + +## Full or incremental + +- **Databases: a full logical dump every time** (`pg_dump -Fc`, MS SQL `BACKUP DATABASE`, + `mongodump`). A dump restores with the store's own tool into a database beside the live one — + issue 241's recovery without the throwaway server — and is consistent, which a copy of a live data + directory is not. +- **Everything goes into one deduplicating repository per node** (restic or borg). Each night is a + complete restore point, yet only changed chunks cost space: the object store's 183 GB is copied + once, then each night adds what changed. This removes the full-vs-delta trade-off rather than + choosing a side. + +Considered for the object store alone: the object store's own versioning with a lifecycle rule. +Rejected as the only copy — it lives inside the store, and a removed bucket or data directory takes +its versions with it. + +## Where + +On the same machine, outside every data directory the mesh manages, on a second filesystem where the +machine has one (the control node does). Not off-site: the scope is mistakes. The repository is +encrypted anyway (both tools require it); its key is a mesh secret (ADR 0085), so a person can +restore without the holder. + +## How often, how long + +- **Nightly**, at a quiet hour, as a scheduled step (ADR 0053). +- **On demand before a risky act** — a migration, a retirement, an operator's experiment — through a + verb; the act's own tooling can call it. +- **Kept: 14 daily, 8 weekly, 6 monthly.** A mistake is usually noticed within days, sometimes weeks + (a deleted file nobody opens). Six months bounds the space a slowly-noticed mistake needs. + +Estimated cost on the control node: ~200 GB for the first night, a few GB a night after; well within +the second filesystem's 2.9 TB. + +## Who runs it + +A node seat, **`node-backup`**, held on every machine that has data by one module (named for the tool +it wraps). It receives the declarations, runs them, keeps the repository, and offers the verbs a +person needs: + +- what is backed up here, and the last good night of each; +- take a backup now; +- restore one item **beside** the live one — a database to `_restore`, a path to + `.restored-` — never over it. Swapping it in stays a person's act, as in issue 241. + +## How it is proven + +- Every run checks its own result; a failed or skipped night goes to the operator's output channel + (research 028), not only a log. +- Weekly: the repository's integrity check, and one database restored from the newest dump into a + throwaway instance and counted against the live one. +- A machine with data and no successful backup in 48 hours is a problem the mesh's status shows. + +## Open questions + +- restic or borg — both fit; restic is a single binary with no server, which suits a module. +- Whether the workstation and laptop take part at all, or only once a module there declares data. +- The mail spool is files and the forge has a dump command of its own; whether the forge's + repositories are worth backing up at all when every clone is a copy (issue 241 says the forge's + *database* is the part with no other copy). diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0002-nodes-communicate-over-a-broker.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0002-nodes-communicate-over-a-broker.md new file mode 100644 index 00000000..b548c9ba --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0002-nodes-communicate-over-a-broker.md @@ -0,0 +1,69 @@ +--- +topic: the mesh +status: accepted +date: 2026-02-25 +deciders: jochen +reconstructed: true +--- + +# 2. Nodes communicate over a message broker, not over HTTP + +> Reconstructed after the fact from the evidence cited below. The decision was taken in +> implementation, not in a record; this document states what was decided and why, not a +> deliberation that happened. + +## Context + +The mesh is a set of machines that must call each other's capabilities. On the day the +repository was founded there was no inter-node transport at all — each node was configured +independently and shared nothing at runtime. + +Three properties were required and are visible in everything built since: + +- A node behind a household NAT must participate fully. It can dial out; nothing can dial in. +- A node that is asleep, rebooting or upgrading must not cause a caller to fail — the request + should wait, not error. +- Adding a node must not require editing anything on the nodes that already exist. + +## Considered options + +1. **HTTP APIs between nodes.** Rejected. Every node becomes a server that every other node + must be able to reach, which the NAT case makes impossible without inbound tunnels to each + participant. It also makes node liveness a caller's problem: a request to a sleeping node + is an error rather than a wait. +2. **Polling a shared database.** Rejected. Latency is the poll interval, load is constant and + independent of demand, and request/reply has to be built on top of it by hand. +3. **A central message broker with per-node exchanges.** Chosen. + +## Decision + +All inter-node communication goes through a message broker. Every node owns a topic exchange +named for itself and a request queue; a shared mesh exchange carries commands and events that +are not addressed to one node. + +Three message shapes, and only three: + +- **RPC** — request/reply, for calling a capability that lives on another node. +- **Commands** — instructions to do a stage of work, addressed by what is to be done. +- **Events** — statements that something happened, addressed to nobody. + +Every node dials the broker outbound. Nothing dials a node. + +## Consequences + +- NAT stops being an architectural concern. A node's reachability is a property of the broker + connection, not of its network position. +- A call to a node that is down waits in that node's queue instead of failing. This is usually + right and occasionally the wrong thing entirely — a queued command for a node that never + returns is a stall with no error, which is the failure shape this mesh keeps rediscovering. +- The broker is a single point of failure and a single point of trust. Its credential is + mesh-wide, so rotating it is a mesh-wide operation. +- Tools never leave the host: a remote call proxies over the broker and the credentials stay + where the capability is. + +## References + +- The broker was stood up on 2026-02-25, the second day of the repository. +- Knowledge base: `mesh` (transport, exchanges, queue naming), `troubleshooting/amqp-credential-rotation`. +- The stall shape is recorded in `troubleshooting/empty-pipeline-blocks-the-queue` and + `troubleshooting/daemon-and-tool-server-share-a-request-queue`. diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0004-a-node-and-how-it-joins.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0004-a-node-and-how-it-joins.md new file mode 100644 index 00000000..a958a769 --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0004-a-node-and-how-it-joins.md @@ -0,0 +1,263 @@ +--- +topic: the tiers +status: accepted +date: 2026-08-28 +deciders: jochen +reconstructed: false +--- + +# 4. A node, and how it joins + +*Consolidated 2026-08-28 from four records.* + +## What a node is + +**A managed machine inside the mesh.** Not a device that is merely known about, not an +unprivileged something. If the mesh does not manage it, it is not a node — it is a client, a +peer, or a thing on the network, and those want their own names rather than a weakened version of +this one. + +**A disconnected node is still a node, in a different situation.** Reachability is **state, not +class**. A machine switched off, roaming, or behind a connection that has dropped has not become +a lesser kind of thing; it has a last-known state and a pending set of declarations. + +The distinction people reach for is real, but it is **capability** — what this machine can be +asked to do — and that belongs in the host's profile rather than in the definition of a node. + +**This is the rule that does the most work elsewhere.** A single control plane is tolerable +because its absence is every node in the ordinary disconnected situation at once. An episodic +host on a phone is that situation more often. Neither needed a new mechanism. + +### A node runs one agent session + +*Written 2026-08-29. It runs on every node today and appeared in no record, which is how something +deliberate comes to look accidental.* + +**A node is a machine. The session is a feature of it** — one of the things running there, like the +host, like any workload. The node does not think; something on the node does. Which is why +[ADR 0001](0001-mesh-brokers-nodes-host-agents-think.md)'s *a node does not authenticate to a model +provider, agents do* holds unchanged: the session authenticates, and it is not the machine. + +**It is permanent, and it remembers.** Anything in the mesh can send it a message; it replies; and +what it was asked ten minutes ago is still there next week, alongside what everything else asked in +between — the same way both sides of any conversation remember it. + +Its system prompt is the node's **engram** — what makes one node's replies recognisably its own +rather than generic. + +**It has its own tools**, and fewer than a session a person is driving directly. So a question can +be answered by going and looking: *what is in our forge*, not only *what is your battery*. + +**Messages travel the broker like everything else** ([ADR 0002](0002-nodes-communicate-over-a-broker.md)). +There is no second transport and nothing is dialled. + +**Any node can message any node, and this is the one part of the system that is genuinely a mesh** +— symmetric, with no centre. A node that is asked something it does not have can ask another, and +how it passes the question on is its own business: it may say who wants to know, or simply ask. A +person relaying a question makes the same choice, and it follows from the engram rather than from a +message format. + +**There is no authorisation between nodes.** Every node is the operator's own, so a message from +one is a message from them, and asking a node something is asking a colleague rather than +presenting credentials. Stated once so it is not discovered later: **the mesh boundary is therefore +the security boundary** — anything inside can reach what any node can reach, which is what puts the +whole perimeter on the token and the overlay +([ADR 0007](0007-connectivity.md)). + +**It can be switched off, and switched off it still answers.** A node whose session is disabled +replies saying so, at once, with no model involved — the queue is still read, and the state is the +reply. That is deliberate and it is the same rule the host follows about a service that does not +exist: **absence must never be indistinguishable from a failure to answer.** A node with nothing +there is a silence somebody has to go and diagnose; a node that says *I am switched off* is not. + +**One per node, always, and it cannot be moved to another machine.** Two and nothing decides which +replies; none and the node is mute; moved, and one machine is answering as another. + +**It is not an employee** ([ADR 0003](0003-agents-are-persistent-employees.md)). Nobody hires it, +it holds no tasks, it drains nothing and it is never reassigned — that vocabulary was written for +workers and does not describe this. It exists because the node does, and it is gone when the node +leaves. + +## How it joins + +**The host has one behaviour and two sources of declaration.** What differs between the first +node and the fiftieth is not what the host does but where the declaration comes from — and, as +above, that is a situation rather than a class. + +| | declaration comes from | +|---|---| +| no mesh reachable | the pinned bundle the host carries | +| mesh reachable | the control plane, over the link | + +**The first node is not a different kind of node.** It is a node whose mesh is not up yet. It +applies the bundle it carries, the control plane comes up on top of it, and from that moment it +takes declarations like everything else. **Its specialness is temporary and self-erasing**, which +is what the hand-run bootstrap scripts never were. + +**A joining node does the minimum to be reachable and nothing else** — an identity, an address, +and one peer to reach. It does **not** compute the overlay: the whole peer set is derived +centrally and pushed down. + +That is also why the migration is smaller than it looked. The hard part of the overlay — every +node's key, address, site and reachability — is only needed to compute the *whole* mesh, and a +joining node needs one peer. + +## The link is the security boundary + +**Everything reaching a node arrives one way**, and four properties make that a boundary rather +than a pipe. + +**It is outbound and node-initiated.** The node dials the control plane; nothing dials a node. Not +only defensive — most nodes sit behind a household connection with no forwarded port, so an +inbound control channel would work for one node and not the rest, and the difference would be +invisible until it mattered. **A node has no listening control surface at all.** + +**A node holds its own identity and nothing else.** No shared secret, no credential to anything it +does not own. **Compromise of a node is compromise of that node** — which the current arrangement +does not have, because every node permanently holds the same database and object-store +credentials, and there is no mechanism that rotates one and informs everything holding it. + +### What that identity is: a keypair the node generates + +*Written 2026-08-29. This is the same rule as the sentence above, and it had been treated as an +open question for weeks because of a word.* + +**The node generates a keypair. The private half never leaves the machine. The mesh records the +public half.** Ed25519, the same as the control plane's signing key, in the other direction: +the mesh proves itself to a node by signing, and a node proves itself to the mesh by signing. + +**This was never open.** [`08-connectivity.md`](../03-DESIGN/01-to-be/08-connectivity.md) already +says it of the overlay keys, in these words: *each node generates its own keypair, the private key +never leaves the machine, the public key is published to the mesh* — and adds that this **is** +ADR 0004's *a node holds its own identity*, applied. What was missing was applying it to the thing +this record is about. + +**The word that caused it:** the lifecycle says a joining node *receives* its own durable identity, +which reads as the mesh issuing something, and then the question becomes *issuing what*. It does +not issue anything. The node arrives holding its identity; what it receives is **being known**. +Enrolment is the moment the mesh writes down a public key it will believe, and the one-time secret +is what buys the right to have it written down. + +**Everything above then holds literally.** Nothing is stored that could be stolen and replayed: the +mesh's copy is a public key, so a copy of the mesh's database grants nothing. *Compromise of a node +is compromise of that node* becomes true rather than aspirational, because the only secret on a +machine is the one that identifies it. + +### What "connecting to the mesh" is, concretely + +*Written 2026-08-29, because it was asked and this record had never said it.* + +**One outbound AMQP connection from the node to the broker, held open.** That is all of it. There +is no second connection and nothing is ever dialled *at* a node. Being in the mesh, operationally, +means that connection is up; being disconnected means it is not +([`09-the-node-lifecycle.md`](../03-DESIGN/01-to-be/09-the-node-lifecycle.md)). + +**Two different things ride on it, and conflating them is what made this confusing:** + +| | what it answers | who issues it | +|---|---|---| +| **an AMQP account** | may this connection be accepted at all | **the mesh, at enrolment** | +| **the node's keypair** | which node is speaking, on every message | **the node**, above | + +**The account is the mesh's to issue**, and per node. The broker has to authenticate somebody +before a connection exists, and a shared account would let any node consume another's queue — +which is the shared-credential fault this record exists to remove, reappearing at the transport. +So enrolment creates that node's account and hands it over, and it is rotatable without touching +the node's identity. + +**The keypair is not made redundant by it.** With only an account, the control plane would know +which node is speaking *because the broker says so* — and that is the same transitive authority +this record refuses in the other direction. A compromised broker could then attribute reports to +whichever node it liked, and the control plane would act on them. Signing is what removes the +broker from the question in both directions. + +**So a node holds two things after enrolment**: a credential the mesh issued for reaching the +broker, and a key it generated itself that the mesh only ever sees the public half of. Both are +its own, neither reaches anything else, and *compromise of a node is compromise of that node* +still holds. + +### Its own key, not the machine's SSH host key + +Reusing the host key is the obvious economy and it is refused, for reasons that are operational +rather than fastidious: + +- **It is regenerated by ordinary events.** A reinstall, an image cloned, `ssh-keygen -A` on a + rebuild — each silently un-enrols the node, and the failure appears as an authentication problem + with no cause anybody changed. +- **It is managed by something else.** Its lifecycle belongs to the machine's SSH daemon, and an + identity the mesh depends on should not rotate on a schedule the mesh does not know about. +- **Not every node has one.** A partial host has no SSH daemon + ([ADR 0005](0005-the-node-host.md)), and an identity scheme that excludes a supported kind of + node is not one. + +**The mesh should still know the host key** — it knows every node, so it can distribute host keys +the same way it distributes authorised keys +([ADR 0006](0006-the-substrate-and-the-control-plane.md)), and node-to-node SSH stops depending on +trust-on-first-use. That is the good half of the idea, kept. + +**Authority is mutual.** The node proves it may join, and the control plane proves it is the +mesh. One-way is not enough: the host applies whatever the link delivers, so a node that cannot +tell the mesh from something impersonating it will apply that something's declarations. + +**What may be pushed is bounded by form, not by trust.** Declarations of known shape, never a +command to run. Stated honestly, **this bounds form and not impact**: a compromised control plane +can declare harmful state and the host will apply it faithfully, because that is what it is for. +What the property buys is that the blast radius is *describable* — exactly what the declaration +language can express, which can be reviewed. An arbitrary command channel has no such bound. + +## The enrolment token carries the mesh + +Mutual authority needs the node to verify something before it trusts anything, and that is a +circle: verifying the mesh needs the mesh's certificate authority, and obtaining one means +trusting whoever hands it over. There is a second circle beside it — a node must reach the mesh +before the mesh has configured it, so it can resolve no mesh name. + +**Both are the same shape: a node needs a fact about the mesh before it has any trustworthy way +to obtain one.** So that fact arrives by a path other than the mesh. + +**The token carries five things**, and it is the only thing a joining node needs: + +| | | +|---|---| +| **who it is** | the name the mesh calls this machine | +| **where** | the broker's **address**, not a name — there is no resolution yet, and this is why none is needed | +| **what it is connecting to** | the fingerprint of the broker's certificate | +| **who it will believe** | the control plane's signing identity | +| **the right to join** | a one-time secret, useless once used and useless after it expires | + +*The first row was added 2026-08-30, from raising a mesh end to end for the first time.* It reads +like an oversight and is not: **the node cannot work its own name out.** The name is the mesh's, +chosen when the record was created, and the broker account the node authenticates as is named +after it — so it must be known *before* the mesh can tell the node anything. It is not a secret, +and whoever issues the token already has it. + +Without it, enrolment fails at the broker with an empty username and a message about credentials, +which points at everything except the cause. **A missing fact that surfaces as an authentication +error is worse than one that surfaces as a missing fact.** + +**Carried out of band**, by the person adopting the machine. That is what breaks both circles: +its authenticity comes from the channel it travelled, not from anything the node can check +afterwards. **Trust on first use, with the first use moved out of band** — the difference between +a pin and a guess. + +**The endpoint and the authority are two identities.** A node connects to the broker and takes +instruction from the control plane behind it. Pinning only the broker would make the control +plane's authority *transitive*, and a compromised broker could then forge declarations — which, +since the host applies whatever the link delivers, is the whole machine. So the transport is +verified once at connect, and **each declaration is verified by its signature, every time**. + +**What this settles:** the mesh's certificate authority is not a bootstrap concern — it certifies +internal names once a node is a member. Nothing needs name resolution before the link. And +nothing is placed on disk beforehand except the token, which is the first moment *a node holds +only its own identity* becomes true rather than aspirational. + +## Consequences + +- **Declarations must be signed**, and the host must tell *this is not from the mesh I joined* + apart from *this is malformed*. Rotating the signing identity is a fleet-wide operation with an + overlapping rollover, and that is the cost of not trusting the broker. +- **The token becomes security-critical**, because it carries the pin. Tampering substitutes the + mesh — which is strictly better than the alternative, where there is nothing to tamper with and + the node trusts the first answer unconditionally. +- **A rejoining node is ordinary.** There is no long-lived secret to recover, so a node that lost + its identity gets a new token. diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0037-where-a-module-lives.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0037-where-a-module-lives.md new file mode 100644 index 00000000..c01e6c65 --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0037-where-a-module-lives.md @@ -0,0 +1,119 @@ +--- +topic: building it +status: accepted +date: 2026-09-01 +deciders: jochen +reconstructed: false +rests-on: 02-DECISIONS/0009-modules-and-the-graph.md +--- + +# 37. Where a module lives + +## The question + +The mesh's own module descriptions currently sit in `examples/` inside the control plane, beside +the small programs that hand out logins. That was fine while there were three of them. It is +wrong now, and the name is doing active harm: everything in `examples/` reads as a sketch, and one +of them shipped naming a container image that nothing in the repository builds. A directory called +*the catalogue* would have made *does this actually work* the obvious question to ask of it. + +So: **one repository holding the modules we ship?** And if so, where does everything that is not +ours go? + +## What a module actually is, counted + +The system being replaced has **126 modules** on its main branch. The shape of them is the whole +argument, so it is measured rather than assumed: + +| | count | what it is | +|---|---|---| +| **the module is software** | 47 | its own source tree lives inside the module — a daemon, a service, a library | +| **helper scripts only** | 44 | no application of its own; scripts it runs at install time or offers to an agent | +| **a description and nothing else** | 35 | a package to install and some files to write | + +**Two thirds of modules contain code.** The largest is a shared library of 182 source files. A +speech-capture module carries a complete daemon — audio capture, mixing, transcription, a model +runner. Treating a module as *a description of something else* is true of barely a quarter of them. + +That kills the simplest answer. A catalogue cannot be "a folder of manifests" when most modules +are programs. + +## The four kinds, which want different homes + +**1. What the mesh is made of.** The control plane, the host, the shared library, the board. +These are not modules that happen to be ours; they are the mesh, expressed as modules so it can +install itself. They belong in the repositories that build them, which already exist. + +**2. Something the world made, that we describe.** A forge, a mail system, an identity provider, +a media server. Nobody upstream ships a description; somebody has to write one, and it is the same +description for everybody who runs it. **This is what a catalogue is for.** It is also where the +small programs that create accounts belong, because such a program is part of describing that +service, not part of the mesh. + +**3. Something we wrote, that runs somewhere.** An application, a site, a side project. The +description belongs **with the code, at the root of its own repository**, because the two change in +the same commit. A repository that gains an environment variable and a description that gains it +elsewhere will drift, and there is no mechanism that could stop it. This is already how it works +and it should stay that way. + +**4. A package and some files.** A tool, a font, a shell. Thirty-five of these, and each is a few +lines. The catalogue. + +## The proposal + +**A `mesh-catalog` repository** holding kinds 2 and 4: descriptions of software we did not write, +and the programs that provision it. Not kind 1, which is the mesh itself. Not kind 3, which lives +with its own code. + +**The mesh's list of modules is not this repository.** It is a table in the control plane, filled +by adding a description to a running mesh. The catalogue is a *source* to add from — one of +several, and the mesh already records which: every module carries where it came from, the branch +followed there, and the commit its description was read at. **Nothing needs inventing to support +modules from anywhere**; a repository of our own is simply the source we curate. + +**A description is checked by the tool, not by a test that imports the tool.** Today a test in the +control plane parses the example manifests by reaching into the control plane's internals, and +another reads the control plane's own build file to check every image a module names can be built. +Two jobs tangled. A `module check` command on the control plane's binary would let the catalogue +hold data validated from outside, and would give the same check to somebody describing their own +application in their own repository — which is the case that matters most and currently has no +check at all. + +## What this costs, and the argument against + +**It is early.** Ten modules exist, four of them ours. Moving ten files is a morning; moving a +hundred is a week — but the hundred is not here yet, and splitting now adds a second repository to +release across before there is anything to release. + +The counter is that the tangle is already producing faults rather than merely threatening to. A +manifest naming an unbuildable image, and a test reading a build file two directories up, are both +symptoms of one repository doing two jobs. And the moment the first module is adopted on a real +machine, the descriptions stop being examples and become the thing deployments come from. **That +is the moment this becomes urgent, and it is close.** + +## What it does not settle + +**Where a provisioning program's image is published**, and how a description pins it. A description +names an image by digest; the image is built from the catalogue; the catalogue must therefore both +produce an image and refer to it, which is the same knot the bootstrap has and solves by writing +the digest down after building. + +**Whether kind 4 deserves a module at all.** Thirty-five descriptions that say *install this and +write these files* may be better as one module with settings than as thirty-five modules. Left +open deliberately; it is a question about the shape of the catalogue, not about whether to have one. + +## Accepted, 2026-09-29, against what was built + +*Marked in a grooming pass: the mesh was built to this record and the record still said `proposed`.* + +The proposal is the arrangement that exists. `mesh-catalog` holds descriptions of software we did +not write and the programs that provision it, and holds neither the mesh's own components nor an +application's own module. The mesh's list of modules is a table in the control plane, filled by +`module add`, and every module records the source it came from with the commit it was read at. + +**One half is not built: `module check` as a command on the control plane's binary.** A manifest is +still validated by a test that reaches into the control plane's internals — which works for this +catalogue and gives nothing at all to somebody describing their own application in their own +repository, which this record says is the case that matters most. That is +[issue 148](../04-ISSUES/148-a-manifest-outside-this-catalogue-has-no-check/00-report.md). + diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0038-the-mesh-assigns-the-port.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0038-the-mesh-assigns-the-port.md new file mode 100644 index 00000000..c8cb2697 --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0038-the-mesh-assigns-the-port.md @@ -0,0 +1,82 @@ +--- +topic: what runs on it +status: accepted +date: 2026-09-01 +deciders: jochen +reconstructed: false +rests-on: 02-DECISIONS/0009-modules-and-the-graph.md +--- + +# 38. The mesh assigns the port, and a module does not care + +## The problem, as met + +A database module cannot start on a machine that runs the control plane. The mesh keeps its own +store there and holds 5432; the module publishes 5432. Nothing notices until a container runtime +three layers down says `port is already allocated` +([`028`](../04-ISSUES/028-two-things-want-one-port-and-nothing-says-so/00-report.md)). + +A module cannot fix this by choosing better, because **a module cannot know what else is on the +machine.** It is written once and assigned anywhere. Any number it picks is a guess about a +machine it has never seen, and two modules guessing the same number is not a mistake either of +them made. + +## The number is written three times, and nothing makes them agree + +Every module says its port in three places: + +| where | for | example | +|---|---|---| +| `listens` | the rule set that lets traffic in | `{port: 5432, from: mesh}` | +| `serves` | what a consumer must know to connect | `{port: 5432}` | +| a container's `ports` | what the runtime publishes | `"5432:5432"` | + +They agree today because one person wrote all three. Nothing checks it. A module whose `serves` +said 5432 and whose container published 5433 would resolve, compose, apply, and hand every +consumer a port that answers nothing. + +## The decision + +**The mesh assigns the machine-side port, and the module says only what it needs.** A module +declares that a container port must be reachable and what it is for. Which number the machine uses +is the mesh's to choose, because the mesh is the only thing that knows what else is there. + +**One source, and the other two are derived.** `serves` carries the assigned port so a consumer is +told where to connect without the module having written it down; the rule set is computed from the +same assignment. Three copies become one fact. + +**An assignment is made once and kept**, exactly as a credential is. A port that moved on every +push would restart both ends each time and would hand consumers a number that was true when it was +read. + +## Some ports cannot move, and that is a claim + +Mail is 25, submission is 587, IMAP over TLS is 993. A mail system on a strange port is not a mail +system. So a module may say a port is **fixed by the protocol** rather than assigned. + +**A fixed port is exactly a claim** — the thing the mesh already has for what is singular on a +machine: one seat, one display server, one artifact store. Two modules wanting 25 on one machine is +the same shape as two wanting the seat, and gets the same answer: the second is refused, by name, +when it is assigned rather than when it is applied. + +That is why this does not need a new mechanism so much as it needs the existing one pointed at +ports. + +## What follows + +- **A module becomes portable in a way it was not.** Two databases on one machine stop being a + collision and become two assignments. +- **The substrate has to be visible.** The mesh cannot assign around its own store while it has + never heard of it. What the bundle holds must be written down somewhere the assignment can read + — which the bundle does not say today. +- **A refusal can be useful.** *25 is held by the mail system on this machine* is a sentence a + person can act on. `port is already allocated` is not. +- **`serves` stops being written by hand**, which is a small vocabulary change with a large + consequence: what a consumer is told is now derived from what actually happened. + +## What this does not settle + +**Whether a module should publish to the machine at all.** Assignment makes publishing safe; it +does not make it necessary. Consumers could instead reach a provider on the module's own network by +name, with nothing published — which would make the question moot for anything inside the mesh, and +would still leave it for anything reached from outside. diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0040-what-a-module-is.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0040-what-a-module-is.md new file mode 100644 index 00000000..8b025787 --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0040-what-a-module-is.md @@ -0,0 +1,103 @@ +--- +topic: what runs on it +status: accepted +date: 2026-09-03 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0009-modules-and-the-graph.md +--- + +# 40. What a module is + +_Reconciliation note (2026-09-05): supersedes the earlier "grouped by domain" decision, which the consolidation folded into how-we-build.md; no standalone record remains to point at._ + +## Context + +[ADR 0009](0009-modules-and-the-graph.md) settled that everything is a module, but never said what a +module *is* beyond "a directory the mesh processes." That gap let the catalogue's breadth read as a +smell: a module can carry a container, a built image, tools, a provisioner, migrations, health, +config, seat claims, requires and provides — so much that the unit seemed ill-defined. +The earlier "grouped by domain" decision (folded in consolidation; see the reconciliation note +above) tried to organise modules by domain, which is the wrong axis. This record states what a module is, drawn from the cases that +stress-tested it: the shell, i3-vs-sway, umami, and "database." + +## Decision + +**A module is one self-contained piece of software the mesh installs and manages** — everything +needed to make that one thing real and integrable: what runs, the seats it claims, what it provides +to other modules, what it requires from them, and what operates it. + +The **software is the module's identity.** Capabilities, seats and provisioned resources are the +**relationships *between* modules**, not what a module is — and that is what binds a module into one +thing. umami is bound by *being umami*: its container runs umami, its provisioner creates umami sites, +its tools query umami, its `requires` gets umami a database. Every feature serves the one software. + +### The three relationships + +1. **Shared seat** — several modules fulfil a capability and coexist; one may be default. bash, zsh + and fish all join `shell`. +2. **Exclusive seat** — modules contend for a single slot; one holds it. i3 (needs x11) and sway + (needs wayland) contend for `display-session`. +3. **Provide / require** — a provider ships the **provisioner** that creates instances of the + resource it offers and returns sealed credentials; a consumer requires it and the mesh wires the + credential in. Symmetric: umami requires a database *and* provides analytics. + +### Interfaces are mesh-owned; providers adapt to them + +The mesh **defines the interface** for a capability — the provider-neutral contract of what a +consumer receives and how it integrates. Both sides conform: a provider's provisioner **adapts** its +software's real API to the mesh contract; a consumer depends on the **interface**, never on a +provider. Swap one provider for another and the consumer does not change. + +### The naming rule — draw the interface at the consumer's real coupling + +Name a `provides`/`requires` at the **widest boundary across which the consumer genuinely does not +care which implementation serves it**: + +- Where the consumer's coupling is thin — an analytics embed snippet and dashboard, opaque to it — + the mesh defines a neutral interface (`analytics`) and providers (umami, amumi) adapt. Swappable + across vendors. +- Where the consumer **speaks a protocol** — a database's wire protocol and query dialect — the + interface *is* the protocol: `postgres-database`, `mssql-database`, `mongodb-database`. Swappable + only among protocol-compatible implementations, **never across**, because the application cannot + cross it either. "database" is not a capability; the protocol is. +- **Never false genericity.** A name must not promise a swap the contract cannot deliver + ([research 005](../01-RESEARCH/005-domain-grouping/analysis.md)). + +This is [ADR 0027](0027-a-provision-names-what-the-consumer-is-coupled-to.md)'s rule made general — "names the protocol, not +the product; a database names the engine because the app targets it" — with the reason stated: the +contract sits where the coupling is. + +### What is not a module + +- A **library** (built against, never deployed — [ADR 0039](0039-what-the-sdk-holds-and-refuses.md)). +- A **control-plane context** (the mesh itself — [ADR 0001](0001-mesh-brokers-nodes-host-agents-think.md)). + +A **swappable machine mechanism** (a firewall — ufw, nftables) *is* a module implementing a +capability. The host hardcodes no firewall, supervisor, package manager or runtime; it owns only the +generic apply primitives and platform detection, so it runs where none of those exist — an Android +phone has no ufw, systemd, pacman or Docker. + +> **The mechanism changed — 2026-10-02, by [ADR 0176](0176-the-login-shell-is-a-node-seat-and-execute-is-its-contract.md).** The shell example above — *bash, zsh and fish all join `shell`; one may be default* — is read as *installed is not holding*: the three may all be installed, and the `login-shell` seat is node-scoped and held by exactly one. The decision — what a module is, and the three relationships — stands; [ADR 0173](0173-the-operators-machine-is-the-meshs-and-a-module-is-what-it-declares.md) applies it to the operator's whole machine. + +## Consequences + +- **Supersedes the earlier "grouped by domain" decision** (folded in consolidation; see the + reconciliation note above). Modules are + organised by their relationships (seats, provisions), not grouped into domain folders. +- **Refines [ADR 0009](0009-modules-and-the-graph.md).** Everything the mesh runs and integrates is + a module — but a module is defined by the *software it delivers*, not by being a bucket of features. +- The target is a **self-fulfilling mesh**: declared wants bound to swappable modules, provisioners + wiring credentials, nothing hardcoded. The control plane's whole job is the binding. +- Converting a module from the old system includes pulling its per-module code out of the shared SDK + ([ADR 0039](0039-what-the-sdk-holds-and-refuses.md)) and shipping its provisioner as an adapter to a + mesh interface — a de-coupling, not just a move. + +## References + +- [ADR 0009](0009-modules-and-the-graph.md) — everything is a module; this says what one is. +- [ADR 0001](0001-mesh-brokers-nodes-host-agents-think.md) — contexts are the mesh, not modules. +- The earlier "grouped by domain" decision — superseded (folded in consolidation; see the note above). +- [ADR 0027](0027-a-provision-names-what-the-consumer-is-coupled-to.md) — protocol-not-product, generalised here. +- [ADR 0039](0039-what-the-sdk-holds-and-refuses.md) — per-module code lives in the module. +- [research 011](../01-RESEARCH/011-the-module-graph/00-overview.md) — the graph of these relationships. diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0085-a-secret-is-a-provision.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0085-a-secret-is-a-provision.md new file mode 100644 index 00000000..f3222317 --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0085-a-secret-is-a-provision.md @@ -0,0 +1,153 @@ +--- +topic: what runs on it +status: accepted +date: 2026-09-20 +amended: 2026-09-20 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0031-the-control-plane-authenticates-nobody.md +--- + +# 85. A secret is a provision, and the vault is the module that provides it + +## Context + +[ADR 0031](0031-the-control-plane-authenticates-nobody.md) decided that identity *runs on the +mesh, not of it* — a module other modules require, rather than a privileged part of the +controller. [ADR 0078](0078-the-store-and-broker-are-modules.md) did the same for the store and +the broker: the twelve-module floor has no specialty left in it. **Secrets are the exception that +survived.** No module owns a secret. + +Secret handling is smeared across three built-in parts of the runtime: + +- the **controller mints** one credential per consumer↔provider pair and seals it to both node + keys ([ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md)); +- the **mesh generates local secrets** — *"generated secrets are the mesh's, never authored"* + (as-is, `06-configuration-and-secrets.md`) — for a value a single module needs for its own use; +- the **synchroniser injects** both into a node's generated files. + +Nothing is the owner of "a secret" the way the store module is the owner of "a database", and the +cost is recorded rather than hypothetical. The as-is design names the weakness in its own words: +*"Rotation is not a mesh operation… there is no mechanism that rotates one and informs everything +holding it."* A `rotate` command has since been built and proven, but it reaches **only** the +provisioned pairs; a secret a module generates for its own fully-local use — the password of a +version-pinned embedded store ([ADR 0084](0084-which-provider-serves-a-consumer.md)), an internal +token — is minted by the mesh and then has no operation that can remake it. Three species of +secret, and only the first has an owner: + +| species | minted by | rotates? | +|---|---|---| +| a provisioned credential (a database login) | controller, sealed to nodes (0048) | yes — `rotate`, per pair | +| a module's own local secret | the mesh, as a generated value | **no owner, no rotation** | +| an operator-delivered secret (an external key) | a person, sealed in (`secret accept`) | no rotation, no audit | + +## Considered Options + +1. **Leave it a property of the controller.** Rejected: it is the smear above — no owner, local + secrets that cannot be rotated, operator secrets that cannot be audited — and it is exactly the + specialty 0078 removed for the store and broker, kept here for no reason anyone recorded. +2. **A dedicated vault built into the foundation, not a module.** Rejected: it reintroduces a + privileged built-in, the thing 0031 and 0078 went out of their way to remove, and a mesh that + wants none would still carry it. +3. **Fold all minting, the controller's provisioning credentials included, into the vault.** + Rejected: the controller must mint in order to **deliver** any provision — the vault's own + credential among them — so making the vault mint the credential of its own delivery is the + store/broker chicken-and-egg for no gain. The provisioned-pair credential already has an owner + and a rotation ([13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md)); this record + does not disturb it. +4. **A secret is a provision; the vault is an ordinary module that provides it.** Adopted. + +## Decision + +**Secret-holding is a module, parallel to identity.** A module that needs a secret **for its own +use** — a local service's password, an internal token, an external key it was handed — requires a +`secret` provision from a vault provider, exactly as it requires a database from the store. The +vault generates the value (or holds one it was given), and because the credential belongs to the +consumer↔vault pair it **rotates, backs up and is audited through the same per-pair machinery** +[13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md) already defines. That is what +gives the second and third species the rotation and audit they lack. A module's own secret stops +being a generated value that nothing owns and becomes an ordinary provision with a provider. + +**The controller's minting of provisioning-pair credentials is unchanged** ([ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md)): +it is how every provision, the vault's own included, is delivered. The vault does not mint the +mesh's delivery credentials; it provides secrets to modules, and is itself provisioned the ordinary +way. + +**The vault is node-scoped like every provider** ([ADR 0084](0084-which-provider-serves-a-consumer.md)): +each node its own, selected the same way, so a module's own secret is held by the vault on the +module's node. **A mesh that wants no vault runs none** — a module requiring no secret needs +nothing, which is the same test 0031 applied to identity. + +## Consequences + +The gap that opened this — a generated local secret with no rotation — **closes without new +machinery**: rotating such a secret is the vault's provisioner remaking a pair credential, the +operation 13 already specifies. Backup, audit and break-glass gain an owner — the vault module — +and become things a design specifies rather than absences. The as-is sentence *"rotation is not a +mesh operation"* is already false for provisioned pairs and, once the vault ships, for local +secrets too; the as-is document is updated when it does, not before. + +What got harder: a break-glass path — recovering a secret when the sealed delivery path is +unavailable — must not reintroduce a key that one place holds, which is the property +[ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) was built to preserve; how +the vault offers recovery without it is left to the design as an open question. And a module that +today bakes a password into its own composition must instead require it from the vault — a +migration taken module by module, not a flag day. + +## Amendment — 2026-09-20, before anything shipped + +Recorded on the record itself rather than as a supersession, by its decider, on the day it was +accepted and before any code was merged against the sentences that change. The original text above +is left as written; this section says what it got wrong and what stands instead. + +**What it got wrong.** The decision treated the vault as a provider like the store — optional, and +one per node — and left the mesh's own root secrets outside it: the store's superuser, the broker's +administrator, the controller's contexts, sealed to a node key and nothing else. Those are the +secrets with no rotation and no recovery, and they are the ones a vault exists for. At genesis they +are not even secret: the foundation raises its store and broker with fixed, well-known credentials +and carries those into the mesh. Leaving that floor in place gave module secrets an owner and the +root secrets none. + +**What stands instead.** + +- **The vault is a foundation module.** It is installed at genesis as part of the foundation + ([ADR 0078](0078-the-store-and-broker-are-modules.md) is the precedent: a foundation piece is + still an ordinary module), not assigned later by a mesh that happens to want one. *"A mesh that + wants no vault runs none"* is withdrawn. A mesh has root secrets, so a mesh has a vault. +- **One per mesh, on the control-node.** *"The vault is node-scoped like every provider"* is + withdrawn. Node scoping ([ADR 0084](0084-which-provider-serves-a-consumer.md)) exists because a + store holds data a consumer is coupled to; the vault holds nothing a consumer is coupled to, and a + second one would be a second place to lose. Consumers on other nodes reach it as they reach + identity. +- **The vault holds the mesh's root secrets under an operator-held key.** The controller mints and + delivers exactly as before; in addition, every secret a module holds for itself is sealed a second + time, to an **operator sealing key** whose private half never enters the mesh. The vault keeps + those operator-sealed copies on its own disk, outside the store, and can hand them out — they are + ciphertext to everything but the operator. This is the break-glass path the original text left + open, and it does **not** reintroduce a key one place holds: the mesh holds blobs it cannot open, + and the operator holds a key with nothing to open until given a blob. Recovery needs both. +- **Genesis mints real root secrets and seals them to the operator key first**, so the fixed + credentials the foundation is raised with are replaced before the mesh is handed over. + +**Unchanged.** A module's own secret is a `secret` provision the controller mints and the vault +records; the provisioned-pair path of [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) +is untouched; the vault stores no plaintext, ever. + +**What it costs.** An operator key is a thing a person must keep, and a mesh whose operator key is +lost has root secrets that can be rotated but not recovered — the same standing as today, stated. +Sealing every own secret twice is a column and a call. Genesis grows a step. A module's +vault-provided secret (a pair credential) is not yet sealed to the operator key; that is the next +increment, not this one. + +## References + +- [ADR 0031](0031-the-control-plane-authenticates-nobody.md) — identity is a module; this is the + same move for secrets. +- [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) — the provisioning-credential + path, left unchanged. +- [ADR 0078](0078-the-store-and-broker-are-modules.md) — the de-specialisation this completes. +- [ADR 0084](0084-which-provider-serves-a-consumer.md) — the node-scoping the vault obeys. +- [issue 068](../04-ISSUES/068-secrets-have-no-owning-module/00-report.md) — the gap. +- [`03-DESIGN/01-to-be/24-the-secrets-vault.md`](../03-DESIGN/01-to-be/24-the-secrets-vault.md) — + the design. The source mesh's `secret_locate` / `secret_backup` / `secret_verify` / + `secret_breakglass` subsystem is the prior art it draws on. diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0259-an-ask-is-a-message-on-the-bus-any-channel-answers-it-and-the-asker-acts-on-a-warrant.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0259-an-ask-is-a-message-on-the-bus-any-channel-answers-it-and-the-asker-acts-on-a-warrant.md new file mode 100644 index 00000000..1ae9b0fa --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0259-an-ask-is-a-message-on-the-bus-any-channel-answers-it-and-the-asker-acts-on-a-warrant.md @@ -0,0 +1,602 @@ +--- +topic: the mesh +status: accepted +date: 2026-10-09 +deciders: jochen +reconstructed: false +supersedes-in-part: + - 02-DECISIONS/0234-the-mesh-holds-a-conversation-with-its-operator.md +extends: 02-DECISIONS/0234-the-mesh-holds-a-conversation-with-its-operator.md +--- + +# 259. An ask is a message on the bus, any channel answers it, and the asker acts on a warrant + +## Context + +On 2026-10-08 the operator chose Telegram as the next channel for the operator's decisions. "Needs you: +release it, or stop it" must be answerable from the phone, with real authorisation, and without an agent +session. + +What the mesh had on its main branches that day: + +- **The desk performs only an acknowledgement** + ([ADR 0258](0258-a-desk-click-performs-only-an-acknowledgement-and-an-answer-is-a-one-time-ticket.md)). + Release, stop, start and restart are said in words: "from the mesh MCP server; this notification cannot do + it". The messenger (the `operator-channel` seat's holder) calls `conditions silence` itself when the + operator chooses Silence on the desk. +- **The messenger holds the Telegram client in its own process**, says conditions there as words without + buttons, and keeps the bot token as its own secret `telegram-token`. Its status said on 2026-10-08 that the + token held is not a bot token: no bot has been given yet. +- **[ADR 0234](0234-the-mesh-holds-a-conversation-with-its-operator.md) §8 gives authorising asks to the + controller**: it holds the ask, keeps the operator's identities and factors, checks the proofs and + performs the action itself. None of it is built ([to-be 46](../03-DESIGN/01-to-be/46-the-conversation-with-the-operator.md) + phase 5). +- **Any node's tool runner may call any tool on any node** (ADR 0175). A tool call therefore proves + nothing about who made it: the laptop's runner carries the agents' calls. What the bus *does* prove is the + publisher of a module's events and a seat's events: only the principal granted a subject may publish it, + which is why a condition's events need no signature (design 29 §2). + +The operator's directions, 2026-10-08, on the first proposal (a Telegram path through the controller): + +- "I don't think we should mangle Telegram into our controller. Now we're using Telegram, but tomorrow we + could be using Signal. And today the controller needs to receive a human decision, but tomorrow a + completely different module might require human approval." +- **Two generic boundaries.** Asking a person is a mesh-wide service any module uses; the asking module + performs the action itself, with its own grants, and records who decided. Channels are pluggable: a + channel renders the question, receives the answer and verifies the sender by its own means. Nothing + outside the channel module knows which service it talks to. +- **The controller only asks and acts.** +- **The flow is native to the bus**: a request is a message, pending requests survive a restart, each + answer is recorded once, the asker reads its request's state from a bucket, and the bus's permissions + decide who may publish what. Events carry no secret; request and reply may. No HTTP between modules. + +The operator approved this record on 2026-10-08, with the word *warrant*, and decided two points it had +left open: + +- **The `telegram` module runs on the control node under a service account of its own, not the + operator's.** Claude Code runs as the operator on every machine; the service account keeps an agent away + from the bot token and the channel's state. +- **A newly linked identity waits one hour before it can approve**; acknowledge works at once. Every link + and unlink is announced on every channel. This holds until a code from an authenticator app exists. + +The operator accepted this record as revised on 2026-10-09 — every part marked "added" or "revised" on that day, +after the security reviews and the confirmation review — and asked that the mesh be able to ask them a question, +and ask for an approval, from then on. + +### Against GENESIS + +- **Core value — sovereignty.** Telegram stays an outside dependency taken deliberately (ADR 0234), and + is now one module that can be removed without touching anything that asks. +- **Core value — failure must be loud.** An ask that no channel can carry is refused in words to its asker + at once, never queued in silence. An answer that is refused is said on the channel it came from. +- **Effect — "when something genuinely needs a decision, you are asked, with the context".** Any module + can now ask, not only the controller. + +## Considered Options + +### Where an ask is held + +1. **The controller holds authorising asks** (ADR 0234 §8). Rejected by the operator: the next module + that needs a person (a backup module, the delivery owner) would ask through the controller, and the + controller would perform actions in other domains. +2. **A new seat for decisions, beside `operator-channel`.** Rejected: an ask needs the router's routing, + the work context and the content rule; a second seat would ask the router for each of them, and two + holders would keep two lists of who the operator is. +3. **The `operator-channel` seat's holder, the router, holds every ask, authorising or not, and judges + the answers.** Chosen. ADR 0234 already gives it ordinary asks; this adds the authorising ones. + +### Who performs the action + +1. **The decision holder calls the asker's verb.** Rejected: the router would act in every domain with a + grant to every verb, which is what ADR 0258 took away from the desk. +2. **The asker performs on hearing "yes" from anyone.** Rejected in ADR 0234: a relayed "yes" is not an + answer. +3. **The asker performs on a warrant**: the router's record that the operator chose an option, published + on a subject only the router may publish, for that asker alone. Chosen. The bus proves the warrant's + source as it proves a condition's. A signature was weighed and rejected: it needs a key in every asker + and proves nothing the server's permissions do not. + +### How an ask, a warrant and an answer travel + +1. **Tool calls** (an `ask` verb, an `answer` verb). Rejected: any tool runner may call any tool, so the + router could not tell the controller's ask from an agent's, nor a channel's answer from anybody's. +2. **HTTP between modules.** Rejected by the operator's direction, and it would carry its own + authentication beside the bus's. +3. **Subjects whose last token names the publisher**, granted to that publisher alone, on streams that + survive a restart. Chosen. The mesh already grants a machine its own token on a node seat's events + (ADR 0219); this is the same rule for a caller's module and a channel's kind. + +### Who is the operator on a channel + +1. **A setting naming the operator's account on the service.** Rejected: any principal granted + `mesh-controller.settings` may write a setting, agents included, so the operator's identity would be + anyone's to set. +2. **The channel module's own allow-list.** Rejected in ADR 0234: the channel would decide who the + operator is. +3. **The router's list of the operator's identities, per channel kind, written only by linking.** + Chosen. A channel module passes the sender as the service authenticated it, as an opaque string; the + router alone compares it with its list. + +### How a channel takes its answers (Telegram) + +1. **A webhook.** The control node is the stably reachable node, so it could take one. Rejected all the + same: it needs a public route, a name and a certificate into the control node, guarded only by a + secret header; a stolen token could point it elsewhere. +2. **Long polling**, as ADR 0234 chose. Chosen: outbound only, unaffected by the control node's address, + answered as fast as a webhook, and a stolen token can read updates (visible as a conflict) but not + inject one. + +### Where the desk's buttons live + +1. **Inside the router**, as its desktop adapter (to-be 46 §11). Rejected: the router would know one + channel, and the operator asked that the desk be just another channel. +2. **The dunst module** claims the channel. Rejected: dunst holds a node seat on each machine, and a + channel kind has one holder for the mesh. +3. **A small module of its own, kind `desktop`**, sending through each machine's `node-notifier` and + reading dunst's `action-chosen`. Chosen. + +## Decision + +### 1. The words + +- An **ask** keeps its glossary sense; an **authorising ask** is one whose chosen option the asker + performs. Each option carries its **assurance level**: `acknowledge`, `approve` or `destroy` (ADR 0234's + tiers, the glossary's word). +- A **warrant** (new) is the router's record that a person chose one option of one ask: the ask, the + option, who chose (the operator, as a channel kind's identity), through which channel, the proofs, and + when. It is the only thing an asker acts on. +- **The router** keeps its sense (the `operator-channel` seat's holder). It now holds every ask and + issues warrants. It never performs an option. + +### 2. The boundaries + +| Party | Does | Never | +|---|---|---| +| **an asker** (any module; first the controller) | publishes an ask under its own name; reads its asks' state; acts on its own warrants with its own grants; records the warrant in its own record of the act | performs without a warrant; reads another asker's asks | +| **the router** (`operator-channel` holder, the `messenger` module) | keeps asks, tokens and the operator's identities; routes an ask to the channels whose capabilities meet each option's level; judges each answer; issues one warrant per ask | calls an asker's verb; knows a service's API | +| **a channel** (a module claiming a kind on `channel` and `intake`: `telegram`, `desktop`) | shows what the router sends, with the router's tokens on its buttons; passes back the token chosen and the sender as its service authenticated it | judges whether the sender is the operator; holds another channel's secret | + +### 3. The subjects, streams and buckets + +Every name below follows the mesh's existing shapes (`mesh.seat...…`, a module's +state bucket, the EVENTS stream). `` is the asking module's name; `` a channel's kind. + +| What | Subject or bucket | Kept in | Publisher (granted) | Reader (granted) | +|---|---|---|---|---| +| an ask | `mesh.seat.operator-channel.accept.ask.` | the seat's work queue `SEAT_OPERATOR_CHANNEL` (kept 7 days, removed when the router has recorded it) | the asker, its own name only | the router | +| an ask cancelled by its asker | `mesh.seat.operator-channel.accept.cancel.` | the same | the asker, its own name only | the router | +| the asks and their state | the router's bucket `asks`, key `.` | its own state, closed records kept 30 days | the router | the router; each asker reads its own prefix only (direct get) | +| answer tickets | in each ask's record in `asks`: one per option and channel kind, used once, dead when the ask closes | its own state | the router | the router | +| the operator's identities | the router's bucket `identities`, key `.` | its own state | the router | the router | +| a warrant, or an ask's end without one | `mesh.seat.operator-channel.event.decided.`, message id `.` | EVENTS (7 days), deduplicated by the message id | the router only | the asker, its own name only, through its durable consumer | +| show, edit or say on a channel | `mesh.seat.channel.accept..` | the seat's work queue `SEAT_CHANNEL`, one worker per kind | the router | the holder of that kind only | +| an answer chosen, a link asked, a message that failed, the channel's standing | `mesh.seat.intake.event..` | EVENTS | the holder of that kind only | the router | +| a code typed (a link code; later a TOTP code) | `mesh.seat.intake.proof.code.`, core request and reply | nothing: never on a stream | the holder of that kind only | the router, which answers | + +**The grants, composed by the controller as it composes every bus account:** + +- **A caller-named verb.** A seat may mark an accepted verb or an emitted event as named by its caller + (`ask.*`, `cancel.*`, `decided.*`). A module that uses the seat is granted to publish the accept, and to + subscribe the event, with its own module name as the last token, and no other. A tool runner carrying + modules is granted the same for each module it carries. The controller asks as `mesh-controller`. +- **A kinded bench** (ADR 0234 §2, built here for `channel` and `intake`). A claim carries `kind`; two + claims of one kind are refused. A holder subscribes its own kind's accepts, publishes its own kind's + events and proofs, and nothing of another kind. Each kind has its own worker on the seat's queue, so a + channel that is offline keeps its work and holds up no other. +- **A seat's `proof` verbs** are core request and reply on `mesh.seat..proof..`: no + stream covers them, so a code is never persisted (the mesh's rule: events carry no secret). A holder + asks with its own kind; the modules that watch the seat answer. +- **The seat declares all of it**: `kinded`, the verbs named `by-caller`, its `proofs`, and the holder's + `records` its users read under their own name. Only `channel` and `intake` may be kinded. +- **A claim of a kinded bench carries `kind` and `capabilities`** (`channel-capabilities/1`, ADR 0234 §2). + The controller names every kind, its holder and its capabilities in the membership of each module that + uses or watches the bench: that is where the router reads them, never from a channel. +- **A record readable by its asker.** The router's `asks` bucket is read by an asker under its own prefix + only: direct get on `$KV...>`. +- **No `answer` tool.** A tool runner may call any tool, so an answer that arrived as a tool call could + come from anyone. An answer arrives only as an intake event of a kind. + +### 4. An ask + +An ask carries: an `id` the asker chose, unique to it; the **question** in plain words, held to the plain +rule and the content rule (ADR 0253, ADR 0234 §6); its **options**, at most four, each a label, what it +does in plain words, its assurance level and an option id; **who may answer** (the role `operator`, the +only role today); an **expiry**; **what happens on expiry**, in words the operator is shown ("the delivery +stays held"); and optionally a subject it is about (a condition's key), so a newer ask about the same +thing replaces the older. + +- **Refused at once, in words, on the decided subject**: an ask whose words break a rule, one none of whose + options any ready channel can carry, or a fourth open ask from one asker. An option no channel can carry + now (`destroy` until a TOTP factor exists; `approve` while no account is linked) is not offered, and the + words say where it can still be given. +- **An authorising ask never defaults.** At its expiry the router ends it with `expired`; the asker does + what it said it would. +- **An approve option expires within 24 hours; a destroy option within 10 minutes of being shown** + (ADR 0234). + +### 5. The router judges an answer + +On an intake `choice` of kind K carrying token T and sender S, the router refuses at the first failure and +says the refusal on K's channel: + +1. T is known, unused, unexpired, and was issued for kind K; +2. the ask is open, and the option T names belongs to it; +3. S is on the router's list of the operator's identities for K; +4. K's declared capabilities, **read from the controller's record of its claim and never from the + channel**, meet the option's level: `acknowledge` needs `choice` and `exact-render`; `approve` needs + `verified-sender` as well (P1); `destroy` needs two proofs, one of them a code (P2); and for `approve` + and above, **the controller's `root-free` verb answers, at that moment, that no agent can become root + without a person** on the router's machine and on K's channel's machine (§8 rule 2, added on 2026-10-09: + an absent condition is no pass); +5. the ask is closed with a compare-and-set on its record, so a second answer loses. + +Then it publishes the warrant on `decided.`, and sends every channel that showed the ask an edit +naming the outcome and removing its buttons. + +**The desk declares no `verified-sender`**, so it answers only `acknowledge` options, as ADR 0258 decided. +Telegram declares it while placed where no agent runs as the operator, so it answers `approve`. + +### 6. The warrant, and the asker's act + +The warrant carries the ask's id and asker, the option, `by` (the operator, as a kind's identity: "the +operator, via telegram, user id verified"), the channel module, the proofs present, the assurance level +met, and when. It carries no secret. + +- **The asker acts once per ask id**, keeping that in its own record, and only on a warrant from its own + `decided` subject. A warrant that says no time it was given, or for an ask that says no expiry, is refused: + neither can be shown to be in time (the confirmation review of 2026-10-09). +- **A silence never takes an approval back** (the confirmation review of 2026-10-09, M1). The controller + asks a condition's acknowledgements (Silence) apart from its approvals (Release, Stop, Restart), and choosing + Silence silences the condition; the first build then took the approval ask back as no longer wanted, so an + acknowledgement, which a desk click may give, ended an approval. An approval asked about a condition that is + silenced since stays open, unchanged, until it is answered on a channel that proves who answered, or expires. +- **A warrant authorises one act, as the person was shown it** (added on 2026-10-09, before the code was + merged). Every option above `acknowledge` carries the digest of exactly what the asker performs when it is + chosen — the verb, the machine and every argument (`binds`); an ask without it is refused at the router. + The router puts the digest of the ask as it took it in the warrant. The asker acts only when that digest + is the one of the ask it keeps, and only when the act it is about to perform is the one the option bound: + an ask published again under the same id, or a record of the act changed after the ask, is refused and + nothing is done. **Both digests are over named fields in a canonical encoding** (each value written as its + length, a colon and its bytes, in a fixed order, versioned), never over a language's encoding of a + structure, so any language reproduces them and a field added to an ask changes no digest until it is + added to the encoding on purpose (the review of 2026-10-09). +- **The router publishes each warrant under a message id nobody can guess** (a random one per publish), on + the asker's `decided` subject, which only the router may publish; the asker deduplicates on its own record + (a compare-and-set claim), never on the bus's duplicate window. A guessed id published first by another + principal into the same stream would otherwise have the real warrant dropped as a duplicate (the review of + 2026-10-09). +- **Its record of the act names the warrant**: a hand-act's `by`, `via`, `ask` and `proofs` (ADR 0234 §8), + a delivery's transition, a silence's why. The hand-act log counts it as a person's decision, not a repair. +- **The controller only asks and acts.** For a condition that needs the operator it asks with the + condition's actions as options, and on the warrant it calls the verb it named, with its own grant. It + holds no channel, no identity and no factor. +- **A warrant is not yet the only way to these acts.** The acts a warrant authorises today — + `mesh-delivery.release` and `stop`, the controller's `plans` go and stop, `node-service-manager.restart`, + and `conditions silence` — are still verbs an agent may call directly with its own grant. Until + [research 039](../01-RESEARCH/039-a-tool-that-needs-a-persons-approval/00-overview.md) (a tool that needs a + person's approval) withholds them from every caller without a warrant, a warrant proves that the operator + chose an act, not that only the operator could have caused it. Said here so no reader takes the warrant for + that second guarantee. + + > **The mechanism changed — 2026-10-10, by [ADR 0286](0286-a-tool-or-verb-marked-in-its-manifest-runs-only-on-the-operators-approval-of-that-one-call-with-its-exact-arguments.md).** A tool or verb marked `needs-approval` in its manifest now runs + > only on a warrant for that one call, with its exact arguments, checked by the tool runner that serves it (or + > by the controller, for its own verbs). For a marked act the warrant is the only way; for the others this + > bullet stands. `mesh-delivery.release` is not marked first: issue 417 tracks how the controller's own warrant + > for it passes the mark. + +### 7. Who the operator is on a channel: linking + +- **An answer from an account that is not the operator's** is refused, and the operator is told on the linked + account which account tried (by name and number); the other account is told nothing (added on 2026-10-09: + a question forwarded, or a phone in other hands, is something the operator must hear of). +- **The router's list is the only list.** It is changed only by linking and unlinking, never by a setting. +- **Linking:** the operator sends the bot `/start`. The channel publishes `link.` with the sender + and the name the service shows. The router shows a six-digit code on the desk, with that name, valid + for ten minutes. The operator sends the code to the bot; the channel passes it on the `proof` subject; + the router links the identity when it matches, and says so on every channel. +- **A code is shown to a person or not at all** (added on 2026-10-09, after the review). The router issues a + code only while the desk channel says it is ready, and tells the sender the desk's reason when it is not; + a code is never queued for later. The desk is ready only while a named desk machine's notifier said within + five minutes that it is not paused (the notifier's `paused` event), and the code goes only to such a + machine; shown nowhere, it is said back to the router as failed. Every message carrying a code is marked secret: no + channel and no router record keeps a copy, the desk channel stores none and hands it to the node's + notifier as transient, and the notifier keeps it out of its history, where an agent could read it later. +- **The desk channel runs as an account of its own**, like the router and Telegram (§8 rule 1): the codes + pass through it. Registration holds every holder of a `private` kind to it, as it holds a `verified-sender` + holder (the confirmation review of 2026-10-09). +- **One identity per kind.** A second link of a kind is refused until the first is unlinked. +- **Unlinking is an approve ask, answered on the account that is linked** (revised after the security + review of 2026-10-08). It is never a plain tool: whoever unlinks the operator's account could link one of + their own in its place. The notice of an unlink, and of any link for an hour after it, is sent to the + account being replaced by name, before anything changes, and to every other channel. +- **A link within a day of an unlink waits for the replaced account's approval**, asked there. When that + account is lost with its phone, the operator acts as root at the router's machine (break-glass, as ADR 0234 + §8 does for a lost factor): a regular file `unlink.` or `allow-link.`, owned by root, in the + router's own state directory (mode 0700, owned by the router's account). The router acts on it within a + minute, deletes it, ends every open ask of its own, and announces on every channel what root did. A file of + any other owner, or not a regular file, is refused and said: an agent without root cannot make one. +- **Linking is bounded**: at most three links asked per account an hour and ten per kind, counted in the + router's own state so a restart forgets nothing (a count that cannot be read or written refuses the link); the code's notice on the desk names the account by + the name and the number the service gives, and only a channel declaring `private` (the desk) may show a + code. **A residual risk of the bound, accepted** (the confirmation review of 2026-10-09): anyone may make + throwaway accounts and send the bot `/start` ten times in an hour, so the operator's own link is refused + for the rest of that hour. It denies a link, never makes one, and every refused link is said; the operator + links when the hour has passed. A bound per sender alone would let the same throwaway accounts flood the + desk with codes, which is worse. +- **The ask to unlink or relink binds the account**, not only the kind (added on 2026-10-09): a warrant for + unlinking one account never unlinks the one linked after it. A link re-checks that its kind has no account + when it is made, and an account just unlinked cannot link again at once from a code shown before. +- **An account still in its first hour can be unlinked without its own approval** (added on 2026-10-09, after + the review): it cannot approve yet, so its consent protects nothing. When another account sends `/start` + while one is in its first hour, the router asks at the desk, as an acknowledgement bound to that account, + whether to unlink it, and tells the sender so. If the hour has passed when the answer arrives, the unlink + is refused. After the hour, unlinking needs the linked account's approval, or break-glass. +- **An unlink at the desk in the first hour bars nothing** (the confirmation review of 2026-10-09, finding + N1-router). A desk click is anyone's at the desk machine — a program there can forge one — and the first + build barred the unlinked account for a day and made the next link wait for it. Another account's `/start` + offered the desk the unlink, a forged click took it, and the operator's own account stayed barred, again + and again, so the operator could never approve. Now such an unlink lets any account, the operator's too, + link again at once with a code shown on the desk. **The day's bar is set only by an unlink the account + itself approved, or root's**; root's `allow-link` at the router's machine lifts any bar. What is said on + every channel names which it was. +- **A new link answers `acknowledge` at once and `approve` only after one hour.** The hour runs from the + moment the link is made, is kept in the identity's record, and is said in the announcement ("it can + approve from 15:40"). An approve answer inside the hour is refused and said on the channel. Decided by + the operator on 2026-10-08, until an authenticator-app code exists. +- **Once a TOTP factor exists**, linking and unlinking are destroy asks (ADR 0234 §8). +- **A residual risk is accepted until then:** on the X11 desk an agent running as the operator can read the + code and send it from an account of its own. It is said loudly on every channel when it happens; during + the hour the account cannot approve, and the operator can unlink it from the desk without its approval (the + rule above — corrected on 2026-10-09: before that rule, the hour gave no way to unlink an account that + would not consent); and the factor closes it. + +### 8. Who may say what on the bus: the trusted parties' own accounts + +*Revised after the security review of 2026-10-08; the revision accepted by the operator the same day.* The first version of +this section rested on the tool runner running as root with a credential only root reads. **On the control +node it runs as the operator's account, its bus credential belongs to that account, and that account may +become root without a password.** Every agent runs as that account. So an agent there could read the runner's +credential and publish anything any carried module may: a warrant, an answer from Telegram, a link. The +service account of the first version kept the bot token from nobody. + +The revision rests on three rules. + +**1. The trusted parties hold bus accounts of their own, never a runner's.** + +- The **router** (`operator-channel`'s holder) and **every channel that declares `verified-sender`** run as + their own process, as their own service account (`messenger`, `telegram`), with a bus account of their + own: an own secret `broker`, owned by that account, mode 0600. The manifest says so with `runs-as`; the + controller composes for such a module a runtime of its own — the machine's runtime program, run as that + account, serving that module alone on that credential — and the machine's runtime never carries it. +- **The controller refuses to compose** into any runner's user, or into any principal whose process runs as + the node's operator account: + - publishing on a by-caller event of `operator-channel` (a warrant); + - writing the router's state (asks, identities); + - publishing on, or answering, a kinded bench's subjects for a kind declaring `verified-sender`. + A manifest asking for it is refused at registration, and a runner carrying such a module is refused at + composition, naming the module. +- **Proofs are answered by the holder of `operator-channel` alone**, not by any module watching the bench; + and **a kind's accepts are published by that holder alone**, not by every user of the bench. +- **The tool runner hands a bundle no bus variable** of its own: the credential and the bus's address are + taken out of every bundle's environment. + +**2. `verified-sender` is honoured only where the controller can show who holds it.** + +The controller passes a kind's `verified-sender` to the router only when its holder runs as its own +account, with its own bus account, on a machine where **no account an agent runs as can become root without +a person**. Otherwise it strips the capability from the membership, and the router answers that kind only +as far as acknowledgements, saying why. The same check holds the router: while it fails on the router's +machine, the router issues no approve warrant, and the condition says what to change. + +*Revised on 2026-10-09, after the review of the first build.* **Only a positive, recent measure is a pass; +the absence of a condition never is.** The first build let the router approve while no `agent-root` +condition was open — so before the first self-check, after a probe error, with the conditions view lost, or +with the router not knowing its own machine, it approved. Now: + +- **The controller's `root-free` verb** answers, for each machine asked, whether no agent can become root + there without a person, judged at the moment it is asked. It is a pass only when the machine names an + account its agents run as, the machine's node-engine judged that account unable to become root (ADR 0266: + the node-engine reads it as root, on every look, and the controller takes a verdict at most fifteen + minutes old), and the login shell's `execute` is not served there (ADR 0268). Anything else — no agent + account named, so agents run as the operator; a verdict missing, stale or unhealthy; a store or the bus not + answering — is not a pass, and says why. +- **The measure is the node-engine's, not the measured account's.** The first build asked the sudo module, + served by the machine's runtime — whose account is the one an agent's calls run as — so the account + being measured answered its own measure. The node-engine runs as root, which an agent that cannot become + root cannot be. The machine's runtime account is counted as able to become root, always: the mesh's + acting tools give it passwordless sudo. +- **The router asks `root-free`** for its own machine and the answering channel's, when it shows an + approve option and again when an approve answer arrives; an error, a missing machine, or a router that does + not know its own machine is not a pass. **The controller asks it too when it composes**, and strips + `verified-sender` from a kind without a pass; the router's live question covers a pass that ends between + compositions. The `root-not-free` condition is raised on the same judgement, so the condition and the + verb never disagree. +- *Added after the confirmation review of 2026-10-09.* **Only the serving controller answers `root-free`.** + Composed from the controller's own manifest, the module principal of the machine running the controller + and that machine's runtime were granted the controller seat's verbs too, and that runtime's credential is + one an agent there may hold. The controller's own seat is now served by the controller's principal alone, + in grants and in memberships, and a grant test composes every principal and finds exactly one that may + answer `root-free`. **A machine waiting for its node-engine's first setuid search is not root-free** + (ADR 0266 keeps the self-check quiet for that bound after a restart; `root-not-free` keeps the same quiet, + and the verb still answers no). The condition has its own key, apart from ADR 0266's + `agent-can-become-root`, which a second probe wrote under the same key with other words. **The router asks + `root-free` before it takes its work**, never while holding it — a controller slow to answer held every + ask, answer and link behind it — and judges an answer with the word it was given then; inside the work it + reads only a word under half a minute old, and an ask that failed drops the last word. + +**3. What remains, said plainly.** + +- **Root on the router's or the channel's machine defeats all of this.** The self-check raises an urgent + condition while an agent can become root there without a person; on the control node today it can (the + operator's account has passwordless sudo, measured on 2026-10-08). **The operator chose, on 2026-10-08, + that agents on the control node run under an account of their own without sudo.** +- **An agent reaches root through the mesh's own tools too.** Any machine's tool runner may call any tool + on any machine, and the login shell seat's `execute` runs a command as the account the runner runs as — + on the control node, the operator's account, with passwordless sudo, which the mesh's acting tools rely + on (measured on 2026-10-08). So an agent on any machine can run a root command on the control node + without touching its own account. The self-check therefore counts both ways in: an account an agent + runs as that can become root without a person, **and** a tool an agent may call that runs a command + given to it as an account that can. It fails until both are closed; the second is decided separately + (the login shell's reach, and what an agent's calls are granted). +- **Since 2026-10-09 the second way is closed on the control node**: + [ADR 0268](0268-the-login-shells-execute-is-withheld-on-the-control-node-until-a-call-to-it-needs-a-persons-approval.md) + withholds `execute` there through its holder's setting. The self-check counts the login shell only where + `execute` is served, failing closed: served while the holder's setting for that machine is `serve`, or + while the bus hears the verb answered there, or while the bus cannot be asked. The first way closes with + ADR 0266 (hq pull request 225, not yet merged): agents on the control node run under an account of their + own, without sudo. +- **What cannot be measured is not a pass** (added on 2026-10-09). A machine where a trusted party runs and + the node-engine's verdict on its agent account is missing, stale or not given raises the same urgent + condition, saying it was not measured, and `root-free` answers no for it. +- **A known gap, accepted by the operator on 2026-10-09: the operator's own Telegram client on a machine + where agents run as the operator** (issue 344, hq pull request 244, not yet merged). The operator's Telegram may also run on such a + machine — their laptop, with Telegram Desktop. There an agent can read the client's data (found readable by + everyone) and drive the X display, and tap Approve as the operator. The router cannot tell such a tap from + one on the phone. Closing it needs agents under an account of their own on that machine, without access to + the operator's X display, and the client's data readable by the operator alone; none of it is built or + measured now, and approvals are not held back for it. Until then, approve from the phone. +- **A known gap, accepted on 2026-10-09: the controller's credential can rewrite the router's state** + (issue 344, hq pull request 244, not yet merged). The controller composes every account and grant on the bus, so a grant withheld from + its own credential — no write to the router's identities or asks — would be withheld by the very principal + that writes the grants, and undone by its next composition. Moving the router's buckets where the + controller cannot write therefore protects nothing against a controller that is not trusted, and is not + done; the controller is trusted with every record, and root on the control node is the mesh. +- **The controller holds the whole JetStream API** today. Narrowing it to the streams and buckets it owns is + planned work of its own; until then the controller is trusted with every record. +- **On the X11 desk an agent can read a code as it is shown or typed** (ADR 0234's accepted risk). The hour + after a link, the replaced account's approval and the announcements bound what a stolen code can do. + +### 9. What is superseded in ADR 0234 + +- **§8 "an ask whose answer performs an action is held by the controller", the controller's verbs + `authorise request`, `authorise answer` and `authorisations`, and the controller performing:** the + router holds the ask and issues a warrant; the asker performs. +- **§3 "the controller's list of the operator's identities"** and §8 "the factors' secrets are the + controller's own": the identities and, when built, the TOTP factor and its recovery codes are the + router's own state, with ADR 0234's rules on showing them once, at a terminal, never on a channel. +- Everything else in ADR 0234 stands: the tiers and proofs, the kinded benches, the content rule, the + desk never authorising, long polling, every tier completable on the away channel. + +### 10. The first channels and the order of work + +1. **The controller:** the kinded bench with `kind` on a claim; the `channel` and `intake` seats; the + caller-named and per-kind grants; `proof` subjects; the asker's read of its own prefix. +2. **The tool runner (mesh-tools)** carries a bundle's seat traffic: publishing an accept or an event + with its own name or kind, taking a held seat's work, asking and answering a proof, a compare-and-set + on its state, and serving one module alone when it runs on that module's own account. +3. **The shared library (mesh-sdk):** an asker's client (ask, cancel, read its state, take its warrants) + and a channel's (take show, edit and send for a kind; publish a choice or a link; send a proof). +4. **The messenger** becomes the router of this record: asks, tokens, identities, judging, warrants, + expiry, linking. Its Telegram client and its desk adapter leave it. +5. **`telegram`**, a new module claiming kind `telegram` on both benches: long polling, buttons carrying + only a token, `/start` linking, the client's defects of to-be 46 §12 fixed in the move (D1, D4 to D8, + D10; D2 and D3 are the router's, D9 the watcher's), its token as its own secret. **`desk-channel`**, a new module claiming kind `desktop`. +6. **The controller as an asker**: conditions that need the operator are asked; it acts on warrants and + records them; the words "from the mesh MCP server" give way to the router's own line, below. +7. **The vault** gains a verb by which the operator gives a module's own secret through a hidden prompt + on the desk, so a bot's token never passes through an agent's session. *As built (2026-10-09):* the + controller's `give`, through its `secret accept` path. It refuses `broker` and any secret the mesh makes + itself; the prompt is written by the desk's launcher from names alone (module, secret, machine), never + from words a caller chose, and says the controller asks; the bus lets only the controller publish the + prompt's verb; no secret value is accepted through a verb otherwise; and every value given raises the + urgent condition `secret-given` on every channel until the operator silences it. + **A trusted party's secret is never given at a desk** (the confirmation review of 2026-10-09, finding + N1-give). The desk prompt's answer comes back over the desk machine's bus, on the subject that machine's + runtime answers, and on a desk machine agents run as the operator, who holds that runtime's credential: + an agent called `give` for the bot token and answered the prompt first with a token of its own, sealed to + the call's key, and the channel the operator's answers are proven on became the agent's. Now the secret of + a module running as an account of its own (the router, Telegram, the desk channel) is refused at the desk + before anybody is asked, naming the line to type at the controller's terminal instead — + `nox secret accept telegram telegram-token` (`nox` is `mesh-cli`'s alias; on the control-node, + from the operator's own account, it is the controller's terminal, ADR 0272). There it is announced + on every channel, the one being replaced among them (it still runs on its old token until the next push), + **before** it is kept, and not kept when that announcement fails. A grant test holds that only the desk + machine's launcher, carried by that machine's runtime, may answer its prompt; for any other module's + secret that remains the accepted risk of a desk where agents run as the operator. + + > **The mechanism changed — 2026-10-10, by [ADR 0277](0277-a-trusted-setting-is-proposed-through-a-verb-and-set-on-the-operators-warrant-and-a-modules-own-secret-is-asked-for-and-typed-at-the-desk.md).** The same prompt is opened by the + > controller's `secret-ask` verb from anywhere on the mesh, with the desk the module's machine unless named, + > and `give` composes its line. It is bounded (one open ask per secret, three an hour, read from the store + > before the prompt opens) and the prompt names who asked, from the bus's word on the caller alone. What + > stands: the value is typed at the desk, sealed to the one call, never in an argument, a log or an event; + > a trusted party's secret is still the terminal's. + +**Where an ask can be answered is said by the router**, from the channels that qualify: "Answer on +Telegram, or from the mesh MCP server." The asker's words never name a channel. + +## Consequences + +- **A channel is a module and nothing else changes when one is added.** Signal or mail is a module + claiming a kind and declaring capabilities. +- **Any module may ask a person**, through one seat, and acts on the answer with its own grants. ADR + 0258's desk Silence becomes an ask of the controller's, and the messenger loses its grant to + `conditions silence`. +- **The controller loses three verbs it never built** and gains an asker's client. +- **The router's state grows**: asks, tokens and identities, each a bucket of its own. +- **Harder:** an asker must keep which warrants it acted on; the shared library carries it. +- **Harder:** the controller composes two new grant rules, and an asking module must use the seat in its + manifest to be granted anything. +- **The bot token moves** from the messenger's own secret to the `telegram` module's, given at the desk + (`give`, §10 step 7). **Nothing the channel or the router trusts is a setting** + ([issue 342](../04-ISSUES/342-a-password-is-taken-as-a-setting-and-outranks-the-vaults/00-report.md): a + password, token or key is not a setting). The operator's Telegram account is the router's identity list, + written by linking alone; no chat id is configured anywhere. The two settings these modules keep — the + router's time zone and the desk channel's machines — are the terminal's, as every mergeable file's are + since [issue 340](../04-ISSUES/340-an-agent-can-plant-a-hook-in-every-agent-session-on-the-mesh/00-report.md). +- **Accepted risk:** a linked Telegram account, if taken, can approve (announced, reversible), never + destroy. Linking before a TOTP factor exists carries the X11 risk in §7. + +## How it is checked + +| Rule | Checked by | +|---|---| +| An asker publishes asks only under its own name, and hears only its own warrants | controller grant tests: a module using `operator-channel` is granted `accept.ask.` and `event.decided.`, no wildcard; the lab's proof `asks` publishing under another name on a real bus is refused by the server | +| Only the router publishes a warrant | controller grant test: `event.decided.*` is in no principal's publish list but the holder's | +| A channel publishes only its own kind's answers, links and proofs | controller grant tests over a kinded claim; a second claim of a kind is refused | +| A code is never on a stream | controller test: no stream's subjects cover `mesh.seat.*.proof.>` | +| A token is used once, expires, and is bound to its ask, option and kind | router tests: the same token twice; a token past its expiry; a token from another kind; a token of a closed ask | +| An answer from an identity not linked is refused and said | router test | +| A level a channel's declared capabilities do not meet is refused | router tests: an approve option answered from `desktop`; capabilities taken from the controller's record, a channel's own claim ignored | +| A second answer to one ask loses | router test: two answers raced; one warrant | +| An authorising ask never defaults | router test: expiry publishes `expired`, never an option | +| An ask no ready channel can carry is refused at once, in words | router test | +| A condition that needs the operator and could not be asked (no router, or the ask refused) is a condition of its own, `asks-undelivered` | controller test (`TestAnAskThatCannotBeDeliveredIsSaid`) | +| Linking needs the code shown on the desk, and is said on every channel | router tests: a wrong code, a late code, a second link of a kind | +| A new link approves only after one hour; it acknowledges at once | router tests: an approve answer at 59 minutes refused and said, at 60 accepted; an acknowledge at once accepted | +| No runner's user, and no principal running as the operator's account, may publish a warrant, write the router's state, or speak for a verified-sender kind | controller composition test over every such grant; registration refuses a manifest asking for it | +| `verified-sender` reaches the router only from a holder with its own account on a machine no agent can become root on | controller test stripping the capability; self-check probe raising the condition | +| Unlinking and a relink within a day are approve asks answered on the linked account; the notice reaches it first | router tests, with a fake channel that honours the addressee | +| No bus variable reaches a bundle | tool runner test over a bundle's environment | +| A module of its own account is served by a runtime of its own, as that account, on its own bus credential, and never by the machine's | controller composition test (`runs-as`); registration refuses root, an account the module does not make, secrets it does not own, no broker; the tool runner test that such a runtime serves that module alone | +| While an agent can become root without a person where the router or a verified channel runs, it is said and nothing proven there approves | controller probe test (fails with agents as the operator's account, and with the login shell running as an account with passwordless sudo; passes once neither holds); the sudo module's `sudo_escalation` test; router test refusing approval | +| A tool call never publishes an intake event or a proof | tool runner test: only a bundle's own channel reaches the seat publish, for its own kind | +| The asker acts once per warrant and records the warrant | shared library test; controller test reading the hand-act after a warrant | +| A warrant authorises only the act its option bound, for the ask as it was shown | shared library tests (`TestAWarrantAuthorisesOnlyTheActItsOptionBound`); controller test (`TestAWarrantPerformsOnlyTheActItsOptionBound`); router test (the warrant names the ask's digest) | +| Only the serving controller answers `root-free`; a machine waiting for its first setuid search is not root-free | controller grant test `TestOnlyTheServingControllerMayAnswerRootFree` (one answerer); `TestAMachineWaitingForItsFirstSetuidSearchIsNotRootFree`; `TestRootNotFreeIsQuietWhileTheFirstSearchRunsAndKeyedApartFromDA` | +| A desk unlink in the first hour bars nothing; root lifts any bar (N1-router) | router tests `TestAnUnlinkFromTheDeskInTheFirstHourBarsNothing`, `TestBreakGlassLiftsABarOnRelinking` | +| A trusted party's secret is never given at a desk; only the desk machine's launcher answers its prompt (N1-give) | controller tests `TestATrustedPartysSecretIsNeverTakenAtADesk`, `TestOnlyTheDeskMachinesLauncherMayAnswerItsPrompt` | +| At the controller's terminal, a trusted party's secret is announced on every channel before it is kept, and not kept when the announcement fails; a machine the mesh does not know is refused before anybody types (N1-give) | controller tests `TestATrustedPartysSecretGivenAtTheTerminalIsAnnouncedBeforeItIsKept` (fails when the announcement before keeping is removed), `TestAGiveNamingAMachineTheMeshDoesNotKnowAsksNobody` | +| A silence never takes an approval back (M1) | controller test `TestAnAcknowledgementNeverSharesAnAskWithAnApproval`, with the condition silenced as the controller silences it | +| The router never waits on the controller while holding its work | router test `TestTheRootFreeWordIsNotAskedWhileTheWorkIsHeld` | +| The self-check counts the login shell only where `execute` is served, and what it cannot measure fails | controller probe tests (`TestTheLoginShellCountsOnlyWhereExecuteIsServed`, the unmeasured case) | +| The whole flow, before anyone uses it | the lab's proof `asks` (mesh-lab): a real bus with the controller's composed grants, the router and the Telegram channel in runtimes of their own accounts, a fake Telegram API — approve acted once, decline, expiry, a replayed tap, a tap from another account, a forged warrant, an ask under another name, an ask from a module that may not ask | +| The controller holds no channel code | review; the controller's module imports no channel package | +| End to end | live acceptance after rollout (mesh-lab `asks/live-acceptance.sh`): a **rehearsal** (`rehearse`), started at the controller's terminal alone, approved on the phone, acted on once and performing nothing; the hand-act reads `warrant rehearsal rehearsal=approve`, by "the operator, as telegram identity …", via "telegram (telegram), user id verified", proofs P1 | +| An answer from another account is refused and the operator is told which account tried | router test (`TestEveryAnswerThatIsNotTheOperatorsIsRefusedAndSaid`); the lab's proof | +| A message carrying an ask's answers is never folded into a digest | router test (`TestAnAsksMessageWithOneAnswerIsNotFoldedIntoADigest`); the lab's proof, where it was found | + +## References + +- [ADR 0234](0234-the-mesh-holds-a-conversation-with-its-operator.md) (the conversation, the tiers and + proofs; §3 and §8 in part superseded here) +- [ADR 0258](0258-a-desk-click-performs-only-an-acknowledgement-and-an-answer-is-a-one-time-ticket.md) + (the desk performs only an acknowledgement; one-time answer tokens) +- [ADR 0253](0253-a-condition-says-itself-to-the-operator-in-plain-words-beside-its-summary.md) (a + condition's actions and plain words) +- [ADR 0246](0246-a-seats-new-verb-is-promised-before-it-is-required.md) (a new seat's verbs are required + from the start) +- [ADR 0255](0255-the-bar-takes-blocks-as-data-from-the-module-that-knows-the-hardware-placed-where-the-machine-has-it.md) + (a generic boundary in front of pluggable modules) +- [To-be 46](../03-DESIGN/01-to-be/46-the-conversation-with-the-operator.md) §3, §4, §10, §11 and §13 +- [Issue 320](../04-ISSUES/320-the-operator-could-not-read-the-meshs-notifications/00-report.md), + [issue 324](../04-ISSUES/324-a-desk-click-could-answer-in-the-operators-name/00-report.md) diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0272-the-operator-reaches-the-controller-with-mesh-cli-on-every-node-and-a-node-is-the-terminal-only-where-no-agent-runs-as-its-operator.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0272-the-operator-reaches-the-controller-with-mesh-cli-on-every-node-and-a-node-is-the-terminal-only-where-no-agent-runs-as-its-operator.md new file mode 100644 index 00000000..20dfa106 --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0272-the-operator-reaches-the-controller-with-mesh-cli-on-every-node-and-a-node-is-the-terminal-only-where-no-agent-runs-as-its-operator.md @@ -0,0 +1,369 @@ +--- +topic: the tiers +status: accepted +date: 2026-10-09 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0181-the-operator-account-is-a-node-fact-and-a-home-is-a-placement-root.md +--- + +# 272. The operator reaches the controller with `mesh-cli` on every node, and a node is the controller's terminal only where no agent runs as its operator + +## Context + +Some changes to the mesh are made **at the controller's terminal** only: + +- the places and accesses a module's settings name, and the keys a provider's trust or a trusted file reads + ([issue 339](../04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/00-report.md)); +- the agent module's managed settings, tool servers and shared text + ([issue 340](../04-ISSUES/340-an-agent-can-plant-a-hook-in-every-agent-session-on-the-mesh/00-report.md)); +- a node's agent account ([ADR 0266](0266-on-the-control-node-agents-run-under-an-account-of-their-own-that-cannot-become-root-without-a-person.md)), typed as + `mesh-controller node agent-account `. + +In the controller's code "the terminal" is a negative: a command line that did not come through a verb. The +serving controller names the verb in the environment of every command it runs for one (`MESH_VERB`, issue 327), +and a terminal-only change is refused when that variable is set. Nothing says who the terminal is, and nothing +lets the operator reach it. + +On 2026-10-09 the operator typed `mesh-controller …` on the control-node and the shell answered +`zsh: command not found: mesh-controller` +([issue 343](../04-ISSUES/343-the-controllers-terminal-has-no-command/00-report.md)). The controller runs as a +mesh process: the node-engine unpacks its bundle under the mesh's own directory, and writes a unit that runs it as +the `mesh-controller` account with an environment composed from the controller's module (its stores' credential +files under a directory only that account may enter, the stores' and the bus's ports). Nothing puts a command on +any `PATH`, and running the binary by hand would mean rebuilding that environment under `sudo`. + +The operator's directions of 2026-10-09 shape what follows: no wrapper script and no link (the mesh creates no +symbolic links); the operator's command is a real program, its own module, on every node; it reaches the +controller over the mesh, not by running the controller's binary; and the person is proven by the account the +command runs from, vouched for by the operating system, never by a credential file a copy of which is as good as +the original. + +## Considered Options + +### How the operator reaches the controller + +1. **A link to the controller's binary on `PATH`.** Refused by the hard rule, and not enough: the binary needs the + unit's account and environment. +2. **A wrapper the controller's module places on the control-node**, running the controller's binary as its + account with the unit's environment through `sudo`. Rejected. It is a second copy of the unit's environment to + keep in step (or a new process field for the node-engine to render), it reaches the controller from the + control-node only, and its proof is `sudo`, which every agent on the control-node holds until ADR 0266 is + switched on. +3. **A program on every node that asks the controller over the mesh.** Chosen. The controller already runs every + command line a verb asks for in a fresh process of its own binary, with its own environment (`runVerb`); one + more way of asking needs no copy of that environment anywhere. + +### Where the program comes from + +1. **A second artifact of the controller's module.** Rejected: that module is assigned to the control-node only, + so its files reach no other node. +2. **A module of its own, assigned to every node.** Chosen. It builds from its own directory, so a change to it + rebuilds it alone and a change to the controller does not rebuild it + ([ADR 0267](0267-an-artifact-is-rebuilt-only-when-its-build-source-changes-and-a-module-moves-only-along-build-dependencies-on-artifacts.md)). + +### How the person is proven + +1. **A credential file in the operator's home.** Rejected. Any agent running as the operator reads it, and a copy + works anywhere. +2. **`sudo`.** Rejected as the proof. It asks the account's password, or nothing where the rule says so, and + agents on the control-node hold the same `sudo` today. +3. **The account the program runs from, read by the node-engine from the kernel.** Chosen. The program connects to + a local socket the node-engine serves; the kernel tells the node-engine the connecting process's user id + (peer credentials). Nothing is presented, so nothing can be copied. The node-engine is on every node and runs + as root, so it is the one part of the mesh that can vouch for a local account. +4. **A second factor**: a one-time code or an approval on a verified channel + ([ADR 0234](0234-the-mesh-holds-a-conversation-with-its-operator.md), ADR 0259). Not now. It is the route for + a node whose agents must share the operator's account, and it waits for the approval flow (research 039). + + > **The mechanism changed — 2026-10-10, by [ADR 0286](0286-a-tool-or-verb-marked-in-its-manifest-runs-only-on-the-operators-approval-of-that-one-call-with-its-exact-arguments.md).** The approval flow for one call exists as a decision: a + > marked tool or verb runs only on the operator's warrant for that call. This option's choice stands. + +### Which account on which node is the terminal + +1. **The operator's account on any node.** Rejected. On every node but the control-node, and on the control-node + until ADR 0266 is switched on, agents run as that account. An agent would be the terminal. +2. **The operator's account on the control-node only.** Safe, and too narrow for later. +3. **The operator's account on a node where agents cannot be that account**: the control-node, and any node that + names an agent account of its own judged unable to become root without a person (ADR 0266 §1, §3, §4). + Chosen. + +## Decision + +### 1. `mesh-cli` is a module and the command it places + +**`mesh-cli` is a module of the catalogue, assigned to every node.** It builds one Go bundle from its own +directory, and an `archive` resource unpacks it into `/usr/local/bin`, so the program is +`/usr/local/bin/mesh-cli` on every node. It is the bundle's own file: no link, no second copy. The node-engine +moves it in beside what is there, refuses to write over a file it did not place, and takes out only what it +placed when the module goes. + +**`nox` is its alias**, `alias nox=mesh-cli`, contributed by the module to zsh's `normal` slot +([ADR 0204](0204-a-module-contributes-shell-code-to-the-login-shell-in-named-slots.md)). `/usr/local/bin` is +already on every account's `PATH`, so the module contributes no environment. The alias exists in interactive zsh +only: a script, `sudo`, a cron line and any document meant to work everywhere say `mesh-cli`. + +### 2. What the operator types + +`mesh-cli ` takes the controller's own command line, in the words the controller's commands and +their refusals already use: `mesh-cli settings set zsh '{"execute":"withhold"}' --node ` is +`settings set zsh … --node ` asked of the controller. It is run as the operator's own account, without +`sudo`: root is not the operator's account, and is refused (§4). + +`mesh-cli` holds no credential and reads no file of the mesh. It does one thing: it asks the node-engine on its own +machine over a local socket, and prints what comes back. A process that carries `MESH_VERB` (a verb's own process) +is refused at once, and told why. **That refusal is a courtesy, not the gate**: a process can unset a variable. +The gate is the controller's judgement of the account the node-engine vouches for (§4). + +### 3. The node-engine vouches for the account; the socket grants nothing + +**The socket grants nothing; the controller judges.** What the node-engine adds is a fact only it can read, the +asking account, and it decides nothing with it. The node-engine serves a socket every local account may connect +to. For each connection it: + +1. sets the connection's deadline before anything else, and reads the connecting process's user id and process id + from the kernel (`SO_PEERCRED`); +2. refuses a process whose mount or user namespace is not the engine's own: a container running with the + operator's user id shares the number, not the machine; +3. takes one of at most two lines at once for that user id and eight for the machine, and answers *busy* at once + when there is none, so one account cannot hold the socket from another and accepting never waits; +4. names the account from the machine's user database, reads the line, and sends the line, the account and the + user id to the controller on `mesh.control..cli`, as a request on core NATS. Only that node's engine may + publish under `mesh.control..`: its bus account is granted that prefix and nothing of any other node's, + and the writers table names the subject as the machine's own, so **the node is a fact the bus server enforces + and the account is a fact the kernel gives**. The CONTROL stream names its own subjects, so the request is + never persisted; +5. follows the line to what came of it (§5) and answers the program. + +Whether the node's agents run under an account of their own is not in the request. The node-engine already +says it: its judgement of the agent account is in its health statement (ADR 0266 §3, §4), and the controller reads +its own record of that statement, so a request cannot claim a judgement. + +The node-engine is trusted for its own node. Root on a node could claim any account there; root on a node that is +the terminal is already the operator's, and §4 makes no other node the terminal. + +**This is no listening control surface** in [ADR 0004](0004-a-node-and-how-it-joins.md)'s sense, and that record's +rule stands. The socket is a file under the node-engine's runtime directory with no network address, so nothing +dials it from another machine. Nothing sent to it acts on the node: the node-engine runs nothing it receives and +changes nothing; it adds what the kernel says and sends the request out over its own link, to a controller that +judges it like any other caller. Only the node's operator account is answered in phase 1 (§4), and it already +reaches the mesh's verbs through the mesh MCP server. + +### 4. The controller judges, and runs the line + +A line from node N, vouched as account A, **runs as the controller's terminal** when A is N's operator account +([ADR 0181](0181-the-operator-account-is-a-node-fact-and-a-home-is-a-placement-root.md)) and either: + +- N is the control-node: the one node the controller's module is assigned to. Two or more, and nobody is the + terminal; or +- (phase 2) N names an agent account of its own (ADR 0266 §1), and the controller's own record of the engine's + judgement, heard within ADR 0266 §4's fifteen minutes, says it cannot become root without a person. + +The controller runs it as it runs a verb's line, in a fresh process of its own binary with its own environment, but +without `MESH_VERB` and without the mark ADR 0266 puts on everything the serving controller starts, and records the +caller as *A through mesh-cli on N*. So the line reads as the terminal wherever the controller asks whether a +command was started there. + +**Which node is the control-node does not follow a verb.** Whoever chooses where the controller's module runs +chooses the terminal, so the module is assigned and unassigned at the controller's terminal only; `assign` and +`unassign` naming it through a verb are refused with the reason. + +**A line from N's operator account that is not the terminal is an ordinary call.** A `settings` line is composed as +the settings verb composes its own and meets that verb's refusals, the terminal-only keys among them. Any other +line meets exactly what the generic `command` verb does with a line: only its reading forms run +([ADR 0266](0266-on-the-control-node-agents-run-under-an-account-of-their-own-that-cannot-become-root-without-a-person.md) §7), and +never a command that is the terminal's alone — `node account` and `node agent-account` above all, which ADR 0266 +makes terminal-only, `token`, `operator`, a secret's accept, recover or export. It runs with `MESH_VERB=mesh-cli`, so every terminal-only change is refused +with the controller's own reason. The answer adds one sentence saying why this account on this node is not the +terminal: the node's agents may run as its operator. That account already reaches the mesh's verbs through the +mesh MCP server, so this widens nothing. + +**A line from any other account is refused, and runs nothing**: root, every service account, and **in phase 1 the +agent account too**. The answer says which account asked and that `mesh-cli` answers the node's operator account. +Phase 2 answers a node's agent account as an ordinary call (§7). + +**The control-node rule is no wider than today, while today lasts.** On the control-node the operator's account +holds passwordless `sudo`, and until ADR 0266 is switched on agents run as that account: either could run the +controller's binary as root with its environment, which is the terminal. `mesh-cli` gives that account the same +reach and no more. Once agents there run as `agent`, `agent` is refused. The claim rests on two measured facts and +fails if one changes alone: if the operator's account lost its passwordless `sudo` while agents still ran as it, +`mesh-cli` would give those agents a terminal they no longer had (§ How it is checked). + +> **Progressive insight — 2026-10-09.** Three facts of the building, after the record was accepted; the decision +> stands. +> +> 1. **How the terminal's line is told from every other.** ADR 0266 marks everything the serving controller starts +> as never the terminal (`MESH_SERVED_BY_THE_CONTROLLER`). A line mesh-cli runs as the terminal therefore runs +> without that mark and without `MESH_VERB`, and carries a mark of its own, `MESH_CLI_TERMINAL`, which the +> controller's check of where a command was started reads as the terminal. The controller says the terminal +> explicitly when it composes a line's environment (`commandEnvironment` in mesh-controller +> `cmd/mesh-controller/seatverbs.go`), and strips `MESH_CLI_TERMINAL` from every other line it runs — a verb's, +> an ordinary mesh-cli line — whatever its own environment holds. A test sets the mark in the serving +> environment and fails when that strip is taken out. +> 2. **Only a login session is the terminal.** §4 said the control-node's operator account; the tool runner and +> the account's user units run as that account too, and are not a person at a terminal. The node-engine reads the +> asking process's cgroup and sends its login session's scope (`session-.scope` under the account's own +> `user-.slice`, which logind's own record must say is this account's, of class `user`), read the moment +> the connection is accepted, or nothing; the controller makes a line from no login session an ordinary call. +> Built in mesh-host and mesh-controller on 2026-10-09, each with a test that fails without it. **What it does +> not do:** a login session includes every process started in it, agents started from the operator's shell +> among them, and any process of the account can reach one (a `tmux` server, a line in `~/.zshrc`). So the rule +> keeps services from being the terminal, not processes of the same account; on the control-node the agents' +> boundary is ADR 0266's account of their own. +> 3. **An ordinary line is reads and settings.** §4 is exact as it stands: an ordinary line is the generic verb's +> reading forms, or a `settings` line composed as the settings verb composes it — nothing else runs. + +> **Progressive insight — 2026-10-09, the operator's rule.** The operator types **one** command, `mesh-cli` +> (`nox`). `nox-mesh-host` (the node-engine) and `mesh-controller` (the controller) are daemons, and a person +> never types their names. The consequence above — a record that says to type `mesh-controller ` means +> `mesh-cli ` — is from this day a rule on every line the mesh addresses to the operator: a condition's +> words, a verb's answer, a module's README, a design document name a `nox` line, never a daemon's. Found on +> 2026-10-09 at 18:01 by the operator typing the node-engine's `hand-over` of +> [issue 339](../04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/00-report.md), which a condition +> had told them to run "at the machine": the command on the machine's `PATH` was a copy of an engine from before +> that subcommand existed, and the act had no channel +> ([issue 356](../04-ISSUES/356-the-operators-command-ran-a-stale-engine-and-a-hand-over-had-no-channel/00-report.md)). +> Now `nox node hand-over ` on the control-node is that act, terminal-only, carried by the controller +> to the node's engine over the bus. **How it is checked:** hq's `words.py` refuses a daemon's line — +> `mesh-controller` or `mesh-host` followed by one of its commands, and `nox-mesh-host hand-over` — in the +> documents it checks from this day, quotations excepted, code blocks and spans included; the catalogue's +> `checks/words` refuses the same lines in every string a module's tools can show and every manifest description, +> from its copy `retired-lines`, which `words.py --list lines` prints and compares. **What is not a line here:** the +> node-engine's install-time subcommands (`nox-mesh-host key`, `enrol`, `overlay take`, `reconcile`, `apply`, +> `profile`, `inventory`, `owned`, `bundle`, `version`) are typed as root at a machine no mesh reaches yet, or at +> one no mesh can reach any more; the engine's usage says so. The daemons' binaries keep their names. The decision +> stands. + +### 5. What comes back + +**Every line is a call**, run, recorded and answered as a seat's verb is: the bus lets a request be answered for a +minute only, so a line answered when it ended lost every answer after that and was said as nothing having run. The +controller answers within ten seconds that a line is still running, with its call; the node-engine follows the +call, as the same account on the same node, until the line ends. The call's record keeps the line's command word +and who asked, never the rest of the line, and the controller's journal says each line, its call, who asked where +and how it ran. **The answer is its asker's alone**: kept in the memory of the controller that ran it for the asker +to follow, never on the bus, and never shown by `calls`, which answers anyone who may call the seat. Lines running +at once are bounded (eight); one over is answered busy, and nothing runs. + +The answer is the command's standard output, its standard error, its exit status, whether it ran as the terminal, +and why or why not. `mesh-cli` prints the two streams and exits with the status. It says one of three things, +never confused: what the line printed; that **nothing ran**, and why, only of a line no controller took; or that +**what came of it is not known** — no answer in time, so it may have run — naming the call to look at. An answer +larger than one bus message is cut there and the cut is said; it is never silently short. Standard input is not +carried: a command that reads it gets nothing and fails saying so. Streaming the output and carrying the input +are later steps. A path in a line names a file where the controller runs, read by the controller's account, never +a file beside `mesh-cli`: settings are given inline, `"$(cat settings.json)"` in the operator's own shell. + +### 6. What is shared, and where it lives + +- **The local protocol is the SDK's.** The socket's path and the program's request and answer are one package of + the Go SDK (`mesh-sdk`), imported by `mesh-cli` and by the node-engine. It is a local protocol a module's + program speaks to its node's runtime, which is what the SDK holds + ([ADR 0039](0039-what-the-sdk-holds-and-refuses.md), as [ADR 0188](0188-a-modules-own-code-is-bundles-in-any-language-and-a-tools-bundle-speaks-mcp-to-the-runtime.md) + reads it). +- **The bus request is the node-engine's and the controller's**, in their own link code, with a test on each side + holding the field names, as for the engine's health statement. A transport lives in no SDK (ADR 0188's note on + ADR 0039). +- **The list of what is terminal-only is not shared.** It stays in the controller, its one judge, and `mesh-cli` + shows the controller's words. + +### 7. In what order it is built + +1. **Phase 1.** The SDK package; the node-engine's socket and its request; the controller's answer, with the + control-node as the terminal for its operator account; the `mesh-cli` module. **After the four merge, the + operator assigns `mesh-cli` to every node** at the terminal; until then it is on no node. From then on the + operator types `nox …` on the control-node, and gets the ordinary calls on every other node. The agent account + is refused. +2. **Phase 2.** The agent account: answered as an ordinary call on a node that names one (ADR 0266 §1), and the + agent-account route of §4 to the terminal on a node other than the control-node, once ADR 0266 runs there (on + the operator's own machines after the follow-up decision ADR 0266 names). Its tests are written with it. +3. **Later.** Streamed output and standard input; a second factor for a node whose agents must share the + operator's account. + +## Consequences + +- The operator has one command on every node, built, checked and moved like every other module, and the records' + "at the controller's terminal" names something that exists: `mesh-cli` (or `nox`) on the control-node. +- A terminal-only change typed on any other node is refused there with the reason, instead of being impossible to + type at all. +- No agent on the control-node gains anything it did not have; once ADR 0266 is switched on there, agents lose the + terminal entirely. +- **Same-account limits stay ADR 0266's boundary.** While an agent runs as the operator, it can do what the + operator's account can: put a `mesh-cli` of its own earlier on `PATH` (the zsh module puts the account's + `~/.local/bin` first), or redefine the alias outside the mesh's block. On a node where agents run as the + operator, the operator's account is not the terminal, so this reaches nothing terminal-only there. +- The alias hides any other `nox` in interactive zsh (Arch's `python-nox` installs `/usr/bin/nox`); + `command nox` still reaches it. +- The node-engine gains a local socket every account on the machine may reach. It accepts one line per + connection, bounded in size, in time and per account, runs nothing itself, and is never reachable from another + machine (ADR 0004 stands, §3). +- Assigning the controller's module moves to the terminal: an agent, or any caller of `assign`, can no longer move + the controller, and so the terminal, to a node of its choosing. +- The controller's own binary is still never on `PATH`. A record that says to type + `mesh-controller ` at the terminal means `mesh-cli ` on the control-node from this record on. + +## How it is checked + +Each claim, and what holds it: + +- **The account is the kernel's** (§3): the node-engine's test connects as the test's own process and the request + carries that process's user id and account; credentials that cannot be read are refused and never sent as + anyone (mesh-host `internal/meshcli`). +- **The socket grants nothing; the controller judges** (§3, §4): the node-engine's package for it runs no command + and changes no file — its one act is the forward, which review holds, and its tests hold every refusal before + it. The controller's judge is tested case by case: the control-node's operator + account runs without `MESH_VERB` (the test reads the variable in the process it ran); another node's operator + account runs with `MESH_VERB=mesh-cli`; every other account, root and the agent account included, is refused + and runs nothing; two control-nodes make nobody the terminal (mesh-controller `cmd/mesh-controller`, the + mesh-cli tests). +- **The node is the bus server's fact**: the writers table names `mesh.control.*.cli` as the machine's own, and + its test refuses another machine or a module granted it (mesh-controller `internal/broker`). +- **Another namespace is refused, and one account cannot hold the socket**: the node-engine's tests refuse a peer + whose namespaces differ, and answer a second line of a busy account *busy* at once while the first is still + asked. +- **An ordinary `settings` line takes the settings verb's path, and every other ordinary line only the generic verb's + reads**: the controller's tests compose each settings form as the verb does and refuse any other, and refuse + `node account`, `node agent-account`, `token issue`, `secret export`, `operator` and `assign` from an ordinary + line, each running nothing (they fail on the commit before ADR 0266's checks were applied to it). +- **The control-node does not follow a verb**: the controller's test asks the `assign` and `unassign` acts + themselves with `MESH_VERB` set and has both refused before any store is touched; with the refusal taken out of + the acts, the test fails. +- **A line is never lost, and never said as nothing having run when it may have**: the controller's test follows a + line that outlasts the answer window to its answer, over a bus, and refuses a follow by another account or + node; the node-engine's tests follow a running line to its answer, say *not known* with the call when no + answer comes or the request fails after it may have been delivered, and say *nothing ran* only of a request the + link knows it never asked. A line's record keeps neither its line's values nor its answer on the bus, and + `calls` never shows its answer; those tests search both the text and the base64 JSON writes bytes in, and each + fails when the withholding it guards is taken out (mesh-controller `internal/link`, `cmd/mesh-controller`). +- **The `MESH_VERB` refusal in `mesh-cli` is a courtesy**: its test holds that the program refuses before asking; + nothing rests on it, since the controller judges the vouched account whatever the program does. +- **The module places one file**: the catalogue's test holds `mesh-cli` to one archive resource unpacking one + compiled Go bundle into `/usr/local/bin`; its shell contribution is the alias. +- **No wider than today, while today lasts** (§4), measured on the control-node, not assumed: the `sudo` module's + own check lists the operator account's rules (on 2026-10-08 it found every command as root without a password, + ADR 0266's context), and `mesh-controller.node` for the control-node shows whether an agent account is named. + Both are read before phase 1 is delivered and said in its pull request. Read on 2026-10-09: the `sudo` module's + `sudo_rules` on the control-node answered `passwordless_all: true` for the operator account (two rules, every + command as root, no password), and `mesh-controller.node` named no agent account there. Neither is checked by the controller at + each line: if the operator account's passwordless `sudo` goes while agents still run as it, this claim is false, + and the remedy is ADR 0266's agent account, not a check here. +- **Live, at acceptance (the operator's view):** after `mesh-cli` is assigned to every node, on the control-node, + in the operator's own shell, `nox settings show zsh --node ` answers as the terminal, and a terminal-only + `nox settings set …` is taken. On another node, the same `settings set` is refused and says why. A line that + runs longer than a minute (`nox push `) is followed to its answer. + +## References + +- [Issue 343](../04-ISSUES/343-the-controllers-terminal-has-no-command/00-report.md): the gap. +- [Issue 339](../04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/00-report.md), + [issue 340](../04-ISSUES/340-an-agent-can-plant-a-hook-in-every-agent-session-on-the-mesh/00-report.md): what is + terminal-only, and how a verb is told from the terminal. +- [ADR 0004](0004-a-node-and-how-it-joins.md): a node has no listening control surface. +- [ADR 0181](0181-the-operator-account-is-a-node-fact-and-a-home-is-a-placement-root.md): the operator account. +- [ADR 0039](0039-what-the-sdk-holds-and-refuses.md), [ADR 0188](0188-a-modules-own-code-is-bundles-in-any-language-and-a-tools-bundle-speaks-mcp-to-the-runtime.md): what the SDK holds. +- [ADR 0204](0204-a-module-contributes-shell-code-to-the-login-shell-in-named-slots.md): shell contributions. +- [ADR 0267](0267-an-artifact-is-rebuilt-only-when-its-build-source-changes-and-a-module-moves-only-along-build-dependencies-on-artifacts.md): + a module builds from its own source. +- [ADR 0234](0234-the-mesh-holds-a-conversation-with-its-operator.md): proof and assurance. +- [ADR 0266](0266-on-the-control-node-agents-run-under-an-account-of-their-own-that-cannot-become-root-without-a-person.md): agents under an account of their own, and what is terminal-only. +- ADR 0259 (hq pull request 211): not yet merged, so named without a link. diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0273-the-agents-home-is-the-modules-and-the-operators-personal-rules-and-lessons-are-registered-through-it.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0273-the-agents-home-is-the-modules-and-the-operators-personal-rules-and-lessons-are-registered-through-it.md new file mode 100644 index 00000000..3e3a810a --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0273-the-agents-home-is-the-modules-and-the-operators-personal-rules-and-lessons-are-registered-through-it.md @@ -0,0 +1,425 @@ +--- +topic: what runs on it +status: accepted +date: 2026-10-09 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0216-the-agents-configuration-is-registered-through-its-module-at-three-scopes-and-served-as-one-plugin.md +--- + +# 273. The agent's home is the module's, and the operator's personal rules and lessons are registered through it + +## Context + +[ADR 0216](0216-the-agents-configuration-is-registered-through-its-module-at-three-scopes-and-served-as-one-plugin.md) +put the coding agent's configuration under its module at three scopes, and gave the home scope the lightest +touch: the module "owns only what it placed" (§5), reports what it did not place and can import it (§6), and +"the stale files already in the homes stay until the person removes them". That followed +[ADR 0182](0182-inside-a-home-the-mesh-owns-what-it-places-and-holds-the-rest-as-found.md): inside a home, the +person's own rules and skills are *found*, never read, rewritten or removed. +[ADR 0243](0243-the-agent-module-removes-a-home-item-it-did-not-place-only-on-the-persons-word-and-keeps-a-copy.md) +added the one channel for removing a found item, on the person's word, with a copy kept. + +[Issue 340](../04-ISSUES/340-an-agent-can-plant-a-hook-in-every-agent-session-on-the-mesh/00-report.md) then +found that the module's tools could not tell the operator from an agent, so an agent confined on the +control-node could register a hook, or a rule, that the operator's next session on another node would run or +obey. The operator closed it on 2026-10-09: shared text and everything that runs is set at the controller's +terminal alone, and the home scope is written only into the home of an agent account of its own +([ADR 0266](0266-on-the-control-node-agents-run-under-an-account-of-their-own-that-cannot-become-root-without-a-person.md)), +never into the operator's. The same day +[ADR 0272](0272-the-operator-reaches-the-controller-with-mesh-cli-on-every-node-and-a-node-is-the-terminal-only-where-no-agent-runs-as-its-operator.md) +gave the operator a way to that terminal, and +[ADR 0259](0259-an-ask-is-a-message-on-the-bus-any-channel-answers-it-and-the-asker-acts-on-a-warrant.md) gave +any module a way to ask the operator a question and act on the answer with a warrant. + +**What the homes hold**, measured on 2026-10-09 through each node's own status tool +([research 040](../01-RESEARCH/040-the-agents-home-is-the-modules-and-the-operators-rules-and-lessons/01-what-the-home-holds-and-the-options.md) §2): + +- a skill [to-be 36](../03-DESIGN/01-to-be/36-the-operators-agent-on-a-machine.md) §1 retired on 2026-10-02 + ("nothing; it named the predecessor's forge") is still in the home on **all four nodes**; +- a person's instruction file is on three; a subagent on one; +- the operator's two hand-written rule files, about how the operator wants text handed over and how a second + forge's repositories are worked in, are on **the laptop alone**; +- the mesh has placed nothing in any home, and raises no condition about any of it. + +On the laptop the directory holds 33 entries: 7 are configuration (the two rules, five skills and a synced +folder, an empty commands folder, a settings file with 6 keys, a local settings file the program writes with +the permission answers a person gives, the program's plugin records), and the rest is the program's own +working state — history, sessions, the memory of 30 projects (1.5 GB), caches, the credentials file +([research 040](../01-RESEARCH/040-the-agents-home-is-the-modules-and-the-operators-rules-and-lessons/01-what-the-home-holds-and-the-options.md) §1). + +**What the agent learns, and where it keeps it.** The agent keeps notes for itself in the program's memory +for one repository on the laptop: 45 on 2026-10-09, at least 9 of them a correction the operator gave of how +the agent works — finish a module before pushing, use the mesh's tools and not ssh, read the merge gate before +merging, no work-arounds for a missing tool. Each was said once, sometimes with irritation, written by the +agent in its own words, shown to nobody, read on no other node and in no other repository, and not said again +at the moment it matters ([research 040](../01-RESEARCH/040-the-agents-home-is-the-modules-and-the-operators-rules-and-lessons/01-what-the-home-holds-and-the-options.md) §3). + +**The operator's direction, 2026-10-09**, in their words: + +> "the home-scope does own ~/.claude, so it should be completely empty and manual edits should be forbidden. +> Rules in my home directory are my 'personal rules' which should also end up under control of the +> claude-code module and all other changes to ~/.claude should go through the claude-code module's mcp tools" + +> "register the decision, but make sure it's clear and that you can/should be pro-active about it. For +> example, if I'm frustrated, try to see why and try to prevent it in the future by warning yourself about +> stuff not to do to frustrate me — kind of like a self-learning process." + +The record was searched first ("personal rules lessons frustration self-learning"; "hand edit home .claude +drift condition"): nothing. + +### Against GENESIS + +- **The mesh owns the truth.** "State lives in the mesh and is derived onto nodes. A file edited on a node + is a bug with a delay on it." A rule file written by hand into the agent's home is that file. +- **The mesh remembers.** "Nothing has to be explained twice." A correction the operator gives once reaches + every session of theirs, and is said again where it applies. +- **Failure must be loud.** A hand edit is a condition, not a silent difference the next render erases. +- **Nodes are personal; the mesh is a guest.** The program's own working state — memory, history, + sessions — stays the program's, excluded by name, exactly as ADR 0182 drew the line. + +## Considered Options + +The options are weighed in full in +[research 040 §5](../01-RESEARCH/040-the-agents-home-is-the-modules-and-the-operators-rules-and-lessons/01-what-the-home-holds-and-the-options.md); +in short: + +### Who owns the configuration under the home + +1. **As today: the module owns what it placed, the rest is found** (ADR 0216 §5). Rejected by the operator, + and by the measurement: a rule on one node reaches no other, a retired skill stays on four, nothing is + raised. And the guard refuses ssh to a machine but not a write to `~/.claude/rules/`: on a node where the + agent runs as the operator, that is a rule planted in every later session, by hand, with no record. +2. **The module owns the whole directory and regenerates it.** Rejected, as ADR 0182 rejected it: the + working state is the program's, and a render touching the projects' memory erases a season of it. +3. **The module owns every configuration file by name, excludes the working state by name, renders the + configuration from registrations, and reports a hand edit as a condition.** Chosen. + +### Whose the rules are + +1. **Per node only**, the home scope as ADR 0216 defines it. Rejected as the only scope: the operator is + one person on four nodes. +2. **Mesh-wide instruction sections.** Rejected as their place: those are the mesh's word about the mesh, + written at the terminal. A personal rule is the operator's word about the operator, and follows the + person's agent into the home under the person's own names. +3. **A new kind, the personal rule, registered for the operator, rendered into the agent's home on every + node.** Chosen; the same scope carries the operator's own skills, commands, subagents and settings. + +### How a lesson becomes a rule + +1. **The agent writes the rule itself.** Rejected: an agent that writes its own rules can loosen them, the + operator never sees what it taught itself, and text every session reads is written on the operator's + word, never an agent's (issue 340). +2. **Lessons stay in the agent's memory.** Rejected by the measurement above. +3. **A lessons register in the module; a lesson is proposed by an agent, accepted or declined by the + operator, rendered as a personal rule on a yes, and said again before the act it bears on.** Chosen. + +### How the operator's yes reaches the operator's home + +1. **Trust the tool's caller.** Rejected in issue 340. +2. **The controller's terminal only.** Safe, and kept as one of two ways; alone, it makes a yes in a + session a trip to the control-node. +3. **A warrant on an authorising ask** (ADR 0259), the exact text bound in the ask. Chosen beside 2. + +## Decision + +### 1. The configuration under the agent's home is the module's, whole + +**What "completely empty" means.** The operator's words, "it should be completely empty", are read as: the +agent's home holds **nothing a person or an agent put there by hand**. Every configuration file in it is the +module's rendering, and the only other entries are the program's own working state, named below. + +**The `claude-code` module owns every configuration file under the agent's home directory** (`~/.claude` of +the account the agent runs as, ADR 0266 §5): the home instruction file, `rules/`, `skills/`, `commands/`, +`agents/`, `hooks/`, `output-styles/`, `settings.json` and `keybindings.json`. **Nothing there is written by +hand.** Each is rendered by the module's code from registrations, on every node, whenever a registration +changes, and nothing else writes them. A configuration file the vendor adds later joins the list by name, in +the module, before the module renders anything into it. + +**The program's working state is excluded by name**, and stays the program's, in ADR 0182's *found* class +or *written by the module's own process*: the credentials file `.credentials.json` (ADR 0183), the history, +the sessions and their environments, the projects directory with every project's memory, `todos/`, +`plans/`, `statsig/`, the shell snapshots, the file history, the debug, cache, paste and telemetry +directories, the statistics, the daemon's files (`daemon/`, its lock, log and status files, +`daemon-auth-status.json`, `daemon-auth-cooldown`), the program's dot-markers (`.last-cleanup`), the +vendor's synced skills (`skills/synced/`), and the backups, worktrees, jobs, feedback, state and IDE +directories. The module never writes or removes any of these, and names the list in its manifest's +documentation. + +**Two entries are the program's to write, and watched**, because each decides what a session runs and +nothing else would see a change to it: `settings.local.json`, where the program records the permission +answers a person gives in a session, and `plugins/`, the program's record of its marketplaces and installed +plugins, and their cache. The module never writes them; §4 says what it raises and what the guard refuses. + +**What is neither, by name.** Measured on the laptop on 2026-10-09: `.hal-binding.json`, a leftover of the +retired predecessor, and `.credentials.json.testbak`, a copy of the credentials file left by a test. **No +copy of the credentials file** (`.credentials.json` followed by anything) belongs in the home: each is +reported, and removed on the operator's word with ADR 0243's tool, never kept as a copy in the module's +state, because it holds a token. **Every other entry the module knows under neither list is a condition** +(§4), so a new vendor file is a question the operator answers, not a default. + +**The keys the program itself writes into `settings.json` on a person's choice** — the model, the spinner, +the drafts and the other preferences the vendor's own `/model` and `/config` commands set — are not hand +edits. The module keeps a short list of them, preferences only. When the program changes one of those keys, +the module takes the new value over into the operator's registered settings (§2), reports it in the status +tool, and the change follows the operator to every node. That is ADR 0182's *written into* class, applied +the other way round: the file is the module's, and the program is allowed into a few named keys of it. This +is the mechanism that keeps a `/model` choice from being reverted, the failure ADR 0182's context records. +**A key that decides what runs is never taken over**: `hooks`, `permissions`, `mcpServers` and the other +tool-server keys, `env`, `apiKeyHelper`, `statusLine`, `autoMode`, and every key not on the list. Changed on +disk, it is written back on the look that finds it (§4). + +**`autoMode` is a permission rule.** It tells the program's unattended mode what it may do without asking, +which ADR 0213 calls policy, not preference. It is barred from the operator scope like `permissions` and +`hooks`, and belongs in the managed settings, set at the controller's terminal in the module's +`managed_settings` for the mesh or a node. + +### 2. A fourth scope, the operator's, and a new kind, the personal rule + +ADR 0216's three scopes stand. A fourth joins them: **operator**. An item registered at the operator scope +is rendered into the agent's home on **every node that names an operator account** (ADR 0181): into the +operator's own home where the agent runs as the operator, and into the agent account's home where the node +names one (ADR 0266). The mesh has one operator; the scope is the person, not a machine. + +The kinds at the operator scope: + +- **personal rule** (new): one short text in the operator's words about how the operator wants the agent + to work with them, rendered as one file under `rules/` and listed in the home instruction file, which the + module renders as the index of the personal rules, each with where it came from (the terminal, or the + lesson it was accepted from) and when. The vendor reads both into every session of that account; +- the operator's own **skills, commands and subagents**, under the person's own names, without the plugin's + prefix; +- the operator's **settings** (`settings.json`), as one registered object of the preference keys of §1, + laid under the managed settings as the vendor lays them; and the operator's **keybindings** and **output + styles**. + +**Never at the operator scope: a hook, a permission rule (`autoMode` included), a tool server, an +environment or key-helper setting, or an item that runs anything.** +Those stay the terminal's, in the managed settings and the managed tool-server file (issue 340). A personal +rule, skill, command or subagent that holds a shell line or the front matter issue 340 refuses is refused at +the operator scope too. + +**The operator's existing items are imported as the first registrations**, through the import tool ADR 0216 +§6 named, at the operator scope: the two rule files on the laptop become the first two personal rules; the +laptop's skills and the workstation's subagent are imported or removed, each on the operator's word +(ADR 0243); the preference keys of `settings.json` on the laptop become the operator's registered settings, +and its `autoMode` key moves to the managed settings at the terminal, because it is a permission rule (§1); +the two leftovers §1 names are removed on the operator's word; the retired `cleanup` skill and the three +instruction files are removed with a copy kept. Until each item is imported or removed it +is a condition (§4). + +### 3. Who may register into the operator's home, and issue 340's protection stands + +A registration at the operator scope, and a registration at the home scope into an operator's own home, is +taken **only** by one of two routes, and refused through any other: + +1. **At the controller's terminal** (ADR 0272), as issue 340 made shared text: the module's settings gain + `personal_rules`, `personal_skills`, `personal_commands`, `personal_agents` and `personal_settings`, set + with `nox settings set claude-code …` on the control-node. The module's whole layer is already the + terminal's, because its settings file is a mergeable file issue 340 counts as trusted. +2. **On a warrant** (ADR 0259): the module is an asker. Its register, import and remove tools, called at the + operator scope or for an operator's home, do not act; they publish an **approve** ask whose words show the + item in full and whose digest binds the exact text, name, kind and scope, and they act once on the warrant + for that ask, recording it. Until a channel that proves its sender answers approve for the module, the + ask is refused at once with the reason and names the terminal's line. + +**The module's tools stay the operator's way in.** A tool called for the operator's home, while no channel +answers approve, does not only refuse: it records the request as **pending** in the module's state (what, +at which scope, asked when, from which node) and answers with the exact line to type: `nox settings set +claude-code ''`, the request merged into the module's current layer, so that typing it changes nothing +else. The status tool lists pending requests; one is dropped when the terminal's layer holds it, or after +7 days. An agent asking for a rule therefore hands the operator one line to check and type, never a +description to translate. + +The home scope into **an agent account's own home** stays as issue 340 left it: that node's own tools take +it, because that home is no person's. A home-scope item never shadows a personal rule: the module renders +the operator's items first and refuses a home-scope name an operator-scope item already uses. + +So an agent still cannot plant anything in the operator's home: a tool call without a warrant writes +nothing there, a warrant is the router's record of the operator's own answer, and the terminal is the +operator's own account on a node where no agent runs as it. + +### 4. A hand edit is a condition, and the guard refuses it + +**On every look**, at least once a minute and at once on a change on disk, the module compares the +configuration paths of §1 with what it last rendered, and reads the two watched entries. Any difference is a +**finding**, said as the module's own health (ADR 0240) on that node, and the controller raises the +condition **`agent-home-drift`** for it, a warning that needs the operator, naming each path and what +differs. **The look that finds a difference acts on it**; nothing waits for the next render: + +- **a rendered file changed by hand**, `settings.json` above all: written back on that look, the hand version + kept as a copy in the module's state (`home-copies`) and shown by the status tool, so the words are not + lost. The one exception is a preference key of §1 the program changed, which is taken over. A `hooks`, + `permissions`, tool-server or other key that decides what runs, added or changed in `settings.json`, is + **always written back**, never taken over; +- **a rendered file missing**: rendered again on that look, and reported once; +- **a file nobody rendered** under `rules/`, `skills/`, `commands/`, `agents/`, `hooks/` or + `output-styles/`: + - **before the node's first adoption** (the imports of §2, done once per node by the operator, and recorded + in the module's state): left in place and reported, never removed by the module. The operator's answers + are the import tool (register it at a scope) or the removal tool (ADR 0243: a reason, a copy kept); + - **after it**: **moved** on that look into `home-copies`, kept and restorable with ADR 0243's restore tool, + and raised. A session never loads a file nobody registered for longer than one look. Importing it from + the copy is the operator's way to keep it; +- **an entry the module knows under neither list of §1**: reported and raised, never touched; the operator + removes it on their word or the module's next build names it; +- **`settings.local.json` holding any key other than `permissions`**: raised, naming the key. The program + writes only that key there, so another is somebody's hand; +- **`plugins/` naming a marketplace or an installed plugin other than the vendor's own marketplace and the + mesh's `nox-mesh`**: raised, naming it. A plugin carries hooks, so one installed by hand runs in every + session. + +The condition clears on the first look that finds no difference. It is never urgent, and the first-node gate +never reads it as a fault of a build (as +[ADR 0254](0254-a-wait-for-a-persons-new-login-is-not-a-gates-failure-and-each-module-of-a-send-keeps-its-own-pass.md) +treats a wait for a person). + +**The guard on the agent's shell** (ADR 0245) refuses, before it runs, naming the module's tool that does the +job and saying that a rule the agent wants is a lesson (§5): + +- any shell command or file edit whose target is under a configuration path of §1 — a write, an append, a + copy or move into it, an in-place edit, a removal; +- the same aimed at `settings.local.json` or at a `.json` file under `plugins/`; +- `claude plugin install`, `claude plugin marketplace add` and the in-session `/plugin install` and + `/plugin marketplace add`, whatever the plugin. + +The operator's override of ADR 0245 §5 applies and is recorded. The guard does not watch the rest of the +working state. + +### 5. Lessons: an agent proposes, the operator decides, and the agent is reminded + +**The module keeps a lessons register** in its state (ADR 0201): one record per lesson, holding when and on +which node it was proposed, the session that proposed it where the agent gave it, the **trigger** (the +operator's words, quoted), the **cause** (what the agent did, in one or two sentences), the **rule** that +would prevent it (written as a personal rule, in the operator's second person: "do not…", "always…"), an +optional **before** (a tool name, a command prefix or a path pattern the rule bears on), and its state: +`proposed`, `accepted` or `declined`, with who decided, how and when. + +**When an agent records a lesson.** The mesh's instruction section tells every agent: when the operator +expresses frustration (repeats a request, says they have asked before, says "again", "stop", "why did you"; +irritation in any words), or corrects a way of working ("don't", "always", "from now on", "never"), the agent +first works out why — what it did, and what the operator wanted instead — and then records a lesson with +the trigger, the cause and the rule, in the same turn, before going on. It says in its answer that it did, +in one line. An agent that is unsure whether a correction is a lesson records it; the operator declines what +is not. + +**A correction goes to the register, not to the agent's own memory.** The program's per-project memory +(§1, working state) stays the agent's notebook for facts about a project; a way of working the operator +corrected is a lesson, so that it reaches every node and the operator sees it. + +**The first lessons are the agent's existing corrections.** Once the register exists on a node, the agent +there proposes, unasked, every note in its own memory that records a correction the operator gave (at least 9 +on the laptop on 2026-10-09), each as a lesson with the note's trigger, cause and rule, and says so in one +line. The operator accepts or declines them like any other. + +**A proposed lesson is not a rule.** It is rendered nowhere and reaches no session as an instruction. The +module shows it to the operator for a yes: an approve ask (§3 route 2) whose words carry the trigger, the +cause and the rule, with Accept and Decline as its options; or, until a channel answers approve, the list +tool names the proposed lessons and the operator accepts or declines each at the terminal (`nox settings +set claude-code …` with the lesson's id in `lessons_accepted` or `lessons_declined`). A proposed lesson +expires after 30 days, declined with that reason. + +**An accepted lesson is a personal rule.** The module registers it at the operator scope with the lesson's +id as its source, so it renders into the agent's home on every node like any personal rule, and the home +instruction file names it as "accepted from lesson N on ". **A declined lesson stays in the register +with its reason**, and the list tool shows it, so an agent does not propose it again; a new lesson whose rule +matches a declined one's is refused with the reason, and the agent is told to say so to the operator rather +than record it again. + +**The agent is reminded, before it acts.** Two things make the lessons proactive rather than a file the agent +may or may not open: + +- **at session start**, the plugin's session-start hook (the module's own binary, beside the guard) prints + the personal rules accepted from lessons in the last 30 days and the ones whose `before` matches the + session's working directory or repository, each in one line, so the session begins knowing them; +- **before a tool runs**, the guard (ADR 0245 §4, asked about every tool) prints the rule of every accepted + lesson whose `before` matches the tool, the command or the path about to be used — a reminder, never a + refusal; a refusal is a guard rule and nothing else. The reminder is printed once per session per rule. + +**The register is readable by any agent on any node** through the list tool, and shows `proposed`, +`accepted` and `declined` apart, each marked, so a session reading it cannot take a proposal for a rule. +Recording a proposal needs no yes: it writes the register and nothing a session obeys. + +### 6. What this changes in the earlier records + +- **ADR 0216 §5 and §6**: the mechanism changes (that record gains a note). The home scope owns the + configuration whole; a found item is reported as a condition rather than listed on request; import stays + the way in; the plugin's hook file holds the module's own hooks, the guard and the session-start + reminder, and nothing registered. +- **ADR 0243** stands as the removal on the person's word, now on a warrant or at the terminal for an + operator's home, and is the operator's one answer to a found item beside importing it. +- **ADR 0182** stands: the class table holds, with the configuration files in the *owned* class and the + working state *found*, each named. +- **ADR 0245**: the guard gains a refusal (§4) and a reminder (§5). +- **Issue 340**: the protection stands; the terminal and the warrant are the two ways into the operator's + home. + +## Consequences + +- The operator's rules follow the operator: a rule written once reaches the agent's home on every node + within a minute, and a session on any node starts with it. +- A hand edit under `~/.claude` is undone within a minute, with the hand version kept, and is said as a + condition: no silent difference, and no lost words. +- The retired skill and the three stale instruction files, found on four nodes since 2026-10-02, are + removed on the operator's word with copies kept, and nothing like them can stay unnoticed again. +- A correction the operator gives once becomes a rule everywhere on one yes, and is said again before the + act it bears on. The operator sees every rule the agent proposes, and can decline it. +- **What got harder:** + - a new rule for the agent is no longer a file: it is a registration at the terminal or a warrant on an + ask. Until a channel answers approve for the module, every yes is typed on the control-node; + - the module watches the home on every node, and its state holds the lessons register and the copies of + hand versions. Clearing old copies is not decided here, as ADR 0243 left it; + - a vendor file the module does not know is a condition until it is classified, which is the point: an + unknown file in the agent's home is a question, not a default; + - an agent is expected to notice frustration and record a lesson in the same turn, which costs the turn a + tool call. A lesson recorded late, or not at all, is found by reading the session against the register. +- **Not decided here:** a per-project lesson (one bound to a repository rather than the operator); lessons + for an agent the mesh runs rather than the operator's; and a channel of the operator's own that can + tell the operator from an agent (issue 340's "once we build a UI"), which would become a third route. + +> **Progressive insight — 2026-10-09.** The four things this section leaves undecided — clearing old copies +> of hand versions, a per-project lesson, lessons for an agent the mesh runs, and a channel of the operator's +> own — were left with no issue, owner or date, which is how ADR 0244's deferred renames went unnoticed +> (issue 354). They are tracked by +> [issue 357](../04-ISSUES/357-what-adr-0273-left-undecided-has-no-owner/00-report.md). The decision stands. + +## How it is checked + +| Rule | Checked by | +|---|---| +| Every configuration path of §1 is rendered from registrations and nothing else; the working state is never written or removed; an entry under neither list is a condition | the module's render test over a seeded home: the configuration files (`keybindings.json` and `output-styles/` included) match the registrations byte for byte; every excluded entry (`todos/`, `plans/`, `statsig/`, `.last-cleanup`, `daemon-auth-cooldown` among them) is byte-identical before and after; an unknown entry, `.hal-binding.json` and a `.credentials.json.*` copy each raise `agent-home-drift` and are left in place; removing a credentials copy keeps no copy of it | +| A program-written preference key of `settings.json` is taken over; a key that decides what runs is always written back; `autoMode` is never taken at the operator scope | the module's test: the model key changed on disk appears in the operator's registered settings on the next look and in the rendered file on every node; `hooks`, `permissions`, `mcpServers`, `env`, `apiKeyHelper`, `statusLine` and `autoMode` added or changed on disk are each written back on the look that finds it and never reach the registration; `autoMode` in a `personal_settings` value is refused | +| A personal rule renders into the agent's home on every node naming an operator account, under `rules/` and in the home instruction file | the module's render test for one machine of three: the operator's own home on two, the agent account's home on the one naming one; live: a rule set at the terminal is in a new session's context on each node | +| Nothing that runs is taken at the operator scope | the module's tests: a hook, a permission rule, a tool server, and a skill, command or subagent holding a shell line or the refused front matter are each refused at the operator scope with issue 340's reason | +| A registration into an operator's home is taken only at the terminal or on a warrant | the module's tests through its real tool dispatch: each register, import and remove tool at the operator scope, or for an operator's home, publishes an ask and writes nothing; the same with a warrant for that ask acts once; a warrant for another ask, or a replayed one, is refused; while no channel answers approve the ask is refused at once naming the terminal's line. The controller's test: the module's layer is the terminal's (issue 340's) | +| A hand edit is a condition, undone on the look that finds it | the module's tests with a clock: a rendered file changed by hand (`settings.json` among them) is written back on the same look, within a minute, its hand version kept and shown; a missing one is rendered again on that look; the condition clears on a clean look; the controller's test that `agent-home-drift` is a warning the first-node gate never reads as a fault | +| Before a node's first adoption a file nobody rendered is left; after it, it is moved to the copies and raised | the module's tests: an unrendered file under each of `rules/`, `skills/`, `commands/`, `agents/`, `hooks/`, `output-styles/` is reported and left before the adoption is recorded; after it, the same file is gone from the home on that look, present in `home-copies`, restorable with the restore tool, and the condition names it | +| `settings.local.json` and `plugins/` are watched | the module's tests: a key other than `permissions` in `settings.local.json` raises the condition naming it; a marketplace or installed plugin other than the vendor's and `nox-mesh` in `plugins/` raises it naming it; neither file is written by the module | +| The guard refuses a write under a configuration path, to `settings.local.json` or `plugins/*.json`, and a plugin install, and lets the working state alone | the module's guard tests: a redirect, append, copy, move, in-place edit and removal under each path, and aimed at `settings.local.json` and a `.json` under `plugins/`, refused naming the tool; `claude plugin install`, `claude plugin marketplace add` and their in-session forms refused; the same write under `projects/` passes; the operator's override honoured and recorded | +| A home-scope name may not shadow a personal rule | the module's test: with a personal rule, skill, command or subagent registered at the operator scope, a home-scope registration of the same name and kind is refused naming the operator-scope item, and the render holds the operator's item | +| A tool called for the operator's home leaves a pending request and the exact terminal line | the module's tests: the call writes nothing into the home, records one pending request, and answers a `nox settings set claude-code '…'` line which, applied to the current layer, adds that item and changes no other key; the request is dropped once the layer holds it, and after 7 days | +| A lesson is recorded with its trigger, cause and rule; a proposal renders nowhere; an accepted one is a personal rule; a declined one is not proposed again | the module's tests over the register: register, list, accept, decline, expiry; the render test that only accepted lessons reach `rules/`; a second proposal matching a declined rule refused with the reason | +| The agent is reminded at session start and before a matching tool | the module's hook tests: the session-start hook prints the recent and matching rules; the guard prints a matching lesson's rule once per session and refuses nothing for it | +| An agent records a lesson when corrected, and proposes its existing corrections first | the module's render test: the rendered managed instruction file's `working-on-the-mesh` section holds the paragraph on when a lesson is recorded, that a correction goes to the register and not to the agent's memory, and that existing corrections are proposed first, and the test fails when the paragraph is taken out of the section; live: a correction given in a session is in the register within that turn, and the laptop's memory corrections are in the register as proposals after WP9 | +| Live, at acceptance (the operator's view) | on the laptop: `~/.claude` holds the rendered configuration and the program's working state and nothing else; a rule written by hand into `rules/` raises `agent-home-drift` within a minute and is listed by the status tool; `nox settings set claude-code '{"personal_rules": …}'` on the control-node puts the rule into a new session on every node; a correction given in a session appears in the lessons list, is accepted at the terminal, and the next session on another node names it at start | + +## References + +- [Research 040](../01-RESEARCH/040-the-agents-home-is-the-modules-and-the-operators-rules-and-lessons/00-overview.md): + the measurement on four nodes, the agent's memory, the options. +- [ADR 0216](0216-the-agents-configuration-is-registered-through-its-module-at-three-scopes-and-served-as-one-plugin.md), + [ADR 0182](0182-inside-a-home-the-mesh-owns-what-it-places-and-holds-the-rest-as-found.md), + [ADR 0243](0243-the-agent-module-removes-a-home-item-it-did-not-place-only-on-the-persons-word-and-keeps-a-copy.md): + the home before this record. +- [ADR 0245](0245-a-verb-says-what-it-replaces-and-the-agent-is-guarded-from-working-round-the-mesh.md): the guard. +- [ADR 0259](0259-an-ask-is-a-message-on-the-bus-any-channel-answers-it-and-the-asker-acts-on-a-warrant.md), + [ADR 0272](0272-the-operator-reaches-the-controller-with-mesh-cli-on-every-node-and-a-node-is-the-terminal-only-where-no-agent-runs-as-its-operator.md): + the two ways the operator's word reaches the mesh. +- [ADR 0266](0266-on-the-control-node-agents-run-under-an-account-of-their-own-that-cannot-become-root-without-a-person.md), + [ADR 0181](0181-the-operator-account-is-a-node-fact-and-a-home-is-a-placement-root.md): whose home the agent's is. +- [ADR 0240](0240-a-module-says-how-it-is-healthy-and-the-node-engine-judges-it.md), + [ADR 0254](0254-a-wait-for-a-persons-new-login-is-not-a-gates-failure-and-each-module-of-a-send-keeps-its-own-pass.md): + the condition and the gate. +- [Issue 340](../04-ISSUES/340-an-agent-can-plant-a-hook-in-every-agent-session-on-the-mesh/00-report.md): the protection kept. +- [To-be 36](../03-DESIGN/01-to-be/36-the-operators-agent-on-a-machine.md) §1, §8 and §10, + [to-be 40](../03-DESIGN/01-to-be/40-building-the-operators-agent-and-its-licence-manager.md) WP7–WP10: the design and the build. diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0278-the-issue-tracker-checks-every-new-issue-for-a-duplicate-through-a-judgement-the-agent-makes-without-tools.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0278-the-issue-tracker-checks-every-new-issue-for-a-duplicate-through-a-judgement-the-agent-makes-without-tools.md new file mode 100644 index 00000000..d9c89e3c --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0278-the-issue-tracker-checks-every-new-issue-for-a-duplicate-through-a-judgement-the-agent-makes-without-tools.md @@ -0,0 +1,219 @@ +--- +topic: how we work +status: accepted +date: 2026-10-10 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0275-the-mesh-numbers-and-tracks-its-issues-in-a-module-of-its-own-and-hq-keeps-only-the-narrative-of-a-design-or-governance-issue.md +--- + +# 278. The issue tracker checks every new issue for a duplicate, through a judgement the agent makes without tools + +## Context + +On 2026-10-10 the operator asked for "a 'groom backlog' tool for our issue tracker which goes over the +non-resolved issues and checks for duplicates and tries to consolidate issues together, 'cancelling' the other +ones with a reference in the open one", and then: "or even better, that runs when a new issue is submitted". + +Agents open issues through `issue-tracker.open` ([ADR 0275](0275-the-mesh-numbers-and-tracks-its-issues-in-a-module-of-its-own-and-hq-keeps-only-the-narrative-of-a-design-or-governance-issue.md)). +Nothing tells an agent that the question it is about to open is already open, so the same symptom is opened +again by the next session that meets it. [Research 042](../01-RESEARCH/042-the-mesh-keeps-its-own-issues/01-findings.md) +§3 planned a duplicates pass over the imported issues as candidate pairs "whose titles share most words", +each confirmed before it is applied. Shared words find the pairs a reader would find by searching; they do not +find the same question asked in other words, which is the case that costs time, and they cannot tell a +duplicate from two issues in one area. That judgement reads both reports, which needs a language model. + +**What the mesh had for a judgement, measured on 2026-10-10.** No seat or tool runs a model for a module: +`mesh_search` for "headless agent", "llm" and "completion" found nothing. The `claude-code` module holds the +agent on every node and the licence the node is bound to ([ADR 0206](0206-a-node-reports-the-anthropic-grant-it-holds-and-the-licence-manager-adopts-a-licence-by-refreshing-it.md)); +on the control-node the agent runs as an account of its own ([ADR 0266](0266-on-the-control-node-agents-run-under-an-account-of-their-own-that-cannot-become-root-without-a-person.md)), +and the operator's account, which the tool runner runs bundles as, may become it. `claude` in print mode with +its built-in tools turned off (`--tools ""`) and in safe mode answers a structured question with only its +answer-shaping tool available: measured on the laptop, the session listed `StructuredOutput` as its only tool, +in 2.2 seconds. Ollama provides `model-access` on the home server only, and the control-node runs no model. + +**The bounds that shape the design.** A tool call on the bus is answered within 30 seconds or the asker hears a +timeout (the tool runner's request bound). An event a module consumes is redelivered after 30 seconds without +an acknowledgement, five times at most; after that the controller raises its `max-deliveries` condition and +keeps the event in `DEAD_LETTERS` until a person delivers it again. The tool runner gives a handler two minutes, +but said nothing to the bus while the handler worked, so an event handled for longer than 30 seconds was offered +again while it was handled, and events waiting in the runner's buffer ran out their 30 seconds before they were +handed over: a burst of opens would have ended in `max-deliveries` with nothing failed. Found in this record's +review and opened as issue 387; the tool runner now keeps an event alive while its handler works and holds one +at a time, and this record's code is merged only after that fix. A module that runs only its bundle declares +no health ([ADR 0240](0240-a-module-says-how-it-is-healthy-and-the-node-engine-judges-it.md) rule 8), and a +module's health is the only way it raises a condition of its own +([ADR 0256](0256-a-machines-hardware-condition-is-read-by-one-module-on-every-machine-and-a-fault-is-said-through-its-health.md)). + +The record was searched first ("duplicate", "groom", "issue opened", "headless", "claude-agent -p", "agent +account"): only research 042's pass and the grooming passes made by hand on 2026-09-29. + +**A word.** The glossary has none for this check. This record names it the **duplicate check** and adds it. + +## Considered Options + +### What starts the check + +1. *A schedule, every two hours, over the backlog.* The operator's first idea, and his own better one replaced + it: a duplicate opened at 10:00 stays open and gathers notes until noon. +2. **The tracker's own `opened` event.** The tracker already says every open on the bus + ([to-be 53](../03-DESIGN/01-to-be/53-the-issue-tracker.md) §4). Consuming it is the mesh's ordinary way for one + change to start work, it reaches an issue opened by any caller, and the event's redelivery is the retry. + Chosen, with a verb for the backlog that exists today. +3. *A step inside `open`.* Refused: `open` answers within the bus's 30 seconds, and a judgement can take most of + them; an open must never fail because a model was slow. + +### Who judges + +1. *The tracker calls the vendor's interface itself, with the node's token.* Refused: the token is the agent + module's, handed to it sealed (ADR 0206); a second holder is a second place it can leak, and a subscription's + token is for the agent. +2. *A local model through `model-access`.* Refused for now: none runs on the control-node, and the one on the + home server is weaker at telling "the same question" from "the same area", the one distinction this check + must get right. +3. **The agent module answers one question, as the node's agent with no tools.** `claude-code` gains a tool that + runs the agent in print mode, as the account agents run as, under the licence the node is bound to, with its + built-in tools off and in safe mode (no tool servers, plugins, hooks or instruction files), and answers the + structured answer. Chosen: the module that holds the licence spends it, and the session can do nothing but + answer. +4. *A seat for judgements, with kinds.* Deferred: one module asks today. When a second module asks, or a + second holder (a local model) is worth offering, the tool becomes a seat's verb; until then it is a module's + tool, called on the tracker's own node. + +### What a verdict may do + +1. *Close whichever of the two is newer, or merge both into a third.* Refused: a third issue loses both + numbers' history, and an agent holding the older number would find it closed. +2. **Close only the newer issue, the one just opened, and only for a clear duplicate.** The older issue stays as + it is and gains the newer one's report as a note, so nothing reported is lost and the number agents already + hold stays open. Chosen. + +## Decision + +### 1. A new issue is checked when it is opened + +`mesh-issues` consumes its own `mesh-issues.opened` event. For each opened issue still at `open`, it judges it +against every **older** issue that is not resolved: at `open`, `diagnosing` or `located`. A resolved or wontfix +issue is never offered, never linked and never changed by the check. + +The judgement has two questions, so each fits a tool call's 30 seconds however long the backlog grows; the +check as a whole, waiting behind a judgement of the backlog's pass included, stays under the two minutes the tool +runner allows a handler (issue 387): + +1. **A shortlist by titles.** The new issue's title and report against every candidate's number and title; the + answer names at most five numbers that could be the same question or closely related. The titles fill at + most 48 KiB of the question, newest first; when the backlog holds more, the note on the issue says how many + were offered of how many. +2. **A verdict on the reports.** The new issue and each shortlisted issue whole, each report cut at 8 KiB; the + answer is one of `duplicate` (the same question: one answer closes both), `related` (the same area or a + cause in common, two questions) or `none`, with the issue it names, whether it is **certain**, and the reason + in one or two sentences. + +An empty shortlist is a verdict of `none` without the second question. + +### 2. What each verdict does + +| Verdict | Done, through the tracker's own `note`, `link` and `move` | +|---|---| +| `duplicate`, certain, naming an offered issue | the new issue is linked `duplicates` to the older one; the older one gains a note holding the new one's title and report, and who found it and why; the new one gains the verdict as a note; the new one moves to `wontfix` with the reason "duplicate of N" | +| `duplicate` not certain, or `related` | the new issue is linked `relates-to` the older one; both gain a note saying so and why; nothing moves | +| `none` | the new issue gains a note: checked against so many issues, none the same | + +The older issue is never moved, so the check never closes both sides. The tracker asks one judgement at a +time, and the backlog's pass (§6) waits while a new issue's check is waiting; one verdict is applied at a time. A verdict naming an issue that was not +offered is a failed judgement (§4), never acted on. The tracker's own code for the verbs does each step, so each +refuses what the verb refuses and says its event; the store is never written around them. Each note and link +records its author as `duplicate-check`, the way an import records `import` (ADR 0275 §5). + +**An issue's report is data in the question, and the session has no tools.** A report written to steer the +verdict can at most close the issue that carries it, or link it: the verdict cannot move an older issue, touch a +resolved one or name an issue it was not offered. + +### 3. Once per issue, whatever is delivered twice + +The verdict note on the new issue is written **first**, and its first line says the verdict in a fixed form +("Duplicate check: duplicate of 412", "related to 412", "none of 85 issues is the same question"). An event +delivered again, or the backlog pass reaching an issue the event already checked, finds that note and does not +judge again: it does only the steps of that verdict not yet done (a link it holds, a note the older issue +holds, a status already `wontfix` are each done). An issue that is no longer at `open` when its event arrives is +not checked. + +### 4. A judgement that fails is loud + +A judgement fails when the agent module does not answer, answers late or refuses, or answers outside the shape +asked for, or names an issue it was not offered. Then the new issue gains one note saying the duplicate check +could not be made and why — once, not at every delivery — and the handler does not take the event. The bus +offers it again after five seconds; after the fifth time the controller raises `max-deliveries` on the tracker's +consumer, an open condition, and keeps the event for a person to deliver again. A tracker that cannot judge is +therefore never silent and never closes anything. + +### 5. The judgement tool + +`claude-code` gains `claude_code_judge`: a question in plain words and the JSON schema of the answer, optionally +a model alias. It runs `claude` in print mode, as the node's agent account, in an empty directory, with built-in +tools off, in safe mode and keeping no session, under the licence the node is bound to; and answers the +structured answer, the model and how long it took. It answers within 25 seconds or not at all (the process is +ended), so its caller hears a refusal rather than the bus's timeout. One judgement runs at a time on a node. +It refuses a question larger than 64 KiB, a schema that is not an object, and a node holding no licence, each in +words. The question is never logged or said in an event; the answer goes only to its caller. + +### 6. The backlog: `groom`, a dry run first + +The tracker's seat gains the verb `groom`, for the issues that were open before this check existed. It runs in +the background, because a judgement per issue takes longer than one call may: + +- **Without `confirm`** it starts a dry run: every issue not resolved, oldest first, judged as in §1 against the + older ones not resolved and not already found a duplicate in this pass. It writes no note, link or move. Its + findings — each duplicate and related pair with the reason, and each judgement that failed — are kept as one + **pass** in the module's new state `grooming`, and `groom` with the pass's id answers its progress and + findings. +- **With `confirm` and a pass's id** it applies exactly that pass's duplicate and related findings, not a fresh + judgement, each checked again against the issues as they stand: a finding whose older issue has since been + resolved, or whose newer one has moved on, is skipped and said. It adds no "checked, none" note to the + backlog's issues. + +One pass runs at a time. A pass's findings are kept until the next pass replaces them. + +### 7. Settings + +`duplicate-check` (`on` by default; `off` stops the check on new issues, said in the log) and `judge-model` +(`sonnet` by default; an alias the judgement tool accepts). Both are the module's ordinary settings. + +> **The mechanism changed — 2026-10-10, by ADR 0279.** The judgement this record decided is now one part of the +> **triage** ([ADR 0279](0279-the-issue-tracker-triages-every-new-issue-and-an-issue-is-ready-when-what-it-depends-on-is-delivered.md)): its second question is always asked and also estimates severity, needs, a suggested +> owner and the issues the new one depends on or relates to. What it decided of duplicates stands; the triage's +> notes and links are recorded as by `triage`, and the notes written as `duplicate-check` are still read. + +## Consequences + +- An issue opened twice is closed as a duplicate within the judgement's time of being opened, with the reason + on both issues and the duplicate's report kept on the older one; agents keep working on one number. +- Every open spends two short judgements of the control-node's licence. At the dozen or so issues a day the + mesh opens, that is noise beside one agent session. +- The duplicate check depends on the agent module and its licence on the tracker's node. When the licence is + gone, every new issue's event ends in `max-deliveries`: loud, and nothing is closed. +- Research 042's word-sharing candidate list for the duplicates pass is replaced by this judgement; to-be 53 + slice 3 says so. + +## How it is checked + +| Rule | Checked by | +|---|---| +| A clear duplicate: linked, its report kept on the older one, the newer moved to wontfix with "duplicate of N" | the tracker's test with the judgement stubbed: the four steps, the verdict note first, each note naming `duplicate-check` | +| Related, or a duplicate not certain: linked `relates-to`, noted on both, nothing moved | the tracker's test, both cases | +| None: one note, nothing else | the tracker's test | +| A resolved or wontfix issue is never offered or changed; the older issue never moves | the tracker's test: a resolved and a wontfix issue in the backlog are absent from both questions; a verdict naming one, or naming an issue not offered, changes nothing and fails | +| Once per issue | the tracker's test: the same event twice judges once; an event after a partial apply finishes only the missing steps | +| A failed judgement is loud | the tracker's test: the handler returns an error, one note is written however often the event comes | +| The dry run changes nothing; confirm applies the pass's findings, re-checked | the tracker's test: issues byte-for-byte unchanged after a dry run; a finding whose older issue was resolved in between is skipped | +| The judgement session can do nothing but answer | the agent module's test of the command line: tools off, safe mode, no session kept, as the agent account, a 25-second bound; a question over 64 KiB and a node without a licence refused | +| Live | an agent opens an issue that repeats one not resolved; within a few minutes `get` shows it `wontfix`, "duplicate of N", linked `duplicates`, and N holds its report as a note; a second, unrelated issue gains only the note that it was checked | + +## References + +- [ADR 0275](0275-the-mesh-numbers-and-tracks-its-issues-in-a-module-of-its-own-and-hq-keeps-only-the-narrative-of-a-design-or-governance-issue.md) — the tracker this record extends +- [Research 042](../01-RESEARCH/042-the-mesh-keeps-its-own-issues/01-findings.md) §3 — the duplicates pass +- [ADR 0206](0206-a-node-reports-the-anthropic-grant-it-holds-and-the-licence-manager-adopts-a-licence-by-refreshing-it.md) and + [ADR 0266](0266-on-the-control-node-agents-run-under-an-account-of-their-own-that-cannot-become-root-without-a-person.md) — + whose licence, and which account, a judgement runs under +- [To-be 53](../03-DESIGN/01-to-be/53-the-issue-tracker.md) — the design this record amends diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0288-the-object-store-stays-on-the-maintained-fork-and-garage-is-chosen-only-after-the-consumers-s3-features-are-counted.md b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0288-the-object-store-stays-on-the-maintained-fork-and-garage-is-chosen-only-after-the-consumers-s3-features-are-counted.md new file mode 100644 index 00000000..15188fbe --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/02-DECISIONS/0288-the-object-store-stays-on-the-maintained-fork-and-garage-is-chosen-only-after-the-consumers-s3-features-are-counted.md @@ -0,0 +1,171 @@ +--- +topic: what runs on it +status: accepted +date: 2026-10-10 +deciders: jochen +reconstructed: false +--- + +# 288. The object store stays on the maintained fork, and Garage is chosen only after the consumers' S3 features are counted + +## Context + +[Research 015](../01-RESEARCH/015-the-object-store-after-minio/00-overview.md) asked which S3 implementation +replaces MinIO as the mesh's object store, and how the data moves. It found two decisions, not one: whether to +repoint to the maintained fork of MinIO or to migrate, and, if migrating, to which product. It ranked the +migration candidates Garage, then SeaweedFS, and not yet RustFS, and it said nothing graduates before two things +are measured: which S3 features the consumers use, and whether the fork reads the incumbent's data. + +**What happened since, measured on 2026-10-10.** + +- [Issue 113](../04-ISSUES/113-the-object-stores-images-were-withdrawn-upstream/00-report.md) (the object store's + images were withdrawn upstream) is resolved in the issue tracker. Its fix repinned the object-store module to + the maintained fork's server image by digest, built the module's client from source, and moved its data off the + predecessor's directory. No decision record chose that fork; the fix did. This record is that decision. +- The object store runs on the control-node from that pin: the tool runner lists the module's four tools there + (`mesh_search "minio"`), and the module's manifest on the catalogue's main branch names the fork's image. +- The manifest declares **one data directory on one node**, not the eight-directory erasure-coded shape research + 015 measured. That shape was the predecessor's. The question the research left open (*is that shape still + warranted?*) is therefore answered for the running module: it was not carried over. +- The manifest's upgrade policy is `record`: a person takes each build of the module, after a backup + ([ADR 0236](0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md)), + and its reason names what the store holds: **the operator's photos, which are irreplaceable**. + +**The count of S3 features, from the catalogue only.** The count was taken from the consumers' manifests and +code in the catalogue repository, and nothing was read from the store itself, because the store holds the +photos and a count needs none of their bytes. What the catalogue shows: + +| Who | Where it was read | S3 features it uses | +|---|---|---| +| The object store's own tools and provisioner | the module's client, tools and provisioner code | list, head, create and delete a bucket; list objects; head an object; a presigned GET or PUT; an admin service account per consumer with a policy scoped to its one bucket | +| A file-sync application, using the store as primary storage | its manifest | path-style addressing, a region, plain HTTP, one bucket, no server-side encryption key configured | +| A second application, keeping its records in the document store and its files in a bucket | its manifest | an endpoint, a port, one bucket and one access key, passed to its own code | + +Two modules require the `s3-bucket` provision, and nothing else in the catalogue does. The provider's own code +uses no object versioning, no object locking, no server-side encryption and no lifecycle rules. **The two +consumers' own S3 calls are not in the catalogue**: the file-sync application's object-store code is its +upstream's, and the second application is built from a repository of its own. So the count cannot be finished +from the catalogue's code and manifests alone. Finishing it means reading those two codebases, which touches +no data. Those four features are the ones Garage lacks, which is why the count decides whether Garage can be +chosen. + +**What nothing declares yet.** Neither consumer says `kept-by` for its bucket, so the store's data is declared +`valuable` under [ADR 0233](0233-a-module-declares-the-data-it-holds-and-the-mesh-protects-and-watches-it-from-that.md), +while the manifest's own words say the photos in it are `irreplaceable`. + +The record was searched first (`records_search` for "object store minio fork garage") and returned nothing. + +## Considered Options + +### What the object store runs now + +1. **Stay on the maintained fork, as issue 113's fix left it.** Chosen. It costs nothing more: the fork is + running, a node that holds nothing can obtain it again, and patches resume. It does not end the dependence on + a codebase its original authors abandoned, and this record does not pretend it does. +2. *Migrate now, to the best-ranked candidate.* Rejected. It moves irreplaceable data on a ranking that one + unmeasured fact (the features the consumers call) can still overturn, and it needs a maintenance window for + the file-sync application, whose copy drifts while it runs. +3. *Migrate now, to RustFS, because its admin API is closest and the handlers port with least work.* Rejected, + for the reason research 015 gave: it reached general availability eight days before the comparison was + written, it is not an in-place drop-in, and it had an open defect in the credential path the mesh relies on. + +### Which product a migration moves to + +1. **Garage first, chosen only once the count shows no consumer needs what it lacks.** Chosen. Its permission + model (one access key, one bucket, read, write or owner) is the mesh's bucket provision stated directly, its + admin API matches how the mesh provisions, and the file-sync application documents it as primary storage. +2. *SeaweedFS first.* Rejected as first: its S3 surface translates onto its own file-system API, and the + file-sync application does not document it. It stays the second candidate if the count rules Garage out. +3. *Ceph RGW.* Rejected, as in research 015: a platform, where the object store is one ordinary module. + +### Where the count comes from + +1. **The consumers' code and manifests, never the store.** Chosen. Listing buckets or objects reads the + operator's data to answer a question the code answers, and a mistake in that reading is a mistake against + irreplaceable data. +2. *The store's access log or its bucket configuration.* Rejected for the same reason, and because it shows + only what was called in the window logged, not what the code can call. + +## Decision + +### 1. The object store is the maintained fork of MinIO, for now + +The object-store module runs the maintained fork, pinned by digest, as issue 113's fix left it. This record is +the decision that fix lacked. The dependency stays on **S3 the protocol**, as +[to-be 07](../03-DESIGN/01-to-be/07-the-foundation.md) states it, and the product can change without a change of +design. + +The fork is reconsidered at once, with an issue opened, when any of these holds: + +- the fork publishes no release for **six months**; +- a security advisory against the pinned release stays unpatched for **30 days**; +- the image at the pinned digest can no longer be pulled by a node that holds nothing. + +**How this is checked: nothing notices any of the three yet.** A check that pulls every pinned third-party image as +a node holding nothing would, and watches the object store's upstream for releases and advisories, is tracked by +issue 421, which is open with no worker. Until it is built, the triggers are read by hand when issue 413 is worked. + +### 2. Garage is the first migration candidate, and it is chosen only after the count + +The first step of any migration is the count: for each consumer of `s3-bucket`, the S3 operations and features +its code calls, read from its code and manifest, written into issue 413. **Tracked by issue 413**, whose worker +writes the count. The count reads no data from the store: not a bucket listing, not an object, not a log. + +Garage is chosen only if no consumer needs object versioning, object locking, server-side encryption or a +lifecycle rule beyond plain expiry. If one does, SeaweedFS is measured next on the same count, and a new record +says which. + +### 3. A migration moves only with a backup first, a check of every object after, and the operator's go + +The object store holds the operator's photos, and they are irreplaceable. Any move of its data, to Garage or to +anything else, follows all three of these, in this order, and none is skipped: + +1. **A backup taken first.** A restore point of the store's data directory, taken by the node's `node-backup` + holder before the first byte is copied, and restored beside the live data once to prove it reads. A move + that starts without a proven restore point is refused. +2. **A check of every object after.** For every bucket: the same object count on both sides, and for every + object the same size and the same checksum. A transfer tool's exit code is not a check. One object that + differs fails the move. +3. **The operator's go.** The operator says go before the copy starts and again before consumers are pointed at + the new store. The binding of a consumer to its data moves only by a person + ([ADR 0232](0232-a-binding-to-a-consumers-data-moves-only-by-a-person.md)), so no agent and no walk does it + alone. + +The incumbent stays running read-only beside the new store as the way back for at least **30 days** after the +consumers are pointed away, and its data is removed only after that, by a person, on its own go. The file-sync +application's final copy is taken with its writes stopped, in a window the operator agrees to. + +### Small limits, set here + +- The reconsideration triggers in §1: six months without a release, 30 days for an unpatched advisory. +- The 30 days the incumbent stays as the way back in §3. +- A check compares every object, not a sample: the measured store held 82,496 objects in research 015, and every + one is compared. + +## Consequences + +- Issue 113's fix now has its decision, and the mesh's object store is no longer an unrecorded choice. +- No to-be design is written by this record, because no design changes: to-be 07 already says the dependency is on + the protocol. The design for a migration is written when the count chooses its product, under issue 413. +- Research 015's open question on mirroring third-party images into the mesh's own registry is not answered + here. It is a general question about every module's images, and it needs a decision of its own; issue 113, + where it was first asked, is resolved, so it is tracked by issue 421 together with the check above. +- What got harder: nothing moves off the fork without the count, a proven restore point, a check of every object + and the operator's go twice. That is slower than a copy, on purpose. +- The gap between the store's declared class (`valuable`) and the photos it holds (`irreplaceable`) stands until a + consumer declares `kept-by`. It is tracked by issue 413 beside the count (the issue's second note), and the + consumer that keeps the photos declares `kept-by` before a migration starts, because the backup a migration takes is + sized and kept by that class. + +## References + +- [Research 015 — the object store after MinIO](../01-RESEARCH/015-the-object-store-after-minio/00-overview.md) and + [its comparison](../01-RESEARCH/015-the-object-store-after-minio/01-candidate-comparison.md). +- [Issue 113](../04-ISSUES/113-the-object-stores-images-were-withdrawn-upstream/00-report.md) and its fix in the + catalogue's object-store module. +- Issue 413 (the count of S3 features, the missing `kept-by`, and any migration) and issue 421 (nothing notices a + withdrawn image or a silent upstream), in the issue tracker. +- [ADR 0232](0232-a-binding-to-a-consumers-data-moves-only-by-a-person.md), + [ADR 0233](0233-a-module-declares-the-data-it-holds-and-the-mesh-protects-and-watches-it-from-that.md), + [ADR 0236](0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md). +- [To-be 07 — the foundation](../03-DESIGN/01-to-be/07-the-foundation.md). diff --git a/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/COMMIT b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/COMMIT new file mode 100644 index 00000000..fa7600b0 --- /dev/null +++ b/modules/mesh-knowledge/cmd/mesh-knowledge/testdata/hq-f0892019/COMMIT @@ -0,0 +1 @@ +f08920199d2540e80724ea86cacb8e7cb8882151