Merge pull request 'Nothing reads decision records from hq: records_decisions retired, reserve asks the knowledge base, an empty import refuses (hq issue 488)' (#214) from fix/488-nothing-reads-records-from-hq into main

This commit was merged in pull request #214.
This commit is contained in:
2026-10-11 10:55:19 +00:00
18 changed files with 403 additions and 330 deletions
+21 -19
View File
@@ -202,20 +202,21 @@ sessions writing a record at once never take one number.
refused, and its number stays spent.
- **A refused reserve takes no number**: a kind other than `decision` or `research`, a `for` that is not one
line of plain text or looks like a secret, an unknown argument.
- **A counter starts once**, at the first reserve of its kind, at the highest of: the numbers hq holds in
`02-DECISIONS` or `01-RESEARCH` (read through the records module's `records_list` at its checkout, the
number before the first `-` of each document or folder), the setting `decision-floor` or `research-floor`,
and the reservations kept. The first number handed out is one above. Until hq can be read, the first
reserve of a kind is refused and nothing is spent.
- **Never at or below a number hq holds**: each reserve reads hq again, and a counter that stands below the
highest number there (one taken by hand) is raised to it first, written only at the revision read. When
hq cannot be read once the counter has started, the counter is used as it stands, and the answer's
`warning` says the number was not checked against hq.
- **Open branches are the floor's job.** The tracker reads hq's checkout of `main` and cannot list a folder on
another branch with the forge tools it may call; a number taken on an open branch and not yet merged is
covered by the floor. Before the first reserve of a kind, the operator raises `decision-floor` or
`research-floor` (the controller's settings verb) to the highest number on any open branch of hq; once the
counter has started, the floor is not read again.
- **A counter starts once**, at the first reserve of its kind, at the highest of: the highest number of that
kind the knowledge base holds (`mesh-knowledge.knowledge_highest`, which counts every number spent there:
accepted, proposed, reserved or withdrawn; hq issue 488), the setting `decision-floor` or `research-floor`,
and the reservations kept. The first number handed out is one above. hq's folders are not read: the records
live in the knowledge base, and those folders empty when they leave hq (hq issue 426). When the knowledge base
does not answer (or holds none of the kind), the first reserve of a kind is refused and nothing is spent,
whatever the floor says: a floor raises the start, and never stands in for the knowledge base's answer.
- **Never at or below a number held**: each reserve asks the knowledge base again, and a counter that stands
below the highest number there is raised to it first, written only at the revision read. When the knowledge
base does not answer once the counter has started, the counter is used as it stands, and the answer's
`warning` says the number was not checked against the records held.
- **Open branches are the floor's job** while records are still written in hq: a number taken on an open
branch and not yet imported is covered by the floor. Before the first reserve of a kind, the operator raises
`decision-floor` or `research-floor` (the controller's settings verb) to the highest number on any open
branch of hq; once the counter has started, the floor is not read again.
## State and events
@@ -279,11 +280,12 @@ in a row. It counts the writes lost to another opener, so it fails when the open
the counter written without its revision check, it fails.
`reserve`'s acceptance — sessions reserving at the same moment never get one number, and the first is above
what hq holds — is `TestFortyReservationsAtOnceFromTwoInstancesGetFortyDistinctConsecutiveNumbers` on the
same bus, with the records module played by the test, and the tests in `reserve_test.go`: the start above
hq, the floor and the reservations kept, a counter behind hq raised, a refused reserve spending nothing,
the first reserve refused while hq cannot be read, and a reservation that cannot be kept leaving its number
spent.
what the knowledge base holds — is `TestFortyReservationsAtOnceFromTwoInstancesGetFortyDistinctConsecutiveNumbers`
on the same bus, with the knowledge base played by the test, and the tests in `reserve_test.go`: the start above
the knowledge base, the floor and the reservations kept, a counter behind the knowledge base raised, a refused
reserve spending nothing, the first reserve refused while the knowledge base does not answer, even with a floor
set, a kind held none of being no answer, hq's folders never read, and a
reservation that cannot be kept leaving its number spent.
The import is tested over `testdata/hq`, copies of seven real reports of hq chosen for their shapes (a
bracketed list with commas inside parentheses, quoted block items, an apostrophe in prose, a `resolved:`
@@ -204,18 +204,14 @@ func (i *instance) answer(m rpc) {
return map[string]any{}, nil
}
if m.Method == "mesh/ask" {
// The records module, as a reservation asks it what numbers hq holds (issue 424).
// The knowledge base, as a reservation asks it the highest numbers it holds (hq issue 488).
var a struct {
Key string `json:"key"`
Body struct {
Folder string `json:"folder"`
} `json:"body"`
Key string `json:"key"`
}
if err := json.Unmarshal(m.Params, &a); err != nil || a.Key != "records.records_list" {
if err := json.Unmarshal(m.Params, &a); err != nil || a.Key != HighestTool {
return nil, fmt.Errorf("the runtime answers no ask %s in this test", a.Key)
}
listing := hqToday()
return map[string]any{"folders": listing.folders[a.Body.Folder], "documents": listing.docs[a.Body.Folder]}, nil
return map[string]any{"decision": 291, "research": 45, "records": 336}, nil
}
kv := i.b.kvs[p.State]
if kv == nil {
@@ -100,7 +100,7 @@ func TestItDeclaresAndHoldsTheIssueTrackerSeatOnceForTheMesh(t *testing.T) {
// the import and a reservation read the record through the records module (issue 424).
if !reflect.DeepEqual(m.Consumes, []string{"mesh-issues." + EventOpened, "mesh-issues." + EventMoved, "mesh-delivery.transition"}) ||
!reflect.DeepEqual(m.Invokes, []string{"claude-code.claude_code_judge", "seat:mesh-delivery.checks", "seat:operator-channel.notify",
"records.records_status", "records.records_list", "records.records_read"}) {
"records.records_status", "records.records_list", "records.records_read", "mesh-knowledge.knowledge_highest"}) {
t.Fatalf("consumes %v, invokes %v", m.Consumes, m.Invokes)
}
if len(m.Build.Artifacts) != 1 || m.Build.Artifacts[0].From != "cmd/mesh-issues" ||
+59 -48
View File
@@ -4,7 +4,6 @@ import (
"encoding/json"
"errors"
"fmt"
"regexp"
"strconv"
"strings"
"time"
@@ -17,6 +16,9 @@ import (
// sessions writing a record at the same moment never take one number. A reserved number is spent forever,
// whether or not its record ever lands; each reservation is kept in the state `reservations` and said as the
// event `reserved`.
//
// A counter starts, and is raised, above the highest number the knowledge base holds (its knowledge_highest,
// hq issue 488): the records live there, and hq's folders empty when they leave it (hq issue 426).
const (
// ReservationsState keeps one key per reserved number, named by ReservationKey.
@@ -27,9 +29,8 @@ const (
KindDecision = "decision"
KindResearch = "research"
// DecisionsFolder and ResearchFolder are where hq keeps the records whose numbers a counter starts above.
DecisionsFolder = "02-DECISIONS"
ResearchFolder = "01-RESEARCH"
// HighestTool is the knowledge base's tool that answers the highest number it holds of each kind.
HighestTool = "mesh-knowledge.knowledge_highest"
// raiseAttempts is how many times a counter behind the record is read and raised again.
raiseAttempts = 16
@@ -38,23 +39,19 @@ const (
// ReserveKinds are the sequences reserve hands numbers from; each is also its counter's key in `counters`.
var ReserveKinds = []string{KindDecision, KindResearch}
// sequence is one kind of record numbered by the tracker: where the record keeps it, how many digits its
// number is written with, and the operator's floor for it.
// sequence is one kind of record numbered by the tracker: how many digits its number is written with, and the
// operator's floor for it.
type sequence struct {
kind string
folder string
digits int
floor func(Settings) uint64
}
var sequences = map[string]sequence{
KindDecision: {KindDecision, DecisionsFolder, 4, func(s Settings) uint64 { return s.DecisionFloor }},
KindResearch: {KindResearch, ResearchFolder, 3, func(s Settings) uint64 { return s.ResearchFloor }},
KindDecision: {KindDecision, 4, func(s Settings) uint64 { return s.DecisionFloor }},
KindResearch: {KindResearch, 3, func(s Settings) uint64 { return s.ResearchFloor }},
}
// recordNumber is a record's name that starts with its number: 0291-some-words.md, 045-some-words.
var recordNumber = regexp.MustCompile(`^([0-9]{1,9})-`)
// ReservationKey is a reservation's key: the kind and the number as the record writes it, such as
// decision-0292 or research-046.
func ReservationKey(kind string, n uint64) string {
@@ -115,12 +112,8 @@ func (t *Tracker) reserve(by string, a ReserveAsk) (*Reserved, error) {
return nil, fmt.Errorf("%v. No number was taken", err)
}
}
records := t.Records
if records == nil {
records = busRecords{}
}
highest, rerr := highestIn(records, seq)
n, unread, err := t.nextOf(seq, highest, rerr)
highest, herr := t.highestHeld(seq)
n, unread, err := t.nextOf(seq, highest, herr)
if err != nil {
return nil, err
}
@@ -152,47 +145,65 @@ func joinWarnings(a, b string) string {
return a + "; and " + b
}
// highestIn is the highest number the record holds in a sequence's folder, sub-folders and documents alike.
func highestIn(records Records, seq sequence) (uint64, error) {
folders, docs, err := records.List(seq.folder)
// highestHeld is the highest number of a sequence the knowledge base holds. A knowledge base that holds none of
// the kind answers nothing: a counter started from 0 would hand out 1.
func (t *Tracker) highestHeld(seq sequence) (uint64, error) {
ask := t.Highest
if ask == nil {
ask = busHighest
}
held, err := ask()
if err != nil {
return 0, err
}
var highest uint64
for _, name := range append(folders, docs...) {
if m := recordNumber.FindStringSubmatch(name); m != nil {
if n, err := strconv.ParseUint(m[1], 10, 64); err == nil && n > highest {
highest = n
}
}
n := held[seq.kind]
if n == 0 {
return 0, fmt.Errorf("the knowledge base holds no %s record", seq.kind)
}
return highest, nil
return n, nil
}
// busHighest asks the knowledge base for the highest number it holds of each kind.
func busHighest() (map[string]uint64, error) {
raw, err := stdio.Ask(HighestTool, map[string]any{})
if err != nil {
return nil, fmt.Errorf("%s did not answer: %v", HighestTool, err)
}
var h struct {
Decision *uint64 `json:"decision"`
Research *uint64 `json:"research"`
}
if err := json.Unmarshal(raw, &h); err != nil || h.Decision == nil || h.Research == nil {
return nil, fmt.Errorf("%s answered no highest numbers: %.200s", HighestTool, raw)
}
return map[string]uint64{KindDecision: *h.Decision, KindResearch: *h.Research}, nil
}
// nextOf takes a sequence's next number. A counter not yet started is started once, above the highest the
// record holds, the operator's floor and every reservation kept; it is never started without the record
// read. A started counter behind the record (a number taken by hand) is raised to it first; a record that
// cannot be read then leaves the counter as it is, and the warning answered says so.
func (t *Tracker) nextOf(seq sequence, highest uint64, recordErr error) (uint64, string, error) {
// knowledge base holds, the operator's floor and every reservation kept; with the knowledge base silent it is not
// started, floor or no floor, and nothing is spent. A started counter
// behind the knowledge base (a number taken by hand, or imported) is raised to it first; a knowledge base that
// does not answer then leaves the counter as it is, and the warning answered says so.
func (t *Tracker) nextOf(seq sequence, highest uint64, heldErr error) (uint64, string, error) {
warning := ""
e, err := t.Store.Get(CountersState, seq.kind)
if err != nil {
return 0, "", fmt.Errorf("the %s counter cannot be read (%v); no number was taken", seq.kind, err)
}
if e == nil {
if recordErr != nil {
return 0, "", fmt.Errorf("the %s counter has not started, and what numbers hq's %s holds cannot be read through "+
"the records module (%v): it starts above them, so no number was taken. Ask again when the records "+
"module answers", seq.kind, seq.folder, recordErr)
if heldErr != nil {
return 0, "", fmt.Errorf("the %s counter has not started, and the knowledge base does not answer the highest %s "+
"number it holds (%v): the counter starts above it, and a floor alone cannot say a number is free, so no "+
"number was taken. Ask again when the knowledge base answers", seq.kind, seq.kind, heldErr)
}
if err := t.startSequence(seq, highest); err != nil {
return 0, "", err
}
} else if recordErr != nil {
t.Logf("[mesh-issues] the record's %s could not be read (%v); the %s counter is used as it stands", seq.folder, recordErr, seq.kind)
warning = fmt.Sprintf("what numbers hq's %s holds could not be read through the records module (%v): the "+
"number comes from the %s counter as it stands, unchecked against hq. Check that hq holds no %s at or above it",
seq.folder, recordErr, seq.kind, seq.kind)
} else if heldErr != nil {
t.Logf("[mesh-issues] the knowledge base did not answer its highest %s (%v); the %s counter is used as it stands", seq.kind, heldErr, seq.kind)
warning = fmt.Sprintf("the knowledge base did not answer the highest %s number it holds (%v): the number comes "+
"from the %s counter as it stands, unchecked against the records held. Check that the knowledge base holds no "+
"%s at or above it", seq.kind, heldErr, seq.kind, seq.kind)
} else if err := t.raise(seq.kind, highest); err != nil {
return 0, "", err
}
@@ -212,7 +223,7 @@ func (t *Tracker) nextOf(seq sequence, highest uint64, recordErr error) (uint64,
"caller %d times; no number was taken. Reserve again", seq.kind, counterRounds*counterAttempts)
}
// startSequence makes a sequence's counter, only when there is none, at the highest of the record, the
// startSequence makes a sequence's counter, only when there is none, at the highest of the knowledge base, the
// floor and the reservations kept: the first number handed out is one above all three.
func (t *Tracker) startSequence(seq sequence, highest uint64) error {
s, err := t.Settings()
@@ -239,8 +250,8 @@ func (t *Tracker) startSequence(seq sequence, highest uint64) error {
return nil
}
// raise sets a counter to the highest number the record holds when it stands below it, written only at the
// revision read: it only ever goes up.
// raise sets a counter to the highest number the knowledge base holds when it stands below it, written only at
// the revision read: it only ever goes up.
func (t *Tracker) raise(key string, highest uint64) error {
for i := 0; i < raiseAttempts; i++ {
e, err := t.Store.Get(CountersState, key)
@@ -258,16 +269,16 @@ func (t *Tracker) raise(key string, highest uint64) error {
}
_, err = t.Store.Update(CountersState, key, highest, e.Revision)
if err == nil {
t.Logf("[mesh-issues] the %s counter stood at %d below %d in the record; raised to it", key, n, highest)
t.Logf("[mesh-issues] the %s counter stood at %d below %d in the knowledge base; raised to it", key, n, highest)
return nil
}
if !errors.Is(err, stdio.ErrChanged) {
return fmt.Errorf("the %s counter stands below the record's %d and could not be raised (%v); no number was "+
return fmt.Errorf("the %s counter stands below the knowledge base's %d and could not be raised (%v); no number was "+
"taken", key, highest, err)
}
t.Sleep(jitter(time.Duration(i+1) * 5 * time.Millisecond))
}
return fmt.Errorf("the %s counter was changed by other callers %d times while it was raised to the record's %d; "+
return fmt.Errorf("the %s counter was changed by other callers %d times while it was raised to the knowledge base's %d; "+
"no number was taken. Reserve again", key, raiseAttempts, highest)
}
@@ -10,46 +10,66 @@ import (
)
// The tracker hands out decision and research numbers too (novox/hq ADR 0275 rule 8, ADR 0290 rule 5, issue
// 424): one counter per sequence, advanced like the issue counter, started above every number the record
// holds and the operator's floor, and a reserved number is spent forever.
// 424): one counter per sequence, advanced like the issue counter, started above every number the knowledge base
// holds (hq issue 488) and the operator's floor, and a reserved number is spent forever.
// listedRecords is the record as a folder listing: what records_list answers per folder, or an error.
type listedRecords struct {
// held is the knowledge base as reserve asks it (knowledge_highest, hq issue 488): the highest number it holds
// of each kind, or an error.
type held struct {
mu sync.Mutex
folders map[string][]string
docs map[string][]string
highest map[string]uint64
err error
lists int
asks int
}
func (l *listedRecords) Standing() (string, error) { return "1ea678c", nil }
func (l *listedRecords) Read(string) (string, string, error) { return "", "", errors.New("not read") }
func (l *listedRecords) List(folder string) ([]string, []string, error) {
l.mu.Lock()
defer l.mu.Unlock()
l.lists++
if l.err != nil {
return nil, nil, l.err
func (k *held) Highest() (map[string]uint64, error) {
k.mu.Lock()
defer k.mu.Unlock()
k.asks++
if k.err != nil {
return nil, k.err
}
return l.folders[folder], l.docs[folder], nil
}
// hqToday is hq as the records module holds it: decisions as documents, research as folders.
func hqToday() *listedRecords {
return &listedRecords{
docs: map[string][]string{DecisionsFolder: {"README.md", "0001-the-first.md", "0290-no-copy-in-hq.md",
"0291-an-issue-lives-whole-in-the-issue-tracker.md"}},
folders: map[string][]string{ResearchFolder: {"001-the-first", "044-what-leaves-the-controller-next",
"045-a-knowledge-base-the-mesh-writes", "_template"}},
out := map[string]uint64{}
for kind, n := range k.highest {
out[kind] = n
}
return out, nil
}
func reserveHarness(t *testing.T) (*harness, *listedRecords) {
func (k *held) set(kind string, n uint64) {
k.mu.Lock()
defer k.mu.Unlock()
k.highest[kind] = n
}
// unread is hq's folders, which reserve never reads for a number: the records leave hq (hq issue 426).
type unread struct{ lists int }
func (u *unread) Standing() (string, error) { return "1ea678c", nil }
func (u *unread) Read(string) (string, string, error) { return "", "", errors.New("not read") }
func (u *unread) List(string) ([]string, []string, error) {
u.lists++
return nil, nil, errors.New("reserve read hq's folders")
}
// heldToday is the knowledge base as it stands: decisions to 0291, research to 045.
func heldToday() *held {
return &held{highest: map[string]uint64{KindDecision: 291, KindResearch: 45}}
}
func reserveHarness(t *testing.T) (*harness, *held) {
t.Helper()
h := newHarness(t)
r := hqToday()
h.Records = r
return h, r
k := heldToday()
hq := &unread{}
h.Records = hq
h.Highest = k.Highest
t.Cleanup(func() {
if hq.lists != 0 {
t.Errorf("reserve read hq's folders %d time(s)", hq.lists)
}
})
return h, k
}
func (h *harness) reserve(t *testing.T, kind string) uint64 {
@@ -74,7 +94,7 @@ func (m *memory) counterOf(t *testing.T, key string) uint64 {
return n
}
func TestTheFirstReservedNumberIsAboveTheHighestTheRecordHolds(t *testing.T) {
func TestTheFirstReservedNumberIsAboveTheHighestTheKnowledgeBaseHolds(t *testing.T) {
h, _ := reserveHarness(t)
if got := h.reserve(t, KindDecision); got != 292 {
t.Fatalf("the first decision number is %d, want 292: one above 0291", got)
@@ -111,14 +131,12 @@ func TestTheCounterNeverStartsBelowAReservationAlreadyKept(t *testing.T) {
}
}
// A number the record holds above the counter — taken by hand while sessions still did — raises it: no
// number at or below one that exists is handed out.
// A number the knowledge base holds above the counter — imported, or taken by hand while sessions still did —
// raises it: no number at or below one held is handed out.
func TestARecordAboveTheCounterRaisesIt(t *testing.T) {
h, r := reserveHarness(t)
h, k := reserveHarness(t)
h.reserve(t, KindDecision) // 292, the counter started
r.mu.Lock()
r.docs[DecisionsFolder] = append(r.docs[DecisionsFolder], "0300-taken-by-hand.md")
r.mu.Unlock()
k.set(KindDecision, 300)
if got := h.reserve(t, KindDecision); got != 301 {
t.Fatalf("got %d, want 301: one above 0300", got)
}
@@ -182,7 +200,7 @@ func TestManyConcurrentReservationsGetDistinctConsecutiveNumbers(t *testing.T) {
}
func TestARefusedReservationSpendsNothing(t *testing.T) {
h, r := reserveHarness(t)
h, k := reserveHarness(t)
refused := []ReserveAsk{
{Kind: ""},
{Kind: "issue"},
@@ -200,39 +218,73 @@ func TestARefusedReservationSpendsNothing(t *testing.T) {
}
keys, _ := h.mem.Keys(CountersState)
kept, _ := h.mem.Keys(ReservationsState)
if len(keys) != 0 || len(kept) != 0 || len(h.said()) != 0 || r.lists != 0 {
t.Fatalf("a refused reservation left counters %v, reservations %v, events %v, record reads %d", keys, kept, h.said(), r.lists)
if len(keys) != 0 || len(kept) != 0 || len(h.said()) != 0 || k.asks != 0 {
t.Fatalf("a refused reservation left counters %v, reservations %v, events %v, knowledge base asks %d", keys, kept, h.said(), k.asks)
}
if got := h.reserve(t, KindDecision); got != 292 {
t.Fatalf("the first valid reservation got %d, want 292", got)
}
}
// Before its counter starts, a sequence must know what the record holds: unreadable, nothing is handed out.
// Once started, the counter alone is enough.
func TestAnUnreadableRecordRefusesTheFirstNumberAndNotTheNext(t *testing.T) {
h, r := reserveHarness(t)
r.err = errors.New("records did not answer")
if _, err := h.Reserve(ReserveAsk{Kind: KindDecision}); err == nil || !strings.Contains(err.Error(), "records did not answer") {
t.Fatalf("a first reservation with the record unreadable: %v", err)
// Before its counter starts, a sequence must know what the knowledge base holds, or have the operator's floor:
// with neither, nothing is handed out and nothing is spent. Once started, the counter alone is enough, and says it
// was not checked.
func TestAKnowledgeBaseThatDoesNotAnswerRefusesTheFirstNumberAndNotTheNext(t *testing.T) {
h, k := reserveHarness(t)
k.err = errors.New("nothing answers mesh-knowledge.knowledge_highest")
if _, err := h.Reserve(ReserveAsk{Kind: KindDecision}); err == nil || !strings.Contains(err.Error(), "knowledge_highest") {
t.Fatalf("a first reservation with the knowledge base silent: %v", err)
}
if h.mem.counterOf(t, KindDecision) != 0 || len(h.said()) != 0 {
t.Fatal("a refused first reservation started the counter or said something")
}
r.err = nil
if keys, _ := h.mem.Keys(ReservationsState); len(keys) != 0 {
t.Fatal("a refused first reservation spent a number")
}
k.err = nil
h.reserve(t, KindDecision) // 292
r.err = errors.New("records did not answer")
k.err = errors.New("nothing answers mesh-knowledge.knowledge_highest")
out, err := h.Reserve(ReserveAsk{Kind: KindDecision})
if err != nil || out.Number != 293 {
t.Fatalf("got %+v (%v), want 293 from the started counter", out, err)
}
// Failure is loud: the answer says the number was not checked against hq.
if !strings.Contains(out.Warning, "records did not answer") || !strings.Contains(out.Warning, "unchecked") {
t.Fatalf("a number unchecked against hq answered warning %q", out.Warning)
// Failure is loud: the answer says the number was not checked against the knowledge base.
if !strings.Contains(out.Warning, "knowledge_highest") || !strings.Contains(out.Warning, "unchecked") {
t.Fatalf("a number unchecked against the knowledge base answered warning %q", out.Warning)
}
r.err = nil
k.err = nil
if out, err := h.Reserve(ReserveAsk{Kind: KindDecision}); err != nil || out.Warning != "" {
t.Fatalf("a reserve with hq read answered %+v (%v): no warning expected", out, err)
t.Fatalf("a reserve with the knowledge base read answered %+v (%v): no warning expected", out, err)
}
}
// A counter not started never starts without the knowledge base's answer, floor or no floor: a floor alone
// cannot say a number above it is free. Nothing is spent. With the answer, the floor still raises the start.
func TestAFloorDoesNotStartACounterTheKnowledgeBaseDoesNotAnswerFor(t *testing.T) {
h, k := reserveHarness(t)
k.err = errors.New("the knowledge base holds no record yet")
h.settings.ResearchFloor = 47
if out, err := h.Reserve(ReserveAsk{Kind: KindResearch}); err == nil || !strings.Contains(err.Error(), "the knowledge base holds no record yet") {
t.Fatalf("a research number from the floor alone: %+v %v", out, err)
}
if keys, _ := h.mem.Keys(ReservationsState); len(keys) != 0 || h.mem.counterOf(t, KindResearch) != 0 {
t.Fatal("a refused first reservation spent a number or started the counter")
}
k.err = nil
if got := h.reserve(t, KindResearch); got != 48 {
t.Fatalf("got %d, want 48: one above the floor 47, above the 045 held", got)
}
}
// A knowledge base that holds no record of a kind answers nothing for it: a 0 would start the counter at 1.
func TestAKindTheKnowledgeBaseHoldsNoneOfIsNoAnswer(t *testing.T) {
h, k := reserveHarness(t)
k.set(KindResearch, 0)
if _, err := h.Reserve(ReserveAsk{Kind: KindResearch}); err == nil || !strings.Contains(err.Error(), "holds no research") {
t.Fatalf("a research number from a knowledge base that holds none: %v", err)
}
if got := h.reserve(t, KindDecision); got != 292 {
t.Fatalf("got %d", got)
}
}
@@ -28,6 +28,9 @@ type Tracker struct {
Sleep func(time.Duration)
// Records is the record the import reads; nil is the records module over the bus.
Records Records
// Highest is the highest decision and research number the knowledge base holds, which reserve starts and
// raises its counters above (hq issue 488); nil is the knowledge base over the bus.
Highest func() (map[string]uint64, error)
// Judge is the triage's way to a model (ADR 0278, ADR 0279); nil judges nothing.
Judge Judgement
// Deliveries answers a pull request's delivery (ADR 0279 §3, ADR 0281); nil asks nothing.
+2 -1
View File
@@ -63,7 +63,8 @@
"seat:operator-channel.notify",
"records.records_status",
"records.records_list",
"records.records_read"
"records.records_read",
"mesh-knowledge.knowledge_highest"
],
"state": [
{
+7 -1
View File
@@ -28,6 +28,11 @@ operator, it is accepted only on the operator's Approve, given on a channel that
`ask`, `get`, `holds` and `history` stay within `bound` bytes (4096 unless asked for, at most 65536) and say what they cut
and how to read the rest. The module's own tool `knowledge_health` answers its health (below).
Its tool `knowledge_highest` answers the highest decision and research number it holds, every number spent
counted (accepted, proposed, reserved or withdrawn): the issue tracker's `reserve` starts and raises its counters
above them (hq issue 488). It refuses while nothing is held. It is a tool of the module, not a verb of the seat,
which serves the verbs ADR 0290 rule 6 names.
**Who acted** is the caller as the runtime proves it, never an argument: no verb takes one. The tool runner takes
the caller's bus user from the caller-named subject a call arrived on and hands it through the SDK
(`stdio.CallerOf`, hq issue 365). A call on the subject that names no caller (served beside it for one release, hq
@@ -123,7 +128,8 @@ as text. `get` serves each, and any other field a record of hq wrote under `fiel
## The import
`import` reads hq through the records module at the commit its checkout stands at. With `commit`, it refuses a
checkout standing elsewhere. It writes every decision record and research effort whole, as version 1, naming the
checkout standing elsewhere. At a commit where hq holds no decision record and no research effort (the records
have left it, hq issue 426), it refuses and says so, rather than answering "nothing created" as a success. It writes every decision record and research effort whole, as version 1, naming the
commit and path it came from. It creates only what is absent and never overwrites. It never asks for a number, so
no counter moves. A second run changes nothing and says so. `dry-run` writes nothing. From a terminal,
`mesh-knowledge dry-run <hq checkout>` gives the same dry run over a checkout on disk, against an empty knowledge
@@ -0,0 +1,62 @@
package main
// The highest number held per kind (hq issue 488): what the issue tracker's reserve starts and raises its decision
// and research counters above (ADR 0275 rule 8, ADR 0290 rule 5), now that the records live here and hq's folders
// empty. Every number held counts — accepted, proposed, reserved or withdrawn — since each is spent.
import (
"context"
"errors"
"fmt"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// HighestTool is the module's tool the issue tracker asks.
const HighestTool = "knowledge_highest"
// Highest is the highest number held of each kind, and how many records are held.
type Highest struct {
Decision int `json:"decision"`
Research int `json:"research"`
Records int `json:"records"`
Says string `json:"says"`
}
// HighestHeld is the highest number held per kind. A knowledge base that holds no record answers none: a counter
// started from it would start at 1.
func (k *KB) HighestHeld() (*Highest, error) {
all, err := k.Store.All(context.Background())
if err != nil {
return nil, fmt.Errorf("the knowledge base cannot be read (%v); it answers no highest number", err)
}
h := &Highest{Records: len(all)}
for _, e := range all {
switch r := e.Current; r.Kind {
case KindDecision:
h.Decision = max(h.Decision, r.Number)
case KindResearch:
h.Research = max(h.Research, r.Number)
}
}
if h.Records == 0 {
return nil, errors.New("the knowledge base holds no record yet (it is imported from hq once): it answers no highest number")
}
h.Says = fmt.Sprintf("The highest numbers held, of %d records, every number spent counted: decision %04d, research %03d.",
h.Records, h.Decision, h.Research)
return h, nil
}
func highestTool(k *KB) stdio.Tool {
return stdio.Tool{Name: HighestTool,
Description: "The highest decision number and the highest research number the knowledge base holds, every number spent " +
"counted: accepted, proposed, reserved or withdrawn. The issue tracker starts and raises its counters above them. " +
"Refused while the knowledge base holds no record, or cannot be read: a number is never guessed.",
Input: schema(nil, map[string]any{}),
Run: func(m map[string]any) (any, error) {
if _, err := readArgs(HighestTool, m); err != nil {
return nil, err
}
return k.HighestHeld()
}}
}
@@ -0,0 +1,88 @@
package main
import (
"context"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
// hq issue 488: the issue tracker's reserve takes the highest decision and research number from the knowledge base,
// which holds every record — reserved, withdrawn and proposed numbers included, since each is spent — rather than
// reading hq's folders, which empty when the records leave.
func TestTheHighestToolAnswersTheHighestNumberHeldPerKind(t *testing.T) {
eachStore(t, func(t *testing.T, s Store) {
w := imported(t, s)
tool := highestTool(w.k)
answer := func() Highest {
t.Helper()
out, err := tool.Run(map[string]any{})
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(out)
var h Highest
if err := json.Unmarshal(raw, &h); err != nil {
t.Fatal(err)
}
return h
}
if h := answer(); h.Decision != 290 || h.Research != 45 || h.Records != 11 {
t.Fatalf("%+v", h)
}
// A number reserved, and later withdrawn, is spent: it is the highest.
w.reserved(t, "author@g14") // 300
w.clock = w.clock.Add(DefaultReservedFor + time.Hour)
if _, err := w.k.Withdraw(); err != nil {
t.Fatal(err)
}
if h := answer(); h.Decision != 300 || h.Research != 45 {
t.Fatalf("after a reservation withdrawn: %+v", h)
}
if _, err := tool.Run(map[string]any{"kind": "decision"}); err == nil {
t.Fatal("an argument it does not take was taken")
}
})
// A knowledge base that holds nothing answers nothing: a 0 would start a counter at 1.
empty := newWorld(t, newMemStore(), fixture())
if _, err := highestTool(empty.k).Run(map[string]any{}); err == nil || !strings.Contains(err.Error(), "holds no record") {
t.Fatalf("an empty knowledge base: %v", err)
}
}
// hq issue 488: once the records have left hq, an import at a later commit finds none. It refuses, saying so,
// rather than reporting nothing created as a success.
func TestTheImportRefusesACommitWhereHqHoldsNoRecords(t *testing.T) {
dir := t.TempDir()
write := func(path, text string) {
t.Helper()
if err := writeFileAll(dir+"/"+path, text); err != nil {
t.Fatal(err)
}
}
write("COMMIT", "0123456789abcdef0123456789abcdef01234567\n")
write("02-DECISIONS/README.md", "# Decisions\n\nThey live in the knowledge base.\n")
write("01-RESEARCH/README.md", "# Research\n")
w := newWorld(t, newMemStore(), dirRecords{dir: dir})
for _, dry := range []bool{true, false} {
out, err := w.k.Import("import@test", ImportAsk{DryRun: dry}, dirRecords{dir: dir})
if err == nil || !strings.Contains(err.Error(), "holds no decision record or research effort") ||
!strings.Contains(err.Error(), "0123456789ab") {
t.Fatalf("dry %v: %+v %v", dry, out, err)
}
}
if all, _ := w.k.Store.All(context.Background()); len(all) != 0 {
t.Fatal("written")
}
}
func writeFileAll(path, text string) error {
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return err
}
return os.WriteFile(path, []byte(text), 0o644)
}
@@ -127,6 +127,12 @@ func (k *KB) Import(caller string, a ImportAsk, records Records) (*Imported, err
cands = append(cands, &importCandidate{path: ResearchFolder + "/" + f})
}
}
if len(cands) == 0 {
// The records left hq (hq issue 426): what it held is here, and an import from it would say "nothing
// created" as a success (hq issue 488).
return nil, fmt.Errorf("hq holds no decision record or research effort at commit %s: its %s and %s hold none. "+
"The records live in the knowledge base now, and nothing was imported", short(commit), DecisionsFolder, ResearchFolder)
}
// Read and parse, a few at a time, every document at the one commit.
var moved error
var movedMu sync.Mutex
@@ -98,7 +98,7 @@ func serve(k *KB, status string) {
k.kick = make(chan struct{}, 1)
go hear(k)
go background(k)
if err := stdio.Serve("", append(Tools(k, callerOf), healthTool(status, k.now))); err != nil {
if err := stdio.Serve("", append(Tools(k, callerOf), healthTool(status, k.now), highestTool(k))); err != nil {
logf("%v", err)
os.Exit(1)
}
@@ -118,7 +118,7 @@ func TestItsProcessesAreThePreparationTheActAndTheJudge(t *testing.T) {
}
h, _ := watch["health"].(map[string]any)
if watch == nil || watch["run"].([]any)[1] != "watch" || h["kind"] != "tool" || h["tool"] != HealthTool ||
!reflect.DeepEqual(m.Tools, []string{HealthTool}) {
!reflect.DeepEqual(m.Tools, []string{HealthTool, HighestTool}) {
t.Fatalf("watch %v, tools %v", watch, m.Tools)
}
if watch["env"].(map[string]any)["MESH_KNOWLEDGE_STATUS"] != m.Build.Artifacts[0].Env["MESH_KNOWLEDGE_STATUS"] {
+2 -1
View File
@@ -51,7 +51,8 @@
}
],
"tools": [
"knowledge_health"
"knowledge_health",
"knowledge_highest"
],
"emits": [
"reserved",
+1 -1
View File
@@ -19,7 +19,7 @@ lagging behind it, and the lag is in `records_status`.
| `records_search` `{query, limit?}` | where a phrase appears, as written: document, line, nearest heading, and the commit read |
| `records_read` `{path}` | one document, whole |
| `records_list` `{folder?}` | what a folder holds |
| `records_decisions` `{topic?}` | the decision records by topic, in the reading order `02-DECISIONS/README.md` writes: number, title, path and status, generated when asked and never stored (hq ADR 0248) |
| `records_decisions` `{topic?}` | retired (hq issue 488): refuses, naming the knowledge base's `knowledge-base.holds`, `ask` and `get`. The decision records and research efforts live in the knowledge base (hq ADR 0290), and an index read from an emptied `02-DECISIONS` would answer every topic empty as a success |
| `records_status` | repository, forge, commit and its date, last sync, document count, last error |
| `records_sync` | bring the checkout up to date now |
-142
View File
@@ -1,142 +0,0 @@
package main
// The decision records, by topic, in reading order (novox/hq ADR 0248, issue 297).
//
// The repository writes its reading order in `02-DECISIONS/README.md`, one line per topic:
//
// 1. **What the mesh is** — `topic: the mesh`. What it is for, ...
//
// and each record names its own `topic:`. The list of records under each topic is generated when it is
// read and never stored, because a stored list was a line every decision added to one file, and any two
// decisions open at once conflicted there. This is the same list hq's own `index.py --print` prints, for a
// reader on the mesh rather than in a checkout. Nothing is kept between calls: every answer is read from
// the checkout at the commit it names.
import (
"fmt"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
)
// DecisionsFolder is where the records and their README live.
const DecisionsFolder = "02-DECISIONS"
var (
topicLine = regexp.MustCompile("(?m)^\\d+\\.\\s+\\*\\*([^*]+)\\*\\*\\s+—\\s+`topic:\\s*([^`]+)`")
recordFile = regexp.MustCompile(`^(\d{4})-.+\.md$`)
recordTitle = regexp.MustCompile(`(?m)^# \d+\.\s*(.+)$`)
)
// Decision is one record, as the list names it.
type Decision struct {
Number string `json:"number"`
Title string `json:"title"`
Path string `json:"path"`
Status string `json:"status"`
}
// Topic is one topic of the reading order and the records under it, by number.
type Topic struct {
Topic string `json:"topic"`
Label string `json:"label"`
Decisions []Decision `json:"decisions"`
}
// DecisionsAnswer is the reading order with its records, and what fits none of its topics.
type DecisionsAnswer struct {
Topics []Topic `json:"topics"`
// Unfiled is every record whose topic is not one of the reading order's — hq's `index.py` fails on it,
// so on a merged commit it is empty.
Unfiled []Decision `json:"unfiled,omitempty"`
Commit string `json:"commit"`
}
// frontField is one `name: value` line of a document's frontmatter.
func frontField(text, name string) string {
if !strings.HasPrefix(text, "---\n") {
return ""
}
end := strings.Index(text[4:], "\n---")
if end < 0 {
return ""
}
for _, line := range strings.Split(text[4:4+end], "\n") {
if v, ok := strings.CutPrefix(line, name+":"); ok {
return strings.TrimSpace(v)
}
}
return ""
}
// Decisions is the records by topic, in the reading order the README writes; with a topic named, that
// topic alone. A repository that writes no reading order is answered with why, not with an empty list.
func (r *Records) Decisions(only string) (DecisionsAnswer, error) {
folder := filepath.Join(r.Dir(), DecisionsFolder)
readme, err := os.ReadFile(filepath.Join(folder, "README.md"))
if err != nil {
return DecisionsAnswer{}, fmt.Errorf("the repository holds no %s/README.md, so it writes no reading order", DecisionsFolder)
}
answer := DecisionsAnswer{Topics: []Topic{}}
at := map[string]int{}
for _, m := range topicLine.FindAllStringSubmatch(string(readme), -1) {
name := strings.TrimSpace(m[2])
at[name] = len(answer.Topics)
answer.Topics = append(answer.Topics, Topic{Topic: name, Label: strings.TrimSpace(m[1]), Decisions: []Decision{}})
}
if len(answer.Topics) == 0 {
return DecisionsAnswer{}, fmt.Errorf("%s/README.md writes no reading order: no line of the form "+
"\"1. **Label** — `topic: name`. ...\"", DecisionsFolder)
}
only = strings.TrimSpace(only)
if _, ok := at[only]; only != "" && !ok {
names := make([]string, len(answer.Topics))
for i, t := range answer.Topics {
names[i] = t.Topic
}
return DecisionsAnswer{}, fmt.Errorf("%q is not a topic of the reading order: %s", only, strings.Join(names, ", "))
}
entries, err := os.ReadDir(folder)
if err != nil {
return DecisionsAnswer{}, err
}
names := []string{}
for _, e := range entries {
if e.Type().IsRegular() && recordFile.MatchString(e.Name()) {
names = append(names, e.Name())
}
}
sort.Strings(names)
for _, name := range names {
raw, err := os.ReadFile(filepath.Join(folder, name))
if err != nil {
continue
}
text := string(raw)
d := Decision{
Number: recordFile.FindStringSubmatch(name)[1],
Title: "(no heading)",
Path: DecisionsFolder + "/" + name,
Status: frontField(text, "status"),
}
if m := recordTitle.FindStringSubmatch(text); m != nil {
d.Title = strings.TrimSpace(m[1])
}
topic := frontField(text, "topic")
i, ok := at[topic]
switch {
case !ok && only == "":
answer.Unfiled = append(answer.Unfiled, d)
case ok && (only == "" || only == topic):
answer.Topics[i].Decisions = append(answer.Topics[i].Decisions, d)
}
}
if only != "" {
answer.Topics = []Topic{answer.Topics[at[only]]}
}
answer.Commit = r.Standing().Commit
return answer, nil
}
+14 -5
View File
@@ -1,11 +1,12 @@
// records: the mesh's record — decisions, designs and issues — read where it is written (novox/hq ADR 0025,
// ADR 0153). A Go bundle the node's runtime launches; it keeps a checkout of one repository current on every
// merge the forge announces and on a timer, and answers six questions about it. stdout is the MCP channel;
// merge the forge announces and on a timer, and answers what it holds. stdout is the MCP channel;
// what this module says, it says on stderr.
package main
import (
"encoding/json"
"errors"
"fmt"
"os"
"time"
@@ -27,6 +28,13 @@ func str(description string) map[string]any {
func strArg(a map[string]any, k string) string { s, _ := a[k].(string); return s }
// errDecisionsMoved is records_decisions' answer now that the decision records live whole in the
// knowledge base (novox/hq ADR 0290 rule 9, hq issue 488): read from an emptied folder, it would answer every topic
// empty as a success. The records module stays the literal reader of what the repository still holds.
var errDecisionsMoved = errors.New("the decision records live in the knowledge base now, not in this repository (hq ADR 0290): " +
"ask knowledge-base.holds {\"topic\": \"...\"} for a topic's clauses in force, knowledge-base.ask {\"question\": \"...\"} in plain " +
"words, or knowledge-base.get {\"number\": N} for one record. records_decisions reads nothing")
func tools(r *Records) []stdio.Tool {
return []stdio.Tool{
{Name: "records_search",
@@ -52,10 +60,11 @@ func tools(r *Records) []stdio.Tool {
Input: map[string]any{"folder": str("a folder inside the repository (optional)")},
Run: func(a map[string]any) (any, error) { return r.List(strArg(a, "folder")) }},
{Name: "records_decisions",
Description: "The decision records by topic, in the reading order the repository writes in 02-DECISIONS/README.md: " +
"each record's number, title, path and status. Generated from the records when asked, never stored (novox/hq ADR 0248).",
Input: map[string]any{"topic": str("one topic of the reading order, e.g. \"how we work\" (optional; every topic when omitted)")},
Run: func(a map[string]any) (any, error) { return r.Decisions(strArg(a, "topic")) }},
Description: "Retired: the decision records and research efforts live in the knowledge base, not in this repository " +
"(novox/hq ADR 0290). Ask knowledge-base.holds with a topic for the clauses in force on it, or knowledge-base.ask " +
"in plain words; knowledge-base.get gives one record whole. This tool reads nothing and says so.",
Input: map[string]any{"topic": str("a topic of the reading order (taken, and answered with where to ask)")},
Run: func(map[string]any) (any, error) { return nil, errDecisionsMoved }},
{Name: "records_status",
Description: "Where the checkout stands: the repository, the forge it is read from, the commit and its date, when it was last brought up to date.",
Run: func(map[string]any) (any, error) { return r.Standing(), nil }},
+26 -48
View File
@@ -5,6 +5,7 @@ package main
// one design document, and get it back. Ported with the TypeScript module's tests, and the layout's own.
import (
stdio "git.novox.be/novox/mesh-sdk/go"
"os"
"os/exec"
"path/filepath"
@@ -146,62 +147,39 @@ func TestTheReaderRefusesToGuess(t *testing.T) {
}
}
// The decision records by topic, in the order the README writes, generated from each record's own
// frontmatter (novox/hq ADR 0248): a record lands under its topic, by number, with its status; a record
// whose topic the reading order does not name is unfiled rather than lost; a topic asked for alone is
// answered alone; a repository with no reading order says so.
func TestTheDecisionsAreListedByTopicInTheReadingOrderTheReadmeWrites(t *testing.T) {
// The decision records live in the knowledge base (novox/hq ADR 0290 rule 9, hq issue 488): records_decisions no
// longer reads hq's 02-DECISIONS, which empties when they leave and would answer every topic empty as a success.
// It refuses, naming where the answer is, whatever the checkout holds.
func TestRecordsDecisionsNamesTheKnowledgeBaseAndReadsNothingOfHq(t *testing.T) {
origin := aRepository(t)
write(t, filepath.Join(origin, "02-DECISIONS/README.md"), "# 02-DECISIONS\n\n## Reading order\n\n"+
"1. **What the mesh is** — `topic: the mesh`. What it is for.\n"+
"2. **How we work** — `topic: how we work`. This repository.\n")
write(t, filepath.Join(origin, "02-DECISIONS/0001-a-decision.md"),
"---\ntopic: how we work\nstatus: accepted\n---\n\n# 1. A decision\n")
write(t, filepath.Join(origin, "02-DECISIONS/0003-a-later-one.md"),
"---\ntopic: the mesh\nstatus: superseded\nsuperseded-by: 02-DECISIONS/0004-x.md\n---\n\n# 3. A later one\n")
write(t, filepath.Join(origin, "02-DECISIONS/0002-an-earlier-one.md"),
"---\ntopic: the mesh\nstatus: accepted\n---\n\n# 2. An earlier one\n")
write(t, filepath.Join(origin, "02-DECISIONS/0005-astray.md"),
"---\ntopic: elsewhere\nstatus: proposed\n---\n\n# 5. Astray\n")
"1. **What the mesh is** — `topic: the mesh`. What it is for.\n")
write(t, filepath.Join(origin, "02-DECISIONS/0001-a-decision.md"), "---\ntopic: the mesh\nstatus: accepted\n---\n\n# 1. A decision\n")
run(t, origin, "add", "-A")
run(t, origin, "commit", "--quiet", "-m", "records")
r := &Records{Base: t.TempDir(), Origin: "http://forge.invalid:3000", Repository: "novox/hq", URL: origin}
r.Sync()
got, err := r.Decisions("")
if err != nil {
t.Fatal(err)
var decisions *stdio.Tool
for _, tool := range tools(r) {
if tool.Name == "records_decisions" {
decisions = &tool
}
}
if len(got.Topics) != 2 || got.Topics[0].Topic != "the mesh" || got.Topics[1].Label != "How we work" {
t.Fatalf("not the reading order: %+v", got.Topics)
if decisions == nil {
t.Fatal("records_decisions is not served: a caller would hear no tool, not where to go")
}
mesh := got.Topics[0].Decisions
if len(mesh) != 2 || mesh[0].Number != "0002" || mesh[1].Number != "0003" || mesh[1].Status != "superseded" ||
mesh[0].Title != "An earlier one" || mesh[0].Path != "02-DECISIONS/0002-an-earlier-one.md" {
t.Fatalf("the mesh's records: %+v", mesh)
for _, args := range []map[string]any{{}, {"topic": "the mesh"}} {
out, err := decisions.Run(args)
if err == nil || out != nil {
t.Fatalf("records_decisions %v answered %v: it reads hq's decision records", args, out)
}
for _, want := range []string{"knowledge base", "knowledge-base.holds", "knowledge-base.ask"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("its refusal does not name %q: %v", want, err)
}
}
}
if len(got.Topics[1].Decisions) != 1 || got.Topics[1].Decisions[0].Number != "0001" {
t.Fatalf("how we work's records: %+v", got.Topics[1].Decisions)
}
if len(got.Unfiled) != 1 || got.Unfiled[0].Number != "0005" {
t.Fatalf("a record off the reading order is not unfiled: %+v", got.Unfiled)
}
if got.Commit == "" {
t.Fatal("the answer names no commit")
}
one, err := r.Decisions("how we work")
if err != nil || len(one.Topics) != 1 || one.Topics[0].Topic != "how we work" || len(one.Unfiled) != 0 {
t.Fatalf("one topic: %+v %v", one, err)
}
if _, err := r.Decisions("nowhere"); err == nil || !strings.Contains(err.Error(), "the mesh, how we work") {
t.Fatalf("an unknown topic is not refused with the topics: %v", err)
}
write(t, filepath.Join(origin, "02-DECISIONS/README.md"), "# 02-DECISIONS\n\nNo order here.\n")
run(t, origin, "commit", "--quiet", "-am", "no order")
r.Sync()
if _, err := r.Decisions(""); err == nil || !strings.Contains(err.Error(), "writes no reading order") {
t.Fatalf("a README with no reading order is not said: %v", err)
if !strings.Contains(decisions.Description, "knowledge-base.holds") {
t.Errorf("its description does not say where the decision records are: %s", decisions.Description)
}
}