diff --git a/01-RESEARCH/042-the-mesh-keeps-its-own-issues/00-overview.md b/01-RESEARCH/042-the-mesh-keeps-its-own-issues/00-overview.md new file mode 100644 index 00000000..5a3e8823 --- /dev/null +++ b/01-RESEARCH/042-the-mesh-keeps-its-own-issues/00-overview.md @@ -0,0 +1,77 @@ +--- +status: active +initiated: 2026-10-10 +touches: [] +became: [] +--- + +# 042 — The mesh keeps its own issues + +## What + +Whether the mesh should keep its issues in a module of its own, on the bus, instead of only as +numbered folders in this repository's `04-ISSUES/`, and what the smallest useful first version of +that module is. The operator's ask of 2026-10-10: work on a ticketing system on the bus, consolidate +every current issue into it, start small and extend the module along the way. + +The effort answers six questions, in [01-findings.md](01-findings.md): + +1. The smallest useful first slice: what the module holds, which tools it serves, which events it + emits, and what it explicitly leaves out. +2. Where the truth lives during and after the move: the folders here, the module, or a split + between them, and what each choice does to the public-repository rule, playbook 03 and the hq + checks. +3. How the 357 issue folders are imported and consolidated (duplicates, still-open ones, grouping). +4. How the controller's conditions, the router's asks and deliveries relate to issues. +5. How the operator sees issues: through the mesh's tools first, through a view onto the bus later. +6. The open questions for the operator, and the slices after the first. + +The operator also named an existing project as inspiration: *hits*, a headless issue tracker built +as NATS micro services on JetStream, with an MCP server for agents. Its model is studied in the +findings, along with whether to reuse its code or build on its model. + +## Why + +Numbers collide. Playbook 03 tells an author to take the next free number across `main` and every +open pull request, and [issue 155](../../04-ISSUES/155-two-records-may-share-a-number-and-nothing-says-so/00-report.md) +records the second collision reaching `main` with every check passing. `cycle.py` now refuses two +records sharing a number, which catches a collision after it happens and does not prevent one. On +`main` at the time of writing, 31 commits mention a renumbering. On 2026-10-07 three decision numbers +collided in one evening. On 2026-10-10 an open pull request still carries a branch named for issue +349 and a title naming issue 350, because 349 was taken on `main` first. The operator's answer on +2026-10-07 was that numbering belongs to the mesh's own issue tracker. + +Status is also hard to see. Of 357 issue folders on `main`, 73 are not closed (27 `open`, 45 +`located`, 1 `diagnosing`). Finding them means reading frontmatter across the repository. +Nothing connects a condition the controller raises to the issue that would fix it: on 2026-10-10 a +condition was silenced with the words "hq issue to follow", and nothing records whether one followed. + +## What it touches + +- [ADR 0019](../../02-DECISIONS/0019-how-this-repository-works.md) (statuses live in frontmatter) and + [ADR 0080](../../02-DECISIONS/0080-the-development-cycle-is-checked.md) (the cycle is checked), with + playbook [03](../../00-META/process/03-issues.md) and `00-META/checks/cycle.py`. +- [ADR 0201](../../02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md) + (a module keeps its state in key-value buckets it declares), + [ADR 0197](../../02-DECISIONS/0197-every-tool-announces-itself-on-the-bus-in-the-nats-services-protocol.md) + (tools announce themselves on the bus) and + [ADR 0041](../../02-DECISIONS/0041-events-are-a-relationship.md) (events are declared). +- The conditions of [ADR 0231](../../02-DECISIONS/0231-a-healer-acts-on-what-observation-raised-and-only-observation-says-it-worked.md) + and the asks of [ADR 0259](../../02-DECISIONS/0259-an-ask-is-a-message-on-the-bus-any-channel-answers-it-and-the-asker-acts-on-a-warrant.md). +- The records module, which indexes this repository for agents. + +`touches:` stays empty until the effort settles which of these it changes. + +## Words + +The glossary's word for one of these is **issue** (*The record*: "a report in `04-ISSUES/`"). This +effort keeps that word for the thing the module holds, and proposes **issue tracker** for the module +that holds them, with the module named `issues`. It does not use *ticket*, the operator's word in +the ask, or *item*, the word hits uses. If the glossary entry is widened (an issue is then held by +the issue tracker, and its narrative may live in `04-ISSUES/`), *ticket* would go on its *Not:* line. +That change belongs to the decision this effort graduates into, not to the effort itself. + +## Open questions for the operator + +Listed in [01-findings.md § 7](01-findings.md#7-open-questions-for-the-operator), each with a +recommendation. diff --git a/01-RESEARCH/042-the-mesh-keeps-its-own-issues/01-findings.md b/01-RESEARCH/042-the-mesh-keeps-its-own-issues/01-findings.md new file mode 100644 index 00000000..c35e210e --- /dev/null +++ b/01-RESEARCH/042-the-mesh-keeps-its-own-issues/01-findings.md @@ -0,0 +1,251 @@ +# 01 — Findings + +Measured on 2026-10-10 against `main` of this repository, the open pull requests on the forge, the +controller's open conditions, the module SDK's source, and a clone of the hits repository at its +release v0.7.0. Each section ends with what this effort recommends; section 7 lists what needs the +operator. + +## 0. What exists today, measured + +| Fact | Measured | +|---|---| +| Issue folders on `main` | 357, numbered up to 362; numbers 283 and 284 were never used, and 338, 341 and 350 sit on open pull requests (363 too) | +| Status in frontmatter | 283 `resolved`, 1 `wontfix`, 45 `located`, 27 `open`, 1 `diagnosing` | +| Frontmatter keys | `status`, `opened`, `located-in` on all 357; `fixed-by` 356; `amended-design` 344; `replay` 22; `replay-none` 44 | +| Size | 1.9 MiB of markdown in all; median folder 4.6 KiB, largest 25 KiB; 244 folders hold only a report, 111 also a diagnosis | +| Links between issues | 168 folders link another issue's folder (265 links); 193 name another issue in prose | +| `located-in` | free text: the controller is written `mesh-controller`, `cmd/mesh-controller`, `mesh-control` and as file paths; 17 issues name nothing | +| Renumberings | 31 commits on `main` mention one; one open pull request still carries the old number in its branch name | +| The forge's own issues | none in this repository or the controller's; [issue 178](../../04-ISSUES/178-a-routed-name-resolves-to-a-provider-merely-told-it/00-report.md) was filed there once and moved here, because "its numbers collide with hq's pull request numbers in conversation" | +| Conditions | 10 open; 5 of kind `healer-wanted` (a hand repair repeated, "an automatic repair is wanted"); one silenced with "hq issue to follow" | + +Two things follow. The collisions come from allocating a number by reading a repository that many +branches write at once; nothing short of one allocator removes them. And the frontmatter is a small, +regular data set (six keys, 1.9 MiB) that fits easily in the bus's key-value store under the limits +[ADR 0201](../../02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md) +§6 sets (256 KiB a value, 64 MiB a bucket). + +## 1. The first slice + +**A Go module `issues`, one instance on the control-node, serving one seat held once for the mesh.** +Its state is in buckets it declares (ADR 0201), its tools announce themselves on the bus +([ADR 0197](../../02-DECISIONS/0197-every-tool-announces-itself-on-the-bus-in-the-nats-services-protocol.md)), +and every change is an event it declares under `emits` +([ADR 0041](../../02-DECISIONS/0041-events-are-a-relationship.md)). The module that delivers pull +requests already has this shape: Go, one mesh seat, two state buckets, declared events. + +**What one issue holds in v1**, the frontmatter of today plus what the folders carry in prose: + +- `number` (the key), `title`, `status` (`open`, `diagnosing`, `located`, `resolved`, `wontfix`), + `opened`, `closed`; +- `located-in`, from a registered list of repositories and modules rather than free text; +- `fixed-by` (pull requests or commits), `amended-design`, `replay` or `replay-none`; +- `notes`, appended and never edited: who wrote, when, and the text; +- `links`, typed: to another issue (`duplicates`, `relates-to`), to a pull request, a decision + record, a design, a condition key or a delivery; +- `record`: the path of the issue's folder in this repository, when it has one (section 2). + +**Storage.** One bucket `issues` with one key per issue, keeping a short history per key, and a +`counter` key for the next number. The events (`opened`, `changed`, `noted`, `linked`) carry what +changed and who changed it, so they are the audit trail, and a view can watch the bucket. Section 6 +compares this with a stream as the source of record. + +**Tools** (verbs of the seat, found with `mesh_search`): + +| Tool | Does | +|---|---| +| `open` | takes the next number atomically and writes the issue; refuses a title or report that is empty | +| `get` | one issue, whole | +| `list` | by status, by `located-in`, by a phrase in title or notes; compact rows, newest first | +| `move` | changes status, enforcing today's cycle rules at the write: `located` needs `located-in`; `resolved` needs `fixed-by`; a core issue needs `replay` or `replay-none`; `wontfix` needs a reason | +| `note` | appends to the trail | +| `link` / `unlink` | adds or removes a typed link | +| `import` | reads this repository's issue folders and writes them as issues with their own numbers (section 3); idempotent, reporting what it would change before it does | + +**The caller is the bus credential, not an argument.** The module records who acted from the +identity the runtime gives it with each call, so an agent cannot write as someone else. + +**Atomic numbering needs compare-and-set.** The SDK's state interface (v0.1.9) offers `get`, `put`, +`delete`, `keys` and `watch`, and no write that is refused when the key changed since it was read. +Whether the runtime's `mesh/state.put` takes an expected revision was not checked; the tool runner's +code was not read for this effort. Two ways out: add `create` and an `update` that takes the revision +read to the state interface (small, and every module with a counter gains it), or run exactly one +instance and number inside it. **Recommended: the first.** One instance is also right for v1, but the +counter should not depend on it. + +**Explicitly left out of v1:** decision and research numbers (slice 2), any change to playbook 03 or +the hq checks (slice 2), duplicate finding beyond links (slice 3), opening an issue from a condition +(slice 4), a web view (slice 5), full-text or semantic search, priorities, claims, notifications, and +the forge's own issues. + +## 2. Where the truth lives + +| | (a) Split: the tracker holds the issue, hq holds the narrative | (b) Everything moves | (c) The tracker mirrors hq | +|---|---|---|---| +| Number | tracker | tracker | hq, so collisions stay | +| Status, `located-in`, `fixed-by`, links | tracker | tracker | hq frontmatter, copied | +| Report and diagnosis | hq folder for a design or governance question; notes in the tracker for the rest | tracker notes | hq | +| Public-repository rule | holds: hq keeps only scrubbed narrative, the tracker is private and may name nodes | the public record of design questions is lost | holds | +| Playbook 03 | changes: open in the tracker first, write a folder only for a design or governance question | replaced | unchanged | +| `cycle.py` | its issue rules move into the tracker's `move`, refused at the write instead of at merge | same | unchanged, and a second copy to keep in step | +| Records module | still finds the narrative; the tracker's `list` and `get` serve status | finds no issues | unchanged | + +Option (c) keeps the collisions and adds a second copy that can drift, so it answers neither of the +operator's problems. Option (b) loses the public record that hq exists to be +([ADR 0019](../../02-DECISIONS/0019-how-this-repository-works.md)), and with it the +symptom-to-component memory that playbook 03 says is never deleted. + +**Recommended: (a).** Every issue gets a number from the tracker, including ones that never need a +document here (a node's quirk, an operational follow-up, which playbook 03's table already sends +elsewhere). A design or governance issue also gets a folder `04-ISSUES/NNN-slug/` with the tracker's +number, whose frontmatter shrinks to a pointer (`issue: NNN`) and whose prose stays reviewed in pull +requests. Status lives in one place, the tracker. This reverses ADR 0019's "statuses live only in +frontmatter" for issues and moves part of ADR 0080's check, so it needs a decision record that +supersedes those parts, written when this effort graduates. + +**Numbering continues.** The tracker's counter starts above the highest number on `main` and on open +branches at the moment of the switch (today 363, so the first new issue is 364). Numbers 283 and 284 +stay unused, and a number never moves again. + +**During the move**, hq frontmatter stays authoritative until the import has run and been checked +against it, and one pull request then removes the status keys from every report and changes playbook +03 and `cycle.py` together. There is no period in which both are written. + +## 3. Importing and consolidating the 357 issues + +**Import.** `import` reads the folders through the records module (`records_list`, `records_read`), +over the bus, rather than from a checkout. Each folder becomes an issue with its own number, its +frontmatter as fields, its title from the heading, the commit it was read at, and its path as +`record`. Folder-to-folder links (265) become `relates-to` links. The four open pull requests' +issues (338, 341, 350, 363) are reserved as numbers and imported when they merge. + +**Consolidating** means four concrete passes, each producing a list for the operator or an agent to +confirm, never applying itself: + +1. **Close what is fixed.** A `located` or `open` issue whose `located-in` repository has a merged + pull request naming its number is a candidate for `resolved`. hits's auditor (section 6) checks + exactly this against git and is the model. +2. **Merge duplicates.** Candidates are pairs that link each other and share a `located-in`, and + pairs whose titles share most words. A confirmed duplicate is closed `wontfix` with a + `duplicates` link and the reason, keeping its number and folder. +3. **Normalise `located-in`** onto the registered list (`mesh-controller`, `mesh-catalog`, the + node-engine's repository, a module's name), keeping the file paths as a note. +4. **Group the 73 not closed** by domain (the glossary's domains, ADR 0244) and by `located-in`, so + the operator sees, for example, how many open issues the delivery domain carries. + +## 4. Conditions, asks and deliveries + +A condition is one open fact, raised and cleared only by observation (ADR 0231); an issue is a +question someone must answer. They are different things, and a condition must not be closed by +closing an issue. But a condition can say that a fix is owed: five of today's ten are `healer-wanted`, +and one silence says "hq issue to follow". + +Recommended for a later slice (slice 4): an issue links a condition by its key; the controller shows +the link on the condition; and for `healer-wanted` the controller opens one issue per condition key +when it first raises it, linked both ways, instead of an agent remembering to. A silence whose reason +names an owed issue should carry its number. Asks +([ADR 0259](../../02-DECISIONS/0259-an-ask-is-a-message-on-the-bus-any-channel-answers-it-and-the-asker-acts-on-a-warrant.md)) +stay with the router: the tracker asks nothing in v1. Deliveries relate through pull requests: when +the forge says a pull request merged, the tracker notes it on every issue the pull request names, and +offers `resolved` (slice 4). Nothing in v1 depends on these. + +## 5. How the operator sees issues + +First through the mesh's tools: `list` returns rows short enough to read in a chat, `get` one issue +whole, and every agent session reaches them through the mesh MCP server. The responder (`@mesh`, +ADR 0234) answers from read verbs, so "what is open in delivery" works on a channel once the seat +exists. Later a view onto the bus: the `issues` bucket watched over the bus (a browser reaches it through a WebSocket listener, which this effort did not check for), +read-only, which the operator named as a future direction for every module's state. Nothing in v1 +blocks it, because the state is already a bucket and every change an event. + +## 6. hits: what it is, and what to take from it + +**What it is.** A headless issue tracker in Go (module `github.com/impire-io/hits`), with no web +interface and no HTTP interface: NATS micro services on JetStream, a command line for people and a +stdio MCP server for agents. One author, 81 commits between 2026-09-02 and 2026-09-14, release +v0.7.0, about 9,700 lines of Go outside tests and 21 test files, tested against an embedded NATS +server. Its model descends from a method like this repository's: a sibling `hits-hq` with numbered +playbooks and decisions. + +**Data model.** An *item* has a type (`bug`, `task`, `improvement`), a status (`open`, `diagnosing`, +`located`, `blocked`, `resolved`, `wontfix`), a priority, `located-in` (from a registered list of +*projects*), `fixed-by` (pull request, commit or action), `amended-design`, append-only notes, typed +links (`duplicates`, `relates-to`), a claim, a block, and a tombstone for a filing mistake. Its +statuses and fields are this repository's frontmatter almost key for key. Projects, *initiatives* +(groups of projects that also scope the numbering) and *releases* are registered lists that are +never reused once retired. + +**How it stores.** The source of record is a stream `hits-ops` with subjects +`hits.ops.item.` (and `project`, `initiative`, `release`). Every write appends an operation, +published with the per-subject expected last sequence, so two writers to one item cannot both win. +A bucket `hits-state` holds the folded snapshot of each item, the registered lists and one counter +per initiative. The counter is advanced by compare-and-set and an item is numbered only after it is +validated, so a refused create spends no number. On start it rebuilds the bucket from the stream. + +**Services and subjects.** One micro service `hits` with 23 endpoints under `hits.api.*` (`create`, +`get`, `edit`, `transition`, `claim`, `release`, `block`, `unblock`, `link`, `unlink`, `note`, +`tombstone`, and register, list and retire for projects, initiatives and releases); three index +services, each a projection of the stream: full text (`hits.search.query`, a local index on disk), +a graph (`hits.graph.neighbors`, `hits.graph.walk`) and an optional semantic search that needs an +outside embedding provider. + +**MCP tools.** 27, one per endpoint: `create_item`, `get_item`, `edit_item`, `transition_item`, +`claim_item`, `release_item`, `block_item`, `unblock_item`, `link_items`, `unlink_items`, +`note_item`, `tombstone_item`, the project, initiative and release tools, and `search_items`, +`semantic_search`, `graph_neighbors`, `graph_walk`. One server serves one agent under one actor. +Its command line also has an auditor that checks every item's `fixed-by` against git and finds +merged pull requests whose items are still open. + +| | Adopt as it is | Adapt to the mesh | +|---|---|---| +| Statuses, fields, append-only notes, typed links, tombstone | yes: they are already ours | — | +| Stream of operations as the record, bucket as its projection | the pattern | the stream and bucket declared in the manifest and created by the controller (ADR 0201 §3), not created by the service on start | +| Expected-sequence writes; compare-and-set counter; number only after validation | yes | needs compare-and-set in the state interface (section 1) | +| The auditor | the idea, as a tool | reads the forge through the gitea module, not a local clone | +| Subjects `hits.api.*`, its own MCP server | no | the seat's verbs, served by the tool runner and found with `mesh_search` | +| Actor named by the caller | no | the caller is the bus credential the mesh issued | +| Contexts, OAuth device flow, semantic search | no | the mesh issues credentials; no outside provider | +| *item*, *project*, *initiative*, *release* | no | *issue*; a repository or module; no initiatives in v1; *release* is already a homonym here (glossary) | + +**Reuse its code, or build on its model?** Its licence is the Sustainable Use License 1.0: use and +modification are allowed for internal business, non-commercial or personal use, and distribution only +free of charge and for non-commercial purposes. It is not an open-source licence. Running it on a +personal mesh is allowed; vendoring it into the catalogue binds the catalogue to that licence for as +long as the code is there. Its services create their own stream and bucket on start, which ADR 0201 +§3 forbids a module to do; its subjects carry no prefix and assume one tracker per bus account; its +words clash with the glossary. Changing those means a fork of a two-week-old, single-author project. +v1 here is the item store without the indexes: roughly the size of the module that delivers pull +requests. + +**Recommended: build our own module on hits's model,** and say so in the module's README. Do not +depend on or vendor its code. Revisit only if its licence changes, or if its author agrees to a +licence for this use. + +## 7. Open questions for the operator + +1. **Should the tracker hold every issue's number and status, while hq keeps a folder only for design + and governance issues, pointing at the tracker by number (option (a) in section 2)?** + Recommended: yes. It ends the collisions, keeps the public record, and moves the cycle rules into + the write, where they refuse instead of report. +2. **Should we build our own `issues` module on hits's model rather than run or vendor hits?** + Recommended: build our own. hits's licence limits distribution, its services create their own bus + resources (against ADR 0201), and its words clash with ours. Its model, its compare-and-set + numbering and its auditor are worth copying. +3. **Should the tracker also hand out decision and research numbers?** Recommended: yes, in slice 2. + Decision numbers collided three times on 2026-10-07; the same counter mechanism serves any + sequence, and the tracker then reserves the number when a branch starts. +4. **Should the word stay *issue*, with the module named `issues` and *ticket* retired?** + Recommended: yes. *Issue* is the glossary's word for this, and one word per thing (ADR 0244). +5. **Should the controller open an issue by itself for a `healer-wanted` condition, or only link + one an agent or the operator opened?** Recommended: link only, until slice 4 has run for a week; + then open by itself for `healer-wanted`, one issue per condition key. + +## 8. Slices after the first + +| Slice | Delivers | Done when the operator sees | +|---|---|---| +| 1 | The `issues` module: open, get, list, move, note, link, import; compare-and-set in the state interface | the next issue number comes from the tracker and the 73 not closed are listed by one call | +| 2 | Decision and research numbers from the tracker; playbook 03 and `cycle.py` changed; status keys removed from every report | two agents opening at once get two numbers, and no hq pull request is renumbered | +| 3 | Consolidation: the four passes of section 3, and an audit tool for `fixed-by` against the forge | the not-closed list shrinks by what was already fixed or duplicated, each with its reason | +| 4 | Conditions and pull requests linked; `healer-wanted` opens an issue | a condition shows its issue, and a merge notes the issues it names | +| 5 | A read-only view onto the `issues` bucket | the operator reads issues in a browser without an agent |