The operator approved measuring from the delivery's own events whether each fix an issue names reached the machines; delivered is not resolved, and a merged fix that does not arrive is said on the issue and the operator's channel instead of found.
mesh-issues
The issue tracker: the mesh numbers and tracks its issues in a module of its own (novox/hq ADR 0275,
to-be 53). An issue is a question about the mesh that someone must answer; this module holds the
mesh's issues and hands out their numbers. It holds the seat issue-tracker, once for the mesh, on the
control-node.
Built on the model of hits, a headless issue tracker on NATS, studied at release v0.7.0: its statuses, fields, append-only notes, typed links and compare-and-set numbering are taken as ideas. It carries none of its code, under a licence that is not an open-source one.
Verbs
Called as issue-tracker.<verb> through the mesh's tools; each says in its description what it does and
what it refuses.
| verb | takes | does |
|---|---|---|
open |
title, report, optionally located-in, record |
checks the issue, then takes the next number and keeps it under that number; answers the number |
get |
number |
the issue whole: fields, notes, links |
list |
optionally status, owner, phrase, limit |
short rows, newest first: number, status, owner, title |
move |
number, status, and what that status needs |
changes the status under the development cycle's rules |
note |
number, text |
appends a note; a note is never edited |
link |
number, kind, target, optionally remove |
adds or removes a typed link |
import |
optionally dry-run, commit |
hq's issue folders, read through the records module, each written as an issue under its own number; see below |
set |
number, any of severity, needs, suggested-owner |
changes them as a person decides |
groom |
optionally pass, confirm |
the triage over the backlog: without arguments a dry run in the background, answering its pass id; with pass its progress and findings; with confirm and pass exactly that pass's findings applied, each checked again; see below |
The cycle's rules, at the write (hq playbook 03, which 00-META/checks/cycle.py enforced on the
folders until the import): a status is one of open,
diagnosing, located, resolved, wontfix; located needs located-in; resolved needs
located-in and fixed-by; a core issue (located in the controller, the node-engine, the tool runner,
the SDK, or the catalogue's bus, forge or build agent) resolves with replay or replay-none; wontfix
needs a reason. A refused change changes nothing and says which rule refused it.
An issue with a folder in hq (record at open: the folder's name, without the number) is whole in the
tracker like any other: the folder holds its narrative, the tracker its status. Until the import of
slice 3 ran, such an issue kept its status in the folder and move refused it; the import filled it.
The triage (hq ADR 0278, ADR 0279)
Every issue opened is triaged by the module itself: it consumes its own mesh-issues.opened event, and for an
issue still at open asks the agent on its node — claude-code.claude_code_judge@<node>, which runs the agent
with no tools under the node's licence — two questions: a shortlist of at most five older issues by their titles,
then, always, a judgement on the reports: whether it repeats one (duplicate, related or none, whether it is
certain, and why), its severity (critical, major, minor), what it needs (none, decision, operator), a
suggested owner when it names none, and the shortlisted issues it depends on or relates to, each with a
one-line reason. Only issues at open, diagnosing or located that are older than the new one are offered.
The judgement is kept on the issue (triage) and said as the event triaged before anything else, then applied
through the module's own note, set, link and move, each recorded as by triage and each only when not
done yet:
- one note saying the whole judgement, a line per value with its reason; where the issuer gave a severity or needs the triage disagrees with, the line says so and the given value stands;
- each empty value filled — never one a person gave;
- each
depends-onandrelates-tolink added, unless it is there, would close a cycle, or its target is resolved or wontfix; the older issue gains a note; - a certain duplicate: linked
duplicates, its title and report noted on the older, moved towontfix"duplicate of N" last. The older issue never moves.
A second delivery of the event finds the kept judgement and finishes what is missing without judging again. A
judgement that fails leaves one note, and the event is not taken: after its fifth delivery the controller raises
max-deliveries on the module's consumer. Notes the duplicate check of ADR 0278 wrote as duplicate-check are
still read as that issue's verdict.
Ready. An issue is ready when it is not resolved or wontfix and every issue it depends on is wontfix, or
resolved with every pull request its fixed-by names (<repository>#<number>) delivered. The module consumes
mesh-delivery.transition and marks a pull request delivered on every issue naming it; at its own moved to
resolved it asks seat:mesh-delivery.checks once for what it has not heard. An issue made ready gains a note and
the event ready. get answers waits-for, needed-by and ready; list filters on severity, needs and
ready — an agent looking for work asks severity=minor, needs=none, ready=true. set changes severity,
needs and suggested owner as a person decides. link refuses a depends-on that would close a cycle.
groom triages every issue not resolved and not yet triaged, oldest first, against the older ones not resolved
and not found a duplicate in the same pass. It writes only the pass, in the state grooming (the last pass, one
key), each finding saying what applying it would do, and applies exactly that pass on confirm, each issue
checked again. One judgement runs at a time; the pass waits while a new issue's triage waits. A triage may take up
to two minutes, which the tool runner allows a handler while keeping its event alive (hq issue 387).
The new values are optional keys of an issue's one JSON value: an issue kept before reads as having none; the key-value state has no schema to migrate.
Whether the fix is delivered (hq ADR 0281)
For each pull request an issue's fixed-by names, the module keeps what mesh-delivery.transition last said —
state, why, merged and when — or asks seat:mesh-delivery.checks once (at its own moved) for one it has not
heard of. Nothing sets it by hand. get and every list row carry delivery, one line derived from it, worst
first: "fix not delivered: …", "held: …", "fix merged, not delivered: …", "fix not merged: …", "nothing heard from
the delivery of …", "delivered to every machine it should reach" (covered by a later merge says so).
Delivered is not resolved. When every fix of an issue not yet resolved is delivered, the issue gains a note and the
event ready-to-verify, once per set of fixes; it is resolved once its acceptance is proven live. A fix is named
while the issue stays located: move to the status it has changes what it holds and is refused when it changes
nothing.
Every five minutes the module looks for a fix merged and not delivered after stuck-after (default 2h): it is
said once — a note on the issue and a warning on the operator's channel (seat:operator-channel.notify, key
issue-tracker.fix-stuck.<number>), cleared there when every fix of the issue is delivered. A module that runs only
its bundle cannot raise a controller condition (hq ADR 0240 rule 8, ADR 0256), so the warning is the channel's, not the controller's.
The import of hq's issues (slice 3)
issue-tracker.import reads every 04-ISSUES/NNN-slug/00-report.md of hq through the records module
over the bus (records_status, records_list, records_read, called as this module through the SDK's
Ask), at the commit the records module's checkout stands at, and writes each as an issue at its own
number: record is the folder, the title is the report's heading, the report is the text under it, and
status, opened, located-in, fixed-by, amended-design, replay and replay-none are the
frontmatter's, as written — the status one to one, no cycle rule applied, because the folder's status is
the snapshot hq kept. The issues and decision records the report names (issue 155, ADR 0275,
04-ISSUES/203-…, 02-DECISIONS/0019-…) become relates-to and decision links.
It is a verb on the seat, callable by any agent, because it can do only this:
- It only creates an issue that is not there (
createon the key, refused when the key exists) or fills the status of one the tracker holds without a status — an issue opened with a folder before this slice, whose folder kept its status; one whose folder hq does not hold at the commit becomesopen. - It never overwrites. An issue the tracker holds differently is reported as
differs, naming the fields, and left alone: the tracker is the truth. An issue it holds the same issame. So a second run changes nothing and says so. - It never touches the counter, and refuses the whole run when a folder is numbered past the counter.
A folder numbered at or past the setting
first-numberwithout an issue in the tracker is refused on its own: a folder from the tracker's time carries a number the tracker gave. - It cannot forge an issue's history. An issue it writes is
opened-by: import, itsopenedthe date the report gave, and it carriesimported: {at, by, commit, from}— when, who ran the import (the bus credential,unprovenuntil the runtime proves it), the commit of hq and the document. Each is animportedevent. What it writes comes from hq'smainas the records module holds it, which is reviewed in pull requests. - What it keeps is bounded as every verb's text is: the title 200 bytes, the report 64 KiB, one issue
256 KiB; a report that trips a credential's shape is refused. An entry of
located-inorfixed-byhq wrote is prose at times, so it is taken within a reason's bound (2000 bytes), not a line's. - An owner hq named that the registered list does not know is kept as written and marked, in
located-in-unverifiedon the issue andunverified-ownersin the answer, never refused: the list is the setting's problem (hq issue 366), and amovethat giveslocated-inanew checks every entry and clears the mark. - With
dry-runit reads everything, writes nothing, and answers the same rows withwould-on each action. Withcommitit refuses a checkout standing elsewhere; it refuses a run during which the checkout moved.
The answer is a row per folder — number, action (create, fill, same, differs, refused, with why),
status, title — the counts, the counts by status, what was odd (a title numbered otherwise than its
folder, a frontmatter key not imported, an amended-design that is no design document) and the unverified
owners. A call that outlives the tool runner's wait is still running: the import goes on, and the answer
is read back through the controller's calls verb by the call's id.
Before the module is rolled out, mesh-issues dry-run <hq checkout> <settings.json> at the terminal runs
the same dry run over a checkout on disk against an empty tracker, and prints the answer. It is not a way
to import: only the verb writes, and it reads the record over the bus.
Every argument is read strictly. One a verb does not take is refused, never dropped. Every field has a
size limit, a one-line field takes no control character, a folder's name is no path, and text shaped like
a credential (a private key, an API key, a forge or bus token, password=…) is refused: an issue is read by
every agent on the mesh. The shapes catch the ordinary mistake, not every secret.
Numbers
One counter, the key issue of the state counters, advanced only by compare-and-set: the SDK's
State(...).Increment reads it and writes the next number only at the revision it read, so two openers —
in one instance or two — never take one number. A write another opener won is read and tried again, a
bounded number of times with a short random wait; past the bound the open is refused and takes no number.
- A refused open takes no number: an issue is checked before its number is taken.
- A number is never handed out twice, and may be skipped. A number taken and then not kept — the bus
failing between the counter and the issue — stays unused, and the answer says which. A write whose answer
was lost is recognised as its own when tried again, so it leaves no gap; when it cannot be confirmed either
way, the answer says the issue may have been written and to
getthat number before opening it again, never that nothing was opened. - An issue is never overwritten. It is kept by a write that is refused when the key exists; if the counter ever hands out a number already kept, the open is refused loudly, the number stays unused, and nothing is overwritten.
- The counter starts once, the first time a number is taken: at the setting
first-number, or after the highest issue already kept, whichever is higher, made only if it does not exist. The setting is never read again, so a number never moves.
State and events
Two states, declared in the manifest and made by the controller; the module never makes a bucket
(ADR 0201): issues, one key per issue by number, keeping 8 values per key; counters, one value per key.
Neither expires. One issue is kept in at most 256 KiB; a note that would pass it is refused.
Every change is an event: mesh-issues.opened, .moved, .noted, .linked, .imported, each with the
number, what changed, who changed it and when. The events are the audit trail. A change whose event the bus did not take
is kept, and its answer says the event was not said (announced: false).
Who acted
ADR 0275 §5 records the caller from the bus credential, never from an argument. The runtime does not yet
tell a module who called it: a tool call reaches a bundle as a name and arguments, and the tool runner is
not told who called it either (hq issue 340). Until it does, every change is recorded as by unproven, no
verb takes an argument naming a caller, and none can be given one.
Settings
duplicate-check (on; off stops the triage of new issues), judge-model (sonnet; haiku or opus) and
forge-owner (the forge's owner of the repositories fixed-by names, for asking a delivery) steer the triage;
stuck-after (2h) is how long a merged fix may take to be delivered before it is said.
{
"first-number": 365,
"owners": ["hq", "mesh-catalog", "mesh-catalog modules/nats", "mesh-controller", …]
}
first-number— the number the first issue gets: the one after the highest issue number on hq's main and every open branch at the moment the module is switched on. Read once for the counter; the import reads it as the boundary between the folders hq numbered before the tracker and the numbers the tracker gave.owners— the registered listlocated-inmay name: repositories, and modules asmesh-catalog modules/<module>. An entry oflocated-inis an owner, or an owner followed by a space or a slash and where in it (mesh-controller internal/broker,mesh-catalog modules/nats/module.json). A module must itself be registered:mesh-catalog modules/<module>is never taken as a place inmesh-catalog. It starts with the mesh's repositories and the modules the catalogue held when this module was written; a module added since is added here through the controller's settings verb.
Tests
go test ./... in this directory. The acceptance — an agent opens an issue and gets the next number; two
agents opening at the same moment never get the same one — is
TestFiftyOpensAtOnceFromTwoInstancesGetFiftyDistinctConsecutiveNumbers: a JetStream server linked into
the test, two instances of this bundle (the test binary started again) speaking to a runtime played by the
test against the buckets the manifest declares, fifty opens at once, fifty distinct numbers from the first
in a row. It counts the writes lost to another opener, so it fails when the openers never collided. With
the counter written without its revision check, it fails.
The import is tested over testdata/hq, copies of seven real reports of hq chosen for their shapes (a
bracketed list with commas inside parentheses, quoted block items, an apostrophe in prose, a resolved:
date, a replay, an empty list, a wontfix): the frontmatter read as written, each issue created at its own
number with the counter untouched, a second run changing nothing, an issue the tracker holds differently
left alone, a folder-backed issue filled once, a folder past the counter or past the first number refused,
a dry run writing nothing, and a report that cannot be read refused alone with why.