A controller restart lost every call's outcome, `status` composed the mesh
while its caller waited (18.6s live on 2026-10-06, past the 10s window), a
repair by hand left no trace, and the core's bounds had nothing measured to
be set from.
- calls: kept in the controller's bucket mesh-controller_calls (last 1000 or
14 days, answers bounded to 64 KiB), read by id across a restart; a
controller starting marks a stopped one's running calls abandoned; each
call names its caller from the inbox its answer goes to.
- status: the serving controller composes it at start, after news from a
machine, a build or an acting verb, and every minute; the verb answers the
last composition at once with when and how long it took. Composing resolves
each machine once instead of twice.
- hand-act log in mesh-controller_hand-acts: push (required through the seat),
plans stop/close, broker consumer-reset and the new hand-act record take
--why/--cause/--condition; `hand-acts` lists them and repeated causes;
status counts the week's.
- durations (migration 0066): apply (send to first report), heartbeat gap,
plan tier and build, recorded as heard; `durations` summarises them.
- the controller's seat row takes this binary's definition of its own verbs,
so the console no longer judges calls against an older build's schema.
- the controller is granted its two buckets' subjects.
musl takes the first reply from any listed nameserver, so a public fallback
beside the mesh's resolver answered NXDOMAIN for mesh names in every Alpine
container (hq ADR 0223). The fix is two mesh resolvers and no public one, which
needs mesh-dns-resolver held on two machines: a seat can now be replicated,
each holder recorded by 'seat <name> --add', checkClaims accepts every holder
on record and still refuses a second holder of any other mesh seat, a holder
answers its own requirement, and a roster fact gives each replicated seat's
holders, this machine first, so resolv-conf can list them. Migration 0062 keys
a holding by seat and assignment.
A seat's protocol lives in the store (migration 0047; seeded additively), a served verb carries its
description and schema, holding a mesh seat requires serving its verbs, a node-scoped seat's tool
carries the node, and the control plane serves status, nodes, node, modules, seats, builds, plan,
assign, unassign, push, build and tools on its seat by running the same commands (novox/hq ADR 0132,
ADR 0154, design 33). A grant of * reaches a role's tools; seat:<seat>.<verb> grants one.
`seat <name> --to <node>/<module>` makes one assignment the holder of a seat in
the same write that removes the previous one. The row is new (migration 0039);
without one, the resolver derives the holder as it always did — the sole eligible
assignment, two refused — so nothing changes for a mesh that never hands a seat
over. With one, the recorded assignment holds and any other whose module could
hold the seat is eligible and silent: not refused, not holding. That is what lets
the next holder run beside the current one until the switch (hq design 26, design
28 task 5.3, ADR 0131).
Why: the controller finds its own bus through a seat, and the day that seat was
left with nobody in it — because two eligible holders could not coexist and the
old one's claim was taken away — the control plane looped for two hours while
every service stayed up. A handover that is never empty in between is the fix,
not a workaround for it.
`CanHold` is the one judgement of whether a module may hold a seat — claims it at
its scope, provides what it delivers, against the store's row — shared by
registration and the handover so they cannot drift apart. The holding belongs to
the assignment and goes when it does, so a seat never points at nothing running.
Tests: the resolver with and without a record, on the same and another machine,
under a former name; the store's row replaced not added, refused for an
unassigned target, removed with its assignment; CanHold's four answers and that
they follow the store. Full suite green against a real NATS and store.
Phase 1 made the set data; a rename still broke every reference to the old
name. This adds the stable identity: a seat's canonical name changes and its
old name becomes an alias that resolves to it forever. SeatNamed and the holder
and display matching resolve a name (former or current) to its seat, so a
manifest's claim, a held record, the git-seat lookup and the build machine's
embedded set all go on working unchanged after a rename. seat_alias table
(migration 0035), inventory Aliases/RenameSeat, openInventory loads them, and a
'seat rename <from> <to>' command does the whole thing — one operation, no
rebuild, no re-registration, no freeze. Behaviour-neutral until a seat is
renamed. Validated against postgres.
The seat set was a Go slice compiled into the controller and referenced by
name everywhere, so changing it meant a rebuild and a freeze-prone deploy. It
is now a table: catalogue keeps the shipped set as defaultSeats (the seed and
the fallback) and a loadable working set; inventory adds the seat table
(migration 0034), Seats to read it, and SeedSeats to fill it idempotently
without overwriting an operator's edit; migrate seeds it; openInventory loads
it, and an empty or unreadable table leaves the compiled defaults in force so
it can never brick the control plane's boot.
Behaviour-neutral: the seeded table equals the defaults. Phase 2 (reference by
a stable id so a rename touches no manifest or code, and the builder reads the
set from the mesh) follows.
jochen
jschoubben