Take-in by the indexer a revision names; a group's merges taken in together (issue 1585, ADR 0633) #153

Merged

mesh-tools merged 19 commits from feat/take-in into main 2026-11-05 00:33:44 +00:00

Conversation 2 Commits 19 Files Changed 88

+4954 -367

mesh-tools commented 2026-11-04 23:34:07 +00:00

Member

Copy Link

Copy Source

Delivery group feat/take-in with mesh-catalog #615. Issue 1585; ADR 0633 (accepted, version 19, the operator's Approve; its insight on the take-in events is version 21).

Why: a merge that changes the indexer was taken in by the indexer that ran before it (ADR 0603 §4), so a group's new manifests reached old node-engines before the new manifest validator could hold them. The registry group waits on this step.

What:

• The indexer takes a revision in whole only when it runs the build the revision names; otherwise it writes only its own entry, manifest record, verdict, graph key and packages and keeps the rest in its state pending (compare-and-set). It emits take-in.pending and take-in.taken as ordinary events, the key in the body. The named indexer takes them in at its start; later merges queue behind them; no timer anywhere.
• The forge bridge sets the group on each member's merged message and publishes forge.group.merged after the last member; the indexer takes the group in as one revision.
• The watchman raises take-in-waits past 60 minutes (a revision) or 30 (a group), seeded at start from indexer_pending.
• Tools: indexer_pending, indexer_take_in_pending, indexer_drop_pending, bridge_groups, bridge_group_end_again; the view names a module waiting for its revision's indexer.
• The checker refuses an indexer change the trunk's manifest validator would refuse or hold.
• Each new body lives with its owner (the operator, 2026-11-04 23:4x): the forge bridge's merged message (Forge) moves whole into modules/mesh-delivery-forge-bridge/forgemsg; the take-in events into modules/mesh-delivery-indexer/takein. mesh-sdk does not change: every module pins its main (5a38a12).

First check (feat/take-in.38623984d6ad, 00:53) failed: the group's manifest validator refused the indexer's own stream take-in. Fixed in c78941e8: the events are ordinary emitted events. Every manifest (171) was run through the group's manifest validator locally before this push; all accepted.

Tests: unit test functions 804 (was 806), integration 71 (unchanged). Run locally: GOWORK=off gofmt, vet, vet -tags integration, go test in every touched module; the indexer's, bridge's and watchman's integration packages against an embedded bus.

Quick fix: none kept the registry group safe. Thorough fix: this step (chosen). Rules bent: development started before the record's Approve; the WIP commit a395b49 keeps its misleading message ("ADR 0633 declined") because rewording it would change every reviewed sha and the gate's pin. Left as a subtask: a unit row that the start goes on when the own build is unknown (T10).

Reviewed-by: mesh-code-reviewer@shanks at c78941e80b

https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

Delivery group `feat/take-in` with mesh-catalog #615. Issue 1585; ADR 0633 (accepted, version 19, the operator's Approve; its insight on the take-in events is version 21). Why: a merge that changes the indexer was taken in by the indexer that ran before it (ADR 0603 §4), so a group's new manifests reached old node-engines before the new manifest validator could hold them. The registry group waits on this step. What: - The indexer takes a revision in whole only when it runs the build the revision names; otherwise it writes only its own entry, manifest record, verdict, graph key and packages and keeps the rest in its state `pending` (compare-and-set). It emits `take-in.pending` and `take-in.taken` as ordinary events, the key in the body. The named indexer takes them in at its start; later merges queue behind them; no timer anywhere. - The forge bridge sets the group on each member's merged message and publishes `forge.group.merged` after the last member; the indexer takes the group in as one revision. - The watchman raises `take-in-waits` past 60 minutes (a revision) or 30 (a group), seeded at start from `indexer_pending`. - Tools: `indexer_pending`, `indexer_take_in_pending`, `indexer_drop_pending`, `bridge_groups`, `bridge_group_end_again`; the view names a module waiting for its revision's indexer. - The checker refuses an indexer change the trunk's manifest validator would refuse or hold. - Each new body lives with its owner (the operator, 2026-11-04 23:4x): the forge bridge's merged message (`Forge`) moves whole into `modules/mesh-delivery-forge-bridge/forgemsg`; the take-in events into `modules/mesh-delivery-indexer/takein`. mesh-sdk does not change: every module pins its main (5a38a12). First check (feat/take-in.38623984d6ad, 00:53) failed: the group's manifest validator refused the indexer's own stream `take-in`. Fixed in c78941e8: the events are ordinary emitted events. Every manifest (171) was run through the group's manifest validator locally before this push; all accepted. Tests: unit test functions 804 (was 806), integration 71 (unchanged). Run locally: GOWORK=off gofmt, vet, vet -tags integration, go test in every touched module; the indexer's, bridge's and watchman's integration packages against an embedded bus. Quick fix: none kept the registry group safe. Thorough fix: this step (chosen). Rules bent: development started before the record's Approve; the WIP commit a395b49 keeps its misleading message ("ADR 0633 declined") because rewording it would change every reviewed sha and the gate's pin. Left as a subtask: a unit row that the start goes on when the own build is unknown (T10). Reviewed-by: mesh-code-reviewer@shanks at c78941e80b59f6a0d727cee43abfad1530df9881 https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

mesh-tools added 18 commits 2026-11-04 23:34:07 +00:00

Checker: judge the new indexer's manifest with the trunk's judge, and fail a held verdict (issue 1585, ADR 0633 clause 6) ... 2cb3198049

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

Watchman: hear the indexer's take-in stream and raise take-in-waits past 60 minutes (revision) or 30 (group) (issue 1585, ADR 0633 §5) ... 0579e4e74d

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

Checker test: the accepted-indexer row asserts the trunk's judge said so (issue 1585, T22) ... 72633722c5

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

indexer: a source revision is taken in only by the indexer it names (ADR 0633 §1-§5, issue 1585) ... 071a28f218

A merged message is kept in the indexer's own state pending and the merges
there are computed together on overlays of the indexer's states: taken in
whole when the indexer's fingerprint is the running build, otherwise only
the indexer's own entry, graph key and verdict are written and the rest
waits for the named indexer. A group's members are gathered until its end.
Waits are said on the stream take-in, read by indexer_pending and steered
by indexer_take_in_pending and indexer_drop_pending.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

indexer: hear a group's end, keep merges pending, take in at start (ADR 0633 §2, §4, issue 1585) ... 531d05ab4b

The consumer tells a group's end apart by its name before the forge decode,
a merged message goes through the pending state, and the pending revisions
naming this build are taken in before the consumer starts; take-in events go
on the indexer's own stream. Until the pending state is on the bus every
merge is taken in at once, as before.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

View: view_tiers names a module the indexer holds back, read through indexer_pending (issue 1585, ADR 0633 §5) ... d11a5af113

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

Watchman: a replayed pending that its taken ends is never raised; a drop then a pending again is covered (issue 1585, ADR 0633 §5) ... 7609f2d9a2

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

Watchman: a take-in wait past its bound heard in the replay is said once the consumer has caught up, not after a time (issue 1585, ADR 0633 §5) ... ed4f07b0d8

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

forge bridge: a member's merged message names its group, and the group's end is said once (ADR 0633 §4, issue 1585) ... d6b6f24e98

The group is read from the gate's merges state the bridge already reads;
the end comes only once every approved member is merged and its merged
message written, never on a refusal, with a message id the stream's
duplicate window holds to one.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

forge bridge: a member's merged message names its group, and the group's end is said once (ADR 0633 §4, issue 1585) ... a762701e3d

The group is read from the gate's merges state the bridge already reads;
the end comes only once every approved member is merged and its merged
message written, never on a refusal, with a message id the stream's
duplicate window holds to one.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

Forge bridge: bridge_groups and bridge_group_end_again, the end said again through the state end-asks (issue 1585, ADR 0633 §4) ... ad9a353484

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

indexer: before the pending state is on the bus, take a merge in at once and start without asking the own build (ADR 0633, issue 1585) ... 9c6e70e555

TestIndexerWhole found the start waiting for ever on a build no node-engine
answered, so the forge consumer never started, though nothing can be pending.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

indexer: TestIndexerWhole runs the take-in step whole, a restart and a group (ADR 0633, issue 1585 I8a, I8b) ... ef13e63268

A revision changing the indexer writes only what the named build needs and is
taken in when that build starts; a gathered group stays across the restart and
its end makes one take-in.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

Merge commit 'ef13e63' into feat/take-in 92dc5f03d0

Forge bridge: the process takes the keys of performed from internal/groups, one copy (issue 1585) ... 1040c8ec46

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

WIP (ADR 0633 declined): watchman pinned to the group's mesh-sdk head; the other pins, the gate's const swap and the full checks not done ... a395b49be2

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

Take-in and forge contracts live with their owners: the indexer's takein package, the forge bridge's forgemsg package; mesh-sdk pinned at main (issue 1585, ADR 0633) ... 4e4601a1eb

The operator, 2026-11-04: logic one module owns lives inside that module as a package of its own, so the take-in step
changes no mesh-sdk. TakeIn, its keys and event names, PendingWait and PendingAnswer move to
modules/mesh-delivery-indexer/takein (its merges a takein.Merge of its own); Forge whole, with its group fields,
ForgeGroupEnd and GroupMerge, moves to modules/mesh-delivery-forge-bridge/forgemsg. Both are nested Go modules that their
readers take by replace, as buildrun is. Every importer moves in this commit: the indexer, the bridge, the watchman and
the view. Forge's private checks (the repository name, the version) are copied small; mesh-sdk main still holds an unused
Forge until its own fold removes it.

The indexer, bridge, watchman, view and checker pin mesh-sdk 5a38a12 (origin/main), no feat/take-in commit.

The bridge's integration bus now grants the state end-asks that bridge_group_end_again added: without it every test of
the package failed on a permissions violation. Test budget: two new test functions (TestForge, TestTakeInEvents) and the
two the join added are paid by merging four pairs in internal/bridge; the repository stays at 806 unit tests.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

Take-in, review round 1: no claim left behind, no sleep at the start, waits seeded at a restart, ends asked readable (issue 1585, ADR 0633) ... 7b7096f37e

- A take-in releases its claims (compare-and-set, only while the claim is still its own) on a lost claim, on a claim it
  cannot read back, and on a merge it fails to take in: a claim left behind refused every later take-in until a
  restart. T5 row with a store that fails once, both ways.
- The start step waits for nothing: with the build it runs unknown it says so and the forge consumer starts; a merged
  message left unacknowledged asks the build again (clause 1). T10 row: the start answers at once and takes nothing in.
- A toolchain report evaluates pending again (ToolchainReported), since the build a revision names moves with the
  toolchain. T7 row: the revision is taken in once the report makes it name the running build.
- The group's end is told by its whole subject (process.ModuleStreamSubject), so a repository named forge/group is
  never taken for one; row in the indexer's handle test.
- The watchman asks the indexer's waits through indexer_pending at its start (invokes), before its durable consumer,
  which does not hear again what it acknowledged before a restart; the comment that said the whole stream is heard is
  corrected. W2 row: a wait seeded within its bound is raised past it.
- bridge_groups names each group's last end ask, its who, why and when, and the bridge's answer (state end-asks).
- The check's words for the indexer's rules: the trunk's manifest validator.
- The bridge's end drops the merged-state test its merge commit already implies.
- Test budget: groupask_test folded into groups_test, and the K1 mesh/check test into the owed status one: 804 unit
  tests (was 806).

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

mesh-tools commented 2026-11-04 23:34:15 +00:00

Author

Member

Copy Link

Copy Source

Delivery novox/mesh-delivery@c78941e80b59 — merged since 2026-11-05T00:33:46Z

Source — source.main moved to novox/mesh-delivery@4596fbe1b7. Whether it reached the machines is mesh-delivery-view's, derived from source.main, the wants and the haves.

Group feat/take-in: novox/mesh-catalog@5ceb6e9e38, novox/mesh-delivery@c78941e80b

• composed together: pass — the checker's verdict at the members' heads (ebd0b6ae80af)

Transitions

• 2026-11-05 00:03 (new) → proposed (announced): novox/mesh-delivery#153's head announced
• 2026-11-05 00:03 proposed → ready (accepted): a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail; or the check gates no merge (check-gates-merge false)
• 2026-11-05 00:33 ready → merged (merged): on the trunk its modules follow: merged there

The commit's note under refs/notes/mesh-plan keeps every transition of this change up to its merge: git log --notes=mesh-plan.

<!-- mesh-delivery:view --> **Delivery** `novox/mesh-delivery@c78941e80b59` — **merged** since 2026-11-05T00:33:46Z **Source** — source.main moved to novox/mesh-delivery@4596fbe1b7ac. Whether it reached the machines is `mesh-delivery-view`'s, derived from `source.main`, the wants and the haves. **Group** `feat/take-in`: novox/mesh-catalog@5ceb6e9e38a4, novox/mesh-delivery@c78941e80b59 - composed together: pass — the checker's verdict at the members' heads (ebd0b6ae80af) **Transitions** - 2026-11-05 00:03 (new) → proposed (announced): novox/mesh-delivery#153's head announced - 2026-11-05 00:03 proposed → ready (accepted): a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail; or the check gates no merge (check-gates-merge false) - 2026-11-05 00:33 ready → merged (merged): on the trunk its modules follow: merged there The commit's note under `refs/notes/mesh-plan` keeps every transition of this change up to its merge: `git log --notes=mesh-plan`.

jschoubben added 1 commit 2026-11-05 00:03:02 +00:00

Take-in events are the indexer's ordinary events: no stream of its own, which the manifest validator refuses (issue 1585, ADR 0633 §5) ... c78941e80b

The group's check failed at 7b7096f3: the group's own manifest validator refuses the indexer's stream "take-in", outside
the bus's predefined forms. The lead's ruling: the indexer says take-in.pending and take-in.taken as its ordinary events
(its emits, mesh.mod.<module>.event.take-in.<kind>), as manifest-accepted is said, and keeps no stream. The wait's key is
in the body only: an emitted event names one event (no wildcard), and a key such as g.<group> is no event name. The
watchman consumes the two events by name on the shared events stream; it needs no replay, since it seeds its waits from
indexer_pending at its start. indexer_pending says the last event of each wait as take-in.<kind>.

Run locally, as the check runs it: cmd/mesh-judge over every manifest of mesh-catalog, mesh-delivery, mesh-host,
mesh-tools, mesh-networking and mesh-media-catalog, with this branch's and mesh-catalog's feat/take-in manifests in the
change case and origin/main's in the trunk case: 171 of 171 accepted in both. The same run with the indexer's manifest at
7b7096f3 refuses it with the check's words.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn

mesh-tools commented 2026-11-05 00:03:10 +00:00

Author

Member

Copy Link

Copy Source

mesh-delivery: the operator's Approve of this pull request's merge into main is ended: its head 7b7096f37e is no longer the pull request's head, or the pull request closed. Nothing is merged on it. The operator is asked again once the checks pass at the new head.

mesh-delivery: the operator's Approve of this pull request's merge into main is ended: its head 7b7096f37ea7 is no longer the pull request's head, or the pull request closed. Nothing is merged on it. The operator is asked again once the checks pass at the new head.

mesh-tools merged commit 4596fbe1b7 into main 2026-11-05 00:33:44 +00:00

mesh-tools deleted branch feat/take-in 2026-11-05 00:33:45 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

agent-ace agent-g14 agent-novox agent-shanks jschoubben mesh-tools miel.truyen nox-agent

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-delivery#153