Commit Graph
66 Commits
Author SHA1 Message Date
jschoubben 06b58bc83b Indexer: take directories and outside-built registrations from the builder's log, watched from its start, so no build record of the controller's is read (issue 1955, ADR 0571 §1) 2026-10-29 23:48:14 +01:00
jschoubben 2488d839ab Bridge, indexer and view: carry the catalogue's predefined list on its merged message, and keep it and each open pull request's config declarations in the indexer's states, so mesh-config reads them in place of the controller's mirror and pull-declares (issue 1953 D1-D3)
The view's tools read both states. Pins mesh-sdk 2d2aa7b.
2026-10-29 22:29:04 +01:00
jschoubben f4dd334b49 Indexer: stop holding the catalogue's placement, it is the seat mesh-placement's now (issue 1126)
Deletes the placement and candidate writes, the reread tool and its forge reads, the two buckets and the placement-changed event.
D1a and D1b are one commit: the index package, the named reads, the tools and the manifest do not compile apart.
2026-10-28 17:58:09 +01:00
jschoubben 3629bcf057 Read only gitea's 404 status as not-found, so a 502 naming a sha with 404 in it is redelivered, not written backwards (issue 1126, review)
mesh/delivery merged: source.main moved to novox/mesh-delivery@e753009fbc71
mesh/check being carried to 3629bcf057c3 on e753009fbc71 | from f39bc10e3be7
2026-10-28 15:01:51 +01:00
jschoubben 743a3618ea Keep back only a merge older than the one taken in, asked once per message before any write (issue 1126, review)
Asking whether the merge contains the held commit refused forward merges
forever on a diverged or rewritten main or a squash-merged registered head,
failed every later merge when the forge no longer had the held commit, and
cost one forge call per held commit. The question is now the reverse: a merge
is kept back only when it is in the taken merge and differs from it. A merged
message asks it once, before it writes anything, and a late message writes no
module, deletion, placement or taken key. A take-in asks it once for its
module. A 404 is not known to be older and is written, and said; any other
failure to say leaves the message for redelivery.
2026-10-28 14:56:40 +01:00
jschoubben 77d738d973 Never let a merge move a module backwards, and make the forge messages and a module's history readable (issue 1126)
The indexer wrote mesh-catalog's merge 9cdac09a back over the later 69056948,
rolling kernel and mesh-build-image back on the nodes: a merged message's
commit was written whatever the held one was. Before writing or deleting a
module's value, the indexer now asks the forge whether the merge contains the
held commit, keeps the held value when it does not and says so, and leaves the
message for redelivery when the forge cannot say. Each applied merged message
is said in one line with the modules written.

No tool showed what the indexer heard of FORGE nor a modules key's history
with times, so the cause could not be proven: the indexer now records each
FORGE message it handles (sequence, deliveries, times, commit, outcome) in its
state heard, read by indexer_heard, and indexer_module_history reads
modules.<module>'s kept values with their times.

Quick fix (Src order in writeOverlay) rejected: Src is 0 once the source moved
on, as here. Thorough fix chosen: the order is the forge's, on every path that
writes a module (merged message, take-in).
2026-10-28 14:48:13 +01:00
jschoubben 2b801f5bd4 indexer_take_in reads a held module again only when its record is behind the source's head, and the two new tests are rows (issue 1126, review)
mesh/delivery merged: source.main moved to novox/mesh-delivery@31bbf207ef3b
mesh/delivery-group group fix/1422-trunk-unknown merged: every member is merged
mesh/check being carried to 2b801f5bd49a on 31bbf207ef3b | from ae7ec702a5e9, mesh-catalog 9fd95693db54
A head older than the record, or one whose order the forge cannot say, would have written
an older manifest back. The new tests become rows of existing ones: the repository is over
its 500 unit tests.
2026-10-28 11:18:01 +01:00
jschoubben 790653dfde indexer_take_in reads a held module again at the source's head when its record stands behind it (issue 1126)
A merge whose merged message never reached the indexer had no recovery: take-in answered
held for mesh-vault, whose record stayed at 78d7ef7 after mesh-catalog#535.
2026-10-28 11:10:09 +01:00
jschoubben 458e403a04 The indexer's taken key moves only by a merge it applied, never by the gate's source head (issue 1126)
A drained forge stream does not say a merge was applied: the forge bridge was still
retrying mesh-catalog#535's merged message when the indexer started, and the source's
head moved taken.mesh-catalog to 9fd9569, so the view and show said the merge was on the
machines while mesh-vault's record stood at 78d7ef7 (ADR 0556 clause 1).
2026-10-28 11:10:09 +01:00
jschoubben 63582e44f3 Delete a closed pull request's placement candidates even when its closed message was missed, and give indexer_placement a compact summary form (issue 1126) 2026-10-28 09:25:09 +01:00
jschoubben 04bb2d1398 Show placement candidates and each node's answer in indexer_placement, add indexer_placement_reread, and drop a group's candidates when its head changes no placement file, so a missed message or a revert is recoverable by tool (issue 1126, ADR 0559 §3, §7, §8) 2026-10-27 02:18:21 +01:00
jschoubben ad60ba3a46 Write a catalogue pull request's changed placement as placement-candidates <group>.<node> and delete them when it merges or closes, so each placed node composes it in shadow and answers (issue 1126) 2026-10-27 02:13:55 +01:00
jschoubben 0b980203fe Taken keys are filled only from merges the indexer applied or a drained forge stream's heads (issue 1280) 2026-10-27 00:18:23 +01:00
jschoubben 73bf290f58 The indexer fills every repository's taken key from the source's head at start and when the state appears (issue 1280) 2026-10-27 00:14:41 +01:00
jschoubben c12b870104 The indexer binds its taken state when it appears, as issue 1790 asks of every state a delivery adds: an indexer started before the bus holds it still takes merges in and says the key it did not write (issue 1280, ADR 0556 clause 1)
mesh/delivery merged: source.main moved to novox/mesh-delivery@4af36c69a761
mesh/delivery-group group fix/delivery-records-true merged: every member is merged
mesh/check checking c12b870104e8 on bd107d20154c | novox/mesh-delivery#80 in group fix/delivery-records-true
2026-10-26 22:25:39 +01:00
jschoubben 971cdb44e2 Merge remote-tracking branch 'origin/main' into fix/delivery-records-true
# Conflicts:
#	modules/mesh-delivery-indexer/cmd/mesh-delivery-indexer/main.go
#	modules/mesh-delivery-indexer/go.mod
#	modules/mesh-delivery-indexer/go.sum
#	modules/mesh-delivery-indexer/manifest_test.go
#	modules/mesh-delivery-indexer/module.json
#	modules/mesh-delivery-view/cmd/mesh-delivery-view-tools/main.go
#	modules/mesh-delivery-view/go.mod
#	modules/mesh-delivery-view/go.sum
#	modules/mesh-delivery-view/internal/view/read.go
#	modules/mesh-delivery-view/manifest_test.go
#	modules/mesh-delivery-view/module.json
2026-10-26 22:20:45 +01:00
jschoubben 6201f58c20 The indexer says per repository the last merge it took in, so the view knows its modules speak for that merge (issue 1280, ADR 0556 clause 1) 2026-10-26 20:49:00 +01:00
jschoubben 352066dde6 Review of L2: the assigner binds the indexer's placement state when it appears, with no restart (D1); the three new unit tests folded into existing ones, so no count rises (D2); pin mesh-sdk 3968849 (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 352066dde611 on 3dd94f714d6e | novox/mesh-delivery#76 in group feat/1126-nodes-compose
2026-10-26 20:12:00 +01:00
jschoubben 4f75925410 Nodes compose: placement from the catalogue, node records, self wants, own and pair credentials, core boxes and the authority from the vault (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: unit: git.novox.be/novox/mesh-catalog/checks/nobinaries.TestTheCatalogueHoldsNoCompiledProgram: modules/mesh-config/cmd/mesh-config…
One commit for the branch feat/1126-nodes-compose of mesh-delivery: its 16 commits squashed so its history does not carry the 17.8 MB compiled mesh-delivery-assigner a commit added by mistake and a later one removed (issue 1577). The tree equals the former head ff6c038d. The operator allowed this rewrite of the feature branch on 2026-10-26.
2026-10-26 19:30:18 +01:00
jschoubben 907e092abc mesh-delivery-indexer reads unread registered modules again in one background run, so a forge timeout holds no merged message or source revision past its ack wait (issue 1945, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-delivery@b3a4b09f5b6f
mesh/check being carried to 907e092abc45 on b3a4b09f5b6f
The take-in tool's bound now says the take-in is still running and where its answer will be.
2026-10-26 14:11:12 +01:00
jschoubben 07b5f13474 mesh-delivery-indexer reads a registered module again once its repository becomes readable, and takes one in on ask (issue 1945)
A registered module whose repository the forge answered 404 (car-hunter, snake, before the forge's account could read
them) was read again only when the indexer started again or the controller wrote a new build record of it, so the grants
part's composed user list stayed short of their subjects and the take-over was refused. A failed read now keeps the
module as unread, read again on the next source revision and the next merged message (no timer of its own), and the new
tool indexer_take_in (the tools bundle mesh-delivery-indexer-tools, through the indexer's takeins state, as the
builder's builder_ask_again) takes one in now and answers what it stored or why not.
2026-10-26 14:01:34 +01:00
jschoubben 699cbfb7c1 Indexer: a registered module takes its registered ref, not a newer branch's record; no read at an older commit after a failure
mesh/delivery merged: source.main moved to novox/mesh-delivery@ead9ee2d138d
mesh/check checking 699cbfb7c131 on ead9ee2d138d
2026-10-26 12:52:29 +01:00
jschoubben 319d56be7e Indexer: a registered module is read from every repository it was built from, whatever the branch, and at the registered commit when the head is gone 2026-10-26 12:45:35 +01:00
jschoubben 646f83642a Indexer: read a registered module at the source's head when it names one, else at the registered commit
mesh/delivery merged: source.main moved to novox/mesh-delivery@11d3678984af
mesh/check pass 646f83642a01 on 11d3678984af | unit 171 run, 502 kept | integration 181 run, 568 kept | composition passed in 39ms on novox
2026-10-26 11:41:46 +01:00
jschoubben d9bfba1e62 Indexer: read one directory of a registered module's repository, leave a repository the source names to its merges, accept master as a trunk 2026-10-26 11:36:22 +01:00
jschoubben c44b9f91cd Indexer: gofmt 2026-10-26 11:23:27 +01:00
jschoubben 82da252cf0 Indexer: index the manifests of modules registered from their own repositories, so the grants part has no gap for them 2026-10-26 11:23:21 +01:00
jschoubben 8d38c6bac9 Write the source revision a candidate was previewed onto, so the builder merges its preview onto that trunk head
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: integration: mesh-delivery-indexer/cmd/mesh-delivery-indexer.TestIndexerCountsOnlyTheAssignedBuildNodesToolchains: within 5 s of n2…
Candidate.Base was never written non-zero (review round 2): the builder's base lookup would have found no trunk for any
live candidate and built no preview at all. The indexer now writes, whenever it writes a candidate's fingerprints, the gate's
source revision whose trunk heads it read the previews onto; that head of the pull request's repository is the preview's
trunk. No mesh-sdk change: the field and its meaning were there. The builder's test now takes the candidate the indexer's
own test writes (testdata/candidate-written.json, kept equal to it), not a hand-made one.
2026-10-25 21:16:20 +01:00
jschoubben eaf27f36af Write a module's value only when the module changed, so a merge no longer rewrites every module of its repository
The operator, 2026-10-25: only if a module has a new version in commit x is a new value written naming commit x. A merge
wrote every module of the repository at the new commit, fingerprint unchanged (about 186 keys per mesh-catalog merge), and
the controller's take-in took each in again: twice tonight it flooded. Now a merge that gives a module the value it holds
but for the commit writes nothing, and the value keeps the commit its fingerprint first appeared at. The kind: a write
that differs only in where it was computed. The other instance, a source revision re-stamping Src on every module at its
repository's head after any merge anywhere, now stamps Src once. The trunk head stays the gate's source.main.
2026-10-25 21:13:59 +01:00
jschoubben 3e7f852472 Count a build node's toolchain report only once written since its assignment, and keep a computed fingerprint while builders disagree (issue 1940)
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check failed: integration: mesh-delivery-indexer/cmd/mesh-delivery-indexer.TestIndexerCountsOnlyTheAssignedBuildNodesToolchains: within 5 s of n2…
On 2026-10-25 20:56 assigning mesh-build-node to ace again brought ace's report of 2026-10-23 (v26.12.0) back into the
agreement before the module ran there; the indexer blanked every module's fingerprint to "builders disagree" and the
controller's take-in refused modules already computed and built.

- The toolchain gate (indexer, builder, checker; still one file in three) leaves a report out, said once, until its node
  writes it since its current assignment: the assignment's own time, or the revision that opened it.
- The indexer keeps a module's held fingerprint while a language has no agreed toolchain, when its kept inputs with the
  toolchains it was computed with give it again; only a new fingerprint (a merge, a base, an agreed toolchain moved)
  needs the agreement.
2026-10-25 21:03:16 +01:00
jschoubben 32542ebc14 Keep an unreadable manifest's inputs and judge its verdict again at start, so an upgraded SDK clears it (issue 1923)
mesh/delivery delivered
mesh/check pass 32542ebc14c3 on 7a88eb0a5609 | unit 122 run, 366 kept | integration 132 run, 414 kept | composition passed on novox
2026-10-23 22:17:09 +02:00
jschoubben fae4d48e09 Merge pull request 'R4g+R5 (mesh-delivery): builder and checker put asks, the assigner and its composition check, the view's wants and backlog (design 47)' (#27) from feat/r4g into main
Reviewed-on: #27
2026-10-23 17:49:01 +00:00
jschoubben d32ec6b7f8 Say what the builder, the indexer and the watchman do since R4g in their comments: no build-agent is asked, no build record moves a fingerprint, no checks-running bucket is mirrored (issue 1887, review round 2 item 6) 2026-10-23 16:47:36 +02:00
jschoubben 527f87f94c Merge branch 'feat/r4g-asks' into feat/r4g
# Conflicts:
#	modules/mesh-delivery-checker/cmd/mesh-delivery-checker/judge.go
#	modules/mesh-delivery-checker/manifest_test.go
#	modules/mesh-delivery-checker/module.json
2026-10-23 16:20:17 +02:00
jschoubben a4629bc649 Compute every module's fingerprint with no builder input and read no build-agent record for it, so a change of the builder rebuilds nothing (issue 1887 I1, ADR 0530 §1) 2026-10-23 16:14:19 +02:00
jschoubben d5868c178f Take the words of a module the change removes from mesh-sdk in the indexer and the assigner, so the assigner's reading of a removal cannot drift from what the indexer writes (issue 1297, B3b) 2026-10-23 15:44:00 +02:00
jschoubben 1dc9d80cfb Keep a key written ahead of the source head and a key with no path when reconciling (issue 1900) 2026-10-23 14:44:14 +02:00
jschoubben 26bfdbb637 Drop the indexer's modules keys the trunk no longer holds, at each merge and at start (issue 1900) 2026-10-23 14:31:55 +02:00
jschoubben 612658e788 Ask the forge for the objects of a merge's named paths in the indexer, the one reader of a manifest's paths, and narrow the module's tree from the answer (issue 1758, I6 reworked; ADR 0499 §2 §3, ADR 0495 §1, L8)
The manifest invokes gitea.gitea_commit_trees. A forge that cannot say gives the module none with its words.
2026-10-22 18:24:38 +02:00
jschoubben ece328d0cc Give a candidate's module its named paths' objects from the preview, so the builder's fetch for it names them (issue 1758, ADR 0499 §4; mesh-sdk CandidateModule.Paths) 2026-10-22 18:20:37 +02:00
jschoubben b39090aab9 Read manifests with mesh-sdk's ParseManifestForInputs in the indexer and the builder, so claude-code, dunst and xterm are built too (issue 1751)
The indexer's own copy goes. Pins mesh-sdk feat/r4e-sdk e76a5a3 in every module of the group.
2026-10-22 18:20:37 +02:00
jschoubben 23daa2c03c Take a module's tree input from the paths its artifacts name, by mesh-sdk's NamedTree, and keep their objects and each artifact's paths, so a module outside them moves for no change there (issue 1758, I7 and the indexer's I8, ADR 0499 §2 §3)
A merged message or a preview without the objects gives the module none, naming the paths. Every other module keeps its
directory's tree. Module.ArtifactPaths is filled for the checker (the lead's interface).
2026-10-22 18:14:30 +02:00
jschoubben e81ef8bb63 Read a manifest whose config value falls back to a predefined key, so claude-code, dunst and xterm get a new-form fingerprint (issue 1751, ADR 0348 §7)
The indexer holds no published list of predefined keys; each key a value names is taken as the value declares it.
No fingerprint input comes from the list: the gate and the controller still judge the value.
2026-10-22 18:11:21 +02:00
jschoubben af4eaefb14 Read the controller's build records only for the builder's own input and to fill a key's missing directory once, so a key written before paths gets one with no merge (issue 1851 I3, issue 1614)
Both reads go in R4g with build-agent. A path a key names is never overwritten, and no fingerprint changes.
2026-10-22 18:09:56 +02:00
jschoubben 9f7e082688 List a repository's modules from the merged message's manifests and the indexer's own bucket, not the controller's build records, so a new module gets a value and a key with no build asked (issue 1851, I1 I2 I4, ADR 0501 §6, ADR 0453 §6)
I1: a merge writes each module its manifests name, at its directory. I2: a pull request's changed directory, a
preview's touched module and a source revision's modules come from the modules bucket (a module a pull request adds is
found in the preview by its manifest). I4: the value of today's form (fp, none) is no longer written, since it was
computed from the build records; mesh-sdk keeps it readable until R4g. One commit: the three share applyMerged and its
tests. Pins mesh-sdk feat/r4e-sdk 9cb4d59.
2026-10-22 18:08:48 +02:00
jschoubben a2a5167d72 Write a preview of the conflicts for a head that does not merge, and give the candidate's modules the reason; pin mesh-sdk to the R4c-5a head (issue 1445) 2026-10-21 21:25:45 +02:00
jschoubben 4242fb5b0b The indexer gives a candidate's touched module the fingerprint its merge preview gives it, and wakes on the checker's preview.written without a restart; pin mesh-sdk to the R4c-5a head (issues 1445, 1681) 2026-10-21 21:17:09 +02:00
jschoubben 61fea8314f Compute from a build of build-agent only once a toolchain is read, so a restart drops no fingerprint; fixtures as live (issue 1727)
mesh/merge-gate head 61fea8314fe8 merged into trunk 318064ce206d — pass: builds mesh-delivery-indexer → novox; no bus step; every machine composes with the…
mesh/repo-check head 61fea8314fe8 merged into trunk 318064ce206d — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-21 16:45:51 +02:00
jschoubben 5f9ed2b886 Take the builder's Builder input from build-agent's newest trunk build record, not its absent fingerprint, so modules get a new-form fingerprint (issue 1727) 2026-10-21 15:49:33 +02:00
jschoubben 3bd2a065f9 The indexer writes '.' for a root module's path and the toolchains each fingerprint was computed with (issue 1615) 2026-10-21 11:33:49 +02:00