A module whose tests could not run fails the repository check (issue 385) #231

Merged
mesh-admin merged 1 commits from fix/385-an-untestable-module-fails-the-check into main 2026-10-12 10:48:12 +00:00
Contributor

Fixes issue 385 (a merge check that reports NOT TESTED still passes). Built to the decision noted on the issue on 2026-10-11 (within ADR 0237 and "failure must be loud").

What changes

  • checks/deps (new Go helper, with its own tests, run first by merge-check.sh). For each touched Go module it runs go mod download. If that fails, every requirement from the mesh's own forge (git.novox.be/novox/<repo>[/<dir>]) is taken from that repository's clone beside the check (MESH_CHECK_BESIDE; the build seat already clones mesh-sdk there, mesh-controller besideRefs, issue 449). The ref is the pseudo-version's commit or the tag <dir>/<version>. The helper exports it with git archive and writes a separate go.mod (-modfile) with a replace. The module's own go.mod is not changed and no forge credential is added. If it still cannot be had, the helper says why, naming the module, the version and the commit or tag the clone lacks.
  • merge-check.sh: NOT TESTED: … dependencies cannot be fetched + pass is gone. That module is now FAILED: modules/<m> could not be tested — <why>. The other touched modules still run, and the check then exits 1, naming every module that could not be tested. go vet and go test run with -modfile when the helper gave one.
  • merge-check-typescript.sh: a TypeScript module that depends on a package the toolchain does not hold was NOT CHECKED + pass. It now fails by name. A module with no tests stays NOT TESTED (case 1 of the decision, as issue 311 kept it).

How it was tested

  • checks/deps tests (offline Go, GOPROXY=off, a git repository standing in for the SDK beside): a pinned commit taken from the clone, and the module builds against it; a tag go/v0.1.7; a commit the clone does not hold, with the reason naming it; no clone beside, with the reason naming mesh-sdk; a module that fetches everything, which gets no go.mod of its own.
  • End to end, with the SDK removed from a fresh module cache and GOPROXY=off (the seat after the credential is forgotten), MESH_CHECK_MODULES=mounts (which pins 33917f91ac3c):
    • main: NOT TESTED: modules/mounts …, exit 0 (the bug);
    • this branch, nothing beside: FAILED: modules/mounts could not be tested — … module lookup disabled …, and nothing was cloned beside the check to take mesh-sdk from, exit 1;
    • this branch, the SDK cloned beside: modules/mounts takes its forge dependencies from the clones beside the check, tests pass, exit 0.
  • MESH_GATE=<controller built from main 22a2f73> MESH_CHECK_BESIDE=<sdk clone> MESH_CHECK_MODULES=mounts,mesh-issues sh merge-check.sh passes: words, agent-home, paths, manifests (69 long-running resources without health, as kept), and both modules' tests.
  • The TypeScript part was not run here: no TypeScript toolchain (/app/node_modules) on this machine. The change there is two lines.

Design limits I chose

  • In mesh-catalog, a Go module whose dependencies cannot be had fails even if it has no tests, because its code cannot be vetted or compiled either.
  • A touched TypeScript module that depends on a package the toolchain lacks will now turn the check red. It was green with a note before.
  • The helper is copied into mesh-media-catalog (its PR for this issue). It is check code for two repositories, not module code, so it does not go in the SDK.

Reviewers' checklist

  • Without the fix the end-to-end case above exits 0. With it, it exits 1 and names the module and the reason.
  • No credential is added. Only the clone the seat already makes is read. The module tree is not modified (-modfile in a temp dir).
  • NOT TESTED stays only for a module that has nothing that could fail.
  • Checks green.

Fixes issue 385.

Fixes issue 385 (a merge check that reports NOT TESTED still passes). Built to the decision noted on the issue on 2026-10-11 (within ADR 0237 and "failure must be loud"). ## What changes - **`checks/deps`** (new Go helper, with its own tests, run first by `merge-check.sh`). For each touched Go module it runs `go mod download`. If that fails, every requirement from the mesh's own forge (`git.novox.be/novox/<repo>[/<dir>]`) is taken from that repository's clone beside the check (`MESH_CHECK_BESIDE`; the build seat already clones `mesh-sdk` there, mesh-controller `besideRefs`, issue 449). The ref is the pseudo-version's commit or the tag `<dir>/<version>`. The helper exports it with `git archive` and writes a separate go.mod (`-modfile`) with a `replace`. The module's own go.mod is not changed and no forge credential is added. If it still cannot be had, the helper says why, naming the module, the version and the commit or tag the clone lacks. - **`merge-check.sh`**: `NOT TESTED: … dependencies cannot be fetched` + pass is gone. That module is now `FAILED: modules/<m> could not be tested — <why>`. The other touched modules still run, and the check then exits 1, naming every module that could not be tested. `go vet` and `go test` run with `-modfile` when the helper gave one. - **`merge-check-typescript.sh`**: a TypeScript module that depends on a package the toolchain does not hold was `NOT CHECKED` + pass. It now fails by name. A module with no tests stays `NOT TESTED` (case 1 of the decision, as issue 311 kept it). ## How it was tested - `checks/deps` tests (offline Go, `GOPROXY=off`, a git repository standing in for the SDK beside): a pinned commit taken from the clone, and the module builds against it; a tag `go/v0.1.7`; a commit the clone does not hold, with the reason naming it; no clone beside, with the reason naming `mesh-sdk`; a module that fetches everything, which gets no go.mod of its own. - End to end, with the SDK removed from a fresh module cache and `GOPROXY=off` (the seat after the credential is forgotten), `MESH_CHECK_MODULES=mounts` (which pins `33917f91ac3c`): - main: `NOT TESTED: modules/mounts …`, **exit 0** (the bug); - this branch, nothing beside: `FAILED: modules/mounts could not be tested — … module lookup disabled …, and nothing was cloned beside the check to take mesh-sdk from`, **exit 1**; - this branch, the SDK cloned beside: `modules/mounts takes its forge dependencies from the clones beside the check`, tests pass, **exit 0**. - `MESH_GATE=<controller built from main 22a2f73> MESH_CHECK_BESIDE=<sdk clone> MESH_CHECK_MODULES=mounts,mesh-issues sh merge-check.sh` passes: words, agent-home, paths, manifests (69 long-running resources without health, as kept), and both modules' tests. - The TypeScript part was not run here: no TypeScript toolchain (`/app/node_modules`) on this machine. The change there is two lines. ## Design limits I chose - In mesh-catalog, a Go module whose dependencies cannot be had fails even if it has no tests, because its code cannot be vetted or compiled either. - A touched TypeScript module that depends on a package the toolchain lacks will now turn the check red. It was green with a note before. - The helper is copied into mesh-media-catalog (its PR for this issue). It is check code for two repositories, not module code, so it does not go in the SDK. ## Reviewers' checklist - [ ] Without the fix the end-to-end case above exits 0. With it, it exits 1 and names the module and the reason. - [ ] No credential is added. Only the clone the seat already makes is read. The module tree is not modified (`-modfile` in a temp dir). - [ ] NOT TESTED stays only for a module that has nothing that could fail. - [ ] Checks green. Fixes issue 385.
mesh-admin added 1 commit 2026-10-12 10:04:49 +00:00
Fail the check when a module's tests could not run, and take a pinned SDK from its clone beside (issue 385)
mesh/delivery delivered
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh, merge-check-typescript.sh): main's version judged it; the change's judges the pull r…
mesh/delivery-group group fix/385-an-untestable-module-fails-the-check merged: every member is merged
de8a866189
A module whose dependencies could not be fetched was said NOT TESTED and the
repository check passed, so a merge on it would have landed untested code.
The build seat clones the SDK beside every check already: a forge requirement
the module cache lacks is now taken from that clone through a go.mod of its
own, and a module that still cannot be built fails the check by name and why.
A TypeScript module whose dependencies the toolchain lacks fails the same way.
Author
Contributor

Delivery novox/mesh-catalog@de8a8661890e — delivered since 2026-10-12T10:48:22Z

Delivery plan — builds nothing: the change touches no module of the mesh's graph

Group fix/385-an-untestable-module-fails-the-check, in order: novox/mesh-catalog@de8a866189 → novox/mesh-media-catalog@8441a50e41 → novox/mesh-tools@847055c758

  • composed together: pass — every machine composes with the change as it did without (4 of 4 compose)

Transitions

  • 2026-10-12 10:05 (new) → proposed (announced): novox/mesh-catalog#231's head announced
  • 2026-10-12 10:05 proposed → checked (checked): the verdict names this commit
  • 2026-10-12 10:05 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
  • 2026-10-12 10:48 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
  • 2026-10-12 10:48 published → delivered (done): nothing for a walk to move

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

<!-- mesh-delivery:view --> **Delivery** `novox/mesh-catalog@de8a8661890e` — **delivered** since 2026-10-12T10:48:22Z **Delivery plan** — builds nothing: the change touches no module of the mesh's graph **Group** `fix/385-an-untestable-module-fails-the-check`, in order: novox/mesh-catalog@de8a8661890e → novox/mesh-media-catalog@8441a50e41c2 → novox/mesh-tools@847055c7588a - composed together: pass — every machine composes with the change as it did without (4 of 4 compose) **Transitions** - 2026-10-12 10:05 (new) → proposed (announced): novox/mesh-catalog#231's head announced - 2026-10-12 10:05 proposed → checked (checked): the verdict names this commit - 2026-10-12 10:05 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail - 2026-10-12 10:48 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move - 2026-10-12 10:48 published → delivered (done): nothing for a walk to move The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.
mesh-admin left a comment
Author
Contributor

Verdict: request changes at de8a866189

Independent review for issue 385. The Go side is right and I would approve it alone; the TypeScript side turns four actively worked modules permanently red, with no way for a pull request to make them green.

Blocking: four TypeScript modules can no longer merge. merge-check-typescript.sh now fails a module that depends on a package the TypeScript toolchain does not hold. The toolchain's /app/node_modules is what mesh-tools' node-tools/package.json installs (@novox/mesh-sdk, nats, @types/node, typescript, esbuild, and their own dependencies; the SDK has no runtime dependencies). Checked against every package.json under modules/:

  • mesh-catalog (pg), model-usage (pg), mongodb (mongodb), mssql (mssql) import packages the toolchain lacks (their .ts sources import them). These four had 9 to 26 commits each in the last month.
  • The other 29 TypeScript modules need only what the toolchain holds, so they are unaffected.

Today those four are said NOT CHECKED; after this merges, every pull request that touches or moves one of them fails mesh/repo-check, and nothing in this repository can fix that. That matches the 2026-10-11 decision (a module whose check could not run fails), but the remedy has to land first. Either add pg, mongodb, mssql as devDependencies of mesh-tools' node-tools (the toolchain stage keeps devDependencies; the runtime stage prunes them), delivered before this or in this delivery group, or keep these four on NOT CHECKED with a named follow-up issue until it lands. I recommend the first option.

Checklist

  1. Go: an untestable module fails, naming module, version and cause. I ran it with forge fetches blocked and nothing beside: FAILED: modules/claude-code could not be tested — … git.novox.be/novox/mesh-sdk/go@v0.1.11-0.20261009143344-f047d0a4a970 cannot be taken from the clone beside the check: no clone of mesh-sdk is beside the check, and the run exits 1. The only remaining NOT TESTED is a TypeScript module with no tests (the issue 311 exemption). The NOT CHECKED (manifests without a controller) and NOT RACE-CHECKED lines are not about a module's tests, so they are out of this issue's scope.
  2. The module's go.mod and go.sum are never written: -modfile points at a copy in mktemp -d, the EXIT trap removes it, and git status stayed clean after every run. No credential is added. (The leftover tmp.* holding known/shared is base_lists, which was already there before this change.)
  3. Build seat risk:
    • The Go toolchain (mesh-tools-go, golang:1-bookworm base, Alpine in practice) runs apk add git gcc musl-dev python3, so git is present.
    • mesh-sdk is cloned beside every check: checkRequestFor adds the controller's siblings mesh-lab and mesh-sdk for every repository, as full clones, so MESH_CHECK_BESIDE=<root> holds mesh-sdk/.git with all pushed commits and the go/vX tags.
    • If that sibling clone fails, checks.go skips it silently, and the helper then fails every SDK-pinning module with "no clone of mesh-sdk is beside the check". That failure is loud, but its cause is the setup, not the change.
    • The TypeScript toolchain: see the blocking finding.
  4. checks/deps is the same 255 lines in both repositories (only the module path and one comment differ). That is acceptable as check code. Putting it in mesh-sdk would bring back the forge fetch it exists to avoid. The proper home later is the build seat: the controller's check would set the replace for any repository. Until then it is two copies to keep in step.
  5. Runs (local Go 1.27.1):
    • checks/deps tests pass in both repositories (6 of 6), and go vet and gofmt are clean.
    • merge-check.sh with MESH_CHECK_MODULES=claude-code (pseudo-version pin), forge blocked, mesh-sdk beside: it took the SDK from beside, the tests passed, and it exited 0.
    • With mesh-sdk not beside, claude-code and messenger both FAILED by name and the run exited 1.
    • mesh-knowledge resolved its dependencies, but one of its tests fails on its own: TestTheKnowledgeBaseOverTheBus reports "dated from the day its number was reserved, 2026-10-12, not 2026-10-11". The test depends on today's date, which has nothing to do with this change. I think it fails on main too; it needs its own issue.
    • Checks on this head: merge-gate pass (no module of the graph touched), and delivery and delivery-group success. repo-check passed, but main's script judged it (THE CHANGE ALTERS ITS OWN CHECK). The new script has not yet run on the build seat. Its first real run is the next pull request after the merge, so prove it then with one pull request that touches an SDK-pinning Go module.
Verdict: request changes at de8a8661890ec2d076e16811b823fc3a80022100 Independent review for issue 385. The Go side is right and I would approve it alone; the TypeScript side turns four actively worked modules permanently red, with no way for a pull request to make them green. **Blocking: four TypeScript modules can no longer merge.** `merge-check-typescript.sh` now fails a module that depends on a package the TypeScript toolchain does not hold. The toolchain's `/app/node_modules` is what mesh-tools' `node-tools/package.json` installs (`@novox/mesh-sdk`, `nats`, `@types/node`, `typescript`, `esbuild`, and their own dependencies; the SDK has no runtime dependencies). Checked against every `package.json` under `modules/`: - `mesh-catalog` (pg), `model-usage` (pg), `mongodb` (mongodb), `mssql` (mssql) import packages the toolchain lacks (their `.ts` sources import them). These four had 9 to 26 commits each in the last month. - The other 29 TypeScript modules need only what the toolchain holds, so they are unaffected. Today those four are said NOT CHECKED; after this merges, every pull request that touches or moves one of them fails `mesh/repo-check`, and nothing in this repository can fix that. That matches the 2026-10-11 decision (a module whose check could not run fails), but the remedy has to land first. Either add `pg`, `mongodb`, `mssql` as devDependencies of mesh-tools' `node-tools` (the toolchain stage keeps devDependencies; the runtime stage prunes them), delivered before this or in this delivery group, or keep these four on NOT CHECKED with a named follow-up issue until it lands. I recommend the first option. **Checklist** 1. Go: an untestable module fails, naming module, version and cause. I ran it with forge fetches blocked and nothing beside: `FAILED: modules/claude-code could not be tested — … git.novox.be/novox/mesh-sdk/go@v0.1.11-0.20261009143344-f047d0a4a970 cannot be taken from the clone beside the check: no clone of mesh-sdk is beside the check`, and the run exits 1. The only remaining NOT TESTED is a TypeScript module with no tests (the issue 311 exemption). The NOT CHECKED (manifests without a controller) and NOT RACE-CHECKED lines are not about a module's tests, so they are out of this issue's scope. 2. The module's go.mod and go.sum are never written: `-modfile` points at a copy in `mktemp -d`, the `EXIT` trap removes it, and `git status` stayed clean after every run. No credential is added. (The leftover `tmp.*` holding `known`/`shared` is `base_lists`, which was already there before this change.) 3. Build seat risk: - The Go toolchain (mesh-tools-go, `golang:1-bookworm` base, Alpine in practice) runs `apk add git gcc musl-dev python3`, so git is present. - mesh-sdk is cloned beside every check: `checkRequestFor` adds the controller's siblings mesh-lab and mesh-sdk for every repository, as full clones, so `MESH_CHECK_BESIDE=<root>` holds `mesh-sdk/.git` with all pushed commits and the `go/vX` tags. - If that sibling clone fails, `checks.go` skips it silently, and the helper then fails every SDK-pinning module with "no clone of mesh-sdk is beside the check". That failure is loud, but its cause is the setup, not the change. - The TypeScript toolchain: see the blocking finding. 4. checks/deps is the same 255 lines in both repositories (only the module path and one comment differ). That is acceptable as check code. Putting it in mesh-sdk would bring back the forge fetch it exists to avoid. The proper home later is the build seat: the controller's check would set the replace for any repository. Until then it is two copies to keep in step. 5. Runs (local Go 1.27.1): - `checks/deps` tests pass in both repositories (6 of 6), and `go vet` and `gofmt` are clean. - `merge-check.sh` with `MESH_CHECK_MODULES=claude-code` (pseudo-version pin), forge blocked, mesh-sdk beside: it took the SDK from beside, the tests passed, and it exited 0. - With mesh-sdk not beside, claude-code and messenger both FAILED by name and the run exited 1. - mesh-knowledge resolved its dependencies, but one of its tests fails on its own: `TestTheKnowledgeBaseOverTheBus` reports "dated from the day its number was reserved, 2026-10-12, not 2026-10-11". The test depends on today's date, which has nothing to do with this change. I think it fails on main too; it needs its own issue. - Checks on this head: merge-gate pass (no module of the graph touched), and delivery and delivery-group success. repo-check passed, but **main's script judged it** (THE CHANGE ALTERS ITS OWN CHECK). The new script has not yet run on the build seat. Its first real run is the next pull request after the merge, so prove it then with one pull request that touches an SDK-pinning Go module.
mesh-admin left a comment
Author
Contributor

Verdict: approve at de8a866189, to be merged only after mesh-tools#31 is merged and its build is published

Independent re-review for issue 385. Review 19 found the Go part right and asked for changes only because the TypeScript toolchain lacked pg, mongodb and mssql. mesh-tools#31 (head 847055c) adds them; this head is unchanged.

What I ran

  1. I built mesh-tools#31's toolchain image (the SDK copied from the current toolchain, because the deps stage needs the registry credential; the rest as written). In it, without a network, this head's merge-check-typescript.sh with MESH_CHECK_MODULES=mesh-catalog,model-usage,mongodb,mssql type-checked all four, ran model-usage's tests (6 pass, 1 skipped without TEST_DATABASE_URL) and mssql's (4 pass), said NOT TESTED for mesh-catalog and mongodb (no tests of their own), and exited 0.
  2. In the current toolchain (mesh-tools-build:4348b428, SDK 0.1.13) the same run fails all four by name and exits 1, as review 19 predicted.
  3. All 33 TypeScript modules at once in the new toolchain: 31 pass or are NOT TESTED. nftables and showcase fail to type-check (tools/index.ts: Module "@novox/mesh-sdk/tools" has no exported member 'tool' and provisioner/index.ts: ... no exported member named 'provisioner'). This is not caused by this change: main's script in the current toolchain fails them the same way. Neither module changed since 2026-10-07. It needs its own issue; the first pull request that touches either module will be red until it is fixed.

Checks on this head: merge-gate pass (touches no module of the graph), repo-check pass (THE CHANGE ALTERS ITS OWN CHECK: main's version judged it), delivery ready, delivery-group ready (composed together they pass).

Merge order, as the mesh does it:

  • A merge check runs in the toolchain the mesh holds: checkSpecOf in mesh-builder takes ToolchainsOf(request.Held), and Held is every artifact the mesh built, i.e. from main. A pull request's head toolchain is never used.
  • The group's composed check (mesh/delivery-group) composes the heads for the gate only; CheckSpec.Group says "the repository's own check is not run for a group". So the green group check does not mean this script ran against #31's toolchain.
  • This pull request itself is safe in any order, because main's script judges it. The risk is the window after it: until mesh-tools#31's build on main is published, any mesh-catalog pull request that touches mesh-catalog, model-usage, mongodb or mssql fails mesh/repo-check.
  • So: merge mesh-tools#31 first, wait until its plan says done (the toolchain is built and published; it is sent to no machine), then merge this and mesh-media-catalog#30. #30 does not depend on #31.

The first real run of the new script is the next pull request after the merge; prove it there with one that touches an SDK-pinning Go module and one of the four TypeScript modules.

Verdict: approve at de8a8661890ec2d076e16811b823fc3a80022100, to be merged only after mesh-tools#31 is merged and its build is published Independent re-review for issue 385. Review 19 found the Go part right and asked for changes only because the TypeScript toolchain lacked pg, mongodb and mssql. mesh-tools#31 (head 847055c) adds them; this head is unchanged. **What I ran** 1. I built mesh-tools#31's toolchain image (the SDK copied from the current toolchain, because the deps stage needs the registry credential; the rest as written). In it, without a network, this head's `merge-check-typescript.sh` with `MESH_CHECK_MODULES=mesh-catalog,model-usage,mongodb,mssql` type-checked all four, ran model-usage's tests (6 pass, 1 skipped without TEST_DATABASE_URL) and mssql's (4 pass), said NOT TESTED for mesh-catalog and mongodb (no tests of their own), and exited 0. 2. In the current toolchain (`mesh-tools-build:4348b428`, SDK 0.1.13) the same run fails all four by name and exits 1, as review 19 predicted. 3. All 33 TypeScript modules at once in the new toolchain: 31 pass or are NOT TESTED. **nftables and showcase fail to type-check** (`tools/index.ts: Module "@novox/mesh-sdk/tools" has no exported member 'tool'` and `provisioner/index.ts: ... no exported member named 'provisioner'`). This is not caused by this change: main's script in the current toolchain fails them the same way. Neither module changed since 2026-10-07. It needs its own issue; the first pull request that touches either module will be red until it is fixed. **Checks on this head:** merge-gate pass (touches no module of the graph), repo-check pass (THE CHANGE ALTERS ITS OWN CHECK: main's version judged it), delivery ready, delivery-group ready (composed together they pass). **Merge order, as the mesh does it:** - A merge check runs in the toolchain the mesh holds: `checkSpecOf` in mesh-builder takes `ToolchainsOf(request.Held)`, and Held is every artifact the mesh built, i.e. from main. A pull request's head toolchain is never used. - The group's composed check (`mesh/delivery-group`) composes the heads for the gate only; `CheckSpec.Group` says "the repository's own check is not run for a group". So the green group check does not mean this script ran against #31's toolchain. - This pull request itself is safe in any order, because main's script judges it. The risk is the window after it: until mesh-tools#31's build on main is published, any mesh-catalog pull request that touches mesh-catalog, model-usage, mongodb or mssql fails `mesh/repo-check`. - So: merge mesh-tools#31 first, wait until its plan says done (the toolchain is built and published; it is sent to no machine), then merge this and mesh-media-catalog#30. #30 does not depend on #31. The first real run of the new script is the next pull request after the merge; prove it there with one that touches an SDK-pinning Go module and one of the four TypeScript modules.
mesh-admin merged commit c5abdf35ed into main 2026-10-12 10:48:12 +00:00
mesh-admin deleted branch fix/385-an-untestable-module-fails-the-check 2026-10-12 10:48:12 +00:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: novox/mesh-catalog#231