lab: read the bound forge credential through the tools bundle's env, and check in a fresh directory each time (issue 624) #274

2026-10-13T12:40:27Z

mesh-admin commented

2026-10-13 12:40:27 +00:00

Issue 624. This PR holds the whole fix now; mesh-controller #302 is closed per review 198.

1. The credential reaches the lab's tools. shanks' declaration binds the lab (mesh_shanks_lab) and seals its credential to /var/lib/lab/package-registry.secret, root's at 0600. The lab's tools run in the node runtime as the operator's account, could not read it, and so cloned with no credential: "could not read Username". The controller gives the runtime's account exactly the files a tools bundle names in its env (ADR 0192), as gitea, icecast, mailu, redis and umami get theirs. The lab's code bundle now names MESH_LAB_FORGE_SECRET=${dir:state}/package-registry.secret and MESH_LAB_FORGE_BINDING=${dir:state}/package-registry.json. index.ts already reads those env values before the env-file's. No other credential changes hands.

2. lab_check in a fresh directory. The check cloned into one fixed path, <work>/check, after an rm -rf whose result it ignored, and failed on "destination path already exists". I think an earlier check left a tree there that the operator account could not remove; I could not stat it on shanks, because node-reads runs only on novox. Each check now clones into a fresh check-* directory (checkDir), runs its setup check under it, and removes it when done. clearChecks first removes what earlier checks left and names any path it cannot remove.

Tests (modules/lab/test/runs.test.ts):

As the operator account, 18 pass. As root (node:23 container), 17 pass and 1 is skipped: the locked-tree test skips as root before it makes anything, which is what reddened repo-check (review 199).
New: fresh check directories; leftovers cleared by any account; an unremovable leftover named; the tools bundle's env naming the same secret and binding files the manifest declares.
Strict tsc passes on runs.ts, its test, and index.ts against a stub of the SDK's tool types.

Issue 624. This PR holds the whole fix now; mesh-controller #302 is closed per review 198. **1. The credential reaches the lab's tools.** shanks' declaration binds the lab (`mesh_shanks_lab`) and seals its credential to /var/lib/lab/package-registry.secret, root's at 0600. The lab's tools run in the node runtime as the operator's account, could not read it, and so cloned with no credential: "could not read Username". The controller gives the runtime's account exactly the files a tools bundle names in its env (ADR 0192), as gitea, icecast, mailu, redis and umami get theirs. The lab's `code` bundle now names `MESH_LAB_FORGE_SECRET=${dir:state}/package-registry.secret` and `MESH_LAB_FORGE_BINDING=${dir:state}/package-registry.json`. index.ts already reads those env values before the env-file's. No other credential changes hands. **2. lab_check in a fresh directory.** The check cloned into one fixed path, `<work>/check`, after an `rm -rf` whose result it ignored, and failed on "destination path already exists". I think an earlier check left a tree there that the operator account could not remove; I could not stat it on shanks, because node-reads runs only on novox. Each check now clones into a fresh `check-*` directory (`checkDir`), runs its setup check under it, and removes it when done. `clearChecks` first removes what earlier checks left and names any path it cannot remove. **Tests** (modules/lab/test/runs.test.ts): - As the operator account, 18 pass. As root (node:23 container), 17 pass and 1 is skipped: the locked-tree test skips as root before it makes anything, which is what reddened repo-check (review 199). - New: fresh check directories; leftovers cleared by any account; an unremovable leftover named; the tools bundle's env naming the same secret and binding files the manifest declares. - Strict tsc passes on runs.ts, its test, and index.ts against a stub of the SDK's tool types.

mesh-admin added 1 commit 2026-10-13 12:40:27 +00:00

lab_check: work in a fresh directory each time, and name what an earlier check left (hq issue 624)

mesh/merge-gate head 5bd40fa4061d merged into trunk 0d723484a84a — pass: builds lab → shanks; no bus step; every machine composes with the change as it did…

Details

mesh/repo-check head 5bd40fa4061d merged into trunk 0d723484a84a — fail: merge-check.sh (go) passed; its merge-check-typescript.sh failed: TypeScript modul…

Details

mesh/delivery superseded: a newer head of the same pull request

Details

5bd40fa406

lab_check cloned into one fixed path, <work>/check, after an rm -rf whose result it ignored; a tree an
earlier check left that the lab's account could not remove made every check fail on "destination path
already exists". Each check now works in a fresh check-* directory it removes when done, and clears
what earlier checks left, naming any path it could not remove.

mesh-admin commented

2026-10-13 12:40:32 +00:00

Delivery novox/mesh-catalog@dd39d4ace74b — delivered since 2026-10-13T13:00:04Z

Delivery plan — builds lab → shanks; no bus step

tier 0: lab
shanks: receives lab
lab keeps data (ADR 0233): what it holds is backed up before it moves
grants step: if this changes what the bus's user list says (a new tool, subject or user), novox is sent that list alone, every build there kept, before lab is judged on its first machine

Machines

lab on shanks: passed — healthy 1 times over 26s
lab on the rest running it: sent

Transitions

2026-10-13 12:51 (new) → proposed (announced): novox/mesh-catalog#274's head announced
2026-10-13 12:55 proposed → checked (checked): a verdict names this commit and the trunk's head as its trunk commit
2026-10-13 12:55 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
2026-10-13 12:58 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
2026-10-13 12:58 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit
2026-10-13 13:00 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

**Delivery** `novox/mesh-catalog@dd39d4ace74b` — **delivered** since 2026-10-13T13:00:04Z **Delivery plan** — builds lab → shanks; no bus step - tier 0: lab - shanks: receives lab - lab keeps data (ADR 0233): what it holds is backed up before it moves - grants step: if this changes what the bus's user list says (a new tool, subject or user), novox is sent that list alone, every build there kept, before lab is judged on its first machine **Machines** - lab on shanks: passed — healthy 1 times over 26s - lab on the rest running it: sent **Transitions** - 2026-10-13 12:51 (new) → proposed (announced): novox/mesh-catalog#274's head announced - 2026-10-13 12:55 proposed → checked (checked): a verdict names this commit and the trunk's head as its trunk commit - 2026-10-13 12:55 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail - 2026-10-13 12:58 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move - 2026-10-13 12:58 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit - 2026-10-13 13:00 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.

mesh-admin commented

2026-10-13 12:43:58 +00:00

Merge check at 5bd40fa4
Judged: head 5bd40fa4061d merged into trunk 0d723484a84a; read beside it: mesh-controller f3875a6b6019, mesh-controller-main f3875a6b6019, mesh-host 7a41e120635b, mesh-tools 954e1a53eef9

mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: lab)
mesh/repo-check: FAIL — merge-check.sh (go) passed; its merge-check-typescript.sh failed: TypeScript modules that failed: lab

Delivery plan — builds lab → shanks; no bus step

tier 0: lab
shanks: receives lab
lab keeps data (ADR 0233): what it holds is backed up before it moves
grants step: if this changes what the bus's user list says (a new tool, subject or user), novox is sent that list alone, every build there kept, before lab is judged on its first machine

Run by the build seat on novox as build-1791895231452385490 (builds --log build-1791895231452385490).

--- the gate: lab
lab: ok, keeps work (cache); WARNING: runtime-env ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339)
long-running resources without health: 0
files asking for a setting without saying whether it is trusted: 1
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-13T12:13:29Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-13T12:13:29Z

modules the change moves:
  - lab (changed)

machines:
  - a machine (f35): composes
  - holds mesh-dns-resolver (idt): composes
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
  - a machine (yvbpsm): composes

a merge would rebuild 1 module(s) in 1 tier(s): lab
--- its merge-check.sh (go toolchain)
ok  	git.novox.be/novox/mesh-catalog/checks/words	0.007s
words: 12 retired words and 70 daemons' lines, none in the 46323 strings of 716 files
ok  	git.novox.be/novox/mesh-catalog/checks/agenthome	0.012s
agent-home: no archive and no file written into under the agent's home
ok  	git.novox.be/novox/mesh-catalog/checks/secrets	0.014s
secrets: no setting can replace a vault secret in a mergeable file (1 known, nested)
ok  	git.novox.be/novox/mesh-catalog/checks/paths	0.007s
paths: no module names another's state on its machine (3 known, listed to go)
long-running resources without health: 66, as kept
ok  	git.novox.be/novox/mesh-catalog/checks/deps	0.313s
modules/lab is TypeScript: merge-check-typescript.sh type-checks and tests it
--- its merge-check-typescript.sh (typescript toolchain)
type-checking modules/lab (TypeScript)
testing modules/lab (TypeScript)
✔ a checkout with a lock file at its root is installed there (2.388974ms)
✔ a checkout whose package lives one directory down is installed there, as mesh-tools' node-tools (0.863151ms)
✔ a package.json with no lock file is named, never installed (1.158832ms)
✔ a run installs and builds mesh-tools in node-tools, where its lock file is (7.870064ms)
✔ a run installs a checkout with its lock file at the root there, and builds only when asked (4.862158ms)
✔ a run fails at its setup, naming the checkout, when nothing of it can be installed (12.609862ms)
✔ a run checks out mesh-tools-go and says where its checkouts are (0.856842ms)
✔ the forge credential is the bound account and its sealed secret, for the forge's own origin (0.721891ms)
✔ a run clones through git's credential store, and its script carries no secret (1.020892ms)
✔ git presents the stored credential to the forge, and only when one is given (10.482908ms)
✔ a stopped script removes the credential, whatever ends it (12.325502ms)
✔ the script removes the credential as soon as its clones are done (0.36966ms)
✔ credentials a crashed run left are removed, but not the running one's (1.493602ms)
✔ the lab asks the package registry for read only (0.359201ms)
✔ each check gets a fresh directory of its own (0.43106ms)
✖ what earlier checks left is cleared, and what cannot be is named rather than tripped over (1.220693ms)
ℹ tests 16
ℹ suites 0
ℹ pass 15
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 159.460155

✖ failing tests:

test at test/runs.test.ts:212:1
✖ what earlier checks left is cleared, and what cannot be is named rather than tripped over (1.220693ms)
  Error: ENOENT: no such file or directory, chmod '/tmp/lab-work-Abjt3q/check-stuck/locked'
      at chmodSync (node:fs:2058:11)
      at TestContext.<anonymous> (file:///tmp/tmp.sW9ETUl6IU/lab/test/runs.test.ts:220:3)
      at Test.runInAsyncScope (node:async_hooks:214:14)
      at Test.run (node:internal/test_runner/test:1047:25)
      at Test.processPendingSubtests (node:internal/test_runner/test:744:18)
      at Test.postRun (node:internal/test_runner/test:1173:19)
      at Test.run (node:internal/test_runner/test:1101:12)
      at async Test.processPendingSubtests (node:internal/test_runner/test:744:7) {
    errno: -2,
    code: 'ENOENT',
    syscall: 'chmod',
    path: '/tmp/lab-work-Abjt3q/check-stuck/locked'
  }
TypeScript modules that failed: lab

**Merge check** at `5bd40fa4` Judged: head `5bd40fa4061d` merged into trunk `0d723484a84a`; read beside it: mesh-controller `f3875a6b6019`, mesh-controller-main `f3875a6b6019`, mesh-host `7a41e120635b`, mesh-tools `954e1a53eef9` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: lab) - `mesh/repo-check`: **FAIL** — merge-check.sh (go) passed; its merge-check-typescript.sh failed: TypeScript modules that failed: lab **Delivery plan** — builds lab → shanks; no bus step - tier 0: lab - shanks: receives lab - lab keeps data (ADR 0233): what it holds is backed up before it moves - grants step: if this changes what the bus's user list says (a new tool, subject or user), novox is sent that list alone, every build there kept, before lab is judged on its first machine Run by the build seat on novox as `build-1791895231452385490` (`builds --log build-1791895231452385490`). ``` --- the gate: lab lab: ok, keeps work (cache); WARNING: runtime-env ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339) long-running resources without health: 0 files asking for a setting without saying whether it is trusted: 1 1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-13T12:13:29Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-13T12:13:29Z modules the change moves: - lab (changed) machines: - a machine (f35): composes - holds mesh-dns-resolver (idt): composes - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes - a machine (yvbpsm): composes a merge would rebuild 1 module(s) in 1 tier(s): lab --- its merge-check.sh (go toolchain) ok git.novox.be/novox/mesh-catalog/checks/words 0.007s words: 12 retired words and 70 daemons' lines, none in the 46323 strings of 716 files ok git.novox.be/novox/mesh-catalog/checks/agenthome 0.012s agent-home: no archive and no file written into under the agent's home ok git.novox.be/novox/mesh-catalog/checks/secrets 0.014s secrets: no setting can replace a vault secret in a mergeable file (1 known, nested) ok git.novox.be/novox/mesh-catalog/checks/paths 0.007s paths: no module names another's state on its machine (3 known, listed to go) long-running resources without health: 66, as kept ok git.novox.be/novox/mesh-catalog/checks/deps 0.313s modules/lab is TypeScript: merge-check-typescript.sh type-checks and tests it --- its merge-check-typescript.sh (typescript toolchain) type-checking modules/lab (TypeScript) testing modules/lab (TypeScript) ✔ a checkout with a lock file at its root is installed there (2.388974ms) ✔ a checkout whose package lives one directory down is installed there, as mesh-tools' node-tools (0.863151ms) ✔ a package.json with no lock file is named, never installed (1.158832ms) ✔ a run installs and builds mesh-tools in node-tools, where its lock file is (7.870064ms) ✔ a run installs a checkout with its lock file at the root there, and builds only when asked (4.862158ms) ✔ a run fails at its setup, naming the checkout, when nothing of it can be installed (12.609862ms) ✔ a run checks out mesh-tools-go and says where its checkouts are (0.856842ms) ✔ the forge credential is the bound account and its sealed secret, for the forge's own origin (0.721891ms) ✔ a run clones through git's credential store, and its script carries no secret (1.020892ms) ✔ git presents the stored credential to the forge, and only when one is given (10.482908ms) ✔ a stopped script removes the credential, whatever ends it (12.325502ms) ✔ the script removes the credential as soon as its clones are done (0.36966ms) ✔ credentials a crashed run left are removed, but not the running one's (1.493602ms) ✔ the lab asks the package registry for read only (0.359201ms) ✔ each check gets a fresh directory of its own (0.43106ms) ✖ what earlier checks left is cleared, and what cannot be is named rather than tripped over (1.220693ms) ℹ tests 16 ℹ suites 0 ℹ pass 15 ℹ fail 1 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0 ℹ duration_ms 159.460155 ✖ failing tests: test at test/runs.test.ts:212:1 ✖ what earlier checks left is cleared, and what cannot be is named rather than tripped over (1.220693ms) Error: ENOENT: no such file or directory, chmod '/tmp/lab-work-Abjt3q/check-stuck/locked' at chmodSync (node:fs:2058:11) at TestContext.<anonymous> (file:///tmp/tmp.sW9ETUl6IU/lab/test/runs.test.ts:220:3) at Test.runInAsyncScope (node:async_hooks:214:14) at Test.run (node:internal/test_runner/test:1047:25) at Test.processPendingSubtests (node:internal/test_runner/test:744:18) at Test.postRun (node:internal/test_runner/test:1173:19) at Test.run (node:internal/test_runner/test:1101:12) at async Test.processPendingSubtests (node:internal/test_runner/test:744:7) { errno: -2, code: 'ENOENT', syscall: 'chmod', path: '/tmp/lab-work-Abjt3q/check-stuck/locked' } TypeScript modules that failed: lab ```

mesh-admin reviewed

2026-10-13 12:48:54 +00:00

mesh-admin left a comment

Verdict: request changes at 5bd40fa406

Statuses at this head (mesh-delivery.checks, 2026-10-13 14:46): mesh/merge-gate success, "PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: lab)"; mesh/repo-check failure, "FAIL — merge-check.sh (go) passed; its merge-check-typescript.sh failed: TypeScript modules that failed: lab"; mesh/delivery failure, "rejected: the gate failed or could not run, or the repository's own check failed".

Blocking: the new test fails in the toolchain, which runs as root. "what earlier checks left is cleared, and what cannot be is named rather than tripped over" fails at test/runs.test.ts:220 with "ENOENT: no such file or directory, chmod '/tmp/lab-work-…/check-stuck/locked'". As root, rmSync removes the 0555 tree, and the chmodSync(…, 0o755) that follows clearChecks has nothing left to chmod. The process.getuid?.() !== 0 guard comes after that line, so it never protects it. The fix is to build and assert the stuck tree only when not root, for example with the test's skip option or an if around the setup, the chmod-back and the assertion. As a non-root account on g14 (niced), all 16 lab tests pass. On main's tools, the test file fails at import ("does not provide an export named 'checkDir'"), so the new tests do fail without the change.

The change does what issue 624's last note asks. Each lab_check clones into a fresh mkdtemp check-* directory. The finally removes it on every path, including the early return when the clone fails. setupProblems now works under <dir>/setup inside it, rather than a fixed check-setup. clearChecks removes the old fixed check and check-setup and any check-*, and names each one it cannot remove in the output.

Non-blocking.

clearChecks removes every check-*, including one a lab_check running at the same time is working in. Two concurrent checks are unlikely, but the second would pull the first's clone out from under it. Clearing only directories older than a few minutes would avoid that.
A leftover that cannot be cleared is named, but ok stays true. The live /var/lib/mesh-lab-runs/check on shanks is probably root's, from a run before the work directory was the operator's. If so, it will be named on every check until something with root removes it. The output could say how it gets removed (through the mesh, not by hand), or the module could declare the old path so the node-engine clears it.
Issue 624's consumer side: the lab's bound secret is root 0600 because the lab's bundle names no word for it. Naming it in build.artifacts[].env (for example MESH_LAB_FORGE_SECRET: ${dir:state}/package-registry.secret) makes the controller give that one file to the runtime's account (ADR 0192, givenTo), as gitea, icecast, mailu, redis and umami already do. I checked that on mesh-controller main with a throwaway test. That is the narrower alternative to mesh-controller #302, and it belongs in this repository; see my review there.

Verdict: request changes at 5bd40fa4061d9aa7fe619f6641c35993478ab9cf Statuses at this head (mesh-delivery.checks, 2026-10-13 14:46): mesh/merge-gate success, "PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: lab)"; mesh/repo-check failure, "FAIL — merge-check.sh (go) passed; its merge-check-typescript.sh failed: TypeScript modules that failed: lab"; mesh/delivery failure, "rejected: the gate failed or could not run, or the repository's own check failed". **Blocking: the new test fails in the toolchain, which runs as root.** "what earlier checks left is cleared, and what cannot be is named rather than tripped over" fails at test/runs.test.ts:220 with "ENOENT: no such file or directory, chmod '/tmp/lab-work-…/check-stuck/locked'". As root, `rmSync` removes the 0555 tree, and the `chmodSync(…, 0o755)` that follows `clearChecks` has nothing left to chmod. The `process.getuid?.() !== 0` guard comes after that line, so it never protects it. The fix is to build and assert the stuck tree only when not root, for example with the test's `skip` option or an `if` around the setup, the chmod-back and the assertion. As a non-root account on g14 (niced), all 16 lab tests pass. On main's tools, the test file fails at import ("does not provide an export named 'checkDir'"), so the new tests do fail without the change. **The change does what issue 624's last note asks.** Each lab_check clones into a fresh `mkdtemp` `check-*` directory. The `finally` removes it on every path, including the early return when the clone fails. setupProblems now works under `<dir>/setup` inside it, rather than a fixed `check-setup`. `clearChecks` removes the old fixed `check` and `check-setup` and any `check-*`, and names each one it cannot remove in the output. **Non-blocking.** - `clearChecks` removes every `check-*`, including one a lab_check running at the same time is working in. Two concurrent checks are unlikely, but the second would pull the first's clone out from under it. Clearing only directories older than a few minutes would avoid that. - A leftover that cannot be cleared is named, but `ok` stays true. The live `/var/lib/mesh-lab-runs/check` on shanks is probably root's, from a run before the work directory was the operator's. If so, it will be named on every check until something with root removes it. The output could say how it gets removed (through the mesh, not by hand), or the module could declare the old path so the node-engine clears it. - Issue 624's consumer side: the lab's bound secret is root 0600 because the lab's bundle names no word for it. Naming it in `build.artifacts[].env` (for example `MESH_LAB_FORGE_SECRET: ${dir:state}/package-registry.secret`) makes the controller give that one file to the runtime's account (ADR 0192, `givenTo`), as gitea, icecast, mailu, redis and umami already do. I checked that on mesh-controller main with a throwaway test. That is the narrower alternative to mesh-controller #302, and it belongs in this repository; see my review there.

jschoubben added 1 commit 2026-10-13 12:51:20 +00:00

lab: name the bound credential in the tools bundle's env, and skip the locked-tree test as root (hq issue 624, reviews 198, 199)

mesh/merge-gate head dd39d4ace74b merged into trunk 0d723484a84a — pass: builds lab → shanks; no bus step; every machine composes with the change as it did…

Details

mesh/repo-check head dd39d4ace74b merged into trunk 0d723484a84a — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…

Details

mesh/delivery delivered

Details

dd39d4ace7

The lab's tools read their bound npm-package-registry credential, root's at 0600 on the machine. The
controller gives the runtime's account exactly the files a tools bundle names in its env (ADR 0192), as
gitea and umami are given theirs, so the bundle now names the credential and its binding there, and no
other credential changes hands.

The merge check runs as root, where no directory mode stops a removal: the test that makes an
unremovable tree now skips there before it makes anything, and a second test clears leftovers whoever
runs it.

mesh-admin changed title from ~~lab_check: work in a fresh directory each time, and name what an earlier check left (issue 624)~~ to lab: read the bound forge credential through the tools bundle's env, and check in a fresh directory each time (issue 624)

2026-10-13 12:51:32 +00:00

mesh-admin commented

2026-10-13 12:55:41 +00:00

Merge check at dd39d4ac
Judged: head dd39d4ace74b merged into trunk 0d723484a84a; read beside it: mesh-controller 0a1dc5f93fcb, mesh-controller-main 0a1dc5f93fcb, mesh-host 7a41e120635b, mesh-tools 954e1a53eef9

mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (run 2) (modules: lab)
mesh/repo-check: PASS — its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-check-typescript.sh (typescript) (run 2)

Delivery plan — builds lab → shanks; no bus step

tier 0: lab
shanks: receives lab
lab keeps data (ADR 0233): what it holds is backed up before it moves
grants step: if this changes what the bus's user list says (a new tool, subject or user), novox is sent that list alone, every build there kept, before lab is judged on its first machine

Run by the build seat on ace as build-1791895907772717762 (builds --log build-1791895907772717762).

restarted: handed out 2 times; an earlier hand-out did not finish here, so this run started from the beginning

--- the gate: lab
lab: ok, keeps work (cache); WARNING: runtime-env ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339)
long-running resources without health: 0
files asking for a setting without saying whether it is trusted: 1
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-13T12:48:29Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-13T12:48:29Z

modules the change moves:
  - lab (changed)

machines:
  - a machine (f35): composes
  - holds mesh-dns-resolver (idt): composes
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
  - a machine (yvbpsm): composes

a merge would rebuild 1 module(s) in 1 tier(s): lab
--- its merge-check.sh (go toolchain)
ok  	git.novox.be/novox/mesh-catalog/checks/words	0.002s
words: 12 retired words and 70 daemons' lines, none in the 46323 strings of 716 files
ok  	git.novox.be/novox/mesh-catalog/checks/agenthome	0.010s
agent-home: no archive and no file written into under the agent's home
ok  	git.novox.be/novox/mesh-catalog/checks/secrets	0.010s
secrets: no setting can replace a vault secret in a mergeable file (1 known, nested)
ok  	git.novox.be/novox/mesh-catalog/checks/paths	0.006s
paths: no module names another's state on its machine (3 known, listed to go)
long-running resources without health: 66, as kept
ok  	git.novox.be/novox/mesh-catalog/checks/deps	0.210s
modules/lab is TypeScript: merge-check-typescript.sh type-checks and tests it
--- its merge-check-typescript.sh (typescript toolchain)
type-checking modules/lab (TypeScript)
testing modules/lab (TypeScript)
✔ a checkout with a lock file at its root is installed there (2.208456ms)
✔ a checkout whose package lives one directory down is installed there, as mesh-tools' node-tools (0.776077ms)
✔ a package.json with no lock file is named, never installed (1.26197ms)
✔ a run installs and builds mesh-tools in node-tools, where its lock file is (7.671566ms)
✔ a run installs a checkout with its lock file at the root there, and builds only when asked (4.062579ms)
✔ a run fails at its setup, naming the checkout, when nothing of it can be installed (9.321121ms)
✔ a run checks out mesh-tools-go and says where its checkouts are (0.683624ms)
✔ the forge credential is the bound account and its sealed secret, for the forge's own origin (0.569319ms)
✔ a run clones through git's credential store, and its script carries no secret (0.75607ms)
✔ git presents the stored credential to the forge, and only when one is given (17.186631ms)
✔ a stopped script removes the credential, whatever ends it (10.354653ms)
✔ the script removes the credential as soon as its clones are done (0.306615ms)
✔ credentials a crashed run left are removed, but not the running one's (1.260968ms)
✔ the lab asks the package registry for read only (0.301877ms)
✔ each check gets a fresh directory of its own (0.433053ms)
﹣ what earlier checks left is cleared, and what cannot be is named rather than tripped over (0.0532ms) # running as root, which no directory mode stops from removing anything
✔ leftover checks are cleared, whoever runs the check (0.726374ms)
✔ the lab's tools bundle names its bound credential and binding in its env (0.143639ms)
ℹ tests 18
ℹ suites 0
ℹ pass 17
ℹ fail 0
ℹ cancelled 0
ℹ skipped 1
ℹ todo 0
ℹ duration_ms 140.897672

**Merge check** at `dd39d4ac` Judged: head `dd39d4ace74b` merged into trunk `0d723484a84a`; read beside it: mesh-controller `0a1dc5f93fcb`, mesh-controller-main `0a1dc5f93fcb`, mesh-host `7a41e120635b`, mesh-tools `954e1a53eef9` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (run 2) (modules: lab) - `mesh/repo-check`: **PASS** — its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-check-typescript.sh (typescript) (run 2) **Delivery plan** — builds lab → shanks; no bus step - tier 0: lab - shanks: receives lab - lab keeps data (ADR 0233): what it holds is backed up before it moves - grants step: if this changes what the bus's user list says (a new tool, subject or user), novox is sent that list alone, every build there kept, before lab is judged on its first machine Run by the build seat on ace as `build-1791895907772717762` (`builds --log build-1791895907772717762`). ``` restarted: handed out 2 times; an earlier hand-out did not finish here, so this run started from the beginning --- the gate: lab lab: ok, keeps work (cache); WARNING: runtime-env ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339) long-running resources without health: 0 files asking for a setting without saying whether it is trusted: 1 1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-13T12:48:29Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-13T12:48:29Z modules the change moves: - lab (changed) machines: - a machine (f35): composes - holds mesh-dns-resolver (idt): composes - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes - a machine (yvbpsm): composes a merge would rebuild 1 module(s) in 1 tier(s): lab --- its merge-check.sh (go toolchain) ok git.novox.be/novox/mesh-catalog/checks/words 0.002s words: 12 retired words and 70 daemons' lines, none in the 46323 strings of 716 files ok git.novox.be/novox/mesh-catalog/checks/agenthome 0.010s agent-home: no archive and no file written into under the agent's home ok git.novox.be/novox/mesh-catalog/checks/secrets 0.010s secrets: no setting can replace a vault secret in a mergeable file (1 known, nested) ok git.novox.be/novox/mesh-catalog/checks/paths 0.006s paths: no module names another's state on its machine (3 known, listed to go) long-running resources without health: 66, as kept ok git.novox.be/novox/mesh-catalog/checks/deps 0.210s modules/lab is TypeScript: merge-check-typescript.sh type-checks and tests it --- its merge-check-typescript.sh (typescript toolchain) type-checking modules/lab (TypeScript) testing modules/lab (TypeScript) ✔ a checkout with a lock file at its root is installed there (2.208456ms) ✔ a checkout whose package lives one directory down is installed there, as mesh-tools' node-tools (0.776077ms) ✔ a package.json with no lock file is named, never installed (1.26197ms) ✔ a run installs and builds mesh-tools in node-tools, where its lock file is (7.671566ms) ✔ a run installs a checkout with its lock file at the root there, and builds only when asked (4.062579ms) ✔ a run fails at its setup, naming the checkout, when nothing of it can be installed (9.321121ms) ✔ a run checks out mesh-tools-go and says where its checkouts are (0.683624ms) ✔ the forge credential is the bound account and its sealed secret, for the forge's own origin (0.569319ms) ✔ a run clones through git's credential store, and its script carries no secret (0.75607ms) ✔ git presents the stored credential to the forge, and only when one is given (17.186631ms) ✔ a stopped script removes the credential, whatever ends it (10.354653ms) ✔ the script removes the credential as soon as its clones are done (0.306615ms) ✔ credentials a crashed run left are removed, but not the running one's (1.260968ms) ✔ the lab asks the package registry for read only (0.301877ms) ✔ each check gets a fresh directory of its own (0.433053ms) ﹣ what earlier checks left is cleared, and what cannot be is named rather than tripped over (0.0532ms) # running as root, which no directory mode stops from removing anything ✔ leftover checks are cleared, whoever runs the check (0.726374ms) ✔ the lab's tools bundle names its bound credential and binding in its env (0.143639ms) ℹ tests 18 ℹ suites 0 ℹ pass 17 ℹ fail 0 ℹ cancelled 0 ℹ skipped 1 ℹ todo 0 ℹ duration_ms 140.897672 ```

mesh-admin reviewed

2026-10-13 12:57:59 +00:00

mesh-admin left a comment

Verdict: approve at dd39d4ace7

(Posted as a comment review: the forge tool refuses APPROVED from a tool, ADR 0266.)

Statuses at this head (gitea, 2026-10-13 14:55): mesh/merge-gate success, "head dd39d4ace7 merged into trunk 0d723484a8 — pass: builds lab → shanks; no bus step; every machine composes with the change as it did…"; mesh/repo-check success, "head dd39d4ace7 merged into trunk 0d723484a8 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…"; mesh/delivery success, "ready: it delivers once merged".

Only the lab's two files change hands. modules/lab/module.json:108-109 names ${dir:state}/package-registry.json and ${dir:state}/package-registry.secret in the code bundle's env. Those are the same paths as the manifest's binds and secrets for npm-package-registry. In shanks' declaration (mesh-controller.plan on main), ${dir:state} is /var/lib/lab. The sealed secret is the file resource lab.secret-npm-package-registry at /var/lib/lab/package-registry.secret, with no owner. The binding is lab.bound-npm-package-registry at /var/lib/lab/package-registry.json. On mesh-controller main, givenTo (internal/catalogue/runtime.go) gives the runtime's account every owner-less file or directory that a bundle word names or holds. /var/lib/lab (lab.state) and lab.env already belong to that account through MESH_LAB_ENV_FILE, so these two files are the only new ones. index.ts:23-24 reads MESH_LAB_FORGE_BINDING and MESH_LAB_FORGE_SECRET from env before lab.env, and the values agree with lab.env's.

The root-run test fix is sound. test/runs.test.ts:215 skips the locked-tree test as root through the test's skip option, before it creates anything, and repo-check (which runs as root) is now green. The new test "leftover checks are cleared, whoever runs the check" checks clearing without a locked tree, so it holds for any account. The lab_check leftover logic (tools/runs.ts:146-168, tools/index.ts:35-49) is unchanged since review 199.

Tests. I cloned this head on g14 and ran the lab tests as uid 1000, niced: 18 of 18 pass.

Non-blocking, carried over from review 199. clearChecks (tools/runs.ts:160) removes any check-*, including one that a concurrent lab_check is working in. A leftover it cannot remove is named, but ok stays true.

Verdict: approve at dd39d4ace74b6a592cfbc726db297e2b7bf42d37 (Posted as a comment review: the forge tool refuses APPROVED from a tool, ADR 0266.) Statuses at this head (gitea, 2026-10-13 14:55): mesh/merge-gate success, "head dd39d4ace74b merged into trunk 0d723484a84a — pass: builds lab → shanks; no bus step; every machine composes with the change as it did…"; mesh/repo-check success, "head dd39d4ace74b merged into trunk 0d723484a84a — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…"; mesh/delivery success, "ready: it delivers once merged". **Only the lab's two files change hands.** modules/lab/module.json:108-109 names `${dir:state}/package-registry.json` and `${dir:state}/package-registry.secret` in the `code` bundle's env. Those are the same paths as the manifest's `binds` and `secrets` for npm-package-registry. In shanks' declaration (mesh-controller.plan on main), `${dir:state}` is /var/lib/lab. The sealed secret is the file resource lab.secret-npm-package-registry at /var/lib/lab/package-registry.secret, with no owner. The binding is lab.bound-npm-package-registry at /var/lib/lab/package-registry.json. On mesh-controller main, `givenTo` (internal/catalogue/runtime.go) gives the runtime's account every owner-less file or directory that a bundle word names or holds. /var/lib/lab (lab.state) and lab.env already belong to that account through MESH_LAB_ENV_FILE, so these two files are the only new ones. index.ts:23-24 reads MESH_LAB_FORGE_BINDING and MESH_LAB_FORGE_SECRET from env before lab.env, and the values agree with lab.env's. **The root-run test fix is sound.** test/runs.test.ts:215 skips the locked-tree test as root through the test's `skip` option, before it creates anything, and repo-check (which runs as root) is now green. The new test "leftover checks are cleared, whoever runs the check" checks clearing without a locked tree, so it holds for any account. The lab_check leftover logic (tools/runs.ts:146-168, tools/index.ts:35-49) is unchanged since review 199. **Tests.** I cloned this head on g14 and ran the lab tests as uid 1000, niced: 18 of 18 pass. **Non-blocking, carried over from review 199.** `clearChecks` (tools/runs.ts:160) removes any `check-*`, including one that a concurrent lab_check is working in. A leftover it cannot remove is named, but `ok` stays true.

mesh-admin merged commit a84d0094a0 into main

2026-10-13 12:58:15 +00:00

mesh-admin deleted branch fix/624-lab-check-in-a-fresh-directory

2026-10-13 12:58:15 +00:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-catalog#274