Merge pull request 'events: an e2e test — an emitted event reaches the audit trail over the mesh's broker' (#2) from events/audit-e2e into initialization

README.md

@@ -131,6 +131,74 @@ Placing needs a built host binary — set `MESH_LAB_HOST_BINARY` to one. It is a
 rather than a search on purpose: the declaration design leaves *where `place:` gets its
 artifacts from* open, and guessing would harden into the answer by accident.
 
+## Pointing a run at the repositories
+
+**Every variable is an explicit path, and none of them has a default.** A test whose artifact was
+not pointed at *skips* — it does not fail — so an unset variable is a green run that proved
+nothing. That is `novox/hq` 04-ISSUES/005 exactly, and it has now been rediscovered twice, so it
+is written down here rather than reconstructed a third time.
+
+```sh
+export MESH_LAB_HOST_BINARY=<mesh-host>/mesh-host
+export MESH_LAB_BUNDLE=<mesh-host>/examples/substrate-first-node.lock
+export MESH_LAB_MODULES=<mesh-control>/examples/modules
+export MESH_LAB_BUILDER=<mesh-control>/build/mesh-builder   # build/, which is git-ignored
+
+# Built with `go build -o <path> ./examples/<name>` in mesh-control.
+export MESH_LAB_PROVISIONER=<somewhere>/postgres-provisioner
+export MESH_LAB_OBJECTSTORE_PROVISIONER=<somewhere>/objectstore-provisioner
+export MESH_LAB_ROUTE_PROXY=<somewhere>/route-proxy
+```
+
+`MESH_LAB_HOST_BINARY` and `MESH_LAB_MODULES` do double duty: the repository each sits in is what
+`suite` rebuilds and what the receipt claims. Point the run at a repository and it is built and
+claimed; leave it out and it is neither.
+
+Check before running a long suite — it says which of these are missing rather than skipping
+quietly:
+
+```sh
+node --experimental-strip-types src/cli.ts check
+```
+
+If it says the daemon is not reachable, the group grant postdates the shell. `newgrp` fixes it,
+but a heredoc into `newgrp` runs the suite as a child of a shell that then exits — start it with
+`setsid nohup … &` inside the heredoc, or the run dies with the shell that launched it.
+
+## When something takes too long
+
+```sh
+export MESH_LAB_LOG=info                 # or debug, or trace
+export MESH_LAB_LOG_FILE=/tmp/lab.log    # unset writes to stderr
+```
+
+| level | what it adds |
+|---|---|
+| `info` | each step of a raise, with how long the previous one took; anything that failed; **and a line every 15s naming whatever is still running** |
+| `debug` | every command the lab runs — incus, docker, and anything local — with its duration, and the stderr of anything that failed |
+| `trace` | what those commands printed |
+
+**The heartbeat is the point.** A stall is a command that started and has not finished, and the
+only thing separating it from ordinary work is how long it has been going — which nothing can tell
+you unless something is still counting. At `info` a run says `… docker load -i /tmp/image.tar —
+still running after 45s` while it happens, rather than nothing until it gives up.
+
+Written with `appendFileSync`, so unlike a redirected stdout it cannot lag behind the run. It is
+off unless asked for.
+
+**A redirected log lags, so do not diagnose a stall from it.** Node block-buffers stdout when it
+is a file rather than a terminal, so `> run.log` can sit unchanged for minutes while the run is
+working normally. On 2026-09-01 that was read as a stall twice, once after a real stall had just
+been fixed — the most expensive kind of false signal, because it argues the fix did not work. Ask
+the machines instead:
+
+```sh
+incus list -c ns
+incus exec <instance>-registry -- systemctl is-active docker
+```
+
+*"I cannot see progress" is not evidence of no progress.*
+
 ## Measured on a workstation
 
 | | one machine | two machines | two machines + a router |
@@ -235,13 +303,35 @@ This repository carries implementation. It does not carry decisions.
 No build step — Node strips the types.
 
 ```
-npm test                   the declaration layer and the diagram, offline, 49 tests
-npm run test:integration   real scenarios against a real hypervisor, 14 tests
+npm test                   the declaration layer and the diagram, offline
+npm run test:integration   real scenarios against a real hypervisor
 npm run typecheck          source and tests both — a test that does not compile is a test
                            that silently never ran
 npm run check              typecheck + both suites — this is the gate
+npm run last-run           when this machine last ran the suite, and whether that still counts
 ```
 
+**The integration suite rebuilds what it tests, and leaves a receipt saying it ran.**
+
+It needs a hypervisor, so it cannot run on every push — which means it runs when somebody
+remembers, and *remembering is not a mechanism*. The harness this replaces had not built for two
+and a half months and nothing said so (`novox/hq` 04-ISSUES/005). So:
+
+- **Before the run**, the host binary, the control-plane image and the builder are rebuilt from
+  source. The last two both parse manifests; building one and not the other is how a rename gets
+  tested against an eleven-hour-old binary.
+- **After the run**, a receipt is written to XDG state — outside the repository, because the
+  question is *has this machine run it*, and a receipt in git would be a claim about everybody's
+  machine made by whoever committed last.
+- `last-run` judges it and exits non-zero when it no longer counts: old, failed, taken against
+  commits the repositories have moved past, taken against a tree with uncommitted work, or a run
+  that never included the end-to-end file.
+  **A receipt that says nothing about something is not a receipt that clears it.**
+
+`suite <paths>` runs something narrower, and the receipt records that it did — a green run of the
+unit tests must not be readable as coverage of the pipeline. `--no-build` skips the rebuild, for
+iterating on a test rather than on the code under it.
+
 **A test names the decision it defends** (`novox/hq` ADR 0017). A decision with no test is one
 that will quietly stop being true, and nobody learns that from a document:
 
@@ -257,6 +347,11 @@ that will quietly stop being true, and nobody learns that from a document:
 | the live diagram distinguishes scenery from a node | ADR 0016 |
 | the live diagram draws what exists, never what was asked for | the diagram design |
 | a picture nobody can open is not a picture | the diagram design |
+| a run that raised no machines is not end-to-end coverage | 04-ISSUES/005 |
+| a run whose result could not be read writes nothing | 04-ISSUES/005 |
+| the control plane's image and builder are always built together | 04-ISSUES/005 |
+| every repository the receipt claims was built by the run | 04-ISSUES/005 |
+| a run against uncommitted work does not cover the commit | 04-ISSUES/005 |
 
 **Mocking the hypervisor is forbidden.** A fake would assert that the fake behaves as expected,
 which is the shape of test this project exists to stop shipping. Integration tests skip with a

package.json

@@ -9,8 +9,9 @@
   "scripts": {
     "typecheck": "tsc --noEmit && tsc --noEmit -p tsconfig.test.json",
     "test": "node --test --experimental-strip-types 'test/*.test.ts'",
-    "test:integration": "node --test --test-concurrency=1 --experimental-strip-types 'test/integration/*.test.ts'",
-    "check": "npm run typecheck && npm test && npm run test:integration"
+    "test:integration": "node --experimental-strip-types src/cli.ts suite",
+    "check": "npm run typecheck && npm test && npm run test:integration",
+    "last-run": "node --experimental-strip-types src/cli.ts last-run"
   },
   "dependencies": {
     "yaml": "^2.6.0"

provisioners/README.md

@@ -22,7 +22,7 @@ and is then given, by the host, from an ordinary declaration:
 | | |
 |---|---|
 | `mesh.json` | every consumer, what it asked for, and where its credential is |
-| `<node>.secret` | one consumer's password, alone in the file, sealed in transit and written in plain by the host |
+| `<node>.<module>.secret` | one consumer's password, alone in the file, sealed in transit and written in plain by the host. Named after both, because a consumer is a module on a machine and a node routinely runs several (`novox/hq` 04-ISSUES/022) |
 
 Two files rather than one because the mesh discarded the plaintext and cannot compose a document
 containing it. The consequence is a good one: the readable half stays readable, and the secret

scenarios/a-public-name.yml

@@ -0,0 +1,26 @@
+# One machine serving a public name with a certificate from an authority it did not run itself.
+#
+# The lab keeps production's two-authority split rather than collapsing it (01-RESEARCH/004): the
+# mesh's own authority certifies `.internal` names, and a name reachable from outside is certified
+# by ACME. A single-authority lab would hide any fault living in that split, so this raises a real
+# ACME server and makes the proxy actually order from it.
+#
+# Pebble rather than a stub, for the reason the lab exists at all: what is under test is whether an
+# HTTP-01 challenge is answered at the name being certified, and a fake would be told to agree.
+scenario: a-public-name
+
+segments:
+  hosting:
+    kind: public
+    cidr: [192.0.2.0/24]
+
+machines:
+  anchor:
+    at: { segment: hosting, address: [192.0.2.10] }
+    inbound: allow
+
+images:
+  - ghcr.io/letsencrypt/pebble:2.5.0
+
+place:
+  all: [runtime]

scenarios/an-object-store.yml

@@ -0,0 +1,31 @@
+# One machine running an object store that other machines use.
+#
+# The same shape as `a-provider`, against a different kind of provision, and that is the whole
+# reason it exists: novox/hq 04-ISSUES and the work breakdown's Phase 1.1 ask whether a module can
+# be given a bucket the way it is given a database. The provisioning model is name-agnostic — the
+# control plane special-cases neither — so what is unproven is not the mesh's half but the last
+# step, where something on the machine turns a delivered secret into a key that works.
+#
+# It also proves the half a database does not: **a consumer must not be able to reach another
+# consumer's bucket.** One store holds everybody's, where one PostgreSQL server holds separate
+# databases, so isolation here is a policy somebody wrote rather than a boundary the product has.
+scenario: an-object-store
+
+segments:
+  hosting:
+    kind: public
+    cidr: [192.0.2.0/24]
+
+machines:
+  anchor:
+    at: { segment: hosting, address: [192.0.2.10] }
+    inbound: allow
+
+images:
+  - minio/minio:RELEASE.2025-09-07T16-13-09Z
+  # The vendor's client, stocked so the provisioner has the thing it drives without reaching a
+  # public registry from a documentation range.
+  - minio/mc:RELEASE.2025-08-13T08-35-41Z
+
+place:
+  all: [runtime]

scenarios/audit-node.yml

@@ -0,0 +1,30 @@
+# One machine that becomes a mesh and then assigns itself the audit logger.
+#
+# The substrate is first-node's — a store, a broker, the control plane — and one module image on
+# top: the tool runtime carrying the audit-logger (mesh-catalog). The node enrols itself and the
+# mesh assigns it the audit logger, so its events account is one the mesh delivered, not the
+# broker's own (novox/hq ADR 0048).
+scenario: audit-node
+
+segments:
+  hosting:
+    kind: public
+    cidr: [192.0.2.0/24]
+
+machines:
+  anchor:
+    at: { segment: hosting, address: [192.0.2.10] }
+    inbound: allow
+    memory: 3GiB
+    cpus: 2
+
+images:
+  - postgres:17-alpine
+  - cloudamqp/lavinmq:latest
+  - mesh-control:development
+  # The tool runtime with the audit-logger, built by scripts/build-runtime-image.sh into the local
+  # daemon and stocked into the scenario's own registry, which is where the host pulls it from.
+  - mesh-runtime-audit:development
+
+place:
+  all: [host, runtime]

scenarios/grafana-node.yml

@@ -0,0 +1,28 @@
+# One machine that becomes a mesh and assigns itself grafana's tool runtime, configured by settings.
+#
+# plex/sonarr prove a runtime that self-detects its credential; this proves the other half of the
+# config story (novox/hq ADR 0051 + 0052): the operator states grafana's URL and token as the
+# assignment's settings, the mesh merges them into the config file the runtime reads, and the
+# runtime registers and serves grafana's tools from that — no credential baked into the manifest.
+scenario: grafana-node
+
+segments:
+  hosting:
+    kind: public
+    cidr: [192.0.2.0/24]
+
+machines:
+  anchor:
+    at: { segment: hosting, address: [192.0.2.10] }
+    inbound: allow
+    memory: 3GiB
+    cpus: 2
+
+images:
+  - postgres:17-alpine
+  - cloudamqp/lavinmq:latest
+  - mesh-control:development
+  - mesh-runtime-grafana:development
+
+place:
+  all: [host, runtime]

scenarios/minio-node.yml

@@ -0,0 +1,32 @@
+# One machine that becomes a mesh and grants a consumer an S3 bucket from an assigned minio provider.
+#
+# The postgres bed proves the provider/consumer contract for a database; this proves it for object
+# storage (novox/hq ADR 0052/0053), on a provider whose code drives the `mc` CLI (so the runtime image
+# carries it): minio is assigned, a consumer that requires s3-bucket is assigned, and the mesh mints
+# one secret key; minio's provisioner creates a bucket and a service account under the access key the
+# mesh derived with the secret it minted, and the consumer reaches its bucket with only that.
+scenario: minio-node
+
+segments:
+  hosting:
+    kind: public
+    cidr: [192.0.2.0/24]
+
+machines:
+  anchor:
+    at: { segment: hosting, address: [192.0.2.10] }
+    inbound: allow
+    memory: 3GiB
+    cpus: 2
+
+images:
+  - postgres:17-alpine
+  - cloudamqp/lavinmq:latest
+  - mesh-control:development
+  - minio/minio:latest
+  # minio's runtime, built by scripts/build-module-runtime.sh minio (it carries mc), stocked into the
+  # scenario's own registry.
+  - mesh-runtime-minio:development
+
+place:
+  all: [host, runtime]

scenarios/plex-node.yml

@@ -0,0 +1,31 @@
+# One machine that becomes a mesh and then assigns itself plex's tool runtime.
+#
+# The audit-node bed proved an assigned *consumer* (novox/hq ADR 0048). This proves an assigned
+# module that *serves tools* (ADR 0052): the same first-node substrate, plus plex's tool runtime on
+# top. The node enrols itself, the mesh issues plex a broker account scoped to serve.plex.* and
+# assigns it, the host runs the runtime container, and a caller invokes plex.plex_reachable over the
+# mesh — proof the module runs its own code as its own process under its own scoped account.
+scenario: plex-node
+
+segments:
+  hosting:
+    kind: public
+    cidr: [192.0.2.0/24]
+
+machines:
+  anchor:
+    at: { segment: hosting, address: [192.0.2.10] }
+    inbound: allow
+    memory: 3GiB
+    cpus: 2
+
+images:
+  - postgres:17-alpine
+  - cloudamqp/lavinmq:latest
+  - mesh-control:development
+  # Plex's tool runtime, built by scripts/build-module-runtime.sh plex into the local daemon and
+  # stocked into the scenario's own registry, which is where the host pulls it from.
+  - mesh-runtime-plex:development
+
+place:
+  all: [host, runtime]

scenarios/postgres-node.yml

@@ -0,0 +1,30 @@
+# One machine that becomes a mesh and grants a consumer a database from an assigned postgres provider.
+#
+# The redis mesh-grant bed proves the whole provider/consumer contract for a cache; this proves it for
+# a database (novox/hq ADR 0052/0053): postgres's runtime carries psql, its provisioner creates a role
+# and database under the login the mesh derived with the password the mesh minted, and a consumer
+# connects to its own database with only what the mesh delivered.
+scenario: postgres-node
+
+segments:
+  hosting:
+    kind: public
+    cidr: [192.0.2.0/24]
+
+machines:
+  anchor:
+    at: { segment: hosting, address: [192.0.2.10] }
+    inbound: allow
+    memory: 3GiB
+    cpus: 2
+
+images:
+  - postgres:17-alpine
+  - cloudamqp/lavinmq:latest
+  - mesh-control:development
+  # postgres's runtime, built by scripts/build-module-runtime.sh postgres (it carries psql), stocked
+  # into the scenario's own registry.
+  - mesh-runtime-postgres:development
+
+place:
+  all: [host, runtime]

scenarios/redis-node.yml

@@ -0,0 +1,31 @@
+# One machine that becomes a mesh and then assigns itself redis — a *provider* module.
+#
+# plex-node proves an assigned module that serves tools (novox/hq ADR 0052). This proves the same
+# for a provider: redis's runtime runs its provisioner AND its tools as one process under one scoped
+# broker account. The provisioner emitting a lifecycle event is the thing 0052 fixes — before it,
+# the provisioner ran in a container with no broker and its emit could not fire.
+scenario: redis-node
+
+segments:
+  hosting:
+    kind: public
+    cidr: [192.0.2.0/24]
+
+machines:
+  anchor:
+    at: { segment: hosting, address: [192.0.2.10] }
+    inbound: allow
+    memory: 3GiB
+    cpus: 2
+
+images:
+  - postgres:17-alpine
+  - cloudamqp/lavinmq:latest
+  - mesh-control:development
+  - redis:7-alpine
+  # Redis's tool+provisioner runtime, built by scripts/build-module-runtime.sh redis into the local
+  # daemon and stocked into the scenario's own registry, which is where the host pulls it from.
+  - mesh-runtime-redis:development
+
+place:
+  all: [host, runtime]

scenarios/sonarr-node.yml

@@ -0,0 +1,28 @@
+# One machine that becomes a mesh and assigns itself sonarr's tool runtime.
+#
+# plex-node proved a tools+events module that self-detects its token from a mounted config dir; this
+# proves the same self-configuring pattern generalises to the Servarr family (novox/hq ADR 0052):
+# sonarr's runtime detects its API key from the server's config.xml and serves sonarr's tools over a
+# mesh-issued scoped account, with no live Sonarr to reach.
+scenario: sonarr-node
+
+segments:
+  hosting:
+    kind: public
+    cidr: [192.0.2.0/24]
+
+machines:
+  anchor:
+    at: { segment: hosting, address: [192.0.2.10] }
+    inbound: allow
+    memory: 3GiB
+    cpus: 2
+
+images:
+  - postgres:17-alpine
+  - cloudamqp/lavinmq:latest
+  - mesh-control:development
+  - mesh-runtime-sonarr:development
+
+place:
+  all: [host, runtime]

scenarios/two-nodes.yml

@@ -16,14 +16,48 @@ machines:
   anchor:
     at: { segment: hosting, address: [192.0.2.10] }
     inbound: allow
+    # The whole substrate, the registry, the builder, an adopted workload and the modules under
+    # test all land here — eleven containers before the forge arrives. At the 1GiB default this
+    # machine thrashes, and it presents as "the mesh hangs": every exec slows from 15s to 105s
+    # and the forge test fails on a status poll that is merely queued behind page-outs.
+    memory: 4GiB
+    cpus: 4
   laptop:
     at: { segment: hosting, address: [192.0.2.20] }
     inbound: allow
+    memory: 2GiB
 
 images:
   - postgres:17-alpine
   - cloudamqp/lavinmq:latest
   - mesh-control:development
+  # So a module can mirror one into a registry of the mesh's own. The scenario's registry serves
+  # what the mesh's registry is built from — the same chicken-and-egg the bootstrap has, resolved
+  # the same way.
+  - registry:2
+  # A real third-party workload, for adopting one the way the conversion will. Its database is
+  # the substrate's postgres image rather than its own: what is under test is the mesh delivering
+  # a module, not which postgres it delivers.
+  - ghcr.io/umami-software/umami:postgresql-latest
+  # And the builder, because it is a module the mesh assigns rather than a program somebody
+  # starts by hand — which is the only way its credential can be one the mesh delivered.
+  - mesh-builder:development
+  # And the provisioner, which is what makes a sealed credential true on a machine — the mesh
+  # discarded the plaintext and cannot tell a database to start accepting it.
+  - mesh-provision-postgres:development
+  # And the proxy, which is what turns a route grant into traffic actually arriving.
+  - mesh-route-proxy:development
+  # And the cache, with its provisioner — the third provision after a database and a bucket,
+  # and the first whose tenancy is a keyspace rather than a namespace something else enforces.
+  - redis:7-alpine
+  - mesh-provision-redis:development
+  # And the object store's provisioner, so the module describing it can be planned. Without it
+  # that module still names an image nothing serves, and planning it is refused — correctly.
+  - mesh-provision-objectstore:development
+  # And a forge, so one of the real module descriptions can be started rather than only planned.
+  # It is the first of them to run: it needs a database from another module, a credential it did
+  # not choose, and a connection string it could not have written itself.
+  - gitea/gitea:1.22
 
 place:
   all: [host, runtime]

scripts/build-module-runtime.sh

@@ -0,0 +1,59 @@
+#!/usr/bin/env bash
+# Build a per-module runtime image (novox/hq ADR 0052): the tool runtime carrying ONE module's
+# compiled code, which serves that module's tools and runs its events/provisioner under the module's
+# own scoped broker account. Generalises build-runtime-image.sh from the audit-logger to any module.
+#
+#   build-module-runtime.sh <module> <output.tar>
+#   -> tags mesh-runtime-<module>:development and saves it to <output.tar>
+set -euo pipefail
+
+MODULE="${1:?usage: build-module-runtime.sh <module> <output.tar>}"
+OUT="${2:?usage: build-module-runtime.sh <module> <output.tar>}"
+HERE="$(cd "$(dirname "$0")/.." && pwd)"; ROOT="$(cd "$HERE/.." && pwd)"
+MESH_TOOLS="${MESH_TOOLS:-$ROOT/mesh-tools}"
+MESH_SDK="${MESH_SDK:-$ROOT/mesh-sdk}"
+MESH_CATALOG="${MESH_CATALOG:-$ROOT/mesh-catalog}"
+MOD="$MESH_CATALOG/modules/$MODULE"
+TAG="${RUNTIME_TAG:-mesh-runtime-$MODULE:development}"
+BASE="${RUNTIME_BASE:-node:22-bookworm-slim}"
+[ -d "$MOD" ] || { echo "no module $MODULE at $MOD" >&2; exit 1; }
+
+( cd "$MESH_SDK" && npm run build >/dev/null )
+( cd "$MESH_TOOLS" && npm run build >/dev/null )
+# Compile whichever of the module's entrypoints exist.
+SRCS=(); for f in client.ts index.ts tools/index.ts provisioner/index.ts; do [ -f "$MOD/$f" ] && SRCS+=("$f"); done
+TSC="$MESH_SDK/node_modules/.bin/tsc"; ( cd "$MOD" && "$TSC" "${SRCS[@]}" --module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist >/dev/null )
+
+STAGE="$(mktemp -d)"; trap 'rm -rf "$STAGE"' EXIT
+cp -r "$MESH_TOOLS/dist" "$STAGE/dist"
+cp -rL "$MESH_TOOLS/node_modules" "$STAGE/node_modules"
+mkdir -p "$STAGE/modules/$MODULE"; cp -r "$MOD/dist" "$STAGE/modules/$MODULE/dist"
+cp "$MESH_TOOLS/package.json" "$STAGE/package.json"
+
+# The entrypoints the runtime loads: tools, events and (a provider's) provisioner, whichever exist.
+ENTRIES=""; for e in tools/index.js index.js provisioner/index.js; do
+  [ -f "$STAGE/modules/$MODULE/dist/$e" ] && ENTRIES="${ENTRIES:+$ENTRIES,}/app/modules/$MODULE/dist/$e"
+done
+
+# A module whose code drives a CLI needs that CLI in the image — postgres shells out to `psql`, minio
+# to `mc`. Everything else speaks a wire protocol or HTTP and needs nothing added.
+EXTRA=""
+case "$MODULE" in
+  postgres) EXTRA='RUN apt-get update && apt-get install -y --no-install-recommends postgresql-client && rm -rf /var/lib/apt/lists/*' ;;
+  minio)    EXTRA='COPY --from=minio/mc:latest /usr/bin/mc /usr/bin/mc' ;;
+esac
+
+cat > "$STAGE/Dockerfile" <<DOCKER
+FROM $BASE
+WORKDIR /app
+$EXTRA
+COPY package.json ./
+COPY node_modules ./node_modules
+COPY dist ./dist
+COPY modules ./modules
+ENV MESH_TOOL_MODULES=$ENTRIES
+ENTRYPOINT ["node", "dist/main.js"]
+DOCKER
+docker build -t "$TAG" "$STAGE"
+docker save -o "$OUT" "$TAG"
+echo "built $TAG (entrypoints: $ENTRIES) -> $OUT"

scripts/build-runtime-image.sh

@@ -0,0 +1,57 @@
+#!/usr/bin/env bash
+# Build the runtime+audit-logger image the events test runs, and save it to a tar.
+#
+# The image is the tier-3 tool runtime (mesh-tools) carrying one tier-4 module (audit-logger) and
+# the sdk it imports. It is what MESH_LAB_RUNTIME points at:
+#
+#   scripts/build-runtime-image.sh /tmp/mesh-runtime-audit.tar
+#   MESH_LAB_RUNTIME=/tmp/mesh-runtime-audit.tar  node --test test/integration/events.test.ts
+#
+# The sdk is vendored (dereferenced), not npm-installed: the sdk is not published, and the lab
+# machine has no route out anyway — the image must be self-contained. Sibling repositories are
+# assumed alongside this one; override with MESH_TOOLS / MESH_SDK / MESH_CATALOG.
+set -euo pipefail
+
+OUT="${1:?usage: build-runtime-image.sh <output.tar>}"
+HERE="$(cd "$(dirname "$0")/.." && pwd)"
+ROOT="$(cd "$HERE/.." && pwd)"
+MESH_TOOLS="${MESH_TOOLS:-$ROOT/mesh-tools}"
+MESH_SDK="${MESH_SDK:-$ROOT/mesh-sdk}"
+MESH_CATALOG="${MESH_CATALOG:-$ROOT/mesh-catalog}"
+AUDIT="$MESH_CATALOG/modules/audit-logger"
+TAG="${RUNTIME_TAG:-mesh-runtime-audit:development}"
+BASE="${RUNTIME_BASE:-node:22-bookworm-slim}"
+
+echo "building $TAG from:"
+echo "  runtime  $MESH_TOOLS"
+echo "  sdk      $MESH_SDK"
+echo "  module   $AUDIT"
+
+# Compile the three, so the image carries current dist. The sdk first — the others import it.
+( cd "$MESH_SDK" && npm run build >/dev/null )
+( cd "$MESH_TOOLS" && npm run build >/dev/null )
+( cd "$AUDIT" && npx tsc audit.ts index.ts --module NodeNext --moduleResolution NodeNext \
+    --target ES2022 --outDir dist >/dev/null )
+
+STAGE="$(mktemp -d)"
+trap 'rm -rf "$STAGE"' EXIT
+cp -r "$MESH_TOOLS/dist" "$STAGE/dist"
+cp -rL "$MESH_TOOLS/node_modules" "$STAGE/node_modules"   # -L materialises the @novox/mesh-sdk symlink
+mkdir -p "$STAGE/modules/audit-logger"
+cp -r "$AUDIT/dist" "$STAGE/modules/audit-logger/dist"
+cp "$MESH_TOOLS/package.json" "$STAGE/package.json"
+
+cat > "$STAGE/Dockerfile" <<DOCKER
+FROM $BASE
+WORKDIR /app
+COPY package.json ./
+COPY node_modules ./node_modules
+COPY dist ./dist
+COPY modules ./modules
+ENV MESH_TOOL_MODULES=/app/modules/audit-logger/dist/index.js
+ENTRYPOINT ["node", "dist/main.js"]
+DOCKER
+
+docker build -t "$TAG" "$STAGE"
+docker save -o "$OUT" "$TAG"
+echo "saved $TAG -> $OUT"

src/cli.ts

@@ -30,6 +30,19 @@ const USAGE = `mesh-lab — raise a disposable mesh on one machine
   diagram <scenario.yml> [out.drawio]      draw what a scenario asks for
   diagram --live <instance> [out.drawio]   draw what is actually raised
 
+  warm                           the scenario kept between runs, and whether it still counts
+  warm cool                      destroy it and forget it
+  suite [paths...] [--no-build]  rebuild the artifacts, run the end-to-end tests, leave a receipt
+  last-run                       whether the last run still counts; non-zero when it does not
+
+  connect [instance]             reach the standing scenario from this workstation, by name
+  disconnect                     give the address back and stop answering those names
+  connected                      what is reachable right now
+
+A scenario is a closed address space, so only one can be reachable at a time: connect refuses
+rather than guessing which you meant. It needs root for an address and a resolver rule, and
+disconnect puts both back.
+
 Set MESH_LAB_INCUS if the daemon needs a different invocation, e.g. "sudo -n incus".
 `;
 
@@ -91,6 +104,63 @@ async function main(): Promise<void> {
     case "check":
       return check();
 
+    // Rebuilding, running, and recording that it ran are one act. Separate commands would mean a
+    // run against a stale artifact, or a run nobody recorded — and both are the state
+    // novox/hq 04-ISSUES/005 is about.
+    case "suite": {
+      const { runSuite } = await import("./suite.ts");
+      process.exitCode = await runSuite(rest);
+      return;
+    }
+
+    // **What 04-ISSUES/005 says nobody was ever told.** The suite needs a machine with a
+    // hypervisor, so it cannot run on every push — which means it runs when somebody remembers,
+    // and remembering is not a mechanism. This asks whether the last run still means anything,
+    // and exits non-zero when it does not, so a timer or a person can act on it.
+    case "last-run": {
+      const { read, judge, whatWasTested } = await import("./lastrun.ts");
+      const said = judge(read(), new Date(), whatWasTested());
+      for (const line of said.lines) console.log(line);
+      if (!said.current) {
+        console.log("\n  `npm run check` runs it.");
+        process.exitCode = 1;
+      }
+      return;
+    }
+
+    // A base state many tests start from, rather than each raising its own mesh.
+    //
+    // **The speed is the lesser half.** Tests that share one long-lived mesh accumulate each
+    // other's state, and a test that reads what the previous one left is a test that passes for
+    // the wrong reason — which has already happened here once. Returning to a named state between
+    // tests makes each of them independent.
+    case "warm": {
+      const { remembered, ready, cool } = await import("./warm.ts");
+      const what = rest[0] ?? "status";
+      if (what === "cool") {
+        const gone = await cool();
+        console.log(gone ? `destroyed ${gone}, and forgot it` : "nothing was being kept warm");
+        return;
+      }
+      const held = remembered();
+      if (!held) {
+        console.log("nothing is being kept warm.");
+        console.log("  a scenario is warmed by whatever brought it to a state worth keeping;");
+        console.log("  the integration suite does it when MESH_LAB_WARM is set.");
+        return;
+      }
+      console.log(`${held.instanceId} — ${held.scenario}, warmed ${held.at}`);
+      for (const [name, commit] of Object.entries(held.against)) {
+        console.log(`  ${name.padEnd(14)} ${commit}`);
+      }
+      const said = await ready(held.scenario);
+      console.log(said.use === "restore"
+        ? "\n  usable: it can be returned to"
+        : `\n  NOT usable: ${said.why}`);
+      if (said.use !== "restore") process.exitCode = 1;
+      return;
+    }
+
     case "base": {
       // `base build` exists because a sealed scenario cannot install a container runtime, and
       // the runtime has to come from somewhere with a network (novox/hq ADR 0006).
@@ -120,6 +190,38 @@ async function main(): Promise<void> {
       return;
     }
 
+    case "connect": {
+      const { connect } = await import("./lifecycle/connect.ts");
+      const reached = await connect(rest[0]);
+      console.log(`connected to ${reached.instanceId} as ${reached.address} on ${reached.bridge}\n`);
+      console.log("these answer here now:");
+      for (const [machine, address] of Object.entries(reached.machines).sort()) {
+        console.log(`  anything.${machine}.internal  →  ${address}`);
+      }
+      console.log(`\ntry:  curl -sI http://${Object.keys(reached.machines)[0]}.internal`);
+      console.log("run `mesh-lab disconnect` when finished — these names are only true while");
+      console.log("that scenario is standing.");
+      return;
+    }
+
+    case "disconnect": {
+      const { disconnect } = await import("./lifecycle/connect.ts");
+      for (const line of await disconnect()) console.log(line);
+      return;
+    }
+
+    case "connected": {
+      const { connection } = await import("./lifecycle/connect.ts");
+      const now = await connection();
+      if (now.length === 0) {
+        console.log("nothing is connected");
+        process.exitCode = 1;
+        return;
+      }
+      for (const line of now) console.log(`  ${line}`);
+      return;
+    }
+
     case "raise": {
       const path = rest[0] ?? fail("raise needs a scenario file");
       const scenario = loadScenario(path);

src/declaration/parse.ts

@@ -37,6 +37,9 @@ function normaliseMachine(raw: unknown): Machine {
   }
   const inbound = machine["inbound"];
   if (inbound === "allow" || inbound === "deny") result.inbound = inbound;
+  if (machine["egress"] === true) result.egress = true;
+  if (machine["memory"] !== undefined) result.memory = String(machine["memory"]);
+  if (machine["cpus"] !== undefined) result.cpus = Number(machine["cpus"]);
   return result;
 }
 

src/declaration/types.ts

@@ -85,6 +85,31 @@ export interface Machine {
    * v6-addressed machine. Without this, v6 addressing would imply reachability.
    */
   inbound?: "allow" | "deny";
+  /**
+   * Whether this machine can reach the world outside the scenario.
+   *
+   * **Off unless asked for.** A scenario is a closed address space, and a machine that could
+   * reach anything would make every test's result depend on what else was reachable that day.
+   * It is declared for the same reason an address is: so what a run proves is what the scenario
+   * says, and not what the workstation happened to have.
+   *
+   * What it is for is the one thing a mesh genuinely cannot do without an outside: a first node
+   * fetching the images it starts from, before there is any mesh to serve them
+   * (novox/hq 04-ISSUES/029).
+   */
+  egress?: boolean;
+  /**
+   * How big the machine is. Absent means the lab's default, which suits a machine running a host
+   * and a handful of containers.
+   *
+   * Declared, because it is a fact about the machine the scenario describes — the node that runs
+   * the whole substrate is bigger than the laptop that joins it, and a test that starves its
+   * anchor at the default answers questions about memory pressure, not about the mesh. The forge
+   * test failed three times as "status hangs" before anyone counted the containers in 1GiB
+   * (novox/hq 04-ISSUES/024 is the same lesson about a different resource).
+   */
+  memory?: string;
+  cpus?: number;
 }
 
 /** Reachability between segments, as a segmented router enforces it. Asymmetric by design. */

src/declaration/validate.ts

@@ -199,11 +199,27 @@ export function validate(scenario: Scenario): void {
     }
   }
 
+  // "uplink" is the one network name the lab itself claims, for the NAT bridge behind
+  // `egress: true`. A segment wearing it would be created first, as an isolated bridge — and the
+  // uplink code, finding a network by that name, would attach egress machines to it. No error
+  // anywhere, no route anywhere: a scenario key silently ignored, which is the fault this file
+  // exists to refuse.
+  if (scenario.segments["uplink"]) {
+    problems.push(`segment 'uplink': the name is reserved for the lab's own NAT bridge — ` +
+      `an egress machine would be silently attached to this segment instead of the world`);
+  }
+
   for (const [name, machine] of Object.entries(scenario.machines)) {
     if (machine.at === "detached") {
       if (machine.published?.length) {
         problems.push(`machine '${name}' is detached but declares published ports`);
       }
+      // The same fault as publishing from nowhere: raising it would drop the key on the floor,
+      // and a scenario key the runtime silently ignores is the thing this lab exists to catch.
+      if (machine.egress) {
+        problems.push(`machine '${name}' is detached but declares egress — a machine on no ` +
+          `segment reaches nothing, the world included`);
+      }
       continue;
     }
 

src/incus/client.ts

@@ -13,6 +13,8 @@
 
 import { spawn } from "node:child_process";
 
+import { around, log, shorten } from "../log.ts";
+
 /**
  * How to invoke incus. Overridable because the socket is group-owned and a session that
  * predates the group grant cannot reach it — which is a real thing that happens on the
@@ -52,6 +54,27 @@ export class IncusError extends Error {
  * that worked perfectly when typed.
  */
 export async function incus(args: string[], timeoutMs = 60_000): Promise<IncusResult> {
+  // **Every command through here is recorded** (novox/hq 04-ISSUES/024). This is one of three
+  // places the lab runs an external program, and ninety-odd call sites reach a hypervisor through
+  // it — so logging here covers all of them and none of them has to remember to.
+  return invoke(args, timeoutMs, false);
+}
+
+/**
+ * `expectedToFail` is not about this command; it is about the caller.
+ *
+ * `incus` rejects and the caller is expected to care. `incusOk` and `succeeds` turn a failure into
+ * an answer — *does this network exist*, *is the agent up yet* — and those are asked constantly
+ * while a scenario comes up. Recording them as faults fills a healthy run with ✗.
+ */
+function invoke(args: string[], timeoutMs: number, expectedToFail: boolean): Promise<IncusResult> {
+  return around(`incus ${shorten(args)}`, () => run(args, timeoutMs), {
+    heartbeatMs: 15_000,
+    expectedToFail,
+  });
+}
+
+function run(args: string[], timeoutMs: number): Promise<IncusResult> {
   const [command, ...prefix] = INCUS;
   if (!command) throw new Error("MESH_LAB_INCUS is empty");
 
@@ -80,10 +103,15 @@ export async function incus(args: string[], timeoutMs = 60_000): Promise<IncusRe
     child.on("close", (code) => {
       clearTimeout(timer);
       if (timedOut) {
+        // Said explicitly. A SIGKILL leaves an empty stderr, so without this the failure arrives
+        // with no explanation at all — which is how the first raise reported "(no output)".
+        log.info(`incus ${shorten(args)} was killed after ${timeoutMs}ms`);
         reject(new IncusError(args, `timed out after ${timeoutMs}ms`, null));
       } else if (code === 0) {
+        if (stdout.trim()) log.trace(`  stdout: ${shorten([stdout.trim()], 400)}`);
         resolve({ stdout, stderr });
       } else {
+        log.debug(`  exit ${code}: ${shorten([stderr.trim() || "(nothing on stderr)"], 400)}`);
         reject(new IncusError(args, stderr, code));
       }
     });
@@ -105,7 +133,7 @@ export async function incus(args: string[], timeoutMs = 60_000): Promise<IncusRe
  */
 export async function incusOk(args: string[], timeoutMs = 60_000): Promise<string | null> {
   try {
-    return (await incus(args, timeoutMs)).stdout;
+    return (await invoke(args, timeoutMs, true)).stdout;
   } catch {
     return null;
   }

src/lastrun.ts

@@ -0,0 +1,213 @@
+/**
+ * When the end-to-end suite last ran, and against what.
+ *
+ * **The fault this exists for is not that the suite breaks — it is that nobody notices it stopped
+ * running** (novox/hq 04-ISSUES/005). The harness it replaces had not built for two and a half
+ * months, and nothing said so; the coverage was assumed rather than checked, and several of the
+ * pipeline's most expensive defects landed inside that window.
+ *
+ * This suite is in a better position and the same danger: it needs a machine with a hypervisor, so
+ * it cannot run on every push, which means it runs when somebody remembers. Remembering is not a
+ * mechanism.
+ *
+ * So a run leaves a receipt, and something can be asked whether the receipt still means anything.
+ * A receipt that is old, or taken against code the repositories have since moved past, is the
+ * thing 005 says nobody was ever told.
+ *
+ * **Kept outside the repository**, because the question is *has this machine run it* rather than
+ * *what is committed* — and a receipt in git would be a claim about everyone's machine made by
+ * whoever committed last.
+ */
+
+import { execFileSync } from "node:child_process";
+import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
+import { homedir } from "node:os";
+import { dirname, join } from "node:path";
+
+import { repositories } from "./repos.ts";
+
+/** What a run was taken against, per repository. */
+export type Against = Record<string, string>;
+
+export interface Receipt {
+  /** When it finished, ISO 8601. */
+  at: string;
+  passed: number;
+  failed: number;
+  /** The commit each repository was at. Absent for anything that was not a git checkout. */
+  against: Against;
+  /** The test files this run was pointed at. See {@link endToEnd}. */
+  ran: string[];
+}
+
+/**
+ * endToEnd is the file that raises real machines. A run that did not include it proved nothing
+ * about the pipeline, however green it was.
+ *
+ * The suite takes paths, so it can be pointed at one quick file — and the receipt from that would
+ * otherwise be indistinguishable from a receipt for the real thing. That is 04-ISSUES/005 again:
+ * not a suite that fails, a record that says more than the run behind it.
+ */
+export const endToEnd = "test/integration/mesh.test.ts";
+
+/** Where the receipt lives: XDG state, which is for exactly this — data a tool keeps between runs. */
+export function receiptPath(): string {
+  const state = process.env["XDG_STATE_HOME"] ?? join(homedir(), ".local", "state");
+  return join(state, "mesh-lab", "last-run.json");
+}
+
+/**
+ * headOf is the commit a directory's repository is at, or "" if it is not one.
+ *
+ * **A tree with uncommitted changes is marked, and never equal to the clean commit it sits on.**
+ * The run tested what was on disk, and that is not what the commit contains — so a receipt naming
+ * the bare hash would claim coverage of code nobody can check out. Nothing else could tell: the
+ * hash is identical either way. That is 04-ISSUES/005's overclaim in its quietest form.
+ */
+export function headOf(directory: string): string {
+  const git = (args: string[]) =>
+    execFileSync("git", ["-C", directory, ...args], {
+      encoding: "utf8",
+      stdio: ["ignore", "pipe", "ignore"],
+    });
+  try {
+    const head = git(["rev-parse", "--short", "HEAD"]).trim();
+    const dirty = git(["status", "--porcelain"]).trim() !== "";
+    return dirty ? `${head}+uncommitted` : head;
+  } catch {
+    // Not a checkout, or no git. Absent rather than guessed: a receipt claiming a commit it did
+    // not read is worse than one that says it could not tell.
+    return "";
+  }
+}
+
+/**
+ * whatWasTested is the repositories this run exercised, by the paths it was given.
+ *
+ * From the environment rather than a fixed list, because the paths are how the suite is told what
+ * to run — so anything it was pointed at is something the receipt should account for, and anything
+ * it was not pointed at was not tested.
+ */
+export function whatWasTested(env: NodeJS.ProcessEnv = process.env): Against {
+  const against: Against = {};
+  for (const [name, directory] of Object.entries(repositories(env))) {
+    const head = headOf(directory);
+    if (head) against[name] = head;
+  }
+  return against;
+}
+
+/** record writes the receipt. Failures are recorded too: a run that failed still ran. */
+export function record(
+  passed: number,
+  failed: number,
+  ran: string[],
+  env = process.env,
+  builtAgainst?: Against,
+): Receipt {
+  const receipt: Receipt = {
+    at: new Date().toISOString(),
+    passed,
+    failed,
+    // What was BUILT, when the caller says — not what the repositories are at when the run ends.
+    // A receipt read at record time names whatever was committed during the twenty minutes the
+    // suite took, and it did: one run's receipt claimed a commit that landed mid-run and was
+    // never in the binaries. A verdict is only worth something attributed to one exact state.
+    against: builtAgainst ?? whatWasTested(env),
+    ran,
+  };
+  const path = receiptPath();
+  mkdirSync(dirname(path), { recursive: true });
+  writeFileSync(path, JSON.stringify(receipt, null, 2) + "\n");
+  return receipt;
+}
+
+/** read returns the receipt, or null when this machine has never run the suite. */
+export function read(): Receipt | null {
+  try {
+    return JSON.parse(readFileSync(receiptPath(), "utf8")) as Receipt;
+  } catch {
+    return null;
+  }
+}
+
+export interface Verdict {
+  /** True when the receipt still says something about the code as it stands. */
+  current: boolean;
+  lines: string[];
+}
+
+/**
+ * judge says whether the last run still means anything.
+ *
+ * Two ways it can stop meaning something, and they read differently: it was long ago, or the code
+ * has moved since. The second is the one that matters — a suite that passed against code nobody
+ * runs any more is coverage in name.
+ */
+export function judge(
+  receipt: Receipt | null,
+  now: Date,
+  against: Against,
+  staleAfterDays = 7,
+): Verdict {
+  if (!receipt) {
+    return {
+      current: false,
+      lines: [
+        "this machine has never run the end-to-end suite.",
+        "  Nothing here has been checked end to end, which is not the same as nothing being wrong.",
+      ],
+    };
+  }
+
+  // A receipt written before this field existed says nothing about what it ran, and the honest
+  // reading of "nothing said" is not "everything".
+  const endToEndRan = (receipt.ran ?? []).some((path) => path.endsWith(endToEnd));
+
+  const days = (now.getTime() - Date.parse(receipt.at)) / 86_400_000;
+  const lines: string[] = [];
+  const outcome = receipt.failed > 0
+    ? `last ran ${ago(days)} and ${receipt.failed} test(s) failed`
+    : `last passed ${ago(days)}, ${receipt.passed} test(s)`;
+  lines.push(`the end-to-end suite ${outcome}`);
+
+  let moved = false;
+  for (const name of Object.keys(against).sort()) {
+    const then = receipt.against[name];
+    const now = against[name];
+    if (!then) {
+      lines.push(`  ${name.padEnd(14)} was not accounted for in that run`);
+      moved = true;
+      continue;
+    }
+    if (then === now) {
+      lines.push(`  ${name.padEnd(14)} at ${then} (unchanged)`);
+      continue;
+    }
+    lines.push(`  ${name.padEnd(14)} at ${then}, now at ${now}`);
+    moved = true;
+  }
+
+  if (!endToEndRan) {
+    lines.push(`  That run did not include ${endToEnd}, so it raised no machines.`);
+  }
+
+  if (receipt.failed > 0) {
+    lines.push("  Nothing has been proven end to end since.");
+  } else if (moved) {
+    lines.push("  What it proved was proven about code that has since changed.");
+  } else if (days > staleAfterDays) {
+    lines.push(`  Nothing has changed since, but that was more than ${staleAfterDays} days ago.`);
+  }
+
+  return {
+    current: endToEndRan && receipt.failed === 0 && !moved && days <= staleAfterDays,
+    lines,
+  };
+}
+
+function ago(days: number): string {
+  if (days < 1 / 24) return "less than an hour ago";
+  if (days < 1) return `${Math.round(days * 24)} hour(s) ago`;
+  return `${Math.round(days)} day(s) ago`;
+}

src/lifecycle/address.ts

@@ -23,6 +23,15 @@ export interface Wire {
   mac: string;
   addresses: string[];
   mtu: number | undefined;
+  /**
+   * Ask the host for an address rather than declaring one.
+   *
+   * **Only the uplink**, and it is the one address a scenario has no business choosing: it is on
+   * the machine's side of the host's own network, and what is routable there is the host's fact,
+   * not the declaration's. Every segment a scenario describes is still static, for the reason
+   * below.
+   */
+  dhcp?: boolean;
 }
 
 /**
@@ -31,6 +40,27 @@ export interface Wire {
  * the declaration is supposed to own.
  */
 function networkUnit(wire: Wire): string {
+  if (wire.dhcp) {
+    return [
+      "[Match]",
+      `MACAddress=${wire.mac}`,
+      "",
+      "[Network]",
+      "DHCP=ipv4",
+      "IPv6AcceptRA=no",
+      "",
+      "[DHCPv4]",
+      // UseDNS matters only where systemd-resolved runs; on a machine whose mesh modules own
+      // /etc/resolv.conf it is inert, and that inertness is load-bearing — the uplink must not
+      // outvote the resolver a scenario is testing.
+      // **Worse than any route the scenario states.** A machine behind a declared gateway must
+      // keep using it: the uplink is a way out of the scenario, not a better way around inside
+      // it. On-link segments win regardless, being connected routes; this only settles which
+      // default route is preferred when a scenario declares one of its own.
+      "RouteMetric=4096",
+      "UseDNS=yes",
+    ].join("\n") + "\n";
+  }
   const lines = [
     "[Match]",
     `MACAddress=${wire.mac}`,
@@ -47,6 +77,36 @@ function networkUnit(wire: Wire): string {
   return lines.join("\n") + "\n";
 }
 
+/**
+ * Give one link a static address through systemd-networkd, and wait until networkd says it is
+ * configured.
+ *
+ * **Not `ip addr add`**, which is what this replaced and what cost a lab that could not finish.
+ * An address set by hand leaves the link `configuring` for ever, because networkd is still
+ * waiting to configure something it was never told about. `systemd-networkd-wait-online` then
+ * never returns — its timeout is `infinity` — so `network-online.target` is never reached, and
+ * **anything ordered after it never starts**. On these machines that is Docker, which meant
+ * `docker load` blocked on a socket whose daemon was queued behind a target that would never
+ * come. The lab stalled for thirty-five minutes with nothing to say.
+ *
+ * Every machine already did it this way. The registry did not, and it was the only one that
+ * needed Docker before anything else ran.
+ */
+export async function addressLink(
+  instanceName: string,
+  wire: Wire,
+  index = 0,
+): Promise<void> {
+  const unit = networkUnit(wire);
+  await incus(
+    ["exec", instanceName, "--", "sh", "-c",
+     `mkdir -p /etc/systemd/network && cat > /etc/systemd/network/10-mlab-${index}.network <<'MLAB'\n${unit}MLAB`],
+    30_000,
+  );
+  await incus(["exec", instanceName, "--", "systemctl", "enable", "--now", "systemd-networkd"], 60_000);
+  await incus(["exec", instanceName, "--", "systemctl", "restart", "systemd-networkd"], 60_000);
+}
+
 export async function applyAddresses(
   scenario: Scenario,
   instanceId: string,
@@ -68,6 +128,16 @@ export async function applyAddresses(
       });
     }
 
+    if (spec.egress) {
+      wires.push({
+        device: `eth${spec.at.length}`,
+        mac: macFor(instanceId, machine, spec.at.length),
+        addresses: [],
+        mtu: undefined,
+        dhcp: true,
+      });
+    }
+
     for (const [index, wire] of wires.entries()) {
       const unit = networkUnit(wire);
       await incus(
@@ -79,7 +149,7 @@ export async function applyAddresses(
 
     await incus(["exec", name, "--", "systemctl", "enable", "--now", "systemd-networkd"], 60_000);
     await incus(["exec", name, "--", "systemctl", "restart", "systemd-networkd"], 60_000);
-    log(`  addressed ${machine} (${wires.map((w) => w.addresses.join(",")).join(" | ")})`);
+    log(`  addressed ${machine} (${wires.map((w) => w.dhcp ? "uplink:dhcp" : w.addresses.join(",")).join(" | ")})`);
   }
 }
 

src/lifecycle/base.ts

@@ -71,6 +71,20 @@ export async function buildBaseImage(
     log("  installing git, so a machine can build modules");
     await incus(["exec", BUILDER, "--", "pacman", "-S", "--noconfirm", "git"], 600_000);
 
+    // And nftables, because the mesh computes a machine's filtering and delivers it as a file
+    // that a service reflects — and neither the file nor the service can install what loads it.
+    // Installed and NOT enabled: whether a machine filters is the mesh's decision, and a lab that
+    // turned it on itself would be testing its own setup.
+    log("  installing nftables, so a machine can enforce what the mesh computed");
+    await incus(["exec", BUILDER, "--", "pacman", "-S", "--noconfirm", "nftables"], 600_000);
+
+    // And dnsmasq, because a service is named under the machine it runs on — postgres.novox.internal
+    // — and only a resolver can answer a name the mesh was never told about. Installed and NOT
+    // started: whether a machine resolves for the mesh is the mesh's decision, and a lab that
+    // turned it on itself would be testing its own setup.
+    log("  installing dnsmasq, so a machine can answer names under another machine");
+    await incus(["exec", BUILDER, "--", "pacman", "-S", "--noconfirm", "dnsmasq"], 600_000);
+
     // Trust the documentation ranges as plain-HTTP registries.
     //
     // A scenario's registry is scenery inside the scenario, serving over HTTP, and a runtime
@@ -108,6 +122,29 @@ export async function buildBaseImage(
     }
     log(`  ${git.trim()}`);
 
+    // The same again, for nftables. A machine that cannot load a rule set applies the mesh's
+    // filtering, reports success, and filters nothing — which is precisely the fault the whole
+    // derivation exists to remove, reappearing in the lab.
+    const nft = await incusOk(["exec", BUILDER, "--", "nft", "--version"], 60_000);
+    if (!nft?.trim()) {
+      throw new BaseImageError(
+        `nftables was installed in ${BUILDER} and \`nft\` does not answer. Publishing this would ` +
+          `give every scenario a machine that cannot enforce what the mesh computed for it.`,
+      );
+    }
+    log(`  ${nft.trim()}`);
+
+    // The same again for dnsmasq. A machine that cannot answer names applies the mesh's resolver
+    // data, reports success, and resolves nothing — the shape of fault this lab exists to catch.
+    const dns = await incusOk(["exec", BUILDER, "--", "dnsmasq", "--version"], 60_000);
+    if (!dns?.trim()) {
+      throw new BaseImageError(
+        `dnsmasq was installed in ${BUILDER} and does not answer. Publishing this would give ` +
+          `every scenario a machine that cannot resolve a name under another machine.`,
+      );
+    }
+    log(`  ${dns.trim().split("\n")[0]}`);
+
     // Read back from the runtime, not from the package manager. An installed package is not a
     // capability (novox/hq 04-ISSUES/007), and this is the one place to catch that — after
     // publishing, every scenario pays for it instead.

src/lifecycle/connect.ts

@@ -0,0 +1,196 @@
+/**
+ * Letting the workstation reach one scenario by name, on purpose and temporarily.
+ *
+ * **A scenario is a closed address space** ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)): two
+ * raised from the same declaration hold the same addresses and never meet, because nothing joins
+ * their links. That is what lets two identical scenarios run at once, and it is why the lab talks
+ * to machines through the hypervisor's own channel rather than over IP.
+ *
+ * Reaching in from the workstation breaks that, so it is **opt-in, one scenario at a time, and
+ * reversible**. It refuses when more than one is standing rather than guessing which was meant —
+ * the failure it exists to avoid is not an error but one scenario's traffic arriving in another.
+ *
+ * **Names resolve to the segment address, not the overlay one.** Inside the mesh a name answers
+ * with a machine's private-network address; from here that would need the workstation on the
+ * overlay, which is a much larger door to open. The segment address reaches the same machine and
+ * the same ports, which is what somebody opening a board in a browser actually needs.
+ */
+
+import { writeFileSync, unlinkSync, existsSync, readFileSync } from "node:fs";
+import { spawnSync } from "node:child_process";
+
+import { incus, incusOk, taggedInstances, taggedNetworks } from "../incus/client.ts";
+import { log } from "../log.ts";
+
+/** Where the rule goes. Its own file — the one beside it belongs to something else. */
+export const RULE = "/etc/dnsmasq.d/mesh-lab.conf";
+
+export interface Reached {
+  instanceId: string;
+  bridge: string;
+  /** The address the workstation took on that link. */
+  address: string;
+  /** Machine name to the address its names now answer with. */
+  machines: Record<string, string>;
+}
+
+export class ConnectError extends Error {}
+
+/** Run something as root, and say plainly when that is what failed. */
+function asRoot(argv: string[]): { ok: boolean; said: string } {
+  const ran = spawnSync("sudo", ["-n", ...argv], { encoding: "utf8" });
+  const said = `${ran.stdout ?? ""}${ran.stderr ?? ""}`.trim();
+  if (ran.status !== 0 && /password|not allowed|no tty/i.test(said)) {
+    throw new ConnectError(
+      `this needs root and sudo asked for a password, which there is nowhere to type here.\n` +
+        `  Run it yourself:  sudo ${argv.join(" ")}`);
+  }
+  return { ok: ran.status === 0, said };
+}
+
+/** The one instance standing, or a refusal naming what it found instead. */
+export async function theOnlyInstance(): Promise<string> {
+  const ids = [...new Set((await taggedInstances()).map((i) => i.instanceId))].sort();
+  if (ids.length === 1) return ids[0]!;
+  if (ids.length === 0) {
+    throw new ConnectError("no scenario is standing, so there is nothing to reach.");
+  }
+  throw new ConnectError(
+    `${ids.length} scenarios are standing and they may hold the same addresses, so there is no ` +
+      `answer to which one you meant: ${ids.join(", ")}.\n` +
+      `  Take the others down, or name one — but only one can be reachable at a time.`);
+}
+
+/** Every machine in an instance, with the address it has on the given link. */
+async function addressesOn(instanceId: string, segment: string): Promise<Record<string, string>> {
+  const out: Record<string, string> = {};
+  for (const machine of (await taggedInstances()).filter((i) => i.instanceId === instanceId)) {
+    const said = await incusOk(
+      ["exec", machine.name, "--", "sh", "-c",
+        `ip -4 -o addr show | awk '{print $4}' | cut -d/ -f1`], 30_000);
+    for (const address of (said ?? "").split("\n").map((l) => l.trim()).filter(Boolean)) {
+      if (address.startsWith("127.")) continue;
+      // The first non-loopback address on the segment. A machine on two links has the one that
+      // matches this segment's range, which is what the caller asked about.
+      if (!out[machine.machine]) out[machine.machine] = address;
+    }
+  }
+  void segment;
+  return out;
+}
+
+/** Names this workstation already answers for, so a scenario cannot quietly shadow one. */
+function alreadyServed(): string[] {
+  const beside = "/etc/dnsmasq.d/hal-dns.conf";
+  if (!existsSync(beside)) return [];
+  return [...readFileSync(beside, "utf8").matchAll(/^address=\/([^/]+)\//gm)].map((m) => m[1]!);
+}
+
+export async function connect(instanceId?: string): Promise<Reached> {
+  const id = instanceId ?? (await theOnlyInstance());
+
+  const link = (await taggedNetworks()).find(
+    (n) => n.instanceId === id && n.kind === "public" && n.cidr.some((c) => !c.includes(":")));
+  if (!link) {
+    throw new ConnectError(
+      `${id} has no public IPv4 segment, so there is nothing for this workstation to join.`);
+  }
+  const range = link.cidr.find((c) => !c.includes(":"))!;
+  const prefix = range.slice(range.lastIndexOf("/") + 1);
+
+  const machines = await addressesOn(id, link.segment);
+  if (Object.keys(machines).length === 0) {
+    throw new ConnectError(`no machine in ${id} has an address yet — is it still coming up?`);
+  }
+
+  // **Refused rather than shadowed.** This workstation already answers for the mesh it really
+  // runs; a scenario machine sharing one of those names would silently take it over, and the
+  // damage would land on the real thing rather than the lab.
+  const clash = Object.keys(machines)
+    .map((m) => `${m}.internal`)
+    .filter((n) => alreadyServed().includes(n));
+  if (clash.length > 0) {
+    throw new ConnectError(
+      `${clash.join(", ")} is already answered on this workstation for something real. ` +
+        `Connecting would point it at the lab instead, which is the wrong thing to break.`);
+  }
+
+  // An address on the link, high in the range so it does not meet what a scenario declares.
+  const base = Object.values(machines)[0]!.split(".").slice(0, 3).join(".");
+  const mine = `${base}.254`;
+  if (Object.values(machines).includes(mine)) {
+    throw new ConnectError(`${mine} is taken by a machine, and that is the address this uses.`);
+  }
+
+  const added = asRoot(["ip", "addr", "add", `${mine}/${prefix}`, "dev", link.name]);
+  if (!added.ok && !/File exists/i.test(added.said)) {
+    throw new ConnectError(`could not take an address on ${link.name}: ${added.said}`);
+  }
+
+  // Everything under a machine's name, answered with that machine. The same shape the mesh's own
+  // resolver writes, because it is answering the same question.
+  const rule = [
+    "# Written by mesh-lab connect. Removed by mesh-lab disconnect.",
+    "# One scenario at a time: these names are only true while that scenario is standing.",
+    ...Object.entries(machines).sort()
+      .map(([machine, address]) => `address=/${machine}.internal/${address}`),
+    "",
+  ].join("\n");
+  writeFileSync("/tmp/mesh-lab-dns.conf", rule);
+  const placed = asRoot(["cp", "/tmp/mesh-lab-dns.conf", RULE]);
+  if (!placed.ok) throw new ConnectError(`could not write ${RULE}: ${placed.said}`);
+  // **Restart, not reload.** A reload is SIGHUP, and dnsmasq answers that by re-reading its hosts
+  // file and clearing its cache — not its configuration. The rule was written, the reload
+  // reported success, and nothing resolved. Measured: the daemon's start time was nine days old
+  // after a "successful" reload.
+  //
+  // It costs a moment of no name resolution on this workstation, which is the honest price and is
+  // paid again by disconnect.
+  const reloaded = asRoot(["systemctl", "restart", "dnsmasq"]);
+  if (!reloaded.ok) throw new ConnectError(`could not restart dnsmasq: ${reloaded.said}`);
+
+  log.info(`connected to ${id} as ${mine} on ${link.name}`);
+  return { instanceId: id, bridge: link.name, address: mine, machines };
+}
+
+export async function disconnect(): Promise<string[]> {
+  const undone: string[] = [];
+
+  if (existsSync(RULE)) {
+    const removed = asRoot(["rm", "-f", RULE]);
+    if (!removed.ok) throw new ConnectError(`could not remove ${RULE}: ${removed.said}`);
+    asRoot(["systemctl", "restart", "dnsmasq"]);
+    undone.push(`removed ${RULE} and reloaded dnsmasq`);
+  }
+
+  // Any address this took, on any lab link still present. Done by looking rather than by
+  // remembering: a workstation that was rebooted, or a scenario destroyed under it, must still
+  // be able to tidy up.
+  for (const link of await taggedNetworks()) {
+    const shown = await incusOk(["network", "info", link.name], 15_000);
+    if (shown === null) continue;
+    const ran = spawnSync("ip", ["-4", "-o", "addr", "show", "dev", link.name], { encoding: "utf8" });
+    for (const line of (ran.stdout ?? "").split("\n")) {
+      const found = line.match(/inet (\d+\.\d+\.\d+\.254\/\d+)/);
+      if (!found) continue;
+      const dropped = asRoot(["ip", "addr", "del", found[1]!, "dev", link.name]);
+      if (dropped.ok) undone.push(`gave up ${found[1]} on ${link.name}`);
+    }
+  }
+
+  if (undone.length === 0) undone.push("nothing was connected");
+  return undone;
+}
+
+/** What is connected now, for a person who cannot remember. */
+export async function connection(): Promise<string[]> {
+  if (!existsSync(RULE)) return [];
+  return readFileSync(RULE, "utf8").split("\n")
+    .filter((l) => l.startsWith("address=/"))
+    .map((l) => {
+      const [, name, address] = l.match(/^address=\/([^/]+)\/(.+)$/) ?? [];
+      return `${name} → ${address}`;
+    });
+}
+
+void incus;

src/lifecycle/place.ts

@@ -17,7 +17,8 @@ import { unlink } from "node:fs/promises";
 import { tmpdir } from "node:os";
 import { join } from "node:path";
 
-import { incus, incusOk } from "../incus/client.ts";
+import { incus, incusOk, succeeds } from "../incus/client.ts";
+import { around, log, shorten } from "../log.ts";
 
 /**
  * What this stage can put inside a machine.
@@ -291,6 +292,7 @@ export async function placeImage(
   }
 
   try {
+    await waitForRuntime(instanceName, machine);
     await incus(["file", "push", tar, `${instanceName}/tmp/image.tar`], 900_000);
 
     // Read back what the runtime says, not that the push returned. A file arriving is not an
@@ -321,6 +323,18 @@ function local(
   command: string,
   args: string[],
   timeoutMs: number,
+): Promise<{ ok: boolean; stdout: string; stderr: string }> {
+  // The third and last place the lab runs an external program (novox/hq 04-ISSUES/024).
+  // `docker save` of a large image is the slowest single thing a raise does.
+  return around(`${command} ${shorten(args)}`, () => runLocal(command, args, timeoutMs), {
+    heartbeatMs: 15_000,
+  });
+}
+
+function runLocal(
+  command: string,
+  args: string[],
+  timeoutMs: number,
 ): Promise<{ ok: boolean; stdout: string; stderr: string }> {
   return new Promise((resolve) => {
     const child = spawn(command, args, { stdio: ["ignore", "pipe", "pipe"] });
@@ -335,7 +349,51 @@ function local(
     });
     child.on("close", (code) => {
       clearTimeout(timer);
+      if (code !== 0) {
+        log.debug(`  exit ${code}: ${shorten([stderr.trim() || "(nothing on stderr)"], 400)}`);
+      }
       resolve({ ok: code === 0, stdout, stderr });
     });
   });
 }
+
+/**
+ * Wait until the machine's container runtime will answer, and say why if it will not.
+ *
+ * **A stall must become a failure with a reason.** `docker load` against a daemon that is not
+ * running does not fail — the socket exists and is socket-activated, so the client blocks while
+ * systemd tries to start a service that may be queued behind something. That turned a
+ * misconfigured link into a thirty-five minute silence and then a timeout naming the wrong
+ * thing entirely (novox/hq 04-ISSUES/024).
+ *
+ * The one cause seen in practice is named in the message, because it is not guessable from
+ * "docker did not start": these machines have no DHCP by design, so a link that networkd was
+ * never told how to configure leaves `network-online.target` unreachable for ever, and Docker
+ * is ordered after it.
+ */
+async function waitForRuntime(instanceName: string, machine: string): Promise<void> {
+  const deadline = Date.now() + 120_000;
+  while (Date.now() < deadline) {
+    if (await succeeds(["exec", instanceName, "--", "docker", "info"], 20_000)) return;
+    await new Promise((r) => setTimeout(r, 2_000));
+  }
+
+  // What it was waiting for, asked once, so the report names the cause rather than the symptom.
+  const jobs = (await incusOk(
+    ["exec", instanceName, "--", "systemctl", "list-jobs", "--no-pager"], 20_000,
+  )) ?? "";
+  const blocked = jobs.includes("network-online.target") || jobs.includes("wait-online");
+
+  throw new PlacementError(
+    machine,
+    `the container runtime on ${machine} did not answer within 120s, so an image cannot be ` +
+      `placed into it.\n` +
+      (blocked
+        ? `  Docker is queued behind network-online.target, which is waiting for a link that ` +
+          `systemd-networkd was never told how to configure. These machines have no DHCP by ` +
+          `design, so that wait never ends — give the link a .network unit rather than an ` +
+          `address set by hand.\n`
+        : "") +
+      `  systemd is waiting on:\n${jobs.trim() || "    (it said nothing)"}`,
+  );
+}

src/lifecycle/raise.ts

@@ -25,6 +25,7 @@ import { applyHostFirewalls } from "./firewall.ts";
 import { IMAGE_PREFIX, BASE_IMAGE_ALIAS, BASE_IMAGE_HOWTO, planPlacements, applyPlacements } from "./place.ts";
 import { baseImageExists, UPSTREAM_IMAGE } from "./base.ts";
 import { discardStock, raiseRegistry, stockRegistry } from "./registry.ts";
+import { log as record } from "../log.ts";
 
 /** Drivers whose snapshots are copy-on-write. On `dir` a snapshot is a full copy. */
 const COW_DRIVERS = ["btrfs", "zfs"];
@@ -130,12 +131,42 @@ async function createNetwork(
   return name;
 }
 
+/**
+ * The one network the lab supplies rather than the declaration.
+ *
+ * Every segment a scenario describes is an isolated bridge with no addresses, no DHCP and no NAT,
+ * because the declaration owns addressing. This is the opposite of that on purpose: it is not part
+ * of the scenario, it carries no scenario traffic, and what is routable on it is the host's fact.
+ *
+ * It exists so a machine can fetch what it starts from. A first node pulls three images before
+ * there is any mesh, and the module that gives a mesh its own store pulls one more
+ * (novox/hq 04-ISSUES/029) — none of which anything inside a scenario can serve.
+ *
+ * Tagged like everything else, so tearing the scenario down takes it too.
+ */
+async function createUplink(instanceId: string): Promise<string> {
+  const name = networkName(instanceId, "uplink");
+  if (await succeeds(["network", "show", name], 15_000)) return name;
+  await incus([
+    "network", "create", name,
+    "ipv4.address=auto",
+    "ipv4.nat=true",
+    "ipv6.address=none",
+    `user.mesh-lab.instance=${instanceId}`,
+    "user.mesh-lab.segment=uplink",
+  ]);
+  return name;
+}
+
 async function createMachine(
   instanceId: string,
   machine: string,
   attachments: { segment: string }[],
   image: string,
   pool: string,
+  egress = false,
+  memory = "1GiB",
+  cpus = 2,
 ): Promise<string> {
   const name = machineName(instanceId, machine);
   if (await succeeds(["config", "show", name], 15_000)) return name;
@@ -147,8 +178,8 @@ async function createMachine(
     // Arch images refuse to boot under secureboot with the shipped keys. Discovered by
     // the first launch failing with exactly that message.
     "-c", "security.secureboot=false",
-    "-c", "limits.memory=1GiB",
-    "-c", "limits.cpu=2",
+    "-c", `limits.memory=${memory}`,
+    "-c", `limits.cpu=${cpus}`,
     "-c", `user.mesh-lab.instance=${instanceId}`,
     "-c", `user.mesh-lab.machine=${machine}`,
   ];
@@ -167,6 +198,17 @@ async function createMachine(
       `hwaddr=${macFor(instanceId, machine, index)}`,
     ]);
   }
+  // After every declared attachment, so eth0..ethN keep meaning what the scenario said and the
+  // uplink is whatever comes next. A machine that never asked for one has no such interface at
+  // all, which is the difference between a closed scenario and an open one.
+  if (egress) {
+    await incus([
+      "config", "device", "add", name, `eth${attachments.length}`, "nic",
+      "nictype=bridged",
+      `parent=${await createUplink(instanceId)}`,
+      `hwaddr=${macFor(instanceId, machine, attachments.length)}`,
+    ]);
+  }
   return name;
 }
 
@@ -174,7 +216,19 @@ export async function raise(
   scenario: Scenario,
   options: RaiseOptions = {},
 ): Promise<RaisedScenario> {
-  const log = options.onProgress ?? (() => {});
+  // **Progress always reaches the file, whether or not anyone asked to see it.**
+  //
+  // This used to be the caller's callback or nothing, and every function below takes its `log`
+  // from here — so a caller that passed none silenced the whole lifecycle. That is exactly what
+  // happened: the end-to-end test called `raise` with no callback, so the one run that mattered
+  // reported not a single step (novox/hq 04-ISSUES/024).
+  //
+  // Teeing rather than replacing: the caller still gets what it asked for, and the record is kept
+  // regardless. A record nobody switched on is the one you want after the thing goes wrong.
+  const log = (message: string): void => {
+    record.info(message);
+    options.onProgress?.(message);
+  };
 
   // A scenario that places a runtime or an image needs machines built from the base image,
   // because a sealed machine cannot install one (novox/hq ADR 0006). Chosen here rather than
@@ -198,55 +252,71 @@ export async function raise(
   // has been created, and there is no wreckage to leave standing.
   assertSupported(scenario);
 
-  let step = "choosing a storage pool";
+  // **The step is the log.** Setting it and recording it are one act, so a step added later
+  // cannot be a step that goes unrecorded — which is the drift that made a thirty-five minute
+  // stall untraceable (novox/hq 04-ISSUES/024). Each entry closes the previous one with its
+  // duration, so the log says where a raise spends its time as well as where it stopped.
+  let step = "";
+  let stepFrom = Date.now();
+  const enter = (next: string): string => {
+    if (step) record.info(`  ${step} — ${((Date.now() - stepFrom) / 1000).toFixed(1)}s`);
+    record.info(`▶ ${next}`);
+    stepFrom = Date.now();
+    step = next;
+    return next;
+  };
+
+  enter("choosing a storage pool");
   try {
     const pool = await choosePool(log);
     log(`instance ${instanceId}  pool ${pool}`);
 
-    step = "creating segments";
+    enter("creating segments");
     const networks: string[] = [];
     for (const [segment, spec] of Object.entries(scenario.segments)) {
       networks.push(await createNetwork(instanceId, segment, spec));
       log(`  segment ${segment}`);
     }
 
-    step = "creating machines";
+    enter("creating machines");
     const created: string[] = [];
     const byMachine = new Map<string, string>();
     for (const [machine, spec] of Object.entries(scenario.machines)) {
       const attachments = spec.at === "detached" ? [] : spec.at;
-      const name = await createMachine(instanceId, machine, attachments, image, pool);
+      const name = await createMachine(
+        instanceId, machine, attachments, image, pool,
+        spec.at !== "detached" && spec.egress === true, spec.memory, spec.cpus);
       created.push(name);
       byMachine.set(machine, name);
       log(`  machine ${machine}${spec.at === "detached" ? " (detached)" : ""}`);
     }
 
-    step = "starting machines";
+    enter("starting machines");
     for (const name of created) {
       await succeeds(["start", name], 60_000);
     }
 
-    step = "waiting for machines to become usable";
+    enter("waiting for machines to become usable");
     await waitUntilAllUsable(created, readyTimeout, log);
 
-    step = "applying declared addresses";
+    enter("applying declared addresses");
     await applyAddresses(scenario, instanceId, byMachine, log);
 
     // Transit first: a gateway's default route points at it, so it has to exist.
-    step = "wiring the public networks together";
+    enter("wiring the public networks together");
     const transit = await raiseTransit(scenario, instanceId, log);
 
-    step = "raising routers";
+    enter("raising routers");
     const routers = await raiseRouters(scenario, instanceId, planRouters(scenario, instanceId), log);
     if (transit) routers.push(transit);
 
     // Stocked on this workstation, where there is a network, and served from inside the
     // scenario, where there is not (novox/hq 04-ISSUES/009).
-    step = "stocking the registry";
+    enter("stocking the registry");
     const stock = await stockRegistry(scenario.images ?? [], log);
     let registry: Awaited<ReturnType<typeof raiseRegistry>> = null;
     try {
-      step = "raising the registry";
+      enter("raising the registry");
       registry = await raiseRegistry(scenario, instanceId, stock, log);
     } finally {
       // Cleaning up scratch must not fail a raise that succeeded. The scenario is standing
@@ -258,17 +328,17 @@ export async function raise(
       }
     }
 
-    step = "routing machines through their gateways";
+    enter("routing machines through their gateways");
     await applyDefaultRoutes(scenario, byMachine, log);
 
     // Last: a machine that refuses inbound must still have been reachable while the lab
     // was configuring it.
-    step = "applying host firewalls";
+    enter("applying host firewalls");
     await applyHostFirewalls(scenario, byMachine, log);
 
     // Last, and only once the underlay is real. Placing before the machines can reach each
     // other would test the host against a network the scenario does not describe.
-    step = "placing";
+    enter("placing");
     await applyPlacements(scenario, byMachine, log);
 
     return {

src/lifecycle/registry.ts

@@ -22,6 +22,8 @@ import { spawn } from "node:child_process";
 
 import { incus, incusOk, succeeds } from "../incus/client.ts";
 import { macFor, networkName } from "./names.ts";
+import { addressLink } from "./address.ts";
+import { around, log, shorten } from "../log.ts";
 import { BASE_IMAGE_ALIAS, placeImage } from "./place.ts";
 import { mkdtemp, rm } from "node:fs/promises";
 import { tmpdir } from "node:os";
@@ -170,6 +172,16 @@ async function waitForRegistry(port: number): Promise<void> {
 function docker(
   args: string[],
   timeoutMs: number,
+): Promise<{ ok: boolean; stdout: string; stderr: string }> {
+  // The second of the three places the lab runs an external program (novox/hq 04-ISSUES/024).
+  // `docker push` of a large image is minutes of legitimate silence, which is exactly when a
+  // heartbeat earns its keep.
+  return around(`docker ${shorten(args)}`, () => runDocker(args, timeoutMs), { heartbeatMs: 15_000 });
+}
+
+function runDocker(
+  args: string[],
+  timeoutMs: number,
 ): Promise<{ ok: boolean; stdout: string; stderr: string }> {
   return new Promise((resolve) => {
     const child = spawn("docker", args, { stdio: ["ignore", "pipe", "pipe"] });
@@ -184,6 +196,11 @@ function docker(
     });
     child.on("close", (code) => {
       clearTimeout(timer);
+      // A docker failure is an answer here rather than an exception, so it would otherwise pass
+      // through the log looking exactly like a success.
+      if (code !== 0) {
+        log.debug(`  exit ${code}: ${shorten([stderr.trim() || "(nothing on stderr)"], 400)}`);
+      }
       resolve({ ok: code === 0, stdout, stderr });
     });
   });
@@ -296,20 +313,33 @@ export async function raiseRegistry(
   await succeeds(["start", name], 60_000);
   await waitForAgent(name);
 
-  // Address it by MAC, never by interface name: a machine with a container runtime has a
-  // `docker0` that sorts before `enp5s0`, and naive selection configures that instead — which
-  // then overlaps the segment and breaks routing on the machine.
-  const mac = macFor(instanceId, "registry", 0);
-  await incus(["exec", name, "--", "sh", "-c",
-    `dev=$(ip -o link | awk -F': ' '/${mac}/ {print $2}' | head -1); ` +
-    `[ -n "$dev" ] && ip addr add ${address}${prefix} dev "$dev" 2>/dev/null; ` +
-    `[ -n "$dev" ] && ip link set "$dev" up`], 60_000);
+  // Addressed the way every other machine is: a systemd-networkd unit matching the MAC.
+  //
+  // **This used to be `ip addr add`, and it stalled the lab.** An address set by hand leaves
+  // networkd waiting to configure a link it was never told about, so the link sits at
+  // `configuring`, `systemd-networkd-wait-online` never returns — its timeout is `infinity` —
+  // and `network-online.target` is never reached. Docker is ordered after that target, so
+  // `docker load` two lines below blocked on a socket whose daemon was queued behind a target
+  // that would never come.
+  //
+  // Matching on MAC and not on interface name is still the rule: a machine with a container
+  // runtime has a `docker0` that sorts before `enp5s0`, and naive selection configures that.
+  await addressLink(name, {
+    device: "eth0",
+    mac: macFor(instanceId, "registry", 0),
+    addresses: [`${address}${prefix}`],
+    // The registry takes the segment's default. It carried no MTU before this and still does
+    // not: what a scenario sets an MTU for is the path under test, and this is scenery.
+    mtu: undefined,
+  });
 
   log(`  registry on ${segment.name} at ${address}`);
 
   // The registry's own image, placed by tag — an archive keeps a tag and cannot keep a digest,
   // which is the whole reason this machine exists.
-  await placeImage(name, "registry", REGISTRY_IMAGE, () => {});
+  // Logged, not silenced. This is the step a stall sat in for thirty-five minutes while the
+  // caller had passed it a callback that threw everything away (novox/hq 04-ISSUES/024).
+  await placeImage(name, "registry", REGISTRY_IMAGE, log);
 
   // The destination must EXIST before a recursive push, or incus copies the source's contents
   // rather than the source — the data lands one directory too shallow, the registry finds

src/lifecycle/router.ts

@@ -62,7 +62,7 @@ export async function ensureRouterImage(log: (message: string) => void = () => {
 
   // Default profile on purpose: this is the one container that needs to reach a repository.
   await incus(["launch", ROUTER_BASE, builder], 300_000);
-  await waitUntilUsable(builder, 120, () => {});
+  await waitUntilUsable(builder, 120, log);
 
   // `exec` works before the container has an address. Usable means a command runs; it does
   // not mean the network is up, and the first attempt failed on DNS because those were
@@ -279,7 +279,7 @@ export async function raiseTransit(
     }
   }
   await succeeds(["start", name], 60_000);
-  await waitUntilUsable(name, 120, () => {});
+  await waitUntilUsable(name, 120, log);
 
   for (const [index, segment] of publicSegments.entries()) {
     const device = `eth${index}`;
@@ -343,7 +343,7 @@ export async function raiseRouters(
   }
 
   for (const name of created) {
-    await waitUntilUsable(name, 120, () => {});
+    await waitUntilUsable(name, 120, log);
   }
 
   for (const plan of plans) {

src/log.ts

@@ -0,0 +1,163 @@
+/**
+ * What the lab was doing, written down while it does it.
+ *
+ * **The lab's own recurring fault, and the one it exists to catch elsewhere.** On 2026-09-01 a run
+ * stalled for thirty-five minutes and said nothing at all. The cause was a link that systemd was
+ * still configuring, three layers down inside a `docker load` that was blocked on a socket — and
+ * every one of those layers knew what it was waiting for. None of them said so
+ * (novox/hq 04-ISSUES/024).
+ *
+ * Three decisions follow from that, and each is doing work:
+ *
+ * **Every external command is logged, at the three places that run one.** Ninety-seven call sites
+ * reach a hypervisor or a container runtime through three wrappers, so instrumenting the wrappers
+ * covers all of them and nothing has to remember to log.
+ *
+ * **A command that is still running says so while it runs.** A line before and a line after tells
+ * you nothing until the after arrives, which is exactly the case that matters. So a command
+ * outstanding past a few seconds reports itself periodically, with how long it has been going.
+ * *"I cannot see progress" is not evidence of no progress* — but it should not be the only thing
+ * available either.
+ *
+ * **It goes to a file, written synchronously.** Node block-buffers stdout when it is redirected,
+ * and a test runner buffers it again, so a console log can sit minutes behind the run. That was
+ * read as a stall twice in one day — the second time straight after the real fix, where a
+ * buffering artifact argues the fix did not work. `appendFileSync` cannot lag.
+ */
+
+import { appendFileSync, mkdirSync } from "node:fs";
+import { dirname } from "node:path";
+
+/** How much to say. `off` writes nothing at all, for a caller that wants none of this. */
+export type Level = "off" | "info" | "debug" | "trace";
+
+const ORDER: Record<Level, number> = { off: 0, info: 1, debug: 2, trace: 3 };
+
+/**
+ * Where the log goes, and how much of it.
+ *
+ * Off unless asked for, because a suite that writes a debug log nobody reads is a suite that
+ * writes a debug log nobody reads. `MESH_LAB_LOG` turns it on; `MESH_LAB_LOG_FILE` says where.
+ */
+function configured(): { level: Level; file: string | null } {
+  const asked = (process.env["MESH_LAB_LOG"] ?? "").trim().toLowerCase();
+  const level: Level = asked === "info" || asked === "debug" || asked === "trace" ? asked : "off";
+  const file = process.env["MESH_LAB_LOG_FILE"]?.trim() || null;
+  return { level, file };
+}
+
+let level: Level = configured().level;
+let file: string | null = configured().file;
+const started = Date.now();
+let opened = false;
+
+/** Re-read the environment. For a test, and for a caller that sets it after import. */
+export function reconfigure(): void {
+  const now = configured();
+  level = now.level;
+  file = now.file;
+  opened = false;
+}
+
+/** Point the log somewhere explicitly, whatever the environment says. */
+export function logTo(path: string | null, at: Level = "debug"): void {
+  file = path;
+  level = path === null ? "off" : at;
+  opened = false;
+}
+
+function elapsed(): string {
+  return ((Date.now() - started) / 1000).toFixed(1).padStart(7) + "s";
+}
+
+/**
+ * Write one line, synchronously.
+ *
+ * A failure to write is swallowed. A log that cannot be written is a nuisance; a run that dies
+ * because its log could not be written is a fault the log invented.
+ */
+function write(at: Level, message: string): void {
+  if (level === "off" || ORDER[at] > ORDER[level]) return;
+  const line = `${new Date().toISOString()} ${elapsed()} ${at.padEnd(5)} ${message}\n`;
+  if (!file) {
+    process.stderr.write(line);
+    return;
+  }
+  try {
+    if (!opened) {
+      mkdirSync(dirname(file), { recursive: true });
+      opened = true;
+    }
+    appendFileSync(file, line);
+  } catch {
+    // Nothing sensible to do, and saying so on every line would be worse than silence.
+  }
+}
+
+export const log = {
+  info: (message: string) => write("info", message),
+  debug: (message: string) => write("debug", message),
+  trace: (message: string) => write("trace", message),
+  /** Whether anything is being recorded, for a caller deciding whether to compose an expensive message. */
+  on: () => level !== "off",
+};
+
+/**
+ * Run something, saying what it is and how long it took — and saying so *while* it runs.
+ *
+ * The heartbeat is the point. A stall is a command that started and has not finished, and the only
+ * thing that distinguishes it from ordinary work is how long it has been going — which nothing can
+ * tell you unless something is still counting.
+ */
+export async function around<T>(
+  what: string,
+  run: () => Promise<T>,
+  options: { heartbeatMs?: number; at?: Level; expectedToFail?: boolean } = {},
+): Promise<T> {
+  const at = options.at ?? "debug";
+  // Only when nothing at all is being recorded. **Skipping the wrapper whenever the step's own
+  // level is too low was wrong twice over**: it dropped the failure line, which is logged louder
+  // than the step precisely so it survives a turned-down log, and it dropped the heartbeat, which
+  // is the whole reason this function exists. Both went missing at exactly the level somebody
+  // would actually run — `info`. The gate belongs in `write`, which already has one.
+  if (level === "off") return run();
+
+  const beat = options.heartbeatMs ?? 15_000;
+  const from = Date.now();
+  write(at, `→ ${what}`);
+
+  const timer = setInterval(() => {
+    // **Reported as still running, not as stuck.** Which it is is not knowable from here, and a
+    // log that calls a slow step a hang teaches people to ignore it.
+    write("info", `… ${what} — still running after ${((Date.now() - from) / 1000).toFixed(0)}s`);
+  }, beat);
+  // Never hold the process open for a heartbeat.
+  timer.unref?.();
+
+  try {
+    const result = await run();
+    write(at, `← ${what} (${((Date.now() - from) / 1000).toFixed(1)}s)`);
+    return result;
+  } catch (err) {
+    // Failures are logged at info even when the step was debug: whoever turned logging down still
+    // wants the thing that went wrong.
+    //
+    // **Unless failing is one of the answers.** Plenty of commands here are questions — does this
+    // network exist, is the agent up yet — and a failure is a no. Logging those as faults fills a
+    // normal run with ✗ and teaches whoever reads it that ✗ means nothing, which is the state to
+    // be in when one of them is real.
+    write(options.expectedToFail ? "debug" : "info",
+      `${options.expectedToFail ? "·" : "✗"} ${what} ` +
+        `(${((Date.now() - from) / 1000).toFixed(1)}s): ${(err as Error).message}`);
+    throw err;
+  } finally {
+    clearInterval(timer);
+  }
+}
+
+/** A command line, shortened for a log without losing which command it was. */
+export function shorten(argv: string[], limit = 200): string {
+  const whole = argv.join(" ");
+  if (whole.length <= limit) return whole;
+  return whole.slice(0, limit - 1) + "…";
+}

src/pinning.ts

@@ -0,0 +1,56 @@
+/**
+ * Rewriting an image reference to the one a scenario's own registry serves.
+ *
+ * **A digest is not knowable until something is built** (novox/hq 04-ISSUES/025). A manifest in a
+ * repository can pin a third-party image, because somebody can ask a registry what a tag points
+ * at. It cannot pin an image the mesh builds itself: that image does not exist yet, and when it
+ * does its digest belongs to whichever registry served it.
+ *
+ * The bundle has always had this problem and solves it by rewriting references once the scenario's
+ * registry is up and its digests are known. Modules have exactly the same problem and were solving
+ * it by shipping sixty-four zeros, which parses, resolves, composes — and stops on the machine.
+ *
+ * So the rewriting is shared rather than copied, and matches on the **repository**, because that
+ * is the part a person writes and the only part that survives being served somewhere else.
+ */
+
+/** `192.0.2.250:5000/ghcr.io/mailu/admin@sha256:…` → `ghcr.io/mailu/admin` */
+export function repositoryOf(pinned: string): string {
+  const at = pinned.indexOf("@");
+  const body = at === -1 ? pinned : pinned.slice(0, at);
+  const slash = body.indexOf("/");
+  // Everything after the registry. A reference with no slash at all is its own repository.
+  return slash === -1 ? body : body.slice(slash + 1);
+}
+
+/**
+ * Replace every reference to a stocked repository with the reference this scenario serves.
+ *
+ * Matching is on the repository and ignores whatever registry and digest were written down —
+ * a file may name `postgres@sha256:7456…` or `mesh-provision-postgres@sha256:0000…` and both mean
+ * *the postgres this scenario has*. That is the whole point: the text says which image, the
+ * scenario says which copy.
+ *
+ * A repository the scenario did not stock is left alone rather than blanked. It may be reachable
+ * some other way, and silently emptying a reference would produce the exact failure this exists to
+ * prevent.
+ */
+export function pinnedInto(text: string, served: string[]): string {
+  let out = text;
+  for (const pinned of served) {
+    const repository = repositoryOf(pinned);
+    const escaped = repository.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
+    // Optionally a registry, then the repository, then any digest. Anchored on a quote or
+    // whitespace so a longer repository ending in a shorter one is not half-replaced.
+    out = out.replaceAll(
+      new RegExp(`(?<=^|["\\s])(?:[A-Za-z0-9_.:-]+\\/)*${escaped}@sha256:[0-9a-f]{64}`, "g"),
+      pinned,
+    );
+  }
+  return out;
+}
+
+/** Whether anything is still pinned to a placeholder, which would fail on the machine. */
+export function stillUnpinned(text: string): string[] {
+  return [...text.matchAll(/([A-Za-z0-9_.:/-]+)@sha256:0{64}/g)].map((m) => m[1]!);
+}

src/rebuild.ts

@@ -0,0 +1,100 @@
+/**
+ * Rebuild what the lab runs, from source, before it runs.
+ *
+ * **A stale artifact reporting success against old rules is the fault this project keeps writing
+ * down** (novox/hq 04-ISSUES/005). The lab consumes three artifacts from two repositories, and they
+ * were rebuilt by hand, one at a time, from memory. A rename in the control plane's catalogue needs
+ * both the control-plane image *and* the builder binary, because both parse manifests; rebuilding
+ * one left a binary eleven hours old refusing a field the mesh had just renamed, and cost a full
+ * run to find out.
+ *
+ * In the repository rather than in a shell script beside it, for the reason 005 is about: a step
+ * that lives in somebody's terminal history runs when they remember, and remembering is not a
+ * mechanism.
+ */
+
+import { spawnSync } from "node:child_process";
+import { repositories } from "./repos.ts";
+
+export interface Build {
+  /** What it produces, for the log. */
+  what: string;
+  /** The repository root to run in. */
+  in: string;
+  argv: string[];
+  env?: NodeJS.ProcessEnv;
+}
+
+/**
+ * planned is what must be built, given where this run has been pointed.
+ *
+ * Derived from the same environment the suite is configured by, so there is one place that says
+ * where a repository is. A repository this run was not pointed at is not built — and, per
+ * {@link whatWasTested}, is not claimed in the receipt either.
+ */
+export function planned(env: NodeJS.ProcessEnv = process.env): Build[] {
+  const builds: Build[] = [];
+  const where = repositories(env);
+  const host = env["MESH_LAB_HOST_BINARY"];
+  if (host && where["mesh-host"]) {
+    builds.push({
+      what: "host",
+      in: where["mesh-host"],
+      argv: ["go", "build", "-ldflags=-s -w -X main.builtFor=arch", "-o", host, "./cmd/mesh-host"],
+      env: { CGO_ENABLED: "0" },
+    });
+  }
+  const control = where["mesh-control"];
+  if (control) {
+    // **Every image the lab runs, not only the control plane's.**
+    //
+    // On 2026-09-01 a suite ran with a control-plane image built that minute and a provisioner
+    // image built the day before. The rotation test failed against a real database, and the
+    // failure looked exactly like the change under test being wrong — the provisioner was
+    // creating logins by a naming rule that had been replaced.
+    //
+    // This is the same fault the builder line below was added for, one target along. A rebuild
+    // that covers most of what a run uses is worse than one that covers none, because the run
+    // that follows it is believed.
+    builds.push({
+      what: "images",
+      in: control,
+      argv: ["make", "image", "builder-image", "provisioner-image", "objectstore-image",
+        "redis-provisioner-image", "proxy-image"],
+    });
+    const builder = env["MESH_LAB_BUILDER"];
+    if (builder) {
+      // Both of these parse manifests. Building one and not the other is the eleven-hour-old
+      // binary above, so they are one step and not two.
+      builds.push({
+        what: "builder",
+        in: control,
+        argv: ["go", "build", "-o", builder, "./cmd/mesh-builder"],
+      });
+    }
+  }
+  return builds;
+}
+
+/** rebuild runs the plan, and throws on the first failure rather than testing a stale artifact. */
+export function rebuild(env: NodeJS.ProcessEnv = process.env): string[] {
+  const built: string[] = [];
+  for (const build of planned(env)) {
+    const [command, ...args] = build.argv;
+    const ran = spawnSync(command!, args, {
+      cwd: build.in,
+      env: { ...env, ...build.env },
+      encoding: "utf8",
+    });
+    if (ran.status !== 0) {
+      // Loudly, and stopping. A suite that runs anyway is a suite reporting on code that is not
+      // the code in front of you, which is the whole of 005.
+      throw new Error(
+        `could not build the ${build.what}: ${build.argv.join(" ")} in ${build.in}\n\n` +
+          `${(ran.stderr || ran.stdout || String(ran.error)).trim()}`,
+      );
+    }
+    built.push(build.what);
+  }
+  return built;
+}

src/repos.ts

@@ -0,0 +1,28 @@
+/**
+ * Where the repositories this run was pointed at are.
+ *
+ * **One place**, because two things need it and they must agree: the rebuild builds these, and the
+ * receipt claims these. A receipt naming a repository the run did not build is exactly the
+ * false coverage novox/hq 04-ISSUES/005 is about, and it would arrive by nobody's decision — just
+ * two derivations drifting.
+ *
+ * Derived from the environment the suite is configured by, so a repository this run was not
+ * pointed at is neither built nor claimed.
+ */
+
+import { dirname } from "node:path";
+
+export interface Repositories {
+  /** Absolute path to the repository root, by name. */
+  [name: string]: string;
+}
+
+export function repositories(env: NodeJS.ProcessEnv = process.env): Repositories {
+  const found: Repositories = {};
+  found["mesh-lab"] = process.cwd();
+  const host = env["MESH_LAB_HOST_BINARY"];
+  if (host) found["mesh-host"] = dirname(host);
+  const modules = env["MESH_LAB_MODULES"];
+  if (modules) found["mesh-control"] = dirname(dirname(modules));
+  return found;
+}

src/suite.ts

@@ -0,0 +1,107 @@
+/**
+ * Run the end-to-end suite, and leave a receipt saying it ran.
+ *
+ * **Here rather than inside the tests, because the tests cannot know their own totals.** Node's
+ * runner reports them to whatever invoked it, and a test file inventing its own count would be a
+ * receipt that says whatever the last edit made it say.
+ *
+ * Here rather than in a shell script for the same reason the rebuild is: a step that lives in
+ * somebody's terminal history is a step that runs when they remember (novox/hq 04-ISSUES/005).
+ */
+
+import { spawn } from "node:child_process";
+import { endToEnd, record, whatWasTested } from "./lastrun.ts";
+import { rebuild } from "./rebuild.ts";
+
+/** counted is what the runner said, or nulls when it said nothing recognisable. */
+export function counted(output: string): { passed: number | null; failed: number | null } {
+  // The runner's own summary lines, each on a line of its own. Anchored, so a test *named*
+  // "pass 3" cannot be mistaken for the total — which is not a hypothetical worry in a suite whose
+  // tests are named in sentences.
+  // Stripped first: the runner colours its summary even when its stdout is a pipe, so the line is
+  // "\x1b[34m\u2139 pass 8\x1b[39m" and an anchored pattern never sees the start of it. Found by
+  // running this against the real runner — the fixture it was first written against was output I
+  // had imagined, which is a test that agrees with the mistake it was written beside.
+  const plain = output.replace(/\u001b\[[0-9;]*m/g, "");
+  const total = (what: RegExp) => {
+    const found = plain.match(what);
+    return found ? Number(found[1]) : null;
+  };
+  return {
+    passed: total(/^\s*(?:\u2139|#)\s*pass\s+(\d+)\s*$/m),
+    failed: total(/^\s*(?:\u2139|#)\s*fail\s+(\d+)\s*$/m),
+  };
+}
+
+export async function runSuite(args: string[]): Promise<number> {
+  const ran = args.filter((a) => a !== "--no-build");
+  const files = ran.length > 0 ? ran : [endToEnd];
+
+  if (!args.includes("--no-build")) {
+    // Before the run, always. The artifacts are built from two other repositories, and a suite
+    // that tests yesterday's binary reports on code nobody is looking at (novox/hq 04-ISSUES/005).
+    const built = rebuild();
+    if (built.length > 0) console.log(`built: ${built.join(", ")}\n`);
+  }
+  // Read now, while it is true. The receipt names these, and reading them when the run ends
+  // names whatever was committed during the twenty minutes in between instead.
+  const against = whatWasTested(process.env);
+
+  // **No canary.** There was one: a second scenario, one machine, raised first so a broken mesh
+  // failed in two minutes rather than in forty. It walked exactly the path the first three tests
+  // of the long run walk — a mesh comes up, a module lands, a consumer gets a credential — and
+  // the long run reaches the end of that path in about 160 seconds.
+  //
+  // So it cost a whole scenario, every passing run, to save about 45 seconds on a failing one.
+  // A scenario is three machines including a registry that boots a kernel to serve files, which
+  // is where the two minutes went. `test/integration/canary.test.ts` is still there and still
+  // runs when it is named; it is no longer raised on the way to everything else.
+
+  const { code, seen } = await runFiles(files);
+  console.log("\n" + reportOn(counted(seen), (p, f) => record(p, f, files, process.env, against)));
+  return code;
+}
+
+/** Run some test files, passing their output through as it arrives. */
+async function runFiles(files: string[]): Promise<{ code: number; seen: string }> {
+  const running = spawn(
+    process.execPath,
+    ["--test", "--test-concurrency=1", "--experimental-strip-types",
+      ...files],
+    { stdio: ["inherit", "pipe", "inherit"] },
+  );
+
+  let seen = "";
+  running.stdout.on("data", (chunk: Buffer) => {
+    // Passed through as it arrives: a suite that takes a quarter of an hour must not look hung.
+    process.stdout.write(chunk);
+    seen += chunk.toString();
+  });
+
+  const code: number = await new Promise((resolve) => {
+    running.on("close", (c) => resolve(c ?? 1));
+  });
+  return { code, seen };
+}
+
+/**
+ * reportOn decides whether this run says anything worth recording, and records it if so.
+ *
+ * Separated from the spawning so the decision can be tested: **no receipt rather than a guessed
+ * one** is the rule that keeps the record meaning something, and it was written where nothing
+ * could check it — which is 04-ISSUES/005 in miniature, inside the fix for it.
+ */
+export function reportOn(
+  counts: { passed: number | null; failed: number | null },
+  write: (passed: number, failed: number) => { against: Record<string, string>; ran: string[] },
+): string {
+  if (counts.passed === null || counts.failed === null) {
+    // A run whose result could not be read is a run nobody can say anything about. Writing
+    // "0 failed" because nothing said otherwise is how a green record comes to mean nothing.
+    return "could not read what the runner reported; no receipt written";
+  }
+  const receipt = write(counts.passed, counts.failed);
+  const against = Object.entries(receipt.against).map(([n, c]) => `${n} ${c}`).join(", ");
+  return `recorded: ${receipt.ran.join(", ")} — ${counts.passed} passed, ` +
+    `${counts.failed} failed, against ${against || "nothing in git"}`;
+}

src/warm.ts

@@ -0,0 +1,210 @@
+/**
+ * A scenario kept between runs, already brought to a state worth starting from.
+ *
+ * **Bootstrapping a mesh takes minutes and proves the same thing every time.** The tests worth
+ * iterating on are the ones after it — assigning a module, adopting a workload, watching something
+ * fail. A warm instance is raised once, brought to that state, snapshotted, and restored on every
+ * later run in seconds.
+ *
+ * **The danger is precisely the one 04-ISSUES/005 is about**, one level down: a mesh snapshotted
+ * against yesterday's binaries will pass today's tests and report green, and nothing about the
+ * result would say what it was actually run against. So a warm instance records the commits it was
+ * built from, and is refused — not silently rebuilt, refused — when they have moved.
+ *
+ * **Fresh stays the default.** This is for iterating. A run that is meant to mean something raises
+ * from nothing, because "it passes" must not quietly come to mean "it passes against a mesh
+ * somebody bootstrapped last week".
+ */
+
+import { readFileSync, writeFileSync, mkdirSync, rmSync } from "node:fs";
+import { dirname, join } from "node:path";
+import { homedir } from "node:os";
+
+import { list, restore, snapshot, snapshots, destroy } from "./lifecycle/operate.ts";
+import type { Against } from "./lastrun.ts";
+import { whatWasTested } from "./lastrun.ts";
+
+/** The state a warm instance is kept at. One label, because a second is a state nobody named. */
+export const label = "warm";
+
+export interface Warm {
+  scenario: string;
+  instanceId: string;
+  /**
+   * The image references the scenario's registry serves, pinned by digest.
+   *
+   * Kept because they are worked out while raising and a restored instance never raises. Without
+   * them a warm run knows nothing about what it can pull, and every test naming an image fails
+   * for a reason that has nothing to do with what it was testing.
+   */
+  images: string[];
+  /** The commit each repository was at when this was brought to its state. */
+  against: Against;
+  at: string;
+}
+
+/** Where the record lives: XDG state, beside the run receipt, for the same reason. */
+export function recordPath(): string {
+  const state = process.env["XDG_STATE_HOME"] ?? join(homedir(), ".local", "state");
+  return join(state, "mesh-lab", "warm.json");
+}
+
+export function remember(warm: Warm): void {
+  const path = recordPath();
+  mkdirSync(dirname(path), { recursive: true });
+  writeFileSync(path, JSON.stringify(warm, null, 2) + "\n");
+}
+
+export function remembered(): Warm | null {
+  try {
+    return JSON.parse(readFileSync(recordPath(), "utf8")) as Warm;
+  } catch {
+    return null;
+  }
+}
+
+export function forget(): void {
+  rmSync(recordPath(), { force: true });
+}
+
+export type Verdict =
+  | { use: "restore"; instanceId: string }
+  | { use: "raise"; why: string };
+
+/**
+ * judge decides whether a remembered instance may be restored.
+ *
+ * **Every reason to refuse is a reason a test would otherwise pass while meaning nothing**, so
+ * each is named rather than collapsed into "not usable".
+ */
+export function judge(
+  warm: Warm | null,
+  scenario: string,
+  standing: string[],
+  hasSnapshot: boolean,
+  against: Against,
+): Verdict {
+  if (!warm) return { use: "raise", why: "nothing is being kept warm" };
+  if (warm.scenario !== scenario) {
+    return { use: "raise", why: `what is kept warm is ${warm.scenario}, and this is ${scenario}` };
+  }
+  if (!standing.includes(warm.instanceId)) {
+    return { use: "raise", why: `${warm.instanceId} is no longer standing` };
+  }
+  if (!hasSnapshot) {
+    return { use: "raise", why: `${warm.instanceId} has no ${label} snapshot to return to` };
+  }
+  // The check that keeps this honest. A mesh built from code that has since moved would pass
+  // today's tests against yesterday's binaries, and say nothing about it.
+  //
+  // **Both directions, because comparing only what is in front of you clears what is not.** The
+  // first version walked the current repositories alone, so running without the environment that
+  // names where they are compared nothing and reported the mesh usable — a warm instance built
+  // from code that had since moved, cleared by a check that had looked at neither. That is
+  // 04-ISSUES/005's rule again: a record that says nothing about something is not a record that
+  // clears it.
+  for (const name of new Set([...Object.keys(warm.against), ...Object.keys(against)])) {
+    const then = warm.against[name];
+    const now = against[name];
+    if (then === now) continue;
+    if (!now) {
+      return {
+        use: "raise",
+        why: `${name} was at ${then} when this was warmed, and nothing says where it is now — ` +
+          `so nothing can say whether it moved`,
+      };
+    }
+    return {
+      use: "raise",
+      why: `${name} was at ${then ?? "nothing recorded"} when this was warmed, ` +
+        `and is now at ${now}`,
+    };
+  }
+  return { use: "restore", instanceId: warm.instanceId };
+}
+
+/** What is standing right now, by instance. */
+export async function standingNow(): Promise<string[]> {
+  return (await list()).map((i) => i.instanceId);
+}
+
+/**
+ * ready returns an instance already at its warm state, or says why one must be raised.
+ *
+ * It never raises: raising needs a scenario, images and a bootstrap, and all of that belongs to
+ * whoever is using this rather than here.
+ */
+export async function ready(
+  scenario: string,
+  env: NodeJS.ProcessEnv = process.env,
+): Promise<Verdict> {
+  const warm = remembered();
+  const standing = await standingNow();
+  const has = warm ? (await snapshots(warm.instanceId)).includes(label) : false;
+  return judge(warm, scenario, standing, has, whatWasTested(env));
+}
+
+/** returnTo puts a warm instance back to its state, and says how long it took. */
+export async function returnTo(
+  instanceId: string,
+  log: (message: string) => void = () => {},
+): Promise<number> {
+  const { usableSeconds } = await restore(instanceId, label, 180, log);
+  return usableSeconds;
+}
+
+/**
+ * keep snapshots an instance as the state to come back to, and records what it was built from.
+ *
+ * Called once the caller has brought the scenario to whatever "ready to work" means for it.
+ */
+export async function keep(
+  scenario: string,
+  instanceId: string,
+  env: NodeJS.ProcessEnv = process.env,
+): Promise<Warm> {
+  await snapshot(instanceId, label);
+  const warm: Warm = {
+    scenario,
+    instanceId,
+    images: stockOf(instanceId),
+    against: whatWasTested(env),
+    at: new Date().toISOString(),
+  };
+  remember(warm);
+  return warm;
+}
+
+/** cool destroys what is being kept and forgets it. */
+export async function cool(): Promise<string | null> {
+  const warm = remembered();
+  forget();
+  if (!warm) return null;
+  if ((await standingNow()).includes(warm.instanceId)) {
+    await destroy(warm.instanceId);
+  }
+  return warm.instanceId;
+}
+
+/**
+ * What a raised scenario stocked, held until it is kept.
+ *
+ * Raising works the images out and snapshotting happens later, so this carries them between the
+ * two without the caller having to hold them.
+ */
+const stock = new Map<string, string[]>();
+
+export function rememberStock(instanceId: string, images: string[]): void {
+  stock.set(instanceId, images);
+}
+
+function stockOf(instanceId: string): string[] {
+  return stock.get(instanceId) ?? [];
+}
+
+/** What a restored instance's registry serves, from when it was warmed. */
+export function warmStock(instanceId: string): { images: string[] } {
+  const warm = remembered();
+  if (!warm || warm.instanceId !== instanceId) return { images: [] };
+  return { images: warm.images };
+}

test/connect.test.ts

@@ -0,0 +1,14 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+
+import { RULE } from "../src/lifecycle/connect.ts";
+
+// The rule goes in its own file, beside the one this workstation already has.
+//
+// **Not into it.** The file next to this belongs to the mesh that really runs here, and it is
+// generated — writing into it would be edited-away at best and would break real name resolution
+// at worst. novox/hq: never edit a file something else owns.
+test("the rule is its own file, not the one already there", () => {
+  assert.match(RULE, /^\/etc\/dnsmasq\.d\/mesh-lab\.conf$/);
+  assert.doesNotMatch(RULE, /hal/, "it would be writing into something else's file");
+});

test/integration/assigned-audit.test.ts

@@ -0,0 +1,221 @@
+/**
+ * The mesh assigns the audit logger, and it consumes over an account the mesh delivered.
+ *
+ * events.test.ts proves the events path with the audit logger started by hand. This proves the
+ * whole of novox/hq ADR 0048: the module is assigned through the control plane, the mesh issues it
+ * a broker account scoped to what it consumes, seals it to the machine, and the host runs it as a
+ * container that connects over amqps with that account — never the broker's own. The trail filling
+ * is the proof the delivered, scoped credential authenticated and the subscription bound.
+ *
+ * It needs the host binary, the substrate bundle, and the runtime image stocked by the scenario:
+ *
+ *   MESH_LAB_HOST_BINARY=.../mesh-host
+ *   MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
+ *   scripts/build-runtime-image.sh builds mesh-runtime-audit:development into the local daemon,
+ *   which scenarios/audit-node.yml stocks — so no MESH_LAB_RUNTIME here; the host pulls it.
+ */
+
+import { test, before, after } from "node:test";
+import assert from "node:assert/strict";
+import { existsSync, readFileSync } from "node:fs";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { raise } from "../../src/lifecycle/raise.ts";
+import { destroy, exec } from "../../src/lifecycle/operate.ts";
+import { hostBinaryPath, HOST_PATH } from "../../src/lifecycle/place.ts";
+import { labIsUsable, destroyAll } from "./harness.ts";
+
+const capability = await labIsUsable();
+const binary = hostBinaryPath();
+const bundle = process.env["MESH_LAB_BUNDLE"] ?? "";
+
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !binary || !existsSync(binary)
+    ? "MESH_LAB_HOST_BINARY is not set to a built mesh-host"
+    : !bundle || !existsSync(bundle)
+      ? "MESH_LAB_BUNDLE is not set to a substrate bundle (mesh-host examples/)"
+      : false;
+
+const SCENARIO = "audit-node";
+const MACHINE = "anchor";
+
+let instanceId = "";
+/** What the scenario's registry serves, by digest. */
+let stocked: string[] = [];
+
+function quote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+async function on(command: string, timeoutMs?: number): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `exec 2>&1\n${command}\necho "__exit=$?"`,
+  ], timeoutMs);
+  const marker = stdout.lastIndexOf("__exit=");
+  if (marker < 0) return { out: stdout, ok: false };
+  return { out: stdout.slice(0, marker), ok: stdout.slice(marker + 7).trim() === "0" };
+}
+
+async function must(command: string, timeoutMs?: number): Promise<string> {
+  const { out, ok } = await on(command, timeoutMs);
+  if (!ok) throw new Error(`${MACHINE}: ${command}\n${out}`);
+  return out;
+}
+
+/** The control plane, a container on the node. */
+async function mesh(command: string, timeoutMs?: number): Promise<string> {
+  return must(`docker exec mesh-control /mesh-control ${command}`, timeoutMs);
+}
+
+/** The pinned reference for one of the scenario's images, by repository. */
+function pinned(repository: string): string {
+  const found = stocked.find((r) => r.slice(r.indexOf("/") + 1, r.indexOf("@")) === repository);
+  assert.ok(found, `the scenario stocks no ${repository}; it serves ${stocked.join(", ")}`);
+  return found;
+}
+
+/** The substrate bundle, its image references pointed at this scenario's own registry. */
+function bundleFor(images: string[]): string {
+  let text = readFileSync(bundle, "utf8");
+  for (const ref of images) {
+    const repository = ref.slice(ref.indexOf("/") + 1, ref.indexOf("@"));
+    const escaped = repository.replaceAll("/", "\\/").replaceAll(".", "\\.");
+    text = text.replaceAll(new RegExp(`[A-Za-z0-9_.:-]+\\/${escaped}@sha256:[0-9a-f]+`, "g"), ref);
+  }
+  return text;
+}
+
+function tokenFrom(said: string): string {
+  const found = said.split("\n").map((l) => l.trim()).find((l) => l.length > 100 && !l.includes(" "));
+  assert.ok(found, `no token in:\n${said}`);
+  return found;
+}
+
+async function settled(withinMs = 480_000): Promise<void> {
+  const until = Date.now() + withinMs;
+  let last = "";
+  while (Date.now() < until) {
+    const asked = await on(`docker exec mesh-control /mesh-control status --json`);
+    if (asked.ok) {
+      try {
+        const state = JSON.parse(asked.out) as {
+          wrong: { node: string; outcome: string }[];
+          waiting: { node: string }[];
+          reported: { node: string; outcome: string; current: boolean }[];
+        };
+        const bad = state.wrong.find((w) => w.node === MACHINE);
+        if (bad) throw new Error(`${MACHINE} did not apply what it was sent: ${bad.outcome}\n${asked.out}`);
+        const word = state.reported.find((r) => r.node === MACHINE);
+        if (!state.waiting.some((w) => w.node === MACHINE) && word?.outcome === "applied" && word.current) return;
+        last = asked.out;
+      } catch (err) {
+        if (err instanceof Error && err.message.includes("did not apply")) throw err;
+        last = asked.out;
+      }
+    }
+    await new Promise((r) => setTimeout(r, 5000));
+  }
+  throw new Error(`${MACHINE} never caught up within ${Math.round(withinMs / 1000)}s. Last:\n${last}`);
+}
+
+before(async () => {
+  if (skip) return;
+
+  const raised = await raise(loadScenario(`scenarios/${SCENARIO}.yml`), {
+    onProgress: (m) => console.log(`raise: ${m}`),
+  });
+  instanceId = raised.instanceId;
+  stocked = raised.images;
+
+  // Raise the substrate — store, broker, control — from the bundle.
+  await must(`cat > /tmp/substrate.lock <<'MESHBUNDLE'\n${bundleFor(raised.images)}\nMESHBUNDLE`);
+  await must(`${HOST_PATH} apply /tmp/substrate.lock`, 600_000);
+  const up = await must(`docker ps --format '{{.Names}}'`);
+  for (const c of ["mesh-store", "mesh-broker", "mesh-control"]) {
+    assert.match(up, new RegExp(c), `the substrate did not raise ${c}:\n${up}`);
+  }
+
+  // The node joins its own mesh, so it is a node the mesh can assign to, and start the host so it
+  // applies what it is pushed.
+  await mesh(`node add ${MACHINE}`);
+  const token = tokenFrom(await mesh(`token issue --node ${MACHINE}`));
+  await must(`${HOST_PATH} enrol --token ${quote(token)}`);
+  await must(`nohup ${HOST_PATH} run > /var/log/mesh-host.log 2>&1 & sleep 3`);
+}, { timeout: 1_800_000 });
+
+after(async () => {
+  if (instanceId) await destroy(instanceId);
+  await destroyAll(`${SCENARIO}-`);
+}, { timeout: 600_000 });
+
+test("the mesh assigns the audit logger, and it consumes over the account the mesh delivered", {
+  skip, timeout: 900_000,
+}, async () => {
+  // The assigned-module manifest (mesh-catalog), its runtime image the digest this registry serves.
+  const manifest = JSON.stringify({
+    module: "audit-logger",
+    version: "1",
+    consumes: ["#"],
+    "own-secrets": { broker: "/var/lib/audit-logger/broker" },
+    resources: [
+      { id: "state", type: "directory", path: "/var/lib/audit-logger", mode: "0700" },
+      { id: "trail", type: "directory", path: "/var/lib/audit-logger/trail", mode: "0700" },
+      {
+        id: "run", type: "container", name: "mesh-audit-logger", image: pinned("mesh-runtime-audit"),
+        network: "host",
+        volumes: [
+          "/var/lib/audit-logger/broker:/run/secrets/broker:ro",
+          "/var/lib/audit-logger/trail:/trail",
+        ],
+        env: { MESH_BROKER_FILE: "/run/secrets/broker", AUDIT_LOG: "/trail/audit.log" },
+      },
+    ],
+  });
+  await must(`printf %s ${quote(manifest)} > /tmp/audit.json && docker cp /tmp/audit.json mesh-control:/audit.json`);
+  await mesh("module add /audit.json");
+
+  // The mesh issues its scoped account and seals it to this machine, then assigns and pushes it.
+  const issued = await mesh(`module issue audit-logger --node ${MACHINE}`);
+  assert.match(issued, /scoped to what it emits and consumes/, issued);
+  await mesh(`assign ${MACHINE} audit-logger`);
+  await mesh(`push ${MACHINE}`);
+  await settled();
+
+  // The container the mesh started is running.
+  const running = await must(`docker ps --format '{{.Names}}'`);
+  assert.match(running, /mesh-audit-logger/,
+    `the audit logger was assigned and is not running:\n${(await on(`tail -30 /var/log/mesh-host.log`)).out}`);
+
+  // The credential on disk is the scoped account over amqps, sealed — not the broker's own.
+  const credential = await must(`cat /var/lib/audit-logger/broker`);
+  assert.match(credential, /"url":"amqps:\/\/anchor-audit-logger:/, `not the scoped account:\n${credential}`);
+  assert.doesNotMatch(credential, /guest:guest/, "the audit logger holds the broker's own account");
+  assert.match(credential, /"fingerprint":"(sha256:)?[0-9a-f]{64}"/, "no fingerprint to pin the broker");
+
+  // An event, emitted by a probe over the bootstrap account (the audit logger's own may not emit).
+  await must(
+    `docker run --rm --network host -e MESH_BROKER_URL=amqp://guest:guest@127.0.0.1:5672/ ` +
+      `-e MESH_MODULE=probe -e MESH_NODE=${MACHINE} ${pinned("mesh-runtime-audit")} ` +
+      `emit module.probe.site.created '{"domain":"my-app"}'`,
+  );
+
+  // It reaches the trail the assigned container writes — proof its delivered account authenticated.
+  let line: Record<string, unknown> | undefined;
+  const until = Date.now() + 60_000;
+  while (Date.now() < until) {
+    const raw = await must(`cat /var/lib/audit-logger/trail/audit.log 2>/dev/null || true`);
+    line = raw.split("\n").map((l) => l.trim()).filter(Boolean).map((l) => JSON.parse(l) as Record<string, unknown>)
+      .find((e) => e.type === "module.probe.site.created");
+    if (line) break;
+    await new Promise((r) => setTimeout(r, 3000));
+  }
+  assert.ok(line, `the event never reached the trail:\n${(await on(`docker logs mesh-audit-logger 2>&1 | tail -20`)).out}`);
+  assert.equal(line.source, "probe");
+  assert.deepEqual(line.body, { domain: "my-app" });
+
+  // And the account the mesh made for it is a real one on the broker — the trail above already
+  // proved it authenticated and read its queue. That it reaches no further than its own queue is
+  // the scope CreateModuleAccount applies, checked as patterns in mesh-control's own tests.
+  const users = await must(`docker exec mesh-broker lavinmqctl list_users 2>&1`);
+  assert.match(users, /anchor-audit-logger/, `the scoped account is not on the broker:\n${users}`);
+});

test/integration/assigned-grafana.test.ts

@@ -0,0 +1,216 @@
+/**
+ * The mesh assigns grafana's tool runtime, configured entirely by the assignment's settings — the
+ * ADR 0051 + 0052 case: config is the assignment's, delivered as a settings-merged file the runtime
+ * reads, not a credential baked into the manifest.
+ *
+ * plex/sonarr prove a runtime that self-detects its key from the app's own config. This proves the
+ * other half: the operator states grafana's URL and an API token as settings for this node, the
+ * control plane merges them into the module's mergeable config file, and the runtime reads that file
+ * at start, registers grafana's tools, and serves them under its scoped account. There is no live
+ * Grafana — that the serve queue is bound is the proof the settings reached the runtime and its
+ * tools loaded from them.
+ *
+ *   MESH_LAB_HOST_BINARY=.../mesh-host   MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
+ *   scripts/build-module-runtime.sh grafana builds mesh-runtime-grafana:development into the local
+ *   daemon, which scenarios/grafana-node.yml stocks — so no MESH_LAB_RUNTIME here; the host pulls it.
+ */
+
+import { test, before, after } from "node:test";
+import assert from "node:assert/strict";
+import { existsSync, readFileSync } from "node:fs";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { raise } from "../../src/lifecycle/raise.ts";
+import { destroy, exec } from "../../src/lifecycle/operate.ts";
+import { hostBinaryPath, HOST_PATH } from "../../src/lifecycle/place.ts";
+import { labIsUsable, destroyAll } from "./harness.ts";
+
+const capability = await labIsUsable();
+const binary = hostBinaryPath();
+const bundle = process.env["MESH_LAB_BUNDLE"] ?? "";
+
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !binary || !existsSync(binary)
+    ? "MESH_LAB_HOST_BINARY is not set to a built mesh-host"
+    : !bundle || !existsSync(bundle)
+      ? "MESH_LAB_BUNDLE is not set to a substrate bundle (mesh-host examples/)"
+      : false;
+
+const SCENARIO = "grafana-node";
+const MACHINE = "anchor";
+
+let instanceId = "";
+let stocked: string[] = [];
+
+function quote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+async function on(command: string, timeoutMs?: number): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `exec 2>&1\n${command}\necho "__exit=$?"`,
+  ], timeoutMs);
+  const marker = stdout.lastIndexOf("__exit=");
+  if (marker < 0) return { out: stdout, ok: false };
+  return { out: stdout.slice(0, marker), ok: stdout.slice(marker + 7).trim() === "0" };
+}
+
+async function must(command: string, timeoutMs?: number): Promise<string> {
+  const { out, ok } = await on(command, timeoutMs);
+  if (!ok) throw new Error(`${MACHINE}: ${command}\n${out}`);
+  return out;
+}
+
+async function mesh(command: string, timeoutMs?: number): Promise<string> {
+  return must(`docker exec mesh-control /mesh-control ${command}`, timeoutMs);
+}
+
+function pinned(repository: string): string {
+  const found = stocked.find((r) => r.slice(r.indexOf("/") + 1, r.indexOf("@")) === repository);
+  assert.ok(found, `the scenario stocks no ${repository}; it serves ${stocked.join(", ")}`);
+  return found;
+}
+
+function bundleFor(images: string[]): string {
+  let text = readFileSync(bundle, "utf8");
+  for (const ref of images) {
+    const repository = ref.slice(ref.indexOf("/") + 1, ref.indexOf("@"));
+    const escaped = repository.replaceAll("/", "\\/").replaceAll(".", "\\.");
+    text = text.replaceAll(new RegExp(`[A-Za-z0-9_.:-]+\\/${escaped}@sha256:[0-9a-f]+`, "g"), ref);
+  }
+  return text;
+}
+
+function tokenFrom(said: string): string {
+  const found = said.split("\n").map((l) => l.trim()).find((l) => l.length > 100 && !l.includes(" "));
+  assert.ok(found, `no token in:\n${said}`);
+  return found;
+}
+
+async function settled(withinMs = 480_000): Promise<void> {
+  const until = Date.now() + withinMs;
+  let last = "";
+  while (Date.now() < until) {
+    const asked = await on(`docker exec mesh-control /mesh-control status --json`);
+    if (asked.ok) {
+      try {
+        const state = JSON.parse(asked.out) as {
+          wrong: { node: string; outcome: string }[];
+          waiting: { node: string }[];
+          reported: { node: string; outcome: string; current: boolean }[];
+        };
+        const bad = state.wrong.find((w) => w.node === MACHINE);
+        if (bad) throw new Error(`${MACHINE} did not apply what it was sent: ${bad.outcome}\n${asked.out}`);
+        const word = state.reported.find((r) => r.node === MACHINE);
+        if (!state.waiting.some((w) => w.node === MACHINE) && word?.outcome === "applied" && word.current) return;
+        last = asked.out;
+      } catch (err) {
+        if (err instanceof Error && err.message.includes("did not apply")) throw err;
+        last = asked.out;
+      }
+    }
+    await new Promise((r) => setTimeout(r, 5000));
+  }
+  throw new Error(`${MACHINE} never caught up within ${Math.round(withinMs / 1000)}s. Last:\n${last}`);
+}
+
+before(async () => {
+  if (skip) return;
+
+  const raised = await raise(loadScenario(`scenarios/${SCENARIO}.yml`), {
+    onProgress: (m) => console.log(`raise: ${m}`),
+  });
+  instanceId = raised.instanceId;
+  stocked = raised.images;
+
+  await must(`cat > /tmp/substrate.lock <<'MESHBUNDLE'\n${bundleFor(raised.images)}\nMESHBUNDLE`);
+  await must(`${HOST_PATH} apply /tmp/substrate.lock`, 600_000);
+  const up = await must(`docker ps --format '{{.Names}}'`);
+  for (const c of ["mesh-store", "mesh-broker", "mesh-control"]) {
+    assert.match(up, new RegExp(c), `the substrate did not raise ${c}:\n${up}`);
+  }
+
+  await mesh(`node add ${MACHINE}`);
+  const token = tokenFrom(await mesh(`token issue --node ${MACHINE}`));
+  await must(`${HOST_PATH} enrol --token ${quote(token)}`);
+  await must(`nohup ${HOST_PATH} run > /var/log/mesh-host.log 2>&1 & sleep 3`);
+}, { timeout: 1_800_000 });
+
+after(async () => {
+  if (instanceId) await destroy(instanceId);
+  await destroyAll(`${SCENARIO}-`);
+}, { timeout: 600_000 });
+
+test("the mesh assigns grafana's runtime, configured by settings, and it serves its tools", {
+  skip, timeout: 900_000,
+}, async () => {
+  // A grafana manifest with no credential in it: its runtime, and a mergeable config file the
+  // settings will fill. This is the whole point of ADR 0051 — the manifest carries defaults and
+  // structure, the assignment carries the URL and token.
+  const manifest = JSON.stringify({
+    module: "grafana",
+    version: "1",
+    emits: ["module.grafana.alert.firing"],
+    "own-secrets": { broker: "/var/lib/mesh/grafana/broker" },
+    resources: [
+      { id: "mesh-state", type: "directory", path: "/var/lib/mesh/grafana", mode: "0700" },
+      { id: "config", type: "file", path: "/var/lib/mesh/grafana/config.json", mode: "0600", content: "{}\n", merge: "json" },
+      {
+        id: "runtime", type: "container", name: "mesh-grafana", image: pinned("mesh-runtime-grafana"),
+        network: "host",
+        volumes: [
+          "/var/lib/mesh/grafana/broker:/run/secrets/broker:ro",
+          "/var/lib/mesh/grafana/config.json:/run/config/config.json:ro",
+        ],
+        env: {
+          MESH_BROKER_FILE: "/run/secrets/broker",
+          MESH_GRAFANA_CONFIG_FILE: "/run/config/config.json",
+        },
+      },
+    ],
+  });
+  await must(`printf %s ${quote(manifest)} > /tmp/grafana.json && docker cp /tmp/grafana.json mesh-control:/grafana.json`);
+  await mesh("module add /grafana.json");
+
+  // The operator states grafana's URL and API token as settings for this node — the config the
+  // runtime will read. Nothing about them is in the manifest.
+  const settings = JSON.stringify({ url: "http://127.0.0.1:3000", token: "lab-grafana-token" });
+  await must(`printf %s ${quote(settings)} > /tmp/grafana-settings.json && docker cp /tmp/grafana-settings.json mesh-control:/grafana-settings.json`);
+  await mesh(`settings set grafana /grafana-settings.json --node ${MACHINE}`);
+
+  const issued = await mesh(`module issue grafana --node ${MACHINE}`);
+  assert.match(issued, /scoped to what it emits and consumes/, issued);
+  await mesh(`assign ${MACHINE} grafana`);
+  await mesh(`push ${MACHINE}`);
+  await settled();
+
+  const running = await must(`docker ps --format '{{.Names}}'`);
+  assert.match(running, /mesh-grafana/,
+    `grafana's runtime was assigned and is not running:\n${(await on(`tail -30 /var/log/mesh-host.log`)).out}`);
+
+  // The settings reached the node: the rendered config file carries what was set, not the manifest's
+  // empty default.
+  const config = await must(`cat /var/lib/mesh/grafana/config.json`);
+  assert.match(config, /lab-grafana-token/, `the settings did not merge into the config file:\n${config}`);
+
+  const credential = await must(`cat /var/lib/mesh/grafana/broker`);
+  assert.match(credential, /"url":"amqps:\/\/anchor-grafana:/, `not the scoped account:\n${credential}`);
+  assert.doesNotMatch(credential, /guest:guest/, "grafana's runtime holds the broker's own account");
+
+  // The runtime read that config, built its client from the settings-provided token, registered its
+  // tools, and bound their serve queues — the queue on the broker is the proof the settings-config
+  // path reached serving, with no credential in the manifest and no live Grafana.
+  let served = "";
+  const untilServing = Date.now() + 60_000;
+  while (Date.now() < untilServing) {
+    served = await must(`docker exec mesh-broker lavinmqctl list_queues name 2>&1 || true`);
+    if (/serve\.grafana\.grafana_status/.test(served)) break;
+    await new Promise((r) => setTimeout(r, 3000));
+  }
+  assert.match(served, /serve\.grafana\.grafana_status/,
+    `grafana's runtime never bound its serve queue (settings not read?):\n` +
+      `${(await on(`docker logs mesh-grafana 2>&1 | tail -20`)).out}\n---\n${served}`);
+
+  const users = await must(`docker exec mesh-broker lavinmqctl list_users 2>&1`);
+  assert.match(users, /anchor-grafana/, `the scoped account is not on the broker:\n${users}`);
+});

test/integration/assigned-plex.test.ts

@@ -0,0 +1,237 @@
+/**
+ * The mesh assigns plex's tool runtime, and it serves plex's tools over an account the mesh
+ * delivered — the whole of novox/hq ADR 0052.
+ *
+ * assigned-audit proves an assigned *consumer* (ADR 0048). This proves an assigned module that runs
+ * its OWN code as its OWN process under its OWN scoped account and *serves tools*: the module is
+ * assigned through the control plane, the mesh issues it an account scoped to serve.plex.* (and its
+ * events), seals it to the machine, and the host runs it as a container that binds amqps with that
+ * account. A caller then invokes plex.plex_reachable over the mesh and gets the tool's own answer —
+ * proof the invocation routed to the assigned runtime, ran plex's real code, and replied, all under
+ * the scoped account and never the broker's own.
+ *
+ * It needs the host binary, the substrate bundle, and the runtime image stocked by the scenario:
+ *
+ *   MESH_LAB_HOST_BINARY=.../mesh-host
+ *   MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
+ *   scripts/build-module-runtime.sh plex builds mesh-runtime-plex:development into the local daemon,
+ *   which scenarios/plex-node.yml stocks — so no MESH_LAB_RUNTIME here; the host pulls it.
+ */
+
+import { test, before, after } from "node:test";
+import assert from "node:assert/strict";
+import { existsSync, readFileSync } from "node:fs";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { raise } from "../../src/lifecycle/raise.ts";
+import { destroy, exec } from "../../src/lifecycle/operate.ts";
+import { hostBinaryPath, HOST_PATH } from "../../src/lifecycle/place.ts";
+import { labIsUsable, destroyAll } from "./harness.ts";
+
+const capability = await labIsUsable();
+const binary = hostBinaryPath();
+const bundle = process.env["MESH_LAB_BUNDLE"] ?? "";
+
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !binary || !existsSync(binary)
+    ? "MESH_LAB_HOST_BINARY is not set to a built mesh-host"
+    : !bundle || !existsSync(bundle)
+      ? "MESH_LAB_BUNDLE is not set to a substrate bundle (mesh-host examples/)"
+      : false;
+
+const SCENARIO = "plex-node";
+const MACHINE = "anchor";
+
+let instanceId = "";
+/** What the scenario's registry serves, by digest. */
+let stocked: string[] = [];
+
+function quote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+async function on(command: string, timeoutMs?: number): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `exec 2>&1\n${command}\necho "__exit=$?"`,
+  ], timeoutMs);
+  const marker = stdout.lastIndexOf("__exit=");
+  if (marker < 0) return { out: stdout, ok: false };
+  return { out: stdout.slice(0, marker), ok: stdout.slice(marker + 7).trim() === "0" };
+}
+
+async function must(command: string, timeoutMs?: number): Promise<string> {
+  const { out, ok } = await on(command, timeoutMs);
+  if (!ok) throw new Error(`${MACHINE}: ${command}\n${out}`);
+  return out;
+}
+
+/** The control plane, a container on the node. */
+async function mesh(command: string, timeoutMs?: number): Promise<string> {
+  return must(`docker exec mesh-control /mesh-control ${command}`, timeoutMs);
+}
+
+/** The pinned reference for one of the scenario's images, by repository. */
+function pinned(repository: string): string {
+  const found = stocked.find((r) => r.slice(r.indexOf("/") + 1, r.indexOf("@")) === repository);
+  assert.ok(found, `the scenario stocks no ${repository}; it serves ${stocked.join(", ")}`);
+  return found;
+}
+
+/** The substrate bundle, its image references pointed at this scenario's own registry. */
+function bundleFor(images: string[]): string {
+  let text = readFileSync(bundle, "utf8");
+  for (const ref of images) {
+    const repository = ref.slice(ref.indexOf("/") + 1, ref.indexOf("@"));
+    const escaped = repository.replaceAll("/", "\\/").replaceAll(".", "\\.");
+    text = text.replaceAll(new RegExp(`[A-Za-z0-9_.:-]+\\/${escaped}@sha256:[0-9a-f]+`, "g"), ref);
+  }
+  return text;
+}
+
+function tokenFrom(said: string): string {
+  const found = said.split("\n").map((l) => l.trim()).find((l) => l.length > 100 && !l.includes(" "));
+  assert.ok(found, `no token in:\n${said}`);
+  return found;
+}
+
+async function settled(withinMs = 480_000): Promise<void> {
+  const until = Date.now() + withinMs;
+  let last = "";
+  while (Date.now() < until) {
+    const asked = await on(`docker exec mesh-control /mesh-control status --json`);
+    if (asked.ok) {
+      try {
+        const state = JSON.parse(asked.out) as {
+          wrong: { node: string; outcome: string }[];
+          waiting: { node: string }[];
+          reported: { node: string; outcome: string; current: boolean }[];
+        };
+        const bad = state.wrong.find((w) => w.node === MACHINE);
+        if (bad) throw new Error(`${MACHINE} did not apply what it was sent: ${bad.outcome}\n${asked.out}`);
+        const word = state.reported.find((r) => r.node === MACHINE);
+        if (!state.waiting.some((w) => w.node === MACHINE) && word?.outcome === "applied" && word.current) return;
+        last = asked.out;
+      } catch (err) {
+        if (err instanceof Error && err.message.includes("did not apply")) throw err;
+        last = asked.out;
+      }
+    }
+    await new Promise((r) => setTimeout(r, 5000));
+  }
+  throw new Error(`${MACHINE} never caught up within ${Math.round(withinMs / 1000)}s. Last:\n${last}`);
+}
+
+before(async () => {
+  if (skip) return;
+
+  const raised = await raise(loadScenario(`scenarios/${SCENARIO}.yml`), {
+    onProgress: (m) => console.log(`raise: ${m}`),
+  });
+  instanceId = raised.instanceId;
+  stocked = raised.images;
+
+  // Raise the substrate — store, broker, control — from the bundle.
+  await must(`cat > /tmp/substrate.lock <<'MESHBUNDLE'\n${bundleFor(raised.images)}\nMESHBUNDLE`);
+  await must(`${HOST_PATH} apply /tmp/substrate.lock`, 600_000);
+  const up = await must(`docker ps --format '{{.Names}}'`);
+  for (const c of ["mesh-store", "mesh-broker", "mesh-control"]) {
+    assert.match(up, new RegExp(c), `the substrate did not raise ${c}:\n${up}`);
+  }
+
+  // The node joins its own mesh, so it is a node the mesh can assign to, and start the host so it
+  // applies what it is pushed.
+  await mesh(`node add ${MACHINE}`);
+  const token = tokenFrom(await mesh(`token issue --node ${MACHINE}`));
+  await must(`${HOST_PATH} enrol --token ${quote(token)}`);
+  await must(`nohup ${HOST_PATH} run > /var/log/mesh-host.log 2>&1 & sleep 3`);
+}, { timeout: 1_800_000 });
+
+after(async () => {
+  if (instanceId) await destroy(instanceId);
+  await destroyAll(`${SCENARIO}-`);
+}, { timeout: 600_000 });
+
+test("the mesh assigns plex's runtime, and it serves plex's tools over the account the mesh delivered", {
+  skip, timeout: 900_000,
+}, async () => {
+  // A minimal plex manifest: its tools/events runtime (no Plex server or media mounts in the lab),
+  // its emits and consumes so the account is scoped to those too, and a token in the environment so
+  // the tools register without a running Plex to detect one from. The runtime image is the digest
+  // this scenario's registry serves.
+  const manifest = JSON.stringify({
+    module: "plex",
+    version: "1",
+    emits: [
+      "module.plex.playback.started",
+      "module.plex.playback.stopped",
+      "module.plex.item.added",
+    ],
+    consumes: ["module.*.download.completed"],
+    "own-secrets": { broker: "/var/lib/mesh/plex/broker" },
+    resources: [
+      { id: "mesh-state", type: "directory", path: "/var/lib/mesh/plex", mode: "0700" },
+      {
+        id: "runtime", type: "container", name: "mesh-plex", image: pinned("mesh-runtime-plex"),
+        network: "host",
+        volumes: ["/var/lib/mesh/plex/broker:/run/secrets/broker:ro"],
+        env: {
+          MESH_BROKER_FILE: "/run/secrets/broker",
+          MESH_PLEX_URL: "http://127.0.0.1:32400",
+          MESH_PLEX_TOKEN: "lab-token",
+        },
+      },
+    ],
+  });
+  await must(`printf %s ${quote(manifest)} > /tmp/plex.json && docker cp /tmp/plex.json mesh-control:/plex.json`);
+  await mesh("module add /plex.json");
+
+  // The mesh issues plex's scoped account and seals it to this machine, then assigns and pushes it.
+  const issued = await mesh(`module issue plex --node ${MACHINE}`);
+  assert.match(issued, /scoped to what it emits and consumes/, issued);
+  await mesh(`assign ${MACHINE} plex`);
+  await mesh(`push ${MACHINE}`);
+  await settled();
+
+  // The runtime container the mesh started is running.
+  const running = await must(`docker ps --format '{{.Names}}'`);
+  assert.match(running, /mesh-plex/,
+    `plex's runtime was assigned and is not running:\n${(await on(`tail -30 /var/log/mesh-host.log`)).out}`);
+
+  // The credential on disk is the scoped account over amqps, sealed — not the broker's own.
+  const credential = await must(`cat /var/lib/mesh/plex/broker`);
+  assert.match(credential, /"url":"amqps:\/\/anchor-plex:/, `not the scoped account:\n${credential}`);
+  assert.doesNotMatch(credential, /guest:guest/, "plex's runtime holds the broker's own account");
+  assert.match(credential, /"fingerprint":"(sha256:)?[0-9a-f]{64}"/, "no fingerprint to pin the broker");
+
+  // The runtime registered and is serving its tools — the queue it declared is on the broker,
+  // named for the scope its account is granted (serve.plex.*).
+  let served = "";
+  const untilServing = Date.now() + 60_000;
+  while (Date.now() < untilServing) {
+    served = await must(`docker exec mesh-broker lavinmqctl list_queues name 2>&1 || true`);
+    if (/serve\.plex\.plex_reachable/.test(served)) break;
+    await new Promise((r) => setTimeout(r, 3000));
+  }
+  assert.match(served, /serve\.plex\.plex_reachable/,
+    `plex's runtime never bound its serve queue:\n${(await on(`docker logs mesh-plex 2>&1 | tail -20`)).out}\n---\n${served}`);
+
+  // A caller invokes plex.plex_reachable over the mesh, from the bootstrap account (a caller, like
+  // mesh-control's command API — plex's own account serves, it does not call). The reply is the
+  // tool's own answer: it ran in the assigned runtime and reported the Plex server is unreachable
+  // (there is none in the lab). A reply at all — not a timeout — is the proof the invocation routed
+  // to the assigned runtime and ran plex's real code under its scoped account.
+  const invoked = await must(
+    `docker run --rm --network host -e MESH_BROKER_URL=amqp://guest:guest@127.0.0.1:5672/ ` +
+      `${pinned("mesh-runtime-plex")} invoke plex plex_reachable`,
+    120_000,
+  );
+  const line = invoked.split("\n").map((l) => l.trim()).filter(Boolean).pop() ?? "";
+  const result = JSON.parse(line) as { reachable: boolean; url: string; error?: string };
+  assert.equal(result.reachable, false, `expected the lab's Plex to be unreachable:\n${invoked}`);
+  assert.match(result.url, /127\.0\.0\.1:32400/, `the tool ran but not against the configured server:\n${invoked}`);
+
+  // And the account the mesh made for it is a real one on the broker, scoped — proven above by the
+  // serve queue authenticating and the invocation round-tripping under it.
+  const users = await must(`docker exec mesh-broker lavinmqctl list_users 2>&1`);
+  assert.match(users, /anchor-plex/, `the scoped account is not on the broker:\n${users}`);
+});

test/integration/assigned-redis.test.ts

@@ -0,0 +1,279 @@
+/**
+ * The mesh assigns redis — a *provider* — and its runtime runs the provisioner AND the tools as one
+ * process under one account the mesh delivered (novox/hq ADR 0052).
+ *
+ * assigned-plex proves an assigned module that serves tools. This proves the provider half: redis's
+ * runtime binds its scoped account, serves redis's tools against the real server (redis_ping →
+ * PONG), and — the thing 0052 fixes — runs its provisioner in that same broker-bound process, so a
+ * grant is provisioned and its lifecycle event is emitted onto the mesh. Before 0052 the provisioner
+ * ran in a container with no broker and its emit could not fire at all.
+ *
+ * A caveat this test makes explicit: runProvisioner needs a seal key ($MESH_SEAL_KEY) and the mesh
+ * has no way yet to deliver one to a provider's runtime (04-ISSUES). The manifest here sets a
+ * lab-local key so the mechanism can be proven; the delivery is a separate, open design question.
+ *
+ * It needs the host binary, the substrate bundle, and the runtime image stocked by the scenario:
+ *
+ *   MESH_LAB_HOST_BINARY=.../mesh-host   MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
+ *   scripts/build-module-runtime.sh redis builds mesh-runtime-redis:development into the local
+ *   daemon, which scenarios/redis-node.yml stocks — so no MESH_LAB_RUNTIME here; the host pulls it.
+ */
+
+import { test, before, after } from "node:test";
+import assert from "node:assert/strict";
+import { existsSync, readFileSync } from "node:fs";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { raise } from "../../src/lifecycle/raise.ts";
+import { destroy, exec } from "../../src/lifecycle/operate.ts";
+import { hostBinaryPath, HOST_PATH } from "../../src/lifecycle/place.ts";
+import { labIsUsable, destroyAll } from "./harness.ts";
+
+const capability = await labIsUsable();
+const binary = hostBinaryPath();
+const bundle = process.env["MESH_LAB_BUNDLE"] ?? "";
+
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !binary || !existsSync(binary)
+    ? "MESH_LAB_HOST_BINARY is not set to a built mesh-host"
+    : !bundle || !existsSync(bundle)
+      ? "MESH_LAB_BUNDLE is not set to a substrate bundle (mesh-host examples/)"
+      : false;
+
+const SCENARIO = "redis-node";
+const MACHINE = "anchor";
+
+let instanceId = "";
+let stocked: string[] = [];
+
+function quote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+async function on(command: string, timeoutMs?: number): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `exec 2>&1\n${command}\necho "__exit=$?"`,
+  ], timeoutMs);
+  const marker = stdout.lastIndexOf("__exit=");
+  if (marker < 0) return { out: stdout, ok: false };
+  return { out: stdout.slice(0, marker), ok: stdout.slice(marker + 7).trim() === "0" };
+}
+
+async function must(command: string, timeoutMs?: number): Promise<string> {
+  const { out, ok } = await on(command, timeoutMs);
+  if (!ok) throw new Error(`${MACHINE}: ${command}\n${out}`);
+  return out;
+}
+
+async function mesh(command: string, timeoutMs?: number): Promise<string> {
+  return must(`docker exec mesh-control /mesh-control ${command}`, timeoutMs);
+}
+
+function pinned(repository: string): string {
+  const found = stocked.find((r) => r.slice(r.indexOf("/") + 1, r.indexOf("@")) === repository);
+  assert.ok(found, `the scenario stocks no ${repository}; it serves ${stocked.join(", ")}`);
+  return found;
+}
+
+function bundleFor(images: string[]): string {
+  let text = readFileSync(bundle, "utf8");
+  for (const ref of images) {
+    const repository = ref.slice(ref.indexOf("/") + 1, ref.indexOf("@"));
+    const escaped = repository.replaceAll("/", "\\/").replaceAll(".", "\\.");
+    text = text.replaceAll(new RegExp(`[A-Za-z0-9_.:-]+\\/${escaped}@sha256:[0-9a-f]+`, "g"), ref);
+  }
+  return text;
+}
+
+function tokenFrom(said: string): string {
+  const found = said.split("\n").map((l) => l.trim()).find((l) => l.length > 100 && !l.includes(" "));
+  assert.ok(found, `no token in:\n${said}`);
+  return found;
+}
+
+async function settled(withinMs = 480_000): Promise<void> {
+  const until = Date.now() + withinMs;
+  let last = "";
+  while (Date.now() < until) {
+    const asked = await on(`docker exec mesh-control /mesh-control status --json`);
+    if (asked.ok) {
+      try {
+        const state = JSON.parse(asked.out) as {
+          wrong: { node: string; outcome: string }[];
+          waiting: { node: string }[];
+          reported: { node: string; outcome: string; current: boolean }[];
+        };
+        const bad = state.wrong.find((w) => w.node === MACHINE);
+        if (bad) throw new Error(`${MACHINE} did not apply what it was sent: ${bad.outcome}\n${asked.out}`);
+        const word = state.reported.find((r) => r.node === MACHINE);
+        if (!state.waiting.some((w) => w.node === MACHINE) && word?.outcome === "applied" && word.current) return;
+        last = asked.out;
+      } catch (err) {
+        if (err instanceof Error && err.message.includes("did not apply")) throw err;
+        last = asked.out;
+      }
+    }
+    await new Promise((r) => setTimeout(r, 5000));
+  }
+  throw new Error(`${MACHINE} never caught up within ${Math.round(withinMs / 1000)}s. Last:\n${last}`);
+}
+
+before(async () => {
+  if (skip) return;
+
+  const raised = await raise(loadScenario(`scenarios/${SCENARIO}.yml`), {
+    onProgress: (m) => console.log(`raise: ${m}`),
+  });
+  instanceId = raised.instanceId;
+  stocked = raised.images;
+
+  await must(`cat > /tmp/substrate.lock <<'MESHBUNDLE'\n${bundleFor(raised.images)}\nMESHBUNDLE`);
+  await must(`${HOST_PATH} apply /tmp/substrate.lock`, 600_000);
+  const up = await must(`docker ps --format '{{.Names}}'`);
+  for (const c of ["mesh-store", "mesh-broker", "mesh-control"]) {
+    assert.match(up, new RegExp(c), `the substrate did not raise ${c}:\n${up}`);
+  }
+
+  await mesh(`node add ${MACHINE}`);
+  const token = tokenFrom(await mesh(`token issue --node ${MACHINE}`));
+  await must(`${HOST_PATH} enrol --token ${quote(token)}`);
+  await must(`nohup ${HOST_PATH} run > /var/log/mesh-host.log 2>&1 & sleep 3`);
+}, { timeout: 1_800_000 });
+
+after(async () => {
+  if (instanceId) await destroy(instanceId);
+  await destroyAll(`${SCENARIO}-`);
+}, { timeout: 600_000 });
+
+test("the mesh assigns redis, and its runtime serves tools and provisions grants over the account the mesh delivered", {
+  skip, timeout: 900_000,
+}, async () => {
+  // A redis manifest with both halves it needs on this node: the redis server, and one broker-bound
+  // runtime that serves redis's tools AND runs its provisioner. Both reach the server over the host
+  // (127.0.0.1:6379) with the same admin password the mesh generated. MESH_SEAL_KEY is lab-local —
+  // the mesh cannot yet deliver one to a provider's runtime (see the file header / 04-ISSUES).
+  const manifest = JSON.stringify({
+    module: "redis",
+    version: "1",
+    emits: ["module.redis.cache.provisioned", "module.redis.cache.deprovisioned"],
+    // redis's events entrypoint subscribes to its own lifecycle events (an audit-trail log), so it
+    // consumes them too — declared, or the substrate never makes the queue the runtime binds and it
+    // crashes on start with a 404 (novox/hq ADR 0046: a consume is declared).
+    consumes: ["module.redis.cache.provisioned", "module.redis.cache.deprovisioned"],
+    "own-secrets": { default: "/var/lib/redis-module/default.secret", broker: "/var/lib/mesh/redis/broker" },
+    resources: [
+      { id: "mesh-state", type: "directory", path: "/var/lib/mesh/redis", mode: "0700" },
+      { id: "state", type: "directory", path: "/var/lib/redis-module", mode: "0700" },
+      { id: "grants-dir", type: "directory", path: "/var/lib/redis-module/grants", mode: "0700" },
+      { id: "data", type: "directory", path: "/services/redis/data", mode: "0700", owner: "999:999" },
+      {
+        id: "server-conf", type: "file", path: "/var/lib/redis-module/redis.conf", mode: "0644",
+        content: "requirepass ${secret:default}\nappendonly no\ndir /data\n",
+      },
+      {
+        id: "server", type: "container", name: "redis", image: pinned("redis"), network: "host",
+        volumes: [
+          "/services/redis/data:/data",
+          "/var/lib/redis-module/redis.conf:/etc/redis/redis.conf:ro",
+        ],
+        args: ["/etc/redis/redis.conf"],
+      },
+      {
+        id: "runtime", type: "container", name: "mesh-redis", image: pinned("mesh-runtime-redis"),
+        network: "host",
+        volumes: [
+          "/var/lib/mesh/redis/broker:/run/secrets/broker:ro",
+          "/var/lib/redis-module/grants:/var/lib/redis-module/grants",
+          "/var/lib/redis-module/default.secret:/run/secrets/default:ro",
+        ],
+        env: {
+          MESH_BROKER_FILE: "/run/secrets/broker",
+          GRANTS: "/var/lib/redis-module/grants",
+          MESH_PROVISION_REDIS: "127.0.0.1:6379",
+          MESH_PROVISION_PASSWORD_FILE: "/run/secrets/default",
+          MESH_SEAL_KEY: "lab-only-seal-key",
+        },
+      },
+    ],
+  });
+  await must(`printf %s ${quote(manifest)} > /tmp/redis.json && docker cp /tmp/redis.json mesh-control:/redis.json`);
+  await mesh("module add /redis.json");
+
+  const issued = await mesh(`module issue redis --node ${MACHINE}`);
+  assert.match(issued, /scoped to what it emits and consumes/, issued);
+  await mesh(`assign ${MACHINE} redis`);
+  await mesh(`push ${MACHINE}`);
+  await settled();
+
+  // The server and the runtime the mesh started are both running.
+  const running = await must(`docker ps --format '{{.Names}}'`);
+  assert.match(running, /\bredis\b/, `redis's server is not running:\n${(await on(`tail -30 /var/log/mesh-host.log`)).out}`);
+  assert.match(running, /mesh-redis/, `redis's runtime is not running:\n${(await on(`docker logs mesh-redis 2>&1 | tail -20`)).out}`);
+
+  // The credential on disk is the scoped account over amqps, sealed — not the broker's own.
+  const credential = await must(`cat /var/lib/mesh/redis/broker`);
+  assert.match(credential, /"url":"amqps:\/\/anchor-redis:/, `not the scoped account:\n${credential}`);
+  assert.doesNotMatch(credential, /guest:guest/, "redis's runtime holds the broker's own account");
+  assert.match(credential, /"fingerprint":"(sha256:)?[0-9a-f]{64}"/, "no fingerprint to pin the broker");
+
+  // The runtime registered and is serving its tools — the serve queue is on the broker.
+  let served = "";
+  const untilServing = Date.now() + 60_000;
+  while (Date.now() < untilServing) {
+    served = await must(`docker exec mesh-broker lavinmqctl list_queues name 2>&1 || true`);
+    if (/serve\.redis\.redis_ping/.test(served)) break;
+    await new Promise((r) => setTimeout(r, 3000));
+  }
+  assert.match(served, /serve\.redis\.redis_ping/,
+    `redis's runtime never bound its serve queue:\n${(await on(`docker logs mesh-redis 2>&1 | tail -30`)).out}\n---\n${served}`);
+
+  // A caller invokes redis.redis_ping over the mesh: the tool runs in the assigned runtime, reaches
+  // the real redis, and answers PONG. A positive round-trip against a real backend.
+  const pinged = await must(
+    `docker run --rm --network host -e MESH_BROKER_URL=amqp://guest:guest@127.0.0.1:5672/ ` +
+      `${pinned("mesh-runtime-redis")} invoke redis redis_ping`,
+    120_000,
+  );
+  const pingLine = pinged.split("\n").map((l) => l.trim()).filter(Boolean).pop() ?? "";
+  const ping = JSON.parse(pingLine) as { ok: unknown };
+  assert.ok(String(ping.ok).toUpperCase().includes("PONG") || ping.ok === true,
+    `redis_ping did not answer PONG through the mesh:\n${pinged}`);
+
+  // The provider path: a grant appears (as the control plane would write it), and the provisioner —
+  // running inside the same broker-bound runtime — creates the ACL user and emits the lifecycle
+  // event. The sealed credential the harness writes only after adapter.create() returns is the
+  // proof create() ran to completion; and because emit() awaits the broker's publish confirm
+  // (ADR 0047), a completed create() means the provisioned event was accepted onto the mesh.
+  const grant = JSON.stringify({ resource: "redis-cache", consumer: "app-one", node: MACHINE, values: {} });
+  await must(`printf %s ${quote(grant)} > /var/lib/redis-module/grants/app-one.grant.json`);
+
+  let credentialWritten = false;
+  const untilProvisioned = Date.now() + 60_000;
+  while (Date.now() < untilProvisioned) {
+    const ls = await on(`ls /var/lib/redis-module/grants/`);
+    if (ls.ok && /app-one\.redis-cache\.credential/.test(ls.out)) { credentialWritten = true; break; }
+    await new Promise((r) => setTimeout(r, 3000));
+  }
+  assert.ok(credentialWritten,
+    `the provisioner never provisioned the grant (no emit under a bound broker?):\n` +
+      `${(await on(`docker logs mesh-redis 2>&1 | tail -30`)).out}`);
+
+  // No emit failed: the provisioner's announce() logs "emit ... failed" only when the broker refused
+  // the publish. Its absence, with the credential written, is the provisioner emitting on the mesh.
+  const runtimeLog = (await on(`docker logs mesh-redis 2>&1`)).out;
+  assert.doesNotMatch(runtimeLog, /emit .*failed/,
+    `the provisioner's emit was refused — the account cannot publish its lifecycle event:\n${runtimeLog}`);
+
+  // And the ACL user the provisioner created is really on the redis server — the provisioning did
+  // its own half, not only the mesh bookkeeping. Asked through the same served tool surface.
+  const acl = await must(
+    `docker run --rm --network host -e MESH_BROKER_URL=amqp://guest:guest@127.0.0.1:5672/ ` +
+      `${pinned("mesh-runtime-redis")} invoke redis redis_command '{"command":"ACL LIST"}'`,
+    120_000,
+  );
+  assert.match(acl, /app-one/, `the provisioner did not create the consumer's ACL user on redis:\n${acl}`);
+
+  // The scoped account the mesh made for it is a real one on the broker.
+  const users = await must(`docker exec mesh-broker lavinmqctl list_users 2>&1`);
+  assert.match(users, /anchor-redis/, `the scoped account is not on the broker:\n${users}`);
+});

test/integration/assigned-sonarr.test.ts

@@ -0,0 +1,220 @@
+/**
+ * The mesh assigns sonarr's tool runtime, and it serves sonarr's tools over an account the mesh
+ * delivered — the Servarr case of novox/hq ADR 0052.
+ *
+ * assigned-plex proved a tools+events module that self-detects its token from a mounted config dir.
+ * This proves that self-configuring pattern generalises to the Servarr family: sonarr's runtime
+ * detects its API key from the server's own config.xml (a file resource stands in for the running
+ * Sonarr here), registers its tools, and serves them under a scoped account. There is no live Sonarr
+ * to reach — that the serve queue is bound is the proof the key was detected and the tools loaded.
+ *
+ *   MESH_LAB_HOST_BINARY=.../mesh-host   MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
+ *   scripts/build-module-runtime.sh sonarr builds mesh-runtime-sonarr:development into the local
+ *   daemon, which scenarios/sonarr-node.yml stocks — so no MESH_LAB_RUNTIME here; the host pulls it.
+ */
+
+import { test, before, after } from "node:test";
+import assert from "node:assert/strict";
+import { existsSync, readFileSync } from "node:fs";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { raise } from "../../src/lifecycle/raise.ts";
+import { destroy, exec } from "../../src/lifecycle/operate.ts";
+import { hostBinaryPath, HOST_PATH } from "../../src/lifecycle/place.ts";
+import { labIsUsable, destroyAll } from "./harness.ts";
+
+const capability = await labIsUsable();
+const binary = hostBinaryPath();
+const bundle = process.env["MESH_LAB_BUNDLE"] ?? "";
+
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !binary || !existsSync(binary)
+    ? "MESH_LAB_HOST_BINARY is not set to a built mesh-host"
+    : !bundle || !existsSync(bundle)
+      ? "MESH_LAB_BUNDLE is not set to a substrate bundle (mesh-host examples/)"
+      : false;
+
+const SCENARIO = "sonarr-node";
+const MACHINE = "anchor";
+
+let instanceId = "";
+let stocked: string[] = [];
+
+function quote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+async function on(command: string, timeoutMs?: number): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `exec 2>&1\n${command}\necho "__exit=$?"`,
+  ], timeoutMs);
+  const marker = stdout.lastIndexOf("__exit=");
+  if (marker < 0) return { out: stdout, ok: false };
+  return { out: stdout.slice(0, marker), ok: stdout.slice(marker + 7).trim() === "0" };
+}
+
+async function must(command: string, timeoutMs?: number): Promise<string> {
+  const { out, ok } = await on(command, timeoutMs);
+  if (!ok) throw new Error(`${MACHINE}: ${command}\n${out}`);
+  return out;
+}
+
+async function mesh(command: string, timeoutMs?: number): Promise<string> {
+  return must(`docker exec mesh-control /mesh-control ${command}`, timeoutMs);
+}
+
+function pinned(repository: string): string {
+  const found = stocked.find((r) => r.slice(r.indexOf("/") + 1, r.indexOf("@")) === repository);
+  assert.ok(found, `the scenario stocks no ${repository}; it serves ${stocked.join(", ")}`);
+  return found;
+}
+
+function bundleFor(images: string[]): string {
+  let text = readFileSync(bundle, "utf8");
+  for (const ref of images) {
+    const repository = ref.slice(ref.indexOf("/") + 1, ref.indexOf("@"));
+    const escaped = repository.replaceAll("/", "\\/").replaceAll(".", "\\.");
+    text = text.replaceAll(new RegExp(`[A-Za-z0-9_.:-]+\\/${escaped}@sha256:[0-9a-f]+`, "g"), ref);
+  }
+  return text;
+}
+
+function tokenFrom(said: string): string {
+  const found = said.split("\n").map((l) => l.trim()).find((l) => l.length > 100 && !l.includes(" "));
+  assert.ok(found, `no token in:\n${said}`);
+  return found;
+}
+
+async function settled(withinMs = 480_000): Promise<void> {
+  const until = Date.now() + withinMs;
+  let last = "";
+  while (Date.now() < until) {
+    const asked = await on(`docker exec mesh-control /mesh-control status --json`);
+    if (asked.ok) {
+      try {
+        const state = JSON.parse(asked.out) as {
+          wrong: { node: string; outcome: string }[];
+          waiting: { node: string }[];
+          reported: { node: string; outcome: string; current: boolean }[];
+        };
+        const bad = state.wrong.find((w) => w.node === MACHINE);
+        if (bad) throw new Error(`${MACHINE} did not apply what it was sent: ${bad.outcome}\n${asked.out}`);
+        const word = state.reported.find((r) => r.node === MACHINE);
+        if (!state.waiting.some((w) => w.node === MACHINE) && word?.outcome === "applied" && word.current) return;
+        last = asked.out;
+      } catch (err) {
+        if (err instanceof Error && err.message.includes("did not apply")) throw err;
+        last = asked.out;
+      }
+    }
+    await new Promise((r) => setTimeout(r, 5000));
+  }
+  throw new Error(`${MACHINE} never caught up within ${Math.round(withinMs / 1000)}s. Last:\n${last}`);
+}
+
+before(async () => {
+  if (skip) return;
+
+  const raised = await raise(loadScenario(`scenarios/${SCENARIO}.yml`), {
+    onProgress: (m) => console.log(`raise: ${m}`),
+  });
+  instanceId = raised.instanceId;
+  stocked = raised.images;
+
+  await must(`cat > /tmp/substrate.lock <<'MESHBUNDLE'\n${bundleFor(raised.images)}\nMESHBUNDLE`);
+  await must(`${HOST_PATH} apply /tmp/substrate.lock`, 600_000);
+  const up = await must(`docker ps --format '{{.Names}}'`);
+  for (const c of ["mesh-store", "mesh-broker", "mesh-control"]) {
+    assert.match(up, new RegExp(c), `the substrate did not raise ${c}:\n${up}`);
+  }
+
+  await mesh(`node add ${MACHINE}`);
+  const token = tokenFrom(await mesh(`token issue --node ${MACHINE}`));
+  await must(`${HOST_PATH} enrol --token ${quote(token)}`);
+  await must(`nohup ${HOST_PATH} run > /var/log/mesh-host.log 2>&1 & sleep 3`);
+}, { timeout: 1_800_000 });
+
+after(async () => {
+  if (instanceId) await destroy(instanceId);
+  await destroyAll(`${SCENARIO}-`);
+}, { timeout: 600_000 });
+
+test("the mesh assigns sonarr's runtime, and it detects its key and serves its tools", {
+  skip, timeout: 900_000,
+}, async () => {
+  // A minimal sonarr manifest: its tool runtime, and a config.xml the runtime detects its API key
+  // from — the file resource stands in for the running Sonarr that would write it. No Sonarr server
+  // or media mounts; the tools simply have nothing live to reach.
+  const manifest = JSON.stringify({
+    module: "sonarr",
+    version: "1",
+    emits: ["module.sonarr.episode.grabbed", "module.sonarr.download.completed"],
+    consumes: [],
+    "own-secrets": { broker: "/var/lib/mesh/sonarr/broker" },
+    resources: [
+      { id: "mesh-state", type: "directory", path: "/var/lib/mesh/sonarr", mode: "0700" },
+      { id: "config", type: "directory", path: "/services/sonarr/config", mode: "0700" },
+      {
+        id: "config-xml", type: "file", path: "/services/sonarr/config/config.xml", mode: "0644",
+        content: "<Config>\n  <Port>8989</Port>\n  <ApiKey>labdetectedapikey0000000000000000</ApiKey>\n</Config>\n",
+      },
+      {
+        id: "runtime", type: "container", name: "mesh-sonarr", image: pinned("mesh-runtime-sonarr"),
+        network: "host",
+        volumes: [
+          "/var/lib/mesh/sonarr/broker:/run/secrets/broker:ro",
+          "/services/sonarr/config:/var/lib/sonarr/config:ro",
+        ],
+        env: {
+          MESH_BROKER_FILE: "/run/secrets/broker",
+          MESH_SONARR_URL: "http://127.0.0.1:8989",
+          MESH_SONARR_CONFIG_DIR: "/var/lib/sonarr/config",
+        },
+      },
+    ],
+  });
+  await must(`printf %s ${quote(manifest)} > /tmp/sonarr.json && docker cp /tmp/sonarr.json mesh-control:/sonarr.json`);
+  await mesh("module add /sonarr.json");
+
+  const issued = await mesh(`module issue sonarr --node ${MACHINE}`);
+  assert.match(issued, /scoped to what it emits and consumes/, issued);
+  await mesh(`assign ${MACHINE} sonarr`);
+  await mesh(`push ${MACHINE}`);
+  await settled();
+
+  const running = await must(`docker ps --format '{{.Names}}'`);
+  assert.match(running, /mesh-sonarr/,
+    `sonarr's runtime was assigned and is not running:\n${(await on(`tail -30 /var/log/mesh-host.log`)).out}`);
+
+  const credential = await must(`cat /var/lib/mesh/sonarr/broker`);
+  assert.match(credential, /"url":"amqps:\/\/anchor-sonarr:/, `not the scoped account:\n${credential}`);
+  assert.doesNotMatch(credential, /guest:guest/, "sonarr's runtime holds the broker's own account");
+  assert.match(credential, /"fingerprint":"(sha256:)?[0-9a-f]{64}"/, "no fingerprint to pin the broker");
+
+  // The runtime detected its API key from config.xml, registered its tools, and bound their serve
+  // queues — the queue on the broker is the proof the whole chain worked with no live Sonarr.
+  let served = "";
+  const untilServing = Date.now() + 60_000;
+  while (Date.now() < untilServing) {
+    served = await must(`docker exec mesh-broker lavinmqctl list_queues name 2>&1 || true`);
+    if (/serve\.sonarr\.sonarr_status/.test(served)) break;
+    await new Promise((r) => setTimeout(r, 3000));
+  }
+  assert.match(served, /serve\.sonarr\.sonarr_status/,
+    `sonarr's runtime never bound its serve queue (key not detected?):\n` +
+      `${(await on(`docker logs mesh-sonarr 2>&1 | tail -20`)).out}\n---\n${served}`);
+
+  // A caller invokes sonarr_status over the mesh: it routes to the assigned runtime, which runs
+  // sonarr's real code and reports Sonarr unreachable (there is none). A reply — not a timeout — is
+  // the proof the invocation reached the runtime under its scoped account.
+  const invoked = await on(
+    `docker run --rm --network host -e MESH_BROKER_URL=amqp://guest:guest@127.0.0.1:5672/ ` +
+      `${pinned("mesh-runtime-sonarr")} invoke sonarr sonarr_status`,
+    120_000,
+  );
+  assert.doesNotMatch(invoked.out, /timed out/,
+    `sonarr_status timed out — nothing served the invocation:\n${invoked.out}`);
+
+  const users = await must(`docker exec mesh-broker lavinmqctl list_users 2>&1`);
+  assert.match(users, /anchor-sonarr/, `the scoped account is not on the broker:\n${users}`);
+});

test/integration/canary.test.ts

@@ -0,0 +1,177 @@
+/**
+ * The shortest run that would have caught today's faults.
+ *
+ * **A suite that takes forty minutes is a suite you find out from once a day.** Every fault found
+ * on 2026-09-01 — a module pinned to an image that does not exist, a consumer given a password and
+ * no name to present with it, a credential file nothing could read, a search for a password that
+ * read the password as an option — would have shown up in the first three minutes of it. The other
+ * thirty-seven proved things that were already working.
+ *
+ * So this runs first, on one machine, with the three images the mesh needs for itself and nothing
+ * else. If it fails there is no point spending the rest.
+ *
+ * It is deliberately *not* a smaller copy of the full suite. It walks one path end to end — a mesh
+ * comes up, a module reaches a machine, and a consumer gets a credential it can actually use —
+ * because that path is where everything went wrong, and a canary that checks many things shallowly
+ * is a canary nobody can read the failure of.
+ */
+
+import { test, before, after } from "node:test";
+import assert from "node:assert/strict";
+import { existsSync, readFileSync } from "node:fs";
+
+import { raise } from "../../src/lifecycle/raise.ts";
+import { exec, destroy } from "../../src/lifecycle/operate.ts";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { labIsUsable } from "./harness.ts";
+import { pinnedInto } from "../../src/pinning.ts";
+
+const capability = await labIsUsable();
+const host = process.env["MESH_LAB_HOST_BINARY"] ?? "";
+const bundle = process.env["MESH_LAB_BUNDLE"] ?? "";
+
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !host || !existsSync(host)
+    ? "MESH_LAB_HOST_BINARY is not set to a built mesh-host"
+    : !bundle || !existsSync(bundle)
+      ? "MESH_LAB_BUNDLE is not set to a substrate bundle"
+      : false;
+
+const SCENARIO = "first-node";
+const MACHINE = "anchor";
+const HOST_PATH = "/usr/local/bin/mesh-host";
+let instanceId = "";
+
+function quote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+async function on(command: string, timeoutMs?: number): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `exec 2>&1\n${command}\necho "__exit=$?"`,
+  ], timeoutMs);
+  const marker = stdout.lastIndexOf("__exit=");
+  return { out: stdout.slice(0, marker), ok: Number(stdout.slice(marker + 7).trim()) === 0 };
+}
+
+async function must(command: string, timeoutMs?: number): Promise<string> {
+  const { out, ok } = await on(command, timeoutMs);
+  if (!ok) throw new Error(`${MACHINE}: ${command}\n${out}`);
+  return out;
+}
+
+const mesh = (command: string, timeoutMs?: number) =>
+  must(`docker exec mesh-control /mesh-control ${command}`, timeoutMs);
+
+before(async () => {
+  if (skip) return;
+  const raised = await raise(loadScenario(`scenarios/${SCENARIO}.yml`), {
+    onProgress: (m) => console.log(`raise: ${m}`),
+  });
+  instanceId = raised.instanceId;
+  await must(`cat > /tmp/substrate.lock <<'MESHBUNDLE'\n${
+    pinnedInto(readFileSync(bundle, "utf8"), raised.images)}\nMESHBUNDLE`);
+  await must(`${HOST_PATH} apply /tmp/substrate.lock`, 300_000);
+
+  // **And the machine joins.** Applying the bundle raises a control plane; it does not tell that
+  // control plane a machine exists. Leaving this out is what the first run of this canary found,
+  // in under three minutes: the mesh answered, said "0 machine(s)", and every assignment after it
+  // failed with `no node of that name`.
+  await mesh(`node add ${MACHINE}`);
+  const said = await mesh(`token issue --node ${MACHINE}`);
+  const token = said.split("\n").map((l) => l.trim())
+    .find((l) => l.length > 100 && !l.includes(" "));
+  assert.ok(token, `no token in:\n${said}`);
+  await must(`${HOST_PATH} enrol --token ${quote(token)}`);
+
+  // The host has to be running for a push to reach it.
+  await must(`pgrep -x mesh-host >/dev/null || ` +
+    `(setsid nohup ${HOST_PATH} run > /var/log/mesh-host.log 2>&1 < /dev/null & sleep 3)`);
+});
+
+after(async () => {
+  // The canary owns its scenario and takes it down. Left standing it would hold a machine for
+  // the forty minutes of the run it exists to protect.
+  if (instanceId) await destroy(instanceId).catch(() => {});
+});
+
+test("a mesh comes up and answers", { skip, timeout: 600_000 }, async () => {
+  const said = await mesh("status");
+  assert.match(said, /anchor/, `the mesh does not know the machine it is running on:\n${said}`);
+});
+
+// One module, no images, nothing to stock. What is under test is the chain — added, assigned,
+// resolved, planned, pushed, applied, reported — not what is at the end of it.
+test("a module reaches the machine", { skip, timeout: 600_000 }, async () => {
+  await must(`printf %s ${quote(JSON.stringify({
+    module: "canary", version: "1",
+    resources: [
+      { id: "state", type: "directory", path: "/var/lib/canary", mode: "0700" },
+      { id: "note", type: "file", path: "/var/lib/canary/it-arrived", content: "yes\n", mode: "0644" },
+    ],
+  }))} > /tmp/canary.json`);
+  await must(`docker cp /tmp/canary.json mesh-control:/canary.json`);
+  await mesh("module add /canary.json");
+  await mesh(`assign ${MACHINE} canary`);
+  await mesh(`push ${MACHINE}`, 300_000);
+
+  assert.equal((await must(`cat /var/lib/canary/it-arrived`)).trim(), "yes");
+  assert.match(await must(`stat -c %a /var/lib/canary`), /^700/);
+});
+
+// **The half that broke all day.** A provider and a consumer on one machine: the mesh makes a
+// credential, tells the provider who asked, and gives the consumer a file it can read — with the
+// name to present, which it could not have known (novox/hq 04-ISSUES/021, 022, 023).
+test("a consumer gets a credential it can use", { skip, timeout: 600_000 }, async () => {
+  await must(`printf %s ${quote(JSON.stringify({
+    module: "canary-store", version: "1",
+    provides: [{ name: "canary-database", scope: "mesh" }],
+    serves: { "canary-database": { port: 5432 } },
+    receives: { "canary-database": "/var/lib/canary-store/asked.json" },
+    grants: { "canary-database": "/var/lib/canary-store/grants" },
+    resources: [
+      { id: "state", type: "directory", path: "/var/lib/canary-store", mode: "0700" },
+      { id: "grants", type: "directory", path: "/var/lib/canary-store/grants", mode: "0700" },
+    ],
+  }))} > /tmp/canary-store.json`);
+  await must(`printf %s ${quote(JSON.stringify({
+    module: "canary-app", version: "1",
+    requires: ["canary-database"],
+    contributes: { "canary-database": { name: "canaryapp" } },
+    binds: { "canary-database": "/var/lib/canary-app/where.json" },
+    secrets: { "canary-database": "/var/lib/canary-app/password" },
+    resources: [
+      { id: "state", type: "directory", path: "/var/lib/canary-app", mode: "0700" },
+      {
+        id: "env", type: "file", path: "/var/lib/canary-app/database.env", mode: "0600",
+        content: "PGHOST=${bound:canary-database:at}\nPGPORT=${bound:canary-database:port}\n" +
+          "PGUSER=${bound:canary-database:as}\nPGPASSWORD=${secret:canary-database}\n",
+      },
+    ],
+  }))} > /tmp/canary-app.json`);
+  for (const name of ["canary-store", "canary-app"]) {
+    await must(`docker cp /tmp/${name}.json mesh-control:/${name}.json`);
+    await mesh(`module add /${name}.json`);
+    await mesh(`assign ${MACHINE} ${name}`);
+  }
+  await mesh(`push ${MACHINE}`, 300_000);
+
+  const password = (await must(`cat /var/lib/canary-app/password`)).trim();
+  assert.ok(password.length >= 40, `the consumer's credential is ${password.length} characters`);
+
+  // Every hole filled, and filled with the right thing.
+  const env = await must(`cat /var/lib/canary-app/database.env`);
+  assert.match(env, /^PGPORT=5432$/m, `the port did not arrive as a port:\n${env}`);
+  assert.match(env, /^PGUSER=mesh_[a-z0-9_]+_canary_app$/m,
+    `the consumer was not told what name to present:\n${env}`);
+  assert.doesNotMatch(env, /\$\{/, `a placeholder reached the machine as a value:\n${env}`);
+  assert.ok(env.includes(`PGPASSWORD=${password}`),
+    `the file holds a different password from the credential file:\n${env}`);
+
+  // And the provider was told who asked, which is what makes the credential real.
+  const asked = await must(`cat /var/lib/canary-store/asked.json`);
+  assert.match(asked, /canary-app/, `the provider was not told who asked:\n${asked}`);
+  assert.match(asked, /"as": "mesh_[a-z0-9_]+_canary_app"/,
+    `the provider was not told what to call the login:\n${asked}`);
+});

test/integration/certificates.test.ts

@@ -0,0 +1,191 @@
+/**
+ * A public name, served with a certificate from an authority the mesh did not run.
+ *
+ * The mesh's own authority certifies `.internal` names and is proven elsewhere. This is the other
+ * half of the split: a name reachable from outside needs a certificate somebody else's browser
+ * already trusts, which means ordering one over ACME and answering a challenge **at the name being
+ * certified**.
+ *
+ * Against a real ACME server rather than a stub, for the reason the lab exists: what is under test
+ * is whether an order, a challenge and a handshake agree with each other, and a stub would be told
+ * to agree.
+ */
+
+import { test, after, before } from "node:test";
+import assert from "node:assert/strict";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { raise } from "../../src/lifecycle/raise.ts";
+import { destroy, exec } from "../../src/lifecycle/operate.ts";
+import { labIsUsable, destroyAll } from "./harness.ts";
+import { incus } from "../../src/incus/client.ts";
+import { machineName } from "../../src/lifecycle/names.ts";
+
+const capability = await labIsUsable();
+const proxy = process.env["MESH_LAB_ROUTE_PROXY"] ?? "";
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !proxy
+    ? "set MESH_LAB_ROUTE_PROXY to a built proxy (mesh-control: go build ./examples/route-proxy)"
+    : false;
+
+const SCENARIO = "a-public-name";
+const MACHINE = "anchor";
+const NAME = "photos.example";
+const ACME = "/var/lib/acme";
+
+let instanceId = "";
+
+function shellQuote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+async function on(command: string): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `${command} 2>&1; echo "__exit=$?"`,
+  ]);
+  const marker = stdout.lastIndexOf("__exit=");
+  return { out: stdout.slice(0, marker), ok: Number(stdout.slice(marker + 7).trim()) === 0 };
+}
+
+async function must(command: string): Promise<string> {
+  const { out, ok } = await on(command);
+  if (!ok) throw new Error(`${command}\n${out}`);
+  return out;
+}
+
+before(async () => {
+  if (skip) return;
+  const scenario = loadScenario(`scenarios/${SCENARIO}.yml`);
+  const instance = await raise(scenario, {});
+  instanceId = instance.instanceId;
+
+  const pebble = instance.images.find((r) => r.includes("pebble"));
+  assert.ok(pebble, `the scenario stocked no ACME server: ${instance.images.join(", ")}`);
+
+  await must(`mkdir -p ${ACME}/cache`);
+
+  // The authority's own API certificate is signed by a root nothing trusts yet. Taken out of the
+  // image rather than disabling verification, which is the same reason the proxy names a bundle:
+  // "skip" would still apply on the day this points at a public authority.
+  await must(`docker create --name pebble-certs ${pebble}`);
+  await must(`docker cp pebble-certs:/test/certs/pebble.minica.pem ${ACME}/authority-api.pem`);
+  await must(`docker rm pebble-certs`);
+
+  // **The challenge must arrive on port 80**, which is where a proxy serving a public name
+  // listens. The authority's own default is 5002 — convenient for its test suite and wrong here,
+  // because the thing being proven is that the real path works.
+  //
+  // **Its own configuration, with one field changed.** The first version of this wrote a config
+  // from scratch and silently dropped two fields the default carries; the order then came back
+  // valid with no certificate to fetch, and the failure looked like a client bug. Take what works
+  // and change the one thing that must differ.
+  await must(`docker create --name pebble-config ${pebble}`);
+  await must(`docker cp pebble-config:/test/config/pebble-config.json ${ACME}/pebble.json`);
+  await must(`docker rm pebble-config`);
+  await must(
+    `python3 -c "import json,sys;` +
+      `c=json.load(open('${ACME}/pebble.json'));` +
+      `c['pebble']['httpPort']=80;` +
+      `json.dump(c,open('${ACME}/pebble.json','w'),indent=2)"`,
+  );
+
+  // The name resolves to this machine, so the authority's challenge reaches the proxy rather than
+  // whatever else on the internet answers to it.
+  await must(`grep -q ${shellQuote(NAME)} /etc/hosts || echo "127.0.0.1 ${NAME}" >> /etc/hosts`);
+
+  await must(
+    `docker run -d --name acme --network host ` +
+      `-v ${ACME}/pebble.json:/test/config/pebble-config.json:ro ` +
+      `${pebble} -config /test/config/pebble-config.json -dnsserver 127.0.0.53:53`,
+  );
+
+  let up = false;
+  for (let i = 0; i < 60 && !up; i++) {
+    ({ ok: up } = await on(
+      `curl -sf --cacert ${ACME}/authority-api.pem https://127.0.0.1:14000/dir -o /dev/null`,
+    ));
+    if (!up) await new Promise((r) => setTimeout(r, 1000));
+  }
+  assert.ok(up, `the ACME server never answered:\n${(await on(`docker logs acme`)).out}`);
+
+  await incus([
+    "file", "push", proxy,
+    `${machineName(instanceId, MACHINE)}/usr/local/bin/mesh-route-proxy`,
+    "--mode", "0755",
+  ], 180_000);
+
+  // Something for the route to point at, so the proxy is serving a real name and not a hole.
+  await must(
+    `printf %s ${shellQuote(JSON.stringify({
+      given: [{ from: "photos", node: "", at: "", values: { name: NAME, port: 8080 } }],
+    }))} > ${ACME}/routes.json`,
+  );
+  await must(
+    `nohup sh -c 'while true; do printf "HTTP/1.1 200 OK\\r\\nContent-Length: 5\\r\\n\\r\\nhello" | nc -l -p 8080 -q 1; done' >/dev/null 2>&1 &`,
+  );
+}, { timeout: 1_200_000 });
+
+after(async () => {
+  if (instanceId) await destroy(instanceId);
+  await destroyAll(`${SCENARIO}-`);
+}, { timeout: 600_000 });
+
+test("a public name is served with a certificate the mesh did not issue", {
+  skip, timeout: 600_000,
+}, async () => {
+  await must(
+    `ROUTES=${ACME}/routes.json LISTEN=:80 TLS_LISTEN=:443 ` +
+      `ACME_CACHE=${ACME}/cache ` +
+      `ACME_DIRECTORY=https://127.0.0.1:14000/dir ` +
+      `ACME_CA_BUNDLE=${ACME}/authority-api.pem ` +
+      `nohup /usr/local/bin/mesh-route-proxy >${ACME}/proxy.log 2>&1 & sleep 3`,
+  );
+
+  // The authority's issuing root, so the handshake can be checked rather than merely completed.
+  await must(
+    `curl -sf --cacert ${ACME}/authority-api.pem https://127.0.0.1:15000/roots/0 > ${ACME}/issuer.pem`,
+  );
+
+  // The first request is what triggers the order: autocert obtains on demand for a name its
+  // policy allows. Retried because ordering, the challenge and issuance take a moment.
+  let served = { out: "", ok: false };
+  for (let i = 0; i < 40 && !served.ok; i++) {
+    served = await on(`curl -sf --cacert ${ACME}/issuer.pem https://${NAME}/ `);
+    if (!served.ok) await new Promise((r) => setTimeout(r, 2000));
+  }
+  if (!served.ok) {
+    // Both sides, gathered before asserting. The proxy's log says what it tried; the authority's
+    // says whether it ever heard from it — and "the client never spoke to it" and "it refused
+    // what the client said" are different faults with nothing in common.
+    const proxyLog = (await on(`cat ${ACME}/proxy.log`)).out;
+    const authority = (await on(`docker logs acme 2>&1 | tail -40`)).out;
+    const directory = (await on(
+      `curl -s --cacert ${ACME}/authority-api.pem https://127.0.0.1:14000/dir`)).out;
+    assert.fail(
+      `the name was never served over TLS: ${served.out}\n\n` +
+      `── the proxy tried:\n${proxyLog}\n` +
+      `── the authority heard:\n${authority}\n` +
+      `── the directory it was pointed at:\n${directory}\n`);
+  }
+  assert.match(served.out, /hello/);
+
+  // And it is the authority's certificate, not something self-signed that happens to work.
+  const issuer = await must(
+    `echo | openssl s_client -connect ${NAME}:443 -servername ${NAME} 2>/dev/null ` +
+      `| openssl x509 -noout -issuer -subject`,
+  );
+  assert.match(issuer, /Pebble/i, `the certificate was not issued by the ACME server:\n${issuer}`);
+  assert.match(issuer, new RegExp(NAME), `the certificate is not for the name asked for:\n${issuer}`);
+});
+
+test("no certificate is ordered for a name the mesh does not route", {
+  skip, timeout: 300_000,
+}, async () => {
+  // The policy that stops a quota being spent by a scan. Refused before any order is placed, so
+  // the authority never sees it.
+  const { out } = await on(
+    `echo | openssl s_client -connect 127.0.0.1:443 -servername nobody-asked-for-this.example 2>&1 | head -20`,
+  );
+  assert.doesNotMatch(out, /Pebble/i,
+    `a certificate was obtained for a name nothing routes here:\n${out}`);
+});

test/integration/events.test.ts

@@ -0,0 +1,221 @@
+/**
+ * An event a module emits reaches an audit trail, over the broker the mesh raised.
+ *
+ * Everything else here proves the broker carries *commands* — a declaration crosses it, a
+ * credential is delivered over it. This proves the other half of the bus (novox/hq ADR 0046): the
+ * events exchange, where a module emits and any number listen, and the audit logger consumes `#`
+ * and writes down what happened. It runs against the `mesh-broker` this scenario's own host raised
+ * from the substrate bundle — not a broker a test stood up — because "the mesh hosts the broker"
+ * (tier-1 substrate) is the thing being relied on.
+ *
+ * The wire shape it asserts is ADR 0047: metadata rides as headers so the body is only the
+ * payload, a consumer gets a durable per-consumer queue `<node>.<module>.events`, and a
+ * dead-letter exchange `mesh.events.dead` stands behind it. The queue and that exchange existing
+ * on the raised broker is the check that the runtime provisioned the contract, not just that a
+ * message happened to arrive.
+ *
+ * It needs the host binary and the substrate bundle, like the mesh walk, plus a runtime image:
+ *
+ *   MESH_LAB_HOST_BINARY=.../mesh-host
+ *   MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
+ *   MESH_LAB_RUNTIME=.../mesh-runtime-audit.tar   (docker save of the runtime+audit-logger image;
+ *                                                  built by scripts/build-runtime-image.sh)
+ *
+ * The runtime is run as a container against the broker here, rather than assigned through the
+ * control plane. Assigning it — so the mesh delivers its broker credential the way it does the
+ * builder's — is the next step; this proves the events path itself first.
+ */
+
+import { test, before, after } from "node:test";
+import assert from "node:assert/strict";
+import { existsSync, readFileSync } from "node:fs";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { raise } from "../../src/lifecycle/raise.ts";
+import { destroy, exec } from "../../src/lifecycle/operate.ts";
+import { hostBinaryPath, HOST_PATH } from "../../src/lifecycle/place.ts";
+import { labIsUsable, destroyAll } from "./harness.ts";
+import { incus } from "../../src/incus/client.ts";
+import { machineName } from "../../src/lifecycle/names.ts";
+
+const capability = await labIsUsable();
+const binary = hostBinaryPath();
+const bundle = process.env["MESH_LAB_BUNDLE"] ?? "";
+const runtime = process.env["MESH_LAB_RUNTIME"] ?? "";
+
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !binary || !existsSync(binary)
+    ? "MESH_LAB_HOST_BINARY is not set to a built mesh-host"
+    : !bundle || !existsSync(bundle)
+      ? "MESH_LAB_BUNDLE is not set to a substrate bundle (mesh-host examples/)"
+      : !runtime || !existsSync(runtime)
+        ? "MESH_LAB_RUNTIME is not set to a runtime image tar (scripts/build-runtime-image.sh)"
+        : false;
+
+const SCENARIO = "first-node";
+const MACHINE = "anchor";
+/** Where the audit-logger container writes its trail, on the machine — mounted from a host dir. */
+const TRAIL_DIR = "/var/lib/mesh-audit";
+const TRAIL = `${TRAIL_DIR}/audit.log`;
+/** The broker the substrate raised, reachable on the node's loopback (novox/hq ADR 0001). */
+const BROKER = "amqp://guest:guest@127.0.0.1:5672/";
+
+let instanceId = "";
+/** The tag `docker load` reported for the runtime image, so the container names what was loaded. */
+let runtimeImage = "";
+
+function quote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+/**
+ * A command on the machine, its exit read from a marker on its own line.
+ *
+ * `exec 2>&1` on its own first line and the marker on its own last line, so a command that carries
+ * a heredoc — the substrate bundle is written with one — terminates where it says it does rather
+ * than swallowing the marker (the fault mesh.test.ts documents).
+ */
+async function on(command: string, timeoutMs?: number): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `exec 2>&1\n${command}\necho "__exit=$?"`,
+  ], timeoutMs);
+  const marker = stdout.lastIndexOf("__exit=");
+  if (marker < 0) return { out: stdout, ok: false };
+  return { out: stdout.slice(0, marker), ok: stdout.slice(marker + 7).trim() === "0" };
+}
+
+async function must(command: string, timeoutMs?: number): Promise<string> {
+  const { out, ok } = await on(command, timeoutMs);
+  if (!ok) throw new Error(`${MACHINE}: ${command}\n${out}`);
+  return out;
+}
+
+/**
+ * The substrate bundle, its image references pointed at this scenario's own registry.
+ *
+ * A digest belongs to whatever registry serves it, so the committed bundle names a registry that
+ * is not this one; matching by repository and rewriting to the digest this registry assigned is
+ * what makes it applicable (the same rewrite mesh.test.ts does).
+ */
+function bundleFor(images: string[]): string {
+  let text = readFileSync(bundle, "utf8");
+  for (const pinned of images) {
+    const repository = pinned.slice(pinned.indexOf("/") + 1, pinned.indexOf("@"));
+    const escaped = repository.replaceAll("/", "\\/").replaceAll(".", "\\.");
+    text = text.replaceAll(
+      new RegExp(`[A-Za-z0-9_.:-]+\\/${escaped}@sha256:[0-9a-f]+`, "g"),
+      pinned,
+    );
+  }
+  return text;
+}
+
+/** Read the trail back as parsed JSON lines. */
+async function trail(): Promise<Record<string, unknown>[]> {
+  const raw = await must(`cat ${TRAIL} 2>/dev/null || true`);
+  return raw
+    .split("\n")
+    .map((l) => l.trim())
+    .filter(Boolean)
+    .map((l) => JSON.parse(l) as Record<string, unknown>);
+}
+
+before(async () => {
+  if (skip) return;
+
+  const raised = await raise(loadScenario(`scenarios/${SCENARIO}.yml`), {
+    onProgress: (m) => console.log(`raise: ${m}`),
+  });
+  instanceId = raised.instanceId;
+
+  // The node raises its substrate — store, broker and the rest — from the bundle, applied from a
+  // file because the digests are this registry's and are not known until it is up.
+  await must(`cat > /tmp/substrate.lock <<'MESHBUNDLE'\n${bundleFor(raised.images)}\nMESHBUNDLE`);
+  await must(`${HOST_PATH} apply /tmp/substrate.lock`, 600_000);
+
+  const running = await must(`docker ps --format '{{.Names}}'`);
+  assert.match(running, /mesh-broker/, `the substrate did not raise a broker:\n${running}`);
+
+  // Bring the runtime+audit-logger image onto the machine. Loaded, not pulled: the machine has no
+  // route out (novox/hq the lab is a closed address space), so the image arrives as a tar the way
+  // the builder binary does, and `docker load` names what it loaded.
+  await incus([
+    "file", "push", runtime, `${machineName(instanceId, MACHINE)}/tmp/runtime.tar`, "--mode", "0644",
+  ], 300_000);
+  const loaded = await must(`docker load < /tmp/runtime.tar`);
+  const named = loaded.match(/Loaded image:\s*(\S+)/)?.[1];
+  assert.ok(named, `docker load did not name the image:\n${loaded}`);
+  runtimeImage = named;
+
+  // Start the audit-logger: the runtime bound to the mesh's broker, consuming `#`, its trail on a
+  // mounted directory so the assertions read what it actually wrote.
+  await must(`mkdir -p ${TRAIL_DIR}`);
+  await must(
+    `docker run -d --name mesh-audit --network host ` +
+      `-e MESH_BROKER_URL=${quote(BROKER)} -e MESH_MODULE=audit-logger -e MESH_NODE=${MACHINE} ` +
+      `-e AUDIT_LOG=/trail/audit.log -v ${TRAIL_DIR}:/trail ` +
+      `${runtimeImage}`,
+  );
+  // Give the subscription a moment to bind before anything is emitted at it.
+  await new Promise((r) => setTimeout(r, 4000));
+}, { timeout: 1_800_000 });
+
+after(async () => {
+  if (instanceId) await destroy(instanceId);
+  await destroyAll(`${SCENARIO}-`);
+}, { timeout: 600_000 });
+
+test("an emitted event reaches the audit trail with its metadata in headers", {
+  skip, timeout: 300_000,
+}, async () => {
+  // Emitted as a different module (source=probe), so the trail's source is the emitter's, read
+  // back from the header — proof it did not come from the body.
+  await must(
+    `docker exec -e MESH_MODULE=probe -e MESH_NODE=${MACHINE} mesh-audit ` +
+      `node dist/main.js emit module.probe.site.created '{"domain":"my-app"}'`,
+  );
+  // A node-origin event too (novox/hq ADR 0047 reserves module.* / mesh.* / node.*), to show the
+  // audit sink takes all of them, not only a module's.
+  await must(
+    `docker exec -e MESH_MODULE=probe -e MESH_NODE=${MACHINE} mesh-audit ` +
+      `node dist/main.js emit node.${MACHINE}.tick '{"n":1}'`,
+  );
+
+  // Poll: the trail is written by a separate container reacting to the broker, so it is not there
+  // the instant emit returns.
+  let lines: Record<string, unknown>[] = [];
+  const until = Date.now() + 30_000;
+  while (Date.now() < until) {
+    lines = await trail();
+    if (lines.length >= 2) break;
+    await new Promise((r) => setTimeout(r, 2000));
+  }
+
+  const types = lines.map((l) => l.type);
+  assert.ok(
+    types.includes("module.probe.site.created") && types.includes(`node.${MACHINE}.tick`),
+    `both events did not reach the trail; it holds ${JSON.stringify(types)}\n` +
+      `${(await on(`docker logs mesh-audit 2>&1 | tail -20`)).out}`,
+  );
+
+  const site = lines.find((l) => l.type === "module.probe.site.created")!;
+  assert.equal(site.source, "probe", "x-source did not survive as the trail's source");
+  assert.equal(site.node, MACHINE, "x-node did not survive");
+  assert.ok(typeof site.id === "string" && site.id.length > 0, "no x-event-id was recorded");
+  assert.deepEqual(site.body, { domain: "my-app" }, "the body was not exactly the payload");
+});
+
+test("the runtime provisioned the ADR 0047 queue and dead-letter on the raised broker", {
+  skip, timeout: 120_000,
+}, async () => {
+  // Asked of the broker itself, so this is the shape that actually exists on the bus, not the
+  // shape the code intends. A durable per-consumer queue and a dead-letter home are what make the
+  // trail survive a restart and set a poison event aside — assert they are there, not assumed.
+  const queues = await must(`docker exec mesh-broker lavinmqctl list_queues name durable`);
+  assert.match(queues, new RegExp(`${MACHINE}\\.audit-logger\\.events`),
+    `the durable per-consumer queue is missing:\n${queues}`);
+
+  const exchanges = await must(`docker exec mesh-broker lavinmqctl list_exchanges name`);
+  assert.match(exchanges, /mesh\.events(\s|$)/m, `the events exchange is missing:\n${exchanges}`);
+  assert.match(exchanges, /mesh\.events\.dead/, `the dead-letter exchange is missing:\n${exchanges}`);
+});

test/integration/mesh-grant-end-to-end.test.ts

@@ -0,0 +1,261 @@
+/**
+ * The whole grant, mesh-driven end to end — novox/hq ADR 0053 with nothing hand-written.
+ *
+ * The earlier provider tests put the contributions file and the password on disk by hand, standing
+ * in for the control plane. This one does not: a provider (redis) and a consumer (a module that
+ * `requires` redis-cache) are both assigned, and the *mesh* mints the password, seals a copy to each
+ * end, writes redis its contributions file and the consumer its bound file, and the host unseals
+ * each side's secret. redis's provisioner — reading only what the mesh wrote — creates the ACL user
+ * under the login the mesh derived, with the password the mesh minted. The proof is the consumer's
+ * end: the credential the mesh delivered *it* authenticates against the login redis created for it.
+ * Mint on one side and create on the other agreeing, with no shared key and nothing placed by the
+ * test, is the entire provider/consumer contract working as one thing.
+ *
+ *   MESH_LAB_HOST_BINARY=.../mesh-host   MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
+ *   scripts/build-module-runtime.sh redis builds mesh-runtime-redis:development, which
+ *   scenarios/redis-node.yml stocks.
+ */
+
+import { test, before, after } from "node:test";
+import assert from "node:assert/strict";
+import { existsSync, readFileSync } from "node:fs";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { raise } from "../../src/lifecycle/raise.ts";
+import { destroy, exec } from "../../src/lifecycle/operate.ts";
+import { hostBinaryPath, HOST_PATH } from "../../src/lifecycle/place.ts";
+import { labIsUsable, destroyAll } from "./harness.ts";
+
+const capability = await labIsUsable();
+const binary = hostBinaryPath();
+const bundle = process.env["MESH_LAB_BUNDLE"] ?? "";
+
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !binary || !existsSync(binary)
+    ? "MESH_LAB_HOST_BINARY is not set to a built mesh-host"
+    : !bundle || !existsSync(bundle)
+      ? "MESH_LAB_BUNDLE is not set to a substrate bundle (mesh-host examples/)"
+      : false;
+
+const SCENARIO = "redis-node";
+const MACHINE = "anchor";
+
+let instanceId = "";
+let stocked: string[] = [];
+
+function quote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+async function on(command: string, timeoutMs?: number): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `exec 2>&1\n${command}\necho "__exit=$?"`,
+  ], timeoutMs);
+  const marker = stdout.lastIndexOf("__exit=");
+  if (marker < 0) return { out: stdout, ok: false };
+  return { out: stdout.slice(0, marker), ok: stdout.slice(marker + 7).trim() === "0" };
+}
+
+async function must(command: string, timeoutMs?: number): Promise<string> {
+  const { out, ok } = await on(command, timeoutMs);
+  if (!ok) throw new Error(`${MACHINE}: ${command}\n${out}`);
+  return out;
+}
+
+async function mesh(command: string, timeoutMs?: number): Promise<string> {
+  return must(`docker exec mesh-control /mesh-control ${command}`, timeoutMs);
+}
+
+function pinned(repository: string): string {
+  const found = stocked.find((r) => r.slice(r.indexOf("/") + 1, r.indexOf("@")) === repository);
+  assert.ok(found, `the scenario stocks no ${repository}; it serves ${stocked.join(", ")}`);
+  return found;
+}
+
+function bundleFor(images: string[]): string {
+  let text = readFileSync(bundle, "utf8");
+  for (const ref of images) {
+    const repository = ref.slice(ref.indexOf("/") + 1, ref.indexOf("@"));
+    const escaped = repository.replaceAll("/", "\\/").replaceAll(".", "\\.");
+    text = text.replaceAll(new RegExp(`[A-Za-z0-9_.:-]+\\/${escaped}@sha256:[0-9a-f]+`, "g"), ref);
+  }
+  return text;
+}
+
+function tokenFrom(said: string): string {
+  const found = said.split("\n").map((l) => l.trim()).find((l) => l.length > 100 && !l.includes(" "));
+  assert.ok(found, `no token in:\n${said}`);
+  return found;
+}
+
+async function settled(withinMs = 480_000): Promise<void> {
+  const until = Date.now() + withinMs;
+  let last = "";
+  while (Date.now() < until) {
+    const asked = await on(`docker exec mesh-control /mesh-control status --json`);
+    if (asked.ok) {
+      try {
+        const state = JSON.parse(asked.out) as {
+          wrong: { node: string; outcome: string }[];
+          waiting: { node: string }[];
+          reported: { node: string; outcome: string; current: boolean }[];
+        };
+        const bad = state.wrong.find((w) => w.node === MACHINE);
+        if (bad) throw new Error(`${MACHINE} did not apply what it was sent: ${bad.outcome}\n${asked.out}`);
+        const word = state.reported.find((r) => r.node === MACHINE);
+        if (!state.waiting.some((w) => w.node === MACHINE) && word?.outcome === "applied" && word.current) return;
+        last = asked.out;
+      } catch (err) {
+        if (err instanceof Error && err.message.includes("did not apply")) throw err;
+        last = asked.out;
+      }
+    }
+    await new Promise((r) => setTimeout(r, 5000));
+  }
+  throw new Error(`${MACHINE} never caught up within ${Math.round(withinMs / 1000)}s. Last:\n${last}`);
+}
+
+before(async () => {
+  if (skip) return;
+
+  const raised = await raise(loadScenario(`scenarios/${SCENARIO}.yml`), {
+    onProgress: (m) => console.log(`raise: ${m}`),
+  });
+  instanceId = raised.instanceId;
+  stocked = raised.images;
+
+  await must(`cat > /tmp/substrate.lock <<'MESHBUNDLE'\n${bundleFor(raised.images)}\nMESHBUNDLE`);
+  await must(`${HOST_PATH} apply /tmp/substrate.lock`, 600_000);
+  const up = await must(`docker ps --format '{{.Names}}'`);
+  for (const c of ["mesh-store", "mesh-broker", "mesh-control"]) {
+    assert.match(up, new RegExp(c), `the substrate did not raise ${c}:\n${up}`);
+  }
+
+  await mesh(`node add ${MACHINE}`);
+  const token = tokenFrom(await mesh(`token issue --node ${MACHINE}`));
+  await must(`${HOST_PATH} enrol --token ${quote(token)}`);
+  await must(`nohup ${HOST_PATH} run > /var/log/mesh-host.log 2>&1 & sleep 3`);
+}, { timeout: 1_800_000 });
+
+after(async () => {
+  if (instanceId) await destroy(instanceId);
+  await destroyAll(`${SCENARIO}-`);
+}, { timeout: 600_000 });
+
+test("the mesh grants a consumer redis's cache, and the credential it delivers authenticates", {
+  skip, timeout: 900_000,
+}, async () => {
+  // The PROVIDER: redis in its committed shape — server and a broker-bound runtime on the private
+  // redis network, the runtime running the provisioner.
+  const redisManifest = JSON.stringify({
+    module: "redis",
+    version: "1",
+    provides: [{ name: "redis-cache", scope: "mesh" }],
+    serves: { "redis-cache": {} },
+    emits: ["module.redis.cache.provisioned", "module.redis.cache.deprovisioned"],
+    consumes: ["module.redis.cache.provisioned", "module.redis.cache.deprovisioned"],
+    receives: { "redis-cache": "/var/lib/redis-module/grants/mesh.json" },
+    grants: { "redis-cache": "/var/lib/redis-module/grants" },
+    "own-secrets": { default: "/var/lib/redis-module/default.secret", broker: "/var/lib/mesh/redis/broker" },
+    resources: [
+      { id: "mesh-state", type: "directory", path: "/var/lib/mesh/redis", mode: "0700" },
+      { id: "state", type: "directory", path: "/var/lib/redis-module", mode: "0700" },
+      { id: "grants-dir", type: "directory", path: "/var/lib/redis-module/grants", mode: "0700" },
+      { id: "data", type: "directory", path: "/services/redis/data", mode: "0700", owner: "999:999" },
+      {
+        id: "server-conf", type: "file", path: "/var/lib/redis-module/redis.conf", mode: "0644",
+        content: "requirepass ${secret:default}\nappendonly no\ndir /data\n",
+      },
+      { id: "net", type: "network", name: "redis" },
+      {
+        id: "server", type: "container", name: "redis", image: pinned("redis"), network: "redis",
+        ports: ["6379"],
+        volumes: ["/services/redis/data:/data", "/var/lib/redis-module/redis.conf:/etc/redis/redis.conf:ro"],
+        args: ["/etc/redis/redis.conf"],
+      },
+      {
+        id: "runtime", type: "container", name: "mesh-redis", image: pinned("mesh-runtime-redis"),
+        network: "redis",
+        volumes: [
+          "/var/lib/mesh/redis/broker:/run/secrets/broker:ro",
+          "/var/lib/redis-module/grants:/var/lib/redis-module/grants:ro",
+          "/var/lib/redis-module/default.secret:/run/secrets/default:ro",
+        ],
+        env: {
+          MESH_BROKER_FILE: "/run/secrets/broker",
+          MESH_RECEIVES: "/var/lib/redis-module/grants/mesh.json",
+          MESH_PROVISION_REDIS: "redis:6379",
+          MESH_PROVISION_PASSWORD_FILE: "/run/secrets/default",
+        },
+      },
+    ],
+  });
+
+  // The CONSUMER: a module that requires redis-cache and no more. It runs no code here — the mesh
+  // delivers it a bound file (where redis is, and the login to present) and its sealed password,
+  // which the host unseals onto the machine. That delivery is exactly what a real consumer reads.
+  const consumerManifest = JSON.stringify({
+    module: "cacheuser",
+    version: "1",
+    requires: ["redis-cache"],
+    // `contributes` (not just `requires`) is what makes a consumer *ask* — the grant forms from a
+    // contribution. It must be non-empty; redis's provisioner ignores the value (it uses the login
+    // the mesh derives), so the name is only what marks this module as wanting a cache.
+    contributes: { "redis-cache": { name: "cacheuser" } },
+    binds: { "redis-cache": "/var/lib/cacheuser/redis.json" },
+    secrets: { "redis-cache": "/var/lib/cacheuser/redis.secret" },
+    resources: [{ id: "state", type: "directory", path: "/var/lib/cacheuser", mode: "0700" }],
+  });
+
+  await must(`printf %s ${quote(redisManifest)} > /tmp/redis.json && docker cp /tmp/redis.json mesh-control:/redis.json`);
+  await mesh("module add /redis.json");
+  await mesh(`module issue redis --node ${MACHINE}`);
+  await mesh(`assign ${MACHINE} redis`);
+
+  await must(`printf %s ${quote(consumerManifest)} > /tmp/cacheuser.json && docker cp /tmp/cacheuser.json mesh-control:/cacheuser.json`);
+  await mesh("module add /cacheuser.json");
+  await mesh(`assign ${MACHINE} cacheuser`);
+
+  await mesh(`push ${MACHINE}`);
+  await settled();
+
+  // The mesh matched the two and wrote redis its contributions file — the test wrote nothing here.
+  const contributions = await must(`cat /var/lib/redis-module/grants/mesh.json`);
+  assert.match(contributions, /"as"/, `the mesh did not write redis a contributions file:\n${contributions}`);
+
+  // The mesh delivered the consumer its bound file and its unsealed password.
+  let boundRaw = "";
+  const untilBound = Date.now() + 60_000;
+  while (Date.now() < untilBound) {
+    const got = await on(`cat /var/lib/cacheuser/redis.json 2>/dev/null`);
+    if (got.ok && /"as"/.test(got.out)) { boundRaw = got.out; break; }
+    await new Promise((r) => setTimeout(r, 3000));
+  }
+  assert.match(boundRaw, /"as"/, `the consumer was never told about its cache:\n${boundRaw}`);
+  const bound = JSON.parse(boundRaw) as { as: string; from: string; provision: string };
+  assert.equal(bound.provision, "redis-cache");
+  assert.ok(bound.from, `the consumer was not told which node serves its cache:\n${boundRaw}`);
+  const as = bound.as;
+  const password = (await must(`cat /var/lib/cacheuser/redis.secret`)).trim();
+  assert.ok(as && password, `the consumer's login or password was empty (as=${as})`);
+
+  // redis's provisioner, reading only the mesh's contributions, created the ACL user. Wait for it.
+  const adminPw = await must(`cat /var/lib/redis-module/default.secret`);
+  let acl = "";
+  const untilAcl = Date.now() + 60_000;
+  while (Date.now() < untilAcl) {
+    acl = (await on(`docker exec redis redis-cli -a ${quote(adminPw)} --no-auth-warning ACL LIST 2>/dev/null`)).out;
+    if (acl.includes(as)) break;
+    await new Promise((r) => setTimeout(r, 3000));
+  }
+  assert.match(acl, new RegExp(as.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")),
+    `redis never created the login the mesh granted (${as}):\n${(await on(`docker logs mesh-redis 2>&1 | tail -30`)).out}\n---\n${acl}`);
+
+  // THE PROOF, from the consumer's side: the credential the mesh delivered *it* — the login from its
+  // bound file, the password from its secret — authenticates against the login redis created. Mint
+  // and create agreeing across the two ends, with no shared key and nothing the test placed.
+  const authed = await on(`docker exec redis redis-cli --user ${quote(as)} --pass ${quote(password)} --no-auth-warning PING 2>&1`);
+  assert.doesNotMatch(authed.out, /WRONGPASS|NOPERM|no password/i,
+    `the consumer's mesh-delivered credential did not authenticate — the two ends do not agree:\n${authed.out}`);
+  assert.match(authed.out, /PONG/, `expected PONG authenticating as the granted consumer:\n${authed.out}`);
+});

test/integration/minio-grant-end-to-end.test.ts

@@ -0,0 +1,258 @@
+/**
+ * The whole grant for an S3 bucket, mesh-driven — novox/hq ADR 0052/0053, the minio case.
+ *
+ * The postgres bed proves the provider/consumer contract for a database. This proves it for object
+ * storage, on a provider whose code drives the `mc` CLI (so the runtime image carries it): minio is
+ * assigned, a consumer that requires s3-bucket is assigned, and the mesh mints one secret key, sealing
+ * a copy to each end. minio's provisioner — reading only the mesh's contributions — creates a bucket
+ * and a service account under the access key the mesh derived, with the secret it minted. The proof is
+ * the consumer reaching its bucket with the access key and secret the mesh delivered it. Nothing is
+ * placed by the test.
+ *
+ *   MESH_LAB_HOST_BINARY=.../mesh-host   MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
+ *   scripts/build-module-runtime.sh minio builds mesh-runtime-minio:development (with mc), which
+ *   scenarios/minio-node.yml stocks. minio/minio:latest must be in the local daemon.
+ */
+
+import { test, before, after } from "node:test";
+import assert from "node:assert/strict";
+import { existsSync, readFileSync } from "node:fs";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { raise } from "../../src/lifecycle/raise.ts";
+import { destroy, exec } from "../../src/lifecycle/operate.ts";
+import { hostBinaryPath, HOST_PATH } from "../../src/lifecycle/place.ts";
+import { labIsUsable, destroyAll } from "./harness.ts";
+
+const capability = await labIsUsable();
+const binary = hostBinaryPath();
+const bundle = process.env["MESH_LAB_BUNDLE"] ?? "";
+
+// 04-ISSUES/010 is fixed by ADR 0054: an S3 access key is capped at 20, and the mesh derives
+// `mesh_<node>_<module>`, so `bucketuser` on `anchor` (22) would overflow — but a consumer declares a
+// short `slug` and its identity fits. This bed's consumer does exactly that.
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !binary || !existsSync(binary)
+    ? "MESH_LAB_HOST_BINARY is not set to a built mesh-host"
+    : !bundle || !existsSync(bundle)
+      ? "MESH_LAB_BUNDLE is not set to a substrate bundle (mesh-host examples/)"
+      : false;
+
+const SCENARIO = "minio-node";
+const MACHINE = "anchor";
+
+let instanceId = "";
+let stocked: string[] = [];
+
+function quote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+async function on(command: string, timeoutMs?: number): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `exec 2>&1\n${command}\necho "__exit=$?"`,
+  ], timeoutMs);
+  const marker = stdout.lastIndexOf("__exit=");
+  if (marker < 0) return { out: stdout, ok: false };
+  return { out: stdout.slice(0, marker), ok: stdout.slice(marker + 7).trim() === "0" };
+}
+
+async function must(command: string, timeoutMs?: number): Promise<string> {
+  const { out, ok } = await on(command, timeoutMs);
+  if (!ok) throw new Error(`${MACHINE}: ${command}\n${out}`);
+  return out;
+}
+
+async function mesh(command: string, timeoutMs?: number): Promise<string> {
+  return must(`docker exec mesh-control /mesh-control ${command}`, timeoutMs);
+}
+
+function pinned(repository: string): string {
+  const found = stocked.find((r) => r.slice(r.indexOf("/") + 1, r.indexOf("@")) === repository);
+  assert.ok(found, `the scenario stocks no ${repository}; it serves ${stocked.join(", ")}`);
+  return found;
+}
+
+function bundleFor(images: string[]): string {
+  let text = readFileSync(bundle, "utf8");
+  for (const ref of images) {
+    const repository = ref.slice(ref.indexOf("/") + 1, ref.indexOf("@"));
+    const escaped = repository.replaceAll("/", "\\/").replaceAll(".", "\\.");
+    text = text.replaceAll(new RegExp(`[A-Za-z0-9_.:-]+\\/${escaped}@sha256:[0-9a-f]+`, "g"), ref);
+  }
+  return text;
+}
+
+function tokenFrom(said: string): string {
+  const found = said.split("\n").map((l) => l.trim()).find((l) => l.length > 100 && !l.includes(" "));
+  assert.ok(found, `no token in:\n${said}`);
+  return found;
+}
+
+/** Same rule minio's client uses to name a bucket for a consumer — recomputed so the test knows it. */
+function bucketFor(as: string): string {
+  const name = as.toLowerCase().replace(/[^a-z0-9-]+/g, "-").replace(/^-+|-+$/g, "").slice(0, 63);
+  return name.length >= 3 ? name : `mesh-${name}`;
+}
+
+async function settled(withinMs = 480_000): Promise<void> {
+  const until = Date.now() + withinMs;
+  let last = "";
+  while (Date.now() < until) {
+    const asked = await on(`docker exec mesh-control /mesh-control status --json`);
+    if (asked.ok) {
+      try {
+        const state = JSON.parse(asked.out) as {
+          wrong: { node: string; outcome: string }[];
+          waiting: { node: string }[];
+          reported: { node: string; outcome: string; current: boolean }[];
+        };
+        const bad = state.wrong.find((w) => w.node === MACHINE);
+        if (bad) throw new Error(`${MACHINE} did not apply what it was sent: ${bad.outcome}\n${asked.out}`);
+        const word = state.reported.find((r) => r.node === MACHINE);
+        if (!state.waiting.some((w) => w.node === MACHINE) && word?.outcome === "applied" && word.current) return;
+        last = asked.out;
+      } catch (err) {
+        if (err instanceof Error && err.message.includes("did not apply")) throw err;
+        last = asked.out;
+      }
+    }
+    await new Promise((r) => setTimeout(r, 5000));
+  }
+  throw new Error(`${MACHINE} never caught up within ${Math.round(withinMs / 1000)}s. Last:\n${last}`);
+}
+
+before(async () => {
+  if (skip) return;
+
+  const raised = await raise(loadScenario(`scenarios/${SCENARIO}.yml`), {
+    onProgress: (m) => console.log(`raise: ${m}`),
+  });
+  instanceId = raised.instanceId;
+  stocked = raised.images;
+
+  await must(`cat > /tmp/substrate.lock <<'MESHBUNDLE'\n${bundleFor(raised.images)}\nMESHBUNDLE`);
+  await must(`${HOST_PATH} apply /tmp/substrate.lock`, 600_000);
+  const up = await must(`docker ps --format '{{.Names}}'`);
+  for (const c of ["mesh-store", "mesh-broker", "mesh-control"]) {
+    assert.match(up, new RegExp(c), `the substrate did not raise ${c}:\n${up}`);
+  }
+
+  await mesh(`node add ${MACHINE}`);
+  const token = tokenFrom(await mesh(`token issue --node ${MACHINE}`));
+  await must(`${HOST_PATH} enrol --token ${quote(token)}`);
+  await must(`nohup ${HOST_PATH} run > /var/log/mesh-host.log 2>&1 & sleep 3`);
+}, { timeout: 1_800_000 });
+
+after(async () => {
+  if (instanceId) await destroy(instanceId);
+  await destroyAll(`${SCENARIO}-`);
+}, { timeout: 600_000 });
+
+test("the mesh grants a consumer an S3 bucket, and the credential it delivers reaches it", {
+  skip, timeout: 900_000,
+}, async () => {
+  // The PROVIDER: minio in its committed shape — server and a broker-bound runtime (carrying mc) on
+  // the private minio network, the runtime running the provisioner. No published port on this
+  // single-node bed; the consumer reaches minio over the private network by name.
+  const minioManifest = JSON.stringify({
+    module: "minio",
+    version: "1",
+    provides: [{ name: "s3-bucket", scope: "mesh" }],
+    serves: { "s3-bucket": { scheme: "http", region: "us-east-1" } },
+    emits: ["module.minio.bucket.created", "module.minio.bucket.removed"],
+    receives: { "s3-bucket": "/var/lib/minio/grants/mesh.json" },
+    grants: { "s3-bucket": "/var/lib/minio/grants" },
+    "own-secrets": { root: "/var/lib/minio/root.secret", broker: "/var/lib/mesh/minio/broker" },
+    resources: [
+      { id: "mesh-state", type: "directory", path: "/var/lib/mesh/minio", mode: "0700" },
+      { id: "state", type: "directory", path: "/var/lib/minio", mode: "0700" },
+      { id: "grants", type: "directory", path: "/var/lib/minio/grants", mode: "0700" },
+      { id: "root-env", type: "file", path: "/var/lib/minio/root.env", mode: "0600", content: "MINIO_ROOT_USER=meshroot\nMINIO_ROOT_PASSWORD=${secret:root}\n" },
+      { id: "data", type: "directory", path: "/services/minio/data/data1-1", mode: "0700" },
+      { id: "net", type: "network", name: "minio" },
+      {
+        id: "server", type: "container", name: "minio", image: pinned("minio/minio"), network: "minio",
+        args: ["server", "/data", "--console-address", ":9001"],
+        "env-file": ["/var/lib/minio/root.env"],
+        volumes: ["/services/minio/data/data1-1:/data"],
+      },
+      {
+        id: "runtime", type: "container", name: "mesh-minio", image: pinned("mesh-runtime-minio"),
+        network: "minio",
+        volumes: [
+          "/var/lib/mesh/minio/broker:/run/secrets/broker:ro",
+          "/var/lib/minio/grants:/var/lib/minio/grants:ro",
+          "/var/lib/minio/root.secret:/run/secrets/root:ro",
+        ],
+        env: {
+          MESH_MINIO_ENDPOINT: "http://minio:9000",
+          MESH_MINIO_ROOT_USER: "meshroot",
+          MESH_MINIO_ROOT_PASSWORD_FILE: "/run/secrets/root",
+          MESH_BROKER_FILE: "/run/secrets/broker",
+          MESH_RECEIVES: "/var/lib/minio/grants/mesh.json",
+        },
+      },
+    ],
+  });
+
+  const consumerManifest = JSON.stringify({
+    module: "bucketuser",
+    version: "1",
+    // A short slug, so the derived identity `mesh_anchor_bkt` fits an S3 access key's 20 chars where
+    // `mesh_anchor_bucketuser` (22) would not (novox/hq ADR 0054, 04-ISSUES/010).
+    slug: "bkt",
+    requires: ["s3-bucket"],
+    contributes: { "s3-bucket": { name: "bucketuser" } },
+    binds: { "s3-bucket": "/var/lib/bucketuser/s3.json" },
+    secrets: { "s3-bucket": "/var/lib/bucketuser/s3.secret" },
+    resources: [{ id: "state", type: "directory", path: "/var/lib/bucketuser", mode: "0700" }],
+  });
+
+  await must(`printf %s ${quote(minioManifest)} > /tmp/minio.json && docker cp /tmp/minio.json mesh-control:/minio.json`);
+  await mesh("module add /minio.json");
+  await mesh(`module issue minio --node ${MACHINE}`);
+  await mesh(`assign ${MACHINE} minio`);
+
+  await must(`printf %s ${quote(consumerManifest)} > /tmp/bucketuser.json && docker cp /tmp/bucketuser.json mesh-control:/bucketuser.json`);
+  await mesh("module add /bucketuser.json");
+  await mesh(`assign ${MACHINE} bucketuser`);
+
+  await mesh(`push ${MACHINE}`);
+  await settled();
+
+  // The mesh delivered the consumer its bound file and its unsealed secret.
+  let boundRaw = "";
+  const untilBound = Date.now() + 60_000;
+  while (Date.now() < untilBound) {
+    const got = await on(`cat /var/lib/bucketuser/s3.json 2>/dev/null`);
+    if (got.ok && /"as"/.test(got.out)) { boundRaw = got.out; break; }
+    await new Promise((r) => setTimeout(r, 3000));
+  }
+  assert.match(boundRaw, /"as"/, `the consumer was never told about its bucket:\n${boundRaw}`);
+  const bound = JSON.parse(boundRaw) as { as: string; provision: string };
+  assert.equal(bound.provision, "s3-bucket");
+  const accessKey = bound.as;
+  const secretKey = (await must(`cat /var/lib/bucketuser/s3.secret`)).trim();
+  assert.ok(accessKey && secretKey, `the consumer's access key or secret was empty (as=${accessKey})`);
+  const bucket = bucketFor(accessKey);
+
+  // THE PROOF: reach the bucket as the consumer, with the access key and secret the mesh delivered
+  // it. mc listing the consumer's own bucket means the service account, the bucket, and the secret all
+  // line up across the two ends. A provisioner that set a different secret answers "Access Denied".
+  const probe =
+    `mc alias set probe http://minio:9000 ${quote(accessKey)} ${quote(secretKey)} >/dev/null 2>&1 && ` +
+    `mc ls probe/${quote(bucket)}/`;
+  let out = { out: "", ok: false };
+  const untilReach = Date.now() + 90_000;
+  while (Date.now() < untilReach) {
+    out = await on(`docker exec mesh-minio sh -c ${quote(probe)} 2>&1`);
+    if (out.ok) break;
+    if (/denied/i.test(out.out)) break; // fast-fail: the credential is wrong
+    await new Promise((r) => setTimeout(r, 3000));
+  }
+  assert.doesNotMatch(out.out, /denied/i,
+    `the consumer could not reach its bucket with the mesh's secret — the two ends do not agree:\n${out.out}`);
+  assert.ok(out.ok,
+    `the consumer could not list its granted bucket ${bucket} as ${accessKey}:\n${out.out}\n---\n${(await on(`docker logs mesh-minio 2>&1 | tail -30`)).out}`);
+});

test/integration/objectstore.test.ts

@@ -0,0 +1,305 @@
+/**
+ * The last step of a credential, against a real object store.
+ *
+ * The mesh generates a secret, seals it to the machine that must accept it, and discards the
+ * plaintext — so it cannot tell the store to start accepting it. Something on that machine reads
+ * what the host wrote and makes it true. This is the step where a secret either becomes a working
+ * key or does not.
+ *
+ * **Phase 1.1 of the work breakdown**, and the finding that shaped it: the control plane
+ * special-cases nothing. `provides`, `requires`, `contributes` and `grants` are name-agnostic, so
+ * asking for a bucket needed no change to the mesh at all — only a provider that answers. What is
+ * proven here is that half.
+ *
+ * **And the half a database does not have.** One PostgreSQL server holds separate databases, and
+ * a role that cannot reach another's is a boundary the product enforces. One object store holds
+ * everybody's buckets behind one endpoint, so a consumer being unable to reach another's is a
+ * policy somebody wrote — which means it is a thing that can be written wrongly, and therefore a
+ * thing to assert rather than assume.
+ */
+
+import { test, after, before } from "node:test";
+import assert from "node:assert/strict";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { raise } from "../../src/lifecycle/raise.ts";
+import { destroy, exec } from "../../src/lifecycle/operate.ts";
+import { labIsUsable, destroyAll } from "./harness.ts";
+import { incus } from "../../src/incus/client.ts";
+import { machineName } from "../../src/lifecycle/names.ts";
+
+const capability = await labIsUsable();
+const provisioner = process.env["MESH_LAB_OBJECTSTORE_PROVISIONER"] ?? "";
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !provisioner
+    ? "set MESH_LAB_OBJECTSTORE_PROVISIONER to a built provisioner " +
+      "(mesh-control: go build ./examples/objectstore-provisioner)"
+    : false;
+
+const SCENARIO = "an-object-store";
+const MACHINE = "anchor";
+const GRANTS = "/var/lib/objectstore/grants";
+const ROOT_USER = "meshroot";
+const ROOT_PASSWORD = "meshroot-super-secret";
+const ROOT_PASSWORD_FILE = "/var/lib/objectstore/root.secret";
+const ENDPOINT = "http://127.0.0.1:9000";
+
+let instanceId = "";
+/** The store's image, by digest, from the registry the scenario raised. */
+let storeImage = "";
+
+function shellQuote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+async function on(command: string): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `${command} 2>&1; echo "__exit=$?"`,
+  ]);
+  const marker = stdout.lastIndexOf("__exit=");
+  const status = Number(stdout.slice(marker + 7).trim());
+  return { out: stdout.slice(0, marker), ok: status === 0 };
+}
+
+/** The same, refusing to continue past a failure nobody would otherwise see. */
+async function must(command: string): Promise<string> {
+  const { out, ok } = await on(command);
+  if (!ok) throw new Error(`${command}\n${out}`);
+  return out;
+}
+
+/** `mc` on the machine, against the store as root. */
+async function admin(args: string): Promise<{ out: string; ok: boolean }> {
+  return on(`mc --config-dir /tmp/root-mc ${args}`);
+}
+
+/**
+ * Write what the host would have written from a declaration: the manifest of who asked, and one
+ * file per consumer holding its secret alone.
+ *
+ * Written here rather than by running the host, because what is under test is the step *after*
+ * the host — and that the host writes these exact shapes is asserted in its own suite.
+ */
+async function meshWrote(
+  consumers: { node: string; module: string; bucket: string; secret: string }[],
+): Promise<void> {
+  const manifest = {
+    contributions: consumers.length,
+    requirement: "s3-bucket",
+    generated: "by the mesh",
+    given: consumers.map((c) => ({
+      from: c.module,
+      node: c.node,
+      secret: `${GRANTS}/${c.node}.${c.module}.secret`,
+      values: { bucket: c.bucket },
+    })),
+  };
+  await must(`mkdir -p ${GRANTS}`);
+  await must(`printf %s ${shellQuote(JSON.stringify(manifest))} > ${GRANTS}/mesh.json`);
+  // Every credential file rewritten from nothing, so a removed consumer's does not linger and
+  // make the revocation test pass for a reason that is not the one being tested.
+  await must(`find ${GRANTS} -name '*.secret' -delete`);
+  for (const c of consumers) {
+    await must(`printf %s ${shellQuote(c.secret)} > ${GRANTS}/${c.node}.${c.module}.secret`);
+    await must(`chmod 600 ${GRANTS}/${c.node}.${c.module}.secret`);
+  }
+}
+
+/** The provisioner, as the module shipping the store would run it. */
+async function provision(): Promise<{ out: string; ok: boolean }> {
+  return on(
+    `GRANTS=${GRANTS} ` +
+      `MESH_OBJECTSTORE_URL=${ENDPOINT} ` +
+      `MESH_OBJECTSTORE_ROOT_USER=${ROOT_USER} ` +
+      `MESH_OBJECTSTORE_ROOT_PASSWORD_FILE=${ROOT_PASSWORD_FILE} ` +
+      `/usr/local/bin/mesh-provision-objectstore`,
+  );
+}
+
+/**
+ * Can this key write to and read from this bucket?
+ *
+ * As the consumer, with its own `mc` configuration directory — never the root one. A check made
+ * with the root alias still in scope would pass for any key at all, which is the object-store
+ * shape of the mistake the database suite records: two of its tests once passed without verifying
+ * a password, because they ran where PostgreSQL trusts the caller.
+ */
+async function canUse(key: string, secret: string, bucket: string): Promise<{ ok: boolean; out: string }> {
+  const dir = `/tmp/as-${key}`;
+  const { out, ok } = await on(
+    `rm -rf ${dir} && mc --config-dir ${dir} alias set probe ${ENDPOINT} ${shellQuote(key)} ${shellQuote(secret)} && ` +
+      `echo hello > /tmp/probe.txt && ` +
+      `mc --config-dir ${dir} cp /tmp/probe.txt probe/${bucket}/probe.txt && ` +
+      `mc --config-dir ${dir} cat probe/${bucket}/probe.txt`,
+  );
+  return { ok: ok && out.includes("hello"), out };
+}
+
+before(async () => {
+  if (skip) return;
+  const scenario = loadScenario(`scenarios/${SCENARIO}.yml`);
+  const instance = await raise(scenario, {});
+  instanceId = instance.instanceId;
+
+  // From the registry the scenario raised, by digest. There is no route to a public registry from
+  // a documentation range, which is the point of the lab having its own.
+  const store = instance.images.find((r) => r.includes("minio/minio"));
+  const client = instance.images.find((r) => r.includes("minio/mc"));
+  assert.ok(store, `the scenario stocked no store image: ${instance.images.join(", ")}`);
+  assert.ok(client, `the scenario stocked no client image: ${instance.images.join(", ")}`);
+  storeImage = store;
+
+  // The client, taken out of the vendor's own image onto the machine. The provisioner drives it,
+  // so it has to be here — and taking it from the stocked image is what keeps this test off any
+  // public network.
+  await must(`docker create --name mc-source ${client}`);
+  await must(`docker cp mc-source:/usr/bin/mc /usr/local/bin/mc && chmod 755 /usr/local/bin/mc`);
+  await must(`docker rm mc-source`);
+
+  await must(`mkdir -p ${GRANTS}`);
+  await must(`printf %s ${shellQuote(ROOT_PASSWORD)} > ${ROOT_PASSWORD_FILE} && chmod 600 ${ROOT_PASSWORD_FILE}`);
+
+  await must(
+    `docker run -d --name mesh-store ` +
+      `-e MINIO_ROOT_USER=${ROOT_USER} -e MINIO_ROOT_PASSWORD=${shellQuote(ROOT_PASSWORD)} ` +
+      `-p 127.0.0.1:9000:9000 ${storeImage} server /data`,
+  );
+
+  // Ready over the endpoint the provisioner will use, not by the container being up. A store that
+  // is starting answers the port and refuses every operation, which is indistinguishable from a
+  // wrong credential if it is not waited for.
+  let ready = false;
+  for (let i = 0; i < 90 && !ready; i++) {
+    ({ ok: ready } = await on(
+      `mc --config-dir /tmp/root-mc alias set root ${ENDPOINT} ${ROOT_USER} ${shellQuote(ROOT_PASSWORD)}`,
+    ));
+    if (!ready) await new Promise((r) => setTimeout(r, 1000));
+  }
+  assert.ok(ready, "the store never became ready");
+
+  await incus([
+    "file", "push", provisioner,
+    `${machineName(instanceId, MACHINE)}/usr/local/bin/mesh-provision-objectstore`,
+    "--mode", "0755",
+  ], 180_000);
+}, { timeout: 1_200_000 });
+
+after(async () => {
+  if (instanceId) await destroy(instanceId);
+  await destroyAll(`${SCENARIO}-`);
+}, { timeout: 600_000 });
+
+test("a secret the mesh generated becomes a key that works", { skip, timeout: 300_000 }, async () => {
+  await meshWrote([
+    { node: "workstation", module: "photos", bucket: "photos", secret: "first-secret-aaaaaaaa" },
+  ]);
+  const { out, ok } = await provision();
+  assert.ok(ok, out);
+
+  const listed = await admin(`admin user list root --json`);
+  assert.ok(listed.out.includes("mesh_workstation_photos"), `no key was made for the consumer:\n${listed.out}`);
+
+  const used = await canUse("mesh_workstation_photos", "first-secret-aaaaaaaa", "photos");
+  assert.ok(used.ok, `the consumer cannot use the bucket the mesh gave it:\n${used.out}`);
+});
+
+test("a consumer cannot reach another consumer's bucket", { skip, timeout: 300_000 }, async () => {
+  // **The assertion this whole scenario exists for.** One store holds every bucket behind one
+  // endpoint, so isolation is a policy rather than a property, and a policy granting
+  // `arn:aws:s3:::*` would pass every other test in this file.
+  await meshWrote([
+    { node: "workstation", module: "photos", bucket: "photos", secret: "first-secret-aaaaaaaa" },
+    { node: "laptop", module: "invoices", bucket: "invoices", secret: "second-secret-bbbbbbbb" },
+  ]);
+  const { out, ok } = await provision();
+  assert.ok(ok, out);
+
+  const own = await canUse("mesh_laptop_invoices", "second-secret-bbbbbbbb", "invoices");
+  assert.ok(own.ok, `a consumer cannot use its own bucket:\n${own.out}`);
+
+  const other = await canUse("mesh_laptop_invoices", "second-secret-bbbbbbbb", "photos");
+  assert.ok(!other.ok, `a consumer reached another consumer's bucket:\n${other.out}`);
+});
+
+test("rotating the secret makes the new one work and the old one stop", { skip, timeout: 300_000 }, async () => {
+  // The failure this guards is a provisioner that only ever creates: the mesh replaces the file,
+  // the user exists, nothing happens, and a rotation reports success while changing nothing.
+  await meshWrote([
+    { node: "workstation", module: "photos", bucket: "photos", secret: "rotated-secret-cccccccc" },
+  ]);
+  const { out, ok } = await provision();
+  assert.ok(ok, out);
+
+  const now = await canUse("mesh_workstation_photos", "rotated-secret-cccccccc", "photos");
+  assert.ok(now.ok, `the rotated secret does not work:\n${now.out}`);
+
+  const before = await canUse("mesh_workstation_photos", "first-secret-aaaaaaaa", "photos");
+  assert.ok(!before.ok, "the secret that was rotated away still works");
+});
+
+test("a consumer that goes away loses its key", { skip, timeout: 300_000 }, async () => {
+  // The half usually missing. Nothing reports a key that outlives its consumer, and it keeps
+  // working for as long as nobody looks.
+  //
+  // **Stages its own precondition rather than inheriting one.** The first version asserted that
+  // `mesh_laptop` was present, having been left by an earlier test — and by then the rotation
+  // test had already rewritten the manifest without it, so revocation had happened for the right
+  // reason two tests too early. The behaviour was correct and the test was measuring residue.
+  await meshWrote([
+    { node: "workstation", module: "photos", bucket: "photos", secret: "rotated-secret-cccccccc" },
+    { node: "laptop", module: "invoices", bucket: "invoices", secret: "second-secret-bbbbbbbb" },
+  ]);
+  const staged = await provision();
+  assert.ok(staged.ok, staged.out);
+  const present = await admin(`admin user list root --json`);
+  assert.ok(present.out.includes("mesh_laptop_invoices"), `the consumer to be removed was never made:\n${present.out}`);
+
+  await meshWrote([
+    { node: "workstation", module: "photos", bucket: "photos", secret: "rotated-secret-cccccccc" },
+  ]);
+  const { out, ok } = await provision();
+  assert.ok(ok, out);
+
+  const after = await admin(`admin user list root --json`);
+  assert.ok(!after.out.includes("mesh_laptop_invoices"), `a key nobody asks for survived:\n${after.out}`);
+  const still = await canUse("mesh_laptop_invoices", "second-secret-bbbbbbbb", "invoices");
+  assert.ok(!still.ok, "a revoked key still works");
+});
+
+test("a key nobody here made is left alone", { skip, timeout: 300_000 }, async () => {
+  // A provisioner that removed every key it did not recognise would be one nobody could safely
+  // run against a store that predates it (novox/hq 04-ISSUES/010).
+  await must(
+    `mc --config-dir /tmp/root-mc admin user add root somebody-elses-key somebody-elses-secret`,
+  );
+  const { out, ok } = await provision();
+  assert.ok(ok, out);
+
+  const listed = await admin(`admin user list root --json`);
+  assert.ok(listed.out.includes("somebody-elses-key"),
+    `a key this provisioner did not make was removed:\n${listed.out}`);
+});
+
+test("a manifest naming a credential that was never written is refused", { skip, timeout: 300_000 }, async () => {
+  // Refused rather than creating a user with no secret — a login nothing can use, which nothing
+  // would report until something tried to connect.
+  await meshWrote([
+    { node: "workstation", module: "photos", bucket: "photos", secret: "rotated-secret-cccccccc" },
+  ]);
+  await must(`rm -f ${GRANTS}/workstation.photos.secret`);
+
+  const { out, ok } = await provision();
+  assert.ok(!ok, `it carried on without the credential:\n${out}`);
+  assert.match(out, /workstation's credential/);
+});
+
+test("a bucket name that would not work is refused by name", { skip, timeout: 300_000 }, async () => {
+  // The refusal names the consumer that asked. The store would refuse it too, as an error inside
+  // a provisioner log with nothing saying whose manifest caused it.
+  await meshWrote([
+    { node: "workstation", module: "photos", bucket: "Photos_2026", secret: "rotated-secret-cccccccc" },
+  ]);
+  const { out, ok } = await provision();
+  assert.ok(!ok, `an unusable bucket name was accepted:\n${out}`);
+  assert.match(out, /workstation asked for a bucket named/);
+});

test/integration/postgres-grant-end-to-end.test.ts

@@ -0,0 +1,247 @@
+/**
+ * The whole grant for a database, mesh-driven — novox/hq ADR 0052/0053, the postgres case.
+ *
+ * The redis bed proves the provider/consumer contract for a cache. This proves it for a database, on
+ * a provider whose code shells out to `psql` (so the runtime image carries it): postgres is assigned,
+ * a consumer that requires postgres-database is assigned, and the mesh mints one password, seals a
+ * copy to each end, and writes each its file. postgres's provisioner — reading only the mesh's
+ * contributions — creates a role and a database under the login the mesh derived, with the password
+ * the mesh minted. The proof is the consumer connecting to its database with the credential the mesh
+ * delivered it. Nothing is placed by the test.
+ *
+ *   MESH_LAB_HOST_BINARY=.../mesh-host   MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
+ *   scripts/build-module-runtime.sh postgres builds mesh-runtime-postgres:development (with psql),
+ *   which scenarios/postgres-node.yml stocks.
+ */
+
+import { test, before, after } from "node:test";
+import assert from "node:assert/strict";
+import { existsSync, readFileSync } from "node:fs";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { raise } from "../../src/lifecycle/raise.ts";
+import { destroy, exec } from "../../src/lifecycle/operate.ts";
+import { hostBinaryPath, HOST_PATH } from "../../src/lifecycle/place.ts";
+import { labIsUsable, destroyAll } from "./harness.ts";
+
+const capability = await labIsUsable();
+const binary = hostBinaryPath();
+const bundle = process.env["MESH_LAB_BUNDLE"] ?? "";
+
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !binary || !existsSync(binary)
+    ? "MESH_LAB_HOST_BINARY is not set to a built mesh-host"
+    : !bundle || !existsSync(bundle)
+      ? "MESH_LAB_BUNDLE is not set to a substrate bundle (mesh-host examples/)"
+      : false;
+
+const SCENARIO = "postgres-node";
+const MACHINE = "anchor";
+
+let instanceId = "";
+let stocked: string[] = [];
+
+function quote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+async function on(command: string, timeoutMs?: number): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `exec 2>&1\n${command}\necho "__exit=$?"`,
+  ], timeoutMs);
+  const marker = stdout.lastIndexOf("__exit=");
+  if (marker < 0) return { out: stdout, ok: false };
+  return { out: stdout.slice(0, marker), ok: stdout.slice(marker + 7).trim() === "0" };
+}
+
+async function must(command: string, timeoutMs?: number): Promise<string> {
+  const { out, ok } = await on(command, timeoutMs);
+  if (!ok) throw new Error(`${MACHINE}: ${command}\n${out}`);
+  return out;
+}
+
+async function mesh(command: string, timeoutMs?: number): Promise<string> {
+  return must(`docker exec mesh-control /mesh-control ${command}`, timeoutMs);
+}
+
+function pinned(repository: string): string {
+  const found = stocked.find((r) => r.slice(r.indexOf("/") + 1, r.indexOf("@")) === repository);
+  assert.ok(found, `the scenario stocks no ${repository}; it serves ${stocked.join(", ")}`);
+  return found;
+}
+
+function bundleFor(images: string[]): string {
+  let text = readFileSync(bundle, "utf8");
+  for (const ref of images) {
+    const repository = ref.slice(ref.indexOf("/") + 1, ref.indexOf("@"));
+    const escaped = repository.replaceAll("/", "\\/").replaceAll(".", "\\.");
+    text = text.replaceAll(new RegExp(`[A-Za-z0-9_.:-]+\\/${escaped}@sha256:[0-9a-f]+`, "g"), ref);
+  }
+  return text;
+}
+
+function tokenFrom(said: string): string {
+  const found = said.split("\n").map((l) => l.trim()).find((l) => l.length > 100 && !l.includes(" "));
+  assert.ok(found, `no token in:\n${said}`);
+  return found;
+}
+
+async function settled(withinMs = 480_000): Promise<void> {
+  const until = Date.now() + withinMs;
+  let last = "";
+  while (Date.now() < until) {
+    const asked = await on(`docker exec mesh-control /mesh-control status --json`);
+    if (asked.ok) {
+      try {
+        const state = JSON.parse(asked.out) as {
+          wrong: { node: string; outcome: string }[];
+          waiting: { node: string }[];
+          reported: { node: string; outcome: string; current: boolean }[];
+        };
+        const bad = state.wrong.find((w) => w.node === MACHINE);
+        if (bad) throw new Error(`${MACHINE} did not apply what it was sent: ${bad.outcome}\n${asked.out}`);
+        const word = state.reported.find((r) => r.node === MACHINE);
+        if (!state.waiting.some((w) => w.node === MACHINE) && word?.outcome === "applied" && word.current) return;
+        last = asked.out;
+      } catch (err) {
+        if (err instanceof Error && err.message.includes("did not apply")) throw err;
+        last = asked.out;
+      }
+    }
+    await new Promise((r) => setTimeout(r, 5000));
+  }
+  throw new Error(`${MACHINE} never caught up within ${Math.round(withinMs / 1000)}s. Last:\n${last}`);
+}
+
+before(async () => {
+  if (skip) return;
+
+  const raised = await raise(loadScenario(`scenarios/${SCENARIO}.yml`), {
+    onProgress: (m) => console.log(`raise: ${m}`),
+  });
+  instanceId = raised.instanceId;
+  stocked = raised.images;
+
+  await must(`cat > /tmp/substrate.lock <<'MESHBUNDLE'\n${bundleFor(raised.images)}\nMESHBUNDLE`);
+  await must(`${HOST_PATH} apply /tmp/substrate.lock`, 600_000);
+  const up = await must(`docker ps --format '{{.Names}}'`);
+  for (const c of ["mesh-store", "mesh-broker", "mesh-control"]) {
+    assert.match(up, new RegExp(c), `the substrate did not raise ${c}:\n${up}`);
+  }
+
+  await mesh(`node add ${MACHINE}`);
+  const token = tokenFrom(await mesh(`token issue --node ${MACHINE}`));
+  await must(`${HOST_PATH} enrol --token ${quote(token)}`);
+  await must(`nohup ${HOST_PATH} run > /var/log/mesh-host.log 2>&1 & sleep 3`);
+}, { timeout: 1_800_000 });
+
+after(async () => {
+  if (instanceId) await destroy(instanceId);
+  await destroyAll(`${SCENARIO}-`);
+}, { timeout: 600_000 });
+
+test("the mesh grants a consumer a postgres database, and the credential it delivers connects", {
+  skip, timeout: 900_000,
+}, async () => {
+  // The PROVIDER: postgres in its committed shape — server and a broker-bound runtime (carrying psql)
+  // on the private postgres network, the runtime running the provisioner.
+  const postgresManifest = JSON.stringify({
+    module: "postgres",
+    version: "1",
+    provides: [{ name: "postgres-database", scope: "mesh" }],
+    serves: { "postgres-database": {} },
+    emits: ["module.postgres.database.provisioned", "module.postgres.database.deprovisioned"],
+    consumes: ["module.postgres.database.provisioned", "module.postgres.database.deprovisioned"],
+    receives: { "postgres-database": "/var/lib/postgres/grants/mesh.json" },
+    grants: { "postgres-database": "/var/lib/postgres/grants" },
+    "own-secrets": { superuser: "/var/lib/postgres/superuser.secret", broker: "/var/lib/mesh/postgres/broker" },
+    resources: [
+      { id: "mesh-state", type: "directory", path: "/var/lib/mesh/postgres", mode: "0700" },
+      { id: "state", type: "directory", path: "/var/lib/postgres", mode: "0700" },
+      { id: "grants", type: "directory", path: "/var/lib/postgres/grants", mode: "0700" },
+      { id: "superuser-env", type: "file", path: "/var/lib/postgres/superuser.env", mode: "0600", content: "POSTGRES_PASSWORD=${secret:superuser}\n" },
+      { id: "data", type: "directory", path: "/services/postgres/db-data", mode: "0700" },
+      { id: "net", type: "network", name: "postgres" },
+      {
+        // No published port here: the substrate's own store already holds host :5432 on this
+        // single-node bed, and the consumer reaches postgres over the private network by name. The
+        // committed manifest publishes it for cross-node consumers, which is a different node.
+        id: "server", type: "container", name: "postgres", image: pinned("postgres"), network: "postgres",
+        env: { POSTGRES_USER: "postgres", POSTGRES_DB: "postgres" },
+        "env-file": ["/var/lib/postgres/superuser.env"],
+        volumes: ["/services/postgres/db-data:/var/lib/postgresql/data"],
+      },
+      {
+        id: "runtime", type: "container", name: "mesh-postgres", image: pinned("mesh-runtime-postgres"),
+        network: "postgres",
+        volumes: [
+          "/var/lib/mesh/postgres/broker:/run/secrets/broker:ro",
+          "/var/lib/postgres/grants:/var/lib/postgres/grants:ro",
+          "/var/lib/postgres/superuser.secret:/run/secrets/superuser:ro",
+        ],
+        env: {
+          MESH_BROKER_FILE: "/run/secrets/broker",
+          MESH_RECEIVES: "/var/lib/postgres/grants/mesh.json",
+          MESH_PROVISION_POSTGRES: "postgres://postgres@postgres:5432/postgres?sslmode=disable",
+          MESH_PROVISION_PASSWORD_FILE: "/run/secrets/superuser",
+        },
+      },
+    ],
+  });
+
+  // The CONSUMER: a module that requires postgres-database and contributes a name so it asks.
+  const consumerManifest = JSON.stringify({
+    module: "dbuser",
+    version: "1",
+    requires: ["postgres-database"],
+    contributes: { "postgres-database": { name: "dbuser" } },
+    binds: { "postgres-database": "/var/lib/dbuser/db.json" },
+    secrets: { "postgres-database": "/var/lib/dbuser/db.secret" },
+    resources: [{ id: "state", type: "directory", path: "/var/lib/dbuser", mode: "0700" }],
+  });
+
+  await must(`printf %s ${quote(postgresManifest)} > /tmp/postgres.json && docker cp /tmp/postgres.json mesh-control:/postgres.json`);
+  await mesh("module add /postgres.json");
+  await mesh(`module issue postgres --node ${MACHINE}`);
+  await mesh(`assign ${MACHINE} postgres`);
+
+  await must(`printf %s ${quote(consumerManifest)} > /tmp/dbuser.json && docker cp /tmp/dbuser.json mesh-control:/dbuser.json`);
+  await mesh("module add /dbuser.json");
+  await mesh(`assign ${MACHINE} dbuser`);
+
+  await mesh(`push ${MACHINE}`);
+  await settled();
+
+  // The mesh delivered the consumer its bound file and its unsealed password.
+  let boundRaw = "";
+  const untilBound = Date.now() + 60_000;
+  while (Date.now() < untilBound) {
+    const got = await on(`cat /var/lib/dbuser/db.json 2>/dev/null`);
+    if (got.ok && /"as"/.test(got.out)) { boundRaw = got.out; break; }
+    await new Promise((r) => setTimeout(r, 3000));
+  }
+  assert.match(boundRaw, /"as"/, `the consumer was never told about its database:\n${boundRaw}`);
+  const bound = JSON.parse(boundRaw) as { as: string; provision: string };
+  assert.equal(bound.provision, "postgres-database");
+  const as = bound.as;
+  const password = (await must(`cat /var/lib/dbuser/db.secret`)).trim();
+  assert.ok(as && password, `the consumer's login or password was empty (as=${as})`);
+
+  // THE PROOF: connect to postgres as the consumer, with the login and password the mesh delivered
+  // it, to the database postgres's provisioner created — a real password-checked TCP connection (the
+  // runtime carries psql). A `1` back means the role, the database, and the password all line up
+  // across the two ends. A provisioner that set a different password answers "authentication failed".
+  const conn = `postgresql://${as}:${encodeURIComponent(password)}@postgres:5432/${as}?sslmode=disable`;
+  let out = { out: "", ok: false };
+  const untilConn = Date.now() + 90_000;
+  while (Date.now() < untilConn) {
+    out = await on(`docker exec mesh-postgres psql ${quote(conn)} -tAc 'select 1' 2>&1`);
+    if (out.ok && /^1$/m.test(out.out)) break;
+    if (/authentication failed/i.test(out.out)) break; // fast-fail: the credential is wrong
+    await new Promise((r) => setTimeout(r, 3000));
+  }
+  assert.doesNotMatch(out.out, /authentication failed/i,
+    `the consumer could not authenticate with the mesh's password — the two ends do not agree:\n${out.out}`);
+  assert.match(out.out, /^1$/m,
+    `the consumer could not connect to its granted database as ${as}:\n${out.out}\n---\n${(await on(`docker logs mesh-postgres 2>&1 | tail -30`)).out}`);
+});

test/integration/provider-on-backend-network.test.ts

@@ -0,0 +1,232 @@
+/**
+ * A provider's runtime, on the backend's own private network, still binds the mesh broker.
+ *
+ * The committed provider manifests keep the backend server on a private container network and put
+ * the tool/provisioner runtime on it too, so the runtime reaches the backend by name (redis:6379).
+ * The runtime also has to bind the mesh broker for its scoped account — reached not on that private
+ * network but via NAT to the broker's node address. This proves that shape: redis's runtime, on the
+ * `redis` network, comes up, provisions a consumer with the mesh's credential, and its scoped account
+ * is on the broker — none of which happens if it could not reach the broker from the bridge.
+ *
+ * This mirrors the redis committed manifest exactly (server on `redis` with a published port, runtime
+ * on `redis`), where provider-uses-mesh-credential used host networking. If this is green, the
+ * private-network shape is the one to roll out to every provider.
+ */
+
+import { test, before, after } from "node:test";
+import assert from "node:assert/strict";
+import { existsSync, readFileSync } from "node:fs";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { raise } from "../../src/lifecycle/raise.ts";
+import { destroy, exec } from "../../src/lifecycle/operate.ts";
+import { hostBinaryPath, HOST_PATH } from "../../src/lifecycle/place.ts";
+import { labIsUsable, destroyAll } from "./harness.ts";
+
+const capability = await labIsUsable();
+const binary = hostBinaryPath();
+const bundle = process.env["MESH_LAB_BUNDLE"] ?? "";
+
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !binary || !existsSync(binary)
+    ? "MESH_LAB_HOST_BINARY is not set to a built mesh-host"
+    : !bundle || !existsSync(bundle)
+      ? "MESH_LAB_BUNDLE is not set to a substrate bundle (mesh-host examples/)"
+      : false;
+
+const SCENARIO = "redis-node";
+const MACHINE = "anchor";
+
+let instanceId = "";
+let stocked: string[] = [];
+
+function quote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+async function on(command: string, timeoutMs?: number): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `exec 2>&1\n${command}\necho "__exit=$?"`,
+  ], timeoutMs);
+  const marker = stdout.lastIndexOf("__exit=");
+  if (marker < 0) return { out: stdout, ok: false };
+  return { out: stdout.slice(0, marker), ok: stdout.slice(marker + 7).trim() === "0" };
+}
+
+async function must(command: string, timeoutMs?: number): Promise<string> {
+  const { out, ok } = await on(command, timeoutMs);
+  if (!ok) throw new Error(`${MACHINE}: ${command}\n${out}`);
+  return out;
+}
+
+async function mesh(command: string, timeoutMs?: number): Promise<string> {
+  return must(`docker exec mesh-control /mesh-control ${command}`, timeoutMs);
+}
+
+function pinned(repository: string): string {
+  const found = stocked.find((r) => r.slice(r.indexOf("/") + 1, r.indexOf("@")) === repository);
+  assert.ok(found, `the scenario stocks no ${repository}; it serves ${stocked.join(", ")}`);
+  return found;
+}
+
+function bundleFor(images: string[]): string {
+  let text = readFileSync(bundle, "utf8");
+  for (const ref of images) {
+    const repository = ref.slice(ref.indexOf("/") + 1, ref.indexOf("@"));
+    const escaped = repository.replaceAll("/", "\\/").replaceAll(".", "\\.");
+    text = text.replaceAll(new RegExp(`[A-Za-z0-9_.:-]+\\/${escaped}@sha256:[0-9a-f]+`, "g"), ref);
+  }
+  return text;
+}
+
+function tokenFrom(said: string): string {
+  const found = said.split("\n").map((l) => l.trim()).find((l) => l.length > 100 && !l.includes(" "));
+  assert.ok(found, `no token in:\n${said}`);
+  return found;
+}
+
+async function settled(withinMs = 480_000): Promise<void> {
+  const until = Date.now() + withinMs;
+  let last = "";
+  while (Date.now() < until) {
+    const asked = await on(`docker exec mesh-control /mesh-control status --json`);
+    if (asked.ok) {
+      try {
+        const state = JSON.parse(asked.out) as {
+          wrong: { node: string; outcome: string }[];
+          waiting: { node: string }[];
+          reported: { node: string; outcome: string; current: boolean }[];
+        };
+        const bad = state.wrong.find((w) => w.node === MACHINE);
+        if (bad) throw new Error(`${MACHINE} did not apply what it was sent: ${bad.outcome}\n${asked.out}`);
+        const word = state.reported.find((r) => r.node === MACHINE);
+        if (!state.waiting.some((w) => w.node === MACHINE) && word?.outcome === "applied" && word.current) return;
+        last = asked.out;
+      } catch (err) {
+        if (err instanceof Error && err.message.includes("did not apply")) throw err;
+        last = asked.out;
+      }
+    }
+    await new Promise((r) => setTimeout(r, 5000));
+  }
+  throw new Error(`${MACHINE} never caught up within ${Math.round(withinMs / 1000)}s. Last:\n${last}`);
+}
+
+before(async () => {
+  if (skip) return;
+
+  const raised = await raise(loadScenario(`scenarios/${SCENARIO}.yml`), {
+    onProgress: (m) => console.log(`raise: ${m}`),
+  });
+  instanceId = raised.instanceId;
+  stocked = raised.images;
+
+  await must(`cat > /tmp/substrate.lock <<'MESHBUNDLE'\n${bundleFor(raised.images)}\nMESHBUNDLE`);
+  await must(`${HOST_PATH} apply /tmp/substrate.lock`, 600_000);
+  const up = await must(`docker ps --format '{{.Names}}'`);
+  for (const c of ["mesh-store", "mesh-broker", "mesh-control"]) {
+    assert.match(up, new RegExp(c), `the substrate did not raise ${c}:\n${up}`);
+  }
+
+  await mesh(`node add ${MACHINE}`);
+  const token = tokenFrom(await mesh(`token issue --node ${MACHINE}`));
+  await must(`${HOST_PATH} enrol --token ${quote(token)}`);
+  await must(`nohup ${HOST_PATH} run > /var/log/mesh-host.log 2>&1 & sleep 3`);
+}, { timeout: 1_800_000 });
+
+after(async () => {
+  if (instanceId) await destroy(instanceId);
+  await destroyAll(`${SCENARIO}-`);
+}, { timeout: 600_000 });
+
+test("redis's runtime, on the backend's private network, binds the broker and provisions with the mesh's credential", {
+  skip, timeout: 900_000,
+}, async () => {
+  // Exactly the committed redis shape: a private `redis` network, the server on it with a published
+  // port, and the runtime on it too — reaching redis by name and the broker by NAT.
+  const manifest = JSON.stringify({
+    module: "redis",
+    version: "1",
+    provides: [{ name: "redis-cache", scope: "mesh" }],
+    serves: { "redis-cache": {} },
+    emits: ["module.redis.cache.provisioned", "module.redis.cache.deprovisioned"],
+    consumes: ["module.redis.cache.provisioned", "module.redis.cache.deprovisioned"],
+    receives: { "redis-cache": "/var/lib/redis-module/grants/mesh.json" },
+    grants: { "redis-cache": "/var/lib/redis-module/grants" },
+    "own-secrets": { default: "/var/lib/redis-module/default.secret", broker: "/var/lib/mesh/redis/broker" },
+    resources: [
+      { id: "mesh-state", type: "directory", path: "/var/lib/mesh/redis", mode: "0700" },
+      { id: "state", type: "directory", path: "/var/lib/redis-module", mode: "0700" },
+      { id: "grants-dir", type: "directory", path: "/var/lib/redis-module/grants", mode: "0700" },
+      { id: "data", type: "directory", path: "/services/redis/data", mode: "0700", owner: "999:999" },
+      {
+        id: "server-conf", type: "file", path: "/var/lib/redis-module/redis.conf", mode: "0644",
+        content: "requirepass ${secret:default}\nappendonly no\ndir /data\n",
+      },
+      { id: "net", type: "network", name: "redis" },
+      {
+        id: "server", type: "container", name: "redis", image: pinned("redis"), network: "redis",
+        ports: ["6379"],
+        volumes: [
+          "/services/redis/data:/data",
+          "/var/lib/redis-module/redis.conf:/etc/redis/redis.conf:ro",
+        ],
+        args: ["/etc/redis/redis.conf"],
+      },
+      {
+        id: "runtime", type: "container", name: "mesh-redis", image: pinned("mesh-runtime-redis"),
+        network: "redis",
+        volumes: [
+          "/var/lib/mesh/redis/broker:/run/secrets/broker:ro",
+          "/var/lib/redis-module/grants:/var/lib/redis-module/grants:ro",
+          "/var/lib/redis-module/default.secret:/run/secrets/default:ro",
+        ],
+        env: {
+          MESH_BROKER_FILE: "/run/secrets/broker",
+          MESH_RECEIVES: "/var/lib/redis-module/grants/mesh.json",
+          MESH_PROVISION_REDIS: "redis:6379",
+          MESH_PROVISION_PASSWORD_FILE: "/run/secrets/default",
+        },
+      },
+    ],
+  });
+  await must(`printf %s ${quote(manifest)} > /tmp/redis.json && docker cp /tmp/redis.json mesh-control:/redis.json`);
+  await mesh("module add /redis.json");
+  await mesh(`module issue redis --node ${MACHINE}`);
+  await mesh(`assign ${MACHINE} redis`);
+  await mesh(`push ${MACHINE}`);
+  await settled();
+
+  const running = await must(`docker ps --format '{{.Names}}'`);
+  assert.match(running, /mesh-redis/, `redis's runtime is not running:\n${(await on(`docker logs mesh-redis 2>&1 | tail -30`)).out}`);
+
+  // It bound the broker from the private network: its scoped account is on the broker. If NAT to the
+  // broker had failed, the runtime would have crashed and never authenticated.
+  const users = await must(`docker exec mesh-broker lavinmqctl list_users 2>&1`);
+  assert.match(users, /anchor-redis/,
+    `the runtime's scoped account is not on the broker — it did not reach the broker from the bridge:\n` +
+      `${(await on(`docker logs mesh-redis 2>&1 | tail -30`)).out}`);
+
+  // The mesh delivers the contribution and the unsealed password; the provisioner creates the login.
+  const password = "mesh-minted-bridge-7c1";
+  await must(`printf %s ${quote(password)} > /var/lib/redis-module/grants/app.secret`);
+  const contributions = JSON.stringify({
+    contributions: 1, requirement: "redis-cache", generated: "by the mesh",
+    given: [{ from: "app", node: "app-node", at: "192.0.2.20:6379", as: "app-one", secret: "/var/lib/redis-module/grants/app.secret", values: {} }],
+  });
+  await must(`printf %s ${quote(contributions)} > /var/lib/redis-module/grants/mesh.json`);
+
+  const adminPw = await must(`cat /var/lib/redis-module/default.secret`);
+  let acl = "";
+  const until = Date.now() + 60_000;
+  while (Date.now() < until) {
+    acl = (await on(`docker exec redis redis-cli -a ${quote(adminPw)} --no-auth-warning ACL LIST 2>/dev/null`)).out;
+    if (/app-one/.test(acl)) break;
+    await new Promise((r) => setTimeout(r, 3000));
+  }
+  assert.match(acl, /app-one/, `the provisioner never created the login:\n${(await on(`docker logs mesh-redis 2>&1 | tail -30`)).out}\n---\n${acl}`);
+
+  const authed = await on(`docker exec redis redis-cli --user app-one --pass ${quote(password)} --no-auth-warning PING 2>&1`);
+  assert.doesNotMatch(authed.out, /WRONGPASS/, `the consumer could not authenticate with the mesh's password:\n${authed.out}`);
+  assert.match(authed.out, /PONG/, `expected PONG:\n${authed.out}`);
+});

test/integration/provider-uses-mesh-credential.test.ts

@@ -0,0 +1,242 @@
+/**
+ * A provider creates the resource with the credential the mesh minted — novox/hq ADR 0053.
+ *
+ * The old provisioner generated its own password, sealed it with a key nothing delivered, and
+ * handed it back. This proves the corrected contract: redis's provisioner reads the mesh's
+ * contributions file and, for each consumer, the password the mesh minted and the host unsealed, and
+ * creates the ACL user under the login the mesh derived, with that exact password. No $MESH_SEAL_KEY
+ * is set anywhere. The proof is authentication: a client logging in as that consumer with the mesh's
+ * password gets PONG — where a provisioner that invented its own password would answer WRONGPASS.
+ *
+ * A hand-written contributions file and secret stand in for the control plane here (a full grant
+ * from a second module is a heavier bed); their SHAPE is exactly what mesh-control writes — a
+ * `receives` doc with `as`/`secret`, and the secret file the host leaves after unsealing.
+ *
+ *   MESH_LAB_HOST_BINARY=.../mesh-host   MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
+ *   scripts/build-module-runtime.sh redis builds mesh-runtime-redis:development, which
+ *   scenarios/redis-node.yml stocks.
+ */
+
+import { test, before, after } from "node:test";
+import assert from "node:assert/strict";
+import { existsSync, readFileSync } from "node:fs";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { raise } from "../../src/lifecycle/raise.ts";
+import { destroy, exec } from "../../src/lifecycle/operate.ts";
+import { hostBinaryPath, HOST_PATH } from "../../src/lifecycle/place.ts";
+import { labIsUsable, destroyAll } from "./harness.ts";
+
+const capability = await labIsUsable();
+const binary = hostBinaryPath();
+const bundle = process.env["MESH_LAB_BUNDLE"] ?? "";
+
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !binary || !existsSync(binary)
+    ? "MESH_LAB_HOST_BINARY is not set to a built mesh-host"
+    : !bundle || !existsSync(bundle)
+      ? "MESH_LAB_BUNDLE is not set to a substrate bundle (mesh-host examples/)"
+      : false;
+
+const SCENARIO = "redis-node";
+const MACHINE = "anchor";
+
+let instanceId = "";
+let stocked: string[] = [];
+
+function quote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+async function on(command: string, timeoutMs?: number): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `exec 2>&1\n${command}\necho "__exit=$?"`,
+  ], timeoutMs);
+  const marker = stdout.lastIndexOf("__exit=");
+  if (marker < 0) return { out: stdout, ok: false };
+  return { out: stdout.slice(0, marker), ok: stdout.slice(marker + 7).trim() === "0" };
+}
+
+async function must(command: string, timeoutMs?: number): Promise<string> {
+  const { out, ok } = await on(command, timeoutMs);
+  if (!ok) throw new Error(`${MACHINE}: ${command}\n${out}`);
+  return out;
+}
+
+async function mesh(command: string, timeoutMs?: number): Promise<string> {
+  return must(`docker exec mesh-control /mesh-control ${command}`, timeoutMs);
+}
+
+function pinned(repository: string): string {
+  const found = stocked.find((r) => r.slice(r.indexOf("/") + 1, r.indexOf("@")) === repository);
+  assert.ok(found, `the scenario stocks no ${repository}; it serves ${stocked.join(", ")}`);
+  return found;
+}
+
+function bundleFor(images: string[]): string {
+  let text = readFileSync(bundle, "utf8");
+  for (const ref of images) {
+    const repository = ref.slice(ref.indexOf("/") + 1, ref.indexOf("@"));
+    const escaped = repository.replaceAll("/", "\\/").replaceAll(".", "\\.");
+    text = text.replaceAll(new RegExp(`[A-Za-z0-9_.:-]+\\/${escaped}@sha256:[0-9a-f]+`, "g"), ref);
+  }
+  return text;
+}
+
+function tokenFrom(said: string): string {
+  const found = said.split("\n").map((l) => l.trim()).find((l) => l.length > 100 && !l.includes(" "));
+  assert.ok(found, `no token in:\n${said}`);
+  return found;
+}
+
+async function settled(withinMs = 480_000): Promise<void> {
+  const until = Date.now() + withinMs;
+  let last = "";
+  while (Date.now() < until) {
+    const asked = await on(`docker exec mesh-control /mesh-control status --json`);
+    if (asked.ok) {
+      try {
+        const state = JSON.parse(asked.out) as {
+          wrong: { node: string; outcome: string }[];
+          waiting: { node: string }[];
+          reported: { node: string; outcome: string; current: boolean }[];
+        };
+        const bad = state.wrong.find((w) => w.node === MACHINE);
+        if (bad) throw new Error(`${MACHINE} did not apply what it was sent: ${bad.outcome}\n${asked.out}`);
+        const word = state.reported.find((r) => r.node === MACHINE);
+        if (!state.waiting.some((w) => w.node === MACHINE) && word?.outcome === "applied" && word.current) return;
+        last = asked.out;
+      } catch (err) {
+        if (err instanceof Error && err.message.includes("did not apply")) throw err;
+        last = asked.out;
+      }
+    }
+    await new Promise((r) => setTimeout(r, 5000));
+  }
+  throw new Error(`${MACHINE} never caught up within ${Math.round(withinMs / 1000)}s. Last:\n${last}`);
+}
+
+before(async () => {
+  if (skip) return;
+
+  const raised = await raise(loadScenario(`scenarios/${SCENARIO}.yml`), {
+    onProgress: (m) => console.log(`raise: ${m}`),
+  });
+  instanceId = raised.instanceId;
+  stocked = raised.images;
+
+  await must(`cat > /tmp/substrate.lock <<'MESHBUNDLE'\n${bundleFor(raised.images)}\nMESHBUNDLE`);
+  await must(`${HOST_PATH} apply /tmp/substrate.lock`, 600_000);
+  const up = await must(`docker ps --format '{{.Names}}'`);
+  for (const c of ["mesh-store", "mesh-broker", "mesh-control"]) {
+    assert.match(up, new RegExp(c), `the substrate did not raise ${c}:\n${up}`);
+  }
+
+  await mesh(`node add ${MACHINE}`);
+  const token = tokenFrom(await mesh(`token issue --node ${MACHINE}`));
+  await must(`${HOST_PATH} enrol --token ${quote(token)}`);
+  await must(`nohup ${HOST_PATH} run > /var/log/mesh-host.log 2>&1 & sleep 3`);
+}, { timeout: 1_800_000 });
+
+after(async () => {
+  if (instanceId) await destroy(instanceId);
+  await destroyAll(`${SCENARIO}-`);
+}, { timeout: 600_000 });
+
+test("redis creates a consumer's login with the password the mesh minted, sealing nothing", {
+  skip, timeout: 900_000,
+}, async () => {
+  // redis as a provider: the server, and a broker-bound runtime that serves its tools AND runs its
+  // provisioner. The provisioner is pointed at the contributions file the mesh would write
+  // (MESH_RECEIVES). There is NO MESH_SEAL_KEY — the whole point of ADR 0053 is that a provider
+  // needs none.
+  const manifest = JSON.stringify({
+    module: "redis",
+    version: "1",
+    emits: ["module.redis.cache.provisioned", "module.redis.cache.deprovisioned"],
+    consumes: ["module.redis.cache.provisioned", "module.redis.cache.deprovisioned"],
+    "own-secrets": { default: "/var/lib/redis-module/default.secret", broker: "/var/lib/mesh/redis/broker" },
+    resources: [
+      { id: "mesh-state", type: "directory", path: "/var/lib/mesh/redis", mode: "0700" },
+      { id: "state", type: "directory", path: "/var/lib/redis-module", mode: "0700" },
+      { id: "grants-dir", type: "directory", path: "/var/lib/redis-module/grants", mode: "0700" },
+      { id: "data", type: "directory", path: "/services/redis/data", mode: "0700", owner: "999:999" },
+      {
+        id: "server-conf", type: "file", path: "/var/lib/redis-module/redis.conf", mode: "0644",
+        content: "requirepass ${secret:default}\nappendonly no\ndir /data\n",
+      },
+      {
+        id: "server", type: "container", name: "redis", image: pinned("redis"), network: "host",
+        volumes: [
+          "/services/redis/data:/data",
+          "/var/lib/redis-module/redis.conf:/etc/redis/redis.conf:ro",
+        ],
+        args: ["/etc/redis/redis.conf"],
+      },
+      {
+        id: "runtime", type: "container", name: "mesh-redis", image: pinned("mesh-runtime-redis"),
+        network: "host",
+        volumes: [
+          "/var/lib/mesh/redis/broker:/run/secrets/broker:ro",
+          "/var/lib/redis-module/grants:/var/lib/redis-module/grants",
+          "/var/lib/redis-module/default.secret:/run/secrets/default:ro",
+        ],
+        env: {
+          MESH_BROKER_FILE: "/run/secrets/broker",
+          MESH_RECEIVES: "/var/lib/redis-module/grants/redis-cache.json",
+          MESH_PROVISION_REDIS: "127.0.0.1:6379",
+          MESH_PROVISION_PASSWORD_FILE: "/run/secrets/default",
+        },
+      },
+    ],
+  });
+  await must(`printf %s ${quote(manifest)} > /tmp/redis.json && docker cp /tmp/redis.json mesh-control:/redis.json`);
+  await mesh("module add /redis.json");
+  await mesh(`module issue redis --node ${MACHINE}`);
+  await mesh(`assign ${MACHINE} redis`);
+  await mesh(`push ${MACHINE}`);
+  await settled();
+
+  const running = await must(`docker ps --format '{{.Names}}'`);
+  assert.match(running, /mesh-redis/, `redis's runtime is not running:\n${(await on(`docker logs mesh-redis 2>&1 | tail -20`)).out}`);
+
+  // What the mesh delivers to the provider: a contributions file naming the consumer's login and
+  // where its password is, and the password itself as the file the host leaves after unsealing.
+  const password = "mesh-minted-9f3c2a";
+  await must(`printf %s ${quote(password)} > /var/lib/redis-module/grants/app.secret`);
+  const contributions = JSON.stringify({
+    contributions: 1,
+    requirement: "redis-cache",
+    generated: "by the mesh — do not edit",
+    given: [
+      { from: "app", node: "app-node", at: "192.0.2.20:6379", as: "app-one", secret: "/var/lib/redis-module/grants/app.secret", values: {} },
+    ],
+  });
+  await must(`printf %s ${quote(contributions)} > /var/lib/redis-module/grants/redis-cache.json`);
+
+  // Within a reconcile tick the provisioner creates the ACL user. It exists on the server.
+  let acl = "";
+  const until = Date.now() + 60_000;
+  while (Date.now() < until) {
+    acl = (await on(`docker exec redis redis-cli -a ${quote(await must(`cat /var/lib/redis-module/default.secret`))} --no-auth-warning ACL LIST 2>/dev/null`)).out;
+    if (/app-one/.test(acl)) break;
+    await new Promise((r) => setTimeout(r, 3000));
+  }
+  assert.match(acl, /app-one/, `the provisioner never created the consumer's login:\n${(await on(`docker logs mesh-redis 2>&1 | tail -30`)).out}\n---\n${acl}`);
+
+  // The proof: authenticate as that consumer with the password the MESH minted. PONG means the
+  // provisioner created the login with exactly that password. A provisioner that invented its own
+  // (the old behaviour) would answer WRONGPASS here.
+  const authed = await on(`docker exec redis redis-cli --user app-one --pass ${quote(password)} --no-auth-warning PING 2>&1`);
+  assert.doesNotMatch(authed.out, /WRONGPASS/,
+    `the consumer could not authenticate with the mesh's password — the provider used a different one:\n${authed.out}`);
+  assert.match(authed.out, /PONG/, `expected PONG authenticating as the consumer:\n${authed.out}`);
+
+  // And it needed no seal key: the runtime came up and provisioned with MESH_SEAL_KEY set nowhere.
+  const env = await must(`docker inspect mesh-redis --format '{{json .Config.Env}}'`);
+  assert.doesNotMatch(env, /MESH_SEAL_KEY/, `a seal key was set after all — ADR 0053 is not what ran:\n${env}`);
+
+  // The provisioner emitted its lifecycle event under the bound account, and no emit was refused.
+  const log = (await on(`docker logs mesh-redis 2>&1`)).out;
+  assert.doesNotMatch(log, /emit .*failed/, `the provisioned event was refused:\n${log}`);
+});

test/integration/provisioner.test.ts

@@ -75,12 +75,12 @@ async function meshWrote(
 ): Promise<void> {
   const manifest = {
     contributions: 1,
-    requirement: "database",
+    requirement: "postgres-database",
     generated: "by the mesh",
     given: consumers.map((c) => ({
       from: c.module,
       node: c.node,
-      secret: `${GRANTS}/${c.node}.secret`,
+      secret: `${GRANTS}/${c.node}.${c.module}.secret`,
       values: { name: c.name },
     })),
   };
@@ -90,8 +90,8 @@ async function meshWrote(
   // make the revocation test pass for a reason that is not the one being tested.
   await must(`find ${GRANTS} -name '*.secret' -delete`);
   for (const c of consumers) {
-    await must(`printf %s ${shellQuote(c.password)} > ${GRANTS}/${c.node}.secret`);
-    await must(`chmod 600 ${GRANTS}/${c.node}.secret`);
+    await must(`printf %s ${shellQuote(c.password)} > ${GRANTS}/${c.node}.${c.module}.secret`);
+    await must(`chmod 600 ${GRANTS}/${c.node}.${c.module}.secret`);
   }
 }
 
@@ -178,9 +178,9 @@ test("a password the mesh generated becomes a login that works", { skip, timeout
   const { out, ok } = await provision();
   assert.ok(ok, out);
 
-  assert.equal(await sql(`select rolcanlogin from pg_roles where rolname = 'mesh_workstation'`), "t");
+  assert.equal(await sql(`select rolcanlogin from pg_roles where rolname = 'mesh_workstation_meshboard'`), "t");
   assert.equal(await sql(`select 1 from pg_database where datname = 'meshboard'`), "1");
-  const attempt = await tryLogIn("mesh_workstation", "first-password-aaa", "meshboard");
+  const attempt = await tryLogIn("mesh_workstation_meshboard", "first-password-aaa", "meshboard");
   assert.ok(attempt.ok, `the consumer cannot log in with the password the mesh gave it:\n${attempt.out}`);
 });
 
@@ -190,7 +190,7 @@ test("running it again reaches the same state and says nothing", { skip, timeout
   const { out, ok } = await provision();
   assert.ok(ok, out);
   assert.equal(out.trim(), "", `it did work on a second run: ${out}`);
-  assert.ok(await canLogIn("mesh_workstation", "first-password-aaa", "meshboard"));
+  assert.ok(await canLogIn("mesh_workstation_meshboard", "first-password-aaa", "meshboard"));
 });
 
 test("rotating the password makes the new one work and the old one stop", { skip, timeout: 300_000 }, async () => {
@@ -203,11 +203,11 @@ test("rotating the password makes the new one work and the old one stop", { skip
   assert.ok(ok, out);
 
   assert.ok(
-    await canLogIn("mesh_workstation", "second-password-bbb", "meshboard"),
+    await canLogIn("mesh_workstation_meshboard", "second-password-bbb", "meshboard"),
     "the rotated password does not work",
   );
   assert.equal(
-    await canLogIn("mesh_workstation", "first-password-aaa", "meshboard"),
+    await canLogIn("mesh_workstation_meshboard", "first-password-aaa", "meshboard"),
     false,
     "the old password still works, so the rotation changed nothing",
   );
@@ -220,11 +220,11 @@ test("a consumer that goes away loses its login", { skip, timeout: 300_000 }, as
   await meshWrote([]);
   const { out, ok } = await provision();
   assert.ok(ok, out);
-  assert.match(out, /revoked mesh_workstation/);
+  assert.match(out, /revoked mesh_workstation_meshboard/);
 
-  assert.equal(await sql(`select rolcanlogin from pg_roles where rolname = 'mesh_workstation'`), "f");
+  assert.equal(await sql(`select rolcanlogin from pg_roles where rolname = 'mesh_workstation_meshboard'`), "f");
   assert.equal(
-    await canLogIn("mesh_workstation", "second-password-bbb", "meshboard"),
+    await canLogIn("mesh_workstation_meshboard", "second-password-bbb", "meshboard"),
     false,
     "a consumer nobody asks for any more can still log in",
   );
@@ -247,12 +247,12 @@ test("a manifest naming a credential that was never written is refused", { skip,
   // report until something tried to connect.
   await meshWrote([]);
   await must(
-    `printf %s '{"contributions":1,"requirement":"database","given":[` +
-      `{"from":"meshboard","node":"ghost","secret":"${GRANTS}/ghost.secret","values":{"name":"ghost"}}` +
+    `printf %s '{"contributions":1,"requirement":"postgres-database","given":[` +
+      `{"from":"meshboard","node":"ghost","secret":"${GRANTS}/ghost.meshboard.secret","values":{"name":"ghost"}}` +
       `]}' > ${GRANTS}/mesh.json`,
   );
   const { out, ok } = await provision();
   assert.equal(ok, false, "it carried on past a missing credential");
-  assert.match(out, /should be at .*ghost\.secret/);
-  assert.equal(await sql(`select count(*) from pg_roles where rolname = 'mesh_ghost'`), "0");
+  assert.match(out, /should be at .*ghost\.meshboard\.secret/);
+  assert.equal(await sql(`select count(*) from pg_roles where rolname = 'mesh_ghost_meshboard'`), "0");
 });

test/integration/runtime-restart-on-config.test.ts

@@ -0,0 +1,209 @@
+/**
+ * A running tool runtime picks up a settings change — novox/hq 04-ISSUES/009, and its fix.
+ *
+ * A runtime reads its settings-merged config file once, at start. Change the settings on an
+ * already-running runtime and, without this, the container keeps the value it read: its spec did
+ * not move (a mounted file's content is not part of it) so the host left it alone, and every check
+ * passed while the mesh did the old thing. The fix gives a container `restart-on`, the same field a
+ * service has: the runtime names its config resource, and the host recreates the container when that
+ * resource changed this pass.
+ *
+ * This assigns grafana configured by settings, then changes the token and pushes again, and asserts
+ * the container was replaced (a new container id) and the config on disk carries the new value.
+ * It builds the host from source (no --no-build), because the behaviour under test is the host's.
+ *
+ *   MESH_LAB_HOST_BINARY=.../mesh-host   MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
+ *   scripts/build-module-runtime.sh grafana builds mesh-runtime-grafana:development, which
+ *   scenarios/grafana-node.yml stocks.
+ */
+
+import { test, before, after } from "node:test";
+import assert from "node:assert/strict";
+import { existsSync, readFileSync } from "node:fs";
+import { loadScenario } from "../../src/declaration/parse.ts";
+import { raise } from "../../src/lifecycle/raise.ts";
+import { destroy, exec } from "../../src/lifecycle/operate.ts";
+import { hostBinaryPath, HOST_PATH } from "../../src/lifecycle/place.ts";
+import { labIsUsable, destroyAll } from "./harness.ts";
+
+const capability = await labIsUsable();
+const binary = hostBinaryPath();
+const bundle = process.env["MESH_LAB_BUNDLE"] ?? "";
+
+const skip = !capability.usable
+  ? `lab not usable: ${capability.why}`
+  : !binary || !existsSync(binary)
+    ? "MESH_LAB_HOST_BINARY is not set to a built mesh-host"
+    : !bundle || !existsSync(bundle)
+      ? "MESH_LAB_BUNDLE is not set to a substrate bundle (mesh-host examples/)"
+      : false;
+
+const SCENARIO = "grafana-node";
+const MACHINE = "anchor";
+
+let instanceId = "";
+let stocked: string[] = [];
+
+function quote(s: string): string {
+  return `'${s.replaceAll("'", `'\\''`)}'`;
+}
+
+async function on(command: string, timeoutMs?: number): Promise<{ out: string; ok: boolean }> {
+  const { stdout } = await exec(instanceId, MACHINE, [
+    "sh", "-c", `exec 2>&1\n${command}\necho "__exit=$?"`,
+  ], timeoutMs);
+  const marker = stdout.lastIndexOf("__exit=");
+  if (marker < 0) return { out: stdout, ok: false };
+  return { out: stdout.slice(0, marker), ok: stdout.slice(marker + 7).trim() === "0" };
+}
+
+async function must(command: string, timeoutMs?: number): Promise<string> {
+  const { out, ok } = await on(command, timeoutMs);
+  if (!ok) throw new Error(`${MACHINE}: ${command}\n${out}`);
+  return out;
+}
+
+async function mesh(command: string, timeoutMs?: number): Promise<string> {
+  return must(`docker exec mesh-control /mesh-control ${command}`, timeoutMs);
+}
+
+function pinned(repository: string): string {
+  const found = stocked.find((r) => r.slice(r.indexOf("/") + 1, r.indexOf("@")) === repository);
+  assert.ok(found, `the scenario stocks no ${repository}; it serves ${stocked.join(", ")}`);
+  return found;
+}
+
+function bundleFor(images: string[]): string {
+  let text = readFileSync(bundle, "utf8");
+  for (const ref of images) {
+    const repository = ref.slice(ref.indexOf("/") + 1, ref.indexOf("@"));
+    const escaped = repository.replaceAll("/", "\\/").replaceAll(".", "\\.");
+    text = text.replaceAll(new RegExp(`[A-Za-z0-9_.:-]+\\/${escaped}@sha256:[0-9a-f]+`, "g"), ref);
+  }
+  return text;
+}
+
+function tokenFrom(said: string): string {
+  const found = said.split("\n").map((l) => l.trim()).find((l) => l.length > 100 && !l.includes(" "));
+  assert.ok(found, `no token in:\n${said}`);
+  return found;
+}
+
+async function settled(withinMs = 480_000): Promise<void> {
+  const until = Date.now() + withinMs;
+  let last = "";
+  while (Date.now() < until) {
+    const asked = await on(`docker exec mesh-control /mesh-control status --json`);
+    if (asked.ok) {
+      try {
+        const state = JSON.parse(asked.out) as {
+          wrong: { node: string; outcome: string }[];
+          waiting: { node: string }[];
+          reported: { node: string; outcome: string; current: boolean }[];
+        };
+        const bad = state.wrong.find((w) => w.node === MACHINE);
+        if (bad) throw new Error(`${MACHINE} did not apply what it was sent: ${bad.outcome}\n${asked.out}`);
+        const word = state.reported.find((r) => r.node === MACHINE);
+        if (!state.waiting.some((w) => w.node === MACHINE) && word?.outcome === "applied" && word.current) return;
+        last = asked.out;
+      } catch (err) {
+        if (err instanceof Error && err.message.includes("did not apply")) throw err;
+        last = asked.out;
+      }
+    }
+    await new Promise((r) => setTimeout(r, 5000));
+  }
+  throw new Error(`${MACHINE} never caught up within ${Math.round(withinMs / 1000)}s. Last:\n${last}`);
+}
+
+async function setToken(token: string): Promise<void> {
+  const settings = JSON.stringify({ url: "http://127.0.0.1:3000", token });
+  await must(`printf %s ${quote(settings)} > /tmp/s.json && docker cp /tmp/s.json mesh-control:/s.json`);
+  await mesh(`settings set grafana /s.json --node ${MACHINE}`);
+}
+
+async function containerId(): Promise<string> {
+  return (await must(`docker inspect mesh-grafana --format '{{.Id}}'`)).trim();
+}
+
+before(async () => {
+  if (skip) return;
+
+  const raised = await raise(loadScenario(`scenarios/${SCENARIO}.yml`), {
+    onProgress: (m) => console.log(`raise: ${m}`),
+  });
+  instanceId = raised.instanceId;
+  stocked = raised.images;
+
+  await must(`cat > /tmp/substrate.lock <<'MESHBUNDLE'\n${bundleFor(raised.images)}\nMESHBUNDLE`);
+  await must(`${HOST_PATH} apply /tmp/substrate.lock`, 600_000);
+  const up = await must(`docker ps --format '{{.Names}}'`);
+  for (const c of ["mesh-store", "mesh-broker", "mesh-control"]) {
+    assert.match(up, new RegExp(c), `the substrate did not raise ${c}:\n${up}`);
+  }
+
+  await mesh(`node add ${MACHINE}`);
+  const token = tokenFrom(await mesh(`token issue --node ${MACHINE}`));
+  await must(`${HOST_PATH} enrol --token ${quote(token)}`);
+  await must(`nohup ${HOST_PATH} run > /var/log/mesh-host.log 2>&1 & sleep 3`);
+}, { timeout: 1_800_000 });
+
+after(async () => {
+  if (instanceId) await destroy(instanceId);
+  await destroyAll(`${SCENARIO}-`);
+}, { timeout: 600_000 });
+
+test("a running runtime is recreated when its settings change, and reads the new value", {
+  skip, timeout: 900_000,
+}, async () => {
+  const manifest = JSON.stringify({
+    module: "grafana",
+    version: "1",
+    emits: ["module.grafana.alert.firing"],
+    "own-secrets": { broker: "/var/lib/mesh/grafana/broker" },
+    resources: [
+      { id: "mesh-state", type: "directory", path: "/var/lib/mesh/grafana", mode: "0700" },
+      { id: "runtime-config", type: "file", path: "/var/lib/mesh/grafana/config.json", mode: "0600", content: "{}\n", merge: "json" },
+      {
+        id: "runtime", type: "container", name: "mesh-grafana", image: pinned("mesh-runtime-grafana"),
+        network: "host",
+        "restart-on": ["runtime-config"],
+        volumes: [
+          "/var/lib/mesh/grafana/broker:/run/secrets/broker:ro",
+          "/var/lib/mesh/grafana/config.json:/run/config/config.json:ro",
+        ],
+        env: { MESH_BROKER_FILE: "/run/secrets/broker", MESH_GRAFANA_CONFIG_FILE: "/run/config/config.json" },
+      },
+    ],
+  });
+  await must(`printf %s ${quote(manifest)} > /tmp/grafana.json && docker cp /tmp/grafana.json mesh-control:/grafana.json`);
+  await mesh("module add /grafana.json");
+
+  await setToken("token-alpha");
+  await mesh(`module issue grafana --node ${MACHINE}`);
+  await mesh(`assign ${MACHINE} grafana`);
+  await mesh(`push ${MACHINE}`);
+  await settled();
+
+  const before = await containerId();
+  const configBefore = await must(`cat /var/lib/mesh/grafana/config.json`);
+  assert.match(configBefore, /token-alpha/, `first settings not rendered:\n${configBefore}`);
+
+  // Change the setting on the already-running runtime, and push. Nothing about the container's
+  // spec changes — only the content of the file it mounts.
+  await setToken("token-bravo");
+  await mesh(`push ${MACHINE}`);
+  await settled();
+
+  const configAfter = await must(`cat /var/lib/mesh/grafana/config.json`);
+  assert.match(configAfter, /token-bravo/, `the settings change did not re-render the config:\n${configAfter}`);
+
+  const after = await containerId();
+  assert.notEqual(after, before,
+    `the runtime was NOT recreated on a config change (issue 009 not fixed): id stayed ${before}\n` +
+      `host log:\n${(await on(`grep -i grafana /var/log/mesh-host.log | tail -10`)).out}`);
+
+  // And it is running on the new container, so the process re-read the new config.
+  const running = await must(`docker ps --format '{{.Names}}'`);
+  assert.match(running, /mesh-grafana/, "the recreated runtime is not running");
+});

test/lastrun.test.ts

@@ -0,0 +1,167 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { execFileSync } from "node:child_process";
+import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { endToEnd, headOf, judge, type Receipt } from "../src/lastrun.ts";
+
+const now = new Date("2026-08-31T12:00:00Z");
+const passing = (at: string, against: Record<string, string>): Receipt =>
+  ({ at, passed: 22, failed: 0, against, ran: [endToEnd] });
+
+// A machine that has never run it is told so, rather than told nothing.
+//
+// novox/hq 04-ISSUES/005: the harness it replaces had not built for two and a half months and
+// nothing said so. Silence and success must never look alike.
+test("a machine that has never run the suite is told so", () => {
+  const said = judge(null, now, { "mesh-lab": "aaa" });
+  assert.equal(said.current, false);
+  assert.match(said.lines.join("\n"), /never run/);
+});
+
+// The one that matters: it passed, and against code nobody runs any more.
+test("a run against code that has since changed is not current", () => {
+  const said = judge(
+    passing("2026-08-31T11:00:00Z", { "mesh-lab": "aaa", "mesh-control": "bbb" }),
+    now,
+    { "mesh-lab": "aaa", "mesh-control": "ccc" },
+  );
+  assert.equal(said.current, false, "a run against changed code was reported as current");
+  const text = said.lines.join("\n");
+  assert.match(text, /mesh-control\s+at bbb, now at ccc/, text);
+  assert.match(text, /code that has since changed/, text);
+});
+
+// Passing, recent, and against exactly this code is the only thing that counts.
+test("a recent run against this code is current", () => {
+  const said = judge(
+    passing("2026-08-31T11:00:00Z", { "mesh-lab": "aaa" }),
+    now,
+    { "mesh-lab": "aaa" },
+  );
+  assert.equal(said.current, true, said.lines.join("\n"));
+  assert.match(said.lines.join("\n"), /unchanged/);
+});
+
+// Old is a different complaint from moved, and says so — otherwise somebody goes looking for a
+// change that did not happen.
+test("a run that is merely old says that, not that something changed", () => {
+  const said = judge(
+    passing("2026-08-01T11:00:00Z", { "mesh-lab": "aaa" }),
+    now,
+    { "mesh-lab": "aaa" },
+  );
+  assert.equal(said.current, false);
+  const text = said.lines.join("\n");
+  assert.match(text, /Nothing has changed since/, text);
+  assert.doesNotMatch(text, /has since changed/, text);
+});
+
+// A failed run is recorded, and does not count as coverage.
+test("a run that failed is not coverage", () => {
+  const said = judge(
+    {
+      at: "2026-08-31T11:00:00Z",
+      passed: 21,
+      failed: 1,
+      against: { "mesh-lab": "aaa" },
+      ran: [endToEnd],
+    },
+    now,
+    { "mesh-lab": "aaa" },
+  );
+  assert.equal(said.current, false);
+  assert.match(said.lines.join("\n"), /1 test\(s\) failed/);
+  assert.match(said.lines.join("\n"), /Nothing has been proven end to end since/);
+});
+
+// A repository the run never accounted for is named, rather than passing silently: a receipt that
+// says nothing about something is not a receipt that clears it.
+test("a repository the run did not account for is named", () => {
+  const said = judge(
+    passing("2026-08-31T11:00:00Z", { "mesh-lab": "aaa" }),
+    now,
+    { "mesh-lab": "aaa", "mesh-host": "ddd" },
+  );
+  assert.equal(said.current, false);
+  assert.match(said.lines.join("\n"), /mesh-host\s+was not accounted for/);
+});
+
+// A green run of something else is not a green run of this.
+//
+// The suite takes paths, so it can be pointed at one quick unit file. Without recording what it
+// ran, that receipt and a receipt for the real thing are the same document — which is the whole
+// fault of novox/hq 04-ISSUES/005, reintroduced by the fix for it.
+test("a run that raised no machines is not end-to-end coverage", () => {
+  const said = judge(
+    {
+      at: "2026-08-31T11:00:00Z",
+      passed: 6,
+      failed: 0,
+      against: { "mesh-lab": "aaa" },
+      ran: ["test/lastrun.test.ts"],
+    },
+    now,
+    { "mesh-lab": "aaa" },
+  );
+  assert.equal(said.current, false, "a unit run was accepted as end-to-end coverage");
+  assert.match(said.lines.join("\n"), /raised no machines/);
+});
+
+// A receipt written before the mesh recorded what it ran claims nothing, and is read as claiming
+// nothing — not as claiming everything.
+test("a receipt from before this was recorded is not read as covering everything", () => {
+  const old = { at: "2026-08-31T11:00:00Z", passed: 22, failed: 0, against: { "mesh-lab": "aaa" } };
+  const said = judge(old as unknown as Receipt, now, { "mesh-lab": "aaa" });
+  assert.equal(said.current, false);
+});
+
+// A dirty tree is never equal to the clean commit it sits on.
+//
+// The run tested what was on disk. Naming the bare hash would claim coverage of code nobody can
+// check out — and nothing else could tell, because the hash is identical either way.
+test("a run taken against uncommitted work does not count as covering the commit", () => {
+  const said = judge(passing("2026-08-31T11:00:00Z", { "mesh-lab": "aaa+uncommitted" }), now, {
+    "mesh-lab": "aaa",
+  });
+  assert.equal(said.current, false, "a run against uncommitted work was read as covering the commit");
+  assert.match(said.lines.join("\n"), /aaa\+uncommitted, now at aaa/);
+});
+
+// headOf against a real repository, because the rule lives in headOf and not in judge.
+//
+// The first test written for this marked a hand-built receipt and passed with the marking removed
+// — it checked how judge reads the value, never that anything produces it. A test that cannot fail
+// when the behaviour is deleted is not defending the behaviour.
+test("a repository with uncommitted work reports a commit that is marked as such", () => {
+  const repo = mkdtempSync(join(tmpdir(), "mesh-lab-headof-"));
+  try {
+    const git = (...args: string[]) =>
+      execFileSync("git", ["-C", repo, ...args], { stdio: ["ignore", "pipe", "ignore"] });
+    git("init", "-q");
+    git("config", "user.email", "test@example.invalid");
+    git("config", "user.name", "test");
+    writeFileSync(join(repo, "a"), "one\n");
+    git("add", "a");
+    git("commit", "-qm", "first");
+
+    const clean = headOf(repo);
+    assert.match(clean, /^[0-9a-f]+$/, `a clean tree was reported as ${clean}`);
+
+    writeFileSync(join(repo, "a"), "two\n");
+    assert.equal(headOf(repo), `${clean}+uncommitted`, "an uncommitted change was not marked");
+  } finally {
+    rmSync(repo, { recursive: true, force: true });
+  }
+});
+
+// A directory that is not a checkout is absent from the receipt, not guessed at.
+test("a directory that is not a repository reports nothing", () => {
+  const plain = mkdtempSync(join(tmpdir(), "mesh-lab-plain-"));
+  try {
+    assert.equal(headOf(plain), "");
+  } finally {
+    rmSync(plain, { recursive: true, force: true });
+  }
+});

test/log.test.ts

@@ -0,0 +1,172 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { readFileSync, mkdtempSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { around, log, logTo, shorten } from "../src/log.ts";
+
+function intoAFile(): string {
+  const path = join(mkdtempSync(join(tmpdir(), "mesh-lab-log-")), "run.log");
+  logTo(path, "debug");
+  return path;
+}
+
+function read(path: string): string {
+  try {
+    return readFileSync(path, "utf8");
+  } catch {
+    return "";
+  }
+}
+
+// Nothing at all unless asked. A suite that writes a debug log nobody reads is a suite that
+// writes a debug log nobody reads.
+test("it is off until it is turned on", () => {
+  const path = join(mkdtempSync(join(tmpdir(), "mesh-lab-log-")), "run.log");
+  logTo(null);
+  log.info("this should go nowhere");
+  assert.equal(read(path), "");
+  assert.equal(log.on(), false);
+});
+
+test("what it records says when, and how far into the run", () => {
+  const path = intoAFile();
+  log.info("a thing happened");
+  const written = read(path);
+  assert.match(written, /a thing happened/);
+  assert.match(written, /^\d{4}-\d{2}-\d{2}T/, `no timestamp:\n${written}`);
+  assert.match(written, /\d+\.\ds/, `no elapsed time:\n${written}`);
+  logTo(null);
+});
+
+test("a level below the one asked for is not written", () => {
+  const path = join(mkdtempSync(join(tmpdir(), "mesh-lab-log-")), "run.log");
+  logTo(path, "info");
+  log.info("kept");
+  log.debug("dropped");
+  log.trace("dropped");
+  const written = read(path);
+  assert.match(written, /kept/);
+  assert.doesNotMatch(written, /dropped/);
+  logTo(null);
+});
+
+// **The assertion this file exists for** (novox/hq 04-ISSUES/024). A line before and a line after
+// tells you nothing until the after arrives, which is precisely the case that matters.
+test("something still running says so while it runs", async () => {
+  const path = intoAFile();
+  await around("a slow thing", async () => {
+    await new Promise((r) => setTimeout(r, 250));
+  }, { heartbeatMs: 60 });
+  const written = read(path);
+  assert.match(written, /still running after/,
+    `nothing was said while it ran:\n${written}`);
+  assert.match(written, /→ a slow thing/);
+  assert.match(written, /← a slow thing \(0\.\ds\)/, `it did not say how long it took:\n${written}`);
+  logTo(null);
+});
+
+// A quick command should not litter the log with heartbeats it never needed.
+test("something quick says only that it happened", async () => {
+  const path = intoAFile();
+  await around("a quick thing", async () => "done", { heartbeatMs: 10_000 });
+  const written = read(path);
+  assert.doesNotMatch(written, /still running/);
+  assert.match(written, /← a quick thing/);
+  logTo(null);
+});
+
+// A failure is worth more than a success, so it survives a level that would have dropped it.
+test("a failure is recorded even when the step was not", async () => {
+  const path = join(mkdtempSync(join(tmpdir(), "mesh-lab-log-")), "run.log");
+  logTo(path, "info");
+  await assert.rejects(
+    around("a failing thing", async () => {
+      throw new Error("it did not work");
+    }, { at: "info" }),
+  );
+  const written = read(path);
+  assert.match(written, /✗ a failing thing/, `the failure was not recorded:\n${written}`);
+  assert.match(written, /it did not work/);
+  logTo(null);
+});
+
+test("what it returns is what the step returned", async () => {
+  logTo(null);
+  assert.equal(await around("a thing", async () => 42), 42);
+});
+
+// A command line is identifiable in the log even when it is very long.
+test("a long command keeps its beginning", () => {
+  const long = shorten(["incus", "exec", "machine", "--", "sh", "-c", "x".repeat(500)]);
+  assert.ok(long.length <= 200, `it is ${long.length} characters`);
+  assert.match(long, /^incus exec machine/);
+  assert.match(long, /…$/);
+});
+
+// A question that answers "no" is not a fault, and must not be logged as one.
+//
+// Plenty of the lab's commands are questions — does this network exist, is the agent up yet — and
+// they fail constantly while a scenario comes up. Recording those as faults fills a healthy run
+// with ✗ and teaches whoever reads it that ✗ means nothing.
+test("a failure the caller expects is recorded quietly", async () => {
+  const path = join(mkdtempSync(join(tmpdir(), "mesh-lab-log-")), "run.log");
+  logTo(path, "info");
+  await assert.rejects(
+    around("asking whether a thing exists", async () => {
+      throw new Error("not found");
+    }, { expectedToFail: true }),
+  );
+  assert.equal(read(path), "", `an expected answer was reported as a fault:\n${read(path)}`);
+
+  // And it is still there for anyone who turns the log up.
+  logTo(path, "debug");
+  await assert.rejects(
+    around("asking again", async () => {
+      throw new Error("not found");
+    }, { expectedToFail: true }),
+  );
+  assert.match(read(path), /· asking again/, `it was dropped entirely:\n${read(path)}`);
+  logTo(null);
+});
+
+// And a failure nobody expected is still loud at the same level.
+test("a failure the caller does not expect is recorded loudly", async () => {
+  const path = join(mkdtempSync(join(tmpdir(), "mesh-lab-log-")), "run.log");
+  logTo(path, "info");
+  await assert.rejects(
+    around("doing a thing", async () => {
+      throw new Error("it broke");
+    }),
+  );
+  assert.match(read(path), /✗ doing a thing/, `a real failure was quiet:\n${read(path)}`);
+  logTo(null);
+});
+
+// A turned-down log still shows a failure and still beats while something runs.
+//
+// **Both were lost, briefly, to an optimisation.** `around` skipped its own wrapper whenever the
+// step's level was below the configured one — which is the ordinary case at `info` — taking the
+// failure line and the heartbeat with it. The two things worth having at a low level were the two
+// that disappeared.
+test("at info, a debug step still reports failing and still beats", async () => {
+  const path = join(mkdtempSync(join(tmpdir(), "mesh-lab-log-")), "run.log");
+  logTo(path, "info");
+
+  await around("a slow debug step", async () => {
+    await new Promise((r) => setTimeout(r, 200));
+  }, { heartbeatMs: 50, at: "debug" });
+  assert.match(read(path), /still running after/, `no heartbeat at info:\n${read(path)}`);
+
+  await assert.rejects(
+    around("a failing debug step", async () => {
+      throw new Error("it broke");
+    }, { at: "debug" }),
+  );
+  assert.match(read(path), /✗ a failing debug step/, `no failure at info:\n${read(path)}`);
+
+  // And the ordinary begin/end pair is still held back, which is what the level asked for.
+  assert.doesNotMatch(read(path), /→ a slow debug step/);
+  logTo(null);
+});

test/pinning.test.ts

@@ -0,0 +1,73 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+
+import { pinnedInto, repositoryOf, stillUnpinned } from "../src/pinning.ts";
+
+const SERVED = [
+  "192.0.2.250:5000/postgres@sha256:" + "a".repeat(64),
+  "192.0.2.250:5000/mesh-provision-postgres@sha256:" + "b".repeat(64),
+  "192.0.2.250:5000/gitea/gitea@sha256:" + "c".repeat(64),
+  "192.0.2.250:5000/ghcr.io/mailu/admin@sha256:" + "d".repeat(64),
+];
+
+test("the repository is what survives being served somewhere else", () => {
+  assert.equal(repositoryOf(SERVED[0]!), "postgres");
+  assert.equal(repositoryOf(SERVED[2]!), "gitea/gitea");
+  assert.equal(repositoryOf(SERVED[3]!), "ghcr.io/mailu/admin");
+  assert.equal(repositoryOf("alpine"), "alpine");
+});
+
+// The case this exists for: an image the mesh builds has no digest until it is built, so a
+// manifest ships sixty-four zeros and would stop on the machine (novox/hq 04-ISSUES/025).
+test("a placeholder for one of our own images becomes the one this scenario serves", () => {
+  const before = `"image": "mesh-provision-postgres@sha256:${"0".repeat(64)}"`;
+  const after = pinnedInto(before, SERVED);
+  assert.match(after, /192\.0\.2\.250:5000\/mesh-provision-postgres@sha256:b{64}/);
+  assert.deepEqual(stillUnpinned(after), []);
+});
+
+// And a real third-party digest is replaced too — the text says which image, the scenario says
+// which copy of it.
+test("a real digest is redirected to this scenario's copy", () => {
+  const before = `"image": "gitea/gitea@sha256:${"f".repeat(64)}"`;
+  assert.match(pinnedInto(before, SERVED), /192\.0\.2\.250:5000\/gitea\/gitea@sha256:c{64}/);
+});
+
+test("a reference that already carries a registry is still redirected", () => {
+  const before = `"image": "docker.io/postgres@sha256:${"e".repeat(64)}"`;
+  assert.match(pinnedInto(before, SERVED), /192\.0\.2\.250:5000\/postgres@sha256:a{64}/);
+});
+
+// **Left alone, not blanked.** A repository this scenario did not stock may be reachable some
+// other way, and emptying the reference would produce the exact failure this prevents.
+test("something the scenario does not serve is untouched", () => {
+  const before = `"image": "redis@sha256:${"9".repeat(64)}"`;
+  assert.equal(pinnedInto(before, SERVED), before);
+});
+
+// A longer repository ending in a shorter one must not be half-replaced.
+test("a repository that ends in another one is not partly rewritten", () => {
+  const before = `"image": "my-postgres@sha256:${"7".repeat(64)}"`;
+  assert.equal(pinnedInto(before, SERVED), before,
+    "'my-postgres' was rewritten because it ends in 'postgres'");
+});
+
+test("every image in a whole manifest is redirected at once", () => {
+  const manifest = JSON.stringify({
+    resources: [
+      { id: "db", image: `postgres@sha256:${"1".repeat(64)}` },
+      { id: "prov", image: `mesh-provision-postgres@sha256:${"0".repeat(64)}` },
+      { id: "app", image: `gitea/gitea@sha256:${"2".repeat(64)}` },
+    ],
+  });
+  const after = pinnedInto(manifest, SERVED);
+  assert.deepEqual(stillUnpinned(after), []);
+  for (const want of ["a".repeat(64), "b".repeat(64), "c".repeat(64)]) {
+    assert.ok(after.includes(want), `missing ${want.slice(0, 6)}… in ${after}`);
+  }
+});
+
+test("what is still a placeholder can be named", () => {
+  const text = `"image": "something-of-ours@sha256:${"0".repeat(64)}"`;
+  assert.deepEqual(stillUnpinned(text), ["something-of-ours"]);
+});

test/rebuild.test.ts

@@ -0,0 +1,68 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { planned } from "../src/rebuild.ts";
+import { repositories } from "../src/repos.ts";
+
+// The control plane's image and the builder are one step, not two.
+//
+// Both parse manifests. On 2026-08-30 a rename was built into the image and not the binary, and
+// the run that found out was a full lab raise. novox/hq 04-ISSUES/005.
+test("the control plane's image and builder are always built together", () => {
+  const builds = planned({
+    MESH_LAB_MODULES: "/repo/control/examples/modules",
+    MESH_LAB_BUILDER: "/repo/control/build/mesh-builder",
+  });
+  const what = builds.map((b) => b.what);
+  assert.ok(what.includes("images"), "the images were not built");
+  assert.ok(what.includes("builder"), "the builder was not built");
+  for (const build of builds) assert.equal(build.in, "/repo/control");
+});
+
+// Every image the lab runs, not only the control plane's.
+//
+// On 2026-09-01 a run had a control-plane image built that minute and a provisioner image built
+// the day before. A test against a real database failed, and it looked exactly like the change
+// under test being wrong: the provisioner was creating logins by a naming rule that had been
+// replaced hours earlier. novox/hq 04-ISSUES/005 again, one target along.
+//
+// Named individually rather than by counting, because the failure this guards is a target that
+// exists and is not run — which a count would not notice.
+test("every image the lab runs is rebuilt, not only the control plane's", () => {
+  const builds = planned({ MESH_LAB_MODULES: "/repo/control/examples/modules" });
+  const images = builds.find((b) => b.what === "images");
+  assert.ok(images, "no image build at all");
+  for (const target of [
+    "image", "builder-image", "provisioner-image", "objectstore-image",
+    "redis-provisioner-image", "proxy-image",
+  ]) {
+    assert.ok(images.argv.includes(target), `${target} is never built, so the lab runs a stale one`);
+  }
+});
+
+// A repository this run was not pointed at is not built, and not claimed.
+test("only what this run was pointed at is built", () => {
+  assert.deepEqual(planned({}), []);
+  const hostOnly = planned({ MESH_LAB_HOST_BINARY: "/repo/host/mesh-host" });
+  assert.deepEqual(hostOnly.map((b) => b.what), ["host"]);
+  assert.equal(hostOnly[0]!.in, "/repo/host");
+});
+
+// What the receipt claims and what the run built come from one derivation.
+//
+// They are separate concerns that must agree: a receipt naming a repository the run did not build
+// is false coverage arriving by nobody's decision — just two derivations drifting apart.
+// novox/hq 04-ISSUES/005.
+test("every repository the receipt claims was built by the run", () => {
+  const env = {
+    MESH_LAB_HOST_BINARY: "/repo/host/mesh-host",
+    MESH_LAB_MODULES: "/repo/control/examples/modules",
+    MESH_LAB_BUILDER: "/repo/control/build/mesh-builder",
+  };
+  const built = new Set(planned(env).map((b) => b.in));
+  for (const [name, directory] of Object.entries(repositories(env))) {
+    // mesh-lab is the exception, and it is not an omission: it is TypeScript run from source, so
+    // the code under test *is* the code running. There is nothing to build and nothing to go stale.
+    if (name === "mesh-lab") continue;
+    assert.ok(built.has(directory), `${name} (${directory}) is claimed but never built`);
+  }
+});

test/suite.test.ts

@@ -0,0 +1,68 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { counted, reportOn } from "../src/suite.ts";
+
+// The totals come from the runner's own summary, and from nothing else.
+test("the runner's totals are read from its summary", () => {
+  const said = counted("✔ something (1ms)\nℹ tests 22\nℹ pass 22\nℹ fail 0\n");
+  assert.deepEqual(said, { passed: 22, failed: 0 });
+});
+
+// A test *named* like a total must not be mistaken for one. The summary is a line of its own, and
+// the pattern says so — otherwise a test called "pass 3" would rewrite the record.
+test("a test named like a total is not a total", () => {
+  const said = counted("✔ a machine reports pass 3 things (1ms)\nℹ pass 22\nℹ fail 0\n");
+  assert.equal(said.passed, 22, "a test name was read as the total");
+});
+
+// A failing run is read as a failing run.
+test("failures are read", () => {
+  const said = counted("ℹ pass 21\nℹ fail 1\n");
+  assert.deepEqual(said, { passed: 21, failed: 1 });
+});
+
+// **No totals is not zero failures.** A run whose result could not be read is a run nobody can say
+// anything about, and writing "0 failed" because nothing said otherwise is how a green record
+// comes to mean nothing — which is the whole of 04-ISSUES/005.
+test("output with no summary yields no totals rather than a clean bill", () => {
+  const said = counted("the runner crashed before it said anything\n");
+  assert.equal(said.passed, null);
+  assert.equal(said.failed, null);
+});
+
+// No receipt rather than a guessed one.
+//
+// The rule that keeps the record meaning something, and it was first written where nothing could
+// check it — 04-ISSUES/005 in miniature, inside the fix for it.
+test("a run whose result could not be read writes nothing", () => {
+  let wrote = false;
+  const said = reportOn({ passed: null, failed: null }, () => {
+    wrote = true;
+    return { against: {}, ran: [] };
+  });
+  assert.equal(wrote, false, "a receipt was written for a run nobody could read");
+  assert.match(said, /no receipt written/);
+});
+
+test("a run that was read is recorded, with what it was read against", () => {
+  let got: [number, number] | null = null;
+  const said = reportOn({ passed: 22, failed: 0 }, (p, f) => {
+    got = [p, f];
+    return { against: { "mesh-lab": "abc1234" }, ran: ["test/integration/mesh.test.ts"] };
+  });
+  assert.deepEqual(got, [22, 0]);
+  assert.match(said, /mesh\.test\.ts — 22 passed, 0 failed, against mesh-lab abc1234/);
+});
+
+// The runner's real output, colours and all.
+//
+// Captured from `node --test` writing into a pipe rather than written by hand: the first version of
+// counted() passed every test and read nothing, because the fixtures were clean text and the runner
+// emits escape codes. A fixture that agrees with the mistake proves the mistake.
+test("the runner's totals are read from output as it actually arrives", () => {
+  const real = "\u001b[34m\u2139 suites 0\u001b[39m\n" +
+    "\u001b[34m\u2139 pass 22\u001b[39m\n" +
+    "\u001b[34m\u2139 fail 0\u001b[39m\n" +
+    "\u001b[34m\u2139 duration_ms 98.9\u001b[39m\n";
+  assert.deepEqual(counted(real), { passed: 22, failed: 0 });
+});

test/supported.test.ts

@@ -84,3 +84,13 @@ segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
 machines: { a: { at: { segment: net, address: [192.0.2.1] }, inbound: allow } }`);
   assert.doesNotThrow(() => assertSupported(scenario));
 });
+
+test("egress is parsed, and off unless asked for", () => {
+  const scenario = parseScenario(`scenario: x
+segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
+machines:
+  a: { at: { segment: net, address: [192.0.2.1] }, egress: true }
+  b: { at: { segment: net, address: [192.0.2.2] } }`);
+  assert.equal(scenario.machines["a"]?.egress, true);
+  assert.equal(scenario.machines["b"]?.egress, undefined);
+});

test/validate.test.ts

@@ -243,3 +243,17 @@ machines:
     /one gateway.*disagree.*forwardable/s,
   );
 });
+
+test("a detached machine cannot declare egress", () => {
+  refuses(`scenario: x
+segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
+machines: { a: { at: detached, egress: true } }`,
+    /detached but declares egress/);
+});
+
+test("a segment may not be named 'uplink' — the lab claims that name for egress", () => {
+  refuses(`scenario: x
+segments: { uplink: { kind: public, cidr: [192.0.2.0/24] } }
+machines: { a: { at: { segment: uplink, address: [192.0.2.1] }, egress: true } }`,
+    /reserved for the lab's own NAT bridge/);
+});

test/warm.test.ts

@@ -0,0 +1,61 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { judge, type Warm } from "../src/warm.ts";
+
+const at = "2026-08-31T20:00:00Z";
+const built = { "mesh-lab": "aaa", "mesh-host": "bbb", "mesh-control": "ccc" };
+const warm = (over: Partial<Warm> = {}): Warm =>
+  ({ scenario: "two-nodes", instanceId: "mlab-two-nodes-1", images: [], against: built, at, ...over });
+
+// The check this exists for: a mesh warmed against code that has since moved would pass today's
+// tests against yesterday's binaries, and the result would say nothing about it.
+//
+// Same fault as novox/hq 04-ISSUES/005, one level down — a green result standing for a run
+// against something other than what is in front of you.
+test("a warm mesh built from code that has moved is refused", () => {
+  const said = judge(warm(), "two-nodes", ["mlab-two-nodes-1"], true,
+    { ...built, "mesh-host": "moved" });
+  assert.equal(said.use, "raise");
+  assert.match(said.use === "raise" ? said.why : "", /mesh-host was at bbb.*now at moved/);
+});
+
+test("a warm mesh built from the same code is used", () => {
+  const said = judge(warm(), "two-nodes", ["mlab-two-nodes-1"], true, built);
+  assert.equal(said.use, "restore");
+});
+
+// Each refusal is named, because each is a different thing being wrong.
+test("every reason to raise instead says which reason it was", () => {
+  const cases: [string, ReturnType<typeof judge>][] = [
+    ["nothing kept", judge(null, "two-nodes", [], true, built)],
+    ["another scenario", judge(warm({ scenario: "first-node" }), "two-nodes",
+      ["mlab-two-nodes-1"], true, built)],
+    ["not standing", judge(warm(), "two-nodes", [], true, built)],
+    ["no snapshot", judge(warm(), "two-nodes", ["mlab-two-nodes-1"], false, built)],
+  ];
+  for (const [what, said] of cases) {
+    assert.equal(said.use, "raise", what);
+    assert.ok(said.use === "raise" && said.why.length > 10,
+      `${what} was refused without saying why: ${JSON.stringify(said)}`);
+  }
+});
+
+// A repository the warm record never accounted for is a difference, not a match.
+test("a repository that was not recorded when it was warmed is refused", () => {
+  const said = judge(warm({ against: { "mesh-lab": "aaa" } }), "two-nodes",
+    ["mlab-two-nodes-1"], true, built);
+  assert.equal(said.use, "raise");
+});
+
+// A repository the current run cannot see is not a repository that agrees.
+//
+// **Found by testing the guard rather than trusting it.** The first version walked only the
+// repositories the current environment names, so running without that environment compared
+// nothing and called a stale mesh usable. The mesh had genuinely moved; the check had looked at
+// neither side.
+test("a repository this run cannot locate is refused, not passed over", () => {
+  const said = judge(warm(), "two-nodes", ["mlab-two-nodes-1"], true, { "mesh-lab": "aaa" });
+  assert.equal(said.use, "raise");
+  assert.match(said.use === "raise" ? said.why : "",
+    /nothing says where it is now|so nothing can say whether it moved/);
+});