mesh/check failed (2): mesh-delivery-checker: no published of the tagged version of go-meshconditions of me…; and 1 more | 5caa760f5eaa on fd023f7473bc
The apply gives each process the declaration's For as MESH_NODE beside its own env, so a process that reads MESH_NODE (mesh-sdk, the zfs follower's writer and its node-service-manager calls) names its node.
Builds: design 47 G A-342
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
The whole test's web manifest keeps G's entry form (no build artifact, the entry read by fingerprint) and takes main's resources under if-config and config-lines.
Builds: design 47 G §421; design 42 zfs-owns-zfs H1-H5
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
mesh/check being carried to 96a869063977 on f8acfaeed79d | from 0193949efe9f, mesh-catalog 117f933dacf0, mesh-delivery 7221a1e5cd7f, mesh-sdk e42b9491…
A size of a million MiB or more reached the file in exponent form and was refused.
Builds: design 42 zfs-owns-zfs §3, A-24
A key a layer holds that the manifest does not declare (scrub-pool during a rename) is not a config value of the module.
Builds: design 42 zfs-owns-zfs §9, A-15, A-24, A-25
The existing node-engine integration test's module gains a file under if-config,
a template with config-lines from a list, and two files that fail and are left
under a value; the test asserts the file appears and goes with its value, the
lines follow the list, and the failing resource fails the module with the later
resource left. No integration test is added.
Builds: design 42 zfs-owns-zfs §5 (and §1 to §3), A-22
The node record's reads come from mesh-sdk's ManifestReads, so building with S1
and S2 states word:resources.if-config, form:config-lines and form:config-json;
the record test now asserts them.
Builds: design 42 zfs-owns-zfs §4, answer A-13
`${config-lines:<key>:<prefix>[:mib-bytes]}` in a file's content writes one line
per entry (nothing for an unset value or empty list; a newline in an entry
refuses the composition); `${config-json}` writes the module's set config values
as one JSON object. Both join the sweep's table of fillers. Their rows join the
config table test (A-22).
Builds: design 42 zfs-owns-zfs §3, §9; answers A-15, A-24, A-22
When one record of a unit goes while another still declares it, what the going
record found of the unit is kept for the staying one when that has no finding of
its own, so the module's leave gives back the state found before the mesh. The
preview test and the new case are one test (A-22).
Builds: design 42 zfs-owns-zfs §2, answers A-21, A-22
The composer leaves out each resource whose `if-config` holds none of its
conditions against the module's config values (with defaults), and drops the key
from the rest, as the node-engine parses strictly. A resource that leaves is
removed by the existing orphan removal. One table test replaces the two tests of
setting_into_test.go.
Builds: design 42 zfs-owns-zfs §2, A-21 (declaration side), A-22
A failed resource of a module gates the rest of that module, which is reported
skipped with the failed resource named; the module's result names each one left.
A resource of no module, and other modules, go on as before. One table test
replaces two in apply_test.go; two fixtures adapted to the rule.
Builds: design 42 zfs-owns-zfs §1 (ADR 0679 §16 (a)), A-22
The composer's Build is made from the catalog half's entry it holds and the parts of the want that names it (buildOf); the
inputs of the indexer's modules and the builder's builds, takeBuildDone, takeModule and the builder read are gone, and mesh-host
names no BuilderBuildsBucket. The shadow takes the entries of the builds it composes from (askBuildsEntries), builds a module a
candidate adds from its entry and the candidate's `added` parts (A-316), and holds, with the reason, a module whose entry
has no part given, whose fingerprint the verb module does not give, or for which it errors (A-312). Its composer is given the
entry call as the node's own is. Rows: a built module added with and without parts, the verb module answering no fingerprint
and erroring.
Builds: design 47 G §421, §628, A-308, A-312, A-316
mesh/check checking 8dc186944e7d on 0193949efe9f | novox/mesh-host#225 in group fix/1748-resolver-wait
TestTheCataloguesResolverComposesOnAMachine takes the options line from the systemd-resolved template beside it instead of the literal timeout:1 attempts:2 edns0, so it passes against the catalogue's main before and after the catalogue's change. Captured copies are left as they are.
Builds: design 50 resolver-wait §3 (and §2 of the main file, line 73)
Reading the catalog half by fingerprint, the node-engine no longer reads the builder's builds, so #224's refused key was
never written there and a reader waited on a build that will never be done. The engine now hears the builder's
build.ended stream, read whole at start then followed (as the assigner hears it); the last trunk end of a placed module
that ended failed is written as the refused key with the event's words, a later done end deletes it, an end naming a
branch moves nothing. The manifest consumes mesh-delivery-builder.build.ended.**.
Builds: design 47 G A-293 (with §421, §628), ADR 0593 §6, issue 1966 U1/U8
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
Brings mesh-host #224 (issue 1966): newestFPs returns a placed module whose
newest build failed, composedValues writes its refused key at that fingerprint,
takeBuildDone starts the pass on a failed build. Conflict in newestFPs settled
by keeping G's by-entries branch first (design 47 G §421, §628: the node reads
neither the indexer's modules nor the builder's builds; it returns no failed),
and main's failed-build path in the other.
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
After a restart, g14's engine restated its reads three seconds after its
first, bootstrapped statement, and the assigner never heard that event: a
say that failed past the bus's retry bound waited for the next link. Now a
say the bus did not take (no responders, a deadline) is said again on the
link until the bus takes it, off the caller, each failed attempt written to
the engine's journal with the event's name; only a lost link, or a refusal
the bus answered, keeps it for the next link.
Builds: design 47 G §590, A-292
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
mesh/check checking 8e993c759a5c on df1412081bed | novox/mesh-host#224 in group fix/1966-unpublished-placed-module
The node-engine writes a refused key, with the build's own error, at the indexer's
fingerprint in composed-<node> for a placed module whose newest build the builder
holds failed; a build queued or running stays unpublished (U1). A failed build of a
placed module's newest fingerprint also moves the pass (takeBuildDone), so the key is
written when it lands (U8).
Builds: design 47 fix-unpublished-placed-module U1, U8, ADR 0593 §6
H-17 made discovery read the node-engine's own name (§527) instead of the constant mesh-host,
but the integration test still gave its link no own name and expected mesh-host, so every
endpoint announced an empty module. It now gives the link an own name and expects that one.
Builds: design 47 G §634, A-149
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
A want with `revert` `live` starts the reverted build with no restore, no aside taken and no catch-up, and
applies as any want; without the field a revert with no restore point stays held.
Builds: design 47 G §710 (G-21-provider-and-revert-ask.md, "On the answer"), §300, §567
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
The node-engine never reads the grants part's state: it asks the seat's verb
withheld in its own name, and says withheld unknown when nothing answers.
Builds: design 46 7 §6 (the composition reads them through the verb withheld; the node-engine's manifest gains mesh-grants in uses; "withheld unknown: the grants part does not answer"), lead answer A-239
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
The grants part decides a node's withheld subjects from the runner's own
reading tied to the runner that stated it; the merge into capabilities stays
for placement.
Builds: design 46 7 §6 (its node's node-engine takes both into the node's record), lead answer A-203 (runner-capabilities, runner-fingerprint)
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
Each finding's id carries its token as a ref (A-14), so machine.<node>.network, .rewritten, .unreachable, .units and module.<module>.<node>.unhealthy and .kept
read as before.
Builds: design 47 G A-197
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
The have's failed phase (apply-failing), unhealthy health (module-unhealthy) and kept words (kept-as-it-runs), and each part of
its health report (machine-network, network-rewritten, network-unreachable, machine-units, relogin-needed,
directory-used-as-found, node-account-fallback), raised and cleared by the held writer where the engine knows the fact.
Builds: design 47 G A-197 (ADR 0596, A-192, A-149)
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
The composer answers the assigner's composition step: the request is read from the file
named by MESH_RUN_INPUT and the verdict is the last line of standard output, exit 0; the
compose-job path stays.
Builds: design 47 G §354, §63, §22; lead answer A-167
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
The delivery repository's contracts and SDK live under modules/ as
mesh-delivery-contracts and mesh-delivery-sdk, their Go module paths
renamed to match the folders (the TypeScript names already did), and
every importer in mesh-delivery, mesh-catalog, mesh-host, mesh-tools
and mesh-sdk follows in the same group: a design as it should be, one
path per module, checked by the group's check building every importer.
Builds: design 47 G §427, §88
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
The long-form invokes entry is read by mesh-sdk's second reader release (09d6070e): packaging's manifest
tests pass with it placed and fail before it is joined.
Builds: design 47 G §558, §617, §633, §635
The codec moved from the mesh-delivery contracts into mesh-delivery's domain SDK (sdk/catalogkey), still a pure package
of the standard library and mesh-sdk; the command's import rule named the old place only.
Builds: design 47 G A-144
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
The manifest declares the file recovery-wait (content ${config:recovery-wait-minutes}); the node-engine reads it at its start and
does not call stdio.Config, which needs the runtime that launches a bundle.
Builds: design 47 G §633, A-112
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
Before a move across a kept data's form change the node agent asks snapshot-kept and waits for the ended of its own act; at a revert it stops the newer build (Env.Stop), asks kept-points, asks restore-kept with the snapshot of the point of the form and fingerprint it reverts from, waits for restored, applies the reverted build, and calls the point's catch-up as the module's tool through the tool runner. The wait is in the have (ask-at, act), bounded by a bounds key of the state bounds (recovery-wait-minutes, 90), and each ending is read as the table of 560 says. The manifest consumes mesh-recovery's ended and restored and invokes its three verbs.
Builds: design 47 G §296, §297, §554, §555, §560, §630, §631, §632, §633, A-77, A-92, A-98, A-108
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
The node's own composer reads the catalog half through meshcatalogclient's Entries cache over its own calls of
mesh-catalog's verb entry, and no longer watches or reads the indexer's modules, manifests and verdicts or the
builder's builds: a module's manifest is its entry's, each artifact's reference is made from the digest its want's
part names (artifact-store://<module>/<artifact>/blobs/<digest>, A-50). A want whose entry is asked or not found is
held, its have the phase held with a detail opening entry-asked: or entry-missing: naming the fingerprint. The
entries and wants it composes from are kept in composed-<node> (entry.<digits> and want.<module>), written before
the composition that uses them, read at start before anything is asked, a kept want standing until the wants'
live read answers, and dropped when nothing names them. The shadow keeps its reads (G §628).
The node grant gains the verb entry and the running stream's consumer calls; OverNATS hears the running stream
as the link's session does.
Builds: design 47 G §421, §683, §614, §628, A-47, A-50
A want that names a node in kept-on, for an artifact its parts name, is fetched through kept-artifact of that node's build seat and
read as an object on the bus, checked against the digest; with no kept-on it is fetched from the store, and the builder's builds
are read no more (the apply hook, the fetch and the object read from bc8fbc7, its node and fingerprint from the want).
Builds: design 47 G §271, §526, §421 (A-50)
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
first-views answers {first-views, entries, wants}, entry-again asks one entry again as a recorded hand act and
answers {asked, read}; first-view-again names no view the engine follows (it follows no branch) and refuses in
words. module.json's claim of node-engine lists every verb the engine serves.
Builds: design 47 G §422, §698
The node's own composer reads the catalog half through meshcatalogclient's Entries cache over its own calls of
mesh-catalog's verb entry, and no longer watches or reads the indexer's modules, manifests and verdicts or the
builder's builds: a module's manifest is its entry's, each artifact's reference is made from the digest its want's
part names (artifact-store://<module>/<artifact>/blobs/<digest>, A-50). A want whose entry is asked or not found is
held, its have the phase held with a detail opening entry-asked: or entry-missing: naming the fingerprint. The
entries and wants it composes from are kept in composed-<node> (entry.<digits> and want.<module>), written before
the composition that uses them, read at start before anything is asked, a kept want standing until the wants'
live read answers, and dropped when nothing names them. The shadow keeps its reads (G §628).
The node grant gains the verb entry and the running stream's consumer calls; OverNATS hears the running stream
as the link's session does.
Builds: design 47 G §421, §683, §614, §628, A-47, A-50
The seat list's five-minute finding and the finding of a held module are raised as they happen, so they use the held
writer (seeded at Start, Held() answers what stands) over the contract of the seat mesh-conditions; the keepers keep no
set or sweep of their own, and the writer is started before the seat list reader writes or clears any. The engine's own
finding stream stays its foundation stream mesh-host-findings.
Builds: design 47 G §529, A-29, A-30, A-37, A-38, A-40
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
The join with mesh-contracts' K6 (7d730a4a) removes the event and its body; H-3's second source of the same fact goes (haveState.capsSaid, capabilitiesToSay, sayCapabilities), the test asserts none is said.
Builds: design 47 G §590
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
A want with parts is applied only when every fetched archive, process and container
digest is among the parts and every part is a fetched digest; otherwise the module
keeps running what it has and its have is held with detail parts-differ naming the
digests. A want with no parts is applied as before.
Builds: design 47 G §524, §379
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
A finding is written in the node's state findings and followed by finding-written or finding-cleared on the engine's own
stream mesh-host-findings; the engine states every finding it observes at its start (the seat list's finding only while the
reader waits for the list, else not) and when the keeper asks, and answers an ask by a new statement. The node's grant in
the golden file gains the stream's publish, the ask stream's consumer, and the whole of its state findings.
Builds: design 47 G §383, §384, §390, §461, §485, §503
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
The module is the one whose catalog entry claims node-engine, else the one this node heard run the build its own
fingerprint names; no name is fixed in the guard or in the compose.
Builds: design 47 G §364, §527, §428
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
The node follows the node-engine seat's running events (an ordered read of the last message of each subject, then a durable
consumer), keeps what each node runs by `at`, acts on none until the first read ended, and hands the composer each node's
heard build with the manifest read by the event's digest; compose resolves the passes through catalogue.SeatsToPassOn
(heard, entries) with the trunk's entry for a node that said nothing.
Builds: design 47 G §199, §249, §459, §536, §556
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
mesh-grants gives the engine the owner's read of its state findings for its own node's keys (EngineFindingsGrants); the golden
grant held only the write, so the engine could not read the finding it clears at the first full read.
Builds: design 47 H-7, ADR 0636 §1 (the finding is cleared at the first full read), ADR 0596 §2 (the owner reads its state), issue 1966
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
A new node-engine process cleared nothing at its first full read of the seat list, because the Waiter remembered in memory whether
it had written the finding; ace and g14 kept the condition for ever. The Waiter now asks the node's state findings whether the
finding stands, clears it at the first full read and writes it at the bound only when it does not stand.
Builds: design 47 H-7, ADR 0636 §1 (the finding is cleared at the first full read), issue 1966
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
mesh/check checking d581acdedae3 on 783a2d22073f | novox/mesh-host#222 in group fix/1966-call-made-once
Since mesh-catalog 6b8e7fc6 the build modules ask the capability seat-settle-delay of the machine, so on
the stand-in builder without it mesh-build-go and mesh-build-node were left out as unhostable and both
toolchain tests failed against the catalogue's main, on mesh-host's trunk as on this group's head.
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
mesh/delivery-group group fix/1966-call-made-once rejected: composed together: the checker's verdict at the members' heads (c54b617d200d): mesh-delivery-checker
mesh/check failed: mesh-delivery-checker: no published of the tagged version of go-meshconditions of mesh-health-and-re… | 9cd03a7430cc on f50442ac7a2d
A failed say was only a line in the engine's log; now it is said again when a link opens, beside the
statement made while no link was open (proved by a step of the node key test, with H1's publish).
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
One publish of manifest-reads-changed with a 10 s deadline was the likely loss behind note 155; the
stream takes the id once, so publishing again is safe.
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
mesh/check checking 5c353cfc7bf0 on f50442ac7a2d | novox/mesh-host#221 in group fix/1966-seat-list-unread
The node-engine seat's definition names "the operator and the healers" for
both seat-list verbs, and design 47 G keeps it; the read was opened to every
proved caller on a ruling the review withdrew. node-tools stays on both: every
console call is proved as its machine's tool runner (issue 627, issue 1966).
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
The live case of ace, in process against a real bus: the stand-in of the seat
mesh-catalog answers as nothing serves it (no responders) while the node's
reader first asks, then serves; the same reader reads the list within the
bus's retry bound, with no event of the seat and no hand act (issue 1966,
ADR 0636 §8). The test of an unread list now reads the reason that names the
bootstrapped seats.
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
On ace, a node-engine left on its bootstrapped seat list could not find the
artifact store's seat, refused avahi's archive reference and with it the whole
pass, so the seat-list hold never ran and every module said "reaches no
artifact store". While the list is unread, a placed module whose seats are not
all bootstrapped is now held as it runs before anything of its build is read
(frombus.Scope.Held), its reason names the bootstrapped seats the node holds,
and a pass refused whole says the unread list first (issue 1966).
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
seat-list-again answered person.<name> and the healers, but every call through
the mesh MCP server is proved as its machine's tool runner (issue 627), so no
operator's session could make a node read its seat list again. node-tools now
stands for that session; the read seat-list is open to every proved caller, as
ADR 0636 §7 limits only the read again. The verb names come from the contract.
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
A request that reached no one is asked again by mesh-sdk's busretry only when
it can see the bus's own error; the node-engine's ask dropped it, so the seat
list's first read, which fell into its holder's re-serve, was never asked
again (issue 1966).
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
The tool runner's unit carries the paths modules own, for the holder of node-files. The engine's own
archive is unpacked into a place named for its build, so every engine replacement changed that word
and restarted the tool runner with nothing of it changed (novox 21:16, 22:25, 22:38 on 2026-11-09).
A resource unpacked into a place named for its build now owns the directory holding every build.
The restart line now names the unit's directive, word and key that differ, never a value.
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
mesh/check being carried to ef41fa2e2796 on 6e461bd24aa5 | from 7c483da93a03
compose.CompositionsHolding returns the modules the declaration leaves out for a claim that cannot be held or a refused
credential; frombus composeEach holds each with those words, so the node-engine keeps what it handed of the module (no
stop of a running module), the candidate answer and the check's sandbox name it refused, and the rest composes.
CredentialsNeeded asks nothing for a held module. Resolve and checkClaims move to a test helper (production uses
ResolveHolding and holdClaims); a pass refused anyway names a held module's reason among the others.
Quick fix: Declared.Held read in the composer only. Thorough, chosen: the same held words reach every reader of the
composition (selfcompose, ComposeSent, the candidate answer) through composeEach's one hold path.
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
The node-engine refused the whole pass when two modules claimed a seat held once, or when a consumer's credential was
refused. Now the composer leaves that module out of the declaration and composes the rest: the module is returned held
(Resolved.Held, resolution.ResolveHolding), reaches the declaration as left out through the filter and the declaration
#218 already keep (Rendering.NotPlaced), and Declared.LeftOut names it. The second claimant of a seat is the one held;
its claim is not this node's holding. The rest-of-mesh view and the network use the same, so a node with a held module
stays on the network. Resolve stays strict for callers that cannot leave a module out.
Quick fix: hold only the credential refusal (resolve.go planForNow), about 20 minutes. Thorough fix, chosen: every
claim problem is attributed to its module (claimsOf) and both kinds go through the one left-out path of ADR 0163 rule 6,
in the declaration, the filter and the network, so no per-kind refusal of a pass remains in the composer. Left for the
next step: a declared-and-not-made own secret (NotMadeError) and the Foreseeing-mode refusal still refuse the pass; the
frombus candidate path holds those by stub already.
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
Rules skipped a left-out module's expose and reaches, so its listens
opened at the manifest's default source and an operator's narrowing for
the node was dropped (ADR 0046). Its exposure is now read like any
module's (exposureOf); where it cannot be read for that module, its
listens open no wider than the mesh (noWiderThanTheMesh), never at the
manifest's default.
Left for the thorough fix, composing a left-out module as what runs of
it (Carried, as a held module is): the filter also opens for a left-out
module that never ran on the node, and it opens the new manifest's
listens rather than what the old composition runs.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
A module left out of the declaration for a setting its new definition no
longer composes (ADR 0163 rule 6) keeps running what it ran, but
resolutionCompose dropped it from the modules the filter is derived from,
so the filter closed the ports it still serves. On novox, 2026-11-09
07:46-09:24 UTC, the artifact store's want at build 6e1dfaa8 (listens on
5100, no container publishes 5000) met the node layer's ports
{"5000": 5100}: GivenPorts refused it, distribution was left out, and
5100 closed while the old registry container served there; every build
failed "dial tcp 10.10.0.1:5100: i/o timeout".
The filter is now derived from the composed modules and the left-out
ones; a left-out module's settings (exposure) are still not read.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
mesh/check checking e504c129401b on 20191223794a | novox/mesh-host#217 in group fix/1966-forge-made-merge
Merges made on the forge directly since 2026-11-09 07:04 UTC have no gate merge record,
and the indexer holds them; an empty merge through the gate covers this repository's head.
Second of its kind: the group's mesh-delivery member removes the need (the forge bridge
reads a forge-made merge's pull request from the forge).
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
The said-once keys no longer hold where the indexer's picture stands; a module composed from the manifest for want of a done
build is named in Pass.Unmade; the agent's wait is told once per want and again after the composition was there; the two tests
are folded into existing ones.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
Every pass names each wanted module it made no composition of, with why (Pass.Unmade, the verb compose, one journal line per
change); a want not taken is said; the agent says once per want that it waits for the composer and why, and tells it, so a
composition the composer believed handed is handed again.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
On 2026-11-08 a failed apply of distribution's new build had already replaced
its store container, yet the node-engine still held the old build as running,
so the put back to it at 23:47 UTC was judged unchanged and applied nothing.
A failed apply now marks what runs of the module failed, and a want of the
composition that ran before is applied again.
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
On 2026-11-08 23:25 UTC the node-engine replaced distribution's store container
and only then fetched distribution's program from the store it had just cut off:
the module was left half applied and the artifact store unreachable from every
other machine. An apply now fetches and checks every archive and process first,
as it already pulls a module's images first; one that cannot be had gates its
module, so none of its resources is replaced and none of its orphans removed.
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
mesh/check checking 71508cc96297 on 25aa110920cd | novox/mesh-host#214 in group fix/1966-gate-record-2
mesh-host #213 (87bdc95a) was merged on the forge under recovery-fixes-merge-and-tell and has no gate merge record; the indexer holds its take-in. An empty merge through the gate covers it, and the forge-made wait is dropped (indexer_drop_pending).
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
mesh/check pass 7b42c94a4d78 on 87bdc95a3d65 | unit 1679 run, 333 kept, 31 skipped | integration 1692 run, 333 kept, 26 skipped | over bound: unit (sl…
The link serves the seat node-engine when it opens, before the node agent
starts the reader of the seat list; since ADR 0635 catalogue holds no seat
until then, so the engine registered no verb at all. Nobody answered
manifest-reads, compose, composition or the hand acts on novox, shanks and
now ace, and the assigner kept ace's old statement, holding its own
successor on seats the new engine states.
Quick and thorough fix are one here: the only seat-list read at start is
this one (the composer waits for the list, the reads statement restates).
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
mesh/check failed (2): mesh-delivery-checker: no published of the tagged version of go-sdk of mesh-delivery…; and 1 more | 9ab85ef92e6c on 875b4ee5ce38
manifestReads froze the statement at its first call, before the engine gave
catalogue even its bootstrapped seats (ADR 0635 empties mesh-sdk's table
until the list is read), so ace's engine stated no seat and its own next
build, naming mesh-catalog, was held on it.
The words and forms stay the build's, computed once; the seats are those of
the list held now. The reader of the seat list restates after each change:
the node record carries the new statement at once and the node seat's event
manifest-reads-changed says it (ADR 0618 §4); an unchanged statement writes
and says nothing; the verb manifest-reads answers when it was made.
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
The review of fe9533c1: a new test function raised a count already over the
budget, so two broker-pin tests are one; KVWatch no longer yields every
change, and its doc says so (issue 1966).
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
Live fault 2026-11-08: node-engines on two machines logged only "nats: slow consumer, messages dropped" and applied nothing. The
client's watch sends into its channel from inside the subscription's callback; KVWatch forwarded that channel to its reader
blocking, and the reader (composing for minutes) stopped draining, so the subscription's pending limit overflowed.
Quick fix: a larger buffer on the watch's channel; it only moves the limit. Thorough fix (chosen): the forwarder always drains the
client's channel and, while its reader is busy, coalesces the pending updates by key (newest value and revision, first-seen
order, Caught kept in its place), so no reader's pace can back the client up, whatever the burst. A slow consumer error now
names its subscription and what it holds pending.
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
mesh/check checking cb7a444a5634 on 70e70eda5a24 | novox/mesh-host#209 in group feat/registry-milestone
When a node refused also with the modules composed without their build held as its node-engine holds them, the
verdict kept the words of the composition from their unbuilt manifests, whose refusals the lift itself made (a tool
runner with no bundle, a resource whose artifact or template no build resolved). In the group check of c5d32074c945
that hid lab's real refusal on shanks among them. The node's refusal is now that of the held composition, named so.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
mesh/check checking 53addd73c358 on e73aae3686a6 | novox/mesh-host#209 in group feat/registry-milestone
The running indexer could not read 17 of the group's manifests, so the running node-engine composed those modules at
their trunk builds and said nothing; the head composer laid those builds over the change's other manifests and ace
failed on "nothing provides artifact-store, wanted by mesh-build-go" (group check 8a80e8e86858). The input a node
answers a check with now names, per module taken in from its build, the content digest of the manifest that build was
made of (ComposedHow.Manifests); the composer compares it with the case's manifest, and, for an input from a
node-engine before it, compares what the two manifests say beyond what a take-in resolves (the lead's ruling, §17).
Each module lifted either way is composed from the case's manifest, its build awaited, and named in NotJudged with
the comparison that lifted it; one that cannot compose without its build is held as its stub. The digest is taken
over one JSON form because a request through a build module arrives compacted.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
The provider held on its machine is composed apart, so no module of the composed set serves the provision; its
manifest's served values reached the consumer's binding raw and the binding refused the consumer for a
${bound:route:name-web} it does not require (the group check of 8a80e8e86858 on novox, mesh-build-node beside a held
gitea). Each such key is now left out and said on the need, as a provider elsewhere leaves it out (ADR 0346).
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
mesh/check failed (4): unit: git.novox.be/novox/mesh-catalog/checks/gitignored.TestNoProgramABuildLeavesIsC…; and 3 more | 5b75e866d079 on e73aae3686a6
The node's own composition named a held module's why only; it now also says whether its manifest's verdict or this
node-engine's decoder holds it, and at which build, as the input a check's composer reads says it. Said in the held
words, so the node-engine seat's answer keeps its shape.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
ADR 0578 §2 and §3: a case's accepted verdict, or none (the running checker drops the change's own), lifts the
input's hold on a manifest the composer reads whole; a case's refusal and bytes that do not decode stay held. It is
composed at the build the input carries (Input.Holds). Without that build the node is not judged and the verdict
names the module, why it was held and that its build is awaited; where it cannot compose without it, it is composed
held as its node-engine holds it, and said so. Never a silent pass (the lead's Q4 ruling). The issue 774 row now
gives the runner the refused verdict the checker gives a manifest the change does not bring.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
A module the running indexer refused stood in the input as its stub with no build, so a check's composer that
judges the change's manifest itself had nothing to compose it at. Each held module now says whether its verdict
or the decoder holds it; for a check's answer the node also takes in the build it would compose it at, when the
builder holds it done, beside the stub and never into what the node composes.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
Eight lists of seat verbs (checker, enrolment, artifact sweep, delivery, node-resolver, node-nfs-server,
node-mounts, mesh-config's input) and their schema helpers were read by nothing but one test, and drifted
from their seats (B0's verb input never reached the checker's copy). A guard row in the seats test finds
any list of the catalogue's verbs written in the node-engine's code.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
The group try of 2026-11-06 10:07 failed its composition on all four
machines because mesh-compose, the node-engine's composer a check builds
at the head, loaded no seat list once the compiled one was deleted (ADR
0635 §1): the uplink's holder no longer stepped back from the resolver
file, and no holder of the registry seats was known. It now takes the
request's seats over the foundation's bootstrapped ones, as the
node-engine takes the bus's; without any it says each machine not judged
(ADR 0636 §1, ADR 0593 §9), never a refusal or pass made without them.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
A case's verdicts are the checker's account of the indexer's modules values, not the indexer's kept values: the
trunk's checker writes them with version 0 and no judge, so the kept value's strict reader refused every one
and the group try failed composition on all four nodes ("the verdict of f1-circuits is not one: ... version 0").
The composer now takes version 0 (or none) and keys.Version, its judge optional, and still refuses a later
version, a value that does not decode and an unknown state. The bus reader of kept verdicts stays strict.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
The marks of the haves (a refusal, the words a module keeps) read and write each module's have in turn, and each write
walks every have: on a slow bus the pass waited minutes on them (shanks, 2026-11-05 22:22). They now run off the pass,
newest word per kind, each bounded; an ask of compose during a pass is answered at once and queues one more pass.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
The state, events and types of the seat mesh-module-assigner say assignments; the legacy node key form is gone from the
reads; the dead seat-verb list of the retired name is deleted. The placement guard and the placed-file sense stay.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
A member's go.mod names the oldest version a module accepts; the check builds the group's own heads whatever it names. go.sum lines of mesh packages are left out.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
The engine's Composed keeps the catalogue's Plan and the declaration types and goes through nodeengineclient's codec; the fingerprint key part is meshcatalogclient's. The candidate answer is nodeengine.CandidateAnswer; the have still holds it through an interim JSON bridge (link.interimAnswers) until the Have group moves.
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
While the indexer holds distribution's new manifest, every node reads main's,
which claims only mesh-artifact-store: a new engine must find the store there.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
The provision artifact-store retires, so the store is found by the holder of
container-image-registry and the listen it opens to the mesh, on the port its
node put it on; every name of the store, oci-artifact-registry's too, answers
that address. The engine states form:seat-reach with its contract.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
The group's mesh-build-go requires go-package-registry, which the forge
now provides; the stand-in listed the forge's offers by hand and missed it,
so the test failed against the group's catalogue. It now takes them from
the catalogue's gitea manifest, so a registry the forge gains needs no edit.
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
mesh/check failed (9): ace: novox/mesh-sdk#167 does not merge into the trunk: files in conflict: go/catalog…; and 8 more | 32c5df952cbf on e73aae3686a6
The store's front is distribution's program, a bundle the mesh builds, so nothing serves it before the builder: the
bootstrap's /v2/ wait and its pushes now go to the loopback port the store's manifest publishes, as the publish contract
allows on the store's own node. The one-off credential and its file go: the runtime could not carry it on a push, and no
push needs it. The tags list is asked for the repository being published, not always the control plane's.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
waitIdle returns once every change sent to the loop is taken in and applied and no backoff wait runs, so a retry the
clock or a wrong settled would start is counted, with no real sleep.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
The owner is the account the manifest names for the directory that holds the file; the bootstrap no longer makes that
directory itself, and a run that reaches it first is refused naming what it waits on.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
The image, the seat and its holder, when the pull began, and the last retry with what triggered it ride in the failed
have's pull-wait (mesh-sdk delivery.Have.PullWait, asked of the lead in j1-sdk-requests.patch), from the agent through
the link's module fact to the have bucket; the agent's own read view of them goes with the verb. Rows in the agent's
pull test and the have bucket's test; each failed without the code.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
ADR 0627 §1, §4: mesh-sdk keeps no asks, asker, vault answers or compose-candidate checks, so the node-engine takes them
from mesh-contracts: operator-channel's contract and client (asks, warrants, the asker's client), mesh-vault's sign verb
and answer, and node-engine's own contract and client for compose-candidate's ask, answer, verdicts and bound. The
lead's ruling on pull-waits (no record gives the seat that verb): the verb and its handler go; the row that read the
waits now holds the seat refusing the verb, and failed before the change. The go.mod requires of mesh-contracts' modules
come with the group's pin.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
mesh-sdk's ComposeCandidateAnswer and ModuleComposed are now the generated nodeengine types (ADR 0613 §3, §5), so the
input sent for the checker's sandbox is decoded once into the object the contract carries, its numbers kept as written
(json.Number), and its size still counted from the bytes it was marshalled to.
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
The verb's answer and the event's body are mesh-contracts' nodeengine types, not mesh-sdk's, and the seat's verb and event
names are the contract's. The two reasons of the event, started and returned, are the engine's own words beside it.
Requires mesh-contracts and the registry milestone's mesh-sdk (a go.work, uncommitted, until the group pins them).
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
mesh/check being carried to e2380404a1fa on e73aae3686a6 | from 998d8f737008
nats on novox, 2026-11-04 11:07: after mesh-host #207 the module stayed held
although its want carried "Back up again", and the engine said nothing. The
engine's composition verb showed it running what its want names (taken from
its held have at the start), so when the answer was in a revision of the want
read after the hold, settleOne returned on "it runs what the want names"
before it looked at the hold: the answer was swallowed, silently. The start
path #207 fixed was right but never met; no test ran the live shape.
- settleOne: the hold is looked at before what runs. An answer (again or
skip) in a want revision after the one the hold took is acted on whatever
runs; with no answer the hold is kept and said with both revisions.
- baseline: the hold kept at the start is said (it was the silent branch).
- composition verb: the want as the engine holds it now carries self, backup,
words and its key's revision, beside the engine's attempt (held, answered
revision, want and composition revisions, transient) and whether the module
was looked at since the start, so which want the engine read is read, never
guessed.
Quick fix: answer once more by hand (a new want revision), which the guard
still swallows while what runs equals the want, so none exists. Thorough fix,
chosen: the order of the guards and the read tool, proved by rows of the live
shape (a self module of policy record, its held have naming the want's own
composition, the answer before the start, and again and skip arriving while
held); the two later rows fail with the old guard order.
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
mesh/check being carried to 5dc3a8e17f97 on 998d8f737008 | from fabfdf04f78a
The journal of an unpack sat at <target>.unpacked.json, in a directory
the module's account may write: a forged one would make the root engine
remove or adopt any path it named. It now lives under
<statedir>/unpacked/<sha256 of the target>.json, mode 0600, names its
target, and is refused whole when an entry is not filepath.IsLocal or a
parent is not above the target. It goes when the archive is taken off,
whether its state record still said what it unpacked or not.
The launcher's stop grace for a host on trial rises from 20 s to 75 s and
the unit's TimeoutStopSec is 120 s (openrc the same), past the host's own
60 s wait for the apply in flight, so neither cuts a save off again.
Tests folded as rows into existing ones (the repository is over its unit
budget): the journal's rows, a forged entry and parent refused, the
journal gone with a record that lost its Unpacked; the test that passed
on main dropped; the stop test waits on its own line, not a sleep, and no
longer consumes the save it waits for.
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
nats on novox, 2026-11-04: the operator answered "Back up again", the
engine restarted, and baseline() took the hold with AnsweredRev = the
want's current revision, so the answer in that very revision was never
acted on and compose refused "held for a failed backup".
Quick fix: answer once more (a new want revision). Thorough fix, chosen:
at baseline, a held module whose want carries again or skip is applied
(again backs it up first, skip goes on without one); a hold with no
answer stays held. The cost said in the code: an answer already acted on
before a stop, whose backup failed again, is tried once more per start.
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
On novox the node-engine's state had lost the records of nats's two
archives, which only a temporary save file of 2026-10-24 still held, and
about 260 such files (~80 MB) lay beside it. The cause: an apply runs
detached from the engine's stop (applyContext, WithoutCancel), and
runLink returned, and the process ended, without waiting for it; a save
cut between its write and its rename left its temporary file and the
state before it, and whatever the apply did was never recorded.
Quick fix: remove the leftover files by hand and restore the records.
Thorough fix, chosen: the engine ends between applies whichever way it
ends (a stop, standing aside, a move to another bus): it waits up to 60 s
for the apply in flight, then lets no save start (store.Quiesce). Save is
one writer per process, flushes the directory after its rename, and says
loudly each record the state saved before held that this save drops and
nothing forgot (Forget, Release mark what is dropped on purpose, and no
longer write into an array an earlier copy reads). The engine's start
removes leftover temporary saves under the apply lock and says how many.
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
mesh/check being carried to 966093d8d067 on fabfdf04f78a | from 4d47ff5ce743, mesh-catalog dd458cb155f5
The vault judged a module handed only when its assignment already carried a handing plan, and the node-engine followed
that word: a module with no plan was composed the old way, refused for a key only the provider's helper answers
(forge-account's ssh_alias), and so never got its plan. resolution.Handing is now the manifest, its build and the
machine alone; the vault's word is read only to say where the two disagree. A refusal of a file or an own secret the
provisioner's client should fill now says "handed module not handed" and why, never only the keys a binding lacks or a
secret to make.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
mesh/check being carried to 8e325830891c on 4d47ff5ce743 | from 6019c6d514a7
The declaration named the account only beside its filled files (node.go), so a node whose every filled-file module
was held this pass named none: on ace from 2026-11-04 01:29 its build modules and ssh-client were held for their
forge binding, and the node-engine refused every write of the files they keep, and of every other module's. The
account is the node's, named whenever the rendering has one. A row holds restic, the provisioned mesh's one
filled-file module, with the client placed: the node's own still names the account; it named none before.
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
mesh/check being carried to 054754482cf4 on 6019c6d514a7 | from 83e135f10187
The review removed the guard and the suite still passed. The guard is now provisionerHeld, a placed client held as
its stand-in, rather than any placed client, so one placed and not held that composes no account keeps none stale.
Rows in the held client's table: the client taken out of the node's placement while what runs names the account, read
from the running own composition and from the client's running process, names no account; a placed client whose
process names no user names none. With the guard removed the first three fail; with it reduced to "placed", the last.
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
Reading ace's node composition gave four resources twice and no way to see that one revision named no provisioner
client's account. The verb now shows, for module node, the revision's provisioner account beside what runs names, its
adoption, how many identities, groups and libraries it names, and each module the composer's last pass held with why.
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
Applied, a node's own composition naming no account refuses every filled write and hands no module, which on ace
went unsaid from 01:29 to 01:33 but for the refusals themselves. The pass now says it once, naming the account what
runs names. A placed client keeps its account by the previous commit; this says the case that is left, a client no
longer composed here.
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
provisionerAccount reads the user of the client's process from this pass's plan, and a client held as its stub (or
carried as it runs) composes no process: on ace, 2026-11-04 01:29 to 01:33, the node's own composition named no
account, the node-engine refused every filled write, and no module was handed (postgres's consumers, restic, power).
A node now carries the account its client runs as (Node.RunningProvisioner: its own composition's account as it runs,
else the user of the client's running process), and a placed client that composes none this pass keeps it. A row holds
the client in the provisioned mesh, both ways: the account stays and restic keeps its filled files; both were lost
before.
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
frombus skipped reading the build of every module in Scope.InOwn, while compose.compositions folds such a module only
at no src2: fingerprint: a want at src2: (nats on novox, its have waiting for a yes) composed it from the indexer's
unresolved manifest and built() refused it, "no build has published" its server artifact. compose.Folds is now the one
rule; frombus takes the build in, or keeps the module at its wanted build, exactly where it says the module is not
folded. A row composes spiegel, in InOwn and wanted at src2:, from its build's upstream image; it read the indexer's
manifest before.
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
mesh/check checking 30b86ea602a8 on 83e135f10187 | novox/mesh-host#203 in group feat/l3-final
The stub's mark (Manifest.Stubbed) is no field of a manifest's JSON, so an input a node-engine sent the check's
sandbox lost it: node-tools, held by its manifest's verdict, read as a tool runner with no bundle and refused every
module of ace, g14 and shanks for "its build produced 0 bundle(s)". compose.Input now writes its held modules (catalogue
and each node's kept) beside its fields and marks them again on reading; an input from an engine before the mark is
read by its contract, every module Refused names held as its stub. The sandbox composer also says, in the module's
read, why the input's node-engine holds a module the case carries, so the check names the cause, not a rule about the
stub. One subtest round-trips an input holding a refused tool runner, in both shapes; it refused for 0 bundles before.
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
mesh/check failed (2): unit: mesh-provisioner-client/cmd/mesh-provisioner-client.TestNatsANodeComingBackFil…; and 1 more | 32a99144f65e on 798d73eb11e9
recordOf writes catalogue.ManifestReads in NodeRecord.ManifestReads, once per build, so the assigner moves a want only
to an engine that reads its manifest. A row holds the engine's own manifest to the baseline's words, so its new build is
never held for a word only it reads.
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
mesh/delivery-group group feat/l3b-bus rejected: composed together: the checker's verdict at the members' heads (a1cb9beeb97a): unit: modules/mesh-provisioner.…
mesh/check failed (4): unit: modules/mesh-provisioner.gofmt: not as gofmt writes them (run gofmt -w): modul…; and 3 more | a24a1d72ad45 on 798d73eb11e9
The second sandbox fault of the same kind on mesh-host #200 (postgres's secret
reader handed, the module not): the engine stripped the whole catalogue from
the input it answers, so the sandbox composed the node's own modules from the
indexer's source manifests while the bus reader composes them taken in from
their builds (artifacts, templates archive texts, what is handed). Both sides
now go through composer.SentInput: the node's own taken-in manifests and kept
builds stay in the input, and the sandbox composes them over the cases'. An
input past what one bus answer carries is refused in words. A conformance row
composes one node through the bus reader and through the sandbox and requires
the same declaration.
Issue 1126.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
mesh/check failed: novox: not composed on novox: postgres: its own secret reader is handed to the provisioner and the m… | 61a3bd996dd4 on 27a956448c0a
A check that composes with a candidate's own composer (any mesh-host change)
gave it the indexer's manifests alone, and the input carries no catalogue, so
mesh-wireguard (shipped with mesh-sdk, no repository) was missing: "nothing
provides mesh-wireguard / mesh-addressing" on novox for mesh-host #200. Each case
now lays the provided module's manifest over its shelf, as the bus reader does.
Issue 1126.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
mesh/check failed: novox: not composed on novox: these assignments cannot be applied: | b8158affa7af on 27a956448c0a
An endpoint's subscription hands its asks over one at a time: two checks asking
one node at once queued, and the second answered at about 100 s, past the
checker's bound and the bus's one-minute reply window. compose-candidate is now
answered in a goroutine of its own, at most four at once, one more refused in
words. The new cases are rows of the existing tests (test budget).
Issue 1126, review round 1.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
Off the control-node an ask of compose-candidate ran into its 60 s bound: each of
its two compositions read every manifest in the mesh one key at a time over the
bus, on scratch copies of the shadow that kept nothing, so each ask read them all
twice again (about 35 s per composition on ace, g14 and shanks; 2 s on novox).
The answer at 60 s then came after the bus's one-minute reply window
(allow_responses ttl), so the bus refused it and the checker heard nothing.
Now the manifests are read eight at a time into the shadow before either
composition, a scratch copy keeps what never changes (manifest, archive, done
build) in the shadow too, the candidate is laid over a copy taken after the
composition without it, and the engine answers within 50 s: unready, naming what
it waits on, when not done. Every answer is said in one journal line with its
reads and its longest wait.
Issue 1126.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
The checker asks by the composition key, <group>.<heads hash> (ADR 0593 §7), but the
indexer keys a candidate by its group alone (delivery.CandidatesKey). The node-engine
read the ask's key as the bucket key, so every check's composition was refused on
every node with "the indexer holds no candidate" (mesh-catalog #599, 2026-11-02).
It now parses the ask, reads CandidatesKey(group), and answers unready when the
group's candidate has moved to other heads. Both tests' fixtures had captured the
wrong key; they now key by group, with a slash in the group name.
Quick fix: this lookup. Thorough fix (a shared SDK reader that takes a composition
key and returns the candidate at those heads) not taken: one reader, one repository.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
mesh/check failed: integration: meshbuildgo/cmd/mesh-build-go.TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus: NOT CHECKED: the node-engine's composer c…
The test read mesh-controller/module.json beside the check, which no longer exists after the
controller's removal (issue 1126, ADR 0593 §11), so it would fail every check of the node-engine.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
mesh/check failed: integration: meshbuildgo/cmd/mesh-build-go.TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus: NOT CHECKED: the node-engine's composer c…
ADR 0593 §7: the environment's rendering left mesh-sdk for the node-engine,
so its two integration tests (the POSIX environment sourced twice, and
the service manager reading the systemd rendering) move with it.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
ADR 0593 §7: the composition is the node-engine's, so its tests hold
what the indexer's judge tests and route-proxy's tests proved by
composing: a bundle's words per machine, a Go bundle's delivery, the
credential form in a handed plan, settings refusals, placeholder and
vault refusals, the session's files, a field set aside composed whole,
route-proxy's manifest composing for a machine, a route endpoint's
machine port in six cases and the plan's route entry end to end. Two
table tests, one subtest per item; route-proxy's manifest is captured
beside the catalogue.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
ADR 0593 §7: the composer with its refusals leaves mesh-sdk, so mesh-sdk
carries no composition rule. The catalogue functions only the
node-engine's composition reaches (Resolve, Resolution's Compose,
Declaration, DataItems, LeftOut, Rules and the rest, the settings,
placeholder and template refusals, the rendering helpers) are copied
with their tests into internal/compose/resolution; a method of a
mesh-sdk shape becomes a function taking it first. The composition
calls its own copies, and the captured catalogue its tests read lives
here now, shared by mesh-compose's tests.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
ADR 0593 §6: a composition read from composed-<node> is a record never
applied, and the engine applies only what it composes itself, so no
composition carries a signature and nothing is verified. The agent's
trusted signers, its signed-composition test and the unused signing
helper of the integration bed go, so mesh-sdk can drop SignComposed,
VerifyComposed and Composed.Signature.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
The test budget: mesh-host is over 500 unit tests, so the three tests
added for ADR 0593 §6 and §7 are paid for by folding the private
network's hub refusals, its MTU cases and its no-self-peer check into
table-driven tests, each case kept.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
ADR 0593 §6, §7, §9: the integration test runs the node agent against
the real bus under the grant the grants part composes for a node: it
publishes web's composition in composed-<node> with the digest its have
names, and answers compose-candidate for the indexer's candidate,
changing nothing. A node that cannot compose now answers unready, so
the checker counts it as no answer by the verdict and never by words.
The golden grant reads the indexer's candidates and verdicts as it
reads the builder's builds.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
ADR 0593 §6: composed-<node> is written again, by the node's engine,
each composition with its plan. The verb composition finds a published
revision by the digest the have and want name, its resources', so a plan
on the record does not hide it. The golden node grant also reads the
indexer's verdicts, which compose-candidate takes for a candidate's
manifests.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
ADR 0593 §7: a check's composition is the checker's, made from each
placed node's answer, since only a node can compose with its own sealed
values. The node composes the indexer's candidate in shadow on a copy of
its shadow's inputs, the candidate's modules laid over the indexer's
with their manifest verdicts, and answers per module composed (changed
or not), refused or held, and its own composition's verdict, within the
answer bound; it applies, writes and hands nothing. With input it also
answers the composition input, for the sandbox path.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
ADR 0593 §6: the provisioner, the assigner and the view read compositions
instead of composing, so the node's engine writes each one it makes to
its composition state: the wanted builds, every placed module at its
newest done build (so the plan exists before the want moves), each with
its provisioning plan, and a module refused alone with its refusal. A
key is written only when it differs, and kept while among the module's
five newest or named by a want. A placed module's trunk build landing
now makes a pass.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
ADR 0593 §1 and §7: compose, its bus reader and runtime reader, takein,
validate, the overlay generator, the composer's run and mesh-compose, and
the judgement of a move leave mesh-sdk for the node-engine's internal
packages, so no other module composes and a change to them rebuilds only
the node-engine. The shapes others read (the composer's request and
verdict, the overlay's names, compositions and answers) stay in mesh-sdk.
The node's grant is now a golden file of what the grants part composes,
since the composer of the bus's user list leaves mesh-sdk too.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
The bus module on the bus node runs from what its agent holds (its own composition, or the declaration kept before the
switch: every novox have read 2026-11-02 names its own composition or none, and nats names a build alone), while
composed-<node> was written by the controller, which ended (issue 1126). The follower now looks at the agent's record
of what runs whenever it changed, writes again only when the file it names moved, and trusts no composition signer.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
Every node composes its own since group A (issue 1126), so the resync that handed the vault an own secret's value
while a node applied the controller's compositions never runs, the shadow's answers read no want, and nothing follows
the heard vault key any more.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
nox now calls the seats' verbs through the console itself (mesh-catalog #593), and the controller the engine forwarded
its lines to ended (issue 1126), so the socket, its forward and the subject it asked on go. The kernel's word on a
peer moves into the filled-file socket, its one remaining reader.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
The CONTROL stream the facts, the enrolment and the overlay rekey were published into was the controller's, which ended
by its unassignment (issue 1126) and whose stream is removed after group B's delivery: a publish there would fail every
facts event before its haves are written. The facts go into the haves as they did after the report, the enrolment goes
through mesh-enrolment alone, and overlay take is refused because no seat takes a rekey now.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
The controller that signed the hand-over, search, judgement, access and move asks ended (issue 1126), so a link
believes none of them by having no signer, and the per-ask key it was told to follow goes with the lease.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
The controller's declared-packages mirror ended with the controller (issue 1126) and is deleted after group B's
delivery, so the verb no longer reads or compares it.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
The controller ended by its unassignment (issue 1126, ADR 0590), so no build is judged by the lease: the lease
witness, its contract and the direct get the witness asked it with are removed, and its tests now prove each rollback
path on the node tools' PING.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
mesh/check being carried to 38d666f22390 on 84737f4cb26d | from 14a12cc7b6f9
The controller ended by its unassignment, so a node that falls back to the
compositions in its bucket follows what nobody writes (issue 1126, A7). The
node agent now always runs its own composer: without a placement key it
composes nothing and keeps what runs, a placement it cannot watch stops the
agent and is said, and a candidate taking its placement away is refused
instead of answered as a return to the controller. The start-time placement
read, the end-when-placed watchers and the returning answer go, with their
tests and the three integration tests of the controller-composed path.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
The controller ended by its unassignment and its lease is never handed, so
the engine's lease gate kept trusting the controller's key and the bus node
kept the controller's list as a fallback nobody writes (issue 1126, A6).
The engine now trusts no controller signature, its hand-over line names the
node seat's hand-over, and the user-list follower writes only the grants
part's list: a key unreadable or deleted keeps the list loaded, and with no
key heard nothing is written. Compositions in the node's bucket name the
file's place alone. The guard no longer asks for the controller's user.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
A fall back to the old login during a rotation is now delivery.NodeHealth's Fallback (the node, the login fallen back from,
why, when), written with the node's health report, from which the watchman raises node-account-fallback; it is said no more
once the vault's key of the node moves on (blocker 7, ADR 0583 §6). mesh-host enrol's fall back to the controller's
enrolment subject says it is the path before the lease write, which the bus refuses once the lease is written. Tests: rows
in the existing rotation test; none added.
(issue 774)
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
A stand-in vault reloads a nats-server's users as the grants part's list is loaded, and the engine's own rotation code runs
against it with real test-dials and a real confirmation request: at every step at least one login authenticates; the engine
switches only once the list holds both logins and confirms in the new login's name; at rotation-done the old login is
retired and its value dropped; with the list never loaded the test-dial is refused and the engine never switches; and an
abandon after the switch brings the engine back to the old login before the new value goes (ADR 0583 §2-§6). One integration
test; mesh-host holds seven of fifty. The real vault and the real engine together are proven live on g14 first.
(issue 774, 774/S65)
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
The engine dials as enrol.<node> on the pinned certificate, calls redeem with the public halves of its keys and a proof over
them (the SDK's RedeemProof, which binds the node and the keys and never the token's secret), asks again while the holder
says to, and verifies the vault's answer against the key its token carries before it keeps anything: an answer signed by
another key, naming another signer or node, or altered is refused and nothing is kept. A signed one is kept (membership,
login, keys), then the node dials as the login it was given, reads its key of node-logins, and calls confirm; nothing is
applied until its placement names a module. When the bus does not serve the seat to the token (the controller still holds its
lease) the controller's enrolment subject is used as before. The token's shape is mesh-sdk's now: identity keeps its names as
aliases. Tests: one added, paid by merging three identity token tests into one (net two fewer).
(issue 774, 774/S64)
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
On node-logins' ready the engine fetches the new value through the vault's own hand-out (own/<node>/mesh-host/<login>, opened
with its sealing key), dials a second connection as the new login on the pinned certificate, and switches only if that
authenticates, then calls mesh-vault.node-account-confirm over the link as the new login (the vault is asked in the name of
the login the open link authenticated as, and only once that link is open). It keeps the old value until the key says the
rotation is done and drops it then, never before; an abandon, after the switch before or after the confirmation, switches
back before the new value is dropped; repeated refusals of the new login's credentials before rotation-done bring it back
to the old one, said as the finding node-account-fallback (in the journal: no route for an engine's finding to the watchman
is decided), and the same rotation is not tried again. Tests: two added, paid by merging two identity sealing tests into one;
more to be paid by deleting the identity token tests that move to mesh-sdk with the token (774/S64).
(issue 774, 774/S62, 774/S63)
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
A membership may name its Logins (ADR 0583 §1): the login in use and the password of each login held, kept beside the
identity, so a restart dials as the login in use. The engine follows its own key of node-logins as a watch on every link
and dials as the login it names when it holds its value, with the node's one inbox; with no key, or a key naming the first
login, it dials node.<n> with the membership's password as before. A change of the login in use drops the link so it is
opened again as the new one, and the vault is asked in the name of the login dialled (the subject names the caller).
Tests: one added, paid by merging the pinned-broker test with the malformed-pin test (two into one).
(issue 774, 774/S61)
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
mesh/check failed: mesh-build-go: the check of novox/mesh-catalog#590: the ask mvyhumdm860c7f's step on shanks did not end within 30m0s of its start |…
A key that later cannot be read or is deleted is said, the key last heard stands and the list loaded stays. A key without
the flag still falls back, as before.
Tests: three rows added to the existing table test of the follower; no new test function. Built against the group's
mesh-sdk through a go.work outside the tree; go.mod is not re-pinned here.
(issue 774, 774/S31)
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
mesh/check checking 74ecda8ea2cf on bc960c51dbf0 | novox/mesh-host#193 in group feat/node-packages
The delivery group feat/node-packages pins one mesh-sdk head in every member: it carries DeclaredPackagesOf and the engine's read of the mirror this change uses.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
pacman still reads the controller's mirror because no engine wrote its key: the engine
now keeps each handed composition's package resources and, after a pass that handed
something new, writes mesh-host_packages.<node> only when the value differs, owed after
the growing bound when refused. The manifest declares the state so the grants part
creates it. The verb packages answers the key beside the mirror's key for the node and
their difference, and with write writes the key again from what was composed. H8's
integration test proves the write under the grant the grants part composes.
Needs mesh-sdk feat/node-packages (0ef7c0c) for DeclaredPackagesOf and the engine's read of the
mirror (go.mod pin left for the lead: the SDK commit is not pushed).
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
Issue 1585, S17: the engine decodes manifests leniently; a module whose manifest the indexer refused is held and named,
its data key stands, and every other module's items are still published. Policy, HoldsData and the certificate check no
longer parse with the judge.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
mesh/check pass e031758e4a37 on 6c1f7cc7f50f | unit 4510 run, 4259 kept, 49 skipped | integration 4589 run, 4361 kept, 51 skipped | over bound: unit, …
The builder claims a done key again as a new revision, so a build kept as done is dropped on any
write of its key. Data items that could not be worked out or written stay due and owe a pass after
the growing bound, so a first write refused before the state or grant exists is tried again
without waiting for an unrelated composition.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
The data items after a pass re-read each placed module's build over the bus (about 55 per pass on
ace, g14 and shanks); a done build never changes, so the composer keeps it. The items are now
worked out with the pass's own kept builds and modules run inside the node's own, as composed.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
After a pass that handed a new composition, the node's composer works out its modules' data
items from the inputs it composed from and writes each module's key <node>.<module> of
mesh-host_data only when it differs, deleting the keys of modules that left. Pins mesh-sdk
feat/data-items for the contract and the engine's grant. Two tool-answer tests are merged so
the unit count stays where it was.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
mesh/check being carried to 71afdf3ad234 on 6c1f7cc7f50f | from ffa63d25a066
novox's postgres stayed held after mesh-host #189: its have, written before that fix,
names no build, so the node's own composition still took it in and was refused for its
secret reader before the start could back it up again. Only that exact record leaves
the node's own; every other have naming nothing keeps main's rule. Completes #189 for a
have written before it (issue 1289).
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
mesh/check being carried to 76c7ddfc5227 on ffa63d25a066 | from a4713e2d28d7
Review round 1: a have naming no build in a phase but running is also written for a
module that runs inside the node's own (waiting for a yes), so taking it out would
drop its resources before the yes. The fp the held have now keeps already fixes
postgres (issue 1289); a test drives an interrupted backup end to end.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
On novox (2026-10-31) postgres's repeat backup was interrupted by the node-engine's
restart: its held have named no build, so the node's own composition took postgres in
and was refused for its secret reader, and nothing ran the backup again. Issue 1289
(R4c, its delivery held).
- A held have names what runs: read from the module's have before backing-up when
the agent has no running entry, as the not-composed mark does.
- A have in any phase but running that names nothing is not run inside the node's own.
- A backup interrupted by the agent's stop is no failure: the next start backs the
module up again and applies while the want asks a backup. A real failure stays
held for the operator's answer.
Quick fix: a new backup ask by hand. Thorough, chosen: the fault fixed where it lives
in the node agent, so every data module's interrupted backup resumes by itself.
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
mesh/check failed: shanks: the check of novox/mesh-controller#487 failed: go mod download in modules/route-proxy: go: git.novox.be/novox/mesh-controll…
mesh/check being carried to da0f8f1cfdb4 on bb8383b065a1 | from 62b211bb6040
Every put under want.<node>.>, every module's take-in and every placement write
made a pass, and each pass asked the vault for the node's own secrets. The
shadow's change check moves onto the composer, gains the want (by its
fingerprint, the only part composed) and serves both loops.
On the bus node restic's manifest replaced its process measure by awake (mesh-catalog 35ff67bd). The record of restic.measure was the node's, made while restic was folded, so restic's own apply never saw it, and the guard against removing a placed module's resources refused the node's own apply whole every two minutes from 2026-10-30 19:42: nothing of the node's own applied.
Quick fix considered: forget the one record by hand (a hand act on the node; refused). Thorough fix (this): the node's own apply keeps such a record and says so, never refusing or removing it; a module's own apply owns a node record whose id names it, so it alone removes it once neither composition declares it.
mesh/check being carried to b0d6684d3731 on 91fb323ce0cf | from mesh-sdk dec3bfe7a8cb | novox/mesh-host#185 in group fix/compose-retries-transient
The verb compose has no warrant (ADR 0563 §2): Agent.Retry released a module held for a failed backup and re-applied
one already running its want. RetryFailed tries again only a failed apply not held, and refuses in words otherwise.
A hand ask no longer restarts the growing bound, and a pass less than askedWithin old is the answer, so a caller in a
loop drives no passes against the vault. A composer whose loop ended refuses at once. The retry line said try N+1.
The test's first shadow now ends before slowAfter is set, a race the race detector found under the new timing.
On 2026-10-31 ace's route-proxy said only that the vault could not be asked, and ace said nothing for five minutes
while a pass waited after the vault restarted; the retry that already existed was invisible. A refusal for a seat or
a read that did not answer now says when the node composes again ("the node composes again at HH:MM:SS (try N)"), a
retry pass says it starts, a pass that runs on says every minute what it waits on, and the node-engine's verb
compose makes a pass now and says how it went, so a person recovers without a restart.
Four providers waited for a yes on 2026-10-31 and nobody could read what waited: the have said only
"where its data is kept". The read verb composition answers it from the engine: the latest revision of the
want's build or the one asked by digest, every revision the bucket keeps, what runs and where that is known from,
the want and the have, the move's judgement and whether it waits for a yes, and with diff the resources that
differ, a received file naming its consumers. revisionNamed now reads the same revisions.
waitsForYes returned at once for a composition equal to what runs, and settleOne then found nothing to apply, so
the have kept phase waiting-yes for a move that no longer existed (ace.postgres, novox.postgres, novox.keycloak and
novox.mongodb on 2026-10-31). The have is now rewritten to running.
Quick fix: rewrite the have where the equal composition is met. Thorough fix (not here): one place that writes a
module's have from the settled state after every settle, so no early return can leave a stale phase.
The review of d5e8458 found that mesh-wireguard, provided by the mesh and never wanted at a fingerprint, kept the kept
declaration read on every node for ever, against ADR 0550 §2 once the lease is written. A selfcompose row shows the
node's own composes it from delivery.ProvidedManifest with nothing running, so the kept read holds nothing of it.
Quick fix: name mesh-wireguard in allHave. Thorough fix: allHave counts every module delivery.ProvidedByMesh says the
mesh provides as covered. Chose thorough: one list of provided modules, mesh-sdk's.
At 19:43:01 on 2026-10-30 shanks' new engine said "the kept declaration is no longer read: each of the 77 modules
placed here has a have of its own", and from 19:43:57 every apply of the node's own was refused: it would remove
systemd-resolved's resources (novox the same for nats and restic). systemd-resolved's have names only a controller
build (build-1791404191702719760), no fingerprint, so the assigner wants it nowhere and it runs folded in the node's
own, of which the kept declaration is the one record; retired, the folded module was composed as nothing.
Quick fix: never retire the kept declaration. Thorough fix: allHave counts only a have naming a fingerprint of the new
form (or removed); a module at a build of the old form keeps the read, named with why, until it has one. Chose
thorough: the read retires by itself once every placed module is wanted.
mesh/check checking 33131dcbfe77 on 2ae7c53376e8 | novox/mesh-host#181 in group fix/build-not-done-holds-one-module
At 19:30 on 2026-10-30 node-tools' want moved to a build not yet done; once it landed, ace, g14 and shanks stayed
refused until some other input moved (g14 and shanks at 19:40, ace after a restart), since the composer watched no
build.
Quick fix: compose again on a timer while anything is held. Thorough fix: the composer watches the builder's builds
(within the node's compose reads, NodeComposeReads) and makes a pass when a done build of a module the node wants, at
its want's or the trunk's build, lands; nothing else moves it. With mesh-sdk 9685990a (pinned here) a running module
whose build is not done is held, keeping what runs, so the pass that waited composes it as soon as it lands. Chose
thorough: event-driven, no poll.
mesh/check checking 4055e0094b48 on ee71ba4dea81 | novox/mesh-host#180 in group fix/one-bad-item-holds-one-module
At 19:13 on 2026-10-30 power's bar item (mesh-catalog ca8d6f33) made frombus.Compose refuse the whole node, and the
composer marked every module not composed (32 of 81 failed on shanks). The composer already holds what Compose names
held (NotComposedEach, issue 1126); the SDK now names a refused manifest's module held instead of refusing the node.
Quick fix: none in mesh-host; thorough fix is the SDK's input (one rule for Compose, Answer and PlansHeld). The test
replays the captured power manifest and fails on the old pin with the live words.
mesh/check checking b9c24a7b42cb on 737bce9c9448 | novox/mesh-host#179 in group fix/sweep-2026-10-30
mesh-vault (mesh-catalog f3b13de2) answers a move it refuses with `refused: {reason, held}` and `why`, and
no read-back. The engine read only the read-back, so every such row failed "answered with no read-back"
and the controller asked it again on every pass.
Quick fix: treat any answer without a read-back as done. Thorough fix (chosen): read `refused`. For
holds-another-value the vault's value is the live one: the move answers done, superseded, at the held
generation with the held record in its words, so the controller records the row and stops asking; the
journal says it once per secret. Any other reason fails in the vault's own words.
restic's composition dropped its process restic.measure on g14 and shanks at 16:14; the engine removed it
and said it down on every look after. The judges were set only when facts were said, and the facts said
inside an apply read the module's composition from before it (the agent records the new one only after
the apply returns), so the removed resource was put back. Judge.Set itself prunes what is not declared.
Quick fix: set the judges after the restic apply. Thorough fix (chosen): one function hands every judge
(liveness, resolver, units, accounts, libraries) what is declared; it runs from the facts as before and
now also whenever a module settles (applied or removed) or is taken as running, so every removed health
entry of every module is pruned at recompose.
On 2026-10-30 ace's redis, postgres and restic (14:03), step-ca (13:55) and mesh-vault on novox (15:22)
were held on a single backup ask that met no responder while the tool runner restarted.
Quick fix: ask node-backup.now twice. Thorough fix (chosen): every node-backup ask (backed-up and now)
is asked again with the engine's backoff while it is known never to have been delivered (ErrNotAsked),
bounded at three minutes; an ask that may have been delivered (a timeout, a refusal) is never asked
twice, so no backup starts twice. Past the bound the backup fails saying how often and how long it was
asked, the module is held and its have carries those words for the assigner's ask and condition. This
was the engine's only one-shot seat ask.
Every engine's node-seat subscriptions were refused at 12:10 by a user list composed without them. nats
asks a refused subscription again only on a new connection, and a user-list reload makes none, so the
seat answered nobody all afternoon and the messenger held every Telegram ask. The endpoints also lacked
the kind and scope discovery reads, so the console said nothing in the mesh is called node-engine.
Quick fix: restart the engines. Thorough fix (chosen, the second time a stale user list left an engine
deaf): a refused subscription of any kind is said and the link is opened again after a pause (1 min,
doubling to 30 min), so every subscription the engine makes is asked again; the seat's endpoints carry
kind, seat, scope, module, description and schema like every tool runner's.
mesh/check checking 015334020ad2 on a3c098a5c618 | novox/mesh-host#177 in group feat/774-l3-reland
The conflicts were go.mod, go.sum and testdata/beside/CAPTURED alone, taken from the release. 2da0de7 is 6e4a301's code;
its conformance events are unchanged.
mesh/check pass 2fd165f5af8e on 8c5af3e4a3a3 | unit 9174 run, 546 kept, 51 skipped | integration 9952 run, 659 kept, 60 skipped | over bound: unit, in…
78c9ec8 is sdk main 307b623, whose tree is main's pin dbcaa29, with one commit that parses the new words (a credential's
form, an artifact's bus, a log's max-age-days) and names the new seats; an offer with no form still means pair. No code
of ours changes. Unit and integration suites pass against a local bus and main's catalogue and controller beside: 1260
tests, 7 skipped for a container or a throwaway machine.
No code of ours changes. Whole unit and integration suites pass at main's code against a local bus: 1260 tests, 7
skipped for a container or a throwaway machine.
The whole integration suite passes at this head against a local bus (JetStream, and a bus with the witness grants) and
the release's mesh-catalog and mesh-controller beside it: 1260 tests, 7 skipped for a container or a throwaway machine.
Every account resource marked root never goes into NodeHealth.Root; the statement's contract into Contract. Relogin,
root and found are judged each on its own, so an account may be both. Pins mesh-sdk f3677eb. Test: the report's row in
TestAHealthStatementIsSaidOnTheLinkOpenNow carries the agent account's verdict and the contract.
delivery.NetworkPart.Targets (mesh-sdk 6eae076) is filled where the report is made, by the controller's own rule:
- the hub's node for "hub";
- each resolver's node by its address on the private network;
- none when the part points at nothing, at this machine, or at an address no other mesh machine has.
The names come from the placements the node's own composer holds (Decided.Address, Decided.Hub). A spoke's tunnel names
only the hub, so the tunnel cannot name a resolver on another spoke. A node that does not compose its own names no
target.
Test: rows in TestAHealthStatementIsSaidOnTheLinkOpenNow (hub, two resolvers, a VPN's resolver, its own, a healthy
part).
delivery.NodeHealth and its parts replace this package's copies. The message's network and units parts are aliases of
the same types, so its JSON is unchanged. The engine keeps the derivation from its statement and the write on a change.
Test: the key's bytes, decoded and encoded again by the sdk, are the same bytes.
Until now the six machine alerts came from a health message passing to the controller. The report is now kept in the
engine's declared state `health`, key `<node>`, created by the grants part as `acts` is. Only this node's engine
writes it, after comparing it on every look and only when what it says changed; it is read first after a restart.
It carries:
- the network parts (machine-network; network-rewritten through the resolver file part's writer and owner);
- the tunnel's peers, each seen within five minutes or not (network-unreachable);
- the failed units no module owns (machine-units);
- the accounts marked relogin needed (relogin-needed);
- the directories used as found (directory-used-as-found).
The streaks and the time of the look are left out, so a look alone writes nothing. A peer is reported seen or not,
never its handshake time, so a renewed handshake writes nothing.
The layout is internal/link's (NodeHealth) until it moves into mesh-sdk. Pins mesh-sdk 58a8324.
Tests:
- rows in TestOnTheHubAnyFreshPeerIsAHealthyTunnel: the peers, and a change only when one is seen or lost;
- a case in TestAHealthStatementIsSaidOnTheLinkOpenNow: what the report takes, and one write per change across a
restart;
- the manifest's states in TestTheManifestConsumesTheBackupEvent.
- A placeholder is `${secret:<name>}`, the name the module calls the credential (catalogue.SecretLocal): the local
name where it keeps several values of one provision, else the provision. The `<provision>.<local>` form goes.
- The engine finds the one key under `<node>.<module>.` of mesh-provisioner's `provisions` going by that name. That is
what node.<node> may read, by a watch to its caught-up mark; plans are not read.
- The reference's ID must be a credential name of that provision (provisions.VaultName). The vault's answer must carry
exactly that name and generation. Anything else refuses the write, naming both.
Pins mesh-sdk 66ff33d. Tests: rows in TestAWriteAndAnApplyOfTheSameFileSerialise (a local name, a name no provision
goes by, a reference to no credential of the provision, a mismatched answer).
The value written is the one the provision names: a1-47 makes Reference.ID the vault's credential name. A mismatch in
either refuses the write, naming both. Test: a row in TestAWriteAndAnApplyOfTheSameFileSerialise.
The provisioner's client fills a handed module's file but for each `${secret:<provision>}` (or a whole `secrets` file
that is the placeholder alone).
- The node-engine reads the provision's key `<node>.<consumer>.<provision>[.<local>]` in mesh-provisioner's
`provisions`. The key must be `provided` and carry the vault's reference.
- It asks the vault's `own` for that pair credential sealed to this machine, at least at the reference's generation,
and opens it with the node's sealing key.
- It writes the file as declared (its owner), 0600 when the declaration gives no mode.
- Anything short of that refuses the write, naming the provision and why.
A module the node composes itself already has its `secrets` files written so: its composition carries the vault's
value from `own`, written 0600 with the declared owner.
Pins mesh-sdk 4a96793 (go/provisions). Tests: rows in TestAWriteAndAnApplyOfTheSameFileSerialise.
The reviewer showed how 76e214a still lets the 07:54 outage happen at the re-landed engine's first start on novox.
applied/ is empty there, so loadApplied loads nothing. The kept declaration retires, because every placed module has a
have. Nothing then carries the node's own composition. The guard misses nats, which has no want there, so its
nats.conf is guessed as nats's and removed.
- At the start, when the store has no node's own composition, the node's own runs as the kept declaration has it, and
the store keeps it now. The kept declaration retires only once the store holds `node`.
- What this engine applied runs until loadHad reads the composition a differing have names. The bucket no longer keeps
the one novox's have names, so the stored one stays.
- In the node's own apply, a resource of another module goes only when that module's want says removed
(Scope.RemovedByWant). A module the node's own composition carried has no want, so Placed never named it.
Cost: the node's own apply cannot drop one of its own dotted resources without such a want. It is refused whole and
said, never removed silently.
Tests, with no new test functions:
- A first-start case in the agent's Everything test (empty store, then a start with the declaration retired). It fails
on 76e214a.
- Two rows in the guard's table: no want refuses; a want saying removed removes.
- A case for the store in the authority test.
On novox on 2026-10-29 at 07:54 UTC the kept declaration was retired and the node's own composition was in no bucket.
The composer's first pass reads what runs before the agent reads any have (the agent waits for the composer's caught,
sent after that pass), so the node's own was composed with its folded modules running nothing, and the bus server and
four held modules were removed.
- Quick fix: read the node's own have in loadHad too. It still waits for the composer's first pass, and still finds
nothing when the bucket no longer keeps the composition the have names.
- Thorough fix, chosen: the engine keeps each composition it applied (the node's own too) in `applied/` beside its
state, 0600. The agent reads it at Run's start, before any watch, as what runs ahead of the bucket and the kept
declaration. A have naming another running composition wins. An apply keeps; a removal forgets.
Test: a case in TestEverythingIsWhatTheAgentRunsAndWhatIsKeptForTheRest (no new test function). It fails on L3's
engine 4f9227a (0 compositions at the first own composition) and passes here.
novox's mesh-vault has failed since 01:16 on 2026-10-29: its have says failed, so loadHad (fbea777) takes no
composition from it, and what runs falls back to the kept declaration, whose export (mesh-vault.kept) names
f099eaa1, an object the controller's sweep removed. The node wrote a newer export since. The composer carries that
file as it runs (mesh-sdk frombus keptOf), so what runs now names the digest and size of what is on disk, but only
where the node's record says it wrote exactly that content at that path (Applied.Wrote, Target). The apply then
finds the file in place and needs no object.
Quick fix weighed: have the controller put the object back. Thorough (chosen, in the node): what runs is what the node
wrote, never a guess from an older declaration. The rows join the existing whole-apply test; the test count stays.
On novox at 01:16 on 2026-10-29, after the node began composing its own modules, mesh-vault's apply failed while the
vault ran: its own composition carries the export as it runs (mesh-sdk frombus KeptRuns) by a digest whose object the
controller's sweep (mesh-controller cmd/mesh-controller/declaration_sweep.go, sweepNode and sweepDeclarationFiles)
had removed, since the sweep keeps only what the controller's compositions in composed-<node> and the wants name.
The node may only read its bucket (ADR 0486 §2), so the node keeps what it had: a file here that verifies, or an
object fetched, goes to <state dir>/objects under the object's name, and a kept one is used before the bus is asked.
An object unused for 14 days is let go. A fetch that still fails says the keep never held it, before its cause.
Quick fix weighed: re-put the object from the controller's store. Thorough (chosen): the node never depends on the
bucket for content it once had, for every module, since the sweep and a node composing its own can always disagree.
The whole-apply test gains three rows that fail on the old code; the test count stays.
The kept-declaration fallback built a composition without Data, so after a restart a data module (mongodb, mssql
on novox) was judged to move its data against its first composition and the operator was asked a false yes. Now
every such composition takes Data from the manifest (agent.Env.HoldsData, the composer's DataItems), and a module
whose manifest cannot be read is said to be unjudgeable rather than taken to hold none. The composition a have
names comes before the kept declaration in the judgement too.
loadHad takes a have only in the phases baseline takes as running (running, held, a composer's mark): one backing
up, applying, checking, failed or waiting names a composition not applied (review of 031e106). Two rows added to
the same table test; the count stays.
A held module on a node composing its own never gets a composition, so the agent never took it and the filling
judged its files by the kept declaration, older than power's three new drop-ins: refused from every engine start
on ace, g14 and shanks until some module settled (2026-10-29 00:09 UTC). The second meeting of issue 924.
Quick fix: knock the filling once the haves are read at the start. Thorough fix (chosen, the kind recurs for every
file added since the switch): what runs of such a module is the composition its have names, from the bucket or
its revisions, ahead of the kept declaration; the filling is knocked on every change of what runs, not only on a
settle. One unit test replaced by a table of four cases; the count stays.
mesh-sdk's composer no longer holds a module behind a held one it reads from; the selfcompose test now expects
proxy handed, composed without de-spiegel's routes, and de-spiegel told alone. Needs mesh-sdk pinned at the fix.
mesh/check pass a6c70803ab72 on bb6e47293909 | unit 3054 run, 234 kept, 36 skipped | integration 3075 run, 266 kept, 29 skipped | over bound: unit (sl…
novox's seed p-20261028t231141-ec84b4 refused 11 modules behind de-spiegel's hold and gitea's handed credential. The
node composes and answers through mesh-sdk's frombus, so the fix is the pin.
- The quick fix: follow one more set of words in the answer. It moves the error again.
- The thorough fix, chosen (mesh-sdk fix/1126-held-as-runs): a held module composes only as what runs of it; a consumer
or collector composes against that and is never held for the held one's stub, and a stub's handed credentials and own
secrets are left out as they run rather than refused.
mesh/delivery-group group feat/774-vault-mints-l2 checking: the checker's verdict is out of date: novox/mesh-delivery@64d364a7686c was judged on trunk fa075dc8…
mesh/check failed: ace: the check of novox/mesh-host#168 failed: the pin of git.novox.be/novox/mesh-sdk/go at commit d4c61ee8ffd0 is on no pull reques…
mesh/delivery-group group feat/774-vault-mints-l2 checking: the checker's verdict is out of date: novox/mesh-host@e0c7f36ca90c was judged on trunk d2a868f99f05…
mesh/check failed: integration: meshbuildgo/cmd/mesh-build-go.TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus: the compose ask mvsg85b34d25b4 never ende…
The answer names its proposal and the revision of the node's own candidate key; a node with no key in a proposal is not asked. The watch for its own placement key is tried again until the seat's state exists, so the node switches when its key is written.
mesh/check being carried to bf667a602d21 on 58b4f8b6bc07 | from 9a8b3870bad0
A reboot left the have running, so a unit that failed to start at boot was never started again and nothing said so. Baseline now asks the machine; the judge's look calls Agent.Reconcile once per down edge, leaving held and unwanted modules alone.
mesh/check pass 24c23a1d066f on a548a61e86d7 | unit 2667 run, 590 kept, 36 skipped | integration 2676 run, 633 kept, 29 skipped | over bound: unit (sl…
frombus.Compose now holds a module refused alone (mesh-sdk 0de3006, the rule
its answer follows too): composed as what runs of it, with what depends on
how it is held held too. The composer hands every other module, keeps what it
handed of a held one, and tells the agent which modules it holds and why;
agent.NotComposedEach marks those alone and clears the rest. Only a refusal
of no single module still marks every module (NotComposed).
heldOut also holds out, of the node's own composition as it runs, the
resources of each module held alone, until a composition of its own is
handed: the node's own apply never removes a module leaving the fold with its
names unset (review of the apply, round 1; issue 1945 by a new path).
Pins mesh-sdk fix/1126-apply-culprit at 0de3006 (a group).
mesh/check pass bc28606d4566 on 690aa662a837 | unit 2192 run, 3794 kept, 57 skipped | integration 2208 run, 3829 kept, 57 skipped | over bound: unit, …
A skipped or held file comes back on every apply; counting it would reload the
service manager on every apply of a node with a gated or held module. The same
allowlist as MovesRoot: created, updated, corrected, removed.
A drop-in that no service reflects, like the power module's bound on
reboot.target, was written but never loaded: the node-engine reloaded units only
for a file named in a service's restart-on, so the next reboot still ran the
target as loaded. Any file written or removed under a system unit directory, or
a drop-in directory below one, now gets one daemon-reload per apply.
mesh/check pass 8bd0b5db15a3 on b4fd2cc210bf | unit 1639 run, 4201 kept, 46 skipped | integration 1674 run, 4215 kept, 47 skipped | over bound: unit, …
Each answer's line now reads 'answering placement candidate G (revision N; woken by X)', X being its first answer,
a retry after a read the bus did not answer, the candidate, or the inputs that changed (placement, records, modules,
values, haves, wants). The single input counter becomes one per input, which also decides whether a group is answered
again, as before.
Quick fix weighed: a counter word in the line. Thorough fix chosen: one count per input, so the line names exactly what
moved since the group's last answer.
mesh/check being carried to 1a021052fe07 on b4fd2cc210bf | from 7a2af2376405
Review round 1 measured it on systemd 262.1: a finished one-shot nothing references is
unloaded, and loaded again it reads inactive, ExecMainStatus=0 and a start timestamp of
0, as one that never ran does. The previous commit's rule read such a unit stopped, so
a one-shot declared running would be started, read stopped and fail on every apply, and
the hold, the take-over and the giving back would judge it wrongly.
ServiceState keeps main's reading. ServiceAtRest and declaredState stay: they alone
satisfy a stopped declaration for a one-shot at rest, which is the kernel cleanup's
case. A one-shot declared running that must be seen to have run lingers
(RemainAfterExit) or is enabled at boot. Unit tests: the arch table has 5 rows, the
apply test 1; 1262 in all.
The kernel module's boot-time cleanup, linux-modules-cleanup.service (Type=oneshot,
declared stopped and enabled at boot), failed its delivery on ace with 'was asked to
be stopped and is running': ServiceState read every inactive one-shot with
ExecMainStatus=0 as running, and systemd says 0 for one that never ran too. Stopping
an inactive unit changes nothing, so the read back said running again; after a boot
that ran it, the same declaration would fail every apply.
ServiceState now reads ExecMainStartTimestampMonotonic: 0 means never ran, read as
stopped (a package's one-shot declared running is then started, and the hold no
longer counts it as in use). The service apply reads a one-shot at rest as stopped
to a declaration that it be stopped (ServiceAtRest), whatever its last run did.
Quick fix weighed: declare the cleanup running in the catalogue. Thorough chosen:
the reading was wrong for every one-shot, and this is its second fault of the kind
(the firewall's was the first). Unit tests: three arch one-shot tests merged into one
table (7 rows), one apply test (2 rows); 1263 to 1262.
Met twice on 2026-10-28: shanks kept 'want.shanks.brother could not be read ... no
responders' as the own-secret step's last word, and ace kept 'the vault's own: not yet
... no responders' as its answer to seed-7, each until another input moved. Every bus
read of the composer (manifests, builds, archives, served values, the vault's own and
credentials, the have of what runs) is noted (BusFailed); an answer or a pass that met
one is made again after a growing bound (5 s to 5 min), and a pass is not handed when
the have of its sealed values did not answer. The own-secret step tells a read or an
ask that did not answer (Retry, taken again) from a key absent or the vault's word.
On shanks probe/1126-seed-7 was answered at revision 84 every 13-17 s: each look rewrote
the haves of its unhealthy modules with new words, and the shadow took any write of its
own module haves, placement, modules or config values as a change. Each answer loaded
the bus, the health tools timed out, the words moved again. Now only what an answer is
composed from wakes it (what runs, folded modules, a changed value), and each group is
answered again only when its keys' revisions or the node's inputs moved.
A new step, internal/ownresync, after each shadow answer: for each own secret masked as another value, while the node
applies the controller's compositions and runs the one its want names, it asks the vault's move-in with
replace_running: true and the running value sealed to the call's key, once per value, and says what it asked and what
the vault answered. The answer written to the node's have key carries values_differ. Pins the group's mesh-sdk head
1062b33 (go.sum is filled once that head is pushed).
A box is reused by slot (resource, field) after the composition, from what runs or the
have's slots; a have written before slots is reused within its module alone. Config
values are laid by mesh-sdk's one rule (never a default). The shadow answers the
newest candidate first and hands each answer as it lands, says when an answer starts,
every minute what it waits on, and when it ends, and fits an answer to the node's key.
Needs mesh-sdk fix/1126-seed-same (16b8d8a1).
mesh/check pass 047237c911a4 on 449cda1b003e | unit 2980 run, 270 kept, 36 skipped | integration 2999 run, 302 kept, 29 skipped | over bound: unit (sl…
ace, g14 and shanks answered nothing to seed #531 and said nothing either. The
shadow now says, after a minute not ready, which inputs it has not read, and
answers each group it knows refused for that; an answer running over a minute
is said while it runs and noted in the answer. The answer's differing resources
carry both bodies, masked, the node saying whether two sealed values open to the
same value (Composer.sameValue: only the comparison leaves the node).
mesh/check pass a6e53a485813 on 199f6b48b1df | unit 2992 run, 1008 kept, 36 skipped | integration 3025 run, 1102 kept, 29 skipped | over bound: unit (…
NotComposed marked from a read have whatever its phase: an apply that failed,
naming the want's build and digest, became a composer's mark, which baseline
then took for what runs, and never applied again. It now marks only a have that
runs, says nothing, or is a composer's mark already. And its loop, whose reads
can take seconds, checks the run's context at each module, so a run that ended
writes no have.
Live 2026-10-28: the own run's last pass, failing on its run's end, marked
every module of the controller-mode agent that replaced it (ace 60, shanks 63),
and the marks dropped the build and digest because the have read failed in the
bus's reload pause. Now: the refusal hook is bound to the run's agent and ctx;
a pass whose context ended returns nothing; JetStream not enabled or no
responders are tried again after 1, 2 and 4 s before any mark; NotComposed reads
the have again while the bus does not answer and marks nothing it cannot read;
baseline takes a have a composer marked as what runs, so a mark it did not make
clears; each run's shadow lives with the run.
mesh/check checking 463626143567 on bb15aab50eda | novox/mesh-host#154 in group fix/1126-shadow-all
selfComposer took a KVHistory error for 'not placed': at any restart a timeout
or a bus not ready flipped a self-composing node to the controller's
compositions for a whole run. selfcompose.Placed chooses only on a read that
succeeds (a missing placement state is one), and tries again otherwise, said
once. The shadow's answer to a head deleting the placement reads the
controller's composition at each want in the node's bucket and compares it with
what runs; a want with none refuses, without returning.
The seed's revert (mesh-catalog #526) was refused on 'would stop running node',
while a self-composing node would have refused its placement's absence, kept
what runs and marked every have failed until a restart. endWhenUnplaced ends the
agent's run when the key goes, so it begins again on the controller's
compositions, as at a start without a key; the composer refuses nothing while
unplaced; the node's have says whose compositions it applies (Composes); the
shadow answers a deleted placement Returns, verdict same.
novox refused its own composition and its answer on de-spiegel, a module it runs
folded into its own composition at a build only the controller holds. The
composer now gives frombus what the node runs (Scope.Running), so such a module
is carried as it runs. The shadow's answer names every refusal (mesh-sdk). A
templates archive read is bounded at 30 s: nothing else held an unanswered store.
H8 found it: a module the node could not compose yet got a failed have naming
no fingerprint, which counted as run inside the node's own composition, so the
next composition took it from its manifest past its build.
The node read a behind module's source manifest digest and composed every module
from the indexer's raw manifest; frombus now takes the build itself (Scope.KeptAt,
a fingerprint). The reader gives frombus the builder's key, a templates archive
fetched as every archive is (apply.Fetch) once per digest and checked against it,
and another node's served keys one by one, as the node's grant allows them.
The fold's watch of every have key ('>') was refused (permissions violation on
CONSUMER.CREATE KV_node-engine_have), and every self-composing agent stopped
after the seed. The composer now watches '*.node' (records) and '<node>.>' (its
own haves), as buscompose grants a node; the shadow watches '<node>.>' too. The
selfcompose tests check every watch and get against PermissionsFor(KindNode),
and the agent's integration tests run the agent as the node under that grant.
mesh/check checking 69b845c0f7b2 on 0d22776b20ef | novox/mesh-host#152 in group fix/1126-fold
A fresh agent composes before it has applied anything (Running empty), so
InOwn of Running folded only mesh-wireguard and the switch would drop nats.
The composer now watches the node's haves: a placed module whose have names no
new-form fingerprint is folded; one with a new-form have whose want is removed
is not folded. The shadow uses the same set, over the candidate's placement.
Pins mesh-sdk 05b67831, whose compositions fold a module no new-form build
names into the node's own. The shadow composed a placed module it runs inside
its own composition (nats, photos) at the trunk's build, as if the candidate
added it; shadowBuilds now leaves those to the fold, and both paths pass
frombus.InOwn of what runs.
mesh/check checking b225c7bb4cf5 on be94d2689e2f | novox/mesh-host#150 in group fix/1496-start-health
On ace, 2026-10-26 21:19, mesh-build-node's cache directory was there before the
mesh and used as found. The health statement says such a directory unhealthy, and
the have folded it into the module's health as failed, with no words, while the
module's own check answered healthy; the assigner read it as its canary failing
and held the version on all four nodes.
The directory waits for the operator's hand-over and keeps its own condition,
raised from the same statement; it no longer counts in the module's have. A have
whose health is failed or waits and whose apply said nothing carries the words of
the resource that made it so, so the next one is readable.
mesh/check failed: unit: ..test: ✔ a transient empty list for four passes retires nothing; five stable passes retire, data kept (290.408814ms) | 58de6…
mesh/check failed: ace: novox/mesh-catalog#505 does not merge into the trunk: files in conflict: .gitignore | cdd3f3fd4f63 on ff9386dcccc2 | unit 1245…
mesh/check failed: ace: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk 91413de33516, which is neither on mesh-sdk's trunk nor the grou…
mesh/check failed: shanks: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk 91413de33516, which is neither on mesh-sdk's trunk nor the g…
mesh/check failed: shanks: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk f1d7c7377dc4, which is neither on mesh-sdk's trunk nor the g…
mesh/check pass 65cd7b47873a on bba9081ca127 | carried from 94f9aa298cb8, mesh-sdk 21252194d530 | unit 971 run, 423 kept, 32 skipped | integration 976…
A process no witness judges had its directory removed before the new bundle
was unpacked and the unit restarted, so the old process ran on in a deleted
working directory: mesh-build-go's `go version` probe failed with "getwd: no
such file or directory" and refused three checks on ace on 2026-10-26, each in
the second a new build of it arrived.
Every process is now replaced the way a judged one is: its build is renamed
under .witness/<name>/retired/ and deleted only once nothing runs from it.
"Runs from it" now counts a process's working directory as well as its image,
so the children a daemon starts keep the directory too. Forget keeps a retired
build that still drains, and the witness's sweep covers every process it keeps
a directory for, judged or not.
Two unit-file tests are folded into one, so the unit count goes down by one.
The share belongs to the part that enforces it: build-cpu-share, default 50,
set per node through the node's settings layer and read by the composer for
the build seats' runtimes. Pin mesh-sdk at fix/build-cpu-yields' head.
Review round 1: one quota per unit let three build modules take three shares.
The engine now writes meshyield.slice, top-level so its idle weight binds
against the operator's session, with the node's share as its CPUQuota; each
yielding unit joins it at nice 19. Comments say the IO controls bind only under
bfq or io.cost, not under the none scheduler NVMe disks use.
mesh/check pass 43f2d6020e11 on 94f9aa298cb8 | carried from 7f04d1204765, mesh-catalog a4f5df797b68, mesh-sdk b4cd95e7c4a4 | unit 972 run, 423 kept, 3…
On the bus node the bus module has no want and has left the declaration, so no
pass of this engine held its fact and the follower's user list was kept in
memory and never written: the vault and the grants part read no hashes and no
key. A module's user list now reaches its key on every pass, merged into what
the key says with every other field kept; a missing key is said once, never
made up (ADR 0547 §8).
A build seat's runtime took 1650% of shanks' 24 processors and slowed the
operator's desktop. The unit of a process declared with yields now carries
CPUWeight=idle, IOWeight=1, Nice=19, the lowest best-effort IO level, and
CPUQuota for its share of the processors this machine has.
Merges fix/1945-adopt-hashes 1a190090. A new follower (one per link) waits for the grants part's key to catch up before
its first write, so a restart after take-over never writes the controller's list first. A refused or ended watch is
tried again on link.Backoff, the have says why it is not watched, and the key last heard, kept beside the node's state,
holds meanwhile. While the grants part's list is followed, the bus module's composition is applied with the list on the
machine in place of the controller's. The grants list is never handed to the vault. Pins mesh-sdk c2a410bb.
On the node whose compositions carry the bus's list, the engine also watches mesh-grants' key user-list.sealed. While it
says follow grants, the object it names is read by name, opened with this machine's sealing key, checked against the
key's digest and the guard, and written to the same file at 0600 only when its digest differs; the bus reloads it
itself. The have says which list is followed and this machine's sealing key, which the grants part seals to.
The follower's test waits for its watch to catch up before a later put, which raced under load. Pins mesh-sdk 175a5567.
The vault holds no value for the controller's own account, the node accounts and the enrolled devices, so it cannot
hash them, and take-over waits on their hashes. The bus node's engine is the one part that has them, in the list the
bus loaded. Its follower of the user list now puts every loaded user's bcrypt hash in the bus module's have, sealed to
the vault's own key (mesh-sdk delivery.AccountsHanded), so of every reader of the have bucket the vault alone opens
it; the clear part says only the list's digest, the key's fingerprint and how many users. The vault's key is heard from
its answer to node-key, which the engine already asks at start; a first or changed key looks at the list again. It is
sealed again only when the list on the machine or the vault's key changes; a refused list hands the list kept on the
machine. The vault takes from it only the users it holds no value for.
Also: the user list follower's test sent its later puts before the watch had handed the keys there before it, so two
of its cases failed on main whenever the goroutine started late; the feed now says when the watch began.
mesh/check pass 99285cb080db on 1195cf45971e | unit 303 run, 1080 kept, 5 skipped | integration 306 run, 1082 kept, 4 skipped | over bound: unit (slow…
The bus node's engine writes the bus user list (ADR 0547 bridge); each write makes the bus reload its accounts, and
for about three seconds every JetStream request meets no responders. An apply that failed then was kept failed until a
key changed (mesh-vault on novox, 2026-10-25 23:04:24), holding every module that needs the secret seat. A transient
bus failure is now retried with a backoff (2 s, 5 s, 15 s, 60 s, then every 5 min), the have saying retrying meanwhile;
after the engine writes the user list, the transiently failed modules are tried again once the reload is over. A have
write that met it is written again the same way. shorten keeps the head and the tail, the cause.
mesh/check pass 5e71ab6ae612 on d75c66dc7e27 | unit 303 run, 1077 kept, 5 skipped | integration 306 run, 1079 kept, 4 skipped | over bound: unit, inte…
B1: the controller sends the user list by digest (over 64 KiB), so its users are counted from the list as resolved, not
from the composition's inline content, which is empty then.
R1: a composition signed for another node or module than its key names is no source.
R2: a list without the controller's own user, or one that drops more than half the users of the file on the machine,
is not written; the have says why and the file on the machine is kept.
The list reached the bus only through a want of the bus module, which nobody writes (ADR 0539 §15): on 2026-10-25 it
stood from 11:23 to 20:50 and was put back by hand. The node agent now follows its composition bucket for the newest
signed composition carrying the user list, writes that file alone through the apply's file step when it differs
(mode 0600, read back, recorded as the module's), moves nothing that reads it (the bus module's entrypoint reloads the
server in place), never goes back to an older list, looks again after every apply of the bus module, and says source,
revision, wanted and written digests, users and any error in the bus module's have (mesh-sdk UserListWritten).
Read with mesh-delivery-view.node_haves (module nats); the bus's own nats_user_can says in_force once it reloaded.
mesh/check pass 338797a52703 on d75c66dc7e27 | unit 2301 run, 3410 kept, 10 skipped | integration 2326 run, 3431 kept, 13 skipped | over bound: unit, …
A write judged by the declaration before the apply settled could record its
refusal just after Knock found nothing owed: the original fault in a small
window. Knock now holds the apply's lock as a write does, reads no declaration
while nothing is owed, and its comment names Settled as its caller.
ace's filling refused mesh-build-go's files as not named as filled 32 s before
its want arrived; after the apply that named them nothing asked again, and the
runtime crash-looped on its missing broker credential. The filling now keeps
what it refused or failed and, when the node agent says a module's apply or
removal settled (agent.Env.Settled, called once what runs holds it), touches its
write socket if the declaration now names such a path, names the provisioner's
account, or a write had failed; the client watches that socket and fills every
module again. Said in the journal each time.
mesh/check pass b32a27c115a2 on 0dfbe1a179f2 | unit 292 run, 1060 kept, 5 skipped | integration 295 run, 1062 kept, 4 skipped | over bound: unit (slow…
The facts look inside an apply runs while the module is still applying and judges nothing; the
running write then left xorg running and unknown until the five-minute statement, holding every
x11-display consumer behind it. The running write now nudges the health loop, which looks and says
at once. A declaration's report gets the same motion and composition check, and a removed module
leaves nothing marked unjudged (review of fix 15).
A have said applying and failed for the controller on novox at 15:27:36, from a look at the
container being replaced, two seconds before its launch said healthy; the assigner held the
canary on it and set novox back. A look now judges only a module that is not in motion, and a
module whose composition moved carries unknown until a look after the move.
xorg's have on g14 and shanks kept the unknown an engine before 958ccbd wrote: the
baseline believed the have and never handed the module to the session, so no look
rewrote its health, and the assigner held every module requiring x11-display back.
mesh/check pass 1bdbf47ffa83 on 0dfbe1a179f2 | unit 302 run, 1044 kept, 5 skipped | integration 305 run, 1046 kept, 4 skipped | over bound: unit (slow…
A unit or container of the node's own that shares only its name with a
module's resource is not the module's: dropped, its record became an orphan
the apply removes, stopping the module's unit (1945/AC17).
The node's own composition and a module's both declared the same files, and
the joined declaration was refused whole ("2 resources declare X"): the
node's own copy is now left out when joining and at the node's own apply, and
said once (1945/AC17). A module still running from the kept declaration now
carries its filled paths, so the filling no longer refuses them as not named
as filled (1945/AC18).
mesh/delivery-group group fix/r6-node-composition held: a member is held: novox/mesh-sdk@10b6e202320f (merged without a passing check: only a person decides wh…
The build agent left mesh-controller with R4g (ADR 0524 section 5); genesis still installs it until issue 1152 decides where a new mesh's builder comes from. The tests that read its manifest say NOT JUDGED and are judged again with 1152.
mesh/merge-gate head 0d12fb11a48b merged into trunk 59e6a6fa2a4a — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/check failed: mesh-build-go: no go build node qualifies: builders disagree: go1.27.1-X:nodwarf5 recipe 74744709d277ded9 on ace, g14, shanks; go1.…
On ace the postgres container was replaced at 22:11:51 mounting superuser.secret, a
file the provisioner's client fills only after the apply. The runtime made the
missing source a directory, and every fill after it failed with "is a directory".
A container whose mount source is a file the declaration declares, not yet on the
machine, now waits and is neither created nor replaced. The write of that file
(WriteFilled) applies what reads it, as before, so the container is made once the
file is there.
What the runtime already left is cured by the mesh: a declared file whose path is
an empty directory in the mesh's own tree (by its path and by where it leads),
owned by the engine's account, has that directory removed before the file is
written, and the outcome says so. A directory with anything in it, or outside the
tree, is never removed.
A root-only link inside /var/lib/mesh pointing out of it would have made a
directory outside the tree, owned by the engine's account, the mesh's to give
away: the fault of issue 1873 again. The path and the tree's root are now both
resolved, links followed, and the directory is the tree's only when the resolved
path lies below the resolved root as well.
postgres's data directory moved to /var/lib/mesh/postgres. On ace and novox that
directory was there, root:root 0700, before the directory resource was applied, so
the engine used it as found (issue 339's rule) and left it to root: the postgres
provider, running as the operator's account, cannot reach its own secrets,
consumers and grants there.
/var/lib/mesh is the engine's tree (root, 0755): only the engine's account makes a
directory in it. A directory below it owned by that account is now the mesh's and
is given its declared owner and mode. Anywhere else, or owned by another account,
issue 339's rule stands.
mesh/merge-gate head 8c0bb9d4531d merged into trunk 4065b45ae84f — pass: group fix/1873-system-state-dirs composed: every machine composes with the change …
The nats module takes its server as an upstream artifact, pinned by digest
(issue 1641), and builds no image on a NATS_BASE argument any more. The test of
the bundle's bus read that argument, found none, and failed every mesh-host check
against the catalogue's main. Bootstrap's own code never read it: the bundle's
bus image comes from the bundle, so genesis was not affected. The test now
compares the bundle's image with the module's server by repository and digest.
The postgres module's own secrets move from /var/lib/postgres, the postgresql
package's directory, to its directory under /var/lib/mesh (mesh-catalog,
fix/1873-system-state-dirs). Genesis writes the store's superuser file at the
module's own-secret path, so the store it raises is the one the module adopts in
place: StoreSuperuserFile follows, to /var/lib/mesh/postgres/superuser.secret.
A directory genesis makes on the way is 0755, the mode of the mesh's directories,
and only the credential's own is 0700: MkdirAll at 0700 would have closed
/var/lib/mesh on a machine that had none yet.
mesh/merge-gate head 09c77d57b823 merged into trunk c231d1031fc7 — pass: group fix/1732-held-moves-asked composed: every machine composes with the change a…
mesh/merge-gate head e597f9791c62 merged into trunk 7e83137aa822 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/delivery superseded: its walk was superseded and names no walk that took it over: a walk that took it over answers its merge, and it follows that wa…
mesh/merge-gate head d9f8bbec0c0b merged into trunk fd3259a3f5d7 — pass: group feat/r4d-groups composed: every machine composes with the change as it did w…
mesh/merge-gate head fdb59bcf4eb7 merged into trunk c5c4caa2a4ab — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head fdb59bcf4eb7 merged into trunk c5c4caa2a4ab — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/merge-gate head 5a2bc0391054 merged into trunk ce1b97ad973d — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/merge-gate head 52600ec7bd61 merged into trunk cdcba9ca7375 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
The asking helper registered a.Close, which closes the shared test connection; the stand-in's cleanup ran after it on a closed connection, so its consumer was never deleted and blocked the next test on the workqueue stream.
The stand-in shared one durable name across tests and let go with Unsubscribe alone, so a test starting before the server dropped the binding was refused with "consumer is already bound to a subscription".
mesh/merge-gate head 150fadfe8bed merged into trunk 619e14aa8e57 — pass: group fix/r3e-before-the-gate composed: every machine composes with the change as …
The set of modules this process wrote lives in memory, so a node-engine restarted before the apply that drops a module never wrote that module's removal. The report now names the modules the kept declaration held before, and they are removal candidates.
A module that left a node kept its have key at running for ever, so nothing could tell it stopped and the gate would never take the seat. The key is written once with the phase removed over the revision read, never deleted. Pins mesh-sdk to the commit that adds the phase.
mesh/merge-gate head 64dfa90c3a6c merged into trunk 454be9b2693d — pass: group feat/r4a-sdk-first composed: every machine composes with the change as it di…
A build module's sandbox needs one, and a module can require only a capability the node-engine detects (ADR 0448 section 5). The probe drops to an unprivileged uid first when run as root, since root always can.
mesh/merge-gate head 671f2840046f merged into trunk 89025ad004d4 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/merge-gate head aab65b81f65c merged into trunk 6469e8238443 — pass: group feat/delivery-have composed: every machine composes with the change as it di…
The node-engine reads a resource's module and a declaration's builds, says it does in every report (reads_modules), groups the apply's outcome per module with every failing resource named, and writes have.<node>.<module> and have.<node>.node to the bucket only when a value differs, never ahead of or in place of the report (novox/hq ADR 0429 section 4).
mesh/merge-gate head dae79c76434c merged into trunk cd13f7726ef0 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
The merge check's build container has no pacman, so the acceptance test now stubs it and names the host it was built for, as other command tests do; nothing in the node-engine itself changes.
mesh/merge-gate head d237dee4c9e3 merged into trunk cd13f7726ef0 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
A directory or archive resource carries `unknown` (move-out, module, program) and `knows`. For move-out under a home, every look moves each path the declaration does not know into ~/.local/state/mesh-moved/<module>/<time>/, the list written before each move, without following a link, a young path waiting, a socket or pipe left and reported, nothing made when nothing moved, and another file system refused. Reviewed in two rounds.
ADR 0380's secret-given announcement asks operator-channel.tell over the
bus; the genesis bundle's first user list must already carry that
publish subject, in the same place the controller composes it
(internal/broker, mesh-controller ms-835 435468c1), or a first node's
controller starts without the grant until its own first composition.
mesh/merge-gate head 62af89f85d0d merged into trunk 1a2677882380 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
Every 15 s the failed-units look read the own manager of every account
the declaration creates, and reading one is a login that starts it: six
module accounts' managers on the control node started and stopped all
day, most of its journal. A look only reads (ADR 0240 rule 6) and never
asks an account's manager (ADR 0177).
An account's manager is read now only when the declaration places a
user-scoped unit in it, and only after the machine's own manager says
user@<uid>.service is active, as the apply asks before it. A manager not
running is said away in the units statement, with no failed units; in
liveness its units read unknown with the apply's words, never down. The
operator's account on a desktop is still read while logged in, since a
module places a user unit there (issue 315's Razer daemon stays said).
mesh/merge-gate head b9a293f8c8bc merged into trunk 0e92ef3d1b21 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
A restart-on trigger that is no file counts as changed only in its own pass,
so after a busy wait the container read unchanged and was never replaced,
while the report stopped listing it (review of #98).
The broker step sent the nats module with a plain push, and the controller refuses a push that would
replace the bus: that is a planned step, bus upgrade, which snapshots the streams first and checks
them after (ADR 0236). Genesis stopped at step 18.
The step now snapshots every stream with the module's own snapshot program (no backup holder exists
yet to be asked), asks bus upgrade with that snapshot and why, says the one restart, and waits until
the bus runs the module's image, answers, and the controller ends the step done. The bundle's bus runs
the release the module's image is built on (2.11.17, by the same digest, where it ran 2.10.29), so the
step changes no server release and is said reversible.
mesh/merge-gate head cb7b71dcc568 merged into trunk 12bc6a56a38a — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
A port reached from local-network is admitted from the private networks the machine is attached to, and only the machine can say which those are. Every network on a link facing outside is said among its facts, and a change is said at the next reconcile.
Genesis's base step built only mesh-tools, the TypeScript toolchain. The store's and the bus's modules
carry Go bundles, compiled in mesh-tools-go, so genesis stopped at the store step: "this mesh holds no
copy of it. Build mesh-tools-go first". The base step now builds mesh-tools-go too, from
--tools-go-source or the repository beside --tools-source, and a test holds every toolchain a genesis
module's bundles compile in, read from the catalogue, to the ones the step builds.
The packages step sealed the build agent its genesis document and pushed, and the push returned as soon
as it was sent. The sdk step then asked for the SDK's build at once, and the agent reads the document
from the file the node writes when it applies that push: one genesis found it in time, the next was
refused for having neither a binding nor the document. That one push now waits until the node reports
it applied it (push --wait); every other push still returns on sending.
The bundle raised the store as mesh-store, on upstream postgres, in a docker volume; the postgres
module keeps it as postgres, on its pgvector image, in /var/lib/mesh-store owned 999:70, reading its
superuser password from a file. The store step refused, and genesis stopped at step 17 of 24.
The bundle now raises the store with the module's name, image, data directory and password file, and
every step of the bundle and the installer reaches it by that name. The refusal to raise a store over
data it cannot open looks for a cluster in the store's directory, no longer for the old volume.
The bus reads its user list from a file that, until the broker step adopts it as the nats module,
only the installer writes, and the installer wrote it only at enrolment. The build agent's account,
minted with step 13's push, never reached the bus; the agent was refused (Authorization Violation)
and the sdk step's build waited until it was killed. Every push genesis makes now places the composed
list and has the bus re-read it, until the nats module is registered (ADR 0331 §5, ADR 0333 §4).
The build agent declares neither the retired builder's npm-password own-secret
nor a serves setting, so the packages step handed its credential to nothing
and genesis's sdk step had no way to publish. The packages step now seals the
genesis-package-registry own-secret to build-agent: the forge on this node's
loopback, the port given for the package registry, its npm path, and the
mesh-builder account with its password. setFoundationSettings no longer gives
the builder a serves.port.
Genesis's builder step read modules/builder from the catalogue, which retired it on 2026-10-03 for
the build agent (node-build-agent on every machine), so no genesis got past step 13. The step now
installs build-agent from the catalogue, puts the builder image it carried and published where the
manifest names the image it builds, and issues its bus account as any module's.
The controller reads its merge window's settings from a file its manifest declares from its
settings (MESH_MERGE_WINDOW_FILE, ADR 0276). Step 11 took every MESH_..._FILE for a secret and
refused this one as a file nobody writes, so no genesis got past step 11. A _FILE naming a file
resource of the module is now the module's own; one naming neither an own-secret nor a file of the
module is still refused.
The bundle raised the bus as mesh-broker on docker volumes; the catalogue's nats module keeps it as
mesh-broker-nats on directories of the machine, so the broker step refused rather than raise a second
bus, and genesis stopped at step 18 of 24.
The bundle now raises mesh-broker-nats with its configuration and user list in /var/lib/nats-module/conf,
its streams in /var/lib/mesh-broker-nats and its certificate in /var/lib/mesh-broker-tls, published on the
node's bus port. The broker step holds that layout to the module's manifest field by field before
registering anything, says the one restart onto the module's image, and waits until the bus runs that image
and its monitor answers. The installer's bridge writes the composed users at the module's bus-users path and
signals mesh-broker-nats. The bundle's bus configuration and user list are seeds, so a re-run after the
adoption does not put the bundle's one user back over the module's.
mesh/delivery-group group feat/335-a-busy-container-is-replaced-when-idle rejected: its order contradicts itself: novox/mesh-catalog@190608193764, novox/mesh-c…
A send that replaced the build agent killed the check it ran, and the queue
started it again elsewhere (issue 594). A container may now declare busy; the
node-engine asks it before replacing it, applies everything else, says the
wait, and replaces it on the first idle pass or at the bound, kept across
restarts (novox/hq ADR 0335).
mesh/merge-gate head c7b534a08c98 merged into trunk 6fe82c61bd32 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
A module's own directories need setgid so what is made beneath keeps its own group,
and a mode of 2750 refused the whole declaration while every apply dropped the bit:
a directory's mode now carries setgid (never setuid or sticky), set, kept and said.
A tool runner's account that is missing or root's failed the whole apply and dropped
the library's outcome: it is now a warning, said as runner:<module> or
runner:library:<name>. And a runner that leaves a group is taken out before the
resources apply, so its unit is restarted in that apply to lose it.
A joining module's own directories belong to its own group, and its tools run in the
tool runner, which could not read them. An identity may now name the tool runner's
account: the engine puts it in mesh-<module> as it puts it in a library's group,
says the change as identity:<module> for the runner's unit to restart once, takes
it back when no identity names it, and refuses an account that is missing or root's.
The engine states contract 7.
A re-run's apply writes the bundle's one-entry user list over the list placed at
enrolment, and an enrolled machine skipped placing it: enrolled was taken for adopted,
against ADR 0331 §5. Until the controller knows the nats module, the installer places
the composed list again. Also drop the previous broker's AMQP and management ports,
which nothing raises any more, and say why the bus always carries a ports setting.
The installer could not raise a first node: the old bundle's control plane had no
MESH_BUS_NATS and crash-looped, and the NATS bundle was refused because rootsecrets
only knew the previous broker's guest:guest@ template.
One bundle now, the NATS one. rootsecrets makes the controller's bus account and puts
it in both placeholders (the address and the bcrypt hash in the carried user list),
kept with its hash so a re-run writes the same bundle. Preflight refuses a template
naming no bus connection it knows, naming the placeholder, instead of waiting out the
verify step. The broker step adopts the bus as the nats module and refuses, by name,
a catalogue without it or a server that is not the foundation's container.
mesh/merge-gate head 281837847e03 merged into trunk 303d4037e84e — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
An earlier change dropped alpine's RemoveAccount and RemoveGroup, so alpine made no
module identity at all. The sample beneath a root in shape had no test of its own.
And an apply that found the library's lock held read the library as kept even when
what held it was a take act about to be refused for a missing disk; it now does so
only while a take walk is running, and otherwise waits and judges as always.
usermod --gid regroups every file under an account's home, and accounts were made
with / as home: they now have /nonexistent, and an older account's home is moved
there (without its files) before its group changes. A sub-path asked for later that
holds data not in the group waits for the operator as the root would. The take act
gives the group to hardlinked files an account owns (the *arr apps' downloads) and
leaves root's; a file's setuid or setgid bit the regroup takes is never put back, so
no setgid file hands out the library's group; each is counted. An apply no longer
waits for a running take walk.
A root in the group with setgid was kept without a warrant even when the data beneath
it was not in the group, which §7 says the operator is asked about. A root is now
kept without asking only when it is empty, or it and a bounded sample beneath it are
all in the group with group write (and setgid on folders). On busybox, an account the
mesh made is given its added groups with addgroup and delgroup; a change of its
primary group is refused, saying why.
ADR 0328 supersedes ADR 0325: shared data is a library with one home, kept usable by
its group rather than by ACL entries. The engine now makes each library's group and
each module's user in its libraries' groups; keeps a library's root and requested
sub-paths in its group with group write and setgid on its home, making missing
sub-paths as the root is; waits for the operator's word before touching a root that
holds data, and on the take act walks it by descriptor (no link, same filesystem,
bounded and resumed, files with other names left); holds the containers of a library
whose disk or mount is not there; and puts the tool runner's account in the group.
The ACL keeping, mask repair and watch are gone; an access is again as before ADR 0325.
mesh/merge-gate head 5c8e768ef520 merged into trunk ef26dba2753a — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
getent passwd reads an all-digit key as a uid, so a tool runner's account named
"1000" resolved to whoever has uid 1000 and the entry went to them. Only an answer
naming the account asked for is taken.
The controller cannot know the operator's account's uid, names an access by its
declared id (which holds a slash), and should not parse words. A tool runner's
account may now come by name alone and is read here (refused when absent or root's,
its uid said in the access's listing); the take act matches the declared id exactly;
an access wait says its reason (keep or disk) and, for a filesystem that refuses
ACLs, whether the take act can enable them.
mesh/merge-gate head d12bba01090d merged into trunk dd9ab7927d96 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
The build seat runs merge-check as root, so every folder a test made was root's
data, which is never kept (§12), and 28 tests failed for that alone. A test's
folders now stand for an operator's: run as root, they are given to nobody before
they are judged. The root rule is unchanged.
mesh/merge-gate head c85c2804a010 merged into trunk dd9ab7927d96 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head c85c2804a010 merged into trunk dd9ab7927d96 — fail: its merge-check.sh failed: --- FAIL: TestAnAccessTheMeshMayNotKeepYetIsSaidAsAWait…
The engine delivers before the controller half, and today's controller sends no
identities, users or taken. Keeping now starts only for an access whose folder has
users, or a declaration with identities; every other access is mounted when present
and refused when absent as on main, with nothing made, kept, held or judged. A
leftover whose walk could not run says why, not always that its disk is away.
A name read that timed out gave an empty name, which differed from the recorded one
and so reset the record: a folder kept while empty asked again, a warranted one was
walked again. An unnamed filesystem now leaves the folder exactly as it is, the
access is said not known for that look, and it resumes once it is named.
NFS and FUSE report a zero f_fsid, so another export or another FUSE mount at the
same point was named alike and kept without a warrant; xfs's f_fsid follows the
device number. A filesystem is now named by its UUID (its device matched in
/dev/disk/by-uuid), else a nonzero f_fsid, else mountinfo's source; one that cannot
be named is never kept or taken. Entries left on a filesystem not back within 30 days
are forgotten, said once, and the naming read is shared per mount.
An ext4 or xfs library mounted by UUID can come back from a reboot as another device
(/dev/sdb1 as /dev/sdc1); naming the filesystem by mountinfo's source read that as a
move, stripped the warranted library's entries and asked for a new Approve. A
filesystem is now named by statfs's f_fsid (from the UUID, or the ZFS dataset), its
type and its mount point. Entries left on the filesystem a folder is on now are
merged back or held, never walked away, and a TakenOn naming a mount point is not
taken.
A folder kept because it was empty, then covered by its disk, was said as unhealthy
leftover entries for ever: its entries sit on a hidden empty folder and grant nothing,
so they are now forgotten, and only entries walked onto data under a warrant are kept
to be taken out. A warrant and a kept folder now name their filesystem by mount point,
source and type, so another pool at the same point is a move; records naming a mount
point are read as its name. A record with no access ids is said under a path-free name.
A container was still created over an unmounted mount point, so its writes landed on
the filesystem below; a folder moved to another disk was kept on the old warrant; and
entries left by a drop or a move that could not run lived only in a log line. Every
container mounting an access whose disk is away now waits, the warrant holds only for
the filesystem it was carried out on (the wait names the new one, path-free), entries
left on a filesystem are recorded per mount and taken out when it is back, and every
leftover is said as unhealthy until it is. A hung stat is shared per path, and a
widened mask takes from other entries only the bits it adds.
A walk run while a dataset was unmounted wrote onto root's empty mount point, or
"dropped" entries from it and forgot the ones on the data; a folder kept while empty
below an unnamed mount was then swept whole once the disk mounted, with no warrant.
Every walk is now judged as an apply is, a changed mount judges the folder afresh, and
systemd .mount units and mounts seen are read for the first-sight rule. Widening a
mask caps the entries it limited, so the owning group gains nothing. The watch lets
go of removed folders, re-arms on lost notice, reads users under its lock and ends
with its context. A drop that cannot read an ACL is counted, a broken folder's words
carry no path, a grant's group that could not be left is tried again, and a folder's
stat is bounded.
A module's containers ran as whatever their image chose and shared folders were
made writable by putting accounts in the folder's group (ADR 0312), which ADR 0325
supersedes. The engine now makes each declared module identity as a system account
with the controller's numbers, runs a container declaring `identity` as it, and keeps
each placed shared folder usable through one POSIX ACL entry per user, set through
the kernel's xattrs on descriptors it opened itself. A folder holding data is kept
only after the operator's warrant (the take act); a missing folder is made where its
data lives and waits where its disk is not; root's data is never kept. A watch and a
bounded sweep put back what an app narrows. The group grant is removed, and what it
put an account in is taken back.
Reuses the descriptor walks, the create act, the missing-path wait and the warrant-
backed act message from the paused issue 567 branch.
mesh/merge-gate head c7a4bf5c11d6 merged into trunk d8e785bb4b4b — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
Go's monotonic clock stops while a laptop is suspended, so a day judged by it could pass a day and a half
of wall time before the machine said what it is. And a hung statfs left a goroutine and thread blocked per
apply; now one check runs at a time and reports after it say the sizes unknown until it ends.
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
The mesh knew a machine's architecture and kernel family and nothing it is made of, and heard even
that only when it sent the machine something. The profile now carries the machine's memory, each local
filesystem's size and free space, processor model, cores and threads, distribution and version, kernel
release, virtual or physical, boot time and timezone; a fact it cannot read is named unknown with why,
never zeroed. The reconcile says the profile unasked once the mesh last heard it 23 hours ago, so on a
machine nobody pushed to nothing the mesh holds of it is older than a day.
The fix is an order, and nothing failed if the trust call moved after the
push or went: steps 7 and 8 are now one function a test drives against a
runtime that refuses an untrusted push. Preflight now refuses, before
anything is raised, a runtime file that is no JSON object, a docker.service
that is not loaded and active, and a dockerd started with
--insecure-registry, which would refuse the reload. The comment names the
order the docker module's takeover rests on.
Genesis pushed the control plane's image to the mesh's plain-HTTP registry
before anything told the container runtime to trust it; the docker module
that writes that trust is applied long after. Genesis now writes the trust
itself at the registry step, with the same into-json writer and a reload,
for the address given and the node's mesh name on the node's registry
port, and refuses before any push unless the runtime says it trusts each.
The docker module later finds the mesh name there and takes it over
without a change.
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
A mesh directory outside every home was chmodded and chowned by path, so a
parent turned into a link between two applies carried root's chmod and chown
to the link's target. It is now made and changed through the descriptor of
the walk from / that follows a link only root alone could have made.
A directory that walk refuses is a refusal like the placement guard's
(review 47): judged before anything is applied, and again when the
directory is applied, used as found or not; refused at either point it gates
its module and refuses what is below it, so no file is written into the
link's target and the report does not say everything was attempted.
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
Once a mesh directory's mode and owner go only through a walk that follows no
link another account could make, a directory behind such a link is refused
and its module gated: on the control-node that could stop the controller.
The operator is to be told which directories that walk would refuse on every
node before it lands. So the engine judges the declaration it keeps with the
same walk, making and changing nothing: on a signed ask from the controller
(mesh.node.<node>.ask.judge-directories, held to the hand-over's checks) and
as `nox-mesh-host judge-directories` at the machine.
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
The engine is built without cgo, so a check by name missed a root group a name
service holds or one number carrying another name: the root groups are resolved
to numbers (now with kmem and shadow) and a number named nothing is refused for
containers too. A group without write is granted nothing (§2). The look reads
each path off the lock within five seconds, so a dead network mount cannot stop
every health statement. A grant's account may not start with a dash. A group a
record leaves for another asker is handed to it, so it is still taken back when
the last asker goes (§5). The scheduler skips paths issue 339's guard refuses.
The engine reads the group owning the accessed path at apply and gives it to the
containers and the tools' account an access's new `grant` names: --group-add for a
container (recreated with it), an account group for the account (recorded, taken
back as ADR 0252 takes back). The path is never changed; an access whose group
cannot write is stated unhealthy for the operator to decide; group 0 and the
root-granting groups are refused by name. The tool runner restarts through the
restart-on the controller gives it, because the access reports updated exactly
when the account's groups moved. Health says GrantContract (4), which tells the
controller it may send `grant`.
The controller is to ask mesh-delivery whether a merge's delivery
group still has a member to merge, so that a merge in no waiting group
is cut at once (ADR 0304, ADR 0306); its bus grant names the verb, and
the installer's first user list must too, as for times (212a3cc).
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
// -catalogue reads <dir>/modules/*/module.json and a module.json at <dir>'s root as the first case's manifests (a
// checkout or `git archive` of the catalogue repository at a ref: the command fetches nothing). -node adds a machine to
// compose; -declarations answers each machine's declared paths with their owners and modes.
//
// Run as a pipeline step it reads its input, a composer.Request at depth compose, from the file named by the environment
// variable MESH_RUN_INPUT and answers the run's output as its composer.Verdict on the last line of standard output
// (design 47 G A-167).
//
// It opens no network connection, bus, store or file beyond its standard input, the -request file, the file
// MESH_RUN_INPUT names and the -catalogue directory, and reads no other environment. Exit codes: 0 the verdict is written (a machine that does not compose is a verdict, not an error);
// 1 the request cannot be read or run, said on standard error; 2 the flags are wrong.
r.Cases[0].Verdicts=bucketstest.Raw(map[string]meshcatalog.VerdictsByDigestValue{"refused":{V:1,Module:"refused",Judge:"src2:j",State:meshcatalogclient.VerdictRefused,Problems:[]string{"it is wrong"}}})
// 1621/AC2 and AC4: the command imports, through every package of this repository it reaches, only the standard
// library, mesh-sdk, the contracts modules' pure packages (seats' types, clients' codecs), the delivery SDK's composer (the command's own request and verdict types) and catalog key codec (a catalog entry's key, design 47 G A-144), the builder's builds package (a build's value and its keys), the private network's overlay package (mesh-wireguard's own pure package: names, graph and
// generator, no bus) and this repository's own packages, and none that reaches a bus, a store, a network service or
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
returnnil,errors.New("this node composes none of its own modules now (it applies the controller's compositions, or its node agent is starting), so it keeps no entry")
{"container not running",fakeRuntime{units:map[string]liveness.Observed{"dnsmasq.service":up},containers:map[string]liveness.Observed{"box":{Found:true}}},nil,true},
{"container held by a window",fakeRuntime{units:map[string]liveness.Observed{"dnsmasq.service":up},containers:map[string]liveness.Observed{"box":{Found:true}}},map[string]bool{"box":true},false},
{"unread",fakeRuntime{err:errors.New("the service manager could not be read")},nil,false},
ifline:="this node's login is not followed yet ("+werr.Error()+"): it dials as it last did, and looks again in "+loginsFollowAgain.String();line!=said{
said=line
say(line)
}
}else{
said=""
foru:=rangeupdates{
ifu.Caught||u.Deleted||len(u.Value)==0{
continue
}
nl,err:=meshvaultclient.DecodeNodeLogin(u.Value)
iferr!=nil{
say("the key "+node+" of "+buckets.NodeLoginsRead+" is not read: "+err.Error())
continue
}
hear(nl)
}
stop()
}
select{
case<-linked.Done():
case<-time.After(loginsFollowAgain):
}
cancel()
}
}
// fetchLogin asks the vault, through its existing hand-out of a machine's own secrets, for the value of login, sealed to this
// node's key, and opens it with the sealing key; no new verb is asked for it.
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
iferr!=nil{// a writer that cannot be made is a fault of its configuration: loud
say("the node-engine cannot make its finding writer: "+err.Error())
return
}
iferr:=w.Start(ctx);err!=nil{
say("the node-engine's findings are kept in its state and not said as events until it is started again: "+err.Error())
}
f.mu.Lock()
f.w=w
f.mu.Unlock()
}
ifname:=f.own.Get();name!=""{
begin(name)
return
}
gofunc(){
ifname,err:=f.own.Wait(ctx);err==nil{
begin(name)
}
}()
}
varerrOwnNameUnknown=errors.New("the node-engine's own module is not yet named by its applied record or its first composing, so its finding writer is not made")
// settledEvent is where the status view says a module has settled on a node: its have reached its want, running and healthy
// (design 47; the view's `settled`, said once for a module on a node at one composition). The engine hears it for every module
// and tells its agent, which tries again, once per settled, a failed apply that waits on that module as the holder of the store
// its image comes from (design 51 *distribution's first delivery*, issue 1966 ENG-11). The engine's manifest consumes it (`mesh-delivery-view.settled`),
// and that line alone grants the engine's account the subject (TestTheSettledEventIsTheManifestsConsume); a refused subscription is said, and made again when the next link opens.
// foundation-first-node-nats.lock — what a machine must be before a mesh exists, on the bus being
// built (novox/hq ADR 0106, design 25).
//
// The same twelve steps as foundation-first-node.lock, with one difference that matters: **the mesh
// writes its own user list, and at genesis there is no mesh yet to write it.** So this carries the
// first one — the controller's own account, at a well-known bootstrap password, exactly as the store
// is reached at `postgres:bootstrap` and the old bus at `guest:guest`. It is rotated with those, and
// from the controller's first composition onward the file is the controller's to write.
//
// The accounts file is its own file beside the server's configuration, because the server's own
// settings belong to whoever raises it and the users belong to the mesh (design 25 §4). Both live in
// one directory, of necessity: an include path is resolved relative to the including file's own
// directory, so a server given an absolute one looks for it underneath that directory and refuses to
// start.
//
// No `verify` on the TLS block, deliberately — that setting makes the server demand a *client*
// certificate, and nothing in the mesh presents one: a host pins this server's exact certificate and
// authenticates with a password (ADR 0004, design 25 §4).
{
"declaration": 1,
"resources": [
{
"id": "container-runtime",
"type": "package",
"package": "docker"
},
{
"id": "container-runtime-running",
"type": "service",
"unit": "docker.service",
"state": "running",
"boot": "enabled"
},
// **A filter before anything listens** (novox/hq issue 054, ADR 0088). The store and the
// broker are adopted as modules later and so bind to every interface from the moment they
// start; the packet filter that governs who may reach them is a module too, installed a
// dozen steps later. Between the two, a control-node facing the network had its store and
// its bus open to anyone who could reach the machine. So the foundation carries a filter of
// its own — the same table the filter module will replace wholesale once it can derive one:
// drop by default, keep loopback, replies, ssh and the mesh's own ports (the bus a node
// enrols over, the registry a node pulls from), and let the container runtime's own
// networks through the forward chain so containers keep working. A published container port
// is forwarded, never input (issue 047), which is why the forward chain is where the store's
// and broker's ports are refused from outside — and a container on this machine dialling a
// port this machine publishes reaches it through the runtime's proxy, which IS input, which
// is why the bus and the registry are opened in both chains, exactly as the derived ruleset
// does.
{
"id": "base-filter-package",
"type": "package",
"package": "nftables"
},
{
"id": "base-filter",
"type": "file",
"path": "/etc/nftables.conf",
"mode": "0644",
"content": "#!/usr/sbin/nft -f\n# the foundation's own filter, until the mesh derives one (novox/hq issue 054)\ntable inet mesh {}\ndelete table inet mesh\n\ntable inet mesh {\n\tchain input {\n\t\ttype filter hook input priority filter; policy drop;\n\t\tct state established,related accept\n\t\tct state invalid drop\n\t\tiif lo accept\n\t\ticmp type echo-request accept\n\t\ticmpv6 type { echo-request, nd-neighbor-solicit, nd-neighbor-advert, nd-router-advert } accept\n\t\t# ssh, from anywhere — never closed\n\t\ttcp dport 22 accept\n\t\t# the mesh's own, from anywhere: the bus a node enrols over and a container on this machine reaches through the proxy, the registry a node pulls from\n\t\ttcp dport 5671 accept\n\t\ttcp dport 5000 accept\n\t}\n\tchain output {\n\t\ttype filter hook output priority filter; policy accept;\n\t}\n\tchain forward {\n\t\ttype filter hook forward priority filter; policy drop;\n\t\tct state established,related accept\n\t\tct state invalid drop\n\t\t# the container runtime's bridge networks, and the networks its compose files are given\n\t\tip saddr 172.16.0.0/12 accept\n\t\tip saddr 192.168.128.0/17 accept\n\t\t# the mesh's own: the bus a node enrols over, the registry a node pulls from\n\t\tct original proto-dst 5671 accept\n\t\tct original proto-dst 5000 accept\n\t}\n}\n"
// Over TCP, not the socket. While the store initialises it runs a temporary server on the
// socket ONLY, then stops it and starts the real one — so a socket check passes, the action
// exits happy, and the verify a moment later lands in the gap and fails. The action and its
// verify must ask the same question, or the action can succeed into a state verify rejects.
{
"id": "store-ready",
"type": "action",
"in": "mesh-store",
"command": ["sh", "-c", "for i in $(seq 1 180); do pg_isready -h 127.0.0.1 -U postgres >/dev/null 2>&1 && exit 0; sleep 1; done; echo 'the store did not answer within 180s; its own last words follow'; pg_isready -h 127.0.0.1 -U postgres; tail -n 20 /var/lib/postgresql/data/log/*.log 2>/dev/null; exit 1"],
// Over TCP, not the socket. While the store initialises it runs a temporary server on the
// socket ONLY, then stops it and starts the real one — so a socket check passes, the action
@@ -99,21 +114,21 @@
{
"id": "store-ready",
"type": "action",
"in": "mesh-store",
"in": "postgres",
"command": ["sh", "-c", "for i in $(seq 1 180); do pg_isready -h 127.0.0.1 -U postgres >/dev/null 2>&1 && exit 0; sleep 1; done; echo 'the store did not answer within 180s; its own last words follow'; pg_isready -h 127.0.0.1 -U postgres; tail -n 20 /var/lib/postgresql/data/log/*.log 2>/dev/null; exit 1"],
// keptStep takes the restore points a move of a module needs before it applies and restores the ones a revert needs: it
// answers whether the module is held (its have says why, and the running build is kept) and, after a restore, the catch-up
// of the point to call once the reverted build runs. A want that says to go on without a backup asks neither, and nor does one that answers a held revert live.
a.env.Say(fmt.Sprintf("%s starts %s on the live data as it is: its want (revision %d) carries the operator's answer %q to the held revert",module,fp,w.Rev,w.Want.Revert))
returnfalse,nil
}
hold:=func(wordsstring,interruptedbool){
ifinterrupted||ctx.Err()!=nil{
words=BackupInterrupted
a.env.Say(fmt.Sprintf("%s waits: %s; it asks again at the next start",module,words))
}else{
a.env.Say(fmt.Sprintf("%s is held before %s: %s",module,fp,words))
Snapshot:"snap-1",Why:"revert of db's cache from "+fp("a")}))
iff.asks[0]!=want{
t.Fatalf("asked %s, want %s",f.asks[0],want)
}
})
t.Run("a revert with no point of its form holds as a restore point missing",func(t*testing.T){
f:=&recoveryFake{changes:down}
b:=moveDB(t,f,"a","b",heldHave)
ifh:=b.haves.last("db");!contains(h.Detail,"restore point missing"){
t.Fatalf("the have %+v",h)
}
})
t.Run("a held revert, then the want with revert live, starts the older build with no restore, no aside and no catch-up (design 47 G section 710)",func(t*testing.T){
f:=&recoveryFake{changes:down}
b:=moveDB(t,f,"a","b",heldHave)
ifh:=b.haves.last("db");!contains(h.Detail,"restore point missing"){
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.