mesh/check failed: shanks: novox/mesh-catalog#505: checks/placement/go.mod names mesh-sdk f1d7c7377dc4, which is neither on mesh-sdk's trunk nor the g…
The share belongs to the part that enforces it: build-cpu-share, default 50,
set per node through the node's settings layer and read by the composer for
the build seats' runtimes. Pin mesh-sdk at fix/build-cpu-yields' head.
Review round 1: one quota per unit let three build modules take three shares.
The engine now writes meshyield.slice, top-level so its idle weight binds
against the operator's session, with the node's share as its CPUQuota; each
yielding unit joins it at nice 19. Comments say the IO controls bind only under
bfq or io.cost, not under the none scheduler NVMe disks use.
A build seat's runtime took 1650% of shanks' 24 processors and slowed the
operator's desktop. The unit of a process declared with yields now carries
CPUWeight=idle, IOWeight=1, Nice=19, the lowest best-effort IO level, and
CPUQuota for its share of the processors this machine has.
Merges fix/1945-adopt-hashes 1a190090. A new follower (one per link) waits for the grants part's key to catch up before
its first write, so a restart after take-over never writes the controller's list first. A refused or ended watch is
tried again on link.Backoff, the have says why it is not watched, and the key last heard, kept beside the node's state,
holds meanwhile. While the grants part's list is followed, the bus module's composition is applied with the list on the
machine in place of the controller's. The grants list is never handed to the vault. Pins mesh-sdk c2a410bb.
On the node whose compositions carry the bus's list, the engine also watches mesh-grants' key user-list.sealed. While it
says follow grants, the object it names is read by name, opened with this machine's sealing key, checked against the
key's digest and the guard, and written to the same file at 0600 only when its digest differs; the bus reloads it
itself. The have says which list is followed and this machine's sealing key, which the grants part seals to.
The follower's test waits for its watch to catch up before a later put, which raced under load. Pins mesh-sdk 175a5567.
The vault holds no value for the controller's own account, the node accounts and the enrolled devices, so it cannot
hash them, and take-over waits on their hashes. The bus node's engine is the one part that has them, in the list the
bus loaded. Its follower of the user list now puts every loaded user's bcrypt hash in the bus module's have, sealed to
the vault's own key (mesh-sdk delivery.AccountsHanded), so of every reader of the have bucket the vault alone opens
it; the clear part says only the list's digest, the key's fingerprint and how many users. The vault's key is heard from
its answer to node-key, which the engine already asks at start; a first or changed key looks at the list again. It is
sealed again only when the list on the machine or the vault's key changes; a refused list hands the list kept on the
machine. The vault takes from it only the users it holds no value for.
Also: the user list follower's test sent its later puts before the watch had handed the keys there before it, so two
of its cases failed on main whenever the goroutine started late; the feed now says when the watch began.
The list reached the bus only through a want of the bus module, which nobody writes (ADR 0539 §15): on 2026-10-25 it
stood from 11:23 to 20:50 and was put back by hand. The node agent now follows its composition bucket for the newest
signed composition carrying the user list, writes that file alone through the apply's file step when it differs
(mode 0600, read back, recorded as the module's), moves nothing that reads it (the bus module's entrypoint reloads the
server in place), never goes back to an older list, looks again after every apply of the bus module, and says source,
revision, wanted and written digests, users and any error in the bus module's have (mesh-sdk UserListWritten).
Read with mesh-delivery-view.node_haves (module nats); the bus's own nats_user_can says in_force once it reloaded.
mesh/delivery-group group fix/r6-node-composition held: a member is held: novox/mesh-sdk@10b6e202320f (merged without a passing check: only a person decides wh…
A module that left a node kept its have key at running for ever, so nothing could tell it stopped and the gate would never take the seat. The key is written once with the phase removed over the revision read, never deleted. Pins mesh-sdk to the commit that adds the phase.
The node-engine reads a resource's module and a declaration's builds, says it does in every report (reads_modules), groups the apply's outcome per module with every failing resource named, and writes have.<node>.<module> and have.<node>.node to the bucket only when a value differs, never ahead of or in place of the report (novox/hq ADR 0429 section 4).
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
A module whose only fault was a secret the operator had not given yet was
stated unhealthy, failed its first-node gate and was put back. The engine
now reads the tool's answer through mesh-sdk go/health and states such a
resource waiting, with what it waits for, on one look; a malformed wait
stays unhealthy.
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
A secret given at the controller's terminal through mesh-cli (secret accept … --from -) never reached the
line: the engine forwarded the words only. It now forwards what mesh-cli's standard input held (at most
64 KiB, the SDK's bound) on the same core request to the controller, says it nowhere, and sends none
with a follow of the same call. mesh-sdk moves to f047d0a.
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
A service running as the operator — the tool runner, a unit of the
account's user manager — shares the operator's user id, and was as much the
terminal on the control-node as a person typing. The engine now reads the
asking process's cgroup and sends its login session's scope
(session-<id>.scope under the account's own user-<uid>.slice), or nothing;
the controller makes nothing else the terminal. The SDK is pinned at its
tag now that go/v0.1.9 exists.
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
- A request that timed out was said as "the controller was not reached,
nothing ran", though the line may have run. The engine now follows a line
the controller answers as running, by its call, until it ends; when no
answer comes in time it says the line may have run and names the call.
"Nothing ran" is said only of a request no controller took.
- One account could hold the socket: accepting waited for a free slot, and
a connection could stall before its deadline was set. The deadline is set
at accept, before any lookup; each user id has two lines at once and the
machine eight, and one over is answered busy at once.
- A container running with the operator's user id shares the number, not
the machine: a peer whose mount or user namespace is not the engine's is
refused, and nothing is asked.
SDK pinned at the go/cli branch's newest commit until go/v0.1.9 is tagged.
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
The records send the operator to "the controller's terminal", and nothing
reaches it (hq issue 343). mesh-cli holds no credential: it asks the engine
on its own machine over a socket. The engine reads the asking account from
the kernel (SO_PEERCRED), never from the program, and asks the controller on
mesh.control.<node>.cli, its own node's subject, so the bus server says which
node asked. The controller decides what the line may do; nothing received
on the socket is run or applied here, and the socket has no network address
(ADR 0004 stands).
The local protocol is the SDK's go/cli package, at a pseudo-version of its
branch until that merges and is tagged go/v0.1.8.
An account could replace ~/.claude with a link to /etc and have the
node-engine chown, chmod or write through it on the next apply. Below a
person's or an agent's home every component is now opened without
following a link, and a link refuses the resource in words.
The root judge also reads doas and polkit rules, the container runtimes'
sockets with their ACLs, ACLs on the secrets, and setuid-root programs no
package owns, in the C locale; Judged and NotJudged write down exactly
what it covers (hq ADR 0266 review).
The mesh runs on the seat's bus alone (novox/hq ADR 0131, design 28 task 5.5). The host's old
dialling and enrolment paths are deleted with the switch that chose between them; a membership or
a token naming another bus is refused before anything is sent, rather than dialled on a transport
that no longer exists.
Step 3.5's first half, mirroring the controller's. A host says exactly two
things unprompted, and the difference between them is the whole interface:
a report must arrive, and a heartbeat must not be insisted on. So a report
goes through JetStream — it is the message the store-window guarantee is
about — and a heartbeat stays on core, because a heartbeat in a stream is
the mesh's least valuable message competing for retention with its most
valuable.
The host still imports nothing of the mesh's own (ADR 0005): this is its
own interface over its own libraries. It agrees with the controller because
a fixture holds both to one envelope, which is the only agreement that
survives two repositories.
Also recorded, where the next person reads it rather than in a plan: the
"newest wins" window narrows at the rollout and does not disappear. Last-
per-subject makes the catch-up half the stream's, and sequence orders them
definitively — but three pushes to a connected node are still three
deliveries. Saying which half goes is worth more than "can probably be
removed", which is how a load-bearing window gets deleted in a hurry.
Everything else in a declaration is visible to whatever carried it. The
message is signed so it cannot be forged, and signing does not make it
unreadable — a password in `content` is a password the broker sees, which
is the transitive trust this design refuses everywhere else.
So a node generates a third key at enrolment and reports the public half,
exactly as it does for its identity and its overlay key. A file may
arrive `sealed` instead of `content`; the host opens it with that key and
writes the result. The control plane can then store a credential it
cannot use, and the broker relays a blob it cannot read.
A third key rather than reusing one of the two. The identity key signs
and is Ed25519; the overlay key is WireGuard's and is tied to being on
the private network, which a machine may not be. A key used for two
purposes is one rotation away from breaking the other.
Details that are not incidental:
- sealed and content together is refused, so "was this the secret or the
placeholder" is answerable by looking
- a sealed file defaults to 0600 rather than 0644, because the
consequence differs; an explicit mode still wins
- a node with no sealing key refuses the file rather than skipping it. A
machine that quietly omits the one resource carrying a credential looks
configured and cannot connect
- what is recorded is a digest of what was written, so drift on a
credential is still detected without the node keeping the value, and
the report that goes back over the broker carries neither
The key is made at enrolment rather than on first use. One made later is
one the mesh was never told about, so nothing could ever be sealed to it,
and the node would look fine and receive nothing.
This is why sealing was borrowed from another mesh's mistakes rather than
its design: there, credentials sit encrypted in the control plane's
database — which guards the database file and nothing else, since the
same value is also in each node's environment file in plain text and
inside every connection string composed from it. Its own tooling has to
search by value rather than by name to find the copies, and says the ones
inside composed URLs are usually the only copies in use.
The last step of the first-node path, and the bundle now carries all of it: a
container runtime, the store, a database per context, their schemas, the broker
with a certificate it generated itself, and the control plane running.
Then the machine enrols against the mesh on its own disk. It dials the broker
over TLS, refuses anything but the pinned certificate, presents the one-time
secret with a public key it generated, and is told the name the mesh has for
it. Its specialness lasted two commands, which is what ADR 0004 asked for.
The identity is saved only after the mesh says it knows this node. A node
holding an identity the mesh never recorded would believe it had joined and be
believed by nobody, which is worse than not joining because nothing looks wrong.
An already-enrolled machine refuses a valid token rather than quietly acquiring
a second identity, and a spent token is refused by the mesh. Both checked.
Containers gained a network field. The control plane must reach the store and
the broker on the machine it was raised on, before there is any mesh to arrange
that; the alternative was publishing ports and guessing an address that works
from inside a container, which fails in a worse way.
The control plane talks to the broker over loopback in plaintext, deliberately.
The TLS on 5671 exists so a node crossing a network can pin a certificate, not
for a hop that never leaves the machine.
Verified on a sealed lab machine: eleven resources applied from bare, the
control plane consuming, a token issued from inside it, and the machine
enrolled -- with the recorded public key matching what the host printed, the
token marked spent, and the profile stored.
Tier 0's first slice, per novox/hq 03-DESIGN/01-to-be/05-the-node-host.md. It
applies nothing, connects to nothing, listens on nothing. 2.9 MB, static, no
dynamic dependencies: copy it onto a machine and run it is the whole install,
which is the property ADR 0041 rests on.
A capability is detected, never assumed. Every detector runs something that only
succeeds if the thing FUNCTIONS — the daemon is asked for its version, the
package database is queried, the firewall is asked to list a ruleset, which
needs the privilege as well as the tool. 04-ISSUES/007 is the fault this
prevents: a client on disk with its daemon down looks exactly like a working
runtime, and a node assigned work on that basis fails when the work arrives.
Every verdict carries the reason and the method. A capability reported absent
with no reason is the same fault in a new place: something nobody can act on.
Two bugs found by running rather than reasoning, both silent:
systemctl is-system-running exits non-zero for every state except `running` —
including `degraded`, which means units failed and the init is emphatically
there. Reading the exit code reported NO service manager on a machine whose init
it was. That is 007 in the mirror, and both directions place work wrongly. A
verdict now reads what a tool says about itself, not only how it exited.
And `mesh-host inventory --json` printed text: the standard library stops
parsing at the first non-flag argument, so the flag sat unread and the command
exited 0 having ignored what was asked. The parser now takes the subcommand off
the front, and a stray or mistyped argument is refused rather than dropped.
Detection deliberately does NOT follow ADR 0008. That rule governs applying
state, where a failed step means the machine is not what was asked for. A failed
probe is a finding — "absent, because the probe failed" — and aborting would
replace one legible absence with total ignorance of the rest.
25 tests: structure and logic with a fake runner, and the same detectors against
this machine, because a test that fakes the system under detection asserts only
that the fake behaves as expected.
jschoubben
mesh-tools
jochen