A second tools process following mirrors would copy again and say published twice. The ask is claimed by
compare-and-set before it is copied, and its outcome carries the ask's id as the event id, so a redelivery is one
message. The cap's publish-refused finding stands on incoming, not on one file, and clears with the next publish
taken.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
A large base could outlast a call's bound. The ask is kept in the state mirrors and the copy runs on that state's
watch, so a copy a restart cut is taken on again; published (or refused, with why) on container-image-registry's
events is the end the caller waits on, the place the seat's contract already gives mirror.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
The holder's own transfer logic goes: parts.Receiver takes the chunks into incoming, checks the hashes, drops an idle
transfer and tells the fill; distribution gives it the builder_versions check, its store and its seats' events, so
the forge's two package seats bind the same receiver instead of a copy.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
The verbs answered the holder's own shapes (missing and ended, a part named digest, versions as bare strings, held as
the publish's event) while the seat definitions, now generated into mesh-contracts, name lacks and held, part,
versions of artifacts, tags with digests and a hold's own body. A build module using mesh-sdk's parts helper would
have read no lacking part and sent nothing. The bundle now takes and answers the contract's types and says each
event in its body; an image's manifest is the part whose digest is the publish's, as the contract names no media
type, and a hold on the image seat is of a copied base.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
The program had no test that runs it whole. This one starts run() on a nats-server of its own, with the registry
container faked on the loopback and node_haves and manifest-reads answered on their call subjects: an away node holds
both windows open, its return closes them, each closing said once on distribution.events, a direct write is then
refused, reads stay open, the health check answers through the program's port, and a waiter on published wakes on
none of the windows' events.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
An http probe of /v2/ said only that a port answered; design 51 asks the health check to call versions per seat and
probe the read side each minute, naming the program when it is down, from outside its process. registry_health asks
versions of each seat through the program's port, writes the read-side finding while it fails and clears it after.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
The windows' logic was built but nothing woke it, so both windows stayed open for ever. The program now reads every
node through node_haves and the node-engine seat's manifest-reads at its start, on each delivered and on each
manifest-reads-changed, steps both windows, says their closing once on distribution.events and raises the seven-day
finding. A build module's have counts as publishing through the seats when it runs a build heard publishing (the
tools bundle records the fingerprint of each publish's proved caller), so a paused build agent running the same
build holds nothing open.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
Until now mirror only answered a copy the store already held, so mesh-build-image had nothing to call for an upstream
base (design 51: an upstream image is copied by mirror with its pinned digest). The copy takes an anonymous pull
token, checks every manifest's and blob's bytes against its digest, copies an index's every platform first, and
fetches only the blobs the store lacks. tags was already served.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
The tools bundle runs in the tool runner's account and cannot remove a file in
distribution's 0700 state directory, so its first answered publish closes the
window bootstrap-push in the state windows and the program removes the file
(design 51 On a fresh mesh).
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
The program serves the store's port 5000 in front of the registry container, now on the node's loopback
(127.0.0.1:5001), listed before the program so the port is free when it starts (E14). The tools bundle serves
each seat's verbs as <seat>.<verb> and registry_incoming, registry_holds, registry_old_name and
registry_direct_writes, on distribution's own account, with the caller the tool runner proved. The manifest
claims container-image-registry and oci-artifact-registry with their verbs, keeps mesh-artifact-store and
provides artifact-store until the old name closes (E15), asks its bus state in requires (incoming, transfers,
holds, windows, findings, events) and declares its health outside the program. The bootstrap's one-off
credential is discarded at the first publish answered.
Not yet: the runtime's object store helper for incoming (mesh-sdk), the windows' driver on delivered and
manifest-reads-changed, the read-side health probe calling versions per seat.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
The holder answers publish and publish-part into incoming (parts of at most 1 MiB, resumed from what is held,
hashed, only the missing layers sent, 20 GiB with its 80 % finding and a refusal at the cap, a transfer idle 24
hours dropped), asks builder_versions before it stores, and says published or refused (design 51 *Publishing*,
ADR 0605 §4, ADR 0611). versions, remove (a tagged version; untagged only for the sweep's retention, which a
hold keeps), hold and release (ADR 0617 §2: the sweep alone; the operator's session by hand; never
distribution's own account), tags and mirror. The caller list is a trusted config value checked against the
caller the tool runner proved. The store's port is the program's: reads open, writes refused but while the
direct-write window is open (never over a tag published through the seat) or with the bootstrap's one-off
credential. The old-name and direct-write windows read every node, away ones included, close once, and name a
node holding them seven days.
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
mesh/delivery-group group feat/774-l3-reland rejected: composed together: the checker's verdict at the members' heads (b48c90fea31f): integration: mesh-self-ch…
mesh/check failed: integration: mesh-self-check/cmd/mesh-self-check.TestTheSelfCheckWholeRunsOnTheBus: the run answered {Run:doctor-1793351737-1 At:20…
One commit with exactly the tree of the reviewed head 35b4509c0a, on main 12c2574c9d, so two compiled programs added and removed on the branch leave no trace in history (issue 1577).
mesh/merge-gate checking again: the gate refused `unknown` because the controller judged with a node-engine validator pinned before mesh-host #117; mesh-co…
mesh/repo-check checking again: the gate refused `unknown` because the controller judged with a node-engine validator pinned before mesh-host #117; mesh-co…
Every directory and archive resource carries `unknown`: move-out for a home folder whose program writes nothing there (the node-engine, delivered in mesh-host #117, moves the rest to ~/.local/state/mesh-moved/<module>/ with a list), module for ~/.ssh and ~/.claude, program elsewhere; `knows` keeps what a program or the operator relies on (powerlevel10k's compiled files, i3's scripts and the asus/g14 drop-ins, dunst's slack rule, betterbird's records, node-env's forge credential, issue 1033). Two shared folders are no longer declared. Each module's README says it. checks/owned holds every manifest to it; checks/sshdir stays while main's merge check runs it (issue 1035). Tests that pin a manifest follow the new key only. Reviewed in three rounds.
mesh/merge-gate pass: builds distribution, new: modules/artifact-store-tools → novox; no bus step; every machine composes with the change as it did without…
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792260860441586760 (novox/mesh-catalog at 5e7567e5 into main,…
A module that provides the artifact store cannot build an artifact: building publishes to the store,
so the store would be needed to create itself, and the module check refuses it. The tools become
artifact-store-tools, assigned beside the store, and read manifests from the store's files through its
container as they already read the listing, so they need no address for the store's door. Named for
the artifact store, because "store" alone is the database server (hq glossary).
mesh/merge-gate fail: builds distribution → novox; no bus step; a manifest the change touches fails the module check: modules/distribution/module.json: thi…
The store had no working tools: its TypeScript client was never built, and nothing could count what
the store holds that no record names. A Go bundle lists the store's files through its own container,
reads each manifest through its door, and sets that beside the controller's records. Collection is
asked of the controller, which decides and records; the store's tools never delete. The image.pushed
event was declared and never emitted, and nothing consumes it, so it goes.
Walks run at most daily and stop after ten minutes or two million files; datasets are read from
their counters and large items from their top level only. The database platform's tables are
dumped with pg_dumpall rather than copied as live files.
Backup lines are derived from each module's data section instead of written by hand; the holder
measures declared items, reads the array under them, and deletes a retired item only after a last
restore point; the Go providers say each held consumer's size so an empty replacement is seen.
The dry run stood until the controller held each kept archive by a manifest
(mesh-controller #53) and an apply stopped reopening the collection window
(mesh-host #23, hq issue 224). The controller's collection command now
reports 134 of 134 kept archives held. hq issue 253.
The stock collector keeps only what a manifest names, and the mesh's bundles
and archives are bare blobs no manifest names: its first run, tonight at
03:30, would delete every one, the current ones included. It now reports
what it would delete and deletes nothing, until the controller holds each
kept archive with a manifest. hq issue 253.
Withdrawn 2026-10-04 to unblock novox: while-stopped named the module-local
id and the host refuses a declaration naming a container it does not have,
whole. mesh-controller#259 fixed the namespacing and it has been live since,
so the window composes as distribution.store and the host will take it.
Unchanged from before: plain garbage-collect at 03:30 with the server held
still. The store is at 40G and nothing reclaims it until this runs.
mesh-controller#259 fixes while-stopped to name the container as the machine
knows it — `distribution.store`, not `store`. Until that controller is the
one composing, novox refuses its whole declaration and takes nothing at all.
The step comes out; deletion stays on, already applied and harmless on its
own. A collect step without its window would be worse than none: garbage
collection against a live registry can sweep a blob a build is pushing.
Put back once the fixed controller is deployed and stays.
REGISTRY_STORAGE_DELETE_ENABLED on the server — the door already accepts a
push — and a scheduled step running the registry's own collector over the
volume at 03:30 with the server held still. Plain garbage-collect: what the
mesh keeps is still a manifest, so --delete-untagged is not needed and would
delete images machines are running.
75 endpoints across 50 modules, named from what each one is for rather than by a
rule: mail's seven protocol ports are smtp, imaps, submission and the rest; unifi's
nine are inform, stun, discovery, the two portal ports and syslog; minio's two are
s3 and console; the resolver's two are dns-udp and dns-tcp.
And 35 route contributions name the endpoint they serve instead of repeating its
port. A route and a listen both carried a port and nothing said they were the same
thing; now one of them does. gitea's path-level deny rule names neither, because it
is a rule about a name rather than an endpoint.
novox/hq ADR 0138. The words shipped a release ahead in mesh-controller #138 and
#139, and the control plane running today is built from that merge — checked before
this was written, because an unknown manifest key is refused and a catalogue using
one against an older control plane would stop resolving.
Two lines of work renamed the same seats differently. The trunk named them for their
scope — node-scoped ones `node-*`, leaving `the-artifact-store`, `npm-package-registry`
and `git` as they were — and this branch had renamed ten of them to `mesh-*`. The trunk's
set is what the live controller loads and what the live seats were actually renamed to, so
a manifest claiming this branch's name is one the running mesh refuses. Three of them
needed reverting by hand: git had auto-merged this branch's names where the trunk had not
touched those lines, which is the quiet kind of merge result.
Event names are this branch's, because the trunk has not converted them and they are what
issue 127 was about.
Verdaccio goes with the trunk's removal of it. The template work on dnsmasq's roster fact
is the trunk's, sitting beside this branch's local event names in the same file — the one
hunk where both changes landed together.
75 manifests, all parsing, no claim outside the trunk's set and no event name left in the
old bus's form.
Every module named its events the way the old bus spelled a routing key —
`module.<module>.<verb>`. Design 29 says a module names an event locally and the
mesh works out where it lands, so all 37 were stale against a rule already
decided. On the new bus that derives into a namespace belonging to a module
called "module", so no cross-module subscription in the mesh matched anything:
nothing failed, nothing reacted (novox/hq 04-ISSUES/127).
36 manifests converted, and 43 files of module code with them. The code mattered
as much as the manifests: the runtime builds the subject from what `emit()` is
handed, so a converted manifest with unconverted code would have had the
permission and the subject disagree.
Three things the new check found on the way:
- `photos` emitted an event its manifest never declared, which the new bus refuses
outright. Declared.
- `showcase` waited for an event nothing emits, so its demo could never be
triggered — only `showcase` may publish under its own name. It emits both halves
now.
- `distribution` declared an event named after a different module. It emits
`image.pushed` under its own name. An event about a *role* belongs on the seat,
where the name outlives whoever holds it, but the sdk has no way to publish on a
seat yet, so that stays recorded rather than declared.
The audit logger's "everything" pattern is `**` rather than the old bus's `#`.
Ten manifests claim mesh-* names now. What they PROVIDE is unchanged: gitea
still provides git and npm-package-registry, and a consumer requires the
interface, not the seat.
hq ADR 0107 / issue 115. HAL's own postgres and lavinmq both used a
directory bind (./db-data, ./data) for exactly this data -- the mesh's
adoption of them, three weeks ago, switched to a named Docker volume
instead, and searxng/distribution followed the same pattern since.
A named volume survives ordinary container recreation, same as a
directory bind -- that was never the problem. The problem is everything
else: docker rm -fv, docker volume rm, and docker system prune --volumes
all target it (one flag away from the docker rm -f this migration already
uses routinely); it is invisible to every tool this migration has used all
night (ls, find, grep across /var/lib, /services); and nothing outside
Docker's own volume machinery can back it up or notice it growing.
mesh-store carries the sharpest version: every database migrated tonight,
including keycloak's, live inside it.
Data already copied and verified on novox before this merges:
- mesh-registry-data -> /var/lib/mesh-registry (11G, diff -rq clean)
- mesh-broker-data -> /var/lib/mesh-broker (37M, cp -a)
- mesh-broker-tls -> /var/lib/mesh-broker-tls (12K, cp -a)
- mesh-store-data -> /var/lib/mesh-store (1.7G) -- mesh-store stopped
cleanly first, so the final copy is crash-consistent, not a live-file
copy of a running postgres; diff -rq clean after.
- searxng/valkey: not yet assigned anywhere, manifest-only fix, nothing
to copy.
Old named volumes left in place, not deleted, as the rollback path.
Review of the registry work found it node-scoped: a second `distribution` on another
machine resolved cleanly there, and only afterwards did the mesh notice `artifact-store`
offered by two nodes, with every consumer elsewhere refusing to choose. Worse, a node-scoped
requirement with one candidate installs that candidate, so anything that wanted the store
beside it would have raised a fresh, empty store on the wrong machine first.
There is one store in a mesh, which was already the effective rule; the claim now says it
where a second one is assigned, by name, instead of leaving consumers to discover it.
A module's identity is the software it is (ADR 0040). Two were named after the
job instead, and the job already had a name.
firewall installs the nftables package and runs nftables.service. The seat it
claims is the-packet-filter, which is correctly named for the role. Calling the
module firewall named neither the software nor the provision, and promised that
any firewall could sit there — the false genericity the naming rule forbids.
registry runs Distribution, the OCI reference implementation, and provides
artifact-store. So registry was a third name for a thing that already had two,
which is how one word ended up meaning the module, the software and the concept
in the same paragraph.
The capability stays firewall, and correctly: a capability IS a functionality, so
a node having one and fail2ban requiring one are both right. Only the module
moves.
Claude-Session: https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx
jschoubben
mesh-tools
jochen