125: a hold is not a line in the apply report — sixteen resources held
for an untaken module while four surfaces reported success, and the
operator stopped the edge's predecessor on their word (the route-proxy
flip outage). 126: a changed volume path neither recreates a running
container nor warns, and a roll-out upgrade policy makes a build a
deployment — together they turned a data-path migration into a forge
outage (the /var/lib move). Filed as 119/121 in the novox session
before syncing; renumbered past the other session's 119-124.
Not every host path names this machine. A system file the mesh owns is at that path on every machine
of the kind — the path is the fact. The operator's shared data is already answered as an access. A
path inside a container is the software's contract. What is left, and what a node's default layout
would replace, is 514: a module's own data, and what the mesh writes for that module.
Documents the reservation model as the records already have it — a root per node, one directory per
assignment, a placement for adopted data — and the three things nothing states: where the root comes
from, what sits beneath it, and the order the 514 are retired in.
Stops there deliberately. Changing where a definition looks without moving the data does not fail: the
mesh creates the directory, the container starts, the service comes up empty. Retiring these is a data
migration with a verification step, module by module, and belongs with whoever can see the machine.
A path inside a container is not a fact about the machine — /run/secrets and the directory a server
keeps its data in are the software's own contract, true in any mesh that runs it. Only the host side
of a mount names where it landed.
The first sweep matched path-shaped strings, so it counted both halves of every mount and every
in-container location a value mentioned: 798. Counted by role — directory and file resources, the
host side of mounts, accesses, and the targets of binds, grants, receives and secrets — it is 698
across 70 definitions.
The five modules this report first named were what a first look found. A sweep of all 71: 49 public
domains across 26, the node's own name 58 times across 19, a routable IP 12 times in one, 798
absolute paths across 70 (issue 119's number, grown), and no email addresses at all.
The sharpest case is not a domain: a mail module states the node's public IPv4 as the address it
trusts a real-IP header from, so a node that moves or gains a second address stops attributing mail
correctly, silently. Two upstream resolvers are excluded deliberately — naming a public DNS service
is a policy default, true of any mesh, not a fact about this one.
The object store derives each consumer's bucket from the login the mesh minted, and never reads the
one a definition named. The consumer still has to tell its own software which bucket to use, and has
no way to be told: bound values come from the provider's serves, which is a literal block identical
for every consumer, and a provisioner returns nothing. So all three consumers wrote the answer down
by hand and one of them wrote the predecessor's bucket — a key scoped to one bucket and software
asking for another, which reads like a credential fault and is not one.
Records the general shape: any interface where the provider names the resource forces the consumer to
reproduce the provider's rule, kept in agreement by hand and checked by nothing.
Three wordings disagree, and the confusion is the damage: the glossary defines artifact as an OCI
image while the build vocabulary already names four kinds in use, two of which are not images; the
image registry's seat is named after its job while ADR 0079 names foundation seats after their servers
and ADR 0109 names package seats after their ecosystem; and prose that says 'the module's image' reads
as though a module were an image.
Records the question the naming hides: ADR 0075 keeps two provisions because packages and images are
two protocols, and already allows the forge to provide the artifact store. The second implementation
rests on a bootstrap argument, and the forge has the same upstream-server shape the store and broker
have, which ADR 0078 raises as plumbing and adopts in place.
Step 4 claimed the forge cannot exist as a container before the builder has built its image. The
forge's server is an upstream public image pinned by digest; only its runtime sidecar is built. The
store and the broker have the same shape, and ADR 0078 raises both at genesis as plumbing and adopts
them in place — so the forge can be raised the same way and serve git, packages and OCI before
anything is built.
What survives: a grant is minted by the provider's runtime sidecar, which is built, so the question
is whether raise-service, grant, build-sidecar simply works. Sequencing inside the mesh, not images.
The wrong version came from taking a record's bootstrap argument at face value instead of comparing it
to how the store and broker are raised — one command away in the manifests.
Two things were treated as one. The mesh's own artifact store holds the store seat and is internal by
design — reached by name over the overlay, no accounts, ADR 0082. Serving a registry publicly is a
service the mesh can host: a module with its own name, accounts and storage, like anything else it
runs for somebody. The conversion this report was written beside gave the seat holder a second public
door over the same filesystem, which is neither.
Keeps the original issue whole — no garbage collection, and the settings a routine needs are not
enabled — drops the two-door complication, and sharpens one thing: deletion on the only door is
deletion on a door with no accounts, which the predecessor kept behind its authenticated one.
Asked whether merging the three reviewed changes would set a precedent. It would not: five modules
already carry a name belonging to this one mesh — a workflow module stating its host, protocol and
absolute webhook URL, and two carrying a full clone URL for a repository on the mesh's own forge.
That changes what the issue is for. There is no version of this catalogue today that does not name
the mesh it was written in, so refusing three changes buys nothing and a mechanism is the only thing
that removes any of them. The three were merged on that reading, each PR saying so.
Three open module changes independently wrote one mesh's names into the catalogue — two literal
public URLs, because the software generates absolute URLs behind a proxy, and one bucket renamed to
match what exists here. None was careless: the mesh composes <label>.<public-domain> for the proxy
and never hands it back to the module that asked for the route, and no interpolation yields a public
name, so writing the answer down is the only expressible option.
Files it rather than blocking the three, because the fix is a mechanism and the instances are live
needs. The cost is stated: a second mesh installing the identity provider gets the first mesh's
hostname, and nothing distinguishes a literal domain from a version number.
Asked first whether the seats change fixed this in passing, since it landed the same day and
touches both manifests the report names. It did not: the seat's holder is consulted only where
several nodes provide the thing, and with none providing it resolution refuses outright. Genesis
has no exemption — the unchecked first pass exists to learn what each node offers, and a
declaration is never built from it.
Records the part that did change: the three tests left failing on purpose were deleted by the
controller's seats PR and replaced with passing seat-based ones, so the gap is invisible again.
Adds the resolver to located-in, since that is where the refusal is.
Renumbered from 117, which is taken on main by 'a module's own code is a container in one record
and a process in another' — two reports claimed the same number and git would not have said so.
Scrubbed the node's name and a real registry path; this repository is public.
Keeps 118: the other claimant to this number is on main as issue 119, where ADR 0112 points.
Scrubbed the service's public name — this repository is public — and completed the report's
frontmatter with the fixed-by and amended-design keys every other report carries.
The harness compares against its own memory, so a backend that loses
what was provisioned (the cache's ACL users on a server restart) is
never provisioned again, silently.
The fact-check found mailu, whose user is its mailbox, so 0114 rotates
over two credentials rather than two logins, the adapter choosing what a
credential is. Also: minio keeps non-empty buckets; five backends take
their admin credential only at first init, so single-party rotation is
staged; postgres ownership moves to a non-login role; the harness keys by
consumer; rotation state lives with the vault. Consistency fixes across
0110-0113, 26 and 27; issue 103 resolved by mesh-host PR #22.
0113 — the plaintext claim was false under its own mechanism: handing a provider's answer to the
controller puts every secret on the broker and in the controller in the clear. The provider now seals
each secret field itself, to the consumer node's public key the mesh hands it, and the controller
carries sealed fields it cannot open. That is stricter than today, where the controller holds every
minted credential in the clear. Option 3 (plaintext to the controller) is recorded and rejected. The
foundation exception now covers root-secret rotation (0085) and forms like the broker admin's hash, so
no phase claims to remove the broker's bootstrap step. To-be 24 and 13 are named among what it amends.
27 — resolution is consistent with 0110: co-location and the only provider apply only where no seat
delivers the provision, so an unheld seat is refused even with one provider. The secret-field rule now
matches 0086 exactly (a declared env-file, never a container environment value). The seat placeholder
is the controller's, and the one module reading it moves to a host port. Contracts are held by the
controller and written down in phase 1, so they can be checked; every rule has a check. An operator's
secret is still the operator's, with the vault as custodian. Which seats a module holds is listed as
not settled.
0110 — the unheld-seat-with-one-provider case and the one-answer-for-everyone rule have checks; the
claim about moved manifests is corrected. 26 — the table governs and the code catches up, not the
reverse; scope and capacity agree with the glossary; moving a seat is described as it really is today.
0112 — aligned with 27, and lists 0049 and 26 among what it changes.
Issue 118 is renumbered 119: another branch took 118 first. 'Control-plane' is gone from 0110 and 0111.
- Secrets follow ADR 0085 as amended: a module's own secret is a provision the controller mints and
the vault records. The previous commit had that backwards. Whether the vault should generate
instead is recorded as an open question, not decided.
- A directory's contract is owner and mode only. The persistence flag was the keep flag ADR 0030
refused; a directory is kept while it holds anything, and disposable data is a named volume (0107).
- An operator's shared data stays an access (ADR 0051), which rejected an operator-owned directory.
Only where its path is written moves to the assignment.
- The records it changes on acceptance are named: 0051, 0091, 0046 (settings keyed by instance),
0084 (a provider is a node and an instance), and the glossary, which gains its new words only
when the record is accepted.
- How it is checked covers every stated rule. Container-side paths are no longer flagged by the
host-path rule, and code fallbacks are covered.
- Provisions are what other modules provide. A seat's occupant is not listed as one, and the vault
is not described as selectable per assignment.
- 'Control plane' becomes 'controller'. The provider count is ten of eleven, not eleven of twelve.
Issue 118 records what a review of where module code reads its files found: 789 host-path strings
in 70 of the catalogue's 71 definitions, every one a decision the definition makes about a machine.
Mounts are checked (ADR 0091); the same paths retyped as values are not. It records what that has
already allowed — a DNS provider that would provision nobody silently, a contributions file that
names credentials by host path and so forces every provider to mount at the identical path, an SDK
loop that treats an unwritten contributions file as empty without a word, defaults in code that
disagree with their own manifests — and that no module can be assigned to one node twice, because
every identity is keyed by the module's name.
ADR 0112, proposed for review, answers it the way ADR 0038 answered ports: a definition names
variables, and installing it resolves every one or refuses, from three sources — the assignment's
own configuration, provisions the mesh resolves against a contract, and what the mesh generates or
knows. A directory becomes a provision: the module requires one by name with its owner, mode and
persistence, and where it lands is the assignment's. The mesh's own files stop carrying host paths.
An assignment gets an identity of its own, so a module may run twice on one node.
Checking copies for agreement was rejected as checking something that should not exist; rewriting
paths per assignment was rejected as inferring which strings are paths by their shape. Syntax, a
node's default layout, and when a second instance becomes possible are left to the design.
Fixing builder's hand-faked package-registry binding tonight (requires:
package-registry, a real mesh grant instead of a hardcoded JSON fragment)
broke three tests describing a deliberate carried-binding fallback for
exactly this: gitea's own image is built by builder, so builder cannot
yet hold a real grant from gitea the first time either has to exist.
Invisible on novox (already bootstrapped, gitea already live) — real on
any genesis from scratch. Fix left in place, tests left failing rather
than reverted or hacked, so the gap stays visible.
Filed 2026-09-24 on a branch of its own and never merged, numbered 113, which is taken. 114 is
free because a sibling branch folded it, so it takes that number and keeps its commit.
Kept separate from issue 117 rather than folded into it. 117 asks the same question of every
module and locates the missing decision; this asks it of the controller, where `network: host`
means container network isolation — the property that resource type usually buys — is not in use.
That observation is this report's own and is nowhere in 117, and folding would lose it.
Its first open question is answered by 117's diagnosis and now says so: the host's `process` shape
is built, applied and tested, restart and run-to-completion semantics included, so deciding this
does not wait on host-side work.
Filed after a session where every mesh-controller interaction went through
docker exec — its manifest runs it as a container with network: host, using
none of the isolation that resource type usually buys, while ADR 0006 makes
it the mesh's single point of coordination. Open question, not a claimed
defect: does type: container get the controller anything type: process
(supervised the way the host supervises its own unit, per ADR 0005) would not.
Asked what the "sidecar" is and whether a supervised process would do instead. The repository
answers both ways. ADR 0047 (accepted, unsuperseded) says a module with tools or events runs a
container carrying its compiled code. To-be 18 and 20 (both proposed) define a `process` resource
type — the module's own code, a unit the machine's supervisor keeps up — and the worked guide says
plainly "it is why these are `process` rather than four containers." Neither design doc names 0047,
and no decision record mentions a `process` shape at all.
Diagnosed rather than left open, because the ground truth settles what the report could not.
The shape is real: mesh-host defines TypeProcess, applies it, and tests it, and the host's
vocabulary is twelve shapes rather than the nine ADR 0029 counted. So the alternative the report
offered — that two proposed documents describe a type that does not exist — is disproven.
ADR 0029's mechanism is intact and was not enough. The vocabulary-count test names the decision
behind each addition: network 0029, access 0051, opening 0100. The eleventh names a *proposed
design document*, and TypeProcess is the only shape in the vocabulary whose doc comment cites no
ADR. Requiring every addition to name something does not require it to name a decision.
The argument this issue asked for already exists — as a Go test comment. "It is a full-host shape
rather than a portable one: it needs a process supervisor to install into. It does NOT need a
container runtime, which is the point — only software that genuinely needs isolation asks for a
container." That is a decision's context and consequences, in another repository.
What the catalogue does is a third thing: 115 container declarations against 3 process, all three
in showcase — the module to-be 20 documents. There the tools resource is a container running
`sleep infinity` on a bare upstream base with the broker credential mounted, and the tools and
provisioner entrypoints are run by nothing. That is the condition 0047 was written to end, back
in a new shape.
Where the isolation argument leaks is narrower than expected and worth having precisely: the
serving key and the credential shape both conform. But serveTools serves every registered module
over one broker connection, the runtime takes its modules from a comma-separated list, and
x-source is stamped from the single credential — so two modules in one runtime means the second's
events are attributed to the first. Nothing refuses it and no test asserts against it.
Located on hq rather than on a code repository: the implementation and the design layer agree,
and the missing thing is the record. Which shape is right is left open, deliberately — this
establishes that the question was answered in practice and never written down, not which answer
is correct.
One correction kept in the trail: the first search here was for len(Vocabulary()), found nothing,
and was two steps from being written up as "the mechanism ADR 0029 relied on is gone." The test
binds the slice to a local first. A negative search result read as a fact about the world is the
same error issue 113 recorded.
The gap is closed in the proxy: policy applies, the four capabilities exist, the table is keyed
by host and path with a total ordering, and the two failure modes that rot quietly are held by
tests — a declaration carrying a credential refused rather than served, an unreadable secret
failing closed.
Resolved rather than left open because the issue reports a gap in the proxy and that gap is
gone. But the record says plainly what it does not yet allow: an operator still cannot move the
affected routes, because that needs the mesh side — a manifest able to declare these values and
the controller minting the secret auth names. Until both exist the capability is reachable only
by writing the routes file by hand. That is the ordinary build-out of a contract this issue's
decision created, and it belongs to to-be 08 rather than here.
The open questions are marked answered and kept rather than deleted, pointing at ADR 0108 —
what was rejected and why is the half worth having, and a section still saying "the fix should
not be written before these are answered" after the fix was written reads as though nobody
looked.
One finding kept in the record: priority was read with the reader for ports, which caps at
65535, and the one real rule this reproduces is declared at 100000. It parsed to zero, so
refusal and path scoping would have shipped looking complete and doing nothing on the only case
that motivated them. A validator borrowed from a neighbouring field is a silent default.
Issue 116 found the mesh's proxy applies nothing to a request — host lookup, forward. Against
what the replaced ingress actually relies on, four capabilities are missing: authentication
(three dependents, each gating an admin surface with no login of its own), refusal scoped to a
path (one, a live incident mitigation), path-scoped routing with priority, and redirect.
Policy goes on the route rather than beside it. A proxy-side settings layer keyed by route name
would keep the grant literally clean, but then "what protects this route" is answered from two
files nothing keeps in step — and a route's protection is part of what a route is.
The set is closed at those four, so a fifth is an amendment and each addition is earned by a
dependent that exists. An open middleware surface was rejected: it recreates what is being
replaced, and narrowing one later is far harder than widening a closed one.
Where policy needs a credential the declaration names a secret and never carries the value,
which keeps the existing secret machinery the only thing holding credentials. Inlining a hash
was rejected as the first credential in a declaration — a precedent easier to set than withdraw.
This re-keys the routing table by host and path with priority, which follows from the decision
rather than being a separate one: two of the four need one host routed more than one way. Equal
priorities must resolve identically every time or the proxy stops being reproducible.
The record says how it is checked, including the negative case that rots quietly — a
declaration carrying a credential value rather than a reference must be refused, so the
rejected option cannot return by accident.
08-connectivity §3 names the record and gains the subsection; issue 116 gains amended-design.
Two things the report got wrong, and one it could not have found the way it looked.
Disclosure first: it carried a real hostname and an absolute node path, in a public
repository. Both are gone; the ingress, the modules and the routes are named by role, as the
rest of 04-ISSUES does.
The count was low. Basic authentication has three dependents in the catalogue, not one — the
key-value store's browser UI, a database web UI, and the ingress's own dashboard. All three
are credential-less admin surfaces whose only gate is a middleware the mesh's proxy lacks.
The earlier version read only the node's dynamic configuration directory, which cannot see
what modules declare as container labels; counting needs both sources, and the report now
says so.
Two gaps were missing entirely. Redirect rules: two live routes canonicalise a www name onto
its apex, they exist only on the node and not in the catalogue, and they fail silently rather
than erroring. And path-scoped routing with priority, which is the one that reorders the
issue: the table maps host to exactly one target, so a host cannot be routed two ways, and
the refusal rule matches a path on a host already routed elsewhere. Authentication and a
source filter would not make it expressible. Path scoping is a prerequisite, not a sibling.
Also corrected: the refusal rule was described as an address-scoped deny. It is an allow-list
holding a single documentation-range address — deny-everyone — so reading it as address-scoped
points at the wrong fix. And its severity was understated: its own header records it as
incident response closing an abused write primitive, which is not "a real exposure" but a live
mitigation.
The open questions now say plainly that they are design questions and the fix should not be
written before they are answered, and one is added: whether a declaration may carry a
credential at all.
Comparing route-proxy against what HAL's actual Traefik config does today,
not Traefik's general feature set, per the standing rule that the nox mesh
must do at minimum what the HAL mesh it replaces already does. Everything
else checked out even or better; these two are real, confirmed gaps —
RedisInsight has no login of its own and depends entirely on Traefik's
basicauth middleware, and the gitea-internal route depends on an IP-scoped
deny rule. Neither has any equivalent in route-proxy's single-lookup
request path.
The object-store module was repinned to a maintained fork of the withdrawn server image, its
runtime sidecar built rather than pulled, and its data moved off the predecessor's live
directory. The instance is closed; the three general points the report makes are not, and What
was done says so rather than letting a resolved status imply otherwise.
Folds in the one thing a duplicate report of this symptom had that this one did not: issue 064
asked whether the build environment can reach a declared vendor image and assumed that, once
declared, it stays fetchable. Withdrawal is the case that assumption does not cover. The
duplicate is not merged — it carried the reading this report's diagnosis retracts.
The diagnosis carried a table headed "claims that could not be substantiated", denying a
module.json, a digest pin, and an all-zeros runtime digest. All three exist. The table is
withdrawn in full and replaced with what is actually true, plus the two claims that remain
genuinely unverified rather than disproven.
The cause: one repository was searched and absence in it was written up as absence. The
catalogue of the mesh being built is a separate repository, not checked out where the search
ran, and all four claims were about that repository. Compounding it, the predecessor's
object-store module and the one being cut over to were treated as one thing — they are
different files in different repositories, one pinning a tag with no sidecar, the other a
digest with two container resources.
Also corrected in the report: located-in named the wrong repository; the "pins a tag" passage
described the predecessor; the open question about pinning by digest is struck, because this
module already does and it made no difference — a deleted digest resolves to nothing either
way. The section on why nothing broke is now scoped explicitly to the predecessor's
machinery.
The lesson kept in the record: "zero occurrences anywhere in the tree" is only as strong as
the tree searched, and a diagnosis must say which tree. A confident rebuttal of a correct
report is worse than no diagnosis — it sends the next person to the wrong place with a
written record behind them.
Records the rule the operator gave directly, mid-session, after checking
that HAL's own postgres and lavinmq both used a directory bind and the
mesh's adoption of them three weeks ago switched to a named volume without
a reason recorded anywhere.
Already built and rolled out on novox (mesh-catalog PR #54) before this
record -- urgent enough to fix first and write down after. Includes the
incident: the new host directories needed the container's own UID, which a
named volume gets for free and a directory bind does not; mesh-store
crash-looped on Permission denied until ownership was matched to what the
original volume already had.
Closes issue 115. Checks pass.
jochen
jschoubben