A new 'package' artifact kind builds a module's own code on a public base image
and publishes it to the mesh's package registry by version (hq ADR 0076) — the
SDK above all, which the toolchain is built from and so cannot be built in the
toolchain. The credential a build needs to resolve or publish packages is
rendered as an .npmrc (basic auth, hq ADR 0048) and given to an image build as a
buildkit secret, never a layer, so a token is not baked into the toolchain image.
Claude-Session: https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx
The logging added a line to stdout, and the genesis path parses the builder's
stdout as JSON — so the first log line broke the parse with "invalid character
'c'", the c from "[clone]". A build that had worked stopped working because of a
print statement.
The installer's runner captures stdout alone (cmd.Output), and the contract was
already stdout=result, stderr=everything else. The fix is to honour it: every
builder diagnostic — the step log, the per-command echo, the module path's own
lines — goes to stderr. Stdout carries only once.go's result JSON.
And a unit test now fails if any fmt.Print to stdout appears in the two builder
command files, except the three that belong there: the result, --version, and
--help. A guard, because this was invisible until a 20-minute run hit it, and the
same class of mistake should fail in milliseconds next time.
Claude-Session: https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx
A build was silent from clone to publish, so a build in progress, one that failed
quietly, and a request that never arrived all looked identical — which cost a long
diagnosis against a running mesh chasing "the handler never fired".
Now: the handler announces a request the instant it lands. Build logs each phase
— clone, commit, manifest, bases, each artifact starting and finishing with what
it produced, resolve, done — through a Log callback that is nil-safe, so the tests
that pass none still build. And the Command runner echoes every command before it
runs, with where and how long it took, because on a hang the last line is exactly
the command it is stuck inside: "git clone waiting on a network that will not
answer" rather than "the builder did nothing".
The unreadable-request path prints to stdout now too, not stderr, so it shows in
docker logs without splitting streams — the split is what hid it.
Claude-Session: https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx
A module is one piece of software and may still carry a daemon in one language,
tools in another and a package in a third. The first cut compiled every bundle
into the toolchain's single output directory, so two of them would have
overwritten each other and then been packed together — one artifact containing
both, published twice.
So output is a property of the artifact, not of the toolchain, and the toolchain
says how it is told where to write rather than where it writes. Under a directory
named for the build rather than beside the source, so a pack never sweeps up the
module's own working files.
A second toolchain is declared so the multi-language path is exercised rather
than asserted — a list with one entry cannot fail the way a list with four will.
And the fake compiler in the tests now writes where it was TOLD to. One that
always wrote to a fixed place would have passed whether or not each artifact got
its own directory, which is the whole of what these tests are for.
Claude-Session: https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx
The bundle recipe: the one that both builds and packs. An archive packs a
directory as it stands, so shipping compiled output meant compiling somewhere
first — which meant a Dockerfile repeating the same incantation in every module.
Two base arguments with no defaults, a working directory chosen so the SDK
resolves upward, the compiler invoked by absolute path because the usual symlink
is resolved away when the base image is assembled, a second stage, an environment
variable naming the entrypoints. Most of the catalogue is unconverted and that is
why; two conversions done in one session were each wrong twice with a working
example open in the next window.
A bundle says a language and a list of entrypoints. The mesh knows what the
language implies. Anything a module could override there it would be writing a
Dockerfile to override, so a toolchain is deliberately not configurable.
Declared rather than inferred, both of them: guessing the language from which
files are present makes a build depend on a directory listing, and guessing the
entrypoints makes it change meaning when somebody adds a helper.
A toolchain the mesh does not hold is refused before anything is compiled, naming
what to build first — the same treatment a missing base already gets, because it
is the same question and somebody can answer it. A language the mesh does not
build is refused saying what would have worked, since the author is usually one
word away.
The list of languages is closed and adding to it is a decision. Every language is
another implementation of the contracts every module shares, and those change
rarely and cascade when they do (ADR 0039) — a mesh whose SDKs disagree about the
envelope fails by ignoring messages rather than by failing to compile.
Claude-Session: https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx
A fingerprint written into a recipe names one particular copy of the base — the
copy on whichever machine the person typing it was using. On any other mesh that
copy has never existed, so the build stops on its first line with a message
about an image nobody can look up. Three modules in the catalogue were in
exactly that state, and the line each of them replaced was equally dead.
A module now names the module and artifact instead, and the mesh answers with
what it holds. The builder is still a thing that clones, builds and answers: the
answer travels with the question, because only the mesh knows what it has.
A base the mesh has not built is refused before anything is built, naming which
module has to exist first.
This is how a mesh is raised: the installer carries this program and runs it
once, before anything exists, to produce the control plane from the same
repository and path every later rebuild will use. What raises the mesh is then
the same thing that maintains it, rather than a second mechanism exercised once
per new mesh — which is how often enough to rot.
With nowhere to publish, an image stays in the machine's own runtime and is
named by the digest of its own configuration: the same identity the installer
has always used for the image it carried.
Answering and announcing are different acts. The reply goes to whoever asked and
is correlated to their request; the announcement says to the whole mesh that a
module now exists at a commit, which is what the catalogue places in the module
graph (novox/hq ADR 0072). A build nobody asked for still has to be announced, or
the graph knows less than the registry does.
What it was built on top of is read out of the build's own inputs rather than
declared, because a declared list drifts from what the code actually uses
(ADR 0009). These are artifact references, which is what a build input names;
resolving them to module-versions is the catalogue's work, since it is what knows
which module-version published which artifact.
Events ride the topic exchange, not the direct one nodes speak over, so the
builder's account is granted both: it must be able to answer and to announce.
The envelope is the sdk's, reproduced exactly — a second shape would be a second
thing for consumers to handle, and they are written against the first.
Announcing is not allowed to fail a build. The work was done and was answered; a
build reported as failed because saying so failed is a lie about it.
Claude-Session: https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx
The builder cloned a repository and read the manifest at its root, which means one
repository per module. Nothing we have is shaped that way, so the builder could be
asked to build nothing that exists (novox/hq ADR 0069).
The path travels the whole way — named when asking, carried in the request, used
to read the manifest and as the context everything is produced from, echoed back
in the result, and recorded as part of where a module came from. Without that last
part the mesh could notice a module was behind its source and then be unable to
rebuild it, which is the worst of both.
A path climbing out of the clone is refused: a machine whose job is building other
people's repositories must not read whatever else is on its disk.
Claude-Session: https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx
A module usually runs software somebody else built: a database module
ships configuration and a provisioner and does not build a database. It
could name the upstream reference directly, and then every machine needs
a route to a public registry and the reference is a tag somebody else can
move — which is what pinning exists to prevent.
So an artifact may be `upstream`: pulled by the reference the module
names, pushed into the mesh's own registry, and pinned by the digest that
registry assigns. This is what the bootstrap already does by hand; it is
now something a module can say.
Refused: an upstream reference with no tag or digest, because what gets
mirrored would be whatever `latest` means today and a module pinned to
that is not pinned. And the rule that a build reads only its own
repository does not apply to it — applying it anyway refused every
reference with a registry host in it, which the test caught.
Written by trying to write a real postgres module and finding it could
not be said. It can now: two directories, two containers pinned by
digest, a superuser password sealed to the machine, and the grants
manifest — six resources from one assignment, all accepted by the host's
own parser.
That exercise also found my manifest wrong rather than the host: a
container declared `restart-on`, which is a service field, and the host
refused it by name. It is right to. A container whose own definition
changes is recreated, and a file it mounts is read by the process inside,
which is that image's business.
One store, and it is the registry the bootstrap already pulls from. An
OCI registry is a content-addressed blob store that also understands
images: PUT a blob and it is retrievable at /v2/<name>/blobs/sha256:… for
ever, by digest. An archive is a content-addressed blob.
A second store beside it was considered and is the right answer for
objects that are mutable, need per-reader access, or are not build output
— somebody's uploads, a backup, a thing with a lifecycle. None of that
describes a digest-pinned archive, and running a second service to hold
one kind of immutable blob is two things to run, two to back up, and two
ways for an artifact to be missing. Overturnable by reading: the manifest
carries a URL and a digest, and neither says what served it.
`build <repository>` clones, reads module.json, builds what it declares,
publishes, and records the manifest with the commit it came from. It is a
command rather than something the control plane does on its own, because
building runs things on a machine and what the control plane may send a
machine is bounded by the declaration language. This is the shape the
builder module takes when it is given work over the broker.
Proven end to end on a real repository and a real registry: a shell
module with a package, a user and a dotfile archive built, published,
fetched back at the digest it declared, rebuilt to the same digest, and
its manifest accepted by the host's own parser — including `user` and
`archive`, which did not exist this morning.
A tag is never accepted as a pin, and a blob already stored is not sent
again — it is named by its content, so re-uploading asks the registry to
store what it already has under the name it already has.
It runs on a node, not in the control plane. Building needs a container
runtime and a working tree, and the control plane deliberately cannot run
commands on a machine — what it may send is bounded by the declaration
language, and "run this build" is not in it. So the builder is something
a node runs as a module, given work over the broker like anything else.
The alternative, the control plane holding a docker socket, would make it
the one component that can do anything anywhere, which is the property
the whole design is arranged to avoid.
A module repository has one file at its root, module.json, saying what it
is and what it builds. A convention somebody can look for beats a setting
somebody has to find.
Properties that are decisions rather than details:
- a fresh clone every time. A build reusing a working tree can succeed
because of something a previous build left behind, and that is a build
nobody can reproduce.
- archives are packed deterministically — sorted, and carrying no
timestamps, uid, gid or original names. Two builds of one commit must
produce one digest, or nothing downstream can tell "this changed" from
"this was built again", and every rebuild looks like a change to every
machine holding it.
- nothing is published until everything is built. Half a module in the
store under a digest the mesh never records is reachable,
unreferenced, and indistinguishable from something in use.
The reproducibility test was passing for the wrong reason: both builds
landed in the same second, so a packer carrying timestamps would still
have agreed. It now stamps the two trees a year apart, and a timestamp in
the header breaks it.
One line is honest about not being independently tested: the sort before
packing is belt and braces over filepath.Walk's documented lexical order,
and no injection can distinguish it.
jschoubben