Design 33 in progress against ADR 0154 (the twelve verbs, the prerequisites built); design 35 for
the records module under ADR 0153, extending 0025; as-is 07 rewritten to a mesh that keeps no store;
as-is 12 and 13 updated; issue 006 built and waiting on its live check.
The work order's group-3 question answered: an ordinary module the mesh assigns to the machine a
person sits at, holding a minted credential, calling tools under a manifest grant (invokes), serving
MCP on loopback. Design 34; pointers in 33, 25 and 0095; module check designed into 12 (issue 148);
README stops claiming an indexing nothing provides (issue 006).
Reported by the operator: ssh by the bare name logs in, by the mesh name
is refused. The predecessor's generator writes the bare name only; the
mesh's ssh-client roster already matches both and is not yet shipped.
Found and fixed the afternoon ADR 0148 landed: mailu-admin lost its
database behind Mailu's own resolver. Two catalogue PRs; an insight on
0148 that a container's dns is a decision, not a preference.
Issue 110's cause was not the filter: the runtime had never been told,
and the resolver dropped a query arriving on a bridge. ADR 0148 step 3
landed once it did (109, 151 resolved). ADR 0151 composes a route's
internal name under the serving node and drops the suffixed alias
(139, 157 resolved). Design 08 amended; a fact in 0148 corrected.
Hosts first, then the controller — a build and a push each, now that the
mesh delivers the host. The host refuses a lower sequence than it kept
and drains a batch by sequence rather than arrival; the controller
numbers each send under the node's hold, inside the signed bytes.
Measured: two pushes, sequence 2 in the kept declaration, counters in
the store agree, no machine reads as behind. That last one is the
subtlety: the mesh compares the digest of what it would send against
what it did, and a number changes the bytes, so the read-only comparison
composes with the last number sent rather than a fresh one.
All four machines run a host the mesh built, published and delivered, the
last delivery unattended: each stood aside once for a genuinely newer
version and the delivered launcher started it. A following push that
delivered nothing new was applied and reported by every machine and stood
nobody aside.
The crossover needs one restart of the unit per machine, once, because
the running launcher executes from its own inode. Measured timing: three
seconds on the machine, 17-20 as the operator sees it, the difference
being the control plane composing before it sends.
107 is unblocked: a declaration field is now a build and a push.
Asked whether a newer host was delivered using the link-time stamp, which
every delivered host carries as 'development build' now that the version
comes from where the binary sits. Never matched, so it stood aside on
every push for ever; standing aside cancels the report, so the mesh never
heard from it. Read as healthy throughout.
The three-minute push wait made it invisible: a wait long enough to
absorb a whole apply is long enough to hide that the machine never
answered.
161 resolved and verified on a machine: the workstation runs a host the
mesh compiled, published, delivered and started, applying declarations
and reporting the version it was delivered as.
The system it was built for comes from the artifact — the one thing a
toolchain takes from a module, which 0142 already allowed because the
target is a property of the artifact. The version comes from where the
binary sits, which 0142 decided and nothing had implemented.
Two mistakes on the way, both caught by reading the output rather than
the line that claimed success. A second -ldflags does not merge with the
first: the binary gained its system and lost -s -w, 12.2MB against 8.5MB.
And the delivered binary was named after its package, so the first
delivery was correct, reported success and was invisible to the launcher.
A delivered host that cannot apply is a machine the mesh cannot repair,
because the declaration that would fix it is the one it cannot apply. The
launcher's fallback is what made that an inconvenience instead of an
expedition.
162 is new and not about the host: an archive has no removal, so a module
using one can never be unassigned, and the attempt takes the whole apply
with it — the machine applies nothing else either. It is how undoing the
first delivery froze the workstation.
The loop closed on the workstation: the version landed, the launcher was
replaced, the running host stood aside, and after one restart the launcher
started a binary the mesh had compiled, published and delivered.
The launcher goes as a file resource rather than inside the archive, and
that is the safety rather than a preference. A file is written atomically,
so the running launcher keeps the inode it started from; an archive writes
in place with truncate and would cut a script a shell is reading. The
manifest carries a second copy and a test refuses any drift from the one
in packaging.
Then it would have refused the first declaration it was asked to apply.
The Makefile links in two facts the mesh's toolchain does not, on purpose,
and one of them is the system the host was built for — read before
anything is applied, so the failure is safe and total. Nothing reports it:
the unit is active, the bus link is up, and the log says it is hearing
what the node should be.
Worse, the declaration that would fix it is the declaration it cannot
apply, so the mesh cannot repair such a machine. Restored by moving the
delivered versions aside and letting the launcher fall back, which is the
fallback working as designed.
0142 already settles the version — it comes from where the component sits,
not from its linker — and that is unimplemented. The system pin has no
answer, and the candidates are a decision rather than a fix: put it in the
path too, carry it in a file beside the binary, or stop pinning at link
time at all, which is 0005's to change.
Filed as a to-do. Nothing is broken by it: every machine here is amd64 and
reports so, and one architecture is enough for now.
When a machine joins, the mesh should collect what it reasonably can about
it and refresh that daily. It already asks what a machine can do; what it
is made of is the same question one level down.
More is already collected than it looks — eight capabilities, the links
that face outside, the host version, and on an adopted machine what it
holds, what is reachable and the firewall and tunnel it was found with.
The architecture and the kernel are in there too, and nothing reads
either: measured, all four machines report amd64 and linux, and node show
prints the capabilities beside them without printing them.
Missing: memory, disk, the processor beyond its architecture, the
distribution and its version, virtual or physical, cores, uptime. Several
are what somebody wants when deciding where a module goes, and the
placement code's own comment already imagines them.
Also missing: the refresh. A machine publishes after an apply, and the
five-minute reconcile publishes nothing, so the mesh's picture is as old
as the last push. Same mechanism 087 wanted.
This is the third thing in one day found to be collected and read nowhere,
after held resources and the host version. Whatever gets added should say
in the same breath which surface shows it, or it will be the fourth.
159 gains the note that the architecture is already reported, so matching
an artifact to a machine needs no new fact — only the comparison and a
compiler told what to target.
Asked whether the host is built for more than one architecture. It is
not, and the reason is worse than a missing feature.
A bundle in a compiled language must name a system, must name one of
alpine, android or arch, and is refused with a careful message if it gets
that wrong. The field is then read by nothing: it does not reach the
compiler, no machine is matched against it, and nothing chooses between
two artifacts by it. The compile runs with no target named and produces a
binary for whatever the build machine happens to be.
The host is x86-64 because the build machine is, not because the
declaration said so. Correct for this mesh by coincidence — four machines,
all x86-64 Arch.
A module declaring two systems would get two identical binaries, both
published and both pinned, and the one sent to the machine it was not
built for would fail at exec. android is the sharp end: not an x86-64
platform, and an artifact declared for it today would be an x86-64 binary
wearing the label.
A field that is checked and ignored is worse than one that does not
exist, because the check is what persuades you it works.
Also noted: the processor is a second dimension the manifest has no word
for, so even implementing the present field would not answer the question
that found this. And since the Go toolchain builds statically, one binary
would run on all three systems anyway — so the pin is a policy rather
than a necessity, which is a decision and not a fix.
Both of the things ADR 0141's insight named as remaining are built. A Go
toolchain based on a new mesh-tools-go module, so the compiler is named
and not pinned; and ${version} in any value of a resource that uses an
archive or a bundle.
Measured rather than asserted: the mesh built the host through its own
toolchain, published it to its own registry, and the bundle fetched back
out is a statically linked stripped binary that runs and says it is the
host.
The cost was larger again than 0141's note said. Three more things in the
path assumed one language or one shape — an entrypoint became a .ts file
whatever the language, the output directory was the compiler's to create,
and a bundle was refused if it named what it is built from — and a fourth
was in the base image, which is Alpine where the first Dockerfile ran
apt-get. That last one is issue 136 in an image, and the build refused
rather than a module failing later.
The version in a path is the digest, not the commit: two builds of one
commit are the same bytes, so a content-addressed version keeps the path
an unchanged build already had.
Still nothing delivers a version to a machine. The host module declares no
resources, so the bundle sits in the registry and no machine is asked to
take it. 0141 carries the insight and 142 the account.
It reported "N machines run an older host than another" by comparing
versions as strings. A host reports its version as a commit, and commits
have no order. On the live mesh it named the three machines running the
NEWER host as the ones behind — ced54d4 sorts above 04a27ca and that is
all it means.
The code even carried a caveat saying versions compare as strings and that
this "is enough for the timestamps and commits this mesh uses". That was
the error, written down and not noticed: enough for timestamps, meaningless
for commits, and the mesh reports commits.
It now reports the split and claims no ordering, which is more useful as
well as more honest — the reader sees who is on which side, and that is
what decides whether a field can be sent. Ordering is left with the host,
which would have to report something ordered for anybody to have it.
This is issue 145 arriving by my own door an hour after I closed it: a
report that confidently says the opposite of the truth is worse than one
that says less.
Measured live. All four machines run the identical host binary — same
digest, installed within eighteen seconds — and at first only one reported
a version, which read as a difference where there was none.
A machine publishes a report after an apply. The five-minute reconcile
publishes nothing, because it is the machine keeping itself as declared
rather than answering anything. So a current, idle machine never says, and
the mesh cannot tell that from a machine running something ancient.
Confirmed by pushing: not reported, then 04a27ca.
Enough for the purpose, not enough for the claim. For deciding whether a
new declaration field is safe it is sufficient — pushing is what the mesh
is about to do, and the answer arrives with the act. For knowing what the
mesh runs it is not, and "not reported" is worded as "nobody has asked
recently" for that reason.
Making a heartbeat carry it would close the gap and would change what a
heartbeat is — a bare word that the node is there, deliberately carrying
nothing else. Left alone rather than widened in passing.
145, partly resolved. The sentence that was true for eleven hours of a
mesh in which no module could reach another now says what it is not a
claim about: that is the mesh and the machines agreeing, and nothing here
dials a provision. It checks nothing and does not pretend to — ADR 0146
decides the check and is deliberately not built. What changed is that the
report no longer implies otherwise. Stays open for that reason.
Carried forward: 0146's check needs an internal name fetched over TLS with
the certificate verified, and until today no machine trusted the mesh's
authority. Three of four do now, so whoever builds it does not have to
solve that first.
107, diagnosed and deliberately not built. The premise is confirmed in the
host's own words — unknown fields are refused because "a field the host
does not know is a thing the control plane believes it asked for" — so the
fix is a flag day, not an addition. 087 now makes the cost measurable, and
the measurement is why it waits: one machine of four runs an older host,
it is parked, and nothing delivers a host at all (142). Shipping the field
means hand-placing binaries and unparking a machine, and one missed in
that sequence is unreachable, not degraded. The fault it prevents has
never been observed.
142 gains the note that it is 107's gate, and that it is what makes a
declaration field cost a rollout instead of an expedition.
A judgement about order, not a refusal, and cheap to overrule.
The machine has reported its host version since ADR 0141, whose own
comment says why it must: without it nothing can say a machine is behind.
The controller's copy of the report did not have the field, so it
unmarshalled into nothing and was thrown away on arrival. Two structs
describe one message and only the sending side had it.
node show names it per machine, "not reported" where the mesh has not been
told. status names every machine running an older host than another does,
and which is newest.
Disagreement rather than staleness, deliberately: nothing delivers a host
version yet, so the mesh holds no canonical current one and "behind" has
no fixed point. What it can say is that the oldest host in the mesh is
what the mesh may send.
Two refusals to guess: a machine that reported nothing is not called
behind, and versions compare as strings — right for the timestamps this
mesh uses, wrong for a scheme where 10 sorts before 9, said at the place
that would have to learn.
107 gains the note that this is what makes its new field safe to consider,
and that one machine of four is behind today, so it is not free yet.
Two of the four surfaces the report named already carried it — the host
has reported Held since ADR 0100, and node show reads the machine's own
list with an `as of` beside it. Recorded as checked rather than assumed.
Two did not. The apply line counted what it applied and said nothing
about the difference; status read the mesh's take-time listing, so a
module assigned after it showed nothing at all.
Both now say it, and the part that carries the weight: a hold suppresses
"all doing what they were told, all heard from, running what the mesh
would send them". That sentence was true for the whole outage, and acting
on it is what stopped the predecessor's proxy. Being adopted still does
not suppress it — a mode somebody chose is not a half-finished action.
Status does not call a hold a fault, deliberately. It is correct
behaviour, and a reader trained to see red for something the mesh did
right stops reading.
Extended after the first was proven. Every converged machine now holds
the anchor and verifies an internal name with a plain client; before,
the two unassigned ones answered 'unable to get local issuer
certificate' and held no entry for the mesh.
ace is excluded on purpose: it is adopted, so a module assigned there is
held rather than run, which is right and is not trust.
Both the resolution and 0147's insight said one machine of four, which
was true for about twenty minutes.
jschoubben