f04294c3c1873a040eb0a322c212b2b883ed6e1f
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jschoubben
|
f04294c3c1 |
A service can be enabled at boot, and a container uses the runtime the machine has
Two gaps found by testing podman rather than reasoning about it.
The service shape could not say "starts at boot". It ran `systemctl start`, so
`service: docker.service, running` started docker now and it would not come
back after a reboot unless something else had enabled it. A declaration that
reports success and stops being true at the next power cut.
`boot: enabled|disabled` is now a separate field, not a fourth value of
`state`, because the two are orthogonal: a unit can be enabled and stopped (it
returns at boot) or disabled and running (started by hand, gone after one).
Absent means the host asserts nothing, so a machine whose operator enabled
something is not silently disabled by a declaration that never mentioned it.
Boot state is made true BEFORE the unit is started. When an apply fails part
way, enabled-and-stopped comes back at the next boot and running-and-disabled
does not, so the more durable half goes first.
`is-enabled` has the same trap as `is-active` had. Its exit code is non-zero
for nearly everything, and `static` is neither enabled nor disabled -- the unit
has no install section and CANNOT be enabled. Reading it as "disabled" would
have the host try, fail, and blame the wrong thing, which is the same shape as
reading a missing unit as "stopped".
The container applier no longer calls `docker` literally. Verified on this
machine against podman 6.1.0:
docker info --format '{{.ServerVersion}}' -> 29.7.2
podman info --format '{{.ServerVersion}}' -> Error: can't evaluate field
ServerVersion
podman info --format '{{.Version.Version}}' -> 6.1.0
So one probe cannot find both, and a host using docker's would report a machine
running podman as having no container runtime at all. Everything else IS
compatible -- run, rm -f, and docker's own Go template syntax for reading state
and labels all work unchanged on podman, confirmed by running them. That is why
this is a two-entry lookup rather than an interface: only the probe differs.
Detected rather than declared, because adoption keeps what the machine already
has (research 012), which hardcoding one runtime contradicts.
A machine with neither now says so, naming both: "docker: command not found" on
a machine deliberately running podman sends the reader after the wrong thing.
Verified end to end against real docker (container created, running, labelled)
and against an empty PATH (refused, naming both runtimes).
Two injections per behaviour, all confirmed to bite. One injection produced a
build failure that my check read as "no bite" for the third time, so the check
now distinguishes them.
|
||
|
jschoubben
|
057f34f924 |
The init is asked for start and restart; a launcher does the rest
ADR 0061. Recovery was the most systemd-specific part of the host, and it is the part that must work on a machine where nothing else does -- which made unit-file syntax a poor place for it, because syntax cannot be tested and the one time it runs is the one time nobody can afford it wrong. So StartLimitBurst and OnFailure move into a launcher script that init starts instead of the host. The unit drops to start-at-boot and restart-on-exit, which OpenRC, runit, s6 and an Android init.rc can all express. Everything 0059 decided is kept: two watchdogs, roll back once, recovery is local, the rollback shares no code with the host. The counter is the whole mechanism, so it is what the tests are mostly about. Three real problems came out of writing them: A counter file holding "1 2" became "12" -- `tr -d [:space:]` concatenates rather than rejecting -- which is past the limit, so a HEALTHY node rolled itself back. Now it reads the first field and insists on a plain integer. The corrupt-counter test used "not-a-number", which shell arithmetic happens to evaluate to 0, so it passed with the guard removed and proved nothing. Replaced with values that discriminate: "5x" errors under set -e and kills the launcher, and "0x10" is read as HEX 16 -- past the limit, so again a healthy node rolls back. And the test harness itself was wrong. With `set -e` and a bare launcher call, removing a guard killed the script at the first corrupt case and silently skipped everything after -- reporting a full pass over tests that never ran. Every launcher call now records its failure instead of aborting. Same class as the placebo assertion found last time, and the reason to keep injecting faults rather than trusting green. Both scripts run in `make check`. 27 launcher tests, 9 rollback tests, all confirmed to bite. |
||
|
jschoubben
|
f4143806c2 |
Build the rollback mechanism, and test it
ADR 0059's recovery path: the pieces that run when the host will not start. internal/upgrade -- two facts, neither of them the host judging its health. Whether the executable this process started from has been replaced on disk, and which version last completed a reconcile. The first design was wrong and the tests caught it, not review. It asked /proc/self/exe whether it was marked deleted. That is Linux procfs behaviour rather than a fact about files, and it catches only unlink -- a binary swapped by rename onto the same path reads as untouched, which is exactly what a package manager does. Now the identity is captured at start and compared later: no procfs, and neither case missed. known-good is one bare line. The reader is a shell script on a machine where the host is failing to start, so it must not need a parser to be present and working. Written only after a clean apply, which is the whole claim -- not health, because a disconnected node is ordinary and a failing resource is the machine's problem rather than the binary's. packaging/ -- the unit, the rollback unit, and the rollback script. The script shares no code with the host and calls none of it: a binary that cannot start cannot be its own recovery. POSIX sh, nothing that has to be installed. The unit carries Restart=always with a comment saying why on-failure would break every upgrade. Both are tested and both sets of tests were confirmed to bite. Injecting five faults broke exactly the intended tests -- except one, and chasing why it did not found a placebo assertion I had written: `check "exits zero" ... "0" "0"` compares a literal to itself and can never fail. Replaced with the real exit code, after which the injection bites. Also caught: an injection that produced a build failure rather than a test failure, which my grep read as "no failure". Re-run so it compiled, and the test did bite. The script test runs in `make check`, so it is a gate rather than something that was run once. Verified against the real binary: known-good is written beside the store after a clean apply and is NOT written after a failed one. |
||
|
jschoubben
|
9a9937b7e6 |
A struct per resource kind, instead of one struct with every field
Jochen asked why we don't simply have dedicated structs. We should, and the
flat struct was me extending an existing pattern rather than questioning it.
Before: one Resource struct carrying path, content, mode, unit, state, package,
image, name, env, ports, volumes, args, command, verify and in. Because a file
and a container shared it, nothing stopped {"type":"file","image":"postgres"},
so a `uses` map listed which fields each kind was allowed to carry -- a second
place to keep current, and the kind nobody updates is the one that silently
accepts a field the host will never read.
Now: Directory, File, Service, Package, Container and Action are separate
structs behind a Resource interface. File has no Image field, so the mistake is
not detected -- it is unrepresentable. Adding a field to a kind is the whole of
adding it; there is nowhere else that has to agree.
Parsing is two passes: read the envelope and each resource's raw bytes, peek at
"type" to choose the struct, then decode into it. Peeking is lenient on purpose
-- reading strictly there would report an unknown field before knowing which
fields are known.
Unknown fields are found by comparing the JSON keys against the struct's own
json tags rather than by catching the decoder's error. The decoder stops at the
first unknown field, and RefusalError promises every problem at once: a caller
fixing one field at a time learns the next only by running again. Caught by
testing the refactor against a real declaration -- a container carrying both
`unit` and `mode` reported only one of them.
apply.go switches on the concrete type instead of a string, so a new kind that
has no applier is a compile error rather than a runtime default branch.
No behaviour change otherwise. All existing tests pass unmodified except two
that reached for fields the interface no longer exposes.
|
||
|
jschoubben
|
337126603e |
Complete the host's vocabulary: package, container, action
The three shapes the substrate bootstrap needs and the host did not have. Until now tier 1 could not be raised at all -- step 0 is a package, step 1 a container, steps 2 and 3 actions -- so every line of the tier 1 and 2 designs was unbuildable. package -- present, never upgraded, never uninstalled. Removal is "forgotten", not "removed": the host cannot know what else needs the package, uninstalling a container runtime because a declaration changed would stop every container on the node, and the machine may have had it before the mesh saw it. Reporting it removed would claim an effect the host declined to have. container -- identified by a label carrying a digest of the declaration that made it. Comparing every field the runtime reports cannot be done reliably: a runtime normalises, defaults and reorders what it is given, and that is indistinguishable from real drift. There is no in-place update; a container's configuration is fixed at creation, so any change is a replacement, and saying so beats a partial update that leaves the running thing half-declared. This is the one shape the host removes, because it is the one the host created. action -- bundle-only, per ADR 0047. Verify is mandatory and does double duty: it is the idempotency check as well as the read-back. The host does not know what a database is, so "is it already there" is a question only the declaration can ask. `in` runs the action inside a named container, which steps 2 and 3 need. Parse now refuses actions; ParseTrusted permits them. The safe path is the default and the permissive one has to be named. The bundle and a local file handed to a root process use ParseTrusted; the link will use Parse. Also replaced the per-type "fields this type ignores" check with a field-set diff stated as what each type USES. The negative form needs every type revisited whenever a field is added, and the one nobody revisits silently accepts a field it will never read. Images must be pinned by digest (ADR 0046). A bundle naming a tag pins nothing. Verified against a real machine, not only fakes: an action ran and was idempotent on the second apply; an action that exits zero and satisfies nothing fails the apply; a real container was created, labelled, replaced when its declaration changed, exec'd into, and removed; a real package query round- tripped. Each new test was also confirmed to fail on an injected fault -- five injections, each breaking exactly its own test. One existing test changed: a vanished unit is now reported "forgotten" rather than "removed", which is what actually happened. |
||
|
jschoubben
|
08a1263a81 |
Stage 2 — the bundle a host carries
novox/hq ADR 0038: one behaviour, two sources of declaration. This is the source that does not need a mesh — the first node's path. The bundle is embedded in the binary rather than shipped beside it, because "copy it onto a machine and run it is the whole installation" stops being true the moment a second file has to arrive with it. `make host BUNDLE=...` builds a host carrying one; `mesh-host reconcile` applies it; `mesh-host bundle` shows it. A default build carries nothing and REFUSES to reconcile, saying why. A host that applied nothing and reported success would look exactly like one that raised a first node, and the difference would surface later as a mesh that never came up with nothing to point at. Proved on a sealed machine: no route out, no name resolution, one binary copied on, and it configured itself from what it carried. Idempotent on the second run. One bug found by running rather than reasoning, and it is a shape worth naming: `mesh-host bundle` validated the carried bundle through a path that strips comments, while `reconcile` handed the raw bytes to the parser. So the command whose whole job is to check the bundle said yes, and the command that uses it said no. Two paths to one artefact, disagreeing. There is one path now, and a test asserts that what validates is what is applied. What this does NOT prove is stated in the README rather than left implied: the claim under stage 2 is that one host can raise the substrate alone, and the substrate is four container services. There is no container type, because a container needs an image and where images come from is open; what belongs in a substrate is not known, because the closure for a one-node mesh is what research 011 and 012 exist to answer; and the machine used to test this cannot install a container runtime through a sealed network. The mechanism is finished. The claim is not, and shipping a host that claimed a substrate it has never raised would be the fault this whole project is about. 65 tests. |
||
|
jschoubben
|
9d8239afe8 |
Stage 2 — the host applies a declaration
A declaration is JSON, versioned, and an ordered list of resources with stable identities (novox/hq ADR 0043). The vocabulary is directory, file and service, and anything outside it — an unknown version, type or field — refuses the WHOLE declaration. A host that skipped what it did not understand would apply most of what it was sent and report success. It converges rather than executes: applying twice changes nothing the second time, and applying to a drifted machine returns it. A mode is maintained rather than set, because a permission applied at creation is not a permission held — this repository has paid for that once already. It owns a footprint and only that. What it applied and is no longer declared is removed; what it did not create is never touched. Removal runs FIRST, because a resource leaving a declaration while another arrives at the same path is an ordinary rename, and removing afterwards would delete the file just written. The store arrives here rather than at stage 3, as ADR 0043 predicted: nothing can be removed without knowing what was applied. It is written atomically, refuses to start empty when it exists and cannot be read — believing it owns nothing would leave everything behind forever — and is saved even when an apply fails, because what was applied before the failure is on the machine either way. Three faults found by running inside a raised machine rather than by reasoning: A unit that DOES NOT EXIST reads as `inactive` from `systemctl is-active`, exactly as a stopped one does. So declaring a unit stopped reported success for a unit the host cannot manage at all — absence read as satisfaction, which is 04-ISSUES/007 wearing a different hat. LoadState separates them. Removing an orphaned service whose unit has since been uninstalled failed the whole apply, and a host holding such a record could then apply NOTHING, ever, with no way out but editing its state by hand. Removal is now idempotent for the same reason os.RemoveAll is. And the flag parser was wrong in the same way twice: fixing `mesh-host inventory --json` by taking the subcommand off the front left `mesh-host apply decl.json --dry-run` broken identically, because the standard library stops at the first non-flag argument wherever that argument is. Parsed in a loop now. 30 new tests, 55 in total. |
||
|
jschoubben
|
73c010e7ef |
Stage 1 — the host reports what a machine is and can do
Tier 0's first slice, per novox/hq 03-DESIGN/01-to-be/05-the-node-host.md. It applies nothing, connects to nothing, listens on nothing. 2.9 MB, static, no dynamic dependencies: copy it onto a machine and run it is the whole install, which is the property ADR 0041 rests on. A capability is detected, never assumed. Every detector runs something that only succeeds if the thing FUNCTIONS — the daemon is asked for its version, the package database is queried, the firewall is asked to list a ruleset, which needs the privilege as well as the tool. 04-ISSUES/007 is the fault this prevents: a client on disk with its daemon down looks exactly like a working runtime, and a node assigned work on that basis fails when the work arrives. Every verdict carries the reason and the method. A capability reported absent with no reason is the same fault in a new place: something nobody can act on. Two bugs found by running rather than reasoning, both silent: systemctl is-system-running exits non-zero for every state except `running` — including `degraded`, which means units failed and the init is emphatically there. Reading the exit code reported NO service manager on a machine whose init it was. That is 007 in the mirror, and both directions place work wrongly. A verdict now reads what a tool says about itself, not only how it exited. And `mesh-host inventory --json` printed text: the standard library stops parsing at the first non-flag argument, so the flag sat unread and the command exited 0 having ignored what was asked. The parser now takes the subcommand off the front, and a stray or mistyped argument is refused rather than dropped. Detection deliberately does NOT follow ADR 0008. That rule governs applying state, where a failed step means the machine is not what was asked for. A failed probe is a finding — "absent, because the probe failed" — and aborting would replace one legible absence with total ignorance of the rest. 25 tests: structure and logic with a fake runner, and the same detectors against this machine, because a test that fakes the system under detection asserts only that the fake behaves as expected. |