mesh-tools
05e6b583e2
mesh/merge-gate head 05e6b583e2e3 merged into trunk ce56b491cffd — pass: builds ssh-client → ace, g14, novox, shanks; no bus step; every machine composes w…
mesh/repo-check head 05e6b583e2e3 merged into trunk ce56b491cffd — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/delivery delivered
The sweep from issue 1017 moved known_hosts2, rc and environment, which ssh and sshd read by default, and moved a link to a folder of keys, breaking every login through it. It could move a file a tool was still writing. When ~/.local was on another file system, every sweep failed at its first move and left an empty folder each round, seen only in the journal. Its list was written after the moves with errors ignored. Now those files and linked key folders stay; a link whose target, or the folder it names, cannot be checked is left and named; a copy of authorized_keys is moved at any depth; a path changed in the last minute, at any depth, waits for the next sweep; a retired folder on another file system fails the sweep before anything is made; the list names each move before it is made; and each sweep's outcome is kept for ssh_client_check, which shows a failed sweep as a problem. A key's first line is read only so far. The catalogue's new checks/sshdir refuses any manifest but ssh-client naming a path under ~/.ssh (ADR 0415 §6). The README cites §7 for config.d.
188 lines
12 KiB
Bash
Executable File
188 lines
12 KiB
Bash
Executable File
#!/bin/sh
|
|
# mesh-check-toolchain: go
|
|
# mesh-check-also: typescript merge-check-typescript.sh
|
|
#
|
|
# The catalogue's own check (novox/hq ADR 0237 as amended): the second layer of a pull request's merge
|
|
# check, `mesh/repo-check`, run by the build seat in the mesh's Go toolchain.
|
|
#
|
|
# The gate — the manifests the change touches through the running controller's module check, every machine
|
|
# of the facts snapshot composed with them and validated by the node-engine's own validator, the replays
|
|
# against this tree — is the build seat's first layer (`mesh/merge-gate`), run because the mesh's module
|
|
# graph builds these modules from here. It is not repeated here. This is the repository's own:
|
|
#
|
|
# 1. every manifest through the controller's module check, so one module's change cannot leave another
|
|
# it shares a rule with refused (MESH_GATE is the controller the mesh runs) — and the count of
|
|
# long-running resources without `health` held to the number in health-undeclared, which only goes down;
|
|
# 2. the Go tests of every module the change touches that has them, under the race detector when the
|
|
# toolchain has a C compiler. A requirement of the mesh's own forge the module cache does not hold (a
|
|
# pinned SDK commit) is taken from its repository's clone beside the check (checks/deps); a module whose
|
|
# dependencies still cannot be had **fails the check**, by name and why — its tests could not run, and a
|
|
# test that could not run is a failure, never a note (novox/hq issue 385). A module written in
|
|
# TypeScript is type-checked and tested by
|
|
# merge-check-typescript.sh, declared above, in the TypeScript toolchain (novox/hq issues 302, 311);
|
|
# 3. the words (novox/hq ADR 0244): no word the glossary retired for the tools, as copied in
|
|
# retired-words, in any text a module's tools can show an agent — every string in its own code that is
|
|
# not a test, and every description in its manifest. Run over every module, not only the touched ones,
|
|
# so a word newly retired is found everywhere it stands. The check's own test runs first.
|
|
# 4. the paths (novox/hq ADR 0260): no module names a path under /run, /var/lib, /usr/local/lib or /etc
|
|
# that another module's manifest places — modules talk over the bus, not through each other's files.
|
|
# Over every module; checks/paths/shared and checks/paths/known narrow it, and known only shrinks.
|
|
# 5. the agent's home (novox/hq ADR 0266): no module unpacks an archive, or writes into a file, under
|
|
# ${machine:agent-home} — the node-engine does both after checking the path, and the agent account owns
|
|
# that home and could swap a link in between.
|
|
# 6. the secrets (novox/hq issue 342): no mergeable file fills a key with a vault secret that a setting could
|
|
# replace — the key is `protected`, or left out of the file's `takes-settings`. checks/secrets/known names
|
|
# the nested ones the controller cannot protect alone yet, and only shrinks.
|
|
# 7. the list of predefined keys (novox/hq ADR 0348 clause 7): a change to predefined.json removes no key a
|
|
# module still names, judged against the trunk's copy at the merge base. Whether a removed key still holds a
|
|
# value at the mesh or a node is mesh-config's to refuse; this check reads no store. The list itself and
|
|
# every manifest against it are judged by the controller's module check in step 1.
|
|
# 8. the config form (novox/hq ADR 0348 clauses 1 and 11): no module says `settings`, `takes-settings` or
|
|
# `${setting:<key>}`, in its manifest or a template it names, or has a mergeable file naming no keys (which
|
|
# takes its whole layer), unless checks/config/unmoved lists it as not yet moved; that list only shrinks,
|
|
# judged against the trunk's copy at the merge base.
|
|
# 9. the file text (novox/hq ADR 0349 §2): no module keeps a file's text inline in its manifest (`content`)
|
|
# unless checks/templates/inline lists it as not yet moved; each names its template in files/. That list
|
|
# only shrinks, judged against the trunk's copy at the merge base.
|
|
# 10. the mail-store seat (novox/hq ADR 0378 §3, issue 800): no manifest but modules/mesh-mail claims it.
|
|
# 11. ~/.ssh (novox/hq ADR 0415 §6, issue 1020): no manifest but modules/ssh-client names a path under it —
|
|
# ssh-client owns it and sweeps out what it does not know.
|
|
set -eu
|
|
|
|
(cd checks/words && go test -count=1 . && go run . ../..)
|
|
# The agent's home (novox/hq ADR 0266): no archive, and no file written into, under it.
|
|
(cd checks/agenthome && go test -count=1 . && go run . ../..)
|
|
# The mail-store seat (novox/hq ADR 0378 §3): only mesh-mail may claim it.
|
|
(cd checks/mailstore && go test -count=1 . && go run . ../..)
|
|
# ~/.ssh (novox/hq ADR 0415 §6): only ssh-client may name a path under it.
|
|
(cd checks/sshdir && go test -count=1 . && go run . ../..)
|
|
# The secrets (novox/hq issue 342): a password is not a setting, so no setting can replace the vault's. The
|
|
# known list only shrinks: judged against the trunk's copy at the merge base, a line it does not hold is refused.
|
|
secrets_base=""
|
|
secrets_ref="origin/${MESH_CHECK_BASE:-main}"
|
|
if merge_base=$(git merge-base HEAD "$secrets_ref" 2>/dev/null); then
|
|
if git cat-file -e "$merge_base:checks/secrets/known" 2>/dev/null; then
|
|
secrets_base=$(mktemp)
|
|
git show "$merge_base:checks/secrets/known" >"$secrets_base"
|
|
fi
|
|
else
|
|
echo "NOT COMPARED WITH THE TRUNK: no merge base with $secrets_ref; checks/secrets/known is held to the list it was first written with"
|
|
fi
|
|
(cd checks/secrets && go test -count=1 . && if [ -n "$secrets_base" ]; then go run . -base "$secrets_base" ../..; else go run . ../..; fi)
|
|
# The list of predefined keys (novox/hq ADR 0348 clause 7): no key removed that a module still names. The trunk's
|
|
# list at the merge base is what it is judged against; a trunk without the file had none.
|
|
predefined_base=""
|
|
if merge_base=$(git merge-base HEAD "origin/${MESH_CHECK_BASE:-main}" 2>/dev/null); then
|
|
predefined_base=$(mktemp)
|
|
if ! git show "$merge_base:predefined.json" >"$predefined_base" 2>/dev/null; then
|
|
echo '{}' >"$predefined_base"
|
|
fi
|
|
else
|
|
echo "NOT COMPARED WITH THE TRUNK: no merge base with origin/${MESH_CHECK_BASE:-main}; no removal from predefined.json can be seen"
|
|
fi
|
|
(cd checks/predefined && go test -count=1 . && if [ -n "$predefined_base" ]; then go run . -base "$predefined_base" ../..; else go run . ../..; fi)
|
|
# The config form (novox/hq ADR 0348 §1, §11): a moved module says no old name, and the list of modules not yet moved
|
|
# only shrinks, judged against the trunk's copy at the merge base; a trunk without one is the list's first copy.
|
|
unmoved_base=""
|
|
if merge_base=$(git merge-base HEAD "origin/${MESH_CHECK_BASE:-main}" 2>/dev/null); then
|
|
if git cat-file -e "$merge_base:checks/config/unmoved" 2>/dev/null; then
|
|
unmoved_base=$(mktemp)
|
|
git show "$merge_base:checks/config/unmoved" >"$unmoved_base"
|
|
fi
|
|
else
|
|
echo "NOT COMPARED WITH THE TRUNK: no merge base with origin/${MESH_CHECK_BASE:-main}; checks/config/unmoved is held to the list it was first written with"
|
|
fi
|
|
(cd checks/config && go test -count=1 . && if [ -n "$unmoved_base" ]; then go run . -base "$unmoved_base" ../..; else go run . ../..; fi)
|
|
# The file text (novox/hq ADR 0349 §2): a file's text is a template in files/, outside the modules not yet moved; the
|
|
# list of those only shrinks, judged against the trunk's copy at the merge base, a trunk without one being its first.
|
|
inline_base=""
|
|
if merge_base=$(git merge-base HEAD "origin/${MESH_CHECK_BASE:-main}" 2>/dev/null); then
|
|
if git cat-file -e "$merge_base:checks/templates/inline" 2>/dev/null; then
|
|
inline_base=$(mktemp)
|
|
git show "$merge_base:checks/templates/inline" >"$inline_base"
|
|
fi
|
|
else
|
|
echo "NOT COMPARED WITH THE TRUNK: no merge base with origin/${MESH_CHECK_BASE:-main}; checks/templates/inline is held to the list it was first written with"
|
|
fi
|
|
(cd checks/templates && go test -count=1 . && if [ -n "$inline_base" ]; then go run . -base "$inline_base" ../..; else go run . ../..; fi)
|
|
# The lists that narrow the paths check may not grow unseen: the base branch's copies are what they are
|
|
# judged against (MESH_CHECK_BASE, main when unset), read from the clone the check runs in.
|
|
base_lists=""
|
|
base_ref="origin/${MESH_CHECK_BASE:-main}"
|
|
if git rev-parse --verify --quiet "$base_ref^{commit}" >/dev/null 2>&1 &&
|
|
git cat-file -e "$base_ref:checks/paths/known" 2>/dev/null && git cat-file -e "$base_ref:checks/paths/shared" 2>/dev/null; then
|
|
base_lists=$(mktemp -d)
|
|
git show "$base_ref:checks/paths/known" >"$base_lists/known"
|
|
git show "$base_ref:checks/paths/shared" >"$base_lists/shared"
|
|
fi
|
|
(cd checks/paths && go test -count=1 . && if [ -n "$base_lists" ]; then go run . -base "$base_lists" ../..; else go run . ../..; fi)
|
|
|
|
if [ -n "${MESH_GATE:-}" ]; then
|
|
checked=$("$MESH_GATE" module check modules/*/module.json) || { printf '%s\n' "$checked"; exit 1; }
|
|
# **The count only goes down** (novox/hq ADR 0240 rule 8): the long-running resources that do not say how
|
|
# they are ready, as the controller counts them, against the number kept in health-undeclared. A change
|
|
# that raises it fails; one that lowers it writes the new number there, so it can never rise again.
|
|
kept=$(sed -n 's/^\([0-9][0-9]*\).*/\1/p' health-undeclared | head -n 1)
|
|
counted=$(printf '%s\n' "$checked" | sed -n 's/^long-running resources without health: \([0-9][0-9]*\)$/\1/p')
|
|
if [ -z "$counted" ]; then
|
|
echo "NOT COUNTED: the controller the mesh runs is older than the count of resources without health (ADR 0240 Phase B)"
|
|
elif [ "$counted" -gt "$kept" ]; then
|
|
echo "the long-running resources without health rose from $kept to $counted: declare how each new one is ready (ADR 0240 rule 8)"
|
|
exit 1
|
|
elif [ "$counted" -lt "$kept" ]; then
|
|
echo "the long-running resources without health fell from $kept to $counted: write $counted in health-undeclared, so the count cannot rise again"
|
|
exit 1
|
|
else
|
|
echo "long-running resources without health: $counted, as kept"
|
|
fi
|
|
else
|
|
echo "NOT CHECKED: no controller was built beside this check, so the manifests were not read by one"
|
|
fi
|
|
|
|
race=""
|
|
if command -v gcc >/dev/null 2>&1; then race="-race"; else echo "NOT RACE-CHECKED: the toolchain holds no C compiler"; fi
|
|
|
|
# The modules the change reaches are the controller's planner's answer (MESH_CHECK_MODULES, hq ADR 0238):
|
|
# a module by its name, a new one by its directory. By hand, without it, the directories the changed files
|
|
# are in.
|
|
if [ -n "${MESH_CHECK_MODULES:-}" ]; then
|
|
touched=$(printf '%s\n' "$MESH_CHECK_MODULES" | tr ',' '\n' | sed -e 's#^modules/##' -e '/\//d' | sort -u)
|
|
else
|
|
touched=$(printf '%s\n' "${MESH_CHECK_CHANGED:-}" | tr ',' '\n' | sed -n 's#^modules/\([^/]*\)/.*#\1#p' | sort -u)
|
|
fi
|
|
# A module's dependencies: fetched, or a forge requirement taken from its clone beside the check
|
|
# (MESH_CHECK_BESIDE); its tests run against them (checks/deps, novox/hq issue 385). The helper's own test
|
|
# runs first.
|
|
(cd checks/deps && go test -count=1 .)
|
|
deps=$(mktemp -d)
|
|
trap 'rm -rf "$deps"' EXIT
|
|
(cd checks/deps && go build -o "$deps/deps" .)
|
|
untestable=""
|
|
for m in $touched; do
|
|
[ -d "modules/$m" ] || continue
|
|
if [ ! -f "modules/$m/go.mod" ]; then
|
|
[ -f "modules/$m/package.json" ] && echo "modules/$m is TypeScript: merge-check-typescript.sh type-checks and tests it"
|
|
continue
|
|
fi
|
|
if ! modfile=$(GOPRIVATE=git.novox.be "$deps/deps" -beside "${MESH_CHECK_BESIDE:-}" -work "$deps" "modules/$m"); then
|
|
echo "FAILED: modules/$m could not be tested — $modfile"
|
|
untestable="$untestable modules/$m"
|
|
continue
|
|
fi
|
|
[ -n "$modfile" ] && echo "modules/$m takes its forge dependencies from the clones beside the check"
|
|
echo "testing modules/$m"
|
|
unformatted=$(cd "modules/$m" && gofmt -l .)
|
|
if [ -n "$unformatted" ]; then
|
|
echo "modules/$m is not gofmt'd: $unformatted"
|
|
exit 1
|
|
fi
|
|
cgo=0
|
|
[ -n "$race" ] && cgo=1
|
|
(cd "modules/$m" && CGO_ENABLED=$cgo GOPRIVATE=git.novox.be go vet ${modfile:+"-modfile=$modfile"} ./... &&
|
|
CGO_ENABLED=$cgo GOPRIVATE=git.novox.be go test ${modfile:+"-modfile=$modfile"} $race -count=1 ./...)
|
|
done
|
|
if [ -n "$untestable" ]; then
|
|
echo "the check fails: the tests of$untestable could not run (novox/hq issue 385)"
|
|
exit 1
|
|
fi
|