jschoubben
668444785d
The review finishes before the push and its record is a Reviewed-by line in the description, so the mark repeated what the push meant. mesh-delivery asks the operator when a pull request's checks pass, keeps an ask through a trunk move by asking the check again at once, and hands the merge tool the approved heads, which replace the forge review of each sibling.
196 lines
13 KiB
Bash
Executable File
196 lines
13 KiB
Bash
Executable File
#!/bin/sh
|
|
# mesh-check-toolchain: go
|
|
# mesh-check-also: typescript merge-check-typescript.sh
|
|
#
|
|
# The catalogue's own check (novox/hq ADR 0237 as amended): the second layer of a pull request's merge
|
|
# check, `mesh/repo-check`, run by the build seat in the mesh's Go toolchain.
|
|
#
|
|
# The gate — the manifests the change touches through the running controller's module check, every machine
|
|
# of the facts snapshot composed with them and validated by the node-engine's own validator, the replays
|
|
# against this tree — is the build seat's first layer (`mesh/merge-gate`), run because the mesh's module
|
|
# graph builds these modules from here. It is not repeated here. This is the repository's own:
|
|
#
|
|
# 1. every manifest through the controller's module check, so one module's change cannot leave another
|
|
# it shares a rule with refused (MESH_GATE is the controller the mesh runs) — and the count of
|
|
# long-running resources without `health` held to the number in health-undeclared, which only goes down;
|
|
# 2. the Go tests of every module the change touches that has them, under the race detector when the
|
|
# toolchain has a C compiler. A requirement of the mesh's own forge the module cache does not hold (a
|
|
# pinned SDK commit) is taken from its repository's clone beside the check (checks/deps); a module whose
|
|
# dependencies still cannot be had **fails the check**, by name and why — its tests could not run, and a
|
|
# test that could not run is a failure, never a note (novox/hq issue 385). The integration tests (`//go:build
|
|
# integration`) run here, with -tags integration, and only here. A module written in
|
|
# TypeScript is type-checked and tested by
|
|
# merge-check-typescript.sh, declared above, in the TypeScript toolchain (novox/hq issues 302, 311);
|
|
# 3. the words (novox/hq ADR 0244): no word the glossary retired for the tools, as copied in
|
|
# retired-words, in any text a module's tools can show an agent — every string in its own code that is
|
|
# not a test, and every description in its manifest. Run over every module, not only the touched ones,
|
|
# so a word newly retired is found everywhere it stands. The check's own test runs first.
|
|
# 4. the paths (novox/hq ADR 0260): no module names a path under /run, /var/lib, /usr/local/lib or /etc
|
|
# that another module's manifest places — modules talk over the bus, not through each other's files.
|
|
# Over every module; checks/paths/shared and checks/paths/known narrow it, and known only shrinks.
|
|
# 5. the agent's home (novox/hq ADR 0266): no module unpacks an archive, or writes into a file, under
|
|
# ${machine:agent-home} — the node-engine does both after checking the path, and the agent account owns
|
|
# that home and could swap a link in between.
|
|
# 6. the secrets (novox/hq issue 342): no mergeable file fills a key with a vault secret that a setting could
|
|
# replace — the key is `protected`, or left out of the file's `takes-settings`. checks/secrets/known names
|
|
# the nested ones the controller cannot protect alone yet, and only shrinks.
|
|
# 7. the list of predefined keys (novox/hq ADR 0348 clause 7): a change to predefined.json removes no key a
|
|
# module still names, judged against the trunk's copy at the merge base. Whether a removed key still holds a
|
|
# value at the mesh or a node is mesh-config's to refuse; this check reads no store. The list itself and
|
|
# every manifest against it are judged by the controller's module check in step 1.
|
|
# 8. the config form (novox/hq ADR 0348 clauses 1 and 11): no module says `settings`, `takes-settings` or
|
|
# `${setting:<key>}`, in its manifest or a template it names, or has a mergeable file naming no keys (which
|
|
# takes its whole layer), unless checks/config/unmoved lists it as not yet moved; that list only shrinks,
|
|
# judged against the trunk's copy at the merge base.
|
|
# 9. the file text (novox/hq ADR 0349 §2): no module keeps a file's text inline in its manifest (`content`)
|
|
# unless checks/templates/inline lists it as not yet moved; each names its template in files/. That list
|
|
# only shrinks, judged against the trunk's copy at the merge base.
|
|
# 10. the mail-store seat (novox/hq ADR 0378 §3, issue 800): no manifest but modules/mesh-mail claims it.
|
|
# 11. owned paths (novox/hq ADR 0422 §8, issue 1024; it holds ~/.ssh for ssh-client as ADR 0415 §6 asked):
|
|
# every directory and archive says `unknown` (move-out, module or program) and a valid `knows`; `move-out`
|
|
# only under a home; and no module declares a path inside a directory of another module.
|
|
# checks/sshdir (ADR 0415 §6) stays while main runs it; it goes under issue 1035 once main runs checks/owned.
|
|
set -eu
|
|
|
|
(cd checks/words && go test -count=1 . && go run . ../..)
|
|
# The agent's home (novox/hq ADR 0266): no archive, and no file written into, under it.
|
|
(cd checks/agenthome && go test -count=1 . && go run . ../..)
|
|
# The mail-store seat (novox/hq ADR 0378 §3): only mesh-mail may claim it.
|
|
(cd checks/mailstore && go test -count=1 . && go run . ../..)
|
|
# ~/.ssh (novox/hq ADR 0415 §6): only ssh-client may name a path under it. Main's merge check runs this line, so
|
|
# checks/sshdir stays until main runs checks/owned; it goes under issue 1035.
|
|
(cd checks/sshdir && go test -count=1 . && go run . ../..)
|
|
# Owned paths (novox/hq ADR 0422 §8): unknown files named per directory; no path inside another module's directory.
|
|
(cd checks/owned && go test -count=1 . && go run . ../..)
|
|
# The secrets (novox/hq issue 342): a password is not a setting, so no setting can replace the vault's. The
|
|
# known list only shrinks: judged against the trunk's copy at the merge base, a line it does not hold is refused.
|
|
secrets_base=""
|
|
secrets_ref="origin/${MESH_CHECK_BASE:-main}"
|
|
if merge_base=$(git merge-base HEAD "$secrets_ref" 2>/dev/null); then
|
|
if git cat-file -e "$merge_base:checks/secrets/known" 2>/dev/null; then
|
|
secrets_base=$(mktemp)
|
|
git show "$merge_base:checks/secrets/known" >"$secrets_base"
|
|
fi
|
|
else
|
|
echo "NOT COMPARED WITH THE TRUNK: no merge base with $secrets_ref; checks/secrets/known is held to the list it was first written with"
|
|
fi
|
|
(cd checks/secrets && go test -count=1 . && if [ -n "$secrets_base" ]; then go run . -base "$secrets_base" ../..; else go run . ../..; fi)
|
|
# The list of predefined keys (novox/hq ADR 0348 clause 7): no key removed that a module still names. The trunk's
|
|
# list at the merge base is what it is judged against; a trunk without the file had none.
|
|
predefined_base=""
|
|
if merge_base=$(git merge-base HEAD "origin/${MESH_CHECK_BASE:-main}" 2>/dev/null); then
|
|
predefined_base=$(mktemp)
|
|
if ! git show "$merge_base:predefined.json" >"$predefined_base" 2>/dev/null; then
|
|
echo '{}' >"$predefined_base"
|
|
fi
|
|
else
|
|
echo "NOT COMPARED WITH THE TRUNK: no merge base with origin/${MESH_CHECK_BASE:-main}; no removal from predefined.json can be seen"
|
|
fi
|
|
(cd checks/predefined && go test -count=1 . && if [ -n "$predefined_base" ]; then go run . -base "$predefined_base" ../..; else go run . ../..; fi)
|
|
# The config form (novox/hq ADR 0348 §1, §11): a moved module says no old name, and the list of modules not yet moved
|
|
# only shrinks, judged against the trunk's copy at the merge base; a trunk without one is the list's first copy.
|
|
unmoved_base=""
|
|
if merge_base=$(git merge-base HEAD "origin/${MESH_CHECK_BASE:-main}" 2>/dev/null); then
|
|
if git cat-file -e "$merge_base:checks/config/unmoved" 2>/dev/null; then
|
|
unmoved_base=$(mktemp)
|
|
git show "$merge_base:checks/config/unmoved" >"$unmoved_base"
|
|
fi
|
|
else
|
|
echo "NOT COMPARED WITH THE TRUNK: no merge base with origin/${MESH_CHECK_BASE:-main}; checks/config/unmoved is held to the list it was first written with"
|
|
fi
|
|
(cd checks/config && go test -count=1 . && if [ -n "$unmoved_base" ]; then go run . -base "$unmoved_base" ../..; else go run . ../..; fi)
|
|
# The file text (novox/hq ADR 0349 §2): a file's text is a template in files/, outside the modules not yet moved; the
|
|
# list of those only shrinks, judged against the trunk's copy at the merge base, a trunk without one being its first.
|
|
inline_base=""
|
|
if merge_base=$(git merge-base HEAD "origin/${MESH_CHECK_BASE:-main}" 2>/dev/null); then
|
|
if git cat-file -e "$merge_base:checks/templates/inline" 2>/dev/null; then
|
|
inline_base=$(mktemp)
|
|
git show "$merge_base:checks/templates/inline" >"$inline_base"
|
|
fi
|
|
else
|
|
echo "NOT COMPARED WITH THE TRUNK: no merge base with origin/${MESH_CHECK_BASE:-main}; checks/templates/inline is held to the list it was first written with"
|
|
fi
|
|
(cd checks/templates && go test -count=1 . && if [ -n "$inline_base" ]; then go run . -base "$inline_base" ../..; else go run . ../..; fi)
|
|
# The lists that narrow the paths check may not grow unseen: the base branch's copies are what they are
|
|
# judged against (MESH_CHECK_BASE, main when unset), read from the clone the check runs in.
|
|
base_lists=""
|
|
base_ref="origin/${MESH_CHECK_BASE:-main}"
|
|
if git rev-parse --verify --quiet "$base_ref^{commit}" >/dev/null 2>&1 &&
|
|
git cat-file -e "$base_ref:checks/paths/known" 2>/dev/null && git cat-file -e "$base_ref:checks/paths/shared" 2>/dev/null; then
|
|
base_lists=$(mktemp -d)
|
|
git show "$base_ref:checks/paths/known" >"$base_lists/known"
|
|
git show "$base_ref:checks/paths/shared" >"$base_lists/shared"
|
|
fi
|
|
(cd checks/paths && go test -count=1 . && if [ -n "$base_lists" ]; then go run . -base "$base_lists" ../..; else go run . ../..; fi)
|
|
|
|
if [ -n "${MESH_GATE:-}" ]; then
|
|
checked=$("$MESH_GATE" module check modules/*/module.json) || { printf '%s\n' "$checked"; exit 1; }
|
|
# **The count only goes down** (novox/hq ADR 0240 rule 8): the long-running resources that do not say how
|
|
# they are ready, as the controller counts them, against the number kept in health-undeclared. A change
|
|
# that raises it fails; one that lowers it writes the new number there, so it can never rise again.
|
|
kept=$(sed -n 's/^\([0-9][0-9]*\).*/\1/p' health-undeclared | head -n 1)
|
|
counted=$(printf '%s\n' "$checked" | sed -n 's/^long-running resources without health: \([0-9][0-9]*\)$/\1/p')
|
|
if [ -z "$counted" ]; then
|
|
echo "NOT COUNTED: the controller the mesh runs is older than the count of resources without health (ADR 0240 Phase B)"
|
|
elif [ "$counted" -gt "$kept" ]; then
|
|
echo "the long-running resources without health rose from $kept to $counted: declare how each new one is ready (ADR 0240 rule 8)"
|
|
exit 1
|
|
elif [ "$counted" -lt "$kept" ]; then
|
|
echo "the long-running resources without health fell from $kept to $counted: write $counted in health-undeclared, so the count cannot rise again"
|
|
exit 1
|
|
else
|
|
echo "long-running resources without health: $counted, as kept"
|
|
fi
|
|
else
|
|
echo "NOT CHECKED: no controller was built beside this check, so the manifests were not read by one"
|
|
fi
|
|
|
|
race=""
|
|
if command -v gcc >/dev/null 2>&1; then race="-race"; else echo "NOT RACE-CHECKED: the toolchain holds no C compiler"; fi
|
|
|
|
# The modules the change reaches are the controller's planner's answer (MESH_CHECK_MODULES, hq ADR 0238):
|
|
# a module by its name, a new one by its directory. By hand, without it, the directories the changed files
|
|
# are in.
|
|
if [ -n "${MESH_CHECK_MODULES:-}" ]; then
|
|
touched=$(printf '%s\n' "$MESH_CHECK_MODULES" | tr ',' '\n' | sed -e 's#^modules/##' -e '/\//d' | sort -u)
|
|
else
|
|
touched=$(printf '%s\n' "${MESH_CHECK_CHANGED:-}" | tr ',' '\n' | sed -n 's#^modules/\([^/]*\)/.*#\1#p' | sort -u)
|
|
fi
|
|
# A module's dependencies: fetched, or a forge requirement taken from its clone beside the check
|
|
# (MESH_CHECK_BESIDE); its tests run against them (checks/deps, novox/hq issue 385). The helper's own test
|
|
# runs first.
|
|
(cd checks/deps && go test -count=1 .)
|
|
deps=$(mktemp -d)
|
|
trap 'rm -rf "$deps"' EXIT
|
|
(cd checks/deps && go build -o "$deps/deps" .)
|
|
untestable=""
|
|
for m in $touched; do
|
|
[ -d "modules/$m" ] || continue
|
|
if [ ! -f "modules/$m/go.mod" ]; then
|
|
[ -f "modules/$m/package.json" ] && echo "modules/$m is TypeScript: merge-check-typescript.sh type-checks and tests it"
|
|
continue
|
|
fi
|
|
if ! modfile=$(GOPRIVATE=git.novox.be "$deps/deps" -beside "${MESH_CHECK_BESIDE:-}" -work "$deps" "modules/$m"); then
|
|
echo "FAILED: modules/$m could not be tested — $modfile"
|
|
untestable="$untestable modules/$m"
|
|
continue
|
|
fi
|
|
[ -n "$modfile" ] && echo "modules/$m takes its forge dependencies from the clones beside the check"
|
|
echo "testing modules/$m"
|
|
unformatted=$(cd "modules/$m" && gofmt -l .)
|
|
if [ -n "$unformatted" ]; then
|
|
echo "modules/$m is not gofmt'd: $unformatted"
|
|
exit 1
|
|
fi
|
|
cgo=0
|
|
[ -n "$race" ] && cgo=1
|
|
# The integration tests (a file whose first line is `//go:build integration`: a test that starts a bus, a store or
|
|
# another process) run here and nowhere else: the unit suite stays under a second (novox/hq playbook 07).
|
|
(cd "modules/$m" && CGO_ENABLED=$cgo GOPRIVATE=git.novox.be go vet -tags integration ${modfile:+"-modfile=$modfile"} ./... &&
|
|
CGO_ENABLED=$cgo GOPRIVATE=git.novox.be go test -tags integration ${modfile:+"-modfile=$modfile"} $race -count=1 ./...)
|
|
done
|
|
if [ -n "$untestable" ]; then
|
|
echo "the check fails: the tests of$untestable could not run (novox/hq issue 385)"
|
|
exit 1
|
|
fi
|